Motion will almost certainly be the most obvious thing that will make humans sit up and take notice. The ability to move, and to sense motion, is the subject of too many studies to mention. Conversely, if a species is sessile (and the spell checker sure doesn't like that word!), it will likely be overlooked for years, if it's ever spotted at all. Note that this isn't a requirement for sentience, just a focal point for human investigation.
Communication is probably the next biggest thing. Vocal communication attracts our attention as a species, and we tend to make big assumptions about species that don't operate this way. Electromagnetic phenomena will also almost certainly be noticed immediately. Body language is something that a trained observer would probably notice, especially if the species is highly gregarious, but this is the point at which arguments will develop about whether or not this form of communication constitutes sentient intelligence. Species that use chemical, "psychic" (for a certain value of psychic), and other more exotic forms of communication will almost certainly be overlooked for a long time.
Gregarity will also be a valuble clue. If a species, especially a spacefaring species, has such a sparse population as to rarely encounter other members of the same species, it might be hard to recognize an extraterrestrial species for what it is, never mind identify it as intelligent.
Technology is, of course, a dead giveaway, or it ought to be. Certainly anyone that thinks the great apes are sentient would see even the most primitive tech as a sign of intelligence. The famous termite fishing sticks come to mind.
Nervous Biology (Let the jokes begin!) For a species that isn't noted as sentient by any other method, biological examination might be used to try to determine the level of intelligence of that species. Current day scientists put a lot of stock in the Anthropic Principle, at least in the sense that they expect intelligent life to have a nervous system that functions like our does. Even xenobiologists that expect a physical nervous system radically different from our own aren't sure that they'll know what they're looking at when they see it. By this measure, if we ever accepted that some species with such a radically different biology was indeed sentient, we'd have much better clues to search for. Until then, only species that really resemble us are likely to be spotted in this way.