Tl/Dr: It's always difficult to write short answers about tricky words like "empathy." However, if we use some philosophy to pin it down to a more abstract concept based on relating to others, we can then expand the concept to how empathy affects societies and groups, and then look for those group patterns. We should see patterns in humans along well recognized scales of 5, 15, 40, 150, and 1500 individuals, and that would be a strong indicator of something we would have to call empathy.
It's actually surprisingly difficult to identify empathy unless you already have an intuitive sense of what it is and understand the being. Its too easy for individuals to fake empathy if you don't have a good enough connection. For example, it can be really difficult for us to determine if the empathy of politicians is genuine or an illusion. The cynics would argue it's always an illusion, but it's hard to tell.
You and I "feel" empathy because we're already bound together in how we view the world. To scientifically measure empathy from an alien perspective, we're going to have to be a bit more specific about it.
Arne Naess is a 20th century philosopher who came up with a concept called the "Ecological Self." He was looking at different definitions of self ("My body is my self" "my mind is my self" "my body and mind is my self", etc), and found great problems with all of them. The one he settled on "The Ecological Self is that with which the self relates to."
In his essays, he gave a story of a scientists who was looking through a microscope into a petri dish. A fly buzzing around the room landed in the dish. The dish itself contained a rather strong acid, rapidly deteriorating the fly's wings so that even if it escaped, it would not survive. But it takes time to be dissolved by acid. The scientist could do nothing but watch as the fly painfully disassociated into tiny molecules and integrated into the liquid of the dish. In those moments, Naess argued the scientist's Ecological Self extended to the fly. He had some sense of relating to what the fly was going through. Of course the scientist had never been dissolved in acid, but he had been splashed with it and burned by it, so there was something with which he related to the fly.
Naess then went on to argue that much of altruism could be explained by selfishness from such a wide concept of self. He argued that Mother Theresa was the most selfish person ever. However, her Self was so wide and all encompassing that acting in her self-interest meant supporting the countless people whom she helped during her life. Her Ecological Self encompassed more people than most of us can even imagine.
I use this philosophical example because it provides a larger more abstract structure the aliens can look for. They can look for evidence of this Ecological Self. If they find that the Ecological Self of any individual extends beyond their trivial body, then they have found empathy.
Thus, the best way to scientifically search for empathy is to put people together into groups and observe whether they exhibit these sorts of behaviors. This, of course, will require trying to categorize human behaviors into simple egocentric behaviors and wider behaviors founded in the Ecological Self.
Measuring this is excruciatingly difficult, which is why we don't have any scientific tests for politicians. However, one fascinating possibility stems from the Integrated Information Theory of Consciousness (IIT). IIT quantifies the gestalt effects which come from bringing processing units together. It studies how much information is contained in the individual units (neurons in the usual IIT case, but test-subjects in this alien case), versus how much information is contained in groups of them. If there's no integration, no larger selves to be had, then a collection contains no more information than the individual units. Put a bunch of CDs together, and they contain little more information than the stamped contents of the individual CDs themselves. Really the only information stored beyond that is a few bits of information stored in the order you stack the CDs. However, put humans together, and they tend to form groups and societies which contain far more information in their structure than in the individual bodies themselves!
A key to using this theory here is that information has a tendency to decay over time unless something is preserving it. If there's a lot of information stored in the collection, it will tend to decay unless it's providing some value to the individual units.
So with this, we can divide up our sample of half a million people into small lots and see if they form structures with measurable information stored in their interactions. You can start with large or small scale samples, but I'd probably start with small because I have a limited population of test subjects to work with.
As it turns out, our aliens will notice really interesting results occurring at some regular intervals. They'll find similarities between groups of 3 4 and 5. Likewise, they'll find patterns in the 5-15 region which are different from the 3-5 region. Another region is 15-40. Then 40-150. 150-500 is a bit fuzzy, then there's 500-1500. 1500 on takes on a very different nature. These numbers have been found by anthropologists in virtually all cultures. If I may gloss them:
- 3-5 people is your close knit group. These are the people you will let you see at your worst, or at your most vulnerable. These are the people you rely upon when everything else has gone wrong. In most militaries, 3-5 people form a fireteam, lead by a corporal.
- 5-15 is your wider friend circle. These are the people you rely on directly on a day to day basis. In hunter-gather societies, these are the sizes of hunting parties in many parts of the world. In most militaries, 12 people form a squad, lead by a corporal or a sergeant.
- 15-40 people is a tricky group to give a single word for. This is smaller than a tribe, but is a close knit group with a lot of tribal elements to it. The hunting party of 15 people will come back to the great group of 40. Here you will have true "leaders" in the sens of them making decisions which everyone else has to follow. In most militaries, 30-50 people will form a platoon, lead by a lieutenant, which is the first officer we've seen in this hierarchy of military organizations.
- 4-150 is a tribe. This is the size where we see real tribal allegiance. Many companies target 150 as the size of a department or center. The upper end of this is a number called "Dunbar's number," as the size of a tribe, though he actually gave it as a range rather than a single number. IDEO, made famous by documentaries in the '90s, would build a new building when they hit 150 people rather than go larger within a building. 80-250 people is a company in the military, lead by a captain or a major.
- 150-500 is a tough region that's been nicknamed the "megatribe." In anthropology, we find this when tribes form alliances. These are the extended tribe. In the military, these are battalions, lead by a Lt. colonel.
- 500-1500 is a fun region to explore. It is currently believed that 1500 is the maximum number of faces we can pair to identities. If you have a society larger than that, you become forced to have people who you do not identify by their identity. You have to identify them by classes or roles, such as "serf" or "plumber" or "teacher." Larger than that, you see written down hierarchies dominate, as our brains can't handle the size of the structures without writing.
I point all of these out because they seem to be very universal, so the aliens would almost certainly notice those structures. They just seem to form when humans are involved, so any reasonable scientific experimentation will eventually find them. From an IIT perspective, this would be sufficient to argue that there's something somewhat-empathic going on. If you stressed one human in a way specific to its individual self, you could watch how the group responds to reach out and help that individual.
It would be very hard to explain the patterns we see by simple rules, as opposed to ants which have very clear hard-coded social structures built into their DNA to handle the 4 thousand to 4 million ant populations. Ours would be more fluid and adapting. The groups always seem to form, though the actual social structures they create vary depending on the circumstances. This could be tested easily by dividing out groups and seeing how they interact before bringing them back into the fold (three cheers for nondestructive testing!) To turn this one around on you, you might ask yourself whether you would consider ants to be empathic in any way at all. There's something to their structure that suggests there's more than meets the eye!
Now that would be the scientific way. The other more intuitive way would be to introduce yourself into the system. You, as an alien, try to extend your Ecological Self to include the humans. Then you see whether this self is returned or not. This, of course, may introduce empathy into the humans where it did not exist before. Depending on your goals, this may be a good thing or a bad thing. Clarke's Rendezvous with Rama series explored this.
Of course, if your alien species has no empathy, this will not work. Then the question will be "what do they think they're looking for?" Is this a cold calculating study before planning an invasion? If so, then the pure scientific approach is best. On the other hand, if they're looking for something they lost, they might try integrating themselves into this human society. Perhaps they can learn empathy from us!