Why bother programming facial expressions for artificial intelligence if humans are bad at recognising them?

Question

Set in the near future, robots and mankind coexist amicably due to the great technological advancement in robotics and artificial intelligence. They are everywhere and some took a human form. All of them are hardwired to obey the robot version of penal code which keeps updating. I kept wondering, since we are good at recognising patterns but not interpreting them, which is why awkwardness exists between people, why even bother giving robot facial expressions? They definitely don't need them for talking to another robot, plus it will undoubtedly be awkward for us too, I suppose, because we all know they are fake!

To clarify I'm not referring to particular group of people with a specific syndrome, I'm talking about human in general as I believe we have no problem identifying pattern in nature or perhaps arguably one of the best species ever in history hooray however the actual meaning the same pattern is trying to convey differs among different people. This could lead to misunderstanding and tragedy especially when comes to AI a very complex machine.

Answer 1

Humans are actually very good at recognizing facial expressions and body language.

While it's true that large numbers of people are terrible at it, it's only in comparison to the average human. Assuming unimpaired vision and intelligence, humans use facial expressions to tell who among a group of people is talking, what or whom the person is referring to (because of their gaze)...at least a good guess, and many other things.

Maybe you don't know if someone is slightly annoyed vs upset, or maybe you can't tell if someone is joking, but that's as much word-choice and tone of voice as it is facial expressions. The number of things that people pick up from other people, or from animals, is much larger than you might think. If you're not sure of that, talk to someone who is blind. Or compare conversations on the phone or online with in-person ones.

Another way to look at facial expressions is as a subset of the larger set of non-verbal communication of body language and signaling.

From Turn Signals Are the Facial Expressions of Automobiles by Donald Norman.

Social cooperation requires signals, ways of letting others know one's actions and intentions. Moreover, it is useful to know reactions to actions: how do others perceive them? The most powerful method of signaling, of course, is through language. Emotions, especially the outward signaling of emotions, play equally important roles. Emotional and facial expressions are simple signal systems that allow us to communicate to others our own internal states. In fact, emotions can act as a communication medium within an individual, helping bridge the gap between internal, subconscious states and conscious ones.

As I study the interaction of people with technology, I am not happy with what I see. In some sense, you might say, my goal is to socialize technology. Right now, technology lacks social graces. The machine sits there, placid, demanding. It tends to interact only in order to demand attention, not to communicate, not to interact gracefully. People and social animals have evolved a wide range of signaling systems, the better to make their interactions pleasant and productive. One way to understand the deficiencies of today's technologies and to see how they might improve is to examine the route that natural evolution has taken. You know the old saying that history repeats itself, that those that who fail to study the lessons of history are doomed to repeat its failures? Well, I think the analogous statement applies to evolution and technology: those who are unaware of the lessons of biological evolution are doomed to repeat its failures.

So, yes, robots need facial expressions. They need external signals in addition to spoken or written or signed language. But they don't have to mimic human expressions. You're right that it is jarring to see fake expressions. If done right, however, they wouldn't be fake, they'd just be specific to a robot. If they are just different enough that they're obviously not human, but not so different that you can't pick up on them right away, you've hit the sweet spot.

Answer 2

Humans are some of the most advanced pattern recognition and actionary machines that have ever existed. So capable in fact that they have theories of matter (physics/chemistry/geology/...) and theories of what matters (psychology/religion/philosophy/...).

Given that it takes nearly 30 years to train most specimens to be expert recognizers in a given field of endeavor, should underline the difficulty of actually grouping various states of the universe into manageable and actionable patterns.

In your world robots act as autonomous embodied entities. They are capable of communicating at speed with other similar entities using a range of network technologies. This allows them to communicate fairly clearly about a range of topics but they too hit limitations.

• Radiowaves have limited bandwidth and everyone can hear them
• lasers require line of sight, and dust can cause errors
• network cables are high bandwidth but tether the robot to a specific vicinity

This places an upper limit on the ability of these entities to communicate effectively. Also there are entities out there that are not robots. It might pay to be able to communicate with those too.

If the robot was the size of a butterfly, aside from having limited mind/communication space, its likely that birds will treat it as food. The robot probably has the ability to shock the bird, but it might pay to communicate to the bird before it tries that eating the robot is a bad idea. Being a bright iridescent red/yellow is the general approach used by poisonous caterpillars, and generally birds respect that. After all the robot might actually be running low on power when the bird shows up.

Scale this up and there are a wide variety of organisms running around where it might pay better to communicate with than ignore. Particularly those two-legged simians, there are a few of them, they are pretty inventive, and quite happy to do boring repetitive tasks. Perhaps if the communication happened well enough they might be put to use inventing, and dealing with irritations?

Even without all of these other organisms hanging around, the simple fact that these technologies have bandwidth limitations is a problem - Communication is insanely important if you desire to reduce the frequency and severity of problems encountered while existing. There are a few more technological channels that would be useful, but it is relatively simple to adopt proven modes of communication:

• colouration (i am poisonous, i don't care if you recognise me/I'm not good news)
• hearing (locate movement, location, as well as conceptual information)
• vocalisation/stridulation (location as well as conceptual information, ready to fight, ready to mate, ready to serve)
• dance/movement displays (are you willing to invest energy into demonstrating capability, should I press the point to the next level?)
• faces
  • eyes (what are you interested in/looking at?)
  • eyebrows (are you exploring/frustrated/contented?)
  • teeth (are you indicating dominance/subservience, will you challenge me?)
• posture/stance (are you ready? what are you ready to do?)

Obviously no two robots need to have the full or even the same range of communication methods. Many creatures even in the same species have varied capacities. An obvious example is red/green colour blindness in humans.

So do robots need faces? No.

Would robots significantly benefit from having faces? Yes.

Answer 3

You say that humans, in general, are poor at recognizing facial expressions. As a person with both autism and a brain injury, I beg to differ. I am poor at both recognizing and using facial expressions (I even have a doctor's note!), and the differences are observable to an outsider.

A few years back, before the brain injury and before we knew I was autistic, my family used to play with the Society for Creative Anachronism (SCA). For those not familiar with the SCA, think of them as adults playing dressup in medieval garb, who get together to gossip, drink beer, and beat on each other with big sticks. For the Brits out there, think of stereotypical rugby players dressing up as King Arthur's knights, then playing rugby with swords (that event was called "Blood of Heroes", by the way, emulating a very bad movie of the same name).

Naturally, there is a lot of banter in such a group, and a lot of good natured roughhousing. They can find a week's worth of innuendo in the world innuendo. Shortly after we moved on to other hobbies, my wife told me that I used to scare them because, on the rare occasions I did join the banter, they couldn't tell whether I was serious or joking.

Post brain injury, it is much worse. Recent;y, when being introduced to a new business partner, they were trying to be friendly and informally ask me about myself. Unfortunately, they asked "What gets you out of bed in the morning?". A metaphorical question about emotional state with social cues, it effectively and awkwardly put an end to the conversation because it exactly hit places I physically lack the machinery to process efficiently.

My wife has learned to be very careful with the use of sarcasm and metaphors around me because, without the social cues that go with them, I tend to take them literally. "Why did you (some action) ?" "You Told me to." "I didn't mean for you to really do that!" "Sorry."

A robot mimicking humanoid characteristics to the point they were indistinguishable from humans, without emotional/social emulation of some sort, would tend to make people feel uneasy and awkward, or as my son would say "it would creep them out".

Worse than not mimicking emotions, would be mimicking them badly. A smile whose timing is off by 1/4 second is perceived as deceptive, for example. In contrast, obviously different emotional responses that are specific to the species (robots) can be learned by humans. I used to keep crustaceans in my aquarium, and it is amazing how expressive crabs and crayfish be without flexible faces.

For a good idea of the issues you might run into, watch the first season of Ninjago and the team's interactions with Zane, before anyone knows he is a robot.

Answer 4

Humans teach other humans in formal positions to use facial expressions when communicating. When someone is representing an organization or group, the message or information they are speaking more often than not has no connection to their personal opinion. As a result we teach humans in these positions to use a facial expression that communicates the "emotive" part of the message the organization/group wants to get across.

Well if we think that's appropriate for humans, we certainly will do it for robots if we can.

They definitely don't need them for talking to another robot

I suspect it could be used as part of robot-robot interaction as well. There are environments where it would be too loud to use audio, too problematic to use e.g. radio based communications and all you may have is the visual. It might be better to implement a custom set of robot-robot expressions for some comms.

plus it will undoubtedly be awkward for us too, I suppose, because we all know they are fake!

I know the smile on the sales person's face is fake too, but it had better be there or no sale ! We expect facial and body language to match words and we treat inconsistencies as suspicious. That's how humans work and why good communicators spend a lot of time learning to make it all look natural, even when it isn't.

The other side of this is that humans need facial expressions to match what they expect. A robot which has a hardwired smile or scowl would give a very disconcerting feeling to a human in the wrong context.

"I hope you enjoyed your stay with us ?" is going to feel very creepy to a human coming from a robot with a scowl or sneer (or something that could be read that way by a human) than from something with a pattern that's more easily interpreted as friendly.

So the human-like expressions would be useful to humans, avoiding emotional dissonance that would put us off.

We even treat animals like this - we learn to interpret their expressions and body language and "map" human meaning onto them. It's an important part of how we interpret interactions with the world. We do extend this to in-animate objects - signs, logos, advertising.

So facial expressions we can read easily would be there to make humans comfortable and make communications more effective.

Answer 5

Even if we're terrible at interpreting facial expressions, their absence would be extremely disconcerting. Therefore, in order to avoid being unsettling, you'd want the robots to mimic human behaviour, just like we'd do for speech patterns, avoiding monotone, etc.

Answer 6

Look for "mentalists" and you will find lots of info about them! They are talented to start with (compared to the average as someone said) and they are sufficiently trained to successfully trick you to think that they can read your mind. They too learn your facial expressions, tone of speech and think ahead of you in choosing a random number. Uri Geller, a famous example is just a first-year student in comparison. Look for two Israeli mentalists like Nimrod Har'el and Hezi Dean:

http://www.nimrodharel.co.il

http://www.hezidean.co.il/ENGLISH/

I think mentalists would have failed if they did not study body language.

Those trained persons would be valuable to AI programmers in training a robot to read your body language. Nobody mentipned that the robots may be sufficiently trained to search for suspects in a crowd.

Answer 7

You raise a very good point.

In many demonstrations, it has been established that robot faces that display really accurate human emotions freak people out. It is very disconcerting to have an inanimate but artificially intelligent object to mimic what is the most basic human attribute - emotion. Rather than trust the robot more, it actually leads to greater mistrust and discomfort.

Personally, I would regard a robot smiling at me with exactly the same trepidation as that of a used car salesperson giving me the 'trust me' smile, or of the clown with that unsettling painted-on grin.

It was much easier to trust the answers from Data of Star Trek fame because they were delivered by a very impartial, non-emotional robotic characterization.

In the movie 'I, Robot' the human connection of Sunny was much more believable BECAUSE he did not display artificial emotions. In fact, Sunny's face was so neutral that the human observer could superimpose their OWN image of emotional expression, that somehow was more appealing than to have Sunny display the emotion. That is, we saw what we believed to be there, not what the artificial intelligence wanted us to see.

Add to this, the fact that these robots will have a hard time being accepted in the first place, and there would already be an element of mistrust, any false impressions that one got from misreading an artificial expression that is dissonant with the message or intent would lead to even greater mistrust and mental tension from cognitive dissonance.

The research into voice responses from our cars, devices, and other voice response technology supports this notion. They all have a bland, neutral tone for the same reasons. People don't WANT a bubbly, upbeat, almost laughing Siri. They don't want any kind of tone or inflection that indicates or that could be interpreted as indicating judgement or acceptance or agreement/disagreement. They want 'just the facts, ma'am' with just enough inflection to make it sound natural.

My bet would be to go with robots having completely neutral expressions, with just the right design that lets the human observer imply their own expected emotion onto the face.

Answer 8

Why bother creating AI when the biological machine is sufficient?

Expressions one could say are also a congantive function of self identity and self awareness. The very thing AI programmers delve volume of topics to unravel. The idea of a fully functioning brain requires all the parts and mechanisms that go along with it. Facial expressions and gestures for example have over thousands of receptors telling our millions of neurons what we are doing with our indvidual muscles. This simple thing like cracking a smile follows a very deep set of instructions if you think about it.