You say that humans, in general, are poor at recognizing facial expressions. As a person with both autism and a brain injury, I beg to differ. I am poor at both recognizing and using facial expressions (I even have a doctor's note!), and the differences are observable to an outsider.
A few years back, before the brain injury and before we knew I was autistic, my family used to play with the Society for Creative Anachronism (SCA). For those not familiar with the SCA, think of them as adults playing dressup in medieval garb, who get together to gossip, drink beer, and beat on each other with big sticks. For the Brits out there, think of stereotypical rugby players dressing up as King Arthur's knights, then playing rugby with swords (that event was called "Blood of Heroes", by the way, emulating a very bad movie of the same name).
Naturally, there is a lot of banter in such a group, and a lot of good natured roughhousing. They can find a week's worth of innuendo in the world innuendo. Shortly after we moved on to other hobbies, my wife told me that I used to scare them because, on the rare occasions I did join the banter, they couldn't tell whether I was serious or joking.
Post brain injury, it is much worse. Recent;y, when being introduced to a new business partner, they were trying to be friendly and informally ask me about myself. Unfortunately, they asked "What gets you out of bed in the morning?". A metaphorical question about emotional state with social cues, it effectively and awkwardly put an end to the conversation because it exactly hit places I physically lack the machinery to process efficiently.
My wife has learned to be very careful with the use of sarcasm and metaphors around me because, without the social cues that go with them, I tend to take them literally. "Why did you (some action) ?" "You Told me to." "I didn't mean for you to really do that!" "Sorry."
A robot mimicking humanoid characteristics to the point they were indistinguishable from humans, without emotional/social emulation of some sort, would tend to make people feel uneasy and awkward, or as my son would say "it would creep them out".
Worse than not mimicking emotions, would be mimicking them badly. A smile whose timing is off by 1/4 second is perceived as deceptive, for example. In contrast, obviously different emotional responses that are specific to the species (robots) can be learned by humans. I used to keep crustaceans in my aquarium, and it is amazing how expressive crabs and crayfish be without flexible faces.
For a good idea of the issues you might run into, watch the first season of Ninjago and the team's interactions with Zane, before anyone knows he is a robot.