News

Facial expressions don t tell the whole story of emotion

Interacting with other people is almost always a game of reading cues and volleying back. We think a smile conveys happiness, so we offer a smile in return. We think a frown shows sadness, and maybe we attempt to cheer that person up.

advertisement

Some businesses are even working on technology to determine customer satisfaction through facial expressions.

But facial expressions might not be reliable indicators of emotion, research indicates. In fact, it might be more accurate to say we should never trust a person s face, new research suggests.

"The question we really asked is: Can we truly detect emotion from facial articulations? " said Aleix Martinez, a professor of electrical and computer engineering at The Ohio State University.

"And the basic conclusion is, no, you can t."

Martinez, whose work has focused on building computer algorithms that analyze facial expressions, and his colleagues presented their findings today (Feb. 16, 2020) at the annual meeting of the American Association for the Advancement of Science in Seattle.

advertisement

The researchers analyzed the kinetics of muscle movement in the human face and compared those muscle movements with a person s emotions. They found that attempts to detect or define emotions based on a person s facial expressions were almost always wrong.

"Everyone makes different facial expressions based on context and cultural background," Martinez said. "And it s important to realize that not everyone who smiles is happy. Not everyone who is happy smiles. I would even go to the extreme of saying most people who do not smile are not necessarily unhappy. And if you are happy for a whole day, you don t go walking down the street with a smile on your face. You re just happy."

It is also true, Martinez said, that sometimes, people smile out of an obligation to the social norms. This would not inherently be a problem, he said -- people are certainly entitled to put on a smile for the rest of the world -- but some companies have begun developing technology to recognize facial muscle movements and assign emotion or intent to those movements.

The research group that presented at AAAS analyzed some of those technologies and, Martinez said, largely found them lacking.

"Some claim they can detect whether someone is guilty of a crime or not, or whether a student is paying attention in class, or whether a customer is satisfied after a purchase," he said. "What our research showed is that those claims are complete baloney. There s no way you can determine those things. And worse, it can be dangerous."

The danger, Martinez said, lies in the possibility of missing the real emotion or intent in another person, and then making decisions about that person s future or abilities.

advertisement

For example, consider a classroom environment, and a teacher who assumes that a student is not paying attention because of the expression on the student s face. The teacher might expect the student to smile and nod along if the student is paying attention. But maybe that student, for reasons the teacher doesn t understand -- cultural reasons, perhaps, or contextual ones -- is listening intently, but not smiling at all. It would be, Martinez argues, wrong for the teacher to dismiss that student because of the student s facial expressions.

After analyzing data about facial expressions and emotion, the research team -- which included scientists from Northeastern University, the California Institute of Technology and the University of Wisconsin -- concluded that it takes more than expressions to correctly detect emotion.

Facial color, for example, can help provide clues.

"What we showed is that when you experience emotion, your brain releases peptides -- mostly hormones -- that change the blood flow and blood composition, and because the face is inundated with these peptides, it changes color," Martinez said.

The human body offers other hints, too, he said: body posture, for example. And context plays a crucial role as well.

In one experiment, Martinez showed study participants a picture cropped to display just a man s face. The man s mouth is open in an apparent scream; his face is bright red.

"When people looked at it, they would think, wow, this guy is super annoyed, or really mad at something, that he s angry and shouting," Martinez said. "But when participants saw the whole image, they saw that it was a soccer player who was celebrating a goal."

In context, it s clear the man is very happy. But isolate his face, Martinez said, and he appears almost dangerous.

Cultural biases play a role, too.

"In the U.S., we tend to smile a lot," Martinez said. "We are just being friendly. But in other cultures, that means different things -- in some cultures, if you walked around the supermarket smiling at everyone, you might get smacked."

Martinez said the research group s findings could indicate that people -- from hiring managers to professors to criminal justice experts -- should consider more than just a facial expression when they evaluate another person.

And while Martinez said he is "a big believer" in developing computer algorithms that try to understand social cues and the intent of a person, he added that two things are important to know about that technology.

"One is you are never going to get 100 percent accuracy," he said. "And the second is that deciphering a person s intent goes beyond their facial expression, and it s important that people -- and the computer algorithms they create -- understand that."

Materials provided by Ohio State University . Original written by Laura Arenschield. Note: Content may be edited for style and length.

Ohio State University. "Facial expressions don t tell the whole story of emotion: Researchers warn of drawing too-quick conclusions about people s feelings." ScienceDaily. ScienceDaily, 16 February 2020. .

Ohio State University. "Facial expressions don t tell the whole story of emotion: Researchers warn of drawing too-quick conclusions about people s feelings." ScienceDaily. www.sciencedaily.com/releases/2020/02/200216184520.htm (accessed February 17, 2020).
Read more on sciencedaily.com
News Topics :
Similar Articles :
Technology
Software that purportedly reads emotions in faces is being deployed or tested for a variety of purposes, including surveillance, hiring, clinical diagnosis, and market research. But a new scientific report...
Science
Disgust needs just one facial expression to get its point across throughout the world. Happiness, on the other hand, has 17 a testament to the many varied forms of...
Science
Close Researchers at Western University have shown that our brains are pre wired to perceive wrinkles around the eyes as conveying more intense and more sincere emotions. This eye wrinkle feature, called...
Technology
Most algorithms have probably never heard the Eagles song, Lyin Eyes. Otherwise, they d do a better job of recognizing duplicity. advertisement Computers aren t very good at...
Science
Facial expressions are important parts of how we communicate and how we develop impressions of the people around us. In Charles Darwin proposed that facial expressions evolved to quickly...