We Can Read Each Other’s Emotions From Subtle Changes in Facial Color

Summary: Researchers reveal people are able to correctly identify, with 75% accuracy, expressions of emotion in others based on subtle changes in color around the nose, eyebrows and chin.

Source: Ohio State University.

Our faces broadcast our feelings in living color—even when we don’t move a muscle.

That’s the conclusion of a groundbreaking study into human expressions of emotion, which found that people are able to correctly identify other people’s feelings up to 75 percent of the time—based solely on subtle shifts in blood flow color around the nose, eyebrows, cheeks or chin.

The study, published in the Proceedings of the National Academy of Sciences this week, demonstrates a never-before-documented connection between the central nervous system and emotional expression in the face. It also enabled researchers to construct computer algorithms that correctly recognize human emotion via face color up to 90 percent of the time.

“We identified patterns of facial coloring that are unique to every emotion we studied,” said Aleix Martinez, cognitive scientist and professor of electrical and computer engineering at The Ohio State University.

“We believe these color patterns are due to subtle changes in blood flow or blood composition triggered by the central nervous system. Not only do we perceive these changes in facial color, but we use them to correctly identify how other people are feeling, whether we do it consciously or not.”

The researchers are patenting the computer algorithms, and hope they will enable future forms of artificial intelligence to recognize and emulate human emotions. They have also formed a spin-off company, Online Emotion, to commercialize the research.

This is the latest in a series of studies in which Martinez and his team have identified unique forms of human facial expression. In prior work, they identified several, previously unknown facial expressions produced though unique patterns of muscle movements, including the “not face,” a frown that they determined to be a universal embodiment of negation in human communication. What’s unusual about their latest work is that it involves color changes that communicate emotion without any movement of facial muscles.

For this study, the researchers first took hundreds of pictures of facial expressions and separated the images into different color channels that correspond to how human eyes see color—a red-green channel and a blue-yellow channel. Via computer analysis, they found that emotions like “happy” or “sad” formed unique color patterns.

Regardless of gender, ethnicity or overall skin tone, everybody displayed similar patterns when expressing the same emotion.

To test whether colors alone could convey emotions—without smiles or frowns to go along with them—the researchers superimposed the different emotional color patterns on pictures of faces with neutral expressions. They showed the neutral faces to 20 study participants and asked them to guess how the person in the picture was feeling, choosing from a list of 18 emotions. The emotions included basic ones like “happy” and “sad” as well as more complex ones such as “sadly angry” or “happily surprised.”

“Admittedly, these images look weird,” Martinez said. “But we told people to go ahead and guess from the list of emotions what emotion they thought those faces were conveying. And they guessed right most of the time.”

About 70 percent of the time, participants thought that a neutral face that had been colorized to look happy actually conveyed happiness. They thought faces colorized to look sad were actually sad about 75 percent of the time, and neutral faces colorized to look angry were actually angry about 65 percent of the time. They perceived the emotion, even though their only clue was the color superimposed on the face, with no facial movements.

Next, researchers showed participants facial expressions of happiness, sadness and other emotions. This time, however, they mixed up the colors on some of the images. For example, they sometimes took a happy face and put angry colors on it, or vice versa. Participants noticed that something about the mixed up images looked “off,” even if they weren’t sure what was wrong.

“Participants could clearly identify which images had the congruent versus the incongruent colors,” Martinez said.

The researchers used what they learned to develop computer algorithms that could detect emotions via face color. Given photographs of people expressing emotion, the computer could match face color to feeling better than the human study participants could.

Happiness was the easiest emotion for the computer to recognize by color alone, and it detected the emotion with 90 percent accuracy. Several emotions related to happiness, such as “happily surprised,” came in second at around 85 percent. Anger was detectable by color alone 80 percent of the time, and sadness 75 percent of the time. Fear was recognizable 70 percent of the time, and the least recognizable emotion was “fearfully disgusted,” which registered only 65 percent of the time.

The findings may ultimately inform research in computer science, cognition, neuroscience and even human evolution.

Language is replete with idioms that equate face color to emotion. When we argue until we’re “blue in the face,” we’re angry. If we look “green around the gills,” something has triggered our disgust. “Blushing brides” are the very picture of happiness.

This study shows that there is some physiological truth to these old sayings, though the color scheme of human emotion is not as simple as a monochromatic blue, green or red.

“There’s a little bit of every color everywhere,” Martinez said. Touches of red, green, blue and yellow characterize every emotion—just in slightly different amounts or locations around the face.

Disgust, for instance, creates a blue-yellow cast around the lips, but with a red-green cast around the nose and forehead.

It’s all the more impressive, then, that our brains are able to decipher the meaning of these color arrangements in an instant. We see a smiling person with red cheeks and temples (with a little blue around the chin) and we automatically read their emotion as “happy.” But the same face with a slightly redder forehead and slightly less blue chin registers as “surprised.”

Humans just may be alone among primates for our ability to change facial color due to emotion. It’s hard to tell, because other primates’ faces are covered by hair.

happy face
This composite image shows a happy face as it was originally photographed (left) and the same photo retouched to emphasize changes in color (right). Researchers at The Ohio State University found that people are able to correctly identify other people’s feelings up to 75 percent of the time—based solely on subtle shifts in blood flow color around the nose, eyebrows, cheeks or chin.The study demonstrates a never-before-documented connection between the central nervous system and emotional expression in the face. NeuroscienceNews.com image is credited to Ohio State University.

Yet, in humans, nowhere else on the body are there so many blood vessels so close to the surface of the skin than our faces. The fact that we evolved much less facial hair than the apes suggests that our early ancestors may have found some advantage to letting their blushes show.

What about fake blushes—ones created with makeup?

“People have always said that we use makeup to look beautiful or younger, but I think that it is possible that we actually do it to appear happier or create a positive perception of emotion—or a negative perception, if you wanted to do that,” Martinez said.

Now that we know a little more about the varied colors of human facial expression, he suggested an intriguing new possibility: We could make “smart” cosmetics to play up certain emotions—or conceal them.

About this neuroscience research article

Martinez’s co-authors on the study included C. Fabian Benitez-Quiroz, a postdoctoral researcher, and Ramprakash Srinivasan, a doctoral student, both in electrical and computer engineering. All three hold appointments in Ohio State’s Center for Cognitive and Brain Sciences.

Funding: Their research was funded by the National Institutes of Health and the Human Frontier Science Program.

Source: Pam Frost Gorder – Ohio State University
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com image is credited to Ohio State University.
Original Research: Open access research in PNAS.

Cite This NeuroscienceNews.com Article

[cbtabs][cbtab title=”MLA”]Ohio State University “We Can Read Each Other’s Emotions From Subtle Changes in Facial Color.” NeuroscienceNews. NeuroscienceNews, 19 March 2018.
<https://neurosciencenews.com/emotion-face-color-8665/>.[/cbtab][cbtab title=”APA”]Ohio State University (2018, March 19). We Can Read Each Other’s Emotions From Subtle Changes in Facial Color. NeuroscienceNews. Retrieved March 19, 2018 from https://neurosciencenews.com/emotion-face-color-8665/[/cbtab][cbtab title=”Chicago”]Ohio State University “We Can Read Each Other’s Emotions From Subtle Changes in Facial Color.” https://neurosciencenews.com/emotion-face-color-8665/ (accessed March 19, 2018).[/cbtab][/cbtabs]


Facial color is an efficient mechanism to visually transmit emotion

Facial expressions of emotion in humans are believed to be produced by contracting one’s facial muscles, generally called action units. However, the surface of the face is also innervated with a large network of blood vessels. Blood flow variations in these vessels yield visible color changes on the face. Here, we study the hypothesis that these visible facial colors allow observers to successfully transmit and visually interpret emotion even in the absence of facial muscle activation. To study this hypothesis, we address the following two questions. Are observable facial colors consistent within and differential between emotion categories and positive vs. negative valence? And does the human visual system use these facial colors to decode emotion from faces? These questions suggest the existence of an important, unexplored mechanism of the production of facial expressions of emotion by a sender and their visual interpretation by an observer. The results of our studies provide evidence in favor of our hypothesis. We show that people successfully decode emotion using these color features, even in the absence of any facial muscle activation. We also demonstrate that this color signal is independent from that provided by facial muscle movements. These results support a revised model of the production and perception of facial expressions of emotion where facial color is an effective mechanism to visually transmit and decode emotion.

Feel free to share this Ohio State University.
Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.