When people talk or sing, they often nod, tilt or bow their heads to reinforce verbal messages. But how effective are these head gestures at conveying emotions?
Very effective, according to researchers from McGill University in Montreal. Steven R. Livingstone and Caroline Palmer, from McGill’s Department of Psychology, found that people were highly accurate at judging emotions based on head movements alone, even in the absence of sound or facial expressions.
This finding suggests that visual information about emotional states available in head movements could aid in the development of automated emotion recognition systems or human-interaction robots, the researchers say. Expressive robots could potentially serve a range of functions, particularly where face-to-face communication is important, such as at hotel reception desks and as interactive care robots for the elderly.
Tracking movement, not sound
Using motion-capture equipment to track people’s head movements in three dimensions, Livingstone and Palmer recorded vocalists while they spoke or sang with a variety of emotions. The researchers then presented these video clips to viewers without any sound, with the facial expressions of vocalists hidden so that only their head movements were visible. Viewers were then asked to identify the emotions that the vocalists intended to convey.
“We found that when people talk, the ways in which they move their head reveal the emotions that they’re expressing. We also found that people are remarkably accurate at identifying a speaker’s emotion, just by seeing their head movements,” says Palmer, who holds the Canada Research Chair in Cognitive Neuroscience of Performance.
Research idea emerged from a noisy pub
“While the head movements for happy and sad emotions differed, they were highly similar across speech and song, despite differences in vocal acoustics”, says Livingstone, a former postdoctoral fellow in the Palmer lab and now a postdoctoral fellow at McMaster University. “Although the research was based on North American English speakers, the focus on head movements creates the possibility for studying emotional communication in contexts where different languages are spoken”.
The idea for the study emerged from a noisy pub. “One night in Montreal I was in a bar with my lab mates”, explains Livingstone, “It was a lively evening, with lots of people, dim lights, and some very loud music. At one point my friend started to talk me; I knew he was excited though I couldn’t make out what he was saying or see his face clearly. Suddenly I realized it was the animated way that he was bobbing his head that told me what he was trying to say”.
Adds Palmer, “Our discovery may lead to new applications in situations where sound is not available, such as automated recognition of emotional states in crowd behavior or in hearing impairments, by making use of head movements when watching someone talk. It also has applications in computing and robotics, where the addition of expressive head movements may help make humanoid robots more lifelike and approachable.”
Funding: This research was funded in part by an NSERC-CREATE Postdoctoral Fellowship to Dr. Steven R. Livingstone and by a Canada Research Chair and NSERC Grant to Dr. Caroline Palmer.
Source: Chris Chipello – McGill University
Image Credit: The image is adapted from the McGill University press release.
Video Source: The video is credited to Sequence Production Lab, Dept of Psychology, McGill University and is available via YouTube
Original Research: Abstract for “Head Movements Encode Emotions During Speech and Song” by Livingstone, Steven R. and Palmer, Caroline in Emotion. Published online October 26 2015 doi:10.1037/emo0000106
Abstract
Head Movements Encode Emotions During Speech and Song
When speaking or singing, vocalists often move their heads in an expressive fashion, yet the influence of emotion on vocalists’ head motion is unknown. Using a comparative speech/song task, we examined whether vocalists’ intended emotions influence head movements and whether those movements influence the perceived emotion. In Experiment 1, vocalists were recorded with motion capture while speaking and singing each statement with different emotional intentions (very happy, happy, neutral, sad, very sad). Functional data analyses showed that head movements differed in translational and rotational displacement across emotional intentions, yet were similar across speech and song, transcending differences in F0 (varied freely in speech, fixed in song) and lexical variability. Head motion specific to emotional state occurred before and after vocalizations, as well as during sound production, confirming that some aspects of movement were not simply a by-product of sound production. In Experiment 2, observers accurately identified vocalists’ intended emotion on the basis of silent, face-occluded videos of head movements during speech and song. These results provide the first evidence that head movements encode a vocalist’s emotional intent and that observers decode emotional information from these movements. We discuss implications for models of head motion during vocalizations and applied outcomes in social robotics and automated emotion recognition.
“Head Movements Encode Emotions During Speech and Song” by Livingstone, Steven R. and Palmer, Caroline in Emotion. Published online October 26 2015 doi:10.1037/emo0000106
