How sounds, shapes, speech and body movements convey emotion through one shared property

Summary: Shapes, sounds, body language, and speech are universal cues to emotional arousal.

Source: Dartmouth College

Death metal band logos often have a spiky look while romance novel titles often have a swirly script. The jaggedness or curviness of a font can be used to express an emotional tone. A Dartmouth study published in the Proceedings of the Royal Society B finds that sounds, shapes, speech and body movements convey emotional arousal the same way across the senses. The findings explain why nearly anything can have an emotional tone, including art, architecture, and music.

“Our study set out to better understand how we express and read emotional arousal, which is fundamental to our core emotional state. We wanted to see if there is a low-level mechanism that allows us to decode emotional arousal information from the movements and sounds that people and animals make,” says lead author Beau Sievers, a postdoctoral student of psychology at Harvard University, who was a graduate student in the department of psychological and brain sciences at Dartmouth at the time of the study.

“Our results show how the spectral centroid, or the balance of high-frequency versus low-frequency energy present in sounds, shapes, and movements, allows us to express and understand emotional arousal,” adds Sievers.

The spectral centroid is essentially a multi-sensory measure of spikyness. The results explain why Zen gardens and brutalist architecture have very different emotional effects, as well as why things like clouds and lullabies seem to go together even though one is seen and the other is heard: We match them based on the spectral centroid.

A pointy shape here represents anger, while a blobby shape represents sadness
Characteristic angry and sad drawings from study 3, after smoothing and corner detection. Corners are marked with red ‘+’ signs. Among the study’s participants, angry drawings had a mean of 23.3 corners, while sad drawings had a mean of 6.6 corners. The image is credited to Beau Sievers et al.

“In a series of studies, we demonstrate that people automatically perceive the frequency spectrum of whatever is coming into their ears and eyes and compute the average–the spectral centroid,” explains senior author Thalia Wheatley, a professor of psychological and brain sciences at Dartmouth, and principal investigator of the Dartmouth Social Systems Laboratory . “This is how people quickly identify the amount of emotional arousal in a person’s voices and movements but also in abstract shapes and sounds, such as why spiky shapes seem to convey higher arousal than rounded shapes,” she added.

To test whether the spectral centroid is used to express and understand emotional arousal, the researchers conducted five mini-studies, some of which asked participants to make judgements about the emotional arousal of shapes, sounds, and movements. The researchers tested if the spectral centroid of the stimulus could be used to predict participants’ emotional arousal judgements. The following are highlights from three of the mini-studies:

  • The authors used a computer program to randomly create hundreds of shapes and sounds. Participants were asked to look at shapes and listen to sounds and judge their levels of emotional arousal. As the study reports, shapes and sounds which had a high spectral centroid were associated with high-arousal emotions (angry, excited), whereas the lower spectral centroid shapes and sounds were associated with low-arousal emotions (sad, peaceful).
  • Participants were asked to draw shapes that were angry, sad, excited, or peaceful. The researchers then estimated the spectral centroids of the drawings by counting how many corners they had. The results revealed that angry and excited shapes had between 17 and 24 corners on average, while sad and peaceful shapes had between 7 and 9 corners on average. The spectral centroid could be used to predict the emotional arousal of shapes with close to 80 percent accuracy.
  • The researchers examined real-world recordings of people’s body movements or of people speaking (in German), to see if the spectral centroid of the voices and movements could be used to predict participants judgements of emotional arousal. The researchers found that higher spectral centroids predicted judgements of higher emotional arousal.

The researchers explain that multi-sensory associations with emotions have been known for a long time but why they occur has been a mystery until now.

About this neuroscience research article

Dartmouth College
Media Contacts:
Beau Sievers – Dartmouth College
Image Source:
The image is credited to Beau Sievers et al.

Original Research: Closed access
“A multi-sensory code for emotional arousal”. Beau Sievers , Caitlyn Lee , William Haslett and Thalia Wheatley.
Proceedings of the Royal Society B. doi:10.1098/rspb.2019.0513


A multi-sensory code for emotional arousal

People express emotion using their voice, face and movement, as well as through abstract forms as in art, architecture and music. The structure of these expressions often seems intuitively linked to its meaning: romantic poetry is written in flowery curlicues, while the logos of death metal bands use spiky script. Here, we show that these associations are universally understood because they are signalled using a multi-sensory code for emotional arousal. Specifically, variation in the central tendency of the frequency spectrum of a stimulus—its spectral centroid—is used by signal senders to express emotional arousal, and by signal receivers to make emotional arousal judgements. We show that this code is used across sounds, shapes, speech and human body movements, providing a strong multi-sensory signal that can be used to efficiently estimate an agent’s level of emotional arousal.

Feel free to share this Psychology News.
Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.