Body language can also be heard

This shows a woman
It’s all about acoustics: the pitch and volume of a voice change together with the movement of arms and hands. Image is in the public domain.

Summary: The human voice contains information about dynamic bodily states. The findings shed light on the evolution of both spoken language and non-verbal communication.

Source: Donders Institute

When people move their hands and arms while using their voices, listeners are able to hear it. Even without seeing the messenger, we can pick up each other’s body language. Wim Pouw of the Donders Institute has published this finding in PNAS this week.

Most people gesture while talking. From a small wrist movement to complete sign language involving arms, hands and fingers. This form of non-verbal communication supports what someone says, and, in some cases, it is even indispensable in order to explain something properly.

This is difficult if you cannot see the other person, you’d think. But even during phone calls, or when shouting from another room, we often talk with our hands. And that is not pointless at all, according to the study conducted by Wim Pouw and his colleagues from the University of Connecticut: your gestures resonate in your voice.

Louder and higher

It’s all about acoustics: the pitch and volume of a voice change together with the movement of arms and hands. “That change is very subtle with a wrist movement,” says Pouw. “It is less subtle with an arm movement. The pitch jumps up slightly whenever a movement slows down.”

According to Pouw, there are two different causes for these acoustic differences. One of the causes is the creation of vibrations. The forces involved in a movement cause vibrations in your body. Through the connective tissue that holds your body together, vibrations end up in your lungs, affecting the pressure in your lungs.

The second cause is muscle tension around your lungs that is needed to maintain balance. We do not merely use our arm muscles when we move our arms. “When starting the process of stopping your arm from moving, for example, other muscles are suddenly addressed to prevent your body from falling over. These muscles that maintain balance include muscles around your lungs.”

Unconsciously hearing an arm movement

As part of this study, Pouw instructed 3 men and 3 women to make a monotone sound, such as ‘aaaa’, while using all kinds of different hand and arm movements. After that, 30 subjects were asked to listen to the recordings. Not only did they guess which movements were made, but, in many cases, they were also able to mimic these movements simultaneously.

It is easy to measure differences in pitch and volume. It is remarkable that listeners unconsciously identify at what point movement causes these differences. “The subjects not only picked up the speed of the movement, but they also heard the location of the movement.”

Emphasize words by hand

Hence, a voice is more than just an abstract collection of sounds, according to the researcher, “When you hear a voice, you literally hear aspects of a person’s entire body.”

These findings go against the assumption that gestures basically only serve to depict or point out something. “It contributes to the understanding that there is a closer relationship between spoken language and gestures. Hand gestures may have been created to support the voice, to emphasize words, for example.”

Google Home

The insights contribute to knowledge about ourselves and to knowledge about speech recognition. Think of systems such as Google Home and Siri. “When developing speech recognition, we have to take movement into account. Think of gestures, but also of someone running while talking, for example.”

By means of information about changes in voice, the way how systems can interpret meanings and weights of words can be improved. Or the other way around: noises in a voice produced by a speaker while running, does not affect the meaning of a word. “We can teach these kinds of systems what they should or should not filter out.”

About this neuroscience research article

Source:
Donders Institute
Media Contacts:
Press Office – Donders Institute
Image Source:
The image is in the public domain.

Original Research: Closed access
“Acoustic information about upper limb movement in voicing”. by Wim Pouw, Alexandra Paxton, Steven J. Harrison, and James A. Dixon.
PNAS doi:10.1073/pnas.2004163117

Abstract

Acoustic information about upper limb movement in voicing

We show that the human voice has complex acoustic qualities that are directly coupled to peripheral musculoskeletal tensioning of the body, such as subtle wrist movements. In this study, human vocalizers produced a steady-state vocalization while rhythmically moving the wrist or the arm at different tempos. Although listeners could only hear and not see the vocalizer, they were able to completely synchronize their own rhythmic wrist or arm movement with the movement of the vocalizer which they perceived in the voice acoustics. This study corroborates recent evidence suggesting that the human voice is constrained by bodily tensioning affecting the respiratory–vocal system. The current results show that the human voice contains a bodily imprint that is directly informative for the interpersonal perception of another’s dynamic physical states.

Feel Free To Share This Neuroscience News.

Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.