Summary: A new study reveals imagining visual stimuli can alter our perception of where auditory stimuli may be coming from.
Source: APS.
Seeing an object at the same time that you hear sound coming from somewhere else can lead to the “ventriloquist illusion” and its aftereffect, but research suggests that simply imagining the object produces the same illusory results. The findings are published in Psychological Science.
“The sensory information we imagine is often treated by the brain in the same way as information streaming in to us from the outside world,” says researcher Christopher C. Berger of the California Institute of Technology. “Our work shows that what we imagine in our ‘mind’s eye’ can lead to changes in perception across our sensory systems, changing how we perceive real information from the world around us in the future.”
The brain is constantly integrating information from multiple senses to produce a coherent experience of the world. This process is seamless and adaptive — for example, in the ventriloquist illusion, the perceived location of a sound shifts toward the location of visual stimulus if they are presented at the same time. And if we experience this pairing repeatedly, the illusion will persist as an aftereffect even when the visual stimulus is gone and we only hear the sound.
We might like to believe that sights and sounds in the real world is categorically different from imagining them – however, brain imaging data suggest that the distinction between real and imagined isn’t quite so clear. Given this overlap, Berger and coauthor H. Henrik Ehrsson hypothesized that the ventriloquism aftereffect would occur with stimuli that are both seen and imagined. They tested their hypothesis in a series of six experiments.
In an adaptation phase, participants imagined a circle appearing in a specific location on screen at the same time that they heard white noise coming from one of three locations behind the screen (e.g., left, center, right). In the test phase, they heard bursts of white noise emanating from randomly assigned locations and indicated whether the noise came from the left- or right-hand side of the screen.
The results showed that imagining a circle simultaneously with hearing sounds was enough to induce the ventriloquism aftereffect — participants’ responses in the test phase were biased in accordance with the specific imagery-sound pairing they experienced during the adaptation phase.
“We were surprised to find that the effects on participants’ perception of acoustic space were almost as strong for imagined stimuli as they were for real visual stimuli,” says Berger. “That is, what we imagine seeing can affect our future perception of sound as much as what we actually see.”
For the aftereffect to emerge, however, the sounds presented in the two phases must be consistent. In four experiments, the ventriloquism aftereffect emerged in response to both mental imagery and visual stimuli, but only when the adaptation and test phase featured the same tone. The aftereffect did not appear when the adaptation phase featured a tone and the test phase featured white noise.
Berger and Ehrsson hope to continue investigating the overlap in processing between mental imagery and visual perception. They note that research in this area may eventually have applications in a variety of domains, including in rehabilitation following injury or stroke and the development of brain-computer interfaces and neural prostheses.
Funding: This research was supported by Riksbankens Jubileumsfond, the James S. McDonnell Foundation, the Swedish Research Council, and Torsten Söderbergs Stiftelse. C. C. Berger was supported by a grant from the Swedish Research Council (2017-00276).
Source: Anna Mikulak – APS
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com image is adapted from the APS news release.
Original Research: Abstract for “Mental Imagery Induces Cross-Modal Sensory Plasticity and Changes Future Auditory Perception” by Christopher C. Berger, and H. Henrik Ehrsson in Psychological Science. Published April 10 2018,
doi:10.1177/0956797617748959
[cbtabs][cbtab title=”MLA”]APS “Imagining an Object Can Change How We Hear Sounds Later.” NeuroscienceNews. NeuroscienceNews, 11 April 2018.
<https://neurosciencenews.com/imagination-sound-8773/>.[/cbtab][cbtab title=”APA”]APS (2018, April 11). Imagining an Object Can Change How We Hear Sounds Later. NeuroscienceNews. Retrieved April 11, 2018 from https://neurosciencenews.com/imagination-sound-8773/[/cbtab][cbtab title=”Chicago”]APS “Imagining an Object Can Change How We Hear Sounds Later.” https://neurosciencenews.com/imagination-sound-8773/ (accessed April 11, 2018).[/cbtab][/cbtabs]
Abstract
Mental Imagery Induces Cross-Modal Sensory Plasticity and Changes Future Auditory Perception
Can what we imagine in our minds change how we perceive the world in the future? A continuous process of multisensory integration and recalibration is responsible for maintaining a correspondence between the senses (e.g., vision, touch, audition) and, ultimately, a stable and coherent perception of our environment. This process depends on the plasticity of our sensory systems. The so-called ventriloquism aftereffect—a shift in the perceived localization of sounds presented alone after repeated exposure to spatially mismatched auditory and visual stimuli—is a clear example of this type of plasticity in the audiovisual domain. In a series of six studies with 24 participants each, we investigated an imagery-induced ventriloquism aftereffect in which imagining a visual stimulus elicits the same frequency-specific auditory aftereffect as actually seeing one. These results demonstrate that mental imagery can recalibrate the senses and induce the same cross-modal sensory plasticity as real sensory stimuli.
