Summary: For over 20 percent of people, visual motion can affect auditory perceptions. A new study reveals over one fifth of people who watched a silent video experienced the illusion of sound.
Source: BPS Digest.
If you have a couple of minutes, click through to this survey site of “noisy gifs” – brief silent movies that, for some people at least, evoke illusory sounds. If you hear a thwack when fists collide with a punchbag, or a yell while watching a man silently scream, then you’re experiencing a “visual-evoked auditory response” (vEAR), also called “hearing motion synaesthesia”.
Ten years after the first, preliminary journal paper on the phenomenon, Christopher Fassnidge and Elliot Freeman at City University, London, report – in a new paper in Cortex – that it’s remarkably common, affecting perhaps 20 – 30 per cent of us. Fassnidge and Freeman also investigated what induces vEAR, providing clues to what’s going on the brain.
More than 4,000 volunteers, plus 126 paid participants, viewed 24, 5-second-long silent videos of real world scenarios and also more abstract images, such as shifting patterns. Using a scale of 0 to 5, they indicated how much auditory sensation they experienced for each video. They also answered a series of other questions, including about past experience of vEAR and of any other synesthesia (when a perception via one sensory modality triggers a sensory perception in another modality).
As the unpaid volunteers were recruited via adverts with text such as “Do you experience ‘hearing motion?’”, it’s certainly possible that there was a self-selection bias. But, on enrolment, the paid participants did not know what the study was about, so, in theory, they should be more representative of the general population. Thirty-one per cent of this paid group (an even higher percentage, in fact, than in the bigger, unpaid group) reported past experience of vEAR.
When it came to the survey results, anyone who rated half of the videos at greater than or equal to 3 was identified as experiencing vEAR. Just over 20 per cent of the paid participants fell into this category. Taken with the self reports of past experience of vEAR, the findings suggest that the phenomenon is far from rare.
The higher-rated videos often depicted relatively familiar events that are reliably associated with particular sounds (like fists hitting a punchbag), suggesting that an understanding of what’s happening in the scene was involved in causing the illusory sounds.
However, in people who reported prior experience of vEAR, even videos containing abstract flickering or moving patterns could trigger sound perceptions.
Fassnidge and Freeman found that these videos had high levels of raw “motion energy” (imagine a flickering neon shop sign, for instance, compared with a silent video of a person screaming, in which case there’s very little movement or “motion energy”).
People who hear silent videos are more likely to report other synesthesia, including seeing flashes upon hearing sounds in the dark. Credit: Joey Graceffa
This specific sensitivity to motion energy suggests, the researchers write, that in some people, at least, visual motion – rather than the meaning of a scene – affects auditory perceptions directly.
Given that participants with prior experience of vEAR were also more likely to report other synesthesia, including seeing flashes in reaction to hearing sudden sounds in the dark, it’s possible the two phenomena are related. Synaesthesia has been associated with higher than normal cortical “excitability”, for instance, and the researchers said cortical excitability might also be involved in vEAR.
This is something that could now be investigated, the researchers write, before concluding: “Given the prevalence of visual-evoked auditory sensations and our new ability to quantify and correlate them, a potentially broad class of related subjective audiovisual phenomena have now become more accessible to scientific study.”
Source: Emma Young – BPS Digest
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com image is credited to giphy.com.
Original Research: Abstract for “Sounds from seeing silent motion: Who hears them, and what looks loudest?” by Christopher J.Fassnidge and Elliot D.Freeman in Cortex. Published March 9 2018.
Sounds from seeing silent motion: Who hears them, and what looks loudest?
Some people hear what they see: car indicator lights, flashing neon shop signs, and people’s movements as they walk may all trigger an auditory sensation, which we call the visual-evoked auditory response (vEAR or ‘visual ear’). We have conducted the first large-scale online survey (N > 4000) of this little-known phenomenon. We analysed the prevalence of vEAR, what induces it, and what other traits are associated with it.
We assessed prevalence by asking whether respondents had previously experienced vEAR. Participants then rated silent videos for vividness of evoked auditory sensations, and answered additional trait questions.
Prevalence appeared higher relative to other typical synesthesia. Prior awareness and video ratings were associated with greater frequency of other synesthesia, including flashes evoked by sounds, and musical imagery. Higher-rated videos often depicted meaningful events that predicted sounds (e.g., collisions). However, even videos containing abstract flickering or moving patterns could also elicit higher ratings, despite having no predictable association with sounds. Such videos had higher levels of raw ‘motion energy’ (ME), which we quantified using a simple computational model of motion processing in early visual cortex. Critically, only respondents reporting prior awareness of vEAR tended to show a positive correlation between video ratings and ME.
This specific sensitivity to ME suggests that in vEAR, signals from visual motion processing may affect audition relatively directly without requiring higher-level interpretative processes. Our other findings challenge the popular assumption that individuals with synaesthesia are rare and have ideosyncratic patterns of brain hyper-connectivity. Instead, our findings of apparently high prevalence and broad associations with other synesthesia and traits are jointly consistent with a common dependence on normal variations in physiological mechanisms of disinhibition or excitability of sensory brain areas and their functional connectivity. The prevalence of vEAR makes it easier to test such hypotheses further, and makes the results more relevant to understanding not only synaesthetic anomalies but also normal perception.