Summary: The synesthesia effect of being able to ‘hear’ silent movements may depend upon disinhibition of signaling between the visual and auditory brain regions. The study found musicians are more likely to experience the ‘visual ear’ phenomena than those with no musical training.
Source: City University London
A synaesthesia-like effect in which people ‘hear’ silent flashes or movement, such as in popular ‘noisy GIFs’ and memes, could be due to a reduction of inhibition of signals that travel between visual and auditory areas of the brain, according to a new study led by researchers at City, University of London.
The study is the first to provide insight into the brain mechanisms underpinning such auditory sensations also known as a ‘visually-evoked auditory response’ (aka vEAR or ‘visual ear’).
The condition has received more attention due to the recent, viral popularity of the ‘skipping pylon GIF’, and other ‘noisy GIFs’ depicting silent motion, which in some people evoke very vivid visual ear sensations.The image is credited to b3ta user HappyToast.
Whilst one theory is that areas of the brain responsible for visual and auditory processing normally compete, this research suggests that they may actually cooperate in people who report visual ear.
It was also found that musicians taking part in the study were significantly more likely to report experiencing visual ear than non-musician participants. This could be because musical training may promote joint attention to both the sound of music and the sight of the coordinated movements of the conductor or other musicians.
Dr Elliot Freeman, Principal Investigator on the study and a Senior Lecturer in Psychology at the University said:
“We already knew that some people hear what they see. Car indicator lights, flashing neon shop signs, and people’s movements as they walk may all trigger an auditory sensation.
“Our latest study reveals normally-occurring individual differences in how our senses of vision and hearing interact.
“We found that people with ‘visual ears’ can use both senses together to see and also ‘hear’ silent motion, while for others hearing is inhibited when watching such visual sequences.”
Some neuroscientists believe visual-ear may be a type of synaesthesia, with other examples including music, letters or numbers that can evoke perceptions of color. However, visual ear appears to be the most prevalent, with as many as 20% of people reporting some experiences of it compared to 4.4 percent for other types.
The condition has received more attention due to the recent, viral popularity of the ‘skipping pylon GIF’, and other ‘noisy GIFs’ depicting silent motion, which in some people evoke very vivid visual ear sensations.
To shed light on what may be going on in the brain when people view such content, the researchers applied a weak alternating current to participants’ scalps, using a technique called transcranial Alternating Current Stimulation (tACS), to explore how the visual and auditory parts of the brain interact in those who experience visual ear and those who don’t.
The first experiment of the study included 36 healthy participants, including 16 classical musicians from the London Royal College of Music. All were shown auditory and visual ‘Morse code’ sequences, while electrical stimulation (tACS) was applied to either the back of the head (visual areas of the brain) or the sides (auditory areas) using ‘alpha-frequency’ tACS stimulation. Participants were then classified as visual or non-visual ear depending on whether they reported ‘hearing’ the silent flashes.
The researchers found that in non-visual ear participants, alpha-frequency stimulation to auditory areas significantly reduced auditory performance but improved visual performance, while the opposite pattern was found for the same frequency of stimulation to visual areas (poorer vision, better audition).
This reciprocal pattern suggests a competitive interaction between visual and auditory brain areas with each normally inhibiting the performance of the other.
However, these interactions were strikingly absent in visual ear participants, suggesting that their auditory and visual areas were not competing but cooperating with each other.
A second experiment was conducted to see whether even people without conscious awareness of ‘visual ear’ sometimes use their auditory brain areas to make purely visual judgments. It found that this might indeed be the case for some, where stimulation to auditory areas of the brain affected the accuracy of visual judgments almost as much as stimulating visual areas.
Taken together, the results of these experiments support a popular theory that some kinds of synaesthesia may depend on disinhibition of pre-existing neural cross-connections between sensory brain areas that are normally inactive. When these connections are disinhibited this may result in conscious awareness of visual ear and other synaesthetic phenomena.
Dr. Freeman said: “We were also interested to find that, on average, participants with visual ear performed better on both visual and auditory tasks than those without. Perhaps their audio-visual cooperation benefits performance because more of the brain is engaged in processing visual stimuli.
“Such cooperation might also benefit musical performance, explaining why so many of the musicians we tested reported experiencing visual ear.”
Dr Freeman is continuing his research in this field with a new online questionnaire, which anyone can complete as a participant.
Original Research: Closed access
“Hearing through Your Eyes: Neural Basis of Audiovisual Cross-activation, Revealed by Transcranial Alternating Current Stimulation”
Christopher Fassnidge, Danny Ball, Zainab Kazaz, Synøve Knudsen, Andrew Spicer, Anthony Tipple, and Elliot Freeman
Journal of Cognitive Neuroscience 0 0:0, 1-14 doi:10.1162/jocn_a_01395
Hearing through Your Eyes: Neural Basis of Audiovisual Cross-activation, Revealed by Transcranial Alternating Current Stimulation
Some people experience auditory sensations when seeing visual flashes or movements. This prevalent synaesthesia-like “visually evoked auditory response” (vEAR) could result either from overexuberant cross-activation between brain areas and/or reduced inhibition of normally occurring cross-activation. We have used transcranial alternating current stimulation (tACS) to test these theories. We applied tACS at 10 Hz (alpha band frequency) or 40 Hz (gamma band), bilaterally either to temporal or occipital sites, while measuring same/different discrimination of paired auditory (A) versus visual (V) “Morse code” sequences. At debriefing, participants were classified as vEAR or non-vEAR, depending on whether they reported “hearing” the silent flashes. In non-vEAR participants, temporal 10-Hz tACS caused impairment of A performance, which correlated with improved V; conversely under occipital tACS, poorer V performance correlated with improved A. This reciprocal pattern suggests that sensory cortices are normally mutually inhibitory and that alpha-frequency tACS may bias the balance of competition between them. vEAR participants showed no tACS effects, consistent with reduced inhibition, or enhanced cooperation between modalities. In addition, temporal 40-Hz tACS impaired V performance, specifically in individuals who showed a performance advantage for V (relative to A). Gamma-frequency tACS may therefore modulate the ability of these individuals to benefit from recoding flashes into the auditory modality, possibly by disrupting cross-activation of auditory areas by visual stimulation. Our results support both theories, suggesting that vEAR may depend on disinhibition of normally occurring sensory cross-activation, which may be expressed more strongly in some individuals. Furthermore, endogenous alpha- and gamma-frequency oscillations may function respectively to inhibit or promote this cross-activation.