Summary: Our brains constantly predict sensory input, and mismatches between expectation and reality trigger prediction error signals. While past research focused on vision, new experiments show that the auditory cortex also produces strong signals when expected sounds are absent.
When both visual and auditory mismatches occur together, the brain’s response is much stronger than to either mismatch alone, indicating non-linear integration of sensory errors. Human experiments using VR and EEG confirm that similar processes occur outside of mice, with potential implications for diagnosing psychiatric conditions.
Key Facts:
- Cross-Modal Prediction Errors: Mismatches in both vision and sound produce amplified brain responses.
- Human-Brain Parallels: Early human studies show similar prediction error signals using VR and EEG.
- Clinical Potential: Abnormal prediction error responses may help diagnose and track psychiatric conditions.
Source: FMI
Our brains are constantly predicting what we should see and hear based on our own movements and previous experiences.
When reality doesn’t match these predictions, certain neurons respond with a distinctive signal — a so-called prediction error.
Previous research from the Keller group at the FMI had shown that when mice run through a virtual tunnel and the visual flow is suddenly paused, their brains produce a strong prediction error signal, but it was unclear whether this was unique to vision.
Magdalena Solyga, a postdoc with Keller, set out to find out. In her experiments, Solyga designed a setup where mice ran through a dark corridor, with the loudness of a sound increasing in proportion to their running speed.
Occasionally, the sound was muted — creating a mismatch between what the mouse expected to hear and what it actually heard. Neurons in the auditory cortex responded strongly, indicating that prediction error signaling is not limited to vision but may be a general function of sensory brain areas.
Next, Solyga introduced mismatches in both sight and sound at the same time. In this version of the experiment, both the visual flow and the sound were tied to the mouse’s running speed — and both were occasionally paused together.
The result was a surprisingly strong brain response, larger than the sum of the individual visual or auditory mismatches. Some neurons responded only to this combined mismatch, suggesting the brain integrates different types of sensory errors in a complex, non-linear way, Solyga says.
Building on results in mice, the team adapted their experiment for humans using EEG recordings and virtual reality headsets. In early tests, human participants walked while navigating a virtual environment.
When the visual scene froze unexpectedly while the participant’s body kept moving, the researchers observed a clear brain response indicating a visual mismatch, similar to the one seen in mice.
The team is now beginning to test combined mismatches in people as well .
One of the long-term goals of this research is to develop reliable brain-based biomarkers for psychiatric conditions. If people with psychosis show abnormal or absent mismatch responses, these brain signals could potentially be used to aid diagnosis or track the effects of treatment — offering a more objective measure than current self-reported symptoms.
However, translating this research into clinical practice will take time. Recording brain signals during movement is technically challenging, as movement introduces noise into EEG data. So far, the team has tested 17 healthy adults and is aiming to recruit up to 50 participants to ensure robust results.
Factors such as hairstyle and movement artifacts can affect signal quality, making it essential to gather data from a large group to be able to draw generalizable conclusions about how the brain works, Solyga says.
The study also raises new scientific questions. “We have no idea how the potentiation of brain response to prediction errors is happening: is it through direct communication between two sensory areas, or is there a region in the brain that gets information about all the mismatches happening?” she says.
“There are so many exciting directions to explore.”
About this neuroscience research news
Author: Magdalena Solyga
Source: FMI
Contact: Magdalena Solyga – FMI
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Multimodal mismatch responses in mouse auditory cortex” by Magdalena Solyga et al. eLife
Abstract
Multimodal mismatch responses in mouse auditory cortex
Our movements result in predictable sensory feedback that is often multimodal. Based on deviations between predictions and actual sensory input, primary sensory areas of cortex have been shown to compute sensorimotor prediction errors.
How prediction errors in one sensory modality influence the computation of prediction errors in another modality is still unclear.
To investigate multimodal prediction errors in mouse auditory cortex (ACx), we used a virtual environment to experimentally couple running to both self-generated auditory and visual feedback.
Using two-photon microscopy, we first characterized responses of layer 2/3 (L2/3) neurons to sounds, visual stimuli, and running onsets and found responses to all three stimuli.
Probing responses evoked by audiomotor mismatches, we found that they closely resemble visuomotor mismatch responses in visual cortex (V1).
Finally, testing for cross modal influence on audiomotor mismatch responses by coupling both sound amplitude and visual flow speed to the speed of running, we found that audiomotor mismatch responses were amplified when paired with concurrent visuomotor mismatches.
Our results demonstrate that multimodal and non-hierarchical interactions shape prediction error responses in cortical L2/3.
