Summary: A newly developed audio-to-tactile sensory substitution device provides immediate and robust improvements in speech recognition without prior training for those with hearing impairments.
Source: IOS Press
A novel study published in Restorative Neurology and Neuroscience provides the first evidence that a simple and inexpensive non-invasive speech-to-touch sensory substitution device has the potential to improve hearing in hearing-impaired cochlear implant patients, as well as individuals with normal hearing, to better discern speech in various situations like learning a second language or trying to deal with the “cocktail party effect.” The device can provide immediate multisensory enhancement without any training..
“Despite recent advancements in hearing aid and cochlear implants, the most widespread surgical implant to restore audition, hearing-impaired users still encounter significant practical and social challenges with or without aids,” explained lead investigator Amir Amedi, PhD, Department of Medical Neurobiology, Institute for Medical Research Israel-Canada, Faculty of Medicine, Hebrew University of Jerusalem, Hadassah Ein-Kerem, and The Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, Israel. “In particular, they all struggle with understanding speech in challenging, noisy acoustic environments, especially in the presence of a competing speaker.”
The number of sensory deprived patients and auditory deprived patients is expected to rise so it is crucial to develop efficient techniques for auditory recovery designed to convey the missing information to patients. “We live in a world that is becoming steadily more multisensory and we really need to understand the mechanisms underlying multisensory perception and integration. Providing relevant information using the sense of touch can significantly help hearing,” commented Dr. Amedi.
In this current proof-of-concept, study investigators hypothesized that they would be able to improve speech understanding under challenging conditions by exploiting the ability of the brain to integrate information coming simultaneously from different senses. They designed a minimalistic auditory-to-tactile sensory substitution device (SSD) that transforms low-frequency speech signals into tactile vibrations delivered on two fingertips. They asked a group of non-native English speakers to repeat a series of sentences, which was degraded by embedding them in speech-like noise.
As expected, when participants could rely only on audition, their understanding of such sentences was poor. Crucially, however, their sentence understanding significantly improved when they paired the degraded speech signal with complementary vibration delivered on the participants’ fingertips. The vibration conveyed a specific set of frequencies known as fundamental frequencies that characterize speech signals.
The reported improvement at the group level was 6 dB – a major difference considering that an increase of 10 dB represents a doubling of the perceived loudness. These results are especially relevant when compared to earlier SSD studies showing effects in behavior only after demanding cognitive training.
“Our results carry important implications for further research, as well as possible clinical and practical solutions,” noted co-author Tomasz Wolak, PhD Eng, Head of the Bioimaging Research Center, Institute of Physiology and Pathology of Hearing, World Hearing Center, Warsaw, Poland. “The ability to ‘hear through one’s fingers can significantly help hearing. Our approach suggests that multisensory stimulations providing the same type of information (in this case spoken language conveyed through touch in addition to hearing) should be processed in the same brain region (in this case spoken language centers), ultimately then predicting that multisensory stimulations (both sounds and touch) should enhance perception.
According to lead author Katarzyna Cieśla, PhD, World Hearing Center, Warsaw, and Hebrew University of Jerusalem, “The most compelling aspect of our study is the fact that learning to use such speech-to-touch sensory substitution device did not require any training. We believe it can also serve as an aid for the elderly population, which finds it challenging to follow extensive training regimes. This might be the first study showing such immediate relevant enhancement of a sensory substitution device and suggests the brain is much more multisensory than the common wisdom.”
“This study is a major step forward to introduce multisensory plasticity of the brain as an innovative paradigm to maximize the potential of patients to compensate for their sensory loss,” commented Bernhard Sabel, PhD, Editor-in-Chief of Restorative Neurology and Neuroscience.
Next, the team plans to further improve the device and training regimes in order to reach the goal of 10 dB enhancement and to test for human brain mechanisms using an MRI-compatible version of the device in various populations (both hearing and hearing-impaired people).
Source:
IOS Press
Media Contacts:
Diana Murray – IOS Press
Image Source:
The image is credited to Institut für Medizinische Psychologie Medizinische Fakultät.
Original Research: Open access
“Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution”. Cieśla, Katarzyna; Wolak, Tomasz; Lorens, Artur; Heimler, Benedetta; Skarżyński, Henryk; Amedi, Amir.
Restorative Neurology and Neuroscience. doi:10.3233/RNN-190898
Abstract
Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution
Background:
Hearing loss is becoming a real social and health problem. Its prevalence in the elderly is an epidemic. The risk of developing hearing loss is also growing among younger people. If left untreated, hearing loss can perpetuate development of neurodegenerative diseases, including dementia. Despite recent advancements in hearing aid (HA) and cochlear implant (CI) technologies, hearing impaired users still encounter significant practical and social challenges, with or without aids. In particular, they all struggle with understanding speech in challenging acoustic environments, especially in presence of a competing speaker.
Objectives:
In the current proof-of-concept study we tested whether multisensory stimulation, pairing audition and a minimal-size touch device would improve intelligibility of speech in noise.
Methods:
To this aim we developed an audio-to-tactile sensory substitution device (SSD) transforming low-frequency speech signals into tactile vibrations delivered on two finger tips. Based on the inverse effectiveness law, i.e., multisensory enhancement is strongest when signal-to-noise ratio is lowest between senses, we embedded non-native language stimuli in speech-like noise and paired it with a low-frequency input conveyed through touch.
Results:
We found immediate and robust improvement in speech recognition (i.e. in the Signal-To-Noise-ratio) in the multisensory condition without any training, at a group level as well as in every participant. The reported improvement at the group-level of 6 dB was indeed major considering that an increase of 10 dB represents a doubling of the perceived loudness.
Conclusions:
These results are especially relevant when compared to previous SSD studies showing effects in behavior only after a demanding cognitive training. We discuss the implications of our results for development of SSDs and of specific rehabilitation programs for the hearing impaired either using or not using HAs or CIs. We also discuss the potential application of such a set-up for sense augmentation, such as when learning a new language.
