Summary: A pioneering new prototype allows those with hearing loss to “listen” to music through the sense of touch.
Source: University of Malaga
People with hearing loss will be able to listen to music through the sense of touch thanks to a pioneering prototype that has been devised by researchers of the Department of Electronics of the University of Malaga, members of the R&D group “Electronics for instrumentation and systems”.
It consists of an audio-tactile algorithm that, by using “tactile illusions”, renders monophonic music into tangible stimuli based on vibration. “It’s like ‘hacking’ the nervous system to receive a different response to the real stimulus sent”, they say.
“What we want to achieve in the long term is for people who do not hear to be able to ‘listen’ to music”, assures the researcher of the PhD Program in Mechatronics Engineering Paul Remache, main author of this paper, who insists on the power of music to influence mood, as well as its possibilities as a therapy for mental disorders and treatment of pain.
This would result in a portable terminal that could be brought to a concert, since this prototype, according to the researchers, will be easily transferable to technological devices such as cell phones.
Mapping music
The algorithm developed by this young researcher, along with the professors at the UMA Andrés Trujillo and Fernando Vidal, is capable of converting musical features and structures extracted from MIDI files –Musical Instrument Digital Interface– to “vibrotactile stimuli”.
“It’s something similar to mapping music”, explains Remache, who adds that this is possible because this type of files not only can be played and generate sound, but also provide “symbolic representations”.
Controlling vibrations
Current models do not warrant the correspondence between the emotional response to music and the vibrotactile version of it. In view of this, these engineers of the UMA propose an arrangement of the “tactile illusions” in order to improve and extend the spectrum of musical features, adding dynamics to the vibration in the form of movement, changes of direction and location.
“It is a challenging process since the perceptible frequency range of the skin is lower than that of the auditory system, which may cause the loss of some musical features”, they explain.
Different emotional response
First experiments completed, in which over fifty volunteers took part, suggest that the arrangement of “tactile illusions” evokes more positive emotions than negative ones. They are also perceived as more agreeable and stimulating than the audio, provoking a different emotional response from that of the original music.
Smart instrumentation and application in healthcare
This first prototype was presented in the 11th International Workshop on Haptic & Audio Interaction Design (United Kingdom) –the biggest international event specialized in these areas of study– after being published in the scientific journal LNCS. At present, the researchers of the UMA are working on a second model and continue with the experiments.
The research is the product of Paul Remache’s doctoral thesis and is part of the National Plan project “Smart instrumentations and application in healthcare”.
About this neurotech research news
Author: María Guerrero
Source: University of Malaga
Contact: María Guerrero – University of Malaga
Image: The image is credited to University of Malaga
Original Research: Closed access.
“Mapping Monophonic MIDI Tracks to Vibrotactile Stimuli Using Tactile Illusions” by Paul Remache et al. LNCS
Abstract
Mapping Monophonic MIDI Tracks to Vibrotactile Stimuli Using Tactile Illusions
In this project, we propose an algorithm to convert musical features and structures extracted from monophonic MIDI files to tactile illusions.
Mapping music to vibrotactile stimuli is a challenging process since the perceptible frequency range of the skin is lower than that of the auditory system, which may cause the loss of some musical features. Moreover, current proposed models do not warrant the correspondence between the emotional response to music and the vibrotactile version of it.
We propose to use tactile illusions as an additional resource to convey more meaningful vibrotactile stimuli. Tactile illusions enable us to add dynamics to vibrotactile stimuli in the form of movement, changes of direction, and localization.
The suggested algorithm converts monophonic MIDI files into arrangements of two tactile illusions: “phantom motion” and “funneling”. The validation of the rendered material consisted of presenting the audio rendered from MIDI files to participants and then adding the vibrotactile component to it. The arrangement of tactile illusions was also evaluated alone.
Results suggest that the arrangement of tactile illusions evokes more positive emotions than negative ones. This arrangement was also perceived as more agreeable and stimulating than the original audio.
Although musical features such as rhythm, tempo, and melody were mostly recognized in the arrangement of tactile illusions, it provoked a different emotional response from that of the original audio.
