Summary: A ‘smart hand exoskeleton’, a custom-made robotic glove, can aid stroke patients in relearning dexterity-based skills like playing music. The glove, equipped with integrated tactile sensors, soft actuators, and artificial intelligence, can mimic natural hand movements and provide tactile sensations.
By applying machine learning, the glove can distinguish between correct and incorrect piano play, potentially offering a novel tool for personalized rehabilitation. Although the current design focuses on music, the technology holds promise for a broader range of rehabilitation tasks.
Key Facts:
- The ‘smart hand exoskeleton’, weighing just 191g, features soft pneumatic actuators and an array of 16 flexible sensors, mimicking natural hand movements and providing tactile feedback.
- The glove’s design was taught via machine learning to discern correct from incorrect piano play, showcasing its potential in helping stroke patients relearn music and other complex tasks.
- Despite the promising initial findings, challenges remain, including improving tactile sensing accuracy, enhancing exoskeleton adaptability, and refining machine learning algorithms for better user input interpretation.
Source: Frontiers
Stroke is the most important cause of disability for adults in the EU, which affects approximately 1.1 million inhabitants each year. After a stroke, patients commonly need rehabilitation to relearn to walk, talk, or perform daily tasks.
Research has shown that besides physical and occupational therapy, music therapy can help stroke patients to recover language and motor function.
But for people trained in music and who suffered a stroke, playing music may itself be a skill that needs to be relearned.
Now, a study in Frontiers in Robotics and AI has shown how novel soft robotics can help recovering patients to relearn playing music and other skills that require dexterity and coordination.
“Here we show that our smart exoskeleton glove, with its integrated tactile sensors, soft actuators, and artificial intelligence, can effectively aid in the relearning of manual tasks after neurotrauma,” said lead author Dr Maohua Lin, an adjunct professor at the Department of Ocean & Mechanical Engineering of Florida Atlantic University.
Whom the glove fits: custom-made ‘smart hand’
Lin and colleagues designed and tested a ‘smart hand exoskeleton’ in the shape of a multi-layered, flexible 3D-printed robo-glove, which weighs only 191g. The entire palm and wrist area of the glove are designed to be soft and flexible, and the shape of the glove can be custom-made to fit each wearer’s anatomy.
Soft pneumatic actuators in its fingertips generate motion and exert force, thus mimicking natural, fine-tuned hand movements. Each fingertip also contains an array of 16 flexible sensors or ‘taxels’, which give tactile sensations to the wearer’s hand upon interaction with objects or surfaces.
Production of the glove is straightforward, as all actuators and sensors are put in place through a single molding process.
“While wearing the glove, human users have control over the movement of each finger to a significant extent,” said senior author Dr Erik Engeberg, a professor at Florida Atlantic University’s Department of Ocean & Mechanical Engineering.
“The glove is designed to assist and enhance their natural hand movements, allowing them to control the flexion and extension of their fingers. The glove supplies hand guidance, providing support and amplifying dexterity.”
The authors foresee that patients might ultimately wear a pair of these gloves, to help both hands independently to regain dexterity, motor skills, and a sense of coordination.
AI trained the glove to be a music teacher
The authors used machine learning to successfully teach the glove to ‘feel’ the difference between playing a correct versus incorrect versions of a beginner’s song on the piano. Here, the glove operated autonomously without human input, with preprogrammed movements. The song was ‘Mary had a little lamb’, which requires four fingers to play.
“We found that the glove can learn to distinguish between correct and incorrect piano play. This means it could be a valuable tool for personalized rehabilitation of people who wish to relearn to play music,” said Engeberg.
Now that the proof-of-principle has been shown, the glove can be programmed to give feedback to the wearer about what went right or wrong in their play, either through haptic feedback, visual cues, or sound. These would enable her or him to understand their performance and make improvements.
Picking up the gauntlet for remaining challenges
Lin added: “Adapting the present design to other rehabilitation tasks beyond playing music, for example object manipulation, would require customization to individual needs. This can be facilitated through 3D scanning technology or CT scans to ensure a personalized fit and functionality for each user.”
“But several challenges in this field need to be overcome. These include improving the accuracy and reliability of tactile sensing, enhancing the adaptability and dexterity of the exoskeleton design, and refining the machine learning algorithms to better interpret and respond to user input.”
About this robotics research news
Author: Mischa Dijkstra
Source: Frontiers
Contact: Mischa Dijkstra – Frontiers
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Feeling the beat: a smart hand exoskeleton for learning to play musical instruments” by Maohua Lin et al. Frontiers in Robotics and AI
Abstract
Feeling the beat: a smart hand exoskeleton for learning to play musical instruments
Individuals who have suffered neurotrauma like a stroke or brachial plexus injury often experience reduced limb functionality. Soft robotic exoskeletons have been successful in assisting rehabilitative treatment and improving activities of daily life but restoring dexterity for tasks such as playing musical instruments has proven challenging.
This research presents a soft robotic hand exoskeleton coupled with machine learning algorithms to aid in relearning how to play the piano by ‘feeling’ the difference between correct and incorrect versions of the same song.
The exoskeleton features piezoresistive sensor arrays with 16 taxels integrated into each fingertip. The hand exoskeleton was created as a single unit, with polyvinyl acid (PVA) used as a stent and later dissolved to construct the internal pressure chambers for the five individually actuated digits. Ten variations of a song were produced, one that was correct and nine containing rhythmic errors.
To classify these song variations, Random Forest (RF), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) algorithms were trained with data from the 80 taxels combined from the tactile sensors in the fingertips.
Feeling the differences between correct and incorrect versions of the song was done with the exoskeleton independently and while the exoskeleton was worn by a person. Results demonstrated that the ANN algorithm had the highest classification accuracy of 97.13% ± 2.00% with the human subject and 94.60% ± 1.26% without.
These findings highlight the potential of the smart exoskeleton to aid disabled individuals in relearning dexterous tasks like playing musical instruments.
