Robotic Arms Connected Directly to Brain of Partially Paralyzed Man Allows Him to Feed Himself

Summary: With the help of brain-machine interface technology and robotic arms, a paralyzed man was able to feed himself for the first time in thirty years.

Source: Frontiers

Two robotic arms—a fork in one hand, a knife in the other—flank a seated man, who sits in front of a table, with a piece of cake on a plate. A computerized voice announces each action: “Moving fork to food,” and “Retracting knife.” Partially paralyzed, the man makes subtle motions with his right and left fists at certain prompts, such as “select cut location,” so that the machine slices off a bite-sized piece. Now: “Moving food to mouth,” and another subtle gesture to align the fork with his mouth.

In less than 90 seconds, a person with very limited upper body mobility, who hasn’t been able to use his fingers in about 30 years, has just fed himself dessert using his mind and some smart robotic hands.

A team led by researchers at the Johns Hopkins Applied Physics Laboratory (APL), in Laurel, Maryland, and the Department of Physical Medicine and Rehabilitation (PMR) in the Johns Hopkins School of Medicine, has published a paper in the journal Frontiers in Neurorobotics that described this latest feat using a brain-machine interface (BMI) and a pair of modular prosthetic limbs.

Also sometimes referred to as a brain-computer interface, BMI systems provide a direct communication link between the brain and a computer, which decodes neural signals and “translates” them to perform various external functions, from moving a cursor on a screen to now enjoying a bite of cake. In this particular experiment, muscle movement signals from the brain helped control the robotic prosthetics.

A new approach

The study built on more than 15 years of research in neural science, robotics, and software, led by APL in collaboration with the Department of PMR, as part of the Revolutionizing Prosthetics program, which was originally sponsored by the US Defense Advanced Research Project Agency (DARPA). The new paper outlines an innovative model for shared control that enables a human to maneuver a pair of robotic prostheses with minimal mental input.

“This shared control approach is intended to leverage the intrinsic capabilities of the brain machine interface and the robotic system, creating a ‘best of both worlds’ environment where the user can personalize the behavior of a smart prosthesis,” said Dr. Francesco Tenore, a senior project manager in APL’s Research and Exploratory Development Department. The paper’s senior author, Tenore focuses on neural interface and applied neuroscience research.

“Although our results are preliminary, we are excited about giving users with limited capability a true sense of control over increasingly intelligent assistive machines,” he added.

Helping people with disabilities

One of the most important advances in robotics demonstrated in the paper is combining robot autonomy with limited human input, with the machine doing most of the work while enabling the user to customize robot behavior to their liking, according to Dr. David Handelman, the paper’s first author and a senior roboticist in the Intelligent Systems Branch of the Research and Exploratory Development Department at APL.

“In order for robots to perform human-like tasks for people with reduced functionality, they will require human-like dexterity. Human-like dexterity requires complex control of a complex robot skeleton,” he explained.

This shows a brain
In this particular experiment, muscle movement signals from the brain helped control the robotic prosthetics. Image is in the public domain

“Our goal is to make it easy for the user to control the few things that matter most for specific tasks.”

Dr. Pablo Celnik, project principal investigator in the department of PMR, said, “The human-machine interaction demonstrated in this project denotes the potential capabilities that can be developed to help people with disabilities.”

Closing the loop

While the DARPA program officially ended in August 2020, the team at APL and at the Johns Hopkins School of Medicine continues to collaborate with colleagues at other institutions to demonstrate and explore the potential of the technology.

The next iteration of the system may integrate previous research that found providing sensory stimulation to amputees enabled them to not only perceive their phantom limb, but use muscle movement signals from the brain to control a prosthetic.

The theory is that the addition of sensory feedback, delivered straight to a person’s brain, may help him or her perform some tasks without requiring the constant visual feedback in the current experiment.

“This research is a great example of this philosophy where we knew we had all the tools to demonstrate this complex bimanual activity of daily living that non-disabled people take for granted,” Tenore said.

“Many challenges still lie ahead, including improved task execution, in terms of both accuracy and timing, and closed-loop control without the constant need for visual feedback.”

Celnik added, “Future research will explore the boundaries of these interactions, even beyond basic activities of daily living.”

About this neurotech and robotics research news

Author: Press Office
Source: Frontiers
Contact: Press Office – Frontiers
Image: The image is in the public domain

Original Research: Open access.
Shared control of bimanual robotic limbs with a BMI for self-feeding” by Francesco Tenore et al. Frontiers in Neurorobotics


Shared control of bimanual robotic limbs with a BMI for self-feeding

Advances in intelligent robotic systems and brain-machine interfaces (BMI) have helped restore functionality and independence to individuals living with sensorimotor deficits; however, tasks requiring bimanual coordination and fine manipulation continue to remain unsolved given the technical complexity of controlling multiple degrees of freedom (DOF) across multiple limbs in a coordinated way through a user input.

To address this challenge, we implemented a collaborative shared control strategy to manipulate and coordinate two Modular Prosthetic Limbs (MPL) for performing a bimanual self-feeding task.

A human participant with microelectrode arrays in sensorimotor brain regions provided commands to both MPLs to perform the self-feeding task, which included bimanual cutting. Motor commands were decoded from bilateral neural signals to control up to two DOFs on each MPL at a time. The shared control strategy enabled the participant to map his four-DOF control inputs, two per hand, to as many as 12 DOFs for specifying robot end effector position and orientation.

Using neurally-driven shared control, the participant successfully and simultaneously controlled movements of both robotic limbs to cut and eat food in a complex bimanual self-feeding task.

This demonstration of bimanual robotic system control via a BMI in collaboration with intelligent robot behavior has major implications for restoring complex movement behaviors for those living with sensorimotor deficits.

Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.