Neural prosthetic devices, which include small electrode arrays implanted in the brain, can allow paralyzed patients to control the movement of a robotic limb, whether that limb is attached to the individual or not. In May 2015, researchers at Caltech, USC, and Rancho Los Amigos National Rehabilitation Center reported the first successful clinical trial of such an implant in a part of the brain that translates intention—the goal to be accomplished through a movement (for example, “I want to reach to the water bottle for a drink”)—into the smooth and fluid motions of a robotic limb. Now, the researchers, led by Richard Andersen, the James G. Boswell Professor of Neuroscience, report that individual neurons in that brain region, known as the posterior parietal cortex (PPC), encode entire hand shapes which can be used for grasping—as when shaking someone’s hand—and hand shapes not directly related to grasping, such as the gestures people make when speaking.
Most neuroprostheses are implanted in the motor cortex, the part of the brain controlling limb motion. But the movement of these robotic arms are jerky, probably due to the complicated mechanics for controlling muscle movement. Having eliminated that problem by implanting the device in the PPC, the brain region that encodes the intent, led Andersen and colleagues to further investigate the role specific neurons play in this part of the brain.
The research appears in the November 18 issue of the Journal of Neuroscience.
“The human hand has the ability to do numerous complex operations beyond just grasping,” says Christian Klaes, a postdoctoral fellow at Caltech and first author of the paper. “We gesture when we speak, we manipulate objects, we use sign language to communicate with the hearing impaired. Tetraplegic patients rate hand and arm function to be of the highest importance to have better control over their environment. So our ultimate goal is to improve the range of neuroprostheses using control signals from the PPC.
“The more precisely we can identify individual neurons involved with hand movements, the better the capability these robotic devices will provide. Ultimately, we hope to mimic in a robotic hand the same freedom of movement of the human hand.”
In the study, the researchers used the rock-paper-scissors game and a variation, rock-paper-scissors-lizard-Spock. The game, says Andersen, is “perfect” for this kind of research. “The addition of a lizard, depicted as a cartoon image of a lizard, and Spock—a picture of Leonard Nimoy in character—was to increase the repertoire of possible hand shapes available to our tetraplegic participant, Erik G. Sorto, whose limbs are completely paralyzed. We assigned a pinch gesture for the lizard and a spherical shape for Mr. Spock.”
The game was played in two phases, first rock-paper-scissors and then the expanded game with the lizard and Spock. In the task, Sorto was briefly shown an object on a screen that corresponded to one of the hand shapes—for example, a picture of a rock or Mr. Spock. The image was followed by a blank screen, and then text appeared instructing Sorto to imagine making the corresponding hand shape with his right hand—a fist for the rock, an open hand for paper, a scissors gesture for scissors, a pinch for the lizard, and a spherical shape (loosely analogous to the Vulcan salute) for Spock—and to say which visual image he had seen, as the neuroprosthetic device recorded the activity of neurons in the PPC.
The researchers were able to identify single neurons in the PPC that fired when Sorto was presented with an image of an object to be grasped—a rock, say—and identified a nearly completely separate class of neurons that responded when Sorto engaged in motor imagery (the mental planning and imagined execution of a movement without the subject actually trying to move the limb).
“We found two mostly separate populations of neurons in the PPC that show either visual responses or motor-imagery responses during the task, the former when Erik identified a cue and the latter when he imagined performing a corresponding hand shape,” says Andersen.
The researchers discovered that individual neurons in the PPC also responded to hand shapes that did not directly correspond to a grasp-related visual stimulus. The paper shape can be related to the initial opening of the hand to grasp a paper, and the rock closing the hand to grasp a rock—and in fact, these imagined hand shapes were used by Sorto to imagine opening a robotic hand by imagining paper and closing the robotic hand around an object by imagining rock. However, scissors, lizard, and Spock call for imagining hand gestures that are more abstract and iconic than those needed to grasp the visual objects, and suggests, says Andersen, that this area of the brain may also be involved in more general hand gestures, such as ones we use when talking, or for sign language.
About this neuroscience and neuroprosthetics research
In addition to Andersen and Klaes, other authors on the study are Spencer Kellis, Tyson Aflalo, and Kelsie Pejsa from Caltech; Brian Lee, Christi Heck, and Charles Liu from USC; and Kathleen Shanfield, Stephanie Hayes-Jackson, and Mindy Aisen from Rancho Los Amigos National Rehabilitation Center.
Source: Deborah Williams-Hedges – CalTech Image Credit: The image is credited to Spencer Kellis/Caltech Original Research:Abstract for “Hand Shape Representations in the Human Posterior Parietal Cortex” by Christian Klaes, Spencer Kellis, Tyson Aflalo, Brian Lee, Kelsie Pejsa, Kathleen Shanfield, Stephanie Hayes-Jackson, Mindy Aisen, Christi Heck, Charles Liu, and Richard A. Andersen in Journal of Neuroscience. Published online November 18 doi:10.1523/JNEUROSCI.2747-15.2015
Hand Shape Representations in the Human Posterior Parietal Cortex
Humans shape their hands to grasp, manipulate objects, and to communicate. From nonhuman primate studies, we know that visual and motor properties for grasps can be derived from cells in the posterior parietal cortex (PPC). Are non-grasp-related hand shapes in humans represented similarly? Here we show for the first time how single neurons in the PPC of humans are selective for particular imagined hand shapes independent of graspable objects. We find that motor imagery to shape the hand can be successfully decoded from the PPC by implementing a version of the popular Rock-Paper-Scissors game and its extension Rock-Paper-Scissors-Lizard-Spock. By simultaneous presentation of visual and auditory cues, we can discriminate motor imagery from visual information and show differences in auditory and visual information processing in the PPC. These results also demonstrate that neural signals from human PPC can be used to drive a dexterous cortical neuroprosthesis.
SIGNIFICANCE STATEMENT This study shows for the first time hand-shape decoding from human PPC. Unlike nonhuman primate studies in which the visual stimuli are the objects to be grasped, the visually cued hand shapes that we use are independent of the stimuli. Furthermore, we can show that distinct neuronal populations are activated for the visual cue and the imagined hand shape. Additionally we found that auditory and visual stimuli that cue the same hand shape are processed differently in PPC. Early on in a trial, only the visual stimuli and not the auditory stimuli can be decoded. During the later stages of a trial, the motor imagery for a particular hand shape can be decoded for both modalities.
“Hand Shape Representations in the Human Posterior Parietal Cortex” by Christian Klaes, Spencer Kellis, Tyson Aflalo, Brian Lee, Kelsie Pejsa, Kathleen Shanfield, Stephanie Hayes-Jackson, Mindy Aisen, Christi Heck, Charles Liu, and Richard A. Andersen in Journal of Neuroscience. Published online November 18 doi:10.1523/JNEUROSCI.2747-15.2015