Summary: A new role for the hippocampus involving pattern separation and conjunctive representation in reinforcement learning is discussed.
Source: Arizona State University
ASU and Stanford study shows how the hippocampus provides information to other brain areas during learning.
Avid hikers know to be cautious of plants with leaves made up of three leaflets if they are red in the spring or fall. Parents worldwide know the precarious relationship between proximity to bedtime and roughhousing with their children.
How do hikers know to link the color of a leaf with the season to determine if it is poison ivy? How do parents know to link the time of day with a child’s excitement level to determine the success of a bedtime routine? Just like Pavlov’s dogs salivated when a bell rang, people learn to recognize poisonous plants or prevent tears of exhaustion in a toddler by forming associations among details in their surroundings and what happens.
Researchers from Arizona State University and Stanford University analyzed patterns of brain activity in humans and discovered a previously unknown role for the hippocampus, a brain area important to memory, in forming associations during learning. The study will be published on March 6 in Nature Communications.
“This study has important implications for understanding how the brain’s memory systems contribute to learning and decision-making” said Vishnu Murty, an assistant professor of psychology at Temple University who was not involved with the research. “The findings could help us understand the complexity of decision-making deficits in populations where the focus has mostly been on memory deficits, like Alzheimer’s disease, other psychopathology disorders and normal aging.”
When people learn, they build associations among features – like the time of day and mood of a toddler – to predict an outcome, like whether or not roughhousing will end in tears.
“These associations have a big influence on behavior,” said Ian Ballard, who recently earned his doctorate in neuroscience from Stanford University and is first author on the paper. “But in the real world, objects or events are defined by more than one feature or combinations of features, and we wanted to understand how the brain builds associations over similar configurations of features.”
To unpack how the brain might handle the problem of building associations over such complex, real-world information, the research team focused on a small brain structure that looks like a snail shell.
A tiny yet powerful brain area
The hippocampus is approximately one-third the size of a gumball, and this small brain structure is crucial for memory formation. Without it, people cannot form new memories about facts or events, like what day it is or the names of coworkers. During memory formation, the hippocampus represents individual details of an event, like where you parked your car, to be as different as possible from each other.
“One challenge with memory is that it is hard to distinguish similar experiences. So, if you use the same parking garage at work every day, you have to remember which floor and space to go to at the end of the day,” said Samuel McClure, associate professor of psychology at ASU and senior author on the paper. “The problem is that it is easy to confuse where you parked on different days. The hippocampus is critical for remembering the combination of where and when.”
How the hippocampus forms memories provided a mechanism for how complex combinations of features could be represented in the brain, but whether the small, coiled structure actually contributed to how people learn about the world was an open question.
To identify how the hippocampus might contribute to how people form associations in the real world, the researchers designed a learning task that required participants to use combinations of features to predict whether an outcome would happen. A series of stimulus images, like a single face or a face paired with a building, appeared sequentially on a screen. The participants then had to predict whether a target image would appear after the stimulus images. The goal for the task was to respond as quickly as possible to any target image that showed up.
It was only the combinations of two stimulus pictures, like a face paired with a townhouse, that that could be used to predict when a target would appear. The individual stimuli, like the face image, were not useful for predicting by themselves.
As the participants worked through the task, the research team used functional magnetic resonance imaging (fMRI) to measure brain activity from the hippocampus and other brain structures known to be involved in learning. The team then examined the patterns of activity during the task and noticed something interesting about the activity in the hippocampus. It was the only brain structure that represented the stimulus images bound together, which is important because success on the task required forming associations about combinations to accurately respond to the target images.
“We found the hippocampus uniquely represented bound features: faces and houses were distinct from a face and house mingled together,” Ballard said.
A new way the hippocampus contributes to learning
When the research team looked at how the patterns of activity in the hippocampus were related to other brain areas, they found hippocampal activity was tightly correlated with activity in the striatum. Nestled beneath the cortex, the striatum consists of three separate structures – the caudate, putamen and nucleus accumbens – and plays an important role in learning what predicts desirable outcomes.
“The hippocampus formed bound associations of multiple features that supported learning in the striatum about the configuration of multiple features in the environment,” Ballard said.
How the striatum handles information during learning is well known, but where all the information comes from is an open question. Ballard added that this study starts to chip away at that question by showing the hippocampus provided information about combinations of features to the striatum and that information was used to learn how to succeed at the task.
Until recently, the brain was thought to have separate learning systems, but the findings suggest the hippocampal memory system and striatal reinforcement learning system are interrelated.
“It is really important to think of the brain as an interconnected structure, with different parts that work together to produce our impressive mental feats. Neuroscience and psychology have done a good job at understanding how individual parts work. It is exciting to start trying to figure out how they begin to work together,” McClure said. “We suspect that understanding both normal and diseased mental functions requires figuring out how all of the parts work together — or not.”
Anthony Wagner, professor of psychology at Stanford University, also contributed to the study.
Source: Arizona State University
Media Contact: Robert Ewing – ASU
Publisher: More Than Just Memories: A New Role for the Hippocampus During Learning organized by Neuroscience News.
Image Source: NeuroscienceNews.com image is credited to Ian C. Ballard. Image credit (top) “Poison ivy” by Stilfehler is licensed under CC BY-SA 3.0. (bottom) “Poison ivy foliage during autumn leaf coloration in Ewing, New Jersey” by Famartin is licensed under CC BY-SA 4.0.
Original Research: Open Access article for “Hippocampal pattern separation supports reinforcement learning” by Ian C. Ballard, Anthony D. Wagner & Samuel M. McClure in Nature Communications. Published March 6,2019
Funding: The work was supported by a Graduate Research Fellowship and Integrative Graduate Education and Research Traineeship from the National Science Foundation and with funds from Stanford University and Arizona State University. The authors declare no competing interests.
Hippocampal pattern separation supports reinforcement learning
Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multi-featural conjunctive representations. Here, we test if the hippocampus forms separable conjunctive representations that enables the learning of response contingencies for stimuli of the form: AB+, B−, AC−, C+. Pattern analyses on functional MRI data show the hippocampus forms conjunctive representations that are dissociable from feature components and that these representations, along with those of cortex, influence striatal prediction errors. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.