Summary: Study explores how the brain utilizes learning by inference by generating cognitive maps.
Source: UC Davis
Both humans and other animals are good at learning by inference, using information we do have to figure out things we cannot observe directly. New research from the Center for Mind and Brain at the University of California, Davis, shows how our brains achieve this by constructing cognitive maps.
“The work suggests a new framework for learning in structured environments that goes beyond incremental, experiential learning of associations,” said Erie Boorman, assistant professor in the UC Davis Department of Psychology and Center for Mind and Brain and senior author on the paper.
In structured environments, individual elements are systematically related to each other as they often are in the real world. The study’s insights could be harnessed to improve educational strategies that promote the use of a cognitive map for accelerated learning through inferences, and potentially, approaches to hasten transfer of learning in machine learning in artificial intelligence, Boorman said.
Learning by inference versus association
Most studies of learning have focused on learning by association—how animals learn to associate one thing with another, through trial and error. The difference between what was expected and what actually happened drives learning in such cases.
When there is a hidden structure behind those associations, you can use direct observations to infer indirect, unseen outcomes, leaping ahead of the chain of direct association.
For example, knowing that the quality of seasonal foods is governed by changes in weather allows you to infer which are best to eat based on which foods are ripe during the same season, Boorman said. Observing ripe apples allows us to infer that pears should also be ripe, but not strawberries. This sort of structure is important to know when making decisions.
Another example is an investor inferring that the drop in Facebook shares can be attributed to a tech bubble, suggesting that Microsoft shares will likely drop soon, too.
“Knowing this hidden relationship means you can learn a lot faster,” Boorman said.
Testing learning in a structured system
To investigate how humans can use a cognitive map to learn information, graduate student Phillip Witkowski, project scientist Seongmin Park, and Boorman created a task. In a series of trials, volunteers were asked to choose between two of four abstract shapes that would lead to either of two different gift cards (e.g., either Starbucks or iTunes).
The volunteers made their choices based on two pieces of information: their estimate of the probability that each shape would lead to a particular gift card, and a randomly assigned payout for each gift card.
The shapes were divided into two pairs. In each pair, the probability that a shape would lead to a particular outcome was the inverse of the other shape.
For example, if there was a 70% chance that shape A would lead to outcome 1, there was a 30% chance that shape B would lead to the same outcome, and vice versa for outcome 2. So the subjects could gain information about the likelihood of one outcome by inference from the other, like Microsoft shares from Facebook shares.
The pairs of shapes were not connected, so the subjects could not learn anything about the results of choosing shapes C or D from the outcomes of choosing A or B.
The researchers followed how the subjects learned about the system by observing their progress over a series of trials. After analyzing the results, they found that the volunteers were using inferential learning to make decisions about which shapes to pick.
Some of the volunteers were invited back for the second part of the experiment, performing the same task while their brain activity was measured with functional magnetic resonance imaging.
Learning is reflected in the brain by a burst of activity, a “belief update” when there is a difference between your prior and newly acquired knowledge. Activity linked to inferential learning was found in the prefrontal cortex and the area of midbrain where the neurotransmitter dopamine is released.
At the same time, the researchers found a representation of the hidden (or latent) probability controlling associations for A and B in the prefrontal cortex.
The fMRI results show the brain representing different outcomes in relation to each other, Boorman said. This representation allows for those “aha” moments.
Conventional thinking holds that incremental learning about rewards from direct experiences is reinforced by the release of dopamine in the brain. The new study also implicates dopamine but for inferential learning.
“Our work suggests a more general role for dopamine signals in updating beliefs through inference,” Boorman said.
About this cognition research news
Author: Andy Fell
Source: UC Davis
Contact: Andy Fell – UC Davis
Image: The image is credited to Phillip Witkowski, UC Davis
Original Research: Closed access.
“Neural mechanisms of credit assignment for inferred relationships in a structured world” by Phillip P. Witkowski et al. Neuron
Abstract
Neural mechanisms of credit assignment for inferred relationships in a structured world
Highlights
- Dopaminergic midbrain and frontal areas update experienced and inferred associations
- mPFC uses a generalized code for observed and inferred credit assignment
- mPFC tracks the evolving estimated “position” in the association space
Summary
Animals abstract compact representations of a task’s structure, which supports accelerated learning and flexible behavior. Whether and how such abstracted representations may be used to assign credit for inferred, but unobserved, relationships in structured environments are unknown.
We develop a hierarchical reversal-learning task and Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific choice-outcome associations via structured knowledge.
We find that the medial prefrontal cortex (mPFC) efficiently represents hierarchically related choice-outcome associations governed by the same latent cause, using a generalized code to assign credit for both experienced and inferred outcomes.
Furthermore, the mPFC and lateral orbitofrontal cortex track the current “position” within a latent association space that generalizes over stimuli.
Collectively, these findings demonstrate the importance of both tracking the current position in an abstracted task space and efficient, generalizable representations in the prefrontal cortex for supporting flexible learning and inference in structured environments.
