Summary: Study explores brain activity in primates and humans when presented with a choice between one option that maximizes information and another which prioritizes reward during decision making.
Source: University of New Mexico
People make countless choices every day. It may be a difficult, complex choice—whether to take the job in a new city or stay in a current position—or be as simple as choosing between visiting a new restaurant or going to an old favorite. Decision-making shapes our knowledge and our perceptions of the world around us.
These simple and complex choices are generally between something familiar—where we mostly know what to expect—and trying something new, where the outcome might be unknown to us. But what happens inside our brain when we make one choice or the other?
Jeremy Hogeveen, assistant professor in the Department of Psychology, details his search for answers in a new article published in the journal Neuron.
The research was a collaboration between The University of New Mexico and Oregon Health & Science University (OHSU), and involved looking at modeling data of rhesus macaques, and observational data of humans using functional Magnetic Resonance Imaging (fMRI) technology.
The article, “The neurocomputational bases of explore-exploit decision-making,” looks at activity in the brain when primates and humans are presented with a choice between one option that maximizes information (explore) and one that prioritizes a known reward (exploit).
“This is the ‘explore-exploit tradeoff,’ and despite exploration being essential for healthy decision-making—a long-standing mystery in the field is why an animal might choose to explore something new and forego a familiar reward,” explains Hogeveen.
“We were hoping to determine the brain computations that help motivate explore-exploit decisions.”
By “brain computations,” Hogeveen is referring to the calculations our brains make through a complex system of neurons, which communicate like circuits on a computer’s motherboard. The team is asking—what is the level of tolerance for uncertainty? Do these brain circuits value an immediate and known reward, or do they prefer to activate motivation to investigate something new?
Using computer modeling of explore-exploit decision-making processes through simulated tests, the team established that for both human adults and rhesus macaques, the estimation of future benefits motivates the choice to try the new options. The multi-disciplinary model, called a POMDP (partially observable Markov decision process) model, can assess numerous real-world situational decision-making processes.
The next step was to collect fMRI data from macaques at OHSU, and humans at UNM and the Mind Research Network as they performed decision-making tasks. The subjects were presented with three choices, with limited opportunities to learn the value of each choice, and a random replacement of one of the choices over time.
Uncertainty always accompanied the new choice because its reward value could not be predicted, and the only way to learn about the choice was to explore it.
Using these methods, Hogeveen and his team were able to merge the computational model with the fMRI data to visualize a partially overlapping, partially distinct network that communicated and calculated the value of exploring versus exploiting, effectively mapping out the decision-making processes of the entire brain.
“Interestingly, this involved distinct encoding of different value signals in brain regions that are highly evolved in primates like the lateral prefrontal cortex, which were more activated during exploration than exploitation,” Hogeveen says.
Basically, newer areas of the brain (evolutionarily speaking) such as the lateral prefrontal cortex, activated more when an unfamiliar choice was made while older brain systems including the amygdala and the orbitofrontal cortex showed increased activation while weighing the value of explorations vs. exploitation.
This suggests that both the newer and older circuits in the brain worked together during the decision-making, process rather than against each other as was previously thought.
Hogeveen and his team took advantage of the research and data analysis resources at the UNM Center for Advanced Research Computing (CARC) for this project. CARC was instrumental in providing time-saving assistance, including the ability to run multiple datasets at once rather than one at a time.
By taking advantage of CARC’s parallel computing capabilities (available to all UNM researchers), Dr. Hogeveen was able to obtain his results in just over a day instead of the 10—25 days it would have taken if the data had been run on a single, stand-alone computer.
This project has promising future paths in cross-species primate research as well as bench-to-bedside interventions in clinical settings.
In addition, according to Hogeveen this kind of explore-exploit decision-making research is crucial for understanding and treating brain disorders with circuit-specific pharmaceuticals or transcranial neural stimulation when decision-making process may be compromised—in people with traumatic brain injuries, for example. But to understand how impaired brains work, scientists first must understand how typical brains function.
“Explore-exploit decision-making is impaired in brain disorders that have a significant public health impact,” says Hogeveen.
“In my view and based on the thinking of lots of others in the field, overly-exploratory and overly-exploitative decision-making traits are likely to play a role in the challenges faced by many patients with psychiatric, neurological, or neurodevelopmental disorders.”
About this decision making research news
Author: Press Office
Source: University of New Mexico
Contact: Press Office – University of New Mexico
Image: The image is credited to University of New Mexico
Original Research: Closed access.
“The neurocomputational bases of explore-exploit decision-making” by Jeremy Hogeveen et al et al. Neuron
Abstract
The neurocomputational bases of explore-exploit decision-making
Highlights
- Value and uncertainty direct explore-exploit decisions in humans and monkeys
- A prefrontal subdivision unique to primates encodes when exploration is valuable
- Frontoparietal brain regions show dissociable encoding of value and uncertainty
- Motivational brain regions complement prefrontal contributions to exploration
Summary
Flexible decision-making requires animals to forego immediate rewards (exploitation) and try novel choice options (exploration) to discover if they are preferable to familiar alternatives.
Using the same task and a partially observable Markov decision process (POMDP) model to quantify the value of choices, we first determined that the computational basis for managing explore-exploit tradeoffs is conserved across monkeys and humans.
We then used fMRI to identify where in the human brain the immediate value of exploitative choices and relative uncertainty about the value of exploratory choices were encoded.
Consistent with prior neurophysiological evidence in monkeys, we observed divergent encoding of reward value and uncertainty in prefrontal and parietal regions, including frontopolar cortex, and parallel encoding of these computations in motivational regions including the amygdala, ventral striatum, and orbitofrontal cortex.
These results clarify the interplay between prefrontal and motivational circuits that supports adaptive explore-exploit decisions in humans and nonhuman primates.
