Summary: Using a comparison to vinyl record collectors, researchers delve into the concept of matching—a behavior where decisions are based on reward expectations—observed from pigeons to humans.
In a recent study, the team found that fruit flies also demonstrate this behavior, deciding based on their reward predictions. By pinpointing this activity in the fly’s brain, scientists can better comprehend decision-making processes across species, and its broader implications in conditions like addiction.
Key Facts:
- “Matching” is a widespread behavior across the animal kingdom, where decisions are made based on reward expectations.
- Janelia researchers demonstrated that fruit flies can engage in value-based decision-making, choosing between odors based on associated reward likelihood.
- The study located the activity within specific synapses in the fruit fly’s mushroom body, providing insights into the synaptic changes enabling decision-making.
Source: HHMI
Like many collectors of L.P. records, James Fitzgerald’s brother-in-law has a favorite store where he consistently finds the best vinyl for his collection. But there are times when he spends hours at the store and comes up empty. He also knows that occasionally he should venture to the record store on the other side of town, where he sometimes scores a hard-to-find gem that was stocked since his last visit.
Fitzgerald’s brother-in-law is making a calculation: weighing probable outcomes to guide his behavior. His favorite record store rewards him more frequently, so he visits that store the most. The second-tier store is less likely to reward him, so he visits that store only occasionally.
Glenn Turner, who like Fitzgerald is a neuroscientist and group leader at HHMI’s Janelia Research Campus, says this “record foraging” habit is a perfect example of a type of behavior called matching that is pervasive in the animal kingdom. Instead of vinyl, non-hipster animals like mice and flies forage for food, using sensory cues like odors to evaluate food quality from a distance.
But, while matching has been observed in everything from pigeons to mice to humans, it was unclear how the brain carried out this value-based decision-making. Researchers had previously proposed a theory for how that might happen, but the idea hadn’t been tested in the real world.
Now, a team of Janelia researchers that includes Fitzgerald, Turner, Janelia Graduate Scholar Adithya Rajagopalan, former Janelia Fellow Ran Darshan and Research Specialist Karen Hibbard has confirmed that the proposed theory works.
Rajagopalan’s experiments showed that, like Fitzgerald’s brother-in-law, fruit flies can make decisions based on their expectations about the likelihood of a reward. The team also pinpointed the site in the fly brain where these value adjustments are made, enabling them to directly test this theory on the level of neural circuits.
“We found that flies are using expectation to assign value to their world,” Turner says. “It also really nicely connects back to this theoretical work that was so elegant and explains this widespread phenomenon.”
Uncovering how the fly brain carries out this ubiquitous behavior could help scientists better understand how similar decision-making happens in the brains of larger animals, including humans. Decision-making goes awry in diseases like addiction, so understanding how this process works in simpler brains has broad value, according to the researchers.
“The kinds of ideas and the theoretical framework that we have identified in this paper feel like a seed for evolution to build on in larger organisms, where more layers are added to allow for more complex behaviors,” says Rajagopalan, the first author of a new paper describing the work.
Investigating matching behavior
Fruit flies, whose brains have been well studied and mapped, were an appealing choice for examining matching and its underlying mechanisms. But first, the team had to design a way to observe fruit fly decisions.
Rajagopalan, who came to the Turner Lab through a joint graduate program with Johns Hopkins University, spearheaded the project. He designed an experiment where a single fly enters one arm of a symmetrical Y-shaped arena. Odors are pumped into the other two arms of the Y. The fly chooses to follow one odor or the other and is rewarded – in this case by having its sugar-sensing neurons activated – but with different probabilities: One odor might translate into a reward 80 percent of the time, while the other odor might yield a reward 20 percent of the time.
The researchers found that the fly learned to expect the rewards in the same proportions they were presented and then made its choice based on those expectations. These actions give the matching behavior its name: 80 percent of the time, the fly chose the odor that gives 80 percent of the rewards. And 20 percent of the time, it chose the odor that yields 20 percent of the rewards.
The team tracked the behavior to specific synapses in the mushroom body, a region of the fly brain responsible for learning and memory. This enabled them to create a model of how the brain carries out this behavior, based on the theory of matching.
In this theory, the values associated with different choices are learned through changes in synaptic strength: Synaptic connections are strengthened or weakened in proportion to the difference between expected and received reward. The team’s model based on this theory and the fly’s behavior allowed them to demonstrate how individual synapses are changing to enable value-based decision-making.
The new work emphasizes the important interplay between experiment and theory, converging on a description of the rules governing how an animal learns – an outcome that the researchers say is satisfying on both a conceptual and mechanistic level.
“To be able to see that you can get these sophisticated economic decisions through this simple mechanistic explanation about how synapses are changing is a great illustration of what mechanistic cognitive neuroscience can mean,” Fitzgerald says. “We’re taking this universal property and using the strengths of these small animals to really nail it mechanistically.”
About this neuroscience research news
Author: Nanci Bompey
Source: HHMI
Contact: Nanci Bompey – HHMI
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Reward expectations direct learning and drive operant matching in Drosophila” by James Fitzgerald et al. PNAS
Abstract
Reward expectations direct learning and drive operant matching in Drosophila
Foraging animals must use decision-making strategies that dynamically adapt to the changing availability of rewards in the environment. A wide diversity of animals do this by distributing their choices in proportion to the rewards received from each option, Herrnstein’s operant matching law.
Theoretical work suggests an elegant mechanistic explanation for this ubiquitous behavior, as operant matching follows automatically from simple synaptic plasticity rules acting within behaviorally relevant neural circuits. However, no past work has mapped operant matching onto plasticity mechanisms in the brain, leaving the biological relevance of the theory unclear.
Here, we discovered operant matching in Drosophila and showed that it requires synaptic plasticity that acts in the mushroom body and incorporates the expectation of reward. We began by developing a dynamic foraging paradigm to measure choices from individual flies as they learn to associate odor cues with probabilistic rewards.
We then built a model of the fly mushroom body to explain each fly’s sequential choice behavior using a family of biologically realistic synaptic plasticity rules. As predicted by past theoretical work, we found that synaptic plasticity rules could explain fly matching behavior by incorporating stimulus expectations, reward expectations, or both.
However, by optogenetically bypassing the representation of reward expectation, we abolished matching behavior and showed that the plasticity rule must specifically incorporate reward expectations. Altogether, these results reveal the first synapse-level mechanisms of operant matching and provide compelling evidence for the role of reward expectation signals in the fly brain.
