Summary: Study reveals mice can learn to manipulate random dopamine impulses for reward.
From the thrill of hearing an ice cream truck approaching to the spikes of pleasure while sipping a fine wine, the neurological messenger known as dopamine has been popularly described as the brain’s “feel good” chemical related to reward and pleasure.
A ubiquitous neurotransmitter that carries signals between brain cells, dopamine, among its many functions, is involved in multiple aspects of cognitive processing. The chemical messenger has been extensively studied from the perspective of external cues, or “deterministic” signals. Instead, University of California San Diego researchers recently set out to investigate less understood aspects related to spontaneous impulses of dopamine.
Their results, published July 23 in the journal Current Biology, have shown that mice can willfully manipulate these random dopamine pulses.
Rather than only occurring when presented with pleasurable, or reward-based expectations, UC San Diego graduate student Conrad Foo led research that found that the neocortex in mice is flooded with unpredictable impulses of dopamine that occur approximately once per minute.
Working with colleagues at UC San Diego (Department of Physics and Section of Neurobiology) and the Icahn School of Medicine at Mount Sinai in New York, Foo investigated whether mice are in fact aware that these impulses–documented in the lab through molecular and optical imaging techniques–are actually occurring. The researchers devised a feedback scheme in which mice on a treadmill received a reward if they showed they were able to control the impromptu dopamine signals. Not only were mice aware of these dopamine impulses, the data revealed, but the results confirmed that they learned to anticipate and volitionally act upon a portion of them.
“Critically, mice learned to reliably elicit (dopamine) impulses prior to receiving a reward,” the researchers note in the paper. “These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.”
The researchers say the study opens a new dimension in the study of dopamine and brain dynamics. They now intend to extend this research to explore if and how unpredictable dopamine events drive foraging, which is an essential aspect of seeking sustenance, finding a mate and as a social behavior in colonizing new home bases.
“We further conjecture that an animal’s sense of spontaneous dopamine impulses may motivate it to search and forage in the absence of known reward-predictive stimuli,” the researchers noted.
In their efforts to control dopamine, the researchers clarified that dopamine appears to invigorate, rather than initiate, motor behavior.
“This started as a serendipitous finding by a talented, and curious, graduate student with intellectual support from a wonderful group of colleagues,” said study senior co-author David Kleinfeld, a professor in the Department of Physics (Division of Physical Sciences) and Section of Neurobiology (Division of Biological Sciences). “As an unanticipated result, we spent many long days expanding on the original study and of course performing control experiments to verify the claims. These led to the current conclusions.”
The full authors list of the paper includes: Conrad Foo, Adrian Lozada, Johnatan Aljadeff, Yulong Li, Jing W. Wang, Paul A. Slesinger and David Kleinfeld.
About this dopamine research news
Source: UCSD Contact: Scott LaFee – UCSD Image: The image is credited to Julia Kuhl
Reinforcement learning links spontaneous cortical dopamine impulses to reward
•Extrasynaptic levels of dopamine in mouse cortex exhibit spontaneous impulses
•Impulses are broadly distributed in amplitude and time, with a rate of about 0.01/s
•Feedback was used to train mice to volitionally control their spontaneous impulses
•Mice learned to reliably modulate dopamine impulses in order to receive a reward
In their pioneering study on dopamine release, Romo and Schultz speculated “…that the amount of dopamine released by unmodulated spontaneous impulse activity exerts a tonic, permissive influence on neuronal processes more actively engaged in preparation of self-initiated movements….”
Motivated by the suggestion of “spontaneous impulses,” as well as by the “ramp up” of dopaminergic neuronal activity that occurs when rodents navigate to a reward, we asked two questions. First, are there spontaneous impulses of dopamine that are released in cortex?
Using cell-based optical sensors of extrasynaptic dopamine, [DA]ex, we found that spontaneous dopamine impulses in cortex of naive mice occur at a rate of ∼0.01 per second. Next, can mice be trained to change the amplitude and/or timing of dopamine events triggered by internal brain dynamics, much as they can change the amplitude and timing of dopamine impulses based on an external cue?
Using a reinforcement learning paradigm based solely on rewards that were gated by feedback from real-time measurements of [DA]ex, we found that mice can volitionally modulate their spontaneous [DA]ex.
In particular, by only the second session of daily, hour-long training, mice increased the rate of impulses of [DA]ex, increased the amplitude of the impulses, and increased their tonic level of [DA]ex for a reward. Critically, mice learned to reliably elicit [DA]ex impulses prior to receiving a reward.
These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.