Summary: A new machine learning algorithm reveals serotonin can help speed up learning rates.
Source: Sainsbury Wellcome Centre.
A new computational-model designed by researchers at UCL based on data from the Champalimaud Centre for the Unknown reveals that serotonin, one of the most widespread chemicals in the brain, can speed up learning.
Serotonin is thought to mediate communications between neural cells and play an essential role in functional, and dysfunctional, cognition. For a long time, serotonin has been recognized as a major target of antidepressants (selective-serotonin-reuptake-inhibitor (SSRIs) that are used to treat various psychiatric conditions, such as depression, obsessive-compulsive-disorder and forms of anxiety. However, serotonin in humans, and other animals, is associated with a bewildering variety of aspects of cognition and decision-making, including punishment, reward and patience.
The new results, published in Nature Communications today, provide additional illumination. In the article, Kiyohito Iigaya and Peter Dayan at Gatsby Computational Neuroscience Unit and the Max Planck UCL Centre for Computational Psychiatry and Ageing Research at UCL, analysed data collected by their collaborators, Madalena Fonseca and Masayoshi Murakami, led by Zachary Mainen at the Champalimaud Centre for Unknown in Portugal.
In the experiments, mice were trained to choose one of the two targets to receive water rewards. Mice continually had to learn which of the targets was more rewarding, as the reward rates changed without warning. Crucially, sometimes serotonin release in the brain was temporarily boosted in mice with genetically modified serotonin neurons by a technique called optogenetics, allowing the effects of serotonin on learning to be assessed.
Iigaya built a computational account of mice behaviour based on reinforcement learning principles, which are widely used in machine-learning and AI. Iigaya found that the learning rate, i.e. how fast the modelled mice learn, was modulated by serotonin stimulation. He compared trials with and without stimulation of serotonin neurons, and observed that the learning rate was significantly faster when stimulation was delivered, meaning that boosting serotonin sped up learning in mice.
The authors also found that when mice made decisions in very quick succession, they followed a simple strategy called ‘win-stay lose-switch’, in which they repeated a choice if it had just been rewarded, and switched to the other choice if it had not been rewarded. Serotonin stimulation did not affect these fast choices. However, on subset of trials, when animals acted slowly and took a long time in-between trials, their decisions did not follow the simple win-stay lose-switch rule. Instead, the mice made decisions based on a longer history of rewards, which was well-characterized by a reinforcement learning account. Serotonin stimulation only affected this slow learning system.
Importantly, the authors found that this slow system tracked reward outcomes every trial, even when the choices were made by the fast, win-stay lose-switch, system. Thus, the effects of the serotonin stimulation to boost the slow system became apparent only occasionally, when the animals spent a long time before making decisions. The authors believe that the way that multiple decision-systems mask each other might explain why scientists have had difficulty in constructing a comprehensive theory as to how serotonin affects learning and decision-making.
The authors conclude: “Our results suggest that serotonin boosts [brain] plasticity by influencing the rate of learning. This resonates, for instance, with the fact that treatment with an SSRI can be more effective when combined with so-called cognitive behavioral therapy, which encourages the breaking of habits in patients.”
Substantial clinical research shows that SSRI treatment is often most effective if combined with cognitive-behavioural-therapy (CBT). The goal of CBT is to change maladaptive thinking and behaviour actively, through sessions that are designed for patients to (re)learn their way to think and behave. However, scientists have had limited understanding of how and why SSRI and CBT work together for treatments. The new findings point to a possible functional link between the two, with serotonin boosting the learning inherent to CBT, providing clues as to one of the roles that this neuromodulator plays in the treatment of psychiatric disorders.
Funding: This research was supported by the Gatsby Charitable Foundation, the Joint Initiative on Computational Psychiatry and Ageing Research between Max Planck Society and UCL, the Japan Society for the Promotion of Science, the European Research Council (250334 and 671251), Fundação para a Ciência e a Tecnologia (PD/BD/52446/2013 and SFRH/BPD/46314/2008), and the Champalimaud Foundation.
Source: April Cashin-Garbutt – Sainsbury Wellcome Centre
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com image is credited to Champalimaud Research/Dario Sarra.
Original Research: Open access research for “An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals” by Kiyohito Iigaya, Madalena S. Fonseca, Masayoshi Murakami, Zachary F. Mainen & Peter Dayan in Nature Communications. Published June 26 2018.
doi:10.1038/s41467-018-04840-2
[cbtabs][cbtab title=”MLA”]Sainsbury Wellcome Centre”Computational Model Analysis Reveals Serotonin Speeds Learning.” NeuroscienceNews. NeuroscienceNews, 26 June 2018.
<https://neurosciencenews.com/ai-serotonin-learning-9447/>.[/cbtab][cbtab title=”APA”]Sainsbury Wellcome Centre(2018, June 26). Computational Model Analysis Reveals Serotonin Speeds Learning. NeuroscienceNews. Retrieved June 26, 2018 from https://neurosciencenews.com/ai-serotonin-learning-9447/[/cbtab][cbtab title=”Chicago”]Sainsbury Wellcome Centre”Computational Model Analysis Reveals Serotonin Speeds Learning.” https://neurosciencenews.com/ai-serotonin-learning-9447/ (accessed June 26, 2018).[/cbtab][/cbtabs]
Abstract
An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals
Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.
