Summary: A new machine learning study reveals behavioral strategies in worms. The study, researchers say, may shed light on how humans make decisions.
Source: Kyoto University.
Scientists have developed a new approach to understanding animal preferences, and the findings could provide greater insight into decision-making in humans
Pet owners are familiar with the challenges of figuring out what their pets want. Building on Pavlov’s work teaching his dogs to associate the ringing of a bell with food, researchers have tried to understand how animals and humans react to rewards under different circumstances. However in the real world, what constitutes a reward and how this may motivate behavior are rarely clear.
Writing in PLOS Computational Biology, a team of scientists at Kyoto University’s Graduate School of Biostudies report how worms evaluate a potential reward, examining their reactions by studying their movements.
“Current behavioral models are based on known rewards, so we can’t use them to study freely behaving animals,” explains lead author Shoichiro Yamaguchi. “We realized that we needed to look at the inverse case, and more accurately define the value of rewards from the behavior of the organism.”
The team observed heat-sensitive worms as they searched for food on surfaces with different temperature zones, applying a machine learning method to understand how potential food rewards guided the movements.
Worms initially fed at a certain temperature were then observed to move toward that temperature zone when transferred to a plate with a range of different surface temperatures. In contrast, worms that were starved at a certain temperature, and then transferred, moved away from that zone.
The team’s model showed that the fed worms sensed not only the environmental temperature, but also the change of temperature as they moved over different zones. The worms combined these sensations into a behavioral strategy to reach food using a minimal amount of energy, comparable to rational decision-making in humans.
Interestingly, the starved worms based their movements only on the temperature of the environment to escape from temperature zones they considered unlikely to contain food.
“Our approach accurately reproduces this simple worm behavior and gives much deeper insights into its underlying mechanism,” elaborates senior scientist Honda Naoki. “Combining our approach with neurological measurements of freely behaving animals could help us better understand the essence of decision-making in higher animals as well as guide developments in artificial intelligence.”
Funding: Japan Society for the Promotion of Science, Kyoto University, Japan Agency for Medical Research and Development funded this study.
Source: Raymond Kunikane Terhune – Kyoto University
Publisher: Organized by NeuroscienceNews.com.
Image Source: NeuroscienceNews.com image is credited to Kyoto University / Honda Lab.
Original Research: Open access research for “Identification of animal behavioral strategies by inverse reinforcement learning” by Shoichiro Yamaguchi, Honda Naoki , Muneki Ikeda, Yuki Tsukada, Shunji Nakano, Ikue Mori, and Shin Ishii in PLOS Computational Biology. Published May 2 2018.
doi:10.1371/journal.pcbi.1006122
[cbtabs][cbtab title=”MLA”]Kyoto University “What Do Animals Want?.” NeuroscienceNews. NeuroscienceNews, 29 May 2018.
<https://neurosciencenews.com/ai-animal-wants-9163/>.[/cbtab][cbtab title=”APA”]Kyoto University (2018, May 29). What Do Animals Want?. NeuroscienceNews. Retrieved May 29, 2018 from https://neurosciencenews.com/ai-animal-wants-9163/[/cbtab][cbtab title=”Chicago”]Kyoto University “What Do Animals Want?.” https://neurosciencenews.com/ai-animal-wants-9163/ (accessed May 29, 2018).[/cbtab][/cbtabs]
Abstract
Identification of animal behavioral strategies by inverse reinforcement learning
Animals are able to reach a desired state in an environment by controlling various behavioral patterns. Identification of the behavioral strategy used for this control is important for understanding animals’ decision-making and is fundamental to dissect information processing done by the nervous system. However, methods for quantifying such behavioral strategies have not been fully established. In this study, we developed an inverse reinforcement-learning (IRL) framework to identify an animal’s behavioral strategy from behavioral time-series data. We applied this framework to C. elegans thermotactic behavior; after cultivation at a constant temperature with or without food, fed worms prefer, while starved worms avoid the cultivation temperature on a thermal gradient. Our IRL approach revealed that the fed worms used both the absolute temperature and its temporal derivative and that their behavior involved two strategies: directed migration (DM) and isothermal migration (IM). With DM, worms efficiently reached specific temperatures, which explains their thermotactic behavior when fed. With IM, worms moved along a constant temperature, which reflects isothermal tracking, well-observed in previous studies. In contrast to fed animals, starved worms escaped the cultivation temperature using only the absolute, but not the temporal derivative of temperature. We also investigated the neural basis underlying these strategies, by applying our method to thermosensory neuron-deficient worms. Thus, our IRL-based approach is useful in identifying animal strategies from behavioral time-series data and could be applied to a wide range of behavioral studies, including decision-making, in other organisms.
