Summary: Researchers use the TV show ‘Sherlock’ to investigate how the brain segments experiences during perception and long-term memory formation.
Source: Cell Press.
“Chunking” has been a concept in cognitive psychology since the mid-1950s. It is the means by which individual items or words are grouped together into larger units so that they can be processed or stored as single ideas. But until recently, there was no way to observe this phenomenon in the neural activity of the brain. Now, in the August 2 issue of Neuron, researchers are reporting a way to use fMRI to investigate how the brain segments experiences during perception and how these experiences become long-term memories.
“We’re interested in examining how we take the kind of continuous environment that we all live in and break it into pieces that we can understand and remember,” says Chris Baldassano, a postdoctoral research associate in the Princeton Neuroscience Institute and the study’s first author. “The main finding of this paper is that we can actually observe what’s going on in the brain during this chunking behavior and see that it’s happening on a lot of different temporal scales, from seconds up to minutes.”
The study made use of two different fMRI datasets, both of which were collected for other research. In the first experiment, participants watched part of the first episode of the BBC show “Sherlock” while being scanned with fMRI. A second group of participants listened to an audio description of the same episode while being scanned. (The research team, led by Uri Hasson and Kenneth Norman, professors in the Department of Psychology at Princeton, frequently makes use of movies and television shows to study brain activity; “Sherlock” is a particular favorite of the lab members.)
Using their new method, the researchers found that brain activity in both cases looked like a sequence of stable activity states punctuated by quick jumps. In some brain regions, these chunks of stable activity lasted for minutes at a time, and the same sequence of activity patterns occurred whether people were watching the show or listening to the audio description. “That suggests these regions are representing something more than just the visual and auditory inputs that are coming in, but a kind of higher-level processing,” Baldassano says.
The primary set of regions representing these high-level chunks is called the default mode network. It was originally identified about two decades ago as a collection of brain regions that’s active “by default”–when people are not doing anything that’s focused on a particular external goal. It lights up during thought activities that are completely internal, like remembering the past or thinking about the future, as well as during daydreaming and mind-wandering.
“This paper, as well as other recent studies from Hasson’s group, challenges the idea that this region of the brain is only about internal activity,” Baldassano says. “We’re arguing that these regions also represent what’s going on in the outside world but on a longer time scale. They are building up representations of what’s happening in the world around you.”
In the first dataset, some of the participants listening to the audio description of the “Sherlock” episode had previously watched it, and some had not. The researchers tracked the brain activity of the two groups and found that it was different in the people who had seen the show and could anticipate what was going to happen next. “They tend to start these chunks a little sooner than people who haven’t seen it,” Baldassano explains.
In the second dataset, participants watched the same episode of “Sherlock” and were then asked to retell the story from memory. The researchers found that chunks in high-level areas reappeared during recall in the same order they had appeared during perception, replaying the same sequence of activity patterns. Furthermore, the hippocampus — a key structure involved in memory–activated at the end of high-level chunks during movie viewing, and the strength of this hippocampal activity during perception predicted how strongly that chunk was later retrieved. These findings suggest that the hippocampus takes “snapshots” of chunks right before they disappear in a manner that supports long-term memory for those chunks.
Baldassano believes that these findings provide vital clues about how these chunks relate to long-term memory. “Almost all the research that’s been done in the field of memory uses a paradigm where people memorize lists of words or photos. This work provides a bridge to be able to understand how we break a continuous, ongoing experience into chunks that can be stored and remembered,” he concludes.
Funding: This research was supported by Intel Labs, the National Institutes of Health, the McKnight Foundation, an NSF CAREER Award, and the Simons Collaboration on the Global Brain.
Source: Joseph Caputo – Cell Press
Image Source: NeuroscienceNews.com image is credited to Baldassano et al./Neuron.
Original Research: Full open access research for “Discovering Event Structure in Continuous Narrative Perception and Memory” by Christopher Baldassano, Janice Chen, Asieh Zadbood, Jonathan W. Pillow, Uri Hasson, and Kenneth A. Norman in Neuron. Published online June 26 2017 doi:10.1016/j.neuron.2017.06.041
[cbtabs][cbtab title=”MLA”]Cell Press “‘Sherlock’ and the Case of Narrative Perception.” NeuroscienceNews. NeuroscienceNews, 2 August 2017.
<https://neurosciencenews.com/sherlock-narrative-perception-7227/>.[/cbtab][cbtab title=”APA”]Cell Press (2017, August 2). ‘Sherlock’ and the Case of Narrative Perception. NeuroscienceNew. Retrieved August 2, 2017 from https://neurosciencenews.com/sherlock-narrative-perception-7227/[/cbtab][cbtab title=”Chicago”]Cell Press “‘Sherlock’ and the Case of Narrative Perception.” https://neurosciencenews.com/sherlock-narrative-perception-7227/ (accessed August 2, 2017).[/cbtab][/cbtabs]
Abstract
Discovering Event Structure in Continuous Narrative Perception and Memory
Highlights
•Event boundaries during perception can be identified from cortical activity patterns
•Event timescales vary from seconds to minutes across the cortical hierarchy
•Hippocampal activity following an event predicts reactivation during recall
•Prior knowledge of a narrative enables anticipatory reinstatement of event patterns
Summary
During realistic, continuous perception, humans automatically segment experiences into discrete events. Using a novel model of cortical event dynamics, we investigate how cortical structures generate event representations during narrative perception and how these events are stored to and retrieved from memory. Our data-driven approach allows us to detect event boundaries as shifts between stable patterns of brain activity without relying on stimulus annotations and reveals a nested hierarchy from short events in sensory regions to long events in high-order areas (including angular gyrus and posterior medial cortex), which represent abstract, multimodal situation models. High-order event boundaries are coupled to increases in hippocampal activity, which predict pattern reinstatement during later free recall. These areas also show evidence of anticipatory reinstatement as subjects listen to a familiar narrative. Based on these results, we propose that brain activity is naturally structured into nested events, which form the basis of long-term memory representations.
“Discovering Event Structure in Continuous Narrative Perception and Memory” by Christopher Baldassano, Janice Chen, Asieh Zadbood, Jonathan W. Pillow, Uri Hasson, and Kenneth A. Norman in Neuron. Published online June 26 2017 doi:10.1016/j.neuron.2017.06.041
