‘Sherlock’ and the Case of Narrative Perception

Summary: Researchers use the TV show ‘Sherlock’ to investigate how the brain segments experiences during perception and long-term memory formation.

Source: Cell Press.

“Chunking” has been a concept in cognitive psychology since the mid-1950s. It is the means by which individual items or words are grouped together into larger units so that they can be processed or stored as single ideas. But until recently, there was no way to observe this phenomenon in the neural activity of the brain. Now, in the August 2 issue of Neuron, researchers are reporting a way to use fMRI to investigate how the brain segments experiences during perception and how these experiences become long-term memories.

“We’re interested in examining how we take the kind of continuous environment that we all live in and break it into pieces that we can understand and remember,” says Chris Baldassano, a postdoctoral research associate in the Princeton Neuroscience Institute and the study’s first author. “The main finding of this paper is that we can actually observe what’s going on in the brain during this chunking behavior and see that it’s happening on a lot of different temporal scales, from seconds up to minutes.”

The study made use of two different fMRI datasets, both of which were collected for other research. In the first experiment, participants watched part of the first episode of the BBC show “Sherlock” while being scanned with fMRI. A second group of participants listened to an audio description of the same episode while being scanned. (The research team, led by Uri Hasson and Kenneth Norman, professors in the Department of Psychology at Princeton, frequently makes use of movies and television shows to study brain activity; “Sherlock” is a particular favorite of the lab members.)

Using their new method, the researchers found that brain activity in both cases looked like a sequence of stable activity states punctuated by quick jumps. In some brain regions, these chunks of stable activity lasted for minutes at a time, and the same sequence of activity patterns occurred whether people were watching the show or listening to the audio description. “That suggests these regions are representing something more than just the visual and auditory inputs that are coming in, but a kind of higher-level processing,” Baldassano says.

The primary set of regions representing these high-level chunks is called the default mode network. It was originally identified about two decades ago as a collection of brain regions that’s active “by default”–when people are not doing anything that’s focused on a particular external goal. It lights up during thought activities that are completely internal, like remembering the past or thinking about the future, as well as during daydreaming and mind-wandering.

“This paper, as well as other recent studies from Hasson’s group, challenges the idea that this region of the brain is only about internal activity,” Baldassano says. “We’re arguing that these regions also represent what’s going on in the outside world but on a longer time scale. They are building up representations of what’s happening in the world around you.”

In the first dataset, some of the participants listening to the audio description of the “Sherlock” episode had previously watched it, and some had not. The researchers tracked the brain activity of the two groups and found that it was different in the people who had seen the show and could anticipate what was going to happen next. “They tend to start these chunks a little sooner than people who haven’t seen it,” Baldassano explains.

Image shows brain scans and event segmental models.
The event segmentation model identifies temporally clustered structure in movie-watching data throughout all regions of cortex with high intersubject correlation. The optimal number of events varied by an order of magnitude across different regions, with a large number of short events in sensory cortex and a small number of long events in high-level cortex. For example, the time point correlation matrix for a region in the precuneus exhibited coarse blocks of correlated patterns, leading to model fits with a small number of events (white squares), while a region in visual cortex was best modeled with a larger number of short events (note that only ∼3 min of the 50-min stimulus are shown and that the highlighted searchlights were selected post hoc for illustration). NeuroscienceNews image is credited to Baldassano et al./Neuron.

In the second dataset, participants watched the same episode of “Sherlock” and were then asked to retell the story from memory. The researchers found that chunks in high-level areas reappeared during recall in the same order they had appeared during perception, replaying the same sequence of activity patterns. Furthermore, the hippocampus — a key structure involved in memory–activated at the end of high-level chunks during movie viewing, and the strength of this hippocampal activity during perception predicted how strongly that chunk was later retrieved. These findings suggest that the hippocampus takes “snapshots” of chunks right before they disappear in a manner that supports long-term memory for those chunks.

Baldassano believes that these findings provide vital clues about how these chunks relate to long-term memory. “Almost all the research that’s been done in the field of memory uses a paradigm where people memorize lists of words or photos. This work provides a bridge to be able to understand how we break a continuous, ongoing experience into chunks that can be stored and remembered,” he concludes.

About this neuroscience research article

Funding: This research was supported by Intel Labs, the National Institutes of Health, the McKnight Foundation, an NSF CAREER Award, and the Simons Collaboration on the Global Brain.

Source: Joseph Caputo – Cell Press
Image Source: NeuroscienceNews.com image is credited to Baldassano et al./Neuron.
Original Research: Full open access research for “Discovering Event Structure in Continuous Narrative Perception and Memory” by Christopher Baldassano, Janice Chen, Asieh Zadbood, Jonathan W. Pillow, Uri Hasson, and Kenneth A. Norman in Neuron. Published online June 26 2017 doi:10.1016/j.neuron.2017.06.041

Cite This NeuroscienceNews.com Article

[cbtabs][cbtab title=”MLA”]Cell Press “‘Sherlock’ and the Case of Narrative Perception.” NeuroscienceNews. NeuroscienceNews, 2 August 2017.
<https://neurosciencenews.com/sherlock-narrative-perception-7227/>.[/cbtab][cbtab title=”APA”]Cell Press (2017, August 2). ‘Sherlock’ and the Case of Narrative Perception. NeuroscienceNew. Retrieved August 2, 2017 from https://neurosciencenews.com/sherlock-narrative-perception-7227/[/cbtab][cbtab title=”Chicago”]Cell Press “‘Sherlock’ and the Case of Narrative Perception.” https://neurosciencenews.com/sherlock-narrative-perception-7227/ (accessed August 2, 2017).[/cbtab][/cbtabs]


Discovering Event Structure in Continuous Narrative Perception and Memory

•Event boundaries during perception can be identified from cortical activity patterns
•Event timescales vary from seconds to minutes across the cortical hierarchy
•Hippocampal activity following an event predicts reactivation during recall
•Prior knowledge of a narrative enables anticipatory reinstatement of event patterns

During realistic, continuous perception, humans automatically segment experiences into discrete events. Using a novel model of cortical event dynamics, we investigate how cortical structures generate event representations during narrative perception and how these events are stored to and retrieved from memory. Our data-driven approach allows us to detect event boundaries as shifts between stable patterns of brain activity without relying on stimulus annotations and reveals a nested hierarchy from short events in sensory regions to long events in high-order areas (including angular gyrus and posterior medial cortex), which represent abstract, multimodal situation models. High-order event boundaries are coupled to increases in hippocampal activity, which predict pattern reinstatement during later free recall. These areas also show evidence of anticipatory reinstatement as subjects listen to a familiar narrative. Based on these results, we propose that brain activity is naturally structured into nested events, which form the basis of long-term memory representations.

“Discovering Event Structure in Continuous Narrative Perception and Memory” by Christopher Baldassano, Janice Chen, Asieh Zadbood, Jonathan W. Pillow, Uri Hasson, and Kenneth A. Norman in Neuron. Published online June 26 2017 doi:10.1016/j.neuron.2017.06.041

Feel free to share this Neuroscience News.
Join our Newsletter
I agree to have my personal information transferred to AWeber for Neuroscience Newsletter ( more information )
Sign up to receive our recent neuroscience headlines and summaries sent to your email once a day, totally free.
We hate spam and only use your email to contact you about newsletters. You can cancel your subscription any time.