Summary: A new study has uncovered how the brain efficiently encodes and recalls visual memories—by organizing them into categories and using the precise timing of neuronal activity. Researchers recorded hippocampal spikes in epilepsy patients as they viewed images from five object categories, then used machine learning to decode which type the patient was remembering.
This reveals that the hippocampus doesn’t store each object individually, but simplifies memory storage by classifying visual inputs. These findings could lead to advanced brain-computer interfaces and memory prostheses for patients with memory loss.
Key Facts:
- Category-Based Storage: The brain simplifies memory encoding by grouping visual objects into categories rather than storing each image individually.
- Temporal Coding: Timing of neural spikes, not just firing rate, carries critical information about memory categories.
- Clinical Potential: Insights may support the development of memory prostheses to help patients with neurological disorders like dementia.
Source: USC
Researchers at USC have made a significant breakthrough in understanding how the human brain forms, stores and recalls visual memories.
A new study, published in Advanced Science, harnesses human patient brain recordings and a powerful machine learning model to shed new light on the brain’s internal code that sorts memories of objects into categories — think of it like the brain’s filing cabinet of imagery.
The results demonstrated that the research team could essentially read subjects’ minds, by pinpointing the category of visual image being recalled, purely from the precise timing of the subject’s neural activity.
The work solves a fundamental neuroscience debate and offers exciting potential for future brain-computer interfaces, including memory prostheses to restore lost memory in patients with neurological disorders like dementia.
The research was led by Dong Song, associate professor in the Department of Neurological Surgery and the Alfred E. Mann Department of Biomedical Engineering. and Charles Liu, the USC Neurorestoration Center director at Keck School of Medicine of USC and professor of biomedical engineering at USC Viterbi School of Engineering.
The first author, Xiwei She, is a former Ph.D. student from the Song Lab and is currently a postdoctoral researcher at Stanford University.
How does the brain store visual information?
The hippocampus is a critical brain region, well-known for its role in creating new episodic memories – the what, where, and when of past events. While its function in encoding spatial (“where”) and temporal (“when”) information is relatively understood, how it manages to encode the vast and high-dimensional world of objects (“what”) has remained a mystery.
It’s simply not feasible for the hippocampus to store every single object separately; instead, scientists hypothesized that the brain might simplify this complexity by encoding objects into categories.
Song, who is Director of the USC Neural Modeling and Interface Laboratory, has been conducting pioneering work in the area of memory prostheses, creating devices that mimic and restore cognitive function, with potential clinical applications for patients with dementia and Alzheimer’s disease.
“We have tested our memory prostheses in a lot of human patients. We created the prostheses and have published several papers showing that it can enhance memory function,” Song said.
“But I also wanted to take the opportunity to answer some fundamental neuroscience questions. And this is one of them.”
Brain recordings from epilepsy patients yield insights
Song, Liu and their team’s latest work harnesses brain recording from 24 epilepsy patients with intracranial depth electrodes implanted in their brains for seizure localization.
Recordings from these patients allowed the team to pinpoint how hippocampal neurons encode complex visual information, not by firing rate alone, but by the precise timing of their activity.
“Working with human patients suffering from memory dysfunction, it was exceptionally exciting to see the current studies reveal a model for the neural basis of memory formation,” Liu said.
The research team developed an innovative experimental-modeling approach to unravel this intricate process. The team recorded the electrical activity, specifically “spikes,” from hippocampal CA3 and CA1 neurons in the epilepsy patients.
The recordings were gathered while patients performed a “delayed match-to-sample” (DMS) task — a popular neuroscience technique to test visual short-term memory.
“We let the patients see five categories of images: ‘animal,’ ‘plant,’ ‘building,’ ‘vehicle,’ and ‘small tools.’ Then we recorded the hippocampal signal,” Song said. “Then, based on the signal, we asked ourselves a question, using our machine learning technique. Can we decode what category image they are recalling purely based on their brain signal?”
The results confirmed the hypothesis that the human brain does indeed recall visual objects by sorting them into categories, and that these visual memory categories that the patients were thinking of were decodable based on their brain signals.
“It’s like reading your hippocampus to see what kind of memory you are trying to form,” Song said. “We found that we can actually do that. We can pretty accurately decode what kind of category of image the patient was trying to remember.”
An efficient strategy for storing diverse memories
The core of the discovery lies in the research team’s interpretable memory decoding model. Unlike previous methods that often rely on averaging neuronal activity over many trials or using pre-selected temporal resolutions, this advanced model analyzes the “spatio-temporal patterns” of spikes from an entire ensemble of neurons.
The study also provides evidence that the hippocampus uses a temporal code to represent visual memory categories. This means that the precise timing of individual neuron spikes, often at the millisecond scale, carries meaningful information.
While previous studies often focused on individual neurons, this research revealed that hippocampal neuron ensembles encode memory categories in a distributed manner.
This means that while a large proportion of neurons (70-80%) were involved in assigning a visual memory to a category, within each individual neuron, only brief, specific moments contributed to this encoding. This efficient strategy allows the brain to store diverse memories while minimizing energy consumption.
“With this knowledge, we can begin to develop clinical tools to restore memory loss and improve lives, including memory prostheses and other neurorestorative strategies,” Liu said.
“While this result may be important to all patients who suffer memory disorders, it has profound relevance specifically to the epilepsy patients who participated in the studies, many of whom suffer from hippocampal dysfunction that manifests in both seizures as well as cognitive/memory disorders.”
About this visual memory research news
Author: Amy Blumenthal
Source: USC
Contact: Amy Blumenthal – USC
Image: The image is credited to Neuroscience News
Original Research: Open access.
“Distributed Temporal Coding of Visual Memory Categories in Human Hippocampal Neurons Revealed by an Interpretable Decoding Model” by Dong Song et al. Advanced Science
Abstract
Distributed Temporal Coding of Visual Memory Categories in Human Hippocampal Neurons Revealed by an Interpretable Decoding Model
The hippocampus is crucial for forming new episodic memories. While its role in encoding spatial and temporal information (where and when) is well understood, how it encodes objects (what) remains unclear due to the high dimensionality of object space.
Rather than encoding each object separately, the hippocampus may encode object categories to reduce complexity.
Here, an experimental-modeling approach to investigate how the hippocampus encodes visual memory categories in humans is developed.
Spikes are recorded from hippocampal CA3 and CA1 neurons in 24 epilepsy patients performing a delayed match-to-sample task involving five image categories.
An interpretable memory decoding model is employed to decode memory categories from hippocampal spiking activity and identify the spatio-temporal characteristics of hippocampal encoding.
Using this model, the optimal temporal resolutions for decoding each visual memory category per neuron are estimated.
Results indicate that visual memory categories can be decoded from hippocampal spike patterns, supporting the presence of category-specific coding.
Hippocampal neuron ensembles encode memory categories in a distributed manner, akin to a population code, while individual neurons use a temporal code.
Additionally, CA3 and CA1 neurons exhibit similar and redundant memory category information, likely due to strong and diffuse feedforward synaptic connections from CA3 to CA1 regions.
