Summary: Objects are not only represented in one form of short term memory but in several forms simultaneously.
Source: RUB
Contrary to previous assumptions, visual short-term memory is not merely based on one kind of information about an object, such as only its colour or only its name. Rather, several types of information can be retained simultaneously in short-term memory. Using complex EEG analyses and deep neural networks, researchers at Beijing Normal University and Ruhr-Universität Bochum have discovered that short-term memory is more complex than previously assumed.
The team describes their findings in the journal Proceedings of the National Academy of Sciences, PNAS for short, published online on 7 December 2020.
For the study, Dr. Hui Zhang, Rebekka Heinen and Professor Nikolai Axmacher from the Department of Neuropsychology collaborated with the team headed by Jing Liu and Professor Gui Xue from Beijing Normal University.
The banana in short-term memory
Visual short-term memory helps us remember objects for a short period of time when these objects are no longer visible. Until now, it has been assumed that short-term memory is based on only one type of brain activity. The German-Chinese research team has now disproved this assumption. The researchers recorded brain activity in epilepsy patients using electrodes that were inserted into the brain for the purpose of surgical planning. The patients saw pictures of objects like a banana and had to remember them for a short time.
Deep neural networks help interpret brain activity
Earlier studies by other groups had shown that deep neural networks process images in similar steps as humans do. If a person or a deep neural network sees a banana, the first step is to process simple characteristics such as its yellow colour and smooth texture. Later on, the processed information becomes more and more complex. Eventually, the human and the network recognise the specific crescent shape and finally identify the banana.
The researchers compared the different processing steps of the neural network with the brain data of the patients. This enabled them to see which activity patterns belong to the processing of simple visual properties like the yellow colour of the banana and which belong to more complex properties like its name.
First simple, then complex
Based on this result, the team then showed that objects are not only represented in one form in short-term memory, as previously assumed, but in several forms simultaneously. When looking at them, initially simple properties of the banana are processed, then complex properties are added. During the memorisation phase, simple and complex information is retained together. The visual short-term memory is thus more complex than has long been assumed.
About this visual memory research news
Source: RUB
Contact: Nikolai Axmacher – RUB
Image: The image is in the public domain
Original Research: Closed access.
“Stable maintenance of multiple representational formats in human visual short-term memory” by Hui Zhang et al. PNAS
Abstract
Stable maintenance of multiple representational formats in human visual short-term memory
Visual short-term memory (VSTM) enables humans to form a stable and coherent representation of the external world. However, the nature and temporal dynamics of the neural representations in VSTM that support this stability are barely understood. Here we combined human intracranial electroencephalography (iEEG) recordings with analyses using deep neural networks and semantic models to probe the representational format and temporal dynamics of information in VSTM. We found clear evidence that VSTM maintenance occurred in two distinct representational formats which originated from different encoding periods. The first format derived from an early encoding period (250 to 770 ms) corresponded to higher-order visual representations. The second format originated from a late encoding period (1,000 to 1,980 ms) and contained abstract semantic representations. These representational formats were overall stable during maintenance, with no consistent transformation across time. Nevertheless, maintenance of both representational formats showed substantial arrhythmic fluctuations, i.e., waxing and waning in irregular intervals. The increases of the maintained representational formats were specific to the phases of hippocampal low-frequency activity. Our results demonstrate that human VSTM simultaneously maintains representations at different levels of processing, from higher-order visual information to abstract semantic representations, which are stably maintained via coupling to hippocampal low-frequency activity.
