Summary: Our concepts of people, objects, and events are composed of sensory impressions and actions associated with them, in addition to abstract symbol-like features.
Source: Max Planck Institute
To understand the world, we arrange individual objects, people, and events into different categories or concepts. Concepts such as ‘the telephone’ consist primarily of visible features, i.e. shape and color, and sounds, such as ringing. In addition, there are actions, i.e. how we use a telephone.
However, the concept of telephone does not only arise in the brain when we have a telephone in front of us. It also appears when the term is merely mentioned. If we read the word “telephone”, our brain also calls up the concept of telephone. The same regions in the brain are activated that would be activated if we actually saw, heard, or used a telephone. The brain thus seems to simulate the characteristics of a telephone when its name alone is mentioned.
Until now, however, it was unclear, depending on the situation, whether the entire concept of a telephone is called up or only individual features such as sounds or actions and whether only the brain areas that process the respective feature become active. So, when we think of a telephone, do we always think of all its features or only the part that is needed at the moment? Do we retrieve our sound knowledge when a phone rings, but our action knowledge when we use it?
Researchers at the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig have now found the answer: It depends on the situation. If, for example, the study participants thought of the sounds associated with the word “telephone”, the corresponding auditory areas in the cerebral cortex were activated, which are also activated during actual hearing. When thinking about using a telephone, the somatomotor areas that underlie the involved movements came into action.
In addition to these sensory-dependent, so-called modality-specific areas, it was found that there are areas that process both sounds and actions together. One of these so-called multimodal areas is the left inferior parietal lobule (IPL). It became active when both features were requested.
The researchers also found out that, in addition to characteristics based on sensory impressions and actions, there must be other criteria by which we understand and classify terms. This became apparent when the participants were only asked to distinguish between real and invented words.
Here, a region that was not active for actions or sounds kicked in: the so-called anterior temporal lobe (ATL). The ATL therefore seems to process concepts abstractly or “amodally”, completely detached from sensory impressions.
From these findings, the scientists finally developed a hierarchical model to reflect how conceptual knowledge is represented in the human brain. According to this model, information is passed on from one hierarchical level to the next and at the same time becomes more abstract with each step.
On the lowest level, therefore, are the modality-specific areas that process individual sensory impressions or actions.
These transmit their information to the multimodal regions such as the IPL, which process several linked perceptions simultaneously, such as sounds and actions.
The amodal ATL, which represents features detached from sensory impressions, operates at the highest level. The more abstract a feature, the higher the level at which it is processed and the further it is removed from actual sensory impressions.
“We thus show that our concepts of things, people, and events are composed, on the one hand, of the sensory impressions and actions associated with them and, on the other hand, of abstract symbol-like features,” explains Philipp Kuhnke, lead author of the study, which was published in the renowned journal Cerebral Cortex.
“Which features are activated depends strongly on the respective situation or task” added Kuhnke.
In a follow-up study in Cerebral Cortex, the researchers also found that modality-specific and multimodal regions work together in a situation-dependent manner when we retrieve conceptual features.
The multimodal IPL interacted with auditory areas when retrieving sounds, and with somatomotor areas when retrieving actions.
This showed that the interaction between modality-specific and multimodal regions determined the behavior of the study participants. The more these regions worked together, the more strongly the participants associated words with actions and sounds.
The scientists investigated these correlations with the help of various word tasks that the participants solved while lying in a functional magnetic resonance imaging (fMRI) scanner.
Here, they had to decide whether they strongly associated the named object with sounds or actions.
The researchers showed them words from four categories: 1) objects associated with sounds and actions, such as “guitar”, 2) objects associated with sounds but not with actions, such as “propeller”, 3) objects not associated with sounds but with actions, such as “napkin”, and 4) objects associated neither with sounds nor with actions, such as “satellite”.
About this neuroscience research news
Source: Max Planck Institute Contact: Verena Müller – Max Planck Institute Image: The image is in the public domain
Task-Dependent Functional and Effective Connectivity during Conceptual Processing
Conceptual knowledge is central to cognition. Previous neuroimaging research indicates that conceptual processing involves both modality-specific perceptual-motor areas and multimodal convergence zones. For example, our previous functional magnetic resonance imaging (fMRI) study revealed that both modality-specific and multimodal regions respond to sound and action features of concepts in a task-dependent fashion (Kuhnke P, Kiefer M, Hartwigsen G. 2020b.
Task-dependent recruitment of modality-specific and multimodal regions during conceptual processing. Cereb Cortex. 30:3938–3959.). However, it remains unknown whether and how modality-specific and multimodal areas interact during conceptual tasks.
Here, we asked 1) whether multimodal and modality-specific areas are functionally coupled during conceptual processing, 2) whether their coupling depends on the task, 3) whether information flows top-down, bottom-up or both, and 4) whether their coupling is behaviorally relevant. We combined psychophysiological interaction analyses with dynamic causal modeling on the fMRI data of our previous study.
We found that functional coupling between multimodal and modality-specific areas strongly depended on the task, involved both top-down and bottom-up information flow, and predicted conceptually guided behavior. Notably, we also found coupling between different modality-specific areas and between different multimodal areas. These results suggest that functional coupling in the conceptual system is extensive, reciprocal, task-dependent, and behaviorally relevant.
We propose a new model of the conceptual system that incorporates task-dependent functional interactions between modality-specific and multimodal areas.