Summary: Researchers report that when the brain “reads” or decodes a sentence in English or Portuguese, its neural activation patterns are the same.
Source: Carnegie Mellon University.
An international research team led by Carnegie Mellon University has found that when the brain “reads” or decodes a sentence in English or Portuguese, its neural activation patterns are the same.
Published in NeuroImage, the study is the first to show that different languages have similar neural signatures for describing events and scenes. By using a machine-learning algorithm, the research team was able to understand the relationship between sentence meaning and brain activation patterns in English and then recognize sentence meaning based on activation patterns in Portuguese. The findings can be used to improve machine translation, brain decoding across languages and, potentially, second language instruction.
“This tells us that, for the most part, the language we happen to learn to speak does not change the organization of the brain,” said Marcel Just, the D.O. Hebb University Professor of Psychology and pioneer in using brain imaging and machine-learning techniques to identify how the brain deciphers thoughts and concepts.
“Semantic information is represented in the same place in the brain and the same pattern of intensities for everyone. Knowing this means that brain to brain or brain to computer interfaces can probably be the same for speakers of all languages,” Just said.
For the study, 15 native Portuguese speakers — eight were bilingual in Portuguese and English — read 60 sentences in Portuguese while in a functional magnetic resonance imaging (fMRI) scanner. A CMU-developed computational model was able to predict which sentences the participants were reading in Portuguese, based only on activation patterns.
The computational model uses a set of 42 concept-level semantic features and six markers of the concepts’ roles in the sentence, such as agent or action, to identify brain activation patterns in English.
With 67 percent accuracy, the model predicted which sentences were read in Portuguese. The resulting brain images showed that the activation patterns for the 60 sentences were in the same brain locations and at similar intensity levels for both English and Portuguese sentences.
Additionally, the results revealed the activation patterns could be grouped into four semantic categories, depending on the sentence’s focus: people, places, actions and feelings. The groupings were very similar across languages, reinforcing the organization of information in the brain is the same regardless of the language in which it is expressed.
“The cross-language prediction model captured the conceptual gist of the described event or state in the sentences, rather than depending on particular language idiosyncrasies. It demonstrated a meta-language prediction capability from neural signals across people, languages and bilingual status,” said Ying Yang, a postdoctoral associate in psychology at CMU and first author of the study.
Discovering that the brain decodes sentences the same in different languages is one of the many brain research breakthroughs to happen at Carnegie Mellon. CMU has created some of the first cognitive tutors, helped to develop the Jeopardy-winning Watson, founded a groundbreaking doctoral program in neural computation, and is the birthplace of artificial intelligence and cognitive psychology. Building on its strengths in biology, computer science, psychology, statistics and engineering, CMU launched BrainHub, an initiative that focuses on how the structure and activity of the brain give rise to complex behaviors.
Funding: The Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via the U.S. Air Force Research Laboratory (AFRL), funded this research.
Source: Carnegie Mellon University
Image Source: This NeuroscienceNews.com image is adapted from the CMU press release.
Original Research: Abstract for “Commonality of neural representations of sentences across languages: Predicting brain activation during Portuguese sentence comprehension using an English-based model of brain function” by Ying Yang, Jing Wang, Cyntia Bailer, Vladimir Cherkassky, Marcel Adam Just in NeuroImage. Published online October 192016 doi:10.1016/j.neuroimage.2016.10.029
[cbtabs][cbtab title=”MLA”]Carnegie Mellon University. “New Theory Debunks Idea That Math Abilities Are Inate.” NeuroscienceNews. NeuroscienceNews, 4 November 2016.
<https://neurosciencenews.com/brain-reading-language-5433/>.[/cbtab][cbtab title=”APA”]Carnegie Mellon University. (2016, November 4). New Theory Debunks Idea That Math Abilities Are Inate. NeuroscienceNews. Retrieved November 4, 2016 from https://neurosciencenews.com/brain-reading-language-5433/[/cbtab][cbtab title=”Chicago”]Carnegie Mellon University. “New Theory Debunks Idea That Math Abilities Are Inate.” https://neurosciencenews.com/brain-reading-language-5433/ (accessed November 4, 2016).[/cbtab][/cbtabs]
Abstract
Commonality of neural representations of sentences across languages: Predicting brain activation during Portuguese sentence comprehension using an English-based model of brain function
The aim of the study was to test the cross-language generative capability of a model that predicts neural activation patterns evoked by sentence reading, based on a semantic characterization of the sentence. In a previous study on English monolingual speakers (Wang et al., submitted), a computational model performed a mapping from a set of 42 concept-level semantic features (Neurally Plausible Semantic Features, NPSFs) as well as 6 thematic role markers to neural activation patterns (assessed with fMRI), to predict activation levels in a network of brain locations. The model used two types of information gained from the English-based fMRI data to predict the activation for individual sentences in Portuguese. First, it used the mapping weights from NPSFs to voxel activation levels derived from the model for English reading. Second, the brain locations for which the activation levels were predicted were derived from a factor analysis of the brain activation patterns during English reading. These meta-language locations were defined by the clusters of voxels with high loadings on each of the four main dimensions (factors), namely people, places, actions and feelings, underlying the neural representations of the stimulus sentences.
This cross-language model succeeded in predicting the brain activation patterns associated with the reading of 60 individual Portuguese sentences that were entirely new to the model, attaining accuracies reliably above chance level. The prediction accuracy was not affected by whether the Portuguese speaker was monolingual or Portuguese-English bilingual. The model’s confusion errors indicated an accurate capture of the events or states described in the sentence at a conceptual level. Overall, the cross-language predictive capability of the model demonstrates the neural commonality between speakers of different languages in the representations of everyday events and states, and provides an initial characterization of the common meta-language neural basis.
“Commonality of neural representations of sentences across languages: Predicting brain activation during Portuguese sentence comprehension using an English-based model of brain function” by Ying Yang, Jing Wang, Cyntia Bailer, Vladimir Cherkassky, Marcel Adam Just in NeuroImage. Published online October 192016 doi:10.1016/j.neuroimage.2016.10.029
