Summary: Combining neuroimaging data and deep learning technologies, researchers have developed a model of the human brain that simulates patterns of neurological impairment better than existing models.
Source: University at Buffalo
He calls it his “chocolate and peanut butter moment.”
A University at Buffalo neuroimaging researcher has developed a computer model of the human brain that more realistically simulates actual patterns of brain impairment than existing methods. The novel advancement represents the union of two established approaches to create a digital simulation environment that could help stroke victims and patients with other brain injuries by serving as a testing ground for hypotheses about specific neurological damage.
“This model is tied accurately to the functional connectivity of the brain and is able to demonstrate realistic patterns of cognitive impairment,” says Christopher McNorgan, an assistant professor of psychology in UB’s College of Arts and Sciences. “Since the model reflects how the brain is connected, we can manipulate it in ways that provide insights, for example, into the areas of a patient’s brain that might be damaged.
“This recent work doesn’t prove that we have a digital facsimile of the human brain, but the findings indicate that the model is performing in a way that is consistent with how the brain performs, and that at least suggests that the model is taking on properties that are moving in the direction of possibly one day creating a facsimile.”
The findings provide a powerful means of identifying and understanding brain networks and how they function, which could lead to what once were unrealized possibilities for discovery and understanding.
Details on the model and the results of its testing appear in the journal NeuroImage.
Explaining McNorgan’s model starts with a look at the two fundamental components of its design: functional connectivity and multivariate pattern analyses (MVPA).
For many years, traditional brain-based models have relied on a general linear approach. This method looks at every spot in the brain and how those areas respond to stimuli. This approach is used in traditional studies of functional connectivity, which rely on functional magnetic resonance imaging (fMRI) to explore how the brain is wired. A linear model assumes a direct relationship between two things, such as the visual region of the brain becoming more or less active when a light flickers on or off.
While linear models excel at identifying which areas are active under certain conditions, they often fail to detect complicated relationships potentially existing among multiple areas. That’s the domain of more recent advances, like MVPA, a “teachable” machine-learning technique that operates on a more holistic level to evaluate how activity is patterned across brain regions.
MVPA is non-linear. Assume for instance that there’s a set of neurons dedicated to recognizing the meaning of a stop sign. These neurons are not active when we see something red or something octagonal because there’s not a one-to-one linear mapping between being red and being a stop sign (an apple isn’t a stop sign), nor between being octagonal and being a stop sign (a board room table isn’t a stop sign).
“A non-linear response ensures that they do light up when we see an object that is both red and octagonal,” explains McNorgan. “For this reason, non-linear methods like MVPA have been at the core of so-called ‘Deep Learning’ approaches behind technologies, such as the computer vision software required for self-driving cars.”
But MVPA uses brute force machine-learning techniques. The process is opportunistic, sometimes confusing coincidence with correlation. Even ideal models require researchers to provide evidence that activity in the theoretical model would also be present under the same conditions in the brain.
On their own, both traditional functional connectivity and MVPA approaches have limitations, and integrating results generated by each of these approaches requires considerable effort and expertise for brain researchers to puzzle out the evidence.
When combined, however, the limitations are mutually constrained — and McNorgan is the first researcher to successfully integrate functional connectivity and MVPA to develop a machine-learning model that’s explicitly grounded in real-world functional connections among brain regions. In other words, the mutually constrained results are a self-assembling puzzle.
“It was my chocolate and peanut butter moment,” says McNorgan, an expert in neuroimaging and computational modeling.
“I’ve had a particular career trajectory that has allowed me to work extensively with different theoretical models. That background provided a particular set of experiences that made the combination seem obvious in hindsight.”
To build his models, McNorgan begins by gathering the brain data that will teach them the patterns of brain activity that are associated with each of three categories – in this case, tools, musical instruments and fruits. These data came from 11 participants who imagined the appearance and sound of familiar category examples, like hammers, guitars and apples, while undergoing an MRI scan. These scans indicate which areas are more or less active based on blood oxygen levels.
“There are certain patterns of activity across the brain that are consistent with thinking about one category versus another,” says McNorgan. “We might think of this as a neural fingerprint.”
These MRI patterns were then digitized and used to train a series of computer models to recognize which activity patterns were associated with each category.
“After training, models are given previously unseen activity patterns,” he explains. “Significantly above-chance classification accuracy indicates that the models have learned a generalizable relationship between specific brain activity patterns and thinking about a specific category.”
To test whether the digital brain models produced by this new method were more realistic, McNorgan gave them “virtual lesions” by disrupting activations in regions known to be important for each of the categories.
He found that the mutually constrained models showed classification errors consistent with the lesion location. For example, lesions to areas thought to be important for representing tools disrupted accuracy for tool patterns, but not the other two categories. By comparison, other versions of models not trained using the new method did not show this behavior.
“The model now suggests how brain areas that might not appear to be important for encoding information when considered individually may be important when it’s functioning as part of a larger configuration or network,” he says. “Knowing these areas may help us understand why someone who suffered a stroke or other injury is having trouble making these distinctions.”
Source:
University at Buffalo
Media Contacts:
Bert Gambini – University at Buffalo
Image Source:
The image is in the public domain.
Original Research: Open access
“Integrating functional connectivity and MVPA through a multiple constraint network analysis”. Christopher McNorgan et al.
Neurolmage doi:10.1016/j.neuroimage.2019.116412.
Abstract
Integrating functional connectivity and MVPA through a multiple constraint network analysis
Traditional general linear model-based brain mapping efforts using functional neuroimaging are complemented by more recent multivariate pattern analyses (MVPA) that apply machine learning techniques to identify the cognitive states associated with regional BOLD activation patterns, and by connectivity analyses that identify networks of interacting regions that support particular cognitive processes. We introduce a novel analysis representing the union of these approaches, and explore the insights gained when MVPA and functional connectivity analyses are allowed to mutually constrain each other within a single model. We explored multisensory semantic representations of concrete object concepts using a self-paced multisensory imagery task. Multilayer neural networks learned the real-world categories associated with macro-scale cortical BOLD activity patterns from the task, with some models additionally encoding regional functional connectivity. Models trained to encode functional connections demonstrated superior classification accuracy and more pronounced lesion-site appropriate category-specific impairments. We replicated these results in a data set from the openneuro.org open fMRI data repository. We conclude that mutually constrained network analyses encourage parsimonious models that may benefit from improved biological plausibility and facilitate discovery.
