Summary: Detecting and decoding activity in brain regions associated with visual processing allowed researchers to determine what a person sees.
Source: WUSTL
Some people are trapped within their own minds, able to think and feel but unable to express themselves because brain injury or disease has damaged their lines of communication with the outside world.
As a step toward helping people in such situations communicate, scientists at Washington University School of Medicine in St. Louis have demonstrated that they can use light to detect what is going on inside someone’s head. The researchers use LED light beamed from the outside of the head inward to detect activity in the area of the brain responsible for visual processing, and then decode brain signals to determine what the person sees. Accomplishing this feat required the development of neuroimaging tools and analysis techniques that move the field a step closer to solving the much more complex problem of decoding language.
The study, available online in the journal NeuroImage, demonstrates that high-density diffuse optical tomography (HD-DOT) — a noninvasive, wearable, light-based brain imaging technology — is sensitive and precise enough to be potentially useful in applications such as augmented communication that are not well suited to other imaging methods.
“MRI could be used for decoding, but it requires a scanner, and you can’t expect someone to go lie in a scanner every time they want to communicate,” said senior author Joseph P. Culver, PhD, the Sherwood Moore Professor of Radiology at Washington University’s Mallinckrodt Institute of Radiology. “With this optical method, users would be able to sit in a chair, put on a cap and potentially use this technology to communicate with people. We’re not quite there yet, but we’re making progress. What we’ve shown in this paper is that, using optical tomography, we can decode some brain signals with an accuracy above 90%, which is very promising.”
When the neuronal activity increases in any region of the brain, oxygenated blood rushes in to fuel the activity. HD-DOT uses light to detect the rush of blood. Participants wear a cap fitted with dozens of fibers that relay light from tiny LEDs to the head. After the light is transmitted through the head, detectors capture dynamic changes in the colors of the brain tissue as a result of changes in blood flow.
Culver, first author and graduate student Kalyan Tripathy, and colleagues set out to evaluate the potential of HD-DOT for decoding brain signals. They started with the visual system because it is one of the best understood brain functions. Neuroscientists long ago worked out a detailed map of the visual part of the brain by showing participants flashing checkerboard patterns on a screen and identifying the 3D units, known as voxels, in the brain that became active in response to each pattern. Decoding is the attempt to reverse the process: Detecting active voxels and then deducing which checkerboard pattern triggered that pattern of brain activity.
“We know what the participant is seeing, so we can verify how well our decoding matches up to reality,” said Culver, also a professor of physics, of electrical and systems engineering, and of biomedical engineering. “By going to something that was well validated, we could optimize the experimental design, push harder on the statistics of the decoding and obtain performance that is really very high.”
The researchers started simple. They recruited five participants for multiple five- to ten-minute runs in which the participants were shown a checkerboard pattern on either the left or the right side of the visual field for a few seconds at a time, interspersed with breaks during which no image was shown.
Using one run as the template, the researchers analyzed the data from another run to determine when the checkerboard was on which side of the screen. They repeated this analysis using different runs as the template and the test until they had analyzed all possible pairings.
The researchers were able to identify the correct position of the checkerboard — left, right or not visible at all — with 75% to 98% accuracy. While decoding was more successful when the same person was used for the template run and the test run, the patterns from one person could be used to decode the brain activity of another person.
Then, the researchers made the problem more complex. They showed participants a checkerboard wedge that rotated at 10 degrees a second. Three participants sat for six seven-minute runs on two separate days. Using the same template and test run strategy, the researchers were able to pinpoint the position of the wedge within 26 degrees.
The results are a first step toward the ultimate goal of facilitating communication for people who struggle to express themselves because of cerebral palsy, stroke or other conditions that result in locked-in syndrome, the researchers said.
“It sounds like a huge jump, from checkerboards to figuring out what words somebody is internally verbalizing to oneself,” Culver said. “But a lot of the principles are the same. The goal is to help people communicate, and what we’ve learned by decoding these visual stimuli is a solid step toward that goal.”
Funding: This work was funded by the National Institutes of Health (NIH), grant numbers U01EB027005 and R01NS090874; and the McDonnell Foundation.
About this neurotech research news
Source: WUSTL
Contact: Judy Martin Finch – WUSTL
Image: The image is in the public domain
Original Research: Closed access.
“Decoding visual information from high-density diffuse optical tomography neuroimaging data” by Tripathy K, Markow ZE, Fishell AK, Sherafati A, Burns-Yocum TM, Schroeder ML, Svoboda AM, Eggebrecht AT, Anastasio MA, Schlaggar BL, Culver JP. NeuroImage
Abstract
Decoding visual information from high-density diffuse optical tomography neuroimaging data
Background
Neural decoding could be useful in many ways, from serving as a neuroscience research tool to providing a means of augmented communication for patients with neurological conditions. However, applications of decoding are currently constrained by the limitations of traditional neuroimaging modalities. Electrocorticography requires invasive neurosurgery, magnetic resonance imaging (MRI) is too cumbersome for uses like daily communication, and alternatives like functional near-infrared spectroscopy (fNIRS) offer poor image quality. High-density diffuse optical tomography (HD-DOT) is an emerging modality that uses denser optode arrays than fNIRS to combine logistical advantages of optical neuroimaging with enhanced image quality. Despite the resulting promise of HD-DOT for facilitating field applications of neuroimaging, decoding of brain activity as measured by HD-DOT has yet to be evaluated.
Objective
To assess the feasibility and performance of decoding with HD-DOT in visual cortex.
Methods and Results
To establish the feasibility of decoding at the single-trial level with HD-DOT, a template matching strategy was used to decode visual stimulus position. A receiver operating characteristic (ROC) analysis was used to quantify the sensitivity, specificity, and reproducibility of binary visual decoding. Mean areas under the curve (AUCs) greater than 0.97 across 10 imaging sessions in a highly sampled participant were observed. ROC analyses of decoding across 5 participants established both reproducibility in multiple individuals and the feasibility of inter-individual decoding (mean AUCs > 0.7), although decoding performance varied between individuals. Phase-encoded checkerboard stimuli were used to assess more complex, non-binary decoding with HD-DOT. Across 3 highly sampled participants, the phase of a 60° wide checkerboard wedge rotating 10° per second through 360° was decoded with a within-participant error of 25.8±24.7°. Decoding between participants was also feasible based on permutation-based significance testing.
Conclusions
Visual stimulus information can be decoded accurately, reproducibly, and across a range of detail (for both binary and non-binary outcomes) at the single-trial level (without needing to block-average test data) using HD-DOT data. These results lay the foundation for future studies of more complex decoding with HD-DOT and applications in clinical populations.
