Summary: Researchers developed an artificial intelligence system that can detect day-to-day changes in speech that may hint of mental health decline.
Source: University of Colorado at Boulder
Thanks to advances in artificial intelligence, computers can now assist doctors in diagnosing disease and help monitor patient vital signs from hundreds of miles away.
Now, CU Boulder researchers are working to apply machine learning to psychiatry, with a speech-based mobile app that can categorize a patient’s mental health status as well as or better than a human can.
“We are not in any way trying to replace clinicians,” says Peter Foltz, a research professor at the Institute of Cognitive Science and co-author of a new paper in Schizophrenia Bulletin that lays out the promise and potential pitfalls of AI in psychiatry. “But we do believe we can create tools that will allow them to better monitor their patients.”
Nearly one in five U.S. adults lives with a mental illness, many in remote areas where access to psychiatrists or psychologists is scarce. Others can’t afford to see a clinician frequently, don’t have time or can’t get in to see one.
Even when a patient does make it in for an occasional visit, therapists base their diagnosis and treatment plan largely on listening to a patient talk – an age-old method that can be subjective and unreliable, notes paper co-author Brita Elvevåg, a cognitive neuroscientist at the University of Tromsø, Norway.
“Humans are not perfect. They can get distracted and sometimes miss out on subtle speech cues and warning signs,” Elvevåg says. “Unfortunately, there is no objective blood test for mental health.”
In pursuit of an AI version of that blood test, Elvevåg and Foltz teamed up to develop machine learning technology able to detect day-to-day changes in speech that hint at mental health decline.
For instance, sentences that don’t follow a logical pattern can be a critical symptom in schizophrenia. Shifts in tone or pace can hint at mania or depression. And memory loss can be a sign of both cognitive and mental health problems.
“Language is a critical pathway to detecting patient mental states,” says Foltz. “Using mobile devices and AI, we are able to track patients daily and monitor these subtle changes.”
The new mobile app asks patients to answer a 5- to 10-minute series of questions by talking into their phone.
Among various other tasks, they’re asked about their emotional state, asked to tell a short story, listen to a story and repeat it and given a series of touch-and-swipe motor skills tests.
In collaboration with Chelsea Chandler, a computer science graduate student at CU Boulder, and other colleagues, they developed an AI system that assesses those speech samples, compares them to previous samples by the same patient and the broader population and rates the patient’s mental state.
In one recent study, the team asked human clinicians to listen to and assess speech samples of 225 participants – half with severe psychiatric issues; half healthy volunteers – in rural Louisiana and Northern Norway. They then compared those results to those of the machine learning system.
“We found that the computer’s AI models can be at least as accurate as clinicians,” says Foltz.
He and his colleagues envision a day when AI systems they’re developing for psychiatry could be in the room with a therapist and a patient to provide additional data-driven insight, or serve as a remote-monitoring system for the severely mentally ill.
If the app detected a worrisome change, it could notify the patient’s doctor to check in.
“Patients often need to be monitored with frequent clinical interviews by trained professionals to avoid costly emergency care and unfortunate events,” says Foltz. ” But there are simply not enough clinicians for that.”
Foltz previously helped develop and commercialize an AI-based essay-grading technology which is now broadly used.
In their new paper, the researchers lay out a call to action for larger studies to prove efficacy and earn public trust before AI technology could be broadly brought into clinical practice for psychiatry.
“The mystery around AI does not nurture trustworthiness, which is critical when applying medical technology,” they write. “Rather than looking for machine learning models to become the ultimate decision-maker in medicine, we should leverage the things that machines do well that are distinct from what humans do well.”
Using Machine Learning in Psychiatry: The Need to Establish a Framework That Nurtures Trustworthiness
The rapid embracing of artificial intelligence in psychiatry has a flavor of being the current “wild west”; a multidisciplinary approach that is very technical and complex, yet seems to produce findings that resonate. These studies are hard to review as the methods are often opaque and it is tricky to find the suitable combination of reviewers. This issue will only get more complex in the absence of a rigorous framework to evaluate such studies and thus nurture trustworthiness. Therefore, our paper discusses the urgency of the field to develop a framework with which to evaluate the complex methodology such that the process is done honestly, fairly, scientifically, and accurately. However, evaluation is a complicated process and so we focus on three issues, namely explainability, transparency, and generalizability, that are critical for establishing the viability of using artificial intelligence in psychiatry. We discuss how defining these three issues helps towards building a framework to ensure trustworthiness, but show how difficult definition can be, as the terms have different meanings in medicine, computer science, and law. We conclude that it is important to start the discussion such that there can be a call for policy on this and that the community takes extra care when reviewing clinical applications of such models.