Scientists translate brain waves of man with paralysis into words.
• Anarthria, which is the loss of the ability to speak, can have a variety of causes, such as a stroke or amyotrophic lateral sclerosis.
• People with anarthria often retain their linguistic ability, but paralysis may prevent them from using technological aids to communicate.
• Neuroscientists used machine learning to translate into words and sentences electrical activity in the part of the brain responsible for speech.
For the first time, scientists have managed to translate into text the brain activity of a person with paralysis who is unable to speak.
Neuroscientists implanted an electrode array the size of a credit card onto the 36-year-old’s sensorimotor cortex, which is the part of the brain that controls the articulation of speech.
The scientists, led by the University of California, San Francisco (UCSF), then used “deep-learning algorithms” to train computer models to recognize and classify words from patterns in the participant’s brain activity.
They asked the man to think about saying words that they presented to him on a screen.
In addition to the deep-learning models, they used a “natural-language model,” which is a program that predicts the next word in a sentence based on the previous words.
“This outcome really is a milestone toward restoring speech for people who are unable to communicate due to paralysis,” said David Moses, a postdoctoral engineer at UCSF and lead author of the study, in an interview with the San Francisco Chronicle.
“There have been other existing approaches, but this is the first proof that someone can attempt to speak and that we can translate what they were trying to say from that brain activity.”
Loss of intelligible speech
The participant, who was 36 at the start of the study, had a stroke in his brain stem when he was 20 that resulted in severe paralysis and loss of intelligible speech, known as anarthria.
Other causes of anarthria include amyotrophic lateral sclerosis, a rare neurological condition that mainly affects the nerves responsible for voluntary movement.
The man in the study could vocalize grunts and moans but was unable to articulate words, despite having healthy cognitive functions.
Using slight movements of his head, he can control a computer-based typing device to communicate. However, his typing speed with this device is only around five correct words per minute.
After 48 training sessions with the new “mind-reading” technology over the course of 81 weeks, he was able to generate around 15 words per minute, with an error rate of 26%.
The scientists, who report their findings in The New England Journal of Medicine, say speech decoding technologies are generally considered usable if they have a word error rate below 30%.
“This study represents a transformational breakthrough in the field of brain-computer interfaces,” said Dr. Lee H. Schwamm, who chairs the advisory committee of the American Stroke Association and was not involved in the new research.
“While the intervention is quite invasive, requiring brain surgery to implant a recording strip on the surface of the brain, and the ‘thought-to-spoken’ conversion accuracy was modest, the paradigm is groundbreaking,” Dr. Schwamm told Medical News Today.
Dr. Schwamm is vice president, virtual care, at Mass General Brigham and professor of neurology at Harvard Medical School in Boston, MA.
He said that while previous research has used a similar interface to translate thoughts about movement into control of a robotic arm, it is an even bigger leap to decipher words.
“What is most astounding is that the area of the brain for sensor implant, the sensory-motor cortex, is not thought to play an important role in language understanding or word generation,” he added.
The area is involved in moving the lips and throat to produce speech sounds, so the next step may be to see whether it can also assist with people with aphasia, which is a more common cause of disabling language disturbance after a stroke, said Dr. Schwamm.
