Researchers at the University of California, San Francisco have unveiled a pioneering brain-computer interface that can, for the first time, decode both speech and associated bodily movements from a single implanted device, enabling paralysed patients to express themselves through a digital avatar. The findings were published in the journal *Nature Neuroscience* on September 14th.
Samantha Brossler, the lead author of the study, explained that the project aims to evolve brain-computer interfaces from restoring a single function at a time towards a scenario where one interface can restore multiple modes of communication and movement. She highlighted that gestures add meaning, emphasis, emotion, and context, and in some cases can fully replace speech, such as nodding for agreement, shaking the head for refusal, or using gestures like waving, shrugging, or giving a thumbs-up.
In this proof-of-concept trial, the system was tested on two participants, who utilised ten and four distinct hand gestures, respectively. Leveraging machine learning algorithms, the technology accurately decoded both words and gestures from the participants' brain signals. The research team's next objective is to broaden the approach to encompass a larger vocabulary and a more continuous range of body movements.
Effective communication prostheses hold the potential to assist millions of individuals with intact or near-intact cognitive function who have lost the ability to speak due to stroke, neurodegenerative conditions like amyotrophic lateral sclerosis, or other brain disorders. In the long run, scientists aspire for this technology to also help those who struggle with vocalisation due to cerebral palsy and autism.
The field of communication prosthetics is drawing growing research attention and investment. Active players in this area include Elon Musk's Neuralink and Precision Neuroscience. Additionally, Echo Neurotechnologies, a company co-founded by Dr. Edward Chang, a principal investigator at UCSF, is developing brain-implant hardware.
Richard Rosch, a senior clinical lecturer and brain dynamics expert at King's College London, called the research innovative, offering new insights into how the brain combines language with other forms of communication. He noted that the study reveals a significant overlap in the brain regions responsible for processing language and gestures, adding evidence to the emerging literature suggesting the human brain may not be a network of highly specialised, interconnected but separate regions, but rather features substantial functional overlap across areas. However, he cautioned that the technology is far from widespread adoption, as it requires major brain surgery and performs less effectively when handling movements on the side of the body opposite the implant.
Dr. Scott Wellington from the University of Bath's Institute for Augmented Human Research praised the study but pointed out that significant hurdles remain before it can be translated into a standard assistive tool, including the need to customise the system for each patient. He stated that these technologies are highly individual-dependent; a model trained on one person's brain signals works for that specific individual but is difficult to generalise to others. Developing a universal model would require substantial investment, which could impede its broad clinical adoption.