Researchers working with a paralyzed participant who has sensors implanted in his brain have deciphered the brain activity associated with trying to write letters by hand.

They used an algorithm to identify letters as he attempted to write them. Then, the system displayed the text on a screen - in real time. 

Brain-computer interfaces convert thought into action. By attempting handwriting, the study participant typed 90 characters per minute - more than double the previous record for typing with such a "brain-computer interface."


Two tiny arrays of implanted electrodes relayed information from the brain area that controls the hands and arms to an algorithm, which translated it into letters that appeared on a screen. Credit: F. Willett et al./Nature 2021/Erika Woodrum
Thought-powered communication

When an injury or disease robs a person of the ability to move, the brain's neural activity for walking, grabbing a cup of coffee, or speaking a sentence remain. Researchers can tap into this activity to help people with paralysis or amputations regain lost abilities.

The need varies with the nature of the disability. Some people who have lost the use of their hands can still use a computer with speech recognition and other software. For those who have difficulty speaking, scientists have been developing other ways to help people communicate.

In recent years, the team has decoded the neural activity associated with speech in the hopes of reproducing it. They also devised a way for participants with implanted sensors to use their thoughts associated with attempted arm movements to move a cursor on a screen. Pointing at and clicking on letters in this way let people type about 40 characters per minute, the previous speed record for typing with a brain computer interface (BCI). 

Dr. Frank Willett wondered if it might be possible to harness the brain signals evoked by putting pen to paper. The team worked with a participant enrolled in a clinical trial called BrainGate2, which is testing the safety of BCIs that relay information directly from a participant's brain to a computer. They implanted two tiny sensors into the part of the brain that controls the hand and arm, making it possible for the person to, for example, move a robotic arm or a cursor on a screen by attempting to move their own paralyzed arm. 



The participant, who was 65 years old at the time of the research, had a spinal cord injury that left him paralyzed from the neck down. Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter. With this system, the man could copy sentences and answer questions at a rate similar to that of someone his age typing on a smartphone.

This so-called "Brain-to-Text" BCI is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another. 

The new system could potentially help those suffering from paralysis caused by a number of conditions, Henderson adds. Those include brain stem stroke, which afflicted Jean-Dominique Bauby, the author of the book The Diving Bell and the Butterfly. "He was able to write this moving and beautiful book by selecting characters painstakingly, one at a time, using eye movement," says Stanford neurosurgeon Dr. Jaimie Henderson. "Imagine what he could have done with Frank's handwriting interface!"