Donate to Science & Enterprise

S&E on Mastodon

S&E on LinkedIn

S&E on Flipboard

Please share Science & Enterprise

Brain Signals Drive Computer Cursor for Faster Typing

Paul Nuyujukian

Paul Nuyujukian (L.A. Cicero, Stanford University)

13 September 2016. A system capturing motor signals in the brain is shown in tests with monkeys to direct a cursor to type text on a keyboard display at rates as high as 12 words per minute. A team from the lab of bioengineering professor Krishna Shenoy at Stanford University published its findings yesterday in the journal Proceedings of the IEEE (paid subscription required).

The research team led by first author and postdoctoral fellow Paul Nuyujukian is seeking a more efficient connection between the brain and digital systems for people with paralysis and other movement disorders to communicate with computers and mobile devices. Most systems being studied or tested now, say the authors, track muscular activity such as eye movements or facial muscles, which may be difficult for some people to control. Results of tests so far with these systems shows producing text is laborious and slow.

Nuyujukian and colleagues are building on previous work that developed small multi-electrode arrays implanted in hand and arm motor regions of the brain, as well as decoder algorithms that improve the speed and reliability of two-dimensional cursor control on display screens. The team previously tested individual algorithms with monkeys, but not together in an integrated system.

The researchers tested the electrode arrays and algorithms on two rhesus monkeys, named J and L, who were earlier trained to type letters displayed on computer screens. J and L then were shown text passages, with corresponding nerve signals to motor regions of the brain captured by the implanted electrodes. Those brain signals were then translated by the decoding algorithms into electronic signals to direct the cursor over a keyboard display on a computer screen.

The monkeys were shown text from a New York Times article and a passage from Shakespeare’s Hamlet, which J typed at an average rate of 10.0 words per minute and L produced at 7.2 words per minute. Adding a clicking sound to the text production increased the rates to 12.0 and 7.8 words per minute respectively. The authors say these rates are 3 times faster than earlier methods.

While the tests show the feasibility of the technology, the authors acknowledge real-life factors could slow down typing speeds. “The interface we tested is exactly what a human would use,” says Nuyujukian in a university statement. “What we cannot quantify is the cognitive load of figuring out what words you are trying to say.” That cognitive load would also include factors such as thinking of correct spellings and filtering out distractions.

Shenoy and Nuyujukian are also taking part in the BrainGate project, which includes an early-stage clinical trial testing the safety and feasibility of direct brain-to-computer signaling technology with individuals having severe movement disorders including spinal cord injuries and amyotrophic lateral sclerosis, or ALS. Science & Enterprise reported several times on this project, including a story in 2012 on development of algorithms by Shenoy and Nuyujukian to control on-screen cursors with brain signals.

The following video gives a demonstration of the technology used to type a classic line from Hamlet.

Read more:

*     *     *

Comments are closed.