Researchers decoded attempted speech and hand gestures at the same time from one brain implant. Three participants with severe paralysis took part. Each had a high-density electrode array placed on the surface of the motor cortex. The team first showed that isolated arm and mouth movements could be classified reliably. Two decoders then ran in parallel to animate a personalised full-body avatar. Participants attempted speech and gestures either together or separately. Models trained only on isolated movements did not transfer well to combined attempts. Adding combined-attempt recordings to the training data improved accuracy in both settings. The authors call this a proof of concept rather than a clinical system. Brosler and colleagues published the work in Nature Neuroscience.
What changed
Earlier implants decoded either speech or gestures, but not both together.
What it unlocks
Driving a virtual avatar's speech and gestures from one implant at once.
- 3 implanted participants
- chance level 2.22%
- 10 cross-validation folds
What you need to act on it
- a surgically implanted high-density ECoG array
- research trial enrollment
Sources