Speech synthesis from neural decoding of spoken sentences.
Level 4 - case-series / case-control
Experimental human neurophysiology and computational decoding study without a control group.
PubMed 31019317 · doi:10.1038/s41586-019-1119-1
What was done
A neural decoding framework was designed to synthesize audible speech from directly recorded human cortical activity. Recurrent neural networks first decoded cortical signals into intermediate representations of vocal tract articulatory movements, which were then transformed into speech acoustics. Performance was evaluated through closed-vocabulary listener transcription tests, cross-participant transfer tests, and during silent sentence miming.
What was found
The abstract reports no numerical values, transcription accuracy percentages, or sample sizes. Qualitatively, listeners were able to identify and transcribe speech synthesized from cortical activity in closed-vocabulary tests. Incorporating intermediate articulatory dynamics improved synthesis performance with limited data, articulatory representations were conserved across speakers allowing partial decoder transferability between individuals, and speech could be synthesized when a participant silently mimed sentences.
Why it matters
This work demonstrates that mapping cortical signals through intermediate articulatory kinematic representations enables the synthesis of audible, intelligible sentences directly from human brain activity, providing a framework for speech neuroprostheses to restore communication.
Limits
The abstract provides no quantitative performance metrics, confidence intervals, or sample size specifications. Evaluation was restricted to closed-vocabulary contexts, and performance during natural, open-vocabulary spontaneous speech was not reported.
Cited by
- supports Neuroengineering research led by Dr. Edward Chang has mapped neural activity to vocal tract control (larynx and pharynx) to decode speech and enable communication in paralyzed patients.