We report on a Deep Neural Network frontend for a continuous speech recognizer based on Surface Electromyography (EMG). Speech data is obtained by facial electrodes capturing the electric activity generated by the articulatory muscles, thus allowing speech processing without making use of the acoustic signal. The electromyographic signal is preprocessed and fed into the neural network, which is trained on framewise targets; the output layer activations are further processed by a Hidden Markov sequence classifier. We show that such a neural network frontend can be trained on EMG data and yields substantial improvements over previous systems, despite the fact that the available amount of data is very small, just amounting to a few tens of sentences: on the EMG-UKA corpus, we obtain average evaluation set Word Error Rate improvements of more than 32% relative on context-independent phone models and 13% relative on versatile Bundled Phonetic feature (BDPF) models, compared to a conventional system using Gaussian Mixture Models. In particular, on simple context-independent phone models, the new system yields results which are almost as good as with BDPF models, which were specifically designed to cope with small amounts of training data.
Cite as: Wand, M., Schmidhuber, J. (2016) Deep Neural Network Frontend for Continuous EMG-Based Speech Recognition. Proc. Interspeech 2016, 3032-3036, doi: 10.21437/Interspeech.2016-340
@inproceedings{wand16_interspeech,
author={Michael Wand and Jürgen Schmidhuber},
title={{Deep Neural Network Frontend for Continuous EMG-Based Speech Recognition}},
year=2016,
booktitle={Proc. Interspeech 2016},
pages={3032--3036},
doi={10.21437/Interspeech.2016-340}
}