Abstract
In real-world applications, robustness against noise is crucial for small-footprint keyword spotting (KWS) systems which are deployed on resource-limited devices. To improve the noise robustness, a reasonable approach is employing a speech enhancement model to enhance the noisy speeches first. However, current enhancement models need a lot of parameters and computation, which do not satisfy the small-footprint requirement. In this paper, we design a lightweight enhancement model, which consists of the convolutional layers for feature extracting, recurrent layers for temporal modeling and deconvolutional layers for feature recovering. To reduce the mismatch between the enhanced features and KWS system desired ones, we further propose an efficient joint training framework, in which the enhancement model and KWS system are concatenated and jointly fine-tuned through a trainable feature transformation block. With the joint training, linguistic information can back-propagate from the KWS system to the enhancement model and guide its training. Our experimental results show that the proposed small-footprint enhancement model significantly improves the noise robustness of KWS systems without much increasing model or computation complexity. Moreover, the recognition performance can be further improved through the proposed joint training framework.
This research was supported in part by the China National Nature Science Foundation (No. 61876214, No. 61866030).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Du, J., Wang, Q., Gao, T., Xu, Y., Dai, L.R., Lee, C.H.: Robust speech recognition with speech enhanced deep neural networks. In: INTERSPEECH (2014)
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML 37, 448–456 (2015)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (Poster) (2015)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: ICML. pp. 807–814 (2010)
Narayanan, A., Wang, D.: Ideal ratio mask estimation using deep neural networks for robust speech recognition. In: ICASSP. pp. 7092–7096 (2013)
Prabhavalkar, R., Alvarez, R., Parada, C., Nakkiran, P., Sainath, T.N.: Automatic gain control and multi-style training for robust small-footprint keyword spotting with deep neural networks. In: ICASSP. pp. 4704–4708 (2015)
Sainath, T., Parada, C.: Convolutional neural networks for small-footprint keyword spotting. In: INTERSPEECH (2015)
Seltzer, M.L., Yu, D., Wang, Y.: An investigation of deep neural networks for noise robust speech recognition. In: ICASSP. pp. 7398–7402 (2013)
Shan, C., Zhang, J., Wang, Y., Xie, L.: Attention-based end-to-end models for small-footprint keyword spotting. pp. 2037–2041 (2018)
Tan, K., Wang, D.: A convolutional recurrent neural network for real-time speech enhancement. In: INTERSPEECH. pp. 3229–3233 (2018)
Tang, R., Lin, J.: Honk: A pytorch reimplementation of convolutional neural net- works for keyword spotting. arXiv preprint arXiv:1710.06554 (2017)
Tang, R., Lin, J.: Deep residual learning for small-footprint keyword spotting. In: ICASSP. pp. 5484–5488 (2018)
Wang, D., Chen, J.: Supervised speech separation based on deep learning: an overview. IEEE/ACM Trans. Audio, Speech, and Language Process. 26(10), 1702–1726 (2018)
Wang, Y., Narayanan, A., Wang, D.: On training targets for supervised speech separation. IEEE/ACM Trans. Audio, Speech, and Language Process. 22(12), 1849–1858 (2014)
Wang, Z.Q., Wang, D.: A joint training framework for robust automatic speech recognition. IEEE/ACM Trans. Audio, Speech, and Language Process. 24(4), 796–806 (2016)
Warden, P.: Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209 (2018)
Yu, M., et al.: Text-dependent speech enhancement for small-footprint robust keyword detection. In: Interspeech. pp. 2613–2617 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Gu, Y., Du, Z., Zhang, H., Zhang, X. (2020). An Efficient Joint Training Framework for Robust Small-Footprint Keyword Spotting. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-63830-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63829-0
Online ISBN: 978-3-030-63830-6
eBook Packages: Computer ScienceComputer Science (R0)