Abstract
Keyword spotting is widely used in device wake-up and user interaction of smart devices. However, the resources in smart devices are limited. In order to ensure that the keyword spotting system can always run efficiently and accurately in smart devices, the size of the model must be compressed to realize a small and compact model. In addition, in actual application scenarios, noise, speech rate, and overlapped speech pose great challenges to the robustness of the model. In order to solve the above problems, we propose a depthwise-separable residual capsule neural network, which uses depthwise-separable convolution to achieve a more compact design, and also uses a multi-scale capsule classifier to improve the model’s robustness in the above complex scenarios. We have achieved the best accuracy on the Google Command dataset, and have fewer calculations and fewer parameters than the previous methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wu, M., Panchapagesan, S., Sun, M., et al.: Monophone-based background modeling for two-stage on-device wake word detection. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5494–5498. IEEE (2018)
Ge, F., Yan, Y.: Deep neural network based wake-up-word speech recognition with two-stage detection. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2761–2765. IEEE (2017)
Chen, G., Parada, C., Heigold, G.: Small-footprint keyword spotting using deep neural networks. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4087–4091. IEEE (2014)
Sainath, T.N., Parada, C.: Convolutional neural networks for small-footprint keyword spotting. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)
Tang, R., Lin, J.: Deep residual learning for small-footprint keyword spotting. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5484–5488. IEEE (2018)
Zhang, Y., Suda, N., Lai, L., et al.: Hello edge: keyword spotting on microcontrollers. arXiv preprint arXiv:1711.07128 (2017)
Arik, S.O., Kliegl, M., Child, R., et al.: Convolutional recurrent neural networks for small-footprint keyword spotting. arXiv preprint arXiv:1703.05390 (2017)
Bai, Y., Yi, J., Tao, J., et al.: A time delay neural network with shared weight self-attention for small-footprint keyword spotting. In: INTERSPEECH 2019, pp. 2190–2194 (2019)
Choi, S., Seo, S., Shin, B., et al.: Temporal convolution for real-time keyword spotting on mobile devices. arXiv preprint arXiv:1904.03814 (2019)
Chen, X., Yin, S., Song, D., et al.: Small-footprint keyword spotting with graph convolutional network. In: 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 539–546. IEEE (2019)
Li, X., Wei, X., Qin, X.: Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution. arXiv preprint arXiv:2010.09960 (2020)
Mo, T., Yu, Y., Salameh, M., et al.: Neural architecture search for keyword spotting. arXiv preprint arXiv:2009.00165 (2020)
Chen, Z., Xiao, X., Yoshioka, T., et al.: Multi-channel overlapped speech recognition with location guided speech extraction network. In: 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 558–565. IEEE (2018)
Boll, S.: Suppression of acoustic noise in speech using spectral subtraction. IEEE Trans. Acoust. Speech Signal Process. 27(2), 113–120 (1979)
Ephraim, Y., Malah, D.: Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator. IEEE Trans. Acoust. Speech Signal Process. 32(6), 1109–1121 (1984)
Bregman, A.S.: Auditory Scene Analysis: The Perceptual Organization of Sound. MIT Press, Cambridge (1994)
Hu, G., Wang, D.L.: A tandem algorithm for pitch estimation and voiced speech segregation. IEEE Trans. Audio Speech Lang. Process. 18(8), 2067–2079 (2010)
Wang, D.L., Chen, J.: Supervised speech separation based on deep learning: an overview. IEEE/ACM Trans. Audio Speech Lang. Process. 26(10), 1702–1726 (2018)
Huang, P.S., Kim, M., Hasegawa-Johnson, M., et al.: Deep learning for monaural speech separation. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1562–1566. IEEE (2014)
Luo, Y., Mesgarani, N.: TasNet: time-domain audio separation network for real-time, single-channel speech separation. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 696–700. IEEE (2018)
Yu, D., Kolbæk, M., Tan, Z.H., et al.: Permutation invariant training of deep models for speaker-independent multi-talker speech separation. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 241–245. IEEE (2017)
Boakye, K., Trueba-Hornero, B., Vinyals, O., et al.: Overlapped speech detection for improved speaker diarization in multiparty meetings. In: 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4353–4356. IEEE (2008)
Yoshioka, T., Erdogan, H., Chen, Z., et al.: Recognizing overlapped speech in meetings: a multichannel separation approach using neural networks. arXiv preprint arXiv:1810.03655 (2018)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. arXiv preprint arXiv:1710.09829 (2017)
LaLonde, R., Bagci, U.: Capsules for object segmentation. arXiv preprint arXiv:1804.04241 (2018)
Afshar, P., Mohammadi, A., Plataniotis, K.N.: Brain tumor type classification via capsule networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3129–3133. IEEE (2018)
Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural. Inf. Process. Syst. 2015(28), 91–99 (2015)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Warden, P.: Speech commands: a dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209 (2018)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, X., Yang, Q., Liu, S. (2022). Depthwise-Separable Residual Capsule for Robust Keyword Spotting. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. https://doi.org/10.1007/978-3-030-98355-0_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-98355-0_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98354-3
Online ISBN: 978-3-030-98355-0
eBook Packages: Computer ScienceComputer Science (R0)