Skip to main content
SpringerLink
Log in

Depthwise-Separable Residual Capsule for Robust Keyword Spotting

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13142))

Included in the following conference series:

Abstract

Keyword spotting is widely used in device wake-up and user interaction of smart devices. However, the resources in smart devices are limited. In order to ensure that the keyword spotting system can always run efficiently and accurately in smart devices, the size of the model must be compressed to realize a small and compact model. In addition, in actual application scenarios, noise, speech rate, and overlapped speech pose great challenges to the robustness of the model. In order to solve the above problems, we propose a depthwise-separable residual capsule neural network, which uses depthwise-separable convolution to achieve a more compact design, and also uses a multi-scale capsule classifier to improve the model’s robustness in the above complex scenarios. We have achieved the best accuracy on the Google Command dataset, and have fewer calculations and fewer parameters than the previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wu, M., Panchapagesan, S., Sun, M., et al.: Monophone-based background modeling for two-stage on-device wake word detection. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5494–5498. IEEE (2018)

    Google Scholar 

  2. Ge, F., Yan, Y.: Deep neural network based wake-up-word speech recognition with two-stage detection. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2761–2765. IEEE (2017)

    Google Scholar 

  3. Chen, G., Parada, C., Heigold, G.: Small-footprint keyword spotting using deep neural networks. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4087–4091. IEEE (2014)

    Google Scholar 

  4. Sainath, T.N., Parada, C.: Convolutional neural networks for small-footprint keyword spotting. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)

    Google Scholar 

  5. Tang, R., Lin, J.: Deep residual learning for small-footprint keyword spotting. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5484–5488. IEEE (2018)

    Google Scholar 

  6. Zhang, Y., Suda, N., Lai, L., et al.: Hello edge: keyword spotting on microcontrollers. arXiv preprint arXiv:1711.07128 (2017)

  7. Arik, S.O., Kliegl, M., Child, R., et al.: Convolutional recurrent neural networks for small-footprint keyword spotting. arXiv preprint arXiv:1703.05390 (2017)

  8. Bai, Y., Yi, J., Tao, J., et al.: A time delay neural network with shared weight self-attention for small-footprint keyword spotting. In: INTERSPEECH 2019, pp. 2190–2194 (2019)

    Google Scholar 

  9. Choi, S., Seo, S., Shin, B., et al.: Temporal convolution for real-time keyword spotting on mobile devices. arXiv preprint arXiv:1904.03814 (2019)

  10. Chen, X., Yin, S., Song, D., et al.: Small-footprint keyword spotting with graph convolutional network. In: 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 539–546. IEEE (2019)

    Google Scholar 

  11. Li, X., Wei, X., Qin, X.: Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution. arXiv preprint arXiv:2010.09960 (2020)

  12. Mo, T., Yu, Y., Salameh, M., et al.: Neural architecture search for keyword spotting. arXiv preprint arXiv:2009.00165 (2020)

  13. Chen, Z., Xiao, X., Yoshioka, T., et al.: Multi-channel overlapped speech recognition with location guided speech extraction network. In: 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 558–565. IEEE (2018)

    Google Scholar 

  14. Boll, S.: Suppression of acoustic noise in speech using spectral subtraction. IEEE Trans. Acoust. Speech Signal Process. 27(2), 113–120 (1979)

    Article  Google Scholar 

  15. Ephraim, Y., Malah, D.: Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator. IEEE Trans. Acoust. Speech Signal Process. 32(6), 1109–1121 (1984)

    Article  Google Scholar 

  16. Bregman, A.S.: Auditory Scene Analysis: The Perceptual Organization of Sound. MIT Press, Cambridge (1994)

    Google Scholar 

  17. Hu, G., Wang, D.L.: A tandem algorithm for pitch estimation and voiced speech segregation. IEEE Trans. Audio Speech Lang. Process. 18(8), 2067–2079 (2010)

    Article  Google Scholar 

  18. Wang, D.L., Chen, J.: Supervised speech separation based on deep learning: an overview. IEEE/ACM Trans. Audio Speech Lang. Process. 26(10), 1702–1726 (2018)

    Article  Google Scholar 

  19. Huang, P.S., Kim, M., Hasegawa-Johnson, M., et al.: Deep learning for monaural speech separation. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1562–1566. IEEE (2014)

    Google Scholar 

  20. Luo, Y., Mesgarani, N.: TasNet: time-domain audio separation network for real-time, single-channel speech separation. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 696–700. IEEE (2018)

    Google Scholar 

  21. Yu, D., Kolbæk, M., Tan, Z.H., et al.: Permutation invariant training of deep models for speaker-independent multi-talker speech separation. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 241–245. IEEE (2017)

    Google Scholar 

  22. Boakye, K., Trueba-Hornero, B., Vinyals, O., et al.: Overlapped speech detection for improved speaker diarization in multiparty meetings. In: 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 4353–4356. IEEE (2008)

    Google Scholar 

  23. Yoshioka, T., Erdogan, H., Chen, Z., et al.: Recognizing overlapped speech in meetings: a multichannel separation approach using neural networks. arXiv preprint arXiv:1810.03655 (2018)

  24. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. arXiv preprint arXiv:1710.09829 (2017)

  25. LaLonde, R., Bagci, U.: Capsules for object segmentation. arXiv preprint arXiv:1804.04241 (2018)

  26. Afshar, P., Mohammadi, A., Plataniotis, K.N.: Brain tumor type classification via capsule networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 3129–3133. IEEE (2018)

    Google Scholar 

  27. Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural. Inf. Process. Syst. 2015(28), 91–99 (2015)

    Google Scholar 

  28. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  29. Warden, P.: Speech commands: a dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209 (2018)

  30. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, Jiangsu, China

    Xianghong Huang, Qun Yang & Shaohan Liu

Authors
  1. Xianghong Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaohan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianghong Huang .

Editor information

Editors and Affiliations

  1. IT University of Copenhagen, Copenhagen, Denmark

    Björn Þór Jónsson

  2. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  3. University of Science, VNU-HCM, Ho Chi Minh City, Vietnam

    Minh-Triet Tran

  4. University of Bergen, Bergen, Norway

    Duc-Tien Dang-Nguyen

  5. National Tsing Hua University, Hsinchu, Taiwan

    Anita Min-Chun Hu

  6. Hanoi University of Science and Technology, Hanoi, Vietnam

    Binh Huynh Thi Thanh

  7. Median Technologies, Valbonne, France

    Benoit Huet

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, X., Yang, Q., Liu, S. (2022). Depthwise-Separable Residual Capsule for Robust Keyword Spotting. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. https://doi.org/10.1007/978-3-030-98355-0_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-98355-0_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98354-3

  • Online ISBN: 978-3-030-98355-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation