Skip to main content
SpringerLink
Log in

Brain-Inspired Binaural Sound Source Localization Method Based on Liquid State Machine

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14449))

Included in the following conference series:

  • 520 Accesses

Abstract

Binaural Sound Source Localization (BSSL) is a remarkable topic in robot design and human hearing aid. A great number of algorithms flourished due to a leap in machine learning. However, prior approaches lack the ability to make a trade-off between parameter size and accuracy, which is a primary obstacle to their further implementation on resource-constrained devices. Spiking Neural Network (SNN)-based models have also emerged due to their inherent computing superiority over sparse event processing. Liquid State Machine (LSM) is a classic Spiking Recurrent Neural Network (SRNN) which has the natural potential of processing spatiotemporal information. LSM has been proved advantageous on numerous tasks once proposed. Yet, to our best knowledge, it is the first proposed BSSL model based on LSM, and we name it BSSL-LSM. BSSL-LSM is lightweight with only 1.04M parameters, which is a considerable reduction compared to CNN (10.1M) and D-BPNN (2.23M) while maintaining comparable or even superior accuracy. Compared to SNN-IID, there is a 10% accuracy improvement for \(10^\circ \) interval localization. To achieve better performance, we introduce Bayesian Optimization (BO) for hyperparameters searching and a novel soft label technique for better differentiating adjacent angles, which can be easily mirrored on related works. Project page: https://github.com/BSSL-LSM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Algazi, V.R., Duda, R.O., Thompson, D., Avendaño, C.: The CIPIC HRTF database. In: Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No. 01TH8575), pp. 99–102 (2001)

    Google Scholar 

  2. Bu, H., Du, J., Na, X., Wu, B., Zheng, H.: AISHELL-1: an open-source mandarin speech corpus and a speech recognition baseline. In: Oriental COCOSDA 2017 (2017, submitted)

    Google Scholar 

  3. Das, D., Bhattacharya, S., Pal, U., Chanda, S.: PLSM: a parallelized liquid state machine for unintentional action detection. ArXiv abs/2105.09909 (2021)

    Google Scholar 

  4. Dávila-Chacón, J., Liu, J., Wermter, S.: Enhanced robot speech recognition using biomimetic binaural sound source localization. IEEE Trans. Neural Netw. Learn. Syst. 30(1), 138–150 (2018)

    Article  Google Scholar 

  5. Desai, D., Mehendale, N.: A review on sound source localization systems. Arch. Comput. Methods Eng. 29(7), 4631–4642 (2022)

    Article  Google Scholar 

  6. Faraji, M.M., Shouraki, S.B., Iranmehr, E.: Spiking neural network for sound localization using microphone array. In: 2015 23rd Iranian Conference on Electrical Engineering, pp. 1260–1265 (2015)

    Google Scholar 

  7. Gerstner, W., Kempter, R., van Hemmen, J.L., Wagner, H.: A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76–78 (1996)

    Article  Google Scholar 

  8. Gerstner, W., Ritz, R., van Hemmen, J.L.: Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern. 69, 503–515 (1993)

    Article  MATH  Google Scholar 

  9. Ghani, A., McGinnity, T.M., Maguire, L.P., McDaid, L.J., Belatreche, A.: Neuro-inspired speech recognition based on reservoir computing (2010)

    Google Scholar 

  10. Glackin, B.P., Wall, J.A., Mcginnity, T.M., Maguire, L.P., McDaid, L.J.: A spiking neural network model of the medial superior olive using spike timing dependent plasticity for sound localization. Frontiers Comput. Neurosci. 4 (2010)

    Google Scholar 

  11. Goodman, D.F.M., Pressnitzer, D., Brette, R.: Sound localization with spiking neural networks. BMC Neurosci. 10, 1 (2009)

    Article  Google Scholar 

  12. Guo, S., et al.: A systolic SNN inference accelerator and its co-optimized software framework. In: Proceedings of the 2019 on Great Lakes Symposium on VLSI (2019)

    Google Scholar 

  13. Jeffress, L.A.: A place theory of sound localization. J. Comp. Physiol. Psychol. 41(1), 35–9 (1948)

    Article  Google Scholar 

  14. Kuang, S., van der Heijden, K., Mehrkanoon, S.: BAST: binaural audio spectrogram transformer for binaural sound localization. ArXiv abs/2207.03927 (2022)

    Google Scholar 

  15. Li, S., Wang, L., Wang, S., Xu, W.: Liquid state machine applications mapping for NoC-based neuromorphic platforms. In: Dong, D., Gong, X., Li, C., Li, D., Wu, J. (eds.) ACA 2020. CCIS, vol. 1256, pp. 277–289. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-8135-9_20

    Chapter  Google Scholar 

  16. Li, Y., Zhang, Y., Zhou, G., Gong, Y.: Bayesian optimization with particle swarm. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–6 (2021)

    Google Scholar 

  17. Liaquat, M.U., Munawar, H.S., Rahman, A., Qadir, Z., Kouzani, A.Z., Mahmud, M.A.P.: Sound localization for ad-hoc microphone arrays. Energies (2021)

    Google Scholar 

  18. Luke, R., McAlpine, D.: A spiking neural network approach to auditory source lateralisation. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1488–1492. IEEE (2019)

    Google Scholar 

  19. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002)

    Article  MATH  Google Scholar 

  20. Pang, C., Liu, H., Li, X.: Multitask learning of time-frequency CNN for sound source localization. IEEE Access 7, 40725–40737 (2019)

    Article  Google Scholar 

  21. Reynolds, J.J.M., Plank, J.S., Schuman, C.D.: Intelligent reservoir generation for liquid state machines using evolutionary optimization. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019)

    Google Scholar 

  22. Rudnicki, M., Schoppe, O., Isik, M., Völk, F., Hemmert, W.: Modeling auditory coding: from sound to spikes. Cell Tissue Res. 361, 159–175 (2015)

    Article  Google Scholar 

  23. Song, H., Liu, X., Yu, S.: Binaural localization algorithm based on deep learning. Technical Acoust. 41 (2022)

    Google Scholar 

  24. Tang, C., Ji, J., Lin, Q., Zhou, Y.: Evolutionary neural architecture design of liquid state machine for image classification. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 91–95 (2022)

    Google Scholar 

  25. Tian, S., Qu, L., Wang, L., Hu, K., Li, N., Xu, W.: A neural architecture search based framework for liquid state machine design. Neurocomputing 443, 174–182 (2021)

    Article  Google Scholar 

  26. Vecchiotti, P., Ma, N., Squartini, S., Brown, G.J.: End-to-end binaural sound localisation from the raw waveform. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 451–455. IEEE (2019)

    Google Scholar 

  27. Wall, J.A., McDaid, L.J., Maguire, L.P., McGinnity, T.M.: Spiking neural network model of sound localization using the interaural intensity difference. IEEE Trans. Neural Netw. Learn. Syst. 23(4), 574–586 (2012)

    Article  Google Scholar 

  28. Wang, S., et al.: A power efficient hardware implementation of the if neuron model. In: Conference on Advanced Computer Architecture (2018)

    Google Scholar 

  29. Wu, J., Chua, Y., Zhang, M., Li, H., Tan, K.C.: A spiking neural network framework for robust sound classification. Frontiers Neurosci. 12 (2018)

    Google Scholar 

  30. Xiao, X., et al.: Dynamic vision sensor based gesture recognition using liquid state machine. In: International Conference on Artificial Neural Networks (2022)

    Google Scholar 

  31. Xu, Y., Afshar, S., Singh, R.K., Wang, R., van Schaik, A., Hamilton, T.J.: A binaural sound localization system using deep convolutional neural networks. In: 2019 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE (2019)

    Google Scholar 

  32. Yang, Q., Zheng, Y.: DeepEar: sound localization with binaural microphones. IEEE Trans. Mob. Comput. (2022)

    Google Scholar 

  33. Youssef, K., Argentieri, S., Zarader, J.L.: A binaural sound source localization method using auditive cues and vision. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 217–220 (2012)

    Google Scholar 

  34. Yu, X., Wang, L., Chen, C., Tie, J., Guo, S.: Multimodal learning of audio-visual speech recognition with liquid state machine. In: International Conference on Neural Information Processing (2022)

    Google Scholar 

  35. Zheng, H., Wu, Y., Deng, L., Hu, Y., Li, G.: Going deeper with directly-trained larger spiking neural networks. In: AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  36. Zhu, J., et al.: An event based gesture recognition system using a liquid state machine accelerator. In: Proceedings of the Great Lakes Symposium on VLSI 2022 (2022)

    Google Scholar 

  37. Zilany, M.S.A., Bruce, I.C., Carney, L.H.: Updated parameters and expanded simulation options for a model of the auditory periphery. J. Acoust. Soc. Am. 135(1), 283–6 (2014)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported in part by the National Natural Science Foundation of China under Grants 62372461, 62032001 and 62203457, and in part by the Key Laboratory of Advanced Microprocessor Chips and Systems.

Author information

Authors and Affiliations

  1. College of Computer, National University of Defence Technology, Changsha, 410071, Hunan, China

    Yuan Li & Xun Xiao

  2. Defense Innovation Institute, Academy of Military Sciences, Beijing, China

    Jingyue Zhao, Renzhi Chen & Lei Wang

Authors
  1. Yuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingyue Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xun Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renzhi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Wang .

Editor information

Editors and Affiliations

  1. Central South University, Changsha, China

    Biao Luo

  2. Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Y., Zhao, J., Xiao, X., Chen, R., Wang, L. (2024). Brain-Inspired Binaural Sound Source Localization Method Based on Liquid State Machine. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14449. Springer, Singapore. https://doi.org/10.1007/978-981-99-8067-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8067-3_15

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8066-6

  • Online ISBN: 978-981-99-8067-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation