Skip to main content
SpringerLink
Log in

Efficient Neuromorphic Signal Processing with Resonator Neurons

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

The biologically inspired spiking neurons used in neuromorphic computing are nonlinear filters with dynamic state variables, which is distinct from the stateless neuron models used in deep learning. The new version of Intel’s neuromorphic research processor, Loihi 2, supports an extended range of stateful spiking neuron models with programmable dynamics. Here, we showcase advanced neuron models that can be used to efficiently process streaming data in simulation experiments on emulated Loihi 2 hardware. In one example, Resonate-and-Fire (RF) neurons are used to compute the Short Time Fourier Transform (STFT) with similar computational complexity but 47x less output bandwidth than the conventional STFT. In another example, we describe an algorithm for optical flow estimation using spatiotemporal RF neurons that requires over 90x fewer operations than a conventional DNN-based solution. We also demonstrate backpropagation methods to train non-linear spiking RF neurons for audio classification tasks, suitable for efficient execution on Loihi 2. We conclude with another application of nonlinear filtering showing a cascade of Hopf resonators exhibiting computational properties seen in the cochlea, such as self-normalization. Taken together, this work presents new techniques for an efficient spike-based spectrogram encoder that can be used for signal processing applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

Notes

  1. Taking Neuromorphic Computing to the Next Level with Loihi 2

  2. https://lava-nc.org. The lava-dl deep learning library is available at https://github.com/lava-nc/lava-dl

References

  1. Davies, M., Srinivasa, N., Lin, T., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., Tse, J., Venkataramanan, G., Weng, Y., Wild, A., Yang, Y., & Wang, H. (2018). Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1), 82–99.

    Article  Google Scholar 

  2. Davies, M., Wild, A., Orchard, G., Sandamirskaya, Y., Guerra, G. A. F., Joshi, P., Plank, P., & Risbud, S. R. (2021). Advancing neuromorphic computing with Loihi: A survey of results and outlook. Proceedings of the IEEE, 109(5), 911–934.

    Article  Google Scholar 

  3. Arthur, J. V., & Boahen, K. A. (2010). Silicon-neuron design: A dynamical systems approach. IEEE Transactions on Circuits and Systems I, 58(5), 1034–1043.

    Article  MathSciNet  Google Scholar 

  4. Frady, E. P., & Sommer, F. T. (2019). Robust computation with rhythmic spike patterns. PNAS, 116(36), 18050–18059.

    Article  Google Scholar 

  5. Polikar, R. (1996). Fundamental concepts & an overview of the wavelet theory.

  6. Gallego, G., Delbruck, T., Orchard, G. M., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A., Conradt, J., Daniilidis, K., & Scaramuzza, D. (2020). Event-based vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.

  7. Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of Optical Society of America, 2(2), 284–299.

    Article  Google Scholar 

  8. Zihao Zhu, A., Thakur, D., Özaslan, T., Pfrommer, B., Kumar, V., & Daniilidis, K. (2018). The multivehicle stereo event camera dataset: An event camera dataset for 3D perception. IEEE Robotics and Automation Letters, 3(3), 2032–2039.

    Article  Google Scholar 

  9. Shrestha, S. B., & Orchard, G. (2018). SLAYER: Spike layer error reassignment in time. In Advances in Neural Information Processing Systems (NeurIPS) (pp. 1412–1421).

  10. Neftci, E. O., Mostafa, H., & Zenke, F. (2019). Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine, 36, 61–63.

    Article  Google Scholar 

  11. Izhikevich, E. M. (2001). Resonate-and-fire neurons. Neural Networks, 14(6), 883–894.

    Article  Google Scholar 

  12. Anumula, J., Neil, D., Delbruck, T., & Liu, S.-C. (2018). Feature representations for neuromorphic audio spike streams. Frontiers in Neuroscience, 12.

  13. Warden, P. (2018). Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209

  14. Zilany, M. S., Bruce, I. C., & Carney, L. H. (2014). Updated parameters and expanded simulation options for a model of the auditory periphery. The Journal of the Acoustical Society of America, 135(1), 283–286.

    Article  Google Scholar 

  15. Yin, B., Corradi, F., & Bohté, S. M. (2021). Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. preprint arXiv:2103.12593

  16. Rybakov, O., Kononenko, N., Subrahmanya, N., Visontai, M., & Laurenzo, S. (2020). Streaming keyword spotting on mobile devices. In Interspeech.

  17. Zhang, Y., Suda, N., Lai, L., & Chandra, V. (2017). Hello edge: Keyword spotting on microcontrollers. arXiv preprint arXiv:1711.07128

  18. Goldstein, J. (1967). Auditory nonlinearity. Journal of the Acoustical Society of America, 41, 676–689.

    Article  Google Scholar 

  19. Tapson, J., Hamilton, T. J., Jin, C., & van Schaik (2008). A. Self-tuned regenerative amplification and the HOPF bifurcation. In IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1768–1771).

  20. Eguíluz, V. M., Ospeck, M., Choe, Y., Hudspeth, A., & Magnasco, M. O. (2000). Essential nonlinearities in hearing. Physical Review Letters, 84(22), 5232–5235.

    Article  Google Scholar 

  21. Kern, A., & Stoop, R. (2003). Essential role of couplings between hearing nonlinearities. Physical Review Letters, 91(12), 128101.

  22. Liu, K. J. R. (1993). Novel Parallel architectures for short-time fourier transform. IEEE TCS II, 40(12), 786–790.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intel Labs, Intel Corporation, Santa Clara, CA, 95054, USA

    E. Paxon Frady, Sophia Sanborn, Sumit Bam Shrestha, Daniel Ben Dayan Rubin, Garrick Orchard, Friedrich T. Sommer & Mike Davies

Authors
  1. E. Paxon Frady
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophia Sanborn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sumit Bam Shrestha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Ben Dayan Rubin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Garrick Orchard
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Friedrich T. Sommer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mike Davies
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. Paxon Frady.

Ethics declarations

Competing Interest

The authors are employees of Intel Labs. Sophia Sanborn contributed to this research while at Intel, and is currently at UC Berkeley. On behalf of Intel, Daniel Ben Dayan Rubin filed a patent application 17/643,652, AD7779-US named “Peak self-normalization gain control based on Hopf resonators cascade signal spectral decomposition”. The authors declare no additional competing interests. E. Paxon Frady, Sophia Sanborn, Sumit Bam Shrestha, Daniel Ben Dayan Rubin, performed experiments and analysis. E. Paxon Frady, Sumit Bam Shreshtha, Daniel Ben Dayan Rubin, Garrick Orchard, Friedrich T. Sommer, and Mike Davies wrote the paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frady, E.P., Sanborn, S., Shrestha, S.B. et al. Efficient Neuromorphic Signal Processing with Resonator Neurons. J Sign Process Syst 94, 917–927 (2022). https://doi.org/10.1007/s11265-022-01772-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-022-01772-5

Keywords

Search

Navigation