Skip to main content
SpringerLink
Log in

Real-time Audio Processing with a Cascade of Discrete-Time Delay Line-Based Reservoir Computers

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Background: Real-time processing of audio or audio-like signals is a promising research topic for the field of machine learning, with many potential applications in music and communications. We present a cascaded delay line reservoir computer capable of real-time audio processing on standard computing equipment, aimed at black-box system identification of nonlinear audio systems. The cascaded reservoir blocks use two-pole filtered virtual neurons to match their timescales to that of the target signals. The reservoir blocks receive both the global input signal and the target estimate from the previous block (local input). The units in the cascade are trained in a successive manner on a single input output training pair, such that a successively better approximation of the target is reached. A cascade of 5 dual-input reservoir blocks of 100 neurons each is trained to mimic the distortion of a measured guitar amplifier. This cascade outperforms both a single delay reservoir having the same total number of neurons as well as a cascade with only single-input blocks. We show that the presented structure is a viable platform for real-time audio applications on present-day computing hardware. A benefit of this structure is that it works directly from the audio samples as input, avoiding computationally intensive preprocessing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. An Intel dual core running at 2.4 GHz, Dell Latitude E4300, built in 2009.

  2. Also after 20 iterations of the RO algorithm, on P = 50 parameter tuples.

  3. This is equivalent to setting the filter coefficients to b 0(n) = 1 and b 1(n) = b 2(n) = 0.

References

  1. Jäger H. The echo state approach to analysing and training recurrent neural networks. Technical report, German National Research Center for Information Technology. 2001.

  2. Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 2002;14(11):2531–2560.

    Article  PubMed  Google Scholar 

  3. Verstraeten D, Schrauwen B, d’Haene M, Stroobandt D. An experimental unification of reservoir computing methods. Neural Netw 2007;20:391–403.

    Article  CAS  PubMed  Google Scholar 

  4. Appeltant L, Soriano M, Van der Sande G, Danckaert J, Massar S, Dambre J, Schrauwen B, Mirasso CR, Fischer I. Information processing using a single dynamical node as complex system. Nat Commun 2011;2:468.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Brünner D., Soriano M, Mirasso CR, Fischer I. Parallel photonic information processing at gigabyte per second data rates using transient states. Nat Commun 2013;4:1364.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Duport F, Schneider B, Smerieri A, Haelterman M, Massar S. All-optical reservoir computing. Opt Exp 2012;20:22,783–22,795.

    Article  Google Scholar 

  7. Soriano M, Ortín S, Keuninckx L, Appeltant L, Danckaert J, Pesquera L, der Sande GV. Delay based reservoir computing: Noise effects in a combined analog and digital implementation. IEEE TNNLS 2015;26(2):388–393.

    Google Scholar 

  8. Larger L, Soriano M, Brünner D, Appeltant L, Gutierrez JM, Pesquera L, Mirasso CR, Fischer I. Photonic information processing beyond Turing: an opto-electronic implementation of reservoir computing. Opt Exp 2012;20:3241–3249.

    Article  CAS  Google Scholar 

  9. Paquot Y, Duport F, Smerieri A, Dambre J, Schrauwen B, Haelterman M, Massar S. Opto-electronic reservoir computing. Sci Rep 2012;2:287.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Holzmann G. Reservoir computing: a powerful black-box framework for nonlinear audio processing. In: Proceedings of the 12th International Conference on Digital Audio Effects (DAFx-09), pp 90–97. 2009.

  11. Scardapane S, Uncini A. Semi-supervised echo state networks for audio classification. Cognitive Computation, pp 1–11. 2016.

  12. Liu X, Bao CC. Audio bandwidth extension using ensemble of recurrent neural networks. EURASIP J Audio Speech, Music Process 2016;2016(1):12.

    Article  Google Scholar 

  13. Triefenbach F, Jalalvand A, Demuynck K, Martens JP. Acoustic modeling with hierarchical reservoirs. IEEE Transactions on Audio. Speech Lang Process 2013;21(11):2439–2450.

    Article  Google Scholar 

  14. Grigoryeva L, Henriques J, Larger L, Ortega JP. Stochastic time series forecasting using time-delay reservoirs. Neural Netw 2014;55:59–71.

    Article  PubMed  Google Scholar 

  15. Furse R, et al. LADSPA: Linux audio developers simple plugin API (2000). prefix http://www.ladspa.org/http://www.ladspa.org/. [Online; accessed 13-January-2016].

  16. Schumacher J, Haslinger R, Pipa G. Statistical modelling approach for detecting generalized synchronization. Phys Rev E. 85 5 Pt 2 (2012). doi:10.1103/PhysRevE.85.056215.

  17. Keen RG. A musical distortion primer. http://www.geofex.com/effxfaq/distn101.htm. 2000. [Online; accessed 28-August-2016].

  18. Tretter SA. Communication system design using DSP algorithms, 1st ed.: Springer; 2008.

  19. Pakarinen J, Yeh DT. A review of digital techniques for modelling vacuum-tube guitar amplifiers. Comput Music J 2009;33(2):85–100.

    Article  Google Scholar 

  20. Yeh DT. Digital implementation of musical distorcion circuits by analysis and simulation. Ph.D. thesis, Helsinki University of Technology. 2009.

  21. Acustica Nebula Series Effects Processors. http://www.acustica-audio.com. [Online; accessed 1-December-2015].

  22. Kemper profiling amplifiers. http://www.kemper-amp.com (2015). [Online; accessed 5-October-2015].

  23. Kemp MJ. Audio effects synthesizer with or without analyzer, u.s. patent no. 7,039,194 b1. http://www.google.com/patents/US7039194. [Filed Aug. 8, 1997 issued May 2, 2006].

  24. Appeltant L. Reservoir computing based on delay dynamical systems. Ph.D. thesis, Vrije Universiteit Brussel (VUB) Universitat de les Illes Balears. 2012.

  25. Mackey MC, Glass L. Oscillation and chaos in physiological control systems. Science 1977;197(4300):287–289.

    Article  CAS  PubMed  Google Scholar 

  26. Bencina R, et al. Portaudio—an open-source cross-platform audio API. http://www.portaudio.com. 2011.

  27. Jones E, Oliphant T, Peterson P, et al. SciPy: Open source scientific tools for Python. http://www.scipy.org. [Online; accessed 2015-11-30]. 2001.

  28. Knapp C, Carter G. Generalized correlation methods for estimation of time delay. IEEE TASSP 1976;24(4):320–327.

    Article  Google Scholar 

  29. Holzmann G. Echo state networks with filter neurons and a delay and sum readout with applications in audio signal processing. Master’s thesis, Institute for Theoretical Computer Science, TU Graz. http://grh.mur.at/misc/MasterThesis.pdf. 2008.

  30. Hauser H. Echo state networks with filter neurons and a delay and sum readout. Neural Netw 2009;23(2):244–256.

    PubMed  Google Scholar 

  31. III JOS. Introduction to digital filter theory with audio applications: W3K Publishing; 2007.

  32. Yates R, Lyons R. DSP tips and tricks: DC blocker algorithms. IEEE Sig Proc Mag 2008;25(2):132–134.

    Article  Google Scholar 

  33. Matyas J. Random optimization. Autom Remote Control 1965;26(2):246–253.

    Google Scholar 

  34. Ortega JP. Time-delay reservoir computers: nonlinear stability of functional differential systems and optimal nonlinear information processing capacity. http://trimestres-lmb.univ-fcomte.fr/Speakers-and-presentations.html. Besançon, France: Workshop on dynamical systems and brain-inspired information processing; 2015.

  35. Smith SW. The Scientist and Engineer’s Guide to Digital Signal Processing. 1st ed. San Diego, CA: California Technical Publishing; 1997.

    Google Scholar 

  36. Lennes M, Lehtokoski A, Alku P, Näätänen R. 1999. Acoustic, psychoacoustic and psychophysiological measures of distance in the Finnish vowel space. In: Proceedings ICPhS99, pp. 2465–2468.

  37. Logan B, Salomon A. A music similarity function based on signal analysis. In: Proceedings International Symposium. Tokyo, Japan: Music Information Retrieval; 2001.

Download references

Author information

Authors and Affiliations

  1. Applied Physics Research Group (APHY), Vrije Universiteit Brussel, Brussels, 1050, Belgium

    Lars Keuninckx, Jan Danckaert & Guy Van der Sande

Authors
  1. Lars Keuninckx
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Danckaert
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guy Van der Sande
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lars Keuninckx.

Ethics declarations

Funding

LK and GVDS were partly funded by the Interuniversitary Attraction Poles Program “Photonics@be” of the Belgian Science Policy Office and by the Science Foundation - Flanders (FWO).

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keuninckx, L., Danckaert, J. & Van der Sande, G. Real-time Audio Processing with a Cascade of Discrete-Time Delay Line-Based Reservoir Computers. Cogn Comput 9, 315–326 (2017). https://doi.org/10.1007/s12559-017-9457-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-017-9457-5

Keywords

Search

Navigation