Skip to main content
SpringerLink
Log in

A Novel Approach to Time Series Complexity via Reservoir Computing

  • Conference paper
  • First Online:
AI 2022: Advances in Artificial Intelligence (AI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13728))

Included in the following conference series:

  • 1498 Accesses

Abstract

When working with time series, it is often beneficial to have an idea as to how complex the signal is. Periodic, chaotic and random signals (from least to most complex) may each be approached in different ways, and knowing when a signal can be identified as belonging to one of these categories can reveal a lot about the underlying system. In the field of time series analysis, permutation entropy has emerged as one of the premier measures of time series complexity due to its ability to be calculated from data alone. We propose an alternative method for calculating complexity based on the machine learning paradigm of reservoir computing, and how the outputs of these neural networks capture similar information regarding signal complexity. We observe similar behaviour in our proposed measure to both the Lyapunov exponent and permutation entropy for well known dynamical systems. Additionally, we assess the dependence of our measure on key hyperparameters of the model, drawing conclusions about the invariance of the measure and possible implications on informing network structure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Appeltant, L., et al.: Information processing using a single dynamical node as complex system. Nature Comm. 2, 468 (2011). https://doi.org/10.1038/ncomms1476

    Article  Google Scholar 

  2. Bandt, C., Pompe, B.: Permutation entropy: a natural complexity measure for time series. Phys. Rev. Lett. 88, 174102 (2002). https://doi.org/10.1103/PhysRevLett.88.174102

  3. Bianchi, F.M., Scardapane, S., Lokse, S., Jenssen, R.: Reservoir computing approaches for representation and classification of multivariate time series. IEEE Trans. Neural Netw. Learn. Syst. 32(5), 2169–2179 (2021). https://doi.org/10.1109/TNNLS.2020.3001377

    Article  Google Scholar 

  4. Cao, Y., Tung, W.w., Gao, J.B., Protopopescu, V.A., Hively, L.M.: Detecting dynamical changes in time series using the permutation entropy. Phys. Rev. E. 70, 046217 (2004). https://doi.org/10.1103/PhysRevE.70.046217

  5. Carroll, T.L.: Using reservoir computers to distinguish chaotic signals. Phys. Rev. E. 98, 052209 (2018). https://doi.org/10.1103/PhysRevE.98.052209

  6. Cellucci, C.J., Albano, A.M., Rapp, P.E.: Comparative study of embedding methods. Phys. Rev. E 67, 066210 (2003). https://doi.org/10.1103/PhysRevE.67.066210

    Article  Google Scholar 

  7. Costa, F.G.d., Duarte, F.S., Vallim, R.M., Mello, R.F.d.: Multidimensional surrogate stability to detect data stream concept drift. Expert Syst. Appl. 87(C), 15–29 (2017). https://doi.org/10.1016/j.eswa.2017.06.005

  8. Eckmann, J.P., Ruelle, D.: Ergodic theory of chaos and strange attractors. Rev. Mod. Phys. 57, 617–656 (1985). https://doi.org/10.1103/RevModPhys.57.617

    Article  MATH  Google Scholar 

  9. Hirata, Y.: Recurrence plots for characterizing random dynamical systems. Commun. Nonlinear Sci. Numer. Simul. 94, 105552 (2021). https://doi.org/10.1016/j.cnsns.2020.105552

    Article  MATH  Google Scholar 

  10. Jaeger, H.: The echo state approach to analysing and training recurrent neural networks-with an erratum note. Ger. Natl. Res. Center for Inf. Technol. GMD Tech. Rep. 148 (2001)

    Google Scholar 

  11. Larger, L., Baylón-Fuentes, A., Martinenghi, R., Udaltsov, V.S., Chembo, Y.K., Jacquot, M.: High-speed photonic reservoir computing using a time-delay-based architecture: Million words per second classification. Phys. Rev. X 7, 011015 (2017). https://doi.org/10.1103/PhysRevX.7.011015

    Article  Google Scholar 

  12. Lee, G.C., Loo, C.K., Liew, W.S., Wermter, S.: Self-organizing kernel-based convolutional echo state network for human actions recognition. In: Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN’20), pp. 591–596 (2020). https://www2.informatik.uni-hamburg.de/wtm/publications/2020/LLLW20/ES2020-99-7.pdf

  13. Lu, Z., Hunt, B.R., Ott, E.: Attractor reconstruction by machine learning. Chaos 28(6), 061104 (2018). https://doi.org/10.1063/1.5039508

    Article  Google Scholar 

  14. Lu, Z., Pathak, J., Hunt, B., Girvan, M., Brockett, R., Ott, E.: Reservoir observers: model-free inference of unmeasured variables in chaotic systems. Chaos 27(4), 041102 (2017). https://doi.org/10.1063/1.4979665

    Article  Google Scholar 

  15. Lymburn, T., Khor, A., Stemler, T., Corréa, D.C., Small, M., Jüngling, T.: Consistency in echo-state networks. Chaos 29(2), 023118 (2019). https://doi.org/10.1063/1.5079686

    Article  Google Scholar 

  16. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002). https://doi.org/10.1162/089976602760407955

    Article  MATH  Google Scholar 

  17. McCullough, M., Small, M., Iu, H.H.C., Stemler, T.: Multiscale ordinal network analysis of human cardiac dynamics. Philos. Trans. Royal Soc. A: Math. Phys. Eng. Sci. 375(2096), 20160292 (2017). https://doi.org/10.1098/rsta.2016.0292

    Article  MATH  Google Scholar 

  18. Yamane, T., et al.: Dimensionality reduction by reservoir computing and its application to IoT edge computing. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11301, pp. 635–643. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04167-0_58

    Chapter  Google Scholar 

  19. Pathak, J., Lu, Z., Hunt, B., Girvan, M., Ott, E.: Using machine learning to replicate chaotic attractors and calculate lyapunov exponents from data. Chaos 27(12), 121102 (2017)

    Article  MATH  Google Scholar 

  20. Pathak, J., Hunt, B., Girvan, M., Lu, Z., Ott, E.: Model-free prediction of large spatiotemporally chaotic systems from data: a reservoir computing approach. Phys. Rev. Lett. 120, 024102 (2018). https://doi.org/10.1103/PhysRevLett.120.024102

    Article  Google Scholar 

  21. Pesin, Y.B.: Dimension theory in dynamical systems contemporary views and applications. Chicago lectures in mathematics series, University of Chicago Press, Chicago (1997)

    Google Scholar 

  22. Qing-Fang, M., Yu-Hua, P., Pei-Jun, X.: A new method of determining the optimal embedding dimension based on nonlinear prediction. Chin. Phys. 16(5), 1252–1257 (2007). https://doi.org/10.1088/1009-1963/16/5/014

  23. Shahriari, Z., Small, M.: Permutation entropy of state transition networks to detect synchronization. Int. J. Bifurcat. Chaos 30(10), 2050154 (2020). https://doi.org/10.1142/S0218127420501540

    Article  MATH  Google Scholar 

  24. Tanaka, G., et al.: Recent advances in physical reservoir computing: a review. Neural Netw. 115, 100–123 (2019). https://doi.org/10.1016/j.neunet.2019.03.005

    Article  Google Scholar 

  25. Thorne, B., Jüngling, T., Small, M., Corrêa, D., Zaitouny, A.: Reservoir time series analysis: using the response of complex dynamical systems as a universal indicator of change. Chaos: Interdisc. J. Nonlinear Sci. 32(3), 033109 (2022). https://doi.org/10.1063/5.0082122

  26. Vertechi, P., Brendel, W., Machens, C.K.: Unsupervised learning of an efficient short-term memory network. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems. vol. 27. Curran Associates, Inc. (2014). https://proceedings.neurips.cc/paper/2014/file/333222170ab9edca4785c39f55221fe7-Paper.pdf

  27. Verzelli, P., Alippi, C., Livi, L.: Learn to synchronize, synchronize to learn. Chaos: Interdisc. J. Nonlinear Sci. 31(8), 083119 (2021). https://doi.org/10.1063/5.0056425

  28. Verzelli, P., Alippi, C., Livi, L., Tiňo, P.: Input-to-state representation in linear reservoirs dynamics. IEEE Trans. Neural Netw. Learn. Syst. PP1–12 (2021). https://doi.org/10.1109/TNNLS.2021.3059389

  29. Xu, X., Zhang, J., Small, M.: Superfamily phenomena and motifs of networks induced from time series. Proc. Natl. Acad. Sci. 105(50), 19601–19605 (2008). https://doi.org/10.1073/pnas.0806082105

    Article  MATH  Google Scholar 

Download references

Acknowledgements

BT is partially supported by the Australian Government Research Training Program and a University Postgraduate Award at The University of Western Australia. BT, MS, and DC are partially supported by the Australian Research Council through the Centre for Transforming Maintenance Through Data Science (grant number IC180100030) funded by the Australian Government.

Author information

Authors and Affiliations

  1. University of Western Australia, Crawley, WA, Australia

    Braden Thorne, Thomas Jüngling, Michael Small, Débora Corrêa & Ayham Zaitouny

  2. ARC Centre for Transforming Maintenance Through Data Science, Crawley, WA, Australia

    Braden Thorne, Michael Small & Débora Corrêa

  3. University of Doha for Science and Technology, Duhail North Doha, Qatar

    Ayham Zaitouny

Authors
  1. Braden Thorne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Jüngling
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Small
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Débora Corrêa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ayham Zaitouny
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Braden Thorne .

Editor information

Editors and Affiliations

  1. University of New South Wales, Sydney, NSW, Australia

    Haris Aziz

  2. University of Western Australia, Perth, WA, Australia

    Débora Corrêa

  3. University of Western Australia, Perth, WA, Australia

    Tim French

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Thorne, B., Jüngling, T., Small, M., Corrêa, D., Zaitouny, A. (2022). A Novel Approach to Time Series Complexity via Reservoir Computing. In: Aziz, H., Corrêa, D., French, T. (eds) AI 2022: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13728. Springer, Cham. https://doi.org/10.1007/978-3-031-22695-3_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-22695-3_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-22694-6

  • Online ISBN: 978-3-031-22695-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation