Skip to main content

Advertisement

Springer Nature Link
Log in

New Results for Prediction of Chaotic Systems Using Deep Recurrent Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Prediction of nonlinear and dynamic systems is a challenging task, however with the aid of machine learning techniques, particularly neural networks, is now possible to accomplish this objective. Most common neural networks used are the multilayer perceptron (MLP) and recurrent neural networks (RNN) using long-short term memory units (LSTM-RNN). In recent years, deep learning neural network models have become more relevant due the improved results they show for various tasks. In this paper the authors compare these neural network models with deep learning neural network models such as long-short term memory deep recurrent neural network (LSTM-DRNN) and gate recurrent unit deep recurrent neural network (GRU-DRNN) when presented with the task of predicting three different chaotic systems such as the Lorenz system, Rabinovich–Fabrikant and the Rossler System. The results obtained show that the deep learning neural network model GRU-DRNN has better results when predicting these three chaotic systems in terms of loss and accuracy than the two other models using less neurons and layers. These results can be very helpful to solve much more complex problems such as the control and synchronization of these chaotic systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Code Availibility

The code used for these work is available at https://github.com/Dajounin/DRNN-Chaos/blob/master/DRNN_Chaos.ipynb.

References

  1. Ahandani MA, Ghiasi AR, Kharrati H (2018) Parameter identification of chaotic systems using a shuffled backtracking search optimization algorithm. Soft Comput 22(24):8317–8339. https://doi.org/10.1007/s00500-017-2779-0

    Article  Google Scholar 

  2. Alstrom RB, Moreau S, Marzocca P, Bollt E (2018) Nonlinear characterization of a Rossler system under periodic closed-loop control via time-frequency and bispectral analysis. Mech Syst Signal Process 99:567–585. https://doi.org/10.1016/j.ymssp.2017.06.001

    Article  Google Scholar 

  3. Azar A (2015) Chaos modeling and control systems design. Springer, Cham

    Book  Google Scholar 

  4. Bildirici M, Sonüstün B (2019) Chaos and exchange rates. In: Economic issues: global and local perspectives. Glasstree Academic Publishing, pp 70–76. https://www.cambridgeint.uk/BFT

  5. Bucci MA, Semeraro O, Allauzen A, Wisniewski G, Cordier L, Mathelin L (2019) Control of chaotic systems by deep reinforcement learning. Proc R Soc A Math Phys Eng Sci 475(2231):20190351. https://doi.org/10.1098/rspa.2019.0351

    Article  MathSciNet  Google Scholar 

  6. Buscarino A, Frasca M, Branciforte M, Fortuna L, Sprott JC (2017) Synchronization of two Rössler systems with switching coupling. Nonlinear Dyn 88(1):673–683. https://doi.org/10.1007/s11071-016-3269-0

    Article  Google Scholar 

  7. Chai X, Gan Z, Yuan K, Chen Y, Liu X (2019) A novel image encryption scheme based on DNA sequence operations and chaotic systems. Neural Comput Appl 31(1):219–237. https://doi.org/10.1007/s00521-017-2993-9

    Article  Google Scholar 

  8. Chattopadhyay A, Hassanzadeh P, Subramanian D (2019) Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: reservoir computing, ANN, and RNN-LSTM, pp 1–21. arXiv:1906.08829

  9. Chen Y, Tan H, Berardi U (2018) A data-driven approach for building energy benchmarking using the Lorenz curve. Energy Build 169:319–331. https://doi.org/10.1016/j.enbuild.2018.03.066

    Article  Google Scholar 

  10. Chen Z, Yuan X, Yuan Y, Iu HHC, Fernando T (2016) Parameter identification of chaotic and hyper-chaotic systems using synchronization-based parameter observer. IEEE Trans Circuits Syst I Regul Pap 63(9):1464–1475. https://doi.org/10.1109/TCSI.2016.2573283

    Article  MathSciNet  Google Scholar 

  11. Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: EMNLP 2014—2014 Conference on empirical methods in natural language processing, proceedings of the conference, pp 1724–1734. https://doi.org/10.3115/v1/d14-1179

  12. Danca M-F, Feckan M, Kuznetsov N, Chen G (2016) Looking more closely at the Rabinovich–Fabrikant system. Int J Bifurc Chaos 26(02):1650038. https://doi.org/10.1142/S0218127416500383. arXiv:1509.09206

    Article  MathSciNet  MATH  Google Scholar 

  13. Danca MF, Bourke P, Kuznetsov N (2019) Graphical structure of attraction basins of hidden chaotic attractors: the Rabinovich–Fabrikant system. Int J Bifurc Chaos 29(1):1–13. https://doi.org/10.1142/S0218127419300015

    Article  MathSciNet  MATH  Google Scholar 

  14. Devaney R (1992) A first course in chaotic dynamical systems? Theory and experiment. Addison-Wesley, Reading

    MATH  Google Scholar 

  15. Dubois P, Gomez T, Planckaert L, Perret L (2020) Data-driven predictions of the Lorenz system. Physica D 408:132495. https://doi.org/10.1016/j.physd.2020.132495

    Article  MathSciNet  Google Scholar 

  16. Eilersen A, Jensen MH, Sneppen K (2020) Chaos in disease outbreaks among prey. Sci Rep. https://doi.org/10.1038/s41598-020-60945-z

    Article  Google Scholar 

  17. Pamina J, Raja JB (2019) Survey on deep learning algorithms. Int J Emerg Technol Innov Eng 5(1):38–43

    Google Scholar 

  18. Fan H, Jiang J, Zhang C, Wang X, Lai YC (2020) Long-term prediction of chaotic systems with machine learning. Phys Rev Res 2(1):1–6. https://doi.org/10.1103/physrevresearch.2.012080

    Article  Google Scholar 

  19. Fei J, Wang H (2020) Recurrent neural network fractional-order sliding mode control of dynamic systems. J Frankl Inst 357(8):4574–4591. https://doi.org/10.1016/j.jfranklin.2020.01.050

    Article  MathSciNet  MATH  Google Scholar 

  20. Goodfellow I (2016) Deep learning. The MIT Press, Cambridge

    MATH  Google Scholar 

  21. Hajiabotorabi Z, Kazemi A, Samavati FF, Maalek Ghaini FM (2019) Improving DWT-RNN model via B-spline wavelet multiresolution to forecast a high-frequency time series. Expert Syst Appl 138:112842. https://doi.org/10.1016/j.eswa.2019.112842

    Article  Google Scholar 

  22. Hilborn R (2000) Chaos and nonlinear dynamics? An introduction for scientists and engineers. Oxford University Press, Oxford

    Book  Google Scholar 

  23. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735. arXiv:1406.1078

    Article  Google Scholar 

  24. Hua Y, Zhao Z, Li R, Chen X, Liu Z, Zhang H (2019) Deep learning with long short-term memory for time series prediction. IEEE Commun Mag 57(6):114–119. https://doi.org/10.1109/MCOM.2019.1800155

    Article  Google Scholar 

  25. Javeed A, Shah T (2020) Design of an S-box using Rabinovich–Fabrikant system of differential equations perceiving third order nonlinearity. Multimed Tools Appl 79(9–10):6649–6660. https://doi.org/10.1007/s11042-019-08393-4

    Article  Google Scholar 

  26. Kutz M (2015) Mechanical engineers handbook. Materials and engineering mechanics. Wiley, Hoboken

    Google Scholar 

  27. Lian HH, Xiao SP, Wang Z, Zhang XH, Xiao HQ (2019) Further results on sampled-data synchronization control for chaotic neural networks with actuator saturation. Neurocomputing 346:30–37. https://doi.org/10.1016/j.neucom.2018.08.090

    Article  Google Scholar 

  28. Liu F, Cai M, Wang L, Lu Y (2019) An ensemble model based on adaptive noise reducer and over-fitting prevention LSTM for multivariate time series forecasting. IEEE Access 7:26102–26115. https://doi.org/10.1109/ACCESS.2019.2900371

    Article  Google Scholar 

  29. Lorenz E (2017) Deterministic nonperiodic flow. Universality in Chaos, CRC Press, Boca Raton, pp 367–378. https://doi.org/10.1201/9780203734636-38

    Book  Google Scholar 

  30. Mandal S, Mandal KK (2020) Optimal energy management of microgrids under environmental constraints using chaos enhanced differential evolution. Renew Energy Focus 34:129–141. https://doi.org/10.1016/j.ref.2020.05.002

    Article  Google Scholar 

  31. Mohajerin N, Waslander SL (2019) Multistep prediction of dynamic systems with recurrent neural networks. IEEE Trans Neural Netw Learn Syst 30(11):3370–3383. https://doi.org/10.1109/TNNLS.2019.2891257

    Article  Google Scholar 

  32. Mohamed ST, B HME, Hassanien AE (2020) The international conference on advanced machine learning technologies and applications (AMLTA2019). In: Advances in intelligent systems and computing, vol 921. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-14118-9. https://doi.org/10.1007/978-3-030-14118-9_74

  33. Mozaffar M, Paul A, Al-Bahrani R, Wolff S, Choudhary A, Agrawal A, Ehmann K, Cao J (2018) Data-driven prediction of the high-dimensional thermal history in directed energy deposition processes via recurrent neural networks. Manuf Lett 18:35–39. https://doi.org/10.1016/j.mfglet.2018.10.002

    Article  Google Scholar 

  34. Ouannas A, Odibat Z, Shawagfeh N (2019) A new Q–S synchronization results for discrete chaotic systems. Differ Equ Dyn Syst 27(4):413–422. https://doi.org/10.1007/s12591-016-0278-x

    Article  MathSciNet  MATH  Google Scholar 

  35. Özkaynak F (2019) Construction of robust substitution boxes based on chaotic systems. Neural Comput Appl 31(8):3317–3326. https://doi.org/10.1007/s00521-017-3287-y

    Article  Google Scholar 

  36. Pappu CS, Carroll TL, Flores BC (2020) Simultaneous radar-communication systems using controlled chaos-based frequency modulated waveforms. IEEE Access 8:48361–48375. https://doi.org/10.1109/ACCESS.2020.2979324

    Article  Google Scholar 

  37. Pascanu R, Mikolov T, Bengio Y (2012) On the difficulty of training recurrent neural networks. In: 30th International conference on machine learning, ICML 2013 (PART 3), pp 2347–2355. arXiv:1211.5063

  38. Poznyak A, Sanchez E, Perez J, Yu W (1997) Robust adaptive nonlinear system identification and trajectory tracking by dynamic neural networks. In: Proceedings of the 1997 American control conference (Cat. No. 97CH36041), vol 1. IEEE, pp 242–246. https://doi.org/10.1109/ACC.1997.611794

  39. Rabinovich M, Fabrikant A (1979) Stochastic self-modulation of waves in nonequilibrium media. Sov J Exp Theor Phys 50(4):311

    Google Scholar 

  40. Raissi M, Perdikaris P, Karniadakis GE (2018) Multistep neural networks for data-driven discovery of nonlinear dynamical systems, pp 1–19. arXiv:1801.01236

  41. Rössler OE (1976) An equation for continuous chaos. Phys Lett A 57(5):397–398. https://doi.org/10.1016/0375-9601(76)90101-8

    Article  MATH  Google Scholar 

  42. Samarasinghe S (2007) Neural networks for applied sciences and engineering? From fundamentals to complex pattern recognition. Auerbach, Boca Raton

    MATH  Google Scholar 

  43. Scott A (2005) Encyclopedia of nonlinear science. Routledge, New York

    MATH  Google Scholar 

  44. Shekofteh Y, Jafari S, Rajagopal K, Pham VT (2019) Parameter identification of chaotic systems using a modified cost function including static and dynamic information of attractors in the state space. Circuits Syst Signal Process 38(5):2039–2054. https://doi.org/10.1007/s00034-018-0967-5

    Article  Google Scholar 

  45. Shih SY, Sun FK, Lee H (2019) Temporal pattern attention for multivariate time series forecasting. Mach Learn 108(8–9):1421–1441. https://doi.org/10.1007/s10994-019-05815-0

    Article  MathSciNet  MATH  Google Scholar 

  46. Shrestha A, Mahmood A (2019) Review of deep learning algorithms and architectures. IEEE Access 7:53040–53065. https://doi.org/10.1109/ACCESS.2019.2912200

    Article  Google Scholar 

  47. Skansi S (2018) Introduction to deep learning? From logical calculus to artificial intelligence. Springer, Cham

    Book  Google Scholar 

  48. Strogatz S (2015) Nonlinear dynamics and chaos: with applications to physics, biology, chemistry, and engineering. Westview Press, a member of the Perseus Books Group, Boulder

    MATH  Google Scholar 

  49. Thompson JMT (2002) Nonlinear dynamics and chaos. Wiley, New York

    MATH  Google Scholar 

  50. Wang R, Kalnay E, Balachandran B (2019) Neural machine-based forecasting of chaotic dynamics. Nonlinear Dyn 98(4):2903–2917. https://doi.org/10.1007/s11071-019-05127-x

    Article  MATH  Google Scholar 

  51. Weiss G, Goldberg Y, Yahav E (2018) On the practical computational power of finite precision rnns for language recognition

  52. Weng T, Yang H, Gu C, Zhang J, Small M (2019) Synchronization of chaotic systems and their machine-learning models. Phys Rev E 99(4):1–7. https://doi.org/10.1103/PhysRevE.99.042203

    Article  MathSciNet  Google Scholar 

  53. Zhang L (2017) Artificial neural networks model design of Lorenz chaotic system for EEG pattern recognition and prediction. In: 2017 IEEE life sciences conference (LSC), Jan. IEEE, pp 39–42. https://doi.org/10.1109/LSC.2017.8268138

  54. Zheng C, Wang S, Liu Y, Liu C (2018) A novel RNN based load modelling method with measurement data in active distribution system. Electr Power Syst Res 166:112–124. https://doi.org/10.1016/j.epsr.2018.09.006

    Article  Google Scholar 

  55. Zhuang L, Cao L, Wu Y, Zhong Y, Zhangzhong L, Zheng W, Wang L (2020) Parameter estimation of Lorenz chaotic system based on a hybrid Jaya–Powell algorithm. IEEE Access 8:20514–20522. https://doi.org/10.1109/ACCESS.2020.2968106

    Article  Google Scholar 

Download references

Acknowledgements

The authors from Universidad Iberoamericana wish to thank the Dirección de Investigación y Posgrado (DIVNP) and the first author to CONACYT for the scholarship given. This work was supported in part by CONACYT under Grant CONACyTA1-S-8216, by CINVESTAV under Grant SEP-CINVESTAV-62 and Grant CNR-CINVESTAV.

Author information

Authors and Affiliations

  1. Department of Physics and Mathematics, Universidad Iberoamerica, Prolongacion Paseo de la Reforma 880, Santa Fe, Zedec Sta Fé, Álvaro Obregón, 01219, Mexico City, Mexico

    José de Jesús Serrano-Pérez, Guillermo Fernández-Anaya & Salvador Carrillo-Moreno

  2. Departamento de Control Automático, CINVESTAV-IPN, Av IPN 2508, 07360, Mexico City, Mexico

    Wen Yu

Authors
  1. José de Jesús Serrano-Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guillermo Fernández-Anaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salvador Carrillo-Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wen Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José de Jesús Serrano-Pérez.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

A Appendix

A Appendix

Fig. 10
figure 10

Accuracy for the Lorenz ’63 System with RNN-GRU, RNN-LSTM and MLP

Full size image
Fig. 11
figure 11

Accuracy for the Rabinovich–Fabrikant equations with RNN-GRU, RNN-LSTM and MLP

Full size image
Fig. 12
figure 12

Accuracy for the Rossler system with RNN-GRU, RNN-LSTM and MLP

Full size image
Table 3 Full results of the neural network models used to predict the chaotic systems with different number of layers and neurons
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Serrano-Pérez, J.d.J., Fernández-Anaya, G., Carrillo-Moreno, S. et al. New Results for Prediction of Chaotic Systems Using Deep Recurrent Neural Networks. Neural Process Lett 53, 1579–1596 (2021). https://doi.org/10.1007/s11063-021-10466-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10466-1

Keywords

Search

Navigation