Skip to main content
Log in

Recurrent neural network system using probability graph model optimization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

We propose a novel recurrent neural network model based on a combination of the echo state network (ESN) and the dynamic Bayesian network (DBN). Our contribution includes the following: (1) A new graph-based echo state network (GESN) model is presented for nonlinear system modeling. The GESN consists of four layers: an input layer, reservoir layer, graph model optimization layer and output layer. Unlike the original ESN model, the GESN model uses probability graph inference to optimize reservoir states, and the optimized GESN is more robust. (2) A novel DBN graph is proposed for describing random probability signals in the GESN. Simultaneously, an inference algorithm of the DBN is deduced according to Bayesian rules and probability graph theory. Finally, GESN performance is tested using the Mackey–Glass time series and laser time series data forecasting The simulations and comparison results show that the proposed model is promising.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, London

    MATH  Google Scholar 

  2. Hopfield JJ, Tank DW (1985) Neural computation of decisions in optimization problems. Biol Cybern 52:141–152

    MATH  Google Scholar 

  3. Jaeger H, Haas H (2004) Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304:78–80

    Article  Google Scholar 

  4. Gross JL, Yellen J (2011) Graph theory and its applications, 2nd edn. Chapman and Hall/CRC, Boca Raton

    MATH  Google Scholar 

  5. Li D, Han M, Wang J (2012) Chaotic time series prediction based on a novel robust echo state network. IEEE Trans Neural Netw Learn Syst 23(5):787–799

    Article  Google Scholar 

  6. Luko M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3(3):127–149

    Article  MATH  Google Scholar 

  7. Yildiz IB, Jaeger H, Kiebel SJ (2012) Re-visiting the echo state property. Neural Netw 35(21):1–9

    Article  MATH  Google Scholar 

  8. Ozturk MC, Xu D, Prncipe JC (2007) Analysis and design of echo state networks. Neural Comput 19 (1):111–138

    Article  MATH  Google Scholar 

  9. Deng Z, Zhang Y (2007) Collective behavior of a small-world recurrent neural system with scale-free distribution. IEEE Trans Neural Netw 18(5):1364–1375

    Article  Google Scholar 

  10. Luko M (2012) Reservoir computing and self-organized neural hierarchies, Ph.D. dissertation, School Eng. Sci., Jacobs Univ., Bremen, Bremen Germany

  11. Luko M (2010) On self-organizing reservoirs and their hierarchies, School Eng. Sci., Jacobs Univ. Bremen, Germany, Tech. Rep. 25

  12. Rodan A, Tino P (2011) Minimum complexity echo state network. IEEE Trans Neural Netw 22(1):131–144

    Article  Google Scholar 

  13. Strauss T, Wustlich W, Labahn R (2012) Design strategies for weight matrices of echo state networks. Neural Comput 24(12):3246–3276

    Article  MathSciNet  MATH  Google Scholar 

  14. Xue Y, Yang L, Haykin S (2007) Decoupled echo state networks with lateral inhibition. Neural Netw 20(3):365–376

    Article  MATH  Google Scholar 

  15. Bas E, Egrioglu E, Hakan Aladag C, Yolcu U (2015) Fuzzy-time-series network used to forecast linear and nonlinear time series. Appl Intell 43:343–355

    Article  Google Scholar 

  16. Bas E, Rezan Uslu V, Yolcu U, Egrioglu E (2014) A modified genetic algorithm for forecasting fuzzy time series. Appl Intell 41:453–463

    Article  Google Scholar 

  17. Najibi E, Rostami H (2015) SCESN, SPESN, SWESN: Three recurrent neural echo state networks with clustered reservoirs for prediction of nonlinear and chaotic time series. Appl Intell 43:460–472

    Article  Google Scholar 

  18. Lorenz EN (1963) Deterministic nonperiodic flow. J Atmos Sci 20:130–141

    Article  Google Scholar 

  19. Mackey MC (1977) Glass L. Oscillation and chaos in physiological control systems. Sci, New Series 197 (4300):287– 289

    Google Scholar 

  20. Weigend AA, Gershenfeld NA (1994) Time series Prediction: forecasting the future and understanding the past. Harlow UK: Addison Wesley

  21. Han M, Xi JH, Xu SG, Yin FL (2004) Prediction of chaotic time series based on the recurrent predictor neural network. IEEE Trans Signal Process 52(12):3409–3416

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was supported by projects 60972095, 61271362, and 61671362 of the National Natural Science Foundation of China.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qinkun Xiao.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, Q. Recurrent neural network system using probability graph model optimization. Appl Intell 46, 889–897 (2017). https://doi.org/10.1007/s10489-016-0879-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-016-0879-7

Keywords

Navigation