Skip to main content
SpringerLink
Log in

A hybrid approach for training recurrent neural networks: application to multi-step-ahead prediction of noisy and large data sets

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Noisy and large data sets are extremely difficult to handle and especially to predict. Time series prediction is a problem, which is frequently addressed by researchers in many engineering fields. This paper presents a hybrid approach to handle a large and noisy data set. In fact, a Self Organizing Map (SOM), combined with multiple recurrent neural networks (RNN) has been trained to predict the components of noisy and large data set. The SOM has been developed to construct incrementally a set of clusters. Each cluster has been represented by a subset of data used to train a recurrent neural network. The back propagation through time has been deployed to train the set of recurrent neural networks. To show the performances of the proposed approach, a problem of instruction addresses prefetching has been treated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Nerrand O, P.Roussel-Ragot, Urbani D, L.Personnaz, Dreyfus G (1994) Training recurrent neural networks: why and how? an illustration in dynamical process modeling. IEEE Trans Neural Netw 5(2):178–184

    Article  Google Scholar 

  2. Inoue H, Narihisa H (2000) Predicting chaotic time series by ensembles self-generating neural networks. International joint conference on neural network (IJCNN’00), p 2231

  3. Connor T, Douglas R (1994) Reccurent neural networks and robust time series prediction. IEEE Trans Neural Netw 5(2):240–254

    Article  Google Scholar 

  4. Deshpande M, Karipys G (2004) Selective Markov models for predicting web pages accesses. ACM Trans Internet Technol 4(2):163–184

    Article  Google Scholar 

  5. Tran N, Reed A (2001) ARIMA time series modeling and forecasting for adaptive I/O prefetching. In: Proceedings of the international conference on supercomputing, pp 473–485

  6. Ho SL, Xie M, Goh TN (2002) A comparative study of neural network and Box–Jenkins ARIMA modeling in time series prediction. In: Proceedings of the 26th international conference on computers and industrial engineering 42(2–4):371–375

  7. Owens AJ (2000) Empirical modeling of very large data sets using neural network. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural network, vol 6, pp 302–307

  8. Principe JC, Wang L, Motter MA (1998) Local dynamic modelling with self-organizing maps and applications to nonlinear system identification and control. Proc IEEE 86(11)

  9. Cho K, Wang B (1996) Radial basis function based adaptive fuzzy systems and their application to system identification and prediction. Fuzzy Sets Syst 83:325–339

    Article  MathSciNet  Google Scholar 

  10. Kohonen T (1999) Self-organizing maps and learning vector quantization for feature sequences. Neural Processing Lett 10(2):151–159

    Article  Google Scholar 

  11. Leung H, Lo T, Wang S (2001) Prediction of noisy chaotic time series using an optimal radial basis function neural network. IEEE Trans Neural Netw 12(5):1163–1172

    Article  Google Scholar 

  12. Lin T, Horn G, Tino P, Lee Giles C (1996) Learning long-term dependencies in NARX recurrent neural networks. IEEE Trans Neural Netw 7(6):1329

    Article  Google Scholar 

  13. Lin T, Giles CL, Horne B, Kung SY (1997) A delay damage model selection algorithm for NARX neural networks. IEEE Trans Signal Process 45(11):2719–2730

    Article  Google Scholar 

  14. Parlos AG, Rais OT, Atiya AF (2000) Multi-step-ahead prediction using dynamic recurrent neural networks. Neural Netw 13(7):765–786

    Article  Google Scholar 

  15. Lee Giles C, Steve Lawrence, Tsoi AC (2001) Noisy time series prediction using a recurrent neural network and grammatical inference. Mach Learning 44(1/2):161–183

    Article  MATH  Google Scholar 

  16. Motter AM (2000) Predictive multiple model switching control with the self-organizing map. In: Proceedings of international joint conference in neural network, 4(4):IEEE-INNS-ENNS, 4317

  17. Wichard JD, Ogorzalek M (2004) Time series prediction with ensemble models. In: Proceedings of international joint conference in neural network, Busdapest

  18. Chtourou S, Chtourou M, Hammami O (2006) Neural network based memory access prediction support for soc dynamic reconfiguration. In: Proceedings of the international joint conference on neural network, pp 5130–5136

  19. Sherwood T, Sair S, Calder B (2003) Phase tracking and prediction. In: Proceedings of the 30th international symposium on computer architecture (ISCA), pp 336–347

  20. Jimenez DA, Lin C (2002) Neural methods for dynamic branch prediction. ACM Trans Comput Syst 20(4):369–397

    Article  Google Scholar 

  21. Sakr MF, Giles CL, Levitan SP, Horne BG, Maggini M, Chiarulli DM (1996) On-line prediction of multiprocessor memory access patterns. In: Proceedings of the IEEE international conference on neural networks, p 1564

  22. Sakr MF, Levitan SP, Chiarulli DM, Horne BG, Giles CL (1997) Predicting multiprocessor memory access patterns with learning models. In: Fisher D (ed) Proceedings of the fourteenth international conference on machine learning. Morgan Kaufmann, San Francisco, pp 305–312

  23. Atiya FA, Parlos AG (2000) New results on recurrent network training: unifying the algorithms and accelerating convergence. IEEE Trans Neural Netw 11(3):697–709

    Article  Google Scholar 

  24. Chalup SK, Blair AD (2003) Incremental training of first order recurrent neural networks to predict a context-sensitive language. Neural Netw 16:955–972

    Article  Google Scholar 

  25. Liu D, Chang T-S, Zhang Y (2002) A constructive algorithm for feedforward neural networks with incremental training. IEEE Trans Circuits Syst I Fundamental Theory Appl 49(12):1876–1879

    Article  Google Scholar 

  26. Dittenbach M, Merkl D, Rauber A (2000) The growing hierarchical self organizing map. In: Proceedings of the international joint conference on neural networks, pp 15–19

  27. Hammond J, MacClean D, Valova I (2006) A parallel implementation of a growing SOM promoting independent neural networks over distributed input space. In: Proceedings of the international joint conference on neural networks, pp 958–965

  28. http://rogue.colorado.edu/Pin/index.html

Download references

Author information

Authors and Affiliations

  1. Intelligent Control, design and Optimization of complex System National School of Engineers of Sfax, (ENIS), Route de Sokra, B.P. W, 3038, Sfax, Tunisie

    S. Chtourou & M. Chtourou

  2. Unité d’Electronique et Informatique, Ecole Nationale Supérieure de Techniques Avancées, ENSTA, 32 bd Victor, 75015, Paris, France

    O. Hammami

Authors
  1. S. Chtourou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Chtourou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. Hammami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Chtourou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chtourou, S., Chtourou, M. & Hammami, O. A hybrid approach for training recurrent neural networks: application to multi-step-ahead prediction of noisy and large data sets. Neural Comput & Applic 17, 245–254 (2008). https://doi.org/10.1007/s00521-007-0116-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-007-0116-8

Keywords

Search

Navigation