Skip to main content
SpringerLink
Log in

A deterministic model selection scheme for incremental RBFNN construction in time series forecasting

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper presents a fast and new deterministic model selection methodology for incremental radial basis function neural network (RBFNN) construction in time series prediction problems. The development of such special designed methodology is motivated by the problems that arise when using a K-fold cross-validation-based model selection methodology for this paradigm: its random nature and the subjective decision for a proper value of K, resulting in large bias for low values and high variance and computational cost for high values. Taking into account these drawbacks, the proposed model selection approach is a combined algorithm that takes advantage of two balanced and representative training and validation sets for their use in RBFNN initialization, optimization and network model evaluation. This way, the model prediction accuracy is improved, getting small variance and bias, reducing the computation time spent in selecting the model and avoiding random and computationally expensive model selection methodologies based on K-fold cross-validation procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Weigend AS, Gershenfeld NA (1993) Time series prediction: forecasting the future and understanding the past. Addison-Wesley, Reading

    Google Scholar 

  2. Qi M, Zhang GP (2001) An investigation of model selection criteria for neural network time series forecasting. Eur J Oper Res 132:666–680

    Article  MATH  Google Scholar 

  3. Ghazali R, Hussain AJ et al (2008) The application of ridge polynomial neural network to multi-step ahead financial time series prediction. Neural Comput Appl 17(3):311–323

    Article  Google Scholar 

  4. Juhos I, Makra L, Tóth B (2008) The behaviour of the multi-layer perceptron and the support vector regression learning methods in the prediction of NO and NO2 concentrations in Szeged, Hungary. Neural Comput Appl 18(2):193–205

    Google Scholar 

  5. Harpham C, Dawson CW (2006) The effect of different basis functions on a radial basis function network for time series prediction: a comparative study. Neurocomputing 69:2161–2170

    Article  Google Scholar 

  6. Du H, Zhang N (2008) Time series prediction using evolving radial basis function neural networks with new encoding scheme. Neurocomputing 71:1388–1400

    Article  Google Scholar 

  7. Zhang G, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14:35–62

    Article  Google Scholar 

  8. Menezes JMP, Barreto GA (2008) Long-term time series prediction with the NARX network: an empirical evaluation. Neurocomputing 71(16):3335–3343

    Article  Google Scholar 

  9. Carpinteiro OAS, Lima I et al (2006) A hierarchical neural model with time windows in long-term electrical load forecasting. Neural Comput Appl 16(4–5):465–470

    Google Scholar 

  10. Coyle D, Prasad G, McGinnity TM (2006) A time-series prediction approach for feature extraction in a brain-computer interface. IEEE Trans Neural Syst Rehabil Eng 13(4):461–467

    Article  Google Scholar 

  11. Atiya AF, El-Shoura SM, Shaheen SI, El-Sherif MS (1999) A comparison between neural-network forecasting techniques-case study: river flow forecasting. IEEE Trans Neural Netw 10((2):402–409

    Article  Google Scholar 

  12. Palit AK, Popovic D (2005) Computational intelligence in time series forecasting. Springer, Berlin

    MATH  Google Scholar 

  13. Park J, Sandberg IW (1991) Universal approximation using radial basis function networks. Neural Comput 3(2):246–257

    Article  Google Scholar 

  14. Aran O, Yildiz OT, Alpaydin E (2009) On incremental framework based on cross-validation for estimating the architecture of a multilayer perceptron. Int J Pattern Recognit Aritif Intell 23(2):159–190

    Article  Google Scholar 

  15. Kaminski W, Strumillo P (1997) Kernel orthonormalization in radial basis function neural networks. IEEE Trans Neural Netw 8(5):1177–1183

    Article  Google Scholar 

  16. Gomm JB, Yu DL (2000) Selecting radial basis function network centers with recursive orthogonal least squares training. IEEE Trans Neural Netw 11(2):306–314

    Article  Google Scholar 

  17. Paetz J (2004) Reducing the number of neurons in radial basis function networks with dynamic decay adjustment. Neuro Comput 62:79–91

    Google Scholar 

  18. Alpaydin E (1991) GAL: networks that grow when they learn and shrink when they forget. Technical report, TR-91-032, ICSI, Berkeley, CA

  19. Kwok T, Yeung D (1997) Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans Neural Netw 8(3):630–645

    Article  Google Scholar 

  20. Leung H, Dubash N, Xie N (2002) Detection of small objects in clutter using a GA-RBF neural network. IEEE Trans Aerosp Eelctron Syst 38(1):98–118

    Article  Google Scholar 

  21. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: International symposium on information theory, 2nd edn. Tsahkadsor, Armenian SSR, pp 267–281

  22. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464

    Article  MATH  Google Scholar 

  23. Rissanen J (1975) Modeling by shortest data description. Automatica 14:465–471

    Article  Google Scholar 

  24. Lendasse A, Wertz V, Verleysen M (2003) Model selection with cross-validations and bootstraps—application to time series prediction RBFN models. In: Joint international conference on artificial neural networks (ICANN)/international on neural information processing (ICONIP), Proceedings, Springer, Lecture notes in computer science, vol 2714, pp 573–580

  25. Constantinopoulos C, Likas A (2006) An incremental training method for the probabilistic RBF network. IEEE Trans Neural Netw 17(4):966–974

    Article  Google Scholar 

  26. Gholipour A, Araabi BN, Lucas C (2006) Predicting chaotic time series using neural and neurofuzzy models: a comparative study. Neural Process Lett 24:217–239

    Article  Google Scholar 

  27. Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th international journal conference on artificial intelligence, Canada, vol 2, pp 1137–1143

  28. Efron B, Tibshirani RJ (1997) On cross-validation: the .632+ bootstrap method. J Am Stat Assoc 92:548–560

    Article  MathSciNet  MATH  Google Scholar 

  29. Neural Network Toolbox 6 User’s Guide (1992–2009) The MathWorks, Inc

  30. Guillén A, González J, Rojas I, Pomares H, Herrera LJ, Valenzuela O, Prieto A (2007) Using fuzzy logic to improve a clustering technique for function approximation. Neurocomputing 70:2853–2860

    Article  Google Scholar 

  31. Rojas I, Pomares H, González J, Bernier JL, Ros E, Pelayo FJ, Prieto A (2000) Analysis of the functional block involved in the designed of radial basis function networks. Neural Process Lett 12(1):1–17

    Article  MATH  Google Scholar 

  32. Guillén A, Pomares H, Rojas I, González J, Herrera LJ, Rojas F, Valenzuela O (2008) Studying possibility in a clustering algorithm for RBFNN design for function approximation. Neural Comput Appl 17(1):75–89

    Google Scholar 

  33. Park J, Sandberg IW (1993) Approximation and radial basis function networks. Neural Comput 5:305–316

    Article  Google Scholar 

  34. Benoudjit N, Verleysen M (2003) On the kernel widths in radial basis function networks. Neural Process Lett 18(2):139–154

    Article  Google Scholar 

  35. Karayiannis NB, Mi GW (1997) Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques. IEEE Trans Neural Netw 8(6):1492–1506

    Article  Google Scholar 

  36. Golub G, Loan CV (1961) Matrix computations. The Johns Hopkins University Press, Baltimore

    Google Scholar 

  37. González J, Rojas I, Ortega J et al (2003) Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation. IEEE Trans Neural Netw 14(6):1478–1495

    Article  Google Scholar 

  38. Singh V, Gupta I, Gupta HO (2007) ANN-based estimator for distillation using Levenberg-Marquardt approach. Eng Appl Artif Ingell 20:249–259

    Article  Google Scholar 

  39. May RM (1976) Simple mathematical models with very complicated dynamics. Nature 261:459–467

    Article  Google Scholar 

  40. Hénon M (1976) A two-dimensional mapping with a strange attractor. Commun Math Phys 50(1):69–77

    Article  MATH  Google Scholar 

  41. Mackey MC, Glass L (1977) Oscillation and chaos in physiological control-systems. Science 197(4300):287–289

    Article  Google Scholar 

  42. Jayawardena AW, Xu PC et al (2006) Determining the structure of a radial basis function network for prediction of nonlinear hydrological time series. Hydrological Sci J 51(1):21–44

    Article  Google Scholar 

  43. Yao X, Liu Y (1996) EPNet for chaotic time-series prediction. In: First Asia-Pacific conference on simulated evolution and learning, proceedings, Springer, Lecture notes in computer science, vol 1285, pp 146–156

  44. Chen YH, Yang B et al (2005) Time-series forecasting using flexible neural tree model. Inf Sci 174(3–4):219–235

    Article  Google Scholar 

  45. Box GEP, Hunter WG, Hunter JS (1978) Statistics for experimenters: an introduction to design, data analysis and model building. Wiley, NewYork

    MATH  Google Scholar 

  46. Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control. Wiley, London

    MATH  Google Scholar 

  47. Balaguer E, Palomares A, Soria E, Martin-Guerrero JD (2008) Predicting service request in support centres based on nonlinear dynamics, ARMA modeling and neural networks. Expert Syst Appl 34(1):665–672

    Article  Google Scholar 

  48. Shiblee M, Kalra PK, Chandra B (2009) Time series prediction with multilayer perceptron (MLP): a new generalized error based approach. Adv Neuro-Inf Process 5507:37–44

    Article  Google Scholar 

  49. Boyacioglu MA, Avci D (2010) An adaptive network-based fuzzy inference system (ANFIS) for the prediction of stock market return: the case of the Istanbul stock exchange. Expert Syst Appl 37(12):7908–7912

    Article  Google Scholar 

  50. Sorjamaa A, Hao J, Lendasse A (2005) Mutual information and K-nearest neighbors approximator for time series prediction. In: 15th International conference on artificial neural networks (ICANN 2005), Proceedings, Springer, Lecture notes in computer science, vol 3697, pp 553–558

  51. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Spanish Project TIN2007-60587 and the FPU research grant AP2007-03009. The authors also want to thank all the anonymous reviewers for their suggestions and comments.

Author information

Authors and Affiliations

  1. Department of Computer Architecture and Computer Technology, CITIC-UGR, University of Granada, 18071, Granada, Spain

    J. P. Florido, H. Pomares, I. Rojas, J. M. Urquiza & M. A. Lopez-Gordo

Authors
  1. J. P. Florido
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Pomares
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I. Rojas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. M. Urquiza
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. A. Lopez-Gordo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. P. Florido.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Florido, J.P., Pomares, H., Rojas, I. et al. A deterministic model selection scheme for incremental RBFNN construction in time series forecasting. Neural Comput & Applic 21, 595–610 (2012). https://doi.org/10.1007/s00521-010-0466-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-010-0466-5

Keywords

Search

Navigation