Skip to main content
SpringerLink
Log in

Time series forecasting with feedforward neural networks trained using particle swarm optimizers for dynamic environments

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Several studies have applied particle swarm optimization (PSO) algorithms to train neural networks (NNs) for time series forecasting and the results indicated good performance. These studies, however, assumed static environments, making the PSO trained NNs unsuitable for forecasting many real-world time series which are generated by non-stationary processes. This study formulates training of a NN forecaster as a dynamic optimization problem, to investigate the application of a dynamic PSO algorithm to train NNs in forecasting time series in non-stationary environments. For this purpose, a set of experiments were conducted on three simulated and seven real-life time series forecasting problems under four different dynamic scenarios. Results obtained are compared to the results of NNs trained using a standard PSO and resilient backpropagation (Rprop). The results show that the NNs trained using dynamic PSO algorithms outperform the NNs trained using PSO and Rprop. These findings highlight the potential of using dynamic PSO in training NNs for real-world forecasting applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abdulkarim S, Garko A (2016) Effectiveness of firefly algorithm based neural network in time series forecasting. Bayero J Pure Appl Sci 9(1):6–10

    Article  Google Scholar 

  2. Abdulkarim SA (2018) Time series forecasting using dynamic particle swarm optimizer trained neural networks. Phd thesis, University of Pretoria

  3. Abdulkarim SA, Engelbrecht AP (2019) Time series forecasting using neural networks: Are recurrent connections necessary? Neural Process Lett 50:2763–2795

    Article  Google Scholar 

  4. Adhikari R, Agrawal RK (2011) Effectiveness of PSO based neural network for seasonal time series forecasting. In: Proceedings of the fifth Indian international conference on artificial intelligence, pp 231–244

  5. Ahmadi MH, Aghaj SSG, Nazeri A (2013) Prediction of power in solar stirling heat engine by using neural network based on hybrid genetic algorithm and particle swarm optimization. Neural Comput Appl 22(6):1141–1150

    Article  Google Scholar 

  6. Blackwell T, Branke J (2006) Multiswarms, exclusion, and anti-convergence in dynamic environments. IEEE Trans Evol Comput 10(4):459–472

    Article  Google Scholar 

  7. Blackwell T, Branke J, Li X (2008) Particle swarms for dynamic optimization problems. Swarm intelligence. Springer, Berlin pp 193–217

  8. Blackwell TM, Bentley PJ (2002) Dynamic search with charged swarms. In: Proceedings of the genetic and evolutionary computation conference, pp 9–16

  9. Box GEP, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: forecasting and control. Wiley, New York

    MATH  Google Scholar 

  10. Brezak D, Bacek T, Majetic D, Kasac J, Novakovic B (2012) A comparison of feed-forward and recurrent neural networks in time series forecasting. In: Proceedings of IEEE conference on computational intelligence for financial engineering and economics, pp 1–6

  11. Carlisle A, Dozier G (2000) Adapting particle swarm optimization to dynamic environments. Int Conf Artif Intell 1:429–434

    Google Scholar 

  12. Carlisle A, Dozler G (2002) Tracking changing extrema with adaptive particle swarm optimizer. In: Automation congress, 2002 proceedings of the 5th biannual world, vol 13. IEEE, pp 265–270

  13. Chatfield C (2016) The analysis of time series: an introduction. CRC Press, Boca Raton

    MATH  Google Scholar 

  14. Deb C, Zhang F, Yang J, Lee SE, Shah KW (2017) A review on time series forecasting techniques for building energy consumption. Renew Sustain Energy Rev 74:902–924

    Article  Google Scholar 

  15. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(Jan):1–30

    MathSciNet  MATH  Google Scholar 

  16. Duhain JGOL (2011) Particle swarm optimization in dynamically changing environment an empirical study. MSc Thesis, University of Pretoria

  17. Eberhart RC, Kennedy J (1995) Particles swarm optimization. In: Proceedings of IEEE international conference on neural networks, Perth, Australia, pp 1942–1948

  18. Engelbrecht AP (2007) Computational intelligence: an introduction. Wiley, New York

    Book  Google Scholar 

  19. Frank RJ, Davey N, Hunt SP (2001) Time series prediction and neural networks. J Intell Robot Syst 31(1–3):91–103

    Article  Google Scholar 

  20. Galvan IM, Isasi P (2001) Multi-step learning rule for recurrent neural models: an application to time series forecasting. Neural Process Lett 13(2):115–133

    Article  Google Scholar 

  21. Gordan B, Armaghani DJ, Hajihassani M, Monjezi M (2016) Prediction of seismic slope stability through combination of particle swarm optimization and neural network. Eng Comput 32(1):85–97

    Article  Google Scholar 

  22. Grimaldi EA, Grimaccia F, Mussetta M, Zich RE (2004) PSO as an effective learning algorithm for neural network applications. In: 2004 3rd International Conference on proceedings of the computational electromagnetics and its applications, 2004 (ICCEA 2004). IEEE, pp 557–560

  23. Hajihassani M, Armaghani DJ, Monjezi M, Mohamad ET, Marto A (2015) Blast-induced air and ground vibration prediction: a particle swarm optimization-based artificial neural network approach. Environ Earth Sci 74(4):2799–2817

    Article  Google Scholar 

  24. Hamzacebi C (2008) Improving artificial neural networks performance in seasonal time series forecasting. Inf Sci 178(23):4550–4559

    Article  Google Scholar 

  25. Harrison KR, Ombuki-berman BM, Engelbrecht AP (2016) A radius-free quantum particle swarm optimization technique for dynamic optimization problems. In: Proceedings of IEEE congress on evolutionary computation, pp 578–585

  26. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

  27. Iqbal S, Zang X, Zhu Y, Liu X (2014) Introducing undergraduate electrical engineering students to chaotic dynamics: computer simulations with logistic map and buck converter. In: IEEE modelling symposium, pp 47–52

  28. Jha GK, Thulasiraman P, Thulasiram RK (2009) PSO based neural network for time series forecasting. In: Proceedings of IEEE international joint conference on neural networks, pp 1422–1427

  29. Jordehi AR (2014) Particle swarm optimisation for dynamic optimisation problems: a review. Neural Comput Appl 25:1507–1516

    Article  Google Scholar 

  30. Kedrowski R, Nelson J, Nair AS, Ranganathan P (2018) Short-term seasonal energy forecasting. In: 2018 IEEE international conference on electro/information technology (EIT). IEEE, pp 696–700

  31. Kennedy J, Mendes R (2002) Population structure and particle swarm performance. Proc IEEE Congr Evol Comput 2:1671–1676

    Google Scholar 

  32. Laepes A, Farben R (1987) Nonlinear signal processing using neural networks: prediction and system modelling. In: Technical Report, Los Alamos National Laboratory, Los Alamos, NM

  33. Lasheras FS, de Cos Juez FJ, Sánchez AS, Krzemień A, Fernández PR (2015) Forecasting the comex copper spot price by means of neural networks and arima models. Resour Policy 45:37–43

    Article  Google Scholar 

  34. LeCun YA, Bottou L, Orr GB, Müller KR (2012) Efficient backprop. Neural networks: tricks of the trade. Springer, Berlin, pp 9–48

    Book  Google Scholar 

  35. Li C, Sprott JC, Thio W (2015) Linearization of the Lorenz system. Phys Lett A 379(10):888–893

    Article  MathSciNet  Google Scholar 

  36. Li X, Dam KH (2003) Comparing particle swarms for tracking extrema in dynamic environments. IEEE Proc Evol Comput 3:1772–1779

    Google Scholar 

  37. Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60

    Article  MathSciNet  Google Scholar 

  38. Milano P, Elettrotecnica D (2004) PSO as an effective learning algorithm for neural network applications. In: Proceedings of the international conference on computational electronics and its applications, pp 557–560

  39. Mohammad AA, Sohrab Z, Ali L, Ali E, Ioannis E (2013) Reservoir permeability prediction by neural networks combined with hybrid genetic algorithm and particle swarm optimization. Geophys Prospect 61(3):582–598

    Article  Google Scholar 

  40. Momeni E, Armaghani DJ, Hajihassani M, Amin M (2015) Prediction of uniaxial compressive strength of rock samples using hybrid particle swarm optimization-based artificial neural networks. Measurement 60:50–63

    Article  Google Scholar 

  41. Morrison RW (2003) Performance measurement in dynamic environments. In: Proceedings of the GECCO workshop on evolutionary algorithms for dynamic optimization problems, pp 5–8

  42. Pampara G, Engelbrecht AP (2015) Towards a generic computational intelligence library: preventing insanity. In: Proceedings of the IEEE symposium series on computational intelligence, pp 1460–1467

  43. Qi M, Zhang GP (2001) An investigation of model selection criteria for neural network time series forecasting. Eur J Oper Res 132(3):666–680

    Article  Google Scholar 

  44. Rakitianskaia A (2011) Using particle swarm optimisation to train feedforward neural networks in dynamic environments. MSc. thesis, University of Pretoria

  45. Rakitianskaia A, Engelbrecht AP (2008) Cooperative charged particle swarm optimiser. In: IEEE congress on evolutionary computation, pp 933–939

  46. Rakitianskaia A, Engelbrecht AP (2009) Training neural networks with PSO in dynamic environments. In: IEEE congress on evolutionary computation, 2009 (CEC’09), pp 667–673

  47. Rakitianskaia A, Engelbrecht AP (2015) Saturation in PSO neural network training: good or evil? IEEE Congr Evol Comput 2:125–132

    Google Scholar 

  48. Riedmiller M, Braun H (1993) A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: IEEE international conference on neural networks, pp 586–591

  49. Röbel A (1994) The dynamic pattern selection algorithm: effective training and controlled generalization of backpropagation neural networks. In: Technical Report, Tech- nische Universität Berlin

  50. Sitte R, Sitte J (2000) Analysis of the predictive ability of time delay neural networks applied to the S&P 500 time series. IEEE Trans Syst Man Cybern C Appl Rev 30(4):568–572

    Article  Google Scholar 

  51. Soodi HA, Vural AM (2018) STATCOM estimation using back-propagation, PSO, shuffled frog leap algorithm, and genetic algorithm based neural networks. Comput Intell Neurosci 2018:6381610. https://doi.org/10.1155/2018/6381610

  52. Suresh A, Harish K, Radhika N (2015) Particle swarm optimization over back propagation neural network for length of stay prediction. Procedia Comput Sci 46:268–275

    Article  Google Scholar 

  53. Tang Z, Fishwick PA (1993) Feedforward neural nets as models for time series forecasting. ORSA J Comput 5(4):374–385

    Article  Google Scholar 

  54. Unger NJ, Ombuki-Berman BM, Engelbrecht AP (2013) Cooperative particle swarm optimization in dynamic environments. In: Proceedings of IEEE symposium on swarm intelligence, pp 172–179

  55. Van den Bergh F (2001) An analysis of particle swarm optimizers. Phd Thesis, University of Pretoria

  56. Van den Bergh F, Engelbrecht AP (2001) Effects of swarm size on cooperative particle swarm optimisers. In: Proceedings of the genetic and evolutionary computation conference, pp 892–899

  57. Van den Bergh F, Engelbrecht AP (2004) A cooperative approach to particle swarm optimization. IEEE Trans Evol Comput 8(3):225–239

    Article  Google Scholar 

  58. Van Wyk AB, Engelbrecht AP (2010) Overfitting by PSO trained feedforward neural networks. In: IEEE congress on evolutionary computation, pp 1–8

  59. Wessels LFA, Barnard E (1992) Avoiding false local minima by proper initialization of connections. IEEE Trans Neural Netw 3(6):899–905

    Article  Google Scholar 

  60. Yalcin N, Tezel G, Karakuzu C (2015) Epilepsy diagnosis using artificial neural network learned by PSO. Turk J Electr Eng Comput Sci 23(2):421–432

    Article  Google Scholar 

  61. Yeh W, Yeh Y, Chang P, Ke Y, Chung V (2014) Forecasting wind power in the Mai Liao wind farm based on the multi-layer perceptron artificial neural network model with improved simplified swarm optimization. Int J Electr Power Energy Syst 55:741–748

    Article  Google Scholar 

  62. Zhang G, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14(1):35–62

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Federal University Dutse, Dutse, Nigeria

    Salihu A. Abdulkarim

  2. Department of Industrial Engineering, Stellenbosch University, Stellenbosch, 7600, South Africa

    Andries P. Engelbrecht

  3. Computer Science Division, Stellenbosch University, Stellenbosch, 7600, South Africa

    Andries P. Engelbrecht

Authors
  1. Salihu A. Abdulkarim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andries P. Engelbrecht
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Salihu A. Abdulkarim.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdulkarim, S.A., Engelbrecht, A.P. Time series forecasting with feedforward neural networks trained using particle swarm optimizers for dynamic environments. Neural Comput & Applic 33, 2667–2683 (2021). https://doi.org/10.1007/s00521-020-05163-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05163-4

Keywords

Search

Navigation