Skip to main content
SpringerLink
Log in

Neural network-based construction of online prediction intervals

  • Brain inspired Computing&Machine Learning Applied Research-BISMLARE
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

With the emergence of online learning systems which generate ever-growing amounts of data, quantifying the uncertainty in predictions regarding the system’s operation is becoming increasingly more important. Prediction intervals offer a powerful tool for assessing prediction uncertainty in artificial neural network applications; nevertheless, little work has been conducted on constructing prediction intervals for online learning applications. In this work, we propose a hybrid approach which employs artificial neural networks to directly estimate prediction intervals for both batch and online approximation scenarios. The aim of the approach is to provide high-quality prediction intervals, combining high coverage probability for future observations with small and thus informative interval widths. Compared with three popular methods for offline construction of prediction intervals, the proposed approach demonstrates a strong capacity for reliably representing prediction uncertainty in real-world regression applications. The approach is extended to adaptive approximation, whereby four online learning schemes are proposed to iteratively update prediction intervals based on recent measurements, requiring a reduced computational cost compared to offline approximation. The four online prediction intervals methods are compared over two synthetic and one real-world regression datasets, whereby data arrive in a sequential manner. Our results suggest the potential of an online learning scheme relying on a human-like memory mechanism, to construct high-quality online prediction intervals, capable of adapting to dynamic changes in data patterns. The proposed method is associated with low computational cost—an attractive feature for online learning applications requiring real-time performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Haykin S (1999) Neural networks: a comprehensive foundation. Prentice Hall, Upper Saddle River

    MATH  Google Scholar 

  2. Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  3. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366. https://doi.org/10.1016/0893-6080(89)90020-8. arXiv:1011.1669v3

    MATH  Google Scholar 

  4. Guresen E, Kayakutlu G, Daim TU (2011) Using artificial neural network models in stock market index prediction. Expert Syst Appl 38(8):10389–10397. https://doi.org/10.1016/j.eswa.2011.02.068

    Google Scholar 

  5. Abhishek K, Singh M, Ghosh S, Anand A (2012) Weather forecasting model using artificial neural network. Procedia Technol 4:311–318. https://doi.org/10.1016/j.protcy.2012.05.047

    Google Scholar 

  6. Reppa V, Polycarpou MM, Panayiotou CG (2014) Adaptive approximation for multiple sensor fault detection and isolation of nonlinear uncertain systems. IEEE Trans Neural Netw Learn Syst 25(1):137–153. https://doi.org/10.1109/TNNLS.2013.2250301

    Google Scholar 

  7. Kabir HM, Khosravi A, Hosen MA, Nahavandi S (2018) Neural network-based uncertainty quantification: a survey of methodologies and applications. IEEE Access 6:36218–36234. https://doi.org/10.1109/ACCESS.2018.2836917

    Google Scholar 

  8. Khosravi A, Nahavandi S, Creighton D, Atiya AF (2011) Lower upper bound estimation method for construction of neural network-based prediction intervals. IEEE Trans Neural Netw 22(3):337–346. https://doi.org/10.1109/TNN.2010.2096824

    Google Scholar 

  9. Wang S, Jia C (2016) Prediction intervals for short-term photovoltaic generation forecasts. Proceedings - 5th International Conference on Instrumentation and Measurement, Computer, Communication, and Control, IMCCC 2015 pp 459–463, https://doi.org/10.1109/IMCCC.2015.103

  10. Zhang C, Wei H, Xie L, Shen Y, Zhang K (2016) Direct interval forecasting of wind speed using radial basis function neural networks in a multi-objective optimization framework. Neurocomputing 205:53–63. https://doi.org/10.1016/j.neucom.2016.03.061

    Google Scholar 

  11. IntHout J, Ioannidis JP, Rovers MM, Goeman JJ (2016) Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open 6(7):e010247. https://doi.org/10.1136/bmjopen-2015-010247

    Google Scholar 

  12. Kim JH, Wong K, Athanasopoulos G, Liu S (2011) Beyond point forecasting: evaluation of alternative prediction intervals for tourist arrivals. Int J Forecast 27(3):887–901. https://doi.org/10.1016/j.ijforecast.2010.02.014

    Google Scholar 

  13. Khosravi A, Nahavandi S, Creighton D, Atiya AF (2011) Comprehensive review of neural network-based prediction intervals and new advances. IEEE Trans Neural Netw Publ IEEE Neural Netw Council 22(9):1341–1356. https://doi.org/10.1109/TNN.2011.2162110

    Google Scholar 

  14. Hwang JTG, Ding AA, Hwang JTG, Ding AA (1997) Prediction intervals for artificial neural networks. J Am Stat Assoc 92(438):748–757

    MathSciNet  MATH  Google Scholar 

  15. Ungar LH, De Veaux RD, Rosengarten E (1996) Estimating Prediction Intervals for Artificial Neural Networks. Proc Of Ninth Yale Workshop on Adaptive and Learning Systems

  16. Nix D, Weigend A (1994) Estimating the mean and variance of the target probability distribution. Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94) pp 55–60 vol.1, https://doi.org/10.1109/ICNN.1994.374138

  17. Heskes T (1997) Practical confidence and prediction intervals. Adv Neural Inf Process Syst i:176–182

    Google Scholar 

  18. Khosravi A, Nahavandi S, Creighton D (2013) Prediction intervals for short-term wind farm power generation forecasts. IEEE Trans Sustain Energy 4(3):602–610. https://doi.org/10.1109/TSTE.2012.2232944

    Google Scholar 

  19. Pearce T, Zaki M, Brintrup A, Neely A (2018) High-quality prediction intervals for deep learning: a distribution-free, ensembled approach. arXiv:1802.07167

  20. Hadjicharalambous M, Polycarpou MM, Panayiotou CG (2018) Online approximation of prediction intervals using artificial neural networks. In: Artificial neural networks and machine learning ICANN 2018, Springer International Publishing 1:566–576. https://doi.org/10.1007/978-3-642-40728-4, arXiv:1412.7927

    Google Scholar 

  21. Pala AA (2015) Online machine learning techniques for predicting operator performance. PhD thesis, Technische Universitšat Berlin. arXiv:1605.01029

  22. Pérez-Cruz F, Van Vaerenbergh S, Murillo-Fuentes JJ, Lázaro-Gredilla M, Santamaria I (2013) Gaussian processes for nonlinear signal processing. IEEE Signal Process Mag 30(4):40–50. https://doi.org/10.1109/MSP.2013.2250352

    Google Scholar 

  23. Ak R, Li Y, Vitelli V, Zio E (2013) Multi-objective generic algorithm optimization of a neural network for estimating wind speed prediction intervals (submitted to applied soft computing)

  24. Taormina R, Chau KW (2015) ANN-based interval forecasting of streamflow discharges using the LUBE method and MOFIPS. Eng Appl Artif Intell 45:429–440. https://doi.org/10.1016/j.engappai.2015.07.019

    Google Scholar 

  25. Quan H, Srinivasan D, Khosravi A (2014) Uncertainty handling using neural network-based prediction intervals for electrical load forecasting. Energy 73:916–925. https://doi.org/10.1016/j.energy.2014.06.104

    Google Scholar 

  26. Galván IM, Valls JM, Cervantes A, Aler R (2017) Multi-objective evolutionary optimization of prediction intervals for solar energy forecasting with neural networks. Inf Sci 418–419:363–382. https://doi.org/10.1016/j.ins.2017.08.039

    Google Scholar 

  27. Sun X, Wang Z, Hu J (2017) Prediction interval construction for byproduct gas flow forecasting using optimized twin extreme learning machine. Math Probl Eng. https://doi.org/10.1155/2017/5120704

    MathSciNet  MATH  Google Scholar 

  28. Lian C, Zeng Z, Yao W, Tang H, Chen CLP (2016) Landslide displacement prediction with uncertainty based on neural networks with random hidden weights. IEEE Trans Neural Netw Learn Syst 27(12):2683–2695. https://doi.org/10.1109/TNNLS.2015.2512283

    Google Scholar 

  29. Ak R, Vitelli V, Zio E (2015) An interval-valued neural network approach for uncertainty quantification in short-term wind speed prediction. IEEE Trans Neural Netw Learn Syst 26(11):2787–2800. https://doi.org/10.1109/TNNLS.2015.2396933

    MathSciNet  Google Scholar 

  30. Shen Y, Wang X, Chen J (2018) Wind power forecasting using multi-objective evolutionary algorithms for wavelet neural network-optimized prediction intervals. Appl Sci 8(2):185. https://doi.org/10.3390/app8020185

    Google Scholar 

  31. Quan H, Srinivasan D, Khosravi A (2014) Particle swarm optimization for construction of neural network-based prediction intervals. Neurocomputing 127(ir):172–180. https://doi.org/10.1016/j.neucom.2013.08.020

    Google Scholar 

  32. Rana M, Koprinska I, Khosravi A, Agelidis VG (2013) Prediction intervals for electricity load forecasting using neural networks. In: The 2013 international joint conference on neural networks (IJCNN), pp 1–8. https://doi.org/10.1109/IJCNN.2013.6706839

  33. Khosravi A, Nahavandi S, Srinivasan D, Khosravi R (2015) Constructing optimal prediction intervals by using neural networks and bootstrap method. IEEE Trans Neural Netw Learn Syst 26(8):1810–1815. https://doi.org/10.1109/TNNLS.2014.2354418

    MathSciNet  Google Scholar 

  34. Leonard J, Kramer M, Ungar L (1992) A neural network architecture that computes its own reliability. Computers Chem Eng 16(9):819–835. https://doi.org/10.1016/0098-1354(92)80035-8

    Google Scholar 

  35. Ebbinghaus H (1885) Memory: a contribution to experimental psychology. Ann Neurosci 20(4):155–156. https://doi.org/10.5214/ans.0972.7531.200408 originally published 1885, translated 1913

    Google Scholar 

  36. Gao K, Zhu Y (2017) Deep data stream analysis model and algorithm with memory mechanism. IEEE Access 5:84–93. https://doi.org/10.1109/ACCESS.2016.2613922

    Google Scholar 

  37. Dua D, Graff C (2019) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine, CA. http://archive.ics.uci.edu/ml

  38. De Vito S, Massera E, Piga M, Martinotto L, Di Francia G (2008) On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuators B Chem 129(2):750–757. https://doi.org/10.1016/j.snb.2007.09.060

    Google Scholar 

Download references

Acknowledgements

This work has been supported by the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 739551 (KIOS CoE) and from the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development. All multi-variate regression datasets were obtained from the UCI Machine Learning Repository [37].

Author information

Authors and Affiliations

  1. KIOS Research and Innovation Center of Excellence, Nicosia, Cyprus

    Myrianthi Hadjicharalambous, Marios M. Polycarpou & Christos G. Panayiotou

Authors
  1. Myrianthi Hadjicharalambous
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marios M. Polycarpou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christos G. Panayiotou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Myrianthi Hadjicharalambous.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hadjicharalambous, M., Polycarpou, M.M. & Panayiotou, C.G. Neural network-based construction of online prediction intervals. Neural Comput & Applic 32, 6715–6733 (2020). https://doi.org/10.1007/s00521-019-04617-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04617-8

Keywords

Search

Navigation