Skip to main content
Log in

Comparative study on local and global strategies for confidence estimation in neural networks and extensions to improve their predictive power

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The use of confidence estimation techniques on neural networks outputs plays an important role when these mathematical models are applied in many practical applications. In general, the method to provide confidence estimation is dependent on the neural network architecture, but traditionally, most popular prediction interval (PI) estimation methods are only valid under strong assumptions, which are rarely satisfied in practical real problems. In this paper, we present a comparative study of local and global strategies for PI calculations and propose novel methods in both approaches to improve the predictive power of multilayer perceptron and radial basis function neural network models when the data are heterogeneous, both in density and residual variance. We apply our methods and make comparisons in a variety of simulated and real problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Bishop C (1994) Novelty detection and neural network validation. Vision, and Image Signal Processing. IEE Proc 141(4):217–222

    Google Scholar 

  2. Bishop C (1995) Neural networks for pattern recognition. Clarendon Press, Oxford

    Google Scholar 

  3. Caers J (2001) Geostatistical reservoir modelling using statistical pattern recognition. J Petrol Sci Eng 29(3-4):177–188

    Article  Google Scholar 

  4. Chinman R, Ding J (1998) Prediction limit estimation for neural network models. Neural Netw IEEE Trans 9(6):1515–1522

    Article  Google Scholar 

  5. Chryssolouris G, Lee M, Ramsey A (1996) Confidence interval prediction for neural network models. Neural Netw IEEE Trans 7(1):229–232

    Article  Google Scholar 

  6. Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst (MCSS) 2:303–314

    Article  MathSciNet  MATH  Google Scholar 

  7. da Silva Neves CAM, Roisenberg M, Neto GS (2009) A method to estimate prediction intervals for artificial neural networks that is sensitive to the noise distribution in the outputs. In: Neural networks, 2009. IJCNN 2009. International joint conference on, IEEE Press, Piscataway, pp 2238–2242

  8. De Veaux RD, Schweinsberg J, Schumi J, Ungar LH (1998) Prediction intervals for neural networks via nonlinear regression. Technometrics 40(4):273–282

    Article  MathSciNet  MATH  Google Scholar 

  9. Dybowski R, Gant V (2001) Confidence intervals and prediction intervals for feedforward neural networks. Cambridge University Press, Cambridge

    Google Scholar 

  10. Giordano F, La Rocca M, Perna C (2009) Neural network sieve bootstrap prediction intervals: Some real data evidence. In: Proceedings of the 2009 conference on new directions in neural networks: 18th Italian Workshop on Neural Networks: WIRN 2008, IOS Press, Amsterdam, pp 205–213

  11. Heskes T (1997) Practical confidence and prediction intervals. In: Advances in neural information processing systems 9, MIT press, pp 176–182

  12. Ho S, Xie M, Tang L, Xu K, Goh T (2001) Neural network modeling with confidence bounds: a case study on the solder paste deposition process. Electron Packag Manuf IEEE Trans 24(4):323–332

    Article  Google Scholar 

  13. Hwang JTG, Ding AA (1997) Prediction intervals for artificial neural networks. J Am Stat Assoc 92(438):748–757

    Article  MathSciNet  MATH  Google Scholar 

  14. Khosravi A, Nahavandi S, Creighton D (2010a) Load forecasting and neural networks: a prediction interval-based perspective. In: Panigrahi B, Abraham A, Das S (eds) Computational intelligence in power engineering, studies in computational intelligence, vol 302, Springer, Berlin, pp 131–150

  15. Khosravi A, Nahavandi S, Creighton D (2010) A prediction interval-based approach to determine optimal structures of neural network metamodels. Expert Syst Appl 37(3):2377–2387

    Article  Google Scholar 

  16. Khosravi A, Nahavandi S, Creighton D, Atiya A (2011) Comprehensive review of neural network-based prediction intervals and new advances. Neural Netw IEEE Trans 22(9):1341–1356

    Article  Google Scholar 

  17. Khosravi A, Nahavandi S, Creighton D, Atiya A (2011) Lower upper bound estimation method for construction of neural network-based prediction intervals. Neural Netw IEEE Trans 22(3):337–346

    Article  Google Scholar 

  18. Leonard J, Kramer M, Ungar L (1992) A neural network architecture that computes its own reliability. Comput Chem Eng 16(9):819–835

    Article  Google Scholar 

  19. Lu T, Viljanen M (2009) Prediction of indoor temperature and relative humidity using neural network models: model comparison. Neural Comput Appl 18:345–357

    Article  Google Scholar 

  20. Mackay D (1995) Probable networks and plausible predictions—a review of practical bayesian methods for supervised neural networks. Netw Comput Neural Syst 6(3):469–505

    Article  MathSciNet  MATH  Google Scholar 

  21. Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1(2):281–294

    Article  Google Scholar 

  22. Neto AR, Roisenberg M, Neto GS (2010) Efficient confidence bounds for rbf networks for sparse and high dimensional data. In: Proceedings of the 20th international conference on artificial neural networks: part III, Springer, Berlin, ICANN’10, pp 423–428

  23. Olatunji SO, Selamat A, Abdulraheem A (2011) Modeling the permeability of carbonate reservoir using type-2 fuzzy logic systems. Comput Ind 62(2):147–163

    Article  Google Scholar 

  24. Pacheco-Vega A, Sen M, Yang K, McClain RL (2001) Neural network analysis of fin-tube refrigerating heat exchanger with limited experimental data. Int J Heat Mass Transf 44(4):763–770

    Article  MATH  Google Scholar 

  25. Pierce SG, Worden K, Bezazi A (2008) Uncertainty analysis of a neural network used for fatigue lifetime prediction. Mech Syst Signal Process 22(6):1395–1411

    Article  Google Scholar 

  26. Prechelt L (1994) Proben1—a set of neural network benchmark problems and benchmarking rules. Tech. rep., Fakultat fur Informatik Universitat Karlsruhe

  27. Shao R, Martin E, Zhang J, Morris A (1997) Confidence bounds for neural network representations. Comput Chem Eng 21(Supplement(0)):S1173–S1178

    Google Scholar 

  28. Terrell GR, Scott DW (1992) Variable kernel density estimation. Ann Stat 20(3):1236–1265

    Article  MathSciNet  MATH  Google Scholar 

  29. Wedding DK II, Cios KJ (1996) Time series forecasting by combining rbf networks, certainty factors, and the box-jenkins model. Neurocomputing 10(2):149–168

    Article  MATH  Google Scholar 

  30. Wedding DK II, Cios KJ (1998) Certainty factors versus parzen windows as reliability measures in rbf networks. Neurocomputing 19(1):151–165

    Article  Google Scholar 

  31. Wong P, Bruce A, Gedeon T (2002) Confidence bounds of petrophysical predictions from conventional neural networks. Geosci Remote Sens IEEE Trans 40(6):1440–1444

    Article  Google Scholar 

  32. Yang L, Kavli T, Carlin M, Clausen S, de Groot PFM (2002) An evaluation of confidence bound estimation methods for neural networks. In: Advances in computational intelligence and learning: methods and applications, Kluwer, B.V., Deventer, The Netherlands, pp 71–84

  33. Yu L, Huang W, Lai KK, Wang S (2006) A reliability-based rbf network ensemble model for foreign exchange rates predication. In: Proceedings of the 13th international conference on neural information processing, volume part III, Springer, Berlin, ICONIP’06, pp 380–389

  34. Zio E (2006) A study of the bootstrap method for estimating the accuracy of artificial neural networks in predicting nuclear transient processes. Nucl Sci IEEE Trans 53(3):1460–1478

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mauro Roisenberg.

Additional information

This work was partially granted by Petrobras S.A. as project: Intelligent Techniques for Reservoir Characterization—SICRES.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rodrigues Neto, A.C., das Neves, C.A.M. & Roisenberg, M. Comparative study on local and global strategies for confidence estimation in neural networks and extensions to improve their predictive power. Neural Comput & Applic 22, 1519–1530 (2013). https://doi.org/10.1007/s00521-012-1051-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1051-x

Keywords

Navigation