Abstract
The use of confidence estimation techniques on neural networks outputs plays an important role when these mathematical models are applied in many practical applications. In general, the method to provide confidence estimation is dependent on the neural network architecture, but traditionally, most popular prediction interval (PI) estimation methods are only valid under strong assumptions, which are rarely satisfied in practical real problems. In this paper, we present a comparative study of local and global strategies for PI calculations and propose novel methods in both approaches to improve the predictive power of multilayer perceptron and radial basis function neural network models when the data are heterogeneous, both in density and residual variance. We apply our methods and make comparisons in a variety of simulated and real problems.










Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Bishop C (1994) Novelty detection and neural network validation. Vision, and Image Signal Processing. IEE Proc 141(4):217–222
Bishop C (1995) Neural networks for pattern recognition. Clarendon Press, Oxford
Caers J (2001) Geostatistical reservoir modelling using statistical pattern recognition. J Petrol Sci Eng 29(3-4):177–188
Chinman R, Ding J (1998) Prediction limit estimation for neural network models. Neural Netw IEEE Trans 9(6):1515–1522
Chryssolouris G, Lee M, Ramsey A (1996) Confidence interval prediction for neural network models. Neural Netw IEEE Trans 7(1):229–232
Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst (MCSS) 2:303–314
da Silva Neves CAM, Roisenberg M, Neto GS (2009) A method to estimate prediction intervals for artificial neural networks that is sensitive to the noise distribution in the outputs. In: Neural networks, 2009. IJCNN 2009. International joint conference on, IEEE Press, Piscataway, pp 2238–2242
De Veaux RD, Schweinsberg J, Schumi J, Ungar LH (1998) Prediction intervals for neural networks via nonlinear regression. Technometrics 40(4):273–282
Dybowski R, Gant V (2001) Confidence intervals and prediction intervals for feedforward neural networks. Cambridge University Press, Cambridge
Giordano F, La Rocca M, Perna C (2009) Neural network sieve bootstrap prediction intervals: Some real data evidence. In: Proceedings of the 2009 conference on new directions in neural networks: 18th Italian Workshop on Neural Networks: WIRN 2008, IOS Press, Amsterdam, pp 205–213
Heskes T (1997) Practical confidence and prediction intervals. In: Advances in neural information processing systems 9, MIT press, pp 176–182
Ho S, Xie M, Tang L, Xu K, Goh T (2001) Neural network modeling with confidence bounds: a case study on the solder paste deposition process. Electron Packag Manuf IEEE Trans 24(4):323–332
Hwang JTG, Ding AA (1997) Prediction intervals for artificial neural networks. J Am Stat Assoc 92(438):748–757
Khosravi A, Nahavandi S, Creighton D (2010a) Load forecasting and neural networks: a prediction interval-based perspective. In: Panigrahi B, Abraham A, Das S (eds) Computational intelligence in power engineering, studies in computational intelligence, vol 302, Springer, Berlin, pp 131–150
Khosravi A, Nahavandi S, Creighton D (2010) A prediction interval-based approach to determine optimal structures of neural network metamodels. Expert Syst Appl 37(3):2377–2387
Khosravi A, Nahavandi S, Creighton D, Atiya A (2011) Comprehensive review of neural network-based prediction intervals and new advances. Neural Netw IEEE Trans 22(9):1341–1356
Khosravi A, Nahavandi S, Creighton D, Atiya A (2011) Lower upper bound estimation method for construction of neural network-based prediction intervals. Neural Netw IEEE Trans 22(3):337–346
Leonard J, Kramer M, Ungar L (1992) A neural network architecture that computes its own reliability. Comput Chem Eng 16(9):819–835
Lu T, Viljanen M (2009) Prediction of indoor temperature and relative humidity using neural network models: model comparison. Neural Comput Appl 18:345–357
Mackay D (1995) Probable networks and plausible predictions—a review of practical bayesian methods for supervised neural networks. Netw Comput Neural Syst 6(3):469–505
Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1(2):281–294
Neto AR, Roisenberg M, Neto GS (2010) Efficient confidence bounds for rbf networks for sparse and high dimensional data. In: Proceedings of the 20th international conference on artificial neural networks: part III, Springer, Berlin, ICANN’10, pp 423–428
Olatunji SO, Selamat A, Abdulraheem A (2011) Modeling the permeability of carbonate reservoir using type-2 fuzzy logic systems. Comput Ind 62(2):147–163
Pacheco-Vega A, Sen M, Yang K, McClain RL (2001) Neural network analysis of fin-tube refrigerating heat exchanger with limited experimental data. Int J Heat Mass Transf 44(4):763–770
Pierce SG, Worden K, Bezazi A (2008) Uncertainty analysis of a neural network used for fatigue lifetime prediction. Mech Syst Signal Process 22(6):1395–1411
Prechelt L (1994) Proben1—a set of neural network benchmark problems and benchmarking rules. Tech. rep., Fakultat fur Informatik Universitat Karlsruhe
Shao R, Martin E, Zhang J, Morris A (1997) Confidence bounds for neural network representations. Comput Chem Eng 21(Supplement(0)):S1173–S1178
Terrell GR, Scott DW (1992) Variable kernel density estimation. Ann Stat 20(3):1236–1265
Wedding DK II, Cios KJ (1996) Time series forecasting by combining rbf networks, certainty factors, and the box-jenkins model. Neurocomputing 10(2):149–168
Wedding DK II, Cios KJ (1998) Certainty factors versus parzen windows as reliability measures in rbf networks. Neurocomputing 19(1):151–165
Wong P, Bruce A, Gedeon T (2002) Confidence bounds of petrophysical predictions from conventional neural networks. Geosci Remote Sens IEEE Trans 40(6):1440–1444
Yang L, Kavli T, Carlin M, Clausen S, de Groot PFM (2002) An evaluation of confidence bound estimation methods for neural networks. In: Advances in computational intelligence and learning: methods and applications, Kluwer, B.V., Deventer, The Netherlands, pp 71–84
Yu L, Huang W, Lai KK, Wang S (2006) A reliability-based rbf network ensemble model for foreign exchange rates predication. In: Proceedings of the 13th international conference on neural information processing, volume part III, Springer, Berlin, ICONIP’06, pp 380–389
Zio E (2006) A study of the bootstrap method for estimating the accuracy of artificial neural networks in predicting nuclear transient processes. Nucl Sci IEEE Trans 53(3):1460–1478
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was partially granted by Petrobras S.A. as project: Intelligent Techniques for Reservoir Characterization—SICRES.
Rights and permissions
About this article
Cite this article
Rodrigues Neto, A.C., das Neves, C.A.M. & Roisenberg, M. Comparative study on local and global strategies for confidence estimation in neural networks and extensions to improve their predictive power. Neural Comput & Applic 22, 1519–1530 (2013). https://doi.org/10.1007/s00521-012-1051-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-012-1051-x