Abstract
To improve estimation results, outputs of multiple neural networks can be aggregated into a committee output. In this paper, we study the usefulness of the leverages based information for creating accurate neural network committees. Based on the approximate leave-one-out error and the suggested, generalization error based, diversity test, accurate and diverse networks are selected and fused into a committee using data dependent aggregation weights. Four data dependent aggregation schemes – based on local variance, covariance, Choquet integral, and the generalized Choquet integral – are investigated. The effectiveness of the approaches is tested on one artificial and three real world data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Taniguchi, M., Tresp, V.: Averaging regularized estimators. Neural Computation 9, 1163–1178 (1997)
Verikas, A., Lipnickas, A., Malmqvist, K., Bacauskiene, M., Gelzinis, A.: Soft combination of neural classifiers: A comparative study. Pattern Recognition Letters 20, 429–444 (1999)
Gader, P.D., Mohamed, M.A., Keller, J.M.: Fusion of handwritten word classifiers. Pattern Recognition Letters 17, 577–584 (1996)
Verikas, A., Lipnickas, A.: Fusing neural networks through space partitioning and fuzzy integration. Neural Processing Letters 16, 53–65 (2002)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans Pattern Analysis and Machine Intelligence 20, 226–239 (1998)
Kim, S.P., Sanchez, J.C., Erdogmus, D., Rao, Y.N., Wessberg, J., Principe, J.C., Nicolelis, M.: Divide-and-conquer approach for brain machine interfaces: nonlinear mixture of competitive linear models. Neural Networks 16, 865–871 (2003)
Woods, K., Kegelmeyer, W.P., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans Pattern Analysis Machine Intelligence 19, 405–410 (1997)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 231–238. MIT Press, Cambridge (1995)
Breiman, L.: Bagging predictors. Technical Report 421, Statistics Departament, University of California, Berkeley (1994)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)
Liu, Y., Yao, X.: Ensemble learning via negatine correlation. Neural Networks 12, 1399–1404 (1999)
Monari, G., Dreyfus, G.: Local overfitting control via leverages. Neural Computation 14, 1481–1506 (2002)
Tresp, V.: A Bayesian committee machine. Neural Computation 12, 2719–2741 (2000)
Yager, R.R.: Generalized OWA aggregation operators. Fuzzy Optimization and Decision Making 3, 93–107 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Verikas, A., Bacauskiene, M., Gelzinis, A. (2004). Leverages Based Neural Networks Fusion. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds) Neural Information Processing. ICONIP 2004. Lecture Notes in Computer Science, vol 3316. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30499-9_68
Download citation
DOI: https://doi.org/10.1007/978-3-540-30499-9_68
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23931-4
Online ISBN: 978-3-540-30499-9
eBook Packages: Springer Book Archive