Abstract
In this paper an investigation of the distribution of the weights and the biases of the Multilayered Perceptron is conducted, in particular the variance of the weight vector (weights and biases) with the aim of indicating the existence of the structural diversity within the ensemble. This will indicate how well the weight vector samples are distributed from the mean and this will be used to serve as an indicator of the structural diversity of the classifiers within the ensemble. This is inspired by the fact that many measures of ensemble diversity are focused on the outcomes and not the classifier’s structure and hence may lose out in diversity measures that correlate well with ensemble performance. Three ensembles were compared, one non-diverse and the other two ensembles made diverse. The generalization across all the ensembles was approximately the same (74 % accuracy). This could be attributed to the data used. Certainty measures were also conducted and indicated that the non-diverse ensemble was biased, even though the performance across the ensembles was the same.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Igelnik, B., Pao, Y., Leclair, S., Shen, C.: The ensemble approach to neural-network learning and generalization. IEEE transactions on neural networks 10(1), 19–30 (1999)
Islam, M.M., Yao, X., Nirjon, S.M.S., Islam, M.A., Murase, K.: Bagging and boosting negatively correlated neural networks. IEEE transactions on systems, man, and cybernetics. Part B 38, 771–784 (2008)
Kuncheva, L.I., Skurichina, M.: An experimental study on diversity for bagging and boosting with linear classifiers. Information Fusion 3, 245–258 (2002)
Polikar, R.: Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 6(3), 21–45 (2006)
Shipp, C.A., Kuncheva, L.I.: Relationships between combination methods and measures of diversity in combining classifiers. Information Fusion 3(2), 135–148 (2002)
Lam, L.: Classifier Combinations: Implementations and Theoretical Issues. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 78–86. Springer, Heidelberg (2000)
Gal-Or, M., May, J.H., Spangler, W.E.: Assessing the predictive accuracy of diversity measures with domain-dependent asymmetric misclassification costs. Information Fusion 6(1), 37–48 (2005)
Brown, G.: Diversity in neural network ensembles. Ph.D. thesis, School of Computer Science, University of Birmingham (2004)
Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles. Machine Learning 51, 181–207 (2003)
Tumer, K., Ghosh, J.: Error correlation and reduction in ensemble classifiers. Connection Science 8, 385–404 (1996)
Masisi, L., Nelwamondo, F.V., Marwala, T.: The effect of structural diversity of an ensemble of classifiers on classification accuracy. In: IASTED International Conference on Modelling and Simulation, Africa-MS (2008)
Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1992)
Cantu-Paz, E., Kamath, C.: An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. Systems, Man, and Cybernetics-Part B: Cybernetics 35, 915–927 (2005)
Izrailev, S., Agrafiotis, D.K.: A method for quantifying and visualizing the diversity of qsar models. Journal of Molecular Graphics and Modelling 22, 275–284 (2004)
Jimenez, D.A., Walsh, N.: Dynamically weighted ensemble neural networks for classification. In: Proceedings of the 1998 International Joint Conference on Neural Networks (1998)
Marwala, T., Lagazio, M.: Modeling and controlling interstate conflict. In: IEEE International Joint Conference on Neural Networks, Budapest, Hungary, pp. 1233–1238 (2004)
Kuncheva, L.I., Whitaker, C.J., Shipp, C.A., Duin, R.P.W.: Is independence good for combining classiffiers? In: Proceedings of 15th International Conference on Parttern Recognition, vol. 2, pp. 168–171 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Masisi, L.M., Nelwamondo, F., Marwala, T. (2009). Investigating Ensemble Weight and the Certainty Distributions for Indicating Structural Diversity. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03040-6_63
Download citation
DOI: https://doi.org/10.1007/978-3-642-03040-6_63
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03039-0
Online ISBN: 978-3-642-03040-6
eBook Packages: Computer ScienceComputer Science (R0)