Abstract
We consider a general scheme of parallel classifier combinations in the framework of statistical pattern recognition. Each statistical classifier defines a set of output variables in terms of a posteriori probabilities, i.e. it is used as a feature extractor. Unlike usual combining schemes the output vectors of classifiers are combined in parallel. The statistical Shannon information is used as a criterion to compare different combining schemes from the point of view of the theoretically available decision information. By means of relatively simple arguments we derive a theoretical hierarchy between different schemes of classifier fusion in terms of information inequalities.
Supported by the grant No. 402/01/0981 of the Czech Grant Agency and partially by the Complex research project No. K1019101 of the Czech Academy of Sciences.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Fukunaga K., Ando S.: The optimum nonlinear features for a scatter criterion and discriminant analysis. In Proc. ICPR 1976, pp 50–54.
Grim J.: Design of multilayer neural networks by information preserving transforms. In E. Pessa M.P. Penna A. Montesanto (Eds.), Proceedings of the Third European Congress on System Science, Roma: Edizzioni Kappa, 1996, pp 977–982.
Grim J., Kittler J., Pudil P., Somol P.: Combining multiple classifiers in probabilistic neural networks. In Multiple Classifier Systems, Kittler J., Roli F., (Eds.), Springer, 2000, pp 157–166.
Grim J., Pudil P.: On virtually binary nature of probabilistic neural networks. In: Advances in Pattern Recognition, Sydney, August 11–13, 1998), A. Amin, D. Dori, P. Pudil, H. Freeman (Eds.), pp. 765–774, Springer: New York, Berlin, 1998.
Grim J., Pudil P., Somol P.: Recognition of handwritten numerals by structural probabilistic neural networks. In Proceedings of the Second ICSC Symposium on Neural Computation, Berlin, 2000. (Bothe H., Rojas R. eds.). ICSC, Wetaskiwin, 2000, pp 528–534.
Hansen L.K., Salamon P.: Neural network ensembles, IEEE Transactions on Pattern Analysis and Machine Intelligence 1990; 12(10):993–1001.
Huang Y.S., Suen C.Y.: A method of combining multiple classifiers–a neural network approach. In Proc. 12th IAPR Int. Conf. on Pattern Recognition and Computer Vision, Los Alamitos: IEEE Comp. Soc. Press., 1994, pp 473–475.
Huang Y.S., Suen C.Y.: Combination of multiple experts for the recognition of unconstrained handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995; 17(1):90–94.
Cho S.B., Kim J.H.: Combining multiple neural networks by fuzzy integral for robust classification, IEEE Transactions on Systems Man and Cybernetics 1995; 25(2):380–384.
Cho S.B., Kim J.H.: Multiple network fusion using fuzzy logic, IEEE Transactions on Neural Networks 1995; 6(2):497–501.
Chuanyi Ji, Sheng Ma.: Combinations of weak classifiers. IEEE Transactions on Neural Networks, 8(1): 32–42.
Kittler J.: Combining classifiers: A theoretical framework. Pattern Analysis and Applications 1998; 1:18–27.
Kittler J., Duin R.P.W., Hatef M., Matas J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998; 20(3):226–239.
Lam L., Suen C.Y.: A theoretical analysis of the application of majority voting to pattern recognition. In Proc. 12th IAPR Int. Conf. on Pattern Recognition and Computer Vision, Los Alamitos: IEEE Comp. Soc. Press., 1994, pp 418–420.
Rogova G.: Combining the results of several neural network classifiers, Neural Networks 1994; 7(5):777–781.
Soulie F.F., Vinnet E., Lamy B.: Multi-Modular Neural Network Architectures: Applications in Optical Character and Human Face Recognition, Int. Journal of Pattern Recognition and Artificial Intelligence, Vol. 5,No. 4, pp 721–755, 1993.
Vajda I., Grim J.: About the maximum information and maximum likelihood principles in neural networks. Kybernetika 1998; 34(4):485–494.
Wolpert D.H.: Stacked generalization. Neural Networks, Vol. 5,No. 2, pp 241–260, 1992.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Grim, J., Kittler, J., Pudil, P., Somol, P. (2001). Information Analysis of Multiple Classifier Fusion?. In: Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2001. Lecture Notes in Computer Science, vol 2096. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48219-9_17
Download citation
DOI: https://doi.org/10.1007/3-540-48219-9_17
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42284-6
Online ISBN: 978-3-540-48219-2
eBook Packages: Springer Book Archive