Abstract
In this paper, we present a maximum entropy (maxent) approach to the fusion of experts opinions, or classifiers outputs, problem. The maxent approach is quite versatile and allows us to express in a clear, rigorous, way the a priori knowledge that is available on the problem. For instance, our knowledge about the reliability of the experts and the correlations between these experts can be easily integrated: Each piece of knowledge is expressed in the form of a linear constraint. An iterative scaling algorithm is used in order to compute the maxent solution of the problem. The maximum entropy method seeks the joint probability density of a set of random variables that has maximum entropy while satisfying the constraints. It is therefore the ”most honest” characterization of our knowledge given the available facts (constraints). In the case of conflicting constraints, we propose to minimise the ”lack of constraints satisfaction” or to relax some constraints and recompute the maximum entropy solution. The maxent fusion rule is illustrated by some simulations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chen, D., Cheng, X.: An asymptotic analysis of some expert fusion methods. Pattern Recognition Letters 22, 901–904 (2001)
Cooke, R.M.: Experts in uncertainty. Oxford University Press, Oxford (1991)
Dubois, D., Grabisch, M., Prade, H., Smets, P.: Assessing the value of a candidate: Comparing belief function and possibility theories. In: Proceedings of the Fifteenth international conference on Uncertainty in Artificial Intelligence, pp. 170–177 (1999)
Fang, S.-C., Rajasekera, J., Tsao, H.-S.J.: Entropy optimization and mathematical programming. Kluwer Academic Publishers, Dordrecht (1997)
Fouss, F., Saerens, M.: A maximum entropy approach to multiple classifiers combination. Technical Report, IAG, Universite catholique de Louvain (2004)
Genest, C., Zidek, J.V.: Combining probability distributions: A critique and an annotated bibliography. Statistical Science 36, 114–148 (1986)
Golan, A., Judge, G., Miller, D.: Maximum entropy econometrics: Robust estimation with limited data. John Wiley and Sons, Chichester (1996)
Jacobs, R.A.: Methods for combining experts’ probability assessments. Neural Computation 7, 867–888 (1995)
Jelinek, F.: Statistical methods for speech recognition. The MIT Press, Cambridge (1997)
Kapur, J.N., Kesavan, H.K.: The generalized maximum entropy principle (with applications). Standford Educational Press, Standford (1987)
Kapur, J.N., Kesavan, H.K.: Entropy optimization principles with applications. Academic Press, London (1992)
Kittler, J., Alkoot, F.: Sum versus vote fusion in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(1), 110–115 (2003)
Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)
Klir, G.J., Folger, T.A.: Fuzzy sets, uncertainty, and information. Prentice-Hall, Englewood Cliffs (1988)
Ku, H., Kullback, S.: Approximating discrete probability distributions. IEEE Transactions on Information Theory 15(4), 444–447 (1969)
Lad, F.: Operational subjective statistical methods. John Wiley and Sons, Chichester (1996)
Levy, W.B., Delic, H.: Maximum entropy aggregation of individual opinions. IEEE Transactions on Systems, Man and Cybernetics 24(4), 606–613 (1994)
Merz, C.: Using correspondence analysis to combine classifiers. Machine Learning 36, 226–239 (1999)
Myung, J., Ramamoorti, S., Andrew, J., Bailey, D.: Maximum entropy aggregation of expert predictions. Management Science 42(10), 1420–1436 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Saerens, M., Fouss, F. (2004). Yet Another Method for Combining Classifiers Outputs: A Maximum Entropy Approach. In: Roli, F., Kittler, J., Windeatt, T. (eds) Multiple Classifier Systems. MCS 2004. Lecture Notes in Computer Science, vol 3077. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-25966-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-25966-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22144-9
Online ISBN: 978-3-540-25966-4
eBook Packages: Springer Book Archive