Abstract
This paper describes a unified approach, based on Gaussian Processes, for achieving sensor fusion under the problematic conditions of missing channels and noisy labels. Under the proposed approach, Gaussian Processes generate separate class labels corresponding to each individual modality. The final classification is based upon a hidden random variable, which probabilistically combines the sensors. Given both labeled and test data, the inference on unknown variables, parameters and class labels for the test data is performed using the variational bound and Expectation Propagation. We apply this method to the challenge of classifying a student’s interest level using observations from the face and postures, together with information from the task the students are performing. Classification with the proposed new approach achieves accuracy of over 83%, significantly outperforming the classification using individual modalities and other common classifier combination schemes.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Beal, M.J.: Variational Algorithms for Approximate Bayesian Inference, Ph.D. Thesis, University College London (2003)
Breiman, L.: Bagging Predictors. Machine Learning 26(2) (1996)
Han, J., Bhanu, B.: Statistical Feature Fusion for Gait-based Human Recognition. In: CVPR (2004)
Hong, L., Jain, A.K.: Integrating Faces and Fingerprints for Personal Identification. In: PAMI, vol. 20(1) (1998)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive Mixtures of Local Experts. Neural Computation 3, 79–87 (1991)
Kapoor, A., Picard, R.W., Ivanov, Y.: Probabilistic Combination of Classifiers to Detect Interest. In: ICPR (2004)
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On Combining Classifiers. In: PAMI, vol. 20(3) (1998)
Miller, D.J., Yan, L.: Critic-Driven Ensemble Classification. Signal Processing 47(10) (1999)
Minka, T.: Expectation Propagation for Approximate Bayesian Inference. In: UAI (2001)
Oliver, N., Garg, A., Horvitz, E.: Layered Representations for Learning and Inferring Office Activity from Multiple Sensory Channels. In: ICMI (2002)
Opper, M., Winther, O.: Mean field methods for Classification with Gaussian Processes. In: NIPS, vol. 11 (1999)
Schapire, R.: A Brief Introduction to Boosting. In: International Conference on Algorithmic Learning Theory (1999)
Toyama, K., Horvitz, E.: Bayesian Modality Fusion: Probabilistic Integration of Multiple Vision Algorithms for Head Tracking. In: ACCV (2000)
Tresp, V.: Mixture of Gaussian Processes. In: NIPS, vol. 13 (2001)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Scholkopf, B.: Learning with Local and Global Consistency. In: NIPS, vol. 16 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kapoor, A., Ahn, H., Picard, R.W. (2005). Mixture of Gaussian Processes for Combining Multiple Modalities. In: Oza, N.C., Polikar, R., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2005. Lecture Notes in Computer Science, vol 3541. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11494683_9
Download citation
DOI: https://doi.org/10.1007/11494683_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26306-7
Online ISBN: 978-3-540-31578-0
eBook Packages: Computer ScienceComputer Science (R0)