Abstract
We propose an algorithm for Sparse Bayesian Classification for multi-class problems using Automatic Relevance Determination(ARD). Unlike other approaches which treat multiclass problem as multiple independent binary classification problem, we propose a method to learn the multiclass predictor directly. The usual approach of “one against rest” and “pairwise coupling” are not only computationally demanding during training stage but also generates dense classifiers which have greater tendency to overfit and have higher classification cost. In this paper we discuss the algorithmic implementation of Multiclass Classification model and compare it with other multi-class classifiers. We also empirically evaluate the classifier on viewpoint learning problem using features extracted from human silhouettes. Our experiments show that our algorithm generates sparser classifiers, with performance comparable to state-of-the-art multi-class classifier.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. The Annals of Statistics (1998)
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research (1995)
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. In: ICML (2000)
Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. Computational Learning Theory (1992)
Breinman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks (1984)
Bredensteiner, E.J., Bennet, K.P.: Multicategory classification using Support Vector Machines. Computational Optimization and Applications (1999)
Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support Vector Learning for Interdependent and Structured Output Spaces. In: ICML (2004)
Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine. In: JMLR (2001)
Nabney, I.T.: Efficient Training of RBF Networks for Classification. International Journal of Neural Systems (2004)
Mackay, D.J.C.: Bayesian Interpolation. Neural Computation (1991)
Crammer, K., Singer, Y.: On Algorithmic Implementation of Multiclass Kernel-bases Vector Machines. JMLR (2001)
Hamamura, T., Mizutani, H., Irie, B.: A Multiclass classification method based on multiple pairwise classifiers. In: ICDAR (2003)
CMU Human Motion Capture Database, http://mocap.cs.cmu.edu
MacKay, D.J.C.: Choice of basis for Laplace approximation. Machine Learning (1998)
Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. PAMI 2002 (2002)
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences (August 1997)
Burges, C.: A tutorial on support vector machines for pattern recognition. KDD (1998)
Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: European Symposium on ANN (April 1999)
Neal, R.M. (ed.): Bayesian Learning for Neural Networks. Springer, Heidelberg (1996)
Berger, J.O.: Statistical decision theory and Bayesian analysis. Springer, Heidelberg (1985)
Nocedal, J.: Updating quasi-Newton matrices with limited storage. Mathematics of Computation (1980)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kanaujia, A., Metaxas, D. (2006). Learning Multi-category Classification in Bayesian Framework. In: Narayanan, P.J., Nayar, S.K., Shum, HY. (eds) Computer Vision – ACCV 2006. ACCV 2006. Lecture Notes in Computer Science, vol 3851. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11612032_27
Download citation
DOI: https://doi.org/10.1007/11612032_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-31219-2
Online ISBN: 978-3-540-32433-1
eBook Packages: Computer ScienceComputer Science (R0)