Skip to main content

Learning Multi-category Classification in Bayesian Framework

  • Conference paper
Computer Vision – ACCV 2006 (ACCV 2006)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 3851))

Included in the following conference series:

Abstract

We propose an algorithm for Sparse Bayesian Classification for multi-class problems using Automatic Relevance Determination(ARD). Unlike other approaches which treat multiclass problem as multiple independent binary classification problem, we propose a method to learn the multiclass predictor directly. The usual approach of “one against rest” and “pairwise coupling” are not only computationally demanding during training stage but also generates dense classifiers which have greater tendency to overfit and have higher classification cost. In this paper we discuss the algorithmic implementation of Multiclass Classification model and compare it with other multi-class classifiers. We also empirically evaluate the classifier on viewpoint learning problem using features extracted from human silhouettes. Our experiments show that our algorithm generates sparser classifiers, with performance comparable to state-of-the-art multi-class classifier.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Hastie, T., Tibshirani, R.: Classification by pairwise coupling. The Annals of Statistics (1998)

    Google Scholar 

  2. Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research (1995)

    Google Scholar 

  3. Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. In: ICML (2000)

    Google Scholar 

  4. Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. Computational Learning Theory (1992)

    Google Scholar 

  5. Breinman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks (1984)

    Google Scholar 

  6. Bredensteiner, E.J., Bennet, K.P.: Multicategory classification using Support Vector Machines. Computational Optimization and Applications (1999)

    Google Scholar 

  7. Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support Vector Learning for Interdependent and Structured Output Spaces. In: ICML (2004)

    Google Scholar 

  8. Tipping, M.E.: Sparse Bayesian Learning and the Relevance Vector Machine. In: JMLR (2001)

    Google Scholar 

  9. Nabney, I.T.: Efficient Training of RBF Networks for Classification. International Journal of Neural Systems (2004)

    Google Scholar 

  10. Mackay, D.J.C.: Bayesian Interpolation. Neural Computation (1991)

    Google Scholar 

  11. Crammer, K., Singer, Y.: On Algorithmic Implementation of Multiclass Kernel-bases Vector Machines. JMLR (2001)

    Google Scholar 

  12. Hamamura, T., Mizutani, H., Irie, B.: A Multiclass classification method based on multiple pairwise classifiers. In: ICDAR (2003)

    Google Scholar 

  13. CMU Human Motion Capture Database, http://mocap.cs.cmu.edu

  14. MacKay, D.J.C.: Choice of basis for Laplace approximation. Machine Learning (1998)

    Google Scholar 

  15. Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. PAMI 2002 (2002)

    Google Scholar 

  16. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences (August 1997)

    Google Scholar 

  17. Burges, C.: A tutorial on support vector machines for pattern recognition. KDD (1998)

    Google Scholar 

  18. Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: European Symposium on ANN (April 1999)

    Google Scholar 

  19. Neal, R.M. (ed.): Bayesian Learning for Neural Networks. Springer, Heidelberg (1996)

    MATH  Google Scholar 

  20. Berger, J.O.: Statistical decision theory and Bayesian analysis. Springer, Heidelberg (1985)

    MATH  Google Scholar 

  21. Nocedal, J.: Updating quasi-Newton matrices with limited storage. Mathematics of Computation (1980)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kanaujia, A., Metaxas, D. (2006). Learning Multi-category Classification in Bayesian Framework. In: Narayanan, P.J., Nayar, S.K., Shum, HY. (eds) Computer Vision – ACCV 2006. ACCV 2006. Lecture Notes in Computer Science, vol 3851. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11612032_27

Download citation

  • DOI: https://doi.org/10.1007/11612032_27

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-31219-2

  • Online ISBN: 978-3-540-32433-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics