Abstract
The Gaussian mixture model (GMM) has been widely used in pattern recognition problems for clustering and probability density estimation. For pattern classification, however, the GMM has to consider two issues: model structure in high-dimensional space and discriminative training for optimizing the decision boundary. In this paper, we propose a classification method using subspace GMM density model and discriminative training. During discriminative training under the minimum classification error (MCE) criterion, both the GMM parameters and the subspace parameters are optimized discriminatively. Our experimental results on the MNIST handwritten digit data and UCI datasets demonstrate the superior classification performance of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum Likelihood from Incomplete Data Via the EM Algorithm. J. Royal Statistical Soc. Ser. B 39(1), 1–38 (1977)
Figueiredo, M.A.T., Jain, A.K.: Unsupervised Learning of Finite Mixture Models. IEEE Trans. Pattern Analysis and Machine Intelligence 24(3), 381–396 (2002)
Gales, M.J.F.: Semi-tied Covariance Matrices for Hidden Markov Models. IEEE Trans. Speech and Audio Processing 7(3), 272–281 (1999)
Moghaddam, B., Pentland, A.: Probabilistic Visual Learning for Object Representation. IEEE Trans. Pattern Analysis and Machine Intelligence 19(7), 696–710 (1997)
Liu, X.-H., Liu, C.-L., Hou, X.-W.: A Pooled Subspace Mixture Density Model for Pattern Classification in High-dimensional Spaces. In: Proc. IJCNN 2008, Hong Kong, pp. 2467–2472 (2008)
Bahl, L., Brown, P., de Souza, P., Mercer, R.: Maximum Mutual Information Estimation of Hidden Markov Model Parameters for Speech Recognition. In: Proc. Int’l. Conf. on Acoustics, Speech and Signal Processing, Tokyo, vol. 1, pp. 49–52 (1986)
Juang, B.-H., Chou, W., Lee, C.-H.: Minimum Classification Error Rate Methods for Speech Recognition. IEEE Trans. Speech Audio Process. 5(3), 257–265 (1997)
Dahmen, J., Schluter, R., Ney, H.: Discriminative Training of Gaussian Mixtures for Image Object Recognition. In: Proc. 21st Symposium of German Association for Pattern Recognition, Bonn, Germany, pp. 205–212 (1999)
Zhou, X., Wang, X.: Optimisation of Gaussian Mixture Model for Satellite Image Classification. IEE Proc.-Vision, Image and Signal Processing 153(3), 349–356 (2006)
Zhang, R., Ding, X.: Offline Handwritten Numeral Recognition Using Orthogonal Gaussian Mixture Model. In: Proc. 6th Int. Conf. Document Analysis and Recognition, pp. 1126–1129 (2001)
Schluter, R., Macherey, W., Muller, B., Ney, H.: Comparison of Discriminative Training Criteria and Optimization Methods for Speech Recognition. Speech Communication 34(1), 287–310 (2001)
Li, Q., Juang, B.-H.: A New Algorithm for Fast Discriminative Training. In: Proc. ICASSP 2002, Orlando, vol. 1, pp. 97–100 (2002)
Moghaddam, B., Jebara, T., Pentland, A.: Bayesian Face Recognition. Pattern Recognition 33(11), 1771–1782 (2000)
Chen, X., Liu, X., Jia, Y.: Unsupervised Selection and Discriminative Estimation of Orthogonal Gaussian Mixture Models for Handwritten Digit Recognition. In: Proc. 10th ICDAR, Barcelona, Spain, pp. 1151–1155 (2009)
Liu, C.-L., Sako, H., Fujisawa, H.: Discriminative Learning Quadratic Discriminant Function for Handwriting Recognition. IEEE Trans. Neural Networks 15(2), 430–444 (2004)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based Learning Applied to Document Recognition. Proc. IEEE 86(11), 2278–2324 (1998)
UCI Machine Learning Repository, http://archive.ics.uci.edu/ml/
Watanabe, H., Katagiri, S.: Subspace Method for Minimum Error Pattern Recognition. IEICE Trans. Information Syst. E80-D(12), 1095–1104 (1997)
Liu, C.-L., Nakashima, K., Sako, H., Fujisawa, H.: Handwritten Digit Recognition: benchmarking of state-of-art techniques. Pattern Recognition 36, 2271–2285 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, XH., Liu, CL. (2010). Discriminative Training of Subspace Gaussian Mixture Model for Pattern Classification. In: Huang, DS., Zhao, Z., Bevilacqua, V., Figueroa, J.C. (eds) Advanced Intelligent Computing Theories and Applications. ICIC 2010. Lecture Notes in Computer Science, vol 6215. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14922-1_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-14922-1_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14921-4
Online ISBN: 978-3-642-14922-1
eBook Packages: Computer ScienceComputer Science (R0)