Skip to main content
SpringerLink
Log in

A New HMM-Based Ensemble Generation Method for Numeral Recognition

  • Conference paper
Book cover Multiple Classifier Systems (MCS 2007)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 4472))

Included in the following conference series:

Abstract

A new scheme for the optimization of codebook sizes for HMMs and the generation of HMM ensembles is proposed in this paper. In a discrete HMM, the vector quantization procedure and the generated codebook are associated with performance degradation. By using a selected clustering validity index, we show that the optimization of HMM codebook size can be selected without training HMM classifiers. Moreover, the proposed scheme yields multiple optimized HMM classifiers, and each individual HMM is based on a different codebook size. By using these to construct an ensemble of HMM classifiers, this scheme can compensate for the degradation of a discrete HMM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arica, N., Vural, F.T.Y.: A shape descriptor based on circular Hidden Markov Model. In: 15th Intl. Conf. on Pattern Recognition (ICPR00) (2000)

    Google Scholar 

  2. Bandyopadhyay, S., Maulik, U.: Non-parametric Genetic Clustering: Comparison of Validity Indices. IEEE Transactions on Systems, Man and Cybernetics Part-C 31(1), 120–125 (2001)

    Article  Google Scholar 

  3. Bengio, Y.: Markovian Models for Sequential Data. Neural Computing Surveys 2, 129–162 (1999)

    Google Scholar 

  4. Britto Jr., A.: A Two-Stage HMM-Based Method for Recognizing Handwritten Numeral Strings. Ph.D. Thesis, Pontifical Catholic University of Paraná (2001)

    Google Scholar 

  5. Britto, A.S., et al.: Recognition of Handwritten Numeral Strings Using a Two-Stage Hmm-Based Method. International Journal on Document Analysis and Recognition (IJDAR) 5(2-3), 102–117 (2003)

    Article  Google Scholar 

  6. Dietterich, T.G.: Machine Learning for Sequential Data: A Review. In: Caelli, T.M., et al. (eds.) SPR 2002 and SSPR 2002. LNCS, vol. 2396, pp. 15–30. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  7. Eppstein, D.: Fast hierarchical clustering and other applications of dynamic closest pairs. In: Proceedings of the Ninth ACM-SIAM Symposium on Discrete Algorithms, pp. 619–628 (1998)

    Google Scholar 

  8. Guenter, S., Bunke, H.: Off-line cursive handwriting recognition using multiple classifier systems - on the influence of vocabulary, ensemble, and training set size. Optics and Lasers in Engineering 43, 437–454 (2005)

    Article  Google Scholar 

  9. Guenter, S., Bunke, H.: Ensembles of classifiers derived from multiple prototypes and their application to handwriting recognition. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 314–323. Springer, Heidelberg (2004)

    Google Scholar 

  10. Guenter, S., Bunke, H.: Off-line Cursive Handwriting Recognition - On the Influence of Training Set and Vocabulary Size in Multiple Classifier Systems. In: Proceedings of the 11th Conference of the International Graphonomics Society (2003)

    Google Scholar 

  11. Guenter, S., Bunke, H.: A new combination scheme for HMM-based classifiers and its application to handwriting recognition. In: Proc. 16th Int. Conference on Pattern Recognition, vol. II, pp. 332–337 (2002)

    Google Scholar 

  12. Bunke, H., Günter, S.: Generating Classifier Ensembles from Multiple Prototypes and Its Application to Handwriting Recognition. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, pp. 179–188. Springer, Heidelberg (2002)

    Google Scholar 

  13. Guenter, S., Bunke, H.: Creation of classifier ensembles for handwritten word recognition using feature selection algorithms. In: Proc. 8th Int. Workshop on Frontiers in Handwriting Recognition, pp. 183–188 (2002)

    Google Scholar 

  14. Guenter, S., Bunke, H.: Ensembles of Classifiers for Handwritten Word Recognition. International Journal of Document Analysis and Recognition 5(4), 224–232 (2003)

    Article  Google Scholar 

  15. Guenter, S., Bunke, H.: New Boosting Algorithms for Classification Problems with Large Number of Classes Applied to a Handwritten Word Recognition Task. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 326–335. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  16. Bunke, H., Günter, S.: Fast Feature Selection in an HMM-Based Multiple Classifier System for Handwriting Recognition. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 289–296. Springer, Heidelberg (2003)

    Google Scholar 

  17. Guenter, S., Bunke, H.: Optimization of weights in a multiple classifier handwritten word recognition system using a genetic algorithm. Electronic Letters of Computer Vision and Image Analysis 3(1), 25–44 (2004)

    Google Scholar 

  18. Halkidi, M., Batistakis, Y., Vazirgiannis, M.: On Clustering Validation Techniques. Journal of Intelligent Information Systems 17(2-3) (2001)

    Google Scholar 

  19. Halkidi, M., Batistakis, Y., Vazirgiannis, M.: Clustering Validity Checking Methods: Part II. SIGMOD Record 31(3), 19–27 (2002)

    Article  Google Scholar 

  20. Huang, X., Acero, A., Hon, H.: Spoken Language Processing - a guide to theory, algorithm, and system development. Prentice Hall PTR, Englewood Cliffs (2001)

    Google Scholar 

  21. Kargupta, H., Johnson, E.L.: Collective, Hierarchical Clustering from Distributed, Heterogeneous Data. In: Zaki, M.J., Ho, C.-T. (eds.) KDD 1999. LNCS (LNAI), vol. 1759, pp. 221–244. Springer, Heidelberg (2000)

    Google Scholar 

  22. Kittler, J., et al.: On Combining Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)

    Article  Google Scholar 

  23. Ko, A., Sabourin, R., Britto, J.A.: Combining Diversity and Classification Accuracy for Ensemble Selection in Random Subspaces. In: IEEE World Congress on Computational Intelligence (WCCI 2006) - International Joint Conference on Neural Networks (IJCNN 2006) (2006)

    Google Scholar 

  24. Ko, A., Sabourin, R., Britto, J.A.: Evolving Ensemble of Classifiers in Random Subspace. In: Genetic and Evolutionary Computation Conference (GECCO 2006) (2006)

    Google Scholar 

  25. Kuncheva, L.I., Skurichina, M., Duin, R.P.W.: An Experimental Study on Diversity for Bagging and Boosting with Linear Classifiers. International Journal of Information Fusion 3(2), 245–258 (2002)

    Article  Google Scholar 

  26. Maulik, U., Bandyopadhyay, S.: Performance Evaluation of Some Clustering Algorithms and Validity Indices. IEEE Trans. Pattern Anal. Mach. Intell 24(12), 1650–1654 (2002)

    Article  Google Scholar 

  27. Pakhira, M.K., Bandyopadhyay, S., Maulik, U.: Validity index for crisp and fuzzy clusters. Pattern Recognition 37(3), 487–501 (2004)

    Article  MATH  Google Scholar 

  28. Pekalska, E., Skurichina, M., Duin, R.P.W.: Combining Dissimilarity-Based One-Class Classifiers. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 122–133. Springer, Heidelberg (2004)

    Google Scholar 

  29. Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  30. Rabiner, L.R., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall, Englewood Cliffs (1993)

    Google Scholar 

  31. Ruta, D., Gabrys, B.: Classifier Selection for Majority Voting. International Journal of Information Fusion, 63–81 (2005)

    Google Scholar 

  32. Seo, J., Shneiderman, B.: Interactively Exploring Hierarchical Clustering Results. IEEE Computer 35(7), 80–86 (2002)

    Google Scholar 

  33. Shipp, C.A., Kuncheva, L.I.: Relationships between combination methods and measures of diversity in combining classifiers. International Journal of Information Fusion 3(2), 135–148 (2002)

    Article  Google Scholar 

  34. Smyth, P., Heckerman, D., Jordan, M.I.: Probabilistic independence networks for hidden Markov probability models. Neural Computation 9, 227–269 (1997)

    Article  MATH  Google Scholar 

  35. Wang, X.: Durationally constrained training of HMM without explicit state durational. Proceedings of the Institute of Phonetic Sciences 18, 111–130 (1994)

    Google Scholar 

  36. Xie, X.L., Beni, G.: A validity measure for fuzzy clustering. IEEE Transactions of Pattern Analysis and Machine Intelligence, 841–847 (1991)

    Google Scholar 

  37. Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)

    Article  Google Scholar 

  38. Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LIVIA, ETS, University of Quebec, 1100 Notre-Dame West Street, Montreal, Quebec, H3C 1K3 Canada, PPGIA, Pontifical Catholic University of Parana, Rua Imaculada Conceicao, 1155, PR 80215-901, Curitiba, Brazil

    Albert Hung-Ren Ko, Robert Sabourin & Alceu de Souza Britto Jr.

Authors
  1. Albert Hung-Ren Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Sabourin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alceu de Souza Britto Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Michal Haindl Josef Kittler Fabio Roli

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this paper

Cite this paper

Ko, A.HR., Sabourin, R., de Souza Britto, A. (2007). A New HMM-Based Ensemble Generation Method for Numeral Recognition. In: Haindl, M., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2007. Lecture Notes in Computer Science, vol 4472. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72523-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-72523-7_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-72481-0

  • Online ISBN: 978-3-540-72523-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation