Skip to main content
SpringerLink
Log in

Temporal Pattern Generation Using Hidden Markov Model Based Unsupervised Classification

  • Conference paper
  • First Online:
Book cover Advances in Intelligent Data Analysis (IDA 1999)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1642))

Included in the following conference series:

Abstract

This paper describes a clustering methodology for temporal data using hidden Markov model(HMM) representation. The proposed method improves upon existing HMM based clustering methods in two ways: (i) it enables HMMs to dynamically change its model structure to obtain a better fit model for data during clustering process, and (ii) it provides objective criterion function to automatically select the clustering partition. The algorithm is presented in terms of four nested levels of searches: (i) the search for the number of clusters in a partition, (ii) the search for the structure for a fixed sized partition, (iii) the search for the HMM structure for each cluster, and (iv) the search for the parameter values for each HMM. Preliminary experiments with artificially generated data demonstrate the effectiveness of the proposed methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baum, L. E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of markov chains. The Annuals of Mathematical Statistics 4(1) (1970) 164–171.

    Article  MathSciNet  Google Scholar 

  2. Forney, G.: The viterbi algorithm. Proceedings of the IEEE 61(3) (1973) 268–277.

    Article  MathSciNet  Google Scholar 

  3. Dempster, A. P., Laird, N. M., Rubin, D. B.: Maximum likelihood from incomplete data via the em algorithm. Journal of Royal Statistical Society Series B(methodological) 39 (1977) 1–38.

    MATH  MathSciNet  Google Scholar 

  4. Bahl, L. R., Brown, P. F., De Souza, P. V., Mercer, R. L.: Maximum mutual information estimation of hidden markov model parameters. Proceedings of the IEEE-IECEJ-AS International Conference on Acoustics, Speech, and Signal Processing 1 (1978) 49–52.

    Google Scholar 

  5. Juang, B. H., Rabiner, L. R.: A probabilistic distance measure for hidden markov models. AT&T Technical Journal 64(2) (1985) 391–408.

    MathSciNet  Google Scholar 

  6. Fisher, D.: Knowledge acquisition via incremental conceptual clustering. Machine Learning 2 (1987) 139–172.

    Google Scholar 

  7. Rabiner, L. R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2) (1989) 257–285.

    Article  Google Scholar 

  8. Rabiner, L. R., Lee, C. H., Juang, B. H., Wilpon, J. G.: Hmm clustering for connected word recognition. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (1989).

    Google Scholar 

  9. Casacuberta, F., Vidal, E., and Mas B.: Learning the structure of hmm’s through grammatical inference techniques. Proceedings of the International Conference on Acoustic, Speech, and Signal Processing (1990) 717–720.

    Google Scholar 

  10. Lee, K. F.: Context-dependent phonetic hidden markov models for speaker-independent continuous speech recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing 38(4) (1990) 599–609.

    Article  Google Scholar 

  11. Casella G., George, E. I.: Explaining the gibbs sampler. The American Statistician 46(3) (1992) 167–174.

    Article  MathSciNet  Google Scholar 

  12. Cooper, G. F., Herskovits, E.: A bayesian method for the induction of probabilistic network from data. Machine Learning 9 (1992) 309–347.

    MATH  Google Scholar 

  13. Omohundro, S. M.: Best-first model merging for dynamic learning and recognition. Advances in Neural Information Processing Systems 4 (1992) 958–965.

    Google Scholar 

  14. Takami, J., Sagayama, S.: A successive state splitting algorithm for Efficient allo-phone modeling. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing 1 (1992) 573–576.

    Google Scholar 

  15. Stolcke, A., Omohundro, S. M.: Best-first model merging for hidden markov model induction. Technical Report TR-94-003, International Computer Science Institute, 1994.

    Google Scholar 

  16. Wallace, C. S., Dowe, D. L.: Intrinsic classificatin by mml-the snob program. Proceedings of the Seventh Australian Joint Conference on Artificial Intelligence (1994) 37–44.

    Google Scholar 

  17. Biswas, G., Weinberg, J., Li, C.: Iterate: A conceptual clustering method for knowledge discovery in databases. Artificial Intelligence in Petroleum Industry: Symbolic and Computational Applications, Braunschweig, B. and Day, R.editors, Teditions Technip, 1995.

    Google Scholar 

  18. Chib, S.: Marginal likelihood from the gibbs sampling. Journal of the American Statistical Association (1995) 1313–1321.

    Google Scholar 

  19. Kass, R. E., Raftery, A. E.: Bayes factor. Journal of the American Statistical Association (1995) 773–795.

    Google Scholar 

  20. Kosaka, T., Masunaga, S., Kuraoka, M.: Speaker-independent phone modeling based on speaker-dependent hmm’s composition and clustering. Proceedings of the ICASSP’95 (1995) 441–444.

    Google Scholar 

  21. Cheeseman, P., Stutz, J.: Bayesian Classification(autoclass): Theory and results. Advances in Knowledge Discovery and Data Mining (1996) chapter 6 153–180, Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P., Uthurusamy, R.editors AAAI-MIT press.

    Google Scholar 

  22. Dermatas, E., Kokkinakis, G.: Algorithm for clustering continuous density hmm by recognition error. IEEE Transactions on Speech and Audio Processing 4(3) (1996) 231–234.

    Article  Google Scholar 

  23. Chickering, D. M., Heckerman, D.: Efficient approximations for the marginal like-lihood of bayesian networks with hidden variables. Machine Learning 29 (1997) 181–212.

    Article  MATH  Google Scholar 

  24. Ghahramani, Z., Jordan, M. I.: Factorial hidden markov models. Machine Leaning 29 (1997) 245–273.

    Article  MATH  Google Scholar 

  25. Ostendorf, M., Singer, H.: Hmm topology design using maximum likelihood successive state splitting. Computer Speech and Language 11 (1997) 17–41.

    Article  Google Scholar 

  26. Smyth, P.: Clustering sequences with hidden markov models. Advances in Neural Information Processing (1997).

    Google Scholar 

  27. Li, C.: Unsupervised Classification on temporal data. Technical Report VU-CS-TR-98-04, Vanderbilt University, April 1998.

    Google Scholar 

  28. Li, C., Biswas, G.: Clustering sequence data using hidden markov model representation. SPIE99 Conference on Data Mining and Knowledge Discovery: Theory, Tools, and Technology (1999) 14–21.

    Google Scholar 

  29. Sebastiani, P., Ramoni, M., Cohen, P., Warwick, J., Davis, J.: Discovering dynamics using bayesian clustering. Proceedings of the 3rd International Symposium on Intelligent Data Analysis (1999).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Vanderbilt University, Nashville, TN, 37235, USA

    Cen Li & Gautam Biswas

Authors
  1. Cen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gautam Biswas
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mathematics, Imperial College, Huxley Building 180 Queen’s Gate, London, SW7 2BZ, UK

    David J. Hand

  2. Leiden Institute for Advanced Computer Science, Leiden University, 2300, RA Leiden, The Netherlands

    Joost N. Kok

  3. Berkeley Initiative in Soft Computing, University of California at Berkeley, 329 Soda Hall, Berkeley, CA, 94720, USA

    Michael R. Berthold

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Li, C., Biswas, G. (1999). Temporal Pattern Generation Using Hidden Markov Model Based Unsupervised Classification. In: Hand, D.J., Kok, J.N., Berthold, M.R. (eds) Advances in Intelligent Data Analysis. IDA 1999. Lecture Notes in Computer Science, vol 1642. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48412-4_21

Download citation

  • DOI: https://doi.org/10.1007/3-540-48412-4_21

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-66332-4

  • Online ISBN: 978-3-540-48412-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation