Skip to main content
Springer Nature Link
Log in

A Nonstationary Hidden Markov Model with Approximately Infinitely-Long Time-Dependencies

  • Conference paper
Advances in Visual Computing (ISVC 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8888))

Included in the following conference series:

  • 2525 Accesses

Abstract

Hidden Markov models (HMMs) are a popular approach for modeling sequential data, typically based on the assumption of a first-order Markov chain. In other words, only one-step back dependencies are modeled which is a rather unrealistic assumption in most applications. In this paper, we propose a method for postulating HMMs with approximately infinitely-long time-dependencies. Our approach considers the whole history of model states in the postulated dependencies, by making use of a recently proposed nonparametric Bayesian method for modeling label sequences with infinitely-long time dependencies, namely the sequence memoizer. We manage to derive training and inference algorithms for our model with computational costs identical to simple first-order HMMs, despite its entailed infinitely-long time-dependencies, by employing a mean-field-like approximation. The efficacy of our proposed model is experimentally demonstrated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer, New York (2005)

    MATH  Google Scholar 

  2. Mari, J., Fohr, D., Junqua, J.: A second-order HMM for high-performance word and phoneme-based continuous speech recognition. In: Proc. of the IEEE Int. Conf. on Acoustics, Speech and Signal Processing, pp. 435–438 (1996)

    Google Scholar 

  3. Mari, J.F., Haton, J.P., Kriouile, A.: Automatic word recognition based on second-order hidden Markov models. IEEE Trans. Speech Audio Process. 5, 22–25 (1997)

    Article  Google Scholar 

  4. Aycard, O., Mari, J.F., Washington, R.: Learning to automatically detect features for mobile robots using second-order HMMs. Int. J. Adv. Robotic Syst. 1, 231–245 (2004)

    Google Scholar 

  5. Engelbrecht, H., du Preez, J.: Efficient backward decoding of high-order hidden markov models. Pattern Recognition 43, 99–112 (2010)

    Article  MATH  Google Scholar 

  6. Wood, F., Gasthaus, J., Archambeau, C., James, L., Teh, Y.W.: The sequence memoizer. Communications of the ACM 54, 91–98 (2011)

    Article  Google Scholar 

  7. Celeux, G., Forbes, F., Peyrard, N.: EM procedures using mean field-like approximations for Markov model-based image segmentation. Patt. Recogn. 36, 131–144 (2003)

    Article  MATH  Google Scholar 

  8. Zhang, J.: The mean field theory in EM procedures for Markov random fields. IEEE Transactions on Image Processing 2, 27–40 (1993)

    Article  Google Scholar 

  9. Pitman, J., Yor, M.: The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Annals of Probability 25, 855–900 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  10. Teh, Y.W.: A hierarchical Bayesian language model based on Pitman-Yor processes. In: Proc. Assoc. for Comp. Linguistics, pp. 985–992 (2006)

    Google Scholar 

  11. Wood, F., Archambeau, C., Gasthaus, J., James, L.F., Teh, Y.: A stochastic memoizer for sequence data. In: Proc. Int. Conference on Machine Learning (ICML) (2009)

    Google Scholar 

  12. Chandler, D.: Introduction to Modern Statistical Mechanics. Oxford Univ. Press (1987)

    Google Scholar 

  13. Geiger, D., Girosi, F.: Parallel and deterministic algorithms from MRFs: surface reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 13, 401–412 (1991)

    Article  Google Scholar 

  14. Zerubia, J., Chellappa, R.: Mean field approximation using compound Gauss-Markov random field for edge detection and image restoration. In: Proc. ICASSP, pp. 2193–2196 (1990)

    Google Scholar 

  15. Jaakkola, T., Jordan, M.: Improving the mean field approximation via the use of mixture distributions. In: Jordan, M. (ed.) Learning in Graphical Models, pp. 163–173. Kluwer (1998)

    Google Scholar 

  16. Hofmann, T., Buhmann, J.: Pairwise data clustering by deterministic annealing. IEEE Trans. Pattern Anal. Mach. Intell. 19, 1–14 (1997)

    Article  Google Scholar 

  17. Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 245–255 (1989)

    Article  Google Scholar 

  18. Rabiner, L.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 245–255 (1989)

    Article  Google Scholar 

  19. McLachlan, G., Peel, D.: Finite Mixture Models. Wiley Ser. Probability and Statistics (2000)

    Google Scholar 

  20. Chatzis, S.P., Varvarigou, T.A.: A fuzzy clustering approach toward hidden Markov random field models for enhanced spatially constrained image segmentation. IEEE Trans. on Fuzzy Systems 16, 1351–1361 (2008)

    Article  Google Scholar 

  21. Chatzis, S.P., Tsechpenakis, G.: The infinite hidden Markov random field model. IEEE Transactions on Neural Networks 21, 1004–1014 (2010)

    Article  Google Scholar 

  22. Voulodimos, A., et al.: A threefold dataset for activity and workflow recognition in complex industrial environments. IEEE Multimedia 19, 42–52 (2012)

    Article  Google Scholar 

  23. Ni, B., Wang, G., Moulin, P.: RGBD-HuDaAct: A color-depth video database for human daily activity recognition. In: ICCV Workshops, pp. 1147–1153 (2011)

    Google Scholar 

  24. Kudo, M., Toyama, J., Shimbo, M.: Multidimensional curve classification using passing through regions. Pattern Recogn. Lett. 20, 1103–1111 (1999)

    Article  Google Scholar 

  25. Jaeger, H., Maass, W., Principe, J.: Special issue on echo state networks and liquid state machines. Neural Networks 20, 287 (2007)

    Article  Google Scholar 

  26. Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453–1484 (2005)

    MATH  MathSciNet  Google Scholar 

  27. McCallum, A., Freitag, D., Pereira, F.C.N.: Maximum entropy markov models for information extraction and segmentation. In: Proc. of the Int. Conf. on Mach. Learning, ICML 2000, pp. 591–598 (2000)

    Google Scholar 

  28. Sha, F., Saul, L.K.: Large margin hidden markov models for automatic speech recognition. In: Advances in Neural Information Processing Systems 19, pp. 1249–1256. MIT Press (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cyprus University of Technology, Cyprus

    Sotirios P. Chatzis

  2. University of Patras, Greece

    Dimitrios I. Kosmopoulos

  3. Technological Educational Institute of Crete, Greece

    George M. Papadourakis

Authors
  1. Sotirios P. Chatzis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dimitrios I. Kosmopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George M. Papadourakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of Nevada at Reno, USA

    George Bebis

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Bahram Parvin

  4. Desert Research Institute, Reno, NV, USA

    Darko Koracin

  5. The University of Texas at Dallas, 75080, Richardson, TX, USA

    Ryan McMahan

  6. NextGen Interactions, 27604, Raleigh, NC, USA

    Jason Jerald

  7. Indiana University, 46202, Indianapolis, IN, USA

    Hui Zhang

  8. Microsoft Research, 1 Microsoft Way, 98052, Redmond, WA, USA

    Steven M. Drucker

  9. University of Delaware, 19716-2712, Newark, DE, USA

    Chandra Kambhamettu

  10. Intel Corp., 95054, Sata Clara, CA, USA

    Maha El Choubassi

  11. Computer Graphics and Interactive Media Lab, Department of Computer Science, University of Houston, 77004, Houston, TX, USA

    Zhigang Deng

  12. NVIDIA, 34788, Leesburg, FL, USA

    Mark Carlson

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Chatzis, S.P., Kosmopoulos, D.I., Papadourakis, G.M. (2014). A Nonstationary Hidden Markov Model with Approximately Infinitely-Long Time-Dependencies. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2014. Lecture Notes in Computer Science, vol 8888. Springer, Cham. https://doi.org/10.1007/978-3-319-14364-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14364-4_6

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14363-7

  • Online ISBN: 978-3-319-14364-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics