Skip to main content
SpringerLink
Log in

Hidden Markov Models Training by a Particle Swarm Optimization Algorithm

  • Published:
Journal of Mathematical Modelling and Algorithms

Abstract

In this work we consider the problem of Hidden Markov Models (HMM) training. This problem can be considered as a global optimization problem and we focus our study on the Particle Swarm Optimization (PSO) algorithm. To take advantage of the search strategy adopted by PSO, we need to modify the HMM's search space. Moreover, we introduce a local search technique from the field of HMMs and that is known as the Baum–Welch algorithm. A parameter study is then presented to evaluate the importance of several parameters of PSO on artificial data and natural data extracted from images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abido, M.: Optimal power flow using particle swarm optimization, Electr. Energy Syst. 24 (2002), 563–571.

    Article  Google Scholar 

  2. Baum, L., Petrie, T., Soules, G. and Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains, Ann. Math. Stat. 41(1) (1970), 164–171.

    MathSciNet  MATH  Google Scholar 

  3. Berg, F. v.: An analysis of Particle Swarm Optimizers, PhD thesis, University of Pretoria, 2001.

  4. Bilmes, J.: What HMMs can do? Technical report, Department of Electrical Engineering, University of Washington, 2002.

  5. Brouard, T., Slimane, M. and Asselin de Beauville, J.-P.: Modélisation de processus simultanés indépendants par chaînes de Markov multidimensionnelles (CMC-MD/I), Technical Report 200, Laboratoire d'Informatique de l'Université de Tours, E3i Tours, 1997.

  6. Cappé, O.: Ten years of HMM. http://ww.tsi.enst.fr/~cappe/docs/hmmbib.html, 2001.

  7. Chen, T.-Y., Mei, X.-D., Pan, J.-S. and Sun, S.-H.: Optimization of HMM by the Tabu Search Algorithm, J. Inf. Sci. Eng. 20(5) (2004), 949–957.

    Google Scholar 

  8. Dempster, A. P., Laird, N. M. and Rubin, D. B.: Maximum-likelihood from incomplete data via the EM algorithm, J. R. Stat. Soc. B 39(1) (1977), 1–39.

    MATH  MathSciNet  Google Scholar 

  9. Do, M.: Fast approximation of Kullback–Leibler distance for dependence trees and Hidden Markov Models, IEEE Signal Process. Lett. 10(8) (2003), 250–254.

    MathSciNet  Google Scholar 

  10. Falkhausen, M., Reininger, H. and Wolf, D.: Calculation of distance measures between Hidden Markov Models, in Proceedings of the Eurospeech'95. Madrid, 1995, pp. 1487–1490.

  11. Fine, S., Singer, Y. and Tishby, N.: The Hierarchical Hidden Markov Model: Analysis and applications, Mach. Learn. 32(1) (1998), 41–62.

    Article  MATH  Google Scholar 

  12. Forney, G.: The Viterbi algorithm, Proc. IEEE 61(3) (1973), 268–278.

    Article  MathSciNet  Google Scholar 

  13. Hu, X., Eberhard, R. and Shi, Y.: Swarm intelligence for Permutation Optimization: A case-study of n-Queens problems, in Proceedings of the IEEE Swarm Intelligence Symposium. Indianapolis, 2003, pp. 243–246.

  14. Jelinek, F., Bahl, L. and Mercer, L.: Design of a linguistic statistical decoder for the recognition of continuous speech, in: IEEE Trans. IT, IT-21, 1975.

  15. Kennedy, J. and Eberhart, R.: Particle Swarm Optimization, in Proceedings of the IEEE International Joint Conference on Neural Networks, Vol. 4, 1995, pp. 1942–1948, IEEE.

  16. Kennedy, J., Eberhart, R. and Shi, Y.: Swarm Intelligence, Morgan Kaufmann, 2002.

  17. Kosmala, A., Willett, D. and Rigoll, G.: Advanced state clustering for very large vocabulary HMM-based on-line handwriting recognition, in Proceeding of the International Conference on Document Analysis and Recognition, 1999, pp. 442–445.

  18. Kundu, A. and Bahl, P.: Recognition of handwritten script: A hidden Markov model based approach, in ICASSP, 1988, pp. 928–931.

  19. Monmarché, N.: Algorithmes de fourmis artificielles : Applicationsàla classification et à l'optimisation, Thèse de doctorat, Laboratoire d'Informatique, Université de Tours, 2000.

  20. Monmarché, N., Venturini, G. and Slimane, M.: Optimisation de Chaînes Markov Cachées par une population de fourmis Pachycondyla apicalis, Rapport interne 195, Laboratoire d'Informatique de l'Université de Tours, E3i Tours, 1997, 24 pages.

  21. Monmarché, N., Venturini, G. and Slimane, M.: On how Pachycondyla apicalis ants suggest a new search algorithm, Future Gener. Comput. Syst. 16(8) (2000), 937–946.

    Article  Google Scholar 

  22. Olivier, C., Avila, M., Courtellemont, P., Paquet, T. and Lecourtier, Y.: Handwritten word recognition by image segmentation and Hidden Markov Model, in Proceedings of IEEE-IES, 1993, pp. 2093–2097.

  23. Omran, M., Salman, A. and Engelbrecht, A.: Image classification using Particle Swarm Optimization, in 4th Asia-Pacific Conference on Simulated Evolution and Learning, 2002.

  24. Paquet, U. and Engelbrecht, A.: Training support vector machines with particle swarms, in International Joint Conference on Neural Networks, Portland, Oregon, 2003.

  25. Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition, Proc. IEEE 77(2) (1989), 257–286.

    Article  Google Scholar 

  26. Rasmussen, T. K. and Krink, T.: Improved Hidden Markov Model training for multiple sequence alignment by a particle swarm optimization–evolutionary algorithm hybrid, Biosystems 72(1–2) (2003), 5–17.

    Article  Google Scholar 

  27. Ratnaweera, A., Halgamuge, S. and Watson, H.: Self-organizing hierarchical Particle Swarm Optimizer with time-varying acceleration coefficients, IEEE Trans. Evol. Comput. 8(3) (2004), 240–255.

    Article  Google Scholar 

  28. Salman, A., Ahmad, I. and Al-Madani, S.: Particle Swarm Optimization for task assignment problem, Microprocess. Microsyst. 26 (2003), 363–371.

    Article  Google Scholar 

  29. Samaria, F. and Harter, A.: Paramatrisation of a stochastic model for human face identification, in 2nd IEEE Workshop on application of computer vision, Sarasota, Florida, 1994.

  30. Slimane, M., Venturini, G., Asselin de Beauville, J.-P. and Brouard, T.: Hybrid genetic learning of hidden Markov models for Time Series Prediction, Biomimetic Approaches in Management Science, Kluwer, 1998.

  31. Slimane, M., Venturini, G., Asselin de Beauville, J.-P., Brouard, T. and Brandeau, A.: Optimizing Hidden Markov Models with a Genetic Algorithm, in Proceedings of Artificial Evolution conference, Vol. 1063 of Lecture Notes in Computer Science, Springer, 1996, pp. 384–396.

  32. Tasgetiren, M., Sevkli, M., Liang, Y. and Gencyilmaz, G.: Particule Swarm Optimization Algorithm for Single Machine Total Weighted Tardiness Problem, in Proceedings of Congress on Evolutionary Computation, 2004, pp. 1412–1419.

  33. Tasgetiren, M.F., Sevkli, M., Liang, Y.-C. and Gencyilmaz, G.: Particle Swarm Optimization Algorithm for Permutation Flowshop, in M. Dorigo, M. Birattari, C. Blum, L. Gambardella, F. Mondada, and T. Stützle (eds.): Proceedings of ANTS 2004 – Fourth International Workshop on Ant Colony Optimization and Swarm Intelligence, Vol. 3172 of Lecture Notes in Computer Science, Brussels, Belgium, Springer, 2004.

  34. Thomsen, R.: Evolving the topology of hidden Markov models using evolutionary algorithms, in Proceedings of Parallel Problem Solving from Nature VII (PPSN-2002), 2002, pp. 861–870.

  35. van der Merwe, D. and Engelbrecht, A.: Data clustering using Particle Swarm Optimization, in IEEE Congress on Evolutionary Computation, Canberra, Australia, 2003, pp. 215–220.

  36. Vihola, M., Harju, M., Salmela, P., Suontausta, J. and Savela, J.: Two dissimilarity measures for HMMs and their application in phoneme model clustering, in Proceedings of ICASSP 2002, 1980, pp. 993–936.

  37. Wachowiak, M., Smolikova, R., Zheng, Y., Zurada, J. and Elmaghraby, A.: An approach to multimodal biomedical image registration utilizing Particle Swarm Optimization, IEEE Trans. Evol. Comput. 8(3) (2004), 289–301.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratoire d'Informatique, Polytech'Tours, Université François-Rabelais de Tours, 64 avenue Jean Portalis, 37200, Tours, France

    Sébastien Aupetit, Nicolas Monmarché & Mohamed Slimane

Authors
  1. Sébastien Aupetit
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicolas Monmarché
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Slimane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sébastien Aupetit.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aupetit, S., Monmarché, N. & Slimane, M. Hidden Markov Models Training by a Particle Swarm Optimization Algorithm. J Math Model Algor 6, 175–193 (2007). https://doi.org/10.1007/s10852-005-9037-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10852-005-9037-7

Mathematics Subject Classifications (2000):

Key words

Search

Navigation