Abstract
Hidden Markov Models (HMMs) are an useful and widely utilized approach to the modeling of data sequences. One of the problems related to this technique is finding the optimal structure of the model, namely, its number of states. Although a lot of work has been carried out in the context of the model selection, few work address this specific problem, and heuristics rules are often used to define the model depending on the tackled application. In this paper, instead, we use the notion of probabilistic bisimulation to automatically and efficiently determine the minimal structure of HMM. Bisimulation allows to merge HMM states in order to obtain a minimal set that do not significantly affect model performances. The approach has been tested on DNA sequence modeling and 2D shape classification. Results are presented in function of reduction rates, classification performances, and noise sensitivity.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Rabiner, L.R.: A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proc. of IEEE 77(2) (1989) 257–286.
Baum, L.E., Petrie, T.E, Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Annals of Math. Statistics. 41(1) (1970) 164–171.
Baum, L.E.: An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequality 3 (1970) 1–8.
Veltman, S.R., Prasad, R.: Hidden Markov Models applied to on-line handwritten isolated character recognition. IEEE Trans. Image Proc., 3(3) (1994) 314–318.
Churchill, G.A.: Hidden Markov Chains and the analysis of genome structure. Computers & Chemistry 16 (1992) 107–115.
Eickeler, S., Kosmala, A., Rigoll, G.: Hidden Markov Model based online gesture recognition. Proc. Int. Conf. on Pattern Recognition (ICPR) (1998) 1755–1757.
Jebara, T., Pentland, A.: Action Reaction Learning: Automatic Visual Analysis and Synthesis of interactive behavior. In 1st Intl. Conf. on Computer Vision Systems (ICVS’99) (1999).
Theodoridis, S., Koutroumbas, K.: Pattern recognition. Academic Press (1999).
He, Y., Kundu, A.: 2-D shape classification using Hidden Markov Model. IEEE Trans. Pattern Analysis Machine Intelligence. 13(11) (1991) 1172–1184.
Figueiredo, M.A.T., Leitao, J.M.N., Jain, A.K.: On Fitting Mixture Models, in E. Hancock and M. Pellilo(Editors), Energy Minimization Methods in Computer Vision and Pattern Recognition, 54–69, Springer Verlag, 1999.
Stolcke, A., Omohundro, S.:Hidden Markov Model Induction by Bayesian Model Merging. Hanson, S.J., Cowan, J.D., Giles, C.L. eds. Advances in Neural Information Processing Systems 5 (1993) 11–18.
Brand, M.: An entropic estimator for structure discovery. Kearns, M.S., Solla, S.A., Cohn, D.A. eds. Advances in Neural Information Processing Systems 11 (1999).
Aczel, P.: Non-well-founded sets. Lecture Notes, Center for the Study of Language and Information 14 (1988).
Baier, C., Engelen, B., Majster-Cederbaum, M.: Deciding Bisimilarity and Similarity for Probabilistic Processes. J. Comp. and System Sciences 60 (1999) 187–231.
Dovier, A., Piazza, C., Policriti, A.: A Fast Bisimulation Algorithm. In proc. of 13th Conference on Computer Aided Verification, CAV’01, 2001. Paris, France.
Kanellakis, P.C., Smolka, S.A.: CCS Expressions, Finite State Processes, and Three Problems of Equivalence. Information and Computation, 86(1) (1990) 43–68.
Lisitsa, A., Sazanov, V.: Bounded Hyperset Theory and Web-like Data Bases. In 5th Kurt Gödel Colloquium. LNCS1289 (1997) 172–185.
Milner, R.: Operational and Algebraic Semantics of Concurrent Processes. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science (1990).
Paige, R., Tarjan, R. E.: Three Partition refinement algorithms. SIAM Journal on Computing 16(6) (1987) 973–989.
VanBenthem, J.: Modal Correspondence Theory. PhD dissertation, Universiteit van Amsterdam, Instituut voor Logica en Grondslagenonderzoek van Exacte Wetenschappen, (1978) 1–148.
Salzberg, S.L.: Gene discovery in DNA sequences. IEEE Intelligent Systems 14(6) (1999) 44–48.
Salzberg, S.L., Searls, D., Kasif, S.: Computational methods in Molecular Biology. Elsevier Science (1998).
Canny, J.F.: A computational approach to edge detection. IEEE Trans. Pattern Analysis Machine Intelligence 8(6) (1986) 679–698.
Jain, R., Kasturi, R., Schunck, B.G.: Machine Vision. McGraw-Hill (1995).
Roberts, S., Husmeier, D., Rezek, I., Penny, W.: Bayesian Approaches to gaussian mixture modelling, IEEE Trans. on P.A.M.I., 20(11) (1998) 1133–1142.
Schwarz, G.: Estimating the dimension of a model, The Annals of Statistics, 6(2) (1978) 461–464.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bicego, M., Dovier, A., Murino, V. (2001). Designing the Minimal Structure of Hidden Markov Model by Bisimulation. In: Figueiredo, M., Zerubia, J., Jain, A.K. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2001. Lecture Notes in Computer Science, vol 2134. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44745-8_6
Download citation
DOI: https://doi.org/10.1007/3-540-44745-8_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42523-6
Online ISBN: 978-3-540-44745-0
eBook Packages: Springer Book Archive