Abstract
Standard hidden Markov models (HMM's) have been studied extensively in the last two decades. It is well known that these models assume state conditional independence of the observations. Therefore, they are inadequate for classification of complex and highly structured patterns. Nowadays, the need for new statistical models that are capable to cope with structural time series data is increasing. We propose in this paper a novel paradigm that we named “structural hidden Markov model” (SHMM). It extends traditional HMM's by partitioning the set of observation sequences into classes of equivalences. These observation sequences are related in the sense they all contribute to produce a particular local structure. We describe four basic problems that are assigned to a structural hidden Markov model: (1) probability evaluation, (2) statistical decoding, (3) local structure decoding, and (4) parameter estimation. We have applied SHMM in order to mine customers' preferences for automotive designs. The results reported in this application show that SHMM's outperform the traditional hidden Markov model with a 9% of increase in accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Asai, S., Hayamizu, K., and Handa, H. 1993. Prediction of protein secondary structures by hidden markov models. Computer Application in the Biosciences (CABIOS), 9(2):141–146.
Bartolucci, F. and Besag, J. 2002. A recursive algorithm for markov random fields. Biometrika, 89(3):724–730.
Baum, L. and Petrie, T. 1966. Statistical inference for probabilistic functions of finite state markov chain. Ann. Math. Stat., 37:1554–1563.
Bellman, R. 1961. On the approximation of curves by line segments using dynamic programming. Communication of the ACM, (4(6)).
Bouchaffra, D., Govindaraju, V., and Srihari, S. 1999. Postprocessing of recognized strings using nonstationary Markovian models. IEEE Transactions: Pattern Analysis and Machine Intelligence, 21(10).
Bouchaffra, D. and Tan, J. 2004. The concept of structural hidden markov models: Application to mining customers' preferences for automotive designs. In 17th international conference on pattern recognition, (icpr). Cambridge, United Kingdom.
Cai, J. and Liu, Z. 1999. Integration of structural and statistical information for unconstrained handwritten numeral recognition. EEE Transactions: Pattern Analysis and Machine Intelligence, 21(3).
Churchill, G. 1989. Stochastic models for heterogeneous DNA sequences. Bulletin of Mathematical Biology, (51):79–94.
Duda, R., Hart, P., and Stork, D. 2001. Pattern classification, New York: Wiley.
Eddy, S. 1998. Profile Hidden Markov models. Bioinformatics, 14(9):755–763.
Efron, B. 1982. In the jackknife, the bootstrap and other resampling plans. SIAM.
Fan, G. and Xia, X. 2001. Improved hidden markov models in the wavelet-domain. IEEE Transactions on Signal Processing, 49(1).
Fellbaum, C. 1998. Wordnet: an electronic lexical database. Bradford Book.
Fodor, J. and Pylyshyn, Z.W. 1988. Connectionism and cognitive architecture: a critical analysis. Cognition, (28):3–71.
Freeman, H. 1961. On the encoding of arbitrary geometric configurations. IRE Trans. Electron. Comput., EC-10, 260–268.
Gales, J. 2000. Cluster adaptive training of hidden markov models. IEEE Transactions on Speech and Audio Processing, 8(4).
Geman, S., Bienenstock, E., Geman, S., and Potter, D. 2004. Compositionality, MDL priors, and object recognition (Tech. Rep.). Division of Applied Mathematics, Brown University.
Gemignani, M. 1990. Elementary topology. second edition. New York: Dover Publications, Inc.
Hernandez-Hernandez, D., Marcus, S., and Fard, P. 1999 May. Analysis of a risk-sensitive control problem for hidden markov chains. IEEE Transactions on Automatic Control, 44(5):1093.
Kim, D. and Bang, S. 2000. A handwritten numeral character classification using tolerant rough set. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9).
Krogh, A., Brown, M., Mian, I., Sjolander, K., and Haussler, D. 1994. Hidden markov models in computational biology: Applications to Protein Modeling. J. Mol. Biol. 235:1501–1531.
Li, J., Najmi, A., and Gray, R. 2000 Feb. Image classification by a two-dimensional hidden markov model. IEEE Transactions on Signal Processing, 482(2):517.
Rabiner, L. and Juang, B. 1993. Fundamentals of speech recognition. Prentice Hall.
Ripley, B. 1996. Pattern recognition and neural networks. Cambridge University Press.
Sanches, I. 2000 Sept. Noise-compensated hidden markov models. IEEE Transactions on Speech and Audio Processing, 8(5).
Zhou, J. and Chen, D. 2002 November. The subsystem of the fuzzed rough sets based on equivalence class. In Proceedings of the First International Conference on Machine Learning and Cybernetics. Beijing.
Zhu, W. and Frias, J. 2004 May. Stochastic context-free grammars and hidden markov models for modeling of bursty channels. IEEE Transactions on Vehicular Technology, 53(3).
Author information
Authors and Affiliations
Corresponding author
Additional information
Note
In other words, it is possible to decrease the resolution level of a complex pattern.
Rights and permissions
About this article
Cite this article
Bouchaffra, D., Tan, J. Structural Hidden Markov Models Using a Relation of Equivalence: Application to Automotive Designs. Data Min Knowl Disc 12, 79–96 (2006). https://doi.org/10.1007/s10618-005-0020-8
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-005-0020-8