Abstract
The problem of learning a classifier from examples is a fundamental task in Machine Learnig and is nowadays actively studied. Most approaches follow a Discrimination based paradigm, where the aim is to find the best way to separate the examples of one class from those of other classes. A less popular approach follows a Characterization based paradigm, where the aim is to build a description (model) of every class from examples, and use this model to recognizes instances of that class. This paper focuses on the specific task of classifying and tagging symbolic sequences, by introducing a Characterization approach, based on Structured Hidden Markov Models, and compares its perfomances against a widely used discriminative approach, i.e. kernel machines. This task is particulary relevant to several applications in many fields, such as molecular biology, web log analysis, network traffic analysis, user profiling, and so on. In order to assess the validity of the proposed approach, an artificial benchmark has been designed in such a way that the regularities to be discovered are well known and allow for a controlled evaluation of the real capabilities the learning algorithms investigated. The obtained results allow to point out the major advantages and weaknesses of the investigated approaches in the specific classification task addressed.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximisation techniques occurring in the statistical analysis of probabilistic functions of markov chains. The Annals of Mathematical Statistics 41(1), 164–171 (1970)
Boser, B.E.: A training algorithm for optimal margin classifiers. In: Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pp. 144–152. ACM Press, New York (1992)
Bouchaffra, D., Tan, J.: Structural hidden markov models using a relation of equivalence: Application to automotive designs. Data Mining and Knowledge Discovery 12, 79–96 (2006)
Carbonell, J.G., Michalski, R.S., Mitchell, T.: An overview of machine learning. In: Carbonell, J.G., Michalski, R.S., Mitchell, T. (eds.) Machine Learning, an Artificial Intelligence Approach. Morgan Kaufmann, San Francisco (1983)
Cortes, C., Vapnik, V.: Support vector networks. Machine Learning, 273–297 (1995)
Davidson, P.: Integrating models of discrimination and characterization. Data Analysis 3(2), 95–109 (1999)
Durbin, R., Eddy, S., Krogh, A., Mitchison, G.: Biological sequence analysis. Cambridge University Press, Cambridge (1998)
Forney, G.D.: The viterbi algorithm. Proceedings of IEEE 61, 268–278 (1973)
Freitag, D., McCallum, A.: Information extraction with HMM structures learned by stochastic optimization. In: Proceedings of the Seventeenth National Conference on Artificial Intelligence, Austin, TX. AAAI Press, Menlo Park (2000)
Fujiwara, Y., Asogawa, M., Konagaya, A.: Stochastic motif extraction using hidden markov model. In: Altman, R.B., Brutlag, D.L., Karp, P.D., Lathrop, R.H., Searls, D.B. (eds.) Proceedings of the Second International Conference on Intelligent Systems for Molecular Biology, ISMB, pp. 121–129. AAAI, Menlo Park (1994)
Galassi, U.: Structured Hidden Markov Models: A General Tool for Modeling Process Behavior. PhD thesis, Università degli Studi di Torino, Dottorato di ricerca in Informatica (April 2008)
Galassi, U., Giordana, A., Saitta, L.: Structured hidden markov models: A general tool for modeling agent behaviors. In: Soft Computing Applications in Business. Studies in Fuzziness and Soft Computing, vol. 230, pp. 273–292. Springer, Heidelberg (2008)
Gussfield, D.: Algorithms on Strings, Trees, and Sequences. Cambridge University Press, Cambridge (1997)
Haussler, D.: Convolution kernels on discrete structures. Technical Report SC-CRL-99-10, University of California in Santa Cruz, Computer Science Department (1999)
Joachims, T.: Text categorization with support vector machines: Learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998)
Lodhi, H., Shawe-Taylor, J., Cristianini, N., Watkins, C.: Text classification using string kernels. Journal of Machine Learning Research 2, 419–444 (2002)
El Manzalawi, Y., Dobbs, D., Honavar, V.: Predicting linear b-call epitopes using string kernels. Journal of Molecular Recognition (2008), doi:10.1002/jmr.893
Moulinier, I., Raskinis, G., Ganascia, J.: Text categorization: a symbolic approach. In: Proceedings of the Fifth Annual Symposium on Document Analysis and Information Retrieval (1996)
Ng, A.Y., Jordan, M.I.: On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) NIPS, pp. 841–848. MIT Press, Cambridge (2001)
Rabiner, L., Juang, B.: Fundamentals of Speech Recognition. Prentice Hall, Englewood Cliffs (1993)
Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of IEEE 77(2), 257–286 (1989)
Rubinstein, Y.D., Hastie, T.: Discriminative vs informative learning. In: Proc. Third Int. Conf. on Knowledge Discovery and Data Mining, pp. 49–53. AAAI Press, Menlo Park (1997)
Schölkopf, B., Burgess, C., Smola, A.: Advances in Kernel Methods. MIP Press (1998)
Stolcke, A., Omohundro, S.: Hidden markov model induction by bayesian model merging. In: Advances in Neural Information Processing Systems, vol. 5, pp. 11–18 (1993)
Vapnik, V.N.: The Nature of Statistical Learning. Springer, Heidelberg (1995)
Watkins, C.: Dynamic alignment kernels. In: Smola, A.J., Bartlett, P.L., Schölkopf, B., Schuurmans, D. (eds.) Advances in Large Margin Classifiers, pp. 39–50. MIT Press, Cambridge (2000)
Yang, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: Fisher, D.H. (ed.) Proceedings of ICML 1997, 14th International Conference on Machine Learning, Nashville, US, pp. 412–420. Morgan Kaufmann Publishers, San Francisco (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Galassi, U., Botta, M., Saitta, L. (2010). Structured Hidden Markov Model versus String Kernel Machines for Symbolic Sequence Classification. In: Koronacki, J., Raś, Z.W., Wierzchoń, S.T., Kacprzyk, J. (eds) Advances in Machine Learning I. Studies in Computational Intelligence, vol 262. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05177-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-05177-7_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05176-0
Online ISBN: 978-3-642-05177-7
eBook Packages: EngineeringEngineering (R0)