Abstract
Sequence labeling problem is commonly encountered in many natural language and query processing tasks. SVM struct is a supervised learning algorithm that provides a flexible and effective way to solve this problem. However, a large amount of training examples is often required to train SVM struct, which can be costly for many applications that generate long and complex sequence data. This paper proposes an active learning technique to select the most informative subset of unlabeled sequences for annotation by choosing sequences that have largest uncertainty in their prediction. A unique aspect of active learning for sequence labeling is that it should take into consideration the effort spent on labeling sequences, which depends on the sequence length. A new active learning technique is proposed to use dynamic programming to identify the best subset of sequences to be annotated, taking into account both the uncertainty and labeling effort. Experiment results show that our SVM struct active learning technique can significantly reduce the number of sequences to be labeled while outperforming other existing techniques.
The work was performed when the first author worked as a summer intern at Yahoo, Inc.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aidan, F.N.: Active learning selection strategies for information extraction
Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Knowledge Discovery and Data Mining, 2 (1998)
Cohn, D.A., Atlas, L., Ladner, R.E.: Improving generalization with active learning. Machine Learning 15(2), 201–221 (1994)
Dagan, I., Engelson, S.P.: Committee-based sampling for training probabilistic classifiers. In: International Conference on Machine Learning, pp. 150–157 (1995)
Forney, G.D.: The viterbi algorithm. Proceedings of the IEEE 61(3), 268–278 (1973)
Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1990)
Jing, F., Li, M.J., Zhang, H.J., Zhang, B.: Entropy-based active learning with support vector machines for content-based image retrieval. In: IEEE International Conference on Multimedia and Expo., June 27–30, 2004. Digital Object Identifier, vol. 1, pp. 85–88 (2004)
Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th International Conf. on Machine Learning, pp. 282–289. Morgan Kaufmann, San Francisco (2001)
McCallum, A., Freitag, D., Pereira, F.: Maximum entropy Markov models for information extraction and segmentation. In: Proc. 17th International Conf. on Machine Learning, pp. 591–598. Morgan Kaufmann, San Francisco (2000)
McCallum, A.K., Nigam, K.: Employing EM in pool-based active learning for text classification. In: Shavlik, J.W. (ed.) Proceedings of 15th International Conference on Machine Learning, Madison, US, pp. 350–358. Morgan Kaufmann Publishers, San Francisco (1998)
Nguyen, N., Guo, Y.S.: Comparisons of sequence labeling algorithms and extensions. In: Proceedings of the 24th international conference on Machine learning, pp. 681–688. ACM Press, New York (2007)
Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)
Roy, N., McCallum, A.: Toward optimal active learning through sampling estimation of error reduction. In: Proc. 18th International Conf. on Machine Learning, pp. 441–448. Morgan Kaufmann, San Francisco (2001)
Sang, E.F.T.K.: Introduction to the conll-2002 shared task: Language-independent named entity recognition. In: Proceedings of CoNLL 2002, Taipei, Taiwan, pp. 155–158 (2002)
Shen, D., Zhang, J., Su, J., Zhou, G.D., Tan, C.L.: Multi-criteria-based active learning for named entity recognition. In: Proceedings of the 42nd Meeting of the Association for Computational Linguistics, Main Volume, Barcelona, Spain, July 2004, pp. 589–596 (2004)
Stegeman, L.: Part-of-speech tagging and chunk parsing of spoken dutch using support vector machines (2006)
Symons, C.T., Samatova, N.F., Krishnamurthy, R., Park, B.H., Umar, T., Buttler, D., Critchlow, T., Hysom, D.: Multi-criterion active learning in conditional random fields. In: Proceedings of the 18th IEEE International Conference on Tools with Artificial Intelligence, Washington, DC, USA, pp. 323–331. IEEE Computer Society, Los Alamitos (2006)
Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. In: Langley, P. (ed.) Proceedings of ICML 2000, 17th International Conference on Machine Learning, Stanford, US, pp. 999–1006. Morgan Kaufmann Publishers, San Francisco (2000)
Tsochantaridis, I., Joachims, T., Hofmann, T., Altun, Y.: Large margin methods for structured and interdependent output variables. J. Mach. Learn. Res. 6, 1453–1484 (2005)
Weston, J., Watkins, C.: Multi-class support vector machines. Technical Report CSD-TR-98-04, Department of Computer Science, Royal Holloway, University of London, Egham, TW20 0EX, UK (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cheng, H., Zhang, R., Peng, Y., Mao, J., Tan, PN. (2008). Maximum Margin Active Learning for Sequence Labeling with Different Length. In: Perner, P. (eds) Advances in Data Mining. Medical Applications, E-Commerce, Marketing, and Theoretical Aspects. ICDM 2008. Lecture Notes in Computer Science(), vol 5077. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70720-2_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-70720-2_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70717-2
Online ISBN: 978-3-540-70720-2
eBook Packages: Computer ScienceComputer Science (R0)