Siddharth Patwardhan
Branimir Boguraev
ABSTRACT
State-of-the-art approaches to token labeling within text documents typically cast the problem either as a classification task, without using complex structural characteristics of the input, or as a sequential labeling task, carried out by a Conditional Random Field (CRF) classifier. Here we explore principled ways for structure to be brought to bear on the task. In line with recent trends in statistical learning of structured natural language input, we use a Support Vector Machine (SVM) classification framework deploying tree kernels. We then propose tree transformations and decorations, as a methodology for modeling complex linguistic phenomena in highly multi-dimensional feature spaces. We develop a general purpose tree engineering framework, which enables us to transcend the typically complex and laborious process of feature engineering. We build kernel based classifiers for two token labeling tasks: fine-grained event recognition, and lexical answer type detection in questions. For both, we show that in comparison with a corresponding linear kernel SVM, our method of using tree kernels improves recognition, thanks to appropriately engineering tree structures for use by the tree kernel. We also observe significant improvements when comparing with a CRF-based realization of structured prediction, itself performing at levels comparable to state-of-the-art.
- R. K. Ando. Exploiting Unannotated Corpora for Tagging and Chunking. In The Companion Volume to the Proceedings of 42st Annual Meeting of the Association for Computational Linguistics, pages 142--145, Barcelona, Spain, July 2004. Google Scholar
Digital Library
- S. Bethard and J. H. Martin. Identification of Event Mentions and their Semantic Class. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 146--154, Sydney, Australia, July 2006. Google ScholarDigital Library
- B. Boguraev and R. K. Ando. TimeML-compliant Text Analysis for Temporal Reasoning. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pages 997--1003, Edinburgh, Scotland, August 2005. Google ScholarDigital Library
- B. Boguraev and R. K. Ando. Analysis of TimeBank as a Resource for TimeML Parsing. In Proceedings of the Fifth International Conference on Language Resources and Evaluation, pages 71--76, Genoa, Italy, May 2006.Google Scholar
- W. Cohen. MinorThird: Methods for Identifying Names and Ontological Relations in Text using Heuristics for Inducing Regularities from Data. http://minorthird.sourceforge.net, 2004.Google Scholar
- M. Collins and N. Duffy. Convolution kernels for Natural Language. In Advances in Neural Information Processing Systems, Vancouver, Canada, December 2001.Google Scholar
- D. Croce, A. Moschitti, and R. Basili. Structured Lexical Similarity via Convolution Kernels on Dependency Trees. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1034--1046, Edinburgh, Scotland, July 2011. Google ScholarDigital Library
- A. Culotta and J. Sorensen. Dependency Tree Kernels for Relation Extraction. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL'04), pages 423--429, Barcelona, Spain, July 2004. Google ScholarDigital Library
- D. Haussler. Convolution Kernels on Discrete Structures. Technical Report UCSC-CRL-99--10, University of California at Santa Cruz, July 1999.Google Scholar
- V. Kecman. Learning and Soft Computing. The MIT Press, Cambridge, MA, 2001.Google Scholar
- T. Kudo and Y. Matsumoto. Chunking with Support Vector Machines. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics}, pages 192--199, Pittsburgh, PA, June 2001. Google ScholarDigital Library
- J. Lafferty, A. McCallum, and F. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. Proceedings of the 18th International Conference on Machine Learning, pages 282--289, June 2001. Google ScholarDigital Library
- A. Lally, J. Prager, M. McCord, B. Boguraev, S. Patwardhan, J. Fan, P. Fodor, and J. Chu-Carroll. Question Analysis: How Watson Reads a Clue. IBM Journal of Research and Development, 56(3/4):2:1--2:14, May/July 2012. Google ScholarDigital Library
- X. Li and D. Roth. Learning Question Classifiers: The Role of Semantic Information. In Proceedings of the 19th International Conference on Computational Linguistics, pages 556--562, Taipei, Taiwan, August 2004. Google ScholarDigital Library
- H. Llorens, E. Saquete, and B. Navarro-Colorado. TimeML Events Recognition and Classification: Learning CRF Models with Semantic Roles. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pages 725--733, Beijing, China, August 2010. Google ScholarDigital Library
- L. Marquez and A. Moschitti. Special Issue on Statistical Learning of Natural Language Structured Input and Output. Natural Language Engineering, 18(2):147--153, April 2012. Google ScholarDigital Library
- M. McCord. Slot Grammar: A System for Simpler Construction of Practical Natural Language Grammars. In Proceedings of the International Symposium on Natural Language and Logic, pages 118--145, Hamburg, Germany, May 1989. Google ScholarDigital Library
- Q. McNemar. Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages. Psychometrika, 12(2):153--157, 1947.Google ScholarCross Ref
- A. Moschitti. A Study on Convolution Kernels for Shallow Statistic Parsing. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL'04), pages 335--342, Barcelona, Spain, July 2004. Google ScholarDigital Library
- A. Moschitti. Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees. In Proceedings of the 17th European Conference on Machine Learning, pages 318--329, Berlin, Germany, September 2006. Google ScholarDigital Library
- A. Moschitti, D. Pighin, and R. Basili. Tree Kernels for Semantic Role Labeling. Computational Linguistics, 34(2):193--224, June 2008. Special Issue on Semantic Role Labeling. Google ScholarDigital Library
- A. Moschitti, S. Quarteroni, R. Basili, and S. Manandhar. Exploiting Syntactic and Shallow Semantic Kernels for Question Answer Classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 776--783, Prague, Czech Republic, June 2007.Google Scholar
- T.-V. T. Nguyen, A. Moschitti, and G. Riccardi. Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1378--1387, Singapore, August 2009. Google ScholarDigital Library
- L. Ramshaw and M. Marcus. Text Chunking using Transformation-Based Learning. In Proceedings of Third Annual Workshop on Very Large Corpora, pages 82--94, Cambridge, MA, June 1995.Google Scholar
- R. Sauri, R. Knippen, M. Verhagen, and J. Pustejovsky. Evita: A Robust Event Recognizer For QA Systems. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 700--707, Vancouver, Canada, October 2005. Google ScholarDigital Library
- V. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, NY, 1995. Google ScholarDigital Library
- L. Wang, editor. Support Vector Machines: Theory and Applications. Springer, Berlin, Germany, 2005. Google ScholarDigital Library
- D. Zelenko, C. Aone, and A. Richardella. Kernel Methods for Relation Extraction. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, pages 71--78, Philadelphia, PA, July 2002. Google ScholarDigital Library
- M. Zhang, J. Zhang, and J. Su. Exploring Syntactic Features for Relation Extraction using a Convolution Tree Kernel. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 288--295, New York City, USA, June 2006. Google ScholarDigital Library
Index Terms
- Labeling by landscaping: classifying tokens in context by pruning and decorating trees
Recommendations
Extending twin support vector machine classifier for multi-category classification problems
Twin support vector machine classifier TWSVM was proposed by Jayadeva et al., which was used for binary classification problems. TWSVM not only overcomes the difficulties in handling the problem of exemplar unbalance in binary classification problems, ...
A subpath kernel for rooted unordered trees
PAKDD'11: Proceedings of the 15th Pacific-Asia conference on Advances in knowledge discovery and data mining - Volume Part IKernel method is one of the promising approaches to learning with tree-structured data, and various efficient tree kernels have been proposed to capture informative structures in trees. In this paper, we propose a new tree kernel function based on "...
All-in-one multicategory least squares nonparallel hyperplanes support vector machine
A least square version of nonparallel hyperplane support vector machine (LSNHSVM) is proposed.The proposed LSNHSVM is obtained by solving a system of linear equations.LSNHSVM is further generalized to solve multiclass classification problem ...
Comments