Abstract
The NLP community has shown a renewed interest in deeper semantic analyses, among them automatic recognition of semantic relations in text. We present the development and evaluation of a semantic analysis task: automatic recognition of relations between pairs of nominals in a sentence. The task was part of SemEval-2007, the fourth edition of the semantic evaluation event previously known as SensEval. Apart from the observations we have made, the long-lasting effect of this task may be a framework for comparing approaches to the task. We introduce the problem of recognizing relations between nominals, and in particular the process of drafting and refining the definitions of the semantic relations. We show how we created the training and test data, list and briefly describe the 15 participating systems, discuss the results, and conclude with the lessons learned in the course of this exercise.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
See http://purl.org/net/semeval/task4 for material related to Task 4.
Because this paper serves also as a documentation of the data set, the order of relations in the table is the same as in the data set.
References
Cafarella, M., Banko, M., & Etzioni, O. (2006). Relational web search. Technical Report 2006-04-02, University of Washington, Department of Computer Science and Engineering.
Girju, R. (2001). Answer fusion with on-line ontology development. In Proceedings of North American chapter of the association for computational linguistics (NAACL-01)—Student Research Workshop. Pittsburgh, PA.
Girju, R., Moldovan, D., Tatu, M., & Antohe, D. (2005). On the semantics of noun compounds. Computer Speech and Language, 19, 479–496.
Hearst, M. (1992). Automatic acquisition of hyponyms from large text corpora. In Proceedings on 14th international conference on computational linguistics (COLING-92). pp. 539–545.
Kim, S. N., & Baldwin, T. (2005). Automatic interpretation of noun compounds using WordNet similarity. In The international joint conference on natural language processing (IJCNLP). Jeju, Korea, pp. 945–956.
Labov, W. (1973). The boundaries of words and their meanings. In Variation in the form and use of language: A sociolinguistic reader. pp. 29–62.
Lapata, M. (2002). The disambiguation of nominalizations. Computational Linguistics, 28(3), 357–388.
Lapata, M., & Keller, F. (2005). Web-based models for natural language processing. ACM Transactions on Speech and Language Processing, 2, 1–31.
Lewis, D. (1991). Evaluating text categorization. In Proceedings of the speech and natural language workshop. Asilomar, pp. 312–318.
Moldovan, D., Badulescu, A., Tatu, M., Antohe, D., & Girju, R. (2004). Models for the semantic classification of noun phrases. In Proceedings of computational lexical semantics workshop at HLT-NAACL 2004 (pp. 60–67). Boston, MA.
Nakov, P., & Hearst, M. (2006). Using verbs to characterize noun-noun relations. In Proceedings on twelfth international conference in artificial intelligence (AIMSA-06) (pp. 233–244). Varna, Bulgaria.
Nastase, V., Sayyad-Shirabad, J., Sokolova, M., & Szpakowicz, S. (2006). Learning noun-modifier semantic relations with corpus-based and WordNet-based features. In Proceedings on 21st national conference on artificial intelligence (AAAI 2006) (pp. 781–787). Boston, MA.
Nastase, V., & Szpakowicz, S. (2003). Exploring noun-modifier semantic relations. In Fifth international workshop on computational semantics (IWCS-5) (pp. 285–301). Tilburg, The Netherlands.
Pantel, P., & Pennacchiotti, M. (2006). Espresso: Leveraging generic patterns for automatically harvesting semantic relations. In The international computational linguistics conference/association for computational linguistics meeting (COLING/ACL) (pp. 113–120). Sydney, Australia.
Pennacchiotti, M., & Pantel, P. (2006). Ontologizing semantic relations. In The international computational linguistics conference/association for computational linguistics meeting (COLING/ACL) (pp. 793–800). Sydney, Australia.
Rosario, B., & Hearst, M. (2001). Classifying the semantic relations in noun-compounds via domain-specific lexical hierarchy. In Proceedings on 2001 conference on empirical methods in natural language processing (EMNLP-01). pp. 82–90.
Rosario, B., Hearst, M., & Fillmore, C. (2002). The descent of hierarchy, and selection in relational semantics. In Proceedings on 40th annual meeting of the association for computational linguistics (ACL-02) (pp. 417–424). Philadelphia, PA.
Stephens, M., Palakal, M., Mukhopadhyay, S., & Raje, R. (2001). Detecting gene relations from MEDLINE abstracts. In Proceedings on sixth annual Pacific symposium on biocomputing. pp. 483–496.
Tatu, M., & Moldovan, D. (2005). A semantic approach to recognizing textual entailment. In Proceedings on human language technology conference and conference on empirical methods in natural language processing (HLT/EMNLP 2005) (pp. 371–378). Vancouver, Canada.
Turney, P. (2005). Measuring semantic similarity by latent relational analysis. In Proceedings on nineteenth international joint conference on artificial intelligence (IJCAI-05) (pp. 1136–1141). Edinburgh, Scotland.
Turney, P., & Littman, M. (2005). Corpus-based learning of analogies and semantic relations. Machine Learning, 60(1–3), 251–278.
Acknowledgments
We thank Eneko Agirre, Lluís Márquez and Richard Wicentowski, the organizers of SemEval-2007, for their guidance and prompt support in all organizational matters. Marti Hearst offered valuable advice throughout the task description and debates on definitions of semantic relations. We also thank all the participants in the SemEval-2007 Task 4 evaluation, without whom this research would not have been possible.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Girju, R., Nakov, P., Nastase, V. et al. Classification of semantic relations between nominals. Lang Resources & Evaluation 43, 105–121 (2009). https://doi.org/10.1007/s10579-009-9083-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10579-009-9083-2