Skip to main content
SpringerLink
Log in

Beyond sentence-level semantic role labeling: linking argument structures in discourse

  • Original Paper
  • Published:
Language Resources and Evaluation Aims and scope Submit manuscript

Abstract

Semantic role labeling is traditionally viewed as a sentence-level task concerned with identifying semantic arguments that are overtly realized in a fairly local context (i.e., a clause or sentence). However, this local view potentially misses important information that can only be recovered if local argument structures are linked across sentence boundaries. One important link concerns semantic arguments that remain locally unrealized (null instantiations) but can be inferred from the context. In this paper, we report on the SemEval 2010 Task-10 on “Linking Events and Their Participants in Discourse”, that addressed this problem. We discuss the corpus that was created for this task, which contains annotations on multiple levels: predicate argument structure (FrameNet and PropBank), null instantiations, and coreference. We also provide an analysis of the task and its difficulties.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://framenet.icsi.berkeley.edu/.

  2. http://verbs.colorado.edu/~mpalmer/projects/ace.html.

  3. http://etext.virginia.edu.

  4. After the completion of the shared task, information about negation was added as another annotation layer to the corpus (Morante et al. 2011). Since the current article focuses on the SemEval-2010 Shared Task on “Linking Events and Their Participants in Discourse”, we will not provide information about the negation annotation here.

  5. PropBank annotations were created semi-automatically using the FrameNet annotations as a starting point (see Sect. 2.3).

  6. http://www.itl.nist.gov/iad/894.02/related_projects/muc/proceedings/co_task.html.

  7. By whole phrases we refer to whole phrases according to human understanding. The nodes we label don’t need to cover complete, single phrases in the imperfect syntax trees we use.

  8. http://www.coli.uni-saarland.de/projects/salsa/salto/request/salsa-tool-request.cgi.

  9. http://verbs.colorado.edu/semlink/.

  10. Palmer et al.’s (1986) treatment of uninstantiated ‘essential roles’ is very similar (see also Palmer 1990).

  11. A more common but idiomatic formulation that conveys a related metaphorical notion of ending is “lay to rest”.

  12. The FrameNet group added a small number of frames for our shared task. These were then part of FrameNet 1.4 alpha, the special release that was made available for task participants.

  13. We allowed annotators to make these assignments as we were in touch with the FrameNet team and new lexical units were included in the FrameNet 1.4 alpha release provided to the task participants.

  14. We provide the comparison for each pair in only one direction. The numbers for the other direction can be found by simply switching recall and precision.

  15. Given that word sense disambiguation had to be performed and that the rates of FE omission were not that high for the two predicates, finding these 200 instances involved inspecting many more instances than those 200 retained at the end.

  16. The annotation was done on plain text in a simple text-editor rather than in Salto on top of a parse tree, which made it more tiring and difficult.

  17. Note that we discuss the usefulness of the above heuristics only in relation to system-building. These heuristics were not used in the manual annotation of the training and test data, which was instance-based and context-sensitive.

References

  • Baker, C., Ellsworth, M., & Erk, K. (2007). SemEval-2007 Task 19: Frame semantic structure extraction. In Proceedings of SemEval-07.

  • Burchardt, A., Frank, A., & Pinkal, M. (2005). Building text meaning representations from contextually related frames—A case study. In Proceedings of IWCS-6.

  • Burchardt, A., Erk, K., Frank, A., Kowalski, A., & Pado, S. (2006). Salto—A versatile multi-level annotation tool. In Proceedings of LREC 2006.

  • Campbell, R. (2004). Using linguistic principles to recover empty categories. In Proceedings of the 42nd annual meeting on association for computational linguistics (pp. 645–652). Association for Computational Linguistics.

  • Carreras, X., & Màrquez, L. (2004). Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of CoNLL-04 (pp. 89–97).

  • Carreras, X., & Màrquez, L. (2005). Introduction to the CoNLL-2005 shared task: Semantic role labeling. In Proceedings of CoNLL-05, (pp. 152–164).

  • Chen, D., Schneider, N., Das, D., & Smith, N. A. (2010). SEMAFOR: Frame argument resolution with log-linear models. In Proceedings of the 5th international workshop on semantic evaluation (pp. 264–267). Uppsala, Sweden: Association for computational linguistics.

  • Collins, M. (1997). Three generative, lexicalised models for statistical parsing. In Proceedings of ACL/EACL 1997 (pp. 16–23).

  • Das, D., Schneider, N., Chen, D., & Smith, N. A. (2010). Probabilistic frame-semantic parsing. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics (pp. 948–956).

  • Delmonte, R. (2008) Computational linguistic text processing—Lexicon, grammar, parsing and anaphora resolution. New York: Nova Science.

    Google Scholar 

  • Diab, M., Alkhalifa, M., ElKateb, S., Fellbaum, C., Mansouri, A., & Palmer, M. (2007). SemEval-2007 Task 18: Arabic semantic labeling. In Proceedings of SemEval-07.

  • Erk, K., & Padó, S. (2004). A powerful and versatile XML format for representing role-semantic annotation. In Proceedings of LREC-2004.

  • Erk, K., & Padó, S. (2006) Shalmaneser—A flexible toolbox for semantic role assignment. In Proceedings of LREC-06.

  • Fillmore, C. J. (1977). Scenes-and-frames semantics, linguistic structures processing. In A. Zampolli (Ed.), Fundamental studies in computer science, no. 59 (pp. 55–88). Amsterdam: North Holland Publishing.

    Google Scholar 

  • Fillmore, C. J. (1986). Pragmatically controlled zero anaphora. In Proceedings of the twelfth annual meeting of the Berkeley liguistics society.

  • Fillmore, C. J., & Baker, C. F. (2001). Frame semantics for text understanding. In Proceedings of the NAACL-01 workshop on WordNet and other lexical resources.

  • Gabbard, R., Marcus, M., & Kulick, S. (2006). Fully parsing the Penn Treebank. In Proceedings of the main conference on human language technology conference of the North American chapter of the association of computational linguistics (pp. 184–191). Association for Computational Linguistics.

  • Gerber, M., & Chai, J. Y. (2010). Beyond NomBank: a study of implicit arguments for nominal predicates. In Proceedings of the 48th annual meeting of the association for computational linguistics, ACL ’10 (pp. 1583–1592). Stroudsburg, PA: Association for Computational Linguistics. http://portal.acm.org/citation.cfm?id=1858681.1858841.

  • Gildea, D., & Jurafsky, D. (2002). Automatic labeling of semantic roles. Computational Linguistics 28(3), 245–288.

    Article  Google Scholar 

  • Iida, R., Inui, K., & Matsumoto, Y. (2007). Zero-anaphora resolution by learning rich syntactic pattern features. ACM Transactions on Asian Language Information Processing (TALIP), 6, 1:1–1:22. ISSN 1530-0226. http://doi.acm.org/10.1145/1316457.1316458.

  • Litkowski, K. (2004). SENSEVAL-3 Task: Automatic labeling of semantic roles. In Proceedings of SENSEVAL-3.

  • Litkowski, K. (2010). CLR: Linking events and their participants in discourse using a comprehensive FrameNet dictionary. In Proceedings of the 5th international workshop on semantic evaluation (pp. 300–303).

  • Liu, H., & Singh, P. (2004). ConceptNet: A practical commonsense reasoning toolkit. BT Technology Journal 22(4), 211–226.

    Article  Google Scholar 

  • Màrquez, L., Villarejo, L., Martí, M. A., & Taulé, M. (2007). SemEval-2007 Task 09: Multilevel semantic annotation of Catalan and Spanish. In Proceedings of SemEval-07.

  • Michaelis, L. (2006). Time and tense. In B. Aarts, & A. MacMahon (Eds.), The handbook of English linguistics, (pp. 220–234). Oxford: Blackwell.

    Chapter  Google Scholar 

  • Michaelis, L. (2010). The Oxford handbook of linguistic analysis, chapter sign-based construction grammar. Oxford: Oxford University Press.

    Google Scholar 

  • Morante, R., Schrauwen, S., & Daelemans, W. (2011). Annotation of negation cues and their scope guidelines v1.0. Technical Report CTR-003, CLiPS, University of Antwerp, Antwerp.

  • Palmer, M. (1990). Semantic processing for finite domains. Cambridge: CUP.

    Book  Google Scholar 

  • Palmer, M., Dahl, D., Passonneau, R., Hirschman, L., Linebarger, M., & Dowding, J. (1986). Recovering implicit information. In Proceedings of ACL-1986.

  • Palomar, M., Moreno, L., Peral, J., Muñoz, R., Ferrández, A., Martínez-Barco, P., & Saiz-Noeda, M. (2001). An algorithm for anaphora resolution in Spanish texts. Computational Linguistics, 27, 545–567. ISSN 0891-2017.

    Google Scholar 

  • Ruppenhofer, J., Gorinski, P., & Sporleder, C. (2011). In search of missing arguments: A linguistic approach. In Proceedings of the international conference recent advances in natural language processing 2011 (pp. 331–338).

  • Ruppenhofer, J., Sporleder, C., Morante, R., Baker, C., & Palmer, M. (2009) Semeval-2010 task 10: Linking events and their participants in discourse. In The NAACL-HLT 2009 workshop on semantic evaluations: Recent achievements and future directions (SEW-09).

  • Surdeanu, M., Johansson, R., Meyers, A., Màrquez, L., & Nivre, J. (2008). The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of CoNLL-2008 (pp. 159–177).

  • Tonelli, S., & Delmonte, R. (2010). Venses++: Adapting a deep semantic processing system to the identification of null instantiations. In Proceedings of the 5th international workshop on semantic evaluation (pp. 296–299). Uppsala: Association for Computational Linguistics.

  • Tonelli, S., & Delmonte, R. (2011). Desperately seeking implicit arguments in text. In Proceedings of the ACL 2011 workshop on relational models of semantics (pp. 54–62).

Download references

Acknowledgments

We would like to thank Collin Baker, Martha Palmer and Jinho D. Choi for their collaboration on the SemEval task. We are also grateful to our annotators Markus Dräger, Lisa Fuchs, and Corinna Schorr and to the anonymous reviewers for their insightful comments and useful feedback. Josef Ruppenhofer was supported by the German Research Foundation DFG under grant PI 154/9-3 and Caroline Sporleder as part of the Cluster of Excellence Multimodal Computing and Interaction (MMCI). Roser Morante’s research was funded by the GOA project BIOGRAPH of the University of Antwerp.

Author information

Authors and Affiliations

  1. University of Hildesheim, Marienburger Platz 22, 31141, Hildesheim, Germany

    Josef Ruppenhofer

  2. International Computer Science Institute, 1947 Center Street, Suite 600, Berkeley, CA, 94704, USA

    Russell Lee-Goldman

  3. Saarland University, PO Box 15 11 50, 66041, Saarbrücken, Germany

    Caroline Sporleder

  4. University of Antwerp, Prinsstraat 13, 2000, Antwerp, Belgium

    Roser Morante

Authors
  1. Josef Ruppenhofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Russell Lee-Goldman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Caroline Sporleder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roser Morante
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Caroline Sporleder.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ruppenhofer, J., Lee-Goldman, R., Sporleder, C. et al. Beyond sentence-level semantic role labeling: linking argument structures in discourse. Lang Resources & Evaluation 47, 695–721 (2013). https://doi.org/10.1007/s10579-012-9201-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10579-012-9201-4

Keywords

Search

Navigation