Skip to main content
SpringerLink
Log in

Early Approaches to Anaphora Resolution: Theoretically Inspired and Heuristic-Based

  • Chapter
  • First Online:
Anaphora Resolution

Abstract

This chapter summarizes the most influential non-statistical approaches to anaphora resolution. Much of the very early work focused on personal pronouns and was based on theoretical proposals concerning anaphora and its interpretation developed in linguistics (e.g., the effect of syntax or semantics on anaphora) and/or psychology (e.g., on the effect of salience or commonsense knowledge). Such systems assumed the resolver would have perfect information available – e.g., on the syntactic structure of the sentence, or the properties of concepts and instances – and as a result, tended to be very brittle (a notable exception being Hobbs’ ‘naive’ algorithm for pronoun resolution). In the first part of this chapter we cover in detail some of these theoretically-motivated algorithms, such as Hobbs’ and Sidner’s, and briefly survey a number of other ones. The availability of the first corpora in the mid-1990s (see chapter “Annotated Corpora and Annotation Tools”) led to the development of the first systems able to operate on a larger scale, and to a widening of the range of anaphoric expressions handled. The fundamental property of these systems was the ability to carry out resolution on the basis of imperfect information only, using a variety of heuristics. In the second part of this chapter, we cover a number of these heuristic-based algorithms. Some of the ideas developed in these heuristic-based systems have come back and are the basis for systems developed in the last few years; of these, we will discuss in some detail the Stanford Deterministic Coreference System.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In Preference Semantics, semantics is expressed in terms of a small number of semantic primitives like FORCE.

  2. 2.

    In which the algorithm discussed in Sect. 2 and since known as “Hobbs’ algorithm” was in fact presented as a baseline against which to evaluate the more sophisticated algorithm using commonsense knowledge.

  3. 3.

    For an alternative account of the inference process leading to the establishment of coherence relations (although, to our knowledge, not of example (2)) see [4]. Systems making heavy use of such inferences for natural language interpretation were actually implemented by SRI, some of which also participated at the early muc competitions, see e.g., [2, 32].

  4. 4.

    rosana = Ro bust S yntax-Based Interpretation of Ana phoric Expressions.

  5. 5.

    The FDG parser is the predecessor of the commercially available Connexor Machinese Syntax parser (www.connexor.com).

  6. 6.

    Notational conventions: round brackets delimit constituents; square brackets emphasize fragment (= parse subtree) boundaries.

  7. 7.

    Between fragments named F d and F e , an embedding relation is assumed, requiring that the parser provides the additional information that the latter fragment is subordinated to the former.

  8. 8.

    The two additional basic patterns that are employed in step 1(b)v for verifying the i-within-i condition of BT are specified in Stuckardt (2001: [72])

  9. 9.

    See www.stuckardt.de/index.php/anaphernresolution.html for details about the distribution; there is as well an implementation available for the German language, which works on the output of the Connexor Machinese Syntax parser.

  10. 10.

    Sentences and mentions are gold, extracted from the Penn Treebank annotation. The mentions and heuristically aligned with the output of a ne recognizer.

  11. 11.

    Soon et al.’s system [68], the first successful machine learning approach, discussed in chapter “The Mention-Pair Model”, obtained an F score of 0.63 for this dataset. As we will see in the rest of this chapter and in the following chapters of the book, it is still the case for coreference that a rule-based system can achieve state-of-the-art performance.

  12. 12.

    Soon et al.’s system obtained an F of 0.605.

  13. 13.

    This figure cannot be compared to the figures obtained by Vieira and Poesio, because the latter evaluate the resolution accuracy for definite descriptions, whereas Kameyama’s evaluation requires both correct identification of a discourse-old noun phrase and the identification of the correct antecedent to be counted.

  14. 14.

    http://nlp.stanford.edu/software/dcoref.shtml

  15. 15.

    The CoNLL coreference shared tasks are discussed in detail in chapter “Evaluation Campaigns”.

  16. 16.

    http://nlp.stanford.edu/software/corenlp.shtml

References

  1. Alshawi, H.: Memory and Context for Language Interpretation. Cambridge University Press, Cambridge (1987)

    Google Scholar 

  2. Alshawi, H. (ed.): The Core Language Engine. MIT, Cambridge (1992)

    Google Scholar 

  3. Appelt, D.E., Hobbs, J.R., Bear, J., Israel, D., Kameyama, M., Tyson, M.: Fastus: a finite-state processor for information extraction from real-world text. In: Proceedings of IJCAI, Chambery (1993)

    Google Scholar 

  4. Asher, N., Lascarides, A.: The Logic of Conversation. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  5. Baldwin, B.: Cogniac: a high precision pronoun resolution engine. In: Proceedings of the ACL’97/EACL’97 Workshop on Operational Factors in Practical, Robust Anaphora Resolution, Madrid, pp. 38–45 (1997)

    Google Scholar 

  6. Brennan, S., Friedman, M., Pollard, C.: A centering approach to pronouns. In: Proceedings of the 25th ACL, Stanford, pp. 155–162 (1987)

    Google Scholar 

  7. Carletta, J.: Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist. 22 (2), 249–254 (1996)

    Google Scholar 

  8. Carter, D.M.: Interpreting Anaphors in Natural Language Texts. Ellis Horwood, Chichester (1987)

    Google Scholar 

  9. Charniak, E.: Towards a model of children’s story comprehension. Ph.D. thesis, MIT (1972). Available as MIT AI Lab TR-266

    Google Scholar 

  10. Charniak, E.: Organization and inference in a frame-like system of commonsense knowledge. In: Proceedings of TINLAP, Cambridge, pp. 42–51 (1975)

    Google Scholar 

  11. Clark, H.H.: Bridging. In: Schank, R.C., Nash-Webber, B.L. (eds.) Proceedings of the 1975 Workshop on Theoretical Issues in Natural Language Processing, pp. 169–174. Association for Computing Machinery, Cambridge (1975)

    Chapter  Google Scholar 

  12. Dale, R.: Generating Referring Expressions. MIT, Cambridge (1992)

    Google Scholar 

  13. Evans, R.: Applying machine learning toward an automatic classification of it. Lit. Linguist. Comput. 16 (1), 45–57 (2001)

    Article  Google Scholar 

  14. Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT, Cambridge (1998)

    MATH  Google Scholar 

  15. Gaizauskas, R., Wakao, T., Humphreys, K., Cunningham, H., Wilks, Y.: University of Sheffield: description of the LaSIE System as used for MUC-6. In: Proceedings of the Sixth Message Understanding Conference (MUC-6), pp. 207–220. Morgan Kauffmann, San Francisco (1995)

    Google Scholar 

  16. Garnham, A.: Mental Models and the Interpretation of Anaphora. Psychology Press, Hove (2001)

    Google Scholar 

  17. Garvey, C., Caramazza, A.: Implicit causality in verbs. Linguist. Inq. 5, 459–464 (1974)

    Google Scholar 

  18. Ge, N., Hale, J., Charniak, E.: A statistical approach to anaphora resolution. In: Proceedings of WVLC/EMNLP (1998)

    Google Scholar 

  19. Gordon, P.C., Grosz, B.J., Gillion, L.A.: Pronouns, names, and the centering of attention in discourse. Cogn. Sci. 17, 311–348 (1993)

    Article  Google Scholar 

  20. Grishman, R., Sundheim, B.: Design of the MUC-6 evaluation. In: Proceedings of the Sixth Message Understanding Conference (MUC-6), Columbia (1995)

    Google Scholar 

  21. Grosz, B.J.: The representation and use of focus in dialogue understanding. Ph.D. thesis, Stanford University (1977)

    Google Scholar 

  22. Grosz, B.J., Joshi, A.K., Weinstein, S.: Centering: a framework for modeling the local coherence of discourse. Comput. Linguist. 21 (2), 202–225 (1995). The paper originally appeared as an unpublished manuscript in 1986

    Google Scholar 

  23. Hawkins, J.: Definiteness and Indefiniteness. Croom Helm, London (1978)

    Google Scholar 

  24. Heim, I.: The semantics of definite and indefinite noun phrases. Ph.D. thesis, University of Massachusetts at Amherst (1982)

    Google Scholar 

  25. Hewitt, C.: Planner: a language for proving theorems in robots. In: Proceedings of IJCAI, Washington DC, pp. 295–302 (1969)

    Google Scholar 

  26. Hirschman, L., Chinchor, N.: MUC-7 coreference task definition (version 3.0). In: Proceedings of the 7th Message Understanding Conference. http://www-nlpir.nist.gov/related_projects/muc/proceedings/co_task.html (1997)

  27. Hirst, G.: Discourse-oriented anaphora resolution: a review. Comput. Linguist. 7, 85–98 (1981)

    Google Scholar 

  28. Hobbs, J.: Resolving pronoun references. Lingua 44, 311–338 (1978)

    Article  Google Scholar 

  29. Hobbs, J.R.: Pronoun resolution. Research Note 76-1, City College, City University of New York (1976)

    Google Scholar 

  30. Hobbs, J.R.: Coherence and coreference. Cogn. Sci. 3, 67–90 (1979)

    Article  Google Scholar 

  31. Hobbs, J.R., Martin, P.: Local pragmatics. In: Proceedings of IJCAI-87, Milano, pp. 520–523 (1987)

    Google Scholar 

  32. Hobbs, J.R., Appelt, D.E., Bear, J., Tyson, M., Magerman, D.: The TACITUS system: the muc-3 experience. SRI Technical Note 511, SRI International, Menlo Park (1991)

    Google Scholar 

  33. Hobbs, J.R., Stickel, M., Appelt, D., Martin, P.: Interpretation as abduction. Artif. Intell. 63, 69–142 (1993)

    Article  Google Scholar 

  34. Humphreys, K., Gaizauskas, R., Azzam, S., Huyck, C., Mitchell, B., Cunningham, H., Wilks, Y.: University of Sheffield: description of the LaSIE-II system as used for muc-7. In: Proceedings of MUC-7, Fairfax (1998)

    Google Scholar 

  35. Järvinen, T., Tapanainen, P.: A dependency parser for English. Technical report TR-1, Department of General Linguistics, University of Helsinki (1997)

    Google Scholar 

  36. Kameyama, M.: Zero anaphora: the case of Japanese. Ph.D. thesis, Stanford University, Stanford (1985)

    Google Scholar 

  37. Kameyama, M.: Recognizing referential links: an information extraction perspective. In: ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts (1997)

    Book  Google Scholar 

  38. Kamp, H., Reyle, U.: From Discourse to Logic. Kluwer Academic, Dordrecht (1993)

    MATH  Google Scholar 

  39. Kantor, R.N.: The management and comprehension of discourse connection by pronouns in English. Ph.D. thesis, Department of Linguistics, Ohio State University (1977)

    Google Scholar 

  40. Karamanis, N., Poesio, M., Oberlander, J., Mellish, C.: Evaluating centering for information ordering using corpora. Comput. Linguist. 35 (1), 29–46 (2009)

    Article  Google Scholar 

  41. Kehler, A., Kertz, L., Rohde, H., Elman, J.: Coherence and coreference revisited. J. Semant. 25 (1), 1–44 (2008)

    Article  Google Scholar 

  42. Kennedy, C., Boguraev, B.: Anaphora for everyone: pronominal anaphora resolution without a parser. In: COLING 1996, Copenhagen (1996)

    Google Scholar 

  43. Kennedy, C., Boguraev, B.: Anaphora for everyone: pronominal anaphora resolution without a parser. In: Proceedings of the 16th International Conference on Computational Linguistics (COLING), Copenhagen, pp 113–118 (1996)

    Google Scholar 

  44. Kibble, R., Power, R.: An integrated framework for text planning and pronominalization. In: Proceedings of the International Conference on Natural Language Generation (INLG), Mitzpe Ramon (2000)

    Google Scholar 

  45. Lappin, S., Leass, H.: An algorithm for pronominal anaphora resolution. Comput. Linguist. 20 (4), 535–561 (1994)

    Google Scholar 

  46. Lee, H., Peirsman, Y., Chang, A., Chambers, N., Surdeanu, M., Jurafsky, D.: Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In: Proceedings of the CoNLL 2011 Shared Task, Portland (2011)

    Google Scholar 

  47. Lee, H., Chang, A., Peirsman, Y., Chambers, N., Surdeanu, M., Jurafsky, D.: Deterministic coreference resolution based on entity-centric, precision-ranked rules. Comput. Linguist. 39 (4), 885–916 (2013)

    Article  Google Scholar 

  48. Linde, C.: Focus of attention and the choice of pronouns in discourse. In: Givon, T. (ed.) Syntax and Semantics, vol. 12. Academic, New York/London (1979)

    Google Scholar 

  49. Löbner, S.: Definites. J. Semant. 4, 279–326 (1985)

    Article  Google Scholar 

  50. Matthews, A., Chodorow, M.S.: Pronoun resolution in two-clause sentences: effects of ambiguity, antecedent location, and depth of embedding. J. Mem. Lang. 27, 245–260 (1988)

    Article  Google Scholar 

  51. Miltsakaki, E.: Towards an aposynthesis of topic continuity and intrasentential anaphora. Comput. Linguist. 28 (3), 319–355 (2002)

    Article  Google Scholar 

  52. Minsky, M.: A framework for representing knowledge. In: Winston, P.H. (ed.) The Psychology of Computer Vision. McGraw-Hill, New York, pp. 211–277 (1975)

    Google Scholar 

  53. Mitkov, R.: Robust pronoun resolution with limited knowledge. In: Proceedings of the 18th COLING, Montreal, pp. 869–875 (1998)

    Google Scholar 

  54. Mitkov, R.: Anaphora Resolution. Longman, London/New York (2002)

    MATH  Google Scholar 

  55. Mitkov, R.: Anaphora resolution. In: Mitkov, R. (ed.) Oxford Handbook of Computational Linguistics. Oxford University Press, Oxford (2005)

    Google Scholar 

  56. Poesio, M., Kabadjov, M.A.: A general-purpose, off-the-shelf anaphora resolution module: implementation and preliminary evaluation. In: LREC 2004. http://privatewww.essex.ac.uk/~malexa/html_files/files/LREC2004.pdf (2004)

  57. Poesio, M., Vieira, R.: A corpus-based investigation of definite description use. Comput. Linguist. 24 (2), 183–216 (1998)

    Google Scholar 

  58. Poesio, M., Vieira, R., Teufel, S.: Resolving bridging descriptions in unrestricted text. In: ACL-97 Workshop on Operational Factors in Practical, Robust, Anaphora Resolution For Unrestricted Texts (1997)

    Google Scholar 

  59. Poesio, M., Stevenson, R., Di Eugenio, B., Hitzeman, J.M.: Centering: a parametric theory and its instantiations. Comput. Linguist. 30 (3), 309–363 (2004)

    Article  Google Scholar 

  60. Prince, E.F.: Toward a taxonomy of given-new information. In: Cole, P. (ed.) Radical Pragmatics, pp. 223–256. Academic, New York (1981)

    Google Scholar 

  61. Prince, E.F.: The ZPG letter: subjects, definiteness and information-status. In: Thompson, S., Mann, W. (eds.) Discourse Description: Diverse Analyses of a Fund Raising Text. John Benjamins, Amsterdam (1992)

    Google Scholar 

  62. Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1 (1), 81–106 (1986)

    Google Scholar 

  63. Raghunathan, K., Lee, H., Rangarajan, S., Chambers, N., Surdeanu, M., Jurafsky, D., Manning, C.: A multi-pass sieve for coreference resolution. In: Proceedings of EMNLP, pp. 492–501. MIT, Boston (2010)

    Google Scholar 

  64. Reichman, R.: Getting Computers to Talk Like You and Me. MIT, Cambridge (1985)

    Google Scholar 

  65. Sanford, A.J., Garrod, S.C.: Understanding Written Language. Wiley, Chichester (1981)

    Google Scholar 

  66. Sidner, C.L.: Towards a computational theory of definite anaphora comprehension in English discourse. Ph.D. thesis, MIT (1979)

    Google Scholar 

  67. Sidner, C.L.: Focusing in the comprehension of definite anaphora. In: Brady, M., Berwick, R. (eds.) Computational Models of Discourse. MIT, Cambridge (1983)

    Google Scholar 

  68. Soon, W.M., Ng, H.T., Lim, D.C.Y.: A machine learning approach to coreference resolution of noun phrases. Comput. Linguist. 27 (4), 521–544 (2001). http://acl.eldoc.ub.rug.nl/mirror/J/J01/J01-4004.pdf

    Article  Google Scholar 

  69. Stevenson, R.J., Crawley, R.A., Kleinman, D.: Thematic roles, focus, and the representation of events. Lang. Cogn. Process. 9, 519–548 (1994)

    Article  Google Scholar 

  70. Strube, M.: Never look back: an alternative to centering. In: Proceedings of COLING-ACL, Montreal, pp. 1251–1257 (1998)

    Google Scholar 

  71. Strube, M., Hahn, U.: Functional centering–grounding referential coherence in information structure. Comput. Linguist. 25 (3), 309–344 (1999)

    Google Scholar 

  72. Stuckardt, R.: Design and enhanced evaluation of a robust anaphor resolution algorithm. Comput. Linguist. 27 (4), 479–506 (2001)

    Article  Google Scholar 

  73. Sundheim, B.M.: Overview of the results of the MUC-6 evaluation. In: Proceedings of the Sixth Message Understanding Conference (MUC-6), Columbia, pp. 13–31 (1995)

    Google Scholar 

  74. Suri, L.Z., McCoy, K.F.: RAFT/RAPR and centering: a comparison and discussion of problems related to processing complex sentences. Comput. Linguist. 20 (2), 301–317 (1994)

    Google Scholar 

  75. Tetrault, J.: A corpus-based evaluation of centering and pronoun resolution. Comput. Linguist. 27 (4), 507–520 (2001)

    Article  Google Scholar 

  76. Vieira, R.: Definite description resolution in unrestricted texts. Ph.D. thesis, Centre for Cognitive Science, University of Edinburgh (1998)

    Google Scholar 

  77. Vieira, R., Poesio, M.: Corpus-based approaches to NLP: a practical prototype. In: Anais do XVI Congresso da Sociedade Brasileira de Computa cão (1996)

    Google Scholar 

  78. Vieira, R., Poesio, M.: Processing definite descriptions in corpora. In: Botley, S., McEnery, M. (eds.) Corpus-Based and Computational Approaches to Discourse Anaphora. UCL Press, London (1997)

    Google Scholar 

  79. Vieira, R., Poesio, M.: Corpus-based development and evaluation of a system for processing definite descriptions. In: Proceedings of 18th COLING, Saarbruecken (2000)

    Google Scholar 

  80. Vieira, R., Poesio, M.: An empirically based system for processing definite descriptions. Comput. Linguist. 26 (4), 539–593 (2000)

    Article  Google Scholar 

  81. Vieira, R., Teufel, S.: Towards resolution of bridging descriptions. In: ACL-EACL, Madrid (1997)

    Google Scholar 

  82. Vilain, M., Burger, J., Aberdeen, J., Connolly, D., Hirschman, L.: A model-theoretic coreference scoring scheme. In: Proceedings of the 6th Message Understanding Conference (MUC-6), pp. 45–52. Morgan Kaufmann, San Francisco (1996). doi:http://dx.doi.org/10.3115/1072399.1072405

  83. Walker, M.A.: Evaluating discourse processing algorithms. In: Proceedings of ACL, Manchester, pp. 251–261 (1989)

    Google Scholar 

  84. Walker, M.A., Iida, M., Cote, S.: Japanese discourse and the process of centering. Comput. Linguist. 20 (2), 193–232 (1994)

    Google Scholar 

  85. Walker, M.A., Joshi, A.K., Prince, E.F. (eds.): Centering Theory in Discourse. Clarendon Press, Oxford (1998)

    MATH  Google Scholar 

  86. Wilks, Y.A.: An intelligent analyzer and understander of English. Commun. ACM 18 (5), 264–274 (1975). Reprinted in Readings in Natural Language Processing, Morgan Kaufmann

    Google Scholar 

  87. Wilks, Y.A.: A preferential pattern-matching semantics for natural language. Artif. Intell. J. 6, 53–74 (1975)

    Article  MATH  Google Scholar 

  88. Winograd, T.: Understanding Natural Language. Academic, New York (1972)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Essex, Colchester, UK

    Massimo Poesio

  2. IT-Beratung, Sprachtechnologie, Medienanalyse, D-60433, Frankfurt am Main, Germany

    Roland Stuckardt

  3. Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany

    Yannick Versley

  4. Universidade Católica do Rio Grande do Sul, Porto Alegre, Brazil

    Renata Vieira

Authors
  1. Massimo Poesio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roland Stuckardt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yannick Versley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renata Vieira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Massimo Poesio .

Editor information

Editors and Affiliations

  1. Trento, Italy

    Massimo Poesio

  2. Frankfurt am Main, Germany

    Roland Stuckardt

  3. Heidelberg, Germany

    Yannick Versley

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Poesio, M., Stuckardt, R., Versley, Y., Vieira, R. (2016). Early Approaches to Anaphora Resolution: Theoretically Inspired and Heuristic-Based. In: Poesio, M., Stuckardt, R., Versley, Y. (eds) Anaphora Resolution. Theory and Applications of Natural Language Processing. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-47909-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-47909-4_3

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-47908-7

  • Online ISBN: 978-3-662-47909-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation