Skip to main content
SpringerLink
Log in
Book cover

European Conference on Information Retrieval

ECIR 2017: Advances in Information Retrieval pp 212–225Cite as

How Do Order and Proximity Impact the Readability of Event Summaries?

  • Conference paper
  • First Online:

  • 2444 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10193))

Abstract

Organizing the structure of fixed-length text summaries for events is important for their coherence and readability. However, typical measures used for evaluation in text summarization tasks often ignore the structure. In this paper, we conduct an empirical study on a crowdsourcing platform to get insights into regularities that make a text summary coherent and readable. For this, we generate four variants of human-written text summaries with 10 sentences for 100 seminal events, and conduct three experiments. Experiment 1 and 2 focus on analyzing the impact of sentence ordering and proximity between originally occurring adjacent sentences, respectively. Experiment 3 analyzes the feasibility of conducting such a study on a crowdsourcing platform. We release our data to facilitate future work like designing dedicated measures to evaluate summary structures.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://en.wikipedia.org/wiki/Timeline_of_modern_history.

  2. 2.

    http://resources.mpi-inf.mpg.de/d5/txtCoherence.

References

  1. Crowdflower platform. https://www.crowdflower.com

  2. Alonso, O., Baeza-Yates, R.: Design and implementation of relevance assessments using crowdsourcing. In: Clough, P., Foley, C., Gurrin, C., Jones, G.J.F., Kraaij, W., Lee, H., Mudoch, V. (eds.) ECIR 2011. LNCS, vol. 6611, pp. 153–164. Springer, Heidelberg (2011). doi:10.1007/978-3-642-20161-5_16

    Chapter  Google Scholar 

  3. Barzilay, R., et al.: Using lexical chains for text summarization. In: Advances in Automatic Text Summarization, pp. 111–121. MIT Press, Cambridge (1999)

    Google Scholar 

  4. Barzilay, R., et al.: Inferring strategies for sentence ordering in multidocument news summarization. J. Artif. Intell. Res. 17, 35–55 (2002)

    MATH  Google Scholar 

  5. Barzilay, R., et al.: Sentence ordering in multidocument summarization. In: HLT 2001

    Google Scholar 

  6. Baumel, T., et al.: Query-chain focused summarization. In: ACL 2014

    Google Scholar 

  7. Bollegala, D., et al.: A preference learning approach to sentence ordering for multi-document summarization. Inf. Sci. 217, 78–95 (2012)

    Article  Google Scholar 

  8. Donghong, J., et al.: Sentence ordering based on cluster adjacency in multi-document summarization. In: IJCNLP 2008

    Google Scholar 

  9. Hobbs, J.R.: Literature and cognition. No. 21, Center for the Study of Language (1990)

    Google Scholar 

  10. Hu, P., Ji, D., et al.: Query-focused multi-document summarization using co-training based semi-supervised learning. In: PACLIC 2009

    Google Scholar 

  11. Hume, D.: Philosophical Essays Concerning Human Understanding. Georg Olms Verlag, Hildesheim (1750)

    Google Scholar 

  12. Ji, P.D., et al.: Sentence ordering with manifold-based classification in multi-document summarization. In: ECML 2006

    Google Scholar 

  13. Jing, H.: Summary generation through intelligent cutting and pasting of the input document. Technical report, Columbia University (1998)

    Google Scholar 

  14. Kaisser, M., et al.: Improving search results quality by customizing summary lengths. In: ACL 2008

    Google Scholar 

  15. Lapata, M.: Probabilistic text structuring: experiments with sentence ordering. In: ACL 2003

    Google Scholar 

  16. Li, Y., et al.: Query-focused multi-document summarization: combining a topic model with graph-based semi-supervised learning. In: COLING 2014

    Google Scholar 

  17. Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. In: ACL-2004 Workshop (2004)

    Google Scholar 

  18. Litkowski, K.C.: Summarization experiments in DUC 2004. In: DUC 2004

    Google Scholar 

  19. Lloret, E., et al.: Analyzing the capabilities of crowdsourcing services for text summarization. Lang. Resour. Eval. 47(2), 337–369 (2013)

    Article  Google Scholar 

  20. Nenkova, A., et al.: Evaluating content selection in summarization: the pyramid method. In: HLT-NAACL 2004

    Google Scholar 

  21. Ono, K., et al.: Abstract generation based on rhetorical structure extraction. In: NLP/CL 1994

    Google Scholar 

  22. Radev, D.R., et al.: Evaluation challenges in large-scale document summarization. In: ACL 2003

    Google Scholar 

  23. Salton, G., et al.: Automatic text structuring and summarization. Inf. Process. Manage. 33(2), 193–207 (1997)

    Article  Google Scholar 

  24. Sorokin, A., et al.: Utility data annotation with Amazon mechanical Turk. Urbana 51(61), 820 (2008)

    Google Scholar 

  25. Steinberger, J., et al.: Evaluation measures for text summarization. Comput. Inform. 28(2), 251–275 (2012)

    MATH  Google Scholar 

  26. Tang, J., Sanderson, M.: Evaluation and user preference study on spatial diversity. In: Gurrin, C., He, Y., Kazai, G., Kruschwitz, U., Little, S., Roelleke, T., Rüger, S., Rijsbergen, K. (eds.) ECIR 2010. LNCS, vol. 5993, pp. 179–190. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12275-0_18

    Chapter  Google Scholar 

  27. Zhong, S., et al.: Query-oriented unsupervised multi-document summarization via deep learning model. Expert Syst. Appl. 42(21), 8146–8155 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Max Planck Institute for Informatics, Saarbrücken, Germany

    Arunav Mishra & Klaus Berberich

  2. htw saar, Saarbrücken, Germany

    Klaus Berberich

Authors
  1. Arunav Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Klaus Berberich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arunav Mishra .

Editor information

Editors and Affiliations

  1. University of Glasgow , Glasgow, United Kingdom

    Joemon M Jose

  2. TU Delft - EWI/ST/WIS , Delft, The Netherlands

    Claudia Hauff

  3. Middle East Technical University , Ankara, Turkey

    Ismail Sengor Altıngovde

  4. Open University , Milton Keynes, United Kingdom

    Dawei Song

  5. Signal Media , London, United Kingdom

    Dyaa Albakour

  6. Toronto, Canada

    Stuart Watt

  7. JohnTait.net Ltd. and BCS IRSG , Sunderland, United Kingdom

    John Tait

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Mishra, A., Berberich, K. (2017). How Do Order and Proximity Impact the Readability of Event Summaries?. In: Jose, J., et al. Advances in Information Retrieval. ECIR 2017. Lecture Notes in Computer Science(), vol 10193. Springer, Cham. https://doi.org/10.1007/978-3-319-56608-5_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-56608-5_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-56607-8

  • Online ISBN: 978-3-319-56608-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation