skip to main content
10.1145/2682571.2797081acmconferencesArticle/Chapter ViewAbstractPublication PagesdocengConference Proceedingsconference-collections
short-paper

A Quantitative and Qualitative Assessment of Automatic Text Summarization Systems

Published: 08 September 2015 Publication History

Abstract

Text summarization is the process of automatically creating a shorter version of one or more text documents. This paper presents a qualitative and quantitative assessment of the 22 state-of-the-art extractive summarization systems using the CNN corpus, a dataset of 3,000 news articles.

References

[1]
Free summarizer. http://freesummarizer.com/, 2011. Last acess: Mar. 2015.
[2]
Autosummarizer. Automatic text summarizer. http://autosummarizer.com/, 2014. Last acess: Mar. 2015.
[3]
Aylien. Aylien text analysis api. http://aylien.com/text-api-doc, 2011. Last acess: Mar. 2015.
[4]
T. Compactor. Text compactor. http://www.textcompactor.com, 2015. Last acess: Mar. 2015.
[5]
H. Dalianis and et al. From swesum to scandsum: Automatic text summarization for the scandinavian languages. pages 153--163. Museum Tusculanums Forlag, 2003.
[6]
R. Ferreira and et al. Assessing sentence scoring techniques for extractive text summarization. Expert Systems with Applications, 40(14):5755--5764, 2013.
[7]
R. Ferreira et al. A context based text summarization system. In DAS 2014, pages 66--70, Apr., 2014.
[8]
Findwise. Findwise multi-document summarizers. http://labdemos.findwise.com/demomds, 2015. Last acess: Mar. 2015.
[9]
M. Hassel and H. Dalianis. Swesum - automatic text summarizer. http://swesum.nada.kth.se/index-eng.html, 2003. Last acess: Mar. 2015.
[10]
C.-Y. Lin. Rouge: A package for automatic evaluation of summaries. In ACL-04 Workshop, pages 74--81, Barcelona, Spain, July 2004. Association for Computational Linguistics.
[11]
R. D. Lins and et al. A multi-tool scheme for summarizing textual documents. In IADIS - WWW/INTERNET, pages 1--8, July 2012.
[12]
E. Lloret and et al. COMPENDIUM: A text summarization system for generating abstracts of research papers. volume 88, pages 164--175. Data Knowl. Eng., 2013.
[13]
E. Lloret and M. Palomar. Text summarisation in progress: a literature review. Artif. Intell. Rev., 37(1):1--41, Jan. 2012.
[14]
N. Lothian. Classifier4j. http://classifier4j.sourceforge.net/, 2003. Last acess: Mar. 2015.
[15]
H. P. Luhn. The automatic creation of literature abstracts. IBM J. Res. Dev., 2(2):159--165, Apr. 1958.
[16]
A. Nenkova and K. McKeown. A survey of text summarization techniques. In Mining Text Data, pages 43--76. Springer, 2012.
[17]
Y. Ouyang and et al. A study on position information in document summarization. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters, pages 919--927. Association for Computational Linguistics, 2010.
[18]
N. Rotem. Open text summarizer. http://libots.sourceforge.net/, 2003. Last acess: Mar. 2015.
[19]
K. Spärck Jones. Automatic summarising: The state of the art. Information Processing Management, 43(6):1449--1481, Nov. 2007.
[20]
J. Steinberger and K. Ježek. Text summarization and singular value decomposition. ADVIS'04, pages 245--254, Berlin, Heidelberg, 2004. Springer-Verlag.
[21]
Sumplify. Sumplify. http://sumplify.com/, 2015. Last acess: Mar. 2015.
[22]
Sumy. Sumy. https://github.com/miso-belica/sumy, 2015. Last acess: Mar. 2015.
[23]
TAO. Text analysis online (). http://textanalysisonline.com/simple-text-summarizer, 2015. Last acess: Mar. 2015.
[24]
D. Teaser. Py teaser. https://github.com/xiaoxu193/PyTeaser, 2013. Last acess: Mar. 2015.
[25]
TextSummarization. Text summarization. http://textsummarization.net/text-summarizer, 2015. Last acess: Mar. 2015.
[26]
Tools4Noobs. Tools4noobs. http://www.tools4noobs.com/summarize/, 2015. Last acess: Mar. 2015.
[27]
C. Writing. Custom writing summarizer. http://custom-writing.org/writing-tools/summarizer, 2006. Last acess: Mar. 2015.

Cited By

View all
  • (2023)EXABSUM: a new text summarization approach for generating extractive and abstractive summariesJournal of Big Data10.1186/s40537-023-00836-y10:1Online publication date: 24-Oct-2023
  • (2022)How Does Corporate Social Responsibility Moderate the Adverse Effects of Product Failure in Social Media?The International Journal of Accounting10.1142/S109440602250003257:01Online publication date: 12-Mar-2022
  • (2019)ESSMArT way to manage customer requestsEmpirical Software Engineering10.1007/s10664-019-09721-w24:6(3755-3789)Online publication date: 21-May-2019
  • Show More Cited By

Index Terms

  1. A Quantitative and Qualitative Assessment of Automatic Text Summarization Systems

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    DocEng '15: Proceedings of the 2015 ACM Symposium on Document Engineering
    September 2015
    248 pages
    ISBN:9781450333078
    DOI:10.1145/2682571
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 September 2015

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. summarization evaluation
    2. survey
    3. text summarization

    Qualifiers

    • Short-paper

    Funding Sources

    • R&D project

    Conference

    DocEng '15
    Sponsor:
    DocEng '15: ACM Symposium on Document Engineering 2015
    September 8 - 11, 2015
    Lausanne, Switzerland

    Acceptance Rates

    DocEng '15 Paper Acceptance Rate 11 of 31 submissions, 35%;
    Overall Acceptance Rate 194 of 564 submissions, 34%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)EXABSUM: a new text summarization approach for generating extractive and abstractive summariesJournal of Big Data10.1186/s40537-023-00836-y10:1Online publication date: 24-Oct-2023
    • (2022)How Does Corporate Social Responsibility Moderate the Adverse Effects of Product Failure in Social Media?The International Journal of Accounting10.1142/S109440602250003257:01Online publication date: 12-Mar-2022
    • (2019)ESSMArT way to manage customer requestsEmpirical Software Engineering10.1007/s10664-019-09721-w24:6(3755-3789)Online publication date: 21-May-2019
    • (2018)Towards coherent single-document summarizationProceedings of the 33rd Annual ACM Symposium on Applied Computing10.1145/3167132.3167211(712-719)Online publication date: 9-Apr-2018
    • (2018)Automatic cohesive summarization with pronominal anaphora resolutionComputer Speech and Language10.1016/j.csl.2018.05.00452:C(141-164)Online publication date: 1-Nov-2018
    • (2017)A Regression-Based Approach Using Integer Linear Programming for Single-Document Summarization2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI.2017.00051(270-277)Online publication date: Nov-2017
    • (2016)Towards Cohesive Extractive Summarization through Anaphoric Expression ResolutionProceedings of the 2016 ACM Symposium on Document Engineering10.1145/2960811.2967159(201-204)Online publication date: 13-Sep-2016
    • (2016)A Concept-Based Integer Linear Programming Approach for Single-Document Summarization2016 5th Brazilian Conference on Intelligent Systems (BRACIS)10.1109/BRACIS.2016.079(403-408)Online publication date: Oct-2016
    • (2016)Assessing shallow sentence scoring techniques and combinations for single and multi-document summarizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2016.08.03065:C(68-86)Online publication date: 15-Dec-2016

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media