Skip to main content
SpringerLink
Log in
Book cover

Complexity and Approximation pp 113–150Cite as

AutoOverview: A Framework for Generating Structured Overviews over Many Documents

  • Chapter
  • First Online:

  • 805 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12000))

Abstract

This article is an exposition of a recent study on automatic generation of a structured overview (SOV) over a very large corpus of documents, where an SOV is organized as sections and subsections according to the latent hierarchy of topics contained in the documents. We present a new framework called AutoOverview that includes and extends our previous scheme called NDORGS (best paper runner-up in ACM DocEng’2019) [47]. Different from the standard NLP task of generating a coherent summary typically over a handful of documents, AutoOverview needs to balance between two competitive objectives of accuracy and efficiency over thousands of documents. It incorporates hierarchical topic clustering, single-document summarization, multiple-document summarization, title generation, and other text mining techniques into a single platform. To assess the quality of an SOV generated over many documents, while it is possible to rely on human annotators to judge its readability, the sheer size of the inputs would make it formidable for human judges to determine if an SOV has covered all major points contained in the original texts. To overcome this obstacle, we present a text mining mechanism to evaluate topic coverage of the SOV against the topics contained in the original documents. We use multi-attribute decision making to help determine a suitable suite of algorithms to implement AutoOverview and the values of parameters for achieving a satisfactory SOV with respect to both accuracy and efficiency. We use NDORGS as an implementation example to address these issues and present evaluation results over a corpus of over 2,000 classified news articles and a corpus of over 5,000 unclassified news articles in a span of 10 years obtained from a search of the same keyword.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    For example, see http://nlpprogress.com/english/dependency_parsing.html.

  2. 2.

    Available at http://nlp.stanford.edu/software/stanford-dependencies.shtml.

  3. 3.

    The six SOVs generated by NDORGS are available at http://www.ndorg.net.

References

  1. Atasu, K., et al.: Linear-complexity relaxed word mover’s distance with GPU acceleration. In: Proceedings of the 2017 IEEE International Conference on Big Data (BigData 2017), Boston, Massachusetts, USA, 11–14 December 2017, pp. 889–896 (2017)

    Google Scholar 

  2. Berman, P., DasGupta, B., Kao, M.Y., Wang, J.: On constructing an optimal consensus clustering from multiple clusterings. Inform. Process. Lett. 104(4), 137–145 (2007)

    Article  MathSciNet  Google Scholar 

  3. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  4. Buyukkokten, O., Garcia-Molina, H., Paepcke, A.: Seeing the whole in parts: text summarization for web browsing on handheld devices. In: Proceedings of the 10th International Conference on World Wide Web (WWW 2001), Hong Kong, China, 1–5 May 2001, pp. 652–662. ACM (2001)

    Google Scholar 

  5. Cao, Z., Li, W., Li, S., Wei, F.: Improving multi-document summarization via text classification. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), San Francisco, USA, 4–9 February 2017, pp. 3053–3059 (2017)

    Google Scholar 

  6. Cao, Z., Li, W., Li, S., Wei, F., Li, Y.: AttSum: joint learning of focusing and summarization with neural attention. In: Proceedings of the 26th International Conference on Computational Linguistics (COLING 2016), Osaka, Japan, 11–16 December 2016, pp. 547–556 (2016)

    Google Scholar 

  7. Christensen, J., Mausam, Soderland, S., Etzioni, O.: Towards coherent multi-document summarization. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2013), Atlanta, Georgia, USA, 9–15 June 2013, pp. 1163–1173 (2013)

    Google Scholar 

  8. Christensen, J., Soderland, S., Bansal, G., et al.: Hierarchical summarization: scaling up multi-document summarization. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL 2014), Baltimore, Maryland, USA, 22–27 June 2014, vol. 1, pp. 902–912 (2014)

    Google Scholar 

  9. DUC: DUC 2004 quality questions (2004). http://duc.nist.gov/duc2004/quality.questions.txt

  10. DUC: Document understanding conference (2014). https://www-nlpir.nist.gov/projects/duc/intro.html

  11. Dueck, D.: Affinity propagation: clustering data by passing messages. Citeseer (2009)

    Google Scholar 

  12. Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)

    Article  Google Scholar 

  13. Florescu, C., Caragea, C.: A position-biased pagerank algorithm for keyphrase extraction. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), San Francisco, California, USA, 4–9 February 2017, pp. 4923–4924 (2017)

    Google Scholar 

  14. Gao, W., Li, P., Darwish, K.: Joint topic modeling for event summarization across news and social media streams. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management (CIKM 2012), Maui, Hawaii, USA, 29 October–2 November 2012, pp. 1173–1182 (2012)

    Google Scholar 

  15. Gillick, D., Favre, B.: A scalable global model for summarization. In: Proceedings of the Workshop on Integer Linear Programming for Natural Langauge Processing, pp. 10–18 (2009)

    Google Scholar 

  16. Gusfield, D.: Partition-distance: a problem and class of perfect graphs arising in clustering. Inform. Process. Lett. 82(3), 159–164 (2002)

    Article  MathSciNet  Google Scholar 

  17. Hartigan, J.A., Wong, M.A.: Algorithm AS 136: a k-means clustering algorithm. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 28(1), 100–108 (1979)

    MATH  Google Scholar 

  18. Hong, K., Conroy, J.M., Favre, B., Kulesza, A., Lin, H., Nenkova, A.: A repository of state of the art and competitive baseline summaries for generic news summarization. In: Proceedings of the 9th edition of the Language Resources and Evaluation Conference (LREC 2014), Reykjavik, Iceland, 26–31 May 2014, pp. 1608–1616 (2014)

    Google Scholar 

  19. Hwang, C.L., Yoon, K.: Methods for multiple attribute decision making. In: Hwang, C.L., Yoon, K. (eds.) Multiple Attribute Decision Making. Lecture Notes in Economics and Mathematical Systems, vol. 186, pp. 58–191. Springer, Heidelberg (1981). https://doi.org/10.1007/978-3-642-48318-9_3

    Chapter  Google Scholar 

  20. Jones, D.: Factiva global news database (2018). https://www.dowjones.com/products/factiva/

  21. Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, Hoboken (2009)

    MATH  Google Scholar 

  22. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the 4th International Conference on Learning Representations (ICLR 2016), San Juan, Puerto Rico, 2–4 May 2016

    Google Scholar 

  23. Kusner, M.J., Sun, Y., Kolkin, N.I., Weinberger, K.Q.: From word embeddings to document distances. In: Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), Lille, France, 06–11 July 2015, vol. 37 (2015)

    Google Scholar 

  24. Lapata, M.: Probabilistic text structuring: experiments with sentence ordering. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics (ACL 2003), Sapporo, Japan, 7–12 July 2003, vol. 1, pp. 545–552 (2003)

    Google Scholar 

  25. Li, C., et al.: LDA meets Word2Vec: a novel model for academic abstract clustering. In: Companion Proceedings of the Web Conference (WWW 2018), pp. 1699–1706 (2018)

    Google Scholar 

  26. Li, C., Qian, X., Liu, Y.: Using supervised bigram-based ILP for extractive summarization. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, 4–9 August 2013, vol. 1, pp. 1004–1013 (2013)

    Google Scholar 

  27. Li, P., Jiang, J., Wang, Y.: Generating templates of entity summaries with an entity-aspect model and pattern mining. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010), Uppsala, Sweden, 11–16 July 2010, pp. 640–649 (2010)

    Google Scholar 

  28. Li, S., Ouyang, Y., Wang, W., Sun, B.: Multi-document summarization using support vector regression. In: Proceedings of DUC. Citeseer (2007)

    Google Scholar 

  29. Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Proceedings of Workshop on Text Summarization Branches Out, Barcelona, Spain, 21–26 July 2004, pp. 74–81 (2004)

    Google Scholar 

  30. Liu, P.J., et al.: Generating Wikipedia by summarizing long sequences. In: Proceedings 6th International Conference on Learning Representation (ICLR 2018), Vancouva, Canada, 30 April-3 May 2018, vol. abs/1801.10198 (2018)

    Google Scholar 

  31. Luhn, H.P.: The automatic creation of literature abstracts. IBM J. Res. Dev. 2(2), 159–165 (1958)

    Article  MathSciNet  Google Scholar 

  32. Lulli, A., Debatty, T., Dell’Amico, M., Michiardi, P., Ricci, L.: Scalable k-NN based text clustering. In: Proceedings of 2015 IEEE International Conference on Big Data (IEEE BigData 2015), Santa Clara, California, USA, 29 October–1 November 2015, pp. 958–963 (2015)

    Google Scholar 

  33. Mihalcea, R., Tarau, P.: TextRank: Bringing order into texts. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), Barcelona, Spain, 25–26 July 2004, pp. 404–411 (2004)

    Google Scholar 

  34. Nallapati, R., Zhai, F., Zhou, B.: SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), San Francisco, USA, 4–9 February 2017, pp. 3075–3081 (2017)

    Google Scholar 

  35. Nayeem, M.T., Chali, Y.: Extract with order for coherent multi-document summarization. In: Proceedings of the Workshop on Graph-based Methods for Natural Language Processing (TextGraphs 2011), Vancouver, Canada, 3 August 2017, pp. 51–56 (2017)

    Google Scholar 

  36. Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems, pp. 849–856 (2002)

    Google Scholar 

  37. Otterbacher, J., Radev, D., Kareem, O.: News to go: hierarchical text summarization for mobile devices. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development on Information Retrieval (SIGIR 2006), Seattle, Washington, USA, 6–11 August 2006, pp. 589–596 (2006)

    Google Scholar 

  38. Pottker, H.: News and its communicative quality: the inverted pyramidwhen and why did it appear? J. Stud. 4(4), 501–511 (2003)

    Google Scholar 

  39. Radev, D., et al.: SummBank 1.0 LDC2003T16. web download. Linguistic Data Consortium, Philadelphia (2003)

    Google Scholar 

  40. Radev, D.R., Jing, H., Sty, M., Tam, D.: Centroid-based summarization of multiple documents. Inf. Process. Manage. 40, 919–938 (2004)

    Article  Google Scholar 

  41. Saaty, T.: The Analytical Hierarchy Process. McGraw Hill, New York (1980)

    MATH  Google Scholar 

  42. Sauper, C., Barzilay, R.: Automatically generating Wikipedia articles: a structure-aware approach. In: Proc of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL 2009), Suntec, Singapore, 2–7 August 2009, pp. 208–216 (2009)

    Google Scholar 

  43. Shao, L., Wang, J.: DTATG: an automatic title generator based on dependency trees. In: Proceedings of the 8th International Joint Conference on Knowledge Discovery and Information Retrieval (KDIR 2016), Porto, Portugal, 9–11 November 2016, pp. 166–173. SCITEPRESS - Science and Technology Publications, Lda, Portugal (2016). https://doi.org/10.5220/0006035101660173

  44. Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical dirichlet processing. J. Am. Stat. Assoc. 101(476), 1566–1581 (2006)

    Article  Google Scholar 

  45. Vandegehinste, V., Pan, Y.: Sentence compression for automated subtitling: a hybrid approach. In: Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistic (ACL 2004), Barcelona, Spain, 21–26 July 2004, pp. 89–95 (2004)

    Google Scholar 

  46. Von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)

    Article  MathSciNet  Google Scholar 

  47. Wang, J., Zhang, H., Zhang, C., Yang, W., Wang, J.: An effective scheme for generating an overview report over a very large corpus of documents. In: Proceedings ACM Symposium on Document Engineering (DocEng 2019), Berlin, Germany, 23–26 September 2019. (Best paper runnerup)

    Google Scholar 

  48. Wang, X., Nishino, M., Hirao, T., Sudoh, K., Nagata, M.: Exploring text links for coherent multi-document summarization. In: Proceedings of the 26th International Conference on Computational Linguistics (COLING 2016), Osaka, Japan, 11–16 December 2016, pp. 213–223 (2016)

    Google Scholar 

  49. Xu, J., et al.: Short text clustering via convolutional neural networks. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies (NAACL HLT 2015), Denver, Colorado, USA, 31 May-5 June 2015, pp. 62–69 (2015)

    Google Scholar 

  50. Yao, C., Jia, X., Shou, S., Feng, S., Zhou, F., Liu, H.: Autopedia: automatic domain-independent Wikipedia article generation. In: Proceedings of the 20th International Conference Companion on World Wide Web (WWW 2011), Hyderabad, India, 28 March–1 April 2011, pp. 161–162 (2011)

    Google Scholar 

  51. Yasunaga, M., Zhang, R., Meelu, K., Pareek, A., Srinivasan, K., Radev, D.R.: Graph-based neural multi-document summarization. In: Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, 3–4 August 2017

    Google Scholar 

  52. Yin, J., Wang, J.: A Dirichlet multinomial mixture model-based approach for short text clustering. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2014), New York, NY, USA, 24–27 August 2014, pp. 233–242 (2014)

    Google Scholar 

  53. Yogatama, D., Liu, F., Smith, N.A.: Extractive summarization by maximizing semantic volume. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), Lisbon, Portugal, 17–21 September 2015, pp. 1961–1966 (2015)

    Google Scholar 

  54. Zhang, H., Wang, J.: Semantic WordRank: generating finer single-document summarizations. In: Yin, H., Camacho, D., Novais, P., Tallón-Ballesteros, A.J. (eds.) IDEAL 2018. LNCS, vol. 11314, pp. 398–409. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03493-1_42

    Chapter  Google Scholar 

  55. Zhang, Y., Peng, C., Wang, H.: Research on Chinese sentence compression for the title generation. In: Ji, D., Xiao, G. (eds.) CLSW 2012. LNCS (LNAI), vol. 7717, pp. 22–31. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36337-5_3

    Chapter  Google Scholar 

Download references

Acklowledgement

During the past five years, a number of students have worked on various parts of AutoOverview for their PhD degrees at the University of Massachusetts Lowell, although the term of AutoOverview and its current framework were not introduced until now in this article. They are (in alphabetical order) Yiqi Bai, Ming Jia, Liqun (Catherine) Shao, Jingwen (Jessica) Wang, Wenjing Yang, Cheng Zhang, and Hao Zhang, and five of them have graduated. Most of their contributions have been published elsewhere. The term of AutoOverview was inspired from a conversation with Prof. Jiawei Han of the University of Illinois at Urbana-Champaign.

Author information

Authors and Affiliations

  1. University of Massachusetts, Lowell, MA, 01854, USA

    Jie Wang

Authors
  1. Jie Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Wang .

Editor information

Editors and Affiliations

  1. The University of Texas at Dallas, Richardson, TX, USA

    Ding-Zhu Du

  2. University of Massachusetts Lowell, Lowell, MA, USA

    Jie Wang

Additional information

In a conversation with Prof. Ker-I Ko about 10 years ago during a visit to Tsinghua University in Beijing, I indicated a desire to venture into a new field that would allow me to integrate algorithm designs, software development, system construction, data modeling, data management, and web technologies into a long-term project, so that a group of PhD students with various backgrounds and interests could work on different parts of the project for their dissertations. Ker-I was supportive and offered his insights. I am honored to dedicate this article on text mining and document engineering in memory of him.

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Wang, J. (2020). AutoOverview: A Framework for Generating Structured Overviews over Many Documents. In: Du, DZ., Wang, J. (eds) Complexity and Approximation. Lecture Notes in Computer Science(), vol 12000. Springer, Cham. https://doi.org/10.1007/978-3-030-41672-0_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-41672-0_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-41671-3

  • Online ISBN: 978-3-030-41672-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation