Skip to main content
Springer Nature Link
Log in

Unsupervised Joint Learning for Headline Generation and Discourse Structure of Reviews

  • Conference paper
  • First Online:
Advances in Artificial Intelligence (JSAI 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1128))

Included in the following conference series:

  • 545 Accesses

Abstract

This is an extension from a selected paper from JSAI2019. Recently, using a large number of reference summaries, supervised neural summarization models have achieved success. However, such data is rare, and trained models cannot be shared across domains. As a solution for such a problem, we propose the first unsupervised end-to-end headline generation model for a single review. We assume that a review can be described as a discourse tree in which the headline is the root and the child sentences elaborate on their parent. By estimating the parent from their children recursively, our model induces the tree and generates the headline that describes the entire review. Through the evaluation of the generated headline on actual reviews, our model achieved competitive performance with supervised models, especially on relatively long reviews. In induced trees, we confirmed that the child sentences explain the parent in detail and the generated headlines abstract for the entire review.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bing, L., Li, P., Liao, Y., Lam, W., Guo, W., Passonneau, R.: Abstractive multi-document summarization via phrase selection and merging. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, vol. 1, pp. 1587–1597 (2015)

    Google Scholar 

  2. Carenini, G., Cheung, J.C.K., Pauls, A.: Multi-document summarization of evaluative text. Comput. Intell. 29(4), 545–576 (2013)

    Article  MathSciNet  Google Scholar 

  3. Chopra, S., Auli, M., Rush, A.M.: Abstractive sentence summarization with attentive recurrent neural networks. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 93–98 (2016)

    Google Scholar 

  4. Chu, E., Liu, P.: MeanSum: a neural model for unsupervised multi-document abstractive summarization. In: Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 1223–1232 (2019)

    Google Scholar 

  5. Di Fabbrizio, G., Stent, A., Gaizauskas, R.: A hybrid approach to multi-document summarization of opinions in reviews. In: Proceedings of the 8th International Natural Language Generation Conference, pp. 54–63 (2014)

    Google Scholar 

  6. Dohare, S., Gupta, V., Karnick, H.: Unsupervised semantic abstractive summarization. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Student Research Workshop, pp. 74–83 (2018)

    Google Scholar 

  7. Erkan, G., Radev, D.R.: Lexpagerank: prestige in multi-document text summarization. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 365–371 (2004)

    Google Scholar 

  8. Fang, Y., Zhu, H., Muszyńska, E., Kuhnle, A., Teufel, S.: A proposition-based abstractive summariser. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics, pp. 567–578 (2016)

    Google Scholar 

  9. Gerani, S., Mehdad, Y., Carenini, G., Ng, R.T., Nejat, B.: Abstractive summarization of product reviews using discourse structure. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1602–1613 (2014)

    Google Scholar 

  10. He, R., McAuley, J.: Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web, pp. 507–517 (2016)

    Google Scholar 

  11. Hirao, T., Yoshida, Y., Nishino, M., Yasuda, N., Nagata, M.: Single-document summarization as a tree knapsack problem. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1515–1520 (2013)

    Google Scholar 

  12. Isonuma, M., Fujino, T., Mori, J., Matsuo, Y., Sakata, I.: Extractive summarization using multi-task learning with document classification. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2101–2110 (2017)

    Google Scholar 

  13. Ji, Y., Smith, N.A.: Neural discourse structure for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 996–1005 (2017)

    Google Scholar 

  14. Joulin, A., Grave, E., Mikolov, P.B.T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, vol. 2, pp. 427–431 (2017)

    Google Scholar 

  15. Kikuchi, Y., Hirao, T., Takamura, H., Okumura, M., Nagata, M.: Single document summarization based on nested tree structure. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol. 2, pp. 315–320 (2014)

    Google Scholar 

  16. Koo, T., Globerson, A., Carreras, X., Collins, M.: Structured prediction models via the matrix-tree theorem. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 141–150 (2007)

    Google Scholar 

  17. Liu, B., Zhang, L.: A survey of opinion mining and sentiment analysis. In: Aggarwal, C., Zhai, C. (eds.) Mining Text Data, pp. 415–463. Springer, Boston (2012)

    Chapter  Google Scholar 

  18. Liu, Y., Lapata, M.: Learning structured text representations. Trans. Assoc. Comput. Linguist. 6, 63–75 (2018)

    Article  Google Scholar 

  19. Ma, S., Sun, X., Lin, J., Ren, X.: A hierarchical end-to-end model for jointly improving text summarization and sentiment classification. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 4251–4257 (2018)

    Google Scholar 

  20. Mann, W.C., Thompson, S.A.: Rhetorical structure theory: toward a functional theory of text organization. Text-Interdiscip. J. Study Discourse 8(3), 243–281 (1988)

    Article  Google Scholar 

  21. McAuley, J., Targett, C., Shi, Q., Van Den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52 (2015)

    Google Scholar 

  22. Miao, Y., Blunsom, P.: Language as a latent variable: discrete generative models for sentence compression. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 319–328 (2016)

    Google Scholar 

  23. Mihalcea, R., Tarau, P.: Textrank: bringing order into texts. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 404–411 (2004)

    Google Scholar 

  24. Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., Xiang, B.: Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp. 280–290 (2016)

    Google Scholar 

  25. Paulus, R., Xiong, C., Socher, R.: A deep reinforced model for abstractive summarization. In: Proceedings of the 6th International Conference on Learning Representations (2018)

    Google Scholar 

  26. Radev, D.R., Jing, H., Styś, M., Tam, D.: Centroid-based summarization of multiple documents. Inf. Process. Manag. 40(6), 919–938 (2004)

    Article  Google Scholar 

  27. Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 379–389 (2015)

    Google Scholar 

  28. See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 1073–1083 (2017)

    Google Scholar 

  29. Tan, J., Wan, X., Xiao, J.: Abstractive document summarization with a graph-based attentional neural model. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 1171–1181 (2017)

    Google Scholar 

  30. Tutte, W.T.: Graph Theory, vol. 21. Addison-Wesley, Boston (1984)

    MATH  Google Scholar 

  31. Wang, H., Ren, J.: A self-attentive hierarchical model for jointly improving text summarization and sentiment classification. In: Proceedings of the 10th Asian Conference on Machine Learning, pp. 630–645 (2018)

    Google Scholar 

  32. Wang, L., Ling, W.: Neural network-based abstract generation for opinions and arguments. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 47–57 (2016)

    Google Scholar 

  33. Yoshida, Y., Suzuki, J., Hirao, T., Nagata, M.: Dependency-based discourse parser for single-document summarization. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1834–1839 (2014)

    Google Scholar 

  34. Yu, N., Huang, M., Shi, Y., Zhu, X.: Product review summarization by exploiting phrase properties. In: Proceedings of the 26th International Conference on Computational Linguistics, pp. 1113–1124 (2016)

    Google Scholar 

Download references

Acknowledgements

This work was supported by CREST, JST, the New Energy and Industrial Technology Development Organization (NEDO) and Deloitte Tohmatsu Financial Advisory LLC.

Author information

Authors and Affiliations

  1. The University of Tokyo, 3-7-1 Hongo, Bunkyo, Tokyo, Japan

    Masaru Isonuma, Junichiro Mori & Ichiro Sakata

  2. RIKEN, 1-4-1 Nihonbashi, Chuo, Tokyo, Japan

    Junichiro Mori

Authors
  1. Masaru Isonuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junichiro Mori
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ichiro Sakata
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masaru Isonuma .

Editor information

Editors and Affiliations

  1. Department of Systems Innovation, University of Tokyo, Tokyo, Japan

    Yukio Ohsawa

  2. Faculty of Business and Commerce, Kansai University, Osaka, Japan

    Katsutoshi Yada

  3. Nagoya Institute of Technology, Nagoya, Japan

    Takayuki Ito

  4. Graduate School of System Design, Tokyo Metropolitan University, Tokyo, Japan

    Yasufumi Takama

  5. Department of Information and Communication, Tokyo Metropolitan University, Tokyo, Japan

    Eri Sato-Shimokawara

  6. Faculty of Letters, Chiba University, Chiba, Japan

    Akinori Abe

  7. School of Engineering, The University of Tokyo, Tokyo, Japan

    Junichiro Mori

  8. Graduate School of Economics, Osaka University, Toyonaka, Osaka, Japan

    Naohiro Matsumura

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Isonuma, M., Mori, J., Sakata, I. (2020). Unsupervised Joint Learning for Headline Generation and Discourse Structure of Reviews. In: Ohsawa, Y., et al. Advances in Artificial Intelligence. JSAI 2019. Advances in Intelligent Systems and Computing, vol 1128. Springer, Cham. https://doi.org/10.1007/978-3-030-39878-1_13

Download citation

Publish with us

Policies and ethics