short-paper

Distant Supervision based Machine Reading Comprehension for Extractive Summarization in Customer Service

Authors:

Jianxin LiaoAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1895 - 1899

https://doi.org/10.1145/3404835.3463046

Published: 11 July 2021 Publication History

Abstract

Given a long text, the summarization system aims to obtain a shorter highlight while keeping important information on the original text. For customer service, the summaries of most dialogues between an agent and a user focus on several fixed key points, such as user's question, user's purpose, the agent's solution, and so on. Traditional extractive methods are difficult to extract all predefined key points exactly. Furthermore, there is a lack of large-scale and high-quality extractive summarization datasets containing key points. In order to solve the above challenges, we propose a Distant Supervision based Machine Reading Comprehension model for extractive Summarization (DSMRC-S). DSMRC-S transforms the summarization task into the machine reading comprehension problem, to fetch key points from the original text exactly according to the predefined questions. In addition, a distant supervision method is proposed to alleviate the lack of eligible extractive summarization datasets. We conduct experiments on a large-scale summarization dataset collected in customer service scenarios, and the results show that the proposed DSMRC-S outperforms the strong baseline methods by 4 points on ROUGE-L.

Supplementary Material

MP4 File (sigir2021.mp4)

Presentation video - short paper

Download
5.93 MB

References

[1]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1409.0473

[2]

Jianpeng Cheng and Mirella Lapata. 2016. Neural Summarization by Extracting Sentences and Words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7--12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics. https://doi.org/10.18653/v1/p16--1046

[3]

Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive Sentence Summarization with Attentive Recurrent Neural Networks. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12--17, 2016, Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). The Association for Computational Linguistics, 93--98. https://doi.org/10.18653/v1/n16--1012

[4]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2--7, 2019, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171--4186. https://doi.org/10.18653/v1/n19--1423

[5]

Jianxiong Dong and Jim Huang. 2018. Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus. CoRR, Vol. abs/1802.02614 (2018). arxiv: 1802.02614 http://arxiv.org/abs/1802.02614

[6]

Sebastian Gehrmann, Yuntian Deng, and Alexander M. Rush. 2018. Bottom-Up Abstractive Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii (Eds.). Association for Computational Linguistics, 4098--4109. https://doi.org/10.18653/v1/d18--1443

[7]

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A Diversity-Promoting Objective Function for Neural Conversation Models. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12--17, 2016, Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). The Association for Computational Linguistics, 110--119. https://doi.org/10.18653/v1/n16--1014

[8]

Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74--81.

[9]

Chunyi Liu, Peng Wang, Jiang Xu, Zang Li, and Jieping Ye. 2019. Automatic Dialogue Summary Generation for Customer Service. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4--8, 2019, Ankur Teredesai, Vipin Kumar, Ying Li, Ró mer Rosales, Evimaria Terzi, and George Karypis (Eds.). ACM, 1957--1965. https://doi.org/10.1145/3292500.3330683

Digital Library

[10]

Yishu Miao and Phil Blunsom. 2016. Language as a Latent Variable: Discrete Generative Models for Sentence Compression. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1--4, 2016, Jian Su, Xavier Carreras, and Kevin Duh (Eds.). The Association for Computational Linguistics, 319--328. https://doi.org/10.18653/v1/d16--1031

[11]

Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4--9, 2017, San Francisco, California, USA, Satinder P. Singh and Shaul Markovitch (Eds.). AAAI Press, 3075--3081. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14636

[12]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6--12, 2002, Philadelphia, PA, USA. ACL, 311--318. https://doi.org/10.3115/1073083.1073135

Digital Library

[13]

Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A Neural Attention Model for Abstractive Sentence Summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17--21, 2015, Llu'i s Mà rquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton (Eds.). The Association for Computational Linguistics, 379--389. https://doi.org/10.18653/v1/d15--1044

[14]

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get To The Point: Summarization with Pointer-Generator Networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1073--1083. https://doi.org/10.18653/v1/P17--1099

[15]

Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8--13 2014, Montreal, Quebec, Canada, Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (Eds.). 3104--3112. https://proceedings.neurips.cc/paper/2014/hash/a14ac55a4f27472c5d894ec1c3c743d2-Abstract.html

Digital Library

[16]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4--9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998--6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

Digital Library

[17]

Danqing Wang, Pengfei Liu, Yining Zheng, Xipeng Qiu, and Xuanjing Huang. 2020. Heterogeneous Graph Neural Networks for Extractive Document Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5--10, 2020, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 6209--6219. https://doi.org/10.18653/v1/2020.acl-main.553

[18]

Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-Aware Neural Extractive Text Summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5--10, 2020, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 5021--5031. https://doi.org/10.18653/v1/2020.acl-main.451

[19]

Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. 2020. Extractive Summarization as Text Matching. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5--10, 2020, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 6197--6208. https://doi.org/10.18653/v1/2020.acl-main.552

[20]

Ming Zhong, Danqing Wang, Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. [n.d.]. A Closer Look at Data Bias in Neural Extractive Summarization Models. ( [n.,d.]).

[21]

Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural Document Summarization by Jointly Learning to Score and Select Sentences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15--20, 2018, Volume 1: Long Papers, Iryna Gurevych and Yusuke Miyao (Eds.). Association for Computational Linguistics, 654--663. https://doi.org/10.18653/v1/P18--1061

[22]

Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2020. A Joint Sentence Scoring and Selection Framework for Neural Extractive Document Summarization. IEEE ACM Trans. Audio Speech Lang. Process., Vol. 28 (2020), 671--681. https://doi.org/10.1109/TASLP.2020.2964427

Digital Library

Cited By

Han QYang ZLin HQin T(2024)Let Topic Flow: A Unified Topic-Guided Segment-Wise Dialogue Summarization FrameworkIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2024.337411232(2021-2032)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1109/TASLP.2024.3374112
Niu CLiu CLiu HJia YZan H(2024)Emotion-Cause Pair Extraction Based on Dependency-injected Dual-MRC2024 International Conference on Asian Language Processing (IALP)10.1109/IALP63756.2024.10661115(222-227)Online publication date: 4-Aug-2024
https://doi.org/10.1109/IALP63756.2024.10661115
Zou JZhang YWu SYang JQin XYing LJiang MHuang Y(2024)A machine reading comprehension framework for recognizing emotion cause in conversationsKnowledge-Based Systems10.1016/j.knosys.2024.111532289:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111532
Show More Cited By

Index Terms

Distant Supervision based Machine Reading Comprehension for Extractive Summarization in Customer Service
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Discourse, dialogue and pragmatics
      2. Information extraction
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Recommendations

Unsupervised Extractive Text Summarization with Distance-Augmented Sentence Graphs
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Supervised summarization has made significant improvements in recent years by leveraging cutting-edge deep learning technologies. However, the true success of supervised methods relies on the availability of large quantity of human-generated summaries of ...
Extractive text summarization using clustering-based topic modeling
Abstract
Text summarization is the process of converting the input document into a short form, provided that it preserves the overall meaning associated with it. Primarily, text summarization is achieved in two ways, i.e., abstractive and extractive. ...
Sentence Relations for Extractive Summarization with Deep Neural Networks

Sentence regression is a type of extractive summarization that achieves state-of-the-art performance and is commonly used in practical systems. The most challenging task within the sentence regression framework is to identify discriminative features to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

the Ministry of Education and China Mobile Joint Fund
the National Postdoctoral Program for Innovative Talents
the Beijing University of Posts and Telecommunications-China Mobile Research Institute Joint Innovation Center
the National Natural Science Foundation of China
the Beijing Municipal Natural Science Foundation
the National Key R&D Program of China

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
340
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Han QYang ZLin HQin T(2024)Let Topic Flow: A Unified Topic-Guided Segment-Wise Dialogue Summarization FrameworkIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2024.337411232(2021-2032)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1109/TASLP.2024.3374112
Niu CLiu CLiu HJia YZan H(2024)Emotion-Cause Pair Extraction Based on Dependency-injected Dual-MRC2024 International Conference on Asian Language Processing (IALP)10.1109/IALP63756.2024.10661115(222-227)Online publication date: 4-Aug-2024
https://doi.org/10.1109/IALP63756.2024.10661115
Zou JZhang YWu SYang JQin XYing LJiang MHuang Y(2024)A machine reading comprehension framework for recognizing emotion cause in conversationsKnowledge-Based Systems10.1016/j.knosys.2024.111532289:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111532
Mai HZhang XWang JZhou X(2024)A machine reading comprehension model with counterfactual contrastive learning for emotion-cause pair extractionKnowledge and Information Systems10.1007/s10115-024-02062-166:6(3459-3476)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10115-024-02062-1
Zhang TCui YYang ZFeng SWang D(2024)Summarizing Doctor’s Diagnoses and Suggestions from Medical DialoguesWeb and Big Data10.1007/978-981-97-2387-4_16(235-249)Online publication date: 28-Apr-2024
https://doi.org/10.1007/978-981-97-2387-4_16
Wang HGuo ZTao RLiu JLuo YYi ZLin Y(2024)MRCJE: A Machine Reading Comprehension Framework with Joint Coding for Emotion-Cause Pair ExtractionAI and Multimodal Services – AIMS 202410.1007/978-3-031-77681-6_5(63-77)Online publication date: 16-Nov-2024
https://doi.org/10.1007/978-3-031-77681-6_5
Cheng ZJiang ZYin YWang CGe SGu Q(2023)A Consistent Dual-MRC Framework for Emotion-cause Pair ExtractionACM Transactions on Information Systems10.1145/355854841:4(1-27)Online publication date: 8-Apr-2023
https://dl.acm.org/doi/10.1145/3558548
Shen HMa YLi YWang XTian DJia THe TLuo S(2023)ADPal: Automatic Detection of Troubled Users in Online Service Systems via Page Access Logs2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00082(638-646)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00082

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten