skip to main content
10.1145/3488560.3498536acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

MTLTS: A Multi-Task Framework To Obtain Trustworthy Summaries From Crisis-Related Microblogs

Published: 15 February 2022 Publication History

Abstract

Occurrences of catastrophes such as natural or man-made disasters trigger the spread of rumours over social media at a rapid pace. Presenting a trustworthy and summarized account of the unfolding event in near real-time to the consumers of such potentially unreliable information thus becomes an important task. In this work, we propose MTLTS, the first end-to-end solution for the task that jointly determines the credibility and summary-worthiness of tweets. Our credibility verifier is designed to recursively learn the structural properties of a Twitter conversation cascade, along with the stances of replies towards the source tweet. We then take a hierarchical multi-task learning approach, where the verifier is trained at a lower layer, and the summarizer is trained at a deeper layer where it utilizes the verifier predictions to determine the salience of a tweet. Different from existing disaster-specific summarizers, we model tweet summarization as a supervised task. Such an approach can automatically learn summary-worthy features, and can therefore generalize well across domains. When trained on the PHEME dataset [29], not only do we outperform the strongest baselines for the auxiliary task of verification/rumour detection, we also achieve 21 - 35% gains in the verified ratio of summary tweets, and 16 - 20% gains in ROUGE1-F1 scores over the existing state-of-the-art solutions for the primary task of trustworthy summarization.

Supplementary Material

MP4 File (WSDM2022_fp870_3488560.3498536.mp4)
We present MTLTS, the first end-to-end solution to obtain trustworthy summaries from large volumes of disaster-related tweets. Different from existing disaster-specific summarizers, our approach is supervised, which enhances the generalizability of our solution to unseen events. We present a novel way to leverage a document summarization technique for summarizing social media posts, here disaster-related tweets. When experimenting on the PHEME dataset, we achieve state-of-the-art results both for the primary task of trustworthy summarization as well as the auxiliary task of tweet credibility verification or rumour detection.

References

[1]
Mingxi Cheng, Shahin Nazarian, and Paul Bogdan. 2020. VRoC: Variational Autoencoder-aided Multi-task Rumor Classifier Based on Text. In WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20--24, 2020, Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen (Eds.). ACM / IW3C2, 2892--2898. https://doi.org/10.1145/3366423.3380054
[2]
Leon Derczynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. 2017. SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). Association for Computational Linguistics, Vancouver, Canada, 69--76. https://doi.org/10.18653/v1/S17--2006
[3]
Francesco Ducci, Mathias Kraus, and Stefan Feuerriegel. 2020. Cascade-LSTM: A Tree-Structured Neural Classifier for Detecting Misinformation Cascades. In KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23--27, 2020, Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (Eds.). ACM, 2666--2676. https://dl.acm.org/doi/10.1145/3394486.3403317
[4]
LLC Gurobi Optimization. 2021. Gurobi Optimizer Reference Manual. http://www.gurobi.com
[5]
Dandan Huang, Leyang Cui, Sen Yang, Guangsheng Bao, Kun Wang, Jun Xie, and Yue Zhang. 2020. What Have We Achieved on Text Summarization?. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Online, 446--469. https://doi.org/10.18653/v1/2020.emnlp-main.33
[6]
Muhammad Imran, Ferda Ofli, Doina Caragea, and Antonio Torralba. 2020. Using AI and Social Media Multimodal Content for Disaster Response and Management: Opportunities, Challenges, and Future Directions. Information Processing & Management, Vol. 57, 5 (2020), 102261. https://doi.org/10.1016/j.ipm.2020.102261
[7]
Chris Kedzie, Kathleen McKeown, and Fernando Diaz. 2015. Predicting Salient Updates for Disaster Summarization. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Beijing, China, 1608--1617. https://doi.org/10.3115/v1/P15--1155
[8]
Ling Min Serena Khoo, Hai Leong Chieu, Zhong Qian, and Jing Jiang. 2020. Interpretable Rumor Detection in Microblogs by Attending to User Interactions. Proceedings of the 34th AAAI Conference on Artificial Intelligence, Vol. 34, 05 (February 2020), 8783--8790. https://aaai.org/ojs/index.php/AAAI/article/view/6405
[9]
Elena Kochkina, Maria Liakata, and Arkaitz Zubiaga. 2018. All-in-one: Multi-task Learning for Rumour Verification. In Proceedings of the 27th International Conference on Computational Linguistics . Association for Computational Linguistics, New Mexico, USA, 3402--3413. https://aclanthology.org/C18--1288
[10]
Sumeet Kumar and Kathleen Carley. 2019. Tree LS™s with Convolution Units to Predict Stance and Rumor Veracity in Social Media Conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 5047--5058. https://doi.org/10.18653/v1/P19--1498
[11]
David M. J. Lazer, Matthew A. Baum, Yochai Benkler, Adam J. Berinsky, Kelly M. Greenhill, Filippo Menczer, Miriam J. Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, Michael Schudson, Steven A. Sloman, Cass R. Sunstein, Emily A. Thorson, Duncan J. Watts, and Jonathan L. Zittrain. 2018. The science of fake news. Science, Vol. 359, 6380 (2018), 1094--1096. https://doi.org/10.1126/science.aao2998 https://doi.org/10.1145/3161603
[12]
Arkaitz Zubiaga, Elena Kochkina, Maria Liakata, Rob Procter, and Michal Lukasik. 2016a. Stance Classification in Rumours as a Sequential Task Exploiting the Tree Structure of Social Media Conversations. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers . The COLING 2016 Organizing Committee, Osaka, Japan, 2438--2448. https://aclanthology.org/C16--1230
[13]
Arkaitz Zubiaga, Maria Liakata, Rob Procter, Kalina Bontcheva, and Peter Tolmie. 2015. Crowdsourcing the Annotation of Rumourous Conversations in Social Media. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15 Companion). Association for Computing Machinery, New York, NY, USA, 347--353. https://doi.org/10.1145/2740908.2743052
[14]
Arkaitz Zubiaga, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Peter Tolmie. 2016b. Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads. PLOS ONE, Vol. 11, 3 (2016), 1--29. https://doi.org/10.1371/journal.pone.0150989

Cited By

View all
  • (2024)Novel Genetic Optimization Techniques for Accurate Social Media Data Summarization and Classification Using Deep Learning ModelsTechnologies10.3390/technologies1210019912:10(199)Online publication date: 15-Oct-2024
  • (2024)MuLX-QA: Classifying Multi-Labels and Extracting Rationale Spans in Social Media PostsACM Transactions on the Web10.1145/365330318:3(1-26)Online publication date: 6-May-2024
  • (2024)Fake Social Media News Detection Based on Forwarding User RepresentationIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.333144611:3(3432-3443)Online publication date: Jun-2024
  • Show More Cited By

Index Terms

  1. MTLTS: A Multi-Task Framework To Obtain Trustworthy Summaries From Crisis-Related Microblogs

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
      February 2022
      1690 pages
      ISBN:9781450391320
      DOI:10.1145/3488560
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 February 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. disaster
      2. rumour detection
      3. trustworthy summarization
      4. twitter

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      WSDM '22

      Acceptance Rates

      Overall Acceptance Rate 498 of 2,863 submissions, 17%

      Upcoming Conference

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)31
      • Downloads (Last 6 weeks)4
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Novel Genetic Optimization Techniques for Accurate Social Media Data Summarization and Classification Using Deep Learning ModelsTechnologies10.3390/technologies1210019912:10(199)Online publication date: 15-Oct-2024
      • (2024)MuLX-QA: Classifying Multi-Labels and Extracting Rationale Spans in Social Media PostsACM Transactions on the Web10.1145/365330318:3(1-26)Online publication date: 6-May-2024
      • (2024)Fake Social Media News Detection Based on Forwarding User RepresentationIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.333144611:3(3432-3443)Online publication date: Jun-2024
      • (2023)Towards Automated Situational Awareness Reporting for Disaster Management—A Case StudySustainability10.3390/su1510796815:10(7968)Online publication date: 13-May-2023
      • (2023)Automatic Short Text Summarization Techniques in Social Media PlatformsFuture Internet10.3390/fi1509031115:9(311)Online publication date: 13-Sep-2023
      • (2023)Advancements in Rumor Detection Research Based on Bibliometrics and S-curve Technology Evolution TheorySage Open10.1177/2158244023121772413:4Online publication date: 23-Dec-2023
      • (2023)Fairness for both Readers and Authors: Evaluating Summaries of User Generated ContentProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591986(1996-2000)Online publication date: 19-Jul-2023
      • (2023)Multimodal Rumour Detection: Catching News that Never Transpired!Document Analysis and Recognition - ICDAR 202310.1007/978-3-031-41682-8_15(231-248)Online publication date: 21-Aug-2023
      • (2022)CrisICSum: Interpretable Classification and Summarization Platform for Crisis Events from MicroblogsProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557191(4941-4945)Online publication date: 17-Oct-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media