research-article

Open access

Towards an Interpretable Approach to Classify and Summarize Crisis Events from Microblogs

Authors:

Thi Huyen Nguyen,

Koustav RudraAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 3641 - 3650

https://doi.org/10.1145/3485447.3512259

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Microblogging platforms like Twitter have been heavily leveraged to report and exchange information about natural disasters. The real-time data on these sites is highly helpful in gaining situational awareness and planning aid efforts. However, disaster-related messages are immersed in a high volume of irrelevant information. The situational data of disaster events also vary greatly in terms of information types ranging from general situational awareness (caution, infrastructure damage, casualties) to individual needs or not related to the crisis. It thus requires efficient methods to handle data overload and prioritize various types of information. This paper proposes an interpretable classification-summarization framework that first classifies tweets into different disaster-related categories and then summarizes those tweets. Unlike existing work, our classification model can provide explanations or rationales for its decisions. In the summarization phase, we employ an Integer Linear Programming (ILP) based optimization technique along with the help of rationales to generate summaries of event categories. Extensive evaluation on large-scale disaster events shows (a). our model can classify tweets into disaster-related categories with an 85% Macro F1 score and high interpretability (b). the summarizer achieves (5-25%) improvement in terms of ROUGE-1 F-score over most state-of-the-art approaches.

References

[1]

Firoj Alam, Shafiq Joty, and Muhammad Imran. 2018. Graph Based Semi-supervised Learning with Convolution Neural Networks to Classify Crisis Related Tweets. In Proceedings of the Eleventh International AAAI Conference on Web and Social Media (ICWSM).

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR).

[3]

Mark A. Cameron, Robert Power, Bella F Robinson, and Jie Yin. 2012. Emergency situation awareness from twitter for crisis management. In Proceedings of the 21st International Conference on World Wide Web (WWW’12 Companion).

Digital Library

[4]

Rich Caruana. 1997. Multitask Learning. Rich Caruana (1997).

[5]

N. Chawla, K. Bowyer, L. Hall, and P. Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research(2002), 321–357.

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).

[7]

J. DeYoung, S. Jain, N. F. Rajani, E. Lehman, C. Xiong, R. Socher, and B. C. Wallace. 2020. ERASER: A Benchmark to Evaluate Rationalized NLP Models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). 4443–4458.

[8]

gurobi. 2015. Gurobi – The overall fastest and best supported solver available. http://www.gurobi.com/

[9]

Huggingface. 2021. Hugging Face – The AI community building the future.https://huggingface.co/

[10]

Muhammad Imran, Carlos Castillo, Ji Lucas, Patrick Meier, and Sarah Vieweg. 2014. AIDR: artificial intelligence for disaster response. In Proceedings of the 23rd International Conference on World Wide Web (WWW’14 Companion).

Digital Library

[11]

Muhammad Imran, Prasenjit Mitra, and Carlos Castillo. 2016. Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages. In Proceedings of the 10th Language Resources and Evaluation Conference (LREC).

[12]

Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (ACL).

[13]

S. Jain, S. Wiegreffe, Y. Pinter, and B. C. Wallace. 2020. Learning to Faithfully Rationalize by Construction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). 4459–4473.

[14]

Ruipeng Jia, Yanan Cao, Hengzhu Tang, Fang Fang, Cong Cao, and Shi Wang. 2020. Neural Extractive Summarization with Hierarchical Attentive Heterogeneous Graph Network. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

[15]

Chris Kedzie, Fernando Diaz, and Kathleen R. McKeown. 2016. Real-Time Web Scale Event Summarization Using Sequential Decision Making. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI, Subbarao Kambhampati (Ed.). 3754–3760.

[16]

Chris Kedzie, Kathleen McKeown, and Fernando Diaz. 2015. Predicting Salient Updates for Disaster Summarization. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (IJCNLP).

[17]

Prashant Khare, Grégoire Burel, Diana Maynard, and Harith Alani. 2018. Cross-Lingual Classification of Crisis Data. In Proceedings of the International International Semantic Web Conference (ISWC).

Digital Library

[18]

Hongmin Li, Doina Caragea, and Cornelia Caragea. 2021. Combining Self-training with Deep Learning for Disaster Tweet Classification. In Proceedings of the 18th International Conference on Information Systems for Crisis Response and Management (ISCRAM).

[19]

Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceddings of Workshop on Text Summarization Branches Out (with ACL).

[20]

Chin-Yew Lin and Eduard Hovy. 2003. Automatic Evaluation of Summaries Using N-gram Co-Occurrence Statistics. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language (NAACL).

Digital Library

[21]

Junhua Liu, Trisha Singhal, Lucienne T.M. Blessing, Kristin L. Wood, and Kwan Hui Lim. 2021. CrisisBERT: A Robust Transformer for Crisis Classification and Contextual Crisis Embedding. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media (HT).

Digital Library

[22]

Yang Liu and Mirella Lapata. 2019. Text Summarization with Pretrained Encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

[23]

Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In Proceddings of the International Conference on Learning Representations (ICLR).

[24]

Reza Mazloom, Hongmin Li, Doina Caragea, Cornelia Caragea, and Muhammad Imran. 2019. A hybrid domain adaptation approach for identifying crisis-relevant tweets. International Journal of Information Systems for Crisis Response and Management (IJISCRAM)2(2019), 1–19.

[25]

Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI).

[26]

Dat Quoc Nguyen, Thanh Vu, and Anh Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English Tweets. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations.

[27]

Dat Tien Nguyen, Kamela Ali Al Mannai, Shafiq Joty, Hassan Sajjad, Muhammad Imran, and Prasenjit Mitra. 2017. Robust Classification of Crisis-Related Data on Social Networks Using Convolutional Neural Networks. In Proceedings of the 11th International AAAI Conference on Web and Social Media (ICWSM).

[28]

Minh-Tien Nguyen, Asanobu Kitamoto, and Tri-Thanh Nguyen. 2015. TSum4act: A Framework for Retrieving and Summarizing Actionable Tweets During a Disaster for Reaction. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD).

[29]

Thi Huyen Nguyen, Tuan-Anh Hoang, and Wolfgang Nejdl. 2019. Efficient Summarizing of Evolving Events from Twitter Streams. In Proceedings of the 2019 SIAM International Conference on Data Mining (SDM).

[30]

Andrei Olariu. 2014. Efficient Online Summarization of Microblogging Streams. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL).

[31]

Miles Osborne, Sean Moran, Richard McCreadie, Alexander Von Lunen, Martin Sykora, Elizabeth Cano, Neil Ireson, Craig Macdonald, Iadh Ounis, Yulan He, Tom Jackson, Fabio Ciravegna, and Ann O’Brien. 2014. Real-Time Detection, Tracking, and Monitoring of Automatically Discovered Events in Social Media. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics (ACL).

[32]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD).

Digital Library

[33]

Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. 2017. Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI).

[34]

Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. In Nature Machine Intelligence.

[35]

Koustav Rudra, Niloy Ganguly, Pawan Goyal, and Saptarshi GhoshACM Transactions on the Web. 2018. Extracting and Summarizing Situational Information from the Twitter Social Media during Disasters. ACM Transactions on the Web(2018).

[36]

Koustav Rudra, Subham Ghosh, and Niloy Ganguly. 2015. Extracting Situational Information from Microblogs during Disaster Events: a Classification-Summarization Approach. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM).

Digital Library

[37]

Koustav Rudra, Pawan Goyal, Niloy Ganguly, Muhammad Imran, and Prasenjit Mitra. 2019. Summarizing Situational Tweets in Crisis Scenarios: An Extractive-Abstractive Approach. In IEEE Transactions on Computational Social Systems.

[38]

Koustav Rudra, Pawan Goyal, Niloy Ganguly, Prasenjit Mitra, and Muhammad Imran. 2018. Identifying Sub-events and Summarizing Disaster-Related Information from Microblogs. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR).

Digital Library

[39]

Swarnadeep Saha, Prateek Yadav, Lisa Bauer, and Mohit Bansal. 2021. EXPLAGRAPHS: An Explanation Graph Generation Task for Structured Commonsense Reasoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (ACL). 7716–7740.

[40]

Naveen Saini, Sriparna Saha, and Pushpak Bhattacharyya. 2019. Multiobjective-Based Approach for Microblog Summarization. IEEE Transactions on Computational Social Systems (2019).

[41]

Sofia Serrano and Noah A. Smith. 2019. Is Attention Interpretable?. In Proceedings of the 57th Annual Meeting of the //Association for Computational Linguistics (ACL).

[42]

Lidan Shou, Zhenhua Wang, Ke Chen, and Gang Chen. 2013. Sumblr: continuous summarization of evolving tweet streams. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval).

Digital Library

[43]

István Varga, Motoki Sano, Kentaro Torisawa, Chikara Hashimoto, Kiyonori Ohtake, Takao Kawai, Jong-Hoon Oh, and Stijn De Saeger. 2013. Aid is Out There: Looking for Help from Tweets during a Large Scale Disaster. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL).

[44]

Sudha Verma, Sarah Vieweg, William J. Corvey, Leysia Palen, James H. Martin, Martha Palmer, Aaron Schram1, and Kenneth M. Anderson. 2011. Natural Language Processing to the Rescue? Extracting “Situational Awareness” Tweets During Mass Emergency. In Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media (ICWSM).

[45]

Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal Adversarial Triggers for Attacking and Analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

[46]

Wikipedia. 2021. Wilcoxon signed-rank test. https://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test

[47]

Ziyi Yang, Chenguang Zhu, Robert Gmyr, Michael Zeng, Xuedong Huang, and Eric Darve1. 2020. TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings.

[48]

Zijian Zhang, Koustav Rudra, and Avishek Anand. 2021. Explain and Predict, and then Predict again. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM).

Digital Library

[49]

Hao Zheng and Mirella Lapata. 2019. Sentence Centrality Revisited for Unsupervised Summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL).

[50]

R. Zhong, S. Shao, and K. McKeown. 2019. Fine-grained sentiment analysis with faithful attention. In arXiv preprint arXiv:1908.06870.

[51]

Arkaitz Zubiaga, Damiano Spina, Enrique Amigó, and Julio Gonzalo. 2012. Towards Real-Time Summarization of Scheduled Events from Twitter Streams. In Proceedings of the 12th ACM conference on Hypertext and Hypermedia (HT).

Digital Library

Cited By

Garg PChakraborty RDandapat S(2025)PORTRAIT: A Hybrid Approach to Create Extractive Ground-truth Summary for Disaster EventACM Transactions on the Web10.1145/371190819:1(1-36)Online publication date: 15-Feb-2025
https://dl.acm.org/doi/10.1145/3711908
Garg PChakraborty RDandapat S(2025)ATSumm: Auxiliary information enhanced approach for abstractive disaster tweet summarization with sparse training dataKnowledge-Based Systems10.1016/j.knosys.2025.112969311(112969)Online publication date: Feb-2025
https://doi.org/10.1016/j.knosys.2025.112969
Chowdhury MDatta SSharma NKhudaBukhsh AChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Infrastructure Ombudsman: Mining Future Failure Concerns from Structural Disaster ResponseProceedings of the ACM Web Conference 202410.1145/3589334.3648153(4664-4673)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648153
Show More Cited By

Recommendations

Learning Faithful Attention for Interpretable Classification of Crisis-Related Microblogs under Constrained Human Budget
WWW '23: Proceedings of the ACM Web Conference 2023

The recent widespread use of social media platforms has created convenient ways to obtain and spread up-to-date information during crisis events such as disasters. Time-critical analysis of crisis data can help human organizations gain actionable ...
CrisICSum: Interpretable Classification and Summarization Platform for Crisis Events from Microblogs
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Microblogging platforms such as Twitter, receive massive messages during crisis events. Real-time insights are crucial for emergency response. Hence, there is a need to develop faithful tools for efficiently digesting information. In this paper, we ...
Rationale Aware Contrastive Learning Based Approach to Classify and Summarize Crisis-Related Microblogs
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Recent fashion of information propagation on Twitter makes the platform a crucial conduit for tactical data and emergency responses during disasters. However, the real-time information about crises is immersed in a large volume of emotional and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

DFG Grant - ManagedForgetting
European Union?s Horizon 2020 research and innovation program - MIRROR

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
629
Total Downloads

Downloads (Last 12 months)180
Downloads (Last 6 weeks)19

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Garg PChakraborty RDandapat S(2025)PORTRAIT: A Hybrid Approach to Create Extractive Ground-truth Summary for Disaster EventACM Transactions on the Web10.1145/371190819:1(1-36)Online publication date: 15-Feb-2025
https://dl.acm.org/doi/10.1145/3711908
Garg PChakraborty RDandapat S(2025)ATSumm: Auxiliary information enhanced approach for abstractive disaster tweet summarization with sparse training dataKnowledge-Based Systems10.1016/j.knosys.2025.112969311(112969)Online publication date: Feb-2025
https://doi.org/10.1016/j.knosys.2025.112969
Chowdhury MDatta SSharma NKhudaBukhsh AChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Infrastructure Ombudsman: Mining Future Failure Concerns from Structural Disaster ResponseProceedings of the ACM Web Conference 202410.1145/3589334.3648153(4664-4673)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648153
Nguyen TRudra KChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Human vs ChatGPT: Effect of Data Annotation in Interpretable Crisis-Related Microblog ClassificationProceedings of the ACM Web Conference 202410.1145/3589334.3648141(4534-4543)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648141
Nguyen TFisichella MRudra K(2024)A Trustworthy Approach to Classify and Analyze Epidemic-Related Information From MicroblogsIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.339139511:5(6229-6241)Online publication date: Oct-2024
https://doi.org/10.1109/TCSS.2024.3391395
Garg PChakraborty RDandapat S(2024)OntoDSumm: Ontology-Based Tweet Summarization for Disaster EventsIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.326602511:2(2724-2739)Online publication date: Apr-2024
https://doi.org/10.1109/TCSS.2023.3266025
Garg PChakraborty RGupta SDandapat S(2024)IKDSummComputer Speech and Language10.1016/j.csl.2024.10164987:COnline publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1016/j.csl.2024.101649
Garg PChakraborty RDandapat S(2024)ADSumm: annotated ground-truth summary datasets for disaster tweet summarizationSocial Network Analysis and Mining10.1007/s13278-024-01323-914:1Online publication date: 5-Aug-2024
https://doi.org/10.1007/s13278-024-01323-9
Schwarz KAranda DHartmann M(2023)Towards Automated Situational Awareness Reporting for Disaster Management—A Case StudySustainability10.3390/su1510796815:10(7968)Online publication date: 13-May-2023
https://doi.org/10.3390/su15107968
Zhang YShang LZong RZeng HYue ZWang D(2023)CollabEquality: A Crowd-AI Collaborative Learning Framework to Address Class-wise Inequality in Web-based Disaster ResponseProceedings of the ACM Web Conference 202310.1145/3543507.3583871(4050-4059)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583871
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten