short-paper

Bias Mitigation for Evidence-aware Fake News Detection by Causal Intervention

Authors:

Shu WuAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2308 - 2313

https://doi.org/10.1145/3477495.3531850

Published: 07 July 2022 Publication History

Abstract

Evidence-based fake news detection is to judge the veracity of news against relevant evidences. However, models tend to memorize the dataset biases within spurious correlations between news patterns and veracity labels as shortcuts, rather than learning how to integrate the information behind them to reason. As a consequence, models may suffer from a serious failure when facing real-life conditions where most news has different patterns. Inspired by the success of causal inference, we propose a novel framework for debiasing evidence-based fake news detection\footnoteCode available at https://github.com/CRIPAC-DIG/CF-FEND by causal intervention. Under this framework, the model is first trained on the original biased dataset like ordinary work, then it makes conventional predictions and counterfactual predictions simultaneously in the testing stage, where counterfactual predictions are based on the intervened evidence. Relatively unbiased predictions are obtained by subtracting intervened outputs from the conventional ones. Extensive experiments conducted on several datasets demonstrate our method's effectiveness and generality on debiased datasets.

Supplementary Material

MP4 File (SIGIR22-sp1776.mp4)

Presentation video of short paper - Bias Mitigation for Evidence-aware Fake News Detection by Causal Intervention.

Download
6.99 MB

References

[1]

Hunt Allcott and Matthew Gentzkow. 2017. Social Media and Fake News in the 2016 Election. CSN: Politics (Topic) (2017).

[2]

Erik Brand, Kevin Roitero, Michael Soprano, and Gianluca Demartini. 2021. EBART: Jointly Predicting and Explaining Truthfulness. Proceedings of the 2021 Truth and Trust Online Conference (2021), 18--27.

[3]

Shantanu Chandra, Pushkar Mishra, Helen Yannakoudakis, and Ekaterina Shutova. 2020. Graph-based Modeling of Online Communities for Fake News Detection. ArXiv abs/2008.06274 (2020).

[4]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186.

[5]

Lisa R Goldberg. 2019. The Book of Why: The New Science of Cause and Effect: by Judea Pearl and Dana Mackenzie, Basic Books (2018). ISBN: 978-0465097609.

[6]

Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation Artifacts in Natural Language Inference Data. In NAACL-HLT. 107--112.

[7]

Casper Hansen, Christian Hansen, and Lucas Chaves Lima. 2021. Automatic Fake News Detection: Are Models Learning to Reason?. In ACL/IJCNLP. 80--86.

[8]

Qiang Liu, Feng Yu, Shu Wu, and Liang Wang. 2018. Mining significant microblogs for misinformation identification: an attention-based approach. ACM Transactions on Intelligent Systems and Technology (TIST) 9, 5 (2018), 1--20.

Digital Library

[9]

Jing Ma, Wei Gao, Shafiq Joty, and Kam-Fai Wong. 2019. Sentence-level evidence embedding for claim verification with hierarchical attention networks. In ACL. 2561--2571.

[10]

Salman Bin Naeem and Rubina Bhatti. 2020. The Covid-19 'infodemic': a new front for information professionals. Health Information and Libraries Journal (2020).

[11]

Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and JiRong Wen. 2021. Counterfactual vqa: A cause-effect look at language bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12700--12710.

[12]

Judea Pearl. 2022. Direct and indirect effects. In Probabilistic and Causal Inference: The Works of Judea Pearl. 373--392.

[13]

Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis Only Baselines in Natural Language Inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. 180--191.

[14]

Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2016. Credibility Assessment of Textual Claims on the Web. In CIKM. 2173--2178.

[15]

Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning. In EMNLP. 22--32.

[16]

Chen Qian, Fuli Feng, Lijie Wen, Chunping Ma, and Pengjun Xie. 2021. Counterfactual Inference for Text Classification Debiasing. In ACL. 5434--5445.

[17]

Qiang Sheng, Xueyao Zhang, Juan Cao, and Lei Zhong. 2021. Integrating Patternand Fact-based Fake News Detection via Model Preference Learning. In CIKM. 1640--1650.

[18]

Nguyen Vo and Kyumin Lee. 2021. Hierarchical Multi-head Attentive Network for Evidence-aware Fake News Detection. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 965--975.

[19]

Svitlana Volkova, Kyle Shaffer, Jin Yea Jang, and Nathan Hodas. 2017. Separating Facts from Fiction: Linguistic Models to Classify Suspicious and Trusted News Posts on Twitter. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 647--653.

[20]

Tan Wang, Jianqiang Huang, Hanwang Zhang, and Qianru Sun. 2020. Visual commonsense r-cnn. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10760--10770.

[21]

Lianwei Wu, Yuan Rao, Xiong Yang, Wanzhen Wang, and Ambreen Nazir. 2021. Evidence-aware hierarchical interactive attention networks for explainable claim verification. In IJCAI. 1388--1394.

[22]

Weizhi Xu, Junfei Wu, Qiang Liu, Shu Wu, and Liang Wang. 2022. Evidence-aware Fake News Detection with Graph Neural Networks. In WWW.

[23]

Feng Yu, Q. Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2017. A Convolutional Approach for Misinformation Identification. In IJCAI. 3901--3907.

[24]

Xueyao Zhang, Juan Cao, Xirong Li, Qiang Sheng, Lei Zhong, and Kai Shu. 2021. Mining Dual Emotion for Fake News Detection (WWW '21). 3465--3476.

Cited By

ZHENG ZLV JWANG LLU RCUI PWANG XWEI W(2024)Cross-scale systematic learning for social big data: theory and methodsSCIENTIA SINICA Informationis10.1360/SSI-2023-040854:9(2083)Online publication date: 2-Sep-2024
https://doi.org/10.1360/SSI-2023-0408
Zeng ZLuo MKong XLiu HGuo HYang HMa ZZhao XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Mitigating World Biases: A Multimodal Multi-View Debiasing Framework for Fake News Video DetectionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681673(6492-6500)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681673
Zhang YKong LTian SFei HXiang CWang HWei XGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Multi-view Counterfactual Contrastive Learning for Fact-checking Fake News DetectionProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658087(385-393)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658087
Show More Cited By

Index Terms

Bias Mitigation for Evidence-aware Fake News Detection by Causal Intervention
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Veracity-aware and Event-driven Personalized News Recommendation for Fake News Mitigation
WWW '22: Proceedings of the ACM Web Conference 2022

Despite the tremendous efforts by social media platforms and fact-check services for fake news detection, fake news and misinformation still spread wildly on social media platforms (e.g., Twitter). Consequently, fake news mitigation strategies are ...
Evolving to the Future: Unseen Event Adaptive Fake News Detection on Social Media
CIKM '24: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

With the rapid development of social media, the wide dissemination of fake news on social media is increasingly threatening both individuals and society. One of the unique challenges for fake news detection on social media is how to detect fake news on ...
Bias Detection and Mitigation in Textual Data: A Study on Fake News and Hate Speech Detection
Advances in Information Retrieval
Abstract
Addressing bias in NLP-based solutions is crucial to promoting fairness, avoiding discrimination, building trust, upholding ethical standards, and ultimately improving their performance and reliability. On the topic of bias detection and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Natural Science Foundation of China

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
704
Total Downloads

Downloads (Last 12 months)139
Downloads (Last 6 weeks)12

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

ZHENG ZLV JWANG LLU RCUI PWANG XWEI W(2024)Cross-scale systematic learning for social big data: theory and methodsSCIENTIA SINICA Informationis10.1360/SSI-2023-040854:9(2083)Online publication date: 2-Sep-2024
https://doi.org/10.1360/SSI-2023-0408
Zeng ZLuo MKong XLiu HGuo HYang HMa ZZhao XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Mitigating World Biases: A Multimodal Multi-View Debiasing Framework for Fake News Video DetectionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681673(6492-6500)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681673
Zhang YKong LTian SFei HXiang CWang HWei XGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Multi-view Counterfactual Contrastive Learning for Fact-checking Fake News DetectionProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658087(385-393)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658087
Zhang DZhang KWu LTian MHong RWang MBaeza-Yates RBonchi F(2024)Path-Specific Causal Reasoning for Fairness-aware Cognitive DiagnosisProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3672049(4143-4154)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3672049
Zhang TMin WLiu TJiang SRui Y(2024)Toward Egocentric Compositional Action Anticipation with Adaptive Semantic DebiasingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363333320:5(1-21)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3633333
Zhao LLiu QYue LChen WChen LSun RSong CHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)COMI: COrrect and MItigate Shortcut Learning Behavior in Deep Neural NetworksProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657729(218-228)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657729
Tao XWang LLiu QWu SWang LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Semantic Evolvement Enhanced Graph Autoencoder for Rumor DetectionProceedings of the ACM Web Conference 202410.1145/3589334.3645478(4150-4159)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645478
Liu GZhang JLiu QWu JWu SWang L(2024)Uni-Modal Event-Agnostic Knowledge Distillation for Multimodal Fake News DetectionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.347797736:12(9490-9503)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3477977
Liu QWu JWu SWang L(2024)Out-of-Distribution Evidence-Aware Fake News Detection via Dual Adversarial DebiasingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.339043136:11(6801-6813)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2024.3390431
Wu JXu WLiu QWu SWang L(2024)Adversarial Contrastive Learning for Evidence-Aware Fake News Detection With Graph Neural NetworksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.334164036:11(5591-5604)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2023.3341640
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten