Avishek Anand
Sourav Saha
ABSTRACT
This tutorial presents explainability of text processing and retrieval methods, an emerging area focused on fostering responsible and trustworthy deployment of machine learning systems in the context of information retrieval. As the field has rapidly evolved in the past 4-5 years, numerous approaches have been proposed that focus on different access modes, stakeholders, and model development stages. This tutorial aims to introduce IR-centric notions, classification, and evaluation styles in explainable information retrieval (ExIR) while focusing on IR-specific tasks such as ranking, text classification, and learning-to-rank systems. We will delve into method families and their adaptations to IR, extensively covering post-hoc methods, axiomatic and probing approaches, and recent advances in interpretability-by-design approaches. We will also discuss ExIR applications for different stakeholders, such as researchers, practitioners, and end-users, in contexts like web search, patent and legal search, and high-stakes decision-making tasks. To facilitate practical understanding, we will provide a hands-on session on applying text processing and ExIR methods, reducing the entry barrier for students, researchers, and practitioners alike. Earlier version of this tutorial has been presented in SIGIR 2023.
- Enrique Amigó, Hui Fang, Stefano Mizzaro, and ChengXiang Zhai. 2017. Axiomatic Thinking for Information Retrieval: And Related Tasks. In Proc. of SIGIR 2017. 1419–1420.Google Scholar
Digital Library
- Avishek Anand, Lijun Lyu, Maximilian Idahl, Yumeng Wang, Jonas Wallat, and Zijian Zhang. 2022. Explainable Information Retrieval: A Survey.Google Scholar
- Yonatan Belinkov. 2022. Probing Classifiers: Promises, Shortcomings, and Advances. Comput. Linguistics1 (2022), 207–219.Google Scholar
- Alexander Bondarenko, Maik Fröbe, Jan Heinrich Reimer, Benno Stein, Michael Völske, and Matthias Hagen. 2022. Axiomatic Retrieval Experimentation with ir_axioms. In Proc. of SIGIR 2022. 3131–3140.Google ScholarDigital Library
- Arthur Câmara and Claudia Hauff. 2020. Diagnosing BERT with Retrieval Heuristics. In Proceedings of ECIR 2020. 605–618.Google Scholar
- Jaekeol Choi, Euna Jung, Sungjun Lim, and Wonjong Rhee. 2022. Finding Inverse Document Frequency Information in BERT. ArXiv preprint (2022).Google Scholar
- Daniel Cohen, Brendan O’Connor, and W. Bruce Croft. 2018. Understanding the Representational Power of Neural Retrieval Models Using NLP Tasks. In Proc. 2018 ACM ICTIR. 67–74.Google ScholarDigital Library
- Marina Danilevsky, Shipi Dhanorkar, Yunyao Li, Lucian Popa, Kun Qian, and Anbang Xu. 2021. Explainability for Natural Language Processing. In Proc. of SIGKDD 2021. 4033–4034.Google ScholarDigital Library
- Laura Dietz, Hannah Bast, Shubham Chatterjee, Jeff Dalton, Edgar Meij, and Arjen de Vries. 2023. Neuro-Symbolic Approaches for Information Retrieval. In Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland. 324–330.Google ScholarDigital Library
- Laura Dietz, Hannah Bast, Shubham Chatterjee, Jeffrey Dalton, Jian-Yun Nie, and Rodrigo Nogueira. 2023. Neuro-Symbolic Representations for Information Retrieval. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (Taipei, Taiwan) (SIGIR ’23). Association for Computing Machinery, New York, NY, USA, 3436–3439.Google ScholarDigital Library
- Yixing Fan, Jiafeng Guo, Xinyu Ma, Ruqing Zhang, Yanyan Lan, and Xueqi Cheng. 2021. A Linguistic Study on Relevance Modeling in Information Retrieval. 1053–1064.Google Scholar
- Hui Fang, Tao Tao, and ChengXiang Zhai. 2004. A Formal Study of Information Retrieval Heuristics. In Proceedings of SIGIR 2004. 49–56.Google ScholarDigital Library
- Zeon Trevor Fernando, Jaspreet Singh, and Avishek Anand. 2019. A study on the Interpretability of Neural Retrieval Models using DeepSHAP. In Proc. of SIGIR 2019. 1005–1008.Google ScholarDigital Library
- Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. A White Box Analysis of ColBERT. In Proc. of ECIR 2021. 257–263.Google ScholarDigital Library
- Matthias Hagen, Michael Völske, Steve Göring, and Benno Stein. 2016. Axiomatic Result Re-Ranking. In Proc. of CIKM 2016. 721–730.Google ScholarDigital Library
- Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing Neural Predictions. In Proc of EMNLP 2016. Austin, Texas, 107–117.Google ScholarCross Ref
- Jurek Leonhardt, Koustav Rudra, and Avishek Anand. 2021. Extractive Explanations for Interpretable Text Ranking. ACM Transactions on Information Systems (2021).Google Scholar
- Claudio Lucchese, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, and Alberto Veneri. 2022. ILMART: Interpretable Ranking with Constrained LambdaMART. In Proc. of SIGIR 2022. 2255–2259.Google ScholarDigital Library
- Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Proc. of NIPS 2017. 4765–4774.Google Scholar
- Lijun Lyu and Avishek Anand. 2023. Listwise Explanations for Ranking Models Using Multiple Explainers. In Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland. Springer, 653–668.Google Scholar
- Sean MacAvaney, Sergey Feldman, Nazli Goharian, Doug Downey, and Arman Cohan. 2020. ABNIRML: Analyzing the Behavior of Neural IR Models. ArXiv preprint (2020).Google Scholar
- Sayantan Polley. 2022. Towards Explainable Search in Legal Text. In European Conference on Information Retrieval. Springer, 528–536.Google Scholar
- Sayantan Polley, Atin Janki, Juliane Thiel, Marcusand Hoebel-Mueller, and Andreas Nuernberger. 2021. ExDocS: Evidence based Explainable Document Search. In Proc. of SIGIR Workshop on Causality in Search and Recommendation 2021.Google Scholar
- Alberto Purpura, Karolina Buchner, Gianmaria Silvello, and Gian Antonio Susto. 2021. Neural feature selection for learning to rank. In Proc. of ECIR 2021. 342–349.Google ScholarDigital Library
- Yifan Qiao, Chenyan Xiong, Zhenghao Liu, and Zhiyuan Liu. 2019. Understanding the Behaviors of BERT in Ranking. ArXiv preprint (2019).Google Scholar
- Razieh Rahimi, Youngwoo Kim, Hamed Zamani, and James Allan. 2021. Explaining Documents’ Relevance to Search Queries. ArXiv preprint (2021).Google Scholar
- David Rau and Jaap Kamps. 2022. The Role of Complex NLP in Transformers for Text Ranking. In Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval (Madrid, Spain) (ICTIR ’22). Association for Computing Machinery, 153–160.Google ScholarDigital Library
- Daniël Rennings, Felipe Moraes, and Claudia Hauff. 2019. An Axiomatic Approach to Diagnosing Neural IR Models. In Proceedings of ECIR 2019. 489–503.Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proc.of SIGKDD 2016. 1135–1144.Google Scholar
- Rishiraj Saha Roy and Avishek Anand. 2021. Question Answering for the Curated Web: Tasks and Methods in QA over Knowledge Bases and Text Collections. Synthesis Lectures onSynthesis Lectures on Information Concepts, Retrieval, and Services 13, 4 (2021), 1–194.Google ScholarCross Ref
- Sourav Saha, Debapriyo Majumdar, and Mandar Mitra. 2022. Explainability of Text Processing and Retrieval Methods: A Critical Survey. arxiv:2212.07126Google Scholar
- Procheta Sen, Debasis Ganguly, Manisha Verma, and Gareth J. F. Jones. 2020. The Curious Case of IR Explainability: Explaining Document Scores within and across Ranking Models. In Proc. of SIGIR 2020. 2069–2072.Google Scholar
- Procheta Sen, Sourav Saha, Debasis Ganguly, Manisha Verma, and Dwaipayan Roy. 2022. Measuring and Comparing the Consistency of IR Models for Query Pairs with Similar and Different Information Needs. In Proc of CIKM 2022. 4449–4453.Google ScholarDigital Library
- Jaspreet Singh and Avishek Anand. 2019. EXS: Explainable Search Using Local Model Agnostic Interpretability. In Proc. of WSDM 2019. 770–773.Google ScholarDigital Library
- Jaspreet Singh and Avishek Anand. 2020. Model agnostic interpretability of rankers via intent modelling. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 618–628.Google ScholarDigital Library
- Jaspreet Singh, Megha Khosla, Wang Zhenye, and Avishek Anand. 2021. Extracting per Query Valid Explanations for Blackbox Learning-to-Rank Models. In Proc. of ICTIR 2021. 203–210.Google ScholarDigital Library
- Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. In Proc of ICML 2017(Proceedings of Machine Learning Research). 3319–3328.Google Scholar
- Manisha Verma and Debasis Ganguly. 2019. LIRME: Locally Interpretable Ranking Model Explanation. In Proc. of SIGIR 2019. 1281–1284.Google ScholarDigital Library
- Michael Völske, Alexander Bondarenko, Maik Fröbe, Benno Stein, Jaspreet Singh, Matthias Hagen, and Avishek Anand. 2021. Towards Axiomatic Explanations for Neural Ranking Models. In Proc. of ICTIR 2021. 13–22.Google ScholarDigital Library
- Jonas Wallat, Fabian Beringer, Abhijit Anand, and Avishek Anand. 2023. Probing BERT for ranking abilities. In Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland. Springer, 255–273.Google ScholarDigital Library
- Puxuan Yu, Razieh Rahimi, and James Allan. 2022. Towards Explainable Search Results: A Listwise Explanation Generator. In Proc. of SIGIR 2022. 669–680.Google ScholarDigital Library
- Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, and Xueqi Cheng. 2020. Query Understanding via Intent Description Generation. In Proc. of CIKM 2020. 1823–1832.Google ScholarDigital Library
- Yongfeng Zhang. 2019. Tutorial on Explainable Recommendation and Search. In Proc. of ICTIR 2019. 255–256.Google ScholarDigital Library
- Yongfeng Zhang, Jiaxin Mao, and Qingyao Ai. 2019. SIGIR 2019 Tutorial on Explainable Recommendation and Search. In Proc. of SIGIR 2019. 1417–1418.Google ScholarDigital Library
- Yongfeng Zhang, Jiaxin Mao, and Qingyao Ai. 2019. WWW’19 Tutorial on Explainable Recommendation and Search. In Proc. of WWW 2019. 1330–1331.Google ScholarDigital Library
- Zijian Zhang, Koustav Rudra, and Avishek Anand. 2021. Explain and Predict, and then Predict Again. In WSDM ’21, Israel, March 8-12, 2021. ACM, 418–426.Google Scholar
- Zijian Zhang, Koustav Rudra, and Avishek Anand. 2021. FaxPlainAC: A Fact-Checking Tool Based on EXPLAINable Models with HumAn Correction in the Loop. In Proceedings of the 30th ACM CIKM. 4823–4827.Google ScholarDigital Library
- Zijian Zhang, Vinay Setty, and Avishek Anand. 2022. SparCAssist: A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals. In Proc. of SIGIR. 3219–3223.Google ScholarDigital Library
- Honglei Zhuang, Xuanhui Wang, Michael Bendersky, Alexander Grushetsky, Yonghui Wu, Petr Mitrichev, Ethan Sterling, Nathan Bell, Walker Ravina, and Hai Qian. 2021. Interpretable Ranking with Generalized Additive Models. In Proc. of WSDM 2021. 499–507.Google ScholarDigital Library
Index Terms
- Explainability of Text Processing and Retrieval Methods
Recommendations
Explainable Information Retrieval
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information RetrievalThis tutorial presents explainable information retrieval (ExIR), an emerging area focused on fostering responsible and trustworthy deployment of machine learning systems in the context of information retrieval. As the field has rapidly evolved in the ...
Phrase processing methods for Japanese text retrieval
This paper examines the effectiveness of different phrase identification and weighting methods for Japanese text retrieval in an operational information retrieval (IR) system, called NACSIS-IR. Based on our previous experiments, we used character-based ...
Text-Based Face Retrieval: Methods and Challenges
Biometric RecognitionAbstractPrevious researches on face retrieval have concentrated on using image-based queries. In this paper, we focus on the task of retrieving faces from a database based on queries given as texts, which holds significant potential for practical ...
Comments