BiTe-REx: An Explainable Bilingual Text Retrieval System in the Automotive Domain
Authors: 
Viju Sudhi, Sabine Wehnert, Norbert Michael Homner, Sebastian Ernst, Mark Gonter, Andreas Krug, 
Ernesto William De LucaAuthors Info & Claims
Viju Sudhi, Sabine Wehnert, Norbert Michael Homner, Sebastian Ernst, Mark Gonter, Andreas Krug, Ernesto William De LucaAuthors Info & ClaimsPages 3251 - 3255
Abstract
To satiate the comprehensive information need of users, retrieval systems surpassing the boundaries of language are inevitable in the present digital space in the wake of an ever-rising multilingualism. This work presents the first-of-its-kind Bilingual Text Retrieval Explanations (BiTe-REx) aimed at users performing competitor or wage analysis in the automotive domain. BiTe-REx supports users to gather a more comprehensive picture of their query by retrieving results regardless of the query language and enables them to make a more informed decision by exposing how the underlying model judges the relevance of documents. With a user study, we demonstrate statistically significant results on the understandability and helpfulness of the explanations provided by the system.
References
[1]
Dominika Basaj, Barbara Rychalska, Przemyslaw Biecek, and Anna Wróblewska. 2018. How much should you ask? On the question structure in QA systems. CoRR, Vol. abs/1809.03734 (2018). [arXiv]1809.03734 http://arxiv.org/abs/1809.03734
[2]
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomás Mikolov. 2017. Enriching Word Vectors with Subword Information. Trans. Assoc. Comput. Linguistics, Vol. 5 (2017), 135--146. https://transacl.org/ojs/index.php/tacl/article/view/999
[3]
Ioannis Chios and Suzan Verberne. 2021. Helping results assessment by adding explainable elements to the deep relevance matching model. CoRR, Vol. abs/2106.05147 (2021). [arXiv]2106.05147 https://arxiv.org/abs/2106.05147
[4]
Jaeho Choi, Seunghyeok Oh, and Joongheon Kim. 2020. The Useful Quantum Computing Techniques for Artificial Intelligence Engineers. In 2020 International Conference on Information Networking, ICOIN 2020, Barcelona, Spain, January 7-10, 2020. IEEE, 1--3. https://doi.org/10.1109/ICOIN48656.2020.9016555
[5]
European Commission. 2019. Artificial Intelligence: Commission Takes Forward its Work on Ethics Guidelines. http://europa.eu/rapid/press-release_IP-19-1893_en.htm
[6]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:, Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171--4186. https://doi.org/10.18653/v1/n19-1423
[7]
Yanzhuo Ding, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Visualizing and Understanding Neural Machine Translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers. Association for Computational Linguistics, 1150--1159. https://doi.org/10.18653/v1/P17-1106
[8]
Shi Feng, Eric Wallace, Alvin Grissom II, Mohit Iyyer, Pedro Rodriguez, and Jordan Boyd-Graber. 2018. Pathologies of neural models make interpretations difficult. arXiv preprint arXiv:1804.07781 (2018).
[9]
Zeon Trevor Fernando, Jaspreet Singh, and Avishek Anand. 2019. A study on the Interpretability of Neural Retrieval Models using DeepSHAP. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1005--1008. https://doi.org/10.1145/3331184.3331312
[10]
Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. 2016. A Deep Relevance Matching Model for Ad-hoc Retrieval. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016, Indianapolis, IN, USA, October 24-28, 2016. ACM, 55--64. https://doi.org/10.1145/2983323.2983769
[11]
Raju Korra, Pothula Sujatha, Sidige Chetana, and Madarapu Naresh Kumar. 2011. Performance evaluation of multilingual information retrieval (MLIR) system over information retrieval (IR) system. In 2011 International Conference on Recent Trends in Information Technology (ICRTIT). IEEE, 722--727.
[12]
Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. Unsupervised Machine Translation Using Monolingual Corpora Only. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rkYTTf-AZ
[13]
Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 4765--4774. https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
[14]
Ling Luo et al. 2018. Beyond Polarity: Interpretable Financial Sentiment Analysis with Hierarchical Query-driven Attention. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, Jérôme Lang (Ed.). ijcai.org, 4244--4250. https://doi.org/10.24963/ijcai.2018/590
[15]
Siwen Luo, Hamish Ivison, Soyeon Caren Han, and Josiah Poon. 2021. Local Interpretations for Explainable Natural Language Processing: A Survey. CoRR, Vol. abs/2103.11072 (2021). [arXiv]2103.11072 https://arxiv.org/abs/2103.11072
[16]
Qianren Mao, Jianxin Li, Senzhang Wang, Yuanning Zhang, Hao Peng, Min He, and Lihong Wang. 2019. Aspect-Based Sentiment Classification with Attentive Neural Turing Machines. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, Sarit Kraus (Ed.). ijcai.org, 5139--5145. https://doi.org/10.24963/ijcai.2019/714
[17]
Leland McInnes and John Healy. 2018. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. CoRR, Vol. abs/1802.03426 (2018). [arXiv]1802.03426 http://arxiv.org/abs/1802.03426
[18]
Pradeepta Mishra. 2022. Explainability for NLP. In Practical Explainable AI Using Python. Springer, 193--227.
[19]
Carol Peters, Martin Braschler, and Paul D. Clough. 2012. Multilingual Information Retrieval - From Research To Practice. Springer. https://doi.org/10.1007/978-3-642-23008-0
[20]
Marco Tú lio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016. ACM, 1135--1144. https://doi.org/10.1145/2939672.2939778
[21]
Procheta Sen, Debasis Ganguly, Manisha Verma, and Gareth JF Jones. 2020. The Curious Case of IR Explainability: Explaining Document Scores within and across Ranking Models. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Jimmy Huang, Yi Chang, Xueqi Cheng, Jaap Kamps, Vanessa Murdock, Ji-Rong Wen, and Yiqun Liu (Eds.). ACM, 2069--2072. https://doi.org/10.1145/3397271.3401286
[22]
Jaspreet Singh and Avishek Anand. 2019. EXS: Explainable Search Using Local Model Agnostic Interpretability. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019. ACM, 770--773. https://doi.org/10.1145/3289600.3290620
[23]
Philipp Sorg and Philipp Cimiano. 2012. Exploiting Wikipedia for cross-lingual and multilingual information retrieval. Data Knowl. Eng., Vol. 74 (2012), 26--45. https://doi.org/10.1016/j.datak.2012.02.003
[24]
Alona Sydorova, Nina Pörner, and Benjamin Roth. 2019. Interpretable Question Answering on Knowledge Bases and Text. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 4943--4951. https://doi.org/10.18653/v1/p19--1488
[25]
Ming Tu et al. 2020. Select, Answer and Explain: Interpretable Multi-Hop Reading Comprehension over Multiple Documents. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7--12, 2020. AAAI Press, 9073--9080. https://aaai.org/ojs/index.php/AAAI/article/view/6441
[26]
Manisha Verma and Debasis Ganguly. 2019. LIRME: Locally Interpretable Ranking Model Explanation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, Benjamin Piwowarski, Max Chevalier, Éric Gaussier, Yoelle Maarek, Jian-Yun Nie, and Falk Scholer (Eds.). ACM, 1281--1284. https://doi.org/10.1145/3331184.3331377
[27]
Yuxuan Wang, Yutai Hou, Wanxiang Che, and Ting Liu. 2020. From static to dynamic word representations: a survey. Int. J. Mach. Learn. Cybern., Vol. 11, 7 (2020), 1611--1630. https://doi.org/10.1007/s13042-020-01069-8
[28]
Abdulrahman Yarali. 2022. Big Data and Artificial Intelligence. 299--326. https://doi.org/10.1002/9781119685265.ch17
Index Terms
- BiTe-REx: An Explainable Bilingual Text Retrieval System in the Automotive Domain
Recommendations
Counterfactual Explanations in Explainable AI: A Tutorial
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data MiningDeep learning has shown powerful performances in many fields, however its black-box nature hinders its further applications. In response, explainable artificial intelligence emerges, aiming to explain the predictions and behaviors of deep learning ...
I-REX: A Lucene Plugin for EXplainable IR
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge ManagementProviding high-level, intuitive explanations of the performance of IR systems is generally difficult due to their complexity, and the various low-level implementation details involved. We present I-REX, a tool built on top of Lucene, that is intended to ...
Algorithmic Recourse: from Counterfactual Explanations to Interventions
FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and TransparencyAs machine learning is increasingly used to inform consequential decision-making (e.g., pre-trial bail and loan approval), it becomes important to explain how the system arrived at its decision, and also suggest actions to achieve a favorable decision. ...
Comments
Information & Contributors
Information
Published In
July 2022
3569 pages
ISBN:9781450387323
DOI:10.1145/3477495
- General Chairs:
- Enrique Amigo,
- Pablo Castells,
- Julio Gonzalo,
- Program Chairs:
- Ben Carterette,
- J. Shane Culpepper,
- Gabriella Kazai
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 July 2022
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
SIGIR '22
Sponsor:
SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
July 11 - 15, 2022
Madrid, Spain
Acceptance Rates
Overall Acceptance Rate 792 of 3,983 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 154Total Downloads
- Downloads (Last 12 months)19
- Downloads (Last 6 weeks)3
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in