Andrew Silva
ABSTRACT
As robots and digital assistants are deployed in the real world, these agents must be able to communicate their decision-making criteria to build trust, improve human-robot teaming, and enable collaboration. While the field of explainable artificial intelligence (xAI) has made great strides to enable such communication, these advances often assume that one xAI approach is ideally suited to each problem (e.g., decision trees to explain how to triage patients in an emergency or feature-importance maps to explain radiology reports). This fails to recognize that users have diverse experiences or preferences for interaction modalities. In this work, we present two user-studies set in a simulated autonomous vehicle (AV) domain. We investigate (1) population-level preferences for xAI and (2) personalization strategies for providing robot explanations. We find significant differences between xAI modes (language explanations, feature-importance maps, and decision trees) in both preference (p < 0.01) and performance (p < 0.05). We also observe that a participant's preferences do not always align with their performance, motivating our development of an adaptive personalization strategy to balance the two. We show that this strategy yields significant performance gains (p < 0.05), and we conclude with a discussion of our findings and implications for xAI in human-robot interactions.
Supplemental Material
Available for Download
- Anna M. H. Abrams, Pia S. C. Dautzenberg, Carla Jakobowsky, Stefan Ladwig, and Astrid M. Rosenthal-von der Pütten. 2021. A Theoretical and Empirical Reflection on Technology Acceptance Models for Autonomous Delivery Robots. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (Boulder, CO, USA) (HRI '21). Association for Computing Machinery, New York, NY, USA, 272--280. https://doi.org/10.1145/3434073.3444662Google Scholar
Digital Library
- Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoe Juozapaitis, Evan Newman, Jed Irvine, Souti Chattopadhyay, Matthew Olson, Alan Fern, and Margaret Burnett. 2020. Mental models of mere mortals with explanations of reinforcement learning. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 2 (2020), 1--37.Google ScholarDigital Library
- Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. 2019. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818 (2019).Google Scholar
- Vijay Arya, Rachel KE Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi?, et al. 2019. One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques. arXiv preprint arXiv:1909.03012 (2019).Google Scholar
- Agnes Axelsson and Gabriel Skantze. 2023. Do You Follow? A Fully Automated System for Adaptive Robot Presenters. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 102--111. https: //doi.org/10.1145/3568162.3576958Google ScholarDigital Library
- Nils Axelsson and Gabriel Skantze. 2019. Modelling Adaptive Presentations in Human-Robot Interaction using Behaviour Trees. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, Satoshi Nakamura, Milica Gasic, Ingrid Zuckerman, Gabriel Skantze, Mikio Nakano, Alexandros Papangelis, Stefan Ultes, and Koichiro Yoshino (Eds.). Association for Computational Linguistics, Stockholm, Sweden, 345--352. https://doi.org/10.18653/v1/W19--5940Google ScholarCross Ref
- Christoph Bartneck, Dana Kulic, Elizabeth Croft, and Susana Zoghbi. 2009. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International journal of social robotics 1, 1 (2009).Google ScholarCross Ref
- Kathleen Boies, John Fiset, and Harjinder Gill. 2015. Communication and trust are key: Unlocking the relationship between leadership and team performance and creativity. The Leadership Quarterly 26, 6 (2015). https://doi.org/10.1016/j. leaqua.2015.07.007Google ScholarCross Ref
- Roel Boumans, René Melis, Tibor Bosse, and Serge Thill. 2023. A Social Robot for Explaining Medical Tests and Procedures: An Exploratory Study in the Wild. In Companion of the 2023 ACM/IEEE International Conference on Human- Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 263--267. https://doi.org/10.1145/3568294. 3580085Google ScholarDigital Library
- Michelle Brachman, Qian Pan, Hyo Jin Do, Casey Dugan, Arunima Chaudhary, James M Johnson, Priyanshu Rai, Tathagata Chakraborti, Thomas Gschwind, Jim A Laredo, et al. 2023. Follow the Successful Herd: Towards Explanations for Improved Use and Mental Models of Natural Language Systems. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 220--239.Google ScholarDigital Library
- Marc-Etienne Brunet, Colleen Alkalay-Houlihan, Ashton Anderson, and Richard Zemel. 2018. Understanding the origins of bias in word embeddings. arXiv preprint arXiv:1810.03611 (2018).Google Scholar
- Rich Caruana, Hooshang Kangarloo, JD Dionisio, Usha Sinha, and David Johnson. 1999. Case-based explanation of non-case-based learning methods.. In Proceedings of the AMIA Symposium. American Medical Informatics Association.Google Scholar
- Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-Day Readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Sydney, NSW, Australia) (KDD '15). Association for Computing Machinery, New York, NY, USA, 1721--1730. https://doi.org/10.1145/2783258.2788613Google ScholarDigital Library
- Hanxiong Chen, Xu Chen, Shaoyun Shi, and Yongfeng Zhang. 2021. Generate natural language explanations for recommendation. arXiv preprint arXiv:2101.03392 (2021).Google Scholar
- Letian Chen, Sravan Jayanthi, Rohan Paleja, Daniel Martin, Viacheslav Zakharov, and Matthew Gombolay. 2022. Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations. In Proceedings of the 6th Conference on Robot Learning (CoRL), 2022.Google Scholar
- Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. 2021. Exploiting Shared Representations for Personalized Federated Learning. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). 2089--2099.Google Scholar
- Cristina Conati, Oswald Barral, Vanessa Putnam, and Lea Rieger. 2021. Toward personalized XAI: A case study in intelligent tutoring systems. Artificial Intelligence 298 (2021), 103503.Google ScholarDigital Library
- Andrew J. Cooper, Luke D. Smillie, and Philip J. Corr. 2010. A confirmatory factor analysis of the Mini-IPIP five-factor model personality scale. Personality and Individual Differences 48, 5 (2010), 688--691. https://doi.org/10.1016/j.paid. 2010.01.004Google ScholarCross Ref
- Devleena Das, Siddhartha Banerjee, and Sonia Chernova. 2021. Explainable AI for Robot Failures: Generating Explanations That Improve User Assistance in Fault Recovery. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (Boulder, CO, USA) (HRI '21). Association for Computing Machinery, New York, NY, USA, 351--360. https: //doi.org/10.1145/3434073.3444657Google ScholarDigital Library
- Devleena Das, Siddhartha Banerjee, and Sonia Chernova. 2021. Explainable AI for Robot Failures: Generating Explanations That Improve User Assistance in Fault Recovery. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (Boulder, CO, USA) (HRI '21). Association for Computing Machinery, New York, NY, USA, 351--360. https: //doi.org/10.1145/3434073.3444657Google ScholarDigital Library
- Devleena Das, Sonia Chernova, and Been Kim. 2023. State2Explanation: Conceptbased explanations to benefit agent learning and user understanding. In Proceedings of the Conference on Neural Information Processing Systems.Google Scholar
- Devleena Das, Been Kim, and Sonia Chernova. 2023. Subgoal-Based Explanations for Unreliable Intelligent Decision Support Systems. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI '23). Association for Computing Machinery, New York, NY, USA, 240--250. https://doi.org/10.1145/3581641.3584055Google ScholarDigital Library
- Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. 2020. Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461 (2020).Google Scholar
- Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2019. Eraser: A benchmark to evaluate rationalized nlp models. arXiv preprint arXiv:1911.03429 (2019).Google Scholar
- Canh T Dinh, Nguyen H Tran, and Tuan Dung Nguyen. 2020. Personalized federated learning with moreau envelopes. arXiv preprint arXiv:2006.08848 (2020).Google Scholar
- Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).Google Scholar
- Finale Doshi-Velez, Mason Kortz, Ryan Budish, Christopher Bavitz, Samuel J. Gershman, David O'Brien, Kate Scott, Stuart Shieber, Jim Waldo, David Weinberger, AdrianWeller, and AlexandraWood. 2017. Accountability of AI Under the Law: The Role of Explanation. SSRN Scholarly Paper ID 3064761. Social Science Research Network, Rochester, NY. https://doi.org/10.2139/ssrn.3064761Google ScholarCross Ref
- Upol Ehsan and Mark O Riedl. 2021. Explainability pitfalls: Beyond dark patterns in explainable AI. arXiv preprint arXiv:2109.12480 (2021).Google Scholar
- Upol Ehsan, Pradyumna Tambwekar, Larry Chan, Brent Harrison, and Mark O Riedl. 2019. Automated rationale generation: a technique for explainable AI and its effects on human perceptions. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 263--274.Google ScholarDigital Library
- Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. 2020. Personalized federated learning: A meta-learning approach. arXiv preprint arXiv:2002.07948 (2020).Google Scholar
- Andrea Ferrario and Michele Loi. 2022. How explainability contributes to trust in AI. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 1457--1466.Google ScholarDigital Library
- Paul D. S. Fink, Anas Abou Allaban, Omoruyi E. Atekha, Raymond J. Perry, Emily S. Sumner, Richard R. Corey, Velin Dimitrov, and Nicholas A. Giudice. 2023. Expanded Situational Awareness Without Vision: A Novel Haptic Interface for Use in Fully Autonomous Vehicles. In Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 54--62. https: //doi.org/10.1145/3568162.3576975Google ScholarDigital Library
- Naomi T. Fitter, Megan Strait, Eloise Bisbee, Maja J. Mataric, and Leila Takayama. 2021. You're Wigging Me Out! Is Personalization of Telepresence Robots Strictly Positive?. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (Boulder, CO, USA) (HRI '21). Association for Computing Machinery, New York, NY, USA, 168--176. https://doi.org/10.1145/ 3434073.3444675Google ScholarDigital Library
- Leilani H Gilpin, Andrew R Paley, Mohammed A Alam, Sarah Spurlock, and Kristian J Hammond. 2022. " Explanation" is Not a Technical Term: The Problem of Ambiguity in XAI. arXiv preprint arXiv:2207.00007 (2022).Google Scholar
- Amar Halilovic and Felix Lindner. 2023. Visuo-Textual Explanations of a Robot's Navigational Choices. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 531--535. https://doi.org/10.1145/3568294.3580141Google ScholarDigital Library
- Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik. 2020. Lower bounds and optimal algorithms for personalized federated learning. arXiv preprint arXiv:2010.02372 (2020).Google Scholar
- Filip Hanzely and Peter Richtárik. 2020. Federated learning of a mixture of global and local models. arXiv preprint arXiv:2002.05516 (2020).Google Scholar
- Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In HumanGoogle Scholar
- Peter Hase and Mohit Bansal. 2020. Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 5540--5552. https://doi.org/10.18653/v1/ 2020.acl-main.491Google ScholarCross Ref
- Bradley Hayes and Julie A. Shah. 2017. Improving Robot Controller Transparency Through Autonomous Policy Explanation. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (Vienna, Austria) (HRI '17). Association for Computing Machinery, New York, NY, USA, 303--312. https://doi.org/10.1145/2909824.3020233Google ScholarDigital Library
- Robert R Hoffman, John W Coffey, Kenneth M Ford, and Mary Jo Carnot. 2001. Storm-lk: A human-centered knowledge model for weather forecasting. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 1. 752--752.Google ScholarCross Ref
- Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608 (2018).Google Scholar
- Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. 2023. Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. Frontiers in Computer Science 5 (2023), 1096257.Google ScholarCross Ref
- Andreas Holzinger, André Carrington, and Heimo Müller. 2020. Measuring the quality of explanations: the system causability scale (SCS). KI-Künstliche Intelligenz (2020), 1--6.Google Scholar
- Fang-I Hsiao, Jui-Hsuan Kuo, and Min Sun. 2019. Learning a multi-modal policy via imitating demonstrations with mixed behaviors. arXiv preprint arXiv:1903.10304 (2019).Google Scholar
- Tobias Huber, Katharina Weitz, Elisabeth André, and Ofra Amir. 2021. Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps. Artificial Intelligence 301 (2021), 103571.Google ScholarCross Ref
- Amanda Hutton, Alexander Liu, and Cheryl Martin. 2012. Crowdsourcing evaluations of classifier interpretability. In 2012 AAAI Spring Symposium Series.Google Scholar
- Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 3543--3556. https://doi.org/10.18653/v1/N19--1357Google ScholarCross Ref
- Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. 2000. Foundations for an empirically determined scale of trust in automated systems. International journal of cognitive ergonomics 4, 1 (2000).Google ScholarCross Ref
- Yihan JiangGoogle Scholar
- y, Keith Rush, and Sreeram Kannan. 2019. Improving federated learning personalization via model agnostic meta learning. arXiv preprint arXiv:1909.12488 (2019).Google Scholar
- Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. 2020. A survey of algorithmic recourse: definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.04050 (2020).Google Scholar
- Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. 2021. Algorithmic recourse: from counterfactual explanations to interventions. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 353--362.Google ScholarDigital Library
- Joongheon Kim, Seunghoon Park, Soyi Jung, and Seehwan Yoo. 2021. Spatiotemporal split learning. In 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks-Supplemental Volume (DSN-S). IEEE, 11--12.Google Scholar
- Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org.Google ScholarDigital Library
- Pigi Kouki, James Schaffer, Jay Pujara, John O'Donovan, and Lise Getoor. 2020. Generating and understanding personalized explanations in hybrid recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 4 (2020), 1--40.Google ScholarDigital Library
- Johannes Maria Kraus, Julia Merger, Felix Gröner, and Jessica Pätz. 2023. 'Sorry' Says the Robot: The Tendency to Anthropomorphize and Technology Affinity Affect Trust in Repair Strategies after Error. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 436--441. https: //doi.org/10.1145/3568294.3580122Google ScholarDigital Library
- Alyssa Kubota, Emma I. C. Peterson, Vaishali Rajendren, Hadas Kress-Gazit, and Laurel D. Riek. 2020. JESSIE: Synthesizing Social Robot Behaviors for Personalized Neurorehabilitation and Beyond. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (Cambridge, United Kingdom) (HRI '20). Association for Computing Machinery, New York, NY, USA, 121--130. https://doi.org/10.1145/3319502.3374836Google ScholarDigital Library
- Vivian Lai, Yiming Zhang, Chacha Chen, Q Vera Liao, and Chenhao Tan. 2023. Selective Explanations: Leveraging Human Input to Align Explainable AI. arXiv preprint arXiv:2301.09656 (2023).Google Scholar
- Seong Hee Lee, Nicholas Britten, Avram Block, Aryaman Pandya, Malte F. Jung, and Paul Schmitt. 2023. Coming In! Communicating Lane Change Intent in Autonomous Vehicles. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 394--398. https://doi.org/10.1145/3568294.3580113Google ScholarDigital Library
- Seong Hee Lee, Vaidehi Patil, Nicholas Britten, Avram Block, Aryaman Pandya, Malte F. Jung, and Paul Schmitt. 2023. Safe to Approach: Insights on Autonomous Vehicle Interaction Protocols with First Responders. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 399--402. https://doi.org/10.1145/3568294.3580114Google ScholarDigital Library
- Daliang Li and Junpu Wang. 2019. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581 (2019).Google Scholar
- Jamy Li, Rebecca Currano, David Sirkin, David Goedicke, Hamish Tennent, Aaron Levine, Vanessa Evers, and Wendy Ju. 2020. On-Road and Online Studies to Investigate Beliefs and Behaviors of Netherlands, US and Mexico Pedestrians Encountering Hidden-Driver Vehicles. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (Cambridge, United Kingdom) (HRI '20). Association for Computing Machinery, New York, NY, USA, 141--149. https://doi.org/10.1145/3319502.3374790Google ScholarDigital Library
- Lei Li, Yongfeng Zhang, and Li Chen. 2023. Personalized prompt learning for explainable recommendation. ACM Transactions on Information Systems 41, 4 (2023), 1--26.Google ScholarDigital Library
- Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems 2 (2020), 429--450.Google Scholar
- Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: informing design practices for explainable AI user experiences. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1--15.Google ScholarDigital Library
- Zhaojiang Lin, Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2019. Personalizing dialogue agents via meta-learning. arXiv preprint arXiv:1905.10033 (2019).Google Scholar
- Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2021. Explainable ai: A review of machine learning interpretability methods. Entropy 23, 1 (2021), 18.Google ScholarCross Ref
- Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2020. Explainable reinforcement learning through a causal lens. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 2493--2500.Google ScholarCross Ref
- Martijn Millecamp, Nyi Nyi Htun, Cristina Conati, and Katrien Verbert. 2020. What's in a User? Towards Personalising Transparency for Music Recommender Interfaces. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 173--182.Google ScholarDigital Library
- Martijn Millecamp, Sidra Naveed, Katrien Verbert, and Jürgen Ziegler. 2019. To explain or not to explain: The effects of personal characteristics when explaining feature-based recommendations in different domains. In Proceedings of the 6th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems, Vol. 2450. CEUR; http://ceur-ws. org/Vol-2450/paper2. pdf, 10--18.Google Scholar
- Dylan Moore, Rebecca Currano, Michael Shanks, and David Sirkin. 2020. Defense Against the Dark Cars: Design Principles for Griefing of Autonomous Vehicles. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (Cambridge, United Kingdom) (HRI '20). Association for Computing Machinery, New York, NY, USA, 201--209. https://doi.org/10.1145/3319502. 3374796Google ScholarDigital Library
- James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explainable prediction of medical codes from clinical text. arXiv preprint arXiv:1802.05695 (2018).Google Scholar
- Dong Nguyen. 2018. Comparing Automatic and Human Evaluation of Local Explanations for Text Classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, New Orleans, Louisiana, 1069--1078. https://doi.org/10.18653/v1/ N18--1097Google ScholarCross Ref
- Tatsuya Nomura, Tomohiro Suzuki, Takayuki Kanda, and Kensuke Kato. 2006. Measurement of negative attitudes toward robots. Interaction Studies 7, 3 (2006), 437--454.Google ScholarCross Ref
- Daniel Omeiza, Helena Webb, Marina Jirotka, and Lars Kunze. 2022. Explanations in Autonomous Driving: A Survey. IEEE Transactions on Intelligent Transportation Systems 23, 8 (2022), 10142--10162. https://doi.org/10.1109/TITS. 2021.3122865Google ScholarCross Ref
- Rohan Paleja, Muyleng Ghuy, Nadun Ranawaka Arachchige, and Matthew Gombolay. 2021. The Utility of Explainable AI in Ad Hoc Human-Machine Teaming. In Proceedings of the Conference on Neural Information Processing Systems.Google Scholar
- Rohan Paleja, Andrew Silva, Letian Chen, and Matthew Gombolay. 2020. Interpretable and Personalized Apprenticeship Scheduling: Learning Interpretable Scheduling Policies from Heterogeneous User Demonstrations. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 6417--6428.Google Scholar
- Raja Parasuraman and Dietrich H. Manzey. 2010. Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors 52, 3 (2010), 381--410. https://doi.org/10.1177/0018720810376055 arXiv:https://doi.org/10.1177/0018720810376055 PMID: 21077562.Google ScholarCross Ref
- Jeyoung Park, Jeeyeon Kim, Da-Young Kim, Juhyun Kim, Min-Gyu Kim, Jihwan Choi, andWonHyong Lee. 2022. User Perception on Personalized Explanation by Science Museum Docent Robot. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 973--975. https://doi.org/10.1109/HRI53351. 2022.9889654Google ScholarCross Ref
- Matthias Paulik, Matt Seigel, Henry Mason, Dominic Telaar, Joris Kluivers, Rogier van Dalen, ChiWai Lau, Luke Carlson, Filip Granqvist, Chris Vandevelde, et al. 2021. Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications. arXiv preprint arXiv:2102.08503 (2021).Google Scholar
- Forough Poursabzi-Sangdeh, Daniel G Goldstein, JakeMHofman, JenniferWortmanWortman Vaughan, and HannaWallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.Google ScholarDigital Library
- Samantha Reig, Michal Luria, Janet Z. Wang, Danielle Oltman, Elizabeth Jeanne Carter, Aaron Steinfeld, Jodi Forlizzi, and John Zimmerman. 2020. Not Some Random Agent: Multi-Person Interaction with a Personalizing Service Robot. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (Cambridge, United Kingdom) (HRI '20). Association for Computing Machinery, New York, NY, USA, 289--297. https://doi.org/10.1145/3319502. 3374795Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13--17, 2016.Google ScholarDigital Library
- Amelie Sophie Robrecht, Markus Rothgänger, and Stefan Kopp. 2023. A Study on the Benefits and Drawbacks of Adaptivity in AI-generated Explanations. In Proceedings of the 23rd ACM International Conference on Intelligent Virtual Agents. Association for Computing Machinery, Würzburg, Germany.Google ScholarDigital Library
- Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. 2021. Interpretable machine learning: Fundamental principles and 10 grand challenges. arXiv preprint arXiv:2103.11251 (2021).Google Scholar
- Ognjen Rudovic, Nicolas Tobis, Sebastian Kaltwang, Björn Schuller, Daniel Rueckert, Jeffrey F Cohn, and Rosalind W Picard. 2021. Personalized Federated Deep Learning for Pain Estimation From Face Images. arXiv preprint arXiv:2101.04800 (2021).Google Scholar
- Nadine Schlicker and Markus Langer. 2021. Towards Warranted Trust: A Model on the Relation Between Actual and Perceived System Trustworthiness. In Proceedings of Mensch Und Computer 2021 (Ingolstadt, Germany) (MuC '21). Association for Computing Machinery, New York, NY, USA, 325--329. https: //doi.org/10.1145/3473856.3474018Google ScholarDigital Library
- Florian Schröder, Sonja Stange, and Stefan Kopp. 2023. Resolving References in Natural Language Explanation Requests about Robot Behavior in HRI. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (Stockholm, Sweden) (HRI '23). Association for Computing Machinery, New York, NY, USA, 772--774. https://doi.org/10.1145/3568294.3579981Google ScholarDigital Library
- Mariah L Schrum, Erin Hedlund-Botti, and Matthew Gombolay. 2022. Reciprocal MIND MELD: Improving Learning From Demonstration via Personalized, Reciprocal Teaching. In Proceedings of the 6th Conference on Robot Learning (CoRL), 2022.Google Scholar
- Mariah L Schrum, Erin Hedlund-Botti, Nina Moorman, and Matthew C Gombolay. 2022. MIND MELD: Personalized Meta-Learning for Robot-Centric Imitation Learning. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction. 157--165.Google ScholarDigital Library
- Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. 2018. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and service robotics. Springer.Google Scholar
- Avital Shulner-Tal, Tsvi Kuflik, and Doron Kliger. 2022. Enhancing Fairness Perception--Towards Human-Centred AI and Personalized Explanations Understanding the Factors Influencing Laypeople's Fairness Perceptions of Algorithmic Decisions. International Journal of Human--Computer Interaction (2022), 1--28.Google Scholar
- Avital Shulner-Tal, Tsvi Kuflik, and Doron Kliger. 2022. Fairness, explainability and in-between: Understanding the impact of different explanation methods on non-expert users' perceptions of fairness toward an algorithmic system. Ethics and Information Technology 24, 1 (2022), 2.Google ScholarDigital Library
- AndrewSilva, Rohit Chopra, and MatthewGombolay. 2022. Cross-Loss Influence Functions to Explain Deep Network Representations. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 151), Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera (Eds.). PMLR, 1--17. https://proceedings.mlr.press/v151/ silva22a.htmlGoogle Scholar
- Andrew Silva, Matthew Gombolay, Taylor Killian, Ivan Jimenez, and Sung-Hyun Son. 2020. Optimization Methods for Interpretable Differentiable Decision Trees Applied to Reinforcement Learning. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Silvia Chiappa and Roberto Calandra (Eds.), Vol. 108. PMLR.Google Scholar
- Andrew Silva, Katherine Metcalf, Nicholas Apostoloff, and Barry-John Theobald. 2022. FedEmbed: Personalized Private Federated Learning. arXiv preprint arXiv:2202.09472 (2022).Google Scholar
- AndrewSilva, Mariah Schrum, Erin Hedlund-Botti, Nakul Gopalan, and Matthew Gombolay. 2022. Explainable artificial intelligence: Evaluating the objective and subjective impacts of xai on human-agent interaction. International Journal of Human--Computer Interaction (2022), 1--15.Google Scholar
- Andrew Silva, Pradyumna Tambwekar, and Matthew Gombolay. 2023. FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings. In Findings of the Association for Computational Linguistics: EACL 2023. Association for Computational Linguistics, Dubrovnik, Croatia.Google ScholarCross Ref
- Kacper Sokol and Peter Flach. 2020. Explainability fact sheets: a framework for systematic assessment of explainable approaches. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency.Google ScholarDigital Library
- Sonja Stange and Stefan Kopp. 2020. Effects of a Social Robot's Self-Explanations on How Humans Understand and Evaluate Its Behavior. In Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (Cambridge, United Kingdom) (HRI '20). Association for Computing Machinery, New York, NY, USA, 619--627. https://doi.org/10.1145/3319502.3374802Google ScholarDigital Library
- Erik Strumbelj and Igor Kononenko. 2014. Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems 41, 3 (2014), 647--665.Google Scholar
- Xavier Suau, Luca Zappella, and Nicholas Apostoloff. 2020. Finding experts in transformer models. arXiv preprint arXiv:2005.07647 (2020).Google Scholar
- Aviv Tamar, Khashayar Rohanimanesh, Yinlam Chow, Chris Vigorito, Ben Goodrich, Michael Kahane, and Derik Pridmore. 2018. Imitation learning from visual data with multiple intentions. In International Conference on Learning Representations.Google Scholar
- Pradyumna Tambwekar and Matthew Gombolay. 2023. Towards Reconciling Usability and Usefulness of Explainable AI Methodologies. arXiv preprint arXiv:2301.05347 (2023).Google Scholar
- Nava Tintarev and Judith Masthoff. 2012. Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction 22, 4--5 (2012), 399--439.Google ScholarDigital Library
- Paul Voigt and Axel Von dem Bussche. 2017. The EU General Data Protection Regulation (GDPR). A Practical Guide, 1st Ed., Cham: Springer International Publishing (2017).Google ScholarCross Ref
- Tong Wang, Cynthia Rudin, Finale Velez-Doshi, Yimin Liu, Erica Klampfl, and Perry MacNeille. 2016. Bayesian rule sets for interpretable classification. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE.Google ScholarCross Ref
- Rebecca Wiczorek and Dietrich Manzey. 2014. Supporting attention allocation in multitask environments: Effects of likelihood alarm systems on trust, behavior, and performance. Human factors 56, 7 (2014), 1209--1221.Google ScholarCross Ref
- Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not Explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 11--20. https://doi.org/10.18653/v1/D19--1002Google ScholarCross Ref
- X Jessie Yang, Vaibhav V Unhelkar, Kevin Li, and Julie A Shah. 2017. Evaluating effects of user experience and system transparency on trust in automation. In 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI. IEEE, 408--416.Google ScholarDigital Library
- Dewei Yi, Jinya Su, Cunjia Liu, Mohammed Quddus, and Wen-Hua Chen. 2019. A machine learning based personalized system for driving state recognition. Transportation Research Part C: Emerging Technologies 105 (2019), 241--261. https: //doi.org/10.1016/j.trc.2019.05.042Google ScholarCross Ref
- Ming Yin, JenniferWortman Vaughan, and HannaWallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1--12.Google ScholarDigital Library
Index Terms
- Towards Balancing Preference and Performance through Adaptive Personalized Explainability
Recommendations
Toward personalized XAI: A case study in intelligent tutoring systems
AbstractOur research is a step toward ascertaining the need for personalization in XAI, and we do so in the context of investigating the value of explanations of AI-driven hints and feedback in Intelligent Tutoring Systems (ITS). We added an ...
A manifesto on explainability for artificial intelligence in medicine
AbstractThe rapid increase of interest in, and use of, artificial intelligence (AI) in computer applications has raised a parallel concern about its ability (or lack thereof) to provide understandable, or explainable, output to users. This ...
Highlights- AI in Medicine becomes increasingly ubiquitous, with new concerns and questions:
Human Centered Explainability for Intelligent Vehicles – A User Study
AutomotiveUI '22: Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular ApplicationsAdvances in artificial intelligence (AI) are leading to an increased use of algorithm-generated user-adaptivity in everyday products. Explainable AI aims to make algorithmic decision-making more transparent to humans. As future vehicles become more ...
Comments