Encoding Social & Ethical Values in Autonomous Navigation: Philosophies Behind an Interactive Online Demonstration
Authors: 
Yun Tang, Luke Moffat, Weisi Guo, Corinne May-Chahal, Joe Deville, Antonios TsourdosAuthors Info & Claims
Yun Tang, Luke Moffat, Weisi Guo, Corinne May-Chahal, Joe Deville, Antonios TsourdosAuthors Info & ClaimsArticle No.: 21, Pages 1 - 9
Abstract
Autonomous Systems (ASs) interacting with human societies raises complex social & ethical challenges. This paper argues that one way of scaffolding human trust in ASs is through the encoding of ethical, legal and social impact (ELSI) considerations in the ASs’ decision-making processes. Existing ELSI-encoding efforts often focus on the implementation of rule-based and risk-based approaches, leaving key questions unanswered - what are the relationships between ELSI-encoding software logic in ASs and human ethical practises; what ethical approaches cannot be easily translated into software rules and numeric risks; and what are the implications of this for ethical AS?
To answer these questions, we review and discuss different ELSI-encoding approaches in ASs from a new perspective, i.e., their relationships with classic human ethics philosophies. We also explore the feasibility of large language models (LLMs)-based ELSI-encoding practices in overcoming the limitations of rule-based and risk-based approaches and the associated challenges. To foster understanding, facilitate knowledge exchange and inspire discussion among cross-disciplinary research communities, we build and publish the first online interactive playground demonstrating different ELSI-encoding approaches on the same AS decision-making process. We welcome feedback and contributions in making this platform truly beneficial to trustworthy autonomous system research communities.
References
[1]
Artificial Analysis. 2024. LLM Leaderboard. https://artificialanalysis.ai/leaderboards/models
[2]
Anthropic. 2024. Claude. https://claude.ai/
[3]
Hugo Araujo, Mohammad Reza Mousavi, and Mahsa Varshosaz. 2023. Testing, validation, and verification of robotic and autonomous systems: a systematic review. ACM Transactions on Software Engineering and Methodology 32, 2 (2023), 1–61.
[4]
Edmond Awad, Sydney Levine, Andrea Loreggia, Nicholas Mattei, Iyad Rahwan, Francesca Rossi, Kartik Talamadupula, Joshua Tenenbaum, and Max Kleiman-Weiner. 2022. When is it acceptable to break the rules? knowledge representation of moral judgement based on empirical data. arXiv preprint arXiv:2201.07763 (2022).
[5]
Davide Bacciu, Siranush Akarmazyan, Eric Armengaud, Manlio Bacco, George Bravos, Calogero Calandra, Emanuele Carlini, Antonio Carta, Pietro Cassarà, Massimo Coppola, 2021. Teaching-trustworthy autonomous cyber-physical applications through human-centred intelligence. In 2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS). IEEE, 1–6.
[6]
Somnath Banerjee, Sayan Layek, Rima Hazra, and Animesh Mukherjee. 2024. How (un) ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries. arXiv preprint arXiv:2402.15302 (2024).
[7]
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17682–17690.
[8]
Monika Buscher, Markus Bylund, Pedro Sanches, Leonardo Ramirez, and Lisa Wood. 2013. A new manhattan project? Interoperability and ethics in emergency response systems of systems. In 10th International ISCRAM Conference. 426–431.
[9]
Monika Büscher, Preben Holst Mogensen, and Margit Kristensen. 2009. When and how (not) to trust IT? Supporting virtual emergency teamwork. International Journal of Information Systems for Crisis Response and Management (IJISCRAM) 1, 2 (2009), 1–15.
[10]
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Changxin Wang, Zhifeng Gao, Yongge Li, Mujie Lin, Shuwen Yang, 2024. Sciassess: Benchmarking llm proficiency in scientific literature analysis. arXiv preprint arXiv:2403.01976 (2024).
[11]
Axel Constant, Hannes Westermann, Bryan Wilson, Alex Kiefer, Ines Hipolito, Sylvain Pronovost, Steven Swanson, Mahault Albarracin, and Maxwell JD Ramstead. 2024. A Path Towards Legal Autonomy: An interoperable and explainable approach to extracting, transforming, loading and computing legal information using large language models, expert systems and Bayesian networks. arXiv preprint arXiv:2403.18537 (2024).
[12]
Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2023. Jailbreaker: Automated jailbreak across multiple large language model chatbots. arXiv preprint arXiv:2307.08715 (2023).
[13]
Harry Frankfurt. 2018. Freedom of the Will and the Concept of a Person. In Agency And Responsiblity. Routledge, 77–91.
[14]
Maximilian Geisslinger, Franziska Poszler, Johannes Betz, Christoph Lütge, and Markus Lienkamp. 2021. Autonomous driving ethics: From trolley problem to ethics of risk. Philosophy & Technology 34, 4 (2021), 1033–1055.
[15]
Google. 2024. Gemini. https://gemini.google.com
[16]
Balint Gyevnar. 2024. Towards Trustworthy Autonomous Systems via Conversations and Explanations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 23389–23390.
[17]
Elliott Hauser, Yao-Cheng Chan, Parth Chonkar, Geethika Hemkumar, Huihai Wang, Daksh Dua, Shikhar Gupta, Efren Mendoza Enriquez, Tiffany Kao, Justin Hart, 2023. "What’s That Robot Doing Here?": Perceptions Of Incidental Encounters With Autonomous Quadruped Robots. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems. 1–15.
[18]
Jianliang He, Siyu Chen, Fengzhuo Zhang, and Zhuoran Yang. 2024. From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems. arXiv preprint arXiv:2405.19883 (2024).
[19]
Shinnosuke Kondo, Kaori Ota, Erina Takeshita, Naoto Yoshimoto, and Yu Nakayama. 2022. Autonomous tethered drone cell for IoT connectivity in 6G communications. In 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring). IEEE, 1–6.
[20]
Joshua Krook and Jan Blockx. 2023. Recommender systems, autonomy and user engagement. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems. 1–9.
[21]
California State Legislature. 2012. DIVISION 16.6. Autonomous Vehicles [38750 - 38755]. https://leginfo.legislature.ca.gov/faces/codes_displayText.xhtml?lawCode=VEH&division=16.6.&title=&part=&chapter=&article=
[22]
Chen Li, Weisi Guo, Schyler Chengyao Sun, Saba Al-Rubaye, and Antonios Tsourdos. 2020. Trustworthy Deep Learning in 6G-Enabled Mass Autonomy: From Concept to Quality-of-Trust Key Performance Indicators. IEEE Vehicular Technology Magazine 15, 4 (2020), 112–121. https://doi.org/10.1109/MVT.2020.3017181
[23]
Lizi Liao, Grace Hui Yang, and Chirag Shah. 2023. Proactive conversational agents in the post-chatgpt world. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3452–3455.
[24]
Michael Liegl, Rachel Oliphant, and Monika Büscher. 2015. Ethically Aware IT Design for Emergency Response: From Co-Design to ELSI Co-Design’. In ISCRAM.
[25]
Guoman Liu, Yufeng Luo, and Jing Sheng. 2024. Applying AVWEWM to ethical decision-making during autonomous vehicle crashes. Scientific reports 14, 1 (2024), 3924.
[26]
Quan Khanh Luu, Xiyu Deng, Anh Van Ho, and Yorie Nakahira. 2024. Context-aware LLM-based Safe Control Against Latent Risks. arXiv preprint arXiv:2403.11863 (2024).
[27]
Roger C Mayer, James H Davis, and F David Schoorman. 1995. An integrative model of organizational trust. Academy of management review 20, 3 (1995), 709–734.
[28]
Luke Moffat. 2023. Relational approaches to autonomous systems ethics. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems. 1–7.
[29]
Mohammad Naiseh and Paurav Shukla. 2023. The well-being of Autonomous Vehicles (AVs) users under uncertain situations. In Proceedings of the first international symposium on trustworthy autonomous systems. 1–8.
[30]
OpenAI. 2024. ChatGPT | OpenAI. https://openai.com/chatgpt/
[31]
OpenAI. 2024. Function Calling - OpenAI API. https://platform.openai.com/docs/guides/function-calling
[32]
Katie Parnell, Siobhan Merriman, Sinem Getir Yaman, Katherine Plant, and Radu Calinescu. 2023. Resilient strategies for socially compliant autonomous assistive dressing robots. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems. 1–9.
[33]
Ali Roshanianfard, Noboru Noguchi, Hiroshi Okamoto, and Kazunobu Ishii. 2020. A review of autonomous agricultural vehicles (The experience of Hokkaido University). Journal of Terramechanics 91 (2020), 155–183.
[34]
YV Satyanarayana. 2009. Ethics: theory and practice. Pearson Education India.
[35]
Shai Shalev-Shwartz, Shaked Shammah, and Amnon Shashua. 2017. On a formal model of safe and scalable self-driving cars. arXiv preprint arXiv:1708.06374 (2017).
[36]
Guangzhi Sun, Xiao Zhan, and Jose Such. 2024. Building Better AI Agents: A Provocation on the Utilisation of Persona in LLM-based Conversational Agents. In Proceedings of the 6th ACM Conference on Conversational User Interfaces. 1–6.
[37]
Yun Tang, Antonio A Bruto Da Costa, Xizhe Zhang, Irvine Patrick, Siddartha Khastgir, and Paul Jennings. 2023. Domain knowledge distillation from large language model: An empirical study in the autonomous driving domain. In 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 3893–3900.
[38]
John J Tilley. 2000. Cultural relativism. Human rights quarterly 22, 2 (2000), 501–547.
[39]
Longcong Wang, Linyu Lin, and Nam Dinh. 2024. Trustworthiness modeling and evaluation for a nearly autonomous management and control system. Reliability Engineering & System Safety 245 (2024), 110008.
[40]
Wikipedia. 2024. A* Search Algorithm - Wikipedia. https://en.wikipedia.org/wiki/A*_search_algorithm
[41]
James Wilson, Greg Chance, Peter Winter, Suet Lee, Emma Milner, Dhaminda Abeywickrama, Shane Windsor, John Downer, Kerstin Eder, Jonathan Ives, 2023. Trustworthy swarms. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems. 1–11.
[42]
Sinem Getir Yaman, Charlie Burholt, Maddie Jones, Radu Calinescu, and Ana Cavalcanti. 2023. Specification and validation of normative rules for autonomous agents. In International Conference on Fundamental Approaches to Software Engineering. Springer Nature Switzerland Cham, 241–248.
[43]
Yizhen Zheng, Huan Yee Koh, Jiaxin Ju, Anh TN Nguyen, Lauren T May, Geoffrey I Webb, and Shirui Pan. 2023. Large language models for scientific synthesis, inference and explanation. arXiv preprint arXiv:2310.07984 (2023).
Index Terms
- Encoding Social & Ethical Values in Autonomous Navigation: Philosophies Behind an Interactive Online Demonstration
Index terms have been assigned to the content through auto-classification.
Recommendations
Toward a Normative Ethical Claims of Ibibio Social Norms, Values, and Beliefs
This study examined the normative ethical claims, social norms and values laden in Ibibio social system and culture. These claims institutes avowal about ethical considerations and make moral judgments. This study presents a socio-cultural description ...
The Internet, ethical values, and conceptual frameworks: an introduction to Cyberethics
What exactly is Cyberethics? How did the field develop? What are some of the central issues and themes in this field, and what methodologies are used by those working in this area of applied ethics? These and related questions are considered in the ...
Ethical approaches and autonomous systems
AbstractIn this paper we consider how the three main approaches to ethics – deontology, consequentialism and virtue ethics – relate to the implementation of ethical agents. We provide a description of each approach and how agents might be ...
Comments
Information & Contributors
Information
Published In
September 2024
335 pages
ISBN:9798400709890
DOI:10.1145/3686038
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Engineering and Physical Sci- ences Research Council Trustworthy Autonomous Systems Security Node
Conference
TAS '24
TAS '24: Second International Symposium on Trustworthy Autonomous Systems
September 16 - 18, 2024
TX, Austin, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 190Total Downloads
- Downloads (Last 12 months)190
- Downloads (Last 6 weeks)47
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in