skip to main content
10.1145/3597512.3600203acmotherconferencesArticle/Chapter ViewAbstractPublication PagestasConference Proceedingsconference-collections
research-article

Designing for Trust A Human-Centered Methodology for Identifying Metrics of Operational Trust in Mission Autonomy

Published: 11 July 2023 Publication History

Abstract

Autonomous systems are becoming increasingly more complex and more ubiquitous across a variety of domains. Specifically in complex, operational scenarios, these systems have the potential to have a large impact, primarily aiding the humans living and working in these environments. Therefore, understanding operational trust in mission autonomy is critical for the appropriate and responsible design, development, implementation, and use of complex, intelligent systems. This paper outlines a human-centered approach taken to begin to identify and develop metrics for understanding operational trust in mission autonomy.

References

[1]
Poornima Madhavan and Douglas A. Wiegmann. 2007. Similarities and differences between human-human and human-automation trust: An integrative review. Theoretical Issues in Ergonomics Science 8, 4 (May 2007), 277–301. https://doi.org/10.1080/14639220500337708.>
[2]
Matthew Johnson and Alonso Vera. 2019. No AI Is an Island: The Case for Teaming Intelligence. AI Magazine 40, 1 (March 2019), 16–28. https://ojs.aaai.org/aimagazine/index.php/aimagazine/issue/view/225.>
[3]
Jaret Riddick. 2022. Operational Trust in Mission Autonomy (OPTIMA). National Security Innovation Network. https://www.nsin.mil/assets/downloads/Operational_Trust_in_Mission_Autonomy_START_20220628.pdf>
[4]
Denise M. Rousseau, Sim B. Sitkin, Ronald S. Burt, and Colin Camerer. 1998. Not So Different After All: A Cross-Discipline View of Trust. Academy of Management Review 23, 3 (July 1998), 393–404. https://journals.aom.org/doi/10.5465/amr.1998.926617.>
[5]
Bonnie M. Muir. 1987. Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies 27, 5-6 (November 1987), 527–539. https://doi.org/10.1016/S0020-7373(87)80013-5>
[6]
Nancy K. Lankton, D. Harrison McKnight, and John Tripp. 2015. Technology, humanness, and trust: Rethinking trust in technology. Journal of the Association for Information Systems 16, 10 (October 2015), 1–33. https://aisel.aisnet.org/jais/vol16/iss10/1>
[7]
Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data & Society 5, 1 (March 2018). https://doi.org/10.1177/2053951718756684>
[8]
Kristin E. Schaefer, Jessie Y.C. Chen, and P.A. Hancock. 2016. A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors 58, 3 (March 2016), 377–400. https://doi.org/10.1177/0018720816634228>
[9]
John D. Lee, and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. Human Factors 46, 1 (August 2004), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392>
[10]
Joseph B. Lyons, Kevin T. Wynne, Sean Mahoney, and Mark A. Roebke. 2019. Chapter 6 - Trust and Human-Machine Teaming: A Qualitative Study. Academic Press, 101–116. https://doi.org/10.1016/B978-0-12-817636-8.00006-5>
[11]
Chad C. Tossell, Boyoung Kim, Bianca Donadio, Ewart J. de Visser, Ryan Holec, and Elizabeth Phillips. 2020. Appropriately Representing Military Tasks for Human-Machine Teaming Research. In Proceedings of the HCI International 2020 - Late Breaking Papers: Virtual and Augmented Reality (Stephanidis, C., Chen, J.Y.C., and Fragomeni, G. eds.). Lecture Notes in Computer Science, vol 12428. Springer, Cham, Article 19, 1–7. https://doi.org/10.1007/978-3-030-59990-4_19>
[12]
Nicole Blatt. 2004. Operational Trust: A New Look at the Human Requirement in Network Centric Warfare. In Proceedings of the 9th International Command and Control Research and Technology Symposium Coalition Transformation: An Evolution of People, Processes, and Technology to Enhance Interoperability. https://apps.dtic.mil/sti/pdfs/ADA466612.pdf>
[13]
Gari Palmer, Anne Selwyn, and Dan Zwillinger. 2014. The “Trust V” - Building and measuring trust in autonomous systems. Raytheon Co., Tewksbury, MA, Tech. Rep., 2014. https://link.springer.com/chapter/10.1007/978-1-4899-7668-0_4>
[14]
The Johns Hopkins Institute for Assured Autonomy. 2022. Johns Hopkins IAA - About. Accessed: 2/16/2022. https://iaa.jhu.edu/about/>
[15]
Jon Kolko. 2012. Wicked Problems: Problems Worth Solving. Stanford Social Innovation Review 10, 3 (March 2012), 1–5. https://doi.org/10.48558/1REG-NX98>
[16]
Susan Gasson. 2003. Human-centered vs. user-centred approaches to information system design. The Journal of Information Technology Theory and Application (JITTA) 5, 2 (January 2003), 29–46. https://cci.drexel.edu/faculty/sgasson/papers/SG-JITTA.pdf>
[17]
Joseph Giacomin. 2014. What Is Human Centered Design? The Design Journal 17, 4 (April 2014), 606–623. https://doi.org/10.2752/175630614X14056185480186>
[18]
Lucy Suchman. 2007. Human-Machine Reconfigurations: Plans and Situated Actions, 2nd ed. Cambridge University Press, Cambridge, UK. https://doi.org/10.1017/CBO9780511808418>
[19]
Marc Steen. 2011. Tensions in human-centred design. CoDesign 7, 1 (June 2011), 45–60. https://doi.org/10.1080/15710882.2011.563314>
[20]
Patrick W. Jordan. 2002. Human factors for pleasure seekers. In Design and the Social Sciences: Making Connections (Frascara, J. ed.). Taylor and Francis, London, UK, 9–23.>
[21]
Ezio Manzini. 2014. Making things happen: Social innovation and design. Design issues 30, 1 (May 2014), 57–66. https://www.jstor.org/stable/24267025>
[22]
Richard Buchanan. 1992. Wicked Problems in Design Thinking. Design Issues 8, 2 (January 1992), 5–21. https://doi.org/10.2307/1511637>
[23]
John C. Camillus. 2008. Strategy as a wicked problem. Harvard Business Review 86, 5 (May 2008), 98–109. https://hbr.org/2008/05/strategy-as-a-wicked-problem>
[24]
Horst Rittel and Melvin Webber. 1973. Planning problems are wicked. Polity 4, 155–169. http://www.jstor.org/stable/4531523?origin=JSTOR-pdf>
[25]
Jeanne Liedtka. 2015. Perspective: linking design thinking with innovation outcomes through cognitive bias reduction. Journal of Product Innovation Management 32, 6 (November 2015), 925–938. https://doi.org/10.1111/jpim.12163>
[26]
Fabrizio Ceschin and Idil Gaziulusoy. 2016. Evolution of design for sustainability: From product design to design for system innovations and transitions. Design studies 47 (November 2016), 118–163. https://doi.org/10.1016/j.destud.2016.09.002>
[27]
Luca Rota, Yanjun Zohu, and Svenja Paege. 2019. Sustainable Product-Service System Design from a strategic sustainable development perspective. http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1340289&dswid=-8792>
[28]
Frederick A. Muckler and Sally A. Seven. 1992. Selecting Performance Measures: “Objective” versus “Subjective” Measurement. Human Factors 34, 4 (November 1992), 441–455.
[29]
SimpleKPI. 2021. What are Metrics and Measures - Explanation and Examples. Accessed: 2/16/2022. Available: https://www.simplekpi.com/Blog/metrics-and-meassures-a-definitive-guide>
[30]
The Johns Hopkins Institute for Assured Autonomy. 2021. Johns Hopkins IAA - Video: IAA Human-Centered Design Team At AUVSI XPonential 2021. Accessed: 2/16/2022. Available: https://iaa.jhu.edu/video-iaa-human-centered-design-team-at-auvsi-xponential-2021/>
[31]
Purdue University. 2022. Institute for Control, ICON Optimization and Networks - About. Accessed: 2/22/2022. Available: https://engineering.purdue.edu/ICON>
[32]
National Security Innovation Network. 2022. Accessed: 2/22/2022. Available: https://www.nsin.mil/>
[33]
Chauncey Wilson. 2013. Brainstorming and beyond: a user-centered design method. Newnes. https://doi.org/10.1016/C2012-0-03533-8>
[34]
Paul B. Paulus and Vincent R. Brown. 2003. Enhancing ideational creativity in groups. Group creativity: Innovation through collaboration, 110–136. https://doi.org/10.1093/acprof:oso/9780195147308.003.0006>
[35]
Paul B. Paulus, Vicky L. Putman, Karen Leggett Dugosh, Mary T. Dzindolet, and Hamit Coskun. 2002. Social and cognitive influences in group brainstorming: Predicting production gains and losses. European review of social psychology 12, 1, 299–325. https://doi.org/10.1080/14792772143000094>
[36]
Rolf Faste. 1993. An Improved Model for Understanding Creativity and Convention, in Cary A. Fisher (ed.), ASME Resource Guide to Innovation in Engineering Design, American Society of Mechanical Engineers. http://fastefoundation.org/publications/an_improved_model.pdf>
[37]
Michael Diehl and Wolfgang Stroebe. 1987. Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of personality and social psychology 53, 3 (September 1987), 497–509. https://doi.org/10.1037/0022-3514.53.3.497>
[38]
Michael Diehl and Wolfgang Stroebe. 1991. Productivity loss in idea-generating groups: Tracking down the blocking effect. Journal of Personality and Social Psychology 61, 3 (September 1991), 392. https://psycnet.apa.org/doi/10.1037/0022-3514.61.3.392>
[39]
Paul B. Paulus and Jared B. Kenworthy. 2019. Effective brainstorming. In The Oxford Handbook of Group Creativity and Innovation, 287-386. https://psycnet.apa.org/doi/10.1093/oxfordhb/9780190648077.013.17>
[40]
Carsten De Dreu, Bernard Nijstad, and Daan van Knippenberg. 2008. Motivated information processing in group judgment and decision making. Personality and Social Psychology Review 12, 1 (February 2008), 22-49. https://doi.org/10.1177/1088868307304092>
[41]
Paul B. Paulus and Mary T. Dzindolet. 1993. Social influence processes in group Brainstorming. Journal of Personality and Social Psychology 64, 4, 575. https://psycnet.apa.org/doi/10.1037/0022-3514.64.4.575>
[42]
Paul B. Paulus, Timothy S. Larey, and Anita H. Ortega. 1995. Performance and perceptions of brainstormers in an organizational setting. Basic and Applied Social Psychology 17, 1-2, 249-265. https://psycnet.apa.org/doi/10.1207/s15324834basp1701&2_15>
[43]
Wolfgang Stroebe, Michael Diehl, and Georgios Abakoumkin. 1992. The illusion of group effectivity. Personality and Social Psychology Bulletin 18, 5 (October 1992), 643-650. https://doi.org/10.1177/0146167292185015>
[44]
Bernard A. Nijstad, Wolfgang Stroebe, and Hein Lodewijkx. 2003. Production blocking and idea generation: Does blocking interfere with cognitive processes? Journal of Experimental Social Psychology 39, 6, 531-548. https://doi.org/10.1016/S0022-1031(03)00040-4>
[45]
Horst Geschka. 1993. The development and assessment of creative thinking techniques: A German perspective. In Nurturing and Developing Creativity: The Emergence of a Discipline, 215-236. https://doi.org/10.1111/j.1467-8691.1996.tb00125.x>
[46]
G. Goodman. 1995. Brainwriting: What to do when there's not a cloud in the brainstorming sky. In Marketing Encyclopedia: Issues and Trends Shaping the Future, 40-46.>
[47]
Arthur B. VanGundy. 1981. Techniques of Structured Problem Solving. New York: Van Nostrand Reinhold. https://link.springer.com/book/9780442288471>
[48]
Robert I. Sutton and Andrew Hargadon. 1996. Brainstorming Groups in Context: Effectiveness in a Product Design Firm. Administrative Science Quarterly 41, 4 (December 1996), 685-718. https://psycnet.apa.org/doi/10.2307/2393872>
[49]
IDEO U. 2017. 7 Simple Rules of Brainstorming. Accessed: 2/22/2022. Available: https://www.ideou.com/blogs/inspiration/7-simple-rules-of-brainstorming>
[50]
Jon Kolko. 2010. Abductive Thinking and Sensemaking: The Drivers of Design Synthesis. Design Issues 26, 1 (January 2010), 15-28. https://doi.org/10.1162/desi.2010.26.1.15>
[51]
Jon Kolko. 2007. Information Architecture and Design Strategy: The Importance of Synthesis during the Process of Design. In IDSA 2007 Educational Conference Proceedings (San Francisco: IDSA). https://www.jonkolko.com/writing/information-architecture-and-design-strategy>
[52]
Gary Klein, B. Moon, and Robert Hoffman. 2006. Making Sense of Sensemaking 1: Alternative Perspectives. Intelligent Systems (IEEE) 21, 4 (July/August 2006), 71. https://doi.org/10.1109/MIS.2006.75>
[53]
Brenda Dervin. 2003. Sense-Making's Journey from Metatheory to Methodology to Methods: An Example Using Information Seeking and Use as Research Focus. In Sense-Making Methodology Reader (Cresskill, NJ: Hampton Press), 141–146.>
[54]
Charles S. Peirce. 1998. On the Logic of Drawing History from Ancient Documents. In The Essential Peirce: Selected Philosophical Writings, 1893–1913, edited by Peirce Edition Project (Bloomington: Indiana University Press), 95>
[55]
David Dunne and Roger Martin. 2006. Design thinking and how it will change management education: An interview and discussion. Academy of Management Learning & Education 5, 4 (December 2006), 512–523. https://psycnet.apa.org/doi/10.5465/AMLE.2006.23473212>
[56]
Richard Coyne. 1988. Logic models of design. Pitman.>
[57]
Charles S. Peirce. 1988. Pragmatism as the Logic of Abduction. In The Essential Peirce: Selected Philosophical Writings, 1893–1913, edited by Peirce Edition Project (Bloomington: Indiana University Press), 227>
[58]
Philip N. Johnson-Laird. 2005. The Shape of Problems. In The Shape of Reason: Essays in Honour of Paolo Legrenzi, edited by V Girotto (Psychology Press), 3–26.>

Index Terms

  1. Designing for Trust A Human-Centered Methodology for Identifying Metrics of Operational Trust in Mission Autonomy
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Information & Contributors

            Information

            Published In

            cover image ACM Other conferences
            TAS '23: Proceedings of the First International Symposium on Trustworthy Autonomous Systems
            July 2023
            426 pages
            ISBN:9798400707346
            DOI:10.1145/3597512
            Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            Published: 11 July 2023

            Permissions

            Request permissions for this article.

            Check for updates

            Qualifiers

            • Research-article
            • Research
            • Refereed limited

            Conference

            TAS '23

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • 0
              Total Citations
            • 101
              Total Downloads
            • Downloads (Last 12 months)61
            • Downloads (Last 6 weeks)0
            Reflects downloads up to 16 Feb 2025

            Other Metrics

            Citations

            View Options

            Login options

            View options

            PDF

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format.

            HTML Format

            Figures

            Tables

            Media

            Share

            Share

            Share this Publication link

            Share on social media