research-article

Planning with Trust for Human-Robot Collaboration

Authors:

Stefanos Nikolaidis,

Siddhartha SrinivasaAuthors Info & Claims

HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

Pages 307 - 315

https://doi.org/10.1145/3171221.3171264

Published: 26 February 2018 Publication History

Abstract

Trust is essential for human-robot collaboration and user adoption of autonomous systems, such as robot assistants. This paper introduces a computational model which integrates trust into robot decision-making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human behaviors, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table-clearing task in simulation (201 participants) and with a real robot (20 participants). The results show that the trust-POMDP improves human-robot team performance in this task. They further suggest that maximizing trust in itself may not improve team performance.

References

[1]

Haoyu Bai, Shaojun Cai, Nan Ye, David Hsu, and Wee Sun Lee . 2015. Intention-aware online POMDP planning for autonomous driving in a crowd 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 454--460.

[2]

Tirthankar Bandyopadhyay, Kok Sung Won, Emilio Frazzoli, David Hsu, Wee Sun Lee, and Daniela Rus . 2013. Intention-aware motion planning. Algorithmic Foundations of Robotics X. Springer, 475--491.

[3]

Samuel Barrett, Noa Agmon, Noam Hazon, Sarit Kraus, and Peter Stone . 2014. Communicating with unknown teammates. In Proceedings of the twenty-first european conference on artificial intelligence. IOS Press, 45--50.

Digital Library

[4]

Bob Carpenter, Andrew Gelman, Matt Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Michael A Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell . 2016. Stan: A probabilistic programming language. Journal of Statistical Software Vol. 20 (2016), 1--37.

[5]

Nathaniel D Daw, John P O'doherty, Peter Dayan, Ben Seymour, and Raymond J Dolan . 2006. Cortical substrates for exploratory decisions in humans. Nature, Vol. 441, 7095 (2006), 876--879.

[6]

Munjal Desai . 2012. Modeling trust to improve human-robot interaction. (2012).

[7]

Anca D Dragan, Rachel M Holladay, and Siddhartha S Srinivasa . 2014. An Analysis of Deceptive Robot Motion. In Robotics: science and systems. 10.

[8]

Anca D Dragan, Kenton CT Lee, and Siddhartha S Srinivasa . 2013. Legibility and predictability of robot motion. In Human-Robot Interaction (HRI), 2013 8th ACM/IEEE International Conference on. IEEE, 301--308.

Digital Library

[9]

Michael W Floyd, Michael Drinkwater, and David W Aha . 2015. Trust-Guided Behavior Adaptation Using Case-Based Reasoning Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. 4261--4267.

Digital Library

[10]

Enric Galceran, Alexander G Cunningham, Ryan M Eustice, and Edwin Olson . 2015. Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction Proc. Robot.: Sci. & Syst. Conf. 2.

[11]

Robert T Golembiewski and Mark McConkie . 1975. The centrality of interpersonal trust in group processes. Theories of group processes Vol. 131 (1975), 185.

[12]

Robert J Hall . 1996. Trusting your assistant. In Knowledge-Based Software Engineering Conference, 1996., Proceedings of the 11th. IEEE, 42--51.

Digital Library

[13]

Guy Hoffman . 2013. Evaluating fluency in human-robot collaboration. International conference on human-robot interaction (HRI), workshop on human robot collaboration, Vol. Vol. 381. 1--8.

[14]

Shervin Javdani, Siddhartha S Srinivasa, and J Andrew Bagnell . 2015. Shared autonomy via hindsight optimization. arXiv preprint arXiv:1503.07619 (2015).

[15]

Edward E Jones, Leslie Rock, Kelly G Shaver, George R Goethals, and Lawrence M Ward . 1968. Pattern of performance and ability attribution: An unexpected primacy effect. Journal of Personality and Social Psychology, Vol. 10, 4 (1968), 317.

[16]

L.P. Kaelbling, M.L. Littman, and A.R. Cassandra . 1998. Planning and acting in partially observable stochastic domains. Artificial Intelligence Vol. 101, 1--2 (1998), 99--134.

Digital Library

[17]

Roderick M Kramer and Tom R Tyler . 1995. Trust in organizations: Frontiers of theory and research. Sage Publications.

[18]

Hanna Kurniawati, David Hsu, and Wee Sun Lee . 2008. SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces. In Robotics: Science and Systems, Vol. Vol. 2008. Zurich, Switzerland.

[19]

John Lee and Neville Moray . 1992. Trust, control strategies and allocation of function in human-machine systems. Ergonomics, Vol. 35, 10 (1992), 1243--1270.

[20]

John D Lee and Katrina A See . 2004. Trust in automation: Designing for appropriate reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 46, 1 (2004), 50--80.

[21]

Owen Macindoe, Leslie Pack Kaelbling, and Tomás Lozano-Pérez . 2012. Pomcop: Belief space planning for sidekicks in cooperative games. (2012).

[22]

Roger C Mayer, James H Davis, and F David Schoorman . 1995. An integrative model of organizational trust. Academy of management review Vol. 20, 3 (1995), 709--734.

[23]

Bonnie Marlene Muir . 1990. Operators' trust in and use of automatic controllers in a supervisory process control task. University of Toronto.

[24]

Stefanos Nikolaidis, David Hsu, and Siddhartha Srinivasa . 2017. Human-robot mutual adaptation in collaborative tasks: Models and experiments. International Journal of Robotics Research Vol. 36, 5--7 (2017), 618--634.

Digital Library

[25]

Stefanos Nikolaidis, Anton Kuznetsov, David Hsu, and Siddhartha Srinivasa . 2016. Formalizing Human-Robot Mutual Adaptation: A Bounded Memory Model HRI. IEEE Press, 75--82.

Digital Library

[26]

Stefanos Nikolaidis, Ramya Ramakrishnan, Keren Gu, and Julie Shah . 2015. Efficient model learning from joint-action demonstrations for human-robot collaborative tasks. In HRI. ACM, 189--196.

Digital Library

[27]

Alyssa Pierson and Mac Schwager . 2016. Adaptive inter-robot trust for robust multi-robot sensor coverage. Robotics Research. Springer, 167--183.

[28]

Charles Pippin and Henrik Christensen . 2014. Trust modeling in multi-robot patrolling. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 59--66.

[29]

Dorsa Sadigh, Shankar Sastry, Sanjit A Seshia, and Anca D Dragan . 2016. Planning for autonomous cars that leverages effects on human actions Proceedings of the Robotics: Science and Systems Conference (RSS).

[30]

Michael Siegrist . 2000. The influence of trust and perceptions of risks and benefits on the acceptance of gene technology. Risk analysis, Vol. 20, 2 (2000), 195--204.

[31]

Ning Wang, David V Pynadath, and Susan G Hill . 2016. Trust calibration within a human-robot team: Comparing automatically generated explanations 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 109--116.

Digital Library

[32]

Anqi Xu and Gregory Dudek . 2015. Optimo: Online probabilistic trust inference model for asymmetric human-robot collaborations HRI. ACM, 221--228.

Digital Library

[33]

Anqi Xu and Gregory Dudek . 2016. Towards Modeling Real-Time Trust in Asymmetric Human--Robot Collaborations. Robotics Research. Springer, 113--129.

[34]

Jessie Yang, Vaibhav Unhelkar, Kevin Li, and Julie Shah . 2017. Evaluating Effects of User Experience and System Transparency on Trust in Automation HRI.

Digital Library

Cited By

Hou MBanbury SCain BFang SWilloughby HFoley LTunstel ERudas I(2025)IMPACTS Homeostasis Trust Management System: Optimizing Trust in Human-AI TeamsACM Computing Surveys10.1145/364944657:6(1-24)Online publication date: 10-Feb-2025
https://dl.acm.org/doi/10.1145/3649446
Noormohammadi-Asl AFan KSmith SDautenhahn K(2025)Human leading or following preferencesRobotics and Autonomous Systems10.1016/j.robot.2024.104821183:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.robot.2024.104821
Natarajan MXue Cvan Waveren SFeigh KGombolay MDastani MSichman JAlechina NDignum V(2024)Mixed-Initiative Human-Robot Teaming under Suboptimality with Online Bayesian AdaptationProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663005(1454-1462)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663005
Show More Cited By

Index Terms

Planning with Trust for Human-Robot Collaboration
1. Computing methodologies
  1. Artificial intelligence
    1. Planning and scheduling
      1. Planning under uncertainty
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Collaborative interaction

Recommendations

Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning

Trust in autonomy is essential for effective human-robot collaboration and user adoption of autonomous systems such as robot assistants. This article introduces a computational model that integrates trust into robot decision making. Specifically, we ...
Human-Robot Collaboration: an analysis of worker’s performance
Abstract
Collaborative robots are an important enabling technology of Industry 4.0. The interaction between humans and robots, called Human-Robot Collaboration (HRC), aims to improve system performance. However, it is necessary to investigate in depth the ...
Concurrent Probabilistic Motion Primitives for Obstacle Avoidance and Human-Robot Collaboration
Intelligent Robotics and Applications
Abstract
The paper proposed a new method to endow a robot with the ability of human-robot collaboration and online obstacle avoidance simultaneously. In other words, we construct a probabilistic model for human-robot collaboration primitives to learn the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

February 2018

468 pages

ISBN:9781450349536

DOI:10.1145/3171221

General Chairs:
Takayuki Kanda
ATR, Japan
,
Selma Ŝabanović
Indiana University Bloomington, USA
,
Program Chairs:
Guy Hoffman
Cornell University, USA
,
Adriana Tapus
ENSTA-ParisTech, France

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation CPS
National Science Foundation NRI
National Institute of Health R01

Conference

HRI '18

Sponsor:

SIGAI
SIGCHI
IEEE-RAS

HRI '18: ACM/IEEE International Conference on Human-Robot Interaction

March 5 - 8, 2018

IL, Chicago, USA

Acceptance Rates

HRI '18 Paper Acceptance Rate 49 of 206 submissions, 24%;

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

143
Total Citations
View Citations
2,956
Total Downloads

Downloads (Last 12 months)390
Downloads (Last 6 weeks)44

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hou MBanbury SCain BFang SWilloughby HFoley LTunstel ERudas I(2025)IMPACTS Homeostasis Trust Management System: Optimizing Trust in Human-AI TeamsACM Computing Surveys10.1145/364944657:6(1-24)Online publication date: 10-Feb-2025
https://dl.acm.org/doi/10.1145/3649446
Noormohammadi-Asl AFan KSmith SDautenhahn K(2025)Human leading or following preferencesRobotics and Autonomous Systems10.1016/j.robot.2024.104821183:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.robot.2024.104821
Natarajan MXue Cvan Waveren SFeigh KGombolay MDastani MSichman JAlechina NDignum V(2024)Mixed-Initiative Human-Robot Teaming under Suboptimality with Online Bayesian AdaptationProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663005(1454-1462)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3663005
Campagna GRehm M(2024)A Systematic Review of Trust Assessments in Human–Robot InteractionACM Transactions on Human-Robot Interaction10.1145/370612314:2(1-35)Online publication date: 29-Nov-2024
https://dl.acm.org/doi/10.1145/3706123
Duan WZhou SScalia MYin XWeng NZhang RFreeman GMcNeese NGorman JTolston M(2024)Understanding the Evolvement of Trust Over Time within Human-AI TeamsProceedings of the ACM on Human-Computer Interaction10.1145/36870608:CSCW2(1-31)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3687060
Furuya HBattistel LDatta Choudhary ZGottsacker MBruder GWelch G(2024)Difficulties in Perceiving and Understanding Robot Reliability Changes in a Sequential Binary TaskProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682083(1-11)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682083
Louca JEder KVrublevskis JTzemanaki A(2024)Impact of Haptic Feedback in High Latency Teleoperation for Space ApplicationsACM Transactions on Human-Robot Interaction10.1145/365199313:2(1-21)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3651993
Rezaei Khavas ZKotturu MAhmadzadeh SRobinette P(2024)Do Humans Trust Robots that Violate Moral Trust?ACM Transactions on Human-Robot Interaction10.1145/365199213:2(1-30)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3651992
Ruocco MJay CParsia B(2024)A Qualitative Study of Human Theorizing about Robot Bodily BehaviorExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651078(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651078
Cai SRam AGou ZShaikh MChen YWan YHara KZhao SHsu D(2024)Navigating Real-World Challenges: A Quadruped Robot Guiding System for Visually Impaired People in Diverse EnvironmentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642227(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642227
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten