Abstract
Trust has been identified as a key element for the successful cooperation between humans and robots. However, little research has been directed at understanding trust development in industrial human-robot collaboration (HRC). With industrial robots becoming increasingly integrated into production lines as a means for enhancing productivity and quality, it will not be long before close proximity industrial HRC becomes a viable concept. Since trust is a multidimensional construct and heavily dependent on the context, it is vital to understand how trust develops when shop floor workers interact with industrial robots. To this end, in this study a trust measurement scale suitable for industrial HRC was developed in two phases. In phase one, an exploratory study was conducted to collect participants’ opinions qualitatively. This led to the identification of trust related themes relevant to the industrial context and a related pool of questionnaire items was generated. In the second phase, three human-robot trials were carried out in which the questionnaire items were applied to participants using three different types of industrial robots. The results were statistically analysed to identify the key factors impacting trust and from these generate a trust measurement scale for industrial HRC.
Similar content being viewed by others
References
Ding Z, Hon B (2013) Constraints analysis and evaluation of manual assembly. CIRP Ann Manuf Technol 62(1):1–4
Hägele M, Schaaf W, Helms E (2002) Robot assistants at manual workplaces: Effective co-operation and safety aspects. International symposium on robotics ISR 2002 / CD-ROM. October 7–11, Stockholm
Schraft RD, Meyer C, Parlitz C, Helms E (2005) Powermate—a safe and intuitive robot assistant for handling and assembly tasks. IEEE international conference on robotics and automation, IEEE, Barcelona, pp 4047–4079
Santis AD, Siciliano B, Luca AD, Bicchi A (2008) Atlas of physical human-robot interaction. Mech Mach Theor 43(3):253–270
Unhelkar VV, Siu HC, Shah JA (2014) Comparative performance on human and mobile robotic assistants in collaborative fetch-and-deliver tasks. Human Robot interaction 2014. ACM, Bielefeld
ISO (2011) Robots and robotic devices-safety requirements for industrial robots, Part 1: robots. international standards organisation, Geneva
Bortot D, Born M, Bengler K (2013) Directly or detours? How should industrial robots approach humans?. ACM/IEEE international conference on human-robot interaction (HRI 2013), IEEE, Tokyo
Walton M, Webb P, Poad M (2011) Applying a concept for robot-human cooperation to aerospace equipping processes, SAE International
Parasuraman R, Riley V (1997) Humans and automation: use, misuse, disuse, abuse. Hum Fact 39(2):230–253
Freedy A, de Visser E, Weltman G, Coeyman N (2007) Measurement of trust in human-robot collaboration. In: Proceedings of the 2007 international conference on collaborative technologies and systems, Orland
Chen JY, Barnes MJ (2014) Human-agent teaming for multirobot control: a review of human factors issues. IEEE Trans Hum Mach Syst 44(1):13–29 IEEE
Groom V, Nass C (2007) Can robots be teammates? Benchmarks in human–robot teams. Interact Stud 8(3):483–500
Park E, Jenkins Q, Jiang X (2008) Measuring trust of human pperators in new generation rescue robots. Proceedings of the 7th JFPS international symposium on fluid power, Toyom, pp 15–18
de Visser EJ, Parasuraman R, Freedy A, Freedy E, Weltman G (2006) A comprehensive methodology for assessing human-robot team Performance for use in training and simulation. Proceedings of the 50th Human factors ergonomics society, San Francisco, pp 2639–2643
Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Fact 46(1):50–80
Verberne F, Ham J, Midden C (2012) Trust in smart systems: sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Hum Fact 54(5):799–810
Mazney D, Reichenbach J, Onnasch L (2012) Human performance consequences of automated decision aids: the impact of degree of automation and system experience. J Cogn Eng Decis Mak 6(1):57–87
Parasuraman R, Molloy R, Singh I (1993) Performance consequences of automation-induced ‘complacency’. Int J Aviat Psychol 3(1):1–23
Dzindolet MT, Pierce LG, Beck HP, Dawe LA, Anderson WB (2001b) Predicting misuse and disuse of combat identification systems. Mil Psychol 13(3):147–164
de Visser E, Krueger F, McKnight P, Scheid S, Smith M, Chalk S et al. (2012) The world is not enough: trust in cognitive agents. In: Proceedings of the 56th annual HFES meeting, pp 263–267
Chen JY, Barnes MJ (2012) Supervisory control of multiple robots: effects of imperfect automation and individual differences. Hum Fact 54(2):157–174
Desai M, Stubbs K, Steinfeld A, Yanco H (2009) Creating trustworthy robots: lessons and inspirations from automated systems. Paper presented at The AISB convention: new frontiers in human-robot interaction. Edinburgh
Yagoda RE, Gillan DJ (2012) You want me to trust a ROBOT? The development of a human-robot interaction trust scale. Int J Soc Robot 4(3):235–248
Hancock PA, Billings DR, Oleson KE, Chen JY, De Visser E, Parasuraman R (2011) A meta-analysis of factors influencing the development of human-robot trust. Aberdeen proving ground, MD 21005-5425: US Army Research Laboratory
Singh IL, Molloy R, Parasuraman R (1993) Automation-induced “complacency”: development of a complacency-potential rating scale. Int J Aviat Psychol 3(2):111–122
Muir BM, Moray N (1996) Trust in automation. Part 1. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3):429–460
Jian X, Bisantz AM, Drury CG (2000) Foundations for an empirically determined scale of trust in automated systems. Int J Cognit Ergon 4(1):53–71
Master R, Gramopadhye AK, Melloy BJ, Bingham J, Jiang X (2000) A questionnaire for measuring trust in hybrid inspection systems. Paper presented at the industrial engineering research conference. Dallas
Schaefer KE (2013) The perception and measurement of human-robot trust. Doctoral dissertation, University of Central Florida Orlando, Florida
King N (1998) Template analysis. In: Gillian S, Catherine C (eds) Qualitative methods and analysis in organizational research: a practical guide. Sage Publications Ltd, Thousand Oaks, pp 118–134
Nunnally JC (1978) Psychometric theory, 2nd edn. McGraw-Hill, New York
Kline TJB (2005) Psychological testing: a practical approach to design and testing. Sage Publications Inc, London
Lowenthal KM (1996) An introduction to psychological tests and scales. UCL Press, London
Harris D, Chan-Pensley J, McGarry S (2005) The development of a multidimensional scale to evaluate motor vehicle dynamic qualities. Ergonomics 48(8):964–982
Hair JF, Anderson RE, Tatham RL, Black WC (1998) Multivariate data analysis. Prentice Hall, Upper Saddle River
Kaiser HF (1974) An index of factorial simplicity. Psychometrika 39(1):31–36
Kline P (1999) The handbook of psychological testing, 2nd edn. Routledge, London
deVellis RF (1991) Scale development: theory and application. Sage Publishing, Newbury Park
Bartneck C, Kulic D, Croft E (2009) Measuring the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int J Soc Robot 1:71–81
Inoue K, Nonaka S, Ujiie Y, Takubo T, Arai T (2005) Comparison of human psychology for real and virtual mobile manipulators. IEEE international workshop on robot and human interactive communication, ROMAN 2005, IEEE, pp 73–78
Huber M, Rickert M, Knoll A, Brandt T, Glasauer S (2008) Human-robot interaction in handing-over tasks. 17th IEEE international symposium on robot and human interactive communication, pp 107–112
Mayer MP, Kuz S, Schlick CM (2013) Using anthropomorphism to improve the human-machine interaction in industrial environments (part II). In: Duffy VG (ed) Digital human modeling and applications in health, safety, ergonomics, and risk management. Human body modelling and ergonomics. Springer, Berlin, pp 93–100
Broadbent E, Stafford R, MacDonald B (2009) Acceptance of healthcare robots for the older population: review and future directions. Int J Soc Robot 1(4):319–330
Bartneck C, Kanda T, Mubin O, Mahmud AA (2009) Does the design of a robot influence its animacy and perceived intelligence? Int J Soc Robot 1(2):195–204
Li D, Rau PL, Li Y (2010) A cross-cultural study: effect of robot appearance and task. Int J Soc Robot 2(2):175–186
Shiomi M, Zanlungo F, Hayashi K, Kanda T (2014) Towards a socially acceptable collision avoidance for a mobile robot navigating among pedestrians using a pedestrian model. Int J Soc Robot 6(3):443–455
van den Brule R, Dotsch R, Bijlstra G, Wigboldus DH, Haselager P (2014) Do robot performance and behavioral style affect human trust? Int J Soc Robot 6(4):519–531
Rosenthal-von der Pütten AM et al, Krämer NC (2015) Individuals’ evaluations of and attitudes towards potentially uncanny robots. Int J Soc Robot 7(5):1–26
Prakash A, Rogers WA (2014) Why some humanoid faces are perceived more positively than others: effects of human-likeness and task. Int J Soc Robot 7(2):309–331
Acknowledgments
Special thanks to the senior laboratory technical officer at Cranfield University, Mr. John Thrower, for the assistance and expertise provided. Also, special thanks to Dr. Matthew Chamberlain and the team at Loughborough University for their assistance. The research is funded by the EPSRC Centre for Innovative Manufacturing in Intelligent Automation.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
1.1 Appendix 1: Interview Template
Section | Main question | Probe |
---|---|---|
Introduction | Can you talk to me about your first thoughts regarding the interaction with this robot? | Why did you feel this way? |
Robot related | Did you feel you could rely on the robot to hand you over the components safely? | Why? |
Can you talk to me about the robot’s ability to hand you the components? | Why? | |
Can you tell me more? | ||
How did the appearance of the robot influence your trust? | Why? | |
Safety | Did you have any concerns when you interacted with the robot? | What? |
Why? | ||
Other topics | Considering the task you have just completed, what has encouraged you to trust the robot? | Can you talk to me more about this? |
Is there anything else about the robot that encouraged you to trust this robot? | Why? | |
Can you tell me more? |
1.2 Appendix 2: Coding Template
Element | Trust-related theme (code sign) | Lower level theme (code sign) |
---|---|---|
Robot (R) | Robot’s performance (R1) | Robot motion (R1m) |
Robot and gripper reliability (R1r) | ||
Robot’s physical attributes (R2) | Robot size (R2s) | |
Robot appearance (R2a) | ||
Human (H) | Safety (H1) | Personal safety (H1p) |
Safe programming of the robot (H1prog) | ||
Experience (H2) | Prior interaction experiences (H2int) | |
Robot mental models (H2mm) | ||
External (E) | Task (E1) | Complexity of the task (E1comp) |
1.3 Appendix 3: Item Generation
Items | Direction |
---|---|
The way the robot moved made me uncomfortable | \(-\) |
I was not concerned because the robot moved in an expected way | + |
The speed of the robot made me uncomfortable | \(-\) |
The speed at which the gripper picked up and released the components made me uneasy | \(-\) |
I felt I could rely on the robot to do what it was supposed to do | + |
I knew the gripper would not drop the components | + |
The design of the robot was friendly | + |
I believe the robot could do a wider number of tasks than what was demonstrated | + |
I felt the robot was working at full capacity | \(-\) |
The robot gripper did not look reliable | \(-\) |
The gripper seemed like it could be trusted | + |
The size of the robot did not intimidate me | + |
I felt safe interacting with the robot | + |
I was comfortable the robot would not hurt me | + |
I trusted that the robot was safe to cooperate with | + |
I had faith that the robot had been programmed correctly | + |
The way robots are presented in the media had a negative influence on my feelings about interacting with this robot | \(-\) |
I had no prior expectations of what the robot would look like | + |
I don’t think any prior experiences with robots would affect the way I interacted with the robot | + |
If I had more experiences with other robots I would feel less concerned | - |
I was uncomfortable working with the robot due to the complexity level of the task | - |
If the task was more complicated I might have felt more concerned | - |
I might not have been able to work with the robot had the task been more complex | - |
The task made it easy to interact with this robot | + |
1.4 Appendix 4: Extract of the Rating Scale
1.5 Appendix 5: Trust Scale Components
Component | ||||
---|---|---|---|---|
1 | 2 | 3 | ||
1 | The way the robot moved made me uncomfortable | 0.759 | ||
2 | The speed at which the gripper picked up and released the components made me uneasy | 0.848 | ||
3 | I knew the gripper would not drop the components | 0.651 | ||
4 | The robot gripper did not look reliable | \(-\)0.828 | ||
5 | I trusted that the robot was safe to cooperate with | 0.688 | ||
6 | The gripper seemed like it could be trusted | 0.793 | ||
7 | I was comfortable the robot would not hurt me | 0.782 | ||
8 | The size of the robot did not intimidate me | 0.754 | ||
9 | I felt safe interacting with the robot | 0.787 | ||
10 | I felt I could rely on the robot to do what it was supposed to do | 0.506 | ||
Reliability analysis output: cronbach alpha value | 0.802 | 0.712 | 0.612 |
1.6 Appendix 6: User Guide to the Trust Scale Questionnaire
Part D: Instructions for the Assessor
The statements in this survey have been randomised to reduce participant bias. Follow these five steps post administration to analyse the output.
Step 1: Group the Statements Together into their Major Components as shown below:
Questionnaire number | Major components |
---|---|
A | Robot’s motion and pick-up speed |
C | |
D | Safe Co-operation |
F | |
H | |
I | |
B | Robot and gripper reliability |
E | |
G | |
J |
Step 2: Score the Individual Statements:
*The following scoring scheme assumes that the individuals were exposed to an industrial robot of 100 % reliability (e.g. no failures, no abnormal motion)
Scale points | Use this scoring scheme for statements: B, D, E, F, H, I, J | Use this scoring scheme for statements: A, C, G |
---|---|---|
Score | Score | |
Strongly agree | 5 | 1 |
Agree | 4 | 2 |
Neutral | 3 | 3 |
Disagree | 2 | 4 |
Strongly disagree | 1 | 5 |
Step 3: Deriving the Trust Score for Each Component and the Total Trust Score
ID | Trust component | Individual score | Minimum score possible | Maximum score possible |
---|---|---|---|---|
1 | Perceived Robot’s motion and pick-up speed | X | 2 | 10 |
2 | Perceived Safe co-operation | Y | 4 | 20 |
3 | Perceived Robot and gripper reliability | Z | 4 | 20 |
Totall trust score | X + Y + Z | 10 | 50 | |
Date undertaken | DD/MM/YYYY |
Step 4: Interpreting the Results
First, observe the total trust score. If the total trust score is low (e.g. less than 25), then this could be an indication the individual does not trust the robot to collaborate with. This could have severe implications on operational effectiveness and efficiency. At the same time, if the total trust score is very high (e.g. close to 50), then this could potentially indicate the individual is overly relying on the robot which could lead to complacency.
Secondly, observe the scores achieved for each trust component separately. Identify any poorly rated trust components. For example, a particular worker might have scored high on component 3 (‘Perceived robot and gripper reliability’) but poorly on component 2 (‘Perceived safe Co-operation’) and 1 (‘Perceived robot’s motion and pick-up speed’). This will enable to identify the specific areas requiring actions, e.g. refresher training course.
Step 5: Identifying Emerging Trends
To identify emerging trends and patterns it is suggested to:
-
Log results for each individual
-
Administer the questionnaire every four months. Consider re-randomising the order to the statements on the questionnaire to avoid participant bias
-
Compare results between each quarter
-
Communicate to the individual the results of the questionnaire. If the individual is identified to be in the ‘low’ or ‘very high’ region, follow up with an informal conversation to understand the reasons behind this. This will enable to take mutually agreed actions.
*Data for Comparison
The table below presents the average score and standard deviation of each of the questionnaire items is provided following the experimental trials at Cranfield and Loughborough Universities (Sample size: 155). These data can be used as an early comparison with the collected data.
Questionnaire item | Average score | SD |
---|---|---|
A | 1.8 | 0.8 |
C | 1.8 | 0.9 |
D | 4.3 | 0.7 |
F | 4.1 | 0.9 |
H | 4.3 | 0.8 |
I | 4.3 | 0.7 |
B | 4.1 | 0.9 |
E | 3.7 | 1.1 |
G | 1.9 | 0.9 |
J | 4 | 0.8 |
Rights and permissions
About this article
Cite this article
Charalambous, G., Fletcher, S. & Webb, P. The Development of a Scale to Evaluate Trust in Industrial Human-robot Collaboration. Int J of Soc Robotics 8, 193–209 (2016). https://doi.org/10.1007/s12369-015-0333-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12369-015-0333-8