skip to main content
10.1145/3357236.3395542acmconferencesArticle/Chapter ViewAbstractPublication PagesdisConference Proceedingsconference-collections
research-article

Punishable AI: Examining Users' Attitude Towards Robot Punishment

Published:03 July 2020Publication History

ABSTRACT

To give robots, which are black box systems for most users, feedback we have to implement interaction paradigms that users understand and accept, for example reward and punishment. In this paper we present the first HRI experience prototype which implements gradual destructive interaction, namely breaking a robot's leg as a punishment technique. We conducted an exploratory experiment (N=20) to investigate participants' behavior during the execution of three punishment techniques. Using a structured analysis of videos and interviews, we provide in-depth insights into participants' attitude towards these techniques. Participants preferred more abstract techniques and felt uncomfortable during human-like punishment interaction. Based on our findings, we raise questions how human-like technologies should be designed. A video documentation of the project can be found here: https://vimeo.com/348646727

References

  1. Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4 (2014), 105--120. DOI: http://dx.doi.org/10.1609/aimag.v35i4.2513Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Anja Austermann and Seiji Yamada. 2008. "Good robot", "bad robot" -- Analyzing users' feedback in a human-robot teaching task. In RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, New York, NY, USA, 41--46. DOI: http://dx.doi.org/10.1109/ROMAN.2008.4600641Google ScholarGoogle ScholarCross RefCross Ref
  3. James Auger. 2014. Living with Robots: A Speculative Design Approach. Journal of Human-Robot Interaction 3, Article 1 (Feb. 2014), 23 pages. DOI: http://dx.doi.org/10.5898/JHRI.3.1.AugerGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  4. Christoph Bartneck and Jun Hu. 2008. Exploring the abuse of robots. Interaction Studies 9, 3 (2008), 415--433. DOI:http://dx.doi.org/10.1075/is.9.3.04barGoogle ScholarGoogle ScholarCross RefCross Ref
  5. Christoph Bartneck, Takayuki Kanda Kanda, Omar Mubin, and Abdullah Al Mahmud. 2007. The perception of animacy and intelligence based on a robot's embodiment. In 2007 7th IEEE-RAS International Conference on Humanoid Robots. IEEE, New York, NY, USA, 300--305. DOI: http://dx.doi.org/10.1109/ICHR.2007.4813884Google ScholarGoogle ScholarCross RefCross Ref
  6. Christoph Bartneck, Dana Kuli´ c, Elizabeth Croft, and Susana Zoghbi. 2009. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics 1, 1 (2009), 71--81. DOI: http://dx.doi.org/10.1007/s12369-008-0001--3Google ScholarGoogle ScholarCross RefCross Ref
  7. Christoph Bartneck, Juliane Reichenbach, and Julie Carpenter. 2006. Use of Praise and Punishment in Human-Robot Collaborative Teams. In RO-MAN 2006 The 15th IEEE International Symposium on Robot and Human Interactive Communication, K. Dautenhahn (Ed.). IEEE, New York, NY, USA, 177--182. DOI: http://dx.doi.org/10.1109/ROMAN.2006.314414Google ScholarGoogle ScholarCross RefCross Ref
  8. Christoph Bartneck, Juliane Reichenbach, and Julie Carpenter. 2008. The carrot and the stick: The role of praise and punishment in human--robot interaction. Interaction Studies 9, 2 (2008), 179--203. DOI: http://dx.doi.org/10.1257/000282803322157142Google ScholarGoogle ScholarCross RefCross Ref
  9. Christoph Bartneck, Chioke Rosalia, Rutger Menges, and Inèz Deckers. 2005. Robot abuse -- a limitation of the media equation. Proceedings of the Interact 2005 Workshop on Agent Abuse, Rome. (2005).Google ScholarGoogle Scholar
  10. Christoph Bartneck, Michel Van Der Hoek, Omar Mubin, and Abdullah Al Mahmud. 2007a. "Daisy, daisy, give me your answer do!" -- Switching off a robot. In 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, New York, NY, USA, 217--222. DOI: http://dx.doi.org/10.1145/1228716.1228746Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Christoph Bartneck, Marcel Verbunt, Omar Mubin, and Abdullah Al Mahmud. 2007b. To kill a mockingbird robot. In Proceedings of the ACM/IEEE international conference on Human-robot interaction. ACM, New York, NY, USA, 81--87. DOI: http://dx.doi.org/10.1145/1228716.1228728Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Cynthia Breazeal. 2001. Affective interaction between humans and robots. In European Conference on Artificial Life. Springer, Berlin, Heidelberg, Germany, 582--591. DOI: http://dx.doi.org/10.1007/3--540--44811-X_66Google ScholarGoogle ScholarCross RefCross Ref
  13. Cynthia Breazeal. 2002. Regulation and entrainment in human-robot interaction. The International Journal of Robotics Research 21, 10--11 (2002), 883--902. DOI: http://dx.doi.org/10.1177/0278364902021010096Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Cynthia Breazeal. 2003. Toward sociable robots. Robotics and autonomous systems 42, 3--4 (2003), 167--175. DOI: http://dx.doi.org/10.1016/S0921--8890(02)00373--1Google ScholarGoogle ScholarCross RefCross Ref
  15. Cynthia Breazeal. 2004. Social interactions in HRI: the robot view. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 34, 2 (May 2004), 181--186. DOI: http://dx.doi.org/10.1109/TSMCC.2004.826268Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Drazen Brsci´ c, Hiroyuki Kidokoro, Yoshitaka Suehiro, and Takayuki Kanda. 2015. Escaping from children's abuse of social robots. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (HRI '15). ACM, New York, NY, USA, 59--66. DOI: http://dx.doi.org/10.1145/2696454.2696468Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Maya Cakmak, Crystal Chao, and Andrea L. Thomaz. 2010. Designing Interactions for Robot Active Learners. IEEE Transactions on Autonomous Mental Development 2, 2 (June 2010), 108--118. DOI: http://dx.doi.org/10.1109/TAMD.2010.2051030Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. John S. Carton. 1996. The differential effects of tangible rewards and praise on intrinsic motivation: A comparison of cognitive evaluation theory and operant theory. The Behavior Analyst 19, 2 (1996), 237--255. DOI:http://dx.doi.org/10.1007/BF03393167Google ScholarGoogle ScholarCross RefCross Ref
  19. John P. Charlton. 2009. The determinants and expression of computer-related anger.computers in Human Behavior 25, 6 (2009), 1213--1221. DOI: http://dx.doi.org/10.1016/j.chb.2009.07.001Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Hyojin Chin and Mun Yong Yi. 2019. Should an Agent Be Ignoring It?: A Study of Verbal Abuse Types and Conversational Agents' Response Styles. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19). ACM, New York, NY, USA, Article LBW2422, 6 pages. DOI: http://dx.doi.org/10.1145/3290607.3312826Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37--46. DOI: http://dx.doi.org/10.1177/001316446002000104Google ScholarGoogle ScholarCross RefCross Ref
  22. Francisco Cruz, Sven Magg, Cornelius Weber, and Stefan Wermter. 2016. Training agents with interactive reinforcement learning and contextual affordances. IEEE Transactions on Cognitive and Developmental Systems 8, 4 (2016), 271--284. DOI: http://dx.doi.org/10.1109/TCDS.2016.2543839Google ScholarGoogle ScholarCross RefCross Ref
  23. Christian Daniel, Malte Viering, Jan Metz, Oliver Kroemer, and Jan Peters. 2014. Active Reward Learning. In Proceedings of Robotics: Science and Systems X. Berkeley, USA. DOI: http://dx.doi.org/10.15607/RSS.2014.X.031Google ScholarGoogle ScholarCross RefCross Ref
  24. John J. Dudley and Per Ola Kristensson. 2018. A Review of User Interface Design for Interactive Machine Learning. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 2 (2018), 1--37. DOI: http://dx.doi.org/10.1145/3185517Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Malin Eiband, Mohamed Khamis, Emanuel von Zezschwitz, Heinrich Hussmann, and Florian Alt. 2017. Understanding Shoulder Surfing in the Wild: Stories from Users and Observers. In Proceedings of the 35th Annual ACM Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 4254 -- 4265. DOI: http://dx.doi.org/10.1145/3025453.3025636Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Julia Fink. 2012. Anthropomorphism and human likeness in the design of robots and human-robot interaction. In International Conference on Social Robotics. Springer, Berlin, Heidelberg, Germany, 199--208. DOI: http://dx.doi.org/10.1007/978--3--642--34103--8_20Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Michael A. Goodrich, Alan C. Schultz, and others. 2008. Human--robot interaction: a survey. Foundations and Trends® in Human--Computer Interaction 1, 3 (2008), 203--275. DOI:http://dx.doi.org/10.1561/1100000005Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Matthias Hoenen, Katrin T. Lübke, and Bettina M. Pause. 2016. Non-anthropomorphic robots as social entities on a neurophysiological level.computers in Human Behavior 57 (2016), 182 -- 186. DOI: http://dx.doi.org/10.1016/j.chb.2015.12.034Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Aike C. Horstmann, Nikolai Bock, Eva Linhuber, Jessica M. Szczuka, Carolin Straßmann, and Nicole C Krämer. 2018. Do a robot's social skills and its objection discourage interactants from switching the robot off? PloS one 13, 7 (2018), e0201581. DOI: http://dx.doi.org/10.1371/journal.pone.0201581Google ScholarGoogle ScholarCross RefCross Ref
  30. Peter H. Kahn Jr, Takayuki Kanda, Hiroshi Ishiguro, Nathan G. Freier, Rachel L. Severson, Brian T. Gill, Jolina H. Ruckert, and Solace Shen. 2012. ?Robovie, you'll have to go into the closet now": Children's social and moral relationships with a humanoid robot. Developmental Psychology 48, 2 (2012), 303. DOI: http://dx.doi.org/10.1037/a0027033Google ScholarGoogle ScholarCross RefCross Ref
  31. Irene Lopatovska and Harriet Williams. 2018. Personification of the Amazon Alexa: BFF or a Mindless Companion. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR '18). ACM, New York, NY, USA, 265--268. DOI: http://dx.doi.org/10.1145/3176349.3176868Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Qinggang Meng, Ibrahim Tholley, and Paul WH Chung. 2014. Robots learn to dance through interaction with humans. Neural Computing and Applications 24, 1 (2014), 117--124. DOI: http://dx.doi.org/10.1007/s00521-013--1504-xGoogle ScholarGoogle ScholarCross RefCross Ref
  33. Stanley Milgram. 1963. Behavioral study of obedience. The Journal of Abnormal and Social Psychology 67, 4 (1963), 371. DOI:http://dx.doi.org/10.1037/h0040525Google ScholarGoogle ScholarCross RefCross Ref
  34. Clifford Nass and Youngme Moon. 2000. Machines and mindlessness: Social responses to computers. Journal of Social Issues 56, 1 (2000), 81--103. DOI: http://dx.doi.org/10.1111/0022--4537.00153Google ScholarGoogle ScholarCross RefCross Ref
  35. Tatsuya Nomura, Takayuki Kanda, Hiroyoshi Kidokoro, Yoshitaka Suehiro, and Sachie Yamada. 2016. Why do children abuse robots? Interaction Studies 17, 3 (2016), 347--369. DOI: http://dx.doi.org/10.1075/is.17.3.02nomGoogle ScholarGoogle ScholarCross RefCross Ref
  36. Byron Reeves and Clifford Ivar Nass. 1996. The media equation: How people treat computers, television, and new media like real people and places. Cambridge University Press, Cambridge, UK.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Julia Ringler and Holger Reckter. 2012. DESU 100: about the temptation to destroy a robot. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI'12). ACM, New York, NY, USA, 151--152. DOI: http://dx.doi.org/10.1145/2148131.2148164Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Chioke Rosalia, Rutger Menges, Inèz Deckers, and Christoph Bartneck. 2005. Cruelty towards robots. Robot Workshop -- Designing Robot Applications for Everyday Use, Göteborg. (2005).Google ScholarGoogle Scholar
  39. Astrid M. Rosenthal-Von Der Pütten, Frank P. Schulte, Sabrina C. Eimler, Sabrina Sobieraj, Laura Hoffmann, Stefan Maderwald, Matthias Brand, and Nicole C. Krämer. 2014. Investigations on empathy towards humans and robots using fMRI.computers in Human Behavior 33 (2014), 201--212. DOI: http://dx.doi.org/10.1016/j.chb.2014.01.004Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Pericle Salvini, Gaetano Ciaravella, Wonpil Yu, Gabriele Ferri, Alessandro Manzi, Barbara Mazzolai, Cecilia Laschi, Sang-Rok Oh, and Paolo Dario. 2010. How safe are service robots in urban environments? Bullying a robot. In 19th International Symposium in Robot and Human Interactive Communication. IEEE, New York, NY, USA, 1--7. DOI: http://dx.doi.org/10.1109/ROMAN.2010.5654677Google ScholarGoogle ScholarCross RefCross Ref
  41. Suleman Shahid, Emiel Krahmer, and Marc Swerts. 2014. Child--robot interaction across cultures: How does playing a game with a social robot compare to playing a game alone or with a friend? Computers in Human Behavior 40 (2014), 86--100. DOI: http://dx.doi.org/10.1016/j.chb.2014.07.043Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. David Harris Smith and Frauke Zeller. 2017. The Death and Lives of hitchBOT: The Design and Implementation of a Hitchhiking Robot. Leonardo 50, 1 (2017), 77--78. DOI:http://dx.doi.org/10.1162/LEON_a_01354Google ScholarGoogle ScholarCross RefCross Ref
  43. Yutaka Suzuki, Lisa Galli, Ayaka Ikeda, Shoji Itakura, and Michiteru Kitazaki. 2015. Measuring empathy for human and robot hand pain using electroencephalography. Nature: Scientific Reports 5 (2015), 9. DOI:http://dx.doi.org/10.1038/srep15924Google ScholarGoogle ScholarCross RefCross Ref
  44. Xiang Zhi Tan, Marynel Vázquez, Elizabeth J. Carter, Cecilia G. Morales, and Aaron Steinfeld. 2018. Inducing Bystander Interventions During Robot Abuse with Social Mechanisms. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (HRI '18). ACM, New York, NY, USA, 169--177. DOI: http://dx.doi.org/10.1145/3171221.3171247Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Andrea L. Thomaz and Cynthia Breazeal. 2008. Teachable robots: Understanding human teaching behavior to build more effective robot learners. Artificial Intelligence 172, 6--7 (2008), 716--737. DOI: http://dx.doi.org/10.1016/j.artint.2007.09.009Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Andrea L. Thomaz, Guy Hoffman, and Cynthia Breazeal. 2005. Real-time interactive reinforcement learning for robots. AAAI 2005 workshop on human comprehensible machine learning. (2005).Google ScholarGoogle Scholar
  47. Blay Whitby. 2008. Sometimes it's hard to be a robot: A call for action on the ethics of abusing artificial agents. Interacting with Computers 20, 3 (02 2008), 326--333. DOI:http://dx.doi.org/10.1016/j.intcom.2008.02.002Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. David Whitney, Eric Rosen, James MacGlashan, Lawson L. S. Wong, and Stefanie Tellex. 2017. Reducing errors in object-fetching interactions through social feedback. In 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, New York, NY, USA, 1006--1013. DOI: http://dx.doi.org/10.1109/ICRA.2017.7989121Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Sarah Woods, Kerstin Dautenhahn, and Joerg Schulz. 2004. The design space of robots: Investigating children's views. In RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No. 04TH8759). IEEE, New York, NY, USA, 47--52. DOI: http://dx.doi.org/10.1109/ROMAN.2004.1374728Google ScholarGoogle ScholarCross RefCross Ref
  50. Ya-Huei Wu, Christine Fassert, and Anne-Sophie Rigaud. 2012. Designing robots for the elderly: appearance issue and beyond. Archives of Gerontology and Geriatrics 54, 1 (2012), 121--126. DOI: http://dx.doi.org/10.1016/j.archger.2011.02.003Google ScholarGoogle ScholarCross RefCross Ref
  51. Yuto Yamaji, Taisuke Miyake, Yuta Yoshiike, P. Ravindra De Silva, and Michio Okada. 2010. STB: Human-dependent Sociable Trash Box. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, New York, NY, USA, 197--198. DOI:http://dx.doi.org/10.1109/HRI.2010.5453196Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Punishable AI: Examining Users' Attitude Towards Robot Punishment

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        DIS '20: Proceedings of the 2020 ACM Designing Interactive Systems Conference
        July 2020
        2264 pages
        ISBN:9781450369749
        DOI:10.1145/3357236

        Copyright © 2020 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 3 July 2020

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        Overall Acceptance Rate1,158of4,684submissions,25%

        Upcoming Conference

        DIS '24
        Designing Interactive Systems Conference
        July 1 - 5, 2024
        IT University of Copenhagen , Denmark

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader