Joseph E. Michaelis
Bengisu Cagiltay
Bilge Mutlu
ABSTRACT
Social robots are becoming increasingly prevalent in the real world. Unsupervised user interactions in a natural and familiar setting, such as the home, can reveal novel design insights and opportunities. This paper presents an analysis and key design insights from family-robot interactions, captured via on-robot recordings during an unsupervised four-week in-home deployment of an autonomous reading companion robot for children. We analyzed interviews and 160 interaction videos involving six families who regularly interacted with a robot for four weeks. Throughout these interactions, we observed how the robot's expressions facilitated unique interactions with the child, as well as how family members interacted with the robot. In conclusion, we discuss five design opportunities derived from our analysis of natural interactions in the wild.
Supplemental Material
- Erin Beneteau, Ashley Boone, Yuxing Wu, Julie A. Kientz, Jason Yip, and Alexis Hiniker. 2020. Parenting with Alexa: Exploring the Introduction of Smart Speakers on Family Dynamics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--13.Google Scholar
Digital Library
- Erin Beneteau, Yini Guan, Olivia K. Richards, Mingrui Ray Zhang, Julie A. Kientz, Jason Yip, and Alexis Hiniker. 2020. Assumptions Checked: How Families Learn About and Use the Echo Dot. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--23.Google ScholarDigital Library
- Erin Beneteau, Olivia K. Richards, Mingrui Zhang, Julie A. Kientz, Jason Yip, and Alexis Hiniker. 2019. Communication breakdowns between families and Alexa. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2022. Thematic analysis: A practical guide. SAGE Publications Ltd, London.Google Scholar
- Bengisu Cagiltay. 2022. STEMMates, Long-Term In-Home Reading Companion Robot Supplemental Resources. https://osf.io/bks8w/?view_only= 7664456907db4dfb8c44d56c1d1e2cfdGoogle Scholar
- Bengisu Cagiltay, Nathan White, Rabia Ibtasar, Bilge Mutlu, and Joseph Michaelis. 2022. Understanding Factors that Shape Children's Long Term Engagement with an In-Home Learning Companion Robot. ACM Interaction Design and Children (2022).Google Scholar
- Caitlyn Clabaugh, Kartik Mahajan, Shomik Jain, Roxanna Pakkar, David Becerra, Zhonghao Shi, Eric Deng, Rhianna Lee, Gisele Ragusa, and Maja Matari?. 2019. Long-term personalization of an in-home socially assistive robot for children with autism spectrum disorders. Frontiers in Robotics and AI 6 (2019), 110.Google ScholarCross Ref
- Alexandre Coninx, Paul Baxter, Elettra Oleari, Sara Bellini, Bert Bierman, O Henkemans, Lola Cañamero, Piero Cosi, Valentin Enescu, R Espinoza, et al. 2016. Towards long-term social child-robot interaction: using multi-activity switching to engage young users. Journal of Human-Robot Interaction (2016).Google Scholar
- Scott Davidoff, Min K. Lee, Charles Yiu, John Zimmerman, and Anind K. Dey. 2006. Principles of smart home control. In International conference on ubiquitous computing. Springer, 19--34.Google Scholar
- Maartje MA de Graaf, Somaya Ben Allouch, and Jan AGM van Dijk. 2016. Longterm evaluation of a social robot in real homes. Interaction studies 17, 3 (2016), 462--491.Google Scholar
- Maartje M. A. de Graaf, Somaya B. Allouch, and Jan A. G. M. van Dijk. 2017. Why do they refuse to use my robot?: Reasons for non-use derived from a long-term home study. In 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 224--233.Google Scholar
- Maartje M. A. de Graaf, Somaya B. Allouch, and Jan A. G. M. van Dijk. 2019. Why would I use this in my home? A model of domestic social robot acceptance. Human--Computer Interaction 34, 2 (March 2019), 115--173.Google ScholarCross Ref
- Brian R Duffy and Karolina Zawieska. 2012. Suspension of disbelief in social robotics. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 484--489.Google ScholarCross Ref
- Brian R. Duffy and Karolina Zawieska. 2012. Suspension of disbelief in social robotics. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. 484--489. https://doi.org/10.1109/ ROMAN.2012.6343798Google Scholar
- Julia Fink, Séverin Lemaignan, Pierre Dillenbourg, Philippe Rétornaz, Florian Vaussard, Alain Berthoud, Francesco Mondada, Florian Wille, and Karmen Franinovi?. 2014. Which robot behavior can motivate children to tidy up their toys? Design and Evaluation of "Ranger". In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. 439--446.Google ScholarDigital Library
- Jeonghye Han, Miheon Jo, Sungju Park, and Sungho Kim. 2005. The educational use of home robots for children. In ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005. IEEE, 378--383.Google Scholar
- Judith M. Harackiewicz, Yoi Tibbetts, Elizabeth Canning, and Janet S. Hyde. 2014. Harnessing Values to Promote Motivation In Education. Advances in Motivation and Achievement: A Research Annual 18 (2014).Google Scholar
- Leslie Rupert Herrenkohl and Lindsay Cornelius. 2013. Investigating Elementary Students' Scientific and Historical Argumentation. Journal of the Learning Sciences 22, 3 (July 2013), 413--461.Google ScholarCross Ref
- William G. Holliday and Stephen D. Cain. 2012. Teaching Science Reading Comprehension: A Realistic, Research-Based Approach. In Second International Handbook of Science Education, Barry J. Fraser, Kenneth Tobin, and Campbell J. McRobbie (Eds.). Springer Netherlands, Dordrecht, 1405--1417.Google Scholar
- Gay Ivey and Karen Broaddus. 2000. Tailoring the Fit: Reading Instruction and Middle School Readers. The Reading Teacher 54, 1 (2000), 68--78.Google Scholar
- Eric W. Jensen, Sherman A. James, W. Thomas Boyce, and Sue A. Hartnett. 1983. The family routines inventory: Development and validation. Social Science & Medicine 17, 4 (1983), 201--211.Google ScholarCross Ref
- Malte Jung and Pamela Hinds. 2018. Robots in the wild: A time for more robust theories of human-robot interaction. ACM Transactions on Human-Robot Interaction (THRI) 7, 1 (2018), 1--5.Google ScholarDigital Library
- Takayuki Kanda, Rumi Sato, Naoki Saiwaki, and Hiroshi Ishiguro. 2007. A twomonth field trial in an elementary school for long-term human--robot interaction. IEEE Transactions on robotics 23, 5 (2007), 962--971.Google ScholarDigital Library
- Takayuki Kanda, Masahiro Shiomi, Zenta Miyashita, Hiroshi Ishiguro, and Norihiro Hagita. 2010. A communication robot in a shopping mall. IEEE Transactions on Robotics 26, 5 (2010), 897--913.Google ScholarDigital Library
- Cory D Kidd and Cynthia Breazeal. 2008. Robots at home: Understanding longterm human-robot interaction. In 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 3230--3235.Google ScholarCross Ref
- SunKyoung Kim, Masakazu Hirokawa, Atsushi Funahashi, and Kenji Suzuki. 2022. What Can We Do with a Robot for Family Playtime?. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI '22). IEEE Press, Sapporo, Hokkaido, Japan, 847--849.Google ScholarDigital Library
- Maximilian Klein, Jinhao Zhao, Jiajun Ni, Isaac Johnson, Benjamin Mako Hill, and Haiyi Zhu. 2017. Quality Standards, Service Orientation, and Power in Airbnb and Couchsurfing. Proc. ACM Hum.-Comput. Interact. 1, CSCW (Dec. 2017), 58:1--58:21.Google ScholarDigital Library
- Jacqueline M. Kory-Westlund and Cynthia Breazeal. 2019. A Long-Term Study of Young Children's Rapport, Social Emulation, and Language Learning With a Peer-Like Robot Playmate in Preschool. Frontiers in Robotics and AI 6 (2019). https://www.frontiersin.org/articles/10.3389/frobt.2019.00081Google Scholar
- Min Kyung Lee, Sara Kiesler, Jodi Forlizzi, and Paul Rybski. 2012. Ripple effects of an embedded social agent: a field study of a social robot in the workplace. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 695--704.Google ScholarDigital Library
- Iolanda Leite, Carlos Martinho, and Ana Paiva. 2013. Social robots for long-term interaction: A survey. International Journal of Social Robotics 5, 2 (2013), 291--308.Google ScholarCross Ref
- James C. Lester and Brian A. Stone. 1997. Increasing believability in animated pedagogical agents. In Proceedings of the first international conference on Autonomous agents. Association for Computing Machinery, New York, NY, USA, 16--21. https://doi.org/10.1145/267658.269943Google ScholarDigital Library
- Mike Ligthart, Mark. A. Neerincx, and Koen V. Hindriks. 2019. Getting Acquainted for a Long-Term Child-Robot Interaction. In Social Robotics (Lecture Notes in Computer Science), Miguel A. Salichs, Shuzhi Sam Ge, Emilia Ivanova Barakova, John-John Cabibihan, Alan R. Wagner, Álvaro Castro-González, and Hongsheng He (Eds.). Springer International Publishing, Cham, 423--433. https://doi.org/10. 1007/978--3-030--35888--4_39Google Scholar
- Mike E.U. Ligthart, Mark A. Neerincx, and Koen V. Hindriks. 2022. Memory- Based Personalization for Fostering a Long-Term Child-Robot Relationship. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, Sapporo, Japan, 80--89. https://doi.org/10.1109/HRI53351.2022.9889446Google ScholarCross Ref
- Allison Master and Andrew N. Meltzoff. 2020. Cultural Stereotypes and Sense of Belonging Contribute to Gender Gaps in STEM. International Journal of Gender, Science and Technology 12, 1 (April 2020), 152--198. Number: 1.Google Scholar
- Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and Inter-rater Reliability in Qualitative Research: Norms and Guidelines for CSCW and HCI Practice. Proc. ACM Hum.-Comput. Interact. 3, CSCW (Nov. 2019), 72:1--72:23.Google ScholarDigital Library
- Joseph E Michaelis and Bilge Mutlu. 2018. Reading socially: Transforming the in-home reading experience with a learning-companion robot. Science Robotics 3, 21 (2018), eaat5999.Google Scholar
- David H. J. Morgan. 1996. Family connections: An introduction to family studies. Vol. 45. Polity Press Cambridge.Google Scholar
- Bilge Mutlu and Jodi Forlizzi. 2008. Robots in organizations: the role of workflow, social, and environmental factors in human-robot interaction. In 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 287-- 294.Google ScholarDigital Library
- Bilge Mutlu, Nicholas Roy, and Selma ?abanovi?. 2016. Cognitive human--robot interaction. Springer handbook of robotics (2016), 1907--1934.Google ScholarCross Ref
- Junya Nakanishi, Itaru Kuramoto, Jun Baba, Kohei Ogawa, Yuichiro Yoshikawa, and Hiroshi Ishiguro. 2020. Continuous Hospitality with Social Robots at a hotel. SN Applied Sciences 2, 3 (2020), 1--13.Google ScholarCross Ref
- Tatsuya Nomura, Takayuki Uratani, Takayuki Kanda, Kazutaka Matsumoto, Hiroyuki Kidokoro, Yoshitaka Suehiro, and Sachie Yamada. 2015. Why do children abuse robots?. In Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction extended abstracts. 63--64.Google ScholarDigital Library
- University of Michigan. 2022. April Tags. https://april.eecs.umich.edu/software/ apriltagGoogle Scholar
- Ana Paiva, Iolanda Leite, and Tiago Ribeiro. 2014. Emotion Modeling for Social Robots. The Oxford handbook of affective computing (2014), 296.Google Scholar
- Misty II Robot. 2022. Misty Robotics. https://www.mistyrobotics.comGoogle Scholar
- Everett M Rogers. 2010. Diffusion of innovations. Simon and Schuster.Google Scholar
- Raquel Ros, Marco Nalin, Rachel Wood, Paul Baxter, Rosemarijn Looije, Yannis Demiris, Tony Belpaeme, Alessio Giusti, and Clara Pozzi. 2011. Child-robot interaction in the wild: advice to the aspiring experimenter. In Proceedings of the 13th international conference on multimodal interfaces. 335--342.Google ScholarDigital Library
- Selma Sabanovic, Marek P Michalowski, and Reid Simmons. 2006. Robots in the wild: Observing human-robot social interaction outside the lab. In 9th IEEE International Workshop on Advanced Motion Control, 2006. IEEE, 596--601.Google ScholarCross Ref
- Magda Saraiva, Hande Ayano?lu, and Beste Özcan. 2019. Emotional Design and Human-Robot Interaction. In Emotional Design in Human-Robot Interaction. Springer, 119--141.Google Scholar
- Allison Sauppé and Bilge Mutlu. 2015. The social impact of a robot co-worker in industrial settings. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 3613--3622.Google ScholarDigital Library
- Brian Scassellati, Laura Boccanfuso, Chien-Ming Huang, Marilena Mademtzi, Meiying Qin, Nicole Salomons, Pamela Ventola, and Frederick Shic. 2018. Improving social skills in children with ASD using a long-term, in-home social robot. Science Robotics 3, 21 (2018), eaat7544.Google Scholar
- Masahiro Shiomi, Takayuki Kanda, Hiroshi Ishiguro, and Norihiro Hagita. 2006. Interactive humanoid robots for a science museum. In Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction. 305--312.Google ScholarDigital Library
- Mary Spagnola and Barbara H Fiese. 2007. Family routines and rituals: A context for development in the lives of young children. Infants & young children 20, 4 (2007), 284--299.Google ScholarCross Ref
- JaYoung Sung, Henrik I Christensen, and Rebecca E Grinter. 2009. Robots in the wild: understanding long-term use. In Proceedings of the 4th ACM/IEEE international conference on Human robot interaction. 45--52.Google ScholarDigital Library
- Sam Thellman, Maartje de Graaf, and Tom Ziemke. 2022. Mental State Attribution to Robots: A Systematic Review of Conceptions, Methods, and Findings. ACM Transactions on Human-Robot Interaction (THRI) 11, 4 (2022), 1--51.Google ScholarDigital Library
- AstridWeiss, Anna Pillinger, and Christiana Tsiourti. 2021. Merely a Conventional ?Diffusion'Problem? On the Adoption Process of Anki Vector. In 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN). IEEE, 712--719.Google Scholar
- SM Wyatt, Graham Thomas, and Tiziana Terranova. 2002. They came, they surfed, they went back to the beach: Conceptualizing. Virtual society (2002), 23--40.Google Scholar
- Zhao Zhao and Rhonda McEwen. 2022. "Let's read a book together": A Long-term Study on the Usage of Pre-school Children with Their Home Companion Robot. In Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI '22). IEEE Press, Sapporo, Hokkaido, Japan, 24--32.Google ScholarCross Ref
Index Terms
- "Off Script:" Design Opportunities Emerging from Long-Term Social Robot Interactions In-the-Wild
Recommendations
Mixed-Method Long-Term Robot Usage: Older Adults' Lived Experience of Social Robots
HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot InteractionIn the past two decades, human-robot interaction (HRI) researchers have increasingly deployed autonomous and reliable robots long-term in various social contexts including the home. Our work provides a mixed-method approach for analyzing older adults' ...
Supporting Long-Term HRI through Shared Family Routines
HRI '24: Companion of the 2024 ACM/IEEE International Conference on Human-Robot InteractionTechnical and practical challenges in human-robot interaction (HRI) research often involve facilitating sustained long-term interactions, fostering engagement with multiple individuals, and taking place in-the-wild. The home environment embodies all ...
Motor-impaired touchscreen interactions in the wild
ASSETS '14: Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibilityTouchscreens are pervasive in mainstream technologies; they offer novel user interfaces and exciting gestural interactions. However, to interpret and distinguish between the vast ranges of gestural inputs, the devices require users to consistently ...
Comments