research-article

Field Trial of an Autonomous Shopworker Robot that Aims to Provide Friendly Encouragement and Exert Social Pressure

Authors:

Sachi Edirisinghe,

Takayuki KandaAuthors Info & Claims

HRI '24: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

Pages 194 - 202

https://doi.org/10.1145/3610977.3635007

Published: 11 March 2024 Publication History

Abstract

We developed an autonomous hatshop robot for encouraging customers to try on hats by providing comments that appropriately fit their actions, and in such a way also indirectly exerting social pressure. To enable it to offer such a service smoothly in a real shop, we developed a large system (around 150k lines of code with 23 ROS packages) integrated with various technologies, like people tracking, shopping activity recognition and navigation. The robot needed to move in narrow corridors, detect customers, and recognise their shopping activities. We employed an iterative development process, repeating trial-and-error integration with the robot in the actual shop, while also collecting real-world data during field-testing. This process enabled us to improve our shopping activity recognition system by collecting real-world data, and to adapt our software modules to the target shop environment. We report the lessons learnt during our system development process. The results of our 11-day field trial show that our robot was able to provide its services reasonably well. Many customers expressed a positive impression of the robot and its services.

References

[1]

2023. https://en.wikipedia.org/wiki/Walmart_greeter

[2]

Francesca Bertacchini, Eleonora Bilotta, and Pietro Pantano. 2017. Shopping with a robotic companion. Computers in Human Behavior 77 (2017), 382--395.

Digital Library

[3]

D Brscic, Francesco Zanlungo, and Takayuki Kanda. 2017. Modelling of pedestrian groups and application to group recognition. In 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). IEEE, 564--569.

[4]

Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6299--6308.

[5]

Yingfeng Chen, Feng Wu, Wei Shuai, and Xiaoping Chen. 2017. Robots serve humans in public places-KeJia robot as a shopping assistant. International Journal of Advanced Robotic Systems 14, 3 (2017), 1729881417703569.

[6]

Laurens De Gauquier, Malaika Brengman, and Kim Willems. 2020. The Rise of Service Robots in Retailing: Literature Review on Success factors and Pitfalls. Emerald Group Publishing Ltd., United Kingdom, 15--35.

[7]

Sachi Edirisinghe, Satoru Satake, and Takayuki Kanda. 2023. Field Trial of a Shopworker Robot with Friendly Guidance and Appropriate Admonishments. ACM Transactions on Human-Robot Interaction 12, 3 (2023), 1--37.

Digital Library

[8]

Jodi Forlizzi, Thidanun Saensuksopa, Natalie Salaets, Mike Shomin, Tekin Mericli, and Guy Hoffman. 2016. Let's be honest: A controlled field study of ethical behavior in the presence of a robot. In 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, 769--774.

Digital Library

[9]

D. Fox, W. Burgard, and S. Thrun. 1997. The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine 4, 1 (1997), 23--33. https://doi.org/10.1109/100.580977

[10]

Ulla H Graneheim and Berit Lundman. 2004. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse education today 24, 2 (2004), 105--112.

[11]

Giorgio Grisetti, Cyrill Stachniss, and Wolfram Burgard. 2007. Improved techniques for grid mapping with rao-blackwellized particle filters. IEEE transactions on Robotics 23, 1 (2007), 34--46.

Digital Library

[12]

H-M Gross, H Boehme, Ch Schroeter, Steffen Müller, Alexander König, Erik Einhorn, Ch Martin, Matthias Merten, and Andreas Bley. 2009. TOOMAS: interactive shopping guide robots in everyday use-final implementation and experiences from long-term field trials. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2005--2012.

[13]

Pranay Gupta, Anirudh Thatipelli, Aditya Aggarwal, Shubh Maheshwari, Neel Trivedi, Sourav Das, and Ravi Kiran Sarvadevabhatla. 2021. Quo vadis, skeleton action recognition? International Journal of Computer Vision 129, 7 (2021), 2097-- 2112.

Digital Library

[14]

Guy Hoffman, Jodi Forlizzi, Shahar Ayal, Aaron Steinfeld, John Antanitis, Guy Hochman, Eric Hochendoner, and Justin Finkenaur. 2015. Robot presence and human honesty: Experimental evidence. In proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction. 181--188.

Digital Library

[15]

Younbo Jung and Kwan Min Lee. 2004. Effects of physical embodiment on social presence of social robots. Proceedings of PRESENCE 2004 (2004), 80--87.

[16]

Koji Kamei, Kazuhiko Shinozawa, Tetsushi Ikeda, Akira Utsumi, Takahiro Miyashita, and Norihiro Hagita. 2010. Recommendation from Robots in a Real- World Retail Shop (ICMI-MLMI '10). Association for Computing Machinery, New York, NY, USA, Article 19, 8 pages. https://doi.org/10.1145/1891903.1891929

Digital Library

[17]

Takayuki Kanda, Dylan F Glas, Masahiro Shiomi, Hiroshi Ishiguro, and Norihiro Hagita. 2008. Who will be the customer? A social robot that anticipates people's behavior from their trajectories. In Proceedings of the 10th international conference on Ubiquitous computing. 380--389.

Digital Library

[18]

Yuya Kaneshige, Satoru Satake, Takayuki Kanda, and Michita Imai. 2021. How to overcome the difficulties in programming and debugging mobile social robots?. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. 361--369.

Digital Library

[19]

Ran Hee Kim, Yeop Moon, Jung Ju Choi, and Sonya S Kwak. 2014. The effect of robot appearance types on motivating donation. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. 210--211.

Digital Library

[20]

Ryo Kitagawa, Yuyi Liu, and Takayuki Kanda. 2021. Human-inspired motion planning for omni-directional social robots. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. 34--42.

Digital Library

[21]

Min Kyung Lee, Sara Kiesler, Jodi Forlizzi, Siddhartha Srinivasa, and Paul Rybski. 2010. Gracefully mitigating breakdowns in robotic services. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 203--210.

[22]

Jun Liu, Amir Shahroudy, Mauricio Perez, GangWang, Ling-Yu Duan, and Alex C Kot. 2019. Ntu rgb d 120: A large-scale benchmark for 3d human activity understanding. IEEE transactions on pattern analysis and machine intelligence 42, 10 (2019), 2684--2701.

[23]

Xingyang Lv, Yue Liu, Jingjing Luo, Yuqing Liu, and Chunxiao Li. 2021. Does a cute artificial intelligence assistant soften the blow? The impact of cuteness on customer tolerance of assistant service failure. Annals of Tourism Research 87 (2021), 103114. https://doi.org/10.1016/j.annals.2020.103114

[24]

Omar Mubin, Massimiliano Cappuccio, Fady Alnajjar, Muneeb Imtiaz Ahmad, and Suleman Shahid. 2020. Can a robot invigilator prevent cheating? AI & SOCIETY 35 (2020), 981--989.

Digital Library

[25]

Marketta Niemelä, Päivi Heikkilä, and Hanna Lammi. 2017. A Social Service Robot in a Shopping Mall: Expectations of the Management, Retailers and Consumers. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (Vienna, Austria) (HRI '17). Association for Computing Machinery, New York, NY, USA, 227--228. https: //doi.org/10.1145/3029798.3038301

Digital Library

[26]

Cagatay Odabasi, Florenz Graf, Jochen Lindermayr, Mayank Patel, Simon D Baumgarten, and Birgit Graf. 2022. Refilling water bottles in elderly care homes with the help of a safe service robot. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 101--110.

Digital Library

[27]

Hao Quan, Yu Hu, and Andrea Bonarini. 2022. POLIMI-ITW-S: A large-scale dataset for human activity recognition in the wild. Data in Brief 43 (2022), 108420.

[28]

Nina Riether, Frank Hegel, Britta Wrede, and Gernot Horstmann. 2012. Social Facilitation with Social Robots?. In Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction (Boston, Massachusetts, USA) (HRI '12). Association for Computing Machinery, New York, NY, USA, 41--48. https://doi.org/10.1145/2157689.2157697

Digital Library

[29]

Emma Roth. 2023. Rite Aid hit with five-year facial recognition ban over ?reckless" use. https://www.theverge.com/2023/12/19/24008516/rite-aid-facialrecognition- ban-ftc-five-years

[30]

Maha Salem, Friederike Eyssel, Katharina Rohlfing, Stefan Kopp, and Frank Joublin. 2013. To Err is Human(-like): Effects of Robot Gesture on Perceived Anthropomorphism and Likability. International Journal of Social Robotics 5, 3 (2013), 313--323. https://doi.org/10.1007/s12369-013-0196--9

[31]

Chao Shi, Satoru Satake, Takayuki Kanda, and Hiroshi Ishiguro. 2016. How would store managers employ social robots?. In 2016 11th acm/ieee international conference on human-robot interaction (hri). IEEE, 519--520.

[32]

Christopher Stanton and Catherine J Stevens. 2014. Robot pressure: the impact of robot eye gaze and lifelike bodily movements upon decision-making and trust. In Social Robotics: 6th International Conference, ICSR 2014, Sydney, NSW, Australia, October 27--29, 2014. Proceedings 6. Springer, 330--339.

[33]

Samuel Wehrli, Corinna Hertweck, Mohammadreza Amirian, Stefan Glüge, and Thilo Stadelmann. 2022. Bias, awareness, and ignorance in deep-learning-based face recognition. AI and Ethics 2, 3 (2022), 509--522.

[34]

Staff Writer. 2018. Worker shortage in Japan to hit 6.4M by 2030, survey finds. https://asia.nikkei.com/Spotlight/Japan-immigration/Workershortage- in-Japan-to-hit-6.4m-by-2030-survey-finds2#: :text=TOKYO%20-- %20Japan%20will%20face%20a%20shortage%20of,about%205%25%20of%20the% 20country%27s%202017%20population%2C%20

[35]

Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.

Cited By

Shiba AObata MKau NBeck ZShah RSudano MLee SJohal WLemaignan SBrščić DVázquez MCharisi V(2025)Look Further: Socially-Compliant Navigation System in Residential BuildingsProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721525(272-282)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721525
Khan MErden Z(2024)A Systematic Review of Social Robots in Shopping EnvironmentsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2426740(1-22)Online publication date: 26-Nov-2024
https://doi.org/10.1080/10447318.2024.2426740
Rihet MSarthou GClodic ARoy R(2024)Electrophysiological Measures for Human–Robot Collaboration Quality AssessmentDiscovering the Frontiers of Human-Robot Interaction10.1007/978-3-031-66656-8_15(363-380)Online publication date: 24-Jul-2024
https://doi.org/10.1007/978-3-031-66656-8_15

Index Terms

Field Trial of an Autonomous Shopworker Robot that Aims to Provide Friendly Encouragement and Exert Social Pressure
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Field Trial of a Shopworker Robot with Friendly Guidance and Appropriate Admonishments
We developed an admonishing service for a shopworker robot and conducted a field trial to investigate the impressions of a shop’s staff and customers. Applying the admonishing service in a real-world robot is difficult due to the high risk of rejection by ...
A semi-autonomous communication robot: a field trial at a train station
HRI '08: Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction

This paper reports an initial field trial with a prototype of a semiautonomous communication robot at a train station. We developed an operator-requesting mechanism to achieve semiautonomous operation for a communication robot functioning in real ...
Enabling Shared Attention with Customers Strengthens a Sales Robot's Social Presence
HAI '22: Proceedings of the 10th International Conference on Human-Agent Interaction

Humanlike robots interact with people in daily situations, and various studies concerning such robots have been conducted. However, robots are easily ignored due to their lack of social presence, with their customer service behavior and sales strategies ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '24: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

March 2024

982 pages

ISBN:9798400703225

DOI:10.1145/3610977

General Chairs:
Dan Grollman
Plus One Robotics, USA
,
Elizabeth Broadbent
University of Auckland, New Zealand
,
Program Chairs:
Wendy Ju
Cornell Tech, USA
,
Harold Soh
National University of Singapore, Singapore
,
Tom Williams
Colorado School of Mines, USA

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 March 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

JST, CREST
JSPS, KAKENHI
JST, AIP Trilateral AI Research
JST, Moonshot R&D

Conference

HRI '24

Sponsor:

HRI '24: ACM/IEEE International Conference on Human-Robot Interaction

March 11 - 15, 2024

CO, Boulder, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
302
Total Downloads

Downloads (Last 12 months)302
Downloads (Last 6 weeks)46

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shiba AObata MKau NBeck ZShah RSudano MLee SJohal WLemaignan SBrščić DVázquez MCharisi V(2025)Look Further: Socially-Compliant Navigation System in Residential BuildingsProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721525(272-282)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721525
Khan MErden Z(2024)A Systematic Review of Social Robots in Shopping EnvironmentsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2426740(1-22)Online publication date: 26-Nov-2024
https://doi.org/10.1080/10447318.2024.2426740
Rihet MSarthou GClodic ARoy R(2024)Electrophysiological Measures for Human–Robot Collaboration Quality AssessmentDiscovering the Frontiers of Human-Robot Interaction10.1007/978-3-031-66656-8_15(363-380)Online publication date: 24-Jul-2024
https://doi.org/10.1007/978-3-031-66656-8_15

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten