research-article

Public Access

Detecting Contingency for HRI in Open-World Environments

Authors:

Elaine Schaertl Short,

Andrea ThomazAuthors Info & Claims

HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

Pages 425 - 433

https://doi.org/10.1145/3171221.3171271

Published: 26 February 2018 Publication History

Abstract

This paper presents a novel algorithm for detecting contingent reactions to robot behavior in noisy real-world environments with naive users. Prior work has established that one way to detect contingency is by calculating a difference metric between sensor data before and after a robot probe of the environment. Our algorithm, CIRCLE (Contingency for Interactive Real-time CLassification of Engagement) provides a new approach to calculating this difference and detecting contingency, improving the running time for the difference calculation from 2.5 seconds to approximately 0.001 seconds on an 1100-sample vector, and effectively enabling real-time detection of contingent events. We show accuracy comparable to the best offline results for detecting contingency in this way (89.5% vs 91% in prior work), and demonstrate the utility of the real-time contingency detection in a field study of a survey-administering robot in a noisy open-world environment with naive users, showing that the robot can decrease the number of requests it makes (from 38 to 13) while more efficiently collecting survey responses (30% response rate rather than 26.3%).

References

[1]

Steven M. Alessandri, Margaret W. Sullivan, and Michael Lewis. 1990. Violation of expectancy and frustration in early infancy. Developmental Psychology 26, 5 (1990), 738--744.

[2]

Minoru Asada, Koh Hosoda, Yasuo Kuniyoshi, Hiroshi Ishiguro, Toshio Inui, Yuichiro Yoshikawa, Masaki Ogino, and Chisato Yoshida. 2009. Cognitive Developmental Robotics: A Survey. IEEE Transactions on Autonomous Mental Development 1, 1 (2009), 12--34.

Digital Library

[3]

M. Asif, M. Sabeel, and Mujeeb ur-Rahman an Z. H. Khan. 2016. Waiter Robot -- Solution to Restaurant Automation. Proceedings of the 1st Student Multi Disciplinary Research Conference (MDSRC) November (2016).

[4]

Ann E Bigelow and Philippe Rochat. 2006. Two-month-old Infants - Sensitivity to Social Contingency in Mother-Infant and Stranger-Infant Interaction. Infancy 9, 3 (2006), 313--325.

[5]

Nicholas J Butko and Javier R Movellan. 2010. Infomax Control of Eye Movements. IEEE Transactions on Autonomous Mental Development 2, 2 (2010), 91--107.

Digital Library

[6]

Richard L Canfield and Marshall M Haith. 1991. Young Infants - Visual Expectations for Symmetric and Asymmetric Stimulus Sequences. Developmental Psychology 27, 2 (1991), 198--208.

[7]

Angelo Cangelosi, Giorgio Metta, Gerhard Sagerer, Stefano Nolfi, Chrystopher Nehaniv, Kerstin Fischer, Jun Tani, Tony Belpaeme, Giulio Sandini, Francesco Nori, Luciano Fadiga, Britta Wrede, Katharina Rohlfing, Elio Tuci, Kerstin Dautenhahn, Joe Saunders, and Arne Zeschel. 2010. Integration of action and language knowledge: A roadmap for Developmental robotics. IEEE Transactions on Autonomous Mental Development 2, 3 (2010), 167--195.

Digital Library

[8]

Vivian Chu, Kalesha Bullard, and Andrea L. Thomaz. 2014. Multimodal real-time contingency detection for HRI. IEEE International Conference on Intelligent Robots and Systems Iros (2014), 3327--3332.

[9]

Gunnar Farneb. 2003. Two-Frame Motion Estimation Based on Polynomial Expansion. Lecture Notes in Computer Science 2749, 1 (2003), 363--370.

Digital Library

[10]

Markus Finke, Kheng Lee Koay, Kerstin Dautenhahn, Chrystopher L. Nehaniv, Michael L. Walters, and Joe Saunders. 2005. Hey, I'm over here - How can a robot attract people's attention? Proceedings - IEEE International Workshop on Robot and Human Interactive Communication 2005 (2005), 7--12.

[11]

Kevin Gold and Brian Scassellati. 2006. Learning acceptable windows of contingency. Connection Science 18, 2 (2006), 217--228.

[12]

Marcel Heerink, Ben Kröse, Vanessa Evers, and Bob Wielinga. 2010. Assessing acceptance of assistive social agent technology by older adults: The almere model. International Journal of Social Robotics 2, 4 (2010), 361--375.

[13]

P Kaewtrakulpong and R Bowden. 2001. An Improved Adaptive Background Mixture Model for Real- time Tracking with Shadow Detection. Advanced Video Based Surveillance Systems (2001), 1--5.

[14]

Takayuki Kanda, Masahiro Shiomi, Zenta Miyashita, Hiroshi Ishiguro, and Norihiro Hagita. 2009. An affective guide robot in a shopping mall. Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction - HRI '09 (2009), 173.

Digital Library

[15]

Takayuki Kanda, Masahiro Shiomi, Zenta Miyashita, Hiroshi Ishiguro, and Norihiro Hagita. 2010. A communication robot in a shopping mall. IEEE Transactions on Robotics 26, 5 (2010), 897--913.

Digital Library

[16]

Simon Keizer, Mary Ellen Foster, Zhuoran Wang, and Oliver Lemon. 2014. Machine Learning for Social Multiparty Human--Robot Interaction. ACM Transactions on Interactive Intelligent Systems 4, 3 (2014), 1--32.

Digital Library

[17]

Jinhan Lee, Crystal Chao, Aaron F. Bobick, and Andrea L. Thomaz. 2012. Multicue Contingency Detection. International Journal of Social Robotics 4, 2 (2012), 147--161.

[18]

MK Lee and Jodi Forlizzi. 2009. Designing adaptive robotic services. Proc. of IASDR'09 November (2009), 1--10.

[19]

Timm Linder, Stefan Breuers, Bastian Leibe, and Kai O Arras. 2016. On multimodal people tracking from mobile platforms in very crowded and dynamic environments. In Robotics and Automation (ICRA), 2016 IEEE International Conference on. IEEE, 5512--5519.

[20]

Sara Ljungblad and Jirina Kotrbova. 2012. Hospital robot at work: something alien or an intelligent colleague? . . . Cooperative Work (2012), 177--186.

Digital Library

[21]

Javier R. Movellan. 2005. An infomax controller for real time detection of social contingency. Proceedings of 2005 4th IEEE International Conference on Development and Learning 2005 (2005), 19--24.

[22]

Steffen Muller, Sven Hellbach, Erik Schaffernicht, Antje Ober, Andrea Scheidig, and Horst Michael Gross. 2008. Whom to talk to? Estimating user interest from movement trajectories. Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication, RO-MAN (2008), 532--538.

[23]

Aastha Nigam and Laurel D Riek. 2015. Social context perception for mobile robots. In Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on. IEEE, 3621--3627.

[24]

Shokoofeh Pourmehr, Jack Thomas, Jake Bruce, Jens Wawerla, and Richard Vaughan. 2017. Robust sensor fusion for finding HRI partners in a crowd. Proceedings - IEEE International Conference on Robotics and Automation i (2017), 3272--3278.

[25]

Satoru Satake, Takayuki Kanda, Dylan F Glas, Michita Imai, Hiroshi Ishiguro, and Norihiro Hagita. 2013. A robot that approaches pedestrians. IEEE Transactions on Robotics 29, 2 (2013), 508--524.

Digital Library

[26]

Chao Shi, Masahiro Shiomi, Christian Smith, Takayuki Kanda, and Hiroshi Ishiguro. 2013. A Model of Distributional Handing Interaction for a Mobile Robot. Robotics: Science and Systems (2013). http://www.diva-portal.org/smash/record. jsf?pid=diva2:631957

[27]

N.M.a Su, L.S.b Liu, and A.b Lazar. 2014. Mundanely miraculous: The robot in healthcare. Proceedings of the NordiCHI 2014: The 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (2014), 391--400.

Digital Library

[28]

Hidenobu Sumioka, Yuichiro Yoshikawa, Masanori Morizono, and Minoru Asada. 2011. Socially Developmental Robot based on Self-Induced Contingency with Multi Latencies. Proceedings of The 3rd International Conference on Cognitive Neurodynamics (2011), 3--6.

[29]

G M Tarabulsy, R Tessier, and a Kappas. 1996. Contingency detection and the contingent organization of behavior in interactions: implications for socioemotional development in infancy. Psychological Bulletin 120, 1 (1996), 25--41.

[30]

John S. Watson. 1972. Smiling, Cooing, and "The Game". Merrill-Palmer Quarterly of Behavior and Development 18, 4 (1972), 323--339. http://www.jstor.org/stable/ 23084026

[31]

Mary-Anne Williams. 2012. Robot Social Intelligence. International Conference on Social Robotics 4 (2012), 45--55.

Digital Library

[32]

Fumitaka Yamaoka, Takayuki Kanda, Hiroshi Ishiguro, and Norihiro Hagita. 2006. How contingent should a communication robot be? Proceeding of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction - HRI '06 (2006), 313--320.

Digital Library

[33]

Zoran Zivkovic. 2004. Improved adaptive Gaussian mixture model for background subtraction. Proceedings of the 17th International Conference on Pattern Recognition 2, 2 (2004), 28--31.

Digital Library

[34]

Zoran Zivkovic and Ferdinand Van Der Heijden. 2006. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letters 27, 7 (2006), 773--780.

Digital Library

Cited By

Knierim MJain SAydoğan MMitra KDesai KSaran ABaraka K(2024)Leveraging Prosody as an Informative Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic ImplicationsProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685735(95-123)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685735
Yasar MIslam MIqbal T(2023)IMPRINT: Interactional Dynamics-aware Motion Prediction in Teams using Multimodal ContextACM Transactions on Human-Robot Interaction10.1145/362695413:3(1-29)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3626954
Nanavati AWalker NTaber LMavrogiannis CTakayama LCakmak MSrinivasa SSakamoto DWeiss AHiatt LShiomi M(2022)Not All Who Wander Are LostProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523817(422-431)Online publication date: 7-Mar-2022
https://dl.acm.org/doi/10.5555/3523760.3523817
Show More Cited By

Index Terms

Detecting Contingency for HRI in Open-World Environments

Recommendations

Contingency Detection in Multi-Agent Interactions
HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

Contingency is the principle of interaction that governs how one partner influences another. This paper presents a formalization for multi-agent contingency detection over discrete intervals of humanagent interaction. A multi-agent virtual sandbox is ...
Vision-based contingency detection
HRI '11: Proceedings of the 6th international conference on Human-robot interaction

We present a novel method for the visual detection of a contingent response by a human to the stimulus of a robot action. Contingency is defined as a change in an agent's behavior within a specific time window in direct response to a signal from another ...
Adaptive integration of multiple cues for contingency detection
HBU'11: Proceedings of the Second international conference on Human Behavior Unterstanding

Critical to natural human-robot interaction is the capability of robots to detect the contingent reactions by humans. In various interaction scenarios, a robot can recognize a human's intention by detecting the presence or absence of a human response to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

February 2018

468 pages

ISBN:9781450349536

DOI:10.1145/3171221

General Chairs:
Takayuki Kanda
ATR, Japan
,
Selma Ŝabanović
Indiana University Bloomington, USA
,
Program Chairs:
Guy Hoffman
Cornell University, USA
,
Adriana Tapus
ENSTA-ParisTech, France

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGCHI: ACM Special Interest Group on Computer-Human Interaction
IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

HRI '18

Sponsor:

SIGAI
SIGCHI
IEEE-RAS

HRI '18: ACM/IEEE International Conference on Human-Robot Interaction

March 5 - 8, 2018

IL, Chicago, USA

Acceptance Rates

HRI '18 Paper Acceptance Rate 49 of 206 submissions, 24%;

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
703
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)10

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Knierim MJain SAydoğan MMitra KDesai KSaran ABaraka K(2024)Leveraging Prosody as an Informative Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic ImplicationsProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685735(95-123)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685735
Yasar MIslam MIqbal T(2023)IMPRINT: Interactional Dynamics-aware Motion Prediction in Teams using Multimodal ContextACM Transactions on Human-Robot Interaction10.1145/362695413:3(1-29)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3626954
Nanavati AWalker NTaber LMavrogiannis CTakayama LCakmak MSrinivasa SSakamoto DWeiss AHiatt LShiomi M(2022)Not All Who Wander Are LostProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523817(422-431)Online publication date: 7-Mar-2022
https://dl.acm.org/doi/10.5555/3523760.3523817
Saran ADesai KChang MLioutikov RThomaz ANiekum S(2022)Understanding Acoustic Patterns of Human Teachers Demonstrating Manipulation Tasks to Robots2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS47612.2022.9981053(9172-9179)Online publication date: 23-Oct-2022
https://doi.org/10.1109/IROS47612.2022.9981053
Nanavati AWalker NTaber LMavrogiannis CTakayama LCakmak MSrinivasa S(2022)Not All Who Wander Are Lost: A Localization-Free System for In-the-Wild Mobile Robot Deployments2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889620(422-431)Online publication date: 7-Mar-2022
https://doi.org/10.1109/HRI53351.2022.9889620
Kontogiorgos DTran MGustafson JSoleymani M(2021)A Systematic Cross-Corpus Analysis of Human Reactions to Robot Conversational FailuresProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479887(112-120)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479887
Staley JShort EBethel CPaiva ABroadbent EFeil-Seifer DSzafir D(2021)Contingency Detection in Multi-Agent InteractionsCompanion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3434074.3447164(225-228)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3434074.3447164
Kontogiorgos DPereira AGustafson J(2021)Grounding behaviours with conversational interfaces: effects of embodiment and failuresJournal on Multimodal User Interfaces10.1007/s12193-021-00366-y15:2(239-254)Online publication date: 24-Mar-2021
https://doi.org/10.1007/s12193-021-00366-y
Mével GJourdan JPottier F(2020)Cosmo: a concurrent separation logic for multicore OCamlProceedings of the ACM on Programming Languages10.1145/34089784:ICFP(1-29)Online publication date: 3-Aug-2020
https://dl.acm.org/doi/10.1145/3408978
Weidner MMiller HMeiklejohn C(2020)Composing and decomposing op-based CRDTs with semidirect productsProceedings of the ACM on Programming Languages10.1145/34089764:ICFP(1-27)Online publication date: 3-Aug-2020
https://dl.acm.org/doi/10.1145/3408976
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten