skip to main content
10.1145/3371382.3378240acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
abstract

There's More than Meets the Eye: Enhancing Robot Control through Augmented Visual Cues

Published: 01 April 2020 Publication History

Abstract

In this paper, we present the design of a visual feedback mechanism using Augmented Reality, which we call augmented visual cues, to assist pick-and-place tasks during robot control. We propose to augment the robot operator's visual space in order to avoid attention splitting and increase situational awareness (SA). In particular, we aim to improve on the SA concepts of perception, comprehension, and projection as well as the overall task performance. For that, we built upon the interaction design paradigm proposed by Walker et al. On the one hand, our design augments the robot to support picking-tasks; and, on the other hand, we augment the environment to support placing-tasks. We evaluated our design in a first user study, and results point to specific design aspects that need improvement while showing promise for the overall approach, in particular regarding user satisfaction and certain SA concepts.

References

[1]
J.W.S. Chong, S. K. Ong, A.Y.C. Nee, and K. Youcef-Youmi. 2009. Robot programming using augmented reality: An interactive method for planning collision-free paths. Robotics and Computer-Integrated Manufacturing, Vol. 25, 3 (2009), 689--701. https://doi.org/10.1016/j.rcim.2008.05.002
[2]
J. L. Drury, J. Scholtz, and H. A. Yanco. 2003. Awareness in human-robot interactions. In SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483), Vol. 1. IEEE, New York, New York, USA, 912--918 vol.1. https://doi.org/10.1109/ICSMC.2003.1243931
[3]
Mica R. Endsley. 1995 a. Measurement of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 37, 1 (1995), 65--84. https://doi.org/10.1518/001872095779049499
[4]
Mica R. Endsley. 1995 b. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 37, 1 (1995), 32--64. https://doi.org/10.1518/001872095779049543
[5]
H. C. Fang, S. K. Ong, and A.Y.C. Nee. 2012. Interactive robot trajectory planning and simulation using Augmented Reality. Robotics and Computer-Integrated Manufacturing, Vol. 28, 2 (2012), 227--237. https://doi.org/10.1016/j.rcim.2011.09.003
[6]
Markus Funk, Andreas B"achler, Liane B"achler, Thomas Kosch, Thomas Heidenreich, and Albrecht Schmidt. 2017. Working with Augmented Reality?. In PETRA 2017 (ICPS: ACM international conference proceeding series), Unknown (Ed.). ACM, New York, NY, USA, 222--229. https://doi.org/10.1145/3056540.3056548
[7]
Yiannis Gatsoulis, Gurvinder S. Virk, and Abbas A. Dehghani-Sanij. 2010. On the Measurement of Situation Awareness for Effective Human-Robot Interaction in Teleoperated Systems. Journal of Cognitive Engineering and Decision Making, Vol. 4, 1 (2010), 69--98. https://doi.org/10.1518/155534310X495591
[8]
G. Gkioxari, R. Girshick, P. Dollár, and K. He. 2018. Detecting and Recognizing Human-Object Interactions. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, New York, New York, USA, 8359--8367. https://doi.org/10.1109/CVPR.2018.00872
[9]
Ian P. Howard and Brian J. Rogers. 2012. Perceiving in depth . Oxford psychology series, Vol. 29. Oxford University Press, New York.
[10]
Arnold Lund. 2001. Measuring usability with the use questionnaire. Usability Interface, Vol. 8, 2 (2001), 3--6.
[11]
Daniel J. Rea, Stela H. Seo, Neil Bruce, and James E. Young. 2017. Movers, Shakers, and Those Who Stand Still: Visual Attention-Grabbing Techniques in Robot Teleoperation. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (HRI '17). Association for Computing Machinery, New York, NY, USA, 398--407. https://doi.org/10.1145/2909824.3020246
[12]
Michael Walker, Hooman Hedayati, Jennifer Lee, and Daniel Szafir. 2018. Communicating Robot Motion Intent with Augmented Reality. In HRI'18, Takayuki Kanda, Selma Sabanović, Guy Hoffman, and Adriana Tapus (Eds.). Association for Computing Machinery, New York, New York, 316--324. https://doi.org/10.1145/3171221.3171253
[13]
Rong Wen, Wei-Liang Tay, Binh P. Nguyen, Chin-Boon Chng, and Chee-Kong Chui. 2014. Hand gesture guided robot-assisted surgery based on a direct augmented reality interface. Computer methods and programs in biomedicine, Vol. 116, 2 (2014), 68--80. https://doi.org/10.1016/j.cmpb.2013.12.018
[14]
H. A. Yanco and J. Drury. op. 2004. textquotedblWhere am i?textquotedbl acquiring situation awareness using a remote robot platform. In 2004 IEEE international conference on systems, man & cybernetics theme, Peter Wieringa (Ed.). IEEE, Piscataway (N.J.), 2835--2840. https://doi.org/10.1109/ICSMC.2004.1400762

Cited By

View all
  • (2024)Cooperative augmented assembly (CAA): augmented reality for on-site cooperative robotic fabricationConstruction Robotics10.1007/s41693-024-00138-68:2Online publication date: 26-Oct-2024
  • (2023)Intuitive Robot Integration via Virtual Reality Workspaces2023 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA48891.2023.10160699(11654-11660)Online publication date: 29-May-2023
  • (2022)Advancing the Design of Visual Debugging Tools for RoboticistsProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523789(195-204)Online publication date: 7-Mar-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction
March 2020
702 pages
ISBN:9781450370578
DOI:10.1145/3371382
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2020

Check for updates

Author Tags

  1. human-robot interaction
  2. mixed reality user interfaces
  3. robot control
  4. situation awareness
  5. visualization

Qualifiers

  • Abstract

Funding Sources

  • BMBF

Conference

HRI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 192 of 519 submissions, 37%

Upcoming Conference

HRI '25
ACM/IEEE International Conference on Human-Robot Interaction
March 4 - 6, 2025
Melbourne , VIC , Australia

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)39
  • Downloads (Last 6 weeks)3
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Cooperative augmented assembly (CAA): augmented reality for on-site cooperative robotic fabricationConstruction Robotics10.1007/s41693-024-00138-68:2Online publication date: 26-Oct-2024
  • (2023)Intuitive Robot Integration via Virtual Reality Workspaces2023 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA48891.2023.10160699(11654-11660)Online publication date: 29-May-2023
  • (2022)Advancing the Design of Visual Debugging Tools for RoboticistsProceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3523760.3523789(195-204)Online publication date: 7-Mar-2022
  • (2022)Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic InterfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517719(1-33)Online publication date: 29-Apr-2022
  • (2022)Advancing the Design of Visual Debugging Tools for Roboticists2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)10.1109/HRI53351.2022.9889392(195-204)Online publication date: 7-Mar-2022
  • (2021)Exploring the Visual Space to Improve Depth Perception in Robot Teleoperation Using Augmented Reality: The Role of Distance and Target’s Pose in Time, Success, and CertaintyHuman-Computer Interaction – INTERACT 202110.1007/978-3-030-85623-6_31(522-543)Online publication date: 30-Aug-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media