skip to main content
10.1145/3334480.3382814acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
abstract

Kinetic AR: A Framework for Robotic Motion Systems in Spatial Computing

Published: 25 April 2020 Publication History

Abstract

We present Kinetic AR, a holistic user experience framework for visual programming of robotic motion systems in Augmented Reality. The Kinetic AR framework facilitates human-robot collaboration in a co-located environment. Our goal is to present a deployable guide for the creation and visualization of manifold robotic interfaces while maintaining a low entry barrier to complex spatial hardware programming. A two phase validation process has been conducted to assess our work. As an initial phase, we have performed a set of interviews with robotics experts. Based on these interviews we have established three main areas that our framework tackles in different time domains. In a second phase, we have developed a set of prototypes using mobile Augmented Reality that apply the principles of Kinetic AR to multiple hardware actors including an AGV, a robotic arm, and a prosthetic system. Additional feedback from experts indicate the potential of the Kinetic ARframework.

Supplemental Material

MP4 File
Preview video
SBV File
Preview video captions

References

[1]
B Akan, B Çürüklü, and others. 2010. Augmented reality meets industry: Interactive robot programming. In Proceedings of SIGRAD 2010: Content aggregation and visualization; November 25--26; 2010; Västerås; Sweden. Linköping University Electronic Press, 55--58.
[2]
Rainer Bischoff and Arif Kazi. 2004. Perspectives on augmented reality based human-robot interaction with industrial robots. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), Vol. 4. IEEE, 3226--3231.
[3]
Yuanzhi Cao, Zhuangying Xu, Fan Li, Wentao Zhong, Ke Huo, and Karthik Ramani. 2019. V. Ra: An In-Situ Visual Authoring System for Robot-IoT Task Planning with Augmented Reality. In Proceedings of the 2019 on Designing Interactive Systems Conference. 1059--1070.
[4]
Matt Carney, Tony Shu, Roman Stolyarov, Jean-Francois Duval, and Hugh Herr. 2019. Design and Preliminary Results of a Reaction Force Series Elastic Actuator for Bionic Ankle Prostheses. engrXiv.
[5]
Ravi Teja Chadalavada, Henrik Andreasson, Robert Krug, and Achim J Lilienthal. 2015. That's on my mind! robot to human intention communication through on-board projection on shared floor space. In 2015 European Conference on Mobile Robots (ECMR). IEEE, 1--6.
[6]
Jonathan Wun Shiung Chong, SKc Ong, Andrew YC Nee, and KB Youcef-Youmi. 2009. Robot programming using augmented reality: An interactive method for planning collision-free paths. Robotics and Computer-Integrated Manufacturing 25, 3 (2009), 689--701.
[7]
HC Fang, SK Ong, and AYC Nee. 2012. Robot path and end-effector orientation planning using augmented reality. Procedia CIRP 3 (2012), 191--196.
[8]
Rolando Fernandez, Nathan John, Sean Kirmani, Justin Hart, Jivko Sinapov, and Peter Stone. 2018. Passive Demonstrations of Light-Based Robot Signals for Improved Human Interpretability. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 234--239.
[9]
Anna Fuste, Judith Amores, Jam3, and Google Creative Lab. 2017. Invisible Highway. (2017). https://experiments.withgoogle.com/invisible-highway
[10]
Sunao Hashimoto, Akihiko Ishida, Masahiko Inami, and Takeo Igarashi. 2011. Touchme: An augmented reality based remote robot manipulation. In The 21st International Conference on Artificial Reality and Telexistence, Proceedings of ICAT2011, Vol. 2. Citeseer.
[11]
Hooman Hedayati, Michael Walker, and Daniel Szafir. 2018. Improving collocated robot teleoperation with augmented reality. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 78--86.
[12]
Valentin Heun, James Hobin, and Pattie Maes. 2013. Reality editor: programming smarter objects. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. ACM, 307--310.
[13]
Dinh Quang Huy, I Vietcheslav, and Gerald Seet Gim Lee. 2017. See-through and spatial augmented reality-a novel framework for human-robot interaction. In 2017 3rd International Conference on Control, Automation and Robotics (ICCAR). IEEE, 719--726.
[14]
Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016. Laplacian vision: Augmenting motion prediction via optical see-through head-mounted displays. In Proceedings of the 7th Augmented Human International Conference 2016. ACM, 16.
[15]
Shunichi Kasahara, Ryuma Niiyama, Valentin Heun, and Hiroshi Ishii. 2013. exTouch: spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. ACM, 223--228.
[16]
Eranda Lakshantha and Simon Egerton. 2014. Human Robot Interaction and Control: Translating Diagrams into an Intuitive Augmented Reality Approach. In 2014 International Conference on Intelligent Environments. IEEE, 111--116.
[17]
Sang-won Leigh and Pattie Maes. 2015. Aftermath: Visualizing consequences of actions through augmented reality. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 941--946.
[18]
Paul Milgram, Shumin Zhai, David Drascic, and Julius Grodski. 1993. Applications of augmented reality for human-robot communication. In Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'93), Vol. 3. IEEE, 1467--1472.
[19]
Faizan Muhammad, Amel Hassan, Andre Cleaver, and Jivko Sinapov. 2019. Creating a Shared Reality with Robots. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 614--615.
[20]
Christopher Reardon, Kevin Lee, and Jonathan Fink. 2018. Come see this! augmented reality to enable human-robot cooperative search. In 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). IEEE, 1--7.
[21]
Takuya Sasai, Yo Takahashi, Mitsunori Kotani, and Akio Nakamura. 2011. Development of a guide robot interacting with the user using information projection-basic system. In 2011 IEEE International Conference on Mechatronics and Automation. IEEE, 1297--1302.
[22]
Daniel Szafir, Bilge Mutlu, and Terrence Fong. 2015. Communicating directionality in flying robots. In 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 19--26.
[23]
Michael Walker, Hooman Hedayati, Jennifer Lee, and Daniel Szafir. 2018. Communicating robot motion intent with augmented reality. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 316--324.
[24]
Tom Williams, Nhan Tran, Josh Rands, and Neil T Dantam. 2018. Augmented, mixed, and virtual reality enabling of robot deixis. In International Conference on Virtual, Augmented and Mixed Reality. Springer, 257--275.

Cited By

View all
  • (2024)Development of Standalone Extended-Reality-Supported Interactive Industrial Robot Programming SystemMachines10.3390/machines1207048012:7(480)Online publication date: 17-Jul-2024
  • (2024)RoboVisAR: Immersive Authoring of Condition-based AR Robot VisualisationsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634972(462-471)Online publication date: 11-Mar-2024
  • (2024)Exploring the Role of AR Cognitive Interface in Enhancing Human-Vehicle Collaborative Driving Safety: A Design PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2023.229569541:1(115-135)Online publication date: 4-Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems
April 2020
4474 pages
ISBN:9781450368193
DOI:10.1145/3334480
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2020

Check for updates

Author Tags

  1. HCI
  2. UX
  3. augmented reality
  4. robotics
  5. spatial computing

Qualifiers

  • Abstract

Conference

CHI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)88
  • Downloads (Last 6 weeks)3
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Development of Standalone Extended-Reality-Supported Interactive Industrial Robot Programming SystemMachines10.3390/machines1207048012:7(480)Online publication date: 17-Jul-2024
  • (2024)RoboVisAR: Immersive Authoring of Condition-based AR Robot VisualisationsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634972(462-471)Online publication date: 11-Mar-2024
  • (2024)Exploring the Role of AR Cognitive Interface in Enhancing Human-Vehicle Collaborative Driving Safety: A Design PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2023.229569541:1(115-135)Online publication date: 4-Jan-2024
  • (2024)Augmented reality cognitive interface in enhancing human vehicle collaborative driving safety: a design perspectiveHuman-Machine Interface for Intelligent Vehicles10.1016/B978-0-443-23606-8.00004-X(357-389)Online publication date: 2024
  • (2024)Cooperative augmented assembly (CAA): augmented reality for on-site cooperative robotic fabricationConstruction Robotics10.1007/s41693-024-00138-68:2Online publication date: 26-Oct-2024
  • (2024)AR-enhanced digital twin for human–robot interaction in manufacturing systemsEnergy, Ecology and Environment10.1007/s40974-024-00327-79:5(530-548)Online publication date: 25-May-2024
  • (2023)Combined Framework of Multicriteria Methods to Identify Quality Attributes in Augmented Reality ApplicationsMathematics10.3390/math1113283411:13(2834)Online publication date: 24-Jun-2023
  • (2023)HAbot: a human-centered augmented reality robot programming method with the awareness of cognitive loadJournal of Intelligent Manufacturing10.1007/s10845-023-02096-235:5(1985-2003)Online publication date: 21-Mar-2023
  • (2022)Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic InterfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517719(1-33)Online publication date: 29-Apr-2022
  • (2021)UX in AR-Supported Industrial Human–Robot Collaborative Tasks: A Systematic ReviewApplied Sciences10.3390/app11211044811:21(10448)Online publication date: 7-Nov-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media