Skip to main content
SpringerLink
Log in
Book cover

International Conference on Human-Computer Interaction

HCII 2022: Virtual, Augmented and Mixed Reality: Applications in Education, Aviation and Industry pp 291–301Cite as

Assisted Human-Robot Interaction for Industry Application Based Augmented Reality

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13318))

Abstract

Augmented Reality provides more possibilities for industrial manufacturing to solve the current problems. However, previous researches have been focused on virtual scene visualization and robot trajectory prediction. The investigation of robotic real-time teleoperation using AR is limited, which is still facing great challenges. In this research, a novel method is presented for human-robot interaction for industrial applications based on AR, like assembly. Users can intuitively teleoperate a 6-DOFs industry robot to recognize and locate entities by multi-channel operation via HoloLens2. Augmented reality, as a medium, bridges human zones and robot zones. The system can transform user instructions from virtual multi-channel user interface to robots by communication module in AR environment. The above approaches and related devices are elaborated in this paper. An industrial case is presented to implement and validate the feasibility of the AR-based human-robot interactive system. The results provide evidence to support that AR is one of the efficient methods to realize multi-channel human-robot interaction.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Wang, X.V., et al.: Closed-loop augmented reality towards accurate human-robot collaboration. CIRP Ann. 69(1), 425–428 (2020)

    Article  Google Scholar 

  2. Guo, J., et al.: Real-time Object Detection with Deep Learning for Robot Vision on Mixed Reality Device. IEEE (2021)

    Google Scholar 

  3. Kästner, L., Lambrecht, J.: Augmented-Reality-Based Visualization of Navigation Data of Mobile Robots on the Microsoft Hololens - Possibilities and Limitations (2019)

    Google Scholar 

  4. Huang, J.: A non-contact measurement method based on HoloLens. Int. J. Performabil. Eng. 14, 144 (2018)

    Google Scholar 

  5. Krupke, D., et al.: Comparison of Multimodal Heading and Pointing Gestures for Co-Located Mixed Reality Human-Robot Interaction. IEEE (2018)

    Google Scholar 

  6. Guhl, J., Nguyen, S.T., Kruger, J.: concept and architecture for programming industrial robots using augmented reality with mobile devices like Microsoft HoloLens. Institute of Electrical and Electronics Engineers Inc., Limassol, Cyprus (2017)

    Google Scholar 

  7. Perzanowski, D., et al.: Building a multimodal human-robot interface. IEEE Intell. Syst. App. 16(1), 16–21 (2001)

    Article  Google Scholar 

  8. Vahrenkamp, N., et al.: Workspace analysis for planning human-robot interaction tasks. IEEE Computer Society, Cancun, Mexico (2016)

    Google Scholar 

  9. Muhammad, F., et al.: Creating a Shared Reality with Robots. IEEE Computer Society, Daegu, Republic of Korea (2019)

    Google Scholar 

  10. Hietanen, A., et al.: AR-based interaction for human-robot collaborative manufacturing. Robot. Comput.-Integr. Manuf. 63, 101891 (2022)

    Google Scholar 

  11. Stone, K.A., et al.: Augmented Reality Interface for Industrial Robot Controllers. Institute of Electrical and Electronics Engineers Inc., Orem, UT, United States (2020)

    Google Scholar 

  12. Guhl, J., Hügle, J., Krüger, J.: Enabling human-robot-interaction via virtual and augmented reality in distributed control systems. Procedia CIRP. 76, 167–170 (2018)

    Article  Google Scholar 

  13. Xue, C., Qiao, Y., Murray, N.: Enabling Human-Robot-Interaction for Remote Robotic Operation via Augmented Reality. Virtual Institute of Electrical and Electronics Engineers Inc., Cork, Ireland (2020)

    Google Scholar 

  14. Liang, C., et al.: Robot Teleoperation System Based on Mixed Reality. Osaka Institute of Electrical and Electronics Engineers Inc., Japan (2019)

    Google Scholar 

  15. Azuma, R.T.M.: Augmented Reality A Reality. San Francisco OSA - The Optical Society CA, United States (2017)

    Google Scholar 

  16. Microsoft HoloLens | Mixed Reality Technology for Business. https://www.microsoft.com/en-us/hololens. Accessed 28 Jan 2022

  17. AUBO Collaborative Robot. https://www.aubo-cobot.com/public/. Accessed 28 Jan 2022

  18. Meng, Y., Zhuang, H.Q.: Autonomous robot calibration using vision technology. Robot. Comput. Integr. Manuf. 23(4), 436–446 (2007)

    Article  Google Scholar 

  19. Fan, L., et al.: Using AR and Unity 3D to Support Geographical Phenomena Simu-lations. Institute of Electrical and Electronics Engineers Inc. Virtual, Nanjing, China (2020)

    Google Scholar 

  20. Furlan, R.: The future of augmented reality: Hololens - Microsoft’s AR headset shines despite rough edges [resources-tools and toys]. IEEE Spectr. 53(6), 21 (2016)

    Article  Google Scholar 

  21. Wang, J., et al.: Research on automatic target detection and recognition based on deep learning. J. Visual Commun. Image Represent. 60, 44–50 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Beijing Institute of Technology, Beijing, China

    Haonan Fang, Jingqian Wen, XiaoNan Yang, Peng Wang & Yinqian Li

Authors
  1. Haonan Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingqian Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. XiaoNan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yinqian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haonan Fang .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA

    Jessie Y. C. Chen

  2. U.S. Army Combat Capabilities Development Command Soldier Center, Orlando, FL, USA

    Gino Fragomeni

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fang, H., Wen, J., Yang, X., Wang, P., Li, Y. (2022). Assisted Human-Robot Interaction for Industry Application Based Augmented Reality. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality: Applications in Education, Aviation and Industry. HCII 2022. Lecture Notes in Computer Science, vol 13318. Springer, Cham. https://doi.org/10.1007/978-3-031-06015-1_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06015-1_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06014-4

  • Online ISBN: 978-3-031-06015-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation