Abstract
Highly intelligent teleoperated robots are playing an increasingly important role in high-risk tasks. This paper proposed a teleoperation vision enhancement method in mixed reality scenarios, aiming at the problems of poor interactive telepresence at the control terminal of the teleoperation system and the difficulty for the operator to obtain real experience. By building the video stream transmission based on this method, the researchers realized the real-time presentation of the remote scene. This research carried out the interface visualization of the mixed reality visual enhancement interactive system by analyzing the target requirements in the teleoperation interaction task and improved the interactive control strategy in the mixed reality scene. This teleoperation interactive system effectively solves the problem of synchronous reproduction of remote operation scenes, improves operators’ interaction efficiency and accuracy, and verifies the advantages and application prospects of mixed reality display technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tachi, S., Inoue, Y., Kato, F.: TELESAR VI: telexistence surrogate anthropomorphic robot VI. Int. J. Hum. Rob. 17(5), 1–33 (2020)
Han, L., Zheng, T., Zhu, Y., et al.: Live semantic 3D perception for immersive augmented reality. IEEE Trans. Vis. Comput. Graph. 26(5), 2012–2022 (2020)
Kastner, L., Lambrecht, J.: Augmented-reality-based visualization of navigation data of mobile robots on the Microsoft HoloLens - possibilities and limitations. In: IEEE Conference on Cybernetics and Intelligent Systems, Robotics, Automation and Mechatronics. IEEE (2019)
Kent, D., Saldanha, C., Chernova, S.: Leveraging depth data in remote robot teleoperation interfaces for general object manipulation. Int. J. Robot. Res. 39(1), 39–53 (2020)
Shin, M., Lee, S., Song, S.W., et al.: Enhancement of perceived body ownership in virtual reality-based teleoperation may backfire in the execution of high-risk tasks. Comput. Hum. Behav. 115(52), 1–11 (2021)
Wang, B., Zhang, R., Xi, C., et al.: Virtual and real-time synchronous interaction for playing table tennis with holograms in mixed reality. Sensors 20(17), 4857 (2020)
Puri, N., Alsadoon, A., Prasad, P.W.C., et al.: Mixed reality using illumination-aware gradient mixing in surgical telepresence: enhanced multi-layer visualization. Multimed. Tools Appl. 81(1), 1153–1178 (2022)
Fang, D., Xu, H., Yang, X., et al.: An augmented reality-based method for remote collaborative real-time assistance: from a system perspective. Mob. Netw. Appl. 25, 412–425 (2020)
Delmerico, J., Poranne, R., Bogo, F., et al.: Spatial computing and intuitive interaction: bringing mixed reality and robotics together. IEEE Robot. Autom. Mag. 29(1), 45–57 (2022)
Ho, S., Liu, P., Palombo, D.J., et al.: The role of spatial ability in mixed reality learning with the HoloLens. Anat. Sci. Educ. (2021)
Malaweera, A., Jogi, R., Wright, M., et al.: A mixed-reality holographic viewing platform enabling interaction with 3D electroanatomical maps using the HoloLens. Eur. Heart J. (2021)
Hu, F., Deng, Y., Zhou, H., et al.: A vision of an XR-aided teleoperation system toward 5G/B5G. IEEE Commun. Mag. IEEE 59(1), 34–40 (2021)
Sakr, M., Uddin, W., Loos, H.F.M.V.D.: Orthographic vision-based interface with motion-tracking system for robot arm teleoperation: a comparative study, pp. 424–426 (2020)
George, J.T., George, M.J.: Human-Computer Interaction Research and Development, pp. 287–300 (2022)
Muoz, A., Mahiques, X., Solanes, J.E., et al.: Mixed reality-based user interface for quality control inspection of car body surfaces. J. Manuf. Syst. 53, 75–92 (2019)
Lewis, J.R.: The system usability scale: past, present, and future. Int. J. Hum.-Comput. Interact. 34(9), 577–590 (2018)
Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv. Psychol. 52(6), 139–183 (1988)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, Y., Li, X., Wang, L., Cheng, Z., Mo, Z., Zhang, S. (2023). Research on Mixed Reality Visual Augmentation Method for Teleoperation Interactive System. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. HCII 2023. Lecture Notes in Computer Science, vol 14027. Springer, Cham. https://doi.org/10.1007/978-3-031-35634-6_35
Download citation
DOI: https://doi.org/10.1007/978-3-031-35634-6_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35633-9
Online ISBN: 978-3-031-35634-6
eBook Packages: Computer ScienceComputer Science (R0)