Abstract
This study developed an augmented reality (AR)-based training system for conventional manual milling operations. An Intel RealSense R200 depth camera and a Leap Motion controller were mounted on an HTC Vive head-mounted display to allow users freely walk around in a room-size AR environment to operate a full-size virtual milling machine with their barehands, using their natural operation behaviors, as if they were operating a real milling machine in the real world, without additional worn or handheld devices. GPU parallel computing was used to handle dynamic occlusions and accelerate the machining simulation to achieve a real-time simulation. Using the developed AR-based training system, novices can receive a hands-on training in a safe environment, without any injury or damage. User test results showed that using the developed AR-based training resulted in lower failure rates and inquiry times than using video training. Users also commented that the AR-based training was interesting and helpful for novices to learn the basic manual milling operation techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brooke J (1996) SUS-A quick and dirty usability scale. Usability Eval Ind 189(194):4–7
Chardonnet J-R, Fromentin G, Outeiro J (2017) Augmented reality as an aid for the use of machine tools. In: The 15th management and innovative technologies (MIT) conference, Sinaia, Romania pp 1–4
Corbett-Davies S, Green R, Clark A (2012) Physically interactive tabletop augmented reality using the Kinect. In: Proceedings of the 27th conference on image and vision computing Dunedin, New Zealand, pp 210–215
Gheorghe C, Rizzotti D, Tièche F, Carrino F, Khaled OA, Mugellini E (2015) Occlusion management in augmented reality systems for machine-tools. In: International conference on virtual, augmented and mixed reality, Los Angeles, CA, USA, pp 438–446
Khattak S, Cowan B, Chepurna I, Hogue A (2014) A real-time reconstructed 3D environment augmented with virtual objects rendered with correct occlusion. In: 2014 IEEE games media entertainment, Toronto, ON, Canada, pp 1–8
Kiswanto G, Ariansyah D (2013) Development of augmented reality (AR) for machining simulation of 3-axis CNC milling. In: International conference on advanced computer science and information systems (ICACSIS), Bali, Indonesia, pp 143–148
Leal-Meléndrez JA, Altamirano-Robles L, Gonzalez JA (2013) Occlusion handling in video-based augmented reality using the Kinect sensor for indoor registration. In: Proceedings of 18th Iberoamerican congress CIARP 2013 on progress in pattern recognition, image analysis, computer vision, and applications, Havana, Cuba, pp 447–454
Lorensen WE, Cline HE (1987) Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput Graphics Anaheim 21(4):163–169
Lu Y, Smith S (2009) GPU-based real-time occlusion in an immersive augmented reality environment. J Comput Inf Sci Eng 9(2):024501
Neugebauer R, Klimant P, Wittstock V (2010) Virtual-reality-based simulation of NC programs for milling machines. In: Proceedings of the 20th CIRP design conference on global product development, Ecole Centrale de Nantes, Nantes, France, pp 697–703
Penelle B, Debeir O (2014) Multi-sensor data fusion for hand tracking using Kinect and Leap Motion. In: Proceedings of the 2014 virtual reality international conference, Laval, France, pp 1–7
Qiu S, Fan X, Wu D, He Q, Zhou D (2013) Virtual human modeling for interactive assembly and disassembly operation in virtual reality environment. Int J Adv Manuf Technol 69:9–12
Regazzoni D, Rizzi C, Vitali A (2018) Virtual reality applications: guidelines to design natural user interface. In: Proceedings of the ASME 2018 international design engineering technical conferences and computers and information in engineering conference, Quebec, Canada, pp DETC2018-85867
Shim J, Yang Y, Kang N, Seo J, Han T-D (2016) Gesture-based interactive augmented reality content authoring system using HMD. Virtual Real 20(1):57–69
Sportillo D, Avveduto G, Tecchia F, Carrozzino M (2015) Training in VR: a preliminary study on learning assembly/disassembly sequences. In: International conference on augmented and virtual reality, Lecce, Italy, pp 332–343
Tullis TS, Stetson JN (2004) A comparison of questionnaires for assessing website usability. In: Usability professional association conference, Minneapolis, Minnesota, USA, pp 1–12
Weichert F, Bachmann D, Rudak B, Fisseler D (2013) Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5):6380–6393
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334
Zhang J, Ong S-K, Nee AY (2008) AR-assisted in situ machining simulation: architecture and implementation. In: Proceedings of the 7th ACM SIGGRAPH international conference on virtual-reality continuum and its applications in industry, p 26
Acknowledgements
The authors would like to thank the Ministry of Science and Technology, Taiwan, Republic of China for financially supporting this research under Contract MOST 108-2221-E-002-161-MY2.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, CK., Chen, YH., Chuang, TJ. et al. An augmented reality-based training system with a natural user interface for manual milling operations. Virtual Reality 24, 527–539 (2020). https://doi.org/10.1007/s10055-019-00415-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-019-00415-8