Abstract
In minimally invasive total knee arthroplasty (MIS-TKA), it is difficult to obtain a global vision of the surgical site in the course of surgery without using X-ray imaging, which brings radiation hazard. To provide the surgeon with accurate image guidance, a Microsoft HoloLens Head-Mounted Display (HMD) based augmented reality approach is proposed. This approach mixes the patient’s virtual models with the reality by obtaining the spatial transformations via HoloLens calibration and image registration. It employs the HoloLens and a custom-made binocular camera to overlay the patient’s preoperative CT image onto the physician’s field of view by combining a hand-eye calibration method and a Sample Consensus Initial Alignment (SAC-IA) based Iterative Closest Point (ICP) registration algorithm. Using the methods above, the preoperative anatomical model of the patient’s knee together with the virtual surgical tool’s path can be displayed at correct location in the view of HoloLens. Experiments show that the final overlay error was evaluated to be within 3 mm. The results have shown the potential use of the HoloLens to provide minimally invasive total knee arthroplasties with real-time intuitive surgical visualization without introducing any X-ray radiation.
L. Wang and Z. Sun—Equal contributors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Traub, J., Sielhorst, T., Heining, S.M., et al.: Advanced display and visualization concepts for image guided surgery. J. Disp. Technol. 4(4), 483–490 (2008)
Sugimoto, M., Yasuda, H., Koda, K., et al.: Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J. Hepato-Biliary-Pancreat. Sci. 17(5), 629–636 (2010)
Gregory, T.M., Gregory, J., Sledge, J., et al.: Surgery guided by mixed reality: presentation of a proof of concept. Acta Orthop. 89(5), 480–483 (2018)
Pratt, P., Ives, M., Lawton, G., et al.: Through the HoloLens looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. Eur. Radiol. Exp. 2(1), 2 (2018)
HoloLens Mixed Reality Surgery, Holographic Augmented Mixed Reality Navigation. https://www.youtube.com/watch?v=qLGD570I1OE. Accessed 20 Apr 2019
Holographic Assisted Spine Surgery with HoloLens. https://www.youtube.com/watch?v=zC5097mA9f4. Accessed 20 Apr 2019
Tzatzairis, T., Fiska, A., Ververidis, A., et al.: Minimally invasive versus conventional approaches in total knee replacement/arthroplasty: a review of the literature. J. Orthop. 15(2), 459–466 (2018)
Wu, M.L., Chien, J.C., Wu, C.T., et al.: An augmented reality system using improved-iterative closest point algorithm for on-patient medical image visualization. Sensors 18(8), 2505 (2018)
Wang, J., Ji, X., Zhang, X., et al.: Real-time robust individual X point localization for stereoscopic tracking. Pattern Recogn. Lett. 112, 138–144 (2018)
Wang, J., Wang, T., Yang, Y., Hu, L.: Nonlinear optimal robot hand-eye calibration. J. Xian Jiaotong Univ. 45(9), 15–10 (2011)
Besl, P.J., Mckay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (2002)
Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. Int. J. Comput. Vis. 13(2), 119–152 (1994)
Radu, R.B., Nico, B., Michael, B.: Fast Point Feature Histograms (FPFH) for 3D registration. In: 2009 IEEE International Conference on Robotics and Automation, Kobe, pp. 3212–3217. IEEE (2009)
Nicolas, M., Dror, A., Niloy, J.M.: Super 4PCS fast global pointcloud registration via smart indexing. Comput. Graph. Forum 33(5), 205–215 (2014)
Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Aligning point cloud views using persistent feature histograms. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, pp. 3384–3391. IEEE (2008)
Hu, Y., Wang, T.-M., Wang, J.-C., et al.: A video-based anterior cruciate ligament reconstruction navigation and evaluation system. Chin. J. Biomed. Eng. 28(2), 231–237 (2009)
OpenGL. https://www.opengl.org. Accessed 22 Apr 2019
PCL-Point Cloud Library, http://pointclouds.org. Accessed 24 Apr 2019
Unity. https://unity.com. Accessed 24 Apr 2019
Acknowledgement
This work was partially supported by National Key R&D Program of China (Grant No. 2017YFB1303004).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, L., Sun, Z., Zhang, X., Sun, Z., Wang, J. (2019). A HoloLens Based Augmented Reality Navigation System for Minimally Invasive Total Knee Arthroplasty. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds) Intelligent Robotics and Applications. ICIRA 2019. Lecture Notes in Computer Science(), vol 11745. Springer, Cham. https://doi.org/10.1007/978-3-030-27529-7_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-27529-7_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27528-0
Online ISBN: 978-3-030-27529-7
eBook Packages: Computer ScienceComputer Science (R0)