Shiyuan Liu
ABSTRACT
Functional endoscopic sinus surgery (FESS) is widely used in head and neck clinical surgery. The nasal cavity is intraoperatively visualised using an endoscope. However, the correct identification of complex structures and the perception of key target spatial relationship are difficult to perform using 2D endoscopic images. Surgeons need to visualise a 3D structure from endoscopic images and patients’ preoperative computed tomography (CT) images. Therefore, this paper presents a fusion rendering method for augmented reality based on endoscopic imaging. Motion consistency was performed to improve the number and accuracy of texture-less endoscopic image matching. The gradient optimisation of volume data was used to enhance the rendering and improve the distance perception of multi-layer information. The surface fusion error of the reconstructed surface and CT extraction reached 0.58mm, 3.86mm, and 4.03mm in the model data, cadaver skull data and clinical data, respectively. Various experimental results proved that our method can provide the accurate surface structure of the nasal cavity and can effectively improve the depth distinction of multiple objects for clinical surgery.
- Totz Johannes, Mountney Peter, Stoyanov Danailet al, Dense surface reconstruction for enhanced navigation in MIS[M], 2011: 89-96.Google Scholar
- Visentini-Scarzanella Marco, Mylonas George P., Stoyanov Danailet al, i-BRUSH: A gaze-contingent virtual paintbrush for dense 3D reconstruction in robotic assisted surgery[M], 2009: 353-360Google Scholar
- Souzaki Ryota, Ieiri Satoshi, Uemura Munenoriet al, An augmented reality navigation system for pediatric oncologic surgery based on preoperative CT and MRI images[M], 2013: 2479-2483.Google ScholarCross Ref
- Lowe D G . Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.Google ScholarDigital Library
- Rublee E, Rabaud V, Konolige K, ORB: An efficient alternative to SIFT or SURF[C]//ICCV. 2011, 11(1): 2.Google Scholar
- Lin B, Sun Y, Sanchez J E, Efficient vessel feature detection for endoscopic image analysis[J]. IEEE Transactions on Biomedical Engineering, 2014, 62(4): 1141-1150.Google ScholarCross Ref
- Yang X, Cheng K T. Learning optimized local difference binaries for scalable augmented reality on mobile devices[J]. IEEE transactions on visualization and computer graphics, 2013, 20(6): 852-865.Google Scholar
- Luo Z, Shen T, Zhou L, Geodesc: Learning local descriptors by integrating geometry constraints[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 168-183.Google Scholar
- Chum O, Matas J, Kittler J. Locally optimized RANSAC[C]//Joint Pattern Recognition Symposium. Springer, Berlin, Heidelberg, 2003: 236-243.Google Scholar
- Mirota Daniel J., Wang Hanzi, Taylor Russell H.et al. A system for video-Based navigation for endoscopic endonasal skull base surgery[J]. IEEE Transactions on Medical Imaging, 2012, 31)4(: 963-976.Google ScholarCross Ref
- Zang Xiao Jun, Weng Dong Dong, Wang Yong Tianet al. Navigation System for Endoscopic Sinus Surgery Based on Augmented Reality[J]. Transactions of Beijing Institute of Technology, 2010.Google Scholar
- Wahid H, Ahmad S, Nor M A M, Prestasi kecekapan pengurusan kewangan dan agihan zakat: perbandingan antara majlis agama islam negeri di Malaysia[J]. Jurnal Ekonomi Malaysia, 2017, 51(2): 39-54.Google Scholar
- Porter T, Duff T. Compositing digital images[C]//ACM Siggraph Computer Graphics. ACM, 1984, 18(3): 253-259.Google Scholar
- Maule M, Comba J L D, Torchelsen R P, A survey of raster-based transparency techniques[J]. Computers & Graphics, 2011, 35(6): 1023-1034.Google ScholarDigital Library
- Lu, A., Morris, C.J., Taylor, J., Ebert, D.S., Hansen, C., Rheingans, P., Hartner, M., 2003. Illustrative interactive stipple rendering. IEEE Transactions on Visualization and Computer Graphics 9, 127–138. https://doi.org/10.1109/TVCG.2003.1196001Google ScholarDigital Library
- Ma J, Zhou H, Zhao J, Robust feature matching for remote sensing image registration via locally linear transforming[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(12): 6469-6481.Google ScholarCross Ref
- Zhou K, Gong M, Huang X, Data-parallel octrees for surface reconstruction[J]. IEEE transactions on visualization and computer graphics, 2010, 17(5): 669-681.Google Scholar
- Heckbert P S. Survey of Texture Mapping[J]. Computer Graphics & Applications IEEE, 1986, 6(11):56-67.Google ScholarDigital Library
- M. Levoy, "Efficient Ray Tracing of Volume Data," ACM Trans. Graph. 9(3), 245–261 (1990).Google ScholarDigital Library
Index Terms
- Surface and Volume Fusion Rendering for Augmented Reality Based Functional Endoscopic Sinus Surgery
Recommendations
Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy
Highlights- Α novel in-situ AR navigation based on the enhanced arthroscopic information is developed to solve the existed problems of in-situ AR navigation.
Abstract PurposeDuring the minimally invasive knee surgery, surgeons insert surgical instruments and arthroscopy through small incisions, and implement treatment assisted by 2D arthroscopic images. However, this 2D arthroscopic ...
Sinus Endoscopy - Application of Advanced GPU Volume Rendering for Virtual Endoscopy
For difficult cases in endoscopic sinus surgery, a careful planning of the intervention is necessary. Due to the reduced field of view during the intervention, the surgeons have less information about the surrounding structures in the working area ...
Augmented Reality System for Dental Implant Surgery
VMR '09: Proceedings of the 3rd International Conference on Virtual and Mixed Reality: Held as Part of HCI International 2009Recently, computer-assisted navigation systems have been developed to realize safe and precise surgery. In conventional systems, surgeons feel anxious intra-operatively because they have to watch a surgical monitor while operating instruments in the ...
Comments