skip to main content
10.1145/3458380.3458398acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicdspConference Proceedingsconference-collections
research-article

Surface and Volume Fusion Rendering for Augmented Reality Based Functional Endoscopic Sinus Surgery

Published: 23 September 2021 Publication History

Abstract

Functional endoscopic sinus surgery (FESS) is widely used in head and neck clinical surgery. The nasal cavity is intraoperatively visualised using an endoscope. However, the correct identification of complex structures and the perception of key target spatial relationship are difficult to perform using 2D endoscopic images. Surgeons need to visualise a 3D structure from endoscopic images and patients’ preoperative computed tomography (CT) images. Therefore, this paper presents a fusion rendering method for augmented reality based on endoscopic imaging. Motion consistency was performed to improve the number and accuracy of texture-less endoscopic image matching. The gradient optimisation of volume data was used to enhance the rendering and improve the distance perception of multi-layer information. The surface fusion error of the reconstructed surface and CT extraction reached 0.58mm, 3.86mm, and 4.03mm in the model data, cadaver skull data and clinical data, respectively. Various experimental results proved that our method can provide the accurate surface structure of the nasal cavity and can effectively improve the depth distinction of multiple objects for clinical surgery.

References

[1]
Totz Johannes, Mountney Peter, Stoyanov Danailet al, Dense surface reconstruction for enhanced navigation in MIS[M], 2011: 89-96.
[2]
Visentini-Scarzanella Marco, Mylonas George P., Stoyanov Danailet al, i-BRUSH: A gaze-contingent virtual paintbrush for dense 3D reconstruction in robotic assisted surgery[M], 2009: 353-360
[3]
Souzaki Ryota, Ieiri Satoshi, Uemura Munenoriet al, An augmented reality navigation system for pediatric oncologic surgery based on preoperative CT and MRI images[M], 2013: 2479-2483.
[4]
Lowe D G . Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[5]
Rublee E, Rabaud V, Konolige K, ORB: An efficient alternative to SIFT or SURF[C]//ICCV. 2011, 11(1): 2.
[6]
Lin B, Sun Y, Sanchez J E, Efficient vessel feature detection for endoscopic image analysis[J]. IEEE Transactions on Biomedical Engineering, 2014, 62(4): 1141-1150.
[7]
Yang X, Cheng K T. Learning optimized local difference binaries for scalable augmented reality on mobile devices[J]. IEEE transactions on visualization and computer graphics, 2013, 20(6): 852-865.
[8]
Luo Z, Shen T, Zhou L, Geodesc: Learning local descriptors by integrating geometry constraints[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 168-183.
[9]
Chum O, Matas J, Kittler J. Locally optimized RANSAC[C]//Joint Pattern Recognition Symposium. Springer, Berlin, Heidelberg, 2003: 236-243.
[10]
Mirota Daniel J., Wang Hanzi, Taylor Russell H.et al. A system for video-Based navigation for endoscopic endonasal skull base surgery[J]. IEEE Transactions on Medical Imaging, 2012, 31)4(: 963-976.
[11]
Zang Xiao Jun, Weng Dong Dong, Wang Yong Tianet al. Navigation System for Endoscopic Sinus Surgery Based on Augmented Reality[J]. Transactions of Beijing Institute of Technology, 2010.
[12]
Wahid H, Ahmad S, Nor M A M, Prestasi kecekapan pengurusan kewangan dan agihan zakat: perbandingan antara majlis agama islam negeri di Malaysia[J]. Jurnal Ekonomi Malaysia, 2017, 51(2): 39-54.
[13]
Porter T, Duff T. Compositing digital images[C]//ACM Siggraph Computer Graphics. ACM, 1984, 18(3): 253-259.
[14]
Maule M, Comba J L D, Torchelsen R P, A survey of raster-based transparency techniques[J]. Computers & Graphics, 2011, 35(6): 1023-1034.
[15]
Lu, A., Morris, C.J., Taylor, J., Ebert, D.S., Hansen, C., Rheingans, P., Hartner, M., 2003. Illustrative interactive stipple rendering. IEEE Transactions on Visualization and Computer Graphics 9, 127–138. https://doi.org/10.1109/TVCG.2003.1196001
[16]
Ma J, Zhou H, Zhao J, Robust feature matching for remote sensing image registration via locally linear transforming[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(12): 6469-6481.
[17]
Zhou K, Gong M, Huang X, Data-parallel octrees for surface reconstruction[J]. IEEE transactions on visualization and computer graphics, 2010, 17(5): 669-681.
[18]
Heckbert P S. Survey of Texture Mapping[J]. Computer Graphics & Applications IEEE, 1986, 6(11):56-67.
[19]
M. Levoy, "Efficient Ray Tracing of Volume Data," ACM Trans. Graph. 9(3), 245–261 (1990).

Cited By

View all
  • (2024)3D reconstruction from endoscopy images: A surveyComputers in Biology and Medicine10.1016/j.compbiomed.2024.108546175(108546)Online publication date: Jun-2024
  • (2024)Augmented and Virtual Reality Applications in Rhinology: A Scoping ReviewThe Laryngoscope10.1002/lary.31602134:11(4433-4440)Online publication date: 25-Jun-2024

Index Terms

  1. Surface and Volume Fusion Rendering for Augmented Reality Based Functional Endoscopic Sinus Surgery
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Other conferences
          ICDSP '21: Proceedings of the 2021 5th International Conference on Digital Signal Processing
          February 2021
          336 pages
          ISBN:9781450389365
          DOI:10.1145/3458380
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 23 September 2021

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Augmented Reality
          2. Endoscopic Image
          3. Fusion Display
          4. Medical Visualization
          5. Volume Rendering

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Conference

          ICDSP 2021

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)21
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 17 Jan 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)3D reconstruction from endoscopy images: A surveyComputers in Biology and Medicine10.1016/j.compbiomed.2024.108546175(108546)Online publication date: Jun-2024
          • (2024)Augmented and Virtual Reality Applications in Rhinology: A Scoping ReviewThe Laryngoscope10.1002/lary.31602134:11(4433-4440)Online publication date: 25-Jun-2024

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Media

          Figures

          Other

          Tables

          Share

          Share

          Share this Publication link

          Share on social media