skip to main content
10.1145/3555776.3577691acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

SLResNet: Neural-Network-Based End-to-End Structure Light 3D Reconstruction for Endoscope

Published: 07 June 2023 Publication History

Abstract

Three-dimensional endoscopes have been widely studied and developed for video-guided mini-invasive surgery to improve depth perception. Modern 3D endoscopes still cannot provide accurate depth readings. Moreover, the users may suffer from symptoms such as dizziness and nausea resulting from the vergence-accommodation conflict on stereo endoscopes. To resolve the aforementioned challenges, this work takes advantage of both structured light techniques and neural-network-based methods to reconstruct depth information on endoscopic images. The developed method includes SLResNet, a neural network model for end-to-end structure light pattern decoding, and a coordinate refinement algorithm. To focus on algorithm design, this work evaluated the algorithms on the projector-camera system. Using a metal gauge block as the targeted object, the maximum relative depth error is 0.396mm. This method can reconstruct at steepness up to 70 degrees stably. The errors in reconstructing the human upper jaw are less than 1mm in depth.

References

[1]
SketchFab.com. 2022. Human mouth animation. (2022). https://sketchfab.com/3d-models/human-mouth-animation.
[2]
Shinichiro Sakata, Philip M. Grove, and Andrew R. L. Stevenson. 2016. Effect of 3-Dimensional Vision on Surgeons Using the da Vinci Robot for Laparoscopy: More Than Meets the Eye. JAMA Surgery, 151, 9, (September 2016), 793--794. ISSN: 2168-6254. eprint: https://jamanetwork.com/journals/jamasurgery/articlepdf/2526247/svp160006.pdf.
[3]
Tangfei Tao, Ja Choon Koo, and Hyouk Ryeol Choi. 2008. A fast block matching algorthim for stereo correspondence. In 2008 IEEE Conference on Cybernetics and Intelligent Systems, 38--41.
[4]
David G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60, 2, (November 2004), 91--110. ISSN: 0920-5691.
[5]
Gregory Kramida. 2016. Resolving the vergence-accommodation conflict in head-mounted displays. IEEE Transactions on Visualization and Computer Graphics, 22, 7, 1912--1931.
[6]
Joaquim Salvi, Jordi Pagès, and Joan Batlle. 2004. Pattern codification strategies in structured light systems. Pattern Recognit., 37, 827--849.
[7]
Zhan Song, Suming Tang, Feifei Gu, Chu Shi, and Jianyang Feng. 2019. Doe-based structured-light method for accurate 3d sensing. Optics and Lasers in Engineering, 120, 21--30. ISSN: 0143--8166.
[8]
Hieu Nguyen, Yuzeng Wang, and Zhaoyang Wang. 2020. Single-shot 3d shape reconstruction using structured light and deep convolutional neural networks. Sensors, 20, 13. ISSN: 1424-8220.
[9]
Maurice Halioua and Hsin-Chu Liu. 1989. Optical three-dimensional sensing by phase measuring profilometry. Optics and Lasers in Engineering, 11, 3, 185--215. ISSN: 0143-8166.
[10]
Tong Jia, Yizhe Liu, Xi Yuan, Wenhao Li, Dongyue Chen, and Yichun Zhang. 2021. Depth measurement based on a convolutional neural network and structured light. Measurement Science and Technology, 33, 2, (December 2021), 025202.
[11]
Christoph Schmalz, Frank Forster, Anton Schick, and Elli Angelopoulou. 2012. An endoscopic 3d scanner based on structured light. Medical image analysis, 16 5, 1063--72.
[12]
Ryo Furukawa, Genki Nagamatsu, Shiro Oka, Takahiro Kotachi, Yuki Okamoto, Shinji Tanaka, and Hiroshi Kawasaki. 2019. Simultaneous shape and camera-projector parameter estimation for 3d endoscopic system using cnn-based grid-oneshot scan. Healthcare Technology Letters, 6, (October 2019).
[13]
Kostiantyn Khabarlak and Larysa Koriashkina. 2021. Fast facial landmark detection and applications: a survey. (2021).
[14]
F.Jessie MacWilliams and N. Sloane. 1977. Pseudo-random sequences and arrays. Proceedings of the IEEE, 64, (January 1977), 1715--1729.
[15]
Z. Song and C. Chung. 2010. Determining both surface position and orientation in structured-light-based sensing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 10, (October 2010), 1770--1780. ISSN: 1939-3539.

Index Terms

  1. SLResNet: Neural-Network-Based End-to-End Structure Light 3D Reconstruction for Endoscope

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing
    March 2023
    1932 pages
    ISBN:9781450395175
    DOI:10.1145/3555776
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 June 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. endoscope image
    2. CNN
    3. ResNet

    Qualifiers

    • Research-article

    Conference

    SAC '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

    Upcoming Conference

    SAC '25
    The 40th ACM/SIGAPP Symposium on Applied Computing
    March 31 - April 4, 2025
    Catania , Italy

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 94
      Total Downloads
    • Downloads (Last 12 months)49
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media