Skip to main content

Advertisement

SpringerLink
Log in

An endoscopic structured light system using multispectral detection

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

In clinical examinations, the tissue surface topology is an important feature for detecting the tissue pathology and implementing augmented reality. We have previously presented a miniaturised structured light (SL) system for recovery of tissue surface shape in minimally invasive surgery (MIS), based on a flexible multispectral structured illumination probe (1.9 mm diameter) (Clancy et al. in Biomed Opt Express 2(11):3119–3128, 2011. doi:10.1364/BOE.2.003119). This paper reports further hardware and analytical developments to improve the light pattern decoding result and increase the reconstruction accuracy.

Methods

The feasibility of using an 8-band multispectral camera with higher pattern-colour discrimination ability than normal RGB camera in this system was studied. Additionally, the “normalised cut” algorithm was investigated to improve pattern segmentation.

Results

The whole SL system was evaluated by phantom and in vivo experiments. Higher pattern identification performance than that of an RGB camera was recorded by using the multispectral camera (average precision >97 %, average sensitivity >62 %). An average of \(0.88\,\pm \,0.67\,\hbox {mm}\) reconstruction error was achieved using the proposed pattern decoding method on a heart phantom at a working distance of approximately 10 cm.

Conclusions

The experiment showed the superiority of the multispectral camera over the RGB camera in the spot identification step. The proposed pattern decoding algorithm underwent evaluations using different experiments, showing that it provided promising reconstruction results. The potential of using this system in MIS environments has been demonstrated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Clancy NT, Stoyanov D, Maier-Hein L, Groch A, Yang G-Z, Elson DS (2011) Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging. Biomed Opt Express 2(11):3119–3128. doi:10.1364/BOE.2.003119

    Article  PubMed Central  PubMed  Google Scholar 

  2. Schwartz JJ, Lichtenstein GR (2004) Magnification endoscopy, chromoendoscopy and other novel techniques in evaluation of patients with IBD. Tech Gastrointest Endosc 6(4):182–188. doi:10.1016/j.tgie.2004.09.006

    Article  Google Scholar 

  3. Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996. doi:10.1016/j.media.2013.04.003

    Article  CAS  PubMed  Google Scholar 

  4. King AP, Edwards PJ, Maurer CR, Cunha DAd, Gaston RP, Clarkson M, Hill DLG, Hawkes DJ, Fenlon MR, Strong AJ, Cox TCS, Gleeson MJ (2000) Stereo augmented reality in the surgical microscope. Presence Teleoper Virtual Environ 9(4):360–368. doi:10.1162/105474600566862

    Article  Google Scholar 

  5. Geng J (2011) Structured-light 3D surface imaging: a tutorial. Adv Opt Photon 3(2):128–160. doi:10.1364/AOP.3.000128

    Article  CAS  Google Scholar 

  6. Ackerman JD, Keller K, Fuchs H (2002) Surface reconstruction of abdominal organs using laparoscopic structured light for augmented reality. Society of photo-optical instrumentation engineers, vol 4661. INTERNATIONAL, Bellingham

  7. Chan M, Lin W, Zhou C, Qu JY (2003) Miniaturized three-dimensional endoscopic imaging system based on active stereovision. Appl Opt 42(10):1888–1898. doi:10.1364/AO.42.001888

    Article  PubMed  Google Scholar 

  8. Wu TT, Qu JY (2007) Optical imaging for medical diagnosis based on active stereo vision and motion tracking. Opt Express 15(16):10421–10426. doi:10.1364/OE.15.010421

    Article  PubMed  Google Scholar 

  9. Maurice X, Albitar C, Doignon C, de Mathelin M (2012) A structured light-based laparoscope with real-time organs’ surface reconstruction for minimally invasive surgery. In:Annual international conference of the IEEE engineering in medicine and biology society (EMBC), Aug. 28 2012-Sept. 1. pp 5769–5772. doi:10.1109/EMBC.2012.6347305

  10. Schmalz C, Forster F, Schick A, Angelopoulou E (2012) An endoscopic 3D scanner based on structured light. Med Image Anal 16(5):1063–1072. doi:10.1016/j.media.2012.04.001

    Article  PubMed  Google Scholar 

  11. Sorg BS, Moeller BJ, Donovan O, Cao Y, Dewhirst MW (2005) Hyperspectral imaging of hemoglobin saturation in tumor microvasculature and tumor hypoxia development. BIOMEDO 10(4):044004-044004-044011. doi:10.1117/1.2003369

    Google Scholar 

  12. Tracy CR, Terrell JD, Francis RP, Wehner EF, Smith J, Litorja M, Hawkins DL, Pearle MS, Cadeddu JA, Zuzak KJ (2010) Characterization of renal ischemia using DLP hyperspectral imaging: a pilot study comparing artery-only occlusion versus artery and vein occlusion. J Endourol 24(3):321–325. doi:10.1089/end.2009.0184

    Article  PubMed  Google Scholar 

  13. Yang VXD, Muller PJ, Herman P, Wilson BC (2003) A multispectral fluorescence imaging system: design and initial clinical tests in intra-operative Photofrin-photodynamic therapy of brain tumors. Lasers Surg Med 32(3):224–232. doi:10.1002/lsm.10131

    Article  PubMed  Google Scholar 

  14. Xu H, Rice BW (2009) In-vivo fluorescence imaging with a multivariate curve resolution spectral unmixing technique. BIOMEDO 14(6):064011-064011-064019. doi:10.1117/1.3258838

    Google Scholar 

  15. Soille P (2003) Morphological image analysis: principles and applications. Springer, New York

    Google Scholar 

  16. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905. doi:10.1109/34.868688

    Article  Google Scholar 

  17. Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In:Proceedings of IEEE computer society conference on computer vision and pattern recognition, 17–19 Jun 1997. pp 1106–1112. doi:10.1109/CVPR.1997.609468

  18. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334. doi:10.1109/34.888718

    Article  Google Scholar 

  19. Li Q, Dai C, Liu H, Liu J (2009) In:Leukemic cells segmentation algorithm based on molecular spectral imaging technology. pp 73830V-73830V-73837

  20. Szeliski R (2010) Computer vision: algorithms and applications. Springer, New York

    Google Scholar 

  21. Ng A, Jordan M, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Citeulike-article-id: 1158897

  22. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416. doi:10.1007/s11222-007-9033-z

    Article  Google Scholar 

  23. Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13(2):119–152. doi:10.1007/BF01427149

    Article  Google Scholar 

Download references

Acknowledgments

This project is funded by ERC grant 242991, and the NIHR Imperial Biomedical Research Centre (BRC)/Imperial Innovations Therapeutic Primer Fund (Imperial Confidence in Concept). The authors thank Northwick Park Institute for Medical Research (NPIMR) and German Cancer Research Center (DKFZ) for collaboration in experimental data acquisition. Neil Clancy is supported by an Imperial College Junior Research Fellowship.

Author information

Authors and Affiliations

  1. Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, Imperial College London, London, SW7 2AZ, UK

    Jianyu Lin, Neil T. Clancy & Daniel S. Elson

  2. Department of Computing, Imperial College London, London, SW7 2AZ, UK

    Jianyu Lin

  3. Department of Surgery and Cancer, Imperial College London, London, SW7 2AZ, UK

    Neil T. Clancy & Daniel S. Elson

Authors
  1. Jianyu Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neil T. Clancy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel S. Elson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianyu Lin.

Ethics declarations

Conflict of interest

None of the authors has a conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, J., Clancy, N.T. & Elson, D.S. An endoscopic structured light system using multispectral detection. Int J CARS 10, 1941–1950 (2015). https://doi.org/10.1007/s11548-015-1264-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-015-1264-4

Keywords

Search

Navigation