Abstract
We developed a multispectral-augmented endoscopic prototype which increases the common number of bands under analysis, allowing exploration in the visible and near infrared range (400–1000 nm). The prototype combines endoscopy with spectroscopy using white light (WL) or Narrow Band Imaging light (NBI) endoscope and two multispectral cameras connected to a twin-cam splitter. The splitter is then connected to an optical fiber and introduced in the endoscope instrument channel.
In this work, we introduce a spectral calibration and an axial displacement correction function to register both modalities. The former is based on a multi-linear transformation of multispectral bands and its performance is assessed using a Digital SG ColorChecker® pattern to report an RMSE of 6.78%. The latter relates the insertion depth of the fiberscope with the required geometric transformation. The performance was assessed using a chessboard pattern and its corner coordinates as ground truth. The mean RMSE error for the registration using our method was 2.3 ± 0.7 pixels, whereas the RMSE error using a frame by frame homographic registration was 1.2 ± 0.4 pixels. Finally, the technique was tested on mouth exploration samples to simulate in-vivo acquisition and display complete spectra for single points of analysis. The results reveal that our method provides similar performance when compared to a homographic transformation which would be impossible to perform in-vivo.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bouguet, J.Y.: Matlab camera calibration toolbox. Caltech Technical report (2000)
Burgers, K., Fessehatsion, Y., Rahmani, S., Seo, J., Wittman, T.: A comparative analysis of dimension reduction algorithms on hyperspectral data. LAMDA Research Group, August 2009
Burschka, D., Li, M., Ishii, M., Taylor, R.H., Hager, G.D.: Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery. Med. Image Anal. 9(5), 413–426 (2005)
Clancy, N.T., et al.: Multispectral image alignment using a three channel endoscope in vivo during minimally invasive surgery. Biomed. Opt. Express 3(10), 2567–2578 (2012)
Deligianni, F.: Visual augmentation for virtual enviromnents in surgical training. Ph.D. thesis, Imperial College London (2006)
Dohi, T., Matsumoto, K., Shimoyama, I.: The micro fabry-perot interferometer for the spectral endoscope. In: 18th IEEE International Conference on Micro Electro Mechanical Systems, MEMS 2005, pp. 830–833. IEEE (2005)
Galeano, J., Jolivot, R., Benezeth, Y., Marzani, F., Emile, J.F., Lamarque, D.: Analysis of multispectral images of excised colon tissue samples based on genetic algorithms. In: 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems (SITIS), pp. 833–838. IEEE (2012)
Georgieva, V.: Homomorphic filtering approach for narrowband images enhancement. J. Appl. Electromagn. (JAE), in print (2015)
Hamamoto, Y., Endo, T., Nosho, K., Arimura, Y., Sato, M., Imai, K.: Usefulness of narrow-band imaging endoscopy for diagnosis of Barrett’s esophagus. J. Gastroenterol. 39(1), 14–20 (2004)
Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1997, pp. 1106–1112. IEEE (1997)
Kiyotoki, S., et al.: New method for detection of gastric cancer by hyperspectral imaging: a pilot study. J. Biomed. Opt. 18(2), 026010–026010 (2013)
Koppel, D., Wang, Y.F., Lee, H.: Image-based rendering and modeling in video-endoscopy. In: IEEE International Symposium on Biomedical Imaging: Nano to Macro, pp. 269–272. IEEE (2004)
Lu, G., Fei, B.: Medical hyperspectral imaging: a review. J. Biomed. Opt. 19(1), 010901 (2014)
Martin, M.E., et al.: Development of an advanced hyperspectral imaging (HSI) system with applications for cancer detection. Ann. Biomed. Eng. 34(6), 1061–1068 (2006)
Martin, R., Thies, B., Gerstner, A.O.: Hyperspectral hybrid method classification for detecting altered mucosa of the human larynx. Int. J. Health Geogr. 11(1), 21 (2012)
Martinez-Herrera, S.E., et al.: Identification of precancerous lesions by multispectral gastroendoscopy. Signal Image Video Process. 10(3), 455–462 (2016)
Nair, J.J., Govindan, V.: Intensity inhomogeneity correction using modified homomorphic unsharp masking. J. Med. Imaging Health Inform. 4(2), 285–290 (2014)
Sauer, F., Khamene, A., Vogt, S.: An augmented reality navigation system with a single-camera tracker: system design and needle biopsy phantom trial. In: Dohi, Takeyoshi, Kikinis, Ron (eds.) MICCAI 2002. LNCS, vol. 2489, pp. 116–124. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45787-9_15
Scholz, M., et al.: Development of an endoscopic navigation system based on digital image processing. Comput. Aided Surg. 3(3), 134–143 (1998)
Sharma, G., Bala, R.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002)
Song, L.M.W.K., et al.: Narrow band imaging and multiband imaging. Gastrointest. Endosc. 67(4), 581–589 (2008)
Van Der Maaten, L., Postma, E., Van den Herik, J.: Dimensionality reduction: a comparative. J. Mach. Learn. Res. 10, 66–71 (2009)
Westwood, J., et al.: Reconstruction and enhancement in monocular laparoscopic imagery. Medicine Meets Virtual Reality 12: Building a Better You: the Next Tools for Medical Education, Diagnosis, and Care, vol. 98, p. 37 (2004)
Zenteno, O., Krebs, A., Treuillet, S., Lucas, Y., Benezeth, Y., Marzani, F.: Dual-channel geometric registration of a multispectral-augmented endoscopic prototype. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, pp. 75–82. INSTICC, SciTePress (2018). https://doi.org/10.5220/0006721200750082
Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 1, pp. 666–673. IEEE (1999)
Acknowledgements
The authors would like to thank M.D. Dominique Lamarque for his expertise in gastro-enterology and his precious help in the project. This work was supported by the EMMIE (Endoscopie MultiModale pour les lésions Inflammatoires de l’Estomac) project funded by the ANR-15-CE17-0015 grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zenteno, O., Krebs, A., Treuillet, S., Lucas, Y., Benezeth, Y., Marzani, F. (2019). Spatial and Spectral Calibration of a Multispectral-Augmented Endoscopic Prototype. In: Bechmann, D., et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2018. Communications in Computer and Information Science, vol 997. Springer, Cham. https://doi.org/10.1007/978-3-030-26756-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-26756-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26755-1
Online ISBN: 978-3-030-26756-8
eBook Packages: Computer ScienceComputer Science (R0)