Abstract
Purpose
Optical colonoscopy is a prominent procedure by which clinicians examine the surface of the colon for cancerous polyps using a flexible colonoscope. One of the main concerns regarding the quality of the colonoscopy is to ensure that the whole colonic surface has been inspected for abnormalities. In this paper, we aim at estimating areas that have not been covered thoroughly by providing a map from the internal colon surface.
Methods
Camera parameters were estimated using optical flow between consecutive colonoscopy frames. A cylinder model was fitted to the colon structure using 3D pseudo stereo vision and projected into each frame. A circumferential band from the cylinder was extracted to unroll the internal colon surface (band image). By registering these band images, drift in estimating camera motion could be reduced, and a visibility map of the colon surface could be generated, revealing uncovered areas by the colonoscope. Hidden areas behind haustral folds were ignored in this study. The method was validated on simulated and actual colonoscopy videos. The realistic simulated videos were generated using a colonoscopy simulator with known ground truth, and the actual colonoscopy videos were manually assessed by a clinical expert.
Results
The proposed method obtained a sensitivity and precision of 98 and 96 % for detecting the number of uncovered areas on simulated data, whereas validation on real videos showed a sensitivity and precision of 96 and 78 %, respectively. Error in camera motion drift could be reduced by almost 50 % using results from band image registration.
Conclusion
Using a simple cylindrical model for the colon and reducing drift by registering band images allows for the generation of visibility maps. The current results also suggest that the provided feedback through the visibility map could enhance clinicians’ awareness of uncovered areas, which in return could reduce the probability of missing polyps.











Similar content being viewed by others
References
Australian Institute of Health and Welfare. http://www.aihw.gov.au/. Accessed 16 Oct 2013
World Health Organization (WHO) Fact sheet # 297: Cancer. In: WHO. http://www.who.int/mediacentre/factsheets/fs297/en/. Accessed 21 Oct 2015
de Groen PC (2010) Advanced systems to assess colonoscopy. Gastrointest Endosc Clin N Am 20:699–716. doi:10.1016/j.giec.2010.07.012
Edakkanambeth Varayil J, Enders F, Tavanapong W, Oh J, Wong J, de Groen PC (2011) Colonoscopy: what endoscopists inspect under optimal conditions. Gastroenterology 140:S-718. doi:10.1016/S0016-5085(11)62982-X
Cotton PB, Williams CB (eds) (2008) Colonoscopy and flexible sigmoidoscopy. In: Practical gastrointestinal endoscopy: the fundamentals, 5th edn. Blackwell Publishing Ltd, Oxford, pp 81–175. doi:10.1002/9780470987032.ch6
Zauber AG, Winawer SJ, O’Brien MJ, Lansdorp-Vogelaar I, van Ballegooijen M, Hankey BF, Shi W, Bond JH, Schapiro M, Panish JF, Stewart ET, Waye JD (2012) Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med 366:687–696. doi:10.1056/NEJMoa1100370
Barclay RL, Vicari JJ, Doughty AS, Johanson JF, Greenlaw RL (2006) Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 355:2533–2541. doi:10.1056/NEJMoa055498
Oh JH, Hwang S, Cao Y, Tavanapong W, Liu D, Wong J, de Groen PC (2009) Measuring objective quality of colonoscopy. IEEE Trans Biomed Eng 56:2190–2196. doi:10.1109/TBME.2008.2006035
Filip D (2012) Colometer: a real-time quality feedback system for screening colonoscopy. World J Gastroenterol 18:4270. doi:10.3748/wjg.v18.i32.4270
Hong D, Tavanapong W, Wong J, Oh J, de Groen PC (2013) 3D reconstruction of virtual colon structures from colonoscopy images. Comput Med Imaging Graph. doi:10.1016/j.compmedimag.2013.10.005
Bergen T, Wittenberg T (2015) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform 20:304–321. doi:10.1109/JBHI.2014.2384134
Armin MA, De Visser H, Chetty G, Dumas C, Conlan D, Grimpen F, Salvado O (2015) Visibility map: a new method in evaluation quality of optical colonoscopy. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Springer, Cham, pp 396–404
Kaufman A, Wang J (2008) 3D surface reconstruction from endoscopic videos. In: Linsen L, Hagen H, Hamann B (eds) Visualization in medicine and life sciences. Springer, Berlin, pp 61–74
Koppel D, Chen C-I,Wang Y-F, Lee H, Gu J, Poirson A,Wolters R (2007) Toward automated model building from video in computer-assisted diagnoses in colonoscopy. In: Cleary KR, Miga MI (eds) Medical imaging 2007: visualization and image-guided procedures, vol 6509. San Diego, CA. doi:10.1117/12.709595
Bao G, Pahlavan K, Mi L (2015) Hybrid localization of microrobotic endoscopic capsule inside small intestine by data fusion of vision and RF sensors. IEEE Sens J 15:2669–2678. doi:10.1109/JSEN.2014.2367495
Drozdzal M, Seguí S, Vitrià J, Malagelada C, Azpiroz F, Radeva P (2013) Adaptable image cuts for motility inspection using WCE. Comput Med Imaging Graph 37:72–80. doi:10.1016/j.compmedimag.2012.09.002
Bao G, Mi L, Pahlavan K (2013) A video aided RF localization technique for the wireless capsule endoscope (WCE) inside small intestine. ACM, Brussels
Mountney P, Stoyanov D, Davison A, Yang G-Z (2006) Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. In: Larsen R, Nielsen M, Sporring J (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006. Springer, Berlin, pp 347–354
Grasa OG, Civera J, Montiel JMM (2011) EKF monocular SLAM with relocalization for laparoscopic sequences. IEEE, Shanghai
Puerto-Souza GA, Staranowicz AN, Bell CS, Valdastri P, Mariottini G-L (2014) A comparative study of ego-motion estimation algorithms for teleoperated robotic endoscopes. In: Luo X, Reichl T, Mirota D, Soper T (eds) Computer-assisted and robotic endoscopy. Springer, Cham, pp 64–76
Liu J, Subramanian KR, Yoo TS (2013) A robust method to track colonoscopy videos with non-informative images. Int J Comput Assist Radiol Surg 8:575–592. doi:10.1007/s11548-013-0814-x
Mori K, Deguchi D, Sugiyama J, Suenaga Y, Toriwaki J, Maurer CR, Takabatake H, Natori H (2002) Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images. A preliminary version of this paper was presented at the Medical Image Computing and Computer-Assisted Intervention (MICCAI) conference, Utrecht, The Netherlands (2001). Med Image Anal 6:321–336. doi:10.1016/S1361-8415(02)00089-0
Rai L, Helferty JP, Higgins WE (2008) Combined video tracking and image-video registration for continuous bronchoscopic guidance. Int J Comput Assist Radiol Surg 3:315–329. doi:10.1007/s11548-008-0241-6
Luó X, Feuerstein M, Deguchi D, Kitasaka T, Takabatake H, Mori K (2012) Development and comparison of new hybrid motion tracking for bronchoscopic navigation. Med Image Anal 16:577–596. doi:10.1016/j.media.2010.11.001
Valdastri P, Ciuti G, Verbeni A, Menciassi A, Dario P, Arezzo A, Morino M (2012) Magnetic air capsule robotic system: proof of concept of a novel approach for painless colonoscopy. Surg Endosc 26:1238–1246. doi:10.1007/s00464-011-2054-x
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge
Wang H, Mirota D, Ishii M, Hager GD (2008) Robust motion estimation and structure recovery from endoscopic image sequences with an Adaptive Scale Kernel Consensus estimator. IEEE, pp 1–7
Scaramuzza D, Martinelli A, Siegwart R (2006) A flexible technique for accurate omnidirectional camera calibration and structure from motion. IEEE, p 45
Shi J, Tomasi C (1994) Good features to track. In: IEEE computer vision and pattern recognition, Society Press, pp 593–600
Armin MA, Chetty G, Jurgen F, Visser HD, Dumas C, Fazlollahi A, Grimpen F, Salvado O (2015) Uninformative frame detection in colonoscopy through motion, edge and color features. In: International workshop on computer-assisted and robotic. doi:10.1007/978-3-319-29965-5_15
Torr PHS, Zisserman A (2000) MLESAC: a new robust estimator with application to estimating image geometry. Comput Vis Image Underst 78:138–156. doi:10.1006/cviu.1999.0832
Nister D (2004) An efficient solution to the five-point relative pose problem. IEEE Trans Pattern Anal Mach Intell 26:756–770. doi:10.1109/TPAMI.2004.17
More J (1978) The Levenberg–Marquardt algorithm: implementation and theory. Springer, Berlin
Scaramuzza D, Fraundorfer F (2011) Visual odometry: Part I—The first 30 years and fundamentals. IEEE Robot Autom Mag 18:80–92. doi:10.1109/MRA.2011.943233
Welch G, Bishop G (1995) An introduction to the Kalman filter. Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill
Nagao J, Mori K, Enjouji T, Deguchi D, Kitasaka T, Suenaga Y, Hasegawa J, Toriwaki J, Takabatake H, Natori H (2004) Fast and accurate bronchoscope tracking using image registration and motion prediction. In: Barillot C, Haynor DR, Hellier P (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2004. Springer, Berlin, pp 551–558
Rabbani T, Dijkman S, van den Heuvel F, Vosselman G (2007) An integrated approach for modelling and global registration of point clouds. ISPRS J Photogramm Remote Sens 61:355–370. doi:10.1016/j.isprsjprs.2006.09.006
Zhen Z, Jinwu Q, Yanan Z, Linyong S (2006) An intelligent endoscopic navigation system. IEEE, pp 1653–1657
Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359. doi:10.1016/j.cviu.2007.09.014
De Visser H, Passenger J, Conlan D, Russ C, Hellier D, Cheng M, Acosta O, Ourselin S, Salvado O (2010) Developing a next generation colonoscopy simulator. Int J Image Graph 10:203–217. doi:10.1142/S0219467810003731
OpenGL—the industry standard for high performance graphics. https://www.opengl.org/. Accessed 1 Feb 2016
Han J (2006) Data mining: concepts and techniques, 2nd edn. Elsevier; Morgan Kaufmann, Amsterdam; Boston
Konda V, Chauhan SS, Abu Dayyeh BK, Hwang JH, Komanduri S, Manfredi MA, Maple JT, Murad FM, Siddiqui UD, Banerjee S (2015) Endoscopes and devices to improve colon polyp detection. Gastrointest Endosc 81:1122–1129. doi:10.1016/j.gie.2014.10.006
Zhou Jin, Das A, Li Feng, Li Baoxin (2008) Circular generalized cylinder fitting for 3D reconstruction in endoscopic imaging based on MRF. In: IEEE computer vision and pattern recognition workshop, pp 1–8
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required.
Conflict of interest
Aspects of the technology might be covered by a patent under review where some of the authors are inventors; the patent would be owned by their institution.
Rights and permissions
About this article
Cite this article
Armin, M.A., Chetty, G., De Visser, H. et al. Automated visibility map of the internal colon surface from colonoscopy video. Int J CARS 11, 1599–1610 (2016). https://doi.org/10.1007/s11548-016-1462-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-016-1462-8