Skip to main content

Advertisement

Springer Nature Link
Log in

Endoscopic navigation in the clinic: registration in the absence of preoperative imaging

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Clinical examinations that involve endoscopic exploration of the nasal cavity and sinuses often do not have a reference preoperative image, like a computed tomography (CT) scan, to provide structural context to the clinician. The aim of this work is to provide structural context during clinical exploration without requiring additional CT acquisition.

Methods

We present a method for registration during clinical endoscopy in the absence of CT scans by making use of shape statistics from past CT scans. Using a deformable registration algorithm that uses these shape statistics along with dense point clouds from video, we simultaneously achieve two goals: (1) register the statistically mean shape of the target anatomy with the video point cloud, and (2) estimate patient shape by deforming the mean shape to fit the video point cloud. Finally, we use statistical tests to assign confidence to the computed registration.

Results

We are able to achieve submillimeter errors in registrations and patient shape reconstructions using simulated data. We establish and evaluate the confidence criteria for our registrations using simulated data. Finally, we evaluate our registration method on in vivo clinical data and assign confidence to these registrations using the criteria established in simulation. All registrations that are not rejected by our criteria produce submillimeter residual errors.

Conclusion

Our deformable registration method can produce submillimeter registrations and reconstructions as well as statistical scores that can be used to assign confidence to the registrations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Amberg B, Romdhani S, Vetter T (2007) Optimal step nonrigid ICP algorithms for surface registration. In: 2007 IEEE conference on CVPR, pp 1–8. https://doi.org/10.1109/CVPR.2007.383165

  2. Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage 54(3):2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025

    Article  PubMed  Google Scholar 

  3. Banerjee A, Dhillon I, Ghosh J, Sra S (2005) Clustering on the unit hypersphere using von Mises–Fisher distributions. J Mach Learn Res 6:1345–1382

    Google Scholar 

  4. Beichel RR, Ulrich EJ, Bauer C, Wahle A, Brown B, Chang T, Plichta KA, Smith BJ, Sunderland JJ, Braun T, Fedorov A, Clunie D, Onken M, Riesmeier J, Pieper S, Kikinis R, Graham MM, Casavant TL, Sonka M, Buatti JM (2015) Data from QIN-HEADNECK. Cancer Imaging Arch https://doi.org/10.7937/K9/TCIA.2015.K0F5CGLI

  5. Besl PJ, McKay ND (1992) A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256. https://doi.org/10.1109/34.121791

    Article  Google Scholar 

  6. Billings SD (2015) Probabilistic feature-based registration for interventional medicine. Ph.D. thesis, The Johns Hopkins University

  7. Billings SD, Taylor RH (2014) Iterative most likely oriented point registration. In: Golland P, Hata N, Barillot C, Hornegger J, Howe R (eds) Medical image computing and computer-assisted intervention–MICCAI 2014. MICCAI 2014. Lecture notes in computer science, vol 8673. Springer, Cham. https://doi.org/10.1007/978-3-319-10404-1_23

    Chapter  Google Scholar 

  8. Billings SD, Taylor RH (2015) Generalized iterative most likely oriented-point (G-IMLOP) registration. Int J CARS 10(8):1213–1226. https://doi.org/10.1007/s11548-015-1221-2

    Article  Google Scholar 

  9. Billings SD, Boctor EM, Taylor RH (2015) Iterative most-likely point registration (IMLP): a robust algorithm for computing optimal shape alignment. PLoS ONE 10(3):1–45. https://doi.org/10.1371/journal.pone.0117688

    Article  CAS  Google Scholar 

  10. Billings SD, Sinha A, Reiter A, Leonard S, Ishii M, Hager GD, Taylor RH (2016) Anatomically Constrained Video-CT Registration via the V-IMLOP Algorithm. In: Ourselin S, Joskowicz L, Sabuncu M, Unal G, Wells W (eds) Medical image computing and computer-assisted intervention–MICCAI 2016. MICCAI 2016. Lecture notes in computer science, vol 9902. Springer, Cham. https://doi.org/10.1007/978-3-319-46726-9_16

    Google Scholar 

  11. Bosch WR, Straube WL, Matthews JW, Purdy JA (2015) Data from Head–Neck\_Cetuximab. Cancer Imaging Arch. https://doi.org/10.7937/K9/TCIA.2015.7AKGJUPZ

  12. Bouaziz S, Tagliasacchi A, Pauly M (2013) Sparse iterative closest point. In: Computer graphics forum, vol 32. Wiley, pp 113–123. https://doi.org/10.1111/cgf.12178

    Article  Google Scholar 

  13. Burschka D, Li M, Ishii M, Taylor RH, Hager GD (2005) Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery. Med Image Anal 9(5):413–426. https://doi.org/10.1016/j.media.2005.05.005

    Article  PubMed  Google Scholar 

  14. Caulley L, Thavorn K, Rudmik L, Cameron C, Kilty SJ (2015) Direct costs of adult chronic rhinosinusitis by using 4 methods of estimation: results of the us medical expenditure panel survey. J Allergy Clin Immunol 136(6):1517–1522. https://doi.org/10.1016/j.jaci.2015.08.037

    Article  PubMed  Google Scholar 

  15. Chen Y, Medioni G (1992) Object modelling by registration of multiple range images. Image Vis Comput 10(3):145–155. https://doi.org/10.1016/0262-8856(92)90066-C

    Article  Google Scholar 

  16. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, Prior F (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057. https://doi.org/10.1007/s10278-013-9622-7

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape modelstheir training and application. Comp Vis Im Underst 61:38–59. https://doi.org/10.1006/cviu.1995.1004

    Article  Google Scholar 

  18. Fedorov A, Clunie D, Ulrich E, Bauer C, Wahle A, Brown B, Onken M, Riesmeier J, Pieper S, Kikinis R, Buatti J, Beichel RR (2016) DICOM for quantitative imaging biomarker development: a standards based approach to sharing clinical data and structured PET/CT analysis results in head and neck cancer research. PeerJ 4:e2057. https://doi.org/10.7717/peerj.2057

    Article  PubMed  PubMed Central  Google Scholar 

  19. Fokkens WJ, Lund VJ, Mullol J, Bachert C, Alobid I, Baroody F, Cohen N, Cervin A, Douglas R, Gevaert P, Georgalas C, Goossens H, Harvey R, Hellings P, Hopkins C, Jones N, Joos G, Kalogjera L, Kern B, Kowalski M, Price D, Riechelmann H, Schlosser R, Senior B, Thomas M, Toskala E, Voegels R, de Wang Y, Wormald PJ (2012) EPOS 2012: European position paper on rhinosinusitis and nasal polyps 2012. Rhinology 50(1):1–12. https://doi.org/10.4193/Rhino50E2

    Article  PubMed  Google Scholar 

  20. Francois R, Fablet R, Barillot C (2003) Robust statistical registration of 3d ultrasound images using texture information. In: Proceedings 2003 international conference on image processing (Cat. No.03CH37429), vol 1, pp I–581. https://doi.org/10.1109/ICIP.2003.1247028

  21. Granger S, Pennec X (2002) Multi-scale EM-ICP: a fast and robust approach for surface registration. In: Heyden A, Sparr G, Nielsen M, Johansen P (eds) Computer Vision–ECCV 2002. ECCV 2002. Lecture notes in computer science, vol 2353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47979-1_28

    Chapter  Google Scholar 

  22. Granger S, Pennec X, Roche A (2001) Rigid point-surface registration using oriented points and an EM variant of ICP for computer guided oral implantology. Tech. Rep. RR-4169, INRIA

  23. Hufnagel H, Pennec X, Ehrhardt J, Ayache N, Handels H (2009) Computation of a probabilistic statistical shape model in a maximum-a-posteriori framework. Methods Inf Med 48(04):314–319. https://doi.org/10.3414/ME9228

    Article  CAS  PubMed  Google Scholar 

  24. Huhle B, Magnusson M, Strasser W, Lilienthal AJ (2008) Registration of colored 3d point clouds with a kernel-based extension to the normal distributions transform. In: 2008 IEEE ICRA, pp 4025–4030. https://doi.org/10.1109/ROBOT.2008.4543829

  25. Kainz J, Stammberger H (1989) The roof of the anterior ethmoid: a place of least resistance in the skull base. Am J Rhinol 3(4):191–199. https://doi.org/10.2500/105065889782009552

    Article  Google Scholar 

  26. Leonard S, Reiter A, Sinha A, Ishii M, Taylor RH, Hager GD (2016) Image-based navigation for functional endoscopic sinus surgery using structure from motion. In: Proceedings of SPIE, medical imaging: image processing, vol 9784, pp 97840V–7. https://doi.org/10.1117/12.2217279

  27. Leonard S, Sinha A, Reiter A, Ishii M, Gallia GL, Taylor RH, Hager GD (2018) Evaluation and stability analysis of video-based navigation system for functional endoscopic sinus surgery on in vivo clinical data. IEEE Trans Med Imaging 37(10):2185–2195. https://doi.org/10.1109/TMI.2018.2833868

    Article  PubMed  Google Scholar 

  28. Liu X, Sinha A, Unberath M, Ishii M, Hager GD, Taylor RH, Reiter A (2018) Self-supervised learning for dense depth estimation in monocular endoscopy. In: Stoyanov D et al (eds) Computer assisted robotic endoscopy–CARE 2018. Lecture notes in computer science, vol 11041. Springer, Cham. https://doi.org/10.1007/978-3-030-01201-4_15

    Google Scholar 

  29. Mardia KV, Jupp PE (2008) Directional statistics. Wiley series in probability and statistics. Wiley, New York, pp 1–432. https://doi.org/10.1002/9780470316979

    Book  Google Scholar 

  30. Min Z, Wang J, Song S, Meng MQ (2018) Robust generalized point cloud registration with expectation maximization considering anisotropic positional uncertainties. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 1290–1297. https://doi.org/10.1109/IROS.2018.8593558

  31. Min Z, Wang J, Meng MQ (2019) Joint rigid registration of multiple generalized point sets with hybrid mixture models. IEEE Trans Autom Sci Eng. https://doi.org/10.1109/TASE.2019.2906391

  32. Myronenko A, Song X (2010) Point set registration: Coherent point drift. IEEE Trans Pattern Anal Mach Intell 32(12):2262–2275. https://doi.org/10.1109/TPAMI.2010.46

    Article  PubMed  Google Scholar 

  33. Phillips JM, Liu R, Tomasi C (2007) Outlier robust icp for minimizing fractional rmsd. In: 6th international conference on 3D digital imaging and modeling, pp 427–434. https://doi.org/10.1109/3DIM.2007.39

  34. Reiter A, Leonard S, Sinha A, Ishii M, Taylor RH, Hager GD (2016) Endoscopic-CT: learning-based photometric reconstruction for endoscopic sinus surgery. In: Proceeding of SPIE, medical imaging: image processing, vol 9784, pp 978418–6. https://doi.org/10.1117/12.2216296

  35. Rusinkiewicz S, Levoy M (2001) Efficient variants of the icp algorithm. In: Proceedings of the 3rd international conference on 3D digital imaging and modeling, pp 145–152. https://doi.org/10.1109/IM.2001.924423

  36. Segal A, Haehnel D, Thrun S (2009) Generalized-ICP. In: Proceedings of robotics: science and systems, Seattle, USA. https://doi.org/10.15607/RSS.2009.V.021

  37. Sinha A (2018) Deformable registration using shape statistics with applications in sinus surgery. Ph.D. thesis. The Johns Hopkins University

  38. Sinha A, Leonard S, Reiter A, Ishii M, Taylor RH, Hager GD (2016) Automatic segmentation and statistical shape modeling of the paranasal sinuses to estimate natural variations. In: Proceedings of SPIE, medical imaging: image processing, vol 9784, pp 97840D–8. https://doi.org/10.1117/12.2217337

  39. Sinha A, Reiter A, Leonard S, Ishii M, Hager GD, Taylor RH (2017) Simultaneous segmentation and correspondence improvement using statistical modes. In: Proceedings of SPIE, medical imaging: image processing, vol 10133, pp 101331B–8. https://doi.org/10.1117/12.2253533

  40. Sinha A, Liu X, Reiter A, Ishii M, Hager GD, Taylor RH (2018) Endoscopic navigation in the absence of CT imaging. In: Frangi A, Schnabel J, Davatzikos C, Alberola-López C, Fichtinger G (eds) Medical image computing and computer assisted intervention–MICCAI 2018. MICCAI 2018. Lecture notes in computer science, vol 11073. Springer, Cham. https://doi.org/10.1007/978-3-030-00937-3_8

    Chapter  Google Scholar 

  41. Sinha A, Billings SD, Reiter A, Liu X, Ishii M, Hager GD, Taylor RH (2019) The deformable most-likely-point paradigm. Med Image Anal 55:148–164. https://doi.org/10.1016/j.media.2019.04.013

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was funded by NIH R01-EB015530, JHU Provost’s Postdoctoral Fellowship, and JHU internal funds. We would also like to acknowledge the comments and input from Seth D. Billings and Mathias Unberath.

Funding

The National Institutes of Health provided a research Grant (NIH R01-EB015530) to conduct the study that yielded the clinical dataset used in our in vivo experiment. A. Sinha was supported partly by the Johns Hopkins University (JHU) Provost’s Postdoctoral Fellowship and partly by other JHU internal funds.

Author information

Authors and Affiliations

  1. Laboratory for Computational and Sensing Robotics, The Johns Hopkins University, Baltimore, MD, 21218, USA

    Ayushi Sinha, Gregory D. Hager & Russell H. Taylor

  2. Department of Otolaryngology - Head and Neck Surgery, Johns Hopkins Medical Institutions, Baltimore, MD, 21205, USA

    Masaru Ishii

Authors
  1. Ayushi Sinha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masaru Ishii
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gregory D. Hager
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Russell H. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ayushi Sinha.

Ethics declarations

Disclaimer

The concepts and information presented in this paper are based on research and are not commercially available.

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the clinical study under JHU IRB NA_00074677.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sinha, A., Ishii, M., Hager, G.D. et al. Endoscopic navigation in the clinic: registration in the absence of preoperative imaging. Int J CARS 14, 1495–1506 (2019). https://doi.org/10.1007/s11548-019-02005-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-019-02005-0

Keywords