Skip to main content
Springer Nature Link
Log in

Depth-based branching level estimation for bronchoscopic navigation

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Bronchoscopists rely on navigation systems during bronchoscopy to reduce the risk of getting lost in the complex bronchial tree-like structure and the homogeneous bronchus lumens. We propose a patient-specific branching level estimation method for bronchoscopic navigation because it is vital to identify the branches being examined in the bronchus tree during examination.

Methods

We estimate the branching level by integrating the changes in the number of bronchial orifices and the camera motions among the frames. We extract the bronchial orifice regions from a depth image, which is generated using a cycle generative adversarial network (CycleGAN) from real bronchoscopic images. We calculate the number of orifice regions using the vertical and horizontal projection profiles of the depth images and obtain the camera-moving direction using the feature point-based camera motion estimation. The changes in the number of bronchial orifices are combined with the camera-moving direction to estimate the branching level.

Results

We used three in vivo and one phantom case to train the CycleGAN model and four in vivo cases to validate the proposed method. We manually created the ground truth of the branching level. The experimental results showed that the proposed method can estimate the branching level with an average accuracy of 87.6%. The processing time per frame was about 61 ms.

Conclusion

Experimental results show that it is feasible to estimate the branching level using the number of bronchial orifices and camera-motion estimation from real bronchoscopic images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Jemal A, Siegel R, Xu J, Ward E (2020) Cancer statistics 2020. CA Cancer J Clin 60(5):277–300

    Article  Google Scholar 

  2. Bricault I, Ferretti G, Cinquin P (1998) Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy. IEEE Trans Med Imaging 17(5):703–714

    Article  CAS  Google Scholar 

  3. Mori K, Deguchi D, Sugiyama J, Suenaga Y, Toriwaki J, Maurer JCR, Takabatake H, Natori H (2002) Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images. Med Image Anal 6(3):321–336

    Article  CAS  Google Scholar 

  4. Deguchi D, Mori K, Feuerstein M, Kitasaka T, Maurer CR, Suenaga Y, Takabatake H, Mori M, Natori H (2009) Selective image similarity measure for bronchoscope tracking based on image registration. Med Image Anal 13:621–633

    Article  Google Scholar 

  5. Shen M, Gu Y, Liu N, Yang GZ (2019) Context-aware depth and pose estimation for bronchoscopic navigation. IEEE Robot Autom Lett 4:732–739

    Article  Google Scholar 

  6. Deguchi D, Feuerstein M, Kitasaka T, Suenaga Y, Ide I, Murase H, Imaizumi K, Hasegawa Y, Mori K (2012) Real-time marker-free patient registration for electromagnetic navigated bronchoscopy: a phantom study. Int J Comput Assist Radiol Surg 7(3):359–369

    Article  Google Scholar 

  7. Wegner I, Biederer J, Tetzlaff R, Wolf I, Meinzer HP (2007) Evaluation and extension of a navigation system for bronchoscopy inside human lungs. In: Medical imaging 2007: international society for optics and photonics, visualization and image-guided procedures, 65091H

  8. Appelbaum L, Sosna J, Nissenbaum Y, Benshtein A, Goldberg SN (2011) Electromagnetic navigation system for CT-guided biopsy of small lesions. Am J Roentgenol 196(5):1194–1200

    Article  Google Scholar 

  9. Kuo SW, Tseng YF, Dai KY, Chang YC, Chen KC, Lee JM (2019) Electromagnetic navigation bronchoscopy localization versus percutaneous CT-guided localization for lung resection via video-assisted thoracoscopic surgery: a propensity-matched study. J Clin Med 8(3):379

    Article  Google Scholar 

  10. Merritt SA, Khare R, Bascom R, Higgins WE (2013) Interactive CT-video registration for the continuous guidance of bronchoscopy. IEEE Trans Med Imaging 32(8):1376–1396

    Article  Google Scholar 

  11. Mori K, Deguchi D, Akiyama K, Kitasaka T, Maurer CR, Suenaga Y, Takabatake H, Mori M, Natori H (2005) Hybrid bronchoscope tracking using a magnetic tracking sensor and image registration. In: International conference on medical image computing and computer-assisted intervention, pp 543–550

  12. Shen M, Giannarou S, Yang GZ (2015) Robust camera localisation with depth reconstruction for bronchoscopic navigation. Int J Comput Assist Radiol Surg 10(6):801–813

    Article  Google Scholar 

  13. Thomas RG, Peter JM, Demet K, Moulay M, Atul CM (2006) Electromagnetic navigation diagnostic bronchoscopy: a prospective study. Am J Respir Crit Care Med 174(9):982–989

  14. Liat A, Jacob S, Yizhak N, Alexander B, Nahum G (2011) Electromagnetic navigation system for CT-guided biopsy of small lesions. Am J Roentgenol 196(5):1194–1200

    Article  Google Scholar 

  15. Mur-Artal R, Montiel JMM, Tardos JD (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Robot 31(5):1147–1163

  16. Wang C, Oda M, Hayashi Y, Villard B, Kitasaka T, Takabatake H, Mori M, Honma H, Natori H, Mori K (2020) A visual SLAM-based bronchoscope tracking scheme for bronchoscopic navigation. Int J Comput Assisted Radiol Surg 15:1619–1630

  17. Sganga J, Eng D, Graetzel C, Camarillo D (2019) Offsetnet: deep learning for localization in the lung using rendered images. In: 2019 international conference on robotics and automation (ICRA), pp 5046–5052

  18. Gil D, Esteban-lansaque A, Borràs A, Ramírez E, Sánchez C (2020) Intraoperative extraction of airways anatomy in videobronchoscopy. IEEE Access 8:159696–159704

    Article  Google Scholar 

  19. Esteban-Lansaque A, Sánchez C, Borràs A, Diez-Ferrer M, Rosell A, Gil D (2016) Stable anatomical structure tracking for video-bronchoscopy navigation. In: Workshop on clinical image-based procedures, pp 18–26

  20. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

  21. Oda M, Tanaka K, Takabatake H, Mori M, Natori H, Mori K (2019) Realistic endoscopic image generation method using virtual-to-real image-domain translation. Healthc Technol Lett 6:214–219

  22. Mori K, Suenaga Y, Toriwaki J (2003) Fast software-based volume rendering using multimedia instructions on PC platforms and its application to virtual endoscopy. Med Imaging 2003 Physiol Funct Methods Syst Appl 5031:111–122

    Google Scholar 

  23. Savitzky A, Golay MJE (1964) Smoothing and differentiation of data by simplified least squares procedures. Anal Chem 36(8):1627–1639

    Article  CAS  Google Scholar 

  24. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: 2011 International conference on computer vision, pp 2564–2571

  25. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge

    Google Scholar 

Download references

Acknowledgements

Parts of this study were supported by JSPS KAKENHI (Grant Nos. 17H00867, 26108006, and 17K20099) and the JSPS Bilateral International Collaboration Grants.

Author information

Authors and Affiliations

  1. Graduate School of Informatics, Nagoya University, Nagoya, Japan

    Cheng Wang, Yuichiro Hayashi & Kensaku Mori

  2. Information Strategy Office, Information and Communications, Graduate School of Informatics, Nagoya University, Nagoya, Japan

    Masahiro Oda

  3. School of Information Science, Aichi Institute of Technology, Toyota, Japan

    Takayuki Kitasaka

  4. Department of Respiratory Medicine, Sapporo Minami-Sanjo Hospital, Sapporo, Japan

    Hirotsugu Takabatake

  5. Department of Respiratory Medicine, Sapporo-Kosei General Hospital, Sapporo, Japan

    Masaki Mori

  6. Medical Examination Department, Seamen’s Insurance Hokkaido Healthcare Center, Sapporo, Japan

    Hirotoshi Honma

  7. Department of Internal Medicine, Keiwakai Nishioka Hospital, Sapporo, Japan

    Hiroshi Natori

Authors
  1. Cheng Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yuichiro Hayashi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Masahiro Oda
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Takayuki Kitasaka
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Hirotsugu Takabatake
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Masaki Mori
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Hirotoshi Honma
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Hiroshi Natori
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Kensaku Mori
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Cheng Wang or Kensaku Mori.

Ethics declarations

Conflicts of interest

K. Mori is receiving research funding from Olympus (Grant No. 30,000 USD).

Ethical approval

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975 (in its most recently amended version).

Informed consent

Informed consent was obtained from all patients included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Hayashi, Y., Oda, M. et al. Depth-based branching level estimation for bronchoscopic navigation. Int J CARS 16, 1795–1804 (2021). https://doi.org/10.1007/s11548-021-02460-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02460-8

Keywords