Skip to main content

Rib region detection for scanning path planning for fully automated robotic abdominal ultrasonography

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Scanning path planning is an essential technology for fully automated ultrasound (US) robotics. During biliary scanning, the subcostal boundary is critical body surface landmarks for scanning path planning but are often invisible, depending on the individual. This study developed a method of estimating the rib region for scanning path planning toward fully automated robotic US systems.

Methods

We proposed a method for determining the rib region using RGB-D images and respiratory variation. We hypothesized that detecting the rib region would be possible based on changes in body surface position due to breathing. We generated a depth difference image by finding the difference between the depth image taken at the resting inspiratory position and the depth image taken at the maximum inspiratory position, which clearly shows the rib region. The boundary position of the subcostal was then determined by applying training using the YOLOv5 object detection model to this depth difference image.

Results

In the experiments with healthy subjects, the proposed method of rib detection using the depth difference image marked an intersection over union (IoU) of 0.951 and average confidence of 0.77. The average error between the ground truth and predicted positions was 16.5 mm in 3D space. The results were superior to rib detection using only the RGB image.

Conclusion

The proposed depth difference imaging method, which measures respiratory variation, was able to accurately estimate the rib region without contact and physician intervention. It will be useful for planning the scan path during the biliary imaging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/ultralytics/yolov5.

References

  1. Kaminski A, Payne A, Roemer S, Ignatowski D, Khandheria BK (2020) Answering to the call of critically ill patients: Limiting sonographer exposure to COVID-19 with focused protocols. J Am Soc Echocardiogr 33:902–903. https://doi.org/10.1016/j.echo.2020.05.006

    Article  PubMed  PubMed Central  Google Scholar 

  2. Li K, Xu Y, Meng MQ (2021) An overview of systems and techniques for autonomous robotic ultrasound acquisitions. IEEE Trans Med Robot Bionics 3:510–524. https://doi.org/10.1109/TMRB.2021.3072190

    Article  Google Scholar 

  3. Koizumi N, Warisawa SI, Nagoshi M, Hashizume H, Mitsuishi M (2009) Construction methodology for a remote ultrasound diagnostic system. IEEE Trans Robot 25:522–538

    Article  Google Scholar 

  4. Zhou J, Koizumi N, Nishiyama Y, Kogiso K, Ishikawa T, Kobayashi K, Watanabe Y, Fujibayashi T, Yamada M, Matsuyama M, Tsukihara H, Tsumura R, Yoshinaka K, Matsumoto N, Ogawa M, Miyazaki H, Numata K, Nagaoka H, Iwai T, Iijima H (2022) A VS ultrasound diagnostic system with kidney image evaluation functions. Int J Comput Assist Radiol Surg 18:227–246. https://doi.org/10.1007/s11548-022-02759-0

    Article  PubMed  Google Scholar 

  5. Mustafa AS, Ishii T, Matsunaga Y, Nakadate R, Ishii H, Ogawa K, Saito A, Sugawara M, Niki K, Takanishi A (2013) Development of robotic system for autonomous liver screening using ultrasound scanning device. In: 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp 804–809. https://doi.org/10.1109/ROBIO.2013.6739561

  6. Newman PG, Rozycki GS (1998) The history of ultrasound. Surg Clin North Am 78:179–195. https://doi.org/10.1016/s0039-6109(05)70308-x

    Article  CAS  PubMed  Google Scholar 

  7. Adams RB (2022) Ultrasound scanning techniques. Surg Open Sci 10:182–207. https://doi.org/10.1016/j.sopen.2022.09.002

    Article  PubMed  PubMed Central  Google Scholar 

  8. Huang Q, Lan J, Li X (2019) Robotic arm based automatic ultrasound scanning for three-dimensional imaging. IEEE Trans Indust Inform 15:1173–1182. https://doi.org/10.1109/TII.2018.2871864

    Article  Google Scholar 

  9. Ma X, Zhang Z, Zhang HK (2021) Autonomous scanning target localization for robotic lung ultrasound imaging. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 9467–9474. https://doi.org/10.1109/iros51168.2021.9635902

  10. Tan J, Li B, Leng Y, Li Y, Peng J, Wu J, Luo B, Chen X, Rong Y, Fu C (2022) Fully automatic dual-probe lung ultrasound scanning robot for screening triage. IEEE Trans Ultrason Ferroelectr Freq Control. https://doi.org/10.1109/TUFFC.2022.3211532

    Article  PubMed  Google Scholar 

  11. Tsumura R, Koseki Y, Nitta N, Yoshinaka K (2022) Towards fully automated robotic platform for remote auscultation. Int J Med Robot Comput Assist Surg. https://doi.org/10.1002/rcs.2461

    Article  Google Scholar 

  12. Virga S, Zettinig O, Esposito M, Pfister K, Frisch B, Neff T, Navab N, Hennersperger C (2016) Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 508–513. https://doi.org/10.1109/IROS.2016.7759101

  13. Hennersperger C, Fuerst B, Virga S, Zettinig O, Frisch B, Neff T, Navab N (2017) Towards MRI-based autonomous robotic us acquisitions: a first feasibility study. IEEE Trans Med Imag 36:538–548. https://doi.org/10.1109/TMI.2016.2620723

    Article  Google Scholar 

  14. Al-Zogbi L, Singh V, Teixeira B, Ahuja A, Bagherzadeh PS, Kapoor A, Saeidi H, Fleiter T, Krieger A (2021) Autonomous robotic point-of-care ultrasound imaging for monitoring of COVID-19-induced pulmonary diseases. Front Robot AI 25:645756. https://doi.org/10.3389/frobt.2021.645756

    Article  Google Scholar 

  15. Nichols KA, Okamura AM (2015) Methods to segment hard inclusions in soft tissue during autonomous robotic palpation. IEEE Trans Rob 31:344–354. https://doi.org/10.1109/TRO.2015.2402531

    Article  Google Scholar 

  16. George Y, Aldeen M, Garnavi R (2019) Automatic nipple detection method for digital skin images with psoriasis lesions. In: 2019 Digital Image Computing: Techniques and Applications (DICTA), pp 1–8

  17. Gao Z, Hidalgo G, Simon T, Wei S, Sheikh Y (2019) OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. arXiv:1812.08008. https://doi.org/10.48550/arXiv.1812.08008

Download references

Acknowledgements

The research is supported by the JSPS KAKENHI Grant (Grant Number 21K20524) and JST FOREST Program (Grant Number JPMJFR215A).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ryosuke Tsumura.

Ethics declarations

Competing interests

All authors declare no competing financial interests.

Ethical approval

The study protocol has been reviewed and approved by the institutional review board at National Institute of Advanced Industrial Science and Technology (No. 2022-1154).

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Okuzaki, K., Koizumi, N., Yoshinaka, K. et al. Rib region detection for scanning path planning for fully automated robotic abdominal ultrasonography. Int J CARS 19, 449–457 (2024). https://doi.org/10.1007/s11548-023-03019-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-03019-5

Keywords