Abstract
Purpose
Scanning path planning is an essential technology for fully automated ultrasound (US) robotics. During biliary scanning, the subcostal boundary is critical body surface landmarks for scanning path planning but are often invisible, depending on the individual. This study developed a method of estimating the rib region for scanning path planning toward fully automated robotic US systems.
Methods
We proposed a method for determining the rib region using RGB-D images and respiratory variation. We hypothesized that detecting the rib region would be possible based on changes in body surface position due to breathing. We generated a depth difference image by finding the difference between the depth image taken at the resting inspiratory position and the depth image taken at the maximum inspiratory position, which clearly shows the rib region. The boundary position of the subcostal was then determined by applying training using the YOLOv5 object detection model to this depth difference image.
Results
In the experiments with healthy subjects, the proposed method of rib detection using the depth difference image marked an intersection over union (IoU) of 0.951 and average confidence of 0.77. The average error between the ground truth and predicted positions was 16.5 mm in 3D space. The results were superior to rib detection using only the RGB image.
Conclusion
The proposed depth difference imaging method, which measures respiratory variation, was able to accurately estimate the rib region without contact and physician intervention. It will be useful for planning the scan path during the biliary imaging.
Similar content being viewed by others
References
Kaminski A, Payne A, Roemer S, Ignatowski D, Khandheria BK (2020) Answering to the call of critically ill patients: Limiting sonographer exposure to COVID-19 with focused protocols. J Am Soc Echocardiogr 33:902–903. https://doi.org/10.1016/j.echo.2020.05.006
Li K, Xu Y, Meng MQ (2021) An overview of systems and techniques for autonomous robotic ultrasound acquisitions. IEEE Trans Med Robot Bionics 3:510–524. https://doi.org/10.1109/TMRB.2021.3072190
Koizumi N, Warisawa SI, Nagoshi M, Hashizume H, Mitsuishi M (2009) Construction methodology for a remote ultrasound diagnostic system. IEEE Trans Robot 25:522–538
Zhou J, Koizumi N, Nishiyama Y, Kogiso K, Ishikawa T, Kobayashi K, Watanabe Y, Fujibayashi T, Yamada M, Matsuyama M, Tsukihara H, Tsumura R, Yoshinaka K, Matsumoto N, Ogawa M, Miyazaki H, Numata K, Nagaoka H, Iwai T, Iijima H (2022) A VS ultrasound diagnostic system with kidney image evaluation functions. Int J Comput Assist Radiol Surg 18:227–246. https://doi.org/10.1007/s11548-022-02759-0
Mustafa AS, Ishii T, Matsunaga Y, Nakadate R, Ishii H, Ogawa K, Saito A, Sugawara M, Niki K, Takanishi A (2013) Development of robotic system for autonomous liver screening using ultrasound scanning device. In: 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp 804–809. https://doi.org/10.1109/ROBIO.2013.6739561
Newman PG, Rozycki GS (1998) The history of ultrasound. Surg Clin North Am 78:179–195. https://doi.org/10.1016/s0039-6109(05)70308-x
Adams RB (2022) Ultrasound scanning techniques. Surg Open Sci 10:182–207. https://doi.org/10.1016/j.sopen.2022.09.002
Huang Q, Lan J, Li X (2019) Robotic arm based automatic ultrasound scanning for three-dimensional imaging. IEEE Trans Indust Inform 15:1173–1182. https://doi.org/10.1109/TII.2018.2871864
Ma X, Zhang Z, Zhang HK (2021) Autonomous scanning target localization for robotic lung ultrasound imaging. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 9467–9474. https://doi.org/10.1109/iros51168.2021.9635902
Tan J, Li B, Leng Y, Li Y, Peng J, Wu J, Luo B, Chen X, Rong Y, Fu C (2022) Fully automatic dual-probe lung ultrasound scanning robot for screening triage. IEEE Trans Ultrason Ferroelectr Freq Control. https://doi.org/10.1109/TUFFC.2022.3211532
Tsumura R, Koseki Y, Nitta N, Yoshinaka K (2022) Towards fully automated robotic platform for remote auscultation. Int J Med Robot Comput Assist Surg. https://doi.org/10.1002/rcs.2461
Virga S, Zettinig O, Esposito M, Pfister K, Frisch B, Neff T, Navab N, Hennersperger C (2016) Automatic force-compliant robotic ultrasound screening of abdominal aortic aneurysms. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 508–513. https://doi.org/10.1109/IROS.2016.7759101
Hennersperger C, Fuerst B, Virga S, Zettinig O, Frisch B, Neff T, Navab N (2017) Towards MRI-based autonomous robotic us acquisitions: a first feasibility study. IEEE Trans Med Imag 36:538–548. https://doi.org/10.1109/TMI.2016.2620723
Al-Zogbi L, Singh V, Teixeira B, Ahuja A, Bagherzadeh PS, Kapoor A, Saeidi H, Fleiter T, Krieger A (2021) Autonomous robotic point-of-care ultrasound imaging for monitoring of COVID-19-induced pulmonary diseases. Front Robot AI 25:645756. https://doi.org/10.3389/frobt.2021.645756
Nichols KA, Okamura AM (2015) Methods to segment hard inclusions in soft tissue during autonomous robotic palpation. IEEE Trans Rob 31:344–354. https://doi.org/10.1109/TRO.2015.2402531
George Y, Aldeen M, Garnavi R (2019) Automatic nipple detection method for digital skin images with psoriasis lesions. In: 2019 Digital Image Computing: Techniques and Applications (DICTA), pp 1–8
Gao Z, Hidalgo G, Simon T, Wei S, Sheikh Y (2019) OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. arXiv:1812.08008. https://doi.org/10.48550/arXiv.1812.08008
Acknowledgements
The research is supported by the JSPS KAKENHI Grant (Grant Number 21K20524) and JST FOREST Program (Grant Number JPMJFR215A).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
All authors declare no competing financial interests.
Ethical approval
The study protocol has been reviewed and approved by the institutional review board at National Institute of Advanced Industrial Science and Technology (No. 2022-1154).
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Okuzaki, K., Koizumi, N., Yoshinaka, K. et al. Rib region detection for scanning path planning for fully automated robotic abdominal ultrasonography. Int J CARS 19, 449–457 (2024). https://doi.org/10.1007/s11548-023-03019-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-023-03019-5