Skip to main content
Log in

A VS ultrasound diagnostic system with kidney image evaluation functions

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

An inevitable feature of ultrasound-based diagnoses is that the quality of the ultrasound images produced depends directly on the skill of the physician operating the probe. This is because physicians have to constantly adjust the probe position to obtain a cross section of the target organ, which is constantly shifting due to patient respiratory motions. Therefore, we developed an ultrasound diagnostic robot that works in cooperation with a visual servo system based on deep learning that will help alleviate the burdens imposed on physicians.

Methods

Our newly developed robotic ultrasound diagnostic system consists of three robots: an organ tracking robot (OTR), a robotic bed, and a robotic supporting arm. Additionally, we used different image processing methods (YOLOv5s and BiSeNet V2) to detect the target kidney location, as well as to evaluate the appropriateness of the obtained ultrasound images (ResNet 50). Ultimately, the image processing results are transmitted to the OTR for use as motion commands.

Results

In our experiments, the highest effective tracking rate (0.749) was obtained by YOLOv5s with Kalman filtering, while the effective tracking rate was improved by about 37% in comparison with cases without such filtering. Additionally, the appropriateness probability of the ultrasound images obtained during the tracking process was also the highest and most stable. The second highest tracking efficiency value (0.694) was obtained by BiSeNet V2 with Kalman filtering and was a 75% improvement over the case without such filtering.

Conclusion

While the most efficient tracking achieved is based on the combination of YOLOv5s and Kalman filtering, the combination of BiSeNet V2 and Kalman filtering was capable of detecting the kidney center of gravity closer to the kidney’s actual motion state. Furthermore, this model could also measure the cross-sectional area, maximum diameter, and other detailed information of the target kidney, which meant it is more practical for use in actual diagnoses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Boudet S, Gariepy J, and Mansour S. (1997) An integrated robotics and medical control device to quantify atheromatous plaques: experiments on the arteries of a patient. In: Proceedings of the 1997 IEEE/RSJ international conference on intelligent robot and systems. innovative robotics for real-world applications. IROS'97 3: pp 1533–1538 IEEE

  2. Berg S, Torp H, Martens D, Steen E, Samstad S, Høivik I, Olstad B (1999) Dynamic three-dimensional freehand echocardiography using raw digital ultrasound data. Ultrasound Med Biol 25(5):745–753

    Article  CAS  Google Scholar 

  3. Davies SC, Hill AL, Holmes RB, Halliwell M, Jackson PC (1994) Ultrasound quantitation of respiratory organ motion in the upper abdomen. Br J Radiol 67(803):1096–1102

    Article  CAS  Google Scholar 

  4. Schwartz LH, Richaud J, Buffat L, Touboul E, Schlienger M (1994) Kidney mobility during respiration. Radiother Oncol 32(1):84–86

    Article  CAS  Google Scholar 

  5. Mebarki R, Krupa A, Chaumette F (2010) 2-d ultrasound probe complete guidance by visual servoing using image moments. IEEE Trans Rob 26(2):296–306

    Article  Google Scholar 

  6. Zettinig O, Fuerst B, Kojcev R, Esposito M, Salehi M, Wein W, and Navab N (2016) Toward real-time 3D ultrasound registration-based visual servoing for interventional navigation. In: 2016 IEEE international conference on robotics and automation (ICRA) pp 945–950 IEEE

  7. Novotny PM, Stoll JA, Dupont PE, and Howe RD (2007) Real-time visual servoing of a robot using three-dimensional ultrasound. In: Proceedings 2007 IEEE international conference on robotics and automation pp 2655–2660 IEEE

  8. Li Y, Yin K, Liang J, Wang C, and Yin G (2020) A multi-task joint framework for real-time person search. arXiv preprint arXiv:2012.06418

  9. Yu C, Gao C, Wang J, Yu G, Shen C, and Sang N (2020) Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation. arXiv preprint arXiv:2004.02147

  10. He K, Zhang X, Ren S, and Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp 770–778

  11. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99

    Google Scholar 

  12. Xinzheng L, Wei J, Gang L, and Caoqian Y (2019) YOLO V2 network with asymmetric convolution kernel for lung nodule detection of CT image, Chin J Biomed Eng

  13. Guo D, Zhang G, Peng H, Yuan J, Paul P, Fu K, Zhu M (2021) Segmentation and measurements of carotid intima-media thickness in ultrasound images using the improved convolutional neural network and support vector machine. J Med Imaging Health Inf 11(1):15–24

    Article  Google Scholar 

  14. Hu Y, Guo L, Lei B, Mao M, Jin Z, Elazab A., Wang T (2019) Fully automatic pediatric echocardiography segmentation using deep convolutional networks based on BiSeNet. In: 2019 41st annual international conference of the ieee engineering in medicine and biology society (EMBC). IEEE, pp 6561–6564

  15. Song R, Tipirneni A, Johnson P, Loeffler RB, Hillenbrand CM (2011) Evaluation of respiratory liver and kidney movements for MRI navigator gating. J Magnetic Res Imaging 33(1):143–148

    Article  Google Scholar 

  16. Xie, S., & Tu, Z. (2015). Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision (pp. 1395–1403).

  17. Welch G,and Bishop G (1995) An introduction to the Kalman filter

  18. Koizumi N, Itagaki Y, Lee D, Tsukihara H, Azuma T, Nomiya A, Yoshinaka K, Sugita N, Homma Y, Matsumoto Y, Mitsuishi M (2013) Construction of kidney phantom model with acoustic shadow by rib bones and respiratory organ motion. J Japan Soc Comput Aid Surg J JSCAS 15(2):228–229

    Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the financial support by the Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Numbers JP20H02113.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Norihiro Koizumi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Human and animal rights

The authors declare that none of the experiments performed in this study involved human participants or animals.

Informed consent

This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Koizumi, N., Nishiyama, Y. et al. A VS ultrasound diagnostic system with kidney image evaluation functions. Int J CARS 18, 227–246 (2023). https://doi.org/10.1007/s11548-022-02759-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02759-0

Keywords

Navigation