Skip to main content

Advertisement

SpringerLink
Log in

A novel noncontact detection method of surgeon’s operation for a master-slave endovascular surgery robot

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Master-slave endovascular interventional surgery (EIS) robots have brought revolutionary advantages to traditional EIS, such as avoiding X-ray radiation to the surgeon and improving surgical precision and safety. However, the master controllers of most of the current EIS robots always lead to bad human-machine interaction, because of the difference in nature between the rigid operating handle and the flexible medical catheter used in EIS. In this paper, a noncontact detection method is proposed, and a novel master controller is developed to realize real-time detection of surgeon’s operation without interference to the surgeon. A medical catheter is used as the operating handle. It is enabled by using FAST corner detection algorithm and optical flow algorithm to track the corner points of the continuous markers on a designed sensing pipe. A mathematical model is established to calculate the axial and rotational motion of the sensing pipe according to the moving distance of the corner points in image coordinates. A master-slave EIS robot system is constructed by integrating the proposed master controller and a developed slave robot. Surgical task performance evaluation in an endovascular evaluator (EVE) is conducted, and the results indicate that the proposed detection method breaks through the axial measuring range limitation of the previous marker-based detection method. In addition, the rotational detection error is reduced by 92.5% compared with the previous laser-based detection method. The results also demonstrate the capability and efficiency of the proposed master controller to control the slave robot for surgical task implementation.

A novel master controller is developed to realize real-time noncontact detection of surgeon’s operation without interference to the surgeon. The master controller is used to remotely control the slave robot to implement certain surgical tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Roth G, Johnson C, Abajobir A (2017) Global regional and national burden of cardiovascular diseases for 10 causes 1990 to 2015. J Am Coll Cardiol 70(1):1–25

    Article  Google Scholar 

  2. Dankelman J, Dobbelsteen J, Breedveld P (2010) Current technology on minimally invasive surgery and interventional techniques. Paper presented at the 2010 IEEE international conference on instrumentation control and automation 12-15

  3. Oonsiri S, Jumpangern C, Sanghangthum T (2007) Radiation dose to medical staff in interventional radiology. J Med Assoc Thail 90(4):823–828

    Google Scholar 

  4. Kamusella P, Scheer F, Ludtke CW (2017) Interventional angiography: radiation protection for the examiner by using lead-free gloves. J Clin Diagn Res 11(7):C26–C29

    Google Scholar 

  5. Naito N (2005) Occupational back pain in the interventional radiologist. J Vasc Interv Radiol 16(2):312–314

    Article  Google Scholar 

  6. Kim Y, Cheng S, Diakite M, Gullapalli R, Simard J, Jaydev JD (2017) Toward the development of a flexible mesoscale MRI-compatible neurosurgical continuum robot. IEEE Trans Robot 33(6):1386–1397

    Article  Google Scholar 

  7. Vandini C, Bergeles B, Glocker P, Giataganas YG (2017) Unified tracking and shape estimation for concentric tube robots. IEEE Trans Robot 33(4):901–915

    Article  Google Scholar 

  8. Zhang L, Guo S, Yu H, Song Y (2017) Performance evaluation of a strain-gauge force sensor for a haptic robot-assisted catheter operating system. Microsyst Technol 23(10):5041–5050

    Article  Google Scholar 

  9. Guo J, Guo S, Yu Y (2016) Design and characteristics evaluation of a novel teleoperated robotic catheterization system with force feedback for vascular interventional surgery. Biomed Microdevices 18:76. https://doi.org/10.1007/s10544-016-0100-0

    Article  PubMed  Google Scholar 

  10. Guo J, Guo S, Tamiya T, Hirata H, Ishihara H (2014) A virtual reality-based method of decreasing transmission time of visual feedback for a tele-operative robotic catheter operating system. Int J Med Robot Comput Assisted Surg 12(1):32–45

    Article  Google Scholar 

  11. Vitiello V, Kwok K, Yang G (2012) Medical robotics: minimally invasive surgery (1st ed). Woodhead Publishing Series in Biomaterials

  12. Rao S (2016) Endovascular robotic catheters: an emerging transformative technology in the interventional radiology suite. J Radiol Nurs 35(3):211–217. https://doi.org/10.1016/j.jradnu.2016.04.022

    Article  Google Scholar 

  13. Antoniou G, Riga C, Mayer E, Cheshire N, Bicknell C (2011) Clinical applications of robotic technology in vascular and endovascular surgery. J Vasc Surg 53(2):493–499. https://doi.org/10.1016/j.jvs.2010.06.154

    Article  PubMed  Google Scholar 

  14. Granada J, Delgado J, Uribe M, Fernandez A, Blanco G, Leon M, Weisz G (2011) First-in-human evaluation of a novel robotic-assisted coronary angioplasty system. J Am Coll Cardiol Intv 4(4):460–465. https://doi.org/10.1016/j.jcin.2010.12.007

    Article  Google Scholar 

  15. Zhao Y, Guo S, Xiao N, Wang Y, Li Y, Jiang Y (2018) Operating force information on-line acquisition of a novel slave manipulator for vascular interventional surgery. Biomed Microdevices 20:33. https://doi.org/10.1007/s10544-018-0275-7

    Article  PubMed  Google Scholar 

  16. Bao X, Guo S, Xiao N, Li Y, Yang C, Shen R, Cui J, Jiang Y, Liu X, Liu K (2018) Operation evaluation in-human of a novel remote-controlled vascular interventional robot. Biomed Microdevices 20:34. https://doi.org/10.1007/s10544-018-0277-5

    Article  CAS  PubMed  Google Scholar 

  17. Bao X, Guo S, Xiao N (2018) Compensatory force measurement and multimodal force feedback for remote-controlled vascular interventional robot. Biomed Microdevices 20:74–11. https://doi.org/10.1007/s10544-018-0318-0

    Article  CAS  PubMed  Google Scholar 

  18. Guo S, Wang Y, Xiao N, Li Y, Jiang Y (2018) Study on real-time force feedback for a master–slave interventional surgical robotic system. Biomed Microdevices 20:37

    Article  Google Scholar 

  19. Yang C, Guo S, Bao X, Xiao N, Shi L, Li Y, Jiang Y (2019, 2019) A vascular interventional surgical robot based on surgeon's operating skills. Med Biol Eng Comput:1–12. https://doi.org/10.1007/s11517-019-02016-8

  20. Zhao Y, Guo S, Wang Y, Cui J, Ma Y, Zeng Y, Liu X, Jiang Y, Li Y, Shi L, Xiao N (2019) A CNNs-based prototype method of unstructured surgical state perception and navigation for an endovascular surgery robot. Med Biol Eng Comput 57:1875–1887. https://doi.org/10.1007/s11517-019-02002-0

    Article  PubMed  Google Scholar 

  21. Lu W, Xu W, Zhang J, Liu D, Wang D, Jia P, Li Z, Wang T, Zhang D, Tian Z, Zeng Y (2011) Application study of medical robots in vascular intervention. Int J Med Robot Comput Assisted Surg 13(3). https://doi.org/10.1002/rcs.406

  22. Thakur Y, Cakiroglu JH, Holdsworth DW, Drangova M (2007) A device for real-time measurement of catheter-motion and input to a catheter navigation system. SPIE. https://doi.org/10.1117/12.709556

  23. Zhou C, Xie L, Shen X, Luo M, Wu Z, Gu L (2015) Cardiovascular interventional surgery virtual training platform and its preliminary evaluation. Int J Med Robot Comput Assisted Surg 11(3):375–387. https://doi.org/10.1002/rcs.1627

    Article  Google Scholar 

  24. Zhang L, Guo S, Yu H, Song Y, Tamiya T, Hirata H, Ishihara H (2018) Design and performance evaluation of collision protection-based safety operation for a haptic robot-assisted catheter operating system. Biomed Microdevices 20(2):22. https://doi.org/10.1007/s10544-018-0266-8

    Article  PubMed  Google Scholar 

  25. Song Y, Guo S, Yin X, Zhang L, Hirata H, Ishihara H, Tamiya T (2018) Performance evaluation of a robot-assisted catheter operating system with haptic feedback. Biomed Microdevices 25:50–16. https://doi.org/10.1007/s10544-018-0294-4

    Article  Google Scholar 

  26. Tercero C, Kodama H, Shi C, Ooe K, Ikeda S, Fukuda T, Arai F, Negoro M, Kwon G, Najdovski Z (2013) Technical skills measurement based on a cyber-physical system for endovascular surgery simulation. Int J Med Robot Comput Assisted Surg 9(3):e25–e33. https://doi.org/10.1002/rcs.1467

    Article  Google Scholar 

  27. Yin X, Guo S, Xiao N, Tamiya T, Hirata H, Ishihara H (2016) Safety operation consciousness realization of a MR fluids-based novel haptic interface for teleoperated catheter minimally invasive neurosurgery. IEEE/ASME Trans Mechatron 21(2):1043–1054. https://doi.org/10.1109/TMECH.2015.2489219

    Article  Google Scholar 

  28. Wang Y, Guo S, Tamiya T, Hirata H, Ishihara H, Yin X (2016) A virtual reality simulator and force sensation combined catheter operation training system and its preliminary evaluation. Int J Med Robot Comp 13:e1769. https://doi.org/10.1002/rcs.1769

    Article  Google Scholar 

  29. Tercero C, Ikeda S, Uchiyama T, Fukuda T, Arai F, Okada Y, Ono Y, Hattori R, Yamamoto T, Negoro M, Takahashi I (2007) Autonomous catheter insertion system using magnetic motion capture sensor for endovascular surgery. Int J Med Robot Comput Assisted Surg 3(1):52–58. https://doi.org/10.1002/rcs.116

    Article  CAS  Google Scholar 

  30. Rafiii H, Liu J, Lee S, Bicknell C, Yang G (2013) Learning-based modeling of endovascular navigation for collaborative robotic catheterization. International Conference on Medical Image Computing and Computer-assisted Intervention 369-377. https://doi.org/10.1007/978-3-642-40763-5_46

  31. Chi W, Liu J, Rafii H, Riga C, Bicknell C, Yang G (2018) Learning-based endovascular navigation through the use of non-rigid registration for collaborative robotic catheterization. Int J Comput Ass Rad 13(6):855–864. https://doi.org/10.1007/s11548-018-1743-5

    Article  Google Scholar 

  32. Koolwal AB, Barbagli F, Carlson C, Liang D (2010) An ultrasound-based localization algorithm for catheter ablation guidance in the left atrium. Int J Robot Res 29(6):643–665

    Article  Google Scholar 

  33. Chen F, Liu J, Liao H (2016) 3D catheter shape determination for endovascular navigation using a two-step particle filter and ultrasound scanning. IEEE Trans Med Imaging 36(3):685–695

    Article  Google Scholar 

  34. Guo J, Guo S, Li M, Tamiya T (2018) A marker-based contactless catheter-sensing method to detect surgeons' operations for catheterization training systems. Biomed Microdevices 20(3):76. https://doi.org/10.1007/s10544-018-0321-5

    Article  CAS  PubMed  Google Scholar 

  35. Wang Y, Guo S, Yin X (2015) Laser mouse-based master-slave catheter operating system for minimally invasive surgery. Proceedings of 2015 IEEE international conference on mechatronics and automation, Beijing, China

  36. Jayarathne UL, Luo X, Chen ECS, Peters TM (2015) Simultaneous Estimation of Feature Correspondence and Stereo Object Pose with Application to Ultrasound Augmented Robotic Laparoscopy the Workshop on Augmented Environments for Computer-Assisted Interventions 9365

  37. Bosch J, Gracias N, Ridao P, Ribas D (2015) Omnidirectional underwater camera design and calibration. Sensors 15(3):33–65

    Article  Google Scholar 

  38. Kannala J, Brandt S (2006) A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans Pattern Anal Mach Intell 28:1335–1340

    Article  Google Scholar 

  39. Huang B, Ye M, Hu Y et al (2018) A Multi-Robot Cooperation Framework for Sewing Personalized Stent Grafts. IEEE Trans Ind Inf 14(4):1776–1785

    Article  Google Scholar 

Download references

Funding

This research is partly supported by National High-tech R&D Program (863 Program) of China (No. 2015AA043202) and National Key Research and Development Program of China (2017YFB1304401).

Author information

Author notes

  1. Yan Zhao and Huiming Xing contributed equally to this work.

Authors and Affiliations

  1. School of Mechanical Engineering, Hebei University of Technology, No.5340 Rd. Xiping, Beichen District, Tianjin, 300401, China

    Yan Zhao

  2. Key Laboratory of Convergence Biomedical Engineering System and Healthcare Technology, The Ministry of Industry and Information Technology, School of Life Science, Beijing Institute of Technology, No.5, Zhongguancun South Street, Haidian District, Beijing, 100081, China

    Yan Zhao, Huiming Xing, Shuxiang Guo, Yuxin Wang, Jinxin Cui, Youchun Ma & Yu Liu

  3. Faculty of Engineering, Kagawa University, 2217-20 Hayashi-cho, Takamatsu, Kagawa, 760-8521, Japan

    Shuxiang Guo

  4. Department of Interventional Neuroradiology, Beijing Neurosurgical Institute and Beijing Tiantan Hospital, Capital Medical University, Beijing, 100050, China

    Xinke Liu, Junqiang Feng & Youxiang Li

  5. Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, 94143, USA

    Xinke Liu

Authors
  1. Yan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiming Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuxiang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuxin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinxin Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Youchun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xinke Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Junqiang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Youxiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuxiang Guo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Y., Xing, H., Guo, S. et al. A novel noncontact detection method of surgeon’s operation for a master-slave endovascular surgery robot. Med Biol Eng Comput 58, 871–885 (2020). https://doi.org/10.1007/s11517-020-02143-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02143-7

Keywords

Search

Navigation