Skip to main content

Advertisement

SpringerLink
Log in

Adaptive Fusion-Based Autonomous Laparoscope Control for Semi-Autonomous Surgery

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

The purpose of this paper is to develop an autonomous tracking algorithm based on adaptive fusion kinematics method, the autonomous laparoscope control algorithm and adaptive fusion kinematics method are proposed for semi-autonomous surgery, focus on solving the problems of autonomous laparoscope field of view control for surgical robot system. A novel autonomous tracking algorithm is proposed. To realize more robust tracking, an adaptive fusion kinematics method based on fuzzy logic is proposed, the method adaptive associates the kinematics information of surgical robot system and the laparoscope information. The proposed methods are implemented on the laparoscopic minimally invasive surgical robot system which is developed by our laboratory. Two experiments are carried out, the results indicate that the accurate autonomous field of view control is achieved with the addition of laparoscope information, laparoscopic motion frequency is reduced, the methods can avoid the laparoscope continuous motion and ensure the stability of field of view. The proposed methods improve the intelligence level of surgical robot system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Yang, G. Z., Cambias, J., Cleary, K., Daimler, E., Drake, J., Drake, P. E., Hata, N., Kazanzides, P., Martel, S., Patel, R. V., Santos, V. J., and Taylor, R. H., Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci. Robot. 2(4):eaam8638, 2017.

    Article  Google Scholar 

  2. Moustris, G. P., Hiridis, S. C., Deliparaschos, K. M., and Konstantinidis, K. M., Evolution of autonomous and semi-autonomous robotic surgical systems: A review of the literature. Int. J. Med. Robot. Comput. Assist. Surg. 7(4):375–392, 2011.

    Article  CAS  Google Scholar 

  3. Kassahun, Y., Yu, B., Tibebu, A. T., Stoyanov, D., Giannarou, S., Metzen, J. H., and Poorten, E. V., Surgical robotics beyond enhanced dexterity instrumentation: A survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int. J. Comput. Assist. Radiol. Surg. 11(4):553–568, 2016.

    Article  Google Scholar 

  4. Wijsman, P. J. M., Broeders, I. A. M. J., Brenkman, H. J., Szold, A., and Kaufman, Y., First experience with the autolap™ system: An image-based robotic camera steering device. Surg. Endosc. 32(5):1–7, 2017.

    Google Scholar 

  5. Voros, S., Haber, G. P., Menudet, J. F., Long, J. A., and Cinquin, P., ViKY robotic scope holder: Initial clinical experience and preliminary results using instrument tracking. IEEE-ASME Trans. Mechatron. 15(6):879–886, 2010.

  6. Azizian, M., Khoshnam, M., Najmaei, N., and Patel, R. V., Visual servoing in medical robotics: A survey. Part I: Endoscopic and direct vision imaging - techniques and applications. Int. J. Med. Robot. Comput. Assist. Surg. 10(3):263–274, 2014.

    Article  Google Scholar 

  7. Rivas-Blanco, I., Lopez-Casado, C., Perez-Del-Pulgar, C. J., Garcia-Vacas, F., Fraile, J. C., and Munoz, V. F., Smart cable-driven camera robotic assistant. IEEE T. Hum.-Mach. Syst. 48(2): 183-196, 2018.

    Article  Google Scholar 

  8. Weede, O., Monnich, H., Muller, B., and Worn, H. (2011) An intelligent and autonomous endoscopic guidance system for minimally invasive surgery. In: 2011 IEEE international conference on robotics and automation (ICRA). IEEE, pp 5762-5768

  9. Allan, M., Ourselin, S., Thompson, S., Hawkes, D. J., Kelly, J., and Stoyanov, D., Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans. Biomed. Eng. 60(4):1050–1058, 2013.

    Article  Google Scholar 

  10. Wang, Z., Zi, B., Ding, H., You, W., and Yu, L., Hybrid grey prediction model-based autotracking algorithm for the laparoscopic visual window of surgical robot. Mech. Mach. Theory 123:107–123, 2018.

    Article  Google Scholar 

  11. Yu, L., Wang, Z., Sun, L., Wang, W., Wang, L., and Du, Z., A new forecasting kinematic algorithm of automatic navigation for a laparoscopic minimally invasive surgical robotic system. Robotica 35(05):1192–1222, 2017.

    Article  Google Scholar 

  12. Kashyap, S. K., and Raol, J. R., Fuzzy logic applications in filtering and fusion for target tracking. Def. Sci. J. 58(1):120–135, 2008.

    Article  Google Scholar 

  13. Sun, S. L., and Deng, Z. L., Multi-sensor optimal information fusion Kalman filter. Automatica 40(6):1017–1023, 2004.

    Article  Google Scholar 

  14. Raol, J. R., Multi sensor data fusion with MATLAB. Boca Raton: CRC Press, Inc, 2009.

    Book  Google Scholar 

  15. Ficocelli M, Janabisharifi F (2001) Adaptive filtering for pose estimation in visual servoing. In: 2001 IEEE/RSJ international conference on Intelligent Robots & Systems (IROS). IEEE, pp 19-24

  16. Lippiello, V., Siciliano, B., and Villani, L., Adaptive extended Kalman filtering for visual motion estimation of 3D objects. Control. Eng. Pract. 15(1):123–134, 2007.

    Article  Google Scholar 

  17. Baek, Y. M., Tanaka, S., Harada, K., Sugita, N., Morita, A., Sora, S., and Mitsuishi, M., Robust visual tracking of robotic forceps under a microscope using kinematic data fusion. IEEE-ASME Trans. Mechatron. 19(1):278–288, 2014.

    Article  Google Scholar 

  18. Richard, S., Computer vision - algorithms and applications. London: Springer-Verlag, 2011.

    Google Scholar 

  19. Liu, H., Lai, X., and Wu, W., Time-optimal and jerk-continuous trajectory planning for robot manipulators with kinematic constraints. Robot. Comput. Integr. Manuf. 29(2):309–317, 2013.

    Article  Google Scholar 

  20. Xiao, Y., Du, Z., and Dong, W., Smooth and near time-optimal trajectory planning of industrial robots for online applications. Ind. Robot 39(2):169–177, 2012.

    Article  Google Scholar 

  21. Ai, Y., Pan, B., Fu, Y., and Wang, S., Design of a novel robotic system for minimally invasive surgery. Ind. Robot 44(3):288–298, 2017.

    Article  Google Scholar 

  22. Kaehler, A., and Bradski, G. R., Learning OpenCV 3. Sebastopol: O’Reilly Media, 2016.

    Google Scholar 

  23. Stockman, G. C., Computer vision. Upper Saddle River: Prentice Hall, 2001.

    Google Scholar 

Download references

Funding

The project is sponsored by National High Technology Research and Development Program (“863”Program) of China, Grant/Award Number: 2012AA041601. The project is also supported by State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Grant/Award Number: SKLRS201601C, and Heilongjiang Postdoctoral Scientific Research Foundation (LBH-Q17070, LBH-Q17068).

Author information

Authors and Affiliations

  1. State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China

    Yanwen Sun, Bo Pan, Shuizhong Zou & Yili Fu

Authors
  1. Yanwen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuizhong Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yili Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanwen Sun.

Ethics declarations

Conflict of Interest

Author Yanwen Sun declares that he has no conflict of interest. Author Bo Pan declares that he has no conflict of interest. Author Shuizhong Zou declares that he has no conflict of interest. Author Yili Fu declares that he has no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Y., Pan, B., Zou, S. et al. Adaptive Fusion-Based Autonomous Laparoscope Control for Semi-Autonomous Surgery. J Med Syst 44, 4 (2020). https://doi.org/10.1007/s10916-019-1460-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-019-1460-9

Keywords

Search

Navigation