Skip to main content
SpringerLink
Log in

A Marker-Free 2D Image-Guided Method for Robot-Assisted Fracture Reduction Surgery

  • Regular Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

The closed fracture reduction is the first step of the minimally invasive treatment for long bone fractures, while it remains a challenging task due to heavy physical burden, excessive radiation exposure and poor accuracy. The robot-assisted fracture reduction (RAFR) shows potential to address these problems, while most existing methods rely on 3D images, optical tracking equipment and invasive robot-bone fixations. Such methods usually have a tedious surgery workflow, complicated hardware requirements, and iatrogenic injury risks, which prevent them from clinical applications. In this paper, we proposed a novel marker-free RAFR method using the image-based visual servoing technique. Firstly, an image feature vector based on 2D intraoperative images are defined for path planning and feedback control, and an online estimated Jacobian matrix is used to map the image variation to robot motion. Secondly, a geometry-based method is developed for Jacobian initialization without exploratory movements. Thirdly, a novel state-dependent weighted least square (SD-WLS) algorithm for online Jacobian estimation is proposed according to the characteristic of fracture reduction tasks. In the proposed RAFR framework, requirements for 3D images, optical markers and rigid robot-bone fixations are avoided. Experiments on fracture models with highly flexible robot-bone fixations are performed. All the tested fracture cases were well reduced within acceptable steps, which shows the feasibility of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Data Availability

All data generated in this study are available from the corresponding author on reasonable request.

References

  1. Russell, T.A.: Intramedullary nailing: evolutions of femoral intramedullary nailing: first to fourth generations. J. Orthop. Trauma. 25(Suppl 3), S135–S138 (2011). https://doi.org/10.1097/BOT.0b013e318237b2eb

    Article  Google Scholar 

  2. Winquist, R.A., Hansen Jr., S.T., Clawson, D.K.: Closed intramedullary nailing of femoral fractures. A report of five hundred and twenty cases. J Bone Joint Surg-Am. 66(4), 529–539 (1984)

    Article  Google Scholar 

  3. Dosch, J.C., Dupuis, M.G.: Time of radiation exposure during intramedullary nailing procedures. In: Kempf, I., Leung, K.S., Grosse, A., Haarman, H.J.T.M., Seidel, H., Taglang, G. (eds.) Practice of Intramedullary Locked Nails, pp. 61–65. Springer (2002). https://doi.org/10.1007/978-3-642-56330-0_7

    Chapter  Google Scholar 

  4. Zhu, Q., Sun, X., Wang, X., Wu, Q., Liang, B.: Development of intraoperative noninvasive force measuring system during femoral fracture reduction. In: Proc. IEEE Int. Symp. Medical Measurements and Applications, 2015, 7–9 May 2015 2015. Proc. IEEE Int. Symp. Medical Measurements and Applications. pp. 335–339 (2015) doi:https://doi.org/10.1109/MeMeA.2015.7145223

  5. Westphal, R., Winkelbach, S., Wahl, F., Gosling, T., Oszwald, M., Hufner, T., Krettek, C.: Robot-assisted long bone fracture reduction. Int. J. Robot. Res. 28(10), 1259–1278 (2009). https://doi.org/10.1177/0278364909101189

    Article  Google Scholar 

  6. Oszwald, M., Westphal, R., Bredow, J., Calafi, A., Hufner, T., Wahl, F., Krettek, C., Gosling, T.: Robot-assisted fracture reduction using three-dimensional intraoperative fracture visualization: an experimental study on human cadaver femora. J. Orthop. Res. 28(9), 1240–1244 (2010). https://doi.org/10.1002/jor.21118

    Article  Google Scholar 

  7. Du, H., Hu, L., Li, C., Wang, T., Zhao, L., Li, Y., Mao, Z., Liu, D., Zhang, L., He, C.: Advancing computer-assisted orthopaedic surgery using a hexapod device for closed diaphyseal fracture reduction. Int. J. Med. Robot. Comput. Assist. Surg. 11(3), 348–359 (2014)

    Article  Google Scholar 

  8. Du, H., Hu, L., Li, C., He, C., Zhang, L., Tang, P.: Preoperative trajectory planning for closed reduction of long-bone diaphyseal fracture using a computer-assisted reduction system. Int. J. Med. Robot. Comput. Assist. Surg. 11(1), 58–66 (2015). https://doi.org/10.1002/rcs.1573

    Article  Google Scholar 

  9. Kim, W.Y., Ko, S.Y.: Hands-on robot-assisted fracture reduction system guided by a linear guidance constraints controller using a pre-operatively planned goal pose. Int. J. Med. Robot. Comput. Assist. Surg. 15(2), e1967 (2019). https://doi.org/10.1002/rcs.1967

    Article  Google Scholar 

  10. Kim, W.Y., Ko, S.Y., Park, J.O., Park, S.: 6-DOF force feedback control of robot-assisted bone fracture reduction system using double F/T sensors and adjustable admittances to protect bones against damage. Mechatronics. 35, 136–147 (2016). https://doi.org/10.1016/j.mechatronics.2016.02.005

    Article  Google Scholar 

  11. Bai, L., Yang, J., Chen, X., Sun, Y., Li, X.: Medical robotics in bone fracture reduction surgery: a review. Sensors-Basel. 19(16), 3593 (2019)

    Article  Google Scholar 

  12. Hoeckelmann, M., Rudas, I.J., Fiorini, P., Kirchner, F., Haidegger, T.: Current capabilities and development potential in surgical robotics. Int. J. Adv. Robot. Syst. 12 (61) (2015) doi: https://doi.org/10.5772/60133

  13. Zhao, J.-X., Li, C., Ren, H., Hao, M., Zhang, L.-C., Tang, P.-F.: Evolution and current applications of robot-assisted fracture reduction: a comprehensive review. Ann. Biomed. Eng. 48(1), 203–224 (2019). https://doi.org/10.1007/s10439-019-02332-y

    Article  Google Scholar 

  14. Wang, L., Wang, T., Tang, P., Hu, L., Liu, W., Han, Z., Hao, M., Liu, H., Wang, K., Zhao, Y., Guo, N., Cao, Y., Li, C.: A new hand-eye calibration approach for fracture reduction robot. Comput. Assist. Surg. 22, 113–119 (2017). https://doi.org/10.1080/24699322.2017.1379254

    Article  Google Scholar 

  15. Leloup, T., Kazzi, W.E., Schuind, F., Warzée, N.: Conception of a navigation system controlling diaphyseal fracture reduction treated with external fixation. Int. J. Med. Robot Comput. Assist. Surg. 5(1), 99–109 (2010)

    Article  Google Scholar 

  16. Dagnino, G., Georgilas, I., Köhler, P., Morad, S., Atkins, R., Dogramadzi, S.: Navigation system for robot-assisted intra-articular lower-limb fracture surgery. Int. J. Comput. Assist. Radiol. Surg. 11(10), 1831–1843 (2016). https://doi.org/10.1007/s11548-016-1418-z

    Article  Google Scholar 

  17. Dagnino, G., Georgilas, I., Morad, S., Gibbons, P., Tarassoli, P., Atkins, R., Dogramadzi, S.: Image-guided surgical robotic system for percutaneous reduction of joint fractures. Ann. Biomed. Eng. 45(11), 2648–2662 (2017)

    Article  Google Scholar 

  18. Georgilas, I., Dagnino, G., Tarassoli, P., Atkins, R., Dogramadzi, S.: Robot-assisted fracture surgery: surgical requirements and system design. Ann. Biomed. Eng. 46(10), 1–13 (2018). https://doi.org/10.1007/s10439-018-2005-y

    Article  Google Scholar 

  19. Zhu, S., Chen, Y., Chen, Y., Sun, J., Zhao, Z., Hu, C., Zheng, G. A. Noninvasive Calibration-Free and Model-Free Surgical Robot for Automatic Fracture Reduction. In: Proc. Int. Conf. Intelligent Robotics and Applications, 2019, 2019. Proc. Int. Conf. Intelligent Robotics and Applications. Springer, pp 285–296 (2019)

  20. Hao, M., Sun, Z., Fujii, M., Song, W. Uncalibrated eye-in-hand visual servoing using recursive least squares. In: Proc. IEEE Int. Conf. Systems, Man and Cybernetics, 2007, 2007. Proc. IEEE Int. Conf. Systems, Man and Cybernetics (2007). IEEE, pp 64–69. doi:https://doi.org/10.1109/WCICA.2008.4593061

  21. Hao, M., Deuflhard, P., Sun, Z., Fujii, M.: Model-free uncalibrated visual Servoing using recursive least squares. J. Comput. 3(11), (2008)

  22. Hao, M., Sun, Z.Q.: A universal state-space approach to uncalibrated model-free visual Servoing. IEEE-ASME Trans. Mechatron. 17(5), 833–846 (2012). https://doi.org/10.1109/Tmech.2011.2131149

    Article  Google Scholar 

  23. Hosoda K, Asada M Versatile visual servoing without knowledge of true jacobian. In: Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 1994, 1994. Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems. pp. 186–193 (1994)

  24. Chaumette, F., Hutchinson, S.: Visual servo control. I. Basic approaches. IEEE Robot. Autom. Magazine. 13(4), 82–90 (2006)

    Article  Google Scholar 

  25. Chaumette, F., Hutchinson, S.: Visual servo control. II. Advanced approaches. IEEE Robot. Autom. Magazine. 14(1), 109–118 (2007)

    Article  Google Scholar 

  26. Sutanto, H., Sharma, R., Varma, V.: The role of exploratory movement in visual servoing without calibration. Robot. Auton. Syst. 23(3), 153–169 (1998). https://doi.org/10.1016/S0921-8890(97)00052-3

    Article  Google Scholar 

  27. Bonkovic, M., Hace, A., Jezernik, K.: Population-based uncalibrated visual servoing. IEEE-ASME Trans. Mechatron. 13(3), 393–397 (2008). https://doi.org/10.1109/Tmech.2008.924135

    Article  Google Scholar 

  28. Piepmeier, J.A., McMurray, G.V., Lipkin, H.: Uncalibrated dynamic visual servoing. IEEE Trans. Robot. Autom. 20(1), 143–147 (2004). https://doi.org/10.1109/Tra.2003.820923

    Article  Google Scholar 

  29. Piepmeier, J.A., Gumpert, B.A., Lipkin, H.: Uncalibrated Eye-in-Hand Visual Servoing. In: Proc. IEEE Int. Conf. Robotics and Automation, 2002, 2002. Proc. IEEE Int. Conf. Robotics and Automation. 561, 568–573 (2002)

    Google Scholar 

  30. Qian JA, Su JB Online estimation of image Jacobian matrix by Kalman-Bucy filter for uncalibrated stereo vision feedback. In: Proc. IEEE Int. Conf. Robotics and Automation, 2002, 2002. Proc. IEEE Int. Conf. Robotics and Automation, 2002. pp. 562–567

  31. Burenius, M., Sullivan, J., Carlsson, S.: Motion capture from dynamic orthographic cameras. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 6–13 Nov. 2011 (2011). pp 1634–1641. doi:https://doi.org/10.1109/ICCVW.2011.6130445

  32. Broyden, C.G.: A Class of Methods for Solving Nonlinear Simultaneous Equations. Math Comput. 19(92), 557 (1965)

    Article  MathSciNet  Google Scholar 

  33. Navarro-Alarcon, D., Liu, Y.H., Romero, J.G.: Model-free visually Servoed deformation control of elastic objects by robot manipulators. IEEE T Robot. 29(6), 1457–1468 (2013). https://doi.org/10.1109/Tro.2013.2275651

    Article  Google Scholar 

  34. Qingjie, Z., Liqun, Z., Yunjiao, C.: Online estimation technique for Jacobian matrix in robot visual servo systems. In: 2008 3rd IEEE Conference on Industrial Electronics and Applications, 3–5 June 2008 2008. pp 1270–1275. doi:https://doi.org/10.1109/ICIEA.2008.4582722

  35. Xiao, G., Bonmati, E., Thompson, S., Evans, J., Hipwell, J., Nikitichev, D., Gurusamy, K., Ourselin, S., Hawkes, D.J., Davidson, B., Clarkson, M.J.: Electromagnetic tracking in image-guided laparoscopic surgery: comparison with optical tracking and feasibility study of a combined laparoscope and laparoscopic ultrasound system. Med. Phys. 45(11), 5094–5104 (2018). https://doi.org/10.1002/mp.13210

    Article  Google Scholar 

  36. Kim, J.-W., Oh, C.-W., Oh, J.-K., Park, I.-H., Kyung, H.-S., Park, K.-H., Yoon, S.-D., Kim, S.-M.: Malalignment after minimally invasive plate osteosynthesis in distal femoral fractures. Injury. 48(3), 751–757 (2017). https://doi.org/10.1016/j.injury.2017.01.019

    Article  Google Scholar 

Download references

Funding

This work is sponsored by Tsinghua University and Cyrus Tang Foundation.

Author information

Author notes

  1. Shijie Zhu and Zhe Zhao contributed equally to this work.

Authors and Affiliations

  1. School of Aerospace Engineering, Tsinghua University, Beijing, China

    Shijie Zhu, Yu Chen, Bicong Zhang, Yitong Chen & Gangtie Zheng

  2. Department of Orthopaedics, Beijing Tsinghua Changgung Hospital, Beijing, China

    Zhe Zhao, Jiuzheng Deng, Jianjin Zhu, Dawei He & Yongwei Pan

Authors
  1. Shijie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhe Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiuzheng Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bicong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianjin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yitong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Dawei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yongwei Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Gangtie Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Shijie Zhu developed the control method, designed the experiment system, analyzed the data and wrote the paper. Zhe Zhao designed and prepared the fracture model, performed the experiment and revised the paper. Yu Chen, Bicong Zhang and Yitong Chen built the experiment system. Jiuzheng Deng, Jianjin Zhu and Dawei He prepared the fracture model, interpreted and evaluated the data. Gangtie Zheng and Yongwei Pan initiated and supervised this study and revised the paper.

Corresponding authors

Correspondence to Yongwei Pan or Gangtie Zheng.

Ethics declarations

Ethical Approval

Not applicable. This research doesn’t involve human or animal subjects, their data or biological material.

Consent to Participate

Not applicable. This research doesn’t involve human subjects.

Consent to Publish

Not applicable. This research doesn’t contain patient data.

Conflict of Interest

The authors have a patent application (US20190125461A1) based on the method described in this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, S., Zhao, Z., Chen, Y. et al. A Marker-Free 2D Image-Guided Method for Robot-Assisted Fracture Reduction Surgery. J Intell Robot Syst 103, 67 (2021). https://doi.org/10.1007/s10846-021-01453-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01453-8

Keywords

Search

Navigation