Skip to main content
SpringerLink
Log in

Robust path planning for flexible needle insertion using Markov decision processes

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Flexible needle has the potential to accurately navigate to a treatment region in the least invasive manner. We propose a new planning method using Markov decision processes (MDPs) for flexible needle navigation that can perform robust path planning and steering under the circumstance of complex tissue–needle interactions.

Methods

This method enhances the robustness of flexible needle steering from three different perspectives. First, the method considers the problem caused by soft tissue deformation. The method then resolves the common needle penetration failure caused by patterns of targets, while the last solution addresses the uncertainty issues in flexible needle motion due to complex and unpredictable tissue–needle interaction.

Results

Computer simulation and phantom experimental results show that the proposed method can perform robust planning and generate a secure control policy for flexible needle steering. Compared with a traditional method using MDPs, the proposed method achieves higher accuracy and probability of success in avoiding obstacles under complicated and uncertain tissue–needle interactions. Future work will involve experiment with biological tissue in vivo.

Conclusion

The proposed robust path planning method can securely steer flexible needle within soft phantom tissues and achieve high adaptability in computer simulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Fuchs KH (2002) Minimally invasive surgery. Endoscopy 34(02):154–159

    Article  PubMed  CAS  Google Scholar 

  2. Rosen M, Ponsky J (2001) Minimally invasive surgery. Endoscopy 33(04):358–366

    Article  PubMed  CAS  Google Scholar 

  3. Abolhassani N, Patel R, Moallem M (2007) Needle insertion into soft tissue: a survey. Med Eng Phys 29(4):413–431

    Article  PubMed  Google Scholar 

  4. Doi K (2007) Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 31(4):198–211

    Article  PubMed  PubMed Central  Google Scholar 

  5. Nath S, Chen Z, Yue N, Trumpore S, Peschel R (2000) Dosimetric effects of needle divergence in prostate seed implant using 125i and 103pd radioactive seeds. Med Phys 27(5):1058–1066

    Article  PubMed  CAS  Google Scholar 

  6. Carr JJ, Hemler PF, Halford PW, Freimanis RI, Choplin RH, Chen MYM (2001) Stereotactic localization of breast lesions: how it works and methods to improve accuracy. Radiographics 21(2):463–473

    Article  PubMed  CAS  Google Scholar 

  7. Taschereau R, Pouliot J, Roy J, Tremblay D (2000) Seed misplacement and stabilizing needles in transperineal permanent prostate implants. Radiother Oncol 55(1):59–63

    Article  PubMed  CAS  Google Scholar 

  8. Narayana V, Roberson PL, Winfield RJ, Kessler ML, McLaughlin PW (1996) Optimal placement of radioisotopes for permanent prostate implants. Radiology 199(2):457–460

    Article  PubMed  CAS  Google Scholar 

  9. Alterovitz R, Branicky M, Goldberg K (2008) Motion planning under uncertainty for image-guided medical needle steering. Int J Robot Res 27(11–12):1361–1374

    Article  Google Scholar 

  10. Donaldson MS, Corrigan JM, Kohn LT (2000) To err is human: building a safer health system, vol 6. National Academies Press, Washington

    Google Scholar 

  11. Robert J, Webster III, Kim JS, Cowan NJ, Chirikjian GS, Okamura AM (2006) Nonholonomic modeling of needle steering. Int J Robot Res 25(5–6):509–525

    Google Scholar 

  12. Webster RJ, Memisevic J, Okamura AM (2005) Design considerations for robotic needle steering. In: Proceedings of the 2005 IEEE international conference on robotics and automation, ICRA 2005. IEEE, pp 3588–3594

  13. Leong F, Huang W-H, Chui C-K (2013) Modeling and analysis of coagulated liver tissue and its interaction with a scalpel blade. Med Biol Eng Comput 51(6):687–695

    Article  PubMed  Google Scholar 

  14. Fu YB, Chui CK (2014) Modelling and simulation of porcine liver tissue indentation using finite element method and uniaxial stress–strain data. J Biomech 47(10):2430–2435

    Article  PubMed  CAS  Google Scholar 

  15. Fu YB, Chui CK, Teo CL (2013) Liver tissue characterization from uniaxial stress-strain data using probabilistic and inverse finite element methods. J Mech Behav Biomed Mater 20:105–112

    Article  PubMed  CAS  Google Scholar 

  16. O’Reilly MS, Boehm T, Shing Y, Fukai N, Vasios G, Lane WS, Flynn E, Birkhead JR, Olsen BR, Folkman J (1997) Endostatin: an endogenous inhibitor of angiogenesis and tumor growth. Cell 88(2):277–285

    Article  PubMed  Google Scholar 

  17. Abayazid M, Vrooijink GJV, Patil S, Alterovitz R, Misra S (2014) Experimental evaluation of ultrasound-guided 3d needle steering in biological tissue. Int J Comput Assist Radiol Surg 9(6):931–939

    Article  PubMed  PubMed Central  Google Scholar 

  18. Vrooijink JG, Abayazid M, Patil S, Alterovitz R, Misra S (2014) Needle path planning and steering in a three-dimensional non-static environment using two-dimensional ultrasound images. Int J Robot Res 33(10):1361–1374

    Article  Google Scholar 

  19. Patil S, Burgner J, Webster RJ, Alterovitz R (2014) Needle steering in 3-d via rapid replanning. IEEE Trans Robot 30(4):853–864

    Article  PubMed  PubMed Central  Google Scholar 

  20. Alterovitz R, Lim A, Goldberg K, Chirikjian GS, Okamura AM (2005) Steering flexible needles under Markov motion uncertainty. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, (IROS 2005). IEEE, pp 1570–1575

  21. Alterovitz R, Siméon T, Goldberg KY (2007) The stochastic motion roadmap: a sampling framework for planning with Markov motion uncertainty. In: Robotics: science and systems, vol 3, pp 233–241

  22. Pengqian Y, Huan X (2016) Distributionally robust counterpart in Markov decision processes. IEEE Trans Autom Control 61(9):2538–2543

    Article  Google Scholar 

  23. DiMaio S, Salcudean SE (2003) Needle insertion modeling and simulation. IEEE Trans Robot Autom 19(5):864–875

    Article  Google Scholar 

  24. DiMaio PS, Salcudean SE (2005) Needle steering and motion planning in soft tissues. IEEE Trans Biomed Eng 52(6):965–974

    Article  PubMed  Google Scholar 

  25. Glozman D, Shoham M (2006) Flexible needle steering for percutaneous therapies. Computer Aided Surg 11(4):194–201

    Article  Google Scholar 

  26. Kallem V, Cowan NJ (2009) Image guidance of flexible tip-steerable needles. IEEE Trans Robot 25(1):191–196

    Article  PubMed  PubMed Central  Google Scholar 

  27. Shahriari N, Roesthuis RJ, van de Berg NJ, van den Dobbelsteen JJ, Misra S (2016) Steering an actuated-tip needle in biological tissue: fusing fbg-sensor data and ultrasound images. In: 2016 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4443–4449

  28. Suetens P (2017) Fundamentals of medical imaging. Cambridge University Press, Cambridge

    Book  Google Scholar 

  29. Thie PR (1983) Markov decision processes. Comap, Incorporated, Bedford

    Google Scholar 

  30. Bertsekas DP, Bertsekas DP, Bertsekas DP, Bertsekas DP (1995) Dynamic programming and optimal control, vol 1. Athena Scientific, Belmont

    Google Scholar 

  31. Mostafa A, Fouad A, Elfattah MA, Hassanien AE, Hefny H, Zhu SY, Schaefer G (2015) Ct liver segmentation using artificial bee colony optimisation. Proc Comput Sci 60:1622–1630

    Article  Google Scholar 

  32. Norajitra T, Meinzer H-P, Maier-Hein KH (2015) 3d statistical shape models incorporating 3d random forest regression voting for robust ct liver segmentation. In: Proceedings of SPIE, vol 9414, p 941406

  33. Duan B, Wen R, Chng C-B, Wang W, Liu P, Qin J, Peneyra JL, Chang Stephen K-Y, Heng P-A, Chui C-K (2015) Image-guided robotic system for radiofrequency ablation of large liver tumor with single incision. In: 2015 12th international conference on ubiquitous robots and ambient intelligence (URAI). IEEE, pp 284–289

  34. Yang L, Wen R, Qin J, Chui C-K, Lim K-B, Chang SK-Y (2010) A robotic system for overlapping radiofrequency ablation in large tumor treatment. IEEE/ASME Trans Mechatron 15(6):887–897

    Article  Google Scholar 

  35. Liu Y, Kerdok AE, Howe RD (2004) A nonlinear finite element model of soft tissue indentation. Lect Notes Comput Sci 3078:67–76

    Article  Google Scholar 

  36. Varadhan G, Manocha D (2006) Accurate Minkowski sum approximation of polyhedral models. Graph Models 68(4):343–355

    Article  Google Scholar 

  37. van de Berg NJ, van Gerwen DJ, Dankelman J, van den Dobbelsteen JJ (2015) Design choices in needle steeringa review. IEEE/ASME Trans Mechatron 20(5):2172–2183

    Article  Google Scholar 

  38. Tan X, Chng C-B, Duan B, Ho Y, Wen R, Chen X, Lim K-B, Chui C-K (2016) Design and implementation of a patient-specific cognitive engine for robotic needle insertion. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 000560–000565

  39. Tan X, Chng C-B, Duan B, Ho Y, Wen R, Chen X, Lim K-B, Chui C-K (2017) Cognitive engine for robot-assisted radio-frequency ablation system. Acta Polytech Hung 14(1):129–145

    Google Scholar 

Download references

Funding

The research and development of the prototype image-guide radio-frequency ablation surgical system was supported in parts by research grants from Singapore Agency of Science and Technology (A*Star) and Ministry of Education, Singapore, respectively.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, National University of Singapore, Singapore, 117575, Singapore

    Xiaoyu Tan, Kah-Bin Lim & Chee-Kong Chui

  2. Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore, 117576, Singapore

    Pengqian Yu

Authors
  1. Xiaoyu Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengqian Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kah-Bin Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chee-Kong Chui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoyu Tan.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human participants

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tan, X., Yu, P., Lim, KB. et al. Robust path planning for flexible needle insertion using Markov decision processes. Int J CARS 13, 1439–1451 (2018). https://doi.org/10.1007/s11548-018-1783-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1783-x

Keywords

Search

Navigation