Abstract
In needle insertion navigation, most researches focus on intraoperative images based navigation system that provides only visual feedback. Besides, few navigation systems are integrated with insertion robot. In this paper, we proposed a digital twin model based robot-assisted needle insertion navigation system with visual and force feedback. Our system can predict needle deflection, tissue deformation for visual feedback and interaction force for force feedback while insertion robot can help steering needle for accurate insertion. The proposed needle insertion navigation system integrates digital twin model and insertion-assisted robot. A digital twin model of target organ, which includes finite element model and visual model, can be generated based on preoperative CT image to predict needle deflection, tissue deformation and interaction force of planned needle path. Optic-based calibration method for our system is developed. A hybrid spring mapping method based on radial-basis function interpolation and spring-mass model is proposed as well for better visual feedback. The proposed navigation system can provide both visual feedback and force feedback in digital twin model for surgeons while robot can help steering needle to target position. Simulations and experiments are carried out for our navigation system and hybrid spring mapping method. Results show the calibrated system is accurate with 4mm targeting accuracy, which meets clinical accuracy requirements. Hybrid spring mapping method can update the visual model smoothly. Both force and visual feedback can be registered to the digital twin coordinate system, allowing for accurate and consistent feedback for navigation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ullrich, S., Grottke, O., Rossaint, R., Staat, M., Deserno, T.M., Kuhlen, T.: Virtual needle simulation with haptics for regional anaesthesia. IEEE Virtual Real. 52(7), 1–3 (2010)
Marchal, M., Promayon, E., Troccaz, J.: Comparisons of needle insertion in brachytherapy protocols using a soft tissue discrete model. In: Pediatrics International, p. 153, September 2007
Hing, J.T., Brooks, A.D., Desai, J.P.: Reality-based needle insertion simulation for haptic feedback in prostate brachytherapy. In: IEEE International Conference on Robotics and Automation, pp. 619–624 (2006)
Moreta-Martínez, R., Rubio-Pérez, I., García-Sevilla, M., García-Elcano, L., Pascau, J.: Evaluation of optical tracking and augmented reality for needle navigation in sacral nerve stimulation. Comput. Methods Programs Biomed. 224, 106991 (2022). https://doi.org/10.1016/j.cmpb.2022.106991
Wu, K., Li, B., Zhang, Y., Dai, X.: Review of research on path planning and control methods of flexible steerable needle puncture robot. Comput. Assist. Surg. 27(1), 91–112 (2022). https://doi.org/10.1080/24699322.2021.2023647
Glozman, D., Shoham, M.: Image-guided robotic flexible needle steering. IEEE Trans. Rob. 23(3), 459–467 (2007). https://doi.org/10.1109/TRO.2007.898972
Fichtinger, G., et al.: Image overlay guidance for needle insertion in CT scanner. IEEE Trans. Biomed. Eng. 52(8), 1415–1424 (2005). https://doi.org/10.1109/TBME.2005.851493
Krieger, A., et al.: Design of a novel MRI compatible manipulator for image guided prostate interventions. IEEE Trans. Biomed. Eng. 52(2), 306–313 (2005). https://doi.org/10.1109/TBME.2004.840497
Krieger, A., et al.: Development and evaluation of an actuated MRI-compatible robotic system for MRI-guided prostate intervention. IEEE/ASME Trans. Mechatron. Joint Publ. IEEE Industrial Electron. Soc. ASME Dyn. Syst. Control Div. 18(1), 273–284 (2012). https://doi.org/10.1109/TMECH.2011.2163523
Tadayyon, H., Lasso, A., Kaushal, A., Guion, P., Fichtinger, G.: Target motion tracking in MRI-guided transrectal robotic prostate biopsy. IEEE Trans. Biomed. Eng. 58(11), 3135–3142 (2011). https://doi.org/10.1109/TBME.2011.2163633
Moreira, P., Boskma, K.J., Misra, S.: Towards MRI-guided flexible needle steering using fiber Bragg grating-based tip tracking. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 4849–4854 (2017). https://doi.org/10.1109/ICRA.2017.7989564
Aboofazeli, M., Abolmaesumi, P., Mousavi, P., Fichtinger, G.: A new scheme for curved needle segmentation in three-dimensional ultrasound images. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1067–1070 (2009). https://doi.org/10.1109/ISBI.2009.5193240
Zhu, M., Salcudean, S.E.: Real-time image-based b-mode ultrasound image simulation of needles using tensor-product interpolation. IEEE Trans. Med. Imaging 30(7), 1391–1400 (2011). https://doi.org/10.1109/TMI.2011.2121091
Boctor, E.M., Choti, M.A., Burdette, E.C., Webster, R.J., III.: Three-dimensional ultrasound-guided robotic needle placement: an experimental evaluation. Int. J. Med. Robot. Comput. Assist. Surg. 4(2), 180–191 (2008). https://doi.org/10.1002/rcs.184
Romano, J.M., Webster, R.J., Okamura, A.M.: Teleoperation of steerable needles. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 934–939 (2007). https://doi.org/10.1109/ROBOT.2007.363105
Duchemin, G., Maillet, P., Poignet, P., Dombre, E., Pierrot, F.: A hybrid position/force control approach for identification of deformation models of skin and underlying tissues. IEEE Trans. Biomed. Eng. 52(2), 160–170 (2005). https://doi.org/10.1109/TBME.2004.840505
Reed, K.B., Kallem, V., Alterovitz, R., Goldbergxz, K., Okamura, A.M., Cowan, N.J.: Integrated planning and image-guided control for planar needle steering. In: 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 819–824 (2008). https://doi.org/10.1109/BIOROB.2008.4762833
Hauser, K., Alterovitz, R., Chentanez, N., Okamura, A., Goldberg, K.: Feedback control for steering needles through 3D deformable tissue using helical paths. Robot. Sci. Syst. Online Proc. V, 37 (2009). https://doi.org/10.15607/rss.2009.v.037
Zhao, X., Guo, H., Ye, D., Huo, B.: Comparison of estimation and control methods for flexible needle in 2D. In: 2016 Chinese Control and Decision Conference (CCDC), pp. 5444–5449 (2016). https://doi.org/10.1109/CCDC.2016.7531970
Aggravi, M., Estima, D.A.L., Krupa, A., Misra, S., Pacchierotti, C.: Haptic teleoperation of flexible needles combining 3D ultrasound guidance and needle tip force feedback. IEEE Robot. Autom. Lett. 6(3), 4859–4866 (2021). https://doi.org/10.1109/LRA.2021.3068635
Gao, D., Lei, Y., Lian, B., Yao, B.: Modeling and simulation of flexible needle insertion into soft tissue using modified local constraints. J. Manuf. Sci. Eng. 138(12), 121012 (2016). https://doi.org/10.1115/1.4034134
Du, S., Li, M., Xu, T., Hu, Y., Wang, Z., Lei, Y.: Design and analysis of a novel experiment platform for 3D needle insertion based on orthogonally arranged dual camera. In: International Manufacturing Science and Engineering Conference, vol. Volume 1: Additive Manufacturing; Biomanufacturing; Life Cycle Engineering; Manufacturing Equipment and Automation; Nano/Micro/Meso Manufacturing (2022). https://doi.org/10.1115/MSEC2022-85764
Amidror, I.: Scattered data interpolation methods for electronic imaging systems: a survey. J. Electron. Imaging 11(2), 157–176 (2002). https://doi.org/10.1117/1.1455013
Guo, Y.Q., et al.: Ultrasound-guided percutaneous needle biopsy for peripheral pulmonary lesions: diagnostic accuracy and influencing factors. Ultrasound Med. Biol. 44(5), 1003–1011 (2018). https://doi.org/10.1016/j.ultrasmedbio.2018.01.016
Acknowledgements
This work was supported by the National Key Scientific Instrument and Equipment Development Project (Grant No. 81827804), Zhejiang Provincial Natural Science Foundation of China (Grant No. LSD19H180004), and Science Fund for Creative Groups of NSFC (Grant No. 51821903).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Du, S. et al. (2023). Digital Twin Model Based Robot-Assisted Needle Insertion Navigation System with Visual and Force Feedback. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14269. Springer, Singapore. https://doi.org/10.1007/978-981-99-6489-5_10
Download citation
DOI: https://doi.org/10.1007/978-981-99-6489-5_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6488-8
Online ISBN: 978-981-99-6489-5
eBook Packages: Computer ScienceComputer Science (R0)