Skip to main content
SpringerLink
Log in

Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

This paper explores methods that make use of visual cues aimed at generating actual haptic sensation to the user, namely pseudo-haptics. We propose a new pseudo-haptic feedback-based method capable of conveying 3D haptic information and combining visual haptics with force feedback to enhance the user’s haptic experience. We focused on an application related to tumor identification during palpation and evaluated the proposed method in an experimental study where users interacted with a haptic device and graphical interface while exploring a virtual model of soft tissue, which represented stiffness distribution of a silicone phantom tissue with embedded hard inclusions. The performance of hard inclusion detection using force feedback only, pseudo-haptic feedback only, and the combination of the two feedbacks was compared with the direct hand touch. The combination method and direct hand touch had no significant difference in the detection results. Compared with the force feedback alone, our method increased the sensitivity by 5 %, the positive predictive value by 4 %, and decreased detection time by 48.7 %. The proposed methodology has great potential for robot-assisted minimally invasive surgery and in all applications where remote haptic feedback is needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Altman DG, Bland J (1994) Diagnostic test 1: sensitivity and specificity. BMJ 308:1552

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  2. Bibin L, Anatole L, Bonnet M, Delbos A, Dillon C (2008) SAILOR: a 3-D medical simulator of loco-regional anaesthesia based on desktop virtual reality and pseudo-haptic feedback. In: ACM symposium on virtual reality software and technology (VRST) 2008, pp 97–100

  3. Conover WJ (1980) Practical nonparametric statistics, 2nd edn. Wiley, UK

    Google Scholar 

  4. De Gersem G (2005) Reliable and enhanced stiffness perception in soft-tissue telemanipulation. Int J Robot Res 24(10):805–822

    Article  Google Scholar 

  5. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415(6870):429–433

    Article  CAS  PubMed  Google Scholar 

  6. Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874

    Article  Google Scholar 

  7. Gwilliam JC, Mahvash M, Vagvolgyi B, Vacharat A, Yuh DD, Okamura AM (2009) Effects of haptic and graphical force feedback on teleoperated palpation. In: Proceedings of IEEE international conference on robotics and automation 2009, pp 677–682

  8. Hachisu T, Cirio G, Marchal M, Luyer A (2011) Pseudo-haptic feedback augmented with visual and tactile vibrations. IEEE international symposium on virtual reality innovation 2011, pp 327–328

  9. Hayward V, Astley O, Cruz-Hernandez M, Grant D, Robles-De-La-Torre G (2004) Haptic interfaces and devices. Sens Rev 24(1):16–29

    Article  Google Scholar 

  10. Hayward V (2008) A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res Bull 75(6):742–752

    Article  PubMed  Google Scholar 

  11. Kim SY, Kyung KU, Park J, Kwon DS (2007) Real-time area-based haptic rendering and the augmented tactile display device for a palpation simulator. Adv Robot 21(9):961–981

    Article  Google Scholar 

  12. Kimura T, Nojima T (2012) Pseudo-haptic feedback on softness induced by grasping motion. In: Isokoski P, Springare J (eds): EuroHaptics 2012, pp 202–205

  13. Klatzky RL, Lederman SJ, Langseth S (2003) Watching a cursor distorts haptically guided reproduction of mouse movement. J Exp Psychol Appl 9(4):228–235

    Article  PubMed  Google Scholar 

  14. Lecuyer A, Burkhardt JM, Tan CH (2008) A study of the modification of the speed and size of the cursor for simulating pseudo-haptic bumps and holes. ACM Trans Appl Percept 5(3):1–21

    Article  Google Scholar 

  15. Lecuyer A, Burkhardt JM, Coquillart S, Coiffet P (2001) Boundary of illusion: an experiment of sensory integration with a pseudo-haptic system. In: Proceedings of the 2001 IEEE virtual reality conference, pp 115–122

  16. Li M, Faragasso A, Konstantinova J, Aminzadeh V, Seneviratne LD, Dasgupta P, Althoefer K (2014) A novel tumor localization method using haptic palpation based on soft tissue probing data. In: Proceedings of IEEE international conference on robotics and automation 2014, pp 4188–4193

  17. Li M, Liu H, Seneviratne LD, Althoefer K (2012) Tissue stiffness simulation and abnormality localization using pseudo-haptic feedback. In: Proceedings of IEEE international conference on robotics and automation 2012, pp 5359–5364

  18. Liu H, Noonan DP, Challacombe BJ, Dasgupta P, Seneviratne LD, Althoefer K (2010) Rolling mechanical imaging for tissue abnormality localization during minimally invasive surgery. IEEE Trans Biomed Eng 57(2):404–414

    Article  PubMed  Google Scholar 

  19. Liu H, Li J, Song X, Seneviratne LD, Althoefer K (2011) Rolling indentation probe for tissue abnormality identification during minimally invasive surgery. IEEE Trans Robot 27(3):450–460

    Article  Google Scholar 

  20. Liu H, Sangpradit K, Li M, Dasgupta P, Althoefer K, Seneviratne LD (2014) Inverse finite-element modeling for tissue parameter identification using a rolling indentation probe. Med Biol Eng Comput 52(1):17–28

    Article  PubMed  Google Scholar 

  21. Masuzaki R, Tateishi R, Yoshida H, Sato T, Ohki T, Goto T, Yoshida H, Sato S, Sugioka Y, Ikeda H, Shiina S, Kawabe T, Omata M (2007) Assessing liver tumor stiffness by transient elastography. Hepatol Int 1(3):394–397

    Article  PubMed Central  PubMed  Google Scholar 

  22. Megumi N, Kuroda T, Komori M, Oyama H (2003) Evaluation and user study of haptic simulator for learning palpation in cardiovascular surgery. In: Proceedings of international conference on artificial reality and telexistence 2003.

  23. Mensvoort K, Vos P, Hermes DJ, Liere RV (2010) Perception of mechanically and optically simulated bumps and holes. ACM Trans Appl Percept 7(2):10:1–24

    Article  Google Scholar 

  24. Nedel LP, Thalmann D (1998) Real-time muscle deformations using mass-spring systems. In: Proceedings computer graphics international 1998, pp 156–165

  25. Netti PA, Berk DA, Swartz MA, Grodzinsky AJ, Jain RK (2000) Role of extracellular matrix assembly in interstitial transport in solid tumors. Cancer Res 60(9):2497–2503

    CAS  PubMed  Google Scholar 

  26. Salomon G, Kollerman J, Thederan I, Chun FKH, Budaus L, Schlomm T, Isbarn H, Heinzer H, Huland H, Graefen M (2008) Evaluation of prostate cancer detection with ultrasound real-time elastography: a comparison with step section pathological analysis after radical prostatectomy. Eur Urol 54(6):1354–1362

    Article  PubMed  Google Scholar 

  27. Sangpradit K, Liu H, Dasgupta P, Althoefer K, Seneviratne LD (2011) Finite-element modeling of soft tissue rolling indentation. IEEE Trans Biomed Eng 58(12):3319–3327

    Article  PubMed  Google Scholar 

  28. Venkatesh SK, Yin M, Glockner JF, Takahashi N, Araoz PA, Talwalkar JA, Ehman RL (2008) MR elastography of liver tumors: preliminary results. AJR Am J Roentgenol 190(6):1534–1540

    Article  PubMed Central  PubMed  Google Scholar 

  29. Wallis S (2013) Binomial confidence intervals and contingency tests: mathematical fundamentals and the evaluation of alternative methods. J Quant Linguist 20(3):178–208

    Article  Google Scholar 

  30. Wellman P, Howe R (1999) Breast tissue stiffness in compression is correlated to histological diagnosis. Harv BioRob Lab Tech Rep

  31. Wilcoxon F (1946) Individual comparisons of grounded data by ranking methods. J Econ Entomol 39:269

    Article  CAS  PubMed  Google Scholar 

  32. Wilson EB (1927) Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 22:209–212

    Article  Google Scholar 

  33. Woodward W, Strom E, Tucker SL, McNeese MD, Perkins GH, Schechter NR, Singletary SE, Theriault RL, Hortobagyi GN, Hunt KK, Buchholz T (2003) Changes in the 2003 American joint committee on cancer staging for breast cancer dramatically affect stage-specific survival. J Clin Oncol 21(17):324–348

    Article  Google Scholar 

  34. Yamamoto T, Abolhassani N (2012) Augmented reality and haptic interfaces for robot assisted surgery. Int J Med Robot Comput Assist Surg 8:45–56

    Article  Google Scholar 

  35. Yau C (2009) R tutoral eBook. r-tutor.com. http://www.r-tutor.com/elementary-statistics/non-parametric-methods/wilcoxon-signed-rank-test Accessed 20 Mar 2014

Download references

Acknowledgments

The research leading to these results has received funding from the National Natural Science Foundation of China (Approval No. 51175412), the China Scholarship Council, the GSTT charity, the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London, and the European Commission’s Seventh Framework Programme under grant agreement 287728 in the framework of EU project STIFF-FLOP. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, 710049, Shaanxi, China

    Min Li

  2. Department of Informatics, Kings College London, London, WC2R 2LS, UK

    Jelizaveta Konstantinova, Emanuele L. Secco, Allen Jiang, Hongbin Liu, Thrishantha Nanayakkara, Lakmal D. Seneviratne, Kaspar Althoefer & Helge A. Wurdemann

  3. Department of Mathematics and Computer Science, Hope University, Liverpool, UK

    Emanuele L. Secco

  4. College of Engineering, Khalifa University of Science, Technology and Research, Abu Dhabi, UAE

    Lakmal D. Seneviratne

  5. Medical Research Council (MRC) Centre for Transplantation, King’s College London, Kings Health Partners, Guys Hospital, London, SE1 9RT, UK

    Prokar Dasgupta

Authors
  1. Min Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jelizaveta Konstantinova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emanuele L. Secco
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Allen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hongbin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Thrishantha Nanayakkara
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lakmal D. Seneviratne
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Prokar Dasgupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kaspar Althoefer
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Helge A. Wurdemann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Konstantinova, J., Secco, E.L. et al. Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue. Med Biol Eng Comput 53, 1177–1186 (2015). https://doi.org/10.1007/s11517-015-1309-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-015-1309-4

Keywords

Search

Navigation