Skip to main content
SpringerLink
Log in

Importance of force feedback for following uneven virtual paths with a stylus

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

It is commonly known that a physical textured path can be followed by indirect touch through a probe also in absence of vision if sufficiently informative cues are delivered by the other sensory channels, but prior research indicates that the level of performance while following a virtual path on a touchscreen depends on the type and channel such cues belong to. The re-enactment of oriented forces, as they are induced by localized obstacles in probe-based exploration, may be important to equalize the performance between physical and virtual path following. Using a stylus attached to a force-feedback arm, an uneven path marked by virtual bars was traversed while time and positions were measured under normal sensory conditions, as well as in absence of vision or hearing. Alternatively, participants followed the same path on a wooden tablet provided with physical bars in relief (i.e., without receiving synthetic force) under the same conditions. The visual conditions were found to be significantly faster than the non-visual conditions. However, there was no significant advantage of traversing either path. In contrast to previous experiments in which the virtual bars were rendered using vibrotactile and/or auditory cues, comparable times to traverse the physical and virtual path were found also when vision was disabled. Our results hence suggest that users who are deprived of vision follow textured virtual paths as efficiently as physical paths, if unevenness is rendered using restorative force cues through a stylus.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://www.immersion.fr/en/phantom-touch/.

  2. https://en.wikipedia.org/wiki/Janka_hardness_test.

  3. https://unity.com/.

  4. https://assetstore.unity.com/.

References

  1. Accot J, Zhai S (1997) Beyond Fitts’ law: models for trajectory-based HCI tasks. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems, CHI ’97, pp 295–302. ACM, New York, NY, USA. https://doi.org/10.1145/258549.258760

  2. Accot J, Zhai S (1999) Performance evaluation of input devices in trajectory-based tasks: an application of the steering law. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’99, pp 466–472. ACM, New York, NY, USA. https://doi.org/10.1145/302979.303133

  3. Benedetti F (2002) Physical response to collision between deformable objects. Technical report. EPFL, Lausanne, Switzerland

  4. Cho Y, Bianchi A, Marquardt N, Bianchi-Berthouze N (2016) RealPen: providing realism in handwriting tasks on touch surfaces using auditory-tactile feedback. In: Proceedings of the 29th annual symposium on user interface software and technology, UIST ’16, pp 195–205. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2984511.2984550

  5. Crossan A, Brewster S (2008) Multimodal trajectory playback for teaching shape information and trajectories to visually impaired computer users. ACM Trans Access Comput 1(2):1–34. https://doi.org/10.1145/1408760.1408766

    Article  Google Scholar 

  6. Culbertson H, Kuchenbecker KJ (2017) Importance of matching physical friction, hardness, and texture in creating realistic haptic virtual surfaces. IEEE Trans Haptics 10(1):63–74

    Article  Google Scholar 

  7. Culbertson H, Unwin J, Kuchenbecker KJ (2014) Modeling and rendering realistic textures from unconstrained tool-surface interactions. IEEE Trans Haptics 7(3):381–393

    Article  Google Scholar 

  8. Del Piccolo A, Rocchesso D, Papetti S (2018) Path following in non-visual conditions. IEEE Trans Haptics 12(1):56–67

    Article  Google Scholar 

  9. Fielder T, Vardar Y (2019) A novel texture rendering approach for electrostatic displays. In: International workshop on haptic and audio interaction design—HAID2019. Lille, France. https://hal.archives-ouvertes.fr/hal-02011782

  10. Fukushi T, Ashe J (2003) Adaptation of arm trajectory during continuous drawing movements in different dynamic environments. Exp Brain Res 148:95–104. https://doi.org/10.1007/s00221-002-1260-0

    Article  Google Scholar 

  11. Gao B, Kim H, Lee H, Lee J, Kim JI (2018) Effects of continuous auditory feedback on drawing trajectory-based finger gestures. IEEE Trans Human-Mach Syst 48(6):658–669. https://doi.org/10.1109/THMS.2018.2850329

    Article  Google Scholar 

  12. Hwang J, Hwang W (2011) Vibration perception and excitatory direction for haptic devices. J Intell Manuf 22:17–27. https://doi.org/10.1007/s10845-009-0277-7

    Article  Google Scholar 

  13. Moore M, Wilhelms J (1988) Collision detection and response for computer animation. SIGGRAPH Comput Graph 22(4):289–298. https://doi.org/10.1145/378456.378528

    Article  Google Scholar 

  14. Moscatelli A, Bianchi M, Serio A, Al Atassi O, Fani S, Terekhov A, Hayward V, Ernst M, Bicchi A (2014) A change in the fingertip contact area induces an illusory displacement of the finger. In: International conference on human haptic sensing and touch enabled computer applications, pp 72–79. Springer

  15. Nancel M, Lank E (2017) Modeling user performance on curved constrained paths. In: Proceedings of the 2017 CHI conference on human factors in computing systems, CHI ’17, pp 244–254. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3025453.3025951

  16. Pastel R (2006) Measuring the difficulty of steering through corners. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’06, pp 1087–1096. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1124772.1124934

  17. Robles-De-La-Torre G, Hayward V (2001) Force can overcome object geometry in the perception of shape through active touch. Nature 412(6845):445–448

    Article  Google Scholar 

  18. Rocchesso D, Delle Monache S, Papetti S (2016) Multisensory texture exploration at the tip of the pen. Int J Human-Comput Stud 85:47–56. https://doi.org/10.1016/j.ijhcs.2015.07.005 (Data Sonification and Sound Design in Interactive Systems)

  19. Shultz C, Peshkin M, Colgate JE (2018) The application of tactile, audible, and ultrasonic forces to human fingertips using broadband electroadhesion. IEEE Trans Haptics 11(2):279–290

    Article  Google Scholar 

  20. Silva AJ, Ramirez OAD, Vega VP, Oliver JPO (2009) Phantom omni haptic device: kinematic and manipulability. In: Proceedings of 2009 electronics, robotics and automotive mechanics conference (CERMA), pp 193–198. Cuernavaca, Morelos, México. https://doi.org/10.1109/CERMA.2009.55

  21. Stanton TR, Spence C (2020) The influence of auditory cues on bodily and movement perception. Front Psychol 10:3001. https://doi.org/10.3389/fpsyg.2019.03001

    Article  Google Scholar 

  22. Sun M, Ren X, Cao X (2010) Effects of multimodal error feedback on human performance in steering tasks. J Inf Process 18:284–292. https://doi.org/10.2197/ipsjjip.18.284

    Article  Google Scholar 

  23. Sun M, Ren X, Zhai S, Mukai T (2012). An investigation of the relationship between texture and human performance in steering tasks. In: Proceedings of the 10th Asia Pacific conference on computer human interaction, APCHI ’12, pp 1–6. ACM, New York, NY, USA. https://doi.org/10.1145/2350046.2350048

  24. Tajadura-Jiménez A, Bianchi-Berthouze N, Furfaro E, Bevilacqua F (2015) Sonification of surface tapping changes behavior, surface perception, and emotion. IEEE MultiMedia 22(1):48–57

    Article  Google Scholar 

  25. Visell Y, Giordano BL, Millet G, Cooperstock JR (2011) Vibration influences haptic perception of surface compliance during walking. PLoS One 6(3):e17697

    Article  Google Scholar 

  26. Wang Q, Ren X, Sarcar S, Sun X (2016) EV-Pen: leveraging electrovibration haptic feedback in pen interaction. In: Proceedings of the 2016 ACM international conference on interactive surfaces and spaces, ISS ’16, pp 57–66. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/2992154.2992161

  27. Wiertlewski M, Fenton Friesen R, Colgate JE (2016) Partial squeeze film levitation modulates fingertip friction. Proc Natl Acad Sci 113(33):9210–9215. https://doi.org/10.1073/pnas.1603908113

    Article  Google Scholar 

  28. Xie M, Niu X (2020) A 3D roaming and collision detection algorithm applicable for massive spatial data. PLoS one 15(2):e0229038

    Article  Google Scholar 

  29. Yamanaka S, Miyashita H (2019) Modeling pen steering performance in a single constant-width curved path. In: Proceedings of the 2019 ACM international conference on interactive surfaces and spaces, ISS ’19, pp 65–76. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3343055.3359697

  30. Yamanaka S, Stuerzlinger W, Miyashita H (2017) Steering through sequential linear path segments. In: Proceedings of the 2017 CHI conference on human factors in computing systems, CHI ’17, pp 232–243. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3025453.3025836

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Computer Science and Physics, University of Udine, Udine, Italy

    Federico Fontana

  2. Udine, Italy

    Francesco Muzzolini

  3. Department of Mathematics and Computer Science, University of Palermo, Palermo, Italy

    Davide Rocchesso

Authors
  1. Federico Fontana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Muzzolini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Davide Rocchesso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Federico Fontana.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fontana, F., Muzzolini, F. & Rocchesso, D. Importance of force feedback for following uneven virtual paths with a stylus. J Multimodal User Interfaces 16, 183–191 (2022). https://doi.org/10.1007/s12193-021-00384-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-021-00384-w

Keywords

Search

Navigation