Skip to main content
SpringerLink
Log in

Design and application of 2D illusory vibrotactile feedback for hand-held tablets

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

In this study, we investigate whether the “out of body” vibrotactile illusion known as funneling could be applied to enrich and thereby improve the interaction performance on a tablet-sized media device. First, a series of pilot tests was taken to determine the appropriate operational conditions and parameters (such as the tablet size, holding position, minimal required vibration amplitude, and the effect of matching visual feedback) for a two-dimensional (2D) illusory tactile rendering method. Two main experiments were then conducted to validate the basic applicability and effectiveness of the rendering method, and to further demonstrate how the illusory tactile feedback could be deployed in an interactive application and actually improve user performance. Our results showed that for a tablet-sized device (e.g., iPad mini and iPad), illusory perception was possible (localization performance of up to 85%) using a rectilinear grid with a resolution of 5 \(\times \) 7 (grid size: 2.5 cm) with matching visual feedback. Furthermore, the illusory feedback was found to be a significant factor in improving the user performance in a 2D object search/attention task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Alles DS (1970) Information transmission by phantom sensations. IEEE Trans Man Mach Syst 11(1):85–91

    Article  MathSciNet  Google Scholar 

  2. Barghout A, Kammerl J, Cha J, Steinbach E, El Saddik A (2009) Spatial resolution of vibrotactile perception on the human forearm when exploiting funneling illusion. In: Proc. IEEE Intl. Workshop on HAVE, pp 19–23

  3. Bekesy GV (1958) Funneling in the nervous system and its role in loudness and sensation intensity on the skin. J Acoust Soc Am 30(5):399–412

    Article  Google Scholar 

  4. Biocca FA, Inoue Y, Lee A, Polinsky H, Tang A (2002) Visual cues and virtual touch: Role of visual stimuli and intersensory integration in cross-modal haptic illusions and the sense of presence. In: Proceedings of presence, pp 376–394

  5. Burdea G, Coiffet P (2003) Virtual reality technology. Presence Teleoper Virtual Environ 12:663–664

    Article  Google Scholar 

  6. Cholewiak RW, Collins AA (2000) The generation of vibrotactile patterns on a linear array: influences of body site, time, and presentation mode. Atten Percept Psychophys 62(6):1220–1235

    Article  Google Scholar 

  7. Eimer M, Forster B, Vibell J (2005) Cutaneous saltation within and across arms: a new measure of the saltation illusion in somatosensation. Percept Psychophys 67(3):458–468

    Article  Google Scholar 

  8. Flach R, Haggard P (2006) The cutaneous rabbit revisited. J Exp Psychol Hum Percept Perform 32:717

    Article  Google Scholar 

  9. Gallace A, Spence C (2008) The cognitive and neural correlates of tactile consciousness: a multisensory perspective. Conscious Cognit 17:370–417

    Article  Google Scholar 

  10. Geldard FA, Sherrick CE (1978) The cutaneous rabbit: a perceptual illusion. Science 178(4507):178–179

    Google Scholar 

  11. Geshelder G (1997) Psychophysics: the fundamentals. Lawrence Erlbaum Associates, Mahwah, New Jersey

    Google Scholar 

  12. Grohn M, Lokki T, Savioja L, Takala T (2001) Some aspects of role of audio in immersive visualization. In Proceedings of SPIE, vol 4302, pp 13–22

  13. Hauptmann A, McAvinney P (1993) Gestures with speech for graphic manipulation. Int J Man Mach Stud 38(2):231–249

    Article  Google Scholar 

  14. Hecht D, Reiner M, Halevy G (2006) Multimodal virtual environments: response times, attention, and presence. Presence Teleoper Virtual Environ 15(5):515–523

    Article  Google Scholar 

  15. Hoggan E, Anwar S, Brewster SA (2007) Mobile multi-actuator tactile displays. In: Proceedings of International Workshop on Haptic and Audio Interaction Design, pp 22–33

  16. Israr A, Poupyrev I (2011) Control space of apparent haptic motion. In: IEEE World Haptics Conf

  17. Israr A, Poupyrev I (2011) Tactile brush: drawing on skin with a tactile grid display. In: Proc. SIGCHI Conf. human factors in computing system, pp 2019–2028

  18. Jahwa Electronics, Vibration motor, Korea [Online]. http://www.jahwa.co.kr/

  19. Jaimes A, Sebe N (2005) Multimodal human computer interaction: a survey. In: IEEE International workshop on human computer interaction

  20. Kilgard MP, Merzenich MM (1995) Anticipated stimuli across skin. Nature 373(6516):663

    Article  Google Scholar 

  21. Kim Y, Lee J, Kim GJ (2015) Extending, “out of the body” tactile phantom sensations to 2D and applying it to mobile interaction. Pers Ubiquitous Comput 19(8):1295–1311

    Article  Google Scholar 

  22. Kim Y, Lee J, Kim GJ (2015) Designing of 2D illusory tactile feedback for hand-held tablets. In: Proceedings of 15th IFIP TC 13 international conference on human-computer interaction, INTERACT, pp 10–17

  23. Kingdom F, Prins N (2010) Psychophysics: a practical introduction. Elsevier-Academic Press, USA

    Google Scholar 

  24. Lecuyer A, Mobuchon P, Megard C, Perret J, Andriot C, Colinot JP (2003) HOMERE: a multimodal system for visually impaired people to explore virtual environments. In: Proceedings of IEEE international conference on virtual reality. IEEE, USA, pp 251–257

  25. Luv K (2009) Exploiting perceptual illusions to enhance passive haptics. In: Proceedings of IEEE VR workshop on perceptual illusions in virtual environment, pp 22–24

  26. Miyazaki M, Hirashima M, Nozaki D (2010) The, “Cutaneous Rabbit” hopping out of the body. J Neurosci 30(5):1856–1860

    Article  Google Scholar 

  27. Mizukami Y, Sawada H (2006) Tactile information transmission by apparent movement phenomenon using shape-memory alloy device. Int J Disabil Hum Dev 5(3):277–284

    Article  Google Scholar 

  28. Morioka M, Griffin MJ (2005) Thresholds for the perception of hand-transmitted vibration: dependence on contact area and contact location. Somatosens Motor Res 22:281–297

    Article  Google Scholar 

  29. Niijima A, Ogawa T (2014) A study of changing locations of vibrotactile perception on a forearm by visual stimulation. In: Proceedings of collaboration technologies and social computing, vol 460, pp 86–95

  30. Niijima A, Ogawa T (2016) A study on control of a phantom sensation by visual stimuli. In: Proceedings of haptics: perception, devices, control, and applications, vol 9974, pp 305–315

  31. Nokia (2013) Bouncing ball on Nokia N900. http://bit.ly/x2-bouncingball

  32. Ooka T, Fujita K (2010) Virtual object manipulation system with substitutive display of tangential force and slip by control of vibrotactile phantom sensation. In: Haptics Symp, IEEE, pp 215–218

  33. Oviatt S (2002) Multimodal interfaces. In: Jacko J, Sears A (eds) Handbook of human-computer interaction. Lawrence Erlbaum, Mahwah, New Jersey

  34. Rahal L, Cha J, Saddik A, Kammerl J, Steinbach E (2009) Investigating the influence of temporal intensity changes on apparent movement phenomenon. In: Virtual environments, human-computer interfaces and measurements syst., IEEE Int. Conf., pp 310–313

  35. Raisamo J, Raisamo R, Surakka V (2009) Evaluating the effect of temporal parameters for vibrotactile saltatory patterns. In: Proc. int. conf. multimodal interfaces, pp 319–326

  36. Richard P, Burdea G, Gomez D, Coiffet P (1994) A comparison of haptic, visual and auditive force feedback for deformable virtual objects. In: Proceedings of ICAT, pp 49–62

  37. Sallnas E, Grohn K, Sjostrom C (2000) Supporting presence in collaborative environment by haptic force feedback. ACM Trans CHI 7(4):461–476

    Google Scholar 

  38. Seo J, Choi S (2010) Initial study for creating linearly moving vibrotactile sensation on mobile device. In: Proc. IEEE haptics symposium, pp 67–70

  39. Sherrick CE (1985) A scale for rate of tactual vibration. J Acoust Soc Am 78:78–83

    Article  Google Scholar 

  40. Sherrick CE, Cholewiak RW, Collins AA (1990) The localization of low and high frequency vibrotactile stimuli. J Acoust Soc Am 88(1):169–179

    Article  Google Scholar 

  41. Stein B, Wallace M, Meredith A (1995) Neural mechanisms mediating attention and orientation to multisensory cues. In: Gazzaniga M (ed) The cognitive neurosciences. M.I.T Press, Cambridge, pp 683–702

    Google Scholar 

  42. Stein E, Meredith A (1993) The merging of the senses. MIT Press, London

    Google Scholar 

  43. Tan HZ, Pentland A (1997) Tactual displays for wearable computing. Pers Technol 1:225–230

    Article  Google Scholar 

  44. Welch TB, Warren DH (1986) Intersensory interactions. In: Boff KK, Kaufman L, Thomas J (eds) Handbook of perception and human performance, vol 1, Chapter 24. Wiley, New York

  45. Yatani K, Truong K (2009) SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices. In: Proc. of UIST, pp 111–120

Download references

Acknowledgements

This work was supported in part by the Institute for Information & communications Technology Promotion (IITP) grant (MSIP No. R0190-16-2011, “Development of Vulnerability Discovery Technologies for IoT Software Security”), the Korea National Research Foundation Basic Science Research Program (MEST/NRF No. 2011-0030079), and also the Forensic Research Program of the National Forensic Service (NFS), Ministry of Government Administration and Home Affairs, Korea (NFS-2016-DIGITAL-05).

Author information

Authors and Affiliations

  1. Digital Experience Laboratory, Korea University, Anam-dong, Seongbuk-gu, Seoul, Korea

    Youngsun Kim, Jaedong Lee & Gerard Jounghyun Kim

Authors
  1. Youngsun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaedong Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerard Jounghyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerard Jounghyun Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, Y., Lee, J. & Kim, G.J. Design and application of 2D illusory vibrotactile feedback for hand-held tablets. J Multimodal User Interfaces 11, 133–148 (2017). https://doi.org/10.1007/s12193-016-0234-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-016-0234-7

Keywords

Search

Navigation