Haptic experience design: What hapticians do and where they need help

https://doi.org/10.1016/j.ijhcs.2017.04.004Get rights and content

Abstract

From simple vibrations to roles in complex multisensory systems, haptic technology is often a critical, expected component of user experience – one face of the rapid progression towards blended physical-digital interfaces. Haptic experience design, which is woven together with other multisensory design efforts, interfaces is now becoming part of many designers' jobs. We can expect it to present unique challenges, and yet we know almost nothing of what it looks like “in the wild” due to the field's relative youth, its technical complexity, the multisensory interactions between haptics, sight, and sound, and the difficulty of accessing practitioners in professional and proprietary environments. In this paper, we analyze interviews with six professional haptic designers to document and articulate haptic experience design by observing designers' goals and processes and finding themes at three levels of scope: the multisensory nature of haptic experiences, a map of the collaborative ecosystem, and the cultural context of haptics. Our findings are augmented by feedback obtained in a recent design workshop at an international haptics conference. We find that haptic designers follow a familiar design process, but face specific challenges when working with haptics. We capture and summarize these challenges, make concrete recommendations to conquer them, and present a vision for the future of haptic experience design.

Introduction

Haptic feedback can provide value in several ways, such as accessibility (Bliss et al., 1970), unintrusive feedback (MacLean, 2009), and motor skill training (Milot et al., 2010). Recently, high-fidelity haptic technology has expanded the available range of user experience, improving support for emotional therapy (Sefidgar et al., 2015, Vaucelle et al., 2009), education (Sato et al., 2008, Minaker et al., 2016), and entertainment (Schneider et al., 2015a). Technological advances enable more compelling haptic sensations in consumer products by making it possible to render variable friction on direct-touch surfaces (Levesque et al., 2011, Winfield et al., 2007), and produce forces without needing to ground devices to a table or wall (Culbertson et al., 2016, Winfree et al., 2009). Even commodity vibrotactile displays are increasing in expressiveness, with high-quality actuation a priority in devices such as the Apple Watch (http://www.apple.com) and the Pebble watch (http://www.pebble.com), although often at the cost of painstaking and costly design effort. Touch is now increasingly studied within market research and business strategy planning because well-designed tactile aspects can improve the quality of product opinions and encourage consumer purchases (Jansson-Boyd, 2011), potentially enhancing the overall multisensory experience (Spence and Gallace, 2011). Part of the power of touch is the emotional, visceral (Norman, 2004) value with it has within a design, giving haptics a close relationship with user experience.

In this paper, we use the engineering and human-computer interaction (HCI) definition5 of “haptic” to refer to one or more perceived sensations of touch; this includes tactile and proprioceptive feedback, active human touch, and passive experience of actuated technology.

We define HaXD as

The design (planning, development, and evaluation) of user experiences deliberately connecting interactive technology to one or more perceived senses of touch, possibly as part of a multisensory experience.

Our focus is on gaining a better understanding of the workflow and processes currently used by hapticians, including those related to integrating haptics into a multisensory experience. We define a haptician as one who is skilled at making haptic sensations, technology, or experiences. We use this term to capture the diversity of people who currently make haptics, and the diversity of their goals. Many people with a need to design haptics may not have formal design training, and may focus on subsets of the entire experience, e.g., technical demonstrations or creating stimuli for psychological tests.

We describe two studies examining how contemporary hapticians design haptic experiences for use in real-world products. We begin by identifying current obstacles to good HaXD and the target audience for our work, then we provide a roadmap to the rest of the paper. Fig. 1 provides a visual overview of our work.

The academic literature suggests many challenges to design for haptic experience. Haptic content remains scarce and design knowledge is limited. Some issues are technological, such as highly variable hardware platforms and communications latency (Kaaresoja et al., 2014). Other issues are human-centered, arising from individual user characteristics in perception and preferences: low-level perceptual variation (Lofvenberg and Johansson, 1984), responses to programmed (Levesque et al., 2011) and natural (Hollins et al., 2000) textures, sensory declines due to aging (Stevens, 1992, Stevens and Choo, 1996), and varied interpretation and appreciation of haptic effects and sensations (Seifi and MacLean, 2013, Seifi et al., 2015) – often because of personal experience (Schneider and MacLean, 2014), or of the close relationship between touch and other senses.

These research findings are reinforced by many interactions the authors have had with practitioners in industry. Prior to our studies, we suspected that there were many challenges related to haptics, but we found little direct evidence in the literature to back this up and guide our research. As we discovered, hapticians who are free to speak about their work are rare because of intellectual property concerns, which may partially account for the lack of prior work in this area within the literature. Thus motivated, we conducted two studies on the workflows used by hapticians when they were engaged in HaXD – an aspect of design that has been largely unexplored.

In our studies, we take a first in-depth look at haptic designers' experiences in order to describe HaXD, identify its unique challenges, and connect it to other fields of design . We focus specifically on HaXD instead of the more general notion of “haptic design,” which can also refer to design practices related to haptics not directly involving user experience, e.g., mechanical design of a new actuator or software design of a new control method. Our definition of HaXD encompasses pseudo-haptics (Pusch and Lécuyer, 2011) and other multisensory illusions that compel a user to perceive a haptic sensation in the absence of direct tactile or proprioceptive tic stimulation, or modify their perception of one on the basis of conflicting input in another sense. These represent ways in which haptic design must involve other perceptual modalities, alongside direct motivations to create fully multisensory experiences. Much of what we discuss can also be gainfully applied to the design of tangible interfaces, even with their lack of actuation, although we leave them out of our scope to focus on actuated interfaces. Similarly, we believe many of our findings can inform general multisensory experience design, but limit our claims to HaXD.

We primarily target readers who are one step removed from HaXD, but who have other design, haptics, or business expertise relevant to haptics.

We expect that experienced haptic experience design experts will be unsurprised by the insights herein. Although not our primary audience, we hope that the articulated challenges and recommendations will nevertheless still be useful for their practice (particularly for those still early in their learning curve) because it consolidates and extends their ad hoc knowledge into a formal framework.

We expect that non-haptic design experts will find our discussion of the specific challenges to HaXD informative because it reveals processes of design that are invisible or are taken for granted in other fields. We also hope non-haptic designers might lend their expertise to accelerate the generation of tools and techniques for creatively working with these complex interactive systems.

We expect that haptic experts engaged in research or device design that is not directly user-facing will develop a further appreciation for how UX design is important for successful haptic technology, and will see ways in which their devices or research findings can be applied in practice. The recommendations we provide may also motivate several avenues of either basic or applied haptic research that these experts could pursue.

We expect that industry practitioners who are not experts in any of these fields will gain insight into how the business case for haptic technology might be more quickly built. This includes those already involved with haptics or similar technologies such as wearables, as well as those looking to become involved. We believe our findings may help cultivate connections between the diverse stakeholders involved with HaXD, and that the challenges (and thus the opportunities) that we identify will inspire people to work more with this emerging modality and that researchers and practitioners engaged in multisensory HCI will find parallels in their work.

In this paper, we describe two studies in which we sought to gain a solid understanding of HaXD as it is currently practiced “in the wild” by actual practitioners (hapticians) in their day-to-day work. After a review of the existing literature in Section 2 , we report on the first study in Section 3: a grounded theory (Corbin and Strauss, 2008) analysis of intensive interviews with six professional haptic designers. We describe observations of haptic designers' process organized into three cross-cutting themes: the holistic, multisensory, and vertical-integrated nature of the experiences they design; the collaborative ecosystem in which haptic experience designers play multiple roles; and the influences of the cultural contexts in which haptic experiences are used and the value and risk this poses. In Section 4 we describe a second study conducted as part of a workshop at a major international haptics conference (World Haptics 2015). The second study complements the first by collecting quantitative and qualitative feedback from a broader sector of industry and academic designers regarding tool use, collaboration, evaluation methods, and challenges facing hapticians. In Section 5, we summarize and discuss our overall findings in three major areas:

  • 1.

    A description of current HaXD practice showing how it has already emerged as a distinct field of design.

  • 2.

    A list of challenges facing haptic experience designers, and some unique considerations HaXD requires compared to other more established fields of design.

  • 3.

    Recommendations for accelerating the development of HaXD as a full-fledged field of design.

We conclude with a few remarks imagining what a mature discipline of HaXD might look like in the near future and its role within multisensory HCI.

Section snippets

Related work

In this section, we discuss key elements of contemporary thinking about user experience design (UX design or XD) and a specific approach known as “design thinking.” We then briefly review haptic technology (hardware and software) and relevant aspects of human perception before providing a critical summary of previous efforts to understand and support HaXD.

Study 1: interviews with hapticians about HaXD in the Wild

In this section, we present findings from our first study, a qualitative analysis of interviews with six professional hapticians.

Study 2: findings from a follow-up workshop

Our second study was conducted during a workshop on haptic experience design at World Haptics 2015, the largest academic haptics conference to date, held that year in Chicago, IL, USA (http://haptics2015.org).

Discussion

As a first step in further exploring the findings from our two studies, we examine in more detail the critical activities practiced by hapticians. This inventory confirms that HaXD is a field of design with familiar processes, but also one that is developing its own identity distinct from general UX design. We then identify major challenges encountered in HaXD that are unique to or exaggerated when the experiences being designed are haptic. We conclude with several concrete recommendations to

Conclusion

We have provided a first exploration of how haptic experience design (HaXD) is being practiced in industry. We report findings from interviews with six hapticians, finding observations about designer process and themes about the holistic nature of haptic experiences and the collaborative ecosystem and cultural context of our participants. We supplement this with broad follow-up data from a recent workshop at a major haptics conference.

We identified the various activities hapticians practice,

Acknowledgements

We deeply thank our designer participants for their time and dedication, Gordon Minaker for helping with survey distribution and workshop administration, and Hasti Seifi for helping to conduct the workshop brainstorming session. More broadly, we thank generations of our own lab's designers and our collaborators for the foundations that led the way to these findings and ideas. This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), the GRAND

References (94)

  • S.J. Lederman et al.

    Hand movements: a window into haptic object recognition

    Cogn. Psychol.

    (1987)
  • J. Lofvenberg et al.

    Regional differences and interindividual variability in sensitivity to vibration in the glabrous skin of the human hand

    Brain Res.

    (1984)
  • J. Bliss et al.

    Optical-to-tactile image conversion for the blind

    IEEE Trans. Man Machine SystemsMach. Syst.

    (1970)
  • Bucci, P., Cang, X. L., Valair, A., Marino, D., Tseng, L., Jung, M., Rantala, J., Schneider, O., MacLean, K., 2017....
  • B. Buxton

    Sketching User Experiences: Getting the Design Right and the Right Design

    (2007)
  • A. Charpentier

    Analyse experimentale de quelques elements de la sensation de poids (Experimental analysis of some elements of the sensation of weight)

    Archives de Physiologie Normale et PathologiqueArch. De. Physiol. Norm. Et. Pathol.

    (1891)
  • S. Choi et al.

    Vibrotactile display: perception, technology, and applications

    Proc. IEEE

    (2013)
  • J. Corbin et al.

    Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory

    (2008)
  • N. Cross

    Designerly Ways of Knowing

    (2006)
  • M. Csikszentmihalyi

    Creativity: Flow and the Psychology of Discovery and Invention

    (1996)
  • H. Culbertson et al.

    Modeling and rendering realistic textures from unconstrained tool-surface interactions

    IEEE Trans. haptics

    (2014)
  • Culbertson, H., Walker, J.M., Okamura, A.M., 2016. Modeling and design of asymmetric vibrations to induce ungrounded...
  • F. Danieau et al.

    Toward haptic cinematographyenhancing movie experiences with camera-based haptic effects

    IEEE Multimed.

    (2014)
  • D. Dougherty

    The maker movement

    Innovations

    (2012)
  • Dow, S., Kulkarni, A., Klemmer, S., Hartmann, B., 2012. Shepherding the crowd yields better work. In: Proceedings of...
  • C.A. Ellis et al.

    Groupware: some issues and experiences

    Commun. ACM

    (1991)
  • Enriquez, M., MacLean, K., 2003. The hapticon editor: A tool in support of haptic communication research. In: HAPTICS...
  • Forsslund, J., Yip, M., Sallnäs, E.-L., 2015. WoodenHaptics. In: Proceedings of the Ninth International Conference on...
  • A. Gallace et al.

    In touch with the future: The sense of touch from cognitive neuroscience to virtual reality

    (2014)
  • Gallacher, C., Mohtat, A., Ding, S., 2016. Toward open-source portable haptic displays with visual-force-tactile...
  • C. Geertz

    The interpretation of cultures: Selected essays

    (1973)
  • Greenberg, S., Fitchett, C., 2001. Phidgets: Easy development of physical interfaces through physical widgets. In:...
  • Guillotel, P., Danieau, F., Fleureau, J., Rouxel, I., 2016. Introducing basic principles of haptic cinematography...
  • Gunther, E., Davenport, G., O′Modhrain, S., 2002. Cutaneous grooves: Composing for the sense of touch. In: NIME ’02....
  • Hartmann, B., Yu, L., Allison, A., Yang, Y., Klemmer, S. R., 2008. Design as exploration: Creating interface...
  • Hassenzahl, M., Burmester, M., Koller, F., 2003. AttrakDiff: Ein fragebogen zur messung wahrgenommener hedonischer und...
  • Haverkamp, M., 2014. Synesthetic Design: Handbook for a multi-sensory approach. Birkhuser Verlag,...
  • Hayward, V., 2016. Tactile illusions. In: Scholarpedia of Touch. Atlantis Press, Paris, pp. 327–342...
  • Herring, S.R., Chang, C.-C., Krantzler, J., Bailey, B.P., Apr 2009. Getting inspired! understanding how and why...
  • M. Hollins et al.

    Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis

    Percept. Psychophys.

    (1993)
  • M. Hollins et al.

    Individual differences in perceptual space for tactile textures: evidence from multidimensional scaling

    Percept. Psychophys.

    (2000)
  • Hong, K., Lee, J., Choi, S., 2013. Demonstration-based vibrotactile pattern authoring. In: TEI ’13. ACM Press, New...
  • Hwang, I., MacLean, K. E., Brehmer, M., Hendy, J., Sotirakopoulos, A., Choi, S., 2011. The haptic crayola effect:...
  • A. Israr et al.

    Feel effects: enriching storytelling with haptic feedback

    Trans. Appl. Percept. (TAP)

    (2014)
  • Israr, A., Zhao, S., McIntosh, K., Schwemler, Z., Fritz, A., Mars, J., Bedford, J., Frisson, C., Huerta, I., Kosek, M.,...
  • C.V. Jansson-Boyd

    Touch matters: exploring the relationship between consumption and tactile interaction

    Social. Semiot.

    (2011)
  • L. Jones

    News from the field: courses in haptics

    Trans. Haptics

    (2014)
  • Jones, L. A. L., Berris, M., 2002. The psychophysics of temperature perception and thermal-interface design. In:...
  • T. Kaaresoja et al.

    Towards the temporally perfect virtual buttontouch-feedback simultaneity and perceived quality in mobile touchscreen press interactions

    ACM Trans. Appl. Percept.

    (2014)
  • E.R. Kandel et al.

    Principles of Neural Science

    (2000)
  • Y. Kim et al.

    Exploring tactile movies: an initial tactile glove design and concept evaluation

    IEEE Multimed. PP

    (2009)
  • Kulkarni, C., Dow, S.P., Klemmer, S.R., 2014. Early and repeated exposure to examples improves creative work. In:...
  • S.J. Lederman et al.

    Haptic perception: a tutorial

    Atten., Percept. Psychophys.

    (2009)
  • Ledo, D., Nacenta, M.A., Marquardt, N., Boring, S., Greenberg, S., 2012. The HapticTouch toolkit. In: Proceedings of...
  • Lee, J., Choi, S., 2012. Evaluation of vibrotactile pattern design using vibrotactile score. In: HAPTICS ’12. IEEE,...
  • Lee, J., Ryu, J., Choi, S., 2009. Vibrotactile score: A score metaphor for designing vibrotactile patterns. In: World...
  • Levesque, V., Oram, L., MacLean, K., Cockburn, A., Marchuk, N. D., Johnson, D., Colgate, J. E., Peshkin, M. A., 2011....
  • Cited by (0)

    1

    Collected workshop data, transcribed interviews, led final qualitative analysis and writing.

    2

    Provided supervision, contributed to research design, writing and ideas.

    3

    Collected interview data and field notes, led initial qualitative analysis, suggested initial ideas.

    4

    Provided supervision, contributed to writing and initial ideas.

    View full text