Haptic experience design: What hapticians do and where they need help
Introduction
Haptic feedback can provide value in several ways, such as accessibility (Bliss et al., 1970), unintrusive feedback (MacLean, 2009), and motor skill training (Milot et al., 2010). Recently, high-fidelity haptic technology has expanded the available range of user experience, improving support for emotional therapy (Sefidgar et al., 2015, Vaucelle et al., 2009), education (Sato et al., 2008, Minaker et al., 2016), and entertainment (Schneider et al., 2015a). Technological advances enable more compelling haptic sensations in consumer products by making it possible to render variable friction on direct-touch surfaces (Levesque et al., 2011, Winfield et al., 2007), and produce forces without needing to ground devices to a table or wall (Culbertson et al., 2016, Winfree et al., 2009). Even commodity vibrotactile displays are increasing in expressiveness, with high-quality actuation a priority in devices such as the Apple Watch (http://www.apple.com) and the Pebble watch (http://www.pebble.com), although often at the cost of painstaking and costly design effort. Touch is now increasingly studied within market research and business strategy planning because well-designed tactile aspects can improve the quality of product opinions and encourage consumer purchases (Jansson-Boyd, 2011), potentially enhancing the overall multisensory experience (Spence and Gallace, 2011). Part of the power of touch is the emotional, visceral (Norman, 2004) value with it has within a design, giving haptics a close relationship with user experience.
In this paper, we use the engineering and human-computer interaction (HCI) definition5 of “haptic” to refer to one or more perceived sensations of touch; this includes tactile and proprioceptive feedback, active human touch, and passive experience of actuated technology.
We define HaXD as
Our focus is on gaining a better understanding of the workflow and processes currently used by hapticians, including those related to integrating haptics into a multisensory experience. We define a haptician as one who is skilled at making haptic sensations, technology, or experiences. We use this term to capture the diversity of people who currently make haptics, and the diversity of their goals. Many people with a need to design haptics may not have formal design training, and may focus on subsets of the entire experience, e.g., technical demonstrations or creating stimuli for psychological tests.The design (planning, development, and evaluation) of user experiences deliberately connecting interactive technology to one or more perceived senses of touch, possibly as part of a multisensory experience.
We describe two studies examining how contemporary hapticians design haptic experiences for use in real-world products. We begin by identifying current obstacles to good HaXD and the target audience for our work, then we provide a roadmap to the rest of the paper. Fig. 1 provides a visual overview of our work.
The academic literature suggests many challenges to design for haptic experience. Haptic content remains scarce and design knowledge is limited. Some issues are technological, such as highly variable hardware platforms and communications latency (Kaaresoja et al., 2014). Other issues are human-centered, arising from individual user characteristics in perception and preferences: low-level perceptual variation (Lofvenberg and Johansson, 1984), responses to programmed (Levesque et al., 2011) and natural (Hollins et al., 2000) textures, sensory declines due to aging (Stevens, 1992, Stevens and Choo, 1996), and varied interpretation and appreciation of haptic effects and sensations (Seifi and MacLean, 2013, Seifi et al., 2015) – often because of personal experience (Schneider and MacLean, 2014), or of the close relationship between touch and other senses.
These research findings are reinforced by many interactions the authors have had with practitioners in industry. Prior to our studies, we suspected that there were many challenges related to haptics, but we found little direct evidence in the literature to back this up and guide our research. As we discovered, hapticians who are free to speak about their work are rare because of intellectual property concerns, which may partially account for the lack of prior work in this area within the literature. Thus motivated, we conducted two studies on the workflows used by hapticians when they were engaged in HaXD – an aspect of design that has been largely unexplored.
In our studies, we take a first in-depth look at haptic designers' experiences in order to describe HaXD, identify its unique challenges, and connect it to other fields of design . We focus specifically on HaXD instead of the more general notion of “haptic design,” which can also refer to design practices related to haptics not directly involving user experience, e.g., mechanical design of a new actuator or software design of a new control method. Our definition of HaXD encompasses pseudo-haptics (Pusch and Lécuyer, 2011) and other multisensory illusions that compel a user to perceive a haptic sensation in the absence of direct tactile or proprioceptive tic stimulation, or modify their perception of one on the basis of conflicting input in another sense. These represent ways in which haptic design must involve other perceptual modalities, alongside direct motivations to create fully multisensory experiences. Much of what we discuss can also be gainfully applied to the design of tangible interfaces, even with their lack of actuation, although we leave them out of our scope to focus on actuated interfaces. Similarly, we believe many of our findings can inform general multisensory experience design, but limit our claims to HaXD.
We primarily target readers who are one step removed from HaXD, but who have other design, haptics, or business expertise relevant to haptics.
We expect that experienced haptic experience design experts will be unsurprised by the insights herein. Although not our primary audience, we hope that the articulated challenges and recommendations will nevertheless still be useful for their practice (particularly for those still early in their learning curve) because it consolidates and extends their ad hoc knowledge into a formal framework.
We expect that non-haptic design experts will find our discussion of the specific challenges to HaXD informative because it reveals processes of design that are invisible or are taken for granted in other fields. We also hope non-haptic designers might lend their expertise to accelerate the generation of tools and techniques for creatively working with these complex interactive systems.
We expect that haptic experts engaged in research or device design that is not directly user-facing will develop a further appreciation for how UX design is important for successful haptic technology, and will see ways in which their devices or research findings can be applied in practice. The recommendations we provide may also motivate several avenues of either basic or applied haptic research that these experts could pursue.
We expect that industry practitioners who are not experts in any of these fields will gain insight into how the business case for haptic technology might be more quickly built. This includes those already involved with haptics or similar technologies such as wearables, as well as those looking to become involved. We believe our findings may help cultivate connections between the diverse stakeholders involved with HaXD, and that the challenges (and thus the opportunities) that we identify will inspire people to work more with this emerging modality and that researchers and practitioners engaged in multisensory HCI will find parallels in their work.
In this paper, we describe two studies in which we sought to gain a solid understanding of HaXD as it is currently practiced “in the wild” by actual practitioners (hapticians) in their day-to-day work. After a review of the existing literature in Section 2 , we report on the first study in Section 3: a grounded theory (Corbin and Strauss, 2008) analysis of intensive interviews with six professional haptic designers. We describe observations of haptic designers' process organized into three cross-cutting themes: the holistic, multisensory, and vertical-integrated nature of the experiences they design; the collaborative ecosystem in which haptic experience designers play multiple roles; and the influences of the cultural contexts in which haptic experiences are used and the value and risk this poses. In Section 4 we describe a second study conducted as part of a workshop at a major international haptics conference (World Haptics 2015). The second study complements the first by collecting quantitative and qualitative feedback from a broader sector of industry and academic designers regarding tool use, collaboration, evaluation methods, and challenges facing hapticians. In Section 5, we summarize and discuss our overall findings in three major areas:
- 1.
A description of current HaXD practice showing how it has already emerged as a distinct field of design.
- 2.
A list of challenges facing haptic experience designers, and some unique considerations HaXD requires compared to other more established fields of design.
- 3.
Recommendations for accelerating the development of HaXD as a full-fledged field of design.
We conclude with a few remarks imagining what a mature discipline of HaXD might look like in the near future and its role within multisensory HCI.
Section snippets
Related work
In this section, we discuss key elements of contemporary thinking about user experience design (UX design or XD) and a specific approach known as “design thinking.” We then briefly review haptic technology (hardware and software) and relevant aspects of human perception before providing a critical summary of previous efforts to understand and support HaXD.
Study 1: interviews with hapticians about HaXD in the Wild
In this section, we present findings from our first study, a qualitative analysis of interviews with six professional hapticians.
Study 2: findings from a follow-up workshop
Our second study was conducted during a workshop on haptic experience design at World Haptics 2015, the largest academic haptics conference to date, held that year in Chicago, IL, USA (http://haptics2015.org).
Discussion
As a first step in further exploring the findings from our two studies, we examine in more detail the critical activities practiced by hapticians. This inventory confirms that HaXD is a field of design with familiar processes, but also one that is developing its own identity distinct from general UX design. We then identify major challenges encountered in HaXD that are unique to or exaggerated when the experiences being designed are haptic. We conclude with several concrete recommendations to
Conclusion
We have provided a first exploration of how haptic experience design (HaXD) is being practiced in industry. We report findings from interviews with six hapticians, finding observations about designer process and themes about the holistic nature of haptic experiences and the collaborative ecosystem and cultural context of our participants. We supplement this with broad follow-up data from a recent workshop at a major haptics conference.
We identified the various activities hapticians practice,
Acknowledgements
We deeply thank our designer participants for their time and dedication, Gordon Minaker for helping with survey distribution and workshop administration, and Hasti Seifi for helping to conduct the workshop brainstorming session. More broadly, we thank generations of our own lab's designers and our collaborators for the foundations that led the way to these findings and ideas. This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), the GRAND
References (94)
- et al.
Hand movements: a window into haptic object recognition
Cogn. Psychol.
(1987) - et al.
Regional differences and interindividual variability in sensitivity to vibration in the glabrous skin of the human hand
Brain Res.
(1984) - et al.
Optical-to-tactile image conversion for the blind
IEEE Trans. Man Machine SystemsMach. Syst.
(1970) - Bucci, P., Cang, X. L., Valair, A., Marino, D., Tseng, L., Jung, M., Rantala, J., Schneider, O., MacLean, K., 2017....
Sketching User Experiences: Getting the Design Right and the Right Design
(2007)Analyse experimentale de quelques elements de la sensation de poids (Experimental analysis of some elements of the sensation of weight)
Archives de Physiologie Normale et PathologiqueArch. De. Physiol. Norm. Et. Pathol.
(1891)- et al.
Vibrotactile display: perception, technology, and applications
Proc. IEEE
(2013) - et al.
Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory
(2008) Designerly Ways of Knowing
(2006)Creativity: Flow and the Psychology of Discovery and Invention
(1996)
Modeling and rendering realistic textures from unconstrained tool-surface interactions
IEEE Trans. haptics
Toward haptic cinematographyenhancing movie experiences with camera-based haptic effects
IEEE Multimed.
The maker movement
Innovations
Groupware: some issues and experiences
Commun. ACM
In touch with the future: The sense of touch from cognitive neuroscience to virtual reality
The interpretation of cultures: Selected essays
Perceptual dimensions of tactile surface texture: a multidimensional scaling analysis
Percept. Psychophys.
Individual differences in perceptual space for tactile textures: evidence from multidimensional scaling
Percept. Psychophys.
Feel effects: enriching storytelling with haptic feedback
Trans. Appl. Percept. (TAP)
Touch matters: exploring the relationship between consumption and tactile interaction
Social. Semiot.
News from the field: courses in haptics
Trans. Haptics
Towards the temporally perfect virtual buttontouch-feedback simultaneity and perceived quality in mobile touchscreen press interactions
ACM Trans. Appl. Percept.
Principles of Neural Science
Exploring tactile movies: an initial tactile glove design and concept evaluation
IEEE Multimed. PP
Haptic perception: a tutorial
Atten., Percept. Psychophys.
Cited by (0)
- 1
Collected workshop data, transcribed interviews, led final qualitative analysis and writing.
- 2
Provided supervision, contributed to research design, writing and ideas.
- 3
Collected interview data and field notes, led initial qualitative analysis, suggested initial ideas.
- 4
Provided supervision, contributed to writing and initial ideas.