1 Introduction

Advancements in touchscreen-based computing devices have amplified our reliance on digital information. Much of this information is based on graphical representations rather than textual content. This has resulted in a significant information access challenge for blind and visually-impaired (BVI) people as there is no commercial solution providing non-visual access to graphical materials. Several researchers and information-access technology (IAT) developers are utilizing touchscreen-based smart devices to address the non-visual graphics accessibility issue, as these solutions offer a multimodal interface based on a commercially available, inexpensive platform incorporating many native universal design and accessibility features (e.g., Voiceover for iOS or TalkBack for Android) [1]. These approaches provide access to on-screen graphical information via auditory [2, 3], vibratory [4,5,6], electrostatic [7], or combinations of one or more of these information sources [8,9,10]. While these approaches are promising, they also offer unique and novel challenges due to the limitations imposed by the touchscreen hardware as well as by the way the on-screen graphical information is accessed via non-visual haptic perception.

Perceiving digital graphical information through vibrotactile stimulation on a touchscreen display is very different from perceiving the same graphical information with vision or perceiving traditional tangible media (e.g., raised line drawings, tactile maps, etc.). With physical tangible media, users can directly touch and perceive the line stimuli with changes in force, friction, and pressure during finger/hand movement leading to skin deformation that innervates mechanoreceptors on the fingertip upon contact with the stimuli (see Fig. 1). Similarly, with a touchscreen-based visual interface, sighted users can perceive the stimuli using various visual cues such as the color of the line, its spatial position, its spatial structure, and angle subtended with respect to the visual axis. By contrast, with a touchscreen-based non-visual interface, the user can only perceive a flat, featureless glass screen that conveys no meaningful tactual information/cutaneous reinforcement, as the stimuli in isolation does not possess any physical attributes that are directly perceivable by the finger. Therefore, haptic interactions must rely on extrinsic feedback such as vibration to indicate contact with an on-screen graphical element. Since the device’s hardware is equipped with only one vibration motor, which vibrates the entire device when triggered, users must employ only one finger to access and extract information. The result is that the focal vibration on the finger touching the display is perceived as a tactile graphical element on the screen. While the extrinsic feedback can indicate contact with on-screen elements, such feedback (in isolation) does not provide any meaningful tactual information, such as the width/height of an element. As a result, it is much more difficult to haptically distinguish fine detail and precise spatial information on a touchscreen using vibrotactile cuing that would otherwise be easily discernible from physical access using tangible graphics or from visual access to the same graphical information presented on touchscreen displays. To tackle these differences imposed by haptic information extraction and to develop truly useful touchscreen-based haptic applications, new approaches must be introduced that go beyond the naïve technique of simply trying to implement a one-to-one haptic analog of the visual graphical rendering on the touchscreen. To be successful, a principled conversion and schematization process of the underlying graphical information must be carried out to optimize effective visual to haptic sensory substitution supporting accurate vibrotactile information extraction [11]. There is an existing body of research based on traditional tangible media that has identified and established perceptual parameters and design guidelines for performing this visual-to-tactile conversion/optimization process for graphical information [12,13,14]. However, these results are limited to studies with tangible media and cannot be applied to extraction and perception of dynamic vibrotactile stimulation from touchscreen-based interfaces due to the previously discussed differences in haptic information extraction and the extrinsic cuing mechanism required for touchscreen-based vibrotactile stimuli. To our knowledge, there are no empirical guidelines and parameters governing the conversion of visual graphical information into haptically perceivable vibrotactile information delivered via commercial touchscreens. This paper builds on a series of studies conducted in the VEMI Lab at the University of Maine aimed at addressing this gap in the literature by developing a set of theoretically-motivated and empirically-validated guidelines for use of vibrotactile stimuli as part of a robust touchscreen-based information access solution. By extension, this work also provides foundational design guidelines that address the long-standing challenge of providing blind and visually-impaired (BVI) people with meaningful access to digital graphical materials.

Fig. 1.
figure 1

Perceptual differences between tangible media and touchscreen displays

Full size image

2 Current Research

The current work is part of a larger corpus of research aimed at empirically evaluating and identifying a set of core nonvisual rendering parameters through a series of psychophysically-inspired usability studies. We posit that, once established, these core usability parameters will serve as a set of much-needed de-facto guidelines specifying the best techniques for accurate rendering and haptic perception of graphical materials via commercial touchscreen-based smart devices (e.g., smartphones and tablets). This paper builds on the findings from four earlier studies [15], which established four key usability parameters, namely:

  1. (1)

    Graphical elements must be rendered at a width of at least 1 mm for tasks requiring simple detection of graphical elements when using vibrotactile feedback on touchscreen displays,

  2. (2)

    A minimum gap width of 4 mm must be maintained for identifying each unique graphical element and accurately detecting gaps between adjacent elements. In addition to the 4 mm minimum gap width, the lines (e.g., borders of the element) must be rendered at a width greater than 2 mm for supporting discrimination of adjacent elements.

  3. (3)

    For tasks requiring accurate orientation judgments of line segments (e.g., paths on a map) using vibrotactile feedback, the line elements must be rendered at a minimum width of at least 2 mm, and

  4. (4)

    For tasks requiring accurate line tracing and learning of multi-line spatial patterns using vibrotactile feedback (e.g., subway maps, road networks, or corridor layouts), the line elements must be rendered at a width of at least 3 mm. (see [15] for details and discussion on the parameters)

Building on these findings, the current research was conducted to empirically identify the minimum angular magnitude that must be maintained for accurate detection and angular judgment of a range of oriented vibrotactile lines. Whether it is a simple line graph or a complex map, the ability to accurately identify an angled line and judging the angle it is subtending with respect to an adjacent line is crucial for extracting information from the graphical material. Consider, for example, a simple corridor map of a shopping mall as shown in Fig. 2. Each of the three corridors are diverging from one vertex and are oriented at different angles. Understanding this layout is a pre-requisite for developing an accurate cognitive map and being able to efficiently navigate within this environment (e.g., way-finding from Macy’s to Sears). For a non-visual interface to effectively communicate this information, the rendering must support users in accurately judging the angle subtended between corridors and the angle subtended by each corridor with respect to some frame of reference (e.g., the frame of the device). As stated earlier, perceiving digital graphical information via vibrotactile feedback on touchscreen-based devices is difficult due to the sparse spatial resolution of touch as well as the extrinsic feedback mechanism. This means the corridors must not only be rendered at a width perceivable by touch but also must be separated by a minimum angular magnitude that allows users to always distinguish one corridor from another.

Fig. 2.
figure 2

Indoor corridor layout of a shopping mall.

Full size image

2.1 Preliminary Studies on Angle Perception and Orientation Judgment

The importance of being able to judge line-orientations has been extensively described in the psychophysical literature with both vision and touch [16, 17]. These studies have shown that blindfolded-sighted people are more accurate when predicting vertical or horizontal orientations over obliquely oriented stimuli. Although formal research has not been conducted on orientation judgments based on active exploration of vibrotactile lines, user feedback and informal observations from earlier studies in our lab revealed that participants found it difficult to trace lines and detect their orientation when they deviated from horizontal or vertical orientations [4, 9, 18]. To investigate whether users were able to judge orientation and perceive angular magnitude between vibrotactile lines, two preliminary studies were conducted. The first study compared performance in a task where blindfolded-sighted participants had to explore and identify the angular magnitude across two touchscreen-based non-visual interfaces (i.e., vibrotactile and electrostatic). Five angle stimuli were generated for each display used in the study, covering a range from near horizontal right to near horizontal left, comprising 25°, 70°, 90°, 125°, and 155° [19]. This study showed that the vibrotactile interface exhibited superior performance over the electrostatic interface and that users were able to accurately identify the angles subtended between two vibrotactile lines with a mean signed error of 0.3° (s.e.m. 1.4°). In a second study, we investigated users’ ability to judge vibrotactile line orientations across 36 angles and three different line widths. The study showed that participants were able to accurately judge vibrotactile line-orientations and that a line width of 2 mm or more must be maintained for efficient tracing and learning of vibrotactile lines [15]. While the findings from these two studies show evidence that users can accurately judge angles subtended between two vibrotactile lines, they do not provide any guidance on the perceptual limitation of detecting angular magnitude (i.e., the minimum perceivable angle between two vibrotactile lines). Identifying this angular threshold and rendering graphical material accordingly is essential for supporting accurate detection of distinct vibrotactile lines that are connected at an intersection (e.g., the intersection shown in Fig. 2). To our knowledge, there is no empirical data from the literature on the minimum angular magnitude that ensures detection of distinct vibrotactile lines. To address this gap in the literature, we designed a psychophysically-inspired usability study aimed at answering the research question: “What is the minimum angular magnitude that best supports the detection of oriented vibrotactile lines on touchscreen interfaces?”.

3 Evaluation of Minimum Perceivable Angular Magnitude

As stated earlier, with the extrinsic cuing mechanism employed on touchscreen devices, users can only detect whether the touched location is on or off of an on-screen graphical element but they cannot directly perceive any other meaningful information such as width/length/angle. For example, consider the triangle in Fig. 3 (right). Based on static contact, the user is able to detect whether they are touching a part of the triangle but they are not able to discern any other meaningful information such as number of edges, length of each edge, angle between two edges, etc. To extract such detailed information, users must actively explore the stimuli by employing finger movements and accurate tracking of proprioceptive information. Because of this basic difference in tactual perception between information rendered on touchscreens vs. tangible media, traditional static psychophysical methods (i.e., measuring perception via direct skin innervation) cannot be utilized for measuring the minimum perceivable angular magnitude on touchscreen-based interfaces, as the contact finger does not receive any meaningful cutaneous sensation as one would receive from tangible media. As stated earlier, the challenge of vibrotactile exploration and tactual learning is further aggravated by technical constraints imposed by touchscreen displays, which typically limit the user to employ only one finger for exploration. This means that users cannot simply maintain static contact with the stimuli to extract meaningful information but must perform exploratory procedures (Eps), which are a stereotyped pattern of manual exploration observed when people are asked to learn about a particular object property during voluntary manual exploration without vision [20]. In contrast to traditional Eps, which generally involve use of all fingers on one or both hands, exploratory procedures with touchscreen-based vibrotactile stimuli must be done using only one finger and involve sequential apprehension/integration of the different graphical elements to develop a coherent mental representation. Germane to the current experiment, for identifying oriented lines and judging the angle subtended between them, we have found that users typically employ a ‘circling’ strategy, where they move their finger in a circular pattern around the intersection (see Fig. 3 (left)) as their exploratory procedure to most accurately identify the geometry and number of legs [8, 18, 21].

Fig. 3.
figure 3

(Left) Intersection circling strategy: adapted from [21], (Right) Geometric representation of cord length ‘c’ and radius ‘r’

Full size image

Based on this exploration strategy, we posit here that the arc of the circle formed between two oriented vibrotactile lines will be perceived by the user as the angular magnitude subtended between the two lines. To be recognized as a distinct vibrotactile line, each of the lines emanating from the intersection must be separated from each other by a minimum perceivable angular magnitude. As stated earlier, our previous work established that a minimum gap of 4 mm must be maintained between adjacent lines for accurate detection of parallel vibrotactile lines. From a geometric standpoint, the straight-line distance between two angled lines is the cord length (see angle-theta and cord length in Fig. 3(right)). The cord length will linearly increase with a corresponding increase in the: (1) θ – angle subtended between the lines, (2) r – the radius of the traced circle, or (3) both 1 and 2. This means that the minimum gap of 4 mm that we have previously identified for detecting two parallel lines [15] should, in theory, be translated into a 4 mm cord length for accurate detection of distinct oriented lines. However, unlike simple gap width, cord length is a variable that is directly proportional to both angle and radius (i.e., an increase in angle or radius leads to a corresponding increase in the cord length). The relation between the three variables is mathematically defined as: cord length = 2r sin (θ/2). This means that the cord length is directly dependent on the radius of the circle formed by the user while performing their circling exploration strategy. For instance, an angle of 5° will lead to a 4 mm cord length with a 1-in. radius circle, and an angle of 2° will lead to a 4 mm cord length with a 2-in. radius circle. Since our interest in this experiment is on identifying the minimum perceivable angle (θ) by varying the cord length, the radius (r) will be kept constant at two levels (i.e., 1-in. and 2-in.).

3.1 Method

Participants.

Eighteen blindfolded-sighted participants (nine males and nine females, ages 19–34) were recruited for the study. All gave informed consent and were paid for their participation. The study was approved by the Institutional Review Board (IRB) of the University of Maine. It is important to note that use of blindfolded-sighted participants was intentional, as although under-studied, sighted individuals can also benefit from haptic information access in eyes-free situations (e.g., Performing a secondary task while driving) and we believe that our interface has significant untapped value in such situations. With respect to traditional information-access technology design, inclusion of blindfolded-sighted participants is widely accepted in the preliminary testing of assistive technology (see [22] for discussion). Furthermore, the graphical information studied here is equally accessible to both groups, a supposition empirically corroborated by our previous studies on touchscreen-based interfaces showing no reliable differences between blindfolded-sighted and blind and visually-impaired participants [4, 23, 24].

Stimuli and Conditions.

The stimulus set was designed as a simple indoor corridor layout (e.g., Shopping mall) where multiple corridors were converging to/diverging from an intersection point at the center (Fig. 4). The number of corridors in each stimuli ranged from 5 to 9 based on Miller’s “The Magical Number Seven, Plus or Minus Two” [25]. To evaluate the influence of radius over perception of oriented lines, two conditions were designed and evaluated. The radius was set as a constant value of 1-in. and 2-in. for conditions 1 and 2 respectively. At a radius of 1-in. from the intersection, the minimum gap width of 4 mm (i.e., cord length in this context) was translated to an angular magnitude of ~9°. Similarly, at a 2-in. radius, the gap width of 4 mm width was translated to a ~5° angular magnitude. To evaluate the influence of cord length (i.e., gap) on the perception of oriented lines, two additional angles (2° and 22°) were also added to the stimuli set that approximately translated to the 4 mm gap width at a radius of 0.5-in. and 4-in. (meaning the radius of the two primary conditions increased and decreased by a factor of 2).

Fig. 4.
figure 4

Experimental angle stimuli 22°, 9°, 5°, 2° from left to right

Full size image

Apparatus.

The stimuli were presented using our prototype, called a vibro-audio interface (VAI) implemented on a touchscreen equipped tablet computer - 10.1 in. Galaxy Tab 3. The interface works by allowing users to freely explore the device screen and whenever an onscreen element is touched, the device’s vibration motor is triggered, creating the perception of focal vibrotactile stimulation on the users finger (more details can be found in [4]). For controlling the circle radius in each condition and for assisting users with the circling strategy, two circular paper stickers of 4 mm width (one at 1-in. from the center and the other at 2-in. from the center) were affixed on the screen (see Fig. 5 (right)). In addition, the intersection point (center of the screen) was also demarcated with a paper sticker of 10 mm radius. To assist participants with orienting themselves on the screen, each circle had a start point (indicated by a tactile marker at the 5 o’clock position).

Fig. 5.
figure 5

(Left) Dimensions of the two conditions and their tracing radius, (right) experimental device with circular stickers for two radiuses and tactile markers for start points

Full size image

Procedure and Design.

The study followed a within-subjects design. A trial rendered 5, 6, 7, 8, or 9 lines on the screen (for example see Fig. 4). In each trial, the angular magnitude between adjacent lines was kept constant irrespective of line number. The order of the conditions (1-in. versus 2-in. radius) was balanced across the participants and the order of stimuli presentation in each condition was randomized within the script. In each trial, participants were asked to start at the reference start point (indicated by a tactile marker) and to count the number of lines perceived in a full 360° circuit by tracing along the circular path (either at 1-in. or 2-in. radius depending on the condition). Upon returning to the start point, they lifted their finger from the display and verbally indicated the number of lines perceived during the 360° scan. In each condition, participants began with 5 practice trials where the experimenter gave corrective feedback with respect to their tracing speed and counting accuracy. They then moved on to the 28 experimental trials in each condition (resulting in 180 observations for each tested angular magnitude). Each participant took between 20 and 40 min to complete the entire experiment.

Experimental Measures.

Based on this design, two measures were compared across the four tested angular magnitudes and two circling conditions.

  • 1. Tracing time: The tracing time is the time taken in each trial from the moment they first touched the reference start point until they returned to the same point after scanning along the circle.

  • 2. Line counting accuracy: Accuracy in line counting was measured based on correctness of line count as self-reported by participants in each trial.

4 Results and Discussion

ANOVA results revealed that in both conditions, the tracing time did not statistically differ between the four tested angles. The f and p values are as follows,

$$ For\;the\;1{\text{-}}inch\;circular\;path,\;{\text{F}}\left( {3,\,500} \right) = 1.043,\;{\text{p}} > 0.05,\;\upeta^{2} = 0.006 $$
$$ For\;the\;2{\text{-}}inch\;circular\;path,\;{\text{F}}\left( {3,\,500} \right) = 1.145,\;{\text{p}} > 0.05,\;\upeta^{2} = 0.006 $$

A one-way ANOVA revealed that in both conditions, the accuracy in line counting was significantly different between the four tested angles. The f and p values are as follows (Fig. 6),

Fig. 6.
figure 6

(Left) Mean tracing time as a function of tested angles and two circling conditions. (Right) Mean error in line counting accuracy as a function of tested angles and two circling conditions

Full size image
$$ For\;the\;1{\text{-}}inch\;circular\;path,\;{\text{F}}\left( {3,\,500} \right) = 14.147,\;{\text{p}} > 0.001,\;\upeta^{2} = 0.019 $$
$$ For\;the\;2{\text{-}}inch\;circular\;path,\;{\text{F}}\left( {3,\,500} \right) = 12.559,\;{\text{p}} > 0.001,\;\upeta^{2} = 0.070 $$

Post-hoc t-tests with Bonferroni correction revealed that the difference in line counting accuracy between observations with a 2° angle compared to the other three angles was significant (p < 0.001). But there was no significant differences between the other angles (5°, 9°, and 22°). This finding indicates that a 4 mm cord length is sufficient to accurately detect distinct oriented vibrotactile lines when using a circling strategy. This parameter is in line with our previous research that also established 4 mm as the gap width for accurate detection of distinct vibrotactile lines that are parallel to each other [15].

On comparing the tracing time between 1-in. and 2-in. circling conditions, a post-hoc paired sample t-Test revealed that the tracing time for the 1-in. radius circle was significantly faster than the tracing time for the 2-in. radius circle (T(503) = −8.060, p < 0.001). This outcome is not surprising as the tracing time is directly proportional to the perimeter of the circle (i.e., the distance they traced) and the 1-in. circle condition has half the perimeter length as the 2-in. circle condition. Similarly, the accuracy in line counting was also significantly different between the two conditions with the 2-in. radius circle condition exhibiting higher accuracy, (T(503) = 6.243, p < 0.001). This finding is also expected as the cord length increases with the corresponding increase in radius, which resulted in a higher chance of line detection for the 2-in. condition than 1-in. condition.

5 Conclusions

This paper investigated the minimum perceivable angular magnitude that is necessary for detecting oriented vibrotactile lines emanating from a common intersection. The work presented here is part of a larger research program aimed at establishing the core usability parameters and design guidelines for governing the conversion of visual graphical information into a haptically perceivable format rendered using touchscreen-based interfaces. We postulated that to accurately detect distinct oriented vibrotactile lines, the spacing between two adjacent lines (i.e., the cord length) must be maintained at a minimum length such that users can accurately detect distinct vibrotactile lines converging or diverging from a common intersection point. As stated earlier, the cord length (and by extension the minimum perceivable angular magnitude) is a variable that is dependent on both the angle subtended between oriented lines and the radius of the circle formed by the user during their exploratory procedure when apprehending the vertex/intersection of these lines. To evaluate the minimum cord length and to assess the influence of the angle and radius on this cord length, accuracy in oriented line detection was compared across four angles (2°, 5°, 9°, and 22°) and two radiuses (1-in. and 2-in.).

The most important outcome of the study is the similarity in perceptual characteristics between parallel lines and oriented lines. The minimum value threshold of a 4 mm cord length for oriented lines established here is congruent with the minimum gap width of 4 mm we previously established for detecting parallel vibrotactile lines [15]. On comparing the two radiuses/conditions, it is evident that the line detection accuracy proportionally increases with an increase in angle magnitude (θ), and/or the radius (r). This validates our hypothesis that the cord length is a variable that depends on two other parameters (i.e., angle and radius) and that the parameters must be manipulated accordingly to maintain a minimum cord length of 4 mm. This suggests that when designing or rendering graphical materials (or converting from a corresponding visual rendering), designers must understand this dependency between angle, radius, and cord length and schematize the angular elements by calculating the minimum perceivable angle (using the formula: θ = 2 arcsin (cord length/2r)) based on the minimum 4 mm cord length. While traditional visual-to-tactile conversion methods generally adopt an 8-sector (45° interval) or 16-sector (22.5° interval) model for schematizing oriented lines [26], the results here clearly suggest that simply relying on angular magnitude will not be sufficient for ensuring accurate haptic perception of oriented vibrotactile lines on touchscreen displays when rendering graphical elements. This difference relates to the nature of haptic perception between these stimuli. That is, with tangible raised stimuli, users can directly perceive fine spatial details via skin deformation that innervates mechanoreceptors even with static finger contact. However, with touchscreen-based vibrotactile cuing, users must perform active exploration using just one finger to extract/perceive these attributes, movement that requires spatial extent and thus mandates incorporation of additional spacing between oriented vibrotactile lines on the display. To produce accurate and efficient vibrotactile renderings, this research demonstrates that designers must consider this difference in stimulus/perceptual coupling. Specifically, when designing/rendering oriented vibrotactile graphical elements on touchscreen-based displays, accurate haptic perception requires considering the relation between the angle (θ), the radius (r), and cord length, rather than adopting traditional parameters/models that are based only on the angular magnitude (i.e., 45°, 30°, or 22.5°).

Caveats are needed before generically adopting this 4 mm cord length threshold, as this value is based on just one exploratory procedure (i.e., circling around an intersection or vertex). Future research will address this limitation and generalize the identified value for different graphical materials (e.g., road networks, edges of a pie chart, building layout maps, etc.,) We will also investigate other exploratory strategies such as four directional scanning, where users start at the intersection/vertex and move their finger in cardinal directions (i.e., north, east, south, and west).

In sum, findings from this work provide foundational guidelines for converting/rendering angular elements and oriented lines on touchscreen-based interfaces for supporting vibrotactile haptic information access. Combining the cord length parameter identified here with the four parameters established from our earlier research (discussed in Sect. 2), we continue to build on our goal of developing a robust set of usability and design guidelines for rendering a wide range of haptically perceivable graphical information on touchscreen displays.