1 Introduction

There have been a numerous reports associated with tactile displays as a potential alternative for the Braille system. Tactile displays can transmit various information to its users, including characters and shapes, by presenting physical stimuli to the skin. The need to develop tactile displays stems from the fact that the proportion of visually disabled persons who can read Braille is quite low (only 12.7 %) [1], and it is difficult for people with acquired blindness to learn Braille [2].

With regard to previous studies on tactile displays, in the study of Mizukami et al. [3], an experiment was conducted where participants were given a set of characters by presenting the vibrotactile information perceived from their palm. They reported that the correct response rates for complex characters such as “A” and “F” were approximately 60 %. Mine et al. [4] conducted an experiment where participants were asked to identify hiragana characters by tracing their palm with a touch pen used for smartphones, reporting that there was an apparent trend between certain shapes with a low recognition rate and the wrong answers. They found that the average correct answer rates were 57.79 % among all participants. These studies commonly reported that the correct answer rates for characters presented by tactile display were low; although various approaches have been discussed, no clear solutions regarding the improvement of recognition rates have been found till date.

In this study, we focused on visualization, visually representing internal concepts formulated through the transmission process to tactile information. Wake et al. [5] suggested that visualization was the process from which we generate image-based information obtained through tactile perception in the brain temporarily; people then recognize the information by matching it with a set of long-term memory images. However, it is not clear how accurately tactile images are visualized. Therefore, considering that perceived tactile information is geometrically distorted from its original information, factors adversely affecting the correct recognition of tactile information can be clarified by evaluating the distortion generated through the process of visualization.

Therefore the objective of this study was to empirically examine the accuracy of visualization of tactile information and the existence of a trend in distortion through the process of visualization by directly comparing the participants’ visualized images with their original images.

2 Method

Figure 1 shows the schematic diagram of the apparatus. Compressed air was generated by an air compressor (Hitachi Koki EC1430H2), and the air-jet stimuli were presented by means of a 12 × 12 air-jet array controlled by a precision pressure regulator (CKD RP2000-8-08-G49PBE). The pressure level and airflow duration were controlled by an electro-pneumatic regulator (CKD EVD-1900-P08 SN), and the selector used for determining the location to generate stimuli was controlled by electromagnetic valves (KOGANEI025E1-2).

Fig. 1.
figure 1

Overview of the experimental apparatus

Full size image

In this experiment, tactile stimuli were presented to the middle of the palm. The level of air pressure was set to 100 kPa, and the duration of each air jet emission was 100 ms. Each trial consisted of presenting two line-shaped stimuli. After two sets of line-shaped stimuli were presented to the participants, they were asked to draw two perceived lines on the tablet. Instructions were given to the participants such that the perceived stimuli should be drawn as per what they just perceived. The inter-stimulus interval between two straight lines was 2000 ms. Prior to the stimulus presentation, an auditory signal was given for preparation. Each set included 240 trials; a total of three sets were conducted in three days. A set is divided into three subsets to confirm the existence of a daily-based difference in geometric transformation of tactile information through visualization.

Six right-handed university students (20 to 24 years of age) participated in the experiment.

Owing to the geometrically precise coupling between the palm and the nozzle matrix, the hand location to the nozzle plate was aligned so that all tactile stimuli were properly applied to the surface of the palm. The hand location was then digitized so that the participants could relocate their hands easily at the identical location.

Ear plugs and headphones with white noise had to be worn by the participants to minimize any clues they may obtain from sounds generated by air-jet emission.

Comparisons between the original and the visualized tactile stimuli were calculated by applying affine transformation. Geographical differences between the original and perceived lines were categorized into displacement [px], rotation angles [deg], and scaling rates [-], for further analysis.

The amount of deviation categorized as three characteristics are derived as follows;

displacement t [mm]

$$ t = \sqrt {\left( {M'_{x} - M_{x} } \right)^{2} + \left( {M'_{y} - M_{y} } \right)^{2} } $$
(1)

  • \( M_{x}\,,\,\,\,M_{y} \) Midpoint coordinates of the preceding line stimulus

  • \( M'_{x} ,\,M'_{y} \) Midpoint coordinates of the subsequent line stimulus

rotation angles \( \theta \) [deg]

$$ \cos \theta = \frac{{C_{x} C_{x} ' + C_{y} C_{y} '}}{{\left| {\vec{C}} \right|\left| {\overrightarrow {C '} } \right|}} $$
(2)

\( \theta \) Angle between two lines

scaling rates A [-]

$$ A = \left\{ {\begin{array}{*{20}c} {\frac{C 'L}{CL} ({\text{if}}\,C^{'} L > CL)} \\ {\frac{CL}{C 'L}({\text{if}}\,CL > C^{'} L)} \\ \end{array} } \right. $$
(3)

  • \( CL \) Length of the presented tactile stimuli

  • \( C^{'} L \) Length of the visualized tactile stimuli

Repeated measures of the analysis of variances were conducted for the above parameters which characterized deviations for the independent variables, including the sets and the order of line stimuli. Significant relationships were further analyzed by multiple comparisons. The significance level was set to 0.05 for the comparison.

3 Results

Figures 2, 3, and 4 show the average displacement, rotation angles, and scaling rates for two consecutive stimuli in each set. These figures represent how accurately the tactile stimuli were visualized; note that they do not show how the perceived stimuli deviated from the original stimuli.

Fig. 2.
figure 2

Changes in displacement of visualized stimuli from the given stimuli for two consecutive stimuli. (Color figure online)

Full size image
Fig. 3.
figure 3

Changes in rotation angles of visualized stimuli from the given stimuli for two consecutive stimuli. (Color figure online)

Full size image
Fig. 4.
figure 4

Changes in scaling rates of visualized stimuli from the given stimuli for two consecutive stimuli. (Color figure online)

Full size image

According to Figs. 2, 3, and 4, the participants perceived that preceding stimuli averaging 210.3 px (about 17.5 mm) deviated from the given stimuli and also perceived that the subsequent stimuli averaging 236.6 px (about 19.7 mm) deviated from the given stimuli. Participants perceived the stimuli, which were displaced on an average of 14.5 % in rotation and approximately 28.3 % in average size changes. Further, the only displacement that was significantly different is for the set (F (2, 2) = 729, p< 0.01).

The deviations for each of the three categories were significantly different between the preceding stimulus and the subsequent stimulus (F (1, 2) = 1240, p< 0.01 for displacement, F (1, 2) = 27.1, p< 0.05 for rotation angles, and F (1, 2) = 19.5, p< 0.05 for scaling rates). The result revealed that deviations from the origin were significantly different between the preceding stimulus and the subsequent stimulus. As shown in Figs. 5, 6 and 7, the deviations of subsequent stimuli were larger than those of the preceding stimuli.

Fig. 5.
figure 5

Deviation by the direction shown in pixels [Participant D]. (Color figure online)

Full size image

The deviations, including the direction of the deviation, was further analyzed. Table 1 shows the number of incorrect answers categorized by eight directions of deviation.

Table 1. Number of incorrect answers categorized by the direction of deviation
Full size table

According to Table 1, the number of incorrect answers tended to shift downward for all participants.

Figure 5 shows the deviation by the above eight directions for the performance by participant D, as an example. As seen in Fig. 5, responses were made that deviated downwards; and the deviation was small when they responded upward, in most cases.

4 Discussion

An empirical study to evaluate visualization of tactile information revealed that errors were present between visualized tactile images and the original tactile images. As reported by the process of modality conversion, where participants who responded by drawing with their hands caused the decay of tactile information [6]. In this study, when participants were asked to visualize and draw a set of two straight lines given as tactile information on their palm, the modality conversion occurred twice, such as Tactile → Vision → Movement of the arm in this experiment; thus the possibility of deviation was smaller at the stage of Tactile → Vision conversion.

In addition, displacement, rotation angles, and scaling rates were significantly different following the order of given stimuli. The deviation of the preceding stimuli was smaller than that of the subsequent stimuli, and the accuracy of the tactile information dropped significantly. This loss of accuracy for the subsequent stimuli was interpreted as the participants not being able to memorize the tactile information of the subsequent stimuli accurately enough to represent them on the tablet, due to a lack of available memory resources for the second stimuli.

The number of incorrect answers shifted downward for all participants. In the previous study [7], an experiment was conducted where participants were asked to answer by providing the locations and the directions of stimuli given as tactile information perceived on their palm. They reported that participants perceived the stimulus position shifting downward, which was consistent with this study. Thus the trend to perceive the stimuli downward would be categorized into as one of the conversion characteristics.

It was also revealed that the deviation upwards decreased, whereas the deviation downward increased. According to Kikuchi et al. [8], the psychological effect related to cautious behavior towards actions away from the body was present; therefore participants were, in an involuntary manner, restrained to draw lines at the upper part of the tablet, which was physically away from the body trunk. This yielded trends where many lines tended to be drawn at near to the body, which was the bottom part of the tablet. Such psychological behaviors may have resulted in suppressed deviation for upward movement.

5 Conclusion

In this study, participants were asked to draw a set of two straight lines given as tactile information perceived on their palm. Geographical differences between the original and perceived lines categorized as displacement, angle of rotation, and the rate of scaling were analyzed by applying affine transformation. As a result, two lines were compared with respect to deviations from the original lines; it was revealed that the participants drew significantly more deviated lines when the subsequent line was given. Participants also tended to draw lines downwards.

Subsequent lines were drawn rather differently from the original line owing to the limitation of short-term memory. It was interpreted that participants could not memorize the geographical information of the subsequent lines accurately enough to represent them on the tablet due to a lack of available memory resources for the second lines. The tendency to deviate the lines downwards was presumably due to the psychological effect related to cautious behavior towards actions away from their bodies, i.e., participants were, in an involuntary manner, restrained to draw lines at the upper part of the tablet, which was physically away from the body trunk, yielding trends where many lines tended to be drawn near the body.