Keywords

1 Introduction

Small mobile devices have been integrated into the lives of most people. Undoubtedly they have become a kind of necessities, and among them the smart phones are the most obvious examples. According to the surveys, the global penetration rate of smart phones has exceeded 62% in 2015. The most commonly used features for surfing the internet with smartphones are “social network visits” (71%), “search engine uses” (64%), “map functions” (60%) [1]. The above-mentioned functions all require text entry, thus it stands for a fact that “text entry” plays an important role in the uses of mobile devices. However, the “lack of realistic tactile feedback” on the touch screens and “the screen sizes would limit the key sizes” have not been fully overcome yet.

Owing to the maturity achieved in the developments of mobile devices and the various hardware and software, their functions and applications are being enhanced day after day, therefore the mobile devices with large screen sizes are increasingly favored by consumers and have gradually become the trend [2, 3]. The larger the screens of the devices, the more information can be displayed, and that would enlarge the virtual keyboards‘sizes and make text entry even more comfortable. However, even if it is a 5.5-inch screen, the key sizes of the virtual keyboard is still subject to great restrictions. For instance, the iPhone 6 s Plus’s virtual keyboard key size is about 5.8 × 7.5 mm only. Those figures are still below the recommendations made in the iOS Human Interface Guidelines [4]; that is, the key sizes for App designs should not be less than 6.74 x 6.74 mm. Thus so far it still deserves our efforts for discussing and studies about the operation habits of the keyboards and how to improve the efficiency of text entry through the virtual keyboards.

Therefore, this research aims at “the relationships between finger movement speeds, distances, directions and accuracies while making continuous data entry inside the keyboard areas of the mobile devices”. The users’ operation behaviors are different when they tap on the mobile devices to perform “text entry” and “non-text entry”. When a “non-text entry” is performed, the user moves his finger to the targeted key (icon) and taps, and the task is completed after repeating several times of such behaviors. If we take taking a picture for example, we tap the camera key firstly, tap on the function keys we want to use (for example: taking picture, video recording, special effects…), and then we press on the shutter key to complete the task. As for “text entry”, although we tap on the targeted keys similarly, we have to quickly, repeatedly and continually move our fingers to tap and select the targets. Such behavior differences would affect the design specifications and research contents of the ordinary buttons and keyboard keys.

The importance of text entry on small mobile devices goes without saying owing to the mobile devices’ becoming popular, but the efficiency of text entry is not as good as traditional physical keyboards [5]. The screen space constraint which makes the keys too small is one of the important reasons lowering down the entry efficiency [6]. Therefore, this research intends to discuss in depth on the behaviors, that is, the finger movement speeds, distances, directions, and accuracies of text entry on mobile devices when we continuously and rapidly make inputs into the panel area of a small mobile device. We anticipate to find out the problems through the understanding of the entry behaviors, so as to improve the efficiency of text entry.

2 Related Work

2.1 Fitts’s Law

Fitts’s [7] law has provided a very useful norm for tapping targets by using pointing devices on traditional screens. Fitts’s law states that the smaller the selection target and the farther it moves, the longer time it would take to reach the target point. In addition, in the case of speed-accuracy tradeoff, the faster the moving speeds, the smaller the target sizes, and the higher the error rates would be. Concerning the study of Fitts’s law [7] and finger touch, Lee and Zhai [8] have studied the use of virtual keys and provided specifications for the use of touch screens, and their researches have also shown that Fitts’s Law is not fully applicable on small mobile devices. Later on, some researchers also modified and extended it based on Fitts’s law and proposed the FFitts Law model [9].

Although the guidelines which the Fitts’s Law has provided are not entirely in line with the relations between target and selection of the current small mobile devices, the theory and core, in other words the corresponding relationships between target sizes, moving distances, speeds and accuracies if there exist any behavior differences between ways of text entry and ordinary icon taping are very valuable for reference that they worth applications and in-depth researches.

2.2 Target Size

There have been extensive discussions on the sizes of touch targets, and there exist absolute correlations on the entry error rate values if the target is too small [6]. Too-small keys would increase the entry time and entry error rate [10]. Different scholars have different opinions on the sizes of the touch screen keys. The traditional key size is 22 mm [8], but there are other studies which claim that the keys can be smaller, for example 11.5 mm [11], 10.5 mm [12], 9 mm [13], and iOS Human Interface Guidelines 2010 [4] has suggested that the keys or target sizes should not be smaller than 6.74 × 6.74 mm while designing an App. At present the keyboard keys on the screen of a small mobile devices are too small. For example, the English keyboard keys of the iPhone 5 are only about 6 mm × 3.9 mm in size, thereby increasing the entry difficulty and reducing the accuracy. But the typing performance on a very small device could be also very well [14].

2.3 Text Entry Performance

Both the indicators of entry speed and error rate have been used to benchmark an input device often [15]. The entry speed measurements can be expressed in characters per second (CPS) or words per minute (WPM), and among them the WPM has been more commonly used. In the WPM calculation rules, a “word” has been defined as “five keystrokes,” meaning that it would be calculated as a “word” as long as the keyboard has been tapped five times rather than denoting a meaningful word. These five strokes may contain all letters, numbers, blank keys, punctuation, backspace keys, etc. [15, 16]. However, scholars of both Arif and Stuerzlinger [17] have further argued that the calculation formula of WPM should be slightly adjusted. They think that the correct total number of keystrokes is the total number of keystrokes minus one, as timing usually begins after the first key is tapped. Currently the entry speeds on small mobile devices range from about 35 to 50 WPM, depending on the mode of operation [18, 19].

Comparing with the measurements of entry speeds, the measurement error rates at data entry are comparatively complicated, because the causes of the occurrences and the calculation measures are very diverse. The entry errors can be divided into four categories according to their attributes: substitution, omission, insertion, and transposition, and the numbers of corrective actions required for different situations are also different [20]. The method which is more commonly known for measurement error rates is to calculate the ratio between number of keystrokes and characters (Key Strokes per Character, KSPC), i.e. actual strokes/expected strokes [17, 21].

Offsets for touch points.

Henze and others have designed a target selection game to collect a large number of touch-point data. They found that when a user selects a target on a mobile device, the occurrence of wrong selections may be affected by the positions and sizes of the keys, and the preference of the touch points also has a tendency of concentration. When taping on the touch screen target, offset conditions have happened quite often. The point which the user gazes at is not consistent with the actual touch point detected on the touch screen [12]. Vogel and Baudisch both argued that the offset can be ascribed to: firstly, the target is obscured by the finger, and secondly, which portion of the finger is to be used for touching the screen is vague and difficult to control [12]. Holz and Baudisch [22, 23] have further explored the ways the fingers touch the screens and pointed out that different users, tilt angles of the fingers, and the rotation angles of the fingers all may cause certain offsets between the gazing points and the touch points, and they stressed that the offsets are the main reasons why the touch screen entry measure is not accurate enough. In addition, when the data entry is made with one thumb, the offset issue would be more serious when the user taps the left and right keys [24, 25].

On the touch screen keyboards, both Azenkot and Zhai [18] have tangibly summarized the offset conditions of each key, and the results are as follows: 1. The actual touch points in most of the cases would be located on the lower portion of the center position 2. The vertical offsets are obvious if the users only use right index fingers to operate 3. Only use right index fingers or right thumbs to operate would result in offsets to the right side 4. The blank key has obvious offset to the right side. The study has showed that even if the keyboard keys have been shifted, the touch-points still could distribute throughout all the positions of the keys.

Based on the above literature about data entry performance, we believe that text entry is of very fast and continuous actions which is different from the non-text data entry. This study would consider the Fitts’s law, key sizes, input offsets and other factors to discuss about the relationships between movement distances, speeds and accuracies while quickly and continually taping on the targets inside the panel areas in order to understand the text entry behaviors and thereby enhance efficiency.

3 Method

3.1 App Design for Usability Test

This research has designed a taping-purpose App to observe, record and analyze the various data and behaviors of the subjects while they taped on the touch panels. We got to know from the literature that the general test methods for text entry efficiency are to have the subjects to input some series of sentences and then calculate the entry speeds and error rates. Under such experimental methods the subjects must look at the title articles first, look at the keyboards to find the correct key positions, and finally watch the entry fields to make sure correct data inputs are made. Hence, the subjects must constantly switch their viewpoints between the subjects, input fields and keyboard fields. Such method not only increases the workloads of the subjects while making data entry, may increase error rates, and on the other hand such measure does not correspond to the real circumstances when the users are making data entry. Because under normal circumstances for data entry the users know what they want to input that their viewpoints do not need to have much switching between input fields, titles and keyboards, and they just need to concentrate on the keyboards searching for the target keys instead. The purpose of the experimental tests is to find out whether the speeds, distances, and directions of the finger movements would affect the taping points. To achieve this purpose, there is no need to test with meaningful texts or sentences, therefore the keyboards used in the experiments would rid off letter symbols and only the blank keys are left for the testing (Fig. 1).

Fig. 1.
figure 1

Experimental keyboard

Full size image

The App operation is very simple, the keys on the keyboard would be lit randomly, and the subjects must tap sequentially. However, in the real text entry scenario the users know the positions of the next keys beforehand; for example, when a user wants to input “key”, he already knows that the next key is the “e” key when he touches the first letter “k”, and then the “y” key that he could be psychologically and motionally prepared. In order to simulate the actual text entry scenario, three keys would be lit randomly, and the numbers “1”, “2” and “3” are respectively displayed in red. The subjects tap according to the numerical sequence, and then another 3 keys individually displaying “A”, “B”, “C” would be lit randomly in blue color after the 2nd key has been taped. The subjects must tap on the numbers in sequence and then tap on the English letters, and the targeted numbers or letters taped would disappear. They would repeat in the same way until the end of the experiment (Fig. 2). In this way, a subject can know the next key and its position to be taped by using peripheral vision or screen scanning, and to proceed with the test process smoother. Such practice is more similar to the typical way of text entry operation.

Fig. 2.
figure 2

Descriptions of entry test keys (color figure online)

Full size image

3.2 Participants

The personal characteristics of the subjects, such as age, gender, dominant hand, familiarity with the operations of the touch devices, etc. may affect the data entry efficiency. Therefore, some constraints must be imposed on the subjects for participating the experiments.

According to the surveys, in Taiwan, the smart phones have a penetration rate of 96% between the ages of 25 and 30, and as high as 95% under the age of 25 (Google, 2015). Therefore, we have chosen 20–30-year-old smart phone users as the experiment subjects. In order to avoid impact from being unfamiliar with the touch screen operations, the subjects must have more than six months of experiences using smart phones. In addition, in order to simplify the experiments, the right-handed users would be invited as the main exploring objects of this study, and they must be able to make text entry by using their right thumbs. Thirty subjects have finally been invited to participate in the experiments, male half and female half for conducting the experiments.

3.3 Collecting Data

The related taping data were recorded via the experimental App, and the data would be stored in the back-end database. The stored data have included: keys taped, taping sequence, touch point coordinates of each taping, each moving distance, each moving speed, each moving direction, distances of touch points to the key centers/offset key centers, entry speeds (WPM), error rates, etc. For moving directions, the top/down and left/right displacements are recorded respectively. As to the center points, the “key center” and “offset key center” data have been recorded at the same time. The term “key center” refers to the center of a key that its position remains unchanged. As for the “offset key center” positions, they could only be obtained after we finished the experiments of each individual subject, figuring out the average position coordinates statistically. Therefore, the “offset key center” of each key varied from person to person, but the offset key center could be regarded as the relative center of all touch points.

The key design of this experiment App has included both visual target (48.48 × 31.03 pixel) and actual target (72.72 × 46.55 pixel), so as to eliminate the condition that any subject mis-identifies or touches the target key by mistake unreasonably. For example, if the target key is the leftmost Q key while the L key which is the rightmost on the screen has been touched by mistake somehow. This type of mistaken entry has nothing to do with the research project, would cause big deviation to seriously affect the experimental results, and thus such data must be deleted before proceeding with the analysis.

3.4 Experiment

This study uses the usability test as the primary research method. The subjects have been placed in a quiet lab to proceed with the experiments, and the entire process has been recorded by video. The iPhone 6S mobile phones have been used as experimental devices with screen resolution of 1334 × 750 pixels, 326 ppi, while the App used for the experiments has configured the screen coordinates to 375 pixels along the horizontal length and 667 pixels along the vertical length, with the upper left corner of the screen as the origin.

In the experiments the right-hand thumbs were used for data entry. The subjects were allowed to practice for two sessions to get familiar with the experimental App prior to the experiments, and there followed with one session of testing as the experiments.

4 Results

This research has collected experimental data of 30 subjects with a total of 3194 valid taping data, average entry speed (WPM) of 22.54 and an error rate of 0.92%. Figure 3 shows the distribution of all the touch points and their 95% confidence ellipse. The data have been used by descriptive statistics, analysis of variance, regression analysis for post analysis, and to proceed with the discussions about the data entry impacts from the directions, distances and speeds of the finger movements.

Fig. 3.
figure 3

Touch points distribution status (left) and 95% confidence ellipse (right)

Full size image

4.1 Data Entry Impacts from the Fingers’ Moving Directions

As shown in Table 1 for the overall movement directions, the proportions of left-movement/right movement, upward movement/downward movements were pretty much the same. At this stage the analyses have been made mainly focusing on the “possibilities which the touch points fall into the key center/offset key center’s corresponding side (Top/Down, Left/Right)” and “the relative positions of the touch points in related to the key centers/offset key centers” when the fingers were moving in different directions.

Table 1. The overall number and proportion of the fingers’ moving directions
Full size table

Possibilities: Touch Points Fall into the Corresponding Side of Key Center/Offset Key Center

In the “possibilities which the touch points fall into the key center/offset key center’s corresponding side” analysis, the touch points’ falling onto the two sides of the center point was regarded as a factor, the appearance percentages of both sides were used as dependent variables for one-way ANOVA analysis, and the finger directions were divided into four directions for discussions; for instance, whether there existed differences on the opportunities if the touch points would fall above or below the key center/offset key center when a finger was moving upwards. When we took the key center as the basis, the Levene’s test results indicated significant, so we did not continue on the analysis. When we took offset key center as the basis, it indicated significant when the finger moved upward (F(1.58) = 136.243, p < .001), so that there existed significant difference in the probability when the touch points fell on the corresponding side of the offset key center. The results were also significant when the fingers moved down (F(1.58) = 210.825, p < .001), the results of fingers’ left-movement (F(1,58) = 6.646, p < .001) and right-movement (F(1.58) = 36.485, p < .001) were similarly significant too.

Table 2 shows the four fingers’ moving directions’ percentage means of the occurrences of touch points’ falling on the corresponding sides. When the fingers moved upward, the touch points appeared above the offset key center with a probability of 56.83%, which was above the bottom possibility of 43.17%. When the fingers moved downwards, the touch points appeared below the offset key centers with a possibility of 59.31%, which was above the top portion of 40.69%, and the horizontal results were similar. Therefore, it could be concluded that when an offset key center was taken as basis, the touch points would have a higher chances of falling on one side of the finger’s moving direction.

Table 2. Fingers’ moving directions and location ratios of the falling points
Full size table

Relative Positions of the Touch Points to the Key Center/Offset Key Center

In the “relative positions of the touch points to the key center/offset key center” analysis, the fingers’ horizontal movements and vertical movements have been analyzed respectively by using one-way ANOVA. The fingers’ moving directions (Top/Down, Left/Right) were the factors, and the touch points in related to the key center/offset key center coordinate values were the dependent variables. In this analysis, each touch point’s relative coordinate values in related to the center of each key has been calculated by the App; for example, if the center point coordinate of the A key was (10, 10) and the touch point coordinate was (5, 15), the relative coordinate of the touch point to the center point was (−5, 5).

When we took the key centers as the basis and the fingers moved horizontally, the results have indicated significant (F (1.3192) = 21.219, p < .001). Therefore, there existed significant differences in the relative coordinates of the touch keys in related to the key centers when the fingers made left/right movements. Likewise, significant results (F (1.3192) = 232.7, p < .001) could be obtained when the fingers moved vertically. When we took an offset key center as the basis, since we have obtained significant results from both horizontal axis (F(1.3192) = 13.86, p < .001) and vertical axis (F(1.3192) = 106.163, p < .001), we thus concluded that the fingers’ moving directions would affect the touch point positions.

Table 3 displays the relative coordinate positions’ average values of the finger movement directions and touch points in relation to the key centers/offset key centers. For instance, when we took a key center as the basis, the touch point was 1.6234 pixel from touch center on the X-axis when the finger made right movement, the touch point would be 0.3579 pixel from the key center when left-movement was made; if we took offset key center as the basis, the touch point would be 0.3754 pixel from offset key center on the X-axis when the finger made right movement, the touch point would be −0.411 pixel from the offset key center when left-movement was made, a negative sign means that the touch point is on the offset key center’s left side.

Table 3. Coordinate means of the touch points in related to the central point v.s. differ finger moving directions
Full size table

4.2 Analyses of Finger Movement Speeds, Distances and Directions in Relation to Accuracy

This analysis focuses on the data entry accuracy. In this research the data entry accuracy does not refer to whether the subjects have correctly taped the target keys that it is referring to the distance between a touch point and the center point. The smaller the distance, the more accurate; and on the contrary, the larger the distance, the less accurate. Therefore, the accuracy value is referring to the absolute value of each individual touch point in related to the touch key’s central coordinate values. In addition, this analysis has solely taken offset key center as the basis because habitual shift happens on every subject while doing the taping, and such habitual shift conditions differ from one to the others. Therefore, it should make more sense to take the distance between the touch point and the offset key center as the basis of accuracy.

The correlation analysis between fingers’ moving speeds, movement distances in relation to accuracy is performed by regression analysis with the moving speeds and movement distances as the independent variable, and the distances between touch points and offset key centers as the dependent variable. The analysis results show that the fingers’ moving speeds on the horizontal axis are significantly related to accuracy (Table 4). The regression equation is:

Table 4. Result of regression analysis on the horizontal axis
Full size table
$$ {\text{Y}}\, = \,3.781\, + \,2.824\,\left( {{\text{X}}1} \right) $$
(1)

while the results are also significant on the vertical axis (Table 5) with the regression equation to be:

Table 5. Result of regression analysis on the vertical axis
Full size table
$$ {\text{Y}}\, = \,3.805\, + \,0.004\,\left( {{\text{X}}2} \right) + \,1.357\,\left( {{\text{X}}1} \right). $$
(2)

Based on the above results, we have learned that accuracy is affected by fingers’ moving speeds and movement distances no matter in vertical or horizontal directions. When the fingers move faster or when the distances get longer, the touch points would get away from offset key centers farther too, and that indicates that the accuracy would be reduced accordingly.

Finally, the finger movement directions and accuracy analyses have been performed by One-way ANOVA with the moving directions as a factor and the distances between touch points and offset key centers are of a dependent variable. As a result, it indicated non-significant both on horizontal axis and vertical axis, therefore finger movement directions do not affect the accuracy.

5 Discussion

Although the last analysis in Sect. 4.2 has shown that the directions of finger movements had no obvious effects on the data entry accuracy, the analysis of Sect. 4.1 clearly has shown that the directions of finger movements do affect the touch point positions. Figures 4 and 5 have presented the probabilities of four kinds of falling points in related to vertical and horizontal movement respectively, and from that we could see the consistent tendency of touch point position deviation. When the offset key centers were used as the basis, the touch points tended to fall on the same side of the fingers’ moving directions. When the fingers moved upwards, the probabilities for the falling points’ appearing above the offset key centers were significantly higher than that of below. For down-move, the probabilities of falling points’ on the lower side were higher than that of on the upper side; for left move, the probabilities of falling points’ on the left side were higher than that of on the right side; as for right move, the probabilities of falling points’ on the right side were higher than that of on the left side. Such tendency was even more obvious along the vertical axis, especially when the fingers moved downwards that the probabilities of the touch points’ falling below the offset key centers were significantly higher than the probabilities of falling above the offset key centers by nearly 20% (Fig. 4).

Fig. 4.
figure 4

Appearance possibilities of touch points’ positions when fingers moved vertically

Full size image
Fig. 5.
figure 5

Appearance possibilities of touch points’ positions when fingers

Full size image

The analysis results of the “relative positions of the touch points in related to the key centers/offset key centers” have also shown that the fingers’ moving directions would affect the touch point positions. Figures 6 and 7 have shown the average values of the coordinates of the touch points in related to the key centers/offset key centers along the horizontal and vertical axes. Regardless of whether the fingers were moving horizontally or vertically, the touch points were biased towards the directions of fingers’ moving directions. When we took offset key centers as the basis, the average falling points even coincidently distributed on both sides of the center points. When the fingers made left moves the mean of the falling points was −0.41 pixel which was located on the left side of the offset key centers. As for right moves, it was 0.38 pixel, located on the right side. There was a gap of 0.79 pixel between both, and it was similar for vertical movement cases. The mean value of the falling points was 0.98 pixel for up-moves, located at the upper position, and the mean value of the falling points was −1.02 pixel for down-moves, located at the lower position. The difference of falling point ranges on both sides was as large as 2 pixels. In the Sect. 4.1 analyses, “the probabilities for the touch points’ falling on the corresponding side of the key centers/offset key centers” and “the relative positions of the touch points in related to the key centers/offset key centers” have shown very consistent results.

Fig. 6.
figure 6

Fingers’ horizontal movements and means of touch points’ location coordinates

Full size image
Fig. 7.
figure 7

Fingers’ vertical movements and means of touch points’ location coordinates

Full size image

When we configured the touch points’ four directions movements to analyze as 95% confidence ellipse setting, we could see the four directions’ distribution status of the finger movements on the keyboards (Fig. 8). Under horizontal movement circumstances, the keys which near the left and right edges of the keyboards, such as the L, O, P, Q, do not have enough data samples due to the effects from finger moving directions, therefore such inappropriate ellipses must be skipped off, for example, unless the P key has been taped twice in succession, the fingers won’t have a chance to make a left-move to P. Under the horizontal movement circumstances, on most of the keys the number of right-biased ellipses are slightly more on the right side than the left-biased ellipses, and under vertical movement circumstances we could tell that there existed significant differences in the positions of the upward and downward ellipses.

Fig. 8.
figure 8

Appearance possibilities of touch points’ positions when fingers moved horizontally

Full size image

Based on the above mentioned, the touch point positions have higher probabilities of shifting towards the directions of finger movements. To speculate the causes, it could be because under the continuous data entry circumstances the movements were rather fast that the fingers could not accurately touch on the keys timely, and thus have taped in the moving directions at convenience. Such results were even more obvious on the fingers’ vertical movement behaviors, and perhaps that was because the vertical lengths of the keys were larger than the horizontal widths which made the subjects to have relatively more spaces for movements in the vertical direction, and thereby made the difference more significant.

In terms of accuracy, the analyses showed that speeds and distances also affected data entry precision. This result was similar to Fitts’s Law except that the results of this research could only show that the touch points would have a higher chances of getting away from the offset key centers when the fingers moved faster or when the fingers were farther away from the targets.

6 Conclusion

This research has designed an App for data entry testing to analyze how the finger movement directions, speeds, distances would affect the falling points of the text entry, and the research has had some findings. First of all, the directions of the finger movements would affect the touch point positions regardless of whether vertical or horizontal finger movements have been made. They all made the touch points to move towards the directions of the finger movements. From the offset key centers‘ point of view the falling points would have higher chances of following the fingers‘ moving directions to fall on the offset key centers‘ corresponding sides. For example, if the fingers moved upwards and then the falling points would have higher chances to fall above the offset key centers, and at the same time the average position of the falling points would distribute on the offset key centers’ corresponding sides according to the fingers’ moving directions; and when the fingers moved upwards the average position of the falling points would be right above the offset key centers. Secondly, according to the above phenomenon mentioned the results along the vertical axis were more significant than what from the horizontal axis despite what the probabilities were and where the average position of falling points was. Thirdly, in terms of taping accuracy, only the fingers’ moving speeds and movement distances would affect the taping accuracy, and the fingers’ moving directions would not have influence on the taping accuracy.

This research has discussed about the operation behaviors of continuous data entry on mobile devices, particularly the effects on touch point positions in related to fingers‘ moving directions. The results of this research would be helpful for the designs of virtual keyboards on the mobile devices, and would upgrade the data entry performance accordingly. In addition, further in-depth studies can be made in the future on such research; for example, analyses can be made on the status of every specific key, or to inspect the differences of keys according to different block divisions.