Keywords

1 Introduction

Largely thanks to the introduction of iPhone in 2007, the landscape of HCI for mobile and ubiquitous computing has been shifted. Such finger gesture interactions as swiping, pinching, tapping have pushed away the traditional and dominant ‘non-smart’ phone inputs–pressing key on small keyboards), which in turn generate a lot of interests among the evaluation of such interactions (Browne and Anand 2011; Findlater et al. 2013; Fitton et al. 2013; MacKenzie et al. 2012; Negulescu et al. 2012; Stößel and Blessing 2010a, b; Teather and MacKenzie 2014; Tran et al. 2013; Trewin et al. 2013; Shi et al. 2008; Gilbertson et al. 2008). However, as much as the appealing features mobile phones can bring, they often pose considerable challenges for game developers, especially regarding user interface design and game control (Gilbertson et al. 2008; Fishkin et al. 2000). Among the many primitive and innovative interface mechanism, tilt, in particular, has been dominantly adopted in mobile games, due to its minimal signal processing and light-weight external references. The interaction is made ascertain with the phone’s built-in accelerometers so as to allow players to incline or tip the device so as to control the game (Lane et al. 2010). Other popular and affordable (in terms of the control’s programmability) mobile game interactions include swipe and tap which have been extensively examined in applications other than mobile games (Fitton et al. 2013; Findlater et al. 2013; Browne and Anand 2011; MacKenzie and Teather 2012; Motti et al. 2014; Negulescu et al. 2012; Volker and Turner 2009; Stößel and Blessing 2010a, b; Teather and MacKenzie 2014).

When it comes to unite and examine users’ interaction preferences and performance of these three most popular mobile interaction patterns in mobile games (Tap, Swipe, and Tilt), there are very few studies here, which motivate our study here. Our study is the first and only an initial comparative study of three popular finger interaction types in mobile games: Tap, Swipe and Tilt under different operation modes.

The rest of the paper is organized as below. Section 2 presents and discusses prior studies on evaluating the usability of swipe, tap and tilt interactions. Our study design will be presented in Sect. 3, followed by experiment results. We conclude this paper by pointing to the limitations of our study as well as show the interesting research paths we will follow.

2 Related Work

2.1 Finger Gesture Interactions

Due to an increasing popularity of mobile touchscreen devices, evaluation studies of finger gesture interaction for these devices also gain attention recently. Shi et al. (2008) designed a set of both single and double finger gesture interaction techniques for digital document sharing on large tabletop; although the result sheds light on the general learnability, usability and naturalness of these finger interaction types, it is not applicable in our study since we focus on exploring the comfort and naturalness of them in a mobile game. A couple of other similar studies on the usability of finger gesture interaction techniques focus on the comparative studies of psychomotor performance between older and younger users (Findlater et al. 2013). In summary, general findings in these studies are consistent in terms of the significant slower performance for older adults than younger ones (Stößel and Blessing 2010a, b; Findlater et al. 2013). Findlater et al. (2013) further reported that when compared with younger adults, older ones showed quicker movement in touchscreen devices. (Stößel and Blessing 2010a) offered gesture design recommendations based on their comparative studies. Results showed that there is a difference between younger and older users on the gesture type they chose. For younger group, 86 % participants chose direct manipulation gesture and only 14 % chose symbolic gesture. It revealed that the symbolic gesture is easier to memorize and therefore easier to be used for the old. When comparing between one-finger and multiple-finger interactions, both two age groups prefer the former over the latter. But older users are not less comfortable with two-finger action than their younger counterparts (Stößel and Blessing 2010a). Motti et al. (2014) focused on the accuracy of drag-and-drop interaction for older adults in tactile puzzle games on two different screen sizes, tablet and smartphones.

Other studies explored the various interactions in mobile devices in general. For instance, Tran et al. (2013) conducted an exploratory study of two prevent gestures (pinch and spread) with seated participants on a tablet and smartphone device; the results revealed that most of the pinch and spread tasks can be finished on an average of 0.9–1.2 s. The study fail to find any significant association between device orientation and gesture performance, One surprising finding did report a good fit to a simple Fitts’s Law model as determined by varying target width and gesture size (Tran et al. 2013).

Although tilt interaction has predominantly been explored in game research, a couple of others have focused how it can help enhance human performance in general. (MacKenzie et al. 2012) employed a mixed methodology to explore the adoption of tilt interaction for mobile users: survey questions were used to obtain users’ subjective evaluation over the usability of tilt; while quantitative methods are used to measure accuracy, maximum tilt, and moving time. (Trewin et al. 2013) compared swipe and tap inputs in the context of smartphone’s physical assess to participants with dexterity impairment, and concluded that swipe input has a better flexibility than tap one due to the latter’s requirement of find finger positioning, which in turn contributed to a higher accuracy rate than tap input. Other similar studies include Teather and MacKenzie (2014), Teather and MacKenzie (2014).

2.2 Swipe, Tap and Tilt in Mobile Games

Evaluating finger gesture interactions in game research is predominantly on tilt control, and the majority of them are qualitative in nature. For instance, mobile phone tilt and the traditional button input were evaluated in a driving game Gilbertson et al. (2008); user experiences such as fun were focused in the testing (informal). Valente et al. (2009) examined tilt interactions for a mobile accessible game for the visually impaired; qualitative measurement was employed via observations and interviews. Participants preferred tilt-based interactions since they are more natural. More recently, evaluation has been shifted to employ some quantitative methods as well. For example, Browne and Anand (2011) examined tilt, gesture and buttons in a shooting game, and reported a similar result as tilt interaction is preferred (Valente et al. 2009). In addition, the results revealed that participants who used tilt can play significantly longer than those who did not.

When it comes to unite and examine users’ interaction preferences and performance of three most popular mobile interaction patterns in mobile games (Tap, Swipe, and Tilt) there are very few studies.

To the best of our knowledge, our study is the first and only an initial comparative study of three popular finger interaction types in mobile games: tap, swipe and tilt. Unique to smartphones and tablet devices is tilt interaction pattern which has primarily been deployed in games (Fitton et al. 2013), particularly, tilt operations can be mapped to specific game inputs and recently has been used in motion sensor-based devices such as Nintendo Wii and Sony PlayStation 3.

3 Our Experiment and Brief Analysis

3.1 Participants and Apparatus

17 participants aged between 18 and 20 are selected from Wenzhou Kean University where the three authors are. The experiment was performed using a Nokia Lumia 730 and a Nokia Lumia 530 running on Windows Phone 8.1.

3.2 Study Design and Testing Applications

We followed recommendations from (Stößel and Blessing 2010b) that the usability of the three types of interaction patterns can only measure realistically in real interactive scenarios. The interface design in this experiment followed two rules proposed in (Negulescu et al. 2012) which centers on the prevention of user distraction should be limited:

  1. 1.

    the need for visual attention during interaction should be limited;

  2. 2.

    the need for more streamlined commands for common tasks;

Following the two principles, we designed four types of interfaces — three as testing applications and one as a testing game.

The Tap Testing Application.

In the Tap applications (known as Tap I, II and III), the interfaces are shown in Fig. 1a. The differences among the three testing interfaces are the size of the active areas (Tap I > II > III). Users can tap the button. Once the user taps on the button, it will be recorded in the upper number and if the user taps out of the button, it will be recorded in the other number.

Fig. 1.
figure 1

The three testing environment with Tap (a), Swipe (b) and Tilt (c) interaction and a testing environment implemented as a game (d).

Full size image

The Swipe Testing Application.

In the Swipe applications (known as Swipe I, II and III), the interfaces are shown in Fig. 1b, with the main difference on the size of the white rectangles (Swipe I > II > III). Users can swipe the rectangle to the right and it will come back to the initial position once the user’s finger released from the screen. If the user swipes the rectangle, it will be recorded in the number on the right down corner.

The Tilt Testing Application.

In the Tilt application (known as Tilt I, II, III), the interfaces are shown in Fig. 1c. User can use the tilt action to move the screen object to the left and right. Once the user tilts to the left or to the right, it will be recorded in the numbers on the left or the right side of the interface. The sensitive of the accelerometer in the three applications are different (known as Tilt I > II > III).

The Game Application.

The last testing environment was implemented as a simple game (Fig. 1d). Users can choose different types of interaction techniques to play the game: — for tap, users can tap the button L, R, U and D; for swipe, users can swipe the rectangle in the middle; and for tilt, users can tilt the phone. The game will randomly make the upper (lower, left and right) half two of the four squares white and the other gray. Once the user provides a correct reaction, he or she can get one point. For example, swipe the rectangle to the right when the right two squares are white. Otherwise, the game will stop.

3.3 Experiment Procedure and Evaluation Protocol

We follow prior study of similar nature on the experiment procedure and evaluation protocol (Negulescu et al. 2012; Stößel and Blessing 2010a, b): both quantitative and qualitative data had been collected. The former includes task completion time, and performance quality; while the latter data examining player experiences and preferences regarding the three interaction techniques. We report the result of our quantitative analysis.

Before the testing, an experimenter provides detailed description on the experiment procedure. They were then asked to fill in a survey and then start the experiment. In the experiment, first, the participants perform the Tap tasks — tap on the button for ten times as fast as possible (in the order: Tap I, II, III). Second, participants will focus on the Swipe tasks — swipe the rectangle for ten times as fast as possible (in the order: Swipe I, II, III). Third, they were asked to manipulate using the Tilt interaction — tilt the phone to the left and back to the right for ten times (in the order Tilt I, II, III). Each task has to repeat for three times in the selected scenarios. At last, the participant plays the testing game. Use different input ways (in the order: Tap, Swipe, Tilt) to play the games three times (30 s per play) in the selected scenarios and playing scores are recorded accordingly. At the end of each task, each participant rates his/her user experiences on the interaction on a 5-point Likert scale where “1” indicates the worst user experience and “5” indicates the best user experience. Figure 2 shows the testing moment when the participant was running on a treadmill.

Fig. 2.
figure 2

The participant is testing the application while running in a treadmill

Full size image

3.4 Experiment Mode

We posit that when users may prefer one interaction over another when in different operation mode, therefore, we invite participants to perform the tasks in the following four modes in our lab and the gym:

  • Standing — interacting while standing

  • Sitting — interacting while sitting on the chair

  • Walking — interaction while walking on the running machine with a comfortable but constant speed in the gym

  • Running — interacting while running on the running machine in the gym

  • Lying — interacting while lying on the sofa

In all scenarios, the participants can choose to finish the task with either one hand or both. The experiment was conducted in the same room or gym (see Fig. 2).

3.5 Experiment Result and Discussions

In this section, we discussed the results of the experiment both the testing environment and the testing game. In each part, we discussed the results in five different scenarios and compared them with each other.

The Testing Environment.

The result of the experiment has been shown in Fig. 3a, b, c respectively. We found no significant difference for tap and swipe operations with a ‘big’ object. In the Sitting mode, the average performance is 10.07 (SD = 1.29) for Tap and 9.80 (SD = 0.48) with Swipe. In the Standing scenario, participants averaged 10.00 (SD = 0.33) using tap and averaged 9.56 (SD = 0.61) with Swipe. In Walking scenario, participants averaged 10.03 (SD = 0.18) using Tap and averaged 9.73 (SD = 0.81) using Swipe. In Lying mode, participants averaged 10.00 (SD = 0.00) with Tap and Swipe. In Running MODE, participants averaged 9.78 (SD = 0.42) with Tap and 9.56 (SD = 0.68) with Swipe. Interestingly, the difference between the results in Running mode when using Tilt is significant. Though the program had the largest fault-tolerance rate, participants only averaged 7.67 (SD = 3.53).

Fig. 3.
figure 3

Experiment results in three testing mode with Tap, Swipe and Tilt operations respectively. (Color figure online)

Full size image

However, when the object or the fault-tolerance gets smaller, we obtained more significant differences across these modes with different manipulation types. In Sitting scenario, participants scored an average of 5.90 (SD = 2.53) using Tap and 7.83 (SD = 1.61) when Swiping. In Standing scenario, participants obtained an average score of 5.10 (SD = 3.70) with Tap and 2.56 (SD = 2.70) with Swipe. In Walking scenario, participants averaged 4.60 (SD = 2.72) in Tap operations and 1.47 (SD = 1.48) in Swipe. But the difference is not significant in both Lying and Running modes despite that the object is small. Particularly, in Lying mode, participants averaged 2.70 (SD = 1.25) using Tap and averaged 2.00 (SD = 0.00) with Swipe. In Running scenario, participants averaged 2.78 (SD = 2.20) using tap and 2.89 (SD = 2.02). When in the Running mode, using Tilt manipulating a small object also lead to a large performance difference between Tap and Swipe operations.

The Testing Game.

In the experiment, participant played a game using three input techniques: Tap, Swipe and Tilt in all modes. In Sitting scenario, participants scored the lowest in Tilt operations (M = 21.74, SD = 13.60), and the highest in Swipe (M = 40.00, SD = 9.00). In Standing mode, participants averaged 40.28 (SD = 9.67) with Swipe mode as the highest score and 9.00 (SD = 9.57) with Tilt mode as the lowest score. In Lying mode, participants scored the highest in Tilt (M = 20.33, SD = 17.52), and the lowest in Tap (M = 16.00, SD = 20.12). In Walking mode, participants averaged 38.63 (SD = 13.00) with Tap mode as the highest score and 19.30 (SD = 13.90) with Tilt mode as the lowest score. In Running mode, the highest performance was received in the Tap manipulations (M = 42.56, SD = 14.46), and the lowest in Tilt operations (M = 4.83, SD = 3.81).

Discussion.

On one hand, if operating in the same mode and with only one input style, especially with Tap or Swipe, the bigger size of a bottom or a control interface can offer a friendlier input experience for the participants. On the other hand, for different input operations’ quality in each mode, using Tap, Swipe and Tilt operations lead to similar performance. For example, in the testing application on relatively stable mode including Sitting, Standing and Lying, operations with Tap and Swipe can help the player obtain a steady improving score. However, in the Walking and Running mode, it is Tilt and Swipe that can achieve most users’ input requirement. For Tap, Swipe and Tilt input styles, we can divide them into three different levels: Level I is entirely static and motionless when the user need to input something, such as in Tap; Level II is a combination of motion and movement, such as in Swipe; as in Level 3, it require a large amount of amplifying motion, like Tilt. It is obvious that in Level I and Level II, Tap and Swipe operations can fulfill most of the basic input requirements in the simple testing environment. However, in the context of the game, interestingly, we obtained some opposite results. We believe that the complexity of the game itself and participants’ limited familiarity contributed to lower and unstable performances. In fact, the game itself poses an intrinsic requirements when compared with the elementary instructions in the simple testing environments. Therefore, we suggest that designers should consider users’ most common operation mode before determining interaction styles.

4 Future Study: Touch-Screen Interaction Styles for Children with ASD

We had observed that Chinese children with ASD are more inclined to interact with the touch-screen-enabled applications during the testing of other applications in a large children’s autism educational development; that is, when given the computer-based application, the majority of children’s first attempt is to touch and tap the screen-objects. This important observation leads us to ponder how our current research can be extended to investigate the interaction styles preferred by these children. To the best of our knowledge, such research is under-explored and yet key to inform the development of appropriate interactions for these children.

We are currently modifying the testing user interfaces and simplifying the testing tasks in order to investigate how children with autism tend to respond to these various interaction styles. Figure 4 below shows the simplified and enhanced testing screens after various interaction styles. Note that we will simplify the testing tasks by only requiring the children to tap, swipe or tilt the screen blocks in order to preclude the learning component of the task.

Fig. 4.
figure 4

New Testing Environment to be tested with Children with ASD: from left to right, the initial environment and the Tap, Swipe and Tilt mode respectively.

Full size image

5 Concluding Remarks

Thanks to the increasing popularity of mobile applications, finger gesture interactions are prevalent; evaluation studies of finger gesture interaction for these devices also gain attention recently. However, prior studies fail to examine users’ interaction preferences and performance of three most popular mobile interaction patterns in mobile games (Tap, Swipe, and Tilt) and link their usage appropriateness with different operation settings (such as lying on the bed, running in a treadmill, etc.), which motivate our study here. In particular, in this paper we offer a comparative study of the three popular finger interaction types in mobile games under different operation modes in terms of users’ performance accuracy and experiences.

Our experiments revealed that in relatively stable operating environment such as Walking, Lying and Sitting, Type and Swipe input styles are preferred over Tilt; however, in such operating modes as Walking and Running, Tilt is preferred. However, when testing these input styles in a game, due to the complexity of the game itself and participants’ limited familiarity, lower performance has been observed. It leads us to believe that the game itself poses intrinsic requirements when compared with the elementary instructions in the simple testing environments. Therefore, we suggest that designers should consider users’ most common operation mode before determining interaction styles. We are currently modifying the testing user interfaces and simplifying the testing tasks in order to investigate how children with autism tend to respond to these various interaction styles.