AttachedShock: Design of a crossing-based target selection technique on augmented reality devices and its implications

https://doi.org/10.1016/j.ijhcs.2014.03.001Get rights and content

Highlights

  • Identify design considerations for selection techniques in AR navigation scenarios.

  • Exploit crossing actions to intercept moving targets.

  • Build an Adobe© Flash application for evaluating selection techniques.

  • Design, prototype, and evaluate AttachedShock for mobile navigation applications.

  • Summarize implications gained from this study for future studies.

Abstract

The prevalence of touch devices in daily lives offers increasing opportunities for users to navigate the real world. However, as users move, on-screen targets move unpredictably, and eventually disappear from the screen in mobile navigation scenarios. The changing target movement pattern creates difficulties in selecting targets in time before targets escape from the screen. This study proposes a novel selecting technique, AttachedShock, for easing target selection tasks on augmented reality devices by crossing a naturally expanding wave pattern attached to targets. We evaluated the effectiveness of our technique by conducting three sets of comparative studies on measuring the performance of four techniques under various mobile navigation scenarios, i.e., various combinations of the identified factors. The results indicate that the proposed technique assists users in selecting moving targets to improve the error rate substantially, by a minimum of 76.51%, 61.75%, and 72.77% in Experiment-I, -II, and -III relative to state-of-the-art techniques, and incurs acceptable distractions to users, compared to other techniques.

Introduction

Designing and improving target selection techniques has long been an important research topic of human–computer interaction research to achieve the accurate performance of target acquisition tasks in traditional navigation or surveillance applications. Since the prevalence of the touch devices, more and more 2D touch devices are carried by users or deployed in our living environments. New touch devices are equipped with sophisticated sensing, computing, and communication capabilities. For example, a touchpad device (such as the iPad 2) is equipped with a variety of sensors, including GPS, accelerometer, digital compass, gyro, Wi-Fi, and cell-ID sensors that can detect the locations and movements of users. Because of their computing power and sensory capabilities, it is possible to locate the existing visual objects in real scenes (e.g., nearby signboards of stores or landmarks) and employ the identified targets as augmented anchor points for users to further access in situ information (Rohs and Oulasvirta, 2008, Rohs et al., 2011). However, this type of augmented reality (AR) applications is frequently used when people are walking or moving in transit vehicles. Users can select these augmented regions (i.e., target regions) to retrieve additional information in the first-person point of view (defined as first-person target-intercepting tasks) under various application scenarios, such as landmark-based navigation in physical (or virtual) world (e.g., Toyota׳s “Window to the world”, 2011 and the MIT׳s AIDA, 2011 project in future car navigation applications), automatic tracking systems (Honey and Milnes, 2013), and first-person action games that require prompt user responses (Fruit Ninja, 2011). Imaging the following scenario:

Ben, who is a first-time visitor to a city, arrives at an airport and plans to take a shuttle bus to commute between the airport and his hotel. To identify fast-moving landmarks or road signboards en route so that he can be aware of where he should get off the shuttle, he glances at an AR device mounted on the window nearby him to search for informative touring spots or road signboards (i.e., augmented indications on the screen) while he is enjoying the beautiful scenery out of the window. As an indication (i.e., the on-screen target) to an informative landmark or a road signboard appears, he notices the indication and then performs target selection actions to select the moving or escaping target for retrieving further descriptions of the landmark or signboard, e.g., estimated time of arrival. However, due to irregular variations of moving speeds and trajectories of targets, he may fail to accurately intercept the desired target to make sure if he is close to his destination before the target escapes from the screen. As a result, he may just miss the station at which he should get off the bus due to the absent information.

Due to the movements of the observer or the objects, the indicated targets change their on-screen locations, and finally escape from the displaying region of the screen. This makes the traditional click-based techniques prone to incorrectly select or completely miss the targets (Hasan et al., 2011). In this study, we present AttachedShock, a crossing-based target selection technique that is used to facilitate the selection of the moving targets. By enlarging the activation area of the target with an attached wave pattern extending in a direction perpendicular to the path of motion, users can accurately intercept the moving targets through goal-crossing actions.

Previous researchers (Kabbash and Buxton, 1995, Grossman and Balakrishnan, 2005, Hasan et al., 2011) extend the traditional click-based techniques (Fig. 1(a)) by enlarging the activation region of the targets (e.g., adding tails on targets in the path of the motion) to improve the performance on selecting static/moving targets. Due to irregular changes on the moving trajectory and speed of the targets, users tend to solve these on-screen moving target interception tasks by exploiting various strategies (e.g., pursuing, interception, etc.), similar to those performed in the physical space (Fajen and Warren, 2007), to predict the future moving directions of the targets. Therefore, users׳ fingertips tend to uncatch or overshoot the target, and thereby incorrectly select other objects or completely miss the targets (Fig. 1(a)). On the contrary, our crossing-based target selection technique, which is not feasible on the desktop interfaces (Findlater et al., 2010), transfers the sub-movement correction tappings to effective goal-crossing selections on AR devices.

Therefore, this study proposes a novel target selection design (Fig. 1(b)) to facilitate the target selection tasks in mobile navigation applications by exploiting the goal-crossing actions (Accot and Zhai, 2002). Fig. 1 illustrates a real-world scene that was enhanced by wave patterns to dynamically stretch the wave according to the on-screen speed of a target. We incorporated goal-crossing actions to trigger target selection by moving the fingertips across the waves. By crossing the stretched pattern growing in a direction that is perpendicular to the path of motion, users can move their fingertips across the target (or wave) without performing the sub-movement correction actions for tapping on the target. The stretched size reaches a maximum as the target approaches the edge of the screen, thus causing less distractions in the center regions (the area usually focused on by the user Ma et al., 2002) of the screen.

The main contributions of this study are two-fold as follows:

  • This study identifies design considerations for selection techniques used in AR navigation scenarios, which future researchers can use to improve their designs on better selection techniques for AR devices. These design considerations also enlighten us on exploiting crossing actions to intercept moving targets on AR devices by breaking through a naturally expanding wave pattern (i.e., an enlarged activation area) attached on targets.

  • Based on identified design considerations, a preliminary pilot study was performed to extract potential factors that affect the performance of selecting the moving targets on AR devices. To emulate the target movement under various mobile navigation scenarios (i.e., various combinations of identified factors), an Adobe® Flash application was built for evaluating target selection techniques on AR devices.

  • We designed, prototyped, and evaluated a target selection technique, AttachedShock, for mobile navigation applications. Three sets of comparative studies were performed for users to select moving targets in various navigation scenarios. The results demonstrate that the AttachedShock technique improved the selection accuracy of moving targets in mobile navigation scenarios substantially, by a minimum of 76.51%, 61.75%, and 72.77% in high-speed, road-direction-changing, and varying-viewing-angle moving scenarios relative to state-of-the-art techniques (Kabbash and Buxton, 1995, Hasan et al., 2011), and incurred a reasonable time for performing the crossing actions.

The rest of this paper is organized as follows. Section 2 reviews related work. Section 3 identifies the design considerations and potential factors that affect the target selection performance on AR devices. Based on these considerations, Section 4 introduces the design concepts of the AttachedShock technique. Section 5 describes the data collection process and presents experimental results to demonstrate system effectiveness under high-speed moving scenarios, while 6 Experiment-II—selection tasks under road-direction-changing scenarios, 7 Experiment-III—selection tasks under different viewing angles present another two sets of experiments to further demonstrate the system effectiveness under road-direction-changing and varying-view-angle scenarios. The implications from all three sets of experiments are summarized in Section 8. Finally, Section 9 offers conclusions and describes future work.

Section snippets

Related work

We organize the related work as follows: first, we review the human performance models for predicting the movement time on selecting static or moving targets. Subsequently, we review various types of actions for interacting with objects or widgets on the screen. Finally, we discuss various feedback designs for visualizing the effects of the targeting actions.

Design guidelines

A number of guidelines affect the design of the AttachedShock technique for selecting moving targets (e.g., signboards of stores) to acquire information (Rohs and Oulasvirta, 2008, Rohs et al., 2011) as follows: (1) user-perception behavior, (2) surrounding scene change, (3) user-responding selecting behavior, and (4) feedback approach.

AttachedShock technique

Based on the design guidelines, we propose the AttachedShock technique, which includes the following concepts: (1) wave propagation and (2) wave-crossing chosen concepts.

Experiment-I—selection tasks under high-speed moving scenarios

The effectiveness of the AttachedShock technique was tested in a controlled experiment, in which participants performed selection tasks by using various targeting techniques: point cursor (i.e., traditional click-based technique), area cursor (Kabbash and Buxton, 1995), comet technique (Hasan et al., 2011) and AttachedShock technique. This section describes the design and experimental results of this comparative user study, which was guided by the following hypotheses:

  • H1: Participants may

Experiment-II—selection tasks under road-direction-changing scenarios

The goal of this set of experiments is to evaluate the target selection performance when users navigate the surrounding environments along an emulated mixed-type of roads (straight/curved paths). From Experiment-I, we have demonstrated the effectiveness of the AttachedShock technique for selecting moving targets in high-speed moving (in straight roads) scenarios. However, in daily navigation scenarios, objects might suddenly change their moving directions as users sit in a vehicle moving along

Experiment-III—selection tasks under different viewing angles

The goal of this set of experiments is to evaluate the selection performance as users move along an emulated straight road with their viewpoints away from the driving direction. After demonstrating the effectiveness of the AttachedShock technique on selecting moving targets in high-speed and road-direction-changing moving scenarios, a third set of experiments was conducted to certify the following hypotheses based on the same apparatus and procedures as those used in previous experiments:

  • H8:

Discussions

This section discusses the pros and cons that are associated with various target selection techniques in these three sets of experiments.

Conclusions and future work

This study presents a novel target selection technique for selecting moving targets in mobile navigation scenarios. The AttachedShock technique expands the activation area of the target with a virtual shock wave front. The shock front patterns stretch outward as the target speed increases. By flicking users׳ fingertips across attached waves, users can accurately intercept moving targets with natural goal-crossing actions on AR devices. To demonstrate the effectiveness on the targeting

References (30)

  • Accot, J., Zhai, S., 2002. More than dotting the i׳s—foundations for crossing-based interfaces. In: Proceedings of the...
  • Artificial Intelligence Driving Agent, 2011....
  • Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P., Bederson, B., Zierlinger, A., 2003. Drag-and-pop...
  • Bezerianos, A., Balakrishnan, R., 2005. The vacuum: facilitating the manipulation of distant objects. In: Proceedings...
  • O.A.R. Board et al.

    OpenGL© Programming Guide: The Official Guide to Learning OpenGL©, Version 2.1

    (2007)
  • Chapuis, O., Labrune, J.-B., Pietriga, E., 2009. DynaSpot: speed-dependent area cursor. In: Proceedings of the 27th...
  • Choe, E.K., Shinohara, K., Chilana, P.K., Dixon, M., Wobbrock, J.O., 2009. Exploring the design of accessible goal...
  • B.R. Fajen et al.

    Behavioral dynamics of intercepting a moving target

    Exp. Brain Res. (Exp. Hirnforsch. Exp. crbrale)

    (2007)
  • Fekete, J.-D., Elmqvist, N., Guiard, Y., 2009. Motion-pointing: target selection using elliptical motions. In:...
  • Findlater, L., Jansen, A., Shinohara, K., Dixon, M., Kamb, P., Rakita, J., Wobbrock, J.O., 2010. Enhanced area cursors:...
  • P.M. Fitts

    The information capacity of the human motor system in controlling the amplitude of movement. 1954

    J. Exp. Psychol. Gen.

    (1992)
  • FruitNinja©, 2011....
  • Girden, E., 1992. ANOVA: repeated measures. Quantitative Applications in the Social Sciences, No. 84. Sage...
  • Grossman, T., Balakrishnan, R., 2004. Pointing at trivariate targets in 3D environments. In: Proceedings of the SIGCHI...
  • Grossman, T., Balakrishnan, R., 2005. The bubble cursor: enhancing target acquisition by dynamic resizing of the...
  • Cited by (6)

    This paper has been recommended for acceptance by Oulasvirta Antti.

    View full text