AttachedShock: Design of a crossing-based target selection technique on augmented reality devices and its implications☆
Introduction
Designing and improving target selection techniques has long been an important research topic of human–computer interaction research to achieve the accurate performance of target acquisition tasks in traditional navigation or surveillance applications. Since the prevalence of the touch devices, more and more 2D touch devices are carried by users or deployed in our living environments. New touch devices are equipped with sophisticated sensing, computing, and communication capabilities. For example, a touchpad device (such as the iPad 2) is equipped with a variety of sensors, including GPS, accelerometer, digital compass, gyro, Wi-Fi, and cell-ID sensors that can detect the locations and movements of users. Because of their computing power and sensory capabilities, it is possible to locate the existing visual objects in real scenes (e.g., nearby signboards of stores or landmarks) and employ the identified targets as augmented anchor points for users to further access in situ information (Rohs and Oulasvirta, 2008, Rohs et al., 2011). However, this type of augmented reality (AR) applications is frequently used when people are walking or moving in transit vehicles. Users can select these augmented regions (i.e., target regions) to retrieve additional information in the first-person point of view (defined as first-person target-intercepting tasks) under various application scenarios, such as landmark-based navigation in physical (or virtual) world (e.g., Toyota׳s “Window to the world”, 2011 and the MIT׳s AIDA, 2011 project in future car navigation applications), automatic tracking systems (Honey and Milnes, 2013), and first-person action games that require prompt user responses (Fruit Ninja, 2011). Imaging the following scenario:
Ben, who is a first-time visitor to a city, arrives at an airport and plans to take a shuttle bus to commute between the airport and his hotel. To identify fast-moving landmarks or road signboards en route so that he can be aware of where he should get off the shuttle, he glances at an AR device mounted on the window nearby him to search for informative touring spots or road signboards (i.e., augmented indications on the screen) while he is enjoying the beautiful scenery out of the window. As an indication (i.e., the on-screen target) to an informative landmark or a road signboard appears, he notices the indication and then performs target selection actions to select the moving or escaping target for retrieving further descriptions of the landmark or signboard, e.g., estimated time of arrival. However, due to irregular variations of moving speeds and trajectories of targets, he may fail to accurately intercept the desired target to make sure if he is close to his destination before the target escapes from the screen. As a result, he may just miss the station at which he should get off the bus due to the absent information.
Due to the movements of the observer or the objects, the indicated targets change their on-screen locations, and finally escape from the displaying region of the screen. This makes the traditional click-based techniques prone to incorrectly select or completely miss the targets (Hasan et al., 2011). In this study, we present AttachedShock, a crossing-based target selection technique that is used to facilitate the selection of the moving targets. By enlarging the activation area of the target with an attached wave pattern extending in a direction perpendicular to the path of motion, users can accurately intercept the moving targets through goal-crossing actions.
Previous researchers (Kabbash and Buxton, 1995, Grossman and Balakrishnan, 2005, Hasan et al., 2011) extend the traditional click-based techniques (Fig. 1(a)) by enlarging the activation region of the targets (e.g., adding tails on targets in the path of the motion) to improve the performance on selecting static/moving targets. Due to irregular changes on the moving trajectory and speed of the targets, users tend to solve these on-screen moving target interception tasks by exploiting various strategies (e.g., pursuing, interception, etc.), similar to those performed in the physical space (Fajen and Warren, 2007), to predict the future moving directions of the targets. Therefore, users׳ fingertips tend to uncatch or overshoot the target, and thereby incorrectly select other objects or completely miss the targets (Fig. 1(a)). On the contrary, our crossing-based target selection technique, which is not feasible on the desktop interfaces (Findlater et al., 2010), transfers the sub-movement correction tappings to effective goal-crossing selections on AR devices.
Therefore, this study proposes a novel target selection design (Fig. 1(b)) to facilitate the target selection tasks in mobile navigation applications by exploiting the goal-crossing actions (Accot and Zhai, 2002). Fig. 1 illustrates a real-world scene that was enhanced by wave patterns to dynamically stretch the wave according to the on-screen speed of a target. We incorporated goal-crossing actions to trigger target selection by moving the fingertips across the waves. By crossing the stretched pattern growing in a direction that is perpendicular to the path of motion, users can move their fingertips across the target (or wave) without performing the sub-movement correction actions for tapping on the target. The stretched size reaches a maximum as the target approaches the edge of the screen, thus causing less distractions in the center regions (the area usually focused on by the user Ma et al., 2002) of the screen.
The main contributions of this study are two-fold as follows:
- •
This study identifies design considerations for selection techniques used in AR navigation scenarios, which future researchers can use to improve their designs on better selection techniques for AR devices. These design considerations also enlighten us on exploiting crossing actions to intercept moving targets on AR devices by breaking through a naturally expanding wave pattern (i.e., an enlarged activation area) attached on targets.
- •
Based on identified design considerations, a preliminary pilot study was performed to extract potential factors that affect the performance of selecting the moving targets on AR devices. To emulate the target movement under various mobile navigation scenarios (i.e., various combinations of identified factors), an Flash application was built for evaluating target selection techniques on AR devices.
- •
We designed, prototyped, and evaluated a target selection technique, AttachedShock, for mobile navigation applications. Three sets of comparative studies were performed for users to select moving targets in various navigation scenarios. The results demonstrate that the AttachedShock technique improved the selection accuracy of moving targets in mobile navigation scenarios substantially, by a minimum of 76.51%, 61.75%, and 72.77% in high-speed, road-direction-changing, and varying-viewing-angle moving scenarios relative to state-of-the-art techniques (Kabbash and Buxton, 1995, Hasan et al., 2011), and incurred a reasonable time for performing the crossing actions.
The rest of this paper is organized as follows. Section 2 reviews related work. Section 3 identifies the design considerations and potential factors that affect the target selection performance on AR devices. Based on these considerations, Section 4 introduces the design concepts of the AttachedShock technique. Section 5 describes the data collection process and presents experimental results to demonstrate system effectiveness under high-speed moving scenarios, while 6 Experiment-II—selection tasks under road-direction-changing scenarios, 7 Experiment-III—selection tasks under different viewing angles present another two sets of experiments to further demonstrate the system effectiveness under road-direction-changing and varying-view-angle scenarios. The implications from all three sets of experiments are summarized in Section 8. Finally, Section 9 offers conclusions and describes future work.
Section snippets
Related work
We organize the related work as follows: first, we review the human performance models for predicting the movement time on selecting static or moving targets. Subsequently, we review various types of actions for interacting with objects or widgets on the screen. Finally, we discuss various feedback designs for visualizing the effects of the targeting actions.
Design guidelines
A number of guidelines affect the design of the AttachedShock technique for selecting moving targets (e.g., signboards of stores) to acquire information (Rohs and Oulasvirta, 2008, Rohs et al., 2011) as follows: (1) user-perception behavior, (2) surrounding scene change, (3) user-responding selecting behavior, and (4) feedback approach.
AttachedShock technique
Based on the design guidelines, we propose the AttachedShock technique, which includes the following concepts: (1) wave propagation and (2) wave-crossing chosen concepts.
Experiment-I—selection tasks under high-speed moving scenarios
The effectiveness of the AttachedShock technique was tested in a controlled experiment, in which participants performed selection tasks by using various targeting techniques: point cursor (i.e., traditional click-based technique), area cursor (Kabbash and Buxton, 1995), comet technique (Hasan et al., 2011) and AttachedShock technique. This section describes the design and experimental results of this comparative user study, which was guided by the following hypotheses:
- •
H1: Participants may
Experiment-II—selection tasks under road-direction-changing scenarios
The goal of this set of experiments is to evaluate the target selection performance when users navigate the surrounding environments along an emulated mixed-type of roads (straight/curved paths). From Experiment-I, we have demonstrated the effectiveness of the AttachedShock technique for selecting moving targets in high-speed moving (in straight roads) scenarios. However, in daily navigation scenarios, objects might suddenly change their moving directions as users sit in a vehicle moving along
Experiment-III—selection tasks under different viewing angles
The goal of this set of experiments is to evaluate the selection performance as users move along an emulated straight road with their viewpoints away from the driving direction. After demonstrating the effectiveness of the AttachedShock technique on selecting moving targets in high-speed and road-direction-changing moving scenarios, a third set of experiments was conducted to certify the following hypotheses based on the same apparatus and procedures as those used in previous experiments:
- •
H8:
Discussions
This section discusses the pros and cons that are associated with various target selection techniques in these three sets of experiments.
Conclusions and future work
This study presents a novel target selection technique for selecting moving targets in mobile navigation scenarios. The AttachedShock technique expands the activation area of the target with a virtual shock wave front. The shock front patterns stretch outward as the target speed increases. By flicking users׳ fingertips across attached waves, users can accurately intercept moving targets with natural goal-crossing actions on AR devices. To demonstrate the effectiveness on the targeting
References (30)
- Accot, J., Zhai, S., 2002. More than dotting the i׳s—foundations for crossing-based interfaces. In: Proceedings of the...
- Artificial Intelligence Driving Agent, 2011....
- Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P., Bederson, B., Zierlinger, A., 2003. Drag-and-pop...
- Bezerianos, A., Balakrishnan, R., 2005. The vacuum: facilitating the manipulation of distant objects. In: Proceedings...
- et al.
Programming Guide: The Official Guide to Learning , Version 2.1
(2007) - Chapuis, O., Labrune, J.-B., Pietriga, E., 2009. DynaSpot: speed-dependent area cursor. In: Proceedings of the 27th...
- Choe, E.K., Shinohara, K., Chilana, P.K., Dixon, M., Wobbrock, J.O., 2009. Exploring the design of accessible goal...
- et al.
Behavioral dynamics of intercepting a moving target
Exp. Brain Res. (Exp. Hirnforsch. Exp. crbrale)
(2007) - Fekete, J.-D., Elmqvist, N., Guiard, Y., 2009. Motion-pointing: target selection using elliptical motions. In:...
- Findlater, L., Jansen, A., Shinohara, K., Dixon, M., Kamb, P., Rakita, J., Wobbrock, J.O., 2010. Enhanced area cursors:...
The information capacity of the human motor system in controlling the amplitude of movement. 1954
J. Exp. Psychol. Gen.
Cited by (6)
Varying Subjective Speed-accuracy Biases to Evaluate the Generalizability of Experimental Conclusions on Pointing-facilitation Techniques
2023, Conference on Human Factors in Computing Systems - ProceedingsAn Exploration of Freehand Crossing Selection in Head-Mounted Augmented Reality
2021, ACM Transactions on Computer-Human InteractionDistractor effects on crossing-based interaction
2021, Conference on Human Factors in Computing Systems - ProceedingsModeling the Endpoint Uncertainty in Crossing-based Moving Target Selection
2020, Conference on Human Factors in Computing Systems - ProceedingsInside-reachable and see-through augmented reality Shell for 3D visualization and tangible interaction
2020, Multimedia Tools and ApplicationsSteering through successive objects
2018, Conference on Human Factors in Computing Systems - Proceedings
- ☆
This paper has been recommended for acceptance by Oulasvirta Antti.