Abstract
This study examines how cognitive processes that support mission planning are influenced by the physical ability to touch and manipulate sand, compared to passively observing the same action. It employs a systematic investigation on terrain conceptual knowledge, terrain recognition and, landmark memory using the ARES sand table. Sand tables are topographic models that support learning through the physical creation of scenarios. In the military, sand tables support strategic exercises for soldiers to practice the process of collective decision making and communication. Operational mission planning typically occurs with one person shaping the sand, followed by a larger group of individuals observing. It is the relationship between the person shaping terrain and those observing that is of specific interest. A total of 96 participants were recruited, from the University of Central Florida (UCF) and the Center for Applied Brain & Cognitive Sciences (CABCS). Results indicate that physically shaping the terrain improved recognition but did not have an effect on conceptual knowledge or recollection of landmarks compared to observers. This experiment supports the need for further investigation to determine how tangible interaction can contribute to cognitive understanding.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
The human being uses the sense of touch to assist in understanding the world. Each sensation experience of touch, also known as a tactile experience, provides information to our brain [21]. A common tactile experience in the military is to represent topography using sand tables. Rehearsal and practice with sand tables are a part of standard military curriculum. A Sand Table Exercise (STEX) is a common military practice for learning terrain features in order to facilitate collective decision making and communication [2, 4, 30]. STEX usually consist of a group of soldiers being briefed on a table that was constructed by a single soldier who shapes the terrain on the table. Therefore, during a STEX only a single soldier will have the tactile experience of shaping the terrain while the majority observe the construction. This research looks at how the role of the individual (e.g., shaper vs. observer) impacts spatial and cognitive metrics related to terrain learning.
The role of the shaper and the role of the observer are expected to affect learning. Literature supports this in the concept of active vs. passive learning. Active learning consists of physically performing activities, while passive learning is characterized by observing those activities [19]. Active learning provides benefits of user engagement, increased motivation and higher-order thinking when synthesizing, analyzing, and evaluating information [5]. For example, research has shown that objects actively explored were recognized faster than objects that were passively viewed [17].
Activities that facilitate active learning by manipulating an interface are known as tangible interactions. The concept of tangible interaction has a foundation across the fields of computing, human-computer interaction (HCI) and product/industrial design [13]. The ability to engage physical touch during tangible interaction can provide a lower barrier to entry, increased user engagement, reduce cognitive load, and create the perception of an intuitive interface [15, 18]. However, Hornecker [14] points out that, just being intuitive is not enough to ensure learning gains. Rather it is consideration of the task, context, and user that maximizes the learning value of tangible interaction. Tangible interaction provides an opportunity for naturalistic interaction to accomplish a task at hand [7] and has been shown to increase speed and accuracy of tasks as well as increase awareness of other interaction types [10].
To support battlespace visualization for mission planning, after action review and the various types of combat interactions, the Combat Capabilities Development Command Solider Center has developed ARES, a distributed interactive visualization architecture. Of its various modalities this study used ARES real-time augmented reality-enhanced sand table. ARES sand table facilitates tangible interaction by intelligently adjusting topographic contour lines projections whiles the user shapes the sand. This is enabled by a combination of commercial, off the shelf (COTS) technologies including a Microsoft Xbox Kinect ā¢ sensor and a short-throw projector and software. The capabilities provided by the ARES sand table present a unique opportunity to evaluate the role of shaper versus observer, and the impact of active and passive learning during collective terrain construction.
1.1 Tangibility Metrics
As a first step to establishing a research design, we developed metrics that assess three knowledge areas; terrain conceptual knowledge, (i.e., knowing what a terrain feature is), terrain recognition, (i.e. recognizing familiar terrains, and landmark memory (i.e., recalling landmark locations relative to terrain features). Overall, three cognitive task were used working under the assumption that active learners would perform better [16]. These tasks were the Terrain Conceptual Knowledge Test (TCKT), Sketch Map Drawing, and Terrain Verification Test (TVT), which assess each of the knowledge areas respectively.
The Terrain Conceptual Knowledge Test (TCKT) assesses terrain knowledge related to terrain features. Terrain knowledge consists of fact-based material that is needed to perform a procedural task such building a terrain feature (e.g., hill). The TKCT consisted of a 40 multiple choice labeling task of pictures of terrain features (5 major, 2 minor). This task was administered prior to and following the experiment.
The Sketch Map Drawing Task involved participants drawing a sketch map of the locations of landmarks embedded in the terrain. Quantitatively analyzing sketch maps can be burdensome so we used the Gardony Map Drawing Analyzer (GMDA) software [12] to easily collect and analyze sketch maps. Participants were required to memorize the locations of eight items; and then recall them on a top-down map. The GMDA evaluates the relative and absolute position of the eight items, providing a variety of quantitative measures that can provide insight on how active versus passive participants remember specific aspects of landmark locations.
The Terrain Verification Task (TVT) evaluated the comprehension of three-dimensional terrain models. Participants were presented with several views of 3D terrain images and were tasked to decide whether the depicted terrain was the one that their group had created or whether it was different (i.e., distractors). To excel at this task, participants had to understand and translate their spatial mental representation of the terrain topography into new orientations as the TVT showed terrains from various angles.
1.2 Research Questions
The primary research questions guiding this experiment are:
-
1.
Is there a difference in terrain conceptual knowledge gained when the shaper constructs the terrain versus passively observing someone else shaping?
-
2.
Is there a difference in the ability to recognize terrain topography between the individual shaper and the individual observing?
-
3.
Is there a difference in landmark memory between shapers and observers?
1.3 Hypotheses
-
H1: The shaper will demonstrate greater terrain knowledge gains (post vs. pre-experiments) on the TCKT compared to the observer.
-
H2: The shaper will demonstrate improved accuracy in recognizing the shaped terrain on the TVT over the observer.
-
H3: The shaper will demonstrate a faster response time on the TVT compared to the observer.
-
H4: The shaper will demonstrate greater landmark placement accuracy on the GMDA over the observer.
2 Method
2.1 Participants
Study trials were conducted at the University of Central Florida (UCF) in Orlando FL, and at the Center for Applied Brain & Cognitive Sciences (CABCS) in Medford, MA. A total of 96 participants ranging in age from 18 to 35 were recruited. The UCF Institute for Simulation and Training, administers a research participation program named SONA which provides volunteer research studies for students in exchange for course credit. This study awarded UCF students one credit hour for participation. Tufts University students recruited at CABCS received payment of $20.00 in exchange for their participation. All participants were randomly selected as shaper or observer upon arrival.
2.2 Apparatus and Materials
The Augmented REality Sandtable (ARES) is a proof-of-concept is a traditional sand table, augmented with a commercial, off the shelf (COTS) projector, LCD monitor, PC, Microsoft Kinect Ā® and Xbox Controllers. ARES allows for the construction of topographic terrain maps through projection as well as the display of tactical graphics and military icons to support real time collaboration for mission rehearsal, planning and after action review. ARES was developed internally at the Combat Capabilities Command Soldier Center, and has been used and validated in several previous experiments [1, 5, 6, 8].
3 Experimental Design
The study employed a yoked control design to investigate the effect of tangibility (i.e., independent variable) on incidentally-learned terrain knowledge and spatial memory (i.e., dependent variables). Participant were placed in dyads and randomly assigned to either of two experimental conditions described below:
-
1.
Shaper condition: In this condition, one participant (i.e., the shaper) shapes various terrain features by hand as described and depicted in a PowerPoint presentation based on an Army Field Manual and included animated GIFs showing the motions needed to create each feature. Shapers were told that they would have someone watching them, but that this person is not assessing them.
-
2.
Observer condition: In this condition, one participant (i.e., the observer) watches the other participant (i.e., the shaper) hand-shaping terrain features. Their goal is to pay close attention to the shapers action but to not comment or influence the shaper in any way.
The dependent variables, described below, relate to the behavioral outcomes of interest:
-
1.
Terrain Conceptual Knowledge: Reaction time and accuracy, evaluated using a multiple choice classification assessment of terrain features.
-
2.
Terrain Verification Test: Reaction time, accuracy, and reaction time slope and intercept, relative to view angular disparity. These variables were evaluated using a terrain verification task that employed a 3-D terrain model generated by ARES software. Participants were required to make quick decisions (i.e., āThis is the terrain we madeā, or āThis is not the terrain we madeā), based on multiple pictures viewed from various angles, as well as comparable views from similar but non-identical 3-D models.
-
3.
Gardony Map Drawing Analyzer: Angle and Distance Accuracy. Participants were required to memorize the locations and positions of eight items (basic colored shapes) placed in the sand; then draw a 2-D map from a top-down view. The GMDA software evaluated the relative positional information, with respect to angles and distances, of the eight items derived from the 2-D drawings.
3.1 Procedure
In this experiment two participants are run simultaneously. One participant is randomly assigned the role of shaper and the other has the role of the observer. Both participantsā first sign informed consent documentation and are assessed on baseline knowledge of terrain features via computer administered surveys. Next they complete the building task in which the shaper is tasked to build basic terrain features and follow directions provided by a PowerPoint slideshow while the observer looks. The TCKT is administered prior to and immediately after the building task while the TVT and the GMDA are both administered after (see Fig. 1).
3.2 Generating TVT Stimuli
To provide terrain images for the TVT, alternative terrain features were created by the authors while participations completed the post-task TCKCT in a separate location. This involved creating alternative terrain by swapping the location of two terrain features. Before shaping the sand attention was paid to the height and orientation of the terrain features in order to make the swapped features resemble how the participant shaped them. In total three swaps were made, creating three sets of distractor terrains.
4 Results
4.1 Terrain Conceptual Knowledge
We first investigated response accuracy on the Terrain Conceptual Knowledge Test (TCKT) to determine differences in conceptual terrain knowledge acquisition between participant roles. We submitted participants response accuracy in the TCKT to a 2 (role: shaper, observer) Ć 2 (session: baseline, post-task) repeated-measures ANOVA.
This analysis revealed a significant main effect of session, F(1,39)ā=ā122.57, pā<ā.0001, \( \eta_{p}^{2} \)ā=ā.76, indicating that participants response accuracy increased after the building task relative to baseline.
This main effect was qualified by a significant session by role interaction, F(1,39)ā=ā4.52, pā=ā.04, \( \eta_{p}^{2} \)ā=ā.10. To examine this interaction further we conducted follow-up pairwise comparisons of the estimated marginal means with Bonferroni p-value adjustments. This analysis revealed no significant differences between roles in either session (all pāsā>ā.1). This finding suggests that while observers showed larger accuracy gains across sessions than shapers these gains did not result in significantly higher post-task knowledge.
We next investigated response times (RTs) on the TCKT. As is common in psychological experiments, RTs were positively skewed and so we applied the natural log transform prior to analysis. We then submitted log transformed RT to the same 2 Ć 2 repeated measures ANOVA. This analysis revealed a significant main effect of session, F(1,39)ā=ā107.23, pā<ā.0001, \( \eta_{p}^{2} \)ā=ā.73, indicating that participants response time decreased across sessions. No interactions emerged.
4.2 Terrain Recognition
We next investigated response accuracy on the Terrain Verification Task (TVT). This task presented images of the constructed terrain interspersed with alternative terrains (i.e., same, different) at different rotations (i.e., angular disparities), similar to the classic mental rotation task (MRT; Shepard & Metzler, 1971). As a first step, we submitted participants response accuracy in the TVT to a 2 (role: shaper, observer) Ć 2 (trial type: same, different) Ć 5 (absolute-valued angular disparity: 0Ā°, 45Ā°, 90Ā°, 135Ā°, 180Ā°) repeated-measures ANOVA. This analysis revealed a significant trial type x angular disparity interaction, F(4,156)ā=ā3.03, pā=ā.02, \( \eta_{p}^{2} \)ā=ā.07. For different trials, response accuracy remained relatively stable as a function of angular disparity. However, for same trials, accuracy increased as a function of angular disparity, peaking at 90Ā°, and then declined. No other main effects or interactions emerged (Fig. 2).
We next investigated RT on the TVT. As with the TCKT, RTs were positively skewed and so we applied the natural log transform prior to analysis. We then submitted log transformed RT to a 2 Ć 2 Ć 5 repeated measures ANOVA. This analyses revealed a main effect of angular disparity, F(4,36)ā=ā2.80, pā=ā.04, \( \eta_{p}^{2} \)ā=ā.24 (see Fig. 3).
This main effect was qualified by a marginally significant condition x angular disparity interaction, F(4,36)ā=ā2.58, pā=ā.054, \( \eta_{p}^{2} \)ā=ā.22. This result is reported due to its medium effect size which provides evidence against it being a false alarm (see Fig. 4).
4.3 Landmark Memory
Last we investigated landmark memory on participantsā maps using GMDA. We assessed two individual landmark measures provided by GMDA, angle and distance accuracy, which reflect the accuracy of inter-landmark relationships with respect to angles and distances, respectively. We first submitted angle accuracy to a one-way repeated measures ANOVA for role. This analysis revealed a marginal main effect of role, F(1,38)ā=ā3.84, pā=ā.06, \( \eta_{p}^{2} \)ā=ā.09. We next submitted distance accuracy to the same ANOVA which revealed a significant main effect of role, F(1,38)ā=ā5.79, pā=ā.02, \( \eta_{p}^{2} \)ā=ā.13, indicating observersā sketch maps better represented inter-landmark distance relationships than shapers.
5 Discussion
This study investigated how physical tangible interaction during terrain construction impacted conceptual knowledge, terrain recognition and landmark memory. Research has shown that active learning through engaging with an interface leads to better understanding than passive learning. Literature supports the use of tangible interaction to facilitate learning [3, 25, 26].
Empirical results have yielded mixed findings on the role of physical touch with some advocating for the use of tangibles [17, 26] others stating no difference [24], and others saying the applicability changes based on task [11, 27]. The outcome of this experiment yielded mixed results.
With respect to conceptual knowledge gains, the hypotheses was that the shaper would gain more terrain knowledge (post vs. pre-experiments) than the observer. As it turned out, this experiment showed no difference between the two groups in terrain knowledge gains. The lack of difference could be due to the task creating a high or low workload state that, in turn, created a ceiling or floor effect respectively. It is also possible that the shapers were in a higher workload state than the observers given that they were responsible for shaping the terrain (i.e., the shapers had more work/tasking). This possibility could have created a masking situation which could not be accounted for in this experiment without a way to address workload as a covariate.
The terrain verification test demonstrates the orientation of content matters through the effect of angular disparity. Participants demonstrating highest accuracy at 90Ā° rotation. The peaking at 90Ā° could be due to the experimental setup where participants tended to move from a 0-degree position to a 90-degree position as the shaper was building the features, therefore providing experience at visualizing from that perspective. This advocates for future studies to carefully control participant position relative to the overall table.
The results from the GMDA were in the opposite direction than expected, with observers performing better than shapers in terms of landmark locations. This finding can be explained due to the differences between roles during the landmark placement task.
Since the observers themselves did not have to be concerned with the positioning of landmarks, they had the ability to maintain an overview of the landscape, whereas the shaper might have been focusing on the task. This focusing could change the key elements recalled, such that shapers and observers emphasized different information. This is supported in the literature through the concept of attentional narrowing. Attentional narrowing is defined as when an individual involuntarily fails to process critical information [23, 29].
If there is an assumption that the observer was experiencing less attentional narrowing and more overview related to the landmarks, this overview would provide them with an advantage on the post-assessment test. This is relevant to sketch drawing map task because the task provides a top down overview of the terrain. Therefore, when it came to taking the post-assessment using an overview image, the observer had less translation to do relative to what they witnessed during the test.
6 Future Research
The task of shaping terrain features was selected because it was a constrained space with clearly defined differences which enabled a foundation for a baseline assessment to be conducted. However, the continued focus of progressing this research to meet the needs of the operational soldier, it will be necessary to create future iterations of this research to be more representative of military sand table scenarios.
From a practical perspective, it is not going to be possible to have each solider shape their own terrain with a physical sand table. However, most soldiers do have access to mobile devices (e.g., smartphone). Since the literature already supports the value of active versus passive learning and research has demonstrated the value of mobile AR devices [16], the next logical step is to provide similar active learning experiences on a mobile device. It may be possible to run a similar experiment where instead of the participant manipulating sand, the participant shapes the terrain using pinching gestures onto a mobile device. This would provide clarity to the type of physical activity necessary to support cognitive process.
7 Conclusion
This research study established a foundation for understanding cognitive processes associated with constructing military terrain features and landmark identification. Data shows that physically shaping the terrain assisted with recognition but did not have an effect on conceptual knowledge or recollection of landmarks. Results indicate the need for further research to better understand how active learning can support recognition and memory. With the increased reliance of technology in military training, providing practical recommendations as to how to best implement tangible interaction to support knowledge acquisition is valuable for operational decision making and training. Through quantitative metrics and the evaluation of spatial representations and modelling, this research provides insight on the cognitive affordances of using AR for learning terrain topography, recognizing terrains, and recalling landmark locations. With the development of technologies such as ARES, determining how active learning can support cognitive processes can serve as a guide to identify appropriate technology for operational decision making and future classroom applications.
References
Abich, J., Eudy, M., Murphy, J., Garneau, C., Raby, Y., Amburn, C.: Use of the Augmented REality Sandtable (ARES) to enhance army CBRN training. In: Stephanidis, C. (ed.) HCI 2018. CCIS, vol. 851, pp. 223ā230. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92279-9_30
Amburn, C., Vey, N., Boyce, M., Mize, J.: The Augmented REality Sandtable (ARES). Army Research Laboratory (2015)
Antle, A., Wang, S.: Comparing motor-cognitive strategies for spatial problem solving with tangible and multi-touch interfaces, pp. 65ā72. ACM Press (2013)
Brewseter, F.W.: Using tactical decision exercises to study tactics. Mil. Rev. 82, 3ā9 (2002)
Bonwell, C.C., Eison, J.A.: Active learning: creating excitement in the classroom. School of Education and Human Development, George Washington University, Washington, DC (1991)
Boyce, M.W., et al.: The impact of surface projection on military tactics comprehension. Mil. Psychol. (American Psychological Association) 31(1), 45ā59 (2019)
Bouabid, A., Lepreux, S., Kolski, C.: Design and evaluation of distributed user interfaces between tangible tabletops. Universal Access in the Information Society (2017)
Boyce, M.W., et al.: Effect of topography on learning military tactics - integration of Generalized Intelligent Framework for Tutoring (GIFT) and Augmented REality Sandtable (ARES). Army Research Laboratory (2016)
Boyce, M.W., et al.: The impact of surface projection on military tactics comprehension. Mil. Psychol. 31, 45ā59 (2018)
Cherek, C., Brocker, A., Voelker, S., Borchers, J.: Tangible awareness: how tangibles on tabletops influence awareness of each otherās actions. In: CHI Conference on Human Factors in Computing Systems (2018)
Clifton, P., Chang, J., Yeboah, G., Doucette, A., Chandrasekharan, S., Nitsche, M.: Design of embodied interfaces for engaging spatial cognition. Cognit. Res. Principles Implications 1, 24 (2016)
Gardony, A.L., Taylor, H.A., BrunyĆ©, T.T.: Gardony map drawing analyzer: software for quantitative analysis of sketch maps. Behav. Res. Meth. 48, 1ā27 (2015)
Hornecker, E., Buur, J.: Getting a grip on tangible interaction: a framework on physical space and social interaction. In: Proceedings of the CHI 2006, pp. 437ā446 (2006)
Hornecker, E.: Beyond affordances: tangiblesā hybrid nature. In: Proceedings of TEI 2012, pp. 175ā182. ACM (2012)
Hornecker, E.: Understanding the benefits of graspable interfaces for cooperative use. In: Proceedings of Cooperative Systems Design, pp. 71ā87 (2002)
James, K.H., Humphrey, G.K., Vilis, T., Corrie, B., Baddour, R., Goodale, M.A.: āActiveā and āpassiveā learning of three-dimensional object structure within an immersive virtual reality environment. Behav. Res. Meth. Instrum. Comput. 34, 383ā390 (2002)
Kim, M.J., Maher, M.L.: The impact of tangible user interfaces on spatial cognition during collaborative design. Des. Stud. 29, 222ā253 (2008)
Klemmer, S.R., Hartmann, B., Takayama, L.: How bodies matter: five themes for interaction design. In: Proceedings of the DIS, pp. 140ā149. ACM Press (2006)
Krathwohl, D.R.: A revision of Bloomās taxonomy: an overview. Theory Pract. 41(4), 212ā218 (2002)
Michel, N., Cater, J., Varela, O.: Active versus passive teaching styles: an empirical study of student learning outcomes. Hum. Resour. Dev. Q. 20(4), 397ā418 (2009)
Obrist, M., Seah, S., Subramanian, S.: Talking about tactile experiences, pp. 1659ā1668 (2013)
Ozdemir, M., Sahin, C., Arcagok, S., Demir, M.K.: The effect of augmented reality applications in the learning process: a meta-analysis study. Eurasian J. Educ. Res. (EJER) 74, 165ā186 (2018)
Prinet, J.C., Mize, A.C., Sarter, N.: Triggering and detecting attentional narrowing in controlled environments. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 60, no. 1, pp. 298ā302. Sage, Los Angeles (2016)
Richardson, R., Sammons, D., Delparte, D.: Augmented affordances support learning: comparing the instructional effects of the augmented reality sandbox and conventional maps to teach topographic map skills. J. Interact. Learn. Res. 2, 231 (2018)
Schneider, B., Blikstein, P.: Unraveling studentsā interaction around a tangible interface using multimodal learning analytics. J. Educ. Data Mining 7, 3 (2015)
Schneider, B., Jermann, P., Zufferey, G., Dillenbourg, P.: Benefits of a tangible interface for collaborative learning and interaction. IEEE Trans. Learn. Technol. 4(3), 222ā232 (2011)
Skulmowski, A., Pradel, S., KĆ¼hnert, T., Brunnett, G., Rey, G.D.: Embodied learning using a tangible user interface: the effects of haptic perception and selective pointing on a spatial learning task. Comput. Educ. 92ā93, 64ā75 (2016)
Vogel-Walcutt, J.J., Fiorella, L., Malone, N.: Review: instructional strategies framework for military training systems. Comput. Hum. Behav. 29, 1490ā1498 (2013)
Wickens, C. D.: Attentional tunneling and task management. In: Proceedings of the 13th International Symposium on Aviation Psychology, pp. 620ā625 (2015)
Wildland Fire Lessons Learned Center. https://www.fireleadership.gov/toolbox/documents/TDGS_STEX_Workbook.pdf. Accessed 5 Feb 2019
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Boyce, M.W. et al. (2019). Characterizing the Cognitive Impact of Tangible Augmented Reality. In: Chen, J., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. Multimodal Interaction. HCII 2019. Lecture Notes in Computer Science(), vol 11574. Springer, Cham. https://doi.org/10.1007/978-3-030-21607-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-21607-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-21606-1
Online ISBN: 978-3-030-21607-8
eBook Packages: Computer ScienceComputer Science (R0)