Keywords

1 Introduction

Research in human-robot interaction (HRI) shows the recent transition for robots, from tools to teammates [1]. Aside from dismounted military members working with physical robots, a development is growing with the implementation of unmanned aerial (UA) and ground vehicles (UGVs) [2,3,4,5]. The interaction with robots is pursued through augmentation of interfaces. Interpretation of the information displayed through such interfaces often requires mental manipulation and interpretation by the human teammate or operator. This changes the work parameters of operators significantly and leads to the importance of considering individual differences in the ability to cognitively process input from the robot/UxV adequately [6]. This process occurs through mental rotation and visualization, both aspects of spatial ability [7]. However, the exact role is still largely unknown. What complicates research of the role of spatial ability, is that the construct is still unclear, despite decades of research.

1.1 Spatial Ability

Factor analyses have shown that spatial ability is a complex construct. In this section, we highlight two major efforts to discuss facets of spatial ability, one by Lohman, Pellegrino, Alderton, and Regian [7] and one by Carroll [8]. Table 1 summarizes and contrasts their factors and definitions of spatial ability.

Table 1. Operationalizations and factors of spatial ability [7, 8].
Full size table

Lohman and colleagues [7] identified ten factors of spatial ability, and Lohman summarizes this human aptitude as the “ability to generate, retain, retrieve, and transform well-structured visual images” [9] (p. 3). By contrast, Carroll’s [8] definition of spatial ability emphasizes a visual component: “spatial and other visual perceptual abilities have to do with individuals’ abilities in searching the visual field, apprehending the forms, shapes, and positions of objects as visually perceived, forming mental representations of those forms, shapes, and positions, and manipulating such representations mentally” (p. 304). His factor analysis of spatial ability classified five major factors (Table 1). The most complex factor is Visualization with several task types that load onto it, including block rotation tasks, assembly tasks, paper folding tasks, perspective-taking tests, and tasks that require participants to determine how an object should be formed into or broken down from a three-dimensional image [8, 10]. Spatial Relations, according to Carroll, constitutes Lohman et al.’s factor Spatial Orientation, with an emphasis on the speed in which the individual is capable of mentally manipulating simple visual patterns, such as determining whether one pattern is a rotated version of another image.

Recent research still scrutinizes these factors and newer measures of spatial ability have been generated [11, 12], indicating that congruence over the constituents of spatial ability has not been reached. This complicates the evaluation of the effect of spatial ability in human-robot interaction as well. However, as Carroll’s [8] analysis is the most recent factor analysis of spatial ability to date, this framework will be used in the remainder of this paper.

1.2 Purpose

The purpose of this paper is to review how spatial ability has been operationalized and measured in research of human-robot interaction (HRI) in the military field, including unmanned vehicles. This is a state-of-the-art assessment, focused on construct operationalization, measurement, and applied task types. The metadata will be synthesized to determine the degree of vergence in the assessment of spatial ability in military HRI research, and how this relates to task performance. The goal is to assess the generalizability of the current modus operandi and to generate recommendations where appropriate.

2 Method

2.1 Information Sources and Inclusion Criteria

The databases Compendex/EngineeringVillage, PsycInfo, and ProQuest were accessed to locate articles that were published in the last 10 years. The following keywords were all required for a study to be included in the qualitative analysis: (a) human-robot interaction (HRI) OR human-agent teaming; AND (b) military OR unmanned vehicle (UxV); AND (c) spatial ability.

2.2 Exclusion Criteria

Studies that included a robot but were not in the realm of the military or unmanned vehicles were excluded, such as assistance and surgical robots. Additionally, only studies that presented primary data from journal or conference papers were included. Papers primarily focused on predictive modeling were excluded as well, as the purpose of the present paper is to relate spatial ability, and its measures, to task performance. The remaining analyzable sample size was N = 23.

2.3 Data Extraction

The following data were extracted for analysis: labeled construct (i.e. the name assigned to the construct by the authors), spatial ability test, task type, and task performance outcome. Any effects on workload, situation awareness, or other outcome measures, were not considered. All data were imported into an Excel worksheet to categorize and qualitatively analyze the data.

3 Results

To evaluate the relationship between spatial ability and performance in military/UxV human-robot interaction, the metrics and constructs of spatial ability were qualitatively analyzed. Next, the tasks in which spatial ability was evaluated were categorized and the metadata were synthesized and related to performance outcome.

3.1 Spatial Ability Metrics

The reviewed publications utilized ten different tests of spatial ability. Figure 1 displays the frequency of the different tests, as well as an overview of the labels authors assigned to the construct associated with the test. The three most used tests were the Cube Comparison Test [13] (CCT), Spatial Orientation Test [14] (SOT), and Mental Rotation Test [15] (MRT). The labels that were most often used by researchers for the tests were “spatial ability” (SpA), “spatial orientation” (SpO), and “spatial visualization” (SpV). The next section will provide a more in-depth analysis the tests and each of these three labels.

Fig. 1.
figure 1

Tests of spatial ability in across included studies. Legend shows the labels authors assigned to the constructs measured by the tests.

Full size image

3.2 Spatial Ability Labels

Table 2 provides an overall summary of the ten spatial ability tests, the associated researcher-assigned label which is contrasted against the construct as categorized by Carroll’s [8] factor analysis, and a frequency of use of the test and label in the current sample. In this section, the three most used labels by researchers (SpA, SpO, SpV) will be discussed further.

Table 2. Overview of tests of spatial ability, label associated with the test, the construct associated with the test based on Carroll [8], and frequency of use in the current sample. Tests that were developed after Carroll’s analysis are coded as N/A for the construct.
Full size table

Spatial Ability (SpA).

SpA was measured 13 times in the current sample (N = 23) with four different tests: CCT, SOT, Hidden Patterns Test [13] (HPT), and Lathan and Tracey’s test battery [17] (LTB). When looking at the term spatial ability, ideally more than one test would be used to measure it, considering its multifaceted nature, such as in LTB. However, in the current sample, 40% of the studies implement one test to assess (a form of) spatial ability. The tests were further examined to compare them to Carroll’s [8] factor analysis.

In the CCT, individuals must evaluate a rotated cube and determine whether or not it is similar to the target cube. Carroll [8] determined that this test loads on the factor Visualization and Spatial Relations (Table 1).

The SOT is based on Gugerty and Brooks’ [14] cardinal direction task, wherein participants are presented with a north-up map that shows the location and (varied) heading of their aircraft and a ground target. Additionally, they see a forward view from the perspective of the aircraft, displaying the ground target, a building, and four parking lots around the building. Participants are required to indicate in which cardinal direction the parking lot with vehicles is (only one): north, east, south, or west of the building. This test was developed after Carroll’s [8] factor analysis; thus formal factor analysis was not applied. However, the perspective-taking properties suggest the test may load on the Carroll’s factor Visualization and potentially Spatial Relations.

The HPT is a task wherein participants are to determine whether or not a specific orientation occurs for the target item. This test loads on the factor Closure Flexibility [8].

The LTB refers to the CTY Spatial Test Battery [18], which includes a complex figure test, a spatial memory test, a block rotation test, and a cube perspective test. This battery was not evaluated by Carroll [8] but incorporates several facets of Visualization.

Spatial Orientation (SpO).

SpO was measured eight times with four different tests, from highest to lowest count: SOT, Perspective Taking Test [11, 12] (PTT), and CCT. As the SOT and CCT have been discussed, only the PTT will be discussed in this section.

The PTT is a more recent test that was developed based on evidence for a dissociation between mental rotation and perspective-taking spatial abilities [11]. In this task, individuals are asked to imagine different perspectives or orientations in space, imaginatively standing at one object while facing another object, and then pointing to where a third object would be. This test was developed after Carroll’s [8] factor analysis. As a perspective-taking task, this test would most likely be categorized as a factor of Visualization and Spatial Relations, although factor analysis needs to confirm this.

Spatial Visualization (SpV).

In the current sample SpV was measured seven times with five different tests: Paper Folding Test [13] (PFT), Part V of the Guilford-Zimmermann Aptitude Test [16] (V-GZ), Mental Rotations Test [15] (MRT), and the PTT, which has been discussed in the previous section.

The PFT shows successive drawings of a folded paper. The last image shows where a hole is punched. The individual then needs to select one out of five options to indicate where the hole would be if the paper is unfolded. This loads on the factor Visualization [8].

The V-GZ asks individuals to look at perspective scenes as if they are looking over the prow of a boat and to indicate how the boat moved between two views. Carroll [8] categorized this task as a Visualization task. The task was not analyzed for the Spatial Relations factor, as Carroll followed the lead of others such as Ekstrom and colleagues [13].

The MRT presents an image consisting of multiple stringed cubes, in different rotational angles. The individual needs to indicate which two of four images are rotated images of the target picture. As a block rotation task, his test loads on the factor Visualization [8].

3.3 Task Analysis

When evaluating the effect of spatial ability on HRI task performance, task type needs to be considered. Task type defines the specific capabilities required from the human teammate. Therefore, tasks were classified into categories, as shown in Table 3, matched with the spatial ability test used, Carroll’s [8] associated construct with the test, and a frequency within the current sample of studies. As expected from military HRI research, tasks are indeed representative of military reconnaissance missions.

Table 3. Categorization of task type with the applied test of spatial ability, associated construct based on Carroll’s [8] factor analysis, and a frequency of use in the present sample. Tests that were developed after Carroll’s analysis are coded as N/A for the construct.
Full size table

Robot control/teleoperation and target detection tasks were most frequently used, presumably as they are primary tasks in military reconnaissance missions with human-robot teams. The ability to perform robot control/teleoperation tasks has been evaluated with the spatial factors Visualization and Spatial Relations, which represent the ability to perceive and to mentally manipulate/rotate images to make a decision or judgment call for the next action, as well as the speed at which this is performed (Table 1).

Target detection ability in military HRI has been primarily associated with the spatial factor Visualization, followed by Spatial Relations, Closure Flexibility, Closure Speed and Perceptual Speed. Target detection indeed is a complex action sequence that involves perception of the environment, potentially mentally rotation depending on the field of view, as well as searching for and discriminating targets, and then marking them in the simulation. This sequence action is likely to hinge on speed of execution as well.

Secondary tasks of the simulated HRI missions in this sample include communication, route planning, target encapsulation, tacton classification when using tactile means of communicating, responding to warning systems, and controlling sensors. These tasks were associated with the spatial ability factors Visualization, Spatial Relations, Perceptual Speed, and Closure Flexibility.

3.4 Performance Outcome

Table 4 summarizes the key findings for task performance in relation to spatial ability, with a representation of the metrics used to measure (aspects of) spatial ability.

Table 4. Integration of relevant key findings for spatial ability according to task type.
Full size table

Table 4 indicates that, in general, primary military reconnaissance tasks are solely evaluated in relation to spatial ability, that is target detection and robot control/teleoperation performance. Secondary tasks such as communication or route planning are seldom evaluated in relation to spatial ability and generally did not show a significant relationship. The relatively small sample size and lack of congruence in utilized tests complicates a clear synthesis of the results.

Based on the findings from the studies in this sample, target detection performance seems to benefit from higher spatial ability. Several studies found a positive relation with capabilities represented by the factors Visualization and Spatial Relations. These findings imply that target detection relies, at least in part, on the ability to perceive, mentally manipulate and rotate, and discriminate or match visual images in a dynamic environment. Perspective-taking ability, through scene perception or block rotation, may be important additional aspects of this aptitude.

Robot control/teleoperation performance generally appears to be fostered by identical factors, i.e., Visualization and Spatial Relations. However, some of the reviewed studies did not find a significant relationship between the score on the spatial ability test and task performance [23], or only when an aid for the target detection performance task was not available [29]. Hence, there may be a weaker relationship between spatial ability and robot control/teleoperation performance than with target detection performance. Another possibility is that a significant relationship was not found due to the perspective used in the task. Indeed, a number of studies made a distinction between direct robot control and remote robot control through teleoperation, and found that teleoperation performance was specifically related to the CCT, while a composite of the CCT and PFT was related to direct robot control performance [25, 31, 32]. This finding indicates that the perspective used in the task may be a moderating factor in the relationship between spatial ability and robot control/teleoperation.

4 Discussion

The purpose of this paper is to review how spatial ability has been researched in military human-robot interaction, including unmanned vehicles (military/UxV HRI). Spatial ability is a multi-faceted construct [7, 8]. In this review, we focused on the factors of spatial ability as defined in the factor analysis by Carroll [8], which focuses on five factors of visualization, as part of spatial ability and as important aspects of interfacing [41]. Spatial ability, according to Carroll, emphasizes “individuals’ abilities in searching the visual field, apprehending the forms, shapes, and positions of objects as visually perceived, forming mental representations of those forms, shapes and positions, and manipulating such representations mentally” (p. 304), and is factor analytically decomposed in the factors Visualization, Spatial Relations, Closure Speed, Closure Flexibility, and Perceptual Speed. The goal of this literature review is to provide a state-of-the-art assessment of spatial ability to assess the generalizability of the current modus operandi and to generate recommendations for the highly specialized field of military/UxV HRI.

The first interesting finding is the relative scarcity of peer-reviewed journal publications, that provide primary data, in the realm of military/UxV HRI that analyze the effect of spatial ability. In a search of the recent decade, 23 viable publications were identified for qualitative analysis. The labels that were most often assigned by researchers to the measures of spatial ability were “spatial ability,” “spatial orientation,” and “spatial visualization.”

The term “spatial ability” is used fairly loosely in this sample. It was the most common measured label and was assessed with four different tests. When “spatial ability” is measured, several tests are required to adequately reflect the multifaceted nature of the construct. In this sample, 60% combined three tests at most when measuring “spatial ability”. This indicates that a large portion used merely one test when claiming to measure “spatial ability”, which connotes a lack of construct validity.

Furthermore, there is a lack of convergence in the definition, i.e., researcher-assigned labels, to the tests. These labels are often not verified by formal factor analysis that would strengthen the validity. One example is the most common used measure in this sample, the Cube Comparison Test [13], which the majority of the researchers labeled as a metric of “spatial ability.” A few researchers more accurately defined this test as “spatial visualization” and “spatial relations”, conform Carroll’s [8] extensive factor analysis. Another frequently used test is the Spatial Orientation Test [14], which was published after Carroll’s factor analysis. Even though the name of the test suggests would be a factor of Spatial Relations/Orientation, the fact that this test requires participants to shift perspectives, implies the task may be a factor of Visualization instead [8, 10].

Next, task types were analyzed in relation to spatial ability. The work parameters of the human interfacing with a robot (or unmanned vehicles) are different from the human actually being in the field. Interfacing with robots relies on the ability to mentally rotate and visualize the information presented through the interface [8, 41]. Therefore, there is a need to analyze the relationship between spatial ability factors and task types, especially in relation to performance. Task type defines the specific capabilities required from the human teammate and therefore should inform which spatial ability tests to implement in the assessment.

The task types used in this sample are similar to military reconnaissance tasks as applied to interactions with robots or unmanned vehicles. The primary tasks are robot control/teleoperation and target detection, with secondary tasks as route planning, communication, responding to system warnings, and tacton classification. In general, tests of Visualization and Spatial Relations have been indexed in relation to robot control/teleoperation tasks. Target detection tasks were represented by several factors of spatial ability, including Visualization, Spatial Relations, Closure Flexibility, Closure Speed, and Perceptual Speed. Target detection tasks may be more complex in nature, therefore relying on different aspects of spatial ability. Other factors that were not reviewed may also play a role in the versatility of the tests used, such as frame of reference of the robot/UxV and interface [41]. Furthermore, although secondary tasks were listed, few studies evaluated spatial ability in relation to secondary task performance. The secondary tasks were perhaps employed to simulate the complexity of a dismounted military reconnaissance scenario, wherein primary task performance is more critical to the mission.

Lastly, the performance outcomes were synthesized and evaluated to find potential patterns. Considering the relatively small sample size and lack of congruence in utilized tests, a generalization of the results was complex and the findings should be interpreted with great caution. There is a conservative indication that performance on robot control/teleoperation and target detection tasks relies at least on some form of spatial ability. Target detection performance may benefit from the ability to perceive, mentally manipulate and rotate, and discriminate or match visual images in a dynamic environment. Perspective-taking ability, through scene perception or block rotation, may be a factor that plays a role in target detection as well. Furthermore, in the current sample, robot control/teleoperation performance does not consistently show a relationship with higher spatial ability. Specifically, there may be a distinction between the aspects of spatial ability that affect direct robot control versus remote robot teleoperation. Thus, robot control/teleoperation performance may depend in some ways on the perspective or frame of reference used in the task.

4.1 Future Research

Based on this literature review, the scientific community is recommended to focus on creating a congruent foundation of assessments in the critical and specialized field of military/unmanned vehicle HRI. We need a general awareness of the factors that form spatial ability, based on validated factor analyses, and how these factors relate to the performance type that is required of the human based on the task. By creating congruence and specificity in how factors of spatial ability are measured, findings can be replicated, which contributes to the foundation of theory in this fast-growing field. Replicated findings, with consistent measures, contributes to the reliability and validity of findings, which is vital when the findings are to be generalized to real-world military missions.

Additionally, the synthesis of the metadata shows at least an indication of a relationship between certain aspects of spatial ability and primary military reconnaissance task performance. Future research should attempt to understand this relationship on a deeper level, by evaluating the effects of spatial ability in such tasks with consistency and with tests that are reflective of factors that are relevant to the context. Replication and validation are key. Furthermore, with a larger database of replicated findings formal meta-analysis can be conducted, which yields statistically tested and generalizable results.

Finally, when looking at the metadata, it seems that researchers tend to have a preference for the specific measures of spatial ability that are implemented. To some extent, this may be a valid choice based on the task paradigm used. However, researchers need to remain cautious of a researcher bias when selecting assessment metrics.

4.2 Limitations

The present study is limited in its scope and size, as only studies that relate to military operations and include robots or unmanned vehicles were included. This limits the generalizability of the results, although specificity is maintained. A selection bias may have occurred due to the small, specialized field in which the literature review was conducted. Quality of the results was attempted to be maintained by only selecting peer-reviewed journals. Furthermore, no formal statistical analyses were performed; therefore the results should be interpreted with caution and only be taken as an indication.

5 Conclusion

To enhance a fluent interaction between human and robot teammates in the military field, future design requirements need to be informed by the human teammate’s capabilities. Individual differences in spatial ability require special attention, as human-robot interfacing relies on the ability to mentally manipulate and interpret the communicated information to inform subsequent actions. The purpose of this literature review is to determine how spatial ability has been operationalized military/unmanned vehicle human-robot interaction, focusing on construct operationalization, measurement, and applied task types. Synthesis of metadata of the sample from the past decade shows that the operationalizations of spatial ability in this highly specialized field are divergent, with evidence of issues with construct validity. There is preliminary evidence for a relationship between aspects of spatial ability and primary military reconnaissance task performance. This relationship should be further investigated with consistent and validated measures of spatial ability that are selected based on the task type and demands posed on the human teammate, so that reliable generalizations can be made real-world military missions.