Keywords

1 Introduction

The Army Learning Concept for 2015 [1] calls for implementing technology-enabled training for demanding tasks within the operational environment. One such role is that of the Joint Fires Observers (JFO), who serve as the link between dismounted units and supporting fires elements. The JFO task domain, roles, and responsibilities require complex decision-making in high-risk environments. Existing Simulation-Based Training (SBT) systems offer increased opportunities for training and certification, and have demonstrated training efficacy, but fail to fully resolve throughput issues. Furthermore, fielded JFO SBT systems lack portability and their associated costs (e.g., hardware, role players) can be prohibitive. One way of reducing cost is to leverage Augmented Virtuality (AV), which theoretically provides comparable cognitive fidelity to augmented reality with less cost. A key element of designing SBT systems is the need to identify what elements of the operational task experience need to be replicated in the simulation environment (i.e., simulator fidelity). Too much fidelity may result in overly expensive systems that do not necessarily provide better training. Too little or the wrong kind of fidelity may result in systems that do not support the required training. Thus, a careful balance is needed to achieve the best Return on Investment (ROI). In order to more fully understand how AV technologies and methods impact Call for Fire (CFF) training effectiveness and compare to existing systems, an empirical evaluation is required. ARL-HRED-STTC is conducting a Training Effectiveness Evaluation (TEE) to evaluate the role of an AV training systems for effectively training CFF as well as inform the academic and applied training literature with regard to the effectiveness of AV as a platform. The goal of the CFFT-AV evaluation is to blend live and virtual worlds to provide individual and team training. The objective of this research is to assess the training effectiveness of AV within the CFF task domain. Specifically, this effort will address four research questions:

  1. 1.

    How much more or less effectiveness does the AV technology, device, or method provide compared to the existing CFFT II system?

  2. 2.

    How much more or less does the AV technology, device, or method of training costs compared to the existing CFFT II system?

  3. 3.

    Does the AV technology provide the appropriate level of fidelity to enable accomplishment of both individual and team learning objectives?

  4. 4.

    What is the ROI of the AV technology, device, or method for CFF training?

This paper describes the TEE CFF to inform how AV technologies and methods impact CFF training.

2 Background

2.1 The Operational Domain

A CFF is a request to execute an attack on a selected target using a specified method of fire [2, 3]. Requests are initiated by a JFO who serves as the communication link between dismounted and fire support units [3]. During a CFF mission, the JFO takes an observation post usually located some distance from the fight, but positioned to enable him to visually locate and identify targets and clearly spot fire rounds [2, 3]. From his post, the JFO identifies the target location and determines an appropriate method of fire. The JFO operational setting is a high-risk and dynamic environment that requires complex decision-making abilities. When selecting the method of fire, JFOs must consider the threat composition and location, position of friendly and neutral elements, potential terrain effects, and direction of incoming fires [3]. Once the firing method is selected, the JFO transmits the CFF request to a Fire Direction Center (FDC) that determines the availability of the requested fire support and disseminates the fire commands to a Firing Unit. Due to the criticality and complexity of the CFF domain, CFF training is highly involved, requiring a JFO trainee to demonstrate proficiency in three specific duty areas. First, essential training and classroom academics include a demonstrated understanding of the individual tasks and information necessary to perform a CFF. Specifically, this portion of the training cycle facilitates a trainee’s ability to evaluate current tactical situations, capabilities of fixed and rotary wing aircraft, effects of weather and terrain, and nuances of radio communications. Next, mission preparation provides familiarization with equipment and effective communication with fire support units. Finally, CFF execution requires a sound understanding of the task domain and demonstrating so through live or simulated execution of the CFF task [4].

2.2 Joint Fires Observer Course

The Army established the Joint Fires Observer Course (JFOC) to train and certify Soldiers in the Tactics, Techniques and Procedures (TTP) learned during combat operations in Afghanistan and Iraq. The course integrates simulation systems in order to enhance the capabilities of Soldiers attending the JFOC and/or Soldiers conducting semi-annual currency requirements. Training Soldiers in a VE has been effective, but falls short of actual training in the field or while deployed. However, virtual training increases the proficiency of the Soldier prior to live training and, ultimately, combat operations. These simulations evaluate a student’s ability to conduct several of the hardest simulation examinations during the JFOC that have a historically high failure rate. These include conducting a Type 2 Close Air Support (CAS) mission with a Joint Terminal Attack Controller (JTAC), estimating distance to a known (or unknown) point, utilizing a laser designation device such as the Special Operations Forces Laser Rangefinder Designator (SOFLAM) or the Lightweight Laser Designator Rangefinder (LLDR) and calling for and adjusting Naval Gunfire (NGF). The Army JFO program is designed to help fill the void that currently exists between the number of JTACs the Army says it needs to conduct operations and what the Air Force can currently provide. The program focuses on training the “Company Fire Support Officers/NCOs, Platoon Forward Observers, Combat Observation Lasing Teams and members of scout/reconnaissance organizations. These individuals will be taught the skill sets necessary to assist JTACs in conducting Type II and Type III CAS. The Army’s goal is to have one JFO per maneuver platoon. The JFO Memorandum of Agreement (MOA), that has been signed between the Departments of Army and Air Force and the United States Special Operations Command, defines the JFO as: “A trained service member who can request, adjust, and control surface-to-surface fires, provide targeting information in support of Type II and III CAS terminal attack controls, and perform autonomous Terminal Guidance Operations (TGO).

2.3 Development of the Call for Fire Trainer

Beginning in the mid 1990’s call for fire system trainers were evolved for various user groups and agencies. The earliest system which has been fielded for the United States Army and is still operational is the Call for Fire Trainer (CFFT) which is currently supported by The Program Executive Office for Simulation, Training and Instrumentation (PEO-STRI). CFFT/CFFT II is an initial call for fire system primarily designed for a classroom environment. The CFFT is fielded around the world in various configurations utilizing an instructor station, one or more enhanced student stations, and a number of additional stations where a student can sit and observe the battlefield view in the front of the classroom. The CFFT is an individual and/or collective training system that provides a simulated battlefield environment for training Forward Observers (FOs) at the institutional and unit level. The system is designed to be transportable and provide advanced distributed learning, Simulated Military Equipment (SME), virtual environments, and computer-generated forces using One Semi-Automated Forces (OneSAF). The CFFT II is a technology upgrade of the guard unit armory device full-crew interactive simulation trainer II (GUARDFIST II). In addition to the 19 basic fire support tasks trained on the GUARDFIST II, the CFFT II can be used to train CAS, NGF, mortar registration, and suppression of enemy air defense tasks. The importability of terrain databases, combined with the inherent flexibility of the OneSAF Test Bed (OTB) scenario, also enhances the realism of the training environment. The CFFT II consists of instructor and student stations. There are two variations of the student station, standard and enhanced. The standard student station is composed of a map and a set of binoculars. The enhanced student station is composed of SME such as the simulated LLDR, virtual military equipment such as virtual Night Vision Goggles (NVGs), and a student control computer. The CFFT II can be configured to accommodate 4, 12, or 30 student stations (1:4, 1:12, and 1:30 configurations). Regardless of the configuration, each CFFT II has at least one enhanced student station and one instructor station; the remainder of the students use standard student stations.

2.4 Joint Fires Effects Training System (JFETS)

The Joint Fires Effects Training System (JFETS) is an evolutionary development of the approved CFFT. It was developed as a test bed at the Fires Battle Lab, Ft. Sill to test new concepts of inserting higher fidelity into the fire support training environment. JFETS is an immersive trainer consisting of tailored modules for Open Terrain (OTM), Urban Terrain (UTM), and close air support (CASM). Two additional modules for After Action Reviews (AAR) and Fires Effects Coordination (FECM) comprise the entire developmental JFETS system. The open terrain, urban terrain, and close air support modules are characterized by a robust visual system embedded in a training space designed to emulate geographical or cultural aspects of the southwest Asia operational area. However, there is a lack of empirical data available to assess the impact of the “immersive” aspects of the developmental JFETS upon trainee performance. Anecdotal data indicate the current configuration may facilitate performance by improving user “buy-in.” As with the CFFT-AV, no empirical data have been collected to support interview responses on this issue. The data provide evidence to conclude that very high fidelity visual systems and realistically modeled (for form, fit, and function) SME produce improved training transfer. The developmental JFETS has been anecdotally reported by the users as an effective training system. There is a desire to advance the training system and more fully integrate it within existing Field Artillery School core curricula. One challenge hindering full integration is throughput due to current configuration parameters and a lack of design recommendations to move integration forward.

2.5 Immersive Dome Technologies

The military training enterprise has explored alternative dome projection technologies for JTAC and JFO training domains. An example of such a dome is shown in (Fig. 1) that is in use at the JTAC School at Nellis, AFB. Domes have been attractive for JTAC/JFO training because of the immersive nature of the dome environment.

Fig. 1.
figure 1

JTAC dome

Full size image

There are two primary approaches to dome architecture, rear projection and front projection. The two main advantages of rear projection are, no projectors are visible to the student, and the student can use tactical night vision goggles to enhance the light projected from the rear. However, there are several disadvantages of the dome architecture. Traditionally, domes have a very large footprint to accommodate the projector throw which hampers point of need training device availability. Additionally, the light level inside domes is diminished by the light attenuation of the screen material and the curved shape of screens present problems in edge blending and projector alignment with the multiple projectors. The geometry in domes also presents various problems. There is a discontinuity where the flat top meets the curved sides and sides tend to have a slight hour-glass shape, which impact visual requirements such as arc and pixel. There are a number of commercially available front projection domes that could be used to meet training requirements and have much smaller footprint, a brighter display, and would present far fewer problems with edge blending and alignment. Regardless, the dome approach is primarily regarded as an option due to the immersive nature that engages the trainee better than traditional projection found in a CFFT type device.

2.6 Call for Fire Trainer – Augmented Virtuality

The Call For Fire Trainer – Augmented Virtuality (CFFT-AV) is a prototype training device designed within the context of the Reality-Virtuality (RV) continuum, specifically for the Joint Forward Observers (JFOs) use case. Augmented Virtuality (AV), the augmentation of a virtual setting with real objects provides a means to merge a richly layered, multi-modal, 3D real experience into a Virtual Environment (VE). The essence of this approach is to provide students with a mixed reality experience by means of an optical see-through head-mounted display (HMD). With this kind of interface, it is possible to create voids in the rendered scene graph of a virtual environment allowing the real-world, physical environment surrounding the student to be viewed normally. With a high-resolution tracking solution, the real-world objects produce scene-graph voids created on-the-fly to match the object’s size, position and orientation relative to the student. This capability eliminates the need in current technology for students to remove HMDs to view real-world objects. In addition to the use of real-world objects, the CFFT-AV incorporates a novel approach to the design of surrogates to simulate additional equipment needed for training in the CFF task domain. This approach utilizes tracked, inert objects with the general shape and heft of devices like M22 and Vector 21 Binoculars, Lensatic Compass, Defense Advanced GPS Receiver (DAGR), Infrared Zoom Laser Illuminator Designator (IZLID), LLDR, and the M4 weapon. These surrogates include knobs, buttons and switches that provide USB inputs to the system. As they are tracked, the objects appear as corresponding virtual models in the student’s field of view. For devices that have a reticle view, the HMD’s view switches to a rendered frustum of the near-eye device view when brought in proximity to the student’s eyes (Fig. 2). Fabricated from inexpensive material, these surrogates offer a significant cost advantage over the SME commonly employed in existing SBT systems. This mix of real-world and surrogate objects mitigates a principal criticism of current virtual reality CFF training systems that require a student to transition between real-world and virtual world interfaces. Such use cases are generally viewed as awkward and distracting, significantly decrementing the immersive qualities of the simulation and the student’s suspension of disbelief.

Fig. 2.
figure 2

Example of an M22 binocular rendered frustum

Full size image

2.7 Mixed Reality – Virtuality Continuum

Milgram [5, 6] describes a taxonomy that identifies how augmented reality and virtual reality are related. He defines the Reality-Virtuality continuum shown in Fig. 3. The real world and a totally virtual environment are at the two ends of this continuum with the middle region called Mixed Reality (MR). Augmented Reality (AR) lies near the real-world end of the spectrum with the predominate perception being the real-world augmented by computer generated data. AV is a term created by Milgram to identify systems that are mostly synthetic with some real world added, such as the CFFT-AV. AV, the augmentation of a virtual setting with real objects provides a means to merge a richly layered, multi-modal, 3D real experience into a VE [7]. Design industry paves the road for more innovations in AV [9], a more expansive form of VR. Despite its potential, AV has not received as much attention as VR and AR. AV has only been applied in very limited domains: as displays on unmanned air vehicles [10], 3D video-conferencing systems [11] and a scientific center [12]. Recognized research effort towards AV applications in the military training domain is fairly limited. .

Fig. 3.
figure 3

Simplified representation of reality-virtuality continuum

Full size image

3 Training Effectiveness Evaluation

3.1 Problem Statement

Existing JFO SBT systems offer increased opportunities for training and certification, and have demonstrated training efficacy, but often do not provide an immersive learning environment. Furthermore, fielded JFO SBT systems have fallen short at addressing the “point of need” training gap and their associated costs (e.g., hardware, support personnel) can be prohibitive. One possible way of reducing cost and increasing immersion is to leverage AV, which theoretically provides comparable cognitive fidelity to AR with less cost. While certain technical challenges persist, a necessary step remains: evaluating the value and effectiveness of AV simulation by identifying what elements of the JFO training experience need to be replicated in the simulation environment (i.e., simulator fidelity). Excessive fidelity may result in a cost prohibitive system that does not provide a measurable improvement in training, while too little or the wrong kind of fidelity may result in a system that does not support the required training requirements. Thus, a careful balance is required to achieve the best ROI for the training acquisition community. In order to more fully understand how AV technologies and methods impact JFO training effectiveness and compare to existing SBT systems, an empirical evaluation is required. This assessment needs to take into account the acquisition and retention of procedural knowledge and cognitive decision-making capability. Practical AR is viewed by many in the military enterprise as the objective end-state for the future of U.S. ground forces training. The goal of this evaluation is to assist the U.S. Army decision making process of future training system acquisition with respect to the JFO use case, combining the flexibility and lower cost of simulation with the fidelity of live training by incorporating mixed reality technologies.

3.2 Methodology

The purpose of this research is to understand and explore the application of AV technologies, devices, and methods aimed at small unit leader and individual Soldier tasks. The objective of the present research effort is to evaluate the training effectiveness of AV within the CFF task domain. The CFFT-AV system includes a head-mounted display (HMD), game-engine software, motion tracking, a compass, binoculars, and a terrain map. The evaluation will focus on the system’s operational and training capabilities and will include four human subject experiments. A brief description of each experiment is provided below.

Experiment 1: Initial Functional Testing. The first experiment will be conducted at UCF-IST utilizing a representative U.S. Army population (e.g., UCF ROTC Cadets). The purpose of Experiment 1 is to observe and collect data on user interaction, subjective responses, and training effectiveness of the CFFT-AV system. Training and performance evaluation utilizing the CFFT-AV will be facilitated by a subject matter expert in collaboration with the experimenters on site. Experimental personnel will collect performance evaluation data and will administer subjective questionnaires related to the participants’ experiences during training within the CFFT-AV system.

Experiment 2: Baseline Data Collection. The second experiment will be conducted at a representative U.S. Army facility (e.g., Ft. Sill, OK) and utilize active duty U.S. Army personnel as participants. The purpose of Experiment 2 is to obtain baseline performance data on the use and effectiveness of the traditional CFFT II training system. The CFFT-II system uses a projected display with accompanying tools to project augmented reality information on the screen to aid in training of the Call for Fire task. Operation of and interaction with the traditional training system will be facilitated by U.S. Army and contracted instructors on site. Experimental personnel will collect instructors’ performance evaluation data and will administer subjective questionnaires related to the Soldiers’ experiences during training within the CFFT II system.

Experiment 3: AV Evaluation (Phase 1). The third experiment will be conducted at a representative U.S. Army facility (e.g., Ft. Sill, OK) and utilize active duty U.S. Army personnel as participants. The purpose of Experiment 3 is to evaluate the training effectiveness metrics of the CFFT-AV system and compare those results to the traditional CFFT II system results obtained during Experiment 2. The task scope for the metrics evaluated will focus on basic CFF skills. Soldier training and performance evaluation utilizing the CFFT-AV will be facilitated by active duty or reserve U.S. Army instructors on site. Experimental personnel will collect instructors’ performance evaluation data and will administer subjective questionnaires related to the Soldiers’ experiences during training within the CFFT-AV system.

Experiment 4: AV Evaluation (Phase 2). The fourth experiment will be conducted at a representative U.S. Army facility (e.g., Ft. Sill, OK) and utilize active duty U.S. Army personnel as participants. The purpose of Experiment 4 is to evaluate the training effectiveness metrics of the CFFT-AV system and compare those results to the traditional CFFT II system results obtained during Experiment 2. The task scope for the metrics evaluated will include advanced CFF skills. Soldier training and performance evaluation utilizing the CFFT-AV will be facilitated by active duty or reserve U.S. Army instructors on site. Experimental personnel will collect instructors’ performance evaluation data and will administer subjective questionnaires related to the Soldiers’ experiences during training within the CFFT-AV system.

Advancements in technology in the recent past have led to immersive 3D, 360 simulations being used successfully to train both civilian and military aviators. The level of this success has led those in the military to seek ways in which to incorporate this technology in other areas of military training. It is hoped that by using immersive VEs a Soldier may be provided training opportunities not afforded in the traditional classroom or in field exercises and can do so without the risk of injury. Moreover, supplemental training in a simulator can greatly reduce the costs associated with using live ammunition during field exercises. Studies designed to ascertain just how humans acquire new concepts and knowledge through the use of immersive and virtual environments lag behind the development and utilization of such sophisticated technology. People often attest that learning in an immersive environment is enjoyable, engaging, and beneficial; however, actual empirical data supporting improved performance is less available.

4 Conclusion

This TEE will provide empirical data to assess the effectiveness of MR, specifically in the area of AV. Undoubtedly, more research will be required to determine if training in immersive environments transfers to live training environments more readily than training in traditional technical training environments. This effort is intended to assess if there are measurable performance and perception differences between Soldiers trained in an AV environment and those trained in a traditional classroom environment. Furthermore, this investigation will inform future research assessing the impact of immersive technologies on the transfer to tactical environments and Warfighter skill sustainment. At a minimum, the CFFT-AV TEE will assist in promoting effective implementation of technology-enabled training for JFO tasks in CFF operations and assist the U.S. Army decision making process of future training system acquisition with respect to the JFO use case.