Contextually Adaptive Multimodal Mixed Reality Interfaces for Dismounted Operator Teaming with Unmanned System Swarms

Abstract

US dismounted Special Operations Forces operating in near-threat environments must maintain their Situation Awareness, survivability, and lethality to maximize their effectiveness on mission. As small unmanned aerial systems and unmanned ground vehicles become more readily available as organic assets at the individual Operator and team level, these Operators must make the decision to divert attention and resources to the control of these assets using touchscreens or controllers to benefit from their support capabilities. This paper provides an end-to-end overview of a solution development process that started with a broad and future-looking capabilities exploration to address this issue of unmanned system control at the dismounted Operator level, and narrowed to a fieldable solution offering immediate value to Special Operation Forces. An overview of this user-centric design process is presented along with lessons learned for others developing complex human-machine interface solutions for dynamic and high-risk environments.