A reinforcement learning approach to convoy scheduling on a contested transportation network

Abstract

Ambushes, in the form of improvised explosive devices (IEDs), have posed grave risk to targeted vehicles operating on supply routes in recent theaters of war. In fact, history shows that this is an enduring problem that US military forces will likely face again in the future. This paper introduces a fundamental reinforcement learning (RL) model for determining convoy schedules and route clearance assignments in light of attack costs on a transportation network subject to IED ambushes. The model represents opponent interaction by assuming dependence between attack probabilities and targeted traffic patterns. There are currently few analytical approaches for this problem in the literature, but RL algorithms offer opportunities for meaningful improvements by optimizing individual movements across an extended planning horizon, accounting for downstream attacker-defender interaction. To our knowledge this approach has not been pursued elsewhere; therefore, this paper introduces the RL methodology with a fundamental formulation and initial computational results which show meaningful performance improvements over a one-step, myopic decision rules.