Computer Models of Sensor and Weapon Systems: Assessing Model Utility Early in War-Game Development

Abstract

“All models are wrong, but some are useful” as the statistician George Box famously remarked (1979). Indeed, all computer models entail approximations that make them quantitatively “wrong” to some degree. Yet some models can be useful, likewise to a degree, for some applications but not for all. This holds when computer models of sensor or weapon systems are used to simulate actual system performance during the action of the game. In war gaming (broadly defined here as any application relying on models of sensor & weapon systems), a model is useful insofar as it supports the outcomes of the gaming, which may be education, training, tactical evaluation, force optimization, concept development, and so forth. To prove that a model suffices for given objectives is difficult, costly, and usually inconclusive. An alternate method for evaluating model adequacy early in a project in light of gaming objectives is described here. The method is based on well-known principles of hypothesis testing. It is therefore 1) objective and evidence based; 2) applicable to a wide range of computer models and gaming objectives; 3) requires moderate technical expertise, without requiring war-game developers to venture into the specialization of model developers. The method provides a framework for soliciting and evaluating the available evidence for model adequacy in light of gaming objectives, when allocating resources and exploring modeling options and suitability, early in a project.