Evaluation of the Effectiveness of Machine-Based Situation Assessment

Abstract

This paper describes a technique for measuring the effectiveness of machine-based situation assessment, one of the levels of data fusion. Using the computer to perform situation assessment assists human operators in comprehending complex situations. The evaluation technique is an iterative one that utilises a metric to measure the divergence between the situation assessment and the ground truth in a simulation environment. Different pieces of divergent information can be weighted separately using methods based on the Hamming distance, the number of antecedents, or a Bayesian approach. The evaluation technique is explored using Random Inference Networks and shows promise. The results are very sensitive to the phase of the inference network, i.e. stable, critical or chaotic phase.