Wagging the Dog: Human vs. Machine Inference of Causality in Visual Sequences

Abstract

Causal inference among pairs of moving objects in a visual scene is compared between human observers and state-of-the-art methods in Machine Learning for causal inference. It is shown that while humans may perform intuitive and/or reasoned statistical decisions with the same overall level of accuracy as machines, they clearly exhibit biases (or priors) in their judgment and are thus able to make decisions based on much less information than is otherwise required by statistical decision algorithms. While there is no simple explanation for how humans perform this task, connectionist learning structures which implement simple time-delayed correlations (both automatic and deliberative) relying on short-term memory mechanisms may suffice to build complex bottom-up models of the physical world and the interaction therewith.