This paper establishes a duality between the calculus of variations, an increasingly common method for trajectory planning, and Hidden Markov Models (HMMs), a common probabilistic graphical model with applications in artificial intelligence and machine learning. This duality allows findings from each field to be applied to the other, namely providing an efficient and robust global optimization tool and machine learning algorithms for variational problems, and fast local solution methods for large state-space HMMs.