The Variational Principles of Action

Abstract

This chapter provides a theoretical perspective on action and the control of movement from the point of view of the free-energy principle. This variational principle offers an explanation for neuronal activity and ensuing behavior that is formulated in terms of dynamical systems and attracting sets. We will see that the free-energy principle emerges when considering the ensemble dynamics of biological systems like ourselves. When we look closely what this principle implies for the behavior of systems like the brain, one finds a fairly straightforward explanation for many aspects of action and perception; in particular, their (approximately Bayesian) optimality. Within the Bayesian brain framework, the ensuing dynamics can be separated into those serving perceptual inference, learning and behavior. Variational principles play a key role in what follows; both in understanding the nature of self-organizing systems but also in explaining the adaptive nature of neuronal dynamics and plasticity in terms of optimization—and the process theories that mediate optimal inference and motor control. A special focus of this chapter is the pre-eminent role of heteroclinic cycles in providing deep and dynamic (generative) models of the sensorium; particularly the sensations that we generate ourselves through action. In what follows, we will briefly rehearse the basic theory and illustrate its implications using simulations of action (handwriting)—and its observation.

Appendix

This appendix provides a brief technical overview of how the free energy principle applies to neuronal dynamics. In this setting, the states of the brain (e.g., the activity of neurons and other systems that are crucial for its function, such as glial cells), are viewed as encoding the sufficient statistics of probability measures on hidden states of the external world. In this view, the main quantities are probabilities measures, denoted by $p(\tilde{s},\psi \,|\,m)$, on the product $S\times \Psi$ of possible values of (generalized) sensory states and hidden states, under a particular model m. Time plays a hidden but fundamental role in this formalism, in the sense that $(S,\Psi )$  are path spaces, and $(\tilde{s},\psi )$ are points in manifolds that depend on time. Particular attention is required by this point in Bayesian modelling [11, 48]).

The underlying premise is that the sufficient statistics $\tilde{\mu }$ and the induced probability $q(\psi \,|\,\tilde{\mu })$ evolve to maximize the marginal likelihood or model evidence:

$$\begin{aligned} p(\tilde{s}\,|\,m)=\int {d\psi p(\tilde{s},\psi \,|\,m)} \end{aligned}$$

(A.1)

However, this marginalization is generally intractable. The main simplification rests on replacing the difficult marginalization in A.1, by the practically easier problem of minimizing free energy:

$$\begin{aligned} {\mathcal {F}}=E_q [{\mathcal {G}}(\tilde{s},\psi )]-H[q(\psi \,|\,\tilde{\mu })] \end{aligned}$$

(A.2)

where $H[q(\psi \,|\,\tilde{\mu })]$ denotes the entropy of the probability law $q(\psi \,|\,\tilde{\mu })$ on $\Psi$, and the first term is the Gibbs internal energy ${\mathcal {G}}(\tilde{s},\psi )=-\ln p(\tilde{s},\psi \,|\,m)$ expected under $q(\psi \,|\,\tilde{\mu })$. It is easy to show that the concavity of the logarithmic function on $]0,\infty [$ implies:

$$\begin{aligned} {\mathcal {F}}(\tilde{s},\tilde{\mu })\ge -\ln p(\tilde{s}\,|\,m) \end{aligned}$$

(A.3)

Then, the minimization of free energy with respect to the sufficient statistics $\tilde{\mu }$ affords a constraint on the good direction for the maximization of the marginal likelihood or model evidence. In general, the problem is further simplified, for instance by an Ansatz of mean-field approximation, or by reducing to a belief propagation algorithm; c.f., [72].