Information Geometry and Its Applications: Survey

Abstract

Information geometry emerged from the study of the geometrical structure of a manifold of probability distributions under the criterion of invariance. It defines a Riemannian metric uniquely, which is the Fisher information metric. Moreover, a family of dually coupled affine connections are introduced. Mathematically, this is a study of a triple (M, g, T), where M is a manifold, g is a Riemannian metric, and T is a third-order symmetric tensor. Information geometry has been applied not only to statistical inferences but also to various fields of information sciences where probability plays an important role. Many important families of probability distributions are dually flat Riemannian manifolds. A dually flat manifold possesses a beautiful structure: It has two mutually coupled flat affine connections and two convex functions connected by the Legendre transformation. It has a canonical divergence, from which all the geometrical structure is derived. The KL-divergence in probability distributions is automatically derived from the invariant flat nature. Moreover, the generalized Pythagorean and geodesic projection theorems hold. Conversely, we can define a dually flat Riemannian structure from a convex function. This is derived through the Legendre transformation and Bregman divergence connected with a convex function. Therefore, information geometry is applicable to convex analysis, even when it is not connected with probability distributions. This widens the applicability of information geometry to convex analysis, machine learning, computer vision, Tsallis entropy, economics, and game theory. The present talk summarizes theoretical constituents of information geometry and surveys a wide range of its applications.