Information-Theoretic Dissimilarities for Graphs

Abstract

This is a survey paper in which we explore the connection between graph representations and dissimilarity measures from an information-theoretic perspective. Firstly, we pose graph comparison (or indexing) in terms of entropic manifold alignment. In this regard, graphs are encoded by multi-dimensional point clouds resulting from their embedding. Once these point clouds are aligned, we explore several dissimilarity measures: multi-dimensional statistical tests (such as the Henze-Penrose Divergence and the Total Variation k-dP Divergence), the Symmetrized Normalized Entropy Square variation (SNESV) and Mutual Information. Most of the latter divergences rely on multi-dimensional entropy estimators. Secondly, we address the representation of graphs in terms of populations of tensors resulting from characterizing topological multi-scale subgraphs in terms of covariances of informative spectral features. Such covariances are mapped to a proper tangent space and then considered zero-mean Gaussian distributions. Therefore each graph can be encoded by a linear combination of Gaussians where the coefficients of the combination rely on unbiased geodesics. Distributional graph representations allows us to exploit a large family of dissimilarities used in information theory. We will focus on Bregman divergences (particularly Total Bregman Divergences) based on the Jensen-Shannon and Jensen-Rényi divergences. This latter approach is referred to as tensor-based distributional comparison for distributions can be also estimated from embeddings through Gaussian mixtures.