On Ordinal VC-Dimension and Some Notions of Complexity

Abstract

We generalize the classical notion of VC-dimension to ordinal VC-dimension, in the context of logical learning paradigms. Logical learning paradigms encompass the numerical learning paradigms commonly studied in Inductive inference. A logical learning paradigm is defined as a set $\mathcal{W}$ of structures over some vocabulary, and a set $\mathcal{D}$ of first-order formulas that represent data. The sets of models of ϕ in $\mathcal{W}$, where ϕ varies over $\mathcal{D}$, generate a natural topology $\mathbb{W}$ over $\mathcal{W}$.

We show that if $\mathcal{D}$ is closed under boolean operators, then the notion of ordinal VC-dimension offers a perfect characterization for the problem of predicting the truth of the members of $\mathcal{D}$ in a member of $\mathcal{W}$, with an ordinal bound on the number of mistakes. This shows that the notion of VC-dimension has a natural interpretation in Inductive Inference, when cast into a logical setting. We also study the relationships between predictive complexity, selective complexity—a variation on predictive complexity—and mind change complexity. The assumptions that $\mathcal{D}$ is closed under boolean operators and that $\mathbb{W}$ is compact often play a crucial role to establish connections between these concepts.