Enumerative Induction and Semi-uniform Convergence to the Truth

Abstract

I propose a new definition of identification in the limit, also called convergence to the truth, as a new success criterion that is meant to complement, but not replace, the classic definition due to Putnam (1963) and Gold (1967). The new definition is designed to explain how it is possible to have successful learning in a kind of scenario that the classic account ignores—the kind of scenario in which the entire infinite data stream to be presented incrementally to the learner is not presupposed to completely determine the correct learning target. For example, suppose that a scientists is interested in whether all ravens are black, and that she will never observe a counterexample in her entire life. This still leaves open whether all ravens (in the universe) are black. From a purely mathematical point of view, the proposed definition of convergence to the truth employs a convergence concept that generalizes net convergence and sits in between pointwise convergence and uniform convergence. Two results are proved to suggest that the proposed definition provides a success criterion that is by no means weak: (i) Between the proposed identification in the limit and the classic one, neither implies the other. (ii) If a learning method identifies the correct target in the limit in the proposed sense, any U-shaped learning involved therein has to be essentially redundant. I conclude that we should have (at least) two success criteria that correspond to two senses of identification in the limit: the classic one and the one proposed here. They are complementary: meeting any one of the two is good; meeting both at the same time, if possible, is even better.

A Convergence Generalized

As mentioned in the introduction, the classic definition of identification in the limit only employs the most familiar concept of convergence—pointwise convergence of sequences—while in analysis and topology mathematicians have already worked out more elaborated concepts of convergence. This section provides a quick review of those concepts.

Definition 9

(Sequence Convergence). Let $f: \omega \rightarrow Y$ be a sequence of points in a topological space Y. Say that f(n) converges to y as n travels upward in $(\omega , \le )$ just in case:

• for any open neighborhood U of y in Y,

• there exists $n \in \omega$ such that

• $f(m) \in U$ for every $m \ge n$ in $\omega$.

The next step is to go from convergence of a sequence to convergence of a “generalized” sequence, a.k.a. “net”. A sequence is defined on a very special set of indices, namely $\omega$, linearly ordered by $\le$. Let us have a more general set I of indices with a weaker ordering structure:

Definition 10

(Directed Poset). A poset $(I, \le )$ consists of a set I partially ordered by $\le$. It is directed just in case $i, j \in I$ implies $i, j \le k$ for some $k \in I$.

Definition 11

(Generalized Sequence, or Net). A generalized sequence or net $f: (I, \le ) \rightarrow Y$, is a function from a nonempty directed poset $(I, \le )$ to a topological space Y.

Definition 12

(Net Convergence). Let $f: (I, \le ) \rightarrow Y$ be a net. Say that f(i) converges to y as i travels in $(I, \le )$ just in case:

• for any open neighborhood U of y in Y,

• there exists i in I such that

• $f(i') \in U$ for every $i' \ge i$ in I.

Note that the definition of net convergence is formally identical to that of sequence convergence, except that the underlying spaces of indices are generalized. Net convergence has a number of equivalent formulations, some of which are very sophisticated (and stated in terms of, say, filters).⁶ Let me introduce the following one:

Proposition 3

(Net Convergence Redefined by “Cofinal Upper”). Let $f: (I, \le ) \rightarrow Y$ be a net. Then net convergence of f can be equivalently redefined in terms of cofinal upper subsets as follows. f(i) converges to y as i travels in $(I, \le )$ if and only if:

• for any open neighborhood U of y,

  there exists a cofinal upper subset S of $(I, \le )$ such that

  $f(i) \in U$ for all $i \in S$.

Proof

The ($\Rightarrow$) side follows immediately from the order-theoretic result that every principal upper subset of a nonempty directed poset is cofinal. The ($\Leftarrow$) side follows immediately from the order-theoretic result that every cofinal upper subset of a nonempty directed poset is nonempty (because of cofinality) and thus (by upward closure) includes a principal upper subset.    $\square$

With the above formulation of net convergence, we can generalize further by relaxing the restriction to directed posets and allowing any nonempty posets:

Definition 13

(Generalized Convergence and Limit). Let f be a function from a nonempty poset $(I, \le )$ to an arbitrary topological space Y. Say that f(i) converges to y as i travels in $(I, \le )$ (in the sense of generalized convergence) just in case:

• for any open neighborhood U of y,

• there exists a cofinal upper subset S of $(I, \le )$ such that

• $f(i) \in U$ for all $i \in S$.

If such a y exists uniquely, also say that y is the generalized limit of f(i) as i travels in $(I, \le )$.

The above is the concept of convergence that underlines the definition of semi-uniform identification in the limit: the index i travels in a partially ordered set I in general, a convergence zone is required to be cofinal and upper, and the underlying topology of the codomain is discrete.