Stochastic Finite Learning

Motivation and Background

Assume that we are given a concept class $\mathcal{C}$ and should design a learner for it. Next, suppose we already know or could prove $\mathcal{C}$ not to be learnable in the model of PAC Learning. But it can be shown that $\mathcal{C}$ is learnable within Gold’s (1967) model of Inductive Inference or learning in the limit. Thus, we can design a learner behaving as follows. When fed any of the data sequences allowed in this model, it converges in the limit to a hypothesis correctly describing the target concept. Nothing more is known. Let M be any fixed learner. If (d _n ) _{n ≥ 0} is any data sequence, then the stage of convergence is the least integer m such that M(d _m ) = M(d _n ) for all n ≥ m, provided such an nexists (and infinite, otherwise). In general, it is undecidable whether or not the learner has already reached the stage of convergence, but even if it is decidable for a particular concept class, it may be practically infeasible to do so....