Average case analysis of pattern language learning algorithms

Abstract

The present paper deals with the comparison of two pattern language learning algorithms with respect to their average case behavior. Pattern languages have been introduced by Angluin (1980) and are defined as follows:

Let Σ={a,b,} be any non-empty finite alphabet containing at least two elements. Furthermore, let X={x _i } i ε In be an infinite set of variables such that Σ ∩ X=$\not 0$. Patterns are non-empty strings from Σ ∪ X, e.g., ab, ax₁ccc, bx₁x₁cx₂x₂ are patterns. L(p), the language generated by pattern p is the set of strings which can be obtained by substituting non-null strings from Σ ^* for the variables of the pattern p. Thus aabbb is generable from pattern ax₁x₂b, while aabba is not. Pat and PAT denote the set of all patterns and of all pattern languages over Σ, respectively. In order to deal with the learnability of pattern languages we have to specify from what information the inference algorithms are supposed to identify the target language. We consider both learning from text and informant. Intuitively, a text for L generates the language L without any information concerning the complement of L, whereas an informant of L decides L by informing the strategy whether or not any word from Σ ^* belongs to L. Note that we allow a text and an informant to be non-effective.

The learning algorithms we are going to analyze learn the class of all pattern languages in the limit. The first one has been established by Angluin (1980). It mainly uses a subprocedure that finds, for any given set of positive and negative examples, a pattern that is descriptive for the input sample. In particular, this learning algorithm exclusively outputs consistent hypotheses. However, its update time is not polynomially bounded in the length of the actual input unless P=NP.

Recently, Lange and Wiehagen (1991) described a learning algorithm that might behave inconsistently. Nevertheless, its update time is polynomially bounded in the length of the actual input. On the other hand, the latter algorithm mainly exploits the fact that every pattern language is uniquely characterized by its corresponding set of all strings that have minimal length.

Hence, it is only natural to compare these algorithms with respect to their average case behavior. This might be also interesting with respect to potential applications pattern language learning algorithms have (cf. Nix (1983)). Finally, we compare the results obtained with the fact that PAT is not PAC-learnable unless P=NP (cf. Schapire (1990)).