The Missing Case in Chomsky-Schützenberger Theorem

Abstract

The theorem by Chomsky and Schützenberger (CST) says that every context-free language L over alphabet $\varSigma$ is representable as $h(D_k \cap R)$ where $D_k$ is the Dyck language over k pairs of brackets, R is a local (i.e., 2-strictly-locally-testable language) regular language, and h is an alphabetic homomorphism that may erase symbols; the Dyck alphabet size depends on the size of the grammar generating L. In the Stanley variant, the Dyck alphabet size only depends on the size of $\varSigma$, but the homomorphism has to erase many more symbols than in the previous version. Berstel found that the number of erasures in CST can be linearly limited if the grammar is in Greibach normal form, and recently Okhotin proved a non-erasing variant of CST for grammars in Double Greibach normal form. In both statements the Dyck alphabet depends on the grammar size. We present a new non-erasing variant of CST that uses a Dyck alphabet independent from the grammar size and a regular language that is strictly-locally-testable, similarly to a recent generalization of Medvedev theorem for regular languages.

Work partially supported by MIUR project PRIN 2010LYA9RH and by CNR-IEIIT.

Appendix: An Example

The example illustrates the crucial part of our constructions, namely the homomorphism $\tau$ defined by formulas (4) and (5). Consider language $L= \{a^{2n+4} b^{6n} \mid n\ge 0\}$ (generated, e.g., by grammar $\{S\rightarrow aaSb^6 \mid a^4\}$), and choose the value $m=2$ for Definition 3, meaning that the substrings of length two occurring in the language are mapped on the 2-tuples $\langle a,a\rangle , \langle a,b\rangle , \langle b,b\rangle$, shortened as $\langle aa\rangle$, etc. The following grammar in Double Greibach normal form, though constructed by hand, takes the place of grammar $G'$ of Lemma 1:

$1: S \rightarrow \langle aa\rangle \, S \,B\, \langle bb\rangle , \; 2: S\rightarrow \langle aa\rangle \,\langle aa\rangle , \; 3:B \rightarrow \langle bb\rangle \, \langle bb\rangle .$

The sentence $a^8 b^{12} \in L$ becomes $\langle aa\rangle ^4 \langle bb\rangle ^6 \in L(G')$, with the syntax tree:

For Okhotin Theorem 1, this sentence is the image by homomorphism h of the following sequence $\gamma$ of labeled parentheses, where the numbers identify the rules and the dash marks the root:

$$ \gamma = (^{-}_{1} \quad (^{1}_{1} \quad (^{1}_{2}\quad )^{1}_{2} \quad (^{1}_{3}\quad )^{1}_{3} \quad )^{1}_{1} \quad (^{1}_{3} \quad )^{1}_{3}\quad )^-_{1} $$

We choose to represent the labeled parentheses with $m=2$ binary digits, defining $\tau$ as:

$$ \begin{array}{c|c | c|c} \omega &{} \omega ' &{} \tau (\omega ) &{} \tau (\omega ')\\ \hline (^{-}_{1} &{} )^-_{1} &{} [_{a,b,0} \; [_{a,b,0} &{} ]_{b,a,0} \; ]_{b,a,0} \\ (^{1}_{1} &{} )^1_{1} &{} [_{a,b,0} \; [_{a,b,1} &{} ]_{b,a,1} \; ]_{b,a,0} \\ (^{1}_{2} &{} )^1_{2} &{} [_{a,a,1} \; [_{a,a,0} &{} ]_{a,a,0} \; ]_{a,a,1} \\ (^{1}_{3} &{} )^1_{3} &{} [_{b,b,1} \; [_{b,b,1} &{} ]_{b,b,1} \; ]_{b,b,1} \end{array} $$

Hence $\tau \left( \pi (h(\gamma ))\right)$ is

$\overbrace{[_{a,b,0}\; [_{a,b,0}}^{(^{-}_{1}}\; \overbrace{[_{a,b,0} \; [_{a,b,1}}^{(^{1}_{1}} \overbrace{[_{a,a,1} \; [_{a,a,0}}^{(^{1}_{2}} \; \overbrace{]_{a,a,0} \; ]_{a,a,1}}^{)^{1}_{2}} \; \overbrace{[_{b,b,1} \; [_{b,b,1}}^{(^{1}_{3}} \overbrace{]_{b,b,1} \; ]_{b,b,1}}^{)^{1}_{3}}$

$\overbrace{]_{b,a,1} \; ]_{b,a,0}}^{)^{1}_{1}} \overbrace{[_{b,b,1} \; [_{b,b,1}}^{(^{1}_{3}} \overbrace{]_{b,b,1} \; ]_{b,b,1}}^{)^{1}_{3}} \overbrace{]_{b,a,0}\; ]_{b,a,0}}^{)^-_{1}}$

Notice that the 2-SLT language of the classical CST (applied to language L) is now replaced by an SLT language of higher width.