Problem solution sustenance in XCS: Markov chain analysis of niche support distributions and the impact on computational complexity

Abstract

Michigan-style learning classifier systems iteratively evolve a distributed solution to a problem in the form of potentially overlapping subsolutions. Each problem niche is covered by subsolutions that are represented by a set of predictive rules, termed classifiers. The genetic algorithm is designed to evolve classifier structures that together cover the whole problem space and represent a complete problem solution. An obvious challenge for such an online evolving, distributed knowledge representation is to continuously sustain all problem subsolutions covering all problem niches, that is, to ensure niche support. Effective niche support depends both on the probability of reproduction and on the probability of deletion of classifiers in a niche. In XCS, reproduction is occurrence-based whereas deletion is support-based. In combination, niche support is assured effectively. In this paper we present a Markov chain analysis of the niche support in XCS, which we validate experimentally. Evaluations in diverse Boolean function settings, which require non-overlapping and overlapping solution structures, support the theoretical derivations. We also consider the effects of mutation and crossover on niche support. With respect to computational complexity, the paper shows that XCS is able to maintain (partially overlapping) niches with a computational effort that is linear in the inverse of the niche occurrence frequency.

Appendices

Appendix A. List of symbols

Table 8

Table 9

Appendix B. Closed-form equation

In this section, we develop the closed form equation for the Markov chain in Fig. 1. We start from the following equation, introduced in Section 4:

$$u_k = r_{k-1}u_{k-1} + s_k u_k + d_{k+1}u_{k+1}\label{eq:fp}$$

(13)

by replacing the probabilities $r_k, d_k$, and $s_k$, with their actual values (see Section 4 for details):

$$ \begin{array}{l}r_k = p\left(1-\displaystyle\frac{k}{N}\right)\\ s_k = \left(1-p\right)\left(1-\displaystyle\frac{k}{N}\right) + p\displaystyle\frac{k}{N}\\ d_k = \left(1-p\right)\displaystyle\frac{k}{N}\end{array} $$

we obtain,

$$\left[ p\left(1-\frac{k}{N}\right) + \frac{k}{N}\left(1-p \right) \right] u_k =\left(1-p \right)\left( \frac{k+1}{N} \right)u_{k+1} +p\left( 1- \frac{k-1}{N}\right)u_{k-1}$$

finally, by dividing by $(1-p)u_{k-1}$, we derive the following equation:

$$\left[ \frac{p}{1-p}(N-k) + k\right]\frac{u_k}{u_{k-1}} = \left(k+1\right)\frac{u_{k+1}}{u_{k-1}} + \frac{p}{1-p}(N-k+1)\label{eq:fixedpoint2}$$

(14)

To derive a closed-form solution for probability $u_k$ we first use Eq. (14) to derive a closed-form equation for the ratio $\frac{u_{k}}{u_{0}}$. Next, we use the equation for $\frac{u_k}{u_0}$ and the condition $\sum_{k=0}^{N}u_k = 1$, to derive the closed-form solution for $u_k$.

B.1. Closed-form equation for $u_k/u_0$

As the very first step, we write the following fixed point equation for the transitions between state 0 and state 1:

$$u_0 = s_0 u_0+d_1u_1.$$

(15)

By substituting the values of $s_0$ and $d_1$ we obtain:

$$ \begin{array}{l}u_0 = (1-p)u_0+(1-p)\displaystyle\frac{1}{N}u_1,\\ pu_0 = (1-p)\displaystyle\frac{1}{N}u_1,\end{array} $$

from which we derive equation:

$$\frac{u_1}{u_0} = \frac{p}{1-p}N.\label{eq:caseu1}$$

(16)

To derive the equation for $u_2/u_0$ we start from Eq. (14) and set $k=1$:

$$\left[ \frac{p}{1-p}(N-1) + 1\right]\frac{u_1}{u_0} = 2\frac{u_2}{u_0} + \frac{p}{1-p}N,$$

so that

$$\frac{u_2}{u_0} = \frac{1}{2}\left[\left(\frac{p}{1-p}(N-1) + 1\right)\frac{u_1}{u_0} - \frac{p}{1-p}N\right].$$

We now replace $u_1/u_0$ with Eq. (16) :

$$\displaylines{ \displaystyle\frac{u_2}{u_0} = \displaystyle\frac{1}{2}\left[\left(\frac{p}{1-p}(N-1) + 1\right)\frac{u_1}{u_0} - \frac{p}{1-p}N\right] \cr = \displaystyle\frac{1}{2}\left[\left(\frac{p}{1-p}(N-1) + 1\right)\frac{p}{1-p}N - \frac{p}{1-p}N\right]\cr = \displaystyle\frac{1}{2}\left[\frac{p}{1-p}(N-1)\frac{p}{1-p}N\right]\cr = \displaystyle\frac{N(N-1)}{2}\left(\frac{p}{1-p}\right)^2\cr = {N\choose 2}\left(\displaystyle\frac{p}{1-p}\right)^2 \label{eq:caseu2} }$$

(17)

This leads us to the hypothesis that

$$ \frac{u_k}{u_0}=\left(\begin{array}{c} N\\ k\\ \end{array}\right) \left(\frac{p}{1-p}\right)^k, $$

(18)

which we prove by induction. Using Eq. (18), we can first derive that:

$$\displaylines{ u_{k+1} = \frac{N-k}{k+1}\frac{p}{1-p}u_k\label{eq:ukp1}\cr u_k = \frac{N-k+1}{k}\frac{p}{1-p}u_{k-1}\label{eq:uk}\cr u_{k-1} = \frac{1-p}{p}\frac{k}{N-k+1}u_k\label{eq:ukm1} }$$

(19)

With Eq. (14) substituting Eq. (21) as the inductive step, we now derive

$$\displaylines{ u_{k+1} = \frac{\big(\big(\frac{p}{1-p}(N-k) + k\big)\frac{u_k}{u_{k-1}} - \frac{p}{1-p} (N-k+1)\big)u_{k-1}}{k+1}\cr = \frac{\big(\frac{p}{1-p}\big)^2\frac{(N-k)(N-k+1)}{k}u_{k-1}}{k+1}\cr = \frac{N-k}{k+1}\frac{p}{1-p}u_k, }$$

(22)

which proves the hypothesis.

B.2. Derivation of $u_0$

To derive a closed-form for $u_k$ from the equation for $u_k/u_0$, we use the subsidiary condition:

$$\sum_{k=0}^{N}u_k=1\label{eq:sub}$$

(25)

We divide both terms by $u_0$

$$\sum_{k=0}^{N}\frac{u_k}{u_0} = \frac{1}{u_0},$$

(26)

where

$$\displaylines{ \sum_{k=0}^{N}\frac{u_k}{u_0} = \sum_{k=0}^{N}{N \choose k}\left(\frac{p}{1-p}\right)^k \cr = \left[\sum_{k=0}^{N}{N \choose k}p^k(1-p)^{N-k}\right]\frac{1}{(1-p)^N}, }$$

where the term “ $\sum_{k=0}^{N}{N\choose k}p^k(1-p)^{N-k}$” is equal to “ $\left[p+(1-p)\right]^N$”, that is 1, so that:

$$\sum_{k=0}^{N}\frac{u_k}{u_0}= \frac{1}{(1-p)^N}$$

(27)

and accordingly,

$$u_0 = (1-p)^N.\label{eq:u0}$$

(28)

B.3. Closed-form equation for $u_k$

By combining Eqs. (18) and (28), we derive the closed-form equation for $u_k$ as follows:

$$ \begin{array}{lll}u_k & = & \displaystyle{N\choose k}\left(\displaystyle\frac{p}{1-p}\right)^ku_0\\ & = & \displaystyle{N\choose k}\left(\displaystyle\frac{p}{1-p}\right)^k(1-p)^N\\ & = & \displaystyle{N\choose k}p^k(1-p)^{N-k}\end{array} $$

Note that the same derivation is possible noting that the proposed Markov chain results in an Engset distribution [35].