Co-evolutionary search in asymmetric spaces

Abstract

Co-evolutionary models have received increasing attention in the research on evolutionary algorithms (EAs), since they seem to offer novel ideas for how to overcome some of the problems inherent in “classical” EAs. This article is concerned with one type of co-evolutionary model, based on the use of hosts and parasites, which allows co-evolution of a population of candidate problem solutions and a population of fitness cases. Although host-parasite algorithms have shown promising results in previous research, there is a lack of work aimed at formalizing this class of algorithms, and studying aspects of their behaviour in more detail. To help remedy this situation, this article explores the role of problem asymmetry in limiting the progress of host-parasite search, and shows how a suite of co-evolutionary function optimization problems can be used to study the behaviour of host-parasite algorithms under varying levels of asymmetry. The article also presents the asymmetry-handling host-parasite algorithm (AHPA) and a set of experiments aimed at identifying the conditions under which AHPA consistently gives improvements over simple host-parasite algorithms.

Introduction

Host-parasite algorithms exploit the idea of co-evolving solutions and test cases in two coupled populations, where fitness of a candidate solution is proportional to the number of test cases it solves, while fitness of a test case is proportional to the number of candidate solutions which fail to solve it. This framework was introduced by Hillis [2] along with promising results in evolving sorting networks. It has later been used for a number of tasks, including the traveling salesperson problem [24], classification neural networks [15], [17], constraint satisfaction [16], cellular automata [18], [10], path planning [19], tic-tac-toe endgame classification [10], game playing [22], and protein sequence classification [12], [14]. The motivation for using host-parasite algorithms for these problems rests on the idea of a coevolutionary “arms race”. When two opposing populations are coevolving, we expect that each will become increasingly efficient at exploiting the weaknesses of the other. In an ideal case, we would like such an arms race to find the most efficient “path” through the two search landscapes – that of candidate solutions and that of candidate fitness cases. The idea is to run two interacting optimisation processes simultaneously – one aimed at optimising the problem solutions, and one aimed at optimising the selection of fitness cases.

To clarify the difference between canonical evolutionary algorithms (EAs) and host-parasite EAs, it is useful to think of the parasite population as a means to sample the most difficult test cases at all stages of the search. Consider the pseudo-code in Fig. 1 which represents a canonical EA with an evolving population H of candidate solutions, which represents a small subset of the whole search space $\text{H}$. For every time-step a small random sample P is drawn from the complete set $\text{P}$ of test cases, and this random sample is used for fitness evaluation. Given this formalization, it is clear that the main difference between a canonical EA and a host-parasite EA is that we replace the random sampling from $\text{P}$ by evolving test cases. In the simple host-parasite algorithm (SHPA) shown in Fig. 2 we treat P as a population where each individual can be assigned a fitness by counting the number of members of H which fail to solve it.

One notable property of SHPA is that it treats the two populations symmetrically, i.e. H is evaluated using P as fitness cases and P is evaluated using H as fitness cases. This is not surprising, since it reflects the very definition of a host-parasite algorithm. However, SHPA also applies selection and reproduction to both H and P in every generation, which reflects a hidden assumption that equivalent numbers of generations are useful for both populations. This assumption could potentially be valid, but only if the problems to which we apply SHPA were also symmetric, i.e. if the difficulty of the tasks faced by H and P was the same. It is reasonable to believe, however, that this is often not the case, and the following chapter will demonstrate that the most well-known application for host-parasite algorithms is in fact heavily asymmetric.

Taking a closer look at SHPA we can see that, despite the restriction of requiring equivalent numbers of generations for hosts and parasites, it does allow other forms of asymmetry. The main way in which SHPA differs from CEA is that evaluation of the candidates in H involves testing them against individuals from P, and vice versa. This is the only form of interaction between the populations, and each is reproduced separately. The two populations therefore have some “autonomy”, meaning that we can use different strategies for the evolution of each. This autonomy includes the following points:

• Representations. It is possible to use different chromosome representations and genotype–phenotype mappings in each population. This form of autonomy makes a host-parasite algorithm applicable to a much wider range of problems since the two populations can represent different forms of phenotypes. Without it, we would restrict the applicability of the algorithm to problems where candidate solutions and fitness cases were identical structures. Examples of such cases include symmetrical games, where the fitness cases can be opponent strategies using the same representation scheme as candidate solution strategies. Allowing different representation schemes for hosts and parasites means that we increase the scope of applicability beyond such symmetrical problems.

• Design of genetic operators. Since different representation schemes can be used, this creates a need for allowing different genetic operators, since the design of a suitable genetic operator is generally dependent on the genotype representation. There is no restriction in SHPA against using different genetic operators in the two populations.

• Frequency of genetic operations. SHPA also allows the use of different mutation rates and crossover probabilities in the different populations.

• Population sizes. Populations sizes may be different in the two populations.

• Selection method. Calculation of fitness in the host population is done by counting the number of parasites which each host solves correctly, while each parasite's fitness is calculated by counting the number of hosts which fail to solve it correctly. These fitness functions are clearly complementary, and given the resulting fitness values, we can use any method for selection of parents. The selection method could be different in the two populations. Although we find no reason to do so in the experiments presented in this article, we could very well use, for example, tournament selection [9] in the host population and fitness-proportional roulette wheel selection [1] in the parasite population.

These forms of autonomy between the two populations give some opportunities for handling asymmetries between the tasks of evolving candidate solutions and fitness cases. Given observations of characteristics of the problem, it is possible to tune the representation schemes, genetic operators, population sizes, and selection schemes of the two populations to adapt SHPA to the problem.

Despite this flexibility, we are on the other hand forced in SHPA to use the same evolutionary time scales in both populations, i.e. one generation for H equals one generation for P. This restriction seems limiting, since we expect to see a coevolutionary arms race between the two populations. If they are evolving in search spaces of very different characteristics, such an arms race will only occur if both populations are given a fair chance to keep up with the other. When the two sub-problems are of very unequal levels of difficulty, this may require the possibility for the algorithm to stop evolution in one of the populations to let the opponent population “catch up”. However, in SHPA there is no such mechanism.