Computing, Artificial Intelligence and Information Management
Learning genetic algorithm parameters using hidden Markov models

https://doi.org/10.1016/j.ejor.2005.04.045Get rights and content

Abstract

Genetic algorithms (GAs) are routinely used to search problem spaces of interest. A lesser known but growing group of applications of GAs is the modeling of so-called “evolutionary processes”, for example, organizational learning and group decision-making. Given such an application, we show it is possible to compute the likely GA parameter settings given observed populations of such an evolutionary process. We examine the parameter estimation process using estimation procedures for learning hidden Markov models, with mathematical models that exactly capture expected GA behavior. We then explore the sampling distributions relevant to this estimation problem using an experimental approach.

Introduction

The area of experimental economics (e.g., Kagel and Roth, 1995) studies complex, multi-agent systems within a computer-simulated environment. Often it is desirable to have the artificial agents adapt to various events and pressures within their environment. One popular type of adaptive behavior is modeled after natural evolutionary processes. A simple, yet powerful, form of this behavior is captured in genetic algorithms (GAs) (see Goldberg, 1989).

Several authors have studied the evolutionary characteristics of systems by creating a simulation environment where a GA is used to mimic the adaptive behavior of some agent or group of agents. For example, Marks et al. (1995) examined oligopoly behavior in an adaptive framework and used a GA to simulate market-pricing movements in the coffee industry. An evolutionary model for electronic commerce was presented in Oliver (1996). Using a GA for learning, automated agents learned strategies for business negotiations within an electronic commerce framework. This investigation focused on a series of simulations of automated negotiation tasks using a genetic algorithm on automated agents. In Sikora and Shaw (1996), a GA was used to simulate group interaction leading to a solution among groups of agents. Genetic programming, (see Koza, 1992), another evolutionary computing technique, was used by Dworman et al. (1996) to automate the discovery of game theory models. A computational model of the organization based on the simple genetic algorithm was created by Bruderer and Singh (1996). Their research is unique in that it specifically uses a genetic algorithm as the model for organizational evolution, as opposed to simply simulating behavior using a genetic algorithm. In the above examples using evolutionary techniques to simulate or model particular processes, the authors have had to provide very rough estimates of parameters for the evolutionary technique used, for example, crossover or mutation rates, typically without much guidance of which values might be appropriate. Clearly, these simulations and models could progress further if more were known about the appropriate settings for the evolutionary parameters.

In Rees and Koehler, 2000, Rees and Koehler, 2002 the process is reversed. Rather than simulate behavior, the authors started with experimental data from an actual adaptive process (in this case a group decision-making case) and used the data to find a best-fit GA to mimic the evolutionary path. This fitting process was used to determine parameters required by the GA. They formulated and tested various hypotheses for these parameters (Rees and Koehler, 2000).

In traditional GA simulation, the GA parameters, χ, the crossover rate, and μ, the mutation rate, are either determined experimentally by running a series of preliminary simulations or are chosen based on previous results or standard values used in the GA community. As discussed above, there are many real-world phenomena that appear to possess evolutionary characteristics, not unlike a GA. Following the lead given in Rees and Koehler, 2000, Rees and Koehler, 2002, we seek to investigate further whether there is a way to characterize these processes in terms of GA instances? In other words, can we reliably learn GA parameters, such as crossover and mutation rates, from real-world processes?

The expected behavior of a GA process can be modeled exactly using a Markov chain (Nix and Vose, 1992). If we know or assume a real-world process is a GA process, then we have a hidden Markov model (HMM)—we know it can be modeled by a Markov chain, we just do not know the specific mutation and crossover rates that generate the transition probabilities. The objective of this research is to use HMM methods to compute the likely GA parameter settings given observed populations of such an evolutionary process. In this paper we study the process of learning or “fitting” GA parameters to such evolutionary processes in more detail than done in Rees and Koehler, 2000, Rees and Koehler, 2002. We examine this issue by using mathematical models that exactly capture expected GA behavior and explore the sampling distributions relevant to this estimation problem using an experimental approach.

The value of this research is to provide researchers with a tool to more accurately simulate real-world evolutionary behavior. Such simulations are becoming commonplace, especially in applications such as information security, artificial markets and retail management. In such simulations, there is no theoretical guidance on how to set GA parameters such as crossover and mutation rates. The technique described in this paper allows researchers to determine these parameter values from existing real-world data. This ability should lead to more accurate and useful simulation studies.

The remainder of this paper is organized as follows: Section 2 presents relevant background on genetic algorithms, on the applications of hidden Markov models to this problem, and specifically on the use of maximum likelihood estimates for computing best-fit genetic algorithm parameters, namely crossover and mutation rates, from experimental data. Section 3 describes the experimental study comparing known GA parameters to estimated parameters using the techniques described in Section 2. Section 4 presents the results of this study and Section 5 provides a discussion of those results. Finally, Section 6 provides conclusions and future research directions.

Section snippets

Background

In the following sections we present the relevant background information on genetic algorithms for this study. The Markov model of GAs is then provided in Section 2.2. We then discuss the hidden Markov model approach used in the study for learning specific parameter settings in Section 2.3.

Experimental study

We wish to study the results of learning or “fitting” GA parameters using the above-described MLE procedures. We do this by generating a large number of controlled cases where we fix the fitness function and GA parameters. In other words, we run a GA with known parameter values for several scenarios. We then use MLE on these generated cases to estimate the GA parameters and study the fit of these versus the known, true values.

We study a collection of problems having string lengths of 10 bits

Results

Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6 show the frequencies of the mutation and crossover rates determined by MLE for the three functions. Each figure provides the distribution of the learned parameters for each given crossover or mutation rate. For example, Fig. 1 shows the distributions of the mutation estimates for each of the three functions tested. The given or fixed value of μ was set at 0.001.

We examined the distribution of the estimates around their true values. For this we use

Discussion

For a majority of the datasets tested, evidence supports the hypothesis that the MLEs distributions are lognormal. This indicates that this HMM technique performs as anticipated and gives results that reasonably approximate the true values of the GA parameters examined.

Overall, the MLEs of both crossover and mutation were good approximations of their fixed values. However, there appears to be an underestimation in both the crossover and mutation rate estimates. This is more pronounced in the

Conclusions and future research

This paper presents a study of a maximum likelihood estimate technique for estimating the crossover and mutation parameters from a hidden Markov model in the form of data generated by a GA. The approach shows reasonable promise for use in many simulation settings where it is desirable to accurately depict evolutionary behaviors of the underlying system. The technique was shown to be relatively accurate over three test functions for various combinations of parameters (mutation rate, crossover

References (27)

  • S.-B. Cho et al.

    Efficient anomaly detection by modeling privilege flows using hidden Markov model

    Computers & Security

    (2003)
  • J. Rees et al.

    Leadership and group search in group decision support systems

    Decision Support Systems

    (2000)
  • D.H. Ackley

    A Connectionist Machine for Genetic Hill Climbing

    (1987)
  • S. Bhattacharyya et al.

    An analysis of genetic algorithms of cardinality 2V

    Complex Systems

    (1994)
  • S. Boykin et al.

    Machine learning of event segmentation for news on demand

    Communications of the ACM

    (2000)
  • E. Bruderer et al.

    Organizational evolution, learning, and selection: A genetic-algorithm-based model

    Academy of Management Journal

    (1996)
  • W.J. Conover

    Practical Nonparametric Statistics

    (1999)
  • De Jong, K., 1975. An analysis of the behavior of a class of genetic adaptive systems, Ph.D. Thesis, University of...
  • Dugad, R., Desai, U.B., 1996. A tutorial on hidden Markov models, Signal Processing and Artificial Neural Networks...
  • G. Dworman et al.

    On automated discovery of models using genetic programming: Bargaining in a three-agent coalitions game

    Journal of Management Information Systems

    (1996)
  • D.E. Goldberg

    Genetic Algorithms in Search, Optimization, and Machine Learning

    (1989)
  • J.H. Kagel et al.

    The Handbook of Experimental Economics

    (1995)
  • G.J. Koehler et al.

    General cardinality genetic algorithms

    Evolutionary Computing

    (1997)
  • Cited by (0)

    1

    Tel.: +1 352 846 2090; fax: +1 352 392 5438.

    View full text