Cartpole Problem with PDL and GP Using Multi-objective Fitness Functions Differing in a Priori Knowledge

Abstract

We present a study looking at the effect of a priori domain knowledge on an EA fitness function. Our experiment has two aims: (1) applying an existing NSGA-II framework for GP with PDL to the cartpole problem—applying GP & PDL to cartpole and a purely behavioral problem for the first time—and (2) contrasting two multi-objective fitness functions: one with high and the other with low a priori domain knowledge. In our experiment we created two populations with an EA, varying in the number of objectives use for the fitness function, 2 objective criteria to represent low a priori knowledge and 3 to represent high. With fitness functions tailored to find specifically prescribed solutions we expect greater discriminating power and more feedback to an evolutionary process. This comes at the cost of excluding some unexpected solutions from the evolutionary process and placing a greater burden on the designer. We address the question: how large is the disadvantage for the low a priori fitness function in a worst-case scenario, where innovative solutions will not enhance performance. This question is interesting because we would prefer to guide EA with simple, easy to create and understand, objective criteria rather than complex and highly specific criteria. Understanding any associated penalty for using simple, easy to create fitness functions, is crucial in assessing how much effort and should be put into designing objective criteria.