Development of an adaptive response surface method for optimization of computation-intensive models

Abstract

In general, optimization techniques involve numerous repeated objective function evaluations. As a consequence, optimization times can become very large depending on the complexity of the model to be optimized. This manuscript describes the development of an adaptive response surface method for optimization of computation-intensive models, capable of reducing optimization times. The response model to be optimized is not built from a pre-defined number of design experiments but is adapted and refined during the optimization routine. Different approximation models are applicable in combination with the developed optimization technique. The proposed optimization technique is evaluated on a standard test problem as well as a finite element model design optimization with multiple parameters.

Introduction

Much of today’s engineering analysis work consists of running complex computer analysis codes with a vector of design variables (x) as inputs and resulting in a vector of responses (y) as outputs. Despite continuous advances in computing power and speed, the expense of running many analysis codes remains non-trivial. Single evaluations of aerodynamic or finite element (FE) codes can take from minutes to hours, if not longer. Statistical techniques are widely used in engineering design to address these concerns. The basic approach is to construct approximations of the analysis codes that are much more efficient to run and that yield insight into the functional relationship between x and y. If the true nature of the relationship between input and output is represented as

$$y=f(x)$$

then the metamodel g(x) for f(x) can be written as

$$\hat{y}=g(x)\text{with}y=\hat{y}+\epsilon$$

where ε represents the approximation error. The most common metamodelling approach is to apply a design of experiments to identify an efficient set of computer runs and then employ regression analysis to create a polynomial approximation of the computer model. These approximations can then replace the existing model while providing a fast analysis tool for the optimization and exploration of the design space. There will be a decrease in optimization time as the optimization takes place on the metamodels or response surface models rather than on the computationally expensive models themselves.

The response surface method (RSM) consists of the following steps (Simpson, Peplinski, Koch, & Allen, 1997):

• Choosing a function that is an acceptable approximation that describes the relationship between input parameters (x₁, … , x_n) and the output y(x₁, … , x_n).

• Performing a design of experiments in order to make an optimal estimation of the regression coefficients for the chosen regression function. Often a central composite design will be applied.

• Estimation of the regression coefficients.

• Validation of the calculated response surface by statistical tests.

Response surface methodology usually consists of a central composite design, second-order polynomials and least-squares regression analysis (Myers & Montgomery, 1995). Usually, a low-order polynomial in some relatively small region of the independent variable space is appropriate. Often the curvature in the true response surface is strong enough that a second-order model will likely be required in these situations.

The remaining part of the introduction will give a summary of the related work concerning optimization in combination with response surface models or regressive techniques. Groenwold, Etman, Snyman, and Rooda (2007) present an incomplete series expansion as a basis for function approximation. The series expansion is expressed in terms of an approximate diagonal Hessian matrix. The approximate diagonal Hessian information is computed from first-order gradient information in points previously visited during the optimization. The main reason for exclusion of the interaction terms is of course the prohibitively high computational (and storage) requirements associated with these terms when many constraints are present. A comparison with popular approximating functions illustrates the high accuracy of the new family of approximation functions.

Toropov, Filatov, and Polynkin (1993) present a new optimization strategy, which uses in each iteration the information gained at several previous design points (multipoint approximations) in order to more accurately fit the constraints and objective functions and to reduce the total number of optimization runs needed to solve the optimization problem. In each iteration, the subregion of the initial region in the space of design variables, defined by move limits, is chosen. In this subregion, several design points are selected using the weighted least-squares method, for which response analysis and design sensitivity analysis are carried out. Three test examples are used to compare the technique with other approximation techniques. The proposed technique required the smallest number of optimization iterations to reach convergence.

Breitkopf, Naceur, Rassineux, and Villon (2005) also focus on a successive response surface method for optimization problems. The use of Moving Least-Squares (MLS) approximations for the regression model is investigated along with strategies for progressive selection of points in the design space using pan-and-zoom search patterns to maximize the accuracy while minimizing the number of function evaluations. When the design of experiments and regression models are selected, any usual descent method can be used to find an optimum design. According to the authors, all suggested MLS patterns perform comparably to gradient-based Gauss–Newton and Levenberg–Marquardt methods. More information on MLS techniques can be found in the work of Levin, 1998, Kiasat et al., 2007.

In Mukherjee and Ray (2006), the existing and frequently used input–output and in-process parameter relationship modelling and optimization techniques, specific to metal cutting processes are reviewed together with the presentation of a generic framework for carrying out process optimization studies. In Shin and Cho, 2005, Steenackers and Guillaume, 2008 different response surface modelling techniques for robust design optimization are suggested and performed for approximating the mean bias and variance in the objective function considered. Second-order polynomial response surfaces are applied to perform a robust design optimization. The different optimization approaches are illustrated with a numerical test case.

This paper addresses the development of an adaptive response surface method (ARSM) for solving design optimization problems. The proposed ARSM is developed to search for the global design optimum for computation-intensive design problems and is capable of reducing optimization times significantly. The developed algorithm only requires a limited number of design evaluations, even for high-dimensional design problems as the optimization takes places on a response surface model instead of the actual design model. The response model to be optimized is not built from a pre-defined number of design experiments but is adapted and refined during the optimization routine itself by adding and removing design points, depending on their objective function value. The proposed optimization technique is tested on a standard test problem as well as a finite element design optimization with multiple parameters. As a result, the optimization approach manages to reduce computation times significantly and demonstrates its usefulness as an optimization technique for computation-intensive design problems such as finite element optimization.