Implementation of Cartesian grids to accelerate Delaunay-based derivative-free optimization

Abstract

This paper introduces a modification of our original Delaunay-based optimization algorithm (developed in JOGO DOI:10.1007/s10898-015-0384-2) that reduces the number of function evaluations on the boundary of feasibility as compared with the original algorithm. A weaknesses we have identified with the original algorithm is the sometimes faulty behavior of the generated uncertainty function near the boundary of feasibility, which leads to more function evaluations along the boundary of feasibility than might otherwise be necessary. To address this issue, a second search function is introduced which has improved behavior near the boundary of the search domain. Additionally, the datapoints are quantized onto a Cartesian grid, which is successively refined, over the search domain. These two modifications lead to a significant reduction of datapoints accumulating on the boundary of feasibility, and faster overall convergence.

Appendix: Modified algorithm for problems without target value

In this appendix, we present a modified algorithm that does not required a target value for the objective function. The algorithm developed is quite similar to Algorithm 2, with the continuous and discrete search functions modified as follows:

$$\begin{aligned} s_c^k(x)= & {} p^k(x)-K^k \, e^k(x), \end{aligned}$$

(30)

$$\begin{aligned} s_d^k(x)= & {} p^k(x)-L^k\, \text {Dis}\{x,S^k_E\}. \end{aligned}$$

(31)

The parameters $L^k$ and $K^k$ are two positive series which are defined as follows:

$$\begin{aligned} L^k=L_0 \,\ell ^k, K^k=K_0 \,\ell ^k. \end{aligned}$$

(32)

Fig. 13

Application of the modified algorithm discussed in the “Appendix”, without leveraging knowledge of the target value, for minimizing the Styblinski Tang test problem (24). The position of the evaluated points (squares) and support points (stars) in the $n=2$ case are depicted in (a). Convergence histories are shown in b for $n=3$ and c for $n=4$. a Location of evaluated and support points, $n=2$. b Convergence history, $n=3$. c Convergence history, $n=4$

where $\ell ^k$ is the level of the grid at step k. The convergence analysis of this modified algorithm is similar to the analysis presented in Sect. 5, with the main differences as follows:

• 1. Equation (14) is modified to:

  $$\begin{aligned} \min \left\{ s_c^k(x^*), \min _{z \in S_U^k} \left\{ s_d^k(z) \right\} \right\} \le f(x^*). \end{aligned}$$

  (33)

  Note that above equation is not true for all iterations k, but it is true once

  $$\begin{aligned} K^k \ge \hat{K} \quad \text {and} \quad L^k \ge \hat{L}; \end{aligned}$$

  note that the series $K^k$ and $L^k$ increase without bound, and thus (33) is satisfied for sufficiently large k.

• 2. Equation (19) is modified to

  $$\begin{aligned} \min _{z \in S^k_E} f(z)-f(x^*) \le \max \left\{ \, (L^k+ \hat{L}) \, \delta _{k}, \, (K^k+\hat{K})\, \delta _{k}^2 \right\} . \end{aligned}$$

  (34)

  Moreover, we have:

  $$\begin{aligned} \lim _{k \rightarrow \infty }L^k \delta _k= & {} \lim _{k \rightarrow \infty } L_0 \delta _0 \frac{\ell ^k}{2^{\ell ^k}}=0,\\ \lim _{k \rightarrow \infty }K^k \delta _k^2= & {} \lim _{k \rightarrow \infty } K_0 \delta _0^2 \frac{2^{\ell }}{4^{\ell }}=0. \end{aligned}$$

  As a result, the right hand size of (34) converges to zero as $k \rightarrow \infty$.

We have implemented this modified algorithm on the problem of minimizing the Styblinski Tang test problem (24), for $n=\{2,3,4\}$, inside the domain $-5 \le x_i \le 5\ \forall i$, with the initial point given by $x^0_i=0.5\ \forall i$. In these computations, the value of $K_0=50$ and $L_0=5$ were used; note that these parameter values happen to be good for this test problem. In general, selecting well these two parameters, which ultimately affect the convergence rate of the resulting algorithm, involves an exercise in trial and error; note, however, that (following the modified analysis described above) convergence is proved for this modification of Algorithm 2 for any choice of $K_0$ and $L_0$. An analogous issue was encountered when selecting K in Algorithm 1 of [8]. Figure 13 shows the positions of the function evaluations and support points for the $n=2$ case, and the convergence histories for the $n=3$ and $n=4$ cases. The convergence of the modified algorithm proposed here is, again, seen to be quite rapid.