On convergence rate of a rectangular partition based global optimization algorithm

Abstract

The convergence rate of a rectangular partition based algorithm is considered. A hyper-rectangle for the subdivision is selected at each step according to a criterion rooted in the statistical models based theory of global optimization; only the objective function values are used to compute the criterion of selection. The convergence rate is analyzed assuming that the objective functions are twice- continuously differentiable and defined on the unit cube in d-dimensional Euclidean space. An asymptotic bound on the convergence rate is established. The results of numerical experiments are included.

Appendices

Appendix A

This appendix contains the proof of Lemma 2.

Denote the eigenvalues of $D^2f(x^*)$ by

$$\begin{aligned} \theta _1\ge \theta _2\ge \cdots \ge \theta _d >0. \end{aligned}$$

Then by Taylor’s theorem (x a column vector),

$$\begin{aligned} f(x^*+x)-f(x^*) = \frac{1}{2} x^T D^2f(x^*) x + o(\left\| x\right\| ^2). \end{aligned}$$

Since $D^2f(x^*)$ is symmetric positive definite, we can express it as

$$\begin{aligned} D^2f(x^*) = V\varTheta V^T \end{aligned}$$

for an orthogonal matrix V and diagonal matrix $\varTheta =\mathrm{diag}(\theta _1, \theta _2,\ldots ,\theta _d)$. Then

$$\begin{aligned} D^2f(x^*) = V \varTheta ^{1/2}\varTheta ^{1/2} V^T \end{aligned}$$

and

$$\begin{aligned} f(x^*+x)-f^* = \frac{1}{2} x^T V \varTheta ^{1/2}\varTheta ^{1/2} V^T x + o(\left\| x\right\| ^2) = \frac{1}{2} \left\| Tx\right\| ^2 + o(\left\| x\right\| ^2), \end{aligned}$$

(24)

where $Tx \equiv \varTheta ^{1/2}V^Tx$.

Let $B_c^d(0)$ denote the ball of radius c, centered at 0, in $\mathbb {R}^d$. For $c>0$, let

$$\begin{aligned} f_c \equiv \inf \{f(x^*+x):x\notin B_c^d(0)\}, \end{aligned}$$

which is positive since the minimizer is assumed unique and f is continuous. Then

$$\begin{aligned} \int _{[0,1]^d\setminus B_c^d(x^*)} \frac{dx}{(f(x)-f(x^*)+\epsilon )^{d/2}}\le f_c^{-d/2} \end{aligned}$$

and

$$\begin{aligned} \int _{[0,1]^d}\frac{dx}{(f(x)-f(x^*)+\epsilon )^{d/2}}&\le f_c^{-d/2}+\int _{B_c^d(0)}\frac{dx}{\left( f(x^*+x)-f(x^*)+\epsilon \right) ^{d/2}}\\&= f_c^{-d/2}+\int _{B_c^d(0)}\frac{dx}{\left( \frac{1}{2}\left\| Tx\right\| ^2+\epsilon +o(\left\| x\right\| ^2)\right) ^{d/2}} \end{aligned}$$

by (24). For any $\eta \in ]0,1/2]$, we can choose $c>0$ small enough so that

$$\begin{aligned} \frac{1}{2}\left\| Tx\right\| ^2+\epsilon +o(\left\| x\right\| ^2)\ge \left( \frac{1}{2}-\eta \right) \left\| Tx\right\| ^2+\epsilon . \end{aligned}$$

Therefore,

$$\begin{aligned} \int _{[0,1]^d} \frac{dx}{\left( f(x) - f(x^*) +\epsilon \right) ^{d/2}} \le \int _{B^d_c(0)} \frac{dx}{\left( (1/2-\eta ) \left\| Tx\right\| ^2 +\epsilon \right) ^{d/2}} +O(1) \end{aligned}$$

(25)

as $\epsilon \downarrow 0$, and similarly

$$\begin{aligned} \int _{[0,1]^d} \frac{dx}{\left( f(x) - f(x^*) +\epsilon \right) ^{d/2}} \ge \int _{B^d_c(0)} \frac{dx}{\left( (\frac{1}{2}+\eta )\left\| Tx\right\| ^2 +\epsilon \right) ^{d/2}} +O(1). \end{aligned}$$

(26)

Using the orthogonality of V, for any $b>0$,

$$\begin{aligned} \int _{B^d_c(0)} \frac{dx}{\left( b\left\| Tx\right\| ^2 +\epsilon \right) ^{d/2}}&= \int _{V(B^d_c(0))} \frac{dx}{\left( b\left\| \varTheta ^{1/2}x\right\| ^2 +\epsilon \right) ^{d/2}}\nonumber \\&= \int _{B^d_c(0)} \frac{dx}{\left( b\left\| \varTheta ^{1/2}x\right\| ^2 +\epsilon \right) ^{d/2}} \nonumber \\&= \int _{E(c,b,\epsilon )} \frac{dy}{\left( \left\| y\right\| ^2 +1\right) ^{d/2}}\frac{1}{b^{d/2}\left( \prod _{i=1}^d \theta _i\right) ^{1/2}}, \end{aligned}$$

(27)

where $E(c,b,\epsilon )$ is the image of the ball of radius c under the map $x_i \mapsto x_i(\theta _ib/\epsilon )^{1/2}$, and we used the substitution $y_i \leftarrow x_i\left( \theta _i b/\epsilon \right) ^{1/2}$ in the last equation. Therefore,

$$\begin{aligned} B^d_{c\sqrt{b\theta _d/\epsilon }}(0) \subset E(c,b,\epsilon ) \subset B^d_{c\sqrt{b\theta _1/\epsilon }}(0), \end{aligned}$$

and (27) gives the bounds

$$\begin{aligned}&\int _{ B^d_{c\sqrt{b\theta _d/\epsilon }}(0) } \frac{dy}{\left( \left\| y\right\| ^2 +1\right) ^{d/2}}\frac{1}{b^{d/2}\left( \prod _{i=1}^d \theta _i\right) ^{1/2}} \le \int _{B^d_c(0)} \frac{dx}{\left( b\left\| Tx\right\| ^2 +\epsilon \right) ^{d/2}}\\&\quad \le \int _{B^d_{c\sqrt{b\theta _1/\epsilon }}(0)} \frac{dy}{\left( \left\| y\right\| ^2 +1\right) ^{d/2}}\frac{1}{b^{d/2}\left( \prod _{i=1}^d \theta _i\right) ^{1/2}}. \end{aligned}$$

Let $\mathcal {V}(x)=\pi ^{d/2}\varGamma (d/2 +1)^{-1}x^d$ denote the volume of the d-dimensional ball of radius x. For $z>0$,

$$\begin{aligned} \int _{B^d_z(0)}\frac{dx}{\left( \left\| x\right\| ^2 + 1\right) ^{d/2}}= & {} \int _{r=0}^z \frac{d\mathcal {V}(r)}{(r^2+1)^{d/2}} =\frac{d \pi ^{d/2}}{\varGamma (d/2 +1)} \int _{r=0}^z \frac{r^{d-1}dr}{(r^2+1)^{d/2}}\\= & {} \frac{d \pi ^{d/2}}{\varGamma (d/2 +1)} \int _{r=0}^z \frac{1}{r}\left( \frac{r^2}{r^2+1}\right) ^{d/2}dr. \end{aligned}$$

For $z>1$,

$$\begin{aligned} \int _{r=0}^z \frac{1}{r}\left( \frac{r^2}{r^2+1}\right) ^{d/2}dr < \int _{r=1}^z \frac{1}{r}dr = \log (z), \end{aligned}$$

(28)

and for $A<z$,

$$\begin{aligned}&\int _{r=0}^z \frac{1}{r}\left( \frac{r^2}{r^2+1}\right) ^{d/2}dr > \left( \frac{A^2}{A^2+1}\right) ^{d/2}\int _{r=A}^z \frac{1}{r} dr\\&\quad = \left( \frac{A^2}{A^2+1}\right) ^{d/2} \left( \log (z)-\log (A)\right) . \end{aligned}$$

Combining this inequality with (28) we conclude that

$$\begin{aligned} \lim _{z\uparrow \infty } \frac{ \int _{r=0}^z \frac{1}{r}\left( \frac{r^2}{r^2+1}\right) ^{d/2} dr}{\log (z)} =1. \end{aligned}$$

(29)

Set

$$\begin{aligned} C_d = \frac{d\pi ^{d/2}}{\varGamma (1+d/2)} \end{aligned}$$

and

$$\begin{aligned} G_d(c) = \int _{r=0}^c\frac{r^{d-1}dr}{(r^2+1)^{d/2}}. \end{aligned}$$

By (29), $G_d(c)/\log (c)\rightarrow 1$ as $c\uparrow \infty$. For any $b>0$, we have the bounds

$$\begin{aligned}&{b^{-d/2}\left( \prod _{i=1}^d\theta _i\right) ^{-1/2}} C_d G_d\left( c\sqrt{b\theta _d/\epsilon }\right) \le \int _{B^d_c(0)} \frac{dx}{\left( b\left\| Tx\right\| ^2 +\epsilon \right) ^{d/2}}\\&\quad \le {b^{-d/2}\left( \prod _{i=1}^d\theta _i\right) ^{-1/2}} C_d G_d\left( c\sqrt{b\theta _1/\epsilon }\right) , \end{aligned}$$

and

$$\begin{aligned}&O(1) + (1/2+\eta ))^{-d/2}\left( \prod _{i=1}^d \theta _i\right) ^{-1/2} C_d G_d\left( c\sqrt{(1/2+\eta )\theta _d/\epsilon }\right) \\&\quad \le {\mathcal I_f}(\epsilon ) \le O(1) + (1/2-\eta )^{-d/2}\left( \prod _{i=1}^d \theta _i\right) ^{-1/2} C_d G_d\left( c\sqrt{(1/2-\eta )\theta _1/\epsilon }\right) . \end{aligned}$$

Since $\eta >0$ is arbitrary, the fact that $G_d(c)/\log (c)\rightarrow 1$ as $c\uparrow \infty$ completes the proof.

Appendix B

This appendix contains the proofs of the lemmas upon which the proof of Theorem 1 are based.

Proof of Lemma 3

Let $m\le n$ be the last time before n that the smallest hyper-rectangle was about to be split, so $v_m = 2v_n$ and by (5),

$$\begin{aligned} \rho ^m\le \frac{1}{\lambda \log (1/v_m)} = \frac{1}{\lambda \log (1/2v_n)}. \end{aligned}$$

Note that $m\ge n_0$ by definition of $n_2$.

We will show by induction that

$$\begin{aligned} \rho ^{m+k} \le \frac{2}{\lambda \log (1/v_{m+k})} \end{aligned}$$

(30)

for $k\in \{1,2,\ldots ,n-m\}$, if the hyper-rectangle that the algorithm is subdividing at iteration $m+k$ has not been previously subdivided since time m. At other times the bound is twice as large.

Let us first consider $\{\rho ^{m+1}_i: i\le m+1\}$. Suppose that i is the split hyper-rectangle at time m, and let j denote a non-split hyper-rectangle, so $\rho ^m_j \le \rho _i^m$. Then there exists a point $c_j\in R_j$ such that

$$\begin{aligned} \rho ^{m+1}_j&= \frac{\left| R_j\right| }{\left( L_{m}(c_j)-M_{m+1} + g(v_{m+1})\right) ^{d/2} }\\&\le \left( \frac{g(2v_n)}{g(v_n)}\right) ^{d/2} \frac{\left| R_j\right| }{\left( L_{m}(c_j)-M_{m} + g(2v_{n})\right) ^{d/2} }\ \ \text {since}~ g(2v_n)>g(v_n)\\&= \left( \frac{g(2v_n)}{g(v_n)}\right) ^{d/2} \rho ^m_j \le \left( 2 \frac{\log (1/2v_n)}{\log (1/v_n)}\right) \frac{1}{\lambda \log (1/v_m)} \ \ \text {by }~(5)\\&= 2 \frac{\log (1/2v_n)}{\log (1/v_n)}\frac{1}{\lambda \log (1/2v_n)} = \frac{2}{\lambda \log (1/v_{n})} = \frac{2}{\lambda \log (1/v_{m+1})}. \end{aligned}$$

Next consider a child of the split subrectangle i. Since we are splitting the smallest subrectangle, $v_m = 2v_n$, and there exists a point $c_i\in R_i$ such that (supposing that one child has index i at time $m+1$)

$$\begin{aligned} \rho ^{m+1}_i = \frac{\left| R_i\right| /2}{\left( L_{m{+1}}(c_i)-M_{m+1} + g(v_{m+1})\right) ^{d/2} } \le \frac{v_{m+1}}{g(v_{m+1})^{d/2}} = \frac{1}{\lambda \log (1/v_{m+1})}. \end{aligned}$$

We have established the base case for the induction. Now consider iteration $m+k+1$, $1\le k<n-m$. For the induction hypothesis, assume that

$$\begin{aligned} \rho _i^{m+k} \le \frac{2}{\lambda \log (1/v_{n})} \end{aligned}$$

if hyper-rectangle i has not been split since time m and

$$\begin{aligned} \rho _i^{m+k} \le \frac{4}{\lambda \log (1/v_{n})} \end{aligned}$$

if hyper-rectangle i has been split since time m.

Suppose that the most promising hyper-rectangle at iteration $m+k+1$ is subrectangle $R_j=\prod _{i=1}^d[a_i,b_i]$,

$$\begin{aligned} \rho ^{m+k+1} = \rho ^{m+k+1}_j = \int _R \frac{ds}{(L_{m+k+1}(s)-M_{m+k+1}+g(v_n))^{d/2}}. \end{aligned}$$

Assume that this hyper-rectangle has not been split since time m. Let us suppose that during this interval (between m and the next time that the smallest hyper-rectangle is split) R is split r times, and denote by $L^\prime$ the piecewise multilinear function defined over R after the splitting evaluations. Then

$$\begin{aligned} \max _{s\in R}\left| L^\prime (s)-L_{m+k+1}(s)\right| \le \max _{s\in R}\left| f(s)-L_{m+k+1}(s)\right| \le \gamma (s), \end{aligned}$$

where

$$\begin{aligned} \gamma (s) = \frac{1}{2}\sum _{i=1}^d(s-a_i)(b_i-s) \left\| D^2f\right\| _{\infty , R} \le \frac{1}{8} q d \left| R\right| ^{2/d} \left\| D^2f\right\| _{\infty , R} \equiv \hat{\gamma }. \end{aligned}$$

Set

$$\begin{aligned} \overline{\gamma }(s) = \min \{\gamma (s), L_{m+k+1}(s)-M_{m+k+1}\}. \end{aligned}$$

Then, denoting by $\rho _1$ the sum of the $\rho$ values for the resulting subrectangles of R at time $m+k+q$, we have

$$\begin{aligned} \rho _1= & {} \int _R \frac{ds}{(L^\prime (s)-M_{m+k+q}+g(v_n))^{d/2}}\\\le & {} \int _R \frac{ds}{(L_{m+k+1}(s) - \overline{\gamma }(s)-M_{m+k+1}+g(v_n))^{d/2}}\\\le & {} \int _R \frac{ds}{(L_{m+k+1}(s) - M_{m+k+1}+g(v_n)-{\hat{\gamma }})^{d/2}}. \end{aligned}$$

Let

$$\begin{aligned} a+Q(s) = L_{m+k+1}(s)-M_{m+k+1}+g(v_n), \end{aligned}$$

where $a=\min _{s\in R} L_{m+k+1}(s)-M_n+g(v_n) >0$. Then

$$\begin{aligned} \frac{\rho _1}{\rho _0} \le \frac{ \int _R\frac{ds}{(a+Q(s)-\hat{\gamma })^{d/2}} }{ \int _R\frac{ds}{(a+Q(s))^{d/2}} }. \end{aligned}$$

The latter ratio is maximized by $Q\equiv 0$, which corresponds to $\rho _0 = \left| R\right| /a^{d/2}$, and so

$$\begin{aligned} \frac{\rho _1}{\rho _0}&\le \frac{ \int _R\frac{ds}{(a-\hat{\gamma })^{d/2}} }{ \int _R\frac{ds}{(a)^{d/2}} } = \frac{1}{\left( 1-\frac{\hat{\gamma }}{a}\right) ^{d/2}} = \frac{1}{\left( 1-\frac{ \frac{1}{8} q d \left| R\right| ^{2/d} \left\| D^2f\right\| _{\infty , R} }{a}\right) ^{d/2}}\\&\le \frac{1}{\left( 1- \frac{1}{8} q d \rho _0^{2/d} \left\| D^2f\right\| _{\infty , R} \right) ^{d/2}} \le \frac{1}{\left( 1- \frac{1}{8} q d \frac{4\cdot 4^{2/d}}{q d^3 \left\| D^2f\right\| _{\infty , R}} \left\| D^2f\right\| _{\infty , R} \right) ^{d/2}}\\&= \frac{1}{\left( 1- \frac{4^{2/d}}{2d^2 } \right) ^{d/2}} \le 2. \end{aligned}$$

We used the inequalities

$$\begin{aligned}&\rho _0^{2/d}\le \left( \frac{4}{\lambda \log (n)}\right) ^{2/d} \le \left( \frac{4}{\lambda \left( d \left\| D^2f\right\| _{\infty , R} \right) ^{d/2}}\right) ^{2/d}\\&=\frac{\left( \frac{4}{\lambda }\right) ^{2/d}}{d \left\| D^2f\right\| _{\infty , R}} =\frac{4\cdot 4^{2/d}}{q d^3 \left\| D^2f\right\| _{\infty , R}}, \end{aligned}$$

which follows from the induction hypothesis.

We have shown that whenever the algorithm is about to subdivide a hyper-rectangle for the first time since m, $\rho ^{m+k}\le 2/\lambda \log (m+k)$. After a hyper-rectangle is subdivided, subsequent subdivisions can never result in $\rho$ more than twice as large.

The proof by induction of (30) is complete. $\square$

Proof of Lemma 4

Suppose (to get a contradiction) that $\left| R_i\right| =v_n^*$, $\left| R_j\right| =v_n$, and $v_n^*=4 v_n$, but yet we are about to split hyper-rectangle $R_j$, resulting in $v_n^*>4v_n$; that is, $\rho ^n_j \ge \rho ^n_i$:

$$\begin{aligned} \rho ^n_j = \frac{\left| R_j\right| =v_n}{\left( L_n(c_j)-M_n + g(v_n)\right) ^{d/2} } \ge \frac{\left| R_i\right| =v_n^*=4v_n}{\left( L_n(c_i)-M_n + g(v_n)\right) ^{d/2} } =\rho ^n_i \end{aligned}$$

for some $c_i\in R_i, c_j\in R_j$. This implies that

$$\begin{aligned} \frac{L_n(c_i)-M_n + g(v_n)}{L_n(c_j)-M_n + g(v_n)} \ge 4^{2/d}. \end{aligned}$$

(31)

But $L_n(c_j) \ge M_n$ and

$$\begin{aligned} L_n(c_i) \le M_n + \frac{1}{8} q d \left\| D^2f\right\| _{\infty ,R_i}\left| R_i\right| ^{2/d}. \end{aligned}$$

Thus

$$\begin{aligned} L_n(c_i)-L_n(c_j) \le \frac{1}{8} q d \left\| D^2f\right\| _{\infty ,[0,1]^d}v_n^{2/d}4^{2/d}. \end{aligned}$$

This means that

$$\begin{aligned}&\frac{L_n(c_i)-M_n + g(v_n)}{L_n(c_j)-M_n + g(v_n)} = 1+ \frac{L_n(c_i)-L_n(c_j)}{L_n(c_j)-M_n + g(v_n)} \\&\quad = 1+ \frac{L_n(c_i)-L_n(c_j)}{v_n^{2/d}} \frac{v_n^{2/d}}{L_n(c_j)-M_n + g(v_n)} \\&\quad = 1+ \frac{L_n(c_i)-L_n(c_j)}{v_n^{2/d}} (\rho ^n_j)^{2/d} \\&\quad \le 1+ \frac{1}{8} q d \left\| D^2f\right\| _{\infty ,[0,1]^d}\left( 4^{2/d}\right) \left( \frac{4}{\lambda \log (n)}\right) ^{2/d} \\&\quad \le 1+ \frac{1}{8} q d \left\| D^2f\right\| _{\infty ,[0,1]^d}\left( 4^{2/d}\right) \frac{4^{2/d}}{(qd^2/4)\log (n)^{2/d}} \\&\quad \le 1+ \frac{1}{2} q d \left\| D^2f\right\| _{\infty ,[0,1]^d}\left( 4^{2/d}\right) \frac{4^{2/d}}{(qd^2) d\left\| D^2f\right\| _{\infty ,[0,1]^d}}\ \ \text {since}~ n\ge n_2 \\&\quad = 1+ \frac{1}{2d^2} 4^{4/d} < 4^{2/d} \end{aligned}$$

since $d\ge 2$. But this contradicts (31), and establishes (9).

The proof of (10) follows from

$$\begin{aligned} \frac{\varDelta _n}{g(v_n)}&\le \frac{ \frac{1}{8} q d \left\| D^2f\right\| _{\infty ,R_i}(4 v_n)^{2/d} }{ \frac{1}{4}q d^2\left( v_n \log (1/v_n)\right) ^{2/d} } = \frac{ \left\| D^2f\right\| _{\infty ,R_i}4^{2/d} }{ 2d\left( \log (1/v_n)\right) ^{2/d} } \le \frac{ \left\| D^2f\right\| _{\infty ,R_i}4^{2/d} }{2d\left( \log (n)\right) ^{2/d} }\\&\le \frac{ \left\| D^2f\right\| _{\infty ,R_i}4^{2/d} }{ 2d\cdot d \left\| D^2f\right\| _{\infty ,[0,1]^d}} \le \frac{ 4^{2/d} }{ 2d^2 } \le \frac{2}{d^2}. \end{aligned}$$

$\square$

Proof of Lemma 5

Recall that $R^*_n$ denotes the hyper-rectangle containing $x^*$ at time n, and that $\left| R^*_n\right| = v^*_n \le 4 v_n$ by Lemma 4. If h is the minimal edge length of $R^*_n$, then $v^*_n = 2^j h^d$ for some $0 \le j <d$, and the diameter of $R^*_n$ is less than $2h\sqrt{d}$. Also $h = 2^{-j/d}(v^*_n)^{1/d}$. Therefore, for $s\in R^*_n$,

$$\begin{aligned} L_n(s) -M_n&\le L_n(s)-f(x^*)\\&\le \max _{s\in R_n^*}f(s)-f(x^*)\\&\le \frac{1}{2}\left( 2h\sqrt{d}\right) ^2 \left\| D^2f\right\| _{\infty ,R^*_n}\ \ \text {by Taylor's theorem} \\&= \frac{1}{2}\left( 2\cdot 2^{-j/d}(v^*_n)^{1/d}\sqrt{d}\right) ^2 \left\| D^2f\right\| _{\infty ,R^*_n}\\&\le 2 d (v^*_n)^{2/d} \left\| D^2f\right\| _{\infty ,[0,1]^d}\\&\le 2 d (4v_n)^{2/d} \left( \frac{\log (n)}{d}\right) ^{2/d}\ \ \ \text { by Lemma}~4~\text { and }~n\ge n_2\\&\le 2 d (4v_n)^{2/d} \left( \frac{\log (1/v_n)}{d}\right) ^{2/d} = 2\cdot 4^{2/d} \frac{d}{d^{2/d}}\lambda ^{-\,2/d}g(v_n)\\&= 2\cdot 4^{2/d} \frac{d}{d^{2/d}}\frac{4}{qd^2}g(v_n) < \frac{32}{dq}g(v_n). \end{aligned}$$

Therefore, by the previous inequality

$$\begin{aligned} \rho ^n&\ge \int _{R^*_n}\frac{ds}{\left( L_n(s)-M_n+g(v_n)\right) ^{d/2}} \ge \frac{v^*_n}{\left( \frac{32}{dq}g(v_n)+g(v_n)\right) ^{d/2}}\\&\ge \frac{v_n}{\lambda v_n \log (1/v_n)\left( 1+\frac{32}{dq}\right) ^{d/2}} = \frac{1}{\lambda \log (1/v_n)\left( 1+\frac{32}{dq}\right) ^{d/2}}\\&\ge \frac{1}{\lambda \log (1/v_n)\exp \left( \frac{16}{q}\right) } = \frac{\exp (-16/q)}{\lambda \log (1/v_n)}. \end{aligned}$$

$\square$

Proof of Lemma 6

Equation (11) follows from Lemma 3.

For $n\ge n_2(f)$, the $\rho$ values for the children of a split hyper-rectangle will not be much smaller than the parent. The largest possible decrease occurs if the values on the boundary of $R_i$ are constant, say with value $M_{m+k}+A$, and the new function values are the largest possible, namely $M_{n}+A+a$, where

$$\begin{aligned} a \le \frac{1}{8}q d \left\| D^2f\right\| _{\infty , R_i} \left| R_i\right| ^{2/d}. \end{aligned}$$

Considering a child of the split hyper-rectangle (say with index i),

$$\begin{aligned} \rho ^{n+1}_i&\ge \frac{\left| R_i\right| /2}{\left( A+a+g({v_n})\right) ^{d/2}} \ge \frac{\left| R_i\right| /2}{\left( A+g({v_n})\right) ^{d/2} \left( 1+\frac{a}{A+g({v_n})} \right) ^{d/2}}\\&\ge \frac{\left| R_i\right| /2}{\left( A+g({v_n})\right) ^{d/2} \left( 1+\frac{ \frac{1}{8}q d \left\| D^2f\right\| _{\infty , R_i} \left| R_i\right| ^{2/d} }{\left| R_i\right| ^{2/d}} \frac{\left| R_i\right| ^{2/d}}{A+g({v_n})} \right) ^{d/2}}\\&\ge \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+\frac{ \frac{1}{8}q d \left\| D^2f\right\| _{\infty , R_i} \left| R_i\right| ^{2/d} }{\left| R_i\right| ^{2/d}} \frac{\left| R_i\right| ^{2/d}}{A+g({v_n})} \right) ^{d/2}}\\&= \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+\frac{ \frac{1}{8}q d \left\| D^2f\right\| _{\infty , R_i} \left| R_i\right| ^{2/d} }{\left| R_i\right| ^{2/d}} \left( \rho ^{n}_i\right) ^{2/d} \right) ^{d/2}}\\&\ge \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+\frac{ \frac{1}{8}q d \left\| D^2f\right\| _{\infty , R_i} \left| R_i\right| ^{2/d} }{\left| R_i\right| ^{2/d}} \left( \frac{4}{\lambda \log (n)}\right) ^{2/d} \right) ^{d/2}}\ \ \ \mathrm{(}\text {by Lemma}~3\mathrm{)}\\&\ge \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+\frac{ \frac{1}{8}q d \left\| D^2f\right\| _{\infty , R_i} \left| R_i\right| ^{2/d} }{\left| R_i\right| ^{2/d}} \left( \frac{{4}}{\lambda d^{d/2}\left\| D^2f\right\| _{\infty ,[0,1]^d}^{d/2} }\right) ^{2/d} \right) ^{d/2}}\ \ \ \mathrm{(}\text {since}~n\ge n_2\mathrm{)}\\&\ge \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+ \frac{1}{8}q d \left( \frac{{4}}{ (q d^3/4)^{d/2} }\right) ^{2/d} \right) ^{d/2}} = \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+ \frac{1}{8}q d \frac{{{4}}^{2/d}}{ q d^3/4 } \right) ^{d/2}}\\&= \frac{1}{2}\rho ^{n}_i \frac{1}{ \left( 1+ \frac{1}{2} \frac{{{4}}^{2/d}}{ d^2} \right) ^{d/2}} \ge \frac{1}{2} \rho ^{n}_i \exp \left( -\frac{1}{4d}{{4}}^{2/d}\right) \ge \frac{1}{2} e^{-1/{{2}}} \rho ^{n}_i, \end{aligned}$$

using the inequality $(1+x/k)^k \le \exp (x)$ and the fact that $d\ge 2$ implies that $4^{2/d}/(4d)\le 1/2$.

Consider the average of the $\{\rho ^n_i\}$ at time n that were the product of splits after time n / 2. Over this time interval we have

$$\begin{aligned} \rho ^k \ge \frac{\exp (-16/q)}{\lambda \log (1/v_k)} \ge \frac{\exp (-16/q)}{\lambda \log (1/v_n)} \end{aligned}$$

by Lemma 5, since $n\ge 2n_2(f)$. Since the children $\rho$ values will not be much smaller than the parent’s,

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^n\rho ^n_i \ge \frac{1}{n}\frac{n/2}{n/2}\sum _{i=n/2}^n\rho ^i \ge \frac{1}{2}\frac{1}{n/2}\sum _{i=n/2}^n \frac{\exp (-16/q)}{\lambda \log (1/v_i)} \ge \frac{1}{{2}} \frac{\exp (-16/q)}{\lambda \log (1/v_n)}, \end{aligned}$$

since $v_i\ge v_n$. $\square$

Proof of Lemma 7

Fix a particular hyper-rectangle $R_i$ with

$$\begin{aligned} \rho ^n_i = \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}}. \end{aligned}$$

We first show that the integrals

$$\begin{aligned} \int _{R_i}\frac{ds}{(f(s)-f^*+g(v_n))^{d/2}},\ \ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} \end{aligned}$$

are close. As in the proof of Lemma 3, let

$$\begin{aligned} a+Q(s) = L(s)-f^*+g(v_n), \end{aligned}$$

where $a=\min _{s\in {R_i}} L_n(s)-f^*+g(v_n) >0$. Then, from (6),

$$\begin{aligned} \max _{x\in {R_i}}\left| f(x)-L_n(x)\right| \le \frac{1}{8} q\cdot d \left\| D^2f\right\| _{\infty , {R_i}} \left| {R_i}\right| ^{2/d}\equiv b, \end{aligned}$$

and

$$\begin{aligned} \frac{ \int _{R_i}\frac{ds}{(f(s)-f^*+g(v_n))^{d/2}} }{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} } \ge \frac{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n)+b)^{d/2} } }{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} } = \frac{ \int _{R_i}\frac{ds}{(a+Q(s)+b)^{d/2} } }{ \int _{R_i}\frac{ds}{(a+Q(s))^{d/2}} }. \end{aligned}$$

The smallest value of the ratio occurs when $Q(s)\equiv 0$.

The case $Q\equiv 0$ corresponds to $\rho = \left| {R_i}\right| /a^{d/2}$, and so

$$\begin{aligned}&\frac{ \int _{R_i}\frac{ds}{(a+Q(s)+b)^{d/2} } }{ \int _{R_i}\frac{ds}{(a+Q(s))^{d/2}} } \ge \frac{ \int _{R_i}\frac{ds}{(a+b)^{d/2} } }{ \int _{R_i}\frac{ds}{a^{d/2}} } =\left( \frac{a}{a+b}\right) ^{d/2}\\&\quad = \frac{1}{\left( 1+ \frac{1}{8} q\cdot d \left\| D^2f\right\| _{\infty , {R_i}}\frac{ \left| {R_i}\right| ^{2/d}}{a}\right) ^{d/2}} = \frac{1}{\left( 1+\frac{1}{8} q\cdot d \left\| D^2f\right\| _{\infty , {R_i}} \rho ^{2/d}\right) ^{d/2}}. \end{aligned}$$

Now

$$\begin{aligned} \frac{1}{8} q\cdot d \left\| D^2f\right\| _{\infty , {R_i}} \rho ^{2/d}&\le \frac{1}{8} q\cdot d \left\| D^2f\right\| _{\infty , {R_i}} \left( \frac{4}{\lambda \log (n)}\right) ^{2/d}\\&\le \frac{1}{8} q\cdot d \left\| D^2f\right\| _{\infty , {R_i}} \frac{4^{2/d}}{\frac{qd^3}{4} \left\| D^2f\right\| _{\infty , {R_i}}} = \frac{4^{2/d}}{2d^2}. \end{aligned}$$

Therefore,

$$\begin{aligned} \frac{ \int _{R_i}\frac{ds}{(a+Q(s)+b)^{d/2} } }{ \int _{R_i}\frac{ds}{(a+Q(s))^{d/2}} } \ge \frac{1}{\left( 1+ \frac{4^{2/d}}{2d^2}\right) ^{d/2}} \ge \frac{2}{3}, \end{aligned}$$

since the last expression is increasing in $d\ge 2$. Turning to an upper bound for

$$\begin{aligned} \frac{ \int _{R_i}\frac{ds}{(f(s)-f^*+g(v_n))^{d/2}} }{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} }, \end{aligned}$$

observe that

$$\begin{aligned} \frac{ \int _{R_i}\frac{ds}{(f(s)-f^*+g(v_n))^{d/2}} }{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} }\le & {} \frac{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n)-b) } }{ \int _{R_i}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} } \equiv \frac{ \int _{R_i}\frac{ds}{(a+Q(s)-b)^{d/2} } }{ \int _{R_i}\frac{ds}{(a+Q(s))^{d/2}} }\\\le & {} \frac{1}{\left( 1-\frac{b}{a}\right) ^{d/2}} \le \frac{1}{\left( 1- \frac{4^{2/d}}{2d^2}\right) ^{d/2}} \le 2. \end{aligned}$$

We have shown that

$$\begin{aligned}&\frac{2}{3}\int _{[0,1]^d}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}} \le \int _{[0,1]^d}\frac{ds}{(f(s)-f^*+g(v_n))^{d/2}}\nonumber \\&\quad \le 2\int _{[0,1]^d}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}}. \end{aligned}$$

(32)

Recall that $v_n^*$ denotes the volume of the hyper-rectangle containing the minimizer $x^*$. Since $n\ge n_2$, by Lemma 4 $v_n^* \le 4 v_n$ and

$$\begin{aligned} M_n-f^* \le \frac{g(v_n)}{d^2/2}. \end{aligned}$$

Therefore,

$$\begin{aligned} \int _R\frac{ds}{(L_n(s)-M_n +g(v_n))^{d/2}}&= \int _R\frac{ds}{(L_n(s)-M_n +g(v_n) +M_n-f^*)^{d/2}}\\&= \int _R\frac{ds}{ (L_n(s)-M_n +g(v_n))^{d/2} \left( 1+\frac{\varDelta _n}{L_n(s)-M_n +g(v_n)}\right) ^{d/2}}\\&\ge \int _R\frac{ds}{(L_n(s)-M_n +g(v_n)(1+2/d^2))^{d/2}}\ \ \text {by}~(10)\\&=\frac{1}{\left( 1+2/d^2\right) ^{d/2}} \int _R\frac{ds}{(L_n(s)-M_n +g(v_n))^{d/2}}\\&\ge \frac{2}{3} \int _R\frac{ds}{(L_n(s)-M_n +g(v_n))^{d/2}}. \end{aligned}$$

Combining these inequalities with (32) gives

$$\begin{aligned}&\frac{4}{9}\int _{[0,1]^d}\frac{ds}{(L_n(s)-M_n+g(v_n))^{d/2}} \le \frac{2}{3}\int _{[0,1]^d}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}}\\&\quad \le \int _{[0,1]^d}\frac{ds}{(f(s)-f^*+g(v_n))^{d/2}} \le 2\int _{[0,1]^d}\frac{ds}{(L_n(s)-f^*+g(v_n))^{d/2}}\\&\quad \le 2\int _{[0,1]^d}\frac{ds}{(L_n(s)-M_n+g(v_n))^{d/2}}. \end{aligned}$$

Since

$$\begin{aligned} \sum _{i=1}^{{n}}\rho _i^n = \int _{[0,1]^d}\frac{ds}{(L_n(s)-M_n+g(v_n))^{d/2}}, \end{aligned}$$

this completes the proof. $\square$