Nonmonotone line searches for unconstrained multiobjective optimization problems

Abstract

In the last two decades, many descent methods for multiobjective optimization problems were proposed. In particular, the steepest descent and the Newton methods were studied for the unconstrained case. In both methods, the search directions are computed by solving convex subproblems, and the stepsizes are obtained by an Armijo-type line search. As a consequence, the objective function values decrease at each iteration of the algorithms. In this work, we consider nonmonotone line searches, i.e., we allow the increase of objective function values in some iterations. Two well-known types of nonmonotone line searches are considered here: the one that takes the maximum of recent function values, and the one that takes their average. We also propose a new nonmonotone technique specifically for multiobjective problems. Under reasonable assumptions, we prove that every accumulation point of the sequence produced by the nonmonotone version of the steepest descent and Newton methods is Pareto critical. Moreover, we present some numerical experiments, showing that the nonmonotone technique is also efficient in the multiobjective case.

Appendix A

Here, we list the test problems used in Sect. 7. For each problem, we state the original reference, the number of variables n, the number of objective functions m, the convexity property, the objective functions, and the bounds L and U of the box constraints.

• 1. Das and Dennis (DD1) [1]: $n=5$, $m=2$, nonconvex,¹$^{,}$²

  $$\begin{aligned} F_{1}(x)&=x_{1}^{2}+x_{2}^{2}+x_{3}^{2}+x_{4}^{2}+x_{5}^{2},\\ F_{2}(x)&=3x_{1}+2x_{2}-\frac{x_{3}}{3}+0.01(x_{4}-x_{5})^{3}, \end{aligned}$$

  $L=(-20, \ldots , -20)^{\top }$, and $U=(20, \ldots , 20)^{\top }$.

• 2. Fliege, Graña Drummond and Svaiter (FDS) [4]: $n=10$, $m=3$, convex,$^2$

  $$\begin{aligned} F_{1}(x)&=\frac{1}{n^{2}}\sum _{i=1}^{n}i(x_{i}-i)^{4},\\ F_{2}(x)&=\exp \left( \sum _{i=1}^{n}\frac{x_{i}}{n}\right) +\Vert x\Vert _{2}^{2},\\ F_{3}(x)&=\frac{1}{n(n+1)}\sum _{i=1}^{n}i(n-i+1)e^{-x_{i}}, \end{aligned}$$

  $L=(-2, \ldots , -2)^{\top }$, and $U=(2,\ldots , 2)^{\top }$.

• 3. Jin, Olhofer and Sendhoff (JOS1) [8]: $n=5$, $m=2$, quadratic convex,$^2$

  $$\begin{aligned} F_{1}(x)&=\frac{1}{n}\sum _{i=1}^{n}x_{i}^{2},\\ F_{2}(x)&=\frac{1}{n}\sum _{i=1}^{n}(x_{i}-2)^{2}, \end{aligned}$$

  $L=(-2,\ldots , -2)^{\top }$, and $U=(2,\ldots , 2)^{\top }$.

• 4. Kim and Weck (KW2) [9]: $n=2$, $m=2$, nonconvex,

  $$\begin{aligned} F_{1}(x) =&-3(1-x_{1})^{2} \exp (-x_{1}^{2}-(x_{2}+1)^{2})\\&+ 10\left( \frac{x_{1}}{5}-x_{1}^{3}-x_{2}^{5}\right) \exp (-x_{1}^{2}-x_{2}^{2})\\&+3 \exp (-(x_{1}+2)^{2}-x_{2}^{2})-0.5(2x_{1}+x_{2}),\\ F_{2}(x) =&-3(1+x_{2})^{2} \exp (-x_{2}^{2}-(1-x_{1})^{2}) \\&+ 10\left( -\frac{x_{2}}{5}+x_{2}^{3}+x_{1}^{5}\right) \exp (-x_{1}^{2}-x_{2}^{2})\\&+ 3 \exp (-(2-x_{2})^{2}-x_{1}^{2}), \end{aligned}$$

  $L=(-3,-3)^{\top }$, and $U=(3,3)^{\top }$.

• 5. Stadler and J. Dauer (SD) [18]: $n=4$, $m=2$, convex,

  $$\begin{aligned} F_{1}(x)&=2x_{1}+\sqrt{2}x_{2}+\sqrt{2}x_{3}+x_{4},\\ F_{2}(x)&=\frac{2}{x_{1}}+\frac{2\sqrt{2}}{x_{2}}+\frac{2\sqrt{2}}{x_{3}}+\frac{2}{x_{4}}, \end{aligned}$$

  $L=\left( 1, \sqrt{2}, \sqrt{2}, 1\right) ^{\top }$, and $U=(3, 3, 3, 3)^{\top }$.

• 6. Zitzler, Deb and Thiele (ZDT1) [21]: $n=30$, $m=2$, convex,$^2$

  $$\begin{aligned} F_{1}(x)&=x_{1},\\ F_{2}(x)&=g(x)\left( 1-\sqrt{\frac{x_{1}}{g(x)}}\right) , \end{aligned}$$

  with $g(x)=1+9\sum _{i=2}^{n} x_{i}/(n-1)$, $L=(0,\ldots , 0)^{\top }$, and $U=\left( \frac{1}{100},\ldots , \frac{1}{100}\right) ^{\top }$.

• 7. Zitzler, Deb and Thiele (ZDT4) [21]: $n=10$, $m=2$, nonconvex,$^2$

  $$\begin{aligned} F_{1}(x)&=x_{1},\\ F_{2}(x)&=g(x)\left( 1-\sqrt{\frac{x_{1}}{g(x)}}\right) , \end{aligned}$$

  with $g(x)=1+10(n-1)+\sum _{i=2}^{n}\left( x_{i}^{2}-10\cos (4\pi x_{i})\right)$, $L=\left( \frac{1}{100}, -5,\ldots , -5\right) ^{\top }$, and $U=(1, 5,\ldots , 5)^{\top }$.

• 8. Toint (TOI4) [19, Problem 4]: $n=4$, $m=2$, convex,³

  $$\begin{aligned} F_1(x)&=x_1^2+x_2^2+1,\\ F_2(x)&=0.5\left( (x_1-x_2)^2+(x_3-x_4)^2\right) +1, \end{aligned}$$

  $L=(-2,\ldots , -2)^{\top }$, and $U=(5,\ldots , 5)^{\top }$.

• 9. TRIDIA [19, Problem 8]: $n=3$, $m=3$, convex,$^{22}$

  $$\begin{aligned} F_1(x)&=(2x_{1}-1)^{2}, \\ F_2(x)&=2(2x_{1}-x_{2})^{2}, \\ F_3(x)&=3(2x_{2}-x_{3})^{2}, \end{aligned}$$

  $L=(-1, -1, -1)^{\top }$, and $U=(1, 1, 1)^{\top }$.

• 10. Shifted TRIDIA [19, Problem 9]: $n=4$, $m=4$, nonconvex,$^{23}$

  $$\begin{aligned} F_{1}(x)&=(2x_{1}-1)^{2}+x_{2}^{2},\\ F_{i}(x)&=i(2x_{i-1}-x_{i})^{2}-(i-1)x_{i-1}^{2}+ix_{i}^{2} \quad i=2,3,\\ F_{4}(x)&=4(2x_{3}-x_{4})^{2}-3x_{3}^{2}, \end{aligned}$$

  $L=(-1,\ldots , -1)^{\top }$, and $U=(1,\ldots , 1)^{\top }$.

• 11. Rosenbrock [19, Problem 10]: $n=4$, $m=3$, nonconvex,$^{23}$

  $$\begin{aligned} F_{i}(x)&=100(x_{i+1}-x_{i}^{2})^{2}+(x_{i+1}-1)^{2}, \quad i=1,2,3, \end{aligned}$$

  $L=(-2,\ldots , -2)^{\top }$, and $U=(2,\ldots , 2)^{\top }$.

• 12. Helical valley [14, Problem (7)]: $n=3$, $m=3$, nonconvex,$^3$

  $$\begin{aligned} F_{1}(x)&=\left\{ \begin{array}{ll} \displaystyle { \left[ 10\left( x_3-\frac{5}{\pi }\arctan \left( \frac{x_2}{x_1}\right) \right) \right] ^2,} &{} \text{ if }\;x_1>0,\\ \displaystyle { \left[ 10\left( x_3-\frac{5}{\pi }\arctan \left( \frac{x_2}{x_1}\right) -5\right) \right] ^2,} &{} \text{ if }\;x_1<0 \end{array}\right. \\ F_{2}(x)&=\left( 10\left( (x_1^2+x_2^2)^{1/2}-1\right) \right) ^2,\\ F_{3}(x)&=x_3^2, \end{aligned}$$

  $L=(-2, -2, -2)^{\top }$, and $U=(2, 2, 2)^{\top }$.

• 13. Gaussian [14, Problem (9)]: $n=3$, $m=15$, nonconvex,$^3$

  $$\begin{aligned} F_{i}(x)&=x_1\exp \left( \frac{-x_2(t_i-x_3)^2}{2}\right) -y_i, \end{aligned}$$

  where $t_i=(8-i)/2$, $i=1,\ldots ,m$ and $y_i$ is given as

  i	1,15	2,14	3,13	4,12	5,11	6,10	7,9	8
  $y_i$	0.0009	0.0044	0.0175	0.0540	0.1295	0.2420	0.3521	0.3989

  $L=(-2, -2, -2)^{\top }$, and $U=(2, -2, 2)^{\top }$.

• 14. Brown and Dennis [14, Problem (16)]: $n=4$, $m=5$, nonconvex,$^3$

  $$\begin{aligned} F_i(x)&=\left( x_1+t_i x_2-e^{t_i}\right) ^2+\left( x_3+x_4\sin (t_i)-\cos (t_i)\right) ^2, \end{aligned}$$

  where $t_i=i/5$, $L=(-25, -5, -5, -1)^{\top }$, and $U=(25, 5, 5, 1)^{\top }$.

• 15. Trigonometric [14, Problem (26)]: $n=4$, $m=4$, nonconvex,$^{23}$

  $$\begin{aligned} F_i(x)&=\left( n-\sum _{j=1}^{n}\cos x_j+i\left( 1-\cos x_i\right) -\sin x_i\right) ^2, \quad i=1,\ldots ,4, \end{aligned}$$

  $L=(-1,\ldots , -1)^{\top }$, and $U=(1,\ldots , 1)^{\top }$.

• 16. Linear function – rank 1 [14, Problem (33)]: $n=10$, $m=4$, convex,$^{23}$

  $$\begin{aligned} F_i(x)&=\left( i\left( \sum _{j=1}^{n}jx_j\right) -1\right) ^2, \quad i=1,\ldots ,4, \end{aligned}$$

  $L=(-1,\ldots , -1)^{\top }$, and $U=(1,\ldots , 1)^{\top }$.