A feasible proximal bundle algorithm with convexification for nonsmooth, nonconvex semi-infinite programming

Abstract

We propose a new numerical method for semi-infinite optimization problems whose objective function is a nonsmooth function. Existing numerical methods for solving semi-infinite programming (SIP) problems make strong assumptions on the structure of the objective function, e.g., differentiability, or are not guaranteed to furnish a feasible point on finite termination. In this paper, we propose a feasible proximal bundle method with convexification for solving this class of problems. The main idea is to derive upper bounding problems for the SIP by replacing the infinite number of constraints with a finite number of convex relaxation constraints, introduce improvement functions for the upper bounding problems, construct cutting plane models of the improvement functions, and reformulate the cutting plane models as quadratic programming problems and solve them. The convex relaxation constraints are constructed with ideas from the α BB method of global optimization. Under mild conditions, we showed that every accumulation point of the iterative sequence is an 𝜖-stationary point of the original SIP problem. Under slightly stronger assumptions, every accumulation point of the iterative sequence is a local solution of the original SIP problem. Preliminary computational results on solving nonconvex, nonsmooth constrained optimization problems and semi-infinite optimization problems are reported to demonstrate the good performance of the new algorithms.

Appendix: 1: Test problems

Problem 1

[23, 40] Dimension: n= 2, $f(x) = 8|{x_{1}^{2}}-x_{2}|+(1-x_{1})^{2}$, $g(x) = \max \limits \left \{\sqrt {2}x_{1},\ 2 x_{2}\right \}-1$, X = [− 2, 2]², x⁰ = (1, 1)^T.

Problem 2

[34] Dimension: n= 2, $f(x) = \max \limits \left \{{{x_{1}^{2}}}+{{x_{2}^{4}}},\ (2-x_{1})^{2}+(2-x_{2})^{2},\ 2\exp (x_{2}-x_{1})\right \}$, $g(x) = \max \limits \left \{-{{x_{1}^{4}}}-2{{x_{2}^{2}}}-1,\ 2 {{x_{1}^{3}}}-{{x_{2}^{2}}}-2.5\right \}$, X = [− 4, 4]², x⁰ = (2, 2)^T.

Problem 3

[34] Dimension: n= 2, $f(x) = \max \limits \left \{{{x_{1}^{2}}}+{{x_{2}^{2}}}+x_{1} x_{2},\ -{{x_{1}^{2}}}-{{x_{2}^{2}}}-x_{1} x_{2},\ \sin \limits {x_{1}},\! -\sin \limits {x_{1}},\\ \cos \limits {x_{2}},\ -\cos \limits {x_{2}}\right \}$, $g(x) = \max \limits \left \{-{{x_{1}^{4}}}-2{{x_{2}^{2}}}-1,\ 2 {{x_{1}^{3}}}-{{x_{2}^{2}}}-2.5\right \}$, X = [− 4, 4]², x⁰ = (3, 1)^T.

Problem 4

[34] Dimension: n= 2, $f(x) = \max \limits \left \{{{x_{1}^{4}}}+{{x_{2}^{2}}},\ (2-x_{1})^{2}+(2-x_{2})^{2},\ 2\exp (x_{2}-x_{1})\right \}$, $g(x) = \max \limits \left \{{{x_{1}^{2}}}-{{x_{2}^{2}}},\ -2{{x_{1}^{3}}}-{{x_{2}^{2}}}\right \}$, X = [− 4, 4]², x⁰ = (0, 1)^T.

Problem 5

[34] Dimension: n= 2, $f(x) = \max \limits \left \{{{x_{1}^{2}}}+{{x_{2}^{2}}},\ (2-x_{1})^{2}+(2-x_{2})^{2},\ 2\exp (x_{2}-x_{1})\right \}$, $g(x) = \max \limits \left \{{x_{1}}+{x_{2}}-2,\ -{{x_{1}^{2}}}-{{x_{2}^{2}}}+2.25\right \}$, X = [− 4, 4]², x⁰ = (2.1, 1.9)^T.

Problem 6

[34] Dimension: n= 2, $f(x) = \max \limits \left \{10(x_{2}-{x_{1}^{2}}),\ 10({x_{1}^{2}}-x_{2}),\ 1-x_{1},\ x_{1}-1\right \}$, $g(x) = \max \limits \left \{100{x_{1}^{2}}+{x_{2}^{2}}-101,\ 80{x_{1}^{2}}-{x_{2}^{2}}-79\right \}$, X = [− 4, 4]², x⁰ = (− 1.2, 1)^T.

Problem 7

[19] Dimension: n= 20, 50, 100, 200, $f(x) =\max \limits \left \{{\sum }_{i=1}^{n-1}({x_{i}^{2}}+(x_{i+1}-1)^{2}+x_{i+1}-1)\right .$, $~~~~~~~~~~~~~\left .{\sum }_{i=1}^{n-1}(-{x_{i}^{2}}-(x_{i+1}-1)^{2}+x_{i+1}+1)\right \}$, $g(x) = {\sum }_{i=1}^{n-1}({x_{i}^{2}}+x_{i+1}^{2}+x_{i} x_{i+1}-2x_{i}-2x_{i+1}+1.0)$, X = [− 10, 10]ⁿ, x⁰ = ones(n, 1).

Problem 8

[19, 40] Dimension: n= 10, 50, 100, 200, 500, 1000, $f(x) = {\sum }_{i=1}^{n-1}(-x_{i}+2({x_{i}^{2}}+x_{i+1}^{2}-1)+1.75|{x_{i}^{2}}+x_{i+1}^{2}-1|)$, $g(x) = {\sum }_{i=1}^{n-2}((3-2 x_{i+1})x_{i+1}-x_{i}-2x_{i+2}+2.5)$, X = [− 10, 10]ⁿ, x⁰ = ones(n, 1).

Problem 9

[41] Dimension: n = 2, p = 1, $f(x) = \frac {1}{3}{x_{1}^{2}}+{x_{2}^{2}}+\frac {1}{2}x_{1}$, $g(x,t) = (1-{x_{1}^{2}} t^{2})^{2}-x_{1} t^{2}-{x_{2}^{2}}+x_{2}$, X = [− 2, 2]², T = [0, 1], x⁰ = (− 1,− 1)^T.

Problem 10

[28] Dimension: n = 3, p = 1, $f(x) = \exp (x_{1})+\exp (x_{2})+\exp (x_{3})$, g(x,t) = 1/(1 + t²) − x₁ − x₂t − x₃t², X = [− 2, 2]³, T = [0, 1], x⁰ = (1, 1, 1)^T.

Problem 11

[28] Dimension: n = 3, p = 1, f(x) = x₁ + x₂/2 + x₃/3, $g(x,t) = \exp (t-1)-x_{1} -x_{2} t-x_{3} t^{2}$, X = [− 2, 2]³, T = [0, 1], x⁰ = (1, 1, 1)^T.

Problem 12

[41] Dimension: n = 3, p = 1, $f(x) = {x_{1}^{2}}+{x_{2}^{2}}+{x_{3}^{2}}$, $g(x,t) = x_{1} +x_{2}\exp (x_{3} t)+\exp (2t)-2\sin \limits (4t)$, X = [− 4, 2]³, T = [0, 1], x⁰ = (1, 1, 1)^T.

Problem 13

[31, 32] Dimension: n = 3, p = 2, f(x) = |x₁| + |x₂| + |x₃|, $g(x,t) = x_{1}+x_{2}\exp (x_{3} t)-\exp (2x_{1} t)+\sin \limits (4 t)$, X = [− 1, 1]³, T = [0, 1], x⁰ = ones(3, 1).

Problem 14

[31, 32] Dimension: n = 3, p = 2, f(x) = |x₁| + |x₂| + |x₃|, $g(x,t) = x_{1}+x_{2}\exp (x_{3} t_{1})-\exp (2 t_{2})+\sin \limits (4 t_{1})$, X = [− 1, 1]³, T = [0, 1]², x⁰ = ones(3, 1).

Problem 15

[31, 32] Dimension: n = 4, p = 2, f(x) = 1/2(|x₁| + |x₂| + |x₃| + |x₄|), $g(x,t) = \sin \limits (t_{1} t_{2})-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} t_{1} t_{2}$, X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(4, 1).

Problem 16

Dimension: n = 6, p = 2, $f(x) = {\sum }_{i=1}^{n-1}(-x_{i}+2({x_{i}^{2}}+x_{i+1}^{2}-1)+1.75|{x_{i}^{2}}+x_{i+1}^{2}-1|)$, $g(x,t) = \sin \limits (t_{1} t_{2})-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$, X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 17

Dimension: n = 6, p = 2, $f(x) = {\sum }_{i=1}^{n-1}(-x_{i}+2({x_{i}^{2}}+x_{i+1}^{2}-1)+1.75|{x_{i}^{2}}+x_{i+1}^{2}-1|)$, $g(x,t) = (1+{t_{1}^{2}}+{t_{2}^{2}})^{2}-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$, X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 18

Dimension: n = 6, p = 2, $f(x) = {\sum }_{i=1}^{n-1}\max \limits \left \{{x_{i}^{2}}+(x_{i+1}-1)^{2}+x_{i+1}-1,\ -{x_{i}^{2}}-(x_{i+1}-1)^{2}+x_{i+1}+1\right \}$, $g(x,t) = \exp ({t_{1}^{2}}+{t_{2}^{2}})-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$, X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 19

Dimension: n = 6, p = 2, $f(x) = {\sum }_{i=1}^{n-1}\max \limits \left \{{x_{i}^{2}}+(x_{i+1}-1)^{2}+x_{i+1}-1,\ -{x_{i}^{2}}-(x_{i+1}-1)^{2}+x_{i+1}+1\right \}$, $g(x,t) = \sin \limits (t_{1} t_{2})-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$, X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 20

Dimension: n = 6, p = 2, $f(x) = {\sum }_{i=1}^{n-1}\max \limits \left \{{x_{i}^{2}}+(x_{i+1}-1)^{2}+x_{i+1}-1,\ -{x_{i}^{2}}-(x_{i+1}-1)^{2}+x_{i+1}+1\right \}$, $g(x,t) = (1+{t_{1}^{2}}+{t_{2}^{2}})^{2}-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$, X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 21

Dimension: n = 6, p = 2,

$f(x)\! =\! \max \limits \left \{{\sum }_{i=1}^{n-1}({x_{i}^{2}}+(x_{i+1}-1)^{2}+x_{i+1}-1)\right .$, $~~~~~~~~~~~~~~~~~\left .{\sum }_{i=1}^{n-1}(-{x_{i}^{2}}-(x_{i+1}-1)^{2}+x_{i+1}+1)\right \}$,

$g(x,t) = \sin \limits (t_{1} t_{2})-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$,

X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 22

Dimension: n = 6, p = 2, $f(x) =\max \limits \left \{\sum \limits _{i=1}^{n-1}({x_{i}^{2}}+(x_{i+1}-1)^{2}+x_{i+1}-1)\right .$, $~~~~~~~~~~~~~\left .\sum \limits _{i=1}^{n-1}(-{x_{i}^{2}}-(x_{i+1}-1)^{2}+x_{i+1}+1)\right \}$,

$g(x,t) = (1+{t_{1}^{2}}+{t_{2}^{2}})^{2}-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$,

X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).

Problem 23

Dimension: n = 6, p = 2,

$f(x) = \max \limits _{1\leq i\leq n }\left \{h(-{\sum }_{i=1}^{n}x_{i}),\ h(x_{i})\right \}$, where $h(y)=\ln (|y|+1),\ \forall y\in \mathbb {R}$,

$g(x,t) = \sin \limits (t_{1} t_{2})-x_{1}-x_{2} t_{1}-x_{3} t_{2}-x_{4} {t_{1}^{2}}-x_{5} t_{1} t_{2}-x_{6} {t_{2}^{2}}$,

X = [− 4, 4]⁶, T = [0, 1]², x⁰ = ones(6, 1).