A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes

Abstract

In this paper, we develop a version of the bundle method to solve unconstrained difference of convex (DC) programming problems. It is assumed that a DC representation of the objective function is available. Our main idea is to utilize subgradients of both the first and second components in the DC representation. This subgradient information is gathered from some neighborhood of the current iteration point and it is used to build separately an approximation for each component in the DC representation. By combining these approximations we obtain a new nonconvex cutting plane model of the original objective function, which takes into account explicitly both the convex and the concave behavior of the objective function. We design the proximal bundle method for DC programming based on this new approach and prove the convergence of the method to an $\varepsilon$-critical point. The algorithm is tested using some academic test problems and the preliminary numerical results have shown the good performance of the new bundle method. An interesting fact is that the new algorithm finds nearly always the global solution in our test problems.

Appendix: Test problems

All test problems are unconstrained DC optimization problems, where objective functions are presented as DC functions:

$$\begin{aligned} f({\varvec{x}}) = f_1({\varvec{x}}) - f_2({\varvec{x}}). \end{aligned}$$

Therefore, in the description of all test problems we present only functions $f_1$ and $f_2$. The following notations are used to describe test problems:

• ${\varvec{x}}_0 \in \mathbb {R}^n$ – starting point;

• ${\varvec{x}}^* \in \mathbb {R}^n$ – known best solution;

• $f^*$ – known best value of the objective function.

Problem 1

[6]

• Dimension: $n = 2$,

• Component functions:$f_1({\varvec{x}}) = \max \{f_1^1({\varvec{x}}), f_1^2({\varvec{x}}), f_1^3({\varvec{x}})\}+f_2^1({\varvec{x}})+f_2^2({\varvec{x}})+f_2^3({\varvec{x}}),$

• $~f_2({\varvec{x}}) = \max \{f_2^1({\varvec{x}})+f_2^2({\varvec{x}}), f_2^2({\varvec{x}})+f_2^3({\varvec{x}}), f_2^1({\varvec{x}})+ f_2^3({\varvec{x}})\},$

• $f_1^1({\varvec{x}}) = x_1^4 + x_2^2, ~f_1^2({\varvec{x}}) = (2-x_1)^2 + (2-x_2)^2, ~f_1^3({\varvec{x}}) = 2e^{-x_1+x_2},$

• $f_2^1({\varvec{x}}) = x_1^2 - 2x_1+x_2^2 - 4x_2 +4, ~f_2^2({\varvec{x}}) = 2x_1^2 - 5x_1+x_2^2 -2x_2 +4,$

• $f_2^3({\varvec{x}}) = x_1^2 + 2x_2^2-4x_2+1,$

• Starting point: ${\varvec{x}}_0=(2,2)^T$,

• Optimum point: ${\varvec{x}}^*=(1, 1)^T$,

• Optimum value: $f^*=2$.

Problem 2

[6]

• Dimension: $n = 2$,

• Component functions: $f_1({\varvec{x}}) = | x_1 - 1| + 200 \max \{0, | x_1| - x_2\},$

• $f_2({\varvec{x}}) = 100( | x_1| - x_2),$

• Starting point: ${\varvec{x}}_0=(-1.2, 1)^T$,

• Optimum point: ${\varvec{x}}^*=(1,1)^T$,

• Optimum value: $f^*=0$.

Problem 3

[6]

• Dimension: $n = 4$,

• Component functions: $f_1({\varvec{x}}) = | x_1 - 1| + 200 \max \{0, | x_1| - x_2\}$

• $~~~~~~~~ + 180 \max \{0, | x_3| - x_4\} + |x_3-1|+10.1 (|x_2-1| + |x_4-1|) + 4.95 |x_2+x_4-2|$,

• $f_2({\varvec{x}}) = 100( | x_1| - x_2) + 90 ( | x_3| - x_4) + 4.95|x_2-x_4|$,

• Starting point: ${\varvec{x}}_0=(1,3,3,1)^T$,

• Optimum point: ${\varvec{x}}^*=(1,1,1,1)^T$,

• Optimum value: $f^*=0$.

Problem 4

[6]

• Dimension: $n = 2, 5, 10, 50, 100, 150, 200, 250, 350, 500, 750,$

• Component functions: $f_1({\varvec{x}}) = n \max \left\{ |x_i|: ~i = 1,\ldots ,n \right\} ,\quad f_2({\varvec{x}}) = \sum _{i=1}^n |x_i|$,

• Starting point: ${\varvec{x}}_0 = (i, ~i = 1,\ldots ,\left\lfloor n/2\right\rfloor , ~-i, ~i = \left\lfloor n/2\right\rfloor +1,\ldots ,n)^T$,

• Optimum point: ${\varvec{x}}^*=(x_1^*, \ldots ,x_n^*)^T,$ $x_i^*=\alpha$ or $x_i^*=-\alpha$, $\alpha \in \mathbb {R}$, $i=1,\ldots ,n$,

• Optimum value: $f^*=0$.

Problem 5

[6]

• Dimension: $n = 2, 5, 10, 50, 100, 150, 200, 250, 300, 350, 400, 500, 1000,$

• 1500, 3000, 10 000, 15 000, 20 000, 50 000, 

• Component functions: $f_1({\varvec{x}}) = 20\, \mathrm{max} \left\{ \left| \sum _{i=1}^n(x_i - x_i^*)t_j^{i-1}\right| : j=1,\ldots ,20 \right\}$,

• $f_2({\varvec{x}}) = \sum _{j=1}^{20} \left| \sum _{i=1}^n(x_i - x_i^*)t_j^{i-1}\right| , ~t_j = 0.05j, ~j = 1,\ldots ,20$,

• Starting point: ${\varvec{x}}_0 = (0,\ldots ,0)^T$,

• Optimum point: ${\varvec{x}}^*=(1/n,\ldots ,1/n)^T$,

• Optimum value: $f^*=0$.

Problem 6

Problem 7

Problem 8

Problem 9

Problem 10