PRIAG: Proximal Reweighted Incremental Aggregated Gradient Algorithm for Distributed Optimizations

Abstract

Large-scale machine learning problems are nowadays tackled by distributed optimization algorithms, i.e., algorithms that leverage multiple workers for training. However, collecting the information from all workers in every iteration is sometimes expensive or even prohibitive. In this paper, we propose an iterative algorithm called proximal reweighted incremental aggregated gradient (PRIAG) for solving a class of nonconvex and nonsmooth problems, which are ubiquitous in machine learning tasks and distributed optimization problems. In each iteration, this algorithm just needs the information from one worker due to the incremental aggregated method. Combined with the reweighted technique, we only require an easy-to-calculate proximal operator to deal with the nonconvex and nonsmooth properties. Using the Lyapunov function analysis method, we prove that the PRIAG algorithm is convergent under some mild assumptions. We apply this approach to nonconvex nonsmooth problems and distributed optimization tasks. Numerical experiments on both synthetic and real data sets show that our algorithm can achieve comparative learning performance, but more efficiently, compared with previous nonconvex solvers.

Appendices

Appendix

Proof of Lemma 1

Note that $x^{k+1}$ is obtained by (7), combining with the Definition 1 and 2 we have

$$\begin{aligned} \frac{x^{k}-x^{k+1}}{\gamma }-v^{k} \in \partial w^k \cdot \tilde{g} \left( x^{k+1}\right) . \end{aligned}$$

(26)

With the convexity of g, we have

$$\begin{aligned} w^k \cdot (\tilde{g} \left( x^{k+1}\right) -\tilde{g} \left( x^{k}\right) )\le \left\langle \frac{x^{k}-x^{k+1}}{\gamma }-v^{k}, x^{k+1}-x^k \right\rangle . \end{aligned}$$

(27)

The assumption (b) implies that the sum function F is differentiable with L-continues gradient, i.e.,

$$\begin{aligned} \Vert \nabla F(w)-\nabla F(\overline{w})\Vert _{2} \le L \Vert w-\overline{w}\Vert _{2}, \end{aligned}$$

(28)

where $L=\sum _{i=1}^{m}L_i$. We can further get

$$\begin{aligned} F \left( x^{k+1}\right) -F \left( x^{k}\right) \le \left\langle \nabla F \left( x^{k}\right) , x^{k+1}-x^k \right\rangle + \frac{L}{2} \left\| x^{k+1}-x^k\right\| _{2}^{2}. \end{aligned}$$

(29)

Then we have

$$\begin{aligned} \begin{aligned}&\varPhi (x^{k+1})-\varPhi (x^{k})=F \left( x^{k+1}\right) -F \left( x^{k}\right) +\sum _{j=1}^{n} h \left( g\left( x_{j}^{k+1}\right) \right) -h \left( g\left( x_{j}^{k}\right) \right) \\&\le \left\langle \nabla F\left( x^{k}\right) , x^{k+1}-x^k \right\rangle +\frac{L}{2}\left\| x^{k+1}-x^k\right\| _{2}^{2}+\sum _{j=1}^{n} h \left( g\left( x_{j}^{k+1}\right) \right) -h\left( g\left( x_{j}^{k}\right) \right) \\&\le \left\langle \nabla F \left( x^{k}\right) , x^{k+1}-x^k \right\rangle +\frac{L}{2}\left\| x^{k+1}-x^k\right\| _{2}^{2}+\sum _{j=1}^{n} w_j^k \left( g\left( x_{j}^{k+1}\right) -g\left( x_{j}^{k} \right) \right) \\&= \left\langle \nabla F \left( x^{k}\right) , x^{k+1}-x^k \right\rangle +\frac{L}{2}\left\| x^{k+1}-x^k\right\| _{2}^{2}+w^k\cdot \left( \tilde{g} \left( x^{k+1}\right) -\tilde{g} \left( x^{k}\right) \right) \\&\le \left\langle \nabla F \left( x^{k}\right) , x^{k+1}-x^k \right\rangle + \frac{L}{2} \left\| x^{k+1}-x^k\right\| _{2}^{2}+\left\langle \frac{x^{k+1}-x^k}{\gamma }+v^{k},-x^{k+1}-x^k \right\rangle \\&=\underbrace{\left\langle \nabla F \left( x^{k}\right) -v^{k}, x^{k+1}-x^k \right\rangle }_{I} + \left( \frac{L}{2}-\frac{1}{\gamma } \right) \left\| x^{k+1}-x^k\right\| _{2}^{2}, \end{aligned} \end{aligned}$$

(30)

where $w_j^k:=(h^{\prime }(g(x_j^k))$ and $\tilde{g}=(g(x_1), g(x_2), \ldots , g(x_N))^{\top }$. The first inequality uses (29). The second inequality uses the concavity of h. The third inequality uses (27). In the following, we will give the bound of I. First, notice that $\text {max}_{i,k}\{\tau _{i,k}\}\le \tau$, then

$$\begin{aligned} \left\| x^{k}-x^{k-\tau _{i, k}}\right\| _2 \le \sum _{d=k-\tau _{i, k}}^{k-1} \left\| x^{d+1}-x^d\right\| _2 \le \sum _{d=k-\tau }^{k-1} \left\| x^{d+1}-x^d\right\| _2. \end{aligned}$$

(31)

Combining with (7), we have

$$\begin{aligned} \begin{aligned} I&=\left\langle \nabla F \left( x^{k}\right) -v^{k}, x^{k+1}-x^k \right\rangle \\&=\left\langle \sum _{i=1}^{m}\left( \nabla f_i \left( x^{k} \right) -\nabla f_i \left( x^{k-\tau _{i,k}} \right) \right) , x^{k+1}-x^k \right\rangle \\&\le \sum _{i=1}^{m} L_{i} \left\| x^{k}-x^{k-\tau _{i, k}}\right\| _2 \cdot \left\| x^{k+1}-x^k\right\| _2\\&\le \sum _{i=1}^{m} L_{i} \left( \sum _{d=k-\tau }^{k-1} \left\| x^{d+1}-x^d \right\| _2 \right) \cdot \left\| x^{k+1}-x^k \right\| _2\\&= L \sum _{d=k-\tau }^{k-1} \left\| x^{d+1}-x^d \right\| _2 \cdot \left\| x^{k+1}-x^k \right\| _2. \end{aligned} \end{aligned}$$

(32)

The first inequality uses the Lipschitz continuity of $\nabla f_{i}$. The second inequality uses (31). Meanwhile, for any $\xi >0$, we have the following Cauchy’s inequality

$$\begin{aligned} \left\| x^{d+1}-x^d\right\| _2 \cdot \left\| x^{k+1}-x^k\right\| _2 \le \frac{1}{2 \xi } \left\| x^{d+1}-x^d\right\| _{2}^{2}+\frac{\xi }{2}\left\| x^{k+1}-x^k \right\| _{2}^{2}. \end{aligned}$$

(33)

Then we have

$$\begin{aligned} I \le \frac{L}{2 \xi } \sum _{d=k-\tau }^{k-1} \left\| x^{d+1}-x^d\right\| _{2}^{2}+\frac{\tau \xi L}{2} \left\| x^{k+1}-x^k\right\| _{2}^{2}. \end{aligned}$$

(34)

Combining (30), (34), we have

$$\begin{aligned} \varPhi (x^{k+1})-\varPhi (x^{k}) \le \frac{L}{2 \xi } \sum _{d=k-\tau }^{k-1} \left\| x^{d+1}-x^d\right\| _{2}^{2}+\left[ \frac{(\tau \xi +1)L}{2}-\frac{1}{\gamma }\right] \left\| x^{k+1}-x^k\right\| _{2}^{2}. \end{aligned}$$

(35)

If $0<\gamma <\frac{2}{(2\tau +1)L}$, we can choose $\xi >0$, such that

$$\begin{aligned} \xi +\frac{1}{\xi }=1+\frac{1}{\tau }\left( \frac{1}{\gamma L}-\frac{1}{2}\right) . \end{aligned}$$

(36)

Then, with direct calculations and substitutions, we have

$$\begin{aligned} \begin{aligned}&\varGamma _{k}(\xi )-\varGamma _{k+1}(\xi ) \\&=\varPhi \left( x^{k}\right) -\varPhi \left( x^{k+1}\right) +\frac{L}{2 \xi }\sum _{d=k-\tau }^{k-1}(d-(k-\tau )+1)\left\| x^{d+1}-x^d \right\| _{2}^{2}\\&\ \ \ -\frac{L}{2\xi }\sum _{d=k+1-\tau }^{k}(d-(k-\tau ))\left\| x^{d+1}-x^d\right\| _{2}^{2}\\&= \varPhi \left( x^{k}\right) -\varPhi \left( x^{k+1}\right) +\frac{L}{2 \xi }\sum _{d=k-\tau }^{k-1}(d-(k-\tau )+1)\left\| x^{d+1}-x^d\right\| _{2}^{2}\\&\ \ \ -\frac{L}{2 \xi }\sum _{d=k-\tau }^{k-1}(d-(k-\tau ))\left\| x^{d+1}-x^d\right\| _{2}^{2}-\frac{L}{2 \xi } \tau \left\| x^{k+1}-x^k \right\| _{2}^{2}\\&= \varPhi \left( x^{k}\right) -\varPhi \left( x^{k+1}\right) +\frac{L}{2 \xi }\sum _{d=k-\tau }^{k-1} \left\| x^{d+1}-x^d\right\| _{2}^{2}-\frac{L}{2\xi } \tau \left\| x^{k+1}-x^k \right\| _{2}^{2}\\&\ge (\frac{1}{\gamma }-\frac{(\tau \xi +1)L}{2}-\frac{L}{2\xi }\tau ) \left\| x^{k+1}-x^k\right\| _{2}^{2}\\&=\frac{1}{4} \left( \frac{1}{\gamma }-\frac{L}{2}-\tau L\right) \left\| x^{k+1}-x^k\right\| _{2}^{2}. \end{aligned} \end{aligned}$$

(37)

The first inequality uses (35). The last equation uses (36). We then prove the first result. By summing the inequality (37), we have:

$$\begin{aligned} \sum _{k=1}^{\infty }\left\| x^{k+1}-x^k \right\| _{2}^{ 2 }<\infty . \end{aligned}$$

(38)

The second then obviously holds. Using [Lemma 3, [9]], we are then led to

$$\begin{aligned} \min _{1\le i\le k}\left\| x^{i+1}-x^i \right\| _{2}^2 =o\left( \frac{1}{k}\right) , \end{aligned}$$

(39)

which directly derives the third one.

Proof of Theorem 1

By the definition of subdifferential, we have

$$\begin{aligned} \frac{x^{k}-x^{k+1}}{\gamma }-v^{k} \in \partial w^k \tilde{g} \left( x^{k+1} \right) . \end{aligned}$$

(40)

That means

$$\begin{aligned} \begin{aligned} \frac{x^{k}-x^{k+1}}{\gamma }+\nabla F \left( x^{k+1}\right) -v^{k} \in \nabla F \left( x^{k+1}\right) +\partial H \left( x^{k+1}\right) =\partial \varPhi \left( x^{k+1}\right) , \end{aligned} \end{aligned}$$

(41)

where $H(x):=\sum _{j=1}^{n} h\left( g\left( x_{j}\right) \right)$. Thus we have

$$\begin{aligned} \begin{aligned} {\text {dist}}^2 \left( \mathbf {0}, \partial \varPhi \left( x^{k+1}\right) \right)&= \Vert \frac{x^{k}-x^{k+1}}{\gamma }+\nabla F \left( x^{k+1}\right) -v^k\Vert _2^2\\&\le \frac{2\left\| x^{k+1}-x^k \right\| _2^2}{\gamma ^2}+2L^2\tau \sum _{d=k-\tau }^{k} \left\| x^{d+1}-x^d\right\| _2^2. \end{aligned} \end{aligned}$$

(42)

Combining with Lemma 1,

$$\begin{aligned} \sum _{k}{\text {dist}}^2\left( \mathbf {0}, \partial \varPhi \left( x^{k+1}\right) \right) <+\infty . \end{aligned}$$

(43)

Still using [Lemma 3, [9]], the result can be proved.