Stochastic Variance Reduction for DR-Submodular Maximization

Abstract

Stochastic optimization has experienced significant growth in recent decades, with the increasing prevalence of variance reduction techniques in stochastic optimization algorithms to enhance computational efficiency. In this paper, we introduce two projection-free stochastic approximation algorithms for maximizing diminishing return (DR) submodular functions over convex constraints, building upon the Stochastic Path Integrated Differential EstimatoR (SPIDER) and its variants. Firstly, we present a SPIDER Continuous Greedy (SPIDER-CG) algorithm for the monotone case that guarantees a $(1-e^{-1})\text {OPT}-\varepsilon$ approximation after $\mathcal {O}(\varepsilon ^{-1})$ iterations and $\mathcal {O}(\varepsilon ^{-2})$ stochastic gradient computations under the mean-squared smoothness assumption. For the non-monotone case, we develop a SPIDER Frank–Wolfe (SPIDER-FW) algorithm that guarantees a $\frac{1}{4}(1-\min _{x\in \mathcal {C}}{\Vert x\Vert _{\infty }})\text {OPT}-\varepsilon$ approximation with $\mathcal {O}(\varepsilon ^{-1})$ iterations and $\mathcal {O}(\varepsilon ^{-2})$ stochastic gradient estimates. To address the practical challenge associated with a large number of samples per iteration, we introduce a modified gradient estimator based on SPIDER, leading to a Hybrid SPIDER-FW (Hybrid SPIDER-CG) algorithm, which achieves the same approximation guarantee as SPIDER-FW (SPIDER-CG) algorithm with only $\mathcal {O}(1)$ samples per iteration. Numerical experiments on both simulated and real data demonstrate the efficiency of the proposed methods.

Appendices

Appendix A

1.1 A.1. The Proof of Lemma 3.2

Proof

We first prove the conclusion for the monotone case. From Lemma 3.1 and the updating rule, we have

$$\begin{aligned} \mathbb {E}\left[ \Vert \Phi _{k}\Vert ^{2}\right]\le & {} \frac{L^{2}}{\vert \mathcal {M}_{k}\vert }\frac{1}{K^{2}}\mathbb {E}\left[ \Vert v_{k}\Vert ^{2}\right] +\mathbb {E}\left[ \Vert \Phi _{k-1}\Vert ^{2}\right] \\\le & {} \frac{kL^{2}R^{2}}{K^{2}\vert \mathcal {M}_{k}\vert }+\mathbb {E}\left[ \Vert \Phi _{0}\Vert ^{2}\right] . \end{aligned}$$

Furthermore, for $k=0$ with $\vert \mathcal {M}_{0}\vert =\frac{\sigma ^{2}K^{2}}{L^{2}R^{2}}$, it is easy to obtain

$$\begin{aligned} \mathbb {E}\left[ \Vert \Phi _{0}\Vert ^{2}\right] \le \frac{\sigma ^{2}}{\vert \mathcal {M}_{0}\vert }=\frac{L^{2}R^{2}}{K^{2}}. \end{aligned}$$

Thus, for $k\ne 0$ with $\vert \mathcal {M}_{k}\vert =K$, we have

$$\begin{aligned} \mathbb {E}\left[ \Vert \Phi _{k}\Vert ^{2}\right] \le \frac{kL^{2}R^{2}}{\vert \mathcal {M}_{k}\vert K^{2}}+\frac{L^{2}R^{2}}{K^{2}} \le \left( 1+\frac{k}{K}\right) \frac{L^{2}R^{2}}{K^{2}}. \end{aligned}$$

It is analogous to derive the conclusion for the non-monotone case with $\alpha _{k}\le \frac{1}{K}$ and the boundedness of $\mathcal {C}$. $\square$

1.2 A.2. The Proof of Lemma 3.4

Proof

From the updating step in Algorithm 2 and $v_{k}\in [0,1]^{n}$, it implies that

$$\begin{aligned} x_{k+1}-x_{k}= & {} \frac{(b_{p_{k+1}}-b_{p_{k}})\sqrt{a_{p_{k}}}}{a_{p_{k+1}}}\left( v_{k}-x_{k}\right) \\\le & {} \frac{(b_{p_{k+1}}-b_{p_{k}})\sqrt{a_{p_{k}}}}{a_{p_{k+1}}}(\textbf{1}-x_{k}). \end{aligned}$$

Note that for $i\in [n]$, there is that,

$$\begin{aligned} 1-(x_{k+1})_{i}\ge & {} \left( 1-\frac{(b_{p_{k+1}}-b_{p_{k}})\sqrt{a_{p_{k}}}}{a_{p_{k+1}}}\right) \left( 1-(x_{k})_{i}\right) \\\ge & {} \left( 1-\frac{b_{p_{k+1}}-b_{p_{k}}}{1+p_{k+1}}\right) \left( 1-(x_{k})_{i}\right) \\\ge & {} \frac{1}{1+p_{k+1}}\left( 1-(x_{0})_{i}\right) , \end{aligned}$$

where $a_{p_{k}}=(1+p_{k})^{2}$ and $b_{p_{k}}=p_{k}$, and because of the initial point $x_{0}\in \arg \min _{x\in \mathcal {C}}\Vert x\Vert _{\infty }$, then,

$$\begin{aligned} (x_{k})_{i}\le 1-\frac{1}{1+p_{k}}(1-(x_{0})_{i})\le 1-\frac{1}{\sqrt{a_{p_{k}}}}\left( 1-\min _{x\in \mathcal {C}}\Vert x\Vert _{\infty }\right) , \quad i\in [n], \end{aligned}$$

which implies that

$$\begin{aligned} \Vert x_{k}\Vert _{\infty }\le 1-\frac{1-\min _{x\in \mathcal {C}}\Vert x\Vert _{\infty }}{\sqrt{a_{p_{k}}}}. \end{aligned}$$

Furthermore, the inequality (9) follows from Property 2. $\square$

Appendix B

1.1 B.1. The Proof of Theorem 4.1

Proof

By the definition of the potential function (4), Lemma 3.3 and the Cauchy–Schwarz inequality, we obtain

$$\begin{aligned}{} & {} E(p_{k+1})-E(p_{k})\\{} & {} \quad \ge -\frac{LR^{2}e^{p_{k+1}}}{2K^{2}}-(e^{p_{k+1}}-e^{p_{k}})\mathbb {E}\left[ \Vert \Phi _{k}\Vert \Vert x^{*}-v_{k}\Vert \right] \\{} & {} \quad \ge e^{p_{k+1}}\left( -\frac{LR^{2}}{2K^{2}}-2R\left( 1-e^{({p_{k}}-p_{k+1})}\right) \mathbb {E}\left[ \Vert \Phi _{k}\Vert \right] \right) \\{} & {} \quad \ge e^{p_{k+1}}\left( -\frac{LR^{2}}{2K^{2}}-\frac{2R}{K}\mathbb {E}\left[ \Vert \Phi _{k}\Vert \right] \right) \\{} & {} \quad \ge e^{p_{k+1}}\left( -\frac{LR^{2}}{2K^{2}}-\frac{2R}{K}\frac{\sqrt{\hat{\Theta }}}{\sqrt{k+1}}\right) , \end{aligned}$$

where the third inequality is from $p_{k}=\frac{k}{K}$, $1-e^{-\frac{1}{K}}\le \frac{1}{K}$ and the last one is because of the inequality (13) in Lemma 4.2. Telescoping the above inequality over $k\in \{0, \ldots , K-1\}$ yields

$$\begin{aligned} E(p_{K})-E(p_{0})= & {} e^{p_{K}}\left( \mathbb {E}\left[ F(x_{K})\right] -F(x^{*})\right) -e^{p_{0}}\left( \mathbb {E}\left[ F(x_{0})\right] -F(x^{*})\right) \\\ge & {} -\sum ^{K}_{k=1}e^{p_{k}}\left( \frac{LR^{2}}{2K^{2}}+\frac{2R}{K}\frac{\sqrt{\hat{\Theta }}}{\sqrt{k}}\right) \\\ge & {} -e^{p_{K}}\left( \frac{LR^{2}}{2K}+\frac{2R\sqrt{\hat{\Theta }}}{K}\cdot 2\sqrt{K}\right) , \end{aligned}$$

where $p_{K}\ge p_{k}, \forall k\le K$ and $\sum ^{K}_{k=1}k^{-\frac{1}{2}}\le \int ^{K}_{0}x^{-\frac{1}{2}}dx=2K^{\frac{1}{2}}$. Rearranging the inequality, we have

$$\begin{aligned} \mathbb {E}[F(x_{K})]\ge & {} F(x^{*})+\frac{e^{p_{0}}\left( \mathbb {E}\left[ F(x_{0})\right] -F(x^{*})\right) -e^{p_{K}}\left( \frac{LR^{2}}{2K}+\frac{2R\sqrt{\hat{\Theta }}}{K}\cdot 2\sqrt{K}\right) }{e^{p_{K}}}\\\ge & {} \left( 1-\frac{1}{e}\right) F(x^{*})-\frac{LR^{2}}{2K}-\frac{4R\sqrt{\hat{\Theta }}}{\sqrt{K}}, \end{aligned}$$

where $F(x_{0})\ge 0, e^{p_{0}}=1$ and $e^{p_{K}}=e$. $\square$

1.2 B.2. The Proof of Theorem 4.2

Proof

Note that the parameter $\alpha _{k}=\frac{(p_{k+1}-p_{k})(1+p_{k})}{(1+p_{k+1})^{2}}\le \frac{1}{K}$, which satisfies the condition of Lemma 4.2 for the non-monotone case. From the definition of the potential function (7) and Lemma 3.5, we have

$$\begin{aligned}{} & {} E(p_{k+1})-E(p_{k})\\{} & {} \quad \ge -\frac{LD^{2}{\alpha _{k}}^{2}a_{p_{k+1}}}{2}-(b_{p_{k+1}}-b_{p_{k}})\sqrt{a_{p_{k}}}\cdot \mathbb {E}\left[ \langle \nabla F(x_{k})-g_{k}, x^{*}-v_{k}\rangle \right] \\{} & {} \quad \ge -\frac{LD^{2}a_{p_{k}}(b_{p_{k+1}}-b_{p_{k}})^{2}}{2a_{p_{k+1}}}-(b_{p_{k+1}}-b_{p_{k}})\sqrt{a_{p_{k}}}\cdot \mathbb {E}\left[ \Vert \Phi _{k}\Vert \Vert x^{*}-v_{k}\Vert \right] \\{} & {} \quad \ge -\frac{LD^{2}}{2K^{2}}-\frac{2D}{K}\frac{\sqrt{\bar{\Theta }}}{\sqrt{k+1}}, \end{aligned}$$

where we use $\alpha _{k} =\frac{(b_{p_{k+1}}-b_{p_{k}})\sqrt{a_{p_{k}}}}{a_{p_{k+1}}}$ and the Cauchy–Schwarz inequality in the second inequality, and the last inequality is from $a_{p_{k+1}} \ge a_{p_{k}}, b_{p_{k}}=\frac{k}{K}, \sqrt{a_{p_{k}}}\le 2$, $\max _{x, y\in \mathcal {C}}\Vert x-y\Vert \le D$ and Lemma 4.2 for non-monotone case.

Summing up the above inequality over $k\in \{0, \ldots , K-1\}$ yields

$$\begin{aligned}{} & {} E(p_{K})-E(p_{0})\\{} & {} \quad =a_{p_{K}}\mathbb {E}\left[ F(x_{K})\right] -(b_{p_{K}}-b_{p_{0}})\left( 1-\min _{x\in \mathcal {C}}{\Vert x\Vert _{\infty }}\right) F(x^{*})-a_{p_{0}}\mathbb {E}\left[ F(x_{0})\right] \\{} & {} \quad \ge -\sum ^{K-1}_{k=0}\left( \frac{LD^{2}}{2K^{2}}+\frac{2D}{K}\frac{\sqrt{\bar{\Theta }}}{\sqrt{k+1}}\right) \\{} & {} \quad \ge -\frac{LR^{2}}{2K}-\frac{2D\sqrt{\bar{\Theta }}}{K}\cdot 2\sqrt{K}, \end{aligned}$$

where the last inequality comes from $\sum ^{K-1}_{k=1}k^{-\frac{1}{2}}\le \int ^{K}_{0}x^{-\frac{1}{2}}dx=2K^{\frac{1}{2}}$. Rearranging the inequality, we attain

$$\begin{aligned}{} & {} \mathbb {E}\left[ F(x_{K})\right] \\{} & {} \quad \ge \frac{(b_{p_{K}}-b_{p_{0}})\left( 1-\min _{x\in \mathcal {C}}{\Vert x\Vert _{\infty }}\right) F(x^{*})-a_{p_{0}}\mathbb {E}\left[ F(x_{0})\right] -\frac{LD^{2}}{2K}-\frac{4D\sqrt{\bar{\Theta }}}{\sqrt{K}}}{a_{p_{K}}}\\{} & {} \quad =\frac{1}{4}\left( 1-\min _{x\in \mathcal {C}}{\Vert x\Vert _{\infty }}\right) F(x^{*})-\frac{LD^{2}}{8K}-\frac{D\sqrt{\bar{\Theta }}}{\sqrt{K}}, \end{aligned}$$

where the equality follows from $b_{p_{K}}=1, b_{p_{0}}=0,a_{p_{K}}=4$ and the non-negativeness of F. $\square$