A Stochastic Subgradient Method for Distributionally Robust Non-convex and Non-smooth Learning

Abstract

We consider a distributionally robust formulation of stochastic optimization problems arising in statistical learning, where robustness is with respect to ambiguity in the underlying data distribution. Our formulation builds on risk-averse optimization techniques and the theory of coherent risk measures. It uses mean–semideviation risk for quantifying uncertainty, allowing us to compute solutions that are robust against perturbations in the population data distribution. We consider a broad class of generalized differentiable loss functions that can be non-convex and non-smooth, involving upward and downward cusps, and we develop an efficient stochastic subgradient method for distributionally robust problems with such functions. We prove that it converges to a point satisfying the optimality conditions. To our knowledge, this is the first method with rigorous convergence guarantees in the context of generalized differentiable non-convex and non-smooth distributionally robust stochastic optimization. Our method allows for the control of the desired level of robustness with little extra computational cost compared to population risk minimization with stochastic gradient methods. We also illustrate the performance of our algorithm on real datasets arising in convex and non-convex supervised learning problems.

Appendices

Appendix A: Generalized Differentiability of Functions

Norkin [42] introduced the following class of functions.

Definition A.1

A function $f:\mathbbm {R}^n\rightarrow \mathbbm {R}$ is differentiable in a generalized sense at a point $x\in \mathbbm {R}^n$, if an open set $U\subset \mathbbm {R}^n$ containing x, and a non-empty, convex, compact valued, and upper semicontinuous multifunction $\hat{\partial } f: U \rightrightarrows \mathbbm {R}^n$ exist, such that for all $y\in U$ and all $g \in \hat{\partial } f(y)$ the following equation is true:

$$\begin{aligned} f(y) = f(x) + \langle g(y), y-x \rangle + o(x,y,g), \end{aligned}$$

with

$$\begin{aligned} \lim _{y\rightarrow x} \sup _{g\in G(y)} \frac{o(x,y,g)}{\Vert y-x\Vert }=0. \end{aligned}$$

The set $\hat{\partial } f(y)$ is the generalized subdifferential of f at y. If a function is differentiable in a generalized sense at every $x \in \mathbbm {R}^n$ with the same generalized subdifferential mapping $\hat{\partial } f:\mathbbm {R}^n\rightrightarrows \mathbbm {R}^n$, we call it differentiable in a generalized sense.

A function $f:\mathbbm {R}^n\rightarrow \mathbbm {R}^m$ is differentiable in a generalized sense, if each of its component functions, $f_i:\mathbbm {R}^n\rightarrow \mathbbm {R}$, $i=1,\dots ,m$, has this property.

The class of such functions is contained in the set of locally Lipschitz functions and contains all subdifferentially regular functions [7], Whitney stratifiable Lipschitz functions [11], semismooth functions [39], and their compositions. The Clarke subdifferential $\partial \! f(x)$ is an inclusion-minimal generalized subdifferential, but the generalized sub-differential mapping $\hat{\partial } f(\cdot )$ is not uniquely defined in Definition A.1. However, if $f:\mathbbm {R}^n\rightarrow \mathbbm {R}$ is differentiable in a generalized sense, then for almost all $x\in \mathbbm {R}^n$ we have $\hat{\partial } f(x)=\{\nabla f(x)\}$.

Compositions of generalized differentiable functions are crucial in our analysis.

Theorem A.1

[40, Thm. 1.6] If $h:\mathbbm {R}^m \rightarrow \mathbbm {R}$ and $f_i:\mathbbm {R}^n\rightarrow \mathbbm {R}$, $i=1,\dots ,m$, are differentiable in a generalized sense, then the composition $\psi (x) = h\big ( f_1(x),\dots ,f_m(x)\big )$ is differentiable in a generalized sense, and at any point $x\in \mathbbm {R}^n$ we can define the generalized subdifferential of $\psi$ as follows:

$$\begin{aligned} \hat{\partial } \psi (x) = \text {conv} \big \{ g\in \mathbbm {R}^n: g = \begin{bmatrix} g_1&\cdots&g_m\end{bmatrix} g_0,\\ \text { with } g_0\in \hat{\partial }{h}\big (f_1(x),\dots ,f_m(x)\big ) \text { and } g_j\in \hat{\partial }{f_j}(x),\ j=1,\dots ,m\big \}. \end{aligned}$$

Even if we take $\hat{\partial }{h}(\cdot )=\partial h(\cdot )$ and $\hat{\partial }{f_j}(\cdot )=\partial \! f_j(\cdot )$, $j=1,\dots ,m$, we may obtain $\hat{\partial }\psi (\cdot ) \ne \partial \psi (\cdot )$, but $\hat{\partial }\psi$ defined above satisfies Definition A.1.

For stochastic optimization, essential is the closure of the class functions differentiable in a generalized sense with respect to expectation.

Theorem A.2

[40, Thm. 23.1] Suppose $(\varOmega ,\mathcal {F},P)$ is a probability space and a function $f:\mathbbm {R}^n\times \varOmega \rightarrow \mathbbm {R}$ is differentiable in a generalized sense with respect to x for all $\omega \in \varOmega$ and integrable with respect to $\omega$ for all $x\in \mathbbm {R}^n$. Let $\hat{\partial } f: \mathbbm {R}^n \times \varOmega \rightrightarrows \mathbbm {R}^n$ be a multifunction, which is measurable with respect to $\omega$ for all $x\in \mathbbm {R}^n$, and which is a generalized subdifferential mapping of $f(\cdot ,\omega )$ for all $\omega \in \varOmega$. If for every compact set $K\subset \mathbbm {R}^n$ an integrable function $L_K:\varOmega \rightarrow \mathbbm {R}$ exists, such that $\sup _{x\in K}\sup _{g\in \hat{\partial } f(x,\omega )}\Vert g\Vert \le L_K(\omega )$, $\omega \in \varOmega$, then the function

$$\begin{aligned} F(x) = \int _\varOmega f(x,\omega )\;P(d\omega ),\quad x\in \mathbbm {R}^n, \end{aligned}$$

is differentiable in a generalized sense, and the multifunction

$$\begin{aligned} \hat{\partial } F(x) = \int _\varOmega \hat{\partial } f(x,\omega )\;P(d\omega ),\quad x\in \mathbbm {R}^n, \end{aligned}$$

is its generalized subdifferential mapping.

A key step in the analysis of stochastic recursive algorithms by the differential inclusion method is the chain rule on a path (see [9] and the references therein). For an absolutely continuous function $p:[0,\infty )\rightarrow \mathbbm {R}^n$, we denote by $\overset{{{\;\,}_\bullet }}{p}(\cdot )$ its weak derivative: a measurable function such that

$$\begin{aligned} p(t) = p(0) + \int _0^t \overset{{{\;\,}_\bullet }}{p}(s)\;{\text {d}}s,\quad \forall \; t \ge 0. \end{aligned}$$

Theorem A.3

[49, Thm. 1] If a function $f:\mathbbm {R}^n \rightarrow \mathbbm {R}^m$ and a path $p:[0,\infty )\rightarrow \mathbbm {R}^n$ are differentiable in a generalized sense, then

$$\begin{aligned} f(p(T))- f(p(0)) = \int _0^T g(p(t)) \, \overset{{{\;\,}_\bullet }}{p}(t) \;{\text {d}}t, \end{aligned}$$

(42)

for all selections $g(\cdot ) \in \hat{\partial } f(\cdot )$, and all $T>0$.

Appendix B: Proof of Lemma 3.3

Proof

Formula (13) and assumptions (A4)(ii) and (iii) yield:

$$\begin{aligned} u^{k+1} = u^k + \tau _k \big [J^{k+1} \big (\bar{y}(x^k,z^k)-x^k\big ) + b \big ( h(x^{k+1})- u^k\big )\big ] + \tau _k \theta _u^{k+1} + \tau _k \epsilon _u^{k+1}, \end{aligned}$$

(43)

with the errors

$$\begin{aligned} \theta _u^{k+1} = E^{k+1}\big (\bar{y}(x^k,z^k)-x^k\big ) + b e_h^{k+1},\\ \epsilon _u^{k+1} = \Delta ^{k+1} \big (\bar{y}(x^k,z^k)-x^k\big ) + b \delta _h^{k+1}. \end{aligned}$$

Due to assumption (A4), for some constant $C_u^{\theta }$,

$$\begin{aligned} \mathbbm {E}\big [\theta _u^{k+1}\,\big |\, \mathcal {F}_k\big ]=0, \quad \mathbbm {E}\big [\Vert \theta _u^{k+1}\Vert ^2\,\big |\,\mathcal {F}_k\big ]\le C_u^{\theta }, \quad k=0,1,\dots \end{aligned}$$

(44)

and

$$\begin{aligned} \lim _{k\rightarrow \infty } \epsilon _u^{k+1} = 0 \quad \text {a.s.}. \end{aligned}$$

To verify the boundedness of $\{u^k\}$, we define the quantities

$$\begin{aligned} \tilde{u}^k = u^k + \sum _{j=k}^\infty \tau _j \theta _u^{j+1}. \end{aligned}$$

Owing to (A3) and (44), by virtue of the martingale convergence theorem, the series in the formula above is convergent a.s., and thus, $\tilde{u}^k - u^k \rightarrow 0$ a.s., when $k\rightarrow \infty$. We can now use (43) to establish the following recursive relation:

$$\begin{aligned} \begin{aligned} \tilde{u}^{k+1}&= (1-b\tau _k) \tilde{u}^k + b\tau _k\Big [ \frac{1}{b}J^{k+1} \big (\bar{y}(x^k,z^k)-x^k\big ) + h(x^{k+1})+ \frac{1}{b} \epsilon _u^{k+1} + (\tilde{u}^k -u^k)\Big ]. \end{aligned} \end{aligned}$$

By (A1), the sequences $\{J^k\}$ and $\{h(x^k)\}$ are bounded. Since $\tilde{u}^k - u^k \rightarrow 0$ and $\epsilon _u^{k}\rightarrow 0$ a.s., the elements in the brackets in the formula above constitute an almost surely bounded sequence. Consequently, the sequence $\{\tilde{u}^k\}$ of their convex combinations is almost surely bounded as well. The same is true for the sequence $\{{u}^k\}$, because $\tilde{u}^k - u^k \rightarrow 0$ a.s.

The boundedness of $\{z^k\}$ can be established in a similar way. We rewrite (12) as

$$\begin{aligned} z^{k+1} = z^k + a\tau _k \Big ({g}_x^{k+1}+\big [{J}^{\,k+1}\big ]^\top {g}_u^{k+1} - z^k\Big ) + a\tau _k \theta _z^{k+1} + a\tau _k \epsilon _z^{k+1}, \end{aligned}$$

(45)

with the errors

$$\begin{aligned} \theta _z^{k+1} = e_{gx}^{k+1} + \big [{J}^{\,k+1}\big ]^\top e_{gu}^{k+1} + \big [{E}^{\,k+1}\big ]^\top {g}_u^{k+1} +\big [{E}^{\,k+1}\big ]^\top e_{gu}^{k+1},\\ \epsilon _z^{k+1} = \delta _{gx}^{k+1} + \big [\tilde{J}^{\,k+1}\big ]^\top \delta _{gu}^{k+1} + \big [\Delta ^{\,k+1}\big ]^\top \tilde{g}_u^{k+1}. \end{aligned}$$

Due to assumption (A4) (note the statistical independence of ${E}^{\,k+1}$ and $e_{gu}^{k+1}$), for some constant $C_z^{\theta }$,

$$\begin{aligned} \mathbbm {E}\big [\theta _z^{k+1}\,\big |\, \mathcal {F}_k\big ]=0, \quad \mathbbm {E}\big [\Vert \theta _z^{k+1}\Vert ^2\,\big |\,\mathcal {F}_k\big ]\le C_z^{\theta }, \quad k=0,1,\dots \end{aligned}$$

and

$$\begin{aligned} \lim _{k\rightarrow \infty } \epsilon _z^{k+1} = 0 \quad \text {a.s.}. \end{aligned}$$

The remaining proof is the same as that for $\{u^k\}$, with relation (45) replacing (43). $\square$