Variational Rectification Inference for Learning with Noisy Labels

Abstract

Label noise has been broadly observed in real-world datasets. To mitigate the negative impact of overfitting to label noise for deep models, effective strategies (e.g., re-weighting, or loss rectification) have been broadly applied in prevailing approaches, which have been generally learned under the meta-learning scenario. Despite the robustness of noise achieved by the probabilistic meta-learning models, they usually suffer from model collapse that degenerates generalization performance. In this paper, we propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions as an amortized variational inference problem and derive the evidence lower bound under the meta-learning framework. Specifically, VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable, which can rectify the loss of the noisy sample with the extra randomness regularization and is, therefore, more robust to label noise. To achieve the inference of the rectifying vector, we approximate its conditional posterior with an amortization meta-network. By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function, which can significantly improve the generalization performance. The elaborated meta-network and prior network adhere to the smoothness assumption, enabling the generation of reliable rectification vectors. Given a set of clean meta-data, VRI can be efficiently meta-learned within the bi-level optimization programming. Besides, theoretical analysis guarantees that the meta-network can be efficiently learned with our algorithm. Comprehensive comparison experiments and analyses validate its effectiveness for robust learning with noisy labels, particularly in the presence of open-set noise.

A Appendix

1.1 Derivations of The ELBO

For a singe observation $(\textbf{x}, \textbf{y} )$, the ELBO can be derived from the perspective of the KL divergence between the variational posterior $q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y})$ and the posterior $p(\textbf{v}| \textbf{x}, \textbf{y})$:

$$\begin{aligned} & D_{\textrm{KL}}[q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y}) || p(\textbf{v}| \textbf{x}, \textbf{y})] \nonumber \\ & = \mathbb {E}_{q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y})} \left[ \log q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y}) - \log p(\textbf{v}| \textbf{x}, \textbf{y})\right] \nonumber \\ & = \mathbb {E}_{q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y})} \left[ \log q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y}) - \log \frac{p(\textbf{v}| \textbf{x}, \textbf{y}) p(\textbf{x}, \textbf{y})}{p(\textbf{x}, \textbf{y})}\right] \nonumber \\ & = \log p(\textbf{y}| \textbf{x}) + \mathbb {E}_{q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y})} \big [\log q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y}) \nonumber \\ & \quad \quad \quad \quad \quad \quad \quad \quad - \log p(\textbf{y}| \textbf{x}, \textbf{v}) - \log p(\textbf{v}| \textbf{x}) \big ] \nonumber \\ & = \log p(\textbf{y}| \textbf{x}) - \mathbb {E}_{q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y})} \left[ \log p(\textbf{y}| \textbf{x}, \textbf{v})\right] \nonumber \\ & \quad \quad \quad \quad \quad \quad \quad \quad +D_{\textrm{KL}}[q_{\phi }(\textbf{v}| \textbf{x}, \textbf{y}) || p(\textbf{v}| \textbf{x})] \nonumber \\ & \ge 0. \end{aligned}$$

(A1)

Specifically, we apply Bayes’ rule to derive Eq. (A1) as

$$\begin{aligned} \begin{aligned} p&(\textbf{v}| \textbf{x}, \textbf{y}) = \frac{p(\textbf{v}| \textbf{x}, \textbf{y}) p(\textbf{x}, \textbf{y})}{p(\textbf{x}, \textbf{y})} \\&= \frac{p(\textbf{y} | \textbf{x},\textbf{v} ) p(\textbf{x},\textbf{v})}{p(\textbf{x}, \textbf{y})} = \frac{p(\textbf{y} | \textbf{x},\textbf{v} ) p(\textbf{v}|\textbf{x})}{p(\textbf{y}| \textbf{x})}. \end{aligned} \end{aligned}$$

(A2)

Therefore, the ELBO for the log-likelihood of the predictive distribution in Eq. (3) can be written as follows

(A3)

1.2 Proof

Lemma 1(Smoothness)

Proof

We begin with computation of the derivation of the meta loss $\widetilde{\mathcal {L}}^{emp}(\hat{\theta })$ w.r.t. the meta-network $\phi$. By using Eq. (9), we have

$$\begin{aligned} \begin{aligned} \frac{\partial \mathcal {L}^{meta}(\hat{\theta })}{\partial \phi }&= \frac{\partial \mathcal {L}^{meta}(\hat{\theta })}{\partial \hat{\theta }} \frac{\partial \hat{\theta }}{\partial V(\phi )} \frac{\partial V(\phi ) }{\partial \phi } \\&=\alpha \frac{\partial \mathcal {L}^{meta}(\hat{\theta })}{\partial \hat{\theta }} \left( \nabla _\theta L(\theta ) + \frac{\partial D_{\textrm{KL}}}{\partial V(\phi )} \right) \frac{\partial V(\phi ) }{\partial \phi }. \end{aligned}\nonumber \\ \end{aligned}$$

(A4)

To simplify the proof, we neglect Monte Carlo estimation in Eq. 6 and consider it as a deterministic rectified vector in the following. This would not affect the result since there ultimately exists a rectified vector for computing the expectation of those sampled losses. Taking the gradient of $\phi$ on both side of Eq. (A4),

(A5)

For the first term ❶ in the right hand, we can obtain the following inequality w.r.t. its norm

(A6)

since we assume $\Vert \frac{\partial ^2 \mathcal {L}^{meta}(\hat{\theta })}{\partial \hat{\theta }^2} \Vert \le \ell$, $\Vert \nabla _\theta L(\theta ) \Vert \le \tau$, $\Vert \frac{\partial D_{\textrm{KL}}}{\partial V(\phi )} \Vert \le o$, and $\Vert \frac{\partial V(\phi ) }{\partial \phi } \Vert \le \delta$.

For the second term ❷, we can also obtain

(A7)

with the assumption $\Vert \frac{\partial ^2 V(\phi ) }{\partial \phi ^2} \Vert \le \zeta$. Therefore, we have

$$\begin{aligned} \begin{aligned} \left\| \frac{\partial ^2 \mathcal {L}^{meta}(\hat{\theta })}{\partial \phi ^2} \right\| \le \alpha (\tau +o)\left( \ell \alpha \delta ^2(\tau +o) + \tau \zeta \right) . \end{aligned} \end{aligned}$$

(A8)

Let $\hat{\ell } = \alpha (\tau +o)\left( \ell \alpha \delta ^2(\tau +o) + \tau \zeta \right)$, we can conclude the proof that

$$\begin{aligned} \Vert \mathcal {L}^{meta}(\hat{\theta }(\phi ^{(t+1)})) - \mathcal {L}^{meta}(\hat{\theta }(\phi ^{(t)})) \Vert \le \hat{\ell } \Vert \phi ^{(t+1)} - \phi ^{(t)} \Vert .\nonumber \\ \end{aligned}$$

(A9)

$\square$

Theorem 1(Convergence Rate)

Proof

Consider

(A10)

For ❸, by Lipschitz smoothness of the meta loss function for $\theta$, we have

$$\begin{aligned} & \mathcal {L}^{meta}(\hat{\theta }^{(t+1)}(\phi ^{(t+1)}))- \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t+1)})) \nonumber \\ & \le \langle \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t+1)})), \hat{\theta }^{(t+1)}(\phi ^{(t+1)})-\hat{\theta }^{(t)}(\phi ^{(t+1)}) \rangle \nonumber \\ & \qquad +\frac{\ell }{2}\Vert \hat{\theta }^{(t+1)}(\phi ^{(t+1)})-\hat{\theta }^{(t)}(\phi ^{(t+1)})\Vert _2^2. \end{aligned}$$

(A11)

We firstly write $\hat{\theta }^{(t+1)}(\phi ^{(t+1)}), \hat{\theta }^{(t)}(\phi ^{(t+1)})$ with Eq. (9). Using Eq. (12), we obtain

$$\begin{aligned} \begin{aligned} \hat{\theta }^{(t+1)}&(\phi ^{(t+1)}) - \hat{\theta }^{(t)}(\phi ^{(t+1)}) \\ &= - \alpha \nabla _\theta \mathcal {L}^{emp}(\hat{\theta }^{(t+1)}(\phi ^{(t+1)})). \end{aligned} \end{aligned}$$

(A12)

and

$$\begin{aligned} \begin{aligned} \Vert \mathcal {L}^{meta}&(\hat{\theta }^{(t+1)}(\phi ^{(t+1)}))- \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t+1)})) \Vert \\&\le \alpha _t \tau ^2+ \frac{\ell \alpha _t^2}{2} \tau ^2 = \alpha _t\tau ^2 (1+\frac{\alpha _t \ell }{2}), \end{aligned} \end{aligned}$$

(A13)

since $\left\| \frac{\partial L(\theta )}{\partial \theta }\Big |_{\theta ^{(t)}}\right\| \le \tau$, $\left\| \!\frac{\partial L_i^{meta} (\hat{\theta })}{\partial \hat{\theta }}\!\Big |_{\hat{\theta }^{(t)}}\!\right\| \!\!\le \!\! \tau$, and the output of $V(\cdot )$ is bounded with the sigmoid function.

For ❹, since the gradient is computed from a mini-batch of training data that is drawn uniformly, we denote the bias of the stochastic gradient $\varepsilon ^{(t)} = \nabla \widetilde{\mathcal {L}}^{meta}\left( \hat{\theta }^{(t)}\left( \phi ^{(t)}\right) \right) - \nabla \mathcal {L}^{meta}\left( \hat{\theta }^{(t)}\left( \phi ^{(t)}\right) \right)$. We then observe its expectation obeys $\mathbb {E}[\varepsilon ^{(t)}] = 0$ and its variance obeys $\mathbb {E}[\Vert \varepsilon ^{(t)}\Vert _2^2 ] \le \sigma ^2$.

By smoothness of $\nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ))$ for $\phi$ in Lemma 1, we have

$$\begin{aligned} & \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t+1)}))-\mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)})) \nonumber \\ & \le \langle \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)})),\phi ^{(t+1)}-\phi ^{(t)} \rangle + \frac{\hat{\ell }}{2} \Vert \phi ^{(t+1)}-\phi ^{(t)}\Vert _2^2 \nonumber \\ & = \langle \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)})), -\eta _t [\nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))+\varepsilon ^{(t)} ] \rangle \nonumber \\ & \quad + \frac{\hat{\ell } \eta _t^2}{2} \Vert \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))+\varepsilon ^{(t)}\Vert _2^2 \nonumber \\ & = -(\eta _t-\frac{\hat{\ell } \eta _t^2}{2}) \Vert \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\Vert _2^2 + \frac{\widetilde{\ell } \eta _t^2}{2}\Vert \varepsilon ^{(t)}\Vert _2^2 \nonumber \\ & \quad - (\eta _t-\hat{\ell } \eta _t^2)\langle \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)})),\varepsilon ^{(t)}\rangle . \end{aligned}$$

(A14)

Thus, Eq.(A10) satisfies

$$\begin{aligned} & \mathcal {L}^{meta}(\hat{\theta }^{(t+1)}(\phi ^{(t+1)}))-\mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\nonumber \\ & \le \alpha _t\tau ^2 (1+\frac{\alpha _t \ell }{2}) -(\eta _t-\frac{\hat{\ell }\eta _t^2}{2}) \Vert \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\Vert _2^2 \nonumber \\ & \quad + \frac{\hat{\ell }\eta _t^2}{2}\Vert \varepsilon ^{(t)}\Vert _2^2 - (\eta _t-\hat{\ell } \eta _t^2)\langle \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)})),\varepsilon ^{(t)}\rangle .\nonumber \\ \end{aligned}$$

(A15)

We take the expectation w.r.t. $\varepsilon ^{(t)}$ over Eq. (A15) and sum up T inequalities. By the property of the bias $\varepsilon ^{(t)}$, we can obtain

$$\begin{aligned} \begin{aligned} \sum \limits _{t = 1}^T&\left( \mathop { \mathbb {E}}_{\varepsilon ^{(t)}} \mathcal {L}^{meta}(\hat{\theta }^{(t+1)}(\phi ^{(t+1)}))-\mathop { \mathbb {E}}_{\varepsilon ^{(t)}}\mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\right) \\&\le \tau ^2 \sum \limits _{t = 1}^T \alpha _t(1+\frac{\alpha _t \ell }{2}) \\&\quad - \sum \limits _{t = 1}^T (\eta _t-\frac{\hat{\ell }\eta _t^2}{2})\mathop { \mathbb {E}}_{\varepsilon ^{(t)}} \left[ \Vert \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\Vert _2^2\right] \\&\quad + \frac{\hat{\ell } \sigma ^2}{2} \sum \limits _{t = 1}^T \eta _t^2. \end{aligned} \end{aligned}$$

(A16)

Taking the total expectation and reordering the terms, we have

$$\begin{aligned} \begin{aligned}&\frac{1}{T}\sum \limits _{t = 1}^T (\eta _t-\frac{\hat{\ell }\eta _t^2}{2}) \mathop { \mathbb {E}} \left[ \Vert \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\Vert _2^2 \right] \\&\le \frac{\mathcal {L}^{meta}(\hat{\theta }^{(0)}(\phi ^{(0)})) - \mathop { \mathbb {E}}\left[ \mathcal {L}^{meta}(\hat{\theta }^{(T+1)}(\phi ^{(T+1)}) ) \right] }{T} \\&+ \frac{\tau ^2 }{T} \sum \limits _{t = 1}^T \alpha _t(1+\frac{\alpha _t \ell }{2}) + \frac{\hat{\ell } \sigma ^2}{2T} \sum \limits _{t = 1}^T \eta _t^2. \end{aligned} \end{aligned}$$

(A17)

Let

(A18)

With the assumption of $\eta _t =\min \{\frac{1}{\hat{\ell }},\frac{C}{\sigma \sqrt{T}}\}$ and $\alpha _t=\min \{1,\frac{\kappa }{T}\}$, we have $\eta _t-\frac{\hat{\ell }\eta _t^2}{2} \ge \eta _t - \frac{\eta _t}{2} = \frac{\eta _t}{2}$ and

$$\begin{aligned} \begin{aligned}&\frac{1}{T}\sum \limits _{t = 1}^T \mathop { \mathbb {E}} \left[ \Vert \nabla \mathcal {L}^{meta}(\hat{\theta }^{(t)}(\phi ^{(t)}))\Vert _2^2 \right] \\&\le \frac{2E}{T\eta _1} + \frac{(2+\ell )\tau ^2 \alpha _1 }{\eta _1} + \hat{\ell } \sigma ^2 \eta _1 \\&= \frac{2E}{T} \max \{\hat{\ell }, \frac{\sigma \sqrt{T}}{C} \} + (2+\ell ) \tau ^2 \min \{1,\frac{\kappa }{T}\} \max \{\hat{\ell }, \frac{\sigma \sqrt{T}}{C} \} \\&\quad + \hat{\ell } \sigma ^2 \min \{\frac{1}{\hat{\ell }},\frac{C}{\sigma \sqrt{T}}\} \\&\le \frac{2\sigma E}{C\sqrt{T}} + \frac{ (2+\ell )\tau ^2 \kappa \sigma }{C \sqrt{T} } + \frac{C\hat{\ell } \sigma ^2}{\sigma \sqrt{T}} = \mathcal {O}(\frac{1}{\sqrt{T}}). r \end{aligned}\nonumber \\ \end{aligned}$$

(A19)

Thus, we conclude our proof. $\square$

1.3 Algorithm for VRI Without the Meta Set

Algorithm 2

Learning without meta data