Continual variational dropout: a view of auxiliary local variables in continual learning

Abstract

Regularization/prior-based approach appears to be one of the critical strategies in continual learning, considering its mechanism for preserving and preventing forgetting the learned knowledge. Without any retraining on previous data or extending the network architecture, the mechanism works by setting a constraint on the important weights of previous tasks when learning the current task. Regularization/prior approach, on the other hand, suffers the challenge of weights being moved intensely to the parameter region, in which the model achieves good performance for the latest task but poor ones for earlier tasks. To that end, we suggest a novel solution to this problem by continually applying variational dropout (CVD), thereby generating task-specific local variables that work as modifying factors for the global variables to fit the task. In particular, as we impose a variational distribution on the auxiliary local variables employed as multiplicative noise to the layers’ input, the model enables the global variables to be retained in a good region for all tasks and reduces the forgetting phenomenon. Furthermore, we obtained theoretical properties that are currently unavailable in existing methods: (1) uncorrelated likelihoods between different data instances reduce the high variance of stochastic gradient variational Bayes; (2) correlated pre-activation improves the representation ability for each task; and (3) data-dependent regularization assures the global variables to be preserved in a good region for all tasks. Throughout our extensive results, adding the local variables shows its significant advantage in enhancing the performance of regularization/prior-based methods by considerable magnitudes on numerous datasets. Specifically, it brings several standard baselines closer to state-of-the-art results.

Appendices

Appendix A: Auxiliary local variables for uncertainty regularized continual learning (UCL)

In this section, we review UCL (Ahn et al., 2019) which is one of the state-of-the-art methods for continual learning and how to apply CVD to this method. UCL uses the same the likelihood term of VCL, but reinterprets the KL term of VCL to improve this term. The KL term is rewritten as follows:

$$\begin{aligned} \frac{1}{2} \sum _{l=1}^L \left[ \left\| \frac{\varvec{\mu }_t^{(l)} - \varvec{\mu }_{t-1}^{(l)}}{\varvec{\sigma }_{t-1}^{(l)}}\right\| _2^2 + {\textbf{1}}^\intercal \left\{ \left( \frac{\varvec{\sigma }_{t}^{(l)}}{\varvec{\sigma }_{t-1}^{(l)}} \right) ^2 - \log \left( \frac{\varvec{\sigma }_{t}^{(l)}}{\varvec{\sigma }_{t-1}^{(l)}} \right) ^2 \right\} \right] \end{aligned}$$

(A1)

where l is the layer index of the neural network. UCL improves VCL by defining node importance and then adding two regularization terms. Based on the node importance, the first term limits the change of weights related to important nodes and the other makes weights more active to learn new tasks. In detail, UCL constrains the standard deviation of all weights connecting to the same node u in layer l to have the same value $\varvec{\sigma }_{u}^{(l)}$ and then uses this parameter to measure node importance. Moreover, it modifies this KL term to freeze the weights related to important nodes:

$$\begin{aligned} KL&= \sum _{l=1}^{L}\Big [\Big ( \frac{1}{2}\Big \Vert \mathbf {\Lambda }^{(l)}\odot (\varvec{\mu }_{t}^{(l)}-\varvec{\mu }_{t-1}^{(l)})\Big \Vert _2^2 \nonumber \\&+ (\varvec{\sigma }_{\text {init}}^{(l)})^2 \Big \Vert \Big (\frac{\varvec{\mu }_{t-1}^{(l)}}{\varvec{\sigma }_{t-1}^{(l)}}\Big )^{2}\odot (\varvec{\mu }_{t}^{(l)}-\varvec{\mu }_{t-1}^{(l)}) \Big \Vert _1\Big )\nonumber \\&+ \frac{\beta }{2}{\textbf{1}}^\top \Big \{\Big (\frac{\varvec{\sigma }_t^{(l)}}{\varvec{\sigma }_{t-1}^{(l)}}\Big )^2-\log \Big (\frac{\varvec{\sigma }_t^{(l)}}{\varvec{\sigma }_{t-1}^{(l)}}\Big )^2 + (\varvec{\sigma }_t^{(l)})^2-\log (\varvec{\sigma }_t^{(l)})^2\Big \}\Big ] \end{aligned}$$

(A2)

where $\varvec{\sigma }^{(l)}_{\text {init}}$ is the initial standard deviation hyperparameter for all weights on the l-th layer. The matrix $\mathbf {\Lambda }^{(l)}_{uv}\triangleq \max \Big \{\frac{\varvec{\sigma }_{\text {init}}^{(l)}}{\varvec{\sigma }_{t-1,u}^{(l)}}, \frac{\varvec{\sigma }_{\text {init}}^{(l-1)}}{\varvec{\sigma }_{t-1,v}^{(l-1)}}\Big \}$ defines the regularization strength for the weight $\varvec{\mu }_{t,uv}^{(l)}$.

CVD for UCL

We also add auxiliary variables to the original model and then maximize the log likelihood: $\log p({\textbf{Y}}_t \vert {\textbf{X}}_t) = \sum _{i=1}^{N_t} \log p({\textbf{y}}_t^{(i)} \vert {\textbf{x}}_t^{(i)})$. Due to the intractability of the likelihood, we use mean-field variational inference with variational distributions $q_t({\varvec{\theta }}), q_t({{\textbf{s}}})$:

$$\begin{aligned}&\sum _{i=1}^{N_t} \log \int _{\varvec{\theta }} \int _{{\textbf{s}}} p({\textbf{y}}_t^{(i)} \vert {\textbf{s}}, \varvec{\theta }, {\textbf{x}}_t^{(i)}) p({\textbf{s}}) p(\varvec{\theta }) d\varvec{\theta } d{\textbf{s}} \nonumber \\&\quad = \sum _{i=1}^{N^{(t)}} \log \int _{\varvec{\theta }} \int _{s} \frac{p(y_{i}^{(t)} \vert s, \varvec{\theta }, x_{i}^{(t)}) p(s) q_{t-1}(\varvec{\theta })}{q(s) q(\varvec{\theta })} q(\varvec{\theta }) q(s) d\varvec{\theta } ds \nonumber \\&\quad \ge \sum _{i=1}^{N_t} E_{q_t({\varvec{\theta }}), q_t({{\textbf{s}}})} \left[ \log p({\textbf{y}}_t^{(i)} \vert {\textbf{s}}, \varvec{\theta }, {\textbf{x}}_t^{(i)}) \right] \nonumber \\&\quad - KL(q_t({\textbf{s}}) \Vert p({\textbf{s}})) - KL(q_t(\varvec{\theta }) \Vert q_{t-1}(\varvec{\theta })) \end{aligned}$$

(A3)

Note that $KL(q_t(\varvec{\theta }) \Vert q_{t-1}(\varvec{\theta }))$ is a different point between CVD for VCL and UCL. While CVD for VCL uses the KL term in Eq. (A1), the KL term in CVD for UCL is expressed in Eq. (A2). The remaining terms are built as in CVD for VCL.

Appendix B: Architectures and settings

Split MNIST and permuted MNIST

For Split MNIST, we use a fully-connected neural network (FCNN) with two hidden layers and multi-head output layer. Table 8 shows the detail of the network for Split MNIST dataset. For Permuted MNIST dataset, we also use FCNN but single-head output layer, and the architecture is shown in Table 9.

Table 8 Network architecture for Split MNIST

Table 9 Network architecture for Permuted MNIST

We tune the hyperparameters of both CVD and the combined methods. Moreover, we again use the strategy of parameter initialization as in UCL’s experiments (Ahn et al., 2019) in the beginning step of training process. All the hyperparameters of methods are listed as below:

• UCL—$\beta :$ {0.0001; 0.001; 0.01; 0.02; 0.03}, $\alpha :$ {0.01; 0.3; 5}

• EWC—$\lambda :$ {40; 400; 4000; 10000; 40000}

• VCL—not needed

• GVCL—$\beta :$ {0.05; 0.1; 0.2}, $\lambda :$ {1; 10; 100; 1000}

• CVD—KL_weight $\kappa$: {0.0001; 0.001; 0.01; 0.1; 1}

Split CIFAR-10/100 and Split CIFAR-100:

The detail of architecture is used in CIFAR experiments is shown in Table 10 and all the hyperparameters are listed as below:

• UCL—$\beta :$ {0.0001; 0.0002; 0.001; 0.002}, $\alpha :$ {0.01; 0.3; 5}, r :  {0.5; 0.125}, $lr(\sigma ):$ {0.01; 0.02}

• EWC—$\lambda :$ {400; 1000; 4000; 10000; 25000; 40000}

• VCL—not needed

• GVCL—$\beta :$ {0.05; 0.1; 0.2}, $\lambda :$ {1; 10; 100; 1000}

• AGS-CL—$\lambda :$ {1.5; 100; 400; 1000; 7000; 10000}, $\mu :$ {0.5; 10; 20}, $\rho :$ {0.1; 0.2; 0.3; 0.4; 0.5}

• CVD—KL_weight $\kappa$: {0.0001; 0.001; 0.01; 0.1; 1}

Table 10 Network architecture for Split CIFAR-10/100

Split Omniglot:

The detail of architecture for Omniglot dataset is given in Table 11. Since, for each task, the number of classes is different, we denoted the classes of $i^{th}$ task as $C_i$. All the hyperparameters are listed as below:

• UCL—$\beta :$ {0.0001; 0.0002; 0.001; 0.002}, $\alpha :$ {0.01; 0.3; 5}, r :  {0.5; 0.125}, $lr(\sigma ):$ {0.01; 0.02}

• EWC—$\lambda :$ {4000; 10000; 25000; 40000; 100000}

• VCL—not needed

• GVCL—$\beta :$ {0.05; 0.1; 0.2}, $\lambda :$ {1; 10; 100; 1000}

• AGS-CL—$\lambda :$ {1.5; 100; 400; 1000; 7000; 10000}, $\mu :$ {0.5; 10; 20}, $\rho :$ {0.1; 0.2; 0.3; 0.4; 0.5}

• CVD—KL_weight $\kappa$: {0.0001; 0.001; 0.01; 0.1; 1}

Table 11 Network architecture for Split Omniglot

Split CUB-200:

We use AlexNet in this experiment, the detail of architecture is given in Table 12. All the hyperparameters are listed as below:

• GVCL—$\beta :$ {0.05; 0.1; 0.2}, $\lambda :$ {1; 10; 100; 1000}

• AGS-CL—$\lambda :$ {1.5; 100; 400; 1000; 7000; 10000}, $\mu :$ {0.5; 10; 20}, $\rho :$ {0.1; 0.2; 0.3; 0.4; 0.5}

• CVD-KL_weight $\kappa$: {0.0001; 0.001; 0.01; 0.1; 1}

Table 12 Network architecture for Split CUB-200

Split ImangeNet-R:

We freeze the pretrained ViT backbone and add 4 dense layers to build model in this experiment, the detail of the architecture is given in Table 13. All the hyperparameters are listed as below:

• GVCL—$\beta :$ {0.001; 0.05; 0.1; 0.2}, $\lambda :$ {1; 10; 50, 100; 1000}

• AGS-CL—$\lambda :$ {1.5; 50; 100; 400; 1000; 7000; 10000}, $\mu :$ {0.5; 10; 20}, $\rho :$ {0.1; 0.2; 0.3; 0.4; 0.5}

• CVD-KL_weight $\kappa$: {0.0001; 0.001; 0.01; 0.1; 1; 1.5}

Table 13 Network architecture for Split ImageNet-R

Appendix C: Supplement visualizations

Figures 15, 16, 17 and 18 are the supplement illustrations for the analysis in Sect. 4.2. Similarly, the charts show the test accuracy of a task corresponding to the trained model of the task on the horizontal axis. As can be seen from this extent, CVD allows AGS-CL, VCL, UCL, and GVCL to be more steady across the tasks, whereas the original performs substantially worse. Based on these findings, it can be concluded that CVD not only enhances performance in most tasks but also effectively minimizes the forgetting phenomena.

Fig. 15

The change of accuracy through tasks on VCL

Fig. 16

The change of accuracy through tasks on UCL

Fig. 17

The change of accuracy through tasks on AGS-CL

Fig. 18

The change of accuracy through tasks on GVCL