Using Flexible Memories to Reduce Catastrophic Forgetting

Abstract

In continual learning, a primary factor of catastrophic forgetting is task-recency bias, which arises when a model is trained on an imbalanced set of new and old task instances. Recent studies have shown the effectiveness of rehearsal-based continual learning methods; however, a major drawback of these methods is the loss of accuracy on older tasks when training is biased towards newer tasks. To bridge this gap, we propose a $\lambda$ Stability Wrapper ($\lambda$SW), where the learner uses a task-based policy to adjust the probability of when instances are replaced in memory to account for task-recency bias to alleviate catastrophic forgetting. The policy results in an increased number of instances seen from older tasks. By construction, $\lambda$SW can be applied with other rehearsal-based continual learning algorithms. We validate the effectiveness of $\lambda$SW with three well known baseline methods: Gradient-based Sample Selection, Experience Replay, and Maximally Interfered Retrieval. Our experimental results show significant gains in accuracy on eleven out of twelve of our experiments across four datasets.