Abstract
In continual learning, a primary factor of catastrophic forgetting is task-recency bias, which arises when a model is trained on an imbalanced set of new and old task instances. Recent studies have shown the effectiveness of rehearsal-based continual learning methods; however, a major drawback of these methods is the loss of accuracy on older tasks when training is biased towards newer tasks. To bridge this gap, we propose a \(\lambda \) Stability Wrapper (\(\lambda \)SW), where the learner uses a task-based policy to adjust the probability of when instances are replaced in memory to account for task-recency bias to alleviate catastrophic forgetting. The policy results in an increased number of instances seen from older tasks. By construction, \(\lambda \)SW can be applied with other rehearsal-based continual learning algorithms. We validate the effectiveness of \(\lambda \)SW with three well known baseline methods: Gradient-based Sample Selection, Experience Replay, and Maximally Interfered Retrieval. Our experimental results show significant gains in accuracy on eleven out of twelve of our experiments across four datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aljundi, R., et al.: Online continual learning with maximal interfered retrieval. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Aljundi, R., Lin, M., Goujaud, B., Bengio, Y.: Gradient based sample selection for online continual learning. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Bang, J., Kim, H., Yoo, Y., Ha, J.W., Choi, J.: Rainbow memory: continual learning with a memory of diverse samples. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8218–8227 (2021)
Belouadah, E., Popescu, A.: IL2M: class incremental learning with dual memory. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 583–592 (2019)
Caccia, L., Aljundi, R., Asadi, N., Tuytelaars, T., Pineau, J., Belilovsky, E.: New insights on reducing abrupt representation change in online continual learning. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=N8MaByOzUfb
Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11216, pp. 241–257. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01258-8_15
Chaudhry, A., et al.: On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486 (2019)
Chen, Z., Liu, B.: Lifelong machine learning. Synth. Lect. Artif. Intell. Mach. Learn. 12(3), 1–207 (2018)
Ebrahimi, S., Elhoseiny, M., Darrell, T., Rohrbach, M.: Uncertainty-guided continual learning in Bayesian neural networks, pp. 75–78 (2019)
Efraimidis, P.S., Spirakis, P.G.: Weighted random sampling with a reservoir. Inf. Process. Lett. 97(5), 181–185 (2006)
Grossberg, S.: How does a brain build a cognitive code? In: Grossberg, S. (ed.) Studies of Mind and Brain, pp. 1–52. Springer, Dordrecht (1982). https://doi.org/10.1007/978-94-009-7758-7_1
Hou, S., Pan, X., Loy, C.C., Wang, Z., Lin, D.: Learning a unified classifier incrementally via rebalancing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 831–839 (2019)
Jastrzebski, S., et al.: The break-even point on optimization trajectories of deep neural networks. In: International Conference on Learning Representations (2019)
Jin, X., Sadhu, A., Du, J., Ren, X.: Gradient-based editing of memory examples for online task-free continual learning. Adv. Neural. Inf. Process. Syst. 34, 29193–29205 (2021)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Citeseer (2009)
Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., Sanner, S.: Online continual learning in image classification: an empirical survey. Neurocomputing 469, 28–51 (2022)
Mallya, A., Lazebnik, S.: Packnet: adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018)
McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: the sequential learning problem. In: Psychology of Learning and Motivation, vol. 24, pp. 109–165. Elsevier (1989)
Nene, S.A., Nayar, S.K., Murase, H., et al.: Columbia object image library (coil-100). Citeseer (1996)
Pan, P., Swaroop, S., Immer, A., Eschenhagen, R., Turner, R., Khan, M.E.E.: Continual deep learning by functional regularisation of memorable past. Adv. Neural. Inf. Process. Syst. 33, 4453–4464 (2020)
Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1320–1328 (2017)
Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H.: ICARL: incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017)
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T., Wayne, G.: Experience replay for continual learning. In: Advances in Neural Information Processing Systems, pp. 350–360 (2019)
Sun, S., Calandriello, D., Hu, H., Li, A., Titsias, M.: Information-theoretic online memory selection for continual learning. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=IpctgL7khPp
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Vitter, J.S.: Random sampling with a reservoir. ACM Trans. Math. Softw. (TOMS) 11(1), 37–57 (1985)
Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021)
Zeng, G., Chen, Y., Cui, B., Yu, S.: Continual learning of context-dependent processing in neural networks. Nat. Mach. Intell. 1(8), 364–372 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wong, W., Koh, Y.S., Dobbie, G. (2023). Using Flexible Memories to Reduce Catastrophic Forgetting. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13936. Springer, Cham. https://doi.org/10.1007/978-3-031-33377-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-33377-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33376-7
Online ISBN: 978-3-031-33377-4
eBook Packages: Computer ScienceComputer Science (R0)