Abstract
Continual learning (CL) addresses the problem of catastrophic forgetting in neural networks, which occurs when a trained model tends to overwrite previously learned information, when presented with a new task. CL aims to instill the lifelong learning characteristic of humans in intelligent systems, making them capable of learning continuously while retaining what was already learned. Current CL problems involve either learning new domains (domain-incremental) or new and previously unseen classes (class-incremental). However, general learning processes are not just limited to learning information, but also refinement of existing information. In this paper, we define CLEO – Continual Learning of Evolving Ontologies, as a new incremental learning setting under CL to tackle evolving classes. CLEO is motivated by the need for intelligent systems to adapt to real-world ontologies that change over time, such as those in autonomous driving. We use Cityscapes, PASCAL VOC, and Mapillary Vistas to define the task settings and demonstrate the applicability of CLEO. We highlight the shortcomings of existing CIL methods in adapting to CLEO and propose a baseline solution, called Modelling Ontologies (MoOn). CLEO is a promising new approach to CL that addresses the challenge of evolving ontologies in real-world applications. MoOn surpasses previous CL approaches in the context of CLEO.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Asghar, N., Mou, L., Selby, K.A., Pantasdo, K.D., Poupart, P., Jiang, X.: Progressive memory banks for incremental domain adaptation. arXiv (2018)
Bang, J., Kim, H., Yoo, Y., Ha, J.W., Choi, J.: Rainbow memory: continual learning with a memory of diverse samples. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Cermelli, F., Mancini, M., Bulo, S.R., Ricci, E., Caputo, B.: Modeling the background for incremental learning in semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Cha, S., kim, b., Yoo, Y., Moon, T.: SSUL: Semantic segmentation with unknown label for exemplar-based class-incremental learning. In: Advances in Neural Information Processing Systems (2021)
Chaudhry, A., et al.: On tiny episodic memories in continual learning. arXiv (2019)
Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv (2017)
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2009)
Douillard, A., Chen, Y., Dapogny, A., Cord, M.: PLOP: learning without forgetting for continual semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vision (IJCV) (2010)
Fernando, C., et al.: PathNet: evolution channels gradient descent in super neural networks. arXiv (2017)
Goswami, D., Schuster, R., van de Weijer, J., Stricker, D.: Attribution-aware weight transfer: a warm-start initialization for class-incremental semantic segmentation. In: Winter Conference on Applications of Computer Vision (2023)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Kemker, R., Kanan, C.: FearNet: brain-inspired model for incremental learning. In: International Conference on Learning Representations (2018)
Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. In: Proceedings of the National Academy of Sciences (2017)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Lee, B.H., Jung, O., Choi, J., Chun, S.Y.: Online continual learning on hierarchical label expansion. In: International Conference on Computer Vision (ICCV) (2023)
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., Díaz-Rodríguez, N.: Continual learning for robotics: definition, framework, learning strategies, opportunities and challenges. Information fusion (2020)
Li, Z., Hoiem, D.: Learning without forgetting. Trans. Pattern Anal. Mach. Intell. (T-PAMI) (2017)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: European Conference on Computer Vision (ECCV) (2014)
Lin, Z., Pathak, D., Wang, Y.X., Ramanan, D., Kong, S.: Continual learning with evolving class ontologies. In: Conference on Neural Information Processing Systems (NeurIPS) (2022)
Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. In: Conference on Neural Information Processing Systems (NeurIPS) (2017)
Mallya, A., Davis, D., Lazebnik, S.: Piggyback: adapting a single network to multiple tasks by learning to mask weights. In: European Conference on Computer Vision (ECCV) (2018)
Mallya, A., Lazebnik, S.: PackNet: adding multiple tasks to a single network by iterative pruning. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: The sequential learning problem. Psychol. Learn. Motivat. (1989)
Mermillod, M., Bugaiska, A., Bonin, P.: The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects (2013)
Michieli, U., Zanuttigh, P.: Incremental learning techniques for semantic segmentation. In: Conference on Computer Vision and Pattern Recognition Workshops (CVPR-W) (2019)
Michieli, U., Zanuttigh, P.: Continual semantic segmentation via repulsion-attraction of sparse and disentangled latent representations. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Neuhold, G., Ollmann, T., Rota Bulo, S., Kontschieder, P.: The mapillary vistas dataset for semantic understanding of street scenes. In: International Conference on Computer Vision (ICCV) (2017)
Phan, M.H., Ta, T.A., Phung, S.L., Tran-Thanh, L., Bouzerdoum, A.: Class similarity weighted knowledge distillation for continual semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Rusu, A.A., et al.: Progressive neural networks. arXiv (2016)
Shin, H., Lee, J.K., Kim, J., Kim, J.: Continual learning with deep generative replay. In: Conference on Neural Information Processing Systems (NeurIPS) (2017)
Sprechmann, P., et al.: Memory-based parameter adaptation. In: International Conference on Learning Representations (ICLR) (2018)
Van de Ven, G.M., Siegelmann, H.T., Tolias, A.S.: Brain-inspired replay for continual learning with artificial neural networks. Nat. Commun. (2020)
Xiao, J.W., Zhang, C.B., Feng, J., Liu, X., van de Weijer, J., Cheng, M.M.: Endpoints weight fusion for class incremental semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
Yang, G., et al.: Uncertainty-aware contrastive distillation for incremental semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2022)
Yang, G., et al.: Continual attentive fusion for incremental learning in semantic segmentation. IEEE Trans. Multimed. (2022)
Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv (2017)
Yu, L., Liu, X., Van de Weijer, J.: Self-training for class-incremental semantic segmentation. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning (ICML) (2017)
Zhang, C.B., Xiao, J.W., Liu, X., Chen, Y.C., Cheng, M.M.: Representation compensation networks for continual semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
Zhao, H., Hu, Q., Zhu, P., Wang, Y., Wang, P.: A recursive regularization based feature selection framework for hierarchical classification. Trans. Knowl. Data Eng. (2021)
Acknowledgments
This work was partially funded by the Federal Ministry of Education and Research Germany under the project DECODE (01IW21001).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Muralidhara, S., Bukhari, S., Schneider, G., Stricker, D., Schuster, R. (2025). CLEO: Continual Learning of Evolving Ontologies. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15112. Springer, Cham. https://doi.org/10.1007/978-3-031-72949-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-72949-2_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72948-5
Online ISBN: 978-3-031-72949-2
eBook Packages: Computer ScienceComputer Science (R0)