Skip to main content

CLEO: Continual Learning of Evolving Ontologies

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Abstract

Continual learning (CL) addresses the problem of catastrophic forgetting in neural networks, which occurs when a trained model tends to overwrite previously learned information, when presented with a new task. CL aims to instill the lifelong learning characteristic of humans in intelligent systems, making them capable of learning continuously while retaining what was already learned. Current CL problems involve either learning new domains (domain-incremental) or new and previously unseen classes (class-incremental). However, general learning processes are not just limited to learning information, but also refinement of existing information. In this paper, we define CLEO – Continual Learning of Evolving Ontologies, as a new incremental learning setting under CL to tackle evolving classes. CLEO is motivated by the need for intelligent systems to adapt to real-world ontologies that change over time, such as those in autonomous driving. We use Cityscapes, PASCAL VOC, and Mapillary Vistas to define the task settings and demonstrate the applicability of CLEO. We highlight the shortcomings of existing CIL methods in adapting to CLEO and propose a baseline solution, called Modelling Ontologies (MoOn). CLEO is a promising new approach to CL that addresses the challenge of evolving ontologies in real-world applications. MoOn surpasses previous CL approaches in the context of CLEO.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  2. Asghar, N., Mou, L., Selby, K.A., Pantasdo, K.D., Poupart, P., Jiang, X.: Progressive memory banks for incremental domain adaptation. arXiv (2018)

    Google Scholar 

  3. Bang, J., Kim, H., Yoo, Y., Ha, J.W., Choi, J.: Rainbow memory: continual learning with a memory of diverse samples. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  4. Cermelli, F., Mancini, M., Bulo, S.R., Ricci, E., Caputo, B.: Modeling the background for incremental learning in semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  5. Cha, S., kim, b., Yoo, Y., Moon, T.: SSUL: Semantic segmentation with unknown label for exemplar-based class-incremental learning. In: Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  6. Chaudhry, A., et al.: On tiny episodic memories in continual learning. arXiv (2019)

    Google Scholar 

  7. Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv (2017)

    Google Scholar 

  8. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  9. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2009)

    Google Scholar 

  10. Douillard, A., Chen, Y., Dapogny, A., Cord, M.: PLOP: learning without forgetting for continual semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  11. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vision (IJCV) (2010)

    Google Scholar 

  12. Fernando, C., et al.: PathNet: evolution channels gradient descent in super neural networks. arXiv (2017)

    Google Scholar 

  13. Goswami, D., Schuster, R., van de Weijer, J., Stricker, D.: Attribution-aware weight transfer: a warm-start initialization for class-incremental semantic segmentation. In: Winter Conference on Applications of Computer Vision (2023)

    Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  15. Kemker, R., Kanan, C.: FearNet: brain-inspired model for incremental learning. In: International Conference on Learning Representations (2018)

    Google Scholar 

  16. Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. In: Proceedings of the National Academy of Sciences (2017)

    Google Scholar 

  17. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)

    Google Scholar 

  18. Lee, B.H., Jung, O., Choi, J., Chun, S.Y.: Online continual learning on hierarchical label expansion. In: International Conference on Computer Vision (ICCV) (2023)

    Google Scholar 

  19. Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., Díaz-Rodríguez, N.: Continual learning for robotics: definition, framework, learning strategies, opportunities and challenges. Information fusion (2020)

    Google Scholar 

  20. Li, Z., Hoiem, D.: Learning without forgetting. Trans. Pattern Anal. Mach. Intell. (T-PAMI) (2017)

    Google Scholar 

  21. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: European Conference on Computer Vision (ECCV) (2014)

    Google Scholar 

  22. Lin, Z., Pathak, D., Wang, Y.X., Ramanan, D., Kong, S.: Continual learning with evolving class ontologies. In: Conference on Neural Information Processing Systems (NeurIPS) (2022)

    Google Scholar 

  23. Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. In: Conference on Neural Information Processing Systems (NeurIPS) (2017)

    Google Scholar 

  24. Mallya, A., Davis, D., Lazebnik, S.: Piggyback: adapting a single network to multiple tasks by learning to mask weights. In: European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  25. Mallya, A., Lazebnik, S.: PackNet: adding multiple tasks to a single network by iterative pruning. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Google Scholar 

  26. McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: The sequential learning problem. Psychol. Learn. Motivat. (1989)

    Google Scholar 

  27. Mermillod, M., Bugaiska, A., Bonin, P.: The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects (2013)

    Google Scholar 

  28. Michieli, U., Zanuttigh, P.: Incremental learning techniques for semantic segmentation. In: Conference on Computer Vision and Pattern Recognition Workshops (CVPR-W) (2019)

    Google Scholar 

  29. Michieli, U., Zanuttigh, P.: Continual semantic segmentation via repulsion-attraction of sparse and disentangled latent representations. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  30. Neuhold, G., Ollmann, T., Rota Bulo, S., Kontschieder, P.: The mapillary vistas dataset for semantic understanding of street scenes. In: International Conference on Computer Vision (ICCV) (2017)

    Google Scholar 

  31. Phan, M.H., Ta, T.A., Phung, S.L., Tran-Thanh, L., Bouzerdoum, A.: Class similarity weighted knowledge distillation for continual semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

    Google Scholar 

  32. Rusu, A.A., et al.: Progressive neural networks. arXiv (2016)

    Google Scholar 

  33. Shin, H., Lee, J.K., Kim, J., Kim, J.: Continual learning with deep generative replay. In: Conference on Neural Information Processing Systems (NeurIPS) (2017)

    Google Scholar 

  34. Sprechmann, P., et al.: Memory-based parameter adaptation. In: International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

  35. Van de Ven, G.M., Siegelmann, H.T., Tolias, A.S.: Brain-inspired replay for continual learning with artificial neural networks. Nat. Commun. (2020)

    Google Scholar 

  36. Xiao, J.W., Zhang, C.B., Feng, J., Liu, X., van de Weijer, J., Cheng, M.M.: Endpoints weight fusion for class incremental semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Google Scholar 

  37. Yang, G., et al.: Uncertainty-aware contrastive distillation for incremental semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2022)

    Google Scholar 

  38. Yang, G., et al.: Continual attentive fusion for incremental learning in semantic segmentation. IEEE Trans. Multimed. (2022)

    Google Scholar 

  39. Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv (2017)

    Google Scholar 

  40. Yu, L., Liu, X., Van de Weijer, J.: Self-training for class-incremental semantic segmentation. IEEE Trans. Neural Netw. Learn. Syst. (2022)

    Google Scholar 

  41. Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning (ICML) (2017)

    Google Scholar 

  42. Zhang, C.B., Xiao, J.W., Liu, X., Chen, Y.C., Cheng, M.M.: Representation compensation networks for continual semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

    Google Scholar 

  43. Zhao, H., Hu, Q., Zhu, P., Wang, Y., Wang, P.: A recursive regularization based feature selection framework for hierarchical classification. Trans. Knowl. Data Eng. (2021)

    Google Scholar 

Download references

Acknowledgments

This work was partially funded by the Federal Ministry of Education and Research Germany under the project DECODE (01IW21001).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shishir Muralidhara .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 8464 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muralidhara, S., Bukhari, S., Schneider, G., Stricker, D., Schuster, R. (2025). CLEO: Continual Learning of Evolving Ontologies. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15112. Springer, Cham. https://doi.org/10.1007/978-3-031-72949-2_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72949-2_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72948-5

  • Online ISBN: 978-3-031-72949-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics