Abstract
Incremental learning is a machine learning strategy that enables the integration of new data to address emerging tasks without retraining the model from scratch. This approach retains previously acquired knowledge and conserves resources, yet it faces the challenge of catastrophic forgetting. Hierarchical classification (HC) improves accuracy and efficiency by assigning labels with hierarchical relationships to objects. In this work, we propose CHFL, an incremental learning method for remote sensing images. CHFL leverages hierarchical relationships through a label relationship graph and class hierarchy information to encode knowledge effectively. It integrates a Feature Pyramid Network (FPN) to process multi-scale features, capturing discriminative region information, and a Learning Without Forgetting (LWF) strategy to efficiently learn new classes while preserving performance on previous ones. Additionally, a Hierarchy and Exclusion (HEX) graph is introduced to constrain label predictions, enhancing consistency and improving classification accuracy. Experimental results on the high-resolution remote sensing dataset HRSC show that CHFL achieves an accuracy of 93.82% on new class classifications while maintaining competitive performance on previous classes. Compared with existing methods, CHFL demonstrates superior classification performance, effectively mitigating catastrophic forgetting and addressing scale variations in remote sensing imagery.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The HRSC dataset belongs to a public database and is widely used in ship detection and recognition research. The data involved in the dataset has obtained all necessary ethical approvals. Our study is based on open-source data, so there are no ethical issues and no conflicts of interest.
References
Rebuffi S, Kolesnikov A, Sperl G, Lampert CH (2017) icarl: Incremental classifier and representation learning. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp 5533–5542
Masana M, Liu X, Twardowski B, Menta M, Bagdanov AD, van de Weijer J (2022) Class-incremental learning: Survey and performance evaluation on image classification. IEEE Trans Pattern Anal Mach Intell (TPAMI) 45(5):5513–5533
Ye Z, Zhang Y, Zhang J, Li W, Bai L (2024) A multiscale incremental learning network for remote sensing scene classification. IEEE Trans Geosci Remote Sens (TGRS)
Sun L, Zhang M, Wang B, Tiwari P (2023) Few-shot class-incremental learning for medical time series classification. IEEE J Biomed Health Inform 28(4):1872–1882
Yu C, Zhao X, Gong B, Hu Y, Song M, Yu H, Chang CI (2024) Distillation-constrained prototype representation network for hyperspectral image incremental classification. IEEE Trans Geosci Remote Sens (TGRS)
Li H, Jiang H, Gu X, Peng J, Li W, Hong L, Tao C (2020) Clrs: Continual learning benchmark for remote sensing image scene classification. Sens 20(4):1226
Tasar O, Tarabalka Y, Alliez P (2019) Incremental learning for semantic segmentation of large-scale remote sensing data. IEEE J Sel Top Appl Earth Obs Remote Sens (J-STARS) 12(9):3524–3537
Chen J, Qian Y (2021) Hierarchical multilabel ship classification in remote sensing images using label relation graphs. IEEE Trans Geosci Remote Sens (TGRS) 60:1–13
Arshad T, Zhang J (2024) Hierarchical attention transformer for hyperspectral image classification. IEEE Geosci Remote Sens Lett (GRSL)
Springstein M, Schneider S, Rahnama J, Stalter J, Kristen M, Müller-Budack E, Ewerth R (2024) Visual narratives: Large-scale hierarchical classification of art-historical images. In: Proceedings of the IEEE/CVF Winter Conf Appl Comput Vision, pp 7220–7230
Cai D, Chen J, Zhao J, Xue Y, Yang S, Yuan W, Feng M, Weng H, Liu S, Peng Y et al (2024) Hicervix: An extensive hierarchical dataset and benchmark for cervical cytology classification. IEEE Trans Med Imaging
Połap D, Jaszcz A (2024) Sonar digital twin layer via multi-attention networks with feature transfer. IEEE Trans Geosci Remote Sens (TGRS)
Wang X, Pan Y, Chen J (2024) Digital twin with data-mechanism-fused model for smart excavation management. Autom Constr 168:105749
Tao F, Sui F, Liu A, Qi Q, Zhang M, Song B, Guo Z, Lu Stephen CY, Nee AY (2019) Digital twin-driven product design framework. Int J Prod Res 57(12):3935–3953
Iqbal I, Younus M, Walayat K, Kakar MU, Ma J (2021) Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images. Comput Med Imaging Graph 88:101843
Li Z, Hoiem D (2016) Learning without forgetting. In: Proc Eur Conf Comput Vis (ECCV), Cham, pp 614–629
Mazumder P, Singh P, Rai P, Namboodiri VP (2022) Rectification-based knowledge retention for task incremental learning. IEEE Trans Pattern Anal Mach Intell (TPAMI)
Zhao H, Fu Y, Kang M, Tian Q, Wu F, Li X (2021) Mgsvf: Multi-grained slow versus fast framework for few-shot class-incremental learning. IEEE Trans Pattern Anal Mach Intell (TPAMI) 46(3):1576–1588
Pan W, Gao T, Zhang Y, Zheng X, Shen Y, Li K, Hu R, Liu Y, Dai P (2024) Semi-supervised blind image quality assessment through knowledge distillation and incremental learning. Proc AAAI Conf Artif Intell (AAAI) 38:4388–4396
Panos A, Kobe Y, Reino DO, Aljundi R, Turner RE (2023) First session adaptation: A strong replay-free baseline for class-incremental learning. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV), pp 18820–18830
Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75
Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
Jodelet Q, Liu X, Phua YJ, Murata T (2023) Class-incremental learning using diffusion model for distillation and replay. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV), pp 3425–3433
Xu S, Meng G, Nie X, Ni B, Fan B, Xiang S (2024) Defying imbalanced forgetting in class incremental learning. Proc AAAI Conf Artif Intell (AAAI) 38:16211–16219
Chen X, Chang X (2023) Dynamic residual classifier for class incremental learning. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV), pp 18743–18752
Aljundi R, Lin M, Goujaud B, Bengio Y (2019) Gradient based sample selection for online continual learning. Adv Neural Inf Process Syst (NeurIPS) 32
Jung S, Ahn H, Cha S, Moon T (2020) Continual learning with node-importance based adaptive group sparse regularization. Adv Neural Inf Process Syst (NeurIPS) 33:3647–3658
Zenke F, Poole B, Ganguli S (2017) Continual learning through synaptic intelligence. In: Proc Int Conf Mach Learn. (ICML), pp 3987–3995. PMLR
Ahn H, Kwak J, Lim S, Bang H, Kim H, Moon T (2021) Ss-il: Separated softmax for incremental learning. In: Proc. IEEE/CVF Int Conf Comput Vis (ICCV), pp 844–853
Kang M, Park J, Han B (2022) Class-incremental learning by knowledge distillation with adaptive feature consolidation. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 16071–16080
Zhao B, Xiao X, Gan G, Zhang B, Xia ST (2020) Maintaining discrimination and fairness in class incremental learning. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 13208–13217
Zhu F, Zhang XY, Wang C, Yin F, Liu CL (2021) Prototype augmentation and self-supervision for incremental learning. In: Proc IEEE/CVF Conf. Comput Vis Pattern Recognit (CVPR), pp 5871–5880
Mallya A, Lazebnik S (2018) Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proc. IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 7765–7773
Wang FY, Zhou DW, Ye HJ, Zhan DC (2022) Foster: Feature boosting and compression for class-incremental learning. In: Proc Eur Conf Comput Vis (ECCV), pp 398–414. Springer
Yan S, Xie J, He X (2021) Der: Dynamically expandable representation for class incremental learning. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 3014–3023
Li X, Zhou Y, Wu T, Socher R, Xiong C (2019) Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In: Proc Int Conf Mach Learn (ICML), pp 3925–3934. PMLR
Shin H, Lee JK, Kim J, Kim J (2017) Continual learning with deep generative replay. Adv Neural Inf Process Syst (NeurIPS) 30
Rusu AA, Rabinowitz NC, Desjardins G, Soyer H, Kirkpatrick J, Kavukcuoglu K, Pascanu R, Hadsell R (2016) Progressive neural networks. arXiv preprint arXiv:1606.04671
Xu M, Zhao Y, Liang Y, Ma X (2022) Hyperspectral image classification based on class-incremental learning with knowledge distillation. Remote Sens 14:2556
Ruan H, Peng J, Chen Y, He S, Zhang Z, Li H (2022) A class-incremental detection method of remote sensing images based on selective distillation. Symmetry 14:2100
Yu CC, Chen TY, Hsu CW, Cheng HY (2024) Incremental scene classification using dual knowledge distillation and classifier discrepancy on natural and remote sensing images. Electron 13(3):583
Gao F, Kong L, Lang R, Sun J, Wang J, Hussain A, Zhou H (2024) Sar target incremental recognition based on features with strong separability. IEEE Trans Geosci Remote Sens (TGRS)
Lefebvre G, Elghazel H, Guillet T, Aussem A, Sonnati M (2024) A new sentence embedding framework for the education and professional training domain with application to hierarchical multi-label text classification. Data Knowl Eng 150:102281
Dumais ST, Chen H (2000) Hierarchical classification of web content. In: Proc Annu Int ACM SIGIR Conf Res Dev Inf Retriev
Guo Z, Cao Y (2022) Sa-cnn: Application to text categorization issues using simulated annealing-based convolutional neural network optimization. Proc Int Conf Electron Inf Technol Comput Eng
Chaki J, Parekh R, Bhattacharya S (2020) Plant leaf classification using multiple descriptors: A hierarchical approach. J King Saud Univ Comput Inf Sci 32:1158–1172
Jiang Y, Chen Z, Han N, Shang J, Wu A (2023) sc-immucc: hierarchical annotation for immune cell types in single-cell rna-seq. Front Immunol 14
Caron DP, Specht WL, Chen D, Wells SB, Szabo PA, Jensen IJ, Farber DL, Sims PA (2023) Multimodal hierarchical classification of cite-seq data delineates immune cell states across lineages and tissues. bioRxiv
Yang Y, Zeng Q, Liu G, Zheng S, Luo T, Guo Y, Tang J, Huang Y (2023) Hierarchical classification-based pan-cancer methylation analysis to classify primary cancer. BMC Bioinformatics 24
Feng S, Fu P, Zheng W (2018) A hierarchical multi-label classification method based on neural networks for gene function prediction. Biotechnol. Biotechnol Equip 32:1613–1621
Romero M, Ramirez OJV, Finke J, Rocha C (2022) Feature extraction with spectral clustering for gene function prediction using hierarchical multi-label classification. Appl Netw Sci 7
Choi K, Lee Y, Kim C, Yoon M (2021) An effective gcn-based hierarchical multi-label classification for protein function prediction. ArXiv abs/2112.02810 (2021)
Cao Y, Feng L, An B (2024) Consistent hierarchical classification with a generalized metric. In: Proc Int Conf Artif Intell Stat (AISTATS)
Yuan Z, Liu H, Zhou H, Zhang D, Zhang X, Wang H, Xiong H (2024) Self-paced unified representation learning for hierarchical multi-label classification. In: Proc AAAI Conf Artif Intell (AAAI)
Zhou Y, Li X, Zhou Y, Wang Y, Hu Q, Wang W (2022) Deep collaborative multi-task network: A human decision process inspired model for hierarchical image classification. Pattern Recognit 124:108449
Noor KT, Robles-Kelly A (2024) H-capsnet: A capsule network for hierarchical image classification. Pattern Recognit. 147:110135
Liu X, Wang L (2024) Multi-granularity sequence generation for hierarchical image classification. Comput. Vis Media 10(2):243–260
Boone Sifuentes T, Nazari A, Bouadjenek MR, Razzak I (2024) Mlt-trans: Multi-level token transformer for hierarchical image classification. In: Proc PacAsia Conf Knowl Discov Data Min (PAKDD), pp 385–396. Springer
Wang P, Chen J, Qian Y (2023) Semantic guided level-category hybrid prediction network for hierarchical image classification. Int J Wavelets Multiresolut Inf Process 21(06):2350023
Tian F, Lei S, Zhou Y, Cheng J, Liang G, Zou Z, Li HC, Shi Z (2024) Hirenet: Hierarchical-relation network for few-shot remote sensing image scene classification. IEEE Trans Geosci Remote Sens (TGRS)
Miao W, Han W, Geng J, Jiang W (2023) Hierarchical feature progressive alignment network for remote sensing image scene classification in multitarget domain adaptation. IEEE Trans Geosci Remote Sens (TGRS)
Zhao Y, Liang J, Huang S, Huang P (2024) Hierarchical deep features progressive aggregation for remote sensing images scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens (J-STARS)
Cao F, Huang X, Yang B, Ye H (2024) Hierarchical structural graph neural network with local relation enhancement for hyperspectral image classification. Digit Signal Process 146:104392
Song T, Zeng Z, Gao C, Chen H, Li J (2024) Joint classification of hyperspectral and lidar data using height information guided hierarchical fusion-and-separation network. IEEE Trans Geosci Remote Sens (TGRS)
Sheng Z, Cao G, Shi H, Zhang Y () Hyperspectral image classification using a double-branch hierarchical partial convolution network. Remote Sens Lett 15(3):203–214
Shi H, Zhang Y, Cao G, Yang D (2023) Mhcformer: Multiscale hierarchical conv-aided fourierformer for hyperspectral image classification. IEEE Trans Instrum Meas
Sun H, Wu S, Chen X, Li M, Kong L, Yang X, Meng Y, Chen S, Zheng J (2024) Sah-net: Structure-aware hierarchical network for clustered microcalcification classification in digital breast tomosynthesis. IEEE Trans Cybern 54(4):2345–2357
Althenayan AS, AlSalamah SA, Aly S, Nouh T, Mahboub B, Salameh L, Alkubeyyer M, Mirza A (2024) Covid-19 hierarchical classification using a deep learning multi-modal. Sens 24(8):2641
Chaushevska M, Dimitrovski I, Deroski S, Gjoreski H (2020) Hierarchical classification of diatom images with transfer learning
Dimitrovski I, Kocev D, Loskovska S, Dzeroski S (2012) Hierarchical classification of diatom images using ensembles of predictive clustering trees. Ecol Inform 7(1):19–29
Cerri R, Barros RC, Carvalho ACPLF, Jin Y (2016) Reduction strategies for hierarchical multi-label classification in protein function prediction. BMC Bioinformatics 17
Wehrmann J, Cerri R, Barros RC (2018) Hierarchical multi-label classification networks. In: Proc Int Conf Mach Learn (ICML)
Giunchiglia E, Lukasiewicz T (2020) Coherent hierarchical multi-label classification networks. ArXiv abs/2010.10151
Jiang J, Chen Z, Lei F, Xu L, Huang J, Yuan X (2024) Multi-granularity hypergraph-guided transformer learning framework for visual classification. Vis Comput 1–18
Iqbal I, Shahzad G, Rafiq N, Mustafa G, Ma J (2020) Deep learning-based automated detection of human knee joint’s synovial fluid from magnetic resonance images with transfer learning. IET Image Process 14(10):1990–1998
Chen J, Wang S, Chen L, Cai H, Qian Y (2020) Incremental detection of remote sensing objects with feature pyramid and knowledge distillation. IEEE Trans Geosci Remote Sens (TGRS) 60:1–13
Chen J, Wang P, Liu J, Qian Yt (2022) Label relation graphs enhanced hierarchical residual network for hierarchical multi-granularity classification. IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), 4848–4857
Chu Y, Wang P, Qian Y (2023) Incremental learning of remote sensing target classification with class hierarchy. In: Proc IEEE Int Geosci Remote Sens Symp, pp 6254–6257
Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. ArXiv abs/1503.02531
Chu Y, Qian Y (2024) Incremental classification of remote sensing images with feature pyramid and class hierarchy. In: Proc IEEE Int Geosci Remote Sens Symp, pp 8797–8800
Chen J, Wang P, Liu J, Qian Y (2022) Label relation graphs enhanced hierarchical residual network for hierarchical multi-granularity classification. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 4848–4857
Funding
This work was supported by the National Natural Science Foundation of China 62071421.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chu, Y., Qian, Y. Incremental classification of remote sensing images using feature pyramid and class hierarchy enhanced by label relationship graphs. Appl Intell 55, 234 (2025). https://doi.org/10.1007/s10489-024-06216-0
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06216-0