Skip to main content

Advertisement

Incremental classification of remote sensing images using feature pyramid and class hierarchy enhanced by label relationship graphs

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Incremental learning is a machine learning strategy that enables the integration of new data to address emerging tasks without retraining the model from scratch. This approach retains previously acquired knowledge and conserves resources, yet it faces the challenge of catastrophic forgetting. Hierarchical classification (HC) improves accuracy and efficiency by assigning labels with hierarchical relationships to objects. In this work, we propose CHFL, an incremental learning method for remote sensing images. CHFL leverages hierarchical relationships through a label relationship graph and class hierarchy information to encode knowledge effectively. It integrates a Feature Pyramid Network (FPN) to process multi-scale features, capturing discriminative region information, and a Learning Without Forgetting (LWF) strategy to efficiently learn new classes while preserving performance on previous ones. Additionally, a Hierarchy and Exclusion (HEX) graph is introduced to constrain label predictions, enhancing consistency and improving classification accuracy. Experimental results on the high-resolution remote sensing dataset HRSC show that CHFL achieves an accuracy of 93.82% on new class classifications while maintaining competitive performance on previous classes. Compared with existing methods, CHFL demonstrates superior classification performance, effectively mitigating catastrophic forgetting and addressing scale variations in remote sensing imagery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The HRSC dataset belongs to a public database and is widely used in ship detection and recognition research. The data involved in the dataset has obtained all necessary ethical approvals. Our study is based on open-source data, so there are no ethical issues and no conflicts of interest.

References

  1. Rebuffi S, Kolesnikov A, Sperl G, Lampert CH (2017) icarl: Incremental classifier and representation learning. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp 5533–5542

  2. Masana M, Liu X, Twardowski B, Menta M, Bagdanov AD, van de Weijer J (2022) Class-incremental learning: Survey and performance evaluation on image classification. IEEE Trans Pattern Anal Mach Intell (TPAMI) 45(5):5513–5533

    Article  Google Scholar 

  3. Ye Z, Zhang Y, Zhang J, Li W, Bai L (2024) A multiscale incremental learning network for remote sensing scene classification. IEEE Trans Geosci Remote Sens (TGRS)

  4. Sun L, Zhang M, Wang B, Tiwari P (2023) Few-shot class-incremental learning for medical time series classification. IEEE J Biomed Health Inform 28(4):1872–1882

    Article  MATH  Google Scholar 

  5. Yu C, Zhao X, Gong B, Hu Y, Song M, Yu H, Chang CI (2024) Distillation-constrained prototype representation network for hyperspectral image incremental classification. IEEE Trans Geosci Remote Sens (TGRS)

  6. Li H, Jiang H, Gu X, Peng J, Li W, Hong L, Tao C (2020) Clrs: Continual learning benchmark for remote sensing image scene classification. Sens 20(4):1226

    Article  MATH  Google Scholar 

  7. Tasar O, Tarabalka Y, Alliez P (2019) Incremental learning for semantic segmentation of large-scale remote sensing data. IEEE J Sel Top Appl Earth Obs Remote Sens (J-STARS) 12(9):3524–3537

  8. Chen J, Qian Y (2021) Hierarchical multilabel ship classification in remote sensing images using label relation graphs. IEEE Trans Geosci Remote Sens (TGRS) 60:1–13

    MATH  Google Scholar 

  9. Arshad T, Zhang J (2024) Hierarchical attention transformer for hyperspectral image classification. IEEE Geosci Remote Sens Lett (GRSL)

  10. Springstein M, Schneider S, Rahnama J, Stalter J, Kristen M, Müller-Budack E, Ewerth R (2024) Visual narratives: Large-scale hierarchical classification of art-historical images. In: Proceedings of the IEEE/CVF Winter Conf Appl Comput Vision, pp 7220–7230

  11. Cai D, Chen J, Zhao J, Xue Y, Yang S, Yuan W, Feng M, Weng H, Liu S, Peng Y et al (2024) Hicervix: An extensive hierarchical dataset and benchmark for cervical cytology classification. IEEE Trans Med Imaging

  12. Połap D, Jaszcz A (2024) Sonar digital twin layer via multi-attention networks with feature transfer. IEEE Trans Geosci Remote Sens (TGRS)

  13. Wang X, Pan Y, Chen J (2024) Digital twin with data-mechanism-fused model for smart excavation management. Autom Constr 168:105749

    Article  MATH  Google Scholar 

  14. Tao F, Sui F, Liu A, Qi Q, Zhang M, Song B, Guo Z, Lu Stephen CY, Nee AY (2019) Digital twin-driven product design framework. Int J Prod Res 57(12):3935–3953

    Article  Google Scholar 

  15. Iqbal I, Younus M, Walayat K, Kakar MU, Ma J (2021) Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images. Comput Med Imaging Graph 88:101843

    Article  Google Scholar 

  16. Li Z, Hoiem D (2016) Learning without forgetting. In: Proc Eur Conf Comput Vis (ECCV), Cham, pp 614–629

  17. Mazumder P, Singh P, Rai P, Namboodiri VP (2022) Rectification-based knowledge retention for task incremental learning. IEEE Trans Pattern Anal Mach Intell (TPAMI)

  18. Zhao H, Fu Y, Kang M, Tian Q, Wu F, Li X (2021) Mgsvf: Multi-grained slow versus fast framework for few-shot class-incremental learning. IEEE Trans Pattern Anal Mach Intell (TPAMI) 46(3):1576–1588

    Article  MATH  Google Scholar 

  19. Pan W, Gao T, Zhang Y, Zheng X, Shen Y, Li K, Hu R, Liu Y, Dai P (2024) Semi-supervised blind image quality assessment through knowledge distillation and incremental learning. Proc AAAI Conf Artif Intell (AAAI) 38:4388–4396

    Google Scholar 

  20. Panos A, Kobe Y, Reino DO, Aljundi R, Turner RE (2023) First session adaptation: A strong replay-free baseline for class-incremental learning. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV), pp 18820–18830

  21. Young T, Hazarika D, Poria S, Cambria E (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75

    Article  MATH  Google Scholar 

  22. Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  23. Jodelet Q, Liu X, Phua YJ, Murata T (2023) Class-incremental learning using diffusion model for distillation and replay. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV), pp 3425–3433

  24. Xu S, Meng G, Nie X, Ni B, Fan B, Xiang S (2024) Defying imbalanced forgetting in class incremental learning. Proc AAAI Conf Artif Intell (AAAI) 38:16211–16219

    Google Scholar 

  25. Chen X, Chang X (2023) Dynamic residual classifier for class incremental learning. In: Proc IEEE/CVF Int Conf Comput Vis (ICCV), pp 18743–18752

  26. Aljundi R, Lin M, Goujaud B, Bengio Y (2019) Gradient based sample selection for online continual learning. Adv Neural Inf Process Syst (NeurIPS) 32

  27. Jung S, Ahn H, Cha S, Moon T (2020) Continual learning with node-importance based adaptive group sparse regularization. Adv Neural Inf Process Syst (NeurIPS) 33:3647–3658

    MATH  Google Scholar 

  28. Zenke F, Poole B, Ganguli S (2017) Continual learning through synaptic intelligence. In: Proc Int Conf Mach Learn. (ICML), pp 3987–3995. PMLR

  29. Ahn H, Kwak J, Lim S, Bang H, Kim H, Moon T (2021) Ss-il: Separated softmax for incremental learning. In: Proc. IEEE/CVF Int Conf Comput Vis (ICCV), pp 844–853

  30. Kang M, Park J, Han B (2022) Class-incremental learning by knowledge distillation with adaptive feature consolidation. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 16071–16080

  31. Zhao B, Xiao X, Gan G, Zhang B, Xia ST (2020) Maintaining discrimination and fairness in class incremental learning. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 13208–13217

  32. Zhu F, Zhang XY, Wang C, Yin F, Liu CL (2021) Prototype augmentation and self-supervision for incremental learning. In: Proc IEEE/CVF Conf. Comput Vis Pattern Recognit (CVPR), pp 5871–5880

  33. Mallya A, Lazebnik S (2018) Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proc. IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 7765–7773

  34. Wang FY, Zhou DW, Ye HJ, Zhan DC (2022) Foster: Feature boosting and compression for class-incremental learning. In: Proc Eur Conf Comput Vis (ECCV), pp 398–414. Springer

  35. Yan S, Xie J, He X (2021) Der: Dynamically expandable representation for class incremental learning. In: Proc IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 3014–3023

  36. Li X, Zhou Y, Wu T, Socher R, Xiong C (2019) Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In: Proc Int Conf Mach Learn (ICML), pp 3925–3934. PMLR

  37. Shin H, Lee JK, Kim J, Kim J (2017) Continual learning with deep generative replay. Adv Neural Inf Process Syst (NeurIPS) 30

  38. Rusu AA, Rabinowitz NC, Desjardins G, Soyer H, Kirkpatrick J, Kavukcuoglu K, Pascanu R, Hadsell R (2016) Progressive neural networks. arXiv preprint arXiv:1606.04671

  39. Xu M, Zhao Y, Liang Y, Ma X (2022) Hyperspectral image classification based on class-incremental learning with knowledge distillation. Remote Sens 14:2556

    Article  MATH  Google Scholar 

  40. Ruan H, Peng J, Chen Y, He S, Zhang Z, Li H (2022) A class-incremental detection method of remote sensing images based on selective distillation. Symmetry 14:2100

    Article  MATH  Google Scholar 

  41. Yu CC, Chen TY, Hsu CW, Cheng HY (2024) Incremental scene classification using dual knowledge distillation and classifier discrepancy on natural and remote sensing images. Electron 13(3):583

    Article  MATH  Google Scholar 

  42. Gao F, Kong L, Lang R, Sun J, Wang J, Hussain A, Zhou H (2024) Sar target incremental recognition based on features with strong separability. IEEE Trans Geosci Remote Sens (TGRS)

  43. Lefebvre G, Elghazel H, Guillet T, Aussem A, Sonnati M (2024) A new sentence embedding framework for the education and professional training domain with application to hierarchical multi-label text classification. Data Knowl Eng 150:102281

    Article  MATH  Google Scholar 

  44. Dumais ST, Chen H (2000) Hierarchical classification of web content. In: Proc Annu Int ACM SIGIR Conf Res Dev Inf Retriev

  45. Guo Z, Cao Y (2022) Sa-cnn: Application to text categorization issues using simulated annealing-based convolutional neural network optimization. Proc Int Conf Electron Inf Technol Comput Eng

  46. Chaki J, Parekh R, Bhattacharya S (2020) Plant leaf classification using multiple descriptors: A hierarchical approach. J King Saud Univ Comput Inf Sci 32:1158–1172

    MATH  Google Scholar 

  47. Jiang Y, Chen Z, Han N, Shang J, Wu A (2023) sc-immucc: hierarchical annotation for immune cell types in single-cell rna-seq. Front Immunol 14

  48. Caron DP, Specht WL, Chen D, Wells SB, Szabo PA, Jensen IJ, Farber DL, Sims PA (2023) Multimodal hierarchical classification of cite-seq data delineates immune cell states across lineages and tissues. bioRxiv

  49. Yang Y, Zeng Q, Liu G, Zheng S, Luo T, Guo Y, Tang J, Huang Y (2023) Hierarchical classification-based pan-cancer methylation analysis to classify primary cancer. BMC Bioinformatics 24

  50. Feng S, Fu P, Zheng W (2018) A hierarchical multi-label classification method based on neural networks for gene function prediction. Biotechnol. Biotechnol Equip 32:1613–1621

    Article  MATH  Google Scholar 

  51. Romero M, Ramirez OJV, Finke J, Rocha C (2022) Feature extraction with spectral clustering for gene function prediction using hierarchical multi-label classification. Appl Netw Sci 7

  52. Choi K, Lee Y, Kim C, Yoon M (2021) An effective gcn-based hierarchical multi-label classification for protein function prediction. ArXiv abs/2112.02810 (2021)

  53. Cao Y, Feng L, An B (2024) Consistent hierarchical classification with a generalized metric. In: Proc Int Conf Artif Intell Stat (AISTATS)

  54. Yuan Z, Liu H, Zhou H, Zhang D, Zhang X, Wang H, Xiong H (2024) Self-paced unified representation learning for hierarchical multi-label classification. In: Proc AAAI Conf Artif Intell (AAAI)

  55. Zhou Y, Li X, Zhou Y, Wang Y, Hu Q, Wang W (2022) Deep collaborative multi-task network: A human decision process inspired model for hierarchical image classification. Pattern Recognit 124:108449

    Article  MATH  Google Scholar 

  56. Noor KT, Robles-Kelly A (2024) H-capsnet: A capsule network for hierarchical image classification. Pattern Recognit. 147:110135

    Article  MATH  Google Scholar 

  57. Liu X, Wang L (2024) Multi-granularity sequence generation for hierarchical image classification. Comput. Vis Media 10(2):243–260

    Article  MATH  Google Scholar 

  58. Boone Sifuentes T, Nazari A, Bouadjenek MR, Razzak I (2024) Mlt-trans: Multi-level token transformer for hierarchical image classification. In: Proc PacAsia Conf Knowl Discov Data Min (PAKDD), pp 385–396. Springer

  59. Wang P, Chen J, Qian Y (2023) Semantic guided level-category hybrid prediction network for hierarchical image classification. Int J Wavelets Multiresolut Inf Process 21(06):2350023

  60. Tian F, Lei S, Zhou Y, Cheng J, Liang G, Zou Z, Li HC, Shi Z (2024) Hirenet: Hierarchical-relation network for few-shot remote sensing image scene classification. IEEE Trans Geosci Remote Sens (TGRS)

  61. Miao W, Han W, Geng J, Jiang W (2023) Hierarchical feature progressive alignment network for remote sensing image scene classification in multitarget domain adaptation. IEEE Trans Geosci Remote Sens (TGRS)

  62. Zhao Y, Liang J, Huang S, Huang P (2024) Hierarchical deep features progressive aggregation for remote sensing images scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens (J-STARS)

  63. Cao F, Huang X, Yang B, Ye H (2024) Hierarchical structural graph neural network with local relation enhancement for hyperspectral image classification. Digit Signal Process 146:104392

    Article  MATH  Google Scholar 

  64. Song T, Zeng Z, Gao C, Chen H, Li J (2024) Joint classification of hyperspectral and lidar data using height information guided hierarchical fusion-and-separation network. IEEE Trans Geosci Remote Sens (TGRS)

  65. Sheng Z, Cao G, Shi H, Zhang Y () Hyperspectral image classification using a double-branch hierarchical partial convolution network. Remote Sens Lett 15(3):203–214

  66. Shi H, Zhang Y, Cao G, Yang D (2023) Mhcformer: Multiscale hierarchical conv-aided fourierformer for hyperspectral image classification. IEEE Trans Instrum Meas

  67. Sun H, Wu S, Chen X, Li M, Kong L, Yang X, Meng Y, Chen S, Zheng J (2024) Sah-net: Structure-aware hierarchical network for clustered microcalcification classification in digital breast tomosynthesis. IEEE Trans Cybern 54(4):2345–2357

    Article  Google Scholar 

  68. Althenayan AS, AlSalamah SA, Aly S, Nouh T, Mahboub B, Salameh L, Alkubeyyer M, Mirza A (2024) Covid-19 hierarchical classification using a deep learning multi-modal. Sens 24(8):2641

    Article  Google Scholar 

  69. Chaushevska M, Dimitrovski I, Deroski S, Gjoreski H (2020) Hierarchical classification of diatom images with transfer learning

  70. Dimitrovski I, Kocev D, Loskovska S, Dzeroski S (2012) Hierarchical classification of diatom images using ensembles of predictive clustering trees. Ecol Inform 7(1):19–29

    Article  MATH  Google Scholar 

  71. Cerri R, Barros RC, Carvalho ACPLF, Jin Y (2016) Reduction strategies for hierarchical multi-label classification in protein function prediction. BMC Bioinformatics 17

  72. Wehrmann J, Cerri R, Barros RC (2018) Hierarchical multi-label classification networks. In: Proc Int Conf Mach Learn (ICML)

  73. Giunchiglia E, Lukasiewicz T (2020) Coherent hierarchical multi-label classification networks. ArXiv abs/2010.10151

  74. Jiang J, Chen Z, Lei F, Xu L, Huang J, Yuan X (2024) Multi-granularity hypergraph-guided transformer learning framework for visual classification. Vis Comput 1–18

  75. Iqbal I, Shahzad G, Rafiq N, Mustafa G, Ma J (2020) Deep learning-based automated detection of human knee joint’s synovial fluid from magnetic resonance images with transfer learning. IET Image Process 14(10):1990–1998

    Article  Google Scholar 

  76. Chen J, Wang S, Chen L, Cai H, Qian Y (2020) Incremental detection of remote sensing objects with feature pyramid and knowledge distillation. IEEE Trans Geosci Remote Sens (TGRS) 60:1–13

    MATH  Google Scholar 

  77. Chen J, Wang P, Liu J, Qian Yt (2022) Label relation graphs enhanced hierarchical residual network for hierarchical multi-granularity classification. IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), 4848–4857

  78. Chu Y, Wang P, Qian Y (2023) Incremental learning of remote sensing target classification with class hierarchy. In: Proc IEEE Int Geosci Remote Sens Symp, pp 6254–6257

  79. Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. ArXiv abs/1503.02531

  80. Chu Y, Qian Y (2024) Incremental classification of remote sensing images with feature pyramid and class hierarchy. In: Proc IEEE Int Geosci Remote Sens Symp, pp 8797–8800

  81. Chen J, Wang P, Liu J, Qian Y (2022) Label relation graphs enhanced hierarchical residual network for hierarchical multi-granularity classification. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 4848–4857

Download references

Funding

This work was supported by the National Natural Science Foundation of China 62071421.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yuntao Qian.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chu, Y., Qian, Y. Incremental classification of remote sensing images using feature pyramid and class hierarchy enhanced by label relationship graphs. Appl Intell 55, 234 (2025). https://doi.org/10.1007/s10489-024-06216-0

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06216-0

Keywords