Skip to main content
SpringerLink
Log in

MGFEEN: a multi-granularity feature encoding ensemble network for remote sensing image classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep convolutional neural networks (DCNNs) have emerged as powerful tools in diverse remote sensing domains, but their optimization remains challenging due to their complex nature and the large number of parameters involved. Researchers have been exploring more sophisticated methodologies to improve image classification accuracy. In this paper, we introduce a multi-granularity feature encoding ensemble network (MGFEEN) that is designed to fine-tune features at different levels of granularity. The network is trained in a two-step process: First, the output of granularity level i is used as the input for the next level; then, a fully connected layer is added to the pre-trained network to advance to the next level. The effectiveness of the MGFEEN’s feature extraction is evaluated by feeding the globally extracted features to a softmax classifier for classification. By applying ensemble learning principles, our proposed MGFEEN achieves more accurate final predictions. We evaluate our model on three widely recognized benchmark datasets: UC-Merced, SIRIWHU, and EAC-Dataset. Notably, on the EAC-Dataset, our results show a significant 0.54% improvement in accuracy over a single-training-network setup, resulting in an impressive 98.70% accuracy level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

All data are available from the corresponding author upon reasonable request.

References

  1. Amankulova K, Farmonov N, Mucsi L (2023) Time-series analysis of sentinel-2 satellite images for sunflower yield estimation. Smart Agric Technol 3:100098

    Article  Google Scholar 

  2. Bhaga TD, Dube T, Shekede MD, Shoko C (2023) Investigating the effectiveness of Landsat-8 OLI and Sentinel-2 MSI satellite data in monitoring the effects of drought on surface water resources in the Western Cape Province, South Africa. Remote Sens Appl Soc Environ 32:101037

    Google Scholar 

  3. Wang Qing, Zhang X, Chen G, Dai F, Gong Y, Zhu K (2022) Change detection based on faster R-CNN for high-resolution remote sensing images. Remote Sens l 9(10):923–932. https://doi.org/10.1080/2150704X.2018.1492172

    Article  Google Scholar 

  4. Fazry L, Ramadhan MML, Jatmiko W (2023) Change detection of high-resolution remote sensing images through adaptive focal modulation on hierarchical feature maps. IEEE Access. https://doi.org/10.1109/ACCESS.2023.3292531

    Article  Google Scholar 

  5. Usmani M, Napolitano M, Bovolo F (2023) Towards global scale segmentation with OpenStreetMap and remote sensing. ISPRS Open J Photogramm Remote Sens 8:100031. https://doi.org/10.1016/j.ophoto.2023.100031

    Article  Google Scholar 

  6. Xu Q, Shi Y, Yuan X, Zhu XX (2023) Universal domain adaptation for remote sensing image scene classification. IEEE Transact Geosci Remote Sens 61:1–15. https://doi.org/10.1109/TGRS.2023.3235988

    Article  CAS  Google Scholar 

  7. PChen Q, Li X, Zhang Z, Zhou C, Guo Z, Liu Z, Zhang H (2023) Remote sensing of photovoltaic scenarios: techniques, applications and future directions. Appl Energy 333:120579. https://doi.org/10.1016/j.apenergy.2022.120579

  8. Zhang C, Zhang Y, Lin H (2023) Multi-scale feature interaction network for remote sensing change detection. Remote Sens 15(11):2880. https://doi.org/10.3390/rs15112880

    Article  ADS  Google Scholar 

  9. Pandiri DK, Murugan R, Goel T (2024) Smart soil image classification system using lightweight convolutional neural network. Exp Syst Appl 238:122185. https://doi.org/10.1016/j.eswa.2023.122185

    Article  Google Scholar 

  10. Kumar GSC, Kumar RK, Kumar KPV, Sai NR, Brahmaiah M (2024) Deep residual convolutional neural network: an efficient technique for intrusion detection system. Exp Syst Appl 238:121912. https://doi.org/10.1016/j.eswa.2023.121912

    Article  Google Scholar 

  11. Kim B, Natarajan Y, Preethaa KS, Song S, An J, Mohan S (2024) Real-time assessment of surface cracks in concrete structures using integrated deep neural networks with autonomous unmanned aerial vehicle. Eng Appl Artif Intell 129:107537

    Article  Google Scholar 

  12. Bo LI, Xiaoyang XIE, Xingxing WEI, Wenting TANG (2021) Ship detection and classification from optical remote sensing images: a survey. Chin J Aeronaut 34(3):145–163. https://doi.org/10.1016/j.cja.2020.09.022

    Article  Google Scholar 

  13. Jena B, Nayak GK, Saxena S (2022) Convolutional neural network and its pretrained models for image classification and object detection: a survey. Concurr Comput Pract Exp 34(6):e6767. https://doi.org/10.1002/cpe.6767

    Article  Google Scholar 

  14. Wu Z, Hou B, Ren B, Ren Z, Wang S, Jiao L (2021) A deep detection network based on interaction of instance segmentation and object detection for SAR images. Remote Sens 13(13):2582. https://doi.org/10.3390/rs13132582

    Article  ADS  Google Scholar 

  15. Bernal J, Kushibar K, Asfaw DS, Valverde S, Oliver A, Mart R, Llad X (2019) Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review. Artif Intell Med 95:64–81. https://doi.org/10.1016/j.artmed.2020.08.008

    Article  PubMed  Google Scholar 

  16. Rahman S, Wang L, Sun C, Zhou L ( 2021) Deep learning based HEp-2 image classification: a comprehensive review. Med Image Anal 101764, https://doi.org/10.1016/j.media.2021.101764

  17. Hwang J-I, Jung H-S (2020) Automatic ship detection using the artificial neural network and support vector machine from X-band SAR satellite images. Remote Sens 10(11):1799

    Article  ADS  Google Scholar 

  18. Miao J, Wang B, Wu X, Zhang L, Hu B, Zhang JQ (2020) Deep feature extraction based on siamese network and auto-encoder for hyperspectral image classification. In: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium 397–400,https://doi.org/10.1109/IGARSS.2019.8899230

  19. Aslam MA, Salik MN, Chughtai F, Ali N, Dar SH, Khalil T (2020) Image classification based on mid-level feature fusion. In: 15th International conference on emerging technologies (ICET) 2020. https://doi.org/10.1109/ICET48972.2019.8994721

  20. Petrovska B, Zdravevski E, Lameski P, Corizzo R, Štajduhar I, Lerga J (2021) Deep learning for feature extraction in remote sensing: a case-study of aerial scene classification. Sensors (Switzerland) 20(14):1–22. https://doi.org/10.3390/s20143906

    Article  Google Scholar 

  21. Aung Su WY, Khaing SS, Aung ST (2020) Multi-label land cover indices classification of satellite images using deep learning. In: International conference big data analysis and deep learning applications, vol 744, pp 94–103

  22. Scott GJ, Marcum RA, Davis CH, Nivin TW (2020) Fusion of deep convolutional neural networks for land cover classification of high-resolution imagery. IEEE Geosci Remote Sens Lett 14(9):1638–1642. https://doi.org/10.1109/LGRS.2020.2722988

    Article  ADS  Google Scholar 

  23. Krizhevsky A, Sutskever I, Geoffrey HE, (2019) ImageNet Classification with Deep Convolutional Neural Networks. Adv Neural Inf Process Syst pp 1–9, https://doi.org/10.1109/5.726791

  24. Yao Y, Hu Q, Yu H, Grzymala-Busse JW (2019) Rough sets, fuzzy sets, data mining, and granular computing: 15th international conference, RSFDGrC 2019 Tianjin, China, November 20-23, In: 2015 proceedings. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics), vol 9437, https://doi.org/10.1007/978-3-319-25783-9

  25. Wang D, Shen Z, Shao J, Zhang W, Xue X, Zhang Z (2019) Multiple granularity descriptors for fine-grained categorization. In: Proceedings of the IEEE international conference on computer vision, 2015, pp 2399–2406, https://doi.org/10.1109/ICCV.2019.276

  26. Liu H, Li L, Wu C, Technology I (2019) Colar image segmentation algorthims based on granular computing clustering. 7(1): 155–168

  27. Missen MMS (2018) Combining granularity-based topic-dependent and topic-independent evidences for opinion detection, Université Paul Sabatier-Toulouse III

  28. Aplin P, Atkinson PM, Curran PJ (2022) Fine spatial resolution simulated satellite sensor imagery for land cover mapping in the United Kingdom. vol 4257, no. 98

  29. Zhang L, Xu F, Fraundorfer F (2022) Deep learning in remote sensing

  30. Szigarski C, Jagdhuber T, Baur M et al (2022) Analysis of the radar vegetation index and potential improvements. Remote Sens 10(11):1776

    Article  ADS  Google Scholar 

  31. Meher SK, Pal SK (2019) Rough-wavelet granular space and classification of multispectral remote sensing image. Appl Soft Comput J 11(8):5662–5673. https://doi.org/10.1016/j.asoc.2019.03.027

    Article  Google Scholar 

  32. Ren Y (2021) Applied sciences small object detection in optical remote sensing images via modified faster R-CNN. https://doi.org/10.3390/app8050813

  33. Ren Y, Zhu C, Xiao S (2021) Small object detection in optical remote sensing images via modified faster R-CNN. Appl Sci 8(5):813

    Article  Google Scholar 

  34. Yang Y, Newsam S (2020) Bag-of-visual-words and spatial extensions for land-use classification. In Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, pp 270–279, https://doi.org/10.1145/1869790.1869829

  35. Hu F, Xia G-S, Yang W, Zhang L (2018) Recent advances and opportunities in scene classification of aerial images with deep models. In: IGARSS 2018-2018 IEEE international geoscience and remote sensing symposium, pp 4371–4374, https://doi.org/10.1109/IGARSS.2020.8518336

  36. Jiang J, Wu D, Jiang Z (2020) A correlation-based bag of visual words for image classification. In: 2020 IEEE 3rd information technology and mechatronics engineering conference (ITOEC) , pp 891–894, https://doi.org/10.1109/ITOEC.2020.8122482

  37. Dong Y, Jiao W, Long T, He G, Gong C (2020) An extension of phase correlation-based image registration to estimate similarity transform using multiple polar Fourier transform. Remote Sens 10(11):1719. https://doi.org/10.3390/rs10111719

    Article  ADS  Google Scholar 

  38. Hu GX, Yang Z, Hu L, et al (2021) Small object detection with multiscale features. Int J Digital Multimed Broadcast, 1-10

  39. Dong Y, Jiao W, Long T et al (2021) An extension of phase correlation-based image registration to estimate similarity transform using multiple polar Fourier transform. Remote Sens 10(11):1719

    Article  ADS  Google Scholar 

  40. Ganaie MA, Hu M, Tanveer M, Suganthan PN (2021) Ensemble deep learning: a review, [Online]. Available: arXiv:2104.02395

  41. Ganaie M, Hu M (2021) Ensemble deep learning: a review,arXiv preprint Available: arXiv:2104.02395

  42. Vali A, Comai S, Matteucci M (2020) Deep learning for land use and land cover classification based on hyperspectral and multispectral earth observation data: A review. Remote Sens 12(15):2495. https://doi.org/10.3390/RS12152495

    Article  ADS  Google Scholar 

  43. Cao L, Liu Q, Yang Y (2019) An Unsupervised Feature Extraction Method based on Multi-granularity Convolution Denoising Autoencoder. In: 2019 IEEE/ACIS 18th International Conference on Computer and Information Science (ICIS). IEEE, pp. 118–123. https://doi.org/10.1109/ICIS46139.2019.8940316

  44. Liu Y, Zhong Y, Qin Q (2020) Scene classification based on multiscale convolutional neural network. IEEE Transact Geosci Remote Sens 56(12):7109–7121. https://doi.org/10.1109/TGRS.2020.2848473

    Article  ADS  Google Scholar 

  45. Kattenborn T, Eichel J, Fassnacht FE (2020) Convolutional Neural Networks enable efficient, accurate, and fine-grained segmentation of plant species and communities from high-resolution UAV imagery. Sci Rep 9(1):1–9. https://doi.org/10.1038/s41598-019-53797-9

    Article  CAS  Google Scholar 

  46. Srivastava S, Vargas Muñoz JE, Lobry S, Tuia D (2020) Fine-grained landuse characterization using ground-based pictures: a deep learning solution based on globally available data. Int J Geogr Inf Sci 00(00):1–20. https://doi.org/10.1080/13658816.2018.1542698

    Article  Google Scholar 

  47. Zheng H, Fu J, Mei T, Luo J, (2017) Learning multi-attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE international conference on computer vision, pp 5209–5217,https://doi.org/10.1109/ICCV.2020.557

  48. Huang S, Xu Z, Tao D, Zhang Y, (2020) Part-stacked cnn for fine-grained visual categorization. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  49. He K, Zhang X, Ren S, Sun J (2019) Deep residual learning for image recognition. https://doi.org/10.1109/CVPR.2019.90

  50. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2019) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 2019, pp 2818–2826, https://doi.org/10.1109/CVPR.2019.308

  51. Bosco MJ, Wang G, Hategekimana Y (2021) Learning multi-granularity neural network encoding image classification using DCNNs for Easter Africa Community Countries. IEEE Access 9:146703–146718. https://doi.org/10.1109/ACCESS.2021.3122569

Download references

Acknowledgements

The work was published with the financial support of the Ministry of Science and Higher Education of the Russian Federation, Priority – 2030. Also, Ahmed A. Abd El-Latif would like to thank Prince Sultan University for their support.

Author information

Authors and Affiliations

  1. School of Science and Technology, Department of Computer Science and Technology, Kigali Independent University (ULK), 2280, Kigali, Rwanda

    Musabe Jean Bosco, Rutarindwa Jean Pierre & Kwizera Jean Pierre

  2. Institute of Computer Technologies and Information Security, Southern Federal University, 347922, Taganrog, Russia

    Mohammed Saleh Ali Muthanna

  3. Peoples’ Friendship University of Russia (RUDN University), 6 Miklukho-Maklaya, 117198, Moscow, Russia

    Ammar Muthanna

  4. EIAS Data Science Lab, College of Computer and Information Sciences, and with Center of Excellence in Quantum and Intelligent Computing, Prince Sultan University, Riyadh, 11586, Saudi Arabia

    Ahmed A. Abd El-Latif

  5. Department of Mathematics and Computer Science, Faculty of Science, Menoufia University, 32511, Shebeen El-Kom, Egypt

    Ahmed A. Abd El-Latif

Authors
  1. Musabe Jean Bosco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rutarindwa Jean Pierre
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Saleh Ali Muthanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kwizera Jean Pierre
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ammar Muthanna
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ahmed A. Abd El-Latif
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed A. Abd El-Latif.

Ethics declarations

Conflict of interest

This work has no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jean Bosco, M., Jean Pierre, R., Muthanna, M.S.A. et al. MGFEEN: a multi-granularity feature encoding ensemble network for remote sensing image classification. Neural Comput & Applic 36, 6547–6558 (2024). https://doi.org/10.1007/s00521-023-09383-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09383-2

Keywords

Search

Navigation