Skip to main content
SpringerLink
Log in

Deep Feature Fusion for High-Resolution Aerial Scene Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The rapid development of remote sensing technology let us acquire a large collection of remote sensing scene images with high resolution. Aerial scene classification has become a crucial problem for understanding high-resolution remote sensing imagery. In this letter, we propose a novel framework for aerial scene classification. Unlike some traditional methods in which the features are produced by using handcrafted feature descriptors, our proposed method uses the raw RGB network stream and the saliency coded network stream to extract two different types of informative features. Then, we further propose a deep feature fusion model to fuse these two sets of features for final classification. The comprehensive performance evaluation of our proposed method is tested on two publicly available remote sensing scene classification benchmarks, i.e., the UC-Merced dataset and the AID dataset. Experimental results show that our proposed method achieves satisfactory results and outperforms the state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Anwer RM, Khan FS, van de Weijer J, Molinier M, Laaksonen J (2018) Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. arXiv preprint arXiv:1706.01171v2

  2. Bian X, Chen C, Sheng Y, Xu Y, Du Q (2017) Fusing two convolutional neural networks for high-resolution scene classification. In: 2017 IEEE International geoscience and remote sensing symposium (IGARSS). IEEE, pp 3242–3245

  3. Bian X, Chen C, Tian L, Du Q (2017) Fusing local and global features for high-resolution scene classification. IEEE J Sel Top Appl Earth Obs Remote Sens 10:2889–2901

    Google Scholar 

  4. Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3(Jan):993–1022

    Google Scholar 

  5. Bosch A, Zisserman A, Muñoz X (2006) Scene classification via pLSA. In: Computer vision—ECCV 2006, pp 517–530

  6. Chaib S, Liu H, Gu Y, Yao H (2017) Deep feature fusion for VHR remote sensing scene classification. IEEE Trans Geosci Remote Sens 55:4775–4784

    Google Scholar 

  7. Ibarrola-Ulzurrun E, Marcello J, Gonzalo-Martin C (2018) Advanced classification of remote sensing high resolution imagery. an application for the management of natural resources. In: Rocha Á (ed) Developments and advances in intelligent systems and applications. Springer, Berlin, pp 1–13

    Google Scholar 

  8. Jegou H, Perronnin F, Douze M, Sánchez J, Perez P, Schmid C (2012) Aggregating local image descriptors into compact codes. IEEE Trans Pattern Anal Mach Intell 34(9):1704–1716

    Google Scholar 

  9. Ji W, Li X, Lu X (2017) Bidirectional adaptive feature fusion for remote sensing scene classification. In: CCF Chinese conference on computer vision. Springer, Berlin, pp 486–497

  10. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 675–678

  11. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp 2169–2178

  12. Liu C, Wechsler H (2001) A shape-and texture-based enhanced Fisher classifier for face recognition. IEEE Trans Image Process 10(4):598–608

    Google Scholar 

  13. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Google Scholar 

  14. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Google Scholar 

  15. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175

    Google Scholar 

  16. Perronnin F, Sánchez J, Mensink T (2010) Improving the fisher kernel for large-scale image classification. In: Computer vision—ECCV 2010, pp 143–156

  17. Ranganath C, Rainer G (2003) Cognitive neuroscience: neural mechanisms for detecting and remembering novel events. Nat Rev Neurosci 4(3):193

    Google Scholar 

  18. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Google Scholar 

  19. Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33(8):2395–2412

    Google Scholar 

  20. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  21. Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: IEEE International conference on computer vision. IEEE, p 1470

  22. Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32

    Google Scholar 

  23. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  24. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE, pp 3360–3367

  25. Wang Y, Zhang L, Tong X, Nie F, Huang H, Mei J (2018) LRAGE: learning latent relationships with adaptive graph embedding for aerial scene classification. IEEE Trans Geosci Remote Sens 56(2):621–634

    Google Scholar 

  26. Weng Q, Mao Z, Lin J, Guo W (2017) Land-use classification via extreme learning classifier based on deep convolutional features. IEEE Geosci Remote Sens Lett 14(5):704–708

    Google Scholar 

  27. Xia GS, Hu J, Hu F, Shi B, Bai X, Zhong Y, Zhang L, Lu X (2017) AID: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55:3965–3981

    Google Scholar 

  28. Yang J, Yang JY, Zhang D, Lu JF (2003) Feature fusion: parallel strategy versus serial strategy. Pattern Recognit 36(6):1369–1381

    Google Scholar 

  29. Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. ACM, pp 270–279

  30. Yu Y, Liu F (2018) Aerial scene classification via multilevel fusion based on deep convolutional neural networks. IEEE Geosci Remote Sens Lett 15(2):287–291

    Google Scholar 

  31. Yu Y (2018) Liu F (2018) A two-stream deep fusion framework for high-resolution aerial scene classification. Comput Intell Neurosci 2018:1–13

    Google Scholar 

  32. Zheng Z, Zhang T, Yan L (2012) Saliency model for object detection: searching for novel items in the scene. Opt Lett 37(9):1580–1582

    Google Scholar 

  33. Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12(11):2321–2325

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Jingchu University of Technology, Jingmen, China

    Heng Wang

  2. Air Force Enginnering University, Xi’an, China

    Yunlong Yu

Authors
  1. Heng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunlong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heng Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Yu, Y. Deep Feature Fusion for High-Resolution Aerial Scene Classification. Neural Process Lett 51, 853–865 (2020). https://doi.org/10.1007/s11063-019-10119-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10119-4

Keywords

Search

Navigation