Skip to main content
Springer Nature Link
Log in

Deep convolution neural network with automatic attribute profiles for hyperspectral image classification

  • 1166: Advances of machine learning in data analytics and visual information
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Performance of deep convolutional neural network (CNN) has shown tremendous improvement when applied to various image classifications including hyperspectral images (HSIs). However, CNN requires a large number of labeled samples to train its parameters in different layers, the scarcity of which, in HSIs leads to overfitting problem. Prior integration of spectral-spatial information acts complementary to the deep features resulting in reduced computational load of deep CNN and also helps in mitigating overfitting problem. In this paper, we propose a CNN based classification model that first integrates spectral-spatial information by using an extended attribute profile constructed by selecting suitable threshold values automatically. Then, the constructed spectral-spatial features are utilized by 2D or 3D deep CNN models for classification. Experimental results on three real HSI data sets show that the proposed model can successfully integrate the individual strength of both the automatic extended attribute profile and deep CNN, and provide better classification accuracies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. Available at: http://www.ehu.eus/ccwintco/index.php?title=Hyperspectral_Remote_Sensing_Scenes

  2. Available at:http://hyperspectral.ee.uh.edu/?page_id=459

References

  1. Aptoula E, Can Ozdemir M, Yanikoglu B (2016) Deep learning with attribute profiles for hyperspectral image classification. IEEE Geosci Remote Sens Lett 13(12):1970–1974

    Article  Google Scholar 

  2. Benediktsson JA, Aevar Palmason J, R Sveinsson J (2005) Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Trans Geosci Remote Sens 43(3):480–491

    Article  Google Scholar 

  3. Bhardwaj K, Patra S (2018) An unsupervised technique for optimal feature selection in attribute profiles for spectral-spatial classification of hyperspectral images. ISPRS J Photogramm Remote Sens 138:139–150

    Article  Google Scholar 

  4. Bhardwaj K, Patra S, Bruzzone L (2019) Threshold-free attribute profile for classification of hyperspectral images. IEEE Trans Geosci Remote Sens 57(10):7731–7742

    Article  Google Scholar 

  5. Cao C, Deng L, Duan W, Xiao F, Yang W, Hu K (2019) Hyperspectral image classification via compact-dictionary-based sparse representation. Multimedia Tools and Applications 78(11):15011–15031

    Article  Google Scholar 

  6. Cavallaro G, Falco N, Dalla Mura M, Benediktsson JA (2017) Automatic attribute profiles. IEEE Trans Image Process 26(4):1859–1872

    Article  MathSciNet  Google Scholar 

  7. Chen Y, Lin Z, Zhao X, Wang G, Gu Y (2014) Deep learning-based classification of hyperspectral data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7(6):2094–2107

    Article  Google Scholar 

  8. Chen Y, Zhao X, Jia X (2015) Spectral–spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 8(6):2381–2392

    Article  Google Scholar 

  9. Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54(10):6232–6251

    Article  Google Scholar 

  10. Chen Y, Zhu L, Ghamisi P, Jia X, Li G, Tang L (2017) Hyperspectral images classification with Gabor filtering and convolutional neural network. IEEE Geosci Remote Sens Lett 14(12):2355–2359

    Article  Google Scholar 

  11. Dalla Mura M, Benediktsson JA, Waske B, Bruzzone L (2010) Extended profiles with morphological attribute filters for the analysis of hyperspectral data. Int J Remote Sens 31(22):5975–5991

    Article  Google Scholar 

  12. Dalla Mura M, Benediktsson JA, Waske B, Bruzzone L (2010) Morphological attribute profiles for the analysis of very high resolution images. IEEE Trans Geosci Remote Sens 48(10):3747–3762

    Article  Google Scholar 

  13. Das A, Patra S (2020) A rough-GA based optimal feature selection in attribute profiles for classification of hyperspectral imagery. Soft Comput 24:12569–12585

    Article  Google Scholar 

  14. Das A, Bhardwaj K, Patra S (2018) Morphological complexity profile for the analysis of hyperspectral images. In: 2018 4th international conference on recent advances in information technology (RAIT), IEEE, pp 1–6

  15. Das A, Bhardwaj K, Patra S (2019) Automatic attribute profiles for spectral-spatial classification of hyperspectral images. In: International conference on pattern recognition and machine intelligence, Springer, pp 13–21

  16. Ghamisi P, Benediktsson JA, Sveinsson JR (2014) Automatic spectral–spatial classification framework based on attribute profiles and supervised feature extraction. IEEE Trans Geosci Remote Sens 52(9):5771–5782

    Article  Google Scholar 

  17. Ghamisi P, Dalla Mura M, Benediktsson JA (2015) A survey on spectral–spatial classification techniques based on attribute profiles. IEEE Trans Geosci Remote Sens 53(5):2335–2353

    Article  Google Scholar 

  18. Ghamisi P, Plaza J, Chen Y, Li J, J Plaza A (2017) Advanced spectral classifiers for hyperspectral images: a review. IEEE Geoscience and Remote Sensing Magazine 5(1):8–32

    Article  Google Scholar 

  19. Ghamisi P, Maggiori E, Li S, Souza R, Tarablaka Y, Moser G, De Giorgi A, Fang L, Chen Y, Chi M, et al. (2018) New frontiers in spectral-spatial hyperspectral image classification: the latest advances based on mathematical morphology, Markov random fields, segmentation, sparse representation, and deep learning. IEEE Geoscience and Remote Sensing Magazine 6 (3):10–43

    Article  Google Scholar 

  20. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

    MATH  Google Scholar 

  21. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  22. Holzwarth S, Muller A, Habermeyer M, Richter R, Hausold A, Thiemann S, Strobl P (2003) HySens-DAIS 7915/ROSIS imaging spectrometers at DLR. In: Proceedings of the 3rd EARSeL workshop on imaging spectroscopy, pp 3–14

  23. Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors 2015:1–12. Article ID 3632943

    Article  Google Scholar 

  24. IEEE GRSS (2013) Image analysis and data fusion. [online]. Available: http://www.grss-ieeeorg/community/technicalcommittees/data-fusion/

  25. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  26. Lagrange A, Le Saux B, Beaupere A, Boulch A, Chan-Hon-Tong A, Herbin S, Randrianarivo H, Ferecatu M (2015) Benchmarking classification of earth-observation data: from learning explicit features to convolutional networks. In: Geoscience and remote sensing symposium (IGARSS), 2015 IEEE international, IEEE, pp 4173–4176

  27. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  28. Li S, Song W, Fang L, Chen Y, Ghamisi P, Benediktsson JA (2019) Deep learning for hyperspectral image classification: an overview. IEEE Trans Geosci Remote Sens 57(9):6690–6709

    Article  Google Scholar 

  29. Li T, Zhang J, Zhang Y (2014) Classification of hyperspectral image based on deep belief networks. In: IEEE International conference on image processing (ICIP), 2014, IEEE, pp 5132–5136

  30. Li W, Wu G, Zhang F, Du Q (2017) Hyperspectral image classification using deep pixel-pair features. IEEE Trans Geosci Remote Sens 55(2):844–853

    Article  Google Scholar 

  31. Li Y, Zhang H, Shen Q (2017) Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens 9(1):67

    Article  Google Scholar 

  32. Liang H, Li Q (2016) Hyperspectral imagery classification using sparse representations of convolutional neural network features. Remote Sens 8 (2):99

    Article  Google Scholar 

  33. Lin Z, Chen Y, Zhao X, Wang G (2013) Spectral-spatial classification of hyperspectral image using autoencoders. In: 2013 9th international conference on information, communications & signal processing, IEEE, pp 1–5

  34. Mahmood Z, Thoonen G, Scheunders P (2012) Automatic threshold selection for morphological attribute profiles. In: Geoscience and remote sensing symposium (IGARSS), 2012 IEEE international, IEEE, pp 4946–4949

  35. Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: Geoscience and remote sensing symposium (IGARSS), 2015 IEEE international, IEEE, pp 4959–4962

  36. Mario Haut J, E Paoletti M, Plaza J, Li J, Plaza A (2018) Active learning with convolutional neural networks for hyperspectral image classification using a new bayesian approach. IEEE Trans Geosci Remote Sens 56(11):6440–6461

    Article  Google Scholar 

  37. Marpu PR, Pedergnana M, Dalla Mura M, Benediktsson JA, Bruzzone L (2013) Automatic generation of standard deviation attribute profiles for spectral–spatial classification of remote sensing data. IEEE Geosci Remote Sens Lett 10 (2):293–297

    Article  Google Scholar 

  38. Paoletti M, Haut J, Plaza J, Plaza A (2018) A new deep convolutional neural network for fast hyperspectral image classification. ISPRS J Photogramm Remote Sens 145:120–147

    Article  Google Scholar 

  39. Pedergnana M, Reddy Marpu P, Dalla Mura M, Benediktsson JA, Bruzzone L (2013) A novel technique for optimal feature selection in attribute profiles based on genetic algorithms. IEEE Trans Geosci Remote Sens 51(6):3514–3528

    Article  Google Scholar 

  40. Salembier P, Oliveras A, Garrido L (1998) Antiextensive connected operators for image and sequence processing. IEEE Trans Image Process 7(4):555–570

    Article  Google Scholar 

  41. Singhal V, Aggarwal HK, Tariyal S, Majumdar A (2017) Discriminative robust deep dictionary learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55(9):5274–5283

    Article  Google Scholar 

  42. Xing C, Ma L, Yang X (2016) Stacked denoise autoencoder based feature extraction and classification for hyperspectral images. Journal of Sensors 2016:1–10. Article ID 3632943

    Google Scholar 

  43. Yang X, Ye Y, Li X, Lau RY, Zhang X, Huang X (2018) Hyperspectral image classification with deep learning models. IEEE Trans Geosci Remote Sens 56(9):5408–5423

    Article  Google Scholar 

  44. Yue J, Zhao W, Mao S, Liu H (2015) Spectral–spatial classification of hyperspectral images using deep convolutional neural networks. Remote Sens Lett 6(6):468–477

    Article  Google Scholar 

  45. Zabalza J, Ren J, Zheng J, Zhao H, Qing C, Yang Z, Du P, Marshall S (2016) Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing 185:1–10

    Article  Google Scholar 

  46. Zhang L, Zhang L, Du B (2016) Deep learning for remote sensing data: a technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine 4(2):22–40

    Article  Google Scholar 

  47. Zhang X, Pan Z, Lu X, Hu B, Zheng X (2018) Hyperspectral image classification based on joint spectrum of spatial space and spectral space. Multimedia Tools and Applications 77(22):29759–29777

    Article  Google Scholar 

  48. Zhao W, Du S (2016) Spectral–spatial feature extraction for hyperspectral image classification: a dimension reduction and deep learning approach. IEEE Trans Geosci Remote Sens 54(8):4544–4554

    Article  Google Scholar 

  49. Zhao W, Guo Z, Yue J, Zhang X, Luo L (2015) On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery. Int J Remote Sens 36(13):3368–3379

    Article  Google Scholar 

  50. Zhong P, Gong Z, Li S, Schönlieb C B (2017) Learning to diversify deep belief networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55(6):3516–3530

    Article  Google Scholar 

  51. Zhu L, Chen Y, Ghamisi P, Benediktsson JA (2018) Generative adversarial networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 56(9):5046–5063

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the RPS-NER Research Grant from the All India Council for Technical Education, New Delhi, India. Authors would like to thank Dr. S. Prasad for providing University of Houston data set and Dr. P. Ghamisi for providing standard training and test sets of the data sets used in the experiments.

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, Tezpur University, Assam, 784028, India

    Arundhati Das & Swarnajyoti Patra

  2. Computer Science and Engineering Department, Indian Institute of Information Technology Senapati, Manipur, 795002, India

    Kaushal Bhardwaj

Authors
  1. Arundhati Das
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kaushal Bhardwaj
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Swarnajyoti Patra
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Swarnajyoti Patra.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Das, A., Bhardwaj, K. & Patra, S. Deep convolution neural network with automatic attribute profiles for hyperspectral image classification. Multimed Tools Appl 80, 35365–35385 (2021). https://doi.org/10.1007/s11042-020-10169-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10169-0

Keywords