Abstract
In high-resolution remote sensing imageries, the scene classification is one of the challenging problems due to the similarity of image structure and available datasets are all small. Performing training with small datasets on new convolutional neural network (CNN) is inclined to overfitting, and attainability is poor. To overcome this, we go for a stream of transfer learning, fine-tuning strategy. Here, we consider AlexNet, VGG 19, and VGG 16 pre-trained CNNs. First, design a network by replacing the classifier stage layers with revised ones through transfer learning. Second, apply fine-tuning from right to left and perform retraining on the classifier stage and part of the feature extraction stage (last convolutional block). Third, form a classifier ensemble by using the majority voting learner strategy to explore better classification results. The datasets called UCM and SIRI-WHU were used and compared with the state-of-the-art methods. Finally, to check the usefulness of our proposed methods, form sub-datasets from AID and WHU-RS19 datasets with likely labeled class names. To assess the performance of the proposed classifiers compute overall accuracy using confusion matrix and F1-score. The results of the proposed methods improve the accuracy from 93.57 to 99.04% for UCM and 91.34 to 99.16% for SIRI-WHU.


















Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets used in this article are publicly available and corresponding hyperlinks mentioned in the references.
Change history
04 November 2024
This article has been retracted. Please see the Retraction Notice for more detail: https://doi.org/10.1007/s00500-024-10266-4
References
Aerial Image Dataset(AID) get from (2021) https://captain-whu.github.io/AID/
Bahmanyar R, Cui S, Datcu M (2015) A comparative study of bag-of-words and bag-of-topics models of EO image patches. IEEE Geosci Remote Sens Lett 12(6):1357–1361
Basha SS, Vinakota SK, Pulabaigari V, Mukherjee S, Dubey SR (2021) Autotune: automatically tuning convolutional neural networks for improved transfer learning. Neural Netw 133:112–122
Boland PJ (2020) Majority systems and the Condorcet jury theorem. J R Stat Soc 38(3):181–189
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Procedings of the conference on computer vision and pattern recognition, San Diego, CA, USA, pp 886–893
Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) “DeCAF: a deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st international conference on machine learning, Beijing, China, pp 647–655
Gao Z, Xie J, Wang Q, Li P (2019) Global second-order pooling convolutional networks. In: Proceedings of the IEEE conference on computer vision pattern recognition, Long Beach, CA, USA, pp 3024–3033
Hinton GE, Osindero S, The YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554
Hu F, Xia G, Hu J, Zhang L (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens 7(11):14680–14707
Karlos S, Kostopoulos G, Kotsiantis S (2020) A soft-voting ensemble based co-training scheme using static selection for binary classification problems. Algorithms 13(1):26
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90
Kuncheva LI (2014) Combining pattern classifiers: methods and algorithms. Wiley, New York
Lazebnik S, Schmid C, Ponce J (2009) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), vol 2, pp 2169–2178
Li X, Wang Q, Liu S, Chanussot J (2018) Scene classification with recurrent attention of VHR remote sensing images. IEEE Trans Geosci Remote Sens 57(2):1155–1167
Li L, Liang P, Ma J, Jiao L, Guo X, Liu F, Sun C (2020) A multiscale self-adaptive attention network for remote sensing scene classification. Remote Sens 12(14):1527–1554
Lienou M, Maitre H, Datcu M (2010) Semantic annotation of satellite images using latent dirichlet allocation. IEEE Geosci Remote Sens Lett 7(1):28–32
Liu Y, Zhong Y, Fei F, Zhu Q, Qin Q (2018) Scene classification based on a deep random-scale stretched convolutional neural network. Remote Sens 10(3):444
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput vis 60(2):91–110
Lv Y, Zhang X, Xiong W, Cui Y, Cai M (2019) An end-to-end local-global-fusion feature extraction network for remote sensing image scene classification. Remote Sens 11(24):3006
Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987
Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33(8):2395–2412
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the international conference learning representation (ICLR), San Diego, CA, USA, pp 1–14
SIRI-WHU Data Set (2020). http://www.lmars.whu.edu.cn/prof_web/zhongyanfei/Num/Google.html
Szegedy C, Liu W, Jia Y, Vanhoucke V (2015) Going deeper with convolutions. In: Proceedings IEEE conference computer vision pattern recognition (CVPR), pp 1–9
UC Merced Data Set (2019). http://weegee.vision.ucmerced.edu/datasets/landuse.html
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol P (2010) Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11(12):3371–3408
Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2019) ECA-Net: efficient channel attention for deep convolutional neural networks. arXiv:1910.03151
WHU-RS 19 Dataset (2021) http://dsp.whu.edu.cn/cn/staff/yw/HRSscene.html, http://captain.whu.edu.cn/datasets/WHU-RS19.zip
Wu J, Cai Z (2011) Attribute weighting via differential evolution algorithm for attribute weighted naive bayes (wnb). J Comput Inf Syst 7(4):1672–1679
Wu F, Wang C, Zhang B, Zhang H, Gong L (2019) Discrimination of collapsed buildings from remote sensing imagery using deep neural networks. In: IGARSS IEEE geoscience remote sensing symposium, pp 2646–2649
Wu Q, Wu B, Hu C, Yan X (2021) Evolutionary multilabel classification algorithm based on cultural algorithm. Symmetry 13(2):322
Xia G-S, Hu J, Hu F, Shi B, Bai X, Zhong Y, Zhang L, Lu X (2017) Aid: A benchmark data set for performance evaluation of aerial scene classification. IEEE Trans Geosci Remote Sens 55(7):3965–3981
Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the international conference on ACM SIGSPATIAL GIS, San Jose, CA, USA, pp 270–279
Zeiler MD, Fergus R (2019) Visualizing and understanding convolutional networks. In: Proceedings of the European conference on computer vision. Springer, Cham, pp 818–833
Zhang F, Du B, Zhang L (2015) Scene classification via a gradient boosting random convolutional network framework. IEEE Trans Geosci Remote Sens 54(3):1793–1802
Zhao B, Zhong Y, Xia G-S, Zhang L (2016) Dirichlet-derived multiple topic scene classification model for high spatial resolution remote sensing imagery. IEEE Trans Geosci Remote Sens 54(4):2108–2123
Zhou W, Newsam S, Li C, Shao Z (2018) PatternNet: a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS J Photogram Remote Sens 145:197–209
Funding
The authors received no specific funding for this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Authors don’t have any conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article has been retracted. Please see the retraction notice for more detail: https://doi.org/10.1007/s00500-024-10266-4
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Thirumaladevi, S., Veera Swamy, K. & Sailaja, M. RETRACTED ARTICLE: Improved transfer learning of CNN through fine-tuning and classifier ensemble for scene classification. Soft Comput 26, 5617–5636 (2022). https://doi.org/10.1007/s00500-022-07145-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07145-1