Abstract
Deep learning (DL) algorithms have shown great potential in classifying satellite imagery but require large amounts of labeled data to make accurate predictions. However, generating large amounts of labeled data is time-consuming, costly, and can be problematic in the case of limited or imbalanced datasets. Data augmentation techniques have been proposed to improve the accuracy and robustness of DL models for satellite image classification. This paper presents a new approach to automated satellite data augmentation leveraging Generative Adversarial Networks (GANs) assisted with Vision Transformers (ViT) and evaluating its effectiveness on satellite image classification. The proposed approach is divided into two main steps: data augmentation using GAN-based transformers and satellite image classification. The GAN generates new images by learning the statistical distribution of the original images and generating new images that are similar to the original ones. ViT are used to learn the images’ features and improve the classification task’s accuracy. The performance of the proposed approach is evaluated through extensive experiments on real-world datasets. The proposed approach achieves an accuracy increase from 76.9% with traditional data augmentation to 98.7%. This is a significant improvement demonstrating the proposed approach’s effectiveness in enhancing the accuracy of satellite image classification.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
Data will be available upon request to the corresponding author.
References
Adedeji O, Owoade P, Ajayi O, Arowolo O (2022) Image augmentation for satellite images., arXiv preprint arXiv:2207.14580
Alzahem A, Boulila W, Driss M, Koubaa A, Almomani I (2022) Towards optimizing malware detection: An approach based on generative adversarial networks and transformers., in: Conference on Computational Collective Intelligence Technologies and Applications., Springer, pp. 598–610. https://doi.org/10.1007/978-3-031-16014-1_47
Boulila W, Ayadi Z, Farah IR (2017) Sensitivity analysis approach to model epistemic and aleatory imperfection: Application to land cover change prediction model. J Comput Sci 23:58–70
Boulila W, Farah IR, Ettabaa KS, Solaiman B, Ghézala HB (2009) Improving spatiotemporal change detection: A high level fusion approach for discovering uncertain knowledge from satellite image databases, in: Icdm, Vol. 9, Citeseer, pp. 222–227
Boulila W, Farah IR, Ettabaa KS, Solaiman B, Ghézala HB (2010) Spatio-temporal modeling for knowledge discovery in satellite image databases., in: CORIA, pp. 35–49
Brigato L, Barz B, Iocchi L, Denzler J (2022) Image classification with small datasets: Overview and benchmark., IEEE Access. https://doi.org/10.1109/ACCESS.2022.3172939
Chatterjee S, Hazra D, Byun Y-C, Kim Y-W (2022) Enhancement of image classification using transfer learning and gan-based synthetic data augmentation. Mathematics 10(9):1541. https://doi.org/10.3390/math10091541
Chen F, Tsou JY (2022) Assessing the effects of convolutional neural network architectural factors on model performance for remote sensing image classification: An in-depth investigation. Int J Appl Earth Observ Geoinf 112:102865. https://doi.org/10.1016/j.jag.2022.102865
Chen J, Wang L, Feng R, Liu P, Han W, Chen X (2020) Cyclegan-stf: Spatiotemporal fusion via cyclegan-based image generation. IEEE Trans Geosci Remote Sens 59(7):5851–5865. https://doi.org/10.1109/TGRS.2020.3023432
Cheng X, He X, Qiao M, Li P, Hu S, Chang P, Tian Z (2022) Enhanced contextual representation with deep neural networks for land cover classification based on remote sensing images. Int J Appl Earth Observ Geoinf 107:102706. https://doi.org/10.1016/j.jag.2022.102706
Chlap P, Min H, Vandenberg N, Dowling J, Holloway L, Haworth A (2021) A review of medical image data augmentation techniques for deep learning applications. J Medi Imaging Radiat Oncol 65(5):545–563
Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath AA (2018) Generative adversarial networks: An overview. IEEE Signal Process Mag 35(1):53–65
Daneshfar F, Jamshidi MB (2023) An octonion-based nonlinear echo state network for speech emotion recognition in metaverse. Neural Netw 163:108–121
Dimitrovski I, Kitanovski I, Kocev D, Simidjievski N (2023) Current trends in deep learning for earth observation: An open-source benchmark arena for image classification. ISPRS J Photogramm Remote Sens 197:18–35. https://doi.org/10.1016/j.isprsjprs.2023.01.014
Dutta AK, Alsanea M, Qureshi B, Alghayadh FY, Sait ARW (2022) Intelligent rider optimization algorithm with deep learning enabled hyperspectral remote sensing imaging classification. Can J Remote Sens 48(5):649–662
Ferchichi A, Boulila W, Farah IR (2017) Propagating aleatory and epistemic uncertainty in land cover change prediction process. Ecological Inform 37:24–37
Ferchichi A, Boulila W, Farah IR (2018) Reducing uncertainties in land cover change models using sensitivity analysis. Knowl Inf Syst 55:719–740
Frid-Adar M, Klang E, Amitai M, Goldberger J, Greenspan H (2018) Synthetic data augmentation using gan for improved liver lesion classification., in: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018)., IEEE, pp. 289–293. https://doi.org/10.1109/ISBI.2018.8363576
Generative adversarial transformers (2023) https://paperswithcode.com/paper/generative-adversarial-transformers
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144. https://doi.org/10.1145/3422622
Huang S-W, Lin C-T, Chen S-P, Wu Y-Y, Hsu P-H, Lai S-H (2018) Auggan: Cross domain adaptation with gan-based data augmentation., in: Proceedings of the European Conference on Computer Vision (ECCV). pp. 718–731
Hudson DA, Zitnick L (2021) Generative adversarial transformers., in: International conference on machine learning., PMLR, pp. 4487–4499
Jamshidi MB, Daneshfar F (2022) A hybrid echo state network for hypercomplex pattern recognition, classification, and big data analysis, in: 2022 12th International Conference on Computer and Knowledge Engineering (ICCKE), IEEE, pp. 007–012
Karras T, Aittala M, Hellsten J, Laine S, Lehtinen J, Aila T (2020) Training generative adversarial networks with limited data. Adv Neural Inf Process Syst 33:12104–12114
Karras T, Aittala M, Laine S, Härkönen E, Hellsten J, Lehtinen J, Aila T (2021) Alias-free generative adversarial networks. Adv Neural Inf Process Syst 34:852–863
Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks., in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401–4410
Khalaj O, Jamshidi M, Hassas P, Hosseininezhad M, Mašek B, Štadler C, Svoboda J (2022) Metaverse and ai digital twinning of 42sicr steel alloys. Mathematics 11(1):4
Khan MZ, Jabeen S, Khan MUG, Saba T, Rehmat A, Rehman A, Tariq U (2020) A realistic image generation of face from text description using the fully trained generative adversarial networks. IEEE Access 9:1250–1260
Kukreja V, Kumar D, Kaur A et al (2020) Gan-based synthetic data augmentation for increased cnn performance in vehicle number plate recognition., in: 2020 4th international conference on electronics, communication and aerospace technology (ICECA)., IEEE, pp. 1190–1195. https://doi.org/10.1109/ICECA49313.2020.9297625
X. Li, G. Zhang, H. Cui, S. Hou, S. Wang, X. Li, Y. Chen, Z. Li, L. Zhang (2022) Mcanet: A joint semantic segmentation framework of optical and sar images for land use classification., International Journal of Applied Earth Observation and Geoinformation. 106:102638. https://doi.org/10.1016/j.jag.2021.102638
Lu Y, Chen D, Olaniyi E, Huang Y (2022) Generative adversarial networks (gans) for image augmentation in agriculture: A systematic review. Comput Electron Agric 200:107208. https://doi.org/10.1016/j.compag.2022.107208
Mariani G, Scheidegger F, Istrate R, Bekas C, Malossi C (2018) Bagan: Data augmentation with balancing gan., arXiv preprint arXiv:1803.09655
Mikołajczyk A, Grochowski M (2018) Data augmentation for improving deep learning in image classification problem, in: 2018 international interdisciplinary PhD workshop (IIPhDW), IEEE, pp. 117–122
Mirza M, Osindero S (2014) Conditional generative adversarial nets, arXiv preprint arXiv:1411.1784
Perez L, Wang J (2017) The effectiveness of data augmentation in image classification using deep learning, arXiv preprint arXiv:1712.04621
Sabry ES, Elagooz S, Abd El-Samie FE, El-Shafai W, El-Bahnasawy NA, El Banby G, Algarni AD, Soliman NF, Ramadan RA (2023) Image retrieval using convolutional autoencoder, infogan, and vision transformer unsupervised models, IEEE Access
Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48
Talukdar S, Singha P, Mahato S, Pal S, Liou Y-A, Rahman A (2020) Land-use land-cover classification by machine learning classifiers for satellite observations-a review. Remote Sens 12(7):1135. https://doi.org/10.3390/rs12071135
Tarasiou M, Chavez E, Zafeiriou S (2023) Vits for sits: Vision transformers for satellite image time series. http://arxiv.org/abs/2301.04944
Tran N-T, Tran V-H, Nguyen N-B, Nguyen T-K, Cheung N-M (2021) On data augmentation for gan training. IEEE Trans Image Process 30:1882–1897. https://doi.org/10.1109/TIP.2021.3049346
Wambugu N, Chen Y, Xiao Z, Tan K, Wei M, Liu X, Li J (2021) Hyperspectral image classification on insufficient-sample and feature learning using deep neural networks: A review. Int J Appl Earth Observ Geoinf 105:102603. https://doi.org/10.1016/j.jag.2021.102603
Zhu X, Liu Y, Li J, Wan T, Qin Z (2018) Emotion classification with data augmentation using generative adversarial networks., in: Pacific-Asia conference on knowledge discovery and data mining., Springer, pp. 349–360. https://doi.org/10.1007/978-3-319-93040-4_28
Acknowledgements
The authors would like to thank Prince Sultan University for their support.
Funding
This research received no external funding.
Author information
Authors and Affiliations
Contributions
Conceptualization, A.A., W.B.; methodology, A.A. and W.B.; validation, A.A., W.B. and A.K.; formal analysis, A.A., W.B. and A.K.; investigation, A.A. and W.B.; resources, A.A., W.B. and A.K.; data curation, A.A and W.B.; writing-original draft preparation, A.A. and W.B.; writing-review and editing, A.A., W.B., A.K., Z.K. and I.A. All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests
Not applicable.
Additional information
Communicated by: H. Babaie.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Alzahem, A., Boulila, W., Koubaa, A. et al. Improving satellite image classification accuracy using GAN-based data augmentation and vision transformers. Earth Sci Inform 16, 4169–4186 (2023). https://doi.org/10.1007/s12145-023-01153-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-023-01153-x