Skip to main content
Springer Nature Link
Log in

AA-TransDeeplabv3 + : a novel semantic segmentation framework for aerial images using adaptive and attentive based Transdeeplabv3 + with hybrid optimization technique

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Aerial imagesemantic segmentation is crucial for various operations such as military observation, land classification, and disaster impact assessments involving unmanned aerial vehicles. Although existing system is unsuited for aerial applications, these algorithms are mostly trained on human-centric datasets like “Cityscapes and Cam Vid”. High-resolution aerial image semantic segmentation is a basic and difficult task with several applications. Even though numerous Convolution Neural Network (CNN) segmentation techniques have shown impressive results, it is still challenging to discriminate semantic parts among regions with comparable spectral properties employing only high-resolution data. Additionally, the typical data-independent up-sampling techniques could produce poor outcomes. Thus, a novel semantic segmentation technique is introduced to resolve the complication presented in the classical segmentation framework in aerial images by utilizing deep learning techniques. Here, an Adaptive and Attentive based TransDeeplabv3 + (AA-TransDeeplabv3 +)-based semantic segmentation model for input images is designed with a novel Hybridized Fire Hawk with Electric Fish Optimization (HFH-EFO). The parameters of Attentive-based TransDeeplabv3 + are tuned by developed HFH-EFO to attain the multi-objective function. The model is implemented using Python, which generates the empirical results. Therefore, the developed method achieves a dice coefficient of 93.02% and an accuracy value of 93.01%, outperforming traditionalapproaches. Hence, the proposedframework secures anexcellent result than the classical technique based on experimental analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1:
Algorithm 2:
Algorithm 3:
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study. In case of benchmark data: The data underlying this article are available in Semantic segmentation of aerial imagery “https://www.kaggle.com/datasets/humansintheloop/semantic-segmentation-of-aerial-imagery?select=Semantic+segmentation+dataset”. Aerial image segmentation dataset http://jiangyeyuan.com/ASD/Aerial%20Image%20Segmentation%20Dataset.html

References

  1. Wang, S., Hou, X., Zhao, X.: Automatic building extraction from high-resolution aerial imagery via fully convolutional encoder–decoder network with non-local block. IEEE Access 8, 7313–7322 (2020)

    Article  MATH  Google Scholar 

  2. Xiang, D., Zhang, X., Wu, W., Liu, H.: DensePPMUNet-a: a robust deep learning network for segmenting water bodies from aerial images. IEEE Trans. Geosci. Remote Sens. 61, 1–11 (2023)

    MATH  Google Scholar 

  3. Hou, J., Guo, Z., Feng, Y., Wu, Y., Diao, W.: SPANet: spatial adaptive convolution based content-aware network for aerial image semantic segmentation. IEEE J. Selected Topics Appl. Earth Observ. Remote Sens. 16, 2192–2204 (2023)

    Article  MATH  Google Scholar 

  4. Niu, R., Sun, X., Tian, Y., Diao, W., Chen, K., Fu, K.: Hybrid multiple attention network for semantic segmentation in aerial images. IEEE Trans. Geosci. Remote Sens. 60, 1–18 (2022)

    MATH  Google Scholar 

  5. Tu, J., Sui, H., Feng, W., Sun, K., Hua, L.: Detection of damaged rooftop areas from high-resolution aerial images based on visual bag-of-words model. IEEE Geosci. Remote Sens. Lett. 13(12), 1817–1821 (2016)

    Article  Google Scholar 

  6. Byeongjun, Y., Jeon, H., Bang, H., Yi, S.S., Min, J.: Fender segmentation in unmanned aerial vehicle images based on densely connected receptive field block. Int. J. Naval Archit. Ocean Eng. 14, 100472 (2022)

    Article  Google Scholar 

  7. Chen, G., Hao, K., Wang, B., Li, Z., Zhao, X.: A power line segmentation model in aerial images based on an efficient multibranch concatenation network. Exp. Syst. Appl. 120359 (2023)

  8. Dutta, K., Talukdar, D., Bora, S.S.: "Segmentation of unhealthy leaves in cruciferous crops for early disease detection using vegetative indices and Otsu thresholding of aerial images. Measurement 189, 110478 (2022)

    Article  MATH  Google Scholar 

  9. Wang, Y., Wang, L., Huchuan, L., He, Y.: Segmentation based rotated bounding boxes prediction and image synthesizing for object detection of high resolution aerial images. Neurocomputing 388, 202–211 (2020). https://doi.org/10.1016/j.neucom.2020.01.039

    Article  MATH  Google Scholar 

  10. Chai, D., Newsam, S., Huang, J.: Aerial image semantic segmentation using DCNN predicted distance maps. ISPRS J. Photogramm. Remote Sens. 161, 309–322 (2020)

    Article  Google Scholar 

  11. Weng, Q., Chen, H., Chen, H., Guo, W., Mao, Z.: A multisensor data fusion model for semantic segmentation in aerial images. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2022)

    Article  MATH  Google Scholar 

  12. Zhao, Y., Guo, P., Gao, H., Chen, X.: Depth-assisted ResiDualGAN for cross-domain aerial images semantic segmentation. IEEE Geosci. Remote Sens. Lett. 20, 1–5 (2023)

    MATH  Google Scholar 

  13. Sani-Mohammed, A., Yao, W., Heurich, M.: Instance segmentation of standing dead trees in dense forest from aerial imagery using deep learning. ISPRS Open J. Photogramm. Remote Sens. 6, 100024 (2022)

    Article  Google Scholar 

  14. Zhou, T., Guo, J., Qilong, Wu., Chuan, Xu.: An unmanned aerial vehicle identification and tracking system based on weakly supervised semantic segmentation technology. Phys. Commun. 54, 101758 (2022)

    Article  MATH  Google Scholar 

  15. Yang, L., Fan, J., Huo, B., Li, E., Liu, Y.: PLE-Net: Automatic power line extraction method using deep learning from aerial images. Exp. Syst. Appl. 198, 116771 (2022)

    Article  Google Scholar 

  16. Park, J., Cho, Y.K., Kim, S.: Deep learning-based UAV image segmentation and inpainting for generating vehicle-free ortho mosaic. Int. J. Appl. Earth Obs. Geoinf. 115, 103111 (2022)

    MATH  Google Scholar 

  17. Dhami, H.S., Ignatyev, D., Tsourdos, A.: Semantic segmentation based mapping systems for the safe and precise landing of flying vehicles. IFAC-PapersOnLine 55(22), 310–315 (2022). https://doi.org/10.1016/j.ifacol.2023.03.052

    Article  MATH  Google Scholar 

  18. Ahmed, S., Biswas, A.: A cross entropy and whale optimization algorithm based image segmentation for aerial images. Int. J. Inf. Technol. (2024). https://doi.org/10.1007/s41870-024-01831-z

    Article  MATH  Google Scholar 

  19. Kapoor, S., Zeya, I.: Chirag Singhal and Satyasai Jagannath Nanda, “a grey wolf optimizer based automatic clustering algorithm for satellite image segmentation.” Proc. Comput. Sci. 115, 415–422 (2017)

    Article  MATH  Google Scholar 

  20. Jia, H., Lang, C., Oliva, D., Song, W., Peng, X.: Hybrid grasshopper optimization algorithm and differential evolution for multilevel satellite image segmentation. Remote Sens. 11(9), 1134 (2019). https://doi.org/10.3390/rs11091134

    Article  MATH  Google Scholar 

  21. Volpi, M., Tuia, D.: Deep multi-task learning for a geographically-regularized semantic segmentation of aerial images. ISPRS J. Photogramm. Remote Sens. 144, 48–60 (2018)

    Article  MATH  Google Scholar 

  22. Behera, T.K., Bakshi, S., Sa, P.K.: A lightweight deep learning architecture for vegetation segmentation using UAV-captured aerial images. Sustain. Comput.: Inf. Syst. 37, 100841 (2023)

    MATH  Google Scholar 

  23. Gupta, A., Watson, S., Yin, H.: Deep learning-based aerial image segmentation with open data for disaster impact assessment. Neurocomputing 439, 22–33 (2021). https://doi.org/10.1016/j.neucom.2020.02.139

    Article  MATH  Google Scholar 

  24. Gevaert, C.M., Belgiu, M.: Assessing the generalization capability of deep learning networks for aerial image classification using landscape metrics. Int. J. Appl. Earth Obs. Geoinf. 114, 103054 (2022)

    MATH  Google Scholar 

  25. Wieland, M., Martinis, S., Kiefl, R., Gstaiger, V.: Semantic segmentation of water bodies in very high-resolution satellite and aerial images. Remote Sens. Environ. 287, 113452 (2023)

    Article  Google Scholar 

  26. Behera, T.K., Bakshi, S., Nappi, M., Sa, P.K.: Superpixel-based multiscale cnn approach toward multiclass object segmentation from UAV-captured aerial images. IEEE J. Selected Topics Appl. Earth Observ. Remote Sens. 16, 1771–1784 (2023). https://doi.org/10.1109/JSTARS.2023.3239119

    Article  Google Scholar 

  27. Li, Bo., Chen, C., Dong, S., Qiao, J.: Transmission line detection in aerial images: An instance segmentation approach based on multitask neural networks. Signal Process.: Image Commun. 96, 116278 (2021)

    MATH  Google Scholar 

  28. Zhang, Y., Gao, X., Duan, Q., Yuan, L., Gao, X.: DHT: deformable hybrid transformer for aerial image segmentation. IEEE Geosci. Remote Sens. Letters 19, 1–5 (2022)

    MATH  Google Scholar 

  29. Jaimes, B.R.A., Ferreira, J.P.K., Castro, C.L.: Unsupervised semantic segmentation of aerial images with application to UAV localization. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2022). https://doi.org/10.1109/LGRS.2021.3113878

    Article  MATH  Google Scholar 

  30. Abdelfattah, R., Wang, X., Wang, S.: PLGAN: generative adversarial networks for power-line segmentation in aerial images. IEEE Trans. Image Process. 32, 6248–6259 (2023)

    Article  MATH  Google Scholar 

  31. Haoyu Yue, Junhong Yue, Xuejun Guo, Yizhen Wang and Liancheng Jiang, "MA-DBFAN: multiple-attention-based dual branch feature aggregation network for aerial image semantic segmentation", Signal, Image and Video Processing, (2024)

  32. Li, Z., Wang, H., Liu, Y.: Semantic segmentation of remote sensing image based on bilateral branch network. Vis. Comput. 40, 3069–3090 (2024)

    Article  MATH  Google Scholar 

  33. Mogaka, O.M., Zewail, R., Inoue, K., Sayed, M.S.: TinyEmergencyNet: a hardware-friendly ultra-lightweight deep learning model for aerial scene image classification. J. Real-Time Image Process. (2024). https://doi.org/10.1007/s11554-024-01430-y

    Article  Google Scholar 

  34. Chen, H., Qin, Y., Liu, X., Wang, H., Zhao, J.: An improved DeepLabv3+ lightweight network for remote-sensing image semantic segmentation. Complex Intell. Syst. 10, 2839–2849 (2024)

    Article  MATH  Google Scholar 

  35. Azizi, M., Talatahari, S., Gandomi, A.H.: Fire Hawk Optimizer: a novel metaheuristic algorithm. Artif. Intell. Rev. 56(1), 287–363 (2022). https://doi.org/10.1007/s10462-022-10173-w

    Article  MATH  Google Scholar 

  36. Yilmaz, S., Sen, S.: Electric fish optimization: a new heuristic algorithm inspired by electrolocation. Neural Comput. Appl. 32, 11543–11578 (2020)

    Article  MATH  Google Scholar 

  37. Tuli, S., Casale, G., Jennings, N.R.: TranAD: deep transformer networks for anomaly detection in multivariate time series data. Proc. VLDB Endowment 15(6), 1201–1214 (2022). https://doi.org/10.14778/3514061.3514067

    Article  Google Scholar 

  38. Maitreya Suin, Kuldeep Purohit, A. N. Rajagopalan, "Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring," computer vision foundation, pp. 3606, (2020)

  39. Xie, L., Han, T., Zhou, H., Zhang, Z.-R., Han, B., Tang, A.: Tuna swarm optimization: a novel swarm‐based metaheuristic algorithm for global optimization. Comput. Intell. Neurosci. (2021). https://doi.org/10.1155/2021/9210050

    Article  MATH  Google Scholar 

  40. Wang, Z.W., Wang, H., Yildizbasi, A.: Developed coyote optimization algorithm and its application to optimal parameters estimation of PEMFC model. Energy Rep. 6, 1106–1117 (2020)

    Article  MATH  Google Scholar 

  41. Zeng, H., Peng, S., Li, D.: Deeplabv3+ semantic segmentation model based on feature cross attention mechanism. J. Phys. Conf. Ser. 1678, 012106 (2020)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

I would like to express my very great appreciation to the co-authors of this manuscript for their valuable and constructive suggestions during the planning and development of this research work.

Funding

This research did not receive any specific funding.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Mother Theresa Institute of Engineering and Technology, Palamaner, Andhra Pradesh, 517408, India

    P. Anilkumar

  2. School of Electronics Engineering, Vellore Institute of Technology, Vellore, Tamil Nadu, 632014, India

    P. Venugopal

  3. Department of Artificial Intelligence and Data Science, Mother Theresa Institute of Engineering and Technology, Palamaner, Andhra Pradesh, 517408, India

    K. Lokesh

  4. Department of Electronics and Communication Engineering, Madanapalle Institute of Technology and Science (Autonomous), Angallu, Andhra Pradesh, 517325, India

    G. NagaJyothi

  5. Department of Electronics and Computer Engineering, Sreenidhi Institute of Science and Technology, Yamnampet, Hyderabad, Telangana, 501301, India

    M. Nanda kumar

Authors
  1. P. Anilkumar
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. P. Venugopal
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. K. Lokesh
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. G. NagaJyothi
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. M. Nanda kumar
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors have made substantial contributions to conception and design, revising the manuscript, and the final approval of the version to be published. Also, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to P. Anilkumar.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Ethical approval

Not Applicable.

Informed consent

Not Applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anilkumar, P., Venugopal, P., Lokesh, K. et al. AA-TransDeeplabv3 + : a novel semantic segmentation framework for aerial images using adaptive and attentive based Transdeeplabv3 + with hybrid optimization technique. SIViP 19, 225 (2025). https://doi.org/10.1007/s11760-024-03617-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03617-z

Keywords