Skip to main content
SpringerLink
Log in

AMGAN: An Attribute-Matched Generative Adversarial Network for UAV Virtual Sample Generation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The recognition and detection of unmanned aerial vehicles (UAV) usually face the difficulty of insufficient samples. Given a limited number of real UAV images, it is a challenging task to generate virtual UAV images to enrich both the diversity and quantity of training samples. Aiming at this problem, we propose a novel attribute-matched generative adversarial network (AMGAN) that can migrate a UAV object from a single background to a complex background. AMGAN consists of three parts: the basic network, the pairing network, and the background constraint. Image attributes are first disentangled and then reorganized by the basic network, which is prone to attribute collapse. Then the pairing network introduces attribute-level discriminators to make the same-type attributes match each other correctly. Furthermore, the background constraint is added to guide model convergence and eliminate the attribute residue problem. Qualitative experimental results show that AMGAN can generate a large number of high-fidelity virtual UAV images in various backgrounds. Quantitative experimental results on small-scale datasets demonstrate that when these generated images are used for data augmentation, both the diversity and quantity of samples can be greatly increased, boosting the UAV recognition performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Zhao J, Zhang J, Li D, Wang D (2022) Vision-based anti-UAV detection and tracking. IEEE Trans Intell Transport Sys 23(12):25323–25334

    Article  Google Scholar 

  2. Jiang N, Wang K, Peng X, Yu X, Wang Q, Xing J, Li G, Ye Q, Jiao J, Han Z et al (2021) Anti-UAV: a large-scale benchmark for vision-based UAV tracking. IEEE Trans Multim. https://doi.org/10.1109/TMM.2021.3128047

    Article  Google Scholar 

  3. Fu C, Li B, Ding F, Lin F, Lu G (2021) Correlation filters for unmanned aerial vehicle-based aerial tracking: a review and experimental evaluation. IEEE Geosci Remote Sens Magaz 10(1):125–160

    Article  Google Scholar 

  4. Sheu B-H, Chiu C-C, Lu W-T, Huang C-I, Chen W-P (2019) Development of UAV tracing and coordinate detection method using a dual-axis rotary platform for an anti-UAV system. Appl Sci 9(13):2583

    Article  Google Scholar 

  5. Hu Y, Wu X, Zheng G, Liu X (2019) Object detection of UAV for anti-UAV based on improved YOLO v3. In: 2019 Chinese control conference (CCC), IEEE, pp. 8386–8390

  6. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  7. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48

    Article  Google Scholar 

  8. Takahashi R, Matsubara T, Uehara K (2019) Data augmentation using random image cropping and patching for deep CNNs. IEEE Trans Circ Sys Video Technol 30(9):2917–2931

    Article  Google Scholar 

  9. Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2017) mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412

  10. Verma V, Lamb A, Beckham C, Najafi A, Mitliagkas I, Lopez-Paz D, Bengio Y (2019) Manifold mixup: better representations by interpolating hidden states. In: International conference on machine learning, pp. 6438–6447. PMLR

  11. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun the ACM 63(11):139–144

    Article  MathSciNet  Google Scholar 

  12. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  13. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  14. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, IEEE, pp. 214–223

  15. Antoniou A, Storkey A, Edwards H (2017) Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340

  16. Xiong W, He Y, Zhang Y, Luo W, Ma L, Luo J (2020) Fine-grained image-to-image transformation towards visual recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5840–5849

  17. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp. 2223–2232

  18. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134

  19. Choi Y, Choi M, Kim M, Ha J-W, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789–8797

  20. Chen X, Xu C, Yang X, Song L, Tao D (2018) Gated-gan: adversarial gated networks for multi-collection style transfer. IEEE Trans Image Process 28(2):546–560

    Article  MathSciNet  MATH  Google Scholar 

  21. Zhou S, Xiao T, Yang Y, Feng D, He Q, He W (2017) Genegan: learning object transfiguration and attribute subspace from unpaired data. arXiv preprint arXiv:1705.04932

  22. Zhao S, Liu Z, Lin J, Zhu J-Y, Han S (2020) Differentiable augmentation for data-efficient GAN training. Adv Neur Inf Process Sys 33:7559–7570

    Google Scholar 

  23. Li Y, Singh KK, Ojha U, Lee YJ (2020) Mixnmatch: multifactor disentanglement and encoding for conditional image generation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8039–8048

  24. Jha AH, Anand S, Singh M, Veeravasarapu VR (2018) Disentangling factors of variation with cycle-consistent variational auto-encoders. In: Proceedings of the European conference on computer vision (ECCV), pp. 805–820

  25. Liu M-Y, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. Advances in neural information processing systems, 30

  26. Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: International conference on machine learning, PMLR, pp. 7354–7363

  27. Mo S, Cho M, Shin J (2020) Freeze the discriminator: a simple baseline for fine-tuning gans. arXiv preprint arXiv:2002.10964

  28. Wen J, Shen Y, Yang J (2022) Multi-view gait recognition based on generative adversarial network. Neural Process Lett 54(3):1855–1877

    Article  Google Scholar 

  29. Huang X, Liu M-Y, Belongie S, Kautz J (2018) Multimodal unsupervised image-to-image translation. In: Proceedings of the European conference on computer vision (ECCV), pp. 172–189

  30. Lee H-Y, Tseng H-Y, Mao Q, Huang J-B, Lu Y-D, Singh M, Yang M-H (2020) Drit++: diverse image-to-image translation via disentangled representations. Int J Comput Vision 128(10):2402–2417

    Article  Google Scholar 

  31. Elasri M, Elharrouss O, Al-Maadeed S, Tairi H (2022) Image generation: a review. Neural Process Lett 54(5):4609–4646

    Article  Google Scholar 

  32. Wah C, Branson S, Welinder P, Perona P, Belongie S (2011) The Caltech-UCSD Birds-200-2011 dataset

  33. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp. 248–255

  34. Xia G-S, Bai X, Ding J, Zhu Z, Belongie S, Luo J, Datcu M, Pelillo M, Zhang L (2018) Dota: a large-scale dataset for object detection in aerial images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3974–3983

  35. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-first AAAI conference on artificial intelligence

  36. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

  37. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training gans. Advances in neural information processing systems 29

  38. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision. Springer, pp. 740–755

  39. Li J, Murray J, Ismaili D, Schindler K, Albl C (2020) Reconstruction of 3D flight trajectories from ad-hoc camera networks. In: 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS), IEEE, pp. 1621–1628

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant No. 61201238, in part by the Aeronautical Science Foundation of China under Grant No. 201801P6002, and in part by the Fundamental Research Funds for the Central Universities under Grant No. 3072022CF0802.

Author information

Author notes

  1. Xinbo Jia, Yahui Shen, Yuanlan Yang and Huiyang Li have contributed equally to this work.

Authors and Affiliations

  1. College of Information and Communication Engineering, Harbin Engineering University, Harbin, 150001, China

    Zhigang Yang, Xinbo Jia, Yahui Shen, Yuanlan Yang & Huiyang Li

  2. Key Laboratory of Advanced Marine Communication and Information Technology, Ministry of Industry and Information Technology, Harbin, 150001, China

    Zhigang Yang

  3. School of Computer Science, University of Adelaide, Adelaide, SA, 5005, Australia

    Wei Emma Zhang

Authors
  1. Zhigang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinbo Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yahui Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuanlan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Huiyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wei Emma Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhigang Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., Jia, X., Shen, Y. et al. AMGAN: An Attribute-Matched Generative Adversarial Network for UAV Virtual Sample Generation. Neural Process Lett 55, 8131–8149 (2023). https://doi.org/10.1007/s11063-023-11304-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11304-2

Keywords

Search

Navigation