Skip to main content
SpringerLink
Log in

Potential of generative adversarial net algorithms in image and video processing applications– a survey

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Generative Adversarial Network (GAN) has gained eminence in a very short period as it can learn deep data distributions with the help of a competitive process among two networks. GANs can synthesize images/videos from latent noise with a minimized adversarial cost function. The cost function plays a deciding factor in GAN training and thus, it is often subjected to new modifications to yield better performance. To date, numerous new GAN models have been proposed owing to changes in cost function according to applications. The main objective of this research paper is to present a gist of major GAN publications and developments in image and video field. Several publications were selected after carrying out a thorough literature survey. Beginning from trends in GAN research publications, basics, literature survey, databases for performance evaluation parameters are presented under one umbrella.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Acharya D (2018) Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs. arXiv,1–22

  2. Agnese J, Herrera J, Tao H, Zhu X (2020) A survey and taxonomy of adversarial neural networks for text-to-image synthesis. Data Min Knowl Disc. https://doi.org/10.1002/widm.1345

  3. Alqahtani H, Kavakli-Thorne M, Kumar G (2019) Applications of generative adversarial networks (GANs): an updated review. Archives Computat Methods Eng. https://doi.org/10.1007/s11831-019-09388-y.1-28

  4. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein GAN. arXiv preprint arXiv:1701.07875

  5. Arora S, Zhang Y (2017) Do GANs actually learn the distribution? An empirical study. arXiv preprint arXiv:1706.08224

  6. Azar MG, Munos R, Kappen HJ (2013) Minimax PAC bounds on the sample complexity of reinforcement learning with a generative model. Mach Learn 91(3):325–349

    MathSciNet  MATH  Google Scholar 

  7. Bansal A, Ma S, Ramanan D, Sheikh Y (2018) Recycle-GAN: Unsupervised video retargeting. In Proceedings of the European Conference on Computer Vision, pp 119–135

  8. BenTaieb A, Hamarneh G (2018) Adversarial stain transfer for histopathology image analysis. IEEE Trans Med Imaging 37(3):792–802

    Google Scholar 

  9. Berg T, Liu J, Woo Lee S, Alexander ML, Jacobs DW, Belhumeur PN (2014) Birdsnap: Large-scale fine-grained visual categorization of birds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2011–2018

  10. Bermudez JD, Happ PN, Feitosa RQ, Oliveira DA (2019) Synthesis of multispectral optical images from SAR/optical multitemporal data using conditional generative adversarial networks. IEEE Geosci Remote Sens Lett 16:1220–1224

    Google Scholar 

  11. Berthelot D, Schumm T, Metz L (2017) BeGAN: boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717

  12. Bhattacharjee P, Das S (2017) Temporal coherency based criteria for predicting video frames using deep multi-stage generative adversarial networks. In advances in neural information processing systems, pp 4268-4277

  13. Borji A (2018) Pros and cons of GAN evaluation measures. Comput Vis Image Underst

  14. Brkić K, Hrkać T, Kalafatić Z, Sikirić I (2017) Face hairstyle and clothing colour de-identification in video sequences. IET Signal Process 11(9):1062–1068

    Google Scholar 

  15. Bulat A, Yang J, Tzimiropoulos G (2018) To learn image superresolution, use a GAN to learn how to do image degradation first. ECCV, 185–200

  16. Cao J, Hu Y, Yu B, He R, Sun Z (2019) 3D aided duet GANs for multi-view face image synthesis. IEEE Trans Inform Forensics Secur 14:2028–2042

    Google Scholar 

  17. Cao Y, Jia LL, Chen YX, Lin N, Yang C, Zhang B, Liu Z, Li X, Dai H (2019) Recent advances of generative adversarial networks in computer vision. IEEE Access, 14985–15006, 7

  18. Chang W, Yang G, Yu J, Liang Z (2018) Real-time segmentation of various insulators using generative adversarial networks. IET Comput Vis 12(5):596–602

    Google Scholar 

  19. Che T, Li Y, Jacob AP, Bengio Y, Li W (2016) Mode regularized generative adversarial networks. arXiv preprint arXiv:1612.02136

  20. Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems, pp 2172–2180

  21. Chen K Y, Lu C Y, Xing Y (2017) Video super-resolution using temporal fusion generative adversarial network

  22. Chen X, Yu J, Kong S, Wu Z, Fang X, Wen L (2019) Towards real-time advancement of underwater visual quality with GAN. IEEE Trans Ind Electron

  23. Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J (2018) StarGAN: Unified Generative Adversarial Networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8789–8797

  24. Chu M, Xie Y, Mayer J, Leal-Taixe L, Thuerey N (2018)Learning Temporal Coherence via Selfsupervision for GAN-based Video generation. arXiv,1–22

  25. Coates A, Ng A, Lee H (2011) An analysis of single-layer Networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 215–223

  26. Costa P, Galdran A, Meyer MI, Niemeijer M, Abràmoff M, Mendonça AM, Campilho A (2018) End-to-end Adversarial retinal image synthesisIEEE transactions on medical imaging 37(3):781–791

  27. Cousins S, Shawe-Taylor J (2017) High-probability minimax probability machines. Mach Learn 106(6):863–886

    MathSciNet  MATH  Google Scholar 

  28. Demir U, Unal G (2018) Patch-based image inpainting with generative adversarial networks. arXiv preprint arXiv:1803.07422

  29. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database

  30. Denton EL, Chintala, Fergus R (2015) Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems, pp 1486–1494

  31. Dinh L, Sohl-Dickstein J, Bengio S (2016) Density estimation using real nvp. arXiv preprint arXiv:1605.08803

  32. Donahue J, Krähenbühl P, Darrell T (2016) Adversarial feature learning. arXiv preprint arXiv:1605.09782

  33. Dong J, Yin R, Sun X, Li Q, Yang Y, Qin X (2019) Inpainting of remote sensing SST images with deep convolutional generative adversarial network. IEEE Geosci Remote Sens Lett 16(2):173–177

    Google Scholar 

  34. Fahlman SE, Hinton GE, Sejnowski TJ (1983) Massively parallel architectures for Al: NETL thistle and Boltzmann machines. In National Conference on Artificial Intelligence

  35. Frey BJ, Hinton GE, Dayan P (1996) Does the wake-sleep algorithm produce good density estimators? In Advances in neural information processing systems, pp 661–667

  36. Frey BJ, Brendan JF, Frey BJ (1998) Graphical models for machine learning and digital communication. MIT press

  37. Gao Y, Liu Y, Wang Y, Shi Z, Yu J (2019) A universal intensity standardization method based on a many-to-one weak-paired cycle generative adversarial network for magnetic resonance images. IEEE Trans Med Imaging 38:2059–2069

    Google Scholar 

  38. Ge H, Yao Y, Chen Z, Sun L (2018) Unsupervised transformation network based on GANs for target-domain oriented image translation. IEEE Access 6:61342–61350

    Google Scholar 

  39. Ghamisi P, Yokoya N (2018) Img2dsm: height simulation from single imagery using conditional generative adversarial net. IEEE Geosci Remote Sens Lett 15(5):794–798

    Google Scholar 

  40. Gong M, Niu X, Zhang P, Li Z (2017) Generative Adversarial Networks for change detection in multispectral imagery. IEEE Geosci Remote Sens Lett 14(12):2310–2314

    Google Scholar 

  41. Goodfellow IJ, Bulatov Y, Ibarz J, Arnoud S, Shet V (2013) Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv:1312.6082

  42. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S and Bengio Y (2014) Generative Adversarial nets. In Advances in neural information processing systems, pp 2672–2680

  43. Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13:723–773

    MathSciNet  MATH  Google Scholar 

  44. Griffin G, Holub A, Perona P (2007) Caltech-101 object category dataset

  45. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset

  46. Grzegorczyk M (2016) A non-homogeneous dynamic Bayesian network with a hidden Markov model dependency structure among the temporal data points. Mach Learn 102(2):155–207

    MathSciNet  MATH  Google Scholar 

  47. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein GANs. In Advances in Neural Information Processing Systems, pp 5767–5777

  48. Gurumurthy S, Kiran Sarvadevabhatla R, Venkatesh Babu R (2017) DeliGAN: Generative Adversarial Networks for diverse and limited data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 166–174

  49. He Y, Zhang J, Shan H, Wang L (2019) Multi-task GANs for view-specific feature learning in gait recognition. IEEE Trans Inform Forensics Secur 14(1):102–113

    Google Scholar 

  50. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) GANs trained by a two time-scale update rule converge to a local Nash equilibriumIn advances in neural information processing systems, pp 6626-6637

  51. Hinton GE, Sejnowski TJ (1986) Learning and relearning in Boltzmann machines parallel distributed processing: explorations in the microstructure of cognition 1(282-317) 2

  52. Hu B, Tang Y, Chang EI, Fan Y, Lai M, Xu Y (2017) Unsupervised learning for cell-level visual representation in histopathology images with generative adversarial networks. arXiv preprint arXiv:1711.11317

  53. Huang Z, Paudel DP, Li G, Wu J, Timofte R, Gool LV (2019) Divide-and-Conquer Adversarial Learning for High-resolution Image and Video enhancement, arXiv,1–17

  54. Huo Y, Xu Z, Bao S, Bermudez C, Moon H, Parvathaneni P, Landman BA (2018) Splenomegaly segmentation on multi-modal MRI using deep convolutional networks. IEEE Trans Med Imaging

  55. Im DJ, Kim CD, Jiang H, Memisevic R (2016) Generating images with recurrent adversarial networks. arXiv preprint arXiv:1602.05110

  56. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional Adversarial Networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  57. Jolicoeur-Martineau A (2018) The relativistic discriminator: a key element missing from standard GAN. arXiv preprint arXiv:1807.00734

  58. Juefei-Xu F, Boddeti VN, Savvides M (2017) GANg of GANs: generative adversarial networks with maximum margin ranking. arXiv preprint arXiv:1704.04865

  59. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of GANs for improved quality and stability and variation. arXiv preprint arXiv:1710.10196

  60. Kim T, Cha M, Kim H, Lee JK, Kim J (2017) Learning to discover cross-domain relations with Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp 1857–1865. JMLRorg

  61. Kim D, Jang HU, Mun SM, Choi S, Lee HK (2018) Median filtered image restoration and anti-forensics using adversarial networks. IEEE Signal Process Lett 25(2):278–282

    Google Scholar 

  62. Kingma DP, Salimans T, Jozefowicz R, Chen X, Sutskever I, Welling M (2016) Improving variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems. arXiv:1606.04934

  63. Kodali N, Abernethy J, Hays J, Kira Z (2017) On convergence and stability of GANs. arXiv preprint arXiv:1705.07215

  64. Krizhevsky A, Hinton G (2010) Convolutional deep belief networks on cifar-10. Unpublished manuscript 40(7)

  65. LeCun Y (1998) The MNIST database of handwritten digits. https://yann.lecun.com/exdb/mnist/

  66. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Shi W (2017) Photo-realistic single image super-resolution using a Generative Adversarial Network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  67. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, (2017) Photo-realistic single image superresolution using a generative adversarial network. CVPR

  68. Lee HJ, Kim ST, Lee H, Ro YM (2019) Lightweight and effective facial landmark detection using adversarial learning with face geometric map generative network. IEEE Trans Circuits Syst Video Technol

  69. Lehmann EL, Romano JP (2006) Testing statistical hypotheses. Springer, Science and Business Media

    MATH  Google Scholar 

  70. Li Y, Shen L (2018) cC-GAN: a robust transfer-learning framework for HEp-2 specimen image segmentation. IEEE Access 6:14048–14058

    Google Scholar 

  71. Li J, Liu S, He H, Li L (2018) A novel framework for gear safety factor prediction. IEEE Trans Industrial Informatics

  72. Li H, Li G, Lin L, Yu H, Yu Y (2018) Context-aware semantic inpainting. IEEE transactions on cybernetics

  73. Li J, Skinner KA, Eustice RM, Matthew JR (2018) WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robotics and Automation Letters 3(1):387–394

    Google Scholar 

  74. Li J, He H, Li L, Chen G (2019) A novel generative model with bounded-GAN for reliability classification of gear safety. IEEE Trans Ind Electron 66(11):8772–8781

    Google Scholar 

  75. Li P, Prieto L, Mery D, Flynn PJ (2019) On low-resolution face recognition in the wild: comparisons and new techniques. IEEE Trans Inform Forensics Secur 14:2000–2012

    Google Scholar 

  76. Liang X, Lee L, Dai W, Xing EP (2017) Dual motion GAN for future-flow embedded video prediction. In proceedings of the IEEE international conference on computer vision, pp 1744–1752

  77. Liao K, Lin C, Zhao Y, Gabbouj M (2020) DR-GAN: automatic radial distortion rectification using conditional GAN in real-time. IEEE Trans Circuits Syst Video Technol 30(3):725–733

    Google Scholar 

  78. Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision. Springer, Cham, pp 740–755

    Google Scholar 

  79. Lin D, Fu K, Wang Y, Xu G, Sun X (2017) MARTA GANs: unsupervised representation learning for remote sensing image classification. IEEE Geosci Remote Sens Lett 14(11):2092–2096

    Google Scholar 

  80. Liu Z, Luo P, Wang X, Tang X (2018) Large-scale celebfaces attributes (celeba) dataset

  81. Lopez-Tapia S, Lucas A, Molina R, Katsaggelos A K (2018) A single video super-resolution GAN for multiple Downsampling operators based on pseudo-inverse image formation models

  82. Lopez-Tapia S, Lucas A, Molina R, Katsaggelos A K (2019) GAN-based video super-resolution with direct regularized inversion of the low-resolution formation model, ICIP conference

  83. Lucas A, Lopez-Tapiad S, Molinae R, Katsaggelos AK (2019) Generative adversarial networks and perceptual losses for video super-resolution. IEEE Trans Image Process 28:3312–3327

    MathSciNet  MATH  Google Scholar 

  84. Lucas A, Lopez-Tapia S, Molina R, Katsaggelos AK (2019) Generative adversarial networks and perceptual losses for video super-resolution. IEEE Trans Image Process 28(7):3312–3327

    MathSciNet  MATH  Google Scholar 

  85. Lucic M, Kurach K, Michalski M, Gelly S, Bousquet O (2018) Are GANs created equal? A large-scale study. In Advances in neural information processing systems, pp 698–707

  86. Lucic M, Kurach K, Michalski M, Bousquet O, Gelly S (2018) Are GANs created equal? A large-scale study. NeurIPS 1–10

  87. Ma S, Fu J, Wen Chen C, Mei T (2018) DA-GAN: Instance-level image translation by deep attention Generative Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5657–5666

  88. Ma D, Tang P, Zhao L (2019) SiftingGAN: generating and sifting labeled samples to improve the remote sensing image scene classification baseline in vitro. IEEE Geosci Remote Sens Lett 16:1046–1050

    Google Scholar 

  89. Mao X, Li Q, Xie H, Lau RY, Wang Z, Paul Smolley S (2017) Least squares generative adversarial networks. In proceedings of the IEEE international conference on computer vision, pp 2794–2802

  90. Mao X, Li Q, Xie H, KLau RY, Wang Z, Smolley SP (2018) On the effectiveness of least squares generative adversarial networks. IEEE Trans Pattern Anal Mach Intell 41:2947–2960. https://doi.org/10.1109/TPAMI.2018.2872043

    Article  Google Scholar 

  91. Mardani M, Gong E, Cheng JY, Vasanawala SS, Zaharchuk G, Xing L, Pauly JM (2019) Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans Med Imaging 38(1):167–179

    Google Scholar 

  92. Mathieu M, Couprie C, Le Y (2016) Deep multi-scale video prediction beyond mean square error. ICLR:1–14

  93. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  94. Miyato T, Kataoka T, Koyama M, Yoshida Y (2018) Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957

  95. Nie D, Trullo R, Lian J, Wang L, Petitjean C, Ruan S, Shen D (2018) Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng 65(12):2720–2730

    Google Scholar 

  96. Nilsback ME and Zisserman A (2008). Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision Graphics and Image Processing, pp 722–729, IEEE

  97. Niu X, Gong M, Zhan T, Yang Y (2019) A conditional adversarial network for change detection in heterogeneous images. IEEE Geosci Remote Sens Lett 16(1):45–49

    Google Scholar 

  98. Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier GANs. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp 2642–2651, JMLRorg

  99. Ohnishi K, Yamamoto S, Ushiku Y, Harada T (2018) Hierarchical video generation from orthogonal information: optical flow and texture. In Thirty-Second AAAI Conference on Artificial Intelligence

  100. Oliveira DA, Ferreira RS, Silva R, Brazil EV (2018) Interpolating seismic data with conditional generative adversarial networks. IEEE Geosci Remote Sens Lett 99:1–5

    Google Scholar 

  101. Oord AVD, Dieleman S, Zen H, Simonyan K, Vinyals O, Graves A, Kavukcuoglu K (2016) Wavenet: a generative model for raw audio. arXiv preprint arXiv:1609.03499

  102. Ouyang X, Cheng Y, Jiang Y, Li C L, Zhou P (2018) Pedestrian-synthesis-GAN: generating pedestrian data in real scene and beyond.arXiv 1–22

  103. Pan Y, Qiu Z, Yao T, Li H, Mei T (2017) To create what you tell: generating videos from captions. In proceedings of the 25th ACM international conference on multimedia, pp 1789–1798. ACM

  104. Pang Y, Xie J, Li X (2018) Visual haze removal by a unified generative adversarial network. IEEE Trans Circuits Syst Video Technol

  105. Pascual S, Bonafonte A, Serrà J (2017) SEGAN: speech enhancement generative adversarial network. arXiv preprint arXiv:1703.09452

  106. Quan TM, Nguyen-Duc T, Jeong WK (2018) Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 37(6):1488–1497

    Google Scholar 

  107. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  108. Rezende DJ, Mohamed S (2015) Variational inference with normalizing flows. arXiv preprint arXiv:1505.05770

  109. Richardson E, Weiss Y (2018) On GANs and gmms. In Advances in Neural Information Processing Systems, pp 5852–5863

  110. Royer A, Bousmalis K, Gouws S, Bertsch F, Mosseri I, Cole F, Murphy K (2017) XGAN: unsupervised image-to-image translation for many-to-many mappings. arXiv preprint arXiv:1711.05139

  111. Saito Y, Takamichi S, Saruwatari H (2018) Statistical parametric speech synthesis incorporating generative adversarial networks. IEEE/ACM Trans Audio Speech Language Process 26(1):84–96

    Google Scholar 

  112. Sakkos D, Ho ES, Shum HP (2019) Illumination-aware multi-task GANs for foreground segmentation. IEEE Access, Illumination-Aware Multi-Task GANs for Foreground Segmentation

  113. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training GANs. In Advances in neural information processing systems, pp 2234–2242

  114. Shan H, Zhang Y, Yang Q, Kruger U, Kalra MK, Sun L, Wang G (2018) 3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network. IEEE Trans Med Imaging 37(6):1522–1534

    Google Scholar 

  115. Shen Z, Sheng W, Xu L T, Kautz J, Yang M H (2018) Deep Semantic Face Deblurring, CVPR

  116. Shen Z, Wang W, Lu X, Shen J, Ling H, Xu T, Shao L (2019) Human-Aware Motion Deblurring, ICCV

  117. Shi Y, Li Q, Zhu XX (2018) Building footprint generation using improved generative adversarial networks. IEEE Geosci Remote Sens Lett

  118. Snell J, Ridgeway K, Liao R, Roads BD, Mozer MC, Zemel RS (2017) Learning to generate images with perceptual similarity metrics. In 2017 IEEE International Conference on Image Processing, pp 4277–4281. IEEE

  119. Theis L, Oord AVD, Bethge M (2015) A note on the evaluation of generative models. arXiv preprint arXiv:1511.01844

  120. Tuan YL, Lee HY (2019) Improving conditional sequence generative adversarial networks by stepwise evaluation. IEEE/ACM Trans Audio Speech Language Process 27(4):788–798

    Google Scholar 

  121. Tulyakov S, Liu MY, Yang X, Kautz J (2018) MocoGAN: Decomposing motion and content for video generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1526–1535

  122. Van Horn G, Branson S, Farrell R, Haber S, Barry J, Ipeirotis P, Belongie S (2015) Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 595–604

  123. Van Horn G, Mac Aodha O, Song Y, Shepard A, Adam H, Perona P, Belongie S (2017) The inaturalist challenge 2017 dataset. arXiv preprint arXiv:1707.06642 1(2)

  124. Vondrick C, Pirsiavash H, Torralba A (2016) Generating videos with scene dynamics. In Advances In Neural Information Processing Systems, pp 613–621

  125. Walker J, Marino K, Gupta A, Hebert M (2017) The pose knows: video forecasting by generating pose futures. In proceedings of the IEEE international conference on computer vision, pp 3332–3341

  126. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

  127. Wang Y, Zhang L, van de Weijer J (2016) Ensembles of generative adversarial networks. arXiv preprint arXiv:1612.00991

  128. Wang W, Huang Q, You S, Yang C, Neumann U (2017) Shape inpainting using 3d generative adversarial network and recurrent convolutional networks. In Proceedings of the IEEE international conference on computer vision, pp 2298–2306

  129. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Loy CC (2018) EsrGAN: enhanced super-resolution generative adversarial networks. In: European Conference on Computer Vision. Springer, Cham, pp 63–79

    Google Scholar 

  130. Wang Y, Zhou L, Yu B, Wang L, Zu C, Lalush DS, Shen D (2018) 3D auto-context-based locality adaptive multi-modality GANs for PET synthesis. IEEE Trans Med Imaging

  131. Wang Y, Zhou L, Yu B, Wang L, Zu C, Lalush DS, Shen D (2018) 3D auto-context-based locality adaptive multi-modality GANs for PET synthesis. IEEE Trans Med Imaging

  132. Wang Z, Chen Z, Wu F (2018) Thermal to visible facial image translation using generative adversarial networks. IEEE Signal Process Lett 25(8):1161–1165

    Google Scholar 

  133. Welinder P, Branson S, Mita T, Wah C, Schroff F, Belongie S, Perona P (2010) Caltech-UCSD birds 200

  134. Wen S, Liu W, Yang Y, Huang T, Zeng Z (2018) Generating realistic videos from keyframes with concatenated GANs. IEEE Trans Circuits Syst Video Technol

  135. Wolterink JM, Leiner T, Viergever MA, Išgum I (2017) Generative Adversarial Networks for noise reduction in low-dose CT. IEEE Transactions on Medical Imaging 36(12):2536–2545

  136. Wu W, Qi H, Rong Z, Liu L, Su H (2018) Scribble-supervised segmentation of aerial building footprints using adversarial learning. IEEE Access 6:58898–58911

    Google Scholar 

  137. Xiang S, Li H (2017) On the effects of batch and weight normalization in generative adversarial networks. arXiv preprint arXiv:1704.03971

  138. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  139. Xiong W, Luo W, Ma L, Liu W, Luo J (2018) Learning to generate time-lapse videos using multi-stage dynamic Generative Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2364–2373

  140. Xu C, Ren J, Zhang D, Zhang Y, Qin Z, Ren K (2019) GANobfuscator: mitigating information leakage under GAN via differential privacy. IEEE Trans Inform Forensics Secur 14:2358–2371

    Google Scholar 

  141. Xuan Q, Chen Z, Liu Y, Huang H, Bao G, Zhang D (2018) Multi-view generative adversarial network and its application in pearl classification. IEEE Trans Ind Electron 66(10):8244–8252

    Google Scholar 

  142. Yang J, Kannan A, Batra D, Parikh D (2017) Lr-GAN: layered recursive generative adversarial networks for image generation. arXiv preprint arXiv:1703.01560

  143. Yang G, Yu S, Dong H, Slabaugh G, Dragotti PL, Ye X, Firmin D (2018) DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging 37(6):1310–1321

    Google Scholar 

  144. Yang Q, Yan P, Zhang Y, Yu H, Shi Y, Mou X, Wang G (2018) Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Trans Med Imaging 37(6):1348–1357

    Google Scholar 

  145. Yang Y, Zhou J, Ai J, Bin Y, Hanjalic A, Shen HT, Ji Y (2018) Video captioning by adversarial lstm. IEEE Trans Image Process 27(11):5600–5611

    MathSciNet  Google Scholar 

  146. Yi Z, Zhang H, Tan P, Gong M (2017) DualGAN: unsupervised dual learning for image-to-image translation. In proceedings of the IEEE international conference on computer vision, pp 2849–2857

  147. Yu F, Seff A, Zhang Y, Song S, Funkhouser T, Xiao J (2015) LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365

  148. Yu J, Lin Z, Yang J, Shen X, Lu X, Huang TS (2018) Free-form image inpainting with gated convolution. arXiv preprint arXiv:1806.03589

  149. Yu J, Lin Z, Yang J, Shen X, Lu X, Huang TS (2018) Generative image inpainting with contextual attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5505–5514

  150. Yu B, Zhou L, Wang L, Shi Y, Fripp J, Bourgeat P (2019) Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Transactions on Medical Imaging

  151. Yuan Y, Tian C, Lu X (2018) Auxiliary Loss Multimodal GRU Model in Audio-Visual Speech Recognition. IEEE Access 6:5573–5583

    Google Scholar 

  152. Zeng Y, Lu H, Borji A (2017) Statistics of deep generated images. arXiv preprint arXiv:1708.02688

  153. Zhan Y, Hu D, Wang Y, Yu X (2018) Semisupervised hyperspectral image classification based on generative adversarial networks. IEEE Geosci Remote Sens Lett 15(2):212–216

    Google Scholar 

  154. Zhang L, Liu P, Gulla (2019) J Artificial Intell Mach Learning https://doi.org/10.1007/s10994-018-05777-9

  155. Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X, Metaxas DN (2017) StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. In proceedings of the IEEE international conference on computer vision, pp 5907–5915

  156. Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X, Metaxas D (2017) StackGAN++: realistic image synthesis with stacked generative adversarial networks. arXiv preprint arXiv:1710.10916

  157. Zhang M, Gong M, Mao Y, Li J, Wu Y (2018) Unsupervised feature extraction in Hyperspectral images based on Wasserstein generative adversarial network. IEEE Trans Geosci Remote Sens

  158. Zhang H, Goodfellow I, Metaxas D, Odena A (2018) Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318

  159. Zhang Z, Song Y, Qi H (2018) Decoupled learning for conditional Adversarial NetworksIn 2018 IEEE Winter Conference on Applications of Computer Vision, pp 700–708. IEEE

  160. Zhang K, Luo W, Zhong Y, Ma L, Liu W, Li H (2019) Adversarial spatio-temporal learning for video deblurring. IEEE Trans Image Process 28(1):291–301

    MathSciNet  Google Scholar 

  161. Zhao J, Mathieu M, LeCun Y (2016) Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126

  162. Zhou Z, Cai H, Rong S, Song Y, Ren K, Zhang W, Wang J (2017) Activation maximization generative adversarial nets. arXiv preprint arXiv:1703.02000

  163. Zhu JY, Zhang R, Pathak D, Darrell T, Efros AA, Wang O, Shechtman E (2017) Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems, pp 465–476

  164. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, pp 2223–2232

  165. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. ICCV

  166. Zhu L, Chen Y, Ghamisi P, Benediktsson JA (2018) Generative adversarial networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 56(9):5046–5063

    Google Scholar 

  167. Zhu X, Zhang L, Zhang L, Liu X, Shen Y, Zhao S (2020) GAN-based image super-resolution with a novel quality loss. Math Probl Eng 2020:1–12. https://doi.org/10.1155/2020/5217429

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Thapar Institute of Engineering and Technology, Patiala, India

    Akanksha Sharma & Neeru Jindal

  2. Department of Computer Science Engineering, Thapar Institute of Engineering and Technology, Patiala, India

    P. S. Rana

Authors
  1. Akanksha Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neeru Jindal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. S. Rana
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Neeru Jindal.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, A., Jindal, N. & Rana, P.S. Potential of generative adversarial net algorithms in image and video processing applications– a survey. Multimed Tools Appl 79, 27407–27437 (2020). https://doi.org/10.1007/s11042-020-09308-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09308-4

Keywords

Search

Navigation