Abstract
The generative adversarial network (GAN) is one of the most popular tools in machine learning. In this work, we present a new framework, RGAN, which is a generative adversarial network with the Rényi loss function. We show that RGAN can generate more stable samples in terms of Fréchet inception distance. Furthermore, we derive the optimal discriminator for the Rényi loss function and show why our loss function can increase the stability by employing functional derivatives.

Similar content being viewed by others
References
Antipov G, Baccouche M, Dugelay JL. Face aging with conditional generative adversarial networks. In: 2017 IEEE international conference on image processing (ICIP). IEEE; 2017.
Arjovsky M, Chintala S, Bottou L. Wasserstein gan. 2017. arXiv:1701.07875.
Baez JC. Rényi entropy and free energy. 2011. arXiv:1102.2098.
Bhatia H, Paul W, Alajaji F, Gharesifard B, Burlina P. Rényi generative adversarial networks. 2020. arXiv:2006.02479.
Borji A. Pros and cons of gan evaluation measures. Comput Vision Image Underst. 2019;179:41–65.
Bousmalis K, et al. Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Brock A, Donahue J, Simonyan K. Large scale gan training for high fidelity natural image synthesis. 2018. arXiv:1809.11096.
Brock A, Lim T, Ritchie JM, Weston N. Neural photo editing with introspective adversarial networks. 2016. arXiv:1609.07093.
Brody DC, Buckley IR, Constantinou IC. Option price calibration from rényi entropy. Phys Lett A. 2007;366(4–5):298–307.
Castillo E, Luceno A, Pedregal P. Composition functionals in calculus of variations: application to products and quotients. Math Models Methods Appl Sci. 2008;18(01):47–75.
Chen X, et al. Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in neural information processing systems. 2016.
Choi Y, et al. Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
Denton E, Gross S, Fergus R. Semi-supervised learning with context-conditional generative adversarial networks. 2016. arXiv:1611.06430.
Denton EL, Chintala S, Fergus R. Deep generative image models using a laplacian pyramid of adversarial networks. In: Advances in neural information processing systems. 2015.
Donahue C, McAuley J, Puckette M. Adversarial audio synthesis. 2018. arXiv:1802.04208.
Donahue J, Krähenbühl P, Darrell T. Adversarial feature learning. 2016. arXiv:1605.09782.
Engel E, Dreizler RM. Density functional theory. New York: Springer; 2013.
Fedus W, Goodfellow I, Dai AM. Maskgan: better text generation via filling in the. 2018. arXiv:1801.07736.
Goodfellow I, et al. Generative adversarial nets. In: Advances in neural information processing systems. 2014.
Gulrajani I, et al. Improved training of wasserstein gans. In: Advances in neural information processing systems. 2017.
Hartmann KG, Schirrmeister RT, Ball T. Eeg-gan: Generative adversarial networks for electroencephalograhic (eeg) brain signals. 2018. arXiv:1806.01875.
Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in neural information processing systems; 2017. pp. 6626–37.
Hjelm RD, Jacob AP, Che T, Trischler A, Cho K, Bengio Y. Boundary-seeking generative adversarial networks. 2017. arXiv:1702.08431.
Ho J, Ermon S. Generative adversarial imitation learning. In: Advances in neural information processing systems. 2016.
Huang R, et al. Beyond face rotation: global and local perception gan for photorealistic and identity preserving frontal view synthesis. In: Proceedings of the IEEE international conference on computer vision. 2017.
Esteban C, Hyland SL, Rätsch G. Real-valued (medical) time series generation with recurrent conditional gans. 2017. arXiv:1706.02633.
Isola P, et al. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Jetchev N, Bergmann U, Vollgraf R. Texture synthesis with spatial generative adversarial networks. 2016. arXiv:1611.08207.
Jin Y, Zhang J, Li M, Tian Y, Zhu H, Fang Z. Towards the automatic anime characters creation with generative adversarial networks. 2017. arXiv:1708.05509.
Karras T, Aila T, Laine S, Lehtinen J. Progressive growing of gans for improved quality, stability, and variation. 2017. arXiv:1710.10196.
Karras T, et al. Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.
Ke L, Barnes M, Sun W, Lee G, Choudhury S, Srinivasa S. Imitation learning as \(f\)-divergence minimization. 2019. arXiv:1905.12888.
Kim T, Cha M, Kim H, Lee JK, Kim J. Learning to discover cross-domain relations with generative adversarial networks. 2017. arXiv:1703.05192.
Kingma DP, Ba J. Adam: A method for stochastic optimization. 2014. arXiv:1412.6980.
Kupyn O, et al. Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
Lassance N, Vrins F. Minimum Rényi entropy portfolios. Ann Oper Res. 2019. https://doi.org/10.1007/s10479-019-03364-2.
Lassner C, Pons-Moll G, Gehler PV. A generative model of people in clothing. In: Proceedings of the IEEE international conference on computer vision. 2017.
Ledig C, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Lenzi E, Mendes R, Da Silva L. Statistical mechanics based on renyi entropy. Phys A Stat Mech Appl. 2000;280(3–4):337–45.
Li C, Wand M. Precomputed real-time texture synthesis with markovian generative adversarial networks. In: European conference on computer vision. Springer; 2016. pp. 702–16.
Li J, et al. Perceptual generative adversarial networks for small object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Li Y, Turner RE. Rényi divergence variational inference. In: Advances in neural information processing systems 29. 2016. pp. 1073–81.
Liu MY, Tuzel O. Coupled generative adversarial networks. In: Advances in neural information processing systems; 2016.
Luc P, Couprie C, Chintala S. Verbeek J. Semantic segmentation using adversarial networks. 2016. arXiv:1611.08408.
Lucic M, et al. Are gans created equal? a large-scale study. In: Advances in neural information processing systems. 2018.
Ma L, et al. Pose guided person image generation. In: Advances in neural information processing systems. 2017.
Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B. Adversarial autoencoders. 2015. arXiv:1511.05644.
Mao X, et al. Least squares generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision. 2017.
Metz L, Poole B, Pfau D, Sohl-Dickstein J. Unrolled generative adversarial networks. 2016. arXiv:1611.02163.
Mirza M, Osindero S. Conditional generative adversarial nets. 2014. arXiv:1411.1784.
Müller-Lennert M, Dupuis F, Szehr O, Fehr S, Tomamichel M. On quantum rényi entropies: A new generalization and some properties. J. Math. Phys. 2013;54(12):122203.
Nowozin S, Cseke B, Tomioka R. f-gan: training generative neural samplers using variational divergence minimization. 2016. arXiv:1606.00709.
Odena A. Semi-supervised learning with generative adversarial networks. 2016. arXiv:1606.01583.
Odena A, Olah C, Shlens J. Conditional image synthesis with auxiliary classifier gans. In: International conference on machine learning. PMLR. 2017.
Oh TH, et al. Speech2face: learning the face behind a voice. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2019.
de Oliveira L, Paganini M, Nachman B. Learning particle physics by example: location-aware generative adversarial networks for physics synthesis. Comput Softw Big Sci. 2017;1(1):4.
Ormos M, Zibriczky D. Entropy-based financial asset pricing. PLoS One. 2014;9(12):e115742.
Paganini M, de Oliveira L, Nachman B. Accelerating science with generative adversarial networks: an application to 3d particle showers in multilayer calorimeters. Phys Rev Lett. 2018;120(4):042003.
Parr RG. Density functional theory of atoms and molecules. In: Fukui K, Pullman B, editors. Horizons of quantum chemistry. Dordrecht: Springer; 1980. p. 5–15.
Pathak D, et al. Context encoders: feature learning by inpainting. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
Perarnau G, Van De Weijer J, Raducanu B, Álvarez JM. Invertible conditional gans for image editing; 2016. arXiv:1611.06355.
Qi G-J. Loss-sensitive generative adversarial networks on lipschitz densities. Int J Comput Vis. 2020;128(5):1118–40.
Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks; 2015. arXiv:1511.06434.
Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H. Generative adversarial text to image synthesis; 2016. arXiv:1605.05396.
Rényi A. On measures of information and entropy. In: Proceedings of the 4th Berkeley symposium on mathematics, statistics and probability, vol. 1, No. 547. 1961.
Sakaguchi H. Renyi entropy and statistical mechanics. Progr Theor Phys. 1989;81(4):732–7.
Salimans T, et al. Improved techniques for training gans. In: Advances in neural information processing systems 29. 2016. p. 2234–42.
Schawinski K, Zhang C, Zhang H, Fowler L, Santhanam GK. Generative adversarial networks recover features in astrophysical images of galaxies beyond the deconvolution limit. Mon Not R Astron Soc Lett. 2017;467(1):L110–4.
Schlegl T, et al. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Niethammer M. et al, editors. International conference on information processing in medical imaging. Cham: Springer; 2017. p. 146–57.
Schmidt V, Luccioni A, Mukkavilli SK, Balasooriya N, Sankaran K, Chayes J, Bengio Y. Visualizing the consequences of climate change using cycle-consistent adversarial networks; 2019. arXiv:1905.03709.
Shmelkov K, Schmid C, Alahari K. How good is my GAN? In: Proceedings of the European Conference on Computer Vision (ECCV). 2018.
Souly N, Spampinato C, Shah M. Semi supervised semantic segmentation using generative adversarial network. In: Proceedings of the IEEE international conference on computer vision. 2017.
Sriram A, et al. Robust speech recognition using generative adversarial networks. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE; 2018.
Taigman Y, Polyak A, Wolf L. Unsupervised cross-domain image generation. 2016. arXiv:1611.02200.
Trefethen LN, Bau III D. Numerical linear algebra, vol. 50. Siam; 1997.
Wang K, Zhang J, Sun S, Wang Y, Xiang F, Xie L. Investigating generative adversarial networks based speech dereverberation for robust speech recognition. 2018. arXiv:1803.10132.
Wang T-C, et al. High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
Wang X, et al. Esrgan: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV). 2018.
Wu H, et al. Gp-gan: towards realistic high-resolution image blending. In: Proceedings of the 27th ACM international conference on multimedia. 2019.
Xu M, Shang P, Zhang S. Multiscale analysis of financial time series by Rényi distribution entropy. Phys A Stat Mech Appl. 2019;536:120916.
Yang LC, Chou SY, Yang YH. Midinet: A convolutional generative adversarial network for symbolic-domain music generation; 2017. arXiv:1703.10847.
Yi Z, et al. Dualgan: unsupervised dual learning for image-to-image translation. In: Proceedings of the IEEE international conference on computer vision. 2017.
Zhang H, Goodfellow I, Metaxas D, Odena A. Self-attention generative adversarial networks; 2018. arXiv:1805.08318.
Zhang H, et al. Stackgan: text to photo-realistic image synthesis with stacked generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision. 2017.
Zhavoronkov A, Ivanenkov YA, Aliper A, Veselov MS, Aladinskiy VA, Aladinskaya AV, Terentiev VA, Polykovskiy DA, Kuznetsov MD, Asadulaev A, et al. Deep learning enables rapid identification of potent ddr1 kinase inhibitors. Nature Biotechnol. 2019;37(9):1038–40.
Zhu J-Y, et al. Generative visual manipulation on the natural image manifold. In: Leibe B, Matas J, Sebe N, Welling M, editors. European conference on computer vision. Cham: Springer. 2016. p. 597–613.
Zhu J-Y, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision. 2017.
Funding
Not applicable (this work is not founded).
Author information
Authors and Affiliations
Contributions
All authors have contributed to this work.
Corresponding author
Ethics declarations
Conflict of Interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Availability of Data and Materials (Data Transparency)
The data used in the experiments are publicly available.
Code Availability (Software Application or Custom Code)
The code for experiments is available upon request.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sarraf, A., Nie, Y. RGAN: Rényi Generative Adversarial Network. SN COMPUT. SCI. 2, 17 (2021). https://doi.org/10.1007/s42979-020-00403-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42979-020-00403-9