research-article

SPGAN: Face Forgery Using Spoofing Generative Adversarial Networks

Authors:

Yuanzhouhan CaoAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 17, Issue 1s

Article No.: 19, Pages 1 - 20

https://doi.org/10.1145/3432817

Published: 31 March 2021 Publication History

Abstract

Current face spoof detection schemes mainly rely on physiological cues such as eye blinking, mouth movements, and micro-expression changes, or textural attributes of the face images [9]. But none of these methods represent a viable mechanism for makeup-induced spoofing, especially since makeup has been widely used. Compared with face alteration techniques such as plastic surgery, makeup is non-permanent and cost efficient, which makes makeup-induced spoofing become a realistic threat to the integrity of a face recognition system. To solve this problem, we propose a generative model to construct spoofing face images (confusing face images) for improving the accuracy and robustness of automatic face recognition. Our network structure is composed of two separate parts, with one using inter-attention mechanism to obtain interested face region, and another using intra-attention to translate imitation style with preserving imitation style-excluding details. These two attention mechanisms can precisely learn imitation style, where inter-attention pays more attention to imitation regions of image and intra-attention learns face attributes with long distance in image. To effectively discriminate generated images, we introduce an imitation style discriminator. Our model (SPGAN) generates face images that transfer the imitation style from target to subject image and preserve the imitation-excluding features. Experimental results demonstrate the performance of our model in improving quality of imitated face images.

References

[1]

Andre Anjos and Sebastien Marcel. 2011. Counter-measures to photo attacks in face recognition: A public database and a bas. In IJCB. 1–7.

Digital Library

[2]

Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein GAN. In ICML.

[3]

Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. 2017. Unsupervised pixel-level domain adaptation with generative adversarial networks. In CVPR.

[4]

Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. 2018. PairedCycleGAN asymmetric style transfer for applying and removing makeup. In CVPR. 40–48.

[5]

Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li. 2017. Mode regularized generative adversarial networks. In ICLR.

[6]

Cunjian Chen, Antitza Dantcheva, and Arun Ross. 2016. An ensemble of patch-based subspaces for makeup-robust face recognition. Inf. Fusion 32, 32 (2016), 80–92.

Digital Library

[7]

Li Chen, Kun Zhou, and Stephen Lin. 2015. Simulating makeup through physics-based manipulation of intrinsic image layers. In CVPR.

[8]

Liang Chieh Chen, George Papandreou, Iasonas Kokkinos, and Kevin Murphy. 2018. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 4 (2018), 834–848.

[9]

WenTing Chen, Xinpeng Xie, Xi Jia, and Linlin Shen. 2018. Texture deformation based generative adversarial networks for face editing. ArXiv Preprint ArXiv:1812.09832 (2018).

[10]

Jianpeng Cheng, Dong Li, and Mirella Lapata. 2016. Long short-term memory-networks for machine reading. In EMNLP. 551–561.

[11]

Songtao Ding, Shiru Qu, Yuling Xi, and Shaohua Wan. 2019. Stimulus-driven and concept-driven analysis for image caption generation. Neurocomputing (2019), 520–530.

[12]

Harshala Gammulle, Tharindu Fernando, Simon Denman, Sridha Sridharan, and Clinton Fookes. 2019. Coupled generative adversarial network for continuous fine-grained action segmentation. In WACV. 200–209.

[13]

Zan Gao, Yinming Li, and Shaohua Wan. 2020. Exploring deep learning for view-based 3D model retrieval. ACM Trans. Multimedia Comput. Commun. Applic. 16, 1 (2020), 1–21.

Digital Library

[14]

Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. Improved training of Wasserstein GANs. In NIPS. 5769–5779.

Digital Library

[15]

Dong Hao, Neekhara Paarth, and Wu Chao. 2017. Unsupervised image-to-image translation with generative adversarial networks. arXiv:CVPR (2017).

[16]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML. 448–456.

Digital Library

[17]

T. Darrell, J. Long, and E. Shelhamer. 2015. Fully convolutional networks for semantic segmentation. In CVPR. 3431–3440.

[18]

Justin Johnson, Alexandre Alahi, and Li Fei Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In ECCV. 694–711.

[19]

Zhu Jun Yan, Park Taesung, and Isola Phillip. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2242–2251.

[20]

Mohammad Ahangar Kiasari, Dennis Singh Moirangthem, and Minho Lee. 2018. Coupled generative adversarial stacked auto-encoder: CoGASA. Neural Netw. 100 (2018), 1–9.

[21]

Olga Komarova. 2016. A neural algorithm of artistic style. J. Vision 16, 12 (2016), 326–326.

[22]

Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017. Fader networks manipulating images by sliding attributes. In NIPS. 5967–5976.

Digital Library

[23]

Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. 2016. Autoencoding beyond pixels using a learned similarity metric. In ICML. 1558–1566.

Digital Library

[24]

Christian Ledig, Lucas Theis, and Huszar. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR. 105–114.

[25]

Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, and Zehan Wang. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR. 105–114.

[26]

Lingzhi Li, Jianmin Bao, Yang Hao, Dong Chen, and Fang Wen. 2019. FaceShifter: Towards high fidelity and occlusion aware face swapping. In CVPR.

[27]

Minjun Li, Haozhi Huang, Lin Ma, Wei Liu, Tong Zhang, and Yu-Gang Jiang. 2018. Unsupervised image-to-image translation with stacked cycle-consistent adversarial networks. In ECCV. 184–199.

[28]

Mu Li, Wangmeng Zuo, and David Zhang. 2016. Convolutional network for attribute-driven and identity-preserving human face generation. Preprint arXiv 1608.06434 (2016).

[29]

Mu Li, Wangmeng Zuo, and David Zhang. 2016. Deep identity aware transfer of facial attributes. Preprint arXiv 1610.05586 (2016).

[30]

Zhouhan Lin, Minwei Feng, Cicero Nogueira Dos Santos, Yu Mo, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. Preprint arXiv:1703.03130 (2017).

[31]

Xiaowei Liu, Kenli Li, and Keqin Li. 2019. Attentive semantic and perceptual faces completion using self-attention generative adversarial networks. Neural Proc. Lett. (2019).

[32]

Chen Long, Hanwang Zhang, Jun Xiao, Liqiang Nie, and Tat Seng Chua. 2017. SCA-CNN: Spatial and channel-wise attention in convolutional networks for image captioning. In CVPR.

[33]

Xiaofeng Mao, Shuhui Wang, Liying Zheng, and Qingming Huang. 2018. Semantic invariant cross-domain image generation with generative adversarial networks [J]. Neurocomputing 293, 7 (2018), 55--63.

[34]

Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. 2017. Unrolled generative adversarial networks. In ICLR.

[35]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. Preprint arXiv 1411.1784 (2014).

[36]

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. Preprint ArXiv (2018).

[37]

Thomas Brox, Olaf Ronneberger, and Philipp Fischer. 2015. U-Net: Convolutional networks for biomedical image segmentation. In MICCAI.

[38]

Andrew Zisserman, Omkar M. Parkhi, and Andrea Vedaldi. 2015. Deep face recognition. In BMVC.

[39]

Ankur P. Parikh, Oscar Tckstrm, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In EMNLP. 2249–2255.

[40]

Keyurkumar Patel, Hu Han, and Anil K. Jain. 2016. Secure face unlock: Spoof detection on smartphones. IEEE Trans. Inf. Forens. Secur. 11, 10 (2016), 2268–2283.

Digital Library

[41]

Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. ArXiv Preprint ArXiv:1705.04304 (2017).

[42]

Guim Perarnau, Joost van de Weijer, Bogdan Raducanu, and Jose M. Álvarez 2016. Invertible conditional GANs for image editing. arXiv preprint arXiv 1611.06355 (2016).

[43]

Albert Pumarola, Antonio Agudo, Aleix M. Martinez, Alberto Sanfeliu, and Francesc Moreno-Noguer. 2018. GANimation: Anatomically-aware facial animation from a single image. In ECCV. 835–851.

[44]

Weichao Qiu and Alan Yuille. 2016. UnrealCV: Connecting computer vision to unreal engine. In ECCV. 909–916.

[45]

Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint ArXiv.

[46]

Stephan R. Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun. 2016. Playing for data: Ground truth from computer games. In ECCV. 102–118.

[47]

Shao Rui, Xiangyuan Lan, Jiawei Li, and Yuen Pong. 2019. Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In CVPR. 10023–10031.

[48]

Andrei A. Rusu, Matej Vecerik, Thomas Rothorl, Nicolas Heess, Razvan Pascanu, and Raia Hadsell. 2016. Sim-to-real robot learning from pixels with progressive nets. Preprint arXiv:1809.07480 (2016).

[49]

Tim Salimans, Han Zhang, Alec Radford, and Dimitris Metaxas. 2018. Improving GANs using optimal transport. In ICLR.

[50]

Mathew Salvaris, Danielle Dean, and Wee Hyong Tok. 2018. Generative adversarial networks. Preprint arXiv 1406.2661 (2018).

[51]

Wei Shen and Rujie Liu. 2017. Learning residual images for face attribute manipulation. In CVPR. 1225–1233.

[52]

Casper Kaae Sonderby, Jose Caballero, Lucas Theis, and Wenzhe Shi. 2017. Amortised MAP inference for image super-resolution. In ICLR.

[53]

C. Dong, T. Li, R. Qian. 2018. BeautyGAN: Instrance-level facial makeup transfer with deep generative adversarial network. In MM. 645–653.

Digital Library

[54]

Dickson Tong, Chi Keung Tang, Michael S. Brown, and Ying Qing Xu. 2008. Example-based cosmetic transfer. Cosmet. Toilet.211–218.

Digital Library

[55]

Eric Tzeng, Coline Devin, Judy Hoffman, Chelsea Finn, and Xingchao Peng. 2016. Towards adapting deep visuomotor representations from simulated to real environments. arXiv preprint arXiv 1511.07111 (2016).

[56]

Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, and Kilian Weinberger. 2017. Deep feature interpolation for image content changes. In CVPR. 6090–6099.

[57]

Shaohua Wan, Yu Xia, Lianyong Qi, Yee Hong Yang, and Mohammed Atiquzzaman. 2020. Automated colorization of a grayscale image with seed points propagation. IEEE Trans. Multimedia 99 (2020), 1–1.

[58]

Shuyang Wang and Fu Yun. 2016. Face behind makeup. In AAAI.

Digital Library

[59]

E. Kolve, Y. Zhu, and R. Mottaghi. 2017. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In ICRA. 3357–3364.

[60]

Junichi Yamagishi, Tomi H. Kinnunen, and Nicholas Evans. 2017. Introduction to the issue on spoofing and countermeasures for automatic speaker verification. IEEE J. Select. Topics Sig. Proc. 11, 4 (2017), 585–587.

[61]

Donggeun Yoo, Namil Kim, Sunggyun Park, Anthony S. Paek, and In So Kweon. 2016. Pixel-level domain transfer. In ECCV. 517–532.

[62]

Munyoung Kim Yunjey Choi, Minje Choi, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN:Unified generative adversarial networks for multidomain image-to-image translation. In CVPR. 8789–8797.

[63]

H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena. 2019. Self-attention generative adversarial networks. In ICML. 7354–7363.

[64]

Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris Metaxas. 2017. StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. In ICCV. 5908–5916.

[65]

Richard Zhang, Phillip Isola, and Alexei A. Efros. 2016. Colorful image colorization. In ECCV. 649–666.

[66]

Junbo Zhao, Michael Mathieu, and Yann Lecun. 2016. Energy-based generative adversarial network. Preprint ArXiv (2016).

[67]

Shuchang Zhou, Taihong Xiao, Yi Yang, Dieqiao Feng, Qinyao He, and Weiran He. 2017. GeneGAN: Learning object transfiguration and attribute subspace from unpaired data. Preprint arXiv 1705.04932 (2017).

[68]

Jun Yan Zhu, Taesung Park, and Phillip Isola. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV. 2242–2251.

[69]

Jun Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros. 2016. Generative visual manipulation on the natural image manifold. In ECCV.

Cited By

Peng FQin LLong MLi J(2024)Detection of Adversarial Facial Accessory Presentation Attacks Using Local Face DifferentialACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364383120:7(1-28)Online publication date: 27-Mar-2024
https://dl.acm.org/doi/10.1145/3643831
Cheng YYan YZhu WPan YPan BYang X(2024)Head3D: Complete 3D Head Generation via Tri-plane Feature DistillationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363571720:6(1-20)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1145/3635717
Huang CPeng XLiu DLu Y(2023)Text Image Super-Resolution Guided by Text Structure and Embedding PriorsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359592419:6(1-18)Online publication date: 5-May-2023
https://dl.acm.org/doi/10.1145/3595924
Show More Cited By

Index Terms

SPGAN: Face Forgery Using Spoofing Generative Adversarial Networks
1. Networks
  1. Network architectures
    1. Network design principles
      1. Naming and addressing

Recommendations

Latent Spatial Features Based on Generative Adversarial Networks for Face Anti-spoofing
Biometric Recognition
Abstract
With the wide deployment of the face recognition system, many face attacks, such as print attack, video attack and 3D face mask, have emerged. Face anti-spoofing is very important to protect face recognition system from attack. This paper proposes ...
Face makeup transfer based on Generative Adversarial Network
ICAIP '23: Proceedings of the 2023 7th International Conference on Advances in Image Processing

The goal of the face makeup transfer algorithm is to transfer the face makeup from the specified reference image to the non-makeup face in the target image while maintaining the face structure of the target image during the transfer process. Aiming at ...
FH-GAN: Face Hallucination and Recognition Using Generative Adversarial Network
Neural Information Processing
Abstract
There are many factors affecting visual face recognition, such as low resolution images, aging, illumination and pose variance, etc. One of the most important problem is low resolution face images which can result in bad performance on face ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 17, Issue 1s

January 2021

353 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3453990

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 March 2021

Accepted: 01 October 2020

Revised: 01 October 2020

Received: 01 March 2020

Published in TOMM Volume 17, Issue 1s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
282
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Peng FQin LLong MLi J(2024)Detection of Adversarial Facial Accessory Presentation Attacks Using Local Face DifferentialACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364383120:7(1-28)Online publication date: 27-Mar-2024
https://dl.acm.org/doi/10.1145/3643831
Cheng YYan YZhu WPan YPan BYang X(2024)Head3D: Complete 3D Head Generation via Tri-plane Feature DistillationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363571720:6(1-20)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1145/3635717
Huang CPeng XLiu DLu Y(2023)Text Image Super-Resolution Guided by Text Structure and Embedding PriorsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359592419:6(1-18)Online publication date: 5-May-2023
https://dl.acm.org/doi/10.1145/3595924
Hsu WJian P(2023)Recurrent Multi-scale Approximation-Guided Network for Single Image Super-ResolutionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359261319:6(1-21)Online publication date: 14-Apr-2023
https://dl.acm.org/doi/10.1145/3592613
Wang ZXu YWu LHan HMa YLi Z(2022)Improving Face Anti-spoofing via Advanced Multi-perspective Feature LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/357566019:6(1-18)Online publication date: 8-Dec-2022
https://dl.acm.org/doi/10.1145/3575660

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents