Skip to main content
SpringerLink
Log in

GAN-generated fake face detection via two-stream CNN with PRNU in the wild

  • 1221: Deep Learning for Image/Video Compression and Visual Quality Assessment
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The rapid development of the generative adversarial networks (GANs) has made it an unprecedented success in image generation. The emergence of BigGAN, StyleGAN, and other advanced GAN makes the generated images more real and deceptive, which poses potential threats to national security, social stability, and personal privacy. In this paper, we proposed a new framework—two-stream CNN to detect GAN generated fake images, which contains RGB stream and Photo Response Non-Uniformity (PRNU) stream, respectively. In the preprocessing stage of the RGB stream, the use of random erasing enhances the diversity of samples and assists the network to pay more attention to the difference in GAN fingerprints in the image content. The PRNU stream’s construction is based on the uniqueness of PRNU features in real images and the robustness of the features to image transformation. The existence of PRNU guides the network to focus on the changes in the image pixel value itself and enhances the generalization performance of the network. Experimental results on multiple datasets show that the proposed method has apparent advantages in accuracy and generalization and is more robust to various image transformations, such as downsampling, JPEG compression, Gaussian noise, and Gaussian blur.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow IJ, Harp A, Irving G, Isard M, Jia Y, Józefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray DG, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker PA, Vanhoucke V, Vasudevan V, Viégas FB, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467

  2. Agarwal S, Farid H, Gu Y, He M, Nagano K, Li H (2019) Protecting world leaders against deep fakes. In: IEEE Conference on computer vision and pattern recognition workshops, CVPR workshops 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, pp 38–45

  3. Albright M, McCloskey S (2019) Source generator attribution via inversion. In: IEEE Conference on computer vision and pattern recognition workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, pp 96–103

  4. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Precup D, Teh YW (ed) Proceedings of the 34th International conference on machine learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of machine learning research. PMLR, pp 214–223

  5. Bay H, Tuytelaars T, Gool LV (2006) SURF: speeded up robust features. In: Leonardis A, Bischof H, Pinz A (ed) Computer Vision - ECCV 2006, 9th European conference on computer vision, Graz, Austria, May 7–13, 2006, Proceedings, Part I, volume 3951 of lecture notes in computer science. Springer, pp 404–417

  6. Berthelot D, Schumm T, Metz L (2017) BEGAN: boundary equilibrium generative adversarial networks. arXiv:1703.10717

  7. Chen M, Fridrich JJ, Goljan M (2007) Digital imaging sensor identification (further study). In: Delp III EJ, Wong PW (eds) Security, steganography, and watermarking of multimedia contents IX, San Jose, CA, USA, January 28, 2007, volume 6505 of SPIE proceedings. SPIE, pp 65050P

  8. Choi Y, Choi M-J, Kim M, Ha J-W, Kim S, Choo J (2018) Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: 2018 IEEE Conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. IEEE Computer Society, pp 8789–8797

  9. Dang H, Liu F, Stehouwer J, Liu X, Jain AK (2020) On the detection of digital face manipulation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, pp 5780–5789

  10. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014) Generative adversarial nets. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems 27: annual conference on neural information processing systems. Montreal, Quebec, Canada, pp 2672–2680

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, pp 770–778

  12. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv:1710.10196

  13. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: IEEE Conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, pp 4401–4410

  14. Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, pp 8107–8116

  15. Ke Tu, Li Hongbo, Sun Fuchun (2015) A statistical learning based image denoising approach. Front Comp Sci 9(5):713–719

    Article  Google Scholar 

  16. Khan RA, Meyer A, Konik H, Bouakaz S (2019) Saliency-based framework for facial expression recognition. Front Comp Sci 13(1):183–198

    Article  Google Scholar 

  17. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio, Y LeCun Y (ed) 3rd International conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, conference track proceedings

  18. Li C-L, Ravanbakhsh S, Póczos B (2016) Annealing gaussian into relu: a new sampling strategy for leaky-relu RBM. arXiv:1611.03879

  19. Li H, Li B, Tan S, Huang J (2018) Detection of deep network generated images using disparities in color components. arXiv:1808.07276

  20. Liu Z, Qi X, Torr PHS (2020) Global texture enhancement for fake face detection in the wild. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, pp 8057–8066

  21. Lukás Jan, Fridrich Jessica J, Goljan Miroslav (2006) Digital camera identification from sensor pattern noise. IEEE Trans Inf Forensics Secur 1(2):205–214

    Article  Google Scholar 

  22. Lukás J, Fridrich JJ, Goljan M (2005) Digital “bullet scratches” for images. In: Proceedings of the 2005 international conference on image processing, ICIP 2005, Genoa, Italy, September 11-14, 2005. IEEE, pp 65–68

  23. Lukás J, Fridrich JJ, Goljan M (2006) Detecting digital image forgeries using sensor pattern noise. In: Delp III EJ, Wong PW (eds) Security, steganography, and watermarking of multimedia contents VIII, San Jose, CA, USA, January 15, 2006, volume 6072 of SPIE Proceedings. SPIE, pp 60720Y

  24. Mao X, Li Q, Xie H, Lau RYK, Wang Z, Smolley SP (2017) Least squares generative adversarial networks. In: IEEE International conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017. IEEE Computer Society, pp 2813–2821

  25. Marra F, Gragnaniello D, Verdoliva L, Poggi G (2019) Do gans leave artificial fingerprints? In: 2nd IEEE conference on multimedia information processing and retrieval, MIPR 2019, San Jose, CA, USA, March 28-30, 2019. IEEE, pp 506–511

  26. McCloskey S, Albright M (2018) Detecting gan-generated imagery using color cues. arXiv:1812.08247

  27. Mo H, Chen B, Luo W (2018) Fake faces identification via convolutional neural network. In: Böhme R, Pasquini C, Boato G, Schöttle P (eds) Proceedings of the 6th ACM workshop on information hiding and multimedia security, Innsbruck, Austria, June 20-22, 2018. ACM, pp 43–47

  28. Nataraj L, Mohammed TM, Manjunath BS, Chandrasekaran S, Flenner A, Bappy JH, Roy-Chowdhury, AK (2020) Detecting GAN generated fake images using co-occurrence matrices. arXiv:1903.06836

  29. Shuang B (2017) Growing random forest on deep convolutional neural networks for scene categorization. Expert Syst Appl 71:279–287

    Article  Google Scholar 

  30. Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems 27: annual conference on neural information processing systems 2014, December 8-13 2014, Montreal, Quebec, Canada. pp 1988–1996

  31. Wang S-Y, Wang O, Zhang R, Owens A, Efros, AA (2019) Detecting photoshopped faces by scripting photoshop. In: 2019 IEEE/CVF International conference on computer vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, pp 10071–10080

  32. Wang X, Thome N, Cord M (2017) Gaze latent support vector machine for image classification improved by weakly supervised region selection. Pattern Recogn 72:59–71

    Article  Google Scholar 

  33. Yang X, Li Y, Lyu, S (2019) Exposing deep fakes using inconsistent head poses. In: IEEE International conference on acoustics, speech and signal processing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019. IEEE, pp 8261–8265

  34. Yu N, Davis L, Fritz, M (2018) Attributing fake images to gans: Analyzing fingerprints in generated images. arXiv:1811.08180

  35. Zhang X, Karaman S, Chang S-F (2019) Detecting and simulating artifacts in GAN fake images. In: IEEE international workshop on information forensics and security, WIFS 2019, Delft, The Netherlands, December 9-12, 2019. IEEE, pp 1–6

  36. Zhang Y, Zheng L, Thing VLL (2017) Automated face swapping and its detection. In: 2017 2nd International conference on signal and image processing

  37. Zheng L, Duffner S, Idrissi K, Garcia C, Baskurt A (2016) Siamese multi-layer perceptrons for dimensionality reduction and face identification. Multimed Tools Appl 75(9):5055–5073

    Article  Google Scholar 

  38. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2020) Random erasing data augmentation. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, The tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press, pp 13001–13008

  39. Zhou Z, Gu Y, Yu G (2021) Adversarial network embedding using structural similarity. Front Comp Sci 15(1):151603

    Article  Google Scholar 

  40. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International conference on computer vision, ICCV 2017, Venice, Italy, October 22-29, 2017. IEEE Computer Society, pp 2242–2251

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No.62072250, 61772281, 61702235, U1636117, U1804263, 62172435, 61872203 and 61802212), the Zhongyuan Science and Technology Innovation Leading Talent Project of China (Grant No.214200510019), the Plan for Scientific Talent of Henan Province (Grant No.2018JR0018), the Opening Project of Guangdong Provincial Key Laboratory of Information Security Technology (Grant No.2020B1212060078), and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund.

Author information

Authors and Affiliations

  1. Department of Computer and Software, Nanjing University of Information Science and Technology, 210044, Jiangsu, China

    Jinwei Wang, Kehui Zeng, Qilin Yin, Guangjie Liu & Sunil Kr. Jha

  2. State Key Laboratory of Mathematical Engineering and Advanced Computing, 450001, Henan, China

    Jinwei Wang & Xiangyang Luo

  3. Engineering Research Center of Digital Forensics, Ministry of Education, 210044, Nanjing, China

    Jinwei Wang

  4. State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, 100093, Beijing, China

    Jinwei Wang

  5. Shandong Provincial Key Laboratory of Computer Networks, Qilu University of Technology, 250353, Shangdong, China

    Bin Ma

  6. Faculty of Information Technology, University of Information Technology and Management, 35-225, Rzeszow, Poland

    Sunil Kr. Jha

Authors
  1. Jinwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kehui Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangyang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qilin Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guangjie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sunil Kr. Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangyang Luo.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Zeng, K., Ma, B. et al. GAN-generated fake face detection via two-stream CNN with PRNU in the wild. Multimed Tools Appl 81, 42527–42545 (2022). https://doi.org/10.1007/s11042-021-11592-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11592-7

Keywords

Search

Navigation