skip to main content
10.1145/3573942.3574030acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaiprConference Proceedingsconference-collections
research-article

A Novel Face Forgery Detection Method Based on Augmented Dual-Stream Networks

Published: 16 May 2023 Publication History

Abstract

The current face forgery methods based on deep learning are becoming more mature and abundant, and existing detection techniques have some limitations and applicability issues that make it difficult to effectively detect such behaviour. In this paper, we propose an enhanced dual-stream FC_2_stream network model based on dual-stream networks to detect forged regions in manipulated face images through end-to-end training of the images. The RGB stream is used to extract features from the RGB image to find the forged traces; the noise stream uses the filtering layer of the SRM (Steganalysis Rich Model) model to extract the noise features and find the inconsistency between the noise in the real region and the forged region in the fake face, then the features of the two streams are fused with a bilinear pooling layer to predict the forged region, and finally the forged region is determined by whether the blending boundary of the forged image is displayed to determine the image authenticity. Experiments conducted on four benchmark datasets show that our model is still effective against forgeries generated by unknown face manipulation methods, and also demonstrate the superior generalisation capability of our model.

References

[1]
DeepFakes. www.github.com/deepfakes/faceswap. Accessed: 2019-09-18. 1, 5
[2]
FaceSwap. www.github.com/MarekKowalski/ FaceSwap. Accessed: 2019-09-30. 1, 5
[3]
Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. 2018. Face2Face: real-time face capture and reenactment of RGB videos. Commun. ACM 62, 1 (January 2019), 96-104. https://doi.org/10.1145/3292039
[4]
Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2019. Deferred neural rendering: image synthesis using neural textures. ACM Trans. Graph. 38, 4, Article 66 (August 2019), 12 pages. https://doi.org/10.1145/3306346.3323035
[5]
Chollet F . Xception: Deep Learning with Depthwise Separable Convolutions[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1800-1807, 2017.
[6]
Li L, Bao J, Zhang T, Face X-Ray for More General Face Forgery Detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5000-5009, 2020.
[7]
Sun K, Zhao Y, Jiang B, High-Resolution Representations for Labeling Pixels and Regions[J]. 2019.
[8]
Rssler A, Cozzolino D, Verdoliva L, FaceForensics++: Learning to Detect Manipulated Facial Images[J], pages 1-11, 2019.
[9]
Jinhui Yuan, Jianmin Li, and Bo Zhang. 2007. Exploiting spatial context constraints for automatic image region annotation. in Proceedings of the 15th ACM international conference on Multimedia (MM '07). Association for Computing Machinery, New York, NY, USA, 595-604. https://doi.org/10.1145/1291233.1291379
[10]
Frank J, Eisenhofer T, Schnherr L, Leveraging Frequency Analysis for Deep Fake Image Recognition[C]// 2020.
[11]
Haliassos A, Vougioukas K, Petridis S, Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection[C]// Computer Vision and Pattern Recognition. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5037-5047, 2021.
[12]
Luo Y, Zhang Y, Yan J, Generalizing Face Forgery Detection with High-frequency Features[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16312-16321, 2021.
[13]
Dang H, Liu F, Stehouwer J, On the Detection of Digital Face Manipulation[J]. 2019. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5780-5789, 2020.
[14]
Yang C, Lim S N . One-Shot Domain Adaptation For Face Generation[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5920-5929, 2020.
[15]
Xuan X, Peng B, Wang W, On the generalization of GAN image forensics[J]. 2019.
[16]
Cozzolino D, Thies J, Rssler A, ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection[J]. 2018.
[17]
Li Y, Lyu S . Exposing DeepFake Videos By Detecting Face Warping Artifacts[C]// 2018.
[18]
Jessica Fridrich and Jan Kodovsky. 2012. Rich Models for Steganalysis of Digital Images. trans. Info. for. sec. 7, 3 (June 2012), 868-882. https://doi.org/10.1109/TIFS.2012.2190402.
[19]
He, K., Gkioxari, G., Piotr Dollár, & Girshick, R. Mask r-cnn. IEEE International Conference on Computer Vision (ICCV), pages 2980-2988, 2017.
[20]
Cozzolino D, Poggi G, Verdoliva L . Splicebuster: A new blind image splicing detector[C]// 2015 IEEE International Workshop on Information Forensics and Security (WIFS), pages 1-6, 2016.
[21]
Yang G, Beijbom O, Ning Z, Compact Bilinear Pooling[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 317-326, 2016.
[22]
Deepfakedetection.https//aigogleblog.com/2019/09/contributing-data-to-deepfake detection.html.Accessed:2020-05-10.
[23]
Dolhansky B, Howes R, Pflaum B, The Deepfake Detection Challenge (DFDC) Preview Dataset[J]. 2019.
[24]
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics[J]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3204-3213, 2020.
[25]
Kingma D, Ba J . Adam: A Method for Stochastic Optimization[J]. Computer Science, 2014.

Index Terms

  1. A Novel Face Forgery Detection Method Based on Augmented Dual-Stream Networks
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Other conferences
          AIPR '22: Proceedings of the 2022 5th International Conference on Artificial Intelligence and Pattern Recognition
          September 2022
          1221 pages
          ISBN:9781450396899
          DOI:10.1145/3573942
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 16 May 2023

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. FC_2_stream Dual-stream Model
          2. Forged Images
          3. Mixing Bounding
          4. Noise Streams
          5. RGB Streams

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          • the Primary Research & Developement Plan of Shaanxi Province
          • the National Key R&D Program of China

          Conference

          AIPR 2022

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • 0
            Total Citations
          • 28
            Total Downloads
          • Downloads (Last 12 months)7
          • Downloads (Last 6 weeks)2
          Reflects downloads up to 01 Mar 2025

          Other Metrics

          Citations

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media