Multi-Scale Feature Enhancement Network for Face Forgery Detection
Authors: 
Zhiyuan Ma, Xue Mei, Haoyang Chen, Jie ShenAuthors Info & Claims
Zhiyuan Ma, Xue Mei, Haoyang Chen, Jie ShenAuthors Info & ClaimsPages 28 - 32
Abstract
Nowadays, synthesizing realistic fake face images and videos becomes easy benefiting from the advance in generation technology. With the popularity of face forgery, abuse of the technology occurs from time to time, which promotes the research on face forgery detection to be an emergency. To deal with the potential risks, we propose a face forgery detection method based on multi-scale feature enhancement. Specifically, we analyze the forgery traces from the perspective of texture and frequency domain, respectively. We find that forgery traces are hard to be perceived by human eyes but noticeable in shallow layers of CNNs and middle-frequency domain and high-frequency domain. Hence, to reserve more forgery information, we design a texture feature enhancement module and a frequency domain feature enhancement module, respectively. The experiments on FaceForensics++ dataset and Celeb-DF dataset show that our method exceeds most existing networks and methods, which proves that our method has strong classification ability.
References
[1]
Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018. MesoNet: a Compact Facial Video Forgery Detection Network. In 2018 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 1–7. https://doi.org/10.1109/WIFS.2018.8630761.
[2]
Belhassen Bayar and Matthew C. Stamm. 2016. A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer. In Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security (Vigo, Galicia, Spain). Association for Computing Machinery, New York, NY, USA, 5–10. https://doi.org/10.1145/2909827.2930786.
[3]
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 8789–8797. https://doi.org/10.1109/CVPR.2018.00916.
[4]
François Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1800–1807. https://doi.org/10.1109/CVPR.2017.195.
[5]
Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil K. Jain. 2020. On the Detection of Digital Face Manipulation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 5780–5789. https://doi.org/10.1109/CVPR42600.2020.00582.
[6]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative Adversarial Networks. Commun. ACM 63, 11 (oct 2020), 139–144. https://doi.org/10.1145/3422622
[7]
Tackhyun Jung, Sangwon Kim, and Keecheon Kim. 2020. DeepVision: Deepfakes Detection Using Human Eye Blinking Pattern. IEEE Access 8 (2020), 83144–83154. https://doi.org/10.1109/ACCESS.2020.2988660.
[8]
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive Growing of GANs for Improved Quality, Stability, and Variation. arXiv preprint arXiv:1710.10196.
[9]
Tero Karras, Samuli Laine, and Timo Aila. 2021. A Style-Based Generator Architecture for Generative Adversarial Networks. IEEE Trans. Pattern Anal. Mach. Intell. 43, 12 (2021), 4217–4228. https://doi.org/10.1109/TPAMI.2020.2970919.
[10]
Davis E King. 2009. Dlib-ml: A machine learning toolkit. J. Mach. Learn. Res. 10 (2009), 1755–1758.
[11]
Aditi Kohli and Abhinav Gupta. 2021. Detecting DeepFake, FaceSwap and Face2Face facial forgeries using frequency CNN. Multimed. Tools Appl. 80, 12 (2021), 18461–18478. https://doi.org/10.1007/s11042-020-10420-8.
[12]
Pavel Korshunov and Sébastien Marcel. 2018. Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685.
[13]
Marek Kowalski. 2018. FaceSwap. https://github.com/marekkowalski/faceswap.
[14]
Jiaming Li, Hongtao Xie, Jiahong Li, Zhongyuan Wang, and Yongdong Zhang. 2021. Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 6458–6467. https://doi.org/10.1109/CVPR46437.2021.00639.
[15]
Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. 2020. Face X-Ray for More General Face Forgery Detection. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 5001–5010. https://doi.org/10.1109/CVPR42600.2020.00505.
[16]
Yuezun Li, Ming-Ching Chang, and Siwei Lyu. 2018. In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking. In 2018 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 1–7. https://doi.org/10.1109/WIFS.2018.8630787.
[17]
Yuezun Li and Siwei Lyu. 2018. Exposing DeepFake Videos By Detecting Face Warping Artifacts. arXiv preprint arXiv:1811.00656.
[18]
Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020. Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 3204–3213. http://doi.org/10.1109/CVPR42600.2020.00327.
[19]
Falko Matern, Christian Riess, and Marc Stamminger. 2019. Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE, 83–92. https://doi.org/10.1109/WACVW.2019.00020.
[20]
Huy H. Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. 2019. Multi-task Learning for Detecting and Segmenting Manipulated Facial Images and Videos. In 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS). IEEE, 1–8. https://doi.org/10.1109/BTAS46853.2019.9185974.
[21]
Huy H. Nguyen, Junichi Yamagishi, and Isao Echizen. 2019. Use of a capsule network to detect fake images and videos. arXiv preprint arXiv:1910.12467.
[22]
Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. 2016. Variational Autoencoder for Deep Learning of Images, Labels and Captions. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.). Vol. 29. Curran Associates, Inc.https://proceedings.neurips.cc/paper/2016/file/eb86d510361fc23b59f18c1bc9802cc6-Paper.pdf
[23]
Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. 2020. Thinking in frequency: Face forgery detection by mining frequency-aware clues. In European conference on computer vision. Springer, 86–103.
[24]
Geirhos Robert, Rubisch Patricia, Michaelis Claudio, Bethge Matthias, Wichmann Felix A., and Brendel Wieland. 2022. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231.
[25]
Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Niessner. 2019. FaceForensics++: Learning to Detect Manipulated Facial Images. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 1–11. http://doi.org/10.1109/ICCV.2019.00009.
[26]
Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2019. Deferred Neural Rendering: Image Synthesis Using Neural Textures. ACM Trans. Graph. 38, 4, Article 66 (jul 2019), 12 pages. https://doi.org/10.1145/3306346.3323035
[27]
Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. 2016. Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2387–2395. https://doi.org/10.1109/CVPR.2016.262.
[28]
Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia. 2020. Deepfakes and beyond: A Survey of face manipulation and fake detection. Information Fusion 64 (2020), 131–148. https://doi.org/10.1016/j.inffus.2020.06.014
[29]
Matt Tora. 2018. DeepFakes. https://github.com/deepfakes/faceswap/tree/v2.0.0.
[30]
Bo Wang, Yucai Li, Xiaohan Wu, Yanyan Ma, Zengren Song, and Mingkan Wu. 2022. Face Forgery Detection Based on the Improved Siamese Network. Secur. Commun. Netw. 2022 (2022), 13 pages. https://doi.org/10.1155/2022/5169873.
[31]
Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing Deep Fakes Using Inconsistent Head Poses. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 8261–8265. https://doi.org/10.1109/ICASSP.2019.8683164.
[32]
Peng Zhou, Xintong Han, Vlad I. Morariu, and Larry S. Davis. 2017. Two-Stream Neural Networks for Tampered Face Detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 1831–1839. https://doi.org/10.1109/CVPRW.2017.229.
[33]
JunYan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2242–2251. https://doi.org/10.1109/ICCV.2017.244.
Index Terms
- Multi-Scale Feature Enhancement Network for Face Forgery Detection
Recommendations
Face Forgery Detection via Texture and Saliency Enhancement
MultiMedia ModelingAbstractIn recent years, AI-driven advancements have resulted in increasingly sophisticated face forgery techniques, posing a challenge in distinguishing genuine images from manipulated ones. This presents significant societal and trust-related concerns. ...
Face forgery detection by progressively enhancing spatial and frequency-aware features
AbstractDue to the security issues caused by face synthesis technology, face forgery detection has received considerable attention. Therefore, there is an increasing necessity to develop effective and generalized face forgery detection models. Some ...
Deep learning technology for face forgery detection: A survey
AbstractCurrently, the rapid development of computer vision and deep learning has enabled the creation or manipulation of high-fidelity facial images and videos via deep generative approaches. This technology, also known as deepfake, has achieved ...
Highlights- A comprehensive review of recent deep facial forgery detection works.
- Review audio and audio-visual forgery detection, unlike most previous surveys.
- Including deepfake attribution, proactive detection, and adversarial learning.
Comments
Information & Contributors
Information
Published In
March 2023
193 pages
ISBN:9781450399531
DOI:10.1145/3589572
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICMVA 2023
ICMVA 2023: 2023 The 6th International Conference on Machine Vision and Applications
March 10 - 12, 2023
Singapore, Singapore
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 62Total Downloads
- Downloads (Last 12 months)19
- Downloads (Last 6 weeks)2
Reflects downloads up to 08 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format