Abstract
The rapid progress of face manipulation technology has attracted people’s attention. At present, a reliable edit detection algorithm is urgently needed to identify real and fake faces to ensure social credibility. Previous deep learning approaches formulate face manipulation detection as a binary classification problem. Many works struggle to focus on specific artifacts and generalize poorly. In this paper, we design reasonable auxiliary supervision to guide the network to learn discriminative and generalizable cues. A multi-scale framework is proposed to estimate the manipulation probability with texture map and blending boundary as auxiliary supervisions. These supervisions will guide the network to focus on the underlying texture information and blending boundary, making the learned features more generalized. Experiments on FaceForensics and FaceForensics++ datasets have demonstrated the effectiveness and generalization of our method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Faceswap. https://github.com/MarekKowalski/FaceSwap/. Accessed 26 Apr 2020
Deepfakes. https://www.deepfakes.club/openfaceswap-deepfakessoftware/. Accessed 26 Apr 2020
Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: MesoNet: a compact facial video forgery detection network. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), p. 17. IEEE (2018)
Ciftci, U.A., Demir, I.: FakeCatcher: detection of synthetic portrait videos using biological signals. arXiv preprint arXiv:1901.02212 (2019)
Cozzolino, D., Poggi, G., Verdoliva, L.: Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In: Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security, pp. 159–164 (2017)
Cozzolino, D., Thies, J., Röossler, A., Riess, C., Nießner, M., Verdoliva, L.: ForensicTransfer: weakly-supervised domain adaptation for forgery detection. arXiv preprint arXiv:1812.02510 (2018)
Cozzolino, D., Verdoliva, L.: Noiseprint: a CNN-based camera model fingerprint. IEEE Trans. Inf. Forensics Secur. 15, 144–159 (2019)
Dale, K., Sunkavalli, K., Johnson, M.K., Vlasic, D., Matusik, W., Pfister, H.: Video face replacement. In: Proceedings of the 2011 SIGGRAPH Asia Conference, pp. 1–10 (2011)
He, D.C., Wang, L.: Texture unit, texture spectrum, and texture analysis. IEEE Trans. Geosci. Remote Sens. 28(4), 509–512 (1990)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Kim, H., et al.: Deep video portraits. ACM Trans. Graph. (TOG) 37(4), 1–14 (2018)
Li, L., et al.: Face x-ray for more general face forgery detection. arXiv preprint arXiv:1912.13458 (2019)
Li, Y., Lyu, S.: Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656 (2018)
Nguyen, H.H., Fang, F., Yamagishi, J., Echizen, I.: Multi-task learning for detecting and segmenting manipulated facial images and videos. arXiv preprint arXiv:1906.06876 (2019)
Nguyen, H.H., Yamagishi, J., Echizen, I.: Use of a capsule network to detect fake images and videos. arXiv preprint arXiv:1910.12467 (2019)
Röossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: FaceForensics: a large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179 (2018)
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: FaceForensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1–11 (2019)
Songsri-in, K., Zafeiriou, S.: Complement face forensic detection and localization with faciallandmarks. arXiv preprint arXiv:1910.05455 (2019)
Sun, K., et al.: High-resolution representations for labeling pixels and regions. arXiv preprint arXiv:1904.04514 (2019)
Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946 (2019)
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2Face: real-time face capture and reenactment of RGB videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2387–2395 (2016)
Wang, S.Y., Wang, O., Zhang, R., Owens, A., Efros, A.A.: CNN-generated images are surprisingly easy to spot... for now. arXiv preprint arXiv:1912.11035 (2019)
Xie, Q., Hovy, E., Luong, M.T., Le, Q.V.: Self-training with noisy student improves imagenet classification. arXiv preprint arXiv:1911.04252 (2019)
Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8261–8265. IEEE (2019)
Zhang, X., Karaman, S., Chang, S.F.: Detecting and simulating artifacts in GAN fake images. arXiv preprint arXiv:1907.06515 (2019)
Acknowledgments
We thank for the support from National Natural Science Foundation of China (61972157, 61902129), Shanghai Pujiang Talent Program (19PJ1403100), Economy and Information Commission of Shanghai (XX-RGZN-01-19-6348), National Key Research and Development Program of China (No. 2019YFC1521104).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, X., Yao, T., Ding, S., Ma, L. (2020). Face Manipulation Detection via Auxiliary Supervision. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-63830-6_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63829-0
Online ISBN: 978-3-030-63830-6
eBook Packages: Computer ScienceComputer Science (R0)