Skip to main content

Advertisement

Log in

Ldsfnet: lightweight dynamic selection fusion network for face forgery detection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Due to the serious security issues caused by face manipulation technology, face forgery detection has received widespread attention. Although existing detection models have achieved impressive results, they still struggle to find the proper balance between detection accuracy and model complexity. To solve this problem, we propose a lightweight dynamic selection fusion network (LDSFNet) to achieve a highly accurate lightweight face forgery detection model. Specifically, we design a two-branch network to capture subtle artifacts in spatial texture features and high-frequency noise features. Firstly, for the spatial texture capture branch, we design a texture feature enhancement (TFE) module, which facilitates the detection performance of the network by extracting the texture difference information between the global texture features and the local texture features, and also introduce a spatial group-wise enhance (SGE) module in the backbone network in order to enhance the forgery traces in the spatial features. Secondly, for the high-frequency noise capture branch, we utilize a learnable steganalysis rich model (SRM) filter to capture the noise inconsistency information in the forged faces, after which we mine and amplify the forged clues through the parameter-free attention (SimAM) module. Finally, we design a dynamic selection fusion (DSF) module to fully fuse spatial texture features and high-frequency noise features, and adaptively select spatial-frequency features to generate feature representations with strong discriminative power. Extensive experiments show that our proposed model outperforms previous work on multiple benchmark dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The original datasets have been published online. The code will be available at https://github.com/xiaozhangxiaowen/LDSFNet.

References

  1. Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: Mesonet: a compact facial video forgery detection network. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7 (2018). IEEE

  2. Matern, F., Riess, C., Stamminger, M.: Exploiting visual artifacts to expose deepfakes and face manipulations. In: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), pp. 83–92 (2019). IEEE

  3. Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8261–8265 (2019). IEEE

  4. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  5. Chollet, F.: Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

  6. Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR

  7. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  8. Zhao, Y., Jin, X., Gao, S., Wu, L., Yao, S., Jiang, Q.: Tan-gfd: generalizing face forgery detection based on texture information and adaptive noise mining. Appl. Intell. 53(16), 19007–19027 (2023)

    Article  MATH  Google Scholar 

  9. Zhao, H., Zhou, W., Chen, D., Wei, T., Zhang, W., Yu, N.: Multi-attentional deepfake detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2185–2194 (2021)

  10. Miao, C., Tan, Z., Chu, Q., Liu, H., Hu, H., Yu, N.: F 2 trans: High-frequency fine-grained transformer for face forgery detection. IEEE Trans. Inf. Forensics Secur. 18, 1039–1051 (2023)

    Article  MATH  Google Scholar 

  11. Luo, Y., Zhang, Y., Yan, J., Liu, W.: Generalizing face forgery detection with high-frequency features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16317–16326 (2021)

  12. Li, X., Hu, X., Yang, J.: Spatial group-wise enhance: Improving semantic feature learning in convolutional networks. arXiv preprint arXiv:1905.09646 (2019)

  13. Yang, L., Zhang, R.-Y., Li, L., Xie, X.: Simam: A simple, parameter-free attention module for convolutional neural networks. In: International Conference on Machine Learning, pp. 11863–11874 (2021). PMLR

  14. Zhou, P., Han, X., Morariu, V.I., Davis, L.S.: Learning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1053–1061 (2018)

  15. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: Faceforensics++: Learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1–11 (2019)

  16. Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-df: A large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3207–3216 (2020)

  17. Dolhansky, B., Howes, R., Pflaum, B., Baram, N., Ferrer, C.C.: The deepfake detection challenge (dfdc) preview dataset. arXiv preprint arXiv:1910.08854 (2019)

  18. Guo, Z., Yang, G., Chen, J., Sun, X.: Fake face detection via adaptive manipulation traces extraction network. Comput. Vis. Image Underst. 204, 103170 (2021)

    Article  Google Scholar 

  19. Yang, X., Li, Y., Qi, H., Lyu, S.: Exposing gan-synthesized faces using landmark locations. In: Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, pp. 113–118 (2019)

  20. Li, H., Li, B., Tan, S., Huang, J.: Identification of deep network generated images using disparities in color components. Signal Process. 174, 107616 (2020)

    Article  MATH  Google Scholar 

  21. Chen, H.-S., Hu, S., You, S., Kuo, C.-C.J., et al.: Defakehop++: An enhanced lightweight deepfake detector. APSIPA Transactions on Signal and Information Processing 11(2) (2022)

  22. Nguyen, H.H., Fang, F., Yamagishi, J., Echizen, I.: Multi-task learning for detecting and segmenting manipulated facial images and videos. In: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–8 (2019). IEEE

  23. Sun, Z., Han, Y., Hua, Z., Ruan, N., Jia, W.: Improving the efficiency and robustness of deepfakes detection through precise geometric features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3609–3618 (2021)

  24. Guo, Z., Wang, L., Yang, W., Yang, G., Li, K.: Ldfnet: Lightweight dynamic fusion network for face forgery detection by integrating local artifacts and global texture information. IEEE Transactions on Circuits and Systems for Video Technology (2023)

  25. Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., Zhao, J.: Deeprhythm: Exposing deepfakes with attentional visual heartbeat rhythms. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4318–4327 (2020)

  26. Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., Guo, B.: Face x-ray for more general face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5001–5010 (2020)

  27. Dong, X., Bao, J., Chen, D., Zhang, W., Yu, N., Chen, D., Wen, F., Guo, B.: Identity-driven deepfake detection. arXiv preprint arXiv:2012.03930 (2020)

  28. Wang, J., Wu, Z., Ouyang, W., Han, X., Chen, J., Jiang, Y.-G., Li, S.-N.: M2tr: Multi-modal multi-scale transformers for deepfake detection. In: Proceedings of the 2022 International Conference on Multimedia Retrieval, pp. 615–623 (2022)

  29. Miao, C., Chu, Q., Li, W., Li, S., Tan, Z., Zhuang, W., Yu, N.: Learning forgery region-aware and id-independent features for face manipulation detection. IEEE Trans. Biometrics, Behavior, and Identity Science 4(1), 71–84 (2021)

    Article  MATH  Google Scholar 

  30. Zhao, T., Xu, X., Xu, M., Ding, H., Xiong, Y., Xia, W.: Learning self-consistency for deepfake detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15023–15033 (2021)

  31. Shiohara, K., Yamasaki, T.: Detecting deepfakes with self-blended images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18720–18729 (2022)

  32. Dong, X., Bao, J., Chen, D., Zhang, T., Zhang, W., Yu, N., Chen, D., Wen, F., Guo, B.: Protecting celebrities from deepfake with identity consistency transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9468–9478 (2022)

  33. Huang, B., Wang, Z., Yang, J., Ai, J., Zou, Q., Wang, Q., Ye, D.: Implicit identity driven deepfake face swapping detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4490–4499 (2023)

  34. Dong, S., Wang, J., Ji, R., Liang, J., Fan, H., Ge, Z.: Implicit identity leakage: The stumbling block to improving deepfake detection generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3994–4004 (2023)

  35. Bai, W., Liu, Y., Zhang, Z., Li, B., Hu, W.: Aunet: Learning relations between action units for face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24709–24719 (2023)

  36. Li, X., Ni, R., Yang, P., Fu, Z., Zhao, Y.: Artifacts-disentangled adversarial learning for deepfake detection. IEEE Trans. Circuits Syst. Video Technol. 33(4), 1658–1670 (2022)

    Article  MATH  Google Scholar 

  37. Durall, R., Keuper, M., Pfreundt, F.-J., Keuper, J.: Unmasking deepfakes with simple features. arXiv preprint arXiv:1911.00686 (2019)

  38. Qian, Y., Yin, G., Sheng, L., Chen, Z., Shao, J.: Thinking in frequency: Face forgery detection by mining frequency-aware clues. In: European Conference on Computer Vision, pp. 86–103 (2020). Springer

  39. Masi, I., Killekar, A., Mascarenhas, R.M., Gurudatt, S.P., AbdAlmageed, W.: Two-branch recurrent network for isolating deepfakes in videos. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16, pp. 667–684 (2020). Springer

  40. Jia, G., Zheng, M., Hu, C., Ma, X., Xu, Y., Liu, L., Deng, Y., He, R.: Inconsistency-aware wavelet dual-branch network for face forgery detection. IEEE Trans. Biometrics, Behavior, and Identity Science 3(3), 308–319 (2021)

    Article  MATH  Google Scholar 

  41. Liu, J., Xie, J., Wang, Y., Zha, Z.-J.: Adaptive texture and spectrum clue mining for generalizable face forgery detection. IEEE Transactions on Information Forensics and Security (2023)

  42. Chen, S., Yao, T., Chen, Y., Ding, S., Li, J., Ji, R.: Local relation learning for face forgery detection. In: Proceedings of the AAAI Conference on Artificial Intelligence 35, 1081–1088 (2021)

    MATH  Google Scholar 

  43. Li, J., Xie, H., Yu, L., Gao, X., Zhang, Y.: Discriminative feature mining based on frequency information and metric learning for face forgery detection. IEEE Trans. Knowl. Data Eng. 35(12), 12167–12180 (2021)

    Article  MATH  Google Scholar 

  44. Gu, Q., Chen, S., Yao, T., Chen, Y., Ding, S., Yi, R.: Exploiting fine-grained face forgery clues via progressive enhancement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence 36, 735–743 (2022)

    Google Scholar 

  45. Wang, Y., Yu, K., Chen, C., Hu, X., Peng, S.: Dynamic graph learning with content-guided spatial-frequency relation reasoning for deepfake detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7278–7287 (2023)

  46. Daubechies, I.: Ten lectures on wavelets, philadelphia. Soc. Ind. Appl. Math. 61, 198–202 (1992)

    MATH  Google Scholar 

  47. Mallat, S.G.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)

    Article  MATH  Google Scholar 

  48. Fei, J., Dai, Y., Yu, P., Shen, T., Xia, Z., Weng, J.: Learning second order local anomaly for general face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20270–20280 (2022)

  49. Guo, Z., Jia, Z., Wang, L., Wang, D., Yang, G., Kasabov, N.: Constructing new backbone networks via space-frequency interactive convolution for deepfake detection. IEEE Transactions on Information Forensics and Security (2023)

  50. Yu, Z., Zhao, C., Wang, Z., Qin, Y., Su, Z., Li, X., Zhou, F., Zhao, G.: Searching central difference convolutional networks for face anti-spoofing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5295–5305 (2020)

  51. Zhou, P., Han, X., Morariu, V.I., Davis, L.S.: Lninearning rich features for image manipulation detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1053–1061 (2018)

  52. Li, X., Wang, W., Hu, X., Yang, J.: Selective kernel networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 510–519 (2019)

  53. King, D.E.: Dlib-ml: A machine learning toolkit. J. Mach. Learning Res. 10, 1755–1758 (2009)

    MATH  Google Scholar 

  54. Deepfakes. https://github.com/deepfakes/faceswap Accessed 2024-4-24

  55. Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2face: Real-time face capture and reenactment of rgb videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2387–2395 (2016)

  56. Faceswap. https://github.com/MarekKowalski/FaceSwap Accessed 2024-4-24

  57. Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: Image synthesis using neural textures. Acm Trans. Graphic (TOG) 38(4), 1–12 (2019)

    Article  Google Scholar 

  58. Guo, Z., Yang, G., Zhang, D., Xia, M.: Rethinking gradient operator for exposing ai-enabled face forgeries. Expert Syst. Appl. 215, 119361 (2023)

    Article  Google Scholar 

  59. Zhou, P., Han, X., Morariu, V.I., Davis, L.S.: Two-stream neural networks for tampered face detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1831–1839 (2017). IEEE

  60. Li, Y., Lyu, S.: Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656 (2018)

  61. Nguyen, H.H., Yamagishi, J., Echizen, I.: Capsule-forensics: Using capsule networks to detect forged images and videos. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2307–2311 (2019). IEEE

  62. Li, X., Lang, Y., Chen, Y., Mao, X., He, Y., Wang, S., Xue, H., Lu, Q.: Sharp multiple instance learning for deepfake video detection. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 1864–1872 (2020)

  63. Sun, K., Liu, H., Ye, Q., Gao, Y., Liu, J., Shao, L., Ji, R.: Domain general face forgery detection by learning to weight. In: Proceedings of the AAAI Conference on Artificial Intelligence 35, 2638–2646 (2021)

    MATH  Google Scholar 

  64. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)

  65. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)

Download references

Funding

The work was supported by the National Natural Science Foundation of China under Grant 62267007, Gansu Provincial Department of Education Industrial Support Plan Project under Grant 2022CYZC-16.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Shengcong Wen or Yongfeng Qi.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethics approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wen, S., Qi, Y., Liang, A. et al. Ldsfnet: lightweight dynamic selection fusion network for face forgery detection. SIViP 19, 100 (2025). https://doi.org/10.1007/s11760-024-03692-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03692-2

Keywords

Navigation