Skip to main content
SpringerLink
Log in

A shared updatable method of content regulation for deepfake videos based on blockchain

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the development of deep learning technologies, video face tampering technologies, represented by Deepfake, can easily generate fake face images of videos by modifying the original video with only a small amount of face images. Therefore, the detection of forged facial videos has become critical to internet content regulation. In this paper, a deep forgery face video detection method with fusion of frequency domain features and spatial domain features (FFS) is proposed to address the problem of deepfake face video detection. At first, the proposed method extracts the wavelet features of images with two-dimensional discrete wavelet transform, then extracts the multidimensional wavelet feature vectors of images according to n-level wavelet decomposition. The frequency domain features of the image extracted by the discrete Fourier transform are also cascaded and fused with the wavelet features. Besides, the proposed method can better address the overfitting problem of detection methods in practical internet application scenarios by establishing a shared updatable strategy. Finally, in order to improve the generalization ability of the detection model and address the problem that the model is vulnerable to malicious attacks under the above shared update strategy, blockchain technology is employed to implement an incentive mechanism. It can motivate participants to provide real and health data, and then achieve the establishment and maintenance of a good shared and renewable environment. Moreover, we use DeepfakeDetection and Celeb-DF datasets to conduct the experiments. Samples with different percentages of high-quality images are selected to simulate the complex environment of image quality in the Internet. Experimental results show that the proposed method can improve the performance of the face forgery detection model effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Nguyen TT, Nguyen CM, Nguyen DT et al (2019) Deep learning for deepfakes creation and detection: A survey[J]. arXiv preprint arXiv:1909.11573

  2. Chesney R, Citron D (2019) Deepfakes and the new disinformation war: The coming age of post-truth geopolitics[J]. Foreign Aff 98:147

    Google Scholar 

  3. Chesney B, Citron D (2019) Deep fakes: A looming challenge for privacy, democracy, and national security[J]. Calif L Rev 107:1753

    Google Scholar 

  4. Korshunov P, Marcel S (2019) Vulnerability assessment and detection of deepfake videos[C]//2019 International Conference on Biometrics (ICB). IEEE, pp 1–6

  5. Tolosana R, Vera-Rodriguez R, Fierrez J et al (2020) Deepfakes and beyond: A survey of face manipulation and fake detection[J]. Information Fusion 64:131–148

    Article  Google Scholar 

  6. Zheng Z, Xie S, Dai HN et al (2018) Blockchain challenges and opportunities: A survey[J]. Int J Web Grid Serv 14(4):352–375

    Article  Google Scholar 

  7. Clack CD, Bakshi VA, Braine L (2016) Smart contract templates: foundations, design landscape and research directions[J]. arXiv preprint arXiv:1608.00771

  8. Matern F, Riess C, Stamminger M (2019) Exploiting visual artifacts to expose deepfakes and face manipulations[C]//2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE, pp 83–92

  9. Nguyen HH, Yamagishi J, Echizen I (2019) Capsule-forensics: Using capsule networks to detect forged images and videos[C]//ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 2307–2311

  10. Dang H, Liu F, Stehouwer J et al (2020) On the detection of digital face manipulation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition. pp 5781–5790

  11. Durall Lopez R, Keuper M, Pfreundt FJ et al (2020) Unmasking DeepFakes with simple Features[J].

  12. Durall R, Keuper M, Keuper J (2020) Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE

  13. Liu H, Li X, Zhou W et al (2021) Spatial-phase shallow learning: rethinking face forgery detection in frequency domain[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 772–781

  14. Yang J, Li A, Xiao S et al (2021) MTD-Net: Learning to Detect Deepfakes Images by Multi-Scale Texture Difference[J]. IEEE Trans Inf Forensics Secur 16:4234–4245

    Article  Google Scholar 

  15. Fernandes S, Raj S, Ortiz E et al (2019) Predicting heart rate variations of deepfake videos using neural ode[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops

  16. Amerini I, Galteri L, Caldelli R et al (2019) Deepfake video detection through optical flow based cnn[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops

  17. Marra F, Gragnaniello D, Verdoliva L et al (2019) Do gans leave artificial fingerprints?[C]//2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, pp 506–511

  18. Yu N, Davis LS, Fritz M (2019) Attributing fake images to gans: Learning and analyzing gan fingerprints[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. pp 7556–7566

  19. Cozzolino D, Poggi G, Verdoliva L (2015) Splicebuster: A new blind image splicing detector[C]//2015 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, pp 1–6

  20. Vo HT, Mehedy L, Mohania M et al (2017) Blockchain-based data management and analytics for micro-insurance applications[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. pp 2539–2542

  21. Wang J, Li M, He Y et al (2018) A blockchain based privacy-preserving incentive mechanism in crowdsensing applications[J]. IEEE Access 6:17545–17556

    Article  Google Scholar 

  22. Liang X, Zhao J, Shetty S et al (2017) Integrating blockchain for data sharing and collaboration in mobile healthcare applications[C]//2017 IEEE 28th annual international symposium on personal, indoor, and mobile radio communications (PIMRC). IEEE, pp 1–5

  23. Truong NB, Sun K, Lee GM et al (2019) Gdpr-compliant personal data management: A blockchain-based solution[J]. IEEE Trans Inf Forensics Secur 15:1746–1761

    Article  Google Scholar 

  24. Waggoner B, Frongillo R, Abernethy JD (2015) A market framework for eliciting private data[J]. Adv Neural Inf Process Syst 28:3510–3518

    Google Scholar 

  25. Yang D, Xue G, Fang X et al (2012) Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing[C]//Proceedings of the 18th annual international conference on Mobile computing and networking. pp 173–184

  26. Rossler A, Cozzolino D, Verdoliva L et al (2019) Faceforensics++: Learning to detect manipulated facial images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. pp 1–11

  27. Li Y, Yang X, Sun P et al (2020) Celeb-df: A large-scale challenging dataset for deepfake forensics[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 3207–3216

  28. Jiang L, Li R, Wu W et al (2020) Deeperforensics-1.0: A large-scale dataset for real-world face forgery detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp 2889–2898

  29. He K, Zhang X, Ren S et al (2016) Identity mappings in deep residual networks[C]//European conference on computer vision. Springer, Cham, pp 630–645

  30. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1251–1258

  31. Afchar D, Nozick V, Yamagishi J et al (2018) Mesonet: a compact facial video forgery detection network[C]//2018 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, pp 1–7

  32. Nguyen HH, Yamagishi J, Echizen I (2019) Use of a capsule network to detect fake images and videos[J]. arXiv preprint arXiv:1910.12467

  33. Chen HS, Rouhsedaghat M, Ghani H et al (2021) DefakeHop: A Light-Weight High-Performance Deepfake Detector[C]//2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 1–6

  34. Yang J, Xiao S, Li A et al (2021) Detecting fake images by identifying potential texture difference[J]. Futur Gener Comput Syst 125:127–135

    Article  Google Scholar 

  35. Bonettini N, Cannas ED, Mandelli S et al (2021) Video face manipulation detection through ensemble of cnns[C]//2020 25th International Conference on Pattern Recognition (ICPR). IEEE, pp 5012–5019

Download references

Funding

This work has been supported by the National Social Science Fund of China under Grant 18BGL202.

Author information

Authors and Affiliations

  1. Department of Beijing Key Laboratory of Big Data Technology for Food Safety, School of Computer Science and Engineering, Beijing Technology and Business University, Beijing, 100048, China

    Dianhui Mao & Shuang Zhao

  2. Department of National Engineering, Laboratory for Agri-Product Quality Traceability, Beijing Technology and Business University, Beijing, 100048, China

    Dianhui Mao

  3. Department of Computer and Information Science, University of Macau, Macao, 999078, China

    Zhihao Hao

Authors
  1. Dianhui Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhihao Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuang Zhao.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mao, D., Zhao, S. & Hao, Z. A shared updatable method of content regulation for deepfake videos based on blockchain. Appl Intell 52, 15557–15574 (2022). https://doi.org/10.1007/s10489-021-03156-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03156-x

Keywords

Search

Navigation