Skip to main content

Adversarially Robust Deepfake Detection via Adversarial Feature Similarity Learning

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14556))

Included in the following conference series:

Abstract

Deepfake technology has raised concerns about the authenticity of digital content, necessitating the development of effective detection methods. However, the widespread availability of deepfakes has given rise to a new challenge in the form of adversarial attacks. Adversaries can manipulate deepfake videos with small, imperceptible perturbations that can deceive the detection models into producing incorrect outputs. To tackle this critical issue, we introduce Adversarial Feature Similarity Learning (AFSL), which integrates three fundamental deep feature learning paradigms. By optimizing the similarity between samples and weight vectors, our approach aims to distinguish between real and fake instances. Additionally, we aim to maximize the similarity between both adversarially perturbed examples and unperturbed examples, regardless of their real or fake nature. Moreover, we introduce a regularization technique that maximizes the dissimilarity between real and fake samples, ensuring a clear separation between these two categories. With extensive experiments on popular deepfake datasets, including FaceForensics++, FaceShifter, and DeeperForensics, the proposed method outperforms other standard adversarial training-based defense methods significantly. This further demonstrates the effectiveness of our approach to protecting deepfake detectors from adversarial attacks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Pretrained model: https://github.com/ahaliassos/RealForensics.

  2. 2.

    Pretrained model: https://github.com/paarthneekhara/AdversarialDeepFakes.

  3. 3.

    https://github.com/yinglinzheng/FTCN.

  4. 4.

    https://github.com/chail/patch-forensics.

References

  1. Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: MesoNet: a compact facial video forgery detection network. In: WIFS, pp. 1–7 (2018)

    Google Scholar 

  2. Alnaim, N.M., Almutairi, Z.M., Alsuwat, M.S., Alalawi, H.H., Alshobaili, A., Alenezi, F.S.: DFFMD: a deepfake face mask dataset for infectious disease era with deepfake detection algorithms. IEEE Access, 16711–16722 (2023)

    Google Scholar 

  3. Carlini, N., Wagner, D.: Adversarial examples are not easily detected: Bypassing ten detection methods. In: AIS, pp. 3–14 (2017)

    Google Scholar 

  4. Chai, L., Bau, D., Lim, S.N., Isola, P.: What makes fake images detectable? understanding properties that generalize. In: ECCV, pp. 103–120 (2020)

    Google Scholar 

  5. Chen, G., et al.: Towards understanding and mitigating audio adversarial examples for speaker recognition. TDSC (2022)

    Google Scholar 

  6. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: CVPR, pp. 1251–1258 (2017)

    Google Scholar 

  7. Deepfakes: faceswap. In: GitHub (2017). Accessed 14 Jun 2023. https://github.com/deepfakes/faceswap

  8. Dong, S., Wang, J., Ji, R., Liang, J., Fan, H., Ge, Z.: Implicit identity leakage: the stumbling block to improving deepfake detection generalization. In: CVPR, pp. 3994–4004 (2023)

    Google Scholar 

  9. Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., Holz, T.: Leveraging frequency analysis for deep fake image recognition. In: ICML, pp. 3247–3258 (2020)

    Google Scholar 

  10. Gandhi, A., Jain, S.: Adversarial perturbations fool deepfake detectors. In: IJCNN, pp. 1–8 (2020)

    Google Scholar 

  11. Gao, G., Huang, H., Fu, C., Li, Z., He, R.: Information bottleneck disentanglement for identity swapping. In: CVPR, pp. 3404–3413 (2021)

    Google Scholar 

  12. Gao, Y., et al.: High-fidelity and arbitrary face editing. In: CVPR, pp. 16115–16124 (2021)

    Google Scholar 

  13. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (2015)

    Google Scholar 

  14. Guan, J., et al.: Delving into sequential patches for deepfake detection. arXiv preprint arXiv:2207.02803 (2022)

  15. Haliassos, A., Mira, R., Petridis, S., Pantic, M.: Leveraging real talking faces via self-supervision for robust forgery detection. In: CVPR, pp. 14950–14962 (2022)

    Google Scholar 

  16. Haliassos, A., Vougioukas, K., Petridis, S., Pantic, M.: Lips don’t lie: a generalisable and robust approach to face forgery detection. In: CVPR, pp. 5039–5049 (2021)

    Google Scholar 

  17. Hou, Y., Guo, Q., Huang, Y., Xie, X., Ma, L., Zhao, J.: Evading deepfake detectors via adversarial statistical consistency. In: CVPR, pp. 12271–12280 (2023)

    Google Scholar 

  18. Hussain, S., Neekhara, P., Jere, M., Koushanfar, F., McAuley, J.: Adversarial deepfakes: evaluating vulnerability of deepfake detectors to adversarial examples. In: WACV, pp. 3348–3357 (2021)

    Google Scholar 

  19. Jiang, L., Li, R., Wu, W., Qian, C., Loy, C.C.: DeeperForensics-1.0: a large-scale dataset for real-world face forgery detection. In: CVPR, pp. 2889–2898 (2020)

    Google Scholar 

  20. Jiang, Z., Chen, T., Chen, T., Wang, Z.: Robust pre-training by adversarial contrastive learning. In: NIPS, pp. 16199–16210 (2020)

    Google Scholar 

  21. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: CVPR, pp. 4401–4410 (2019)

    Google Scholar 

  22. Kireev, K., Andriushchenko, M., Flammarion, N.: On the effectiveness of adversarial training against common corruptions. In: UAI, pp. 1012–1021 (2022)

    Google Scholar 

  23. Li, L., Bao, J., Yang, H., Chen, D., Wen, F.: Advancing high fidelity identity swapping for forgery detection. In: CVPR, pp. 5074–5083 (2020)

    Google Scholar 

  24. Li, Z., et al.: Sibling-attack: rethinking transferable adversarial attacks against face recognition. In: CVPR, pp. 24626–24637 (2023)

    Google Scholar 

  25. Liang, K., Xiao, B.: StyLess: boosting the transferability of adversarial examples. In: CVPR, pp. 8163–8172 (2023)

    Google Scholar 

  26. Liu, B., Liu, B., Ding, M., Zhu, T., Yu, X.: TI2Net: temporal identity inconsistency network for deepfake detection. In: WACV, pp. 4691–4700 (2023)

    Google Scholar 

  27. Lo, S.Y., Patel, V.M.: Defending against multiple and unforeseen adversarial videos. In: TIP, pp. 962–973 (2021)

    Google Scholar 

  28. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: ICLR (2018)

    Google Scholar 

  29. Menon, A.K., Jayasumana, S., Rawat, A.S., Jain, H., Veit, A., Kumar, S.: Long-tail learning via logit adjustment. In: ICLR (2021)

    Google Scholar 

  30. Mumcu, F., Doshi, K., Yilmaz, Y.: Adversarial machine learning attacks against video anomaly detection systems. In: CVPR, pp. 206–213 (2022)

    Google Scholar 

  31. Neekhara, P.: Adversarial deepfake. In: GitHub (2019). Accessed 14 Jun 2023. https://github.com/paarthneekhara/AdversarialDeepFakes

  32. Neekhara, P., Dolhansky, B., Bitton, J., Ferrer, C.C.: Adversarial threats to deepfake detection: a practical perspective. In: CVPR, pp. 923–932 (2021)

    Google Scholar 

  33. Neekhara, P., Hussain, S., Pandey, P., Dubnov, S., McAuley, J., Koushanfar, F.: Universal adversarial perturbations for speech recognition systems. arXiv preprint arXiv:1905.03828 (2019)

  34. Qin, Y., Carlini, N., Cottrell, G., Goodfellow, I., Raffel, C.: Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In: ICML, pp. 5231–5240 (2019)

    Google Scholar 

  35. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: FaceForensics++: learning to detect manipulated facial images. In: CVPR, pp. 1–11 (2019)

    Google Scholar 

  36. Shahzad, S.A., Hashmi, A., Khan, S., Peng, Y.T., Tsao, Y., Wang, H.M.: Lip sync matters: a novel multimodal forgery detector. In: APSIPA, pp. 1885–1892 (2022)

    Google Scholar 

  37. Songja, R., Promboot, I., Haetanurak, B., Kerdvibulvech, C.: Deepfake AI images: should deepfakes be banned in Thailand? AI and Ethics, pp. 1–13 (2023)

    Google Scholar 

  38. Spivak, R.: deepfakes: the newest way to commit one of the oldest crimes. HeinOnline, p. 339 (2018)

    Google Scholar 

  39. Tran, D., Wang, H., Torresani, L., Feiszli, M.: Video classification with channel-separated convolutional networks. In: ICCV, pp. 5552–5561 (2019)

    Google Scholar 

  40. Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: CVPR, pp. 6450–6459 (2018)

    Google Scholar 

  41. Wang, H., et al.: Understanding the robustness of skeleton-based action recognition under adversarial attack. In: CVPR, pp. 14656–14665 (2021)

    Google Scholar 

  42. Yadlin-Segal, A., Oppenheim, Y.: Whose dystopia is it anyway? deepfakes and social media regulation. In: Convergence, pp. 36–51 (2021)

    Google Scholar 

  43. Yang, C., Ding, L., Chen, Y., Li, H.: Defending against GAN-based deepfake attacks via transformation-aware adversarial faces. In: IJCNN, pp. 1–8 (2021)

    Google Scholar 

  44. Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In: ICML, pp. 7472–7482 (2019)

    Google Scholar 

  45. Zheng, Y., Bao, J., Chen, D., Zeng, M., Wen, F.: Exploring temporal coherence for more general video face forgery detection. In: ICCV, pp. 15044–15054 (2021)

    Google Scholar 

Download references

Acknowledgment

This research is supported by National Science and Technology Council, Taiwan (R.O.C), under the grant number of NSTC-111-2634-F-002-022, 110-2221-E-001-009-MY2, 112-2634-F-001-001-MBK, and Academia Sinica under the grant number of AS-CDA-112-M09. In addition, we would like to express our gratitude for the valuable contributions and guidance from these organizations, which have been instrumental in achieving the goals of this research.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sarwar Khan .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khan, S., Chen, JC., Liao, WH., Chen, CS. (2024). Adversarially Robust Deepfake Detection via Adversarial Feature Similarity Learning. In: Rudinac, S., et al. MultiMedia Modeling. MMM 2024. Lecture Notes in Computer Science, vol 14556. Springer, Cham. https://doi.org/10.1007/978-3-031-53311-2_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53311-2_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53310-5

  • Online ISBN: 978-3-031-53311-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics