Skip to main content
Springer Nature Link
Log in

Exposing Audio-Visual Forgeries in Frequency Domain

  • Conference paper
  • First Online:
Biometric Recognition (CCBR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15352))

Included in the following conference series:

Abstract

Recently, the rapid development of deepfake technology attracted strong attention from the community. Some previous work on deepfake detection achieved good results in the frequency domain, which inspires us to combine frequency-domain information with temporal and spatial domains of visual to detect deepfakes. In addition, the audio signal can be represented in the frequency domain, so we can explore multimodal frequency-domain cues by combining audio and visual modalities. In this paper, we propose a Frequency-aware Audio-Visual Deepfake Detection(FAVDD) method. Specifically, we design a Frequency-Temporal-Spatial(FTS) visual encoder that extracts spatial, frequency, and temporal forgery cues and embeds them into visual features to form a unified representation. In addition, we project the audio signal into the frequency domain by Fourier transform and capture the forgery traces, which are later combined with visual features for deepfake detection. The results show that our proposed framework effectively combines multiple cues and achieves good results on three multimodal deepfake datasets.

Supported by National Natural Science Foundation of China under Grant 62076131 and Postdoctoral Fellowship Program of CPSF under Grant Number GZC20240743.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sun, Y., Tang, J., Sun, Z., Tistarelli, M.: Facial age and expression synthesis using ordinal ranking adversarial networks. IEEE Trans. Inf. Forensics Secur. 15, 2960–2972 (2020)

    Article  MATH  Google Scholar 

  2. Sun, Y., Tang, J., Shu, X., Sun, Z., Tistarelli, M.: Facial age synthesis with label distribution-guided generative adversarial network. IEEE Trans. Inf. Forensics Secur. 15, 2679–2691 (2020)

    Article  MATH  Google Scholar 

  3. Li, Z., Lv, X., Yu, W., Liu, Q., Lin, J., Zhang, S.: Face shape transfer via semantic warping. Vis. Intell. 2(1), 1–11 (2024)

    Article  Google Scholar 

  4. Qian, Y., Yin, G., Sheng, L., Chen, Z., Shao, J.: Thinking in frequency: face forgery detection by mining frequency-aware clues. In: European Conference on Computer Vision, pp. 86–103. Springer (2020)

    Google Scholar 

  5. Li, J., Xie, H., Li, J., Wang, Z., Zhang, Y.: Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6458–6467 (2021)

    Google Scholar 

  6. Gu, Q., Chen, S., Yao, T., Chen, Y., Ding, S., Yi, R.: Exploiting fine-grained face forgery clues via progressive enhancement learning. Proc. AAAI Conf. Artif. Intell. 36, 735–743 (2022)

    Google Scholar 

  7. Haliassos, A., Vougioukas, K., Petridis, S., Pantic, M.: Lips don’t lie: a generalisable and robust approach to face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5039–5049 (2021)

    Google Scholar 

  8. Haliassos, A., Mira, R., Petridis, S., Pantic, M.: Leveraging real talking faces via self-supervision for robust forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14950–14962 (2022)

    Google Scholar 

  9. Tariq, S., Lee, S., Woo, S.S.: A convolutional LSTM based residual network for deepfake video detection. arXiv preprint arXiv:2009.07480 (2020)

  10. Wang, J., Du, X., Cheng, Y., Sun, Y., Tang, J.: SI-Net: spatial interaction network for deepfake detection. Multimedia Syst. 29(5), 3139–3150 (2023)

    Article  MATH  Google Scholar 

  11. Wang, J., Sun, Y., Tang, J.: LiSiam: localization invariance Siamese network for deepfake detection. IEEE Trans. Inf. Forensics Secur. 17, 2425–2436 (2022)

    Article  MATH  Google Scholar 

  12. Frank, J., Eisenhofer, T., Schönherr, L., Fischer, A., Kolossa, D., Holz, T.: Leveraging frequency analysis for deep fake image recognition. In: International Conference on Machine Learning, pp. 3247–3258. PMLR (2020)

    Google Scholar 

  13. Luo, Y., Zhang, Y., Yan, J., Liu, W.: Generalizing face forgery detection with high-frequency features. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16317–16326 (2021)

    Google Scholar 

  14. Jeong, Y., Kim, D., Min, S., Joe, S., Gwon, Y., Choi, J.: BiHPF: bilateral high-pass filters for robust deepfake detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 48–57 (2022)

    Google Scholar 

  15. Feng, C., Chen, Z., Owens, A.: Self-supervised video forensics by audio-visual anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10491–10503 (2023)

    Google Scholar 

  16. Dolhansky, B., et al.: The deepfake detection challenge (dfdc) dataset. arXiv preprint arXiv:2006.07397 (2020)

  17. Khalid, H., Tariq, S., Kim, M., Woo, S.S.: Fakeavceleb: a novel audio-video multimodal deepfake dataset. arXiv preprint arXiv:2108.05080 (2021)

  18. Chung, J.S., Nagrani, A., Zisserman, A.: VoxCeleb2: deep speaker recognition. arXiv preprint arXiv:1806.05622 (2018)

  19. Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: Mesonet: a compact facial video forgery detection network. In: 2018 IEEE International Workshop on Information Forensics and Security, pp. 1–7. IEEE (2018)

    Google Scholar 

  20. Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  21. Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: Faceforensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1–11 (2019)

    Google Scholar 

  22. Nguyen, H.H., Yamagishi, J., Echizen, I.: Capsule-forensics: using capsule networks to detect forged images and videos. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2307–2311. IEEE (2019)

    Google Scholar 

  23. Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)

    Google Scholar 

  24. Chumachenko, K., Iosifidis, A., Gabbouj, M.: Self-attention fusion for audiovisual emotion recognition with incomplete data. In: 2022 26th International Conference on Pattern Recognition, pp. 2822–2828. IEEE (2022)

    Google Scholar 

  25. Chugh, K., Gupta, P., Dhall, A., Subramanian, R.: Not made for each other-audio-visual dissonance-based deepfake detection and localization. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 439–447 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Yuanfei Wan & Yunlian Sun

  2. School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing, China

    Jian Wang

  3. South China Agricultural University, Guangzhou, China

    Jinrong Cui

Authors
  1. Yuanfei Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinrong Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunlian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunlian Sun .

Editor information

Editors and Affiliations

  1. Southern University of Science and Technology, Shenzhen, China

    Shiqi Yu

  2. Hefei University of Technology, Hefei, China

    Wei Jia

  3. Nanjing University of Science and Technology, Nanjing, China

    Xiangbo Shu

  4. Nanjing University, Nanjing, China

    Xiaotong Yuan

  5. Southeast University, Nanjing, China

    Jie Gui

  6. Nanjing University of Science and Technology, Nanjing, China

    Jinhui Tang

  7. Nanjing University, Nanjing, China

    Caifeng Shan

  8. Nanjing University of Posts and Telecommunications, Nanjing, China

    Qingshan Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wan, Y., Wang, J., Cui, J., Sun, Y. (2025). Exposing Audio-Visual Forgeries in Frequency Domain. In: Yu, S., et al. Biometric Recognition. CCBR 2024. Lecture Notes in Computer Science, vol 15352. Springer, Singapore. https://doi.org/10.1007/978-981-96-1068-6_23

Download citation

  • DOI: https://doi.org/10.1007/978-981-96-1068-6_23

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-96-1067-9

  • Online ISBN: 978-981-96-1068-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics