Skip to main content
SpringerLink
Log in

DT-TransUNet: A Dual-Task Model for Deepfake Detection and Segmentation

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14431))

Included in the following conference series:

  • 366 Accesses

Abstract

The swift advancement of deepfake technology has given rise to concerns regarding its potential misuse, necessitating the detection of fake images and acquisition of supportive evidence. In response to this need, we present a novel dual-task network model called DT-TransUNet, which concurrently performs segmentation for both deepfake detection and deepfake segmentation. Additionally, we propose a new Multi-Scale Spatial Frequency Feature (MSSFF) module that employs the Stationary Wavelet Transform (SWT) to extract multi-scale high-frequency components and enhance these features using a texture activation function. When evaluated on multiple datasets, DT-TransUNet surpasses comparable methods in performance and visual segmentation quality, thereby validating the effectiveness and capability of the MSSFF module for deepfake detection and segmentation tasks.

This work was supported by the National Natural Science Foundation of China (62172227) and National Key RD Program of China (2021YFF0602101).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amerini, I., Galteri, L., Caldelli, R., Del Bimbo, A.: Deepfake video detection through optical flow based CNN. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)

    Google Scholar 

  2. Chamot, F., Geradts, Z., Haasdijk, E.: Deepfake forensics: cross-manipulation robustness of feedforward-and recurrent convolutional forgery detection methods. Forensic Sci. Int. Digit. Invest. 40, 301374 (2022)

    Google Scholar 

  3. Chen, B., Li, T., Ding, W.: Detecting deepfake videos based on spatiotemporal attention and convolutional LSTM. Inf. Sci. 601, 58–70 (2022)

    Article  Google Scholar 

  4. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

    Google Scholar 

  5. Coccomini, D.A., Messina, N., Gennaro, C., Falchi, F.: Combining EfficientNet and vision transformers for video deepfake detection. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds.) Image Analysis and Processing, ICIAP 2022, Part III. LNCS, vol. 13233, pp. 219–229. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06433-3_19

  6. Cozzolino, D., Rössler, A., Thies, J., Nießner, M., Verdoliva, L.: ID-Reveal: identity-aware deepfake video detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15108–15117 (2021)

    Google Scholar 

  7. Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., Bharath, A.A.: Generative adversarial networks: an overview. IEEE Sig. Process. Mag. 35(1), 53–65 (2018)

    Article  Google Scholar 

  8. Deng, L., Suo, H., Li, D.: Deepfake video detection based on EfficientNet-V2 network. Comput. Intell. Neurosci. 2022, 1–13 (2022)

    Google Scholar 

  9. Doersch, C.: Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 (2016)

  10. Durall, R., Keuper, M., Pfreundt, F.J., Keuper, J.: Unmasking deepfakes with simple features. arXiv preprint arXiv:1911.00686 (2019)

  11. Ganguly, S., Ganguly, A., Mohiuddin, S., Malakar, S., Sarkar, R.: ViXNet: vision transformer with Xception network for deepfakes based video and image forgery detection. Exp. Syst. Appl. 210, 118423 (2022)

    Article  Google Scholar 

  12. Guan, W., Wang, W., Dong, J., Peng, B., Tan, T.: Collaborative feature learning for fine-grained facial forgery detection and segmentation. arXiv preprint arXiv:2304.08078 (2023)

  13. Guarnera, L., Giudice, O., Battiato, S.: Fighting deepfake by exposing the convolutional traces on images. IEEE Access 8, 165085–165098 (2020)

    Article  Google Scholar 

  14. He, Z., Zuo, W., Kan, M., Shan, S., Chen, X.: AttGAN: facial attribute editing by only changing what you want. IEEE Trans. Image Process. 28(11), 5464–5478 (2019)

    Article  MathSciNet  Google Scholar 

  15. Hernandez-Ortega, J., Tolosana, R., Fierrez, J., Morales, A.: DeepFakesON-Phys: deepfakes detection based on heart rate estimation. arXiv preprint arXiv:2010.00400 (2020)

  16. Khalid, F., Akbar, M.H., Gul, S.: SWYNT: swin y-net transformers for deepfake detection. In: 2023 International Conference on Robotics and Automation in Industry (ICRAI), pp. 1–6. IEEE (2023)

    Google Scholar 

  17. Li, L., et al.: Face X-ray for more general face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5001–5010 (2020)

    Google Scholar 

  18. Li, Y., Chang, M.C., Lyu, S.: In Ictu Oculi: exposing AI created fake videos by detecting eye blinking. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7. IEEE (2018)

    Google Scholar 

  19. Li, Y., Lyu, S.: Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656 (2018)

  20. Masi, I., Killekar, A., Mascarenhas, R.M., Gurudatt, S.P., AbdAlmageed, W.: Two-branch recurrent network for isolating deepfakes in videos. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020, Part VII. LNCS, vol. 12352, pp. 667–684. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58571-6_39

    Chapter  Google Scholar 

  21. Nguyen, H.H., Fang, F., Yamagishi, J., Echizen, I.: Multi-task learning for detecting and segmenting manipulated facial images and videos. In: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2019)

    Google Scholar 

  22. Qi, H., et al.: DeepRhythm: exposing deepfakes with attentional visual heartbeat rhythms. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 4318–4327 (2020)

    Google Scholar 

  23. Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: FaceForensics: a large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179 (2018)

  24. Saif, S., Tehseen, S., Ali, S.S., Kausar, S., Jameel, A.: Generalized deepfake video detection through time-distribution and metric learning. IT Prof. 24(2), 38–44 (2022)

    Article  Google Scholar 

  25. Tjon, E., Moh, M., Moh, T.S.: Eff-YNet: a dual task network for deepfake detection and segmentation. In: 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM), pp. 1–8. IEEE (2021)

    Google Scholar 

  26. Yang, J., Xiao, S., Li, A., Lan, G., Wang, H.: Detecting fake images by identifying potential texture difference. Futur. Gener. Comput. Syst. 125, 127–135 (2021)

    Article  Google Scholar 

  27. Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2019, pp. 8261–8265. IEEE (2019)

    Google Scholar 

  28. Zhang, X., Karaman, S., Chang, S.F.: Detecting and simulating artifacts in GAN fake images. In: 2019 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6. IEEE (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nanjing University of Science and Technology, Nanjing, China

    Junshuai Zheng, Yichao Zhou, Xiyuan Hu & Zhenmin Tang

Authors
  1. Junshuai Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yichao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiyuan Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenmin Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiyuan Hu .

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  2. Xiamen University, Xiamen, China

    Hanzi Wang

  3. Beijing University of Posts and Telecommunications, Beijing, China

    Zhanyu Ma

  4. Sun Yat-sen University, Guangzhou, China

    Weishi Zheng

  5. Peking University, Beijing, China

    Hongbin Zha

  6. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  8. Xiamen University, Xiamen, China

    Rongrong Ji

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zheng, J., Zhou, Y., Hu, X., Tang, Z. (2024). DT-TransUNet: A Dual-Task Model for Deepfake Detection and Segmentation. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14431. Springer, Singapore. https://doi.org/10.1007/978-981-99-8540-1_20

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8540-1_20

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8539-5

  • Online ISBN: 978-981-99-8540-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation