Skip to main content
SpringerLink
Log in

Universal Detection and Source Attribution of Diffusion Model Generated Images with High Generalization and Robustness

  • Conference paper
  • First Online:
Pattern Recognition and Machine Intelligence (PReMI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14301))

Abstract

The proliferation of synthetic media over the internet is posing a significant social threat. Recent advancements in Diffusion Models (DM) have made it easier to create astonishingly photo-realistic synthetic media with high stability and control. Moreover, applications like DALLE-2, powered by DM and Large Language Models (LLM), permit visual content generation from natural language description, enabling opportunities for everyone to generate visual media. Hence, there is an immediate need to identify synthetic images and attribute them to their source architectures. In this work, we propose a synthetic image detector as universal detector and a source model attributor based on a popular transfer-learning model ResNet-50 and compare the results with other popular models, including Visual Geometry Group (VGG) 16, XceptionNet and InceptionNet. The proposed universal detector attains over 96% accuracy, with a source attribution, accuracy over 93% for detection of Diffusion Model generated images. The model also succeeds in achieving significant generalization and robustness capabilities under different training-testing configurations, as proven by our experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://edition.cnn.com/2020/02/20/tech/fake-faces-deepfake/.

  2. 2.

    https://www.theregreview.org/2023/01/24/penava-ai-art-is-in-legal-greyscale/.

  3. 3.

    https://www.christies.com/features/A-collaboration-between-two-artists-one-human-one-a-machine-9332-1.aspx.

References

  1. Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  Google Scholar 

  2. Lago, F., Pasquini, C., Böhme, R., Dumont, H., Goffaux, V., Boato, G.: More real than real: a study on human visual perception of synthetic faces [applications corner]. IEEE Signal Process. Mag. 39(1), 109–116 (2021)

    Article  Google Scholar 

  3. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of styleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110–8119 (2020)

    Google Scholar 

  4. Hancock, J.T., Bailenson, J.N.: The social impact of DeepFakes. Cyberpsychol. Behav. Soc. Netw. 24(3), 149–152 (2021). PMID: 33760669

    Article  Google Scholar 

  5. Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. In: Advances in Neural Information Processing, vol. 34, pp. 8780–8794 (2021)

    Google Scholar 

  6. Nichol, A., Ramesh, A., Dhariwal, P.: Hierarchical text-conditional image generation with clip Latents (2022)

    Google Scholar 

  7. Lorenz, D., Rombach, R., Blattmann, A.: High-resolution image synthesis with latent diffusion models (2022)

    Google Scholar 

  8. Passos, L. A., Jodas, D., da Costa, K. A., Júnior, L. A. S., Colombo, D., Papa, J. P.: A review of deep learning-based approaches for DeepFake content detection. arXiv preprint arXiv:2202.06095 (2022)

  9. Cozzolino, D., Gragnaniello, D., Poggi, G., Verdoliva, L.: Towards universal GAN image detection. In: 2021 International Conference on Visual Communications and Image Processing (VCIP), pp. 1–5. IEEE (2021)

    Google Scholar 

  10. Zingarini, G., Corvi, R., Cozzolino, D.: On the detection of synthetic images generated by diffusion models (2022)

    Google Scholar 

  11. Yu, N., Sha, Z., Li, Z.: De-fake: detection and attribution of fake images generated by text-to-image generation models (2023)

    Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  13. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695 (2022)

    Google Scholar 

  14. Ramesh, A., Nichol, A., Dhariwal, P.: GLIDE: towards photorealistic image generation and editing with text-guided diffusion models (2022)

    Google Scholar 

  15. Dayma, B.: Dall\(\cdot \)e mini, vol. 7 (2021)

    Google Scholar 

  16. Hodosh, M., Young, P., Hockenmaier, J.: Framing image description as a ranking task: data, models and evaluation metrics. J. Artif. Intell. Res. 47, 853–899 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  18. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)

    Google Scholar 

  19. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)

    Google Scholar 

  20. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  21. Towsley, D., Atwood, J.: Diffusion-convolutional neural networks (2016)

    Google Scholar 

  22. Simonyan, K., Brock, A., Donahue, J.: Large scale GAN training for high fidelity natural image synthesis (2018)

    Google Scholar 

  23. Laine, S., Lehtinen, J., Karras, T., Aila, T.: Progressive growing of GANs for improved quality, stability, and variation (2018)

    Google Scholar 

  24. Marra, F., Gragnaniello, D., Cozzolino, D.: Are GAN generated images easy to detect? A critical analysis of the state-of-the-art (2021)

    Google Scholar 

  25. Wang, Z.J., Montoya, E., Munechika, D., Yang, H., Hoover, B., Chau, D.H.: DiffusionDB: a large-scale prompt gallery dataset for text-to-image generative models. arXiv:2210.14896 [cs] (2022)

  26. Fried, O., Sinitsa, S.: Deep image fingerprint: accurate and low budget synthetic image detector (2023)

    Google Scholar 

  27. Zhang, R., Wang, S.-Y., Wang, O.: CNN-generated images are surprisingly easy to spot... for now (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Indian Institute of Engineering, Science and Technology, Shibpur, Howrah, 711103, India

    Sanandita Das, Dibyarup Dutta, Tanusree Ghosh & Ruchira Naskar

Authors
  1. Sanandita Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dibyarup Dutta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tanusree Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruchira Naskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tanusree Ghosh .

Editor information

Editors and Affiliations

  1. Indian Statistical Institute, Kolkata, India

    Pradipta Maji

  2. Texas A&M University at Qatar, Doha, Qatar

    Tingwen Huang

  3. Indian Statistical Institute, Kolkata, West Bengal, India

    Nikhil R. Pal

  4. Indian Institute of Technology Jodhpur, Jodhpur, India

    Santanu Chaudhury

  5. Indian Statistical Institute, Kolkata, West Bengal, India

    Rajat K. De

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Das, S., Dutta, D., Ghosh, T., Naskar, R. (2023). Universal Detection and Source Attribution of Diffusion Model Generated Images with High Generalization and Robustness. In: Maji, P., Huang, T., Pal, N.R., Chaudhury, S., De, R.K. (eds) Pattern Recognition and Machine Intelligence. PReMI 2023. Lecture Notes in Computer Science, vol 14301. Springer, Cham. https://doi.org/10.1007/978-3-031-45170-6_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45170-6_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45169-0

  • Online ISBN: 978-3-031-45170-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation