Abstract
Diffusion models have demonstrated remarkable capabilities across a range of tasks and have become the backbone of various web applications, such as text-to-image, image-to-image, and text-to-video generation. Obtaining large, high-performance diffusion models demands significant resources, highlighting their importance as Intellectual Property (IP) worth protecting. Watermarking is widely adopted as the mainstream technique for model IP protection. However, existing watermarking methods designed for discriminative models are insufficient for protection diffusion models. This paper introduces WDM, a novel watermarking solution for diffusion models without imprinting the watermark during task generation. It involves training a model to concurrently learn a Watermark Diffusion Process (WDP) for embedding watermarks alongside the standard diffusion process for task generation. We provide a detailed theoretical analysis of the training and sampling in WDP, relating it to a shifted Gaussian diffusion process via the same reverse noise. Watermarks are extracted using a designated trigger, ensuring they stay unexposed during the primary task sampling. We further present a complete framework for verifying copyright infringement through hypothesis testing. Extensive experiments have validated the effectiveness and robustness of our approach in various trigger and watermark data configurations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adi, Y., Baum, C., Cissé, M., Pinkas, B., Keshet, J.: Turning your weakness into a strength: watermarking deep neural networks by backdooring. In: USENIX Security (2018)
Chen, H., Rouhani, B.D., Fu, C., Zhao, J., Koushanfar, F.: Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In: ICMR (2019)
Chen, W., Song, D., Li, B.: Trojdiff: trojan attacks on diffusion models with diverse targets. In: CVPR (2023)
Chou, S., Chen, P., Ho, T.: How to backdoor diffusion models? In: CVPR (2022)
Fei, J., Xia, Z., Tondi, B., Barni, M.: Supervised GAN watermarking for intellectual property protection. In: WIFS (2022)
Garg, S., Dhamo, H., Farshad, A., Musatian, S., Navab, N., Tombari, F.: Unconditional scene graph generation. In: ICCV (2021)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS (2017)
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020)
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. In: NeurIPS (2022)
Jia, H., Choquette-Choo, C.A., Chandrasekaran, V., Papernot, N.: Entangled watermarks as a defense against model extraction. In: USENIX Security (2021)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: ICLR (2014)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
LeCun, Y., et al.: Learning algorithms for classification: a comparison on handwritten digit recognition. Neural networks: the statistical mechanics perspective (1995)
Li, Y., Wang, H., Barni, M.: A survey of deep neural network watermarking techniques. In: Neurocomputing (2021)
Liu, Y., Li, Z., Backes, M., Shen, Y., Zhang, Y.: Watermarking diffusion model. arXiv preprint arXiv:2305.12502 (2023)
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: ICCV (2015)
Merrer, E.L., Pérez, P., Trédan, G.: Adversarial Frontier Stitching for Remote Neural Network Watermarking. Neural Comput, Appl (2020)
Ong, D.S., Chan, C.S., Ng, K.W., Fan, L., Yang, Q.: Protecting intellectual property of generative adversarial networks from ambiguity attacks. In: CVPR (2021)
van den Oord, A., Vinyals, O., Kavukcuoglu, K.: Neural discrete representation learning. In: NeurIPS (2017)
Peng, S., Chen, Y., Xu, J., Chen, Z., Wang, C., Jia, X.: Intellectual property protection of dnn models. World Wide Web (2022)
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022)
Ramesh, A., et al.: Zero-shot text-to-image generation. In: ICML (2021)
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: CVPR (2022)
Saharia, C., et al.: Palette: Image-to-image diffusion models. In: SIGGRAPH (2022)
Salimans, T., Goodfellow, I.J., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. In: NeurIPS (2016)
Sohl-Dickstein, J., Weiss, E.A., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: ICML (2015)
Szyller, S., Atli, B.G., Marchal, S., Asokan, N.: DAWN: dynamic adversarial watermarking of neural networks. In: MM (2021)
Uchida, Y., Nagai, Y., Sakazawa, S., Satoh, S.: Embedding watermarks into deep neural networks. In: ICMR (2017)
Wei, Y., et al.: Meta-learning without data via unconditional diffusion models. IEEE Transactions on Circuits and Systems for Video Technology (2024)
Wu, D., Wang, Y.: Adversarial neuron pruning purifies backdoored deep models. In: NeurIPS (2021)
Wu, J.Z., et al.: Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In: CVPR (2023)
Yu, N., Skripniuk, V., Abdelnabi, S., Fritz, M.: Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In: ICCV (2021)
Zeng, Y., Chen, S., Park, W., Mao, Z.M., Jin, M., Jia, R.: Adversarial unlearning of backdoors via implicit hypergradient. arXiv preprint arXiv:2110.03735 (2021)
Zhang, Z., Han, L., Ghosh, A., Metaxas, D.N., Ren, J.: Sine: Single image editing with text-to-image diffusion models. In: CVPR (2023)
Zhao, Y., Pang, T., Du, C., Yang, X., Cheung, N., Lin, M.: A recipe for watermarking diffusion models. arXiv preprint arXiv:2303.10137 (2023)
Acknowledgments
The authors would like to acknowledge the financial support of this work by the Hong Kong RGC RIF grant No. R1012-21.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Peng, S., Chen, Y., Wang, C., Jia, X. (2025). Intellectual Property Protection of Diffusion Models via the Watermark Diffusion Process. In: Barhamgi, M., Wang, H., Wang, X. (eds) Web Information Systems Engineering – WISE 2024. WISE 2024. Lecture Notes in Computer Science, vol 15437. Springer, Singapore. https://doi.org/10.1007/978-981-96-0567-5_21
Download citation
DOI: https://doi.org/10.1007/978-981-96-0567-5_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-96-0566-8
Online ISBN: 978-981-96-0567-5
eBook Packages: Computer ScienceComputer Science (R0)