Suppressing High-Frequency Artifacts for Generative Model Watermarking by Anti-Aliasing
Authors: 
Li Zhang, Yong Liu, Xinpeng Zhang, Hanzhou WuAuthors Info & Claims
Li Zhang, Yong Liu, Xinpeng Zhang, Hanzhou WuAuthors Info & ClaimsPages 223 - 234
Abstract
Protecting deep neural networks (DNNs) against intellectual property (IP) infringement has attracted an increasing attention in recent years. Recent advances focus on IP protection of generative models, which embed the watermark information into the image generated by the model to be protected. Although the generated marked image has good visual quality, it introduces noticeable artifacts to the marked image in high-frequency area, which severely impairs the imperceptibility of the watermark and thereby reduces the security of the watermarking system. To deal with this problem, we propose a novel framework for generative model watermarking that can suppress the high-frequency artifacts. The main idea is to design a new watermark embedding network that can suppress high-frequency artifacts by applying anti-aliasing. To realize anti-aliasing, we use low-pass filtering for the internal sampling layers of the new watermark embedding network. Meanwhile, joint loss optimization and adversarial training are applied to enhance the effectiveness and robustness. Experimental results indicate that the marked model not only maintains the performance very well on the original task, but also demonstrates better imperceptibility and robustness on the watermarking task. This work reveals the importance of suppressing high-frequency artifacts for enhancing imperceptibility and security of generative model watermarking.
References
[1]
Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. 2018. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In Proc. USENIX Conference on Security Symposium. 1615--1631.
[2]
Gwern Branwen and A Gokaslan. 2019. Danbooru2019: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset.
[3]
Geoffrey J Burton and Ian R Moorhead. 1987. Color and spatial structure in natural scenes. Applied Optics, Vol. 26, 1 (1987), 157--170.
[4]
Faten Chaabane, Maha Charfeddine, William Puech, and Chokri Ben Amaf. 2015. A QR-code based audio watermarking technique for tracing traitors. In Proc. European Signal Processing Conference. 51--55.
[5]
Huili Chen, Bita Darvish Rouhani, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. 2019. Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In Proc. International Conference on Multimedia Retrieval. 105--113.
[6]
Francc ois Chollet. 2017. Xception: Deep learning with depthwise separable convolutions. In Proc. IEEE Conference on Computer Vision and Pattern Recognition. 1251--1258.
[7]
Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker. 2007. Digital watermarking and steganography. Morgan kaufmann.
[8]
Lixin Fan, Kam Woh Ng, and Chee Seng Chan. 2019. Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks. In Proc. Neural Information Processing Systems, Vol. 32. 10.
[9]
Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer, Dorothea Kolossa, and Thorsten Holz. 2020. Leveraging frequency analysis for deep fake image recognition. In Proc. International Conference on Machine Learning. 3247--3258.
[10]
Rafael C Gonzalez. 2009. Digital image processing. Pearson education india.
[11]
Jia Guo and Miodrag Potkonjak. 2018. Watermarking deep neural networks for embedded systems. In Proc. IEEE/ACM International Conference on Computer-Aided Design (ICCAD). 1--8.
[12]
Geoffrey Hinton, Li Deng, Dong Yu, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, Vol. 29, 6 (2012), 82--97.
[13]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. International Conference on Machine Learning. 448--456.
[14]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition. 1125--1134.
[15]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In Proc. European Conference on Computer Vision. 694--711.
[16]
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of gans for improved quality, stability, and variation. arXiv Preprint arXiv:1710.10196 (2017).
[17]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4401--4410.
[18]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[19]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2017. Imagenet classification with deep convolutional neural networks. Commun. ACM, Vol. 60, 6 (2017), 84--90.
[20]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature, Vol. 521, 7553 (2015), 436--444.
[21]
Huiying Li, Emily Wenger, Shawn Shan, Ben Y Zhao, and Haitao Zheng. 2019b. Piracy resistant watermarks for deep neural networks. arXiv Preprint arXiv:1910.01226 (2019).
[22]
Yue Li, Benedetta Tondi, and Mauro Barni. 2021. Spread-transform dither modulation watermarking of deep neural network. Journal of Information Security and Applications, Vol. 63 (2021), 103004.
[23]
Zheng Li, Chengyu Hu, Yang Zhang, and Shanqing Guo. 2019a. How to prove your model belongs to you: A blind-watermark based framework to protect intellectual property of DNN. In Proc. Annual Computer Security Applications Conference. 126--137.
[24]
Lina Lin and Hanzhou Wu. 2022. Verifying Integrity of Deep Ensemble Models by Lossless Black-box Watermarking with Sensitive Samples. In Proc. IEEE International Symposium on Digital Forensics and Security. 1--6.
[25]
Xiao-Long Liu, Chia-Chen Lin, and Shyan-Ming Yuan. 2018. Blind dual watermarking for color images' authentication and copyright protection. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 28, 5 (2018), 1047--1055.
[26]
Nelson Morgan. 2011. Deep and wide: Multiple layers in automatic speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, Vol. 20, 1 (2011), 7--13.
[27]
Augustus Odena, Vincent Dumoulin, and Chris Olah. 2016. Deconvolution and checkerboard artifacts. Distill (2016).
[28]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Proc. International Conference on Medical Image Computing and Computer-assisted Intervention. 234--241.
[29]
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv Preprint arXiv:1508.07909 (2015).
[30]
Hamid R Sheikh and Alan C Bovik. 2006. Image information and visual quality. IEEE Transactions on Image Processing, Vol. 15, 2 (2006), 430--444.
[31]
David Silver, Julian Schrittwieser, Karen Simonyan, et al. 2017. Mastering the game of go without human knowledge. Nature, Vol. 550, 7676 (2017), 354--359.
[32]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv Preprint arXiv:1409.1556 (2014).
[33]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, Vol. 15, 1 (2014), 1929--1958.
[34]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition. 1--9.
[35]
Enzo Tartaglione, Marco Grangetto, Davide Cavagnino, and Marco Botta. 2021. Delving in the loss landscape to embed robust watermarks into neural networks. In Proc. International Conference on Pattern Recognition. 1243--1250.
[36]
David J Tolhurst, Yoav Tadmor, and Tang Chao. 1992. Amplitude spectra of natural images. Ophthalmic and Physiological Optics, Vol. 12, 2 (1992), 229--232.
[37]
Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. 2017. Embedding watermarks into deep neural networks. In Proc. ACM International Conference on Multimedia Retrieval. 269--277.
[38]
Jiangfeng Wang, Hanzhou Wu, Xinpeng Zhang, and Yuwei Yao. 2020. Watermarking in deep neural networks via error back-propagation. In Proc. IS&T Electronic Imaging, Media Watermarking, Security, and Forensics. 22--1--22--9(9).
[39]
Tianhao Wang and Florian Kerschbaum. 2021. Riga: Covert and robust white-box watermarking of deep neural networks. In Proc. Web Conference. 993--1004.
[40]
Yumin Wang and Hanzhou Wu. 2022. Protecting the Intellectual Property of Speaker Recognition Model by Black-Box Watermarking in the Frequency Domain. Symmetry, Vol. 14, 3 (2022), 619.
[41]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, Vol. 13, 4 (2004), 600--612.
[42]
Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale structural similarity for image quality assessment. In Proc. Asilomar Conference on Signals, Systems & Computers, 2003, Vol. 2. 1398--1402.
[43]
Hanzhou Wu, Gen Liu, Yuwei Yao, and Xinpeng Zhang. 2020. Watermarking neural networks with watermarked images. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, 7 (2020), 2591--2601.
[44]
Hanzhou Wu, Gen Liu, and Xinpeng Zhang. 2023. Hiding Data Hiding. Pattern Recognition Letters, Vol. 165 (2023), 122--127.
[45]
HanZhou Wu, YunQing Shi, HongXia Wang, and LinNa Zhou. 2017. Separable reversible data hiding for encrypted palette images with color partitioning and flipping verification. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, 8 (2017), 1620--1631.
[46]
Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li. 2015. Empirical evaluation of rectified activations in convolutional network. arXiv Preprint arXiv:1505.00853 (2015).
[47]
Wenhan Yang, Robby T Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, and Jiaying Liu. 2019. Joint rain detection and removal from a single image with contextualized deep networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, 6 (2019), 1377--1393.
[48]
Yi Zeng, Won Park, Z Morley Mao, and Ruoxi Jia. 2021. Rethinking the backdoor attacks' triggers: A frequency perspective. In Proc. IEEE/CVF International Conference on Computer Vision. 16473--16481.
[49]
Chaoning Zhang, Philipp Benz, Adil Karjauv, Geng Sun, and In So Kweon. 2020a. Udh: Universal deep hiding for steganography, watermarking, and light field messaging. Proc. Neural Information Processing Systems, Vol. 33 (2020), 10223--10234.
[50]
Jie Zhang, Dongdong Chen, Jing Liao, Han Fang, Weiming Zhang, Wenbo Zhou, Hao Cui, and Nenghai Yu. 2020b. Model watermarking for image processing networks. In Proc. AAAI Conference on Artificial Intelligence, Vol. 34. 12805--12812.
[51]
Jie Zhang, Dongdong Chen, Jing Liao, Weiming Zhang, Huamin Feng, Gang Hua, and Nenghai Yu. 2021. Deep model intellectual property protection via deep watermarking. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
[52]
Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph Stoecklin, Heqing Huang, and Ian Molloy. 2018. Protecting intellectual property of deep neural networks with watermarking. In Proc. Asia Conference on Computer and Communications Security. 159--172.
[53]
Li Zhang, Yong Liu, Shaoteng Liu, Tianshu Yang, Yexin Wang, Xinpeng Zhang, and Hanzhou Wu. 2022. Generative Model Watermarking Based on Human Visual System. arXiv Preprint arXiv:2209.15268 (2022).
[54]
Richard Zhang. 2019. Making convolutional networks shift-invariant again. In Proc. International Conference on Machine Learning. 7324--7334.
[55]
Xiangyu Zhao, Hanzhou Wu, and Xinpeng Zhang. 2021a. Watermarking graph neural networks by random graphs. In Proc. IEEE International Symposium on Digital Forensics and Security. 1--6.
[56]
Xiangyu Zhao, Yinzhe Yao, Hanzhou Wu, and Xinpeng Zhang. 2021b. Structural watermarking to deep neural networks via network channel pruning. In Proc. IEEE International Workshop on Information Forensics and Security. 1--6.
[57]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proc. IEEE International Conference on Computer Vision. 2223--2232. io
Index Terms
- Suppressing High-Frequency Artifacts for Generative Model Watermarking by Anti-Aliasing
Recommendations
Cryptanalysis on majority-voting based self-recovery watermarking scheme
In 2007, Wang-Chen proposed a majority-voting based self-recovery watermarking scheme, in which the additional authentication watermark of each block was embedded in the less significant bits and used to examine the authenticity of the same block. In ...
Nonlinear scrambling-based reversible watermarking for 2D-vector maps
The reversible watermarking technique is suitable for vector maps due to its reversibility after watermark extraction. In this paper, a novel reversible watermarking scheme based on the idea of nonlinear scrambling is proposed. It begins with feature ...
Ambiguity attack against text-to-image diffusion model watermarking
Highlights- The proposed scheme is the first work for ambiguity attack of SDM watermarking.
- The implementation of the scheme only requires a small cost.
- Our method does not need the original training data.
- The proposed method is effective ...
AbstractIn recent years, the text-to-image diffusion models have achieved excellent performance. Among them, stable diffusion models (SDMs) have become one of the most widely used models because of their excellent performance. Scholars have proposed many ...
Comments
Information & Contributors
Information
Published In
June 2024
305 pages
ISBN:9798400706370
DOI:10.1145/3658664
- General Chair:
- Fernando Pérez-González,
- Program Chairs:
- Pedro Comesaña-Alfaro,
- Christian Krätzer,
- Hong Vicky Zhao
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
IH&MMSEC '24
Sponsor:
IH&MMSEC '24: ACM Workshop on Information Hiding and Multimedia Security
June 24 - 26, 2024
Baiona, Spain
Acceptance Rates
Overall Acceptance Rate 128 of 318 submissions, 40%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 111Total Downloads
- Downloads (Last 12 months)111
- Downloads (Last 6 weeks)29
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in