research-article

A Novel Deep Video Watermarking Framework with Enhanced Robustness to H.264/AVC Compression

Authors:

Xin LiaoAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 8095 - 8104

https://doi.org/10.1145/3581783.3612270

Published: 27 October 2023 Publication History

Abstract

The recent success of deep image watermarking has demonstrated the potential of deep learning for watermarking, which has drawn increasing attention to deep video watermarking with the objective to improve its robustness and perceptual quality. Compared to images, video watermarking is much more challenging due to the rich structures of video data and the diversity of attacks in video transmission pipeline. The existing deep video watermarking schemes are far from satisfactory in dealing with temporal attacks, e.g., frame averaging, frame dropping and transcoding. To this end, a novel deep framework for Robustness Enhanced Video watermarking (REVMark) is proposed in this paper, aiming at improving the overall robustness, especially in dealing with H.264/AVC compression, while maintaining good visual quality. REVMark has an encoder/decoder structure with a pre-processing block (TAsBlock) to effectively extract the temporal-associated features on aligned frames. To ensure the end-to-end robust training, a distortion layer is integrated into the REVMark to resemble various attacks in real-world scenarios, among which, a new differentiable simulator of video compression, namely DiffH264, is developed to approximately simulate the process of H.264/AVC compression. In addition, the mask loss is incorporated to guide the encoder to embed the watermark in the human-imperceptible regions, thus improving the perceptual quality of the watermarked video. Experimental results demonstrate that the proposed scheme can outperform other SOTA methods while achieving 10X faster inference.

References

[1]

Mahdi Ahmadi, Alireza Norouzi, Nader Karimi, Shadrokh Samavi, and Ali Emami. 2020. ReDMark: Framework for residual diffusion watermarking based on deep networks. Expert Systems with Applications 146 (2020), 113157.

Digital Library

[2]

Md Asikuzzaman, Md Jahangir Alam, Andrew J Lambert, and Mark Richard Pickering. 2014. Imperceptible and robust blind video watermarking using chrominance embedding: A set of approaches in the DT CWT domain. IEEE Transactions on Information Forensics and Security 9, 9 (2014), 1502--1517.

Digital Library

[3]

Md Asikuzzaman, Md Jahangir Alam, Andrew J Lambert, and Mark R Pickering. 2016. Robust DT CWT-based DIBR 3D video watermarking using chrominance embedding. IEEE Transactions on Multimedia 18, 9 (2016), 1733--1748.

Digital Library

[4]

Thomas Brox, Andrés Bruhn, Nils Papenberg, and Joachim Weickert. 2004. High accuracy optical flow estimation based on a theory for warping. In Proceedings of European Conference on Computer Vision (ECCV). Springer, 25--36.

[5]

Patrizio Campisi and Alessandro Neri. 2005. Video watermarking in the 3D-DWT domain using perceptual masking. In Proceedings of IEEE International Conference on Image Processing (ICIP), Vol. 1. IEEE, I--997.

[6]

Pik-Wah Chan and Michael R Lyu. 2003. A DWT-based digital video watermarking scheme with error correcting code. In Proceedings of International Conference on Information and Communications Security (ICICS). Springer, 202--213.

[7]

Qinwei Chang, Leichao Huang, Shaoteng Liu, Hualuo Liu, Tianshu Yang, and Yexin Wang. 2022. Blind robust video watermarking based on adaptive region selection and channel reference. In Proceedings of the 30th ACM International Conference on Multimedia (ACM MM). 2344--2350.

Digital Library

[8]

Lino E Coria, Mark R Pickering, Panos Nasiopoulos, and Rabab Kreidieh Ward. 2008. A video watermarking scheme based on the dual-tree complex wavelet transform. IEEE Transactions on Information Forensics and Security 3, 3 (2008), 466--474.

Digital Library

[9]

Ingemar Cox, Matthew Miller, Jeffrey Bloom, and Chris Honsinger. 2002. Digital watermarking. Journal of Electronic Imaging 11, 3 (2002), 414--414.

[10]

Frederic Deguillaume, Gabriela Csurka, Joseph JK O'Ruanaidh, and Thierry Pun. 1999. Robust 3D DFT video watermarking. In Proceedings of Security and Watermarking of Multimedia Contents, Vol. 3657. SPIE, 113--124.

[11]

Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. 2015. Flownet: Learning optical flow with convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2758--2766.

Digital Library

[12]

Oleg Evsutin and Kristina Dzhanashia. 2022. Watermarking schemes for digital images: Robustness overview. Signal Processing: Image Communication 100 (2022), 116523.

Digital Library

[13]

Han Fang, Zhaoyang Jia, Zehua Ma, Ee-Chien Chang, and Weiming Zhang. 2022. PIMoG: An effective screen-shooting noise-layer simulation for deep-learningbased watermarking network. In Proceedings of the 30th ACM International Conference on Multimedia (ACM MM). 2267--2275.

Digital Library

[14]

David Fleet and Yair Weiss. 2006. Optical flow estimation. Handbook of Mathematical Models in Computer Vision (2006), 237--257.

[15]

Palak Garg, Lakshita Dodeja, and Mayank Dave. 2019. Hybrid color image watermarking algorithm based on DSWT-DCT-SVD and Arnold transform. In Proceedings of International Conference on Signal Processing and Communication (ICSC). Springer, 327--336.

[16]

Frank C Glazer. 1987. Hierarchical motion detection. University of Massachusetts.

[17]

Mohamed Hamidi, Mohamed El Haziti, Hocine Cherifi, and Driss Aboutajdine. 2015. A blind robust image watermarking approach exploiting the DFT magnitude. In Proceedings of 2015 IEEE/ACS 12th International Conference of Computer Systems and Applications (AICCSA). IEEE, 1--6.

[18]

Hwai-Tsu Hu and Ling-Yuan Hsu. 2017. Collective blind image watermarking in DWT-DCT domain with adaptive embedding strength governed by quality metrics. Multimedia Tools and Applications 76, 5 (2017), 6575--6594.

Digital Library

[19]

Jun Jia, Zhongpai Gao, Kang Chen, Menghan Hu, Xiongkuo Min, Guangtao Zhai, and Xiaokang Yang. 2020. RIHOOP: Robust invisible hyperlinks in offline and online photographs. IEEE Transactions on Cybernetics 52, 7 (2020), 7094--7106.

[20]

Jun Jia, Zhongpai Gao, Dandan Zhu, Xiongkuo Min, Menghan Hu, and Guangtao Zhai. 2022. RIVIE: Robust inherent video information embedding. IEEE Transactions on Multimedia (2022).

[21]

Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. 2017. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017).

[22]

Hongmei Liu, Nuo Chen, Jiwu Huang, Xialing Huang, and Yun Q Shi. 2002. A robust DWT-based video watermarking algorithm. In Proceedings of 2002 IEEE International Symposium on Circuits and Systems (ISCAS), Vol. 3. IEEE, III-III.

[23]

Junxiong Lu, Jiangqun Ni, Wenkang Su, and Hao Xie. 2022. Wavelet-based CNN for robust and high-capacity image watermarking. In Proceedings of 2022 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1--6.

[24]

Zhe-Ming Lu, Dian-Guo Xu, and Sheng-He Sun. 2005. Multipurpose image watermarking algorithm based on multistage vector quantization. IEEE Transactions on Image Processing 14, 6 (2005), 822--831.

Digital Library

[25]

Xiyang Luo, Yinxiao Li, Huiwen Chang, Ce Liu, Peyman Milanfar, and Feng Yang. 2023. DVMark: a deep multiscale framework for video watermarking. IEEE Transactions on Image Processing (2023).

[26]

S Maheswari, K Rameshwaran, and KM Malarselvi. 2015. DCT-PCA based watermarking on E-governance documents. Research Journal of Applied Sciences, Engineering and Technology 9, 7 (2015), 507--511.

[27]

Hannes Mareen, Johan De Praeter, Glenn Van Wallendael, and Peter Lambert. 2018. A scalable architecture for uncompressed-domain watermarked videos. IEEE Transactions on Information Forensics and Security 14, 6 (2018), 1432--1444.

[28]

Chi-Man Pun. 2006. A novel DFT-based digital watermarking system for images. In Proceedings of IEEE International Conference on Signal Processing (ICSP), Vol. 2. IEEE.

[29]

Anurag Ranjan and Michael J Black. 2017. Optical flow estimation using a spatial pyramid network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4161--4170.

[30]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). Springer, 234--241.

[31]

Soumitra Roy and Arup Kumar Pal. 2017. A blind DCT based color watermarking algorithm for embedding multiple watermarks. AEU-International Journal of Electronics and Communications 72 (2017), 149--161.

[32]

Richard Shin and Dawn Song. 2017. Jpeg-resistant adversarial images. In Proceedings of NIPS 2017 Workshop on Machine Learning and Computer Security, Vol. 1. 8.

[33]

Matthew Tancik, Ben Mildenhall, and Ren Ng. 2020. Stegastamp: Invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2117--2126.

[34]

Rohit Thanki, Ashish Kothari, and Deven Trivedi. 2019. Hybrid and blind watermarking scheme in DCuT--RDWT domain. Journal of Information Security and Applications 46 (2019), 231--249.

Digital Library

[35]

Matthieu Urvoy, Dalila Goudia, and Florent Autrusseau. 2014. Perceptual DFT watermarking with improved detection and robustness to geometrical distortions. IEEE Transactions on Information Forensics and Security 9, 7 (2014), 1108--1119.

Digital Library

[36]

Eric Wengrowski and Kristin Dana. 2019. Light field messaging with deep photographic steganography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1515--1524.

[37]

Thomas Wiegand, Gary J Sullivan, Gisle Bjontegaard, and Ajay Luthra. 2003. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (2003), 560--576.

Digital Library

[38]

Jonas Wulff and Michael J Black. 2015. Efficient sparse-to-dense optical flow estimation using a learned basis and layers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 120--130.

[39]

Innfarn Yoo, Huiwen Chang, Xiyang Luo, Ondrej Stava, Ce Liu, Peyman Milanfar, and Feng Yang. 2022. Deep 3d-to-2d watermarking: Embedding messages in 3d meshes and extracting them from 2d renderings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10031--10040.

[40]

Aditi Zear, Amit Kumar Singh, and Pardeep Kumar. 2018. A proposed secure multiple watermarking technique based on DWT, DCT and SVD for application in medicine. Multimedia Tools and Applications 77 (2018), 4863--4882.

[41]

Chaoning Zhang, Adil Karjauv, Philipp Benz, and In So Kweon. 2021. Towards robust deep hiding under non-differentiable distortions for practical blind watermarking. In Proceedings of the 29th ACM International Conference on Multimedia (ACM MM). 5158--5166.

Digital Library

[42]

Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 586--595.

[43]

Yifeng Zhang, Yingying Li, and Yibo Sun. 2019. Digital watermarking based on joint DWT-DCT and OMP reconstruction. Circuits, Systems, and Signal Processing 38 (2019), 5135--5148.

Digital Library

[44]

Zhen Zhou, Shuyu Chen, and Guiping Wang. 2017. A robust digital image watermarking algorithm based on dct domain for copyright protection. In Proceedings of International Symposium on Smart Graphics (SG). Springer, 132--142.

[45]

Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. 2018. Hidden: Hiding data with deep networks. In Proceedings of European Conference on Computer Vision (ECCV). 657--672.

Digital Library

Cited By

Chen LWang CZhou XQin Z(2025)Robust and Compatible Video Watermarking via Spatio-Temporal Enhancement and Multiscale Pyramid AttentionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.347189135:2(1548-1561)Online publication date: Feb-2025
https://doi.org/10.1109/TCSVT.2024.3471891
Liang YNiu KZhang YMeng YHu F(2024)Adaptive Video Dual Domain Watermarking Scheme Based on PHT Moment and Optimized Spread Transform Dither ModulationComputers, Materials & Continua10.32604/cmc.2024.05643881:2(2457-2492)Online publication date: 2024
https://doi.org/10.32604/cmc.2024.056438
Zhang XXu YLi RYu JLi WXu ZZhang JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright ProtectionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680904(9818-9827)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680904
Show More Cited By

Index Terms

A Novel Deep Video Watermarking Framework with Enhanced Robustness to H.264/AVC Compression
1. Security and privacy
  1. Human and societal aspects of security and privacy
    1. Social aspects of security and privacy
  2. Security services

Recommendations

A robust video watermarking algorithm in H.264/AVC compressed domain
SAC '15: Proceedings of the 30th Annual ACM Symposium on Applied Computing

In this paper, a low complexity video watermarking scheme for H.264 has been presented. Our contribution is to attain lower complexity in embedding procedure and extracting watermark. At the same time, we avoid a bit-rate increase and improve the ...
Hiding depth information into H.264 compressed video using reversible watermarking
CMBAS-EH '12: Proceedings of the 1st ACM multimedia international workshop on Cloud-based multimedia applications and services for e-health

A scheme is proposed to hide 3D information (depth map) into H.264 compressed video using reversible watermarking. The watermark embedder works jointly with the H.264 encoder with concern of bit rate control and low complexity. The depth information is ...
Blind Digital Watermarking of Low Bit-Rate Advanced H.264/AVC Compressed Video
IWDW '09: Proceedings of the 8th International Workshop on Digital Watermarking

H.264/AVC is becoming a popular video codec for its better compression ratio, lower distortion and applicability to portable electronic devices. Thus, issues of copyright protection and authentication that are appropriate for this standard become very ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Guangdong Province
Peng Cheng Laboratory Project
National Natural Science Foundation of China
Key-Area Research and Development Program of Guangdong Province

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
371
Total Downloads

Downloads (Last 12 months)242
Downloads (Last 6 weeks)36

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen LWang CZhou XQin Z(2025)Robust and Compatible Video Watermarking via Spatio-Temporal Enhancement and Multiscale Pyramid AttentionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.347189135:2(1548-1561)Online publication date: Feb-2025
https://doi.org/10.1109/TCSVT.2024.3471891
Liang YNiu KZhang YMeng YHu F(2024)Adaptive Video Dual Domain Watermarking Scheme Based on PHT Moment and Optimized Spread Transform Dither ModulationComputers, Materials & Continua10.32604/cmc.2024.05643881:2(2457-2492)Online publication date: 2024
https://doi.org/10.32604/cmc.2024.056438
Zhang XXu YLi RYu JLi WXu ZZhang JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright ProtectionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680904(9818-9827)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680904
Xiang LLiu YYang Z(2024)A reversible natural language watermarking for sensitive information protectionInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10366161:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103661

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten