Well-performed deep neural networks (DNNs) generally require massive labeled data and computational resources for training. Various watermarking techniques are proposed to protect such intellectual properties (IPs), wherein the DNN providers can claim IP ownership by retrieving their embedded watermarks. While promising results are reported in the literature, existing solutions suffer from watermark removal attacks, such as model fine-tuning, model pruning, and model extraction. In this paper, we propose a novel DNN watermarking solution that can effectively defend against the above attacks. Our key insight is to enhance the coupling of the watermark and model functionalities such that removing the watermark would inevitably degrade the model’s performance on normal inputs. Specifically, on one hand, we sample inputs from the original training dataset and fuse them as watermark images. On the other hand, we randomly mask model weights during training to distribute the watermark information in the network. Our method can successfully defend against common watermark removal attacks, watermark ambiguity attacks, and existing widely used backdoor detection methods, outperforming existing solutions as demonstrated by evaluation results on various benchmarks. Our code is available at: https://github.com/cure-lab/Function-Coupled-Watermark.