Coherent adversarial deepfake video generation

Highlights

• Adversarial attacks can be ancillary technique for deepfake videos to fool CNN-based detector.

• Adversarial perturbations generated frame by frame disrupt the consistency of adjacent deepfake videos frames and only obtaining weakly adversarial deepfake videos.

• A robust detection method which utilizing the coherence error is proposed to distinguish weakly adversarial deepfake videos from clean ones.

• Coherent adversarial deepfake videos are not sensitive to human eyes and also can evade the detection of CNN-based deepfake video detection.

• Optical flow helps restrict the temporal coherence of adversarial perturbations among frames and a well-designed adaptive distortion can measure the complexity of each frame to keep the adversarial modification imperceptible.

Abstract

Deepfake video has been rapidly developed and attracted public concerns due to its potential wide applications, deepfake videos can be easily distinguished by DNN-based detection approaches. As the vulnerability of DNNs, the adversarial attack can be an effective way to deteriorate the ability of deepfake detection, but current adversarial attack techniques are commonly designed for individual images, which are easily perceived at the video level. To reveal the weakness of current attack methods, we first propose a robust detector utilizing the temporal consistency to discriminate between the clean and perturbed ones aiming at weakly adversarial deepfake videos, achieving maximum success rates of 100%. Then we propose a novel framework for generating high-quality adversarial deepfake videos which can fool deepfake detectors and evade the detection of adversarial perturbations simultaneously. Two pivotal techniques are utilized for improving the visual quality and the imperceptibility of adversarial perturbations: (i) Optical flow is adopted to restrict the temporal coherence of adversarial perturbations among frames; (ii) An adaptive distortion cost that can measure the complexity of each frame and help to keep the adversarial modification imperceptible. We demonstrate the effectiveness of our methods in disrupting representative DNN-based deepfake detectors. Extensive experiments are conducted to show the great improvement in coherence, visual quality, and imperceptibility of the adversarial deepfake videos. Furthermore, We hope that our adversarial deepfake generation framework can shed some light on the detection methods to fix their weakness.

Introduction

With the tremendous success of deep generative models [1], [2], [3], [4] and extensive entertainment needs, face creation and manipulation have been one of the most popular topics in recent years, known as DeepFake. Depending on the manipulation goal, current methods can be categorized into identity swap [5], [6] that replaces the target identity with the source identity instead, attributes editing [7], [8] which modifies the expression, hairstyle or other attributes while keeping the original identity, and face synthesis [3], [4] aiming to generate non-existent face images. Some of these methods generate high-quality fake videos that are imperceptible to human eyes.

However, invisible artifacts of deepfake videos can be easily perceived by deepfake detectors. Many face manipulation detection methods have been developed to mitigate the potential risk of deepfake videos. According to the ways of feature extraction, current face manipulation detection methods can be roughly categorized into two types: handcrafted feature-based detection and deep learning-based detection. And deep learning-based techniques usually achieve better performance especially faced with massive amounts of data.

CNN-based face manipulation detection methods have increased rapidly in recent years due to the development of deep learning. This type of detection methods usually extracts features with convolutional layers [9], [10], [11] construct a face forensics network based on image classification network architecture. Recently, Rossler et al. [12] proposed a general framework of face manipulation detection task, which adopts XceptionNet [13] as the main architecture and obtains a good performance. The CNN-based detection methods outperform human heuristic due to the powerful ability of deep neural networks and rich data sources. Thus many more effective methods are proposed [14], [15], [16] for better accuracy and transferability following this framework and brings a new challenge to deepfake video generation.

However, CNN-based detection methods can be easily fooled by adversarial attacks[17], which is an inherent vulnerability of CNN. This techniques can be an effective way to help to mislead the classification of CNN based detectors and can be be introduced as an ancillary way for face manipulation to fool the human eyes and CNN-based detectors simultaneously. To generate adversarial deepfake videos, current adversarial attack methods [18], [19], [20] commonly generate the adversarial perturbation frame by frame that will cause visual vibrations and are sensitive to human eyes at the video level. Moreover, the independently adversarial perturbation generation on each frame can also be easily detected from the video level. To demonstrate that the frame-independent adversarial perturbation generation can be easily detected, in this paper, we first propose a simple but novel framework inspired by the intrinsic coherence in time domain. We first utilize a denoising network to obtain an intermediate video, and calculate the coherence score [21] of original video and intermediate video respectively. We can easily classify whether the video has been perturbed according to the disparity of the coherence score. Therefore, how to simultaneously ensure the attack effectiveness and temporal consistency in perturbed manipulated videos is worthy of further research.

To address the problem caused by independent adversarial perturbation generation and preserve the temporal coherence of adjacent frames, we consider warping adversarial noises from the previous frame to the current one as a restriction. In this case, adversarial noises change position absolutely guided by the motion of the original video. Such a preliminary strategy is capable of making perturbed videos as coherent as possible. Although the preliminary strategy makes the perturbed videos smooth and stable, directly adding adversarial noise warped from the previous frame will lead to a reduction in attack success rate to a certain extent. To balance visual stability and attack success rate, we design a loss function to restrict the generation of adversarial noises. Thus we utilize the video optical flow between adjacent frames to restrict the generation of adversarial perturbation. Last but not least, an adaptive distortion cost is introduced, which is motivated by steganography, to metric the texture degrees of image regions and help to constrain the perturbations in complex texture areas (e.g., eyes, eyebrows, nose and hair) to keep the visual quality of videos. By doing this, we can not only guarantee the effectiveness of attack, but also preserve the coherence between frames as much as possible.

To verify the effectiveness of our method, we have conducted extensive experiments for different challenging cases. It demonstrates the effectiveness of our method when attacking different defense models, and the superiority of our method over several state-of-the-art ones. To summarize, the main contributions of our method are threefold as below:

• We propose a robust adversarial deepfake videos detector utilizing the coherence error over consecutive frames to distinguish the adversarial videos and clean ones. The detector can effectively detect weakly adversarial deepfake videos generated by current adversarial attack methods which add perturbations frame by frame on deepfake videos.

• To the best of our knowledge, we are the first attempt on generating coherent adversarial deepfake videos for fool the human eyes and CNN-based detectors simultaneously. And we demonstrate the effectiveness of our method in attacking different deepfake video detectors.

• We leverage the optical flow for restricting the generation of adversarial perturbations to keep the consistency between adjacent frames. We also introduce an adaptive distortion cost to constrain the total perturbations for improving the visual quality and the imperceptibility of adversarial deepfake videos.

The rest of the paper is organized as follows: we first make a brief summary of previous related work in Section 2. Next, our approach is described in detail in Section 3, including robust detection for weakly adversarial deepfake videos and coherent adversarial deepfake videos generation. Then we exhibit the experimental validation in Section 4. Finally, we conclude our work by discussing the significance of our result and related research prospects in Section 5.