Exposing Face Forgery Clues via Retinex-Based Image Enhancement

Abstract

Public concerns about deepfake face forgery are continually rising in recent years. Existing deepfake detection approaches typically use convolutional neural networks (CNNs) to mine subtle artifacts under high-quality forged faces. However, most CNN-based deepfake detectors tend to over-fit the content-specific color textures, and thus fail to generalize across different data sources, forgery methods, and/or post-processing operations. It motivates us to develop a method to expose the subtle forgery clues in RGB space. Herein, we propose to utilize multi-scale retinex-based enhancement of RGB space and compose a novel modality, named MSR, to complementary capture the forgery traces. To take full advantage of the MSR information, we propose a two-stream network combined with salience-guided attention and feature re-weighted interaction modules. The salience-guided attention module guides the RGB feature extractor to concentrate more on forgery traces from an MSR perspective. The feature re-weighted interaction module implicitly learns the correlation between the two complementary modalities to promote feature learning for each other. Comprehensive experiments on several benchmarks show that our method outperforms the state-of-the-art face forgery detection methods in detecting severely compressed deepfakes. Besides, our method also shows superior performances on cross-datasets evaluation.

H. Chen and Y. Lin—Contributed equally to this work.