Feature Space Based Loss for Face Photo-Sketch Synthesis

Abstract

Learning-based face photo-sketch synthesis has made great progress in the past few years because of the development of the generate adversarial networks (GANs)  [1]. However, these existing GAN-based methods mostly yield poor texture and details on the synthesized sketch/photo which leads low perceptual similarity between the synthesized sketch/photo and real sketch/photo. In order to tackle this problem, we first introduce the perceptual loss into our objective loss function which can measure the difference of content and style between synthesized sketch/photo and real sketch/photo in feature level. Second, we propose a feature map based loss termed content feature loss, which are utilized to supervise the generator in our network to make the synthesized sketch/photo have more perceptual quality. To achieve this, we use the pre-trained VGG network to extract the feature maps of the sketch/photo as a feature extractor, and calculate the Euclidean difference between these feature maps and the feature maps from the hidden layers of the generator. Extensive experiments both synthesis quality and recognition ability assessment of the public face photo-sketch database are conducted to show that our method can obtain better results in comparison with existing state-of-the-art methods.

Supported in part by the National Key Research and Development Program of China under Grant 2018AAA0103202, in part by the National Natural Science Foundation of China under Grant 61922066, Grant 61876142, Grant 61671339, Grant 61772402, Grant U1605252, Grant 61976166, and Grant 62036007, in part by the National High-Level Talents Special Support Program of China under Grant CS31117200001, in part by the Fundamental Research Funds for the Central Universities under Grant JB190117, in part by the Xidian University Intellifusion Joint Innovation Laboratory of Artificial Intelligence, and in part by the Innovation Fund of Xidian University.