Stepwise Change and Refine Network for Human Pose Transfer

Abstract

Achieving person image synthesis through pose guidance is an arduous and profound endeavor. Addressing the limitations of previous approaches, namely inaccurate pose generation and inconsistencies with the target texture, we present a new network. This two-stage network is designed not only to reposition a given person’s image to the intended pose but also to produce outcomes that are more believable and closely mirror authentic images. In the initial stage of our network, we employ Coarse Blocks, a series of modules sharing a uniform structure, to generate coarse images. This process involves gradually transforming the raw images towards the target pose, fostering improved shape consistency. Subsequently, we extract style features from the reference image using semantic distribution. These features play a pivotal role in optimizing the coarse image, leading to the production of the final generated image. Significantly, this enhanced image demonstrates increased fidelity to the visual characteristics of the target image. To further enhance the perceptual quality of the generated images, we introduce a novel loss function. This loss function is instrumental in aligning the generated images more closely with our cognition. We compare and ablate experiments with current state-of-the-art models, attest to the remarkable improvements achieved by our model, as evidenced by significant enhancements in SSIM, PSNR, and LPIPS scores.

Supported by Guizhou Provincial Key Technology R &D Program [2023] General 326.