GaitGANv2: Invariant gait feature extraction using generative adversarial networks

Abstract

The performance of gait recognition can be adversely affected by many sources of variation such as view angle, clothing, presence of and type of bag, posture, and occlusion, among others. To extract invariant gait features, we proposed a method called GaitGANv2 which is based on generative adversarial networks (GAN). In the proposed method, a GAN model is taken as a regressor to generate a canonical side view of a walking gait in normal clothing without carrying any bag. A unique advantage of this approach is that, unlike other methods, GaitGANv2 does not need to determine the view angle before generating invariant gait images. Indeed, only one model is needed to account for all possible sources of variation such as with or without carrying accessories and varying degrees of view angle. The most important computational challenge, however, is to address how to retain useful identity information when generating the invariant gait images. To this end, our approach differs from the traditional GAN in that GaitGANv2 contains two discriminators instead of one. They are respectively called fake/real discriminator and identification discriminator. While the first discriminator ensures that the generated gait images are realistic, the second one maintains the human identity information. The proposed GaitGANv2 represents an improvement over GaitGANv1 in that the former adopts a multi-loss strategy to optimize the network to increase the inter-class distance and to reduce the intra-class distance, at the same time. Experimental results show that GaitGANv2 can achieve state-of-the-art performance.

Introduction

Gait is a behavioural biometric modality with a great potential for person identification because of its unique advantages such as being contactless, hard to fake and passive in nature, i.e., it requires no explicit cooperation from the subjects. Furthermore, the gait features can be captured at a distance in uncontrolled scenarios. Therefore, gait recognition is a very valuable technique in video surveillance, with a wide-ranging applications. This is particular so since many surveillance cameras have already been installed in major cities around world. Therefore, by continually improving its accuracy, the gait recognition technology will certainly add to the repertoire of tools available for crime prevention and forensic identification. For this reason, gait recognition is and will become an ever more important research topic in the computer vision community.

Unfortunately, automatic gait recognition remains a challenging task because it suffers from many potential sources of variation that can alter the human appearance drastically, such as, but not limited to aspects such as viewpoint, clothing, and objects being carried. These variations can affect the recognition accuracy greatly. Among these sources of variation, view angle is one of the most common one because we can not control the walking directions of subjects in real applications, and that is the central focus of our work here.

As a proof of concept, we shall consider variability in conditions of consisting of view angle, choice of clothing and type of objects being carried by the subject. The proposed generative adversarial networks (GAN) can handle all these variations simultaneously by using only one model. GAN acts as a regressor which takes an gait image captured with any combination of the above-mentioned sources of variation and then transforms it into a canonical side view image The method can do so without any knowledge regarding the factors that contribute to the gait variability. The most important computational challenge, however, is to address how to retain useful identity information when generating the canonical, invariant gait images.

The rest of the paper is organized as follows. Section 2 presents the state-of-the-art literature in gait recognition that deals with invariance in gait recognition. Section 3 describes the proposed method. Experiments and evaluation are presented in Section 4. The last section, Section 5, gives the conclusions and identifies future work.