Abstract
Gait and face are the two major biometric features that need to be de-identified to preserve the privacy of an individual and conceal his/her identity. Unlike face de-identification, no research has been conducted on gait de-identification to date. A few existing body/silhouette de-identification approaches use blurring and other primitive image processing techniques that are not robust to varying input environments and tend to remove non-biometric features like appearance and activity. In this paper, we propose a plausible deep learning-based solution to the gait de-identification problem. First, a set of key walking poses is determined from a large gallery set. Next, given an input sequence, a graph-based path search algorithm is employed to classify each frame of the sequence into the appropriate key pose. Next, a random frame with matched key pose chosen from the subset of the gallery sequences is considered the target frame. The dense pose features of the input and target frames are then fused using our proposed gait transformation network (GTNet), which is trained using a combination of perceptual loss, L1 loss, and adversarial loss. Training and testing of the model have been conducted using the RGB sequences present in the TUM-GAID and CASIA-B gait data. Promising de-identification results are obtained both qualitatively and quantitatively.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-022-02386-x/MediaObjects/11760_2022_2386_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-022-02386-x/MediaObjects/11760_2022_2386_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-022-02386-x/MediaObjects/11760_2022_2386_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-022-02386-x/MediaObjects/11760_2022_2386_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-022-02386-x/MediaObjects/11760_2022_2386_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11760-022-02386-x/MediaObjects/11760_2022_2386_Fig6_HTML.png)
Similar content being viewed by others
Availability of data and materials
Pretrained model will be made available on paper acceptance.
References
Agarwal, A., Chattopadhyay, P., Wang, L.: Privacy preservation through facial de-identification with simultaneous emotion preservation. Signal Image Video Process. 15(5), 951–958 (2021)
Gafni, O., Wolf, L., Taigman, Y.: Live face de-identification in video. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision, pp. 9378–9387. (2019)
Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., Van Gool, L.: Pose guided person image generation. In: Proc. of the Advances in Neural Information Processing Systems, 30. (2017)
Ma, L., Sun, Q., Georgoulis, S., Van Gool, L., Schiele, B., Fritz, M.: Disentangled person image generation. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 99–108. (2018)
Agrawal, P., Narayanan, P.J.: Person de-identification in videos. IEEE Trans. Circuits Syst. Video Technol. 21(3), 299–310 (2011)
Boyle, M., Edwards, C., Greenberg, S.: The effects of filtered video on awareness and privacy. In: Proc. of the 2000 ACM Conf. on Computer Supported Cooperative Work, pp. 1–10. (2000)
Park, S., Trivedi, M. M.: A track-based human movement analysis and privacy protection system adaptive to environmental contexts. In: Proc. of the IEEE Conf. on Advanced Video and Signal Based Surveillance, pp. 171–176. (2005)
Brkic, K., Sikiric, I., Hrkac, T., Kalafatic, Z.: I know that person: Generative full body and face de-identification of people in images. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshops, pp. 1319–1328. (2017)
Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Proc. of the Advances in Neural Information Processing Systems, 28. (2015)
Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576. (2015)
Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: feed-forward synthesis of textures and stylized images. In: Proc. of the Intl. Conf. on Machine Learning, pp. 1349–1357. (2016)
Gross, R., Sweeney, L., De la Torre, F., Baker, S.: Model-based face de-identification. In: Proc. of the Conf. on Computer Vision and Pattern Recognition Workshop, pp. 161–168. (2006)
Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput. Hum. Interact. 13(1), 1–36 (2006)
Oh, S.J., Benenson, R., Fritz, M., Schiele, B.: Faceless person recognition: privacy implications in social media. In: Proc. of the European Conf. on Computer Vision, pp. 19–35. (2016)
Aggarwal, A., Rathore, R., Chattopadhyay, P., Wang, L.: EPD-net: A GAN-based architecture for face de-identification from images. In: Proc. of the IEEE Intl. IOT, Electronics and Mechatronics Conf., pp. 1–7. (2020)
Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005)
Gross, R., Airoldi, E., Malin, B., Sweeney, L.: Integrating utility into face de-identification. In: Proc. of the Intl. Workshop on Privacy Enhancing Technologies, pp. 227–242. (2005)
Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Proc. of the Advances in Neural Information Processing Systems, 27. (2014)
Wen, Y., Bo, L., Cao, J., Xie, R., Song, L., Li, Z.: IdentityMask: deep motion flow guided reversible face video de-identification. IEEE Trans. Circuits Syst. Video Technol. (2022). https://doi.org/10.1109/TCSVT.2022.3191982
Korshunova, I., Shi, W., Dambre, J., Theis, L.: Fast face-swap using convolutional neural networks. In: Proc. of the IEEE Intl. Conf. on Computer Vision, pp. 3677–3685. (2017)
Dou, S., Jiang, X., Zhao, Q., Li, D., Zhao, C.: Towards privacy-preserving person re-identification via person identify shift. arXiv preprint arXiv:2207.07311. (2022)
Ren, Z., Lee, Y.J., Ryoo, M.S.: Learning to anonymize faces for privacy preserving action detection. In: Proc. of the European Conf. on Computer Vision, pp. 620–636. (2018)
Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. (2014)
Gong, M., Liu, J., Li, H., Xie, Yu., Tang, Z.: Disentangled representation learning for multiple attributes preserving face deidentification. IEEE Trans. Neural Netw. Learn. Syst. (2020). https://doi.org/10.1109/TNNLS.2020.3027617
Chen, D., Chang, Y., Yan, R., Yang, J.: Tools for protecting the privacy of specific individuals in video. EURASIP J. Adv. Signal Process. 2007, 1–9 (2007)
Roy, A., Sural, S., Mukherjee, J.: Gait recognition using pose kinematics and pose energy image. Signal Process. 92(3), 780–792 (2012)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 3431–3440. (2015)
Gupta, S.K., Chattopadhyay, P.: Gait recognition in the presence of co-variate conditions. Neurocomputing 454, 76–87 (2021)
Güler, RA., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 7297–7306. (2018)
Albahar, B., Jingwan, L., Yang, J., Shu, Z., Shechtman, E., Huang, J.-B.: Pose with style: Detail-preserving pose-guided image synthesis with conditional stylegan. ACM Trans. Graph. 40(6), 1–11 (2021)
Zhu, Z., Huang, T., Shi, B., Yu, M., Wang, B., Bai, X.: Progressive pose attention transfer for person image generation. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 2347–2356. (2019)
SSimonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. (2014)
Hofmann, M., Geiger, J., Bachmann, S., Schuller, B., Rigoll, G.: The tum gait from audio, image and depth (GAID) database: multimodal recognition of subjects and traits. J. Vis. Commun. Image Represent. 25(1), 195–206 (2014)
Yu, S., Tan, D., Tan, T.: A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: Proc. of the \(18^{th}\) Intl. Conf. on Pattern Recognition, vol. 4, pp. 441–444. (2006)
Chao, H., He, Y., Zhang, J., Feng, J.: Gaitset: regarding gait as a set for cross-view gait recognition. In: Proc. of the AAAI Conf. on Artificial Intelligence, vol. 33, 01, pp. 8126–8133. (2019)
Hou, S., Cao, C., Liu, X., Huang, Y.: Gait lateral network: Learning discriminative and compact representations for gait recognition. In: Proc. of the European Conf. on Computer Vision, pp. 382–398, (2020)
Fan, C., Peng, Y., Cao, C., Liu, X., Hou, S., Chi, J., Huang, Y., Li, Q., He, Z.: Gaitpart: temporal part-based model for gait recognition. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 14225–14233. (2020)
Lin, B., Zhang, S., Yu, X.: Gait recognition via effective global-local feature representation and local temporal aggregation. In: Proc. of the IEEE/CVF Intl. Conf. on Computer Vision, pp. 14648–14656. (2021)
Acknowledgements
The authors would like to thank SERB, DST, GoI for supporting this work with a server.
Funding
The server used in the work is purchased from grant CRG/2020/005465 (SERB, DST, GoI).
Author information
Authors and Affiliations
Contributions
AH has done the concept modeling, implementation, and manuscript writing. PC supervised the entire work and helped AH in formulating the concepts and also thoroughly checked the manuscript. SK has also helped in concept building and proof-reading of the entire paper.
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Consent for publication
Not applicable.
Ethics approval and consent to participate
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Halder, A., Chattopadhyay, P. & Kumar, S. Gait transformation network for gait de-identification with pose preservation. SIViP 17, 1753–1761 (2023). https://doi.org/10.1007/s11760-022-02386-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-022-02386-x