Skip to main content

Advertisement

SpringerLink
Log in

L-GAN: landmark-based generative adversarial network for efficient face de-identification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

A large amount of high-quality data are collected through autonomous vehicles, CCTVs, guidance service robots, and web map services (Google Street View). However, the data collected through them include personal information such as peoples’ faces and vehicle license plates. Currently, personal information contained in data is de-identified using methods such as face blur, pixelation, and masking. Consequently, it loses value as data for artificial intelligence (AI) learning. Therefore, in this study, we propose a model to generate a fake face that maintains the basic structure of the human face. There are several methods for generating faces. One is to generate them using a generative adversarial network (GAN) model. The GAN is an AI algorithm used for unsupervised learning and is implemented by a system in which two neural networks compete. However, because GAN operates as an input of a random noise vector, it is difficult to obtain results for the desired face angle and shape. Therefore, pre- and post-processing is required to generate a fake face that maintains the basic structure and angles; however, the calculation is complicated, and it is difficult to generate a natural image. To solve this problem, we propose a method for generating a fake face that maintains the basic structure and angle of the real face by applying a facial landmark. Using the proposed method, it was possible to generate a fake face with a different impression while maintaining the basic structure and angle of the face.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

  2. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Comput Intell Neurosci 2018:1–13

    Google Scholar 

  3. Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 160(1):106

    Article  Google Scholar 

  4. Fukushima K (1988) Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw 1(2):119–130

    Article  Google Scholar 

  5. Ide H, Kurita T (2017) Improvement of learning for CNN with ReLU activation by sparse regularization. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, pp. 2684–2691.

  6. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  7. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

  8. Ribaric S, Ariyaeeinia A, Pavesic N (2016) De-identification for privacy protection in multimedia content: a survey. Signal Process Image Commun 47:131–151

    Article  Google Scholar 

  9. Meden B, Rot P, Terhörst P, Damer N, Kuijper A, Scheirer WJ et al (2021) Privacy–enhancing face biometrics: a comprehensive survey. IEEE Trans Inf Forensics Secur 16:4147–4183

  10. Oh SJ, Benenson R, Fritz M, Schiele B (2016) Faceless person recognition: privacy implications in social media. European Conference on Computer Vision. Springer, Cham, pp 19–35

    Google Scholar 

  11. McPherson R, Shokri R, Shmatikov V (2016) Defeating image obfuscation with deep learning. arXiv preprint arXiv:1609.00408

  12. Newton EM, Sweeney L, Malin B (2005) Preserving privacy by de-identifying face images. IEEE Trans Knowl Data Eng 17(2):232–243

    Article  Google Scholar 

  13. Gross R, Sweeney L, De la Torre F, Baker S (2006) Model-based face de-identification. In: 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06). IEEE, pp. 161–161

  14. Gross R, Airoldi E, Malin B, Sweeney L (2005) Integrating utility into face de-identification. International Workshop on Privacy Enhancing Technologies. Springer, Berlin, pp 227–242

    Google Scholar 

  15. Sweeney L (2002) k-anonymity: a model for protecting privacy. Int J Uncertain Fuzziness Knowl Based Syst 10(05):557–570

    Article  MathSciNet  MATH  Google Scholar 

  16. Cootes T, Edwards G, Taylor C (2001) Robust real-time periodic motion detection, analysis, and applications. IEEE Trans Patt Analy Mach Intell 23(6):681–685

    Article  Google Scholar 

  17. Du L, Yi M, Blasch E, Ling H (2014) GARP-face: Balancing privacy protection and utility preservation in face de-identification. In: IEEE International Joint Conference on Biometrics. IEEE, pp. 1–8

  18. Jourabloo A, Yin X, Liu X (2015) Attribute preserved face de-identification. In: 2015 International Conference on Biometrics (ICB). IEEE, pp. 278–285

  19. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S et al (2014) Generative adversarial nets. In: Advances in neural information processing systems, vol 27, pp 2672–2680

  20. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint at arXiv:1511.06434

  21. Chen J, Konrad J, Ishwar P (2018) Vgan-based image representation learning for privacy-preserving facial expression recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1570–1579

  22. Ren Z, Lee YJ, Ryoo MS (2018) Learning to anonymize faces for privacy preserving action detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 620–636

  23. Li T, Lin L (2019) Anonymousnet: natural face de-identification with measurable privacy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops

  24. Wang HP, Orekondy T, Fritz M (2021) Infoscrub: towards attribute privacy by targeted obfuscation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3281–3289

  25. Hukkelås H, Mester R, Lindseth F (2019) Deepprivacy: a generative adversarial network for face anonymization. International symposium on visual computing. Springer, Cham, pp 565–578

    Google Scholar 

  26. Sun Q, Ma L, Oh SJ, Van Gool L, Schiele B, Fritz M (2018) Natural and effective obfuscation by head inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5050–5059

  27. Mao X, Li Q, Xie H, Lau RY, Wang Z, Paul Smolley S (2017) Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2794–2802

  28. Wen Y, Song L, Liu B, Ding M, Xie R (2021) Identitydp: Differential private identification protection for face images. Preprint at arXiv:2103.01745

  29. Cho D, Lee JH, Suh IH (2020) CLEANIR: controllable attribute-preserving natural identity remover. Appl Sci 10(3):1120

    Article  Google Scholar 

  30. Zhan F, Zhu H, Lu S (2019) Spatial fusion gan for image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3653–3662

  31. Kuang Z, Liu H, Yu J, Tian A, Wang L, Fan J, Babaguchi N (2021) Effective de-identification generative adversarial network for face anonymization. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3182–3191

  32. Zhang X, Wang X, Shi C, Yan Z, Li X, Kong B, Mumtaz I (2022) De-gan: domain embedded gan for high quality face image inpainting. Pattern Recogn 124:108415

    Article  Google Scholar 

  33. Li Y, Lu Q, Tao Q, Zhao X, Yu Y (2021) SF-GAN: face de-identification method without losing facial attribute information. IEEE Signal Process Lett 28:1345–1349

    Article  Google Scholar 

  34. Mirza M, Osindero S (2014) Conditional generative adversarial nets. Preprint at arXiv:1411.1784

  35. Murez Z, Kolouri S, Kriegman D, Ramamoorthi R, Kim K (2018) Image to image translation for domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4500–4509

  36. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134

  37. Demir U, Unal G (2018) Patch-based image inpainting with generative adversarial networks. Preprint at arXiv:1803.07422

  38. Berthelot D, Schumm T, Metz L (2017) Began: Boundary equilibrium generative adversarial networks. Preprint at arXiv:1703.10717

  39. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410

  40. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232

  41. Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. European Conference on Computer Vision. Springer, Cham, pp 94–108

    Google Scholar 

  42. Nguyen TT, Nguyen QVH, Nguyen CM, Nguyen D, Nguyen DT, Nahavandi S (2019) Deep learning for deepfakes creation and detection: a survey. Preprint at arXiv:1909.11573.

  43. Liu Y, Jourabloo A, Ren W, Liu X (2017) Dense face alignment. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1619–1628

  44. King DE (2009) Dlib-ml: a machine learning toolkit. J Mach Learn Res 10:1755–1758

    Google Scholar 

  45. Shu C, Ding X, Fang C (2011) Histogram of the oriented gradient for face recognition. Tsinghua Sci Technol 16(2):216–224

    Article  Google Scholar 

  46. Culjak I, Abram D, Pribanic T, Dzapo H, Cifrek M (2012) A brief introduction to OpenCV. In: 2012 Proceedings of the 35th International Convention MIPRO. IEEE, pp. 1725–1730

  47. Liu Z, Luo P, Wang X, Tang X (2018) Large-scale celebfaces attributes (celeba) dataset. Retrieved August 15(2018):11

    Google Scholar 

  48. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, pp 234–241

    Google Scholar 

  49. Li J, Wang Y, Wang C, Tai Y, Qian J, Yang J, Huang F (2019) DSFD: dual shot face detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5060–5069

Download references

Acknowledgements

This paper was supported by Konkuk University Researcher Fund in 2021.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Konkuk University, Neungdong-ro, Gwangjin-gu, Seoul, 143-701, Korea

    Sung-su Jang, Cheol-jin Kim, Seong-yeon Hwang, Myung-jae Lee & Young-guk Ha

Authors
  1. Sung-su Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cheol-jin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seong-yeon Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Myung-jae Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Young-guk Ha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Young-guk Ha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jang, Ss., Kim, Cj., Hwang, Sy. et al. L-GAN: landmark-based generative adversarial network for efficient face de-identification. J Supercomput 79, 7132–7159 (2023). https://doi.org/10.1007/s11227-022-04954-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04954-x

Keywords

Search

Navigation