Skip to main content

Advertisement

SpringerLink
Log in

Adaptive semantic attribute decoupling for precise face image editing

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Precisely editing user specified facial attributes has wide applications in areas such as virtual makeup, face aging, facial expression transfer, face synthesis. However, it is difficult to explicitly control individual facial attribute due to the gap between high level semantics in human perception and feature vectors in latent space. In this paper, a semantic disentanglement algorithm interpreting the latent space of GAN is proposed, which can be employed to extract attribute control vector adaptive to individual face. By adjusting the coefficient of extracted control vector, variation of single attribute is realized. Then, comprehensive modification effect of facial attributes is obtained through the superposition of control vector. Classification and content loss functions are introduced to limit modification occurs to the specified attribute without affecting the other attributes. As a result, precise editing control is realized.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems pp. 2672–2680, (2014)

  2. Shen, Y., Gu, J., Tang, X., et al.: Interpreting the latent space of gans for semantic face editing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 9243–9252, (2020)

  3. Mirza, M., Osindero, S.: Conditional generative adversarial nets, arXiv preprint arXiv:1411.1784 (2014)

  4. Chen, X., Duan, Y., Houthooft, R., et al.: Infogan: interpretable representation learning by information maximizing gener-ative adversarial nets. In: Advances in Neural Information Processing Systems pp. 2172–2180, (2016)

  5. Shoshan, A., Bhonker, N., Kviatkovsky, I., et al.: GAN-Control: explicitly controllable GANs, arXiv preprint arXiv:2101.02477 (2021)

  6. Zhu, J.Y., Park, T., Isola, P., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer vision pp. 2223–2232, (2017)

  7. Choi, Y., Choi, M., Kim, M., et al.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 8789–8797, (2018)

  8. KIM, T., CHA, M., KIM, H., et al.: Learning to discover cross-domain relations with generative adversarial networks, arXiv preprint arXiv:1703.05192 (2017)

  9. Lample, G., Zeghidour, N., Usunier, N., et al.: Fader networks: manipulating images by sliding attributes. In: Advances in Neural Information Processing Systems pp. 5967–5976, (2017)

  10. Shen, W., Liu, R.: Learning residual images for face attribute manipulation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 4030–4038, (2017)

  11. Perarnau, G., Van De Weijer, J., Raducanu, B., et al.: Invertible conditional gans for image editing, arXiv preprint arXiv:1611.06355 (2016)

  12. Harkonen, E., Hertzmann, A., Lehtinen, J., et al.: Ganspace: Discovering interpretable gan controls, arXiv preprint arXiv:2004.02546 (2020)

  13. Shen, Y., Zhou, B.: Closed-form factorization of latent semantics in gans, arXiv preprint arXiv:2007.06600 (2020)

  14. Zhang, Z., Song, Y., Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE conference on computer vision and pattern recognition pp. 5810–5818, (2017)

  15. HE, Z., ZUO, W., KAN, M., et al.: Arbitrary facial attribute editing: only change what you want, arXiv preprint arXiv:1711.10678, 1(3) (2017)

  16. Isola, P., Zhu, J.Y., Zhou, T., et al.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern recognition pp. 1125–1134, (2017)

  17. Yin, W., Fu, Y., Sigal, L., et al.: Semi-latent gan: learning to generate and modify facial images from attributes, arXiv preprint arXiv:1704.02166 (2017)

  18. Natsume, R., Yatagawa, T., Morishima, S.: Rsgan: face swapping and editing using face and hair representation in latent spaces, arXiv preprint arXiv:1804.03447 (2018)

  19. JIANG, S., TAO, Z., FU, Y.: Segmentation guided image-to-image translation with adversarial networks. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1-7 (2019)

  20. Jo, Y., Park, J.: SC-FEGAN: face editing generative adversarial network with user’s sketch and color. In: Proceedings of the IEEE International Conference on Computer Vision pp. 1745–1753, (2019)

  21. Zhu, J., Zhao, D., Zhang, B.: Lia: latently invertible autoencoder with adversarial learning, arXiv preprint arXiv:1906.08090 (2019)

  22. Karras, T., Laine, S., Aila, T.: A style-based generator archi-tecture for generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 4401–4410, (2019)

  23. Karras, T., Aila, T., Laine, S., et al.: Progressive growing of gans for improved quality, stability, and variation. In: International Conference on Learning Representations (2018)

  24. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein gan, arXiv preprint arXiv:1701.07875 (2017)

Download references

Acknowledgements

This paper was sponsored by the Public Welfare Research Project of Zhejiang Province, China (Grant No. LGF18F020015), Opening Foundation of Key Laboratory of Fundamental Science for National Defense on Vision Synthetization, Sichuan University, China (Grant No. 2020SCUVS007), Opening Foundation of Zhejiang Police College, China (Grant No.2020DSJSYS002), and JSPS Grants–in–Aid for Scientific Research, Japan (Grant No. 17H00737), and Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, China.

Author information

Authors and Affiliations

  1. School of Computer Science, Hangzhou Dianzi University, Xiasha Higher Education Zone, Hangzhou, 310018, China

    Yixuan Ju, Jianhai Zhang, Xiaoyang Mao & Jiayi Xu

  2. Department of Computer Science and Engineering, University of Yamanashi, Kofu, 400-8511, Japan

    Xiaoyang Mao

Authors
  1. Yixuan Ju
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyang Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiayi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiayi Xu.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ju, Y., Zhang, J., Mao, X. et al. Adaptive semantic attribute decoupling for precise face image editing. Vis Comput 37, 2907–2918 (2021). https://doi.org/10.1007/s00371-021-02198-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02198-z

Keywords

Search

Navigation