Skip to main content
Springer Nature Link
Log in

CoPrGAN: Image-to-Image Translation via Content Preservation

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2022 (ICANN 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13531))

Included in the following conference series:

  • 2165 Accesses

Abstract

Image-to-image translation is an interesting and challenging application. At present, it has developed from single-domain to multi-domain and many other aspects. But losing content structure is still an inevitable problem, which is manifested in the details of the objects or the global features of the images. We propose a new framework named CoPrGAN to alleviate the lack of content. CoPrGAN focuses on the expressive ability of content and style in different dimensions. In this way, the model uses multiple dynamic paths between the content encoder and the decoder to transform domains while preserving content structure. The content structure we are concerned with is not just posture and location, but also birthmarks, hair color, environment, etc. Experiments are arranged in animal face change that focus on local details and seasonal change that focus on global information. Both the comparative experiments with state-of-the-art and the ablation experiments demonstrate the superiority of CoPrGAN.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Anokhin, I., et al.: High-resolution daytime translation without domain labels. In: CVPR, pp. 7488–7497 (2020)

    Google Scholar 

  2. Barzilay, N., Shalev, T.B., Giryes, R.: Miss GAN: a multi-illustrator style generative adversarial network for image to illustration translation. Pattern Recogn. Lett. 151, 140–147 (2021)

    Article  Google Scholar 

  3. Chang, H.-Y., Wang, Z., Chuang, Y.-Y.: Domain-specific mappings for generative adversarial style transfer. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12353, pp. 573–589. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58598-3_34

    Chapter  Google Scholar 

  4. Chen, J., Liu, G., Chen, X.: AnimeGAN: a novel lightweight GAN for photo animation. In: Li, K., Li, W., Wang, H., Liu, Y. (eds.) ISICA 2019. CCIS, vol. 1205, pp. 242–256. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-5577-0_18

    Chapter  Google Scholar 

  5. Chen, R., Chen, X., Ni, B., Ge, Y.: SimSwap: an efficient framework for high fidelity face swapping. In: ACM MM, pp. 2003–2011 (2020)

    Google Scholar 

  6. Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., Liu, Z.: Dynamic convolution: attention over convolution kernels. In: CVPR, pp. 11030–11039 (2020)

    Google Scholar 

  7. Cho, W., Choi, S., Park, D.K., Shin, I., Choo, J.: Image-to-image translation via group-wise deep whitening-and-coloring transformation. In: CVPR, pp. 10639–10647 (2019)

    Google Scholar 

  8. Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: CVPR, pp. 8789–8797 (2018)

    Google Scholar 

  9. Choi, Y., Uh, Y., Yoo, J., Ha, J.W.: StarGAN V2: diverse image synthesis for multiple domains. In: CVPR, pp. 8188–8197 (2020)

    Google Scholar 

  10. Dong, H., Neekhara, P., Wu, C., Guo, Y.: Unsupervised image-to-image translation with generative adversarial networks. arXiv preprint arXiv:1701.02676 (2017)

  11. Gabbay, A., Hoshen, Y.: Scaling-up disentanglement for image translation. In: ICCV, pp. 6783–6792 (2021)

    Google Scholar 

  12. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: NeurIPS, vol. 30 (2017)

    Google Scholar 

  13. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: ICCV, pp. 1501–1510 (2017)

    Google Scholar 

  14. Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: ECCV, pp. 172–189 (2018)

    Google Scholar 

  15. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR, pp. 1125–1134 (2017)

    Google Scholar 

  16. Jang, E., Gu, S., Poole, B.: Categorical reparameterization with Gumbel-Softmax. arXiv preprint arXiv:1611.01144 (2016)

  17. Jeong, S., Lee, J., Sohn, K.: Multi-domain unsupervised image-to-image translation with appearance adaptive convolution. In: ICASSP, pp. 1750–1754. IEEE (2022)

    Google Scholar 

  18. Jia, X., De Brabandere, B., Tuytelaars, T., Gool, L.V.: Dynamic filter networks. In: NeurIPS, vol. 29 (2016)

    Google Scholar 

  19. Kim, J., Kim, M., Kang, H., Lee, K.H.: U-GAT-IT: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation. In: ICLR (2020)

    Google Scholar 

  20. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes (2014)

    Google Scholar 

  21. Lee, H.Y., Tseng, H.Y., Huang, J.B., Singh, M., Yang, M.H.: Diverse image-to-image translation via disentangled representations. In: ECCV, pp. 35–51 (2018)

    Google Scholar 

  22. Li, B., et al.: DyStyle: dynamic neural network for multi-attribute-conditioned style editing. arXiv preprint arXiv:2109.10737 (2021)

  23. Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Universal style transfer via feature transforms. In: NeurIPS, vol. 30 (2017)

    Google Scholar 

  24. Liu, A.H., Liu, Y.C., Yeh, Y.Y., Wang, Y.C.F.: A unified feature disentangler for multi-domain image translation and manipulation. In: NeurIPS, vol. 31 (2018)

    Google Scholar 

  25. Liu, M.Y., et al.: Few-shot unsupervised image-to-image translation. In: ICCV, pp. 10551–10560 (2019)

    Google Scholar 

  26. Mao, Q., Lee, H.Y., Tseng, H.Y., Ma, S., Yang, M.H.: Mode seeking generative adversarial networks for diverse image synthesis. In: CVPR, pp. 1429–1437 (2019)

    Google Scholar 

  27. Mao, Q., Tseng, H.Y., Lee, H.Y., Huang, J.B., Ma, S., Yang, M.H.: Continuous and diverse image-to-image translation via signed attribute vectors. In: IJCV, pp. 1–33 (2022)

    Google Scholar 

  28. Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Paul Smolley, S.: Least squares generative adversarial networks. In: ICCV, pp. 2794–2802 (2017)

    Google Scholar 

  29. Nguyen, T.P., Lathuiliére, S., Ricci, E.: Multi-domain image-to-image translation with adaptive inference graph. In: ICPR, pp. 5368–5375. IEEE (2021)

    Google Scholar 

  30. Veit, A., Belongie, S.: Convolutional networks with adaptive inference graphs. In: ECCV, pp. 3–18 (2018)

    Google Scholar 

  31. Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs. In: CVPR, pp. 8798–8807 (2018)

    Google Scholar 

  32. Wu, P.W., Lin, Y.J., Chang, C.H., Chang, E.Y., Liao, S.W.: RelGAN: multi-domain image-to-image translation via relative attributes. In: ICCV, pp. 5914–5922 (2019)

    Google Scholar 

  33. Yang, G., Fei, N., Ding, M., Liu, G., Lu, Z., Xiang, T.: L2M-GAN: learning to manipulate latent space semantics for facial attribute editing. In: CVPR, pp. 2951–2960 (2021)

    Google Scholar 

  34. Yu, X., Chen, Y., Liu, S., Li, T., Li, G.: Multi-mapping image-to-image translation via learning disentanglement. In: NeurIPS, vol. 32 (2019)

    Google Scholar 

  35. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV, pp. 2223–2232 (2017)

    Google Scholar 

Download references

Acknowledgements

We thank the reviewers for their constructive comments. And we thank predecessors in the field of I2I translation for their inspiring works.

Author information

Authors and Affiliations

  1. National Computer System Engineering Research Institute of China, Beijing, China

    Xiaoming Yu & Gan Zhou

Authors
  1. Xiaoming Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Yu .

Editor information

Editors and Affiliations

  1. University of the West of England, Bristol, UK

    Elias Pimenidis

  2. Lancaster University, Lancaster, UK

    Plamen Angelov

  3. Digital Innovation, Teeside University, Middlesbrough, UK

    Chrisina Jayne

  4. Democritus University of Thrace, Xanthi, Greece

    Antonios Papaleonidas

  5. The University of the West of England, Bristol, UK

    Mehmet Aydin

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yu, X., Zhou, G. (2022). CoPrGAN: Image-to-Image Translation via Content Preservation. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13531. Springer, Cham. https://doi.org/10.1007/978-3-031-15934-3_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15934-3_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15933-6

  • Online ISBN: 978-3-031-15934-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics