Skip to main content

Towards Compact Reversible Image Representations for Neural Style Transfer

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Abstract

Arbitrary neural style transfer aims to stylise a content image by referencing a provided style image. Despite various efforts to achieve both content preservation and style transferability, learning effective representations for this task remains challenging since the redundancy of content and style features leads to unpleasant image artefacts. In this paper, we learn compact neural representations for style transfer motivated from an information theoretical perspective. In particular, we enforce compressive representations across sequential modules of a reversible flow network in order to reduce feature redundancy without losing content preservation capability. We use a Barlow twins loss to reduce channel dependency and thus to provide better content expressiveness, and optimise the Jensen-Shannon divergence of style representations between reference and target images to avoid under- and over-stylisation. We comprehensively demonstrate the effectiveness of our proposed method in comparison to other state-of-the-art style transfer approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alemi, A.A., Fischer, I., Dillon, J.V., Murphy, K.: Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 (2016)

  2. An, J., Huang, S., Song, Y., Dou, D., Liu, W., Luo, J.: ArtFlow: unbiased image style transfer via reversible neural flows. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 862–871 (2021)

    Google Scholar 

  3. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223 (2017)

    Google Scholar 

  4. Barlow, H.: Redundancy reduction revisited. Netw. Comput. Neural Syst. 12(3), 241 (2001)

    Article  MathSciNet  Google Scholar 

  5. Barlow, H.B.: Possible principles underlying the transformation of sensory messages. Sens. Commun. 1(01), 217–233 (1961)

    Google Scholar 

  6. Chen, D., Yuan, L., Liao, J., Yu, N., Hua, G.: StyleBank: an explicit representation for neural image style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1897–1906 (2017)

    Google Scholar 

  7. Chen, H., et al.: Artistic style transfer with internal-external learning and contrastive learning. In: Advances in Neural Information Processing Systems, vol. 34, pp. 26561–26573 (2021)

    Google Scholar 

  8. Cheng, J., Jaiswal, A., Wu, Y., Natarajan, P., Natarajan, P.: Style-aware normalized loss for improving arbitrary style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 134–143 (2021)

    Google Scholar 

  9. Deng, Y., et al.: StyTr2: image style transfer with transformers. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11326–11336 (2022)

    Google Scholar 

  10. Dinh, L., Krueger, D., Bengio, Y.: NICE: non-linear independent components estimation. arXiv preprint arXiv:1410.8516 (2014)

  11. Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. arXiv preprint arXiv:1610.07629 (2016)

  12. Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. Roy. Soc. Lond. Ser. A 222(594–604), 309–368 (1922)

    Google Scholar 

  13. Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Advances in Neural Information Processing Systems 28 (2015)

    Google Scholar 

  14. Gatys, L.A., Bethge, M., Hertzmann, A., Shechtman, E.: Preserving color in neural artistic style transfer. arXiv preprint arXiv:1606.05897 (2016)

  15. Gomez, A.N., Ren, M., Urtasun, R., Grosse, R.B.: The reversible residual network: backpropagation without storing activations. In: Advances in Neural Information Processing Systems 30 (2017)

    Google Scholar 

  16. Huang, S., et al.: Parameter-free style projection for arbitrary image style transfer. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2070–2074 (2022)

    Google Scholar 

  17. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: IEEE/CVF International Conference on Computer Vision, pp. 1501–1510 (2017)

    Google Scholar 

  18. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  19. Karayev, S., et al.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)

  20. Kingma, D.P., Dhariwal, P.: Glow: generative flow with invertible \(1\times 1\) convolutions. In: Advances in Neural Information Processing Systems 31 (2018)

    Google Scholar 

  21. Kitaev, N., Kaiser, Ł., Levskaya, A.: Reformer: the efficient transformer. arXiv preprint arXiv:2001.04451 (2020)

  22. Lee, J., Kim, H., Shim, J., Hwang, E.: Cartoon-Flow: a flow-based generative adversarial network for arbitrary-style photo cartoonization. In: ACM International Conference on Multimedia, pp. 1241–1251 (2022)

    Google Scholar 

  23. Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Diversified texture synthesis with feed-forward networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3920–3928 (2017)

    Google Scholar 

  24. Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Universal style transfer via feature transforms. In: Advances in Neural Information Processing Systems 30 (2017)

    Google Scholar 

  25. Li, Z., Meng, Q., Liu, Q., Zhang, Z., Zhang, L., Zhang, S.: Face shape transfer via semantic warping. Available at SSRN 3997660

    Google Scholar 

  26. Lin, T., et al.: Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5141–5150 (2021)

    Google Scholar 

  27. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  28. Liu, S., et al.: AdaAttN: revisit attention mechanism in arbitrary neural style transfer. In: IEEE/CVF International Conference on Computer Vision, pp. 6649–6658 (2021)

    Google Scholar 

  29. Park, D.Y., Lee, K.H.: Arbitrary style transfer with style-attentional networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5880–5888 (2019)

    Google Scholar 

  30. Pluim, J.P., Maintz, J.A., Viergever, M.A.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003)

    Article  Google Scholar 

  31. Risser, E., Wilmot, P., Barnes, C.: Stable and controllable neural texture synthesis and style transfer using histogram losses. arXiv preprint arXiv:1701.08893 (2017)

  32. Sheng, L., Lin, Z., Shao, J., Wang, X.: Avatar-Net: multi-scale zero-shot style transfer by feature decoration. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8242–8250 (2018)

    Google Scholar 

  33. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  34. Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.: Texture networks: feed-forward synthesis of textures and stylized images. arXiv preprint arXiv:1603.03417 (2016)

  35. Wang, X., Oxholm, G., Zhang, D., Wang, Y.F.: Multimodal transfer: a hierarchical deep convolutional neural network for fast artistic style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5239–5247 (2017)

    Google Scholar 

  36. Wen, L., Gao, C., Zou, C.: CAP-VSTNet: content affinity preserved versatile style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18300–18309 (2023)

    Google Scholar 

  37. Yoo, J., Uh, Y., Chun, S., Kang, B., Ha, J.W.: Photorealistic style transfer via wavelet transforms. In: IEEE/CVF International Conference on Computer Vision, pp. 9036–9045 (2019)

    Google Scholar 

  38. Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: self-supervised learning via redundancy reduction. In: International Conference on Machine Learning, pp. 12310–12320 (2021)

    Google Scholar 

  39. Zhang, Y., Li, M., Li, R., Jia, K., Zhang, L.: Exact feature distribution matching for arbitrary style transfer and domain generalization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8035–8045 (2022)

    Google Scholar 

Download references

Acknowledgements

This research is supported by the Natural Science Foundation of Hunan Province, China (2022GK5002,2024JK2015,2024JJ5440), the National Natural Science Foundation of China (61602527), the Special Foundation for Distinguished Young Scientists of Changsha (kq2209003), the Changsha Municipal Natural Science Foundation (kq2202109), the National Foreign Expert Project (G2023041039L), the 111 Project (D23006), and the High Performance Computing Center of Central South University.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Jian Zhang or Hui Fang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, X. et al. (2025). Towards Compact Reversible Image Representations for Neural Style Transfer. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15124. Springer, Cham. https://doi.org/10.1007/978-3-031-72848-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72848-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72847-1

  • Online ISBN: 978-3-031-72848-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics