Abstract
Arbitrary neural style transfer aims to stylise a content image by referencing a provided style image. Despite various efforts to achieve both content preservation and style transferability, learning effective representations for this task remains challenging since the redundancy of content and style features leads to unpleasant image artefacts. In this paper, we learn compact neural representations for style transfer motivated from an information theoretical perspective. In particular, we enforce compressive representations across sequential modules of a reversible flow network in order to reduce feature redundancy without losing content preservation capability. We use a Barlow twins loss to reduce channel dependency and thus to provide better content expressiveness, and optimise the Jensen-Shannon divergence of style representations between reference and target images to avoid under- and over-stylisation. We comprehensively demonstrate the effectiveness of our proposed method in comparison to other state-of-the-art style transfer approaches.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alemi, A.A., Fischer, I., Dillon, J.V., Murphy, K.: Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 (2016)
An, J., Huang, S., Song, Y., Dou, D., Liu, W., Luo, J.: ArtFlow: unbiased image style transfer via reversible neural flows. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 862–871 (2021)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223 (2017)
Barlow, H.: Redundancy reduction revisited. Netw. Comput. Neural Syst. 12(3), 241 (2001)
Barlow, H.B.: Possible principles underlying the transformation of sensory messages. Sens. Commun. 1(01), 217–233 (1961)
Chen, D., Yuan, L., Liao, J., Yu, N., Hua, G.: StyleBank: an explicit representation for neural image style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1897–1906 (2017)
Chen, H., et al.: Artistic style transfer with internal-external learning and contrastive learning. In: Advances in Neural Information Processing Systems, vol. 34, pp. 26561–26573 (2021)
Cheng, J., Jaiswal, A., Wu, Y., Natarajan, P., Natarajan, P.: Style-aware normalized loss for improving arbitrary style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 134–143 (2021)
Deng, Y., et al.: StyTr2: image style transfer with transformers. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11326–11336 (2022)
Dinh, L., Krueger, D., Bengio, Y.: NICE: non-linear independent components estimation. arXiv preprint arXiv:1410.8516 (2014)
Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. arXiv preprint arXiv:1610.07629 (2016)
Fisher, R.A.: On the mathematical foundations of theoretical statistics. Philos. Trans. Roy. Soc. Lond. Ser. A 222(594–604), 309–368 (1922)
Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Advances in Neural Information Processing Systems 28 (2015)
Gatys, L.A., Bethge, M., Hertzmann, A., Shechtman, E.: Preserving color in neural artistic style transfer. arXiv preprint arXiv:1606.05897 (2016)
Gomez, A.N., Ren, M., Urtasun, R., Grosse, R.B.: The reversible residual network: backpropagation without storing activations. In: Advances in Neural Information Processing Systems 30 (2017)
Huang, S., et al.: Parameter-free style projection for arbitrary image style transfer. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2070–2074 (2022)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: IEEE/CVF International Conference on Computer Vision, pp. 1501–1510 (2017)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Karayev, S., et al.: Recognizing image style. arXiv preprint arXiv:1311.3715 (2013)
Kingma, D.P., Dhariwal, P.: Glow: generative flow with invertible \(1\times 1\) convolutions. In: Advances in Neural Information Processing Systems 31 (2018)
Kitaev, N., Kaiser, Ł., Levskaya, A.: Reformer: the efficient transformer. arXiv preprint arXiv:2001.04451 (2020)
Lee, J., Kim, H., Shim, J., Hwang, E.: Cartoon-Flow: a flow-based generative adversarial network for arbitrary-style photo cartoonization. In: ACM International Conference on Multimedia, pp. 1241–1251 (2022)
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Diversified texture synthesis with feed-forward networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3920–3928 (2017)
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.H.: Universal style transfer via feature transforms. In: Advances in Neural Information Processing Systems 30 (2017)
Li, Z., Meng, Q., Liu, Q., Zhang, Z., Zhang, L., Zhang, S.: Face shape transfer via semantic warping. Available at SSRN 3997660
Lin, T., et al.: Drafting and revision: Laplacian pyramid network for fast high-quality artistic style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5141–5150 (2021)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, S., et al.: AdaAttN: revisit attention mechanism in arbitrary neural style transfer. In: IEEE/CVF International Conference on Computer Vision, pp. 6649–6658 (2021)
Park, D.Y., Lee, K.H.: Arbitrary style transfer with style-attentional networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5880–5888 (2019)
Pluim, J.P., Maintz, J.A., Viergever, M.A.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22(8), 986–1004 (2003)
Risser, E., Wilmot, P., Barnes, C.: Stable and controllable neural texture synthesis and style transfer using histogram losses. arXiv preprint arXiv:1701.08893 (2017)
Sheng, L., Lin, Z., Shao, J., Wang, X.: Avatar-Net: multi-scale zero-shot style transfer by feature decoration. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8242–8250 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.: Texture networks: feed-forward synthesis of textures and stylized images. arXiv preprint arXiv:1603.03417 (2016)
Wang, X., Oxholm, G., Zhang, D., Wang, Y.F.: Multimodal transfer: a hierarchical deep convolutional neural network for fast artistic style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5239–5247 (2017)
Wen, L., Gao, C., Zou, C.: CAP-VSTNet: content affinity preserved versatile style transfer. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18300–18309 (2023)
Yoo, J., Uh, Y., Chun, S., Kang, B., Ha, J.W.: Photorealistic style transfer via wavelet transforms. In: IEEE/CVF International Conference on Computer Vision, pp. 9036–9045 (2019)
Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: self-supervised learning via redundancy reduction. In: International Conference on Machine Learning, pp. 12310–12320 (2021)
Zhang, Y., Li, M., Li, R., Jia, K., Zhang, L.: Exact feature distribution matching for arbitrary style transfer and domain generalization. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8035–8045 (2022)
Acknowledgements
This research is supported by the Natural Science Foundation of Hunan Province, China (2022GK5002,2024JK2015,2024JJ5440), the National Natural Science Foundation of China (61602527), the Special Foundation for Distinguished Young Scientists of Changsha (kq2209003), the Changsha Municipal Natural Science Foundation (kq2202109), the National Foreign Expert Project (G2023041039L), the 111 Project (D23006), and the High Performance Computing Center of Central South University.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X. et al. (2025). Towards Compact Reversible Image Representations for Neural Style Transfer. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15124. Springer, Cham. https://doi.org/10.1007/978-3-031-72848-8_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-72848-8_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72847-1
Online ISBN: 978-3-031-72848-8
eBook Packages: Computer ScienceComputer Science (R0)