skip to main content
10.1145/3599589.3599605acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmipConference Proceedingsconference-collections
research-article

Arbitrary Style Transfer with Multiple Self-Attention

Published: 11 August 2023 Publication History

Abstract

Style transfer aims to transfer the style information of a given style image to the other images, but most existing methods cannot transfer the texture details in style images well while maintaining the content structure. This paper proposes a novel arbitrary style transfer network that achieves arbitrary style transfer with more local style details through the cross-attention mechanism in visual transforms. The network uses a pre-trained VGG network to extract content and style features. The self-attention-based content and style enhancement module is utilized to enhance content and style feature representation. The transformer-based style cross-attention module is utilized to learn the relationship between content features and style features to transfer appropriate styles at each position of the content feature map and achieve style transfer with local details. Extensive experiments show that the proposed arbitrary style transfer network can generate high-quality stylized images with better visual quality.

References

[1]
Gatys L A, Ecker A S, Bethge M. A neural algorithm of artistic style[J]. arXiv preprint arXiv:1508.06576, 2015.
[2]
Gatys L A, Ecker A S, Bethge M. Image style transfer using convolutional neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2414-2423.
[3]
Huang X, Belongie S. Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of the IEEE international conference on computer vision. 2017: 1501-1510.
[4]
Kitov V, Kozlovtsev K, Mishustina M. Depth-aware arbitrary style transfer using instance normalization[J]. arXiv preprint arXiv:1906.01123, 2019.
[5]
Park D Y, Lee K H. Arbitrary style transfer with style-attentional networks[C]//proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 5880-5888.
[6]
Deng Y, Tang F, Dong W, Arbitrary style transfer via multi-adaptation network[C]//Proceedings of the 28th ACM international conference on multimedia. 2020: 2719-2727.
[7]
Liu S, Lin T, He D, Adaattn: Revisit attention mechanism in arbitrary neural style transfer[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 6649-6658.
[8]
Yao Y, Ren J, Xie X, Attention-aware multi-stroke style transfer[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 1467-1475.
[9]
Deng Y, Tang F, Pan X, StyTr^ 2: Unbiased Image Style Transfer with Transformers[J]. arXiv preprint arXiv:2105.14576, 2021.
[10]
Wang J, Yang H, Fu J, Fine-Grained Image Style Transfer with Visual Transformers[C]//Proceedings of the Asian Conference on Computer Vision. 2022: 841-857.
[11]
Li C, Wand M. Combining markov random fields and convolutional neural networks for image synthesis[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 2479-2486.
[12]
Li Y, Fang C, Yang J, Universal style transfer via feature transforms[J]. Advances in neural information processing systems, 2017, 30.
[13]
Vaswani A, Shazeer N, Parmar N, Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.
[14]
Fu J, Liu J, Tian H, Dual attention network for scene segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 3146-3154.
[15]
Dumoulin V, Shlens J, Kudlur M. A learned representation for artistic style[J]. arXiv preprint arXiv:1610.07629, 2016.
[16]
Lin T Y, Maire M, Belongie S, Microsoft coco: Common objects in context[C]//Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer International Publishing, 2014: 740-755.
[17]
Phillips F, Mackintosh B. Wiki Art Gallery, Inc.: A case for critical thinking[J]. Issues in Accounting Education, 2011, 26(3): 593-608.

Index Terms

  1. Arbitrary Style Transfer with Multiple Self-Attention

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICMIP '23: Proceedings of the 2023 8th International Conference on Multimedia and Image Processing
    April 2023
    131 pages
    ISBN:9781450399586
    DOI:10.1145/3599589
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 August 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. arbitrary style transfer
    2. cross-attention
    3. self-attention
    4. visual transformer

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICMIP 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 73
      Total Downloads
    • Downloads (Last 12 months)27
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media