Skip to main content

Arbitrary Scale Texture Synthesis with Feature Map Swapping

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14868))

Included in the following conference series:

Abstract

Texture synthesis is a technique widely used in computer vision. Existing learning-based methods typically use fixed network structure, and they can only generate images that are the same size or integer multiples of the input sample. In this paper, we propose a swapping-aware texture synthesis method based on feature mapping using a deep generative model. To optimize the loss function, we conduct a dedicated exchange algorithm that operates directly in the feature map space. The texture matching is optimized by matching the feature map between the original image and the generated image. The model can generate texture images of any size based on input samples. The generated results can be extended from the inside of the image to the surrounding, resulting in an image larger than the original input size. The experimental results show that the proposed method effectively preserves more high-frequency details while maintaining the consistency of the generated content and texture, and obtains more realistic synthesized results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Wei, L.-Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Tecniques. SIGGRAPH ’00, pp. 479–488. ACM Press/Addison Wesley Publishing Co., New York, NY, USA (2000)

    Google Scholar 

  2. Ashikhmin,M.: Synthesizing natural textures. In: Symposium on Interactive 3d Graphics (2001)

    Google Scholar 

  3. Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005)

    Article  Google Scholar 

  4. Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceeings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’01, pp. 341–346. ACM, New York, NY, USA (2001)

    Google Scholar 

  5. Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 262–270. Springer, Heidelberg (2015)

    Google Scholar 

  6. Wilmot, P., Risser, E., Barnes, C.: Stable and controllable neural texture synthesis and style transfer using histogram losses. CoRR abs/1701.08893 (2017)

    Google Scholar 

  7. Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.: Texture Networks: feed-forward synthesis of textures and stylized images. arXiv e-prints (2016)

    Google Scholar 

  8. Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.-H.: Diversified texture synthesis with feed-forward networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  9. Zhou, Y., Zhu, Z., Bai, X., Lischinski, D., Cohen-Or, D., Huang, H.: Nonstationary texture synthesis by adversarial expansion. CoRR abs/1805.04487 (2018)

    Google Scholar 

  10. Yu, N., Barnes, C., Shechtman, E., Amirghodsi, S., Lukac, M.: Texture mixer: A network for controllable synthesis and interpolation of texture. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

    Google Scholar 

  11. Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc., RedHook (2014)

    Google Scholar 

  12. Sabini, M., Rusak, G.: Painting outside the box: image outpainting with GANs. CoRR abs/1808.08483 (2018)

    Google Scholar 

  13. Darabi, S., Shechtman, E., Barnes, C., Dan, B.G., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 1–10 (2012)

    Article  Google Scholar 

  14. Zhou, Y., Chen, K., Xiao, R., Huang, H.: Neural texture synthesis with guided correspondence. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18095–18104 (2023)

    Google Scholar 

  15. Ntavelis, E., Shahbazi, M., Kastanis, I., Timofte, R., Danelljan, M., Van Gool, L.: Arbitrary-scale image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11533–11542 (2022)

    Google Scholar 

  16. Chen, T.Q., Schmidt, M.: Fast patch-based style transfer of arbitrary style. CoRR abs/1612.04337 (2016)

    Google Scholar 

  17. Li, C., Wand, M.: Combining markov random fields and convolutional neural networks for image synthesis. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  18. Song, Y., et al.: Contextual-based image inpainting: infer, match, and translate. In: The European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  19. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

    Google Scholar 

  20. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

    Google Scholar 

  21. Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107–110714 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qihang Wang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sun, D., Lin, Y., Shen, S., Zeng, Z., Zhang, S., Wang, Q. (2024). Arbitrary Scale Texture Synthesis with Feature Map Swapping. In: Huang, DS., Zhang, C., Zhang, Q. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14868. Springer, Singapore. https://doi.org/10.1007/978-981-97-5600-1_28

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5600-1_28

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5599-8

  • Online ISBN: 978-981-97-5600-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics