Abstract
Texture synthesis is a technique widely used in computer vision. Existing learning-based methods typically use fixed network structure, and they can only generate images that are the same size or integer multiples of the input sample. In this paper, we propose a swapping-aware texture synthesis method based on feature mapping using a deep generative model. To optimize the loss function, we conduct a dedicated exchange algorithm that operates directly in the feature map space. The texture matching is optimized by matching the feature map between the original image and the generated image. The model can generate texture images of any size based on input samples. The generated results can be extended from the inside of the image to the surrounding, resulting in an image larger than the original input size. The experimental results show that the proposed method effectively preserves more high-frequency details while maintaining the consistency of the generated content and texture, and obtains more realistic synthesized results.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wei, L.-Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Tecniques. SIGGRAPH ’00, pp. 479–488. ACM Press/Addison Wesley Publishing Co., New York, NY, USA (2000)
Ashikhmin,M.: Synthesizing natural textures. In: Symposium on Interactive 3d Graphics (2001)
Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005)
Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proceeings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’01, pp. 341–346. ACM, New York, NY, USA (2001)
Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 262–270. Springer, Heidelberg (2015)
Wilmot, P., Risser, E., Barnes, C.: Stable and controllable neural texture synthesis and style transfer using histogram losses. CoRR abs/1701.08893 (2017)
Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.: Texture Networks: feed-forward synthesis of textures and stylized images. arXiv e-prints (2016)
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.-H.: Diversified texture synthesis with feed-forward networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Zhou, Y., Zhu, Z., Bai, X., Lischinski, D., Cohen-Or, D., Huang, H.: Nonstationary texture synthesis by adversarial expansion. CoRR abs/1805.04487 (2018)
Yu, N., Barnes, C., Shechtman, E., Amirghodsi, S., Lukac, M.: Texture mixer: A network for controllable synthesis and interpolation of texture. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27, pp. 2672–2680. Curran Associates, Inc., RedHook (2014)
Sabini, M., Rusak, G.: Painting outside the box: image outpainting with GANs. CoRR abs/1808.08483 (2018)
Darabi, S., Shechtman, E., Barnes, C., Dan, B.G., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. 31(4), 1–10 (2012)
Zhou, Y., Chen, K., Xiao, R., Huang, H.: Neural texture synthesis with guided correspondence. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18095–18104 (2023)
Ntavelis, E., Shahbazi, M., Kastanis, I., Timofte, R., Danelljan, M., Van Gool, L.: Arbitrary-scale image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11533–11542 (2022)
Chen, T.Q., Schmidt, M.: Fast patch-based style transfer of arbitrary style. CoRR abs/1612.04337 (2016)
Li, C., Wand, M.: Combining markov random fields and convolutional neural networks for image synthesis. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Song, Y., et al.: Contextual-based image inpainting: infer, match, and translate. In: The European Conference on Computer Vision (ECCV) (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4), 107–110714 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, D., Lin, Y., Shen, S., Zeng, Z., Zhang, S., Wang, Q. (2024). Arbitrary Scale Texture Synthesis with Feature Map Swapping. In: Huang, DS., Zhang, C., Zhang, Q. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14868. Springer, Singapore. https://doi.org/10.1007/978-981-97-5600-1_28
Download citation
DOI: https://doi.org/10.1007/978-981-97-5600-1_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5599-8
Online ISBN: 978-981-97-5600-1
eBook Packages: Computer ScienceComputer Science (R0)