Abstract
Recently, deep learning technology has made a breakthrough in computer vision, image processing, and other fields. Some researchers suggested neural style transfer method using a convolutional neural network (CNN). They established the correlation of features in a neural network to be treated as the style. However, their performance is unacceptable for Chinese landscape painting. According to the property of the Chinese landscape painting, this paper proposes a novel two stage style transfer method that imitates multiple styles of Chinese landscape painting based on deep learning. The structure of an input photo was simplified in the first stage. Then, the result of the first stage was transferred into the final stylized image in second stage. A generative adversarial network (GAN) is applied to train in each stage. Besides, a novel loss function was proposed to keep the shape of the content image. Finally, our method haves successfully imitated several styles of Chinese Landscape ink painting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, D., Yuan, L., Liao, J., Yu, N., Hua, G.: Stylebank: an explicit representation for neural image style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1897–1906 (2017)
Chen, T.Q, Schmidt, M.: Fast patchÂbased style transfer of arbitrary style. arXiv preprint arXiv:1612.04337 (2016)
Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. arXiv preprint arXiv:1610.07629 (2016)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, X., Belongie, S.: Arbitrary style transfer in realÂtime with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)
Huang, X., Liu, M.-Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11207, pp. 179–196. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01219-9_11
Isola, P., Zhu, J.ÂY., Zhou, T., Efros, A.A.: ImageÂtoÂimage translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Jing, Y., et al.: Stroke controllable fast style transfer with adaptive receptive fields. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 244–260. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_15
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Kim, T., Cha, M., Kim, H., Lee, J.K., Kim, J.: Learning to discover crossÂdomain relations with generative adversarial networks. In: Proceedings of the 34th International Conference on Machine Learning, vol. 70, pp. 1857–1865. JMLR.org (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: GradientÂbased learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278–2324 (1998)
Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., Yang, M.ÂH.: Universal style transfer via feature transforms. In: Advances in Neural Information Processing Systems, pp. 386–396 (2017)
Liu, M.ÂY., Breuel, T., Kautz, J.: Unsupervised imageÂtoÂimage translation networks. In: Advances in Neural Information Processing Systems, pp. 700–708 (2017)
Liu, X.ÂC., Cheng, M.ÂM., Lai, Y.ÂK., Rosin, P. L.: DepthÂaware neural style transfer. In: Proceedings of the Symposium on NonÂPhotorealistic Animation and Rendering, pp. 1–10 (2017)
Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4990–4998 (2017)
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
Rosenfeld, A., Pfaltz, J.L.: Sequential operations in digital picture processing. J. ACM (JACM) 13(4), 471–494 (1966)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for largeÂscale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Taigman, Y., Polyak, A., Wolf, L.: Unsupervised crossÂdomain image generation. arXiv preprint arXiv:1611.02200 (2016)
Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: feedÂforward synthesis of textures and stylized images. In: ICML, Vol. 1, pp. 4 (2016)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Improved texture networks: maximizing quality and diversity in feedÂforward stylization and texture synthesis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6924–6932 (2017)
Wang, T.ÂC., Liu, M.ÂY., Zhu, J.ÂY., Tao, A., Kautz, J., Catanzaro, B.: HighÂresolution image synthesis and semantic manipulation with conditional GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8798–8807 ( 2018)
Xie, S., Tu, Z.: HolisticallyÂnested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1395–1403 (2015)
Acknowledgement
The authors would like to thank the Ministry of Science and Technology of the Republic of China, Taiwan, for financially supporting this research under Contract No. MOST 110-2221-E-119-002.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Style transfer results of Chinese landscape painting.
Styles transfer results of Chinese landscape painting.
Styles transfer results of Chinese landscape painting.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Way, DL., Lo, CH., Wei, YH., Shih, ZC. (2023). A Structure-Aware Deep Learning Network for the Transfer of Chinese Landscape Painting Style. In: Rauterberg, M. (eds) Culture and Computing. HCII 2023. Lecture Notes in Computer Science, vol 14035. Springer, Cham. https://doi.org/10.1007/978-3-031-34732-0_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-34732-0_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34731-3
Online ISBN: 978-3-031-34732-0
eBook Packages: Computer ScienceComputer Science (R0)