Abstract
With the advance in deep learning techniques, increasing attention has been paid to studying the creation capacity of artificial intelligence (AI) models. AI-based painting image generation not only streamlines tedious tasks of professional users but also enlightens a broad range of casual users for artistic creation. This paper focuses on customizing painting images with limited inputs, i.e., a reference painting image as the exemplar and a conditional input that users could easily specify to reflect their ideas. We show the challenges of directly applying existing solutions to the problem of painting customizing and provide a solution to address them. We also develop the EasyPainter, a user-friendly interface system to better understand AI’s creative and exploratory potential in customizing painting images. Our demo video is available here.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bansal, A., Sheikh, Y., Ramanan, D.: Shapes and context: in-the-wild image synthesis and manipulation. In: Proceedings of Computer Vision and Pattern Recognition, pp. 2317–2326 (2019)
Cetinic, E., She, J.: Understanding and creating art with AI: review and outlook. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 18(2), 1–22 (2022)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems 27 (2014)
He, M., Chen, D., Liao, J., Sander, P.V., Yuan, L.: Deep exemplar-based colorization. ACM Trans. Graph. (TOG) 37(4), 1–16 (2018)
Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: Proceedings of European Conference on Computer Vision, pp. 172–189 (2018)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
kazuto1011: Deeplab with pytorch. https://github.com/kazuto1011/deeplab-pytorch
Kim, H., Jhoo, H.Y., Park, E., Yoo, S.: Tag2Pix: line art colorization using text tag with secat and changing loss. In: Proceedings of Computer Vision and Pattern Recognition, pp. 9056–9065 (2019)
Kim, J., Kim, M., Kang, H., Lee, K.H.: U-GAT-IT: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation. In: International Conference on Learning Representations (2019)
Ma, L., Jia, X., Georgoulis, S., Tuytelaars, T., Van Gool, L.: Exemplar guided unsupervised image-to-image translation with semantic consistency. In: International Conference on Learning Representations (2018)
Qi, X., Chen, Q., Jia, J., Koltun, V.: Semi-parametric image synthesis. In: Proceedings of Computer Vision and Pattern Recognition, pp. 8808–8816 (2018)
Rozière, B., Riviere, M., Teytaud, O., Rapin, J., LeCun, Y., Couprie, C.: Inspirational adversarial image generation. IEEE Trans. Image Process. 30, 4036–4045 (2021)
Villani, C.: Optimal transport: old and new, vol. 338. Springer (2009). 10.1007/978-3-540-71050-9
Wang, M., et al.: Example-guided style-consistent image synthesis from semantic labeling. In: Proceedings of Computer Vision and Pattern Recognition, pp. 1495–1504 (2019)
Zhan, F., et al.: Unbalanced feature transport for exemplar-based image translation. In: Proceedings of Computer Vision and Pattern Recognition, pp. 15028–15038 (2021)
Zhang, B., et al.: Deep exemplar-based video colorization. In: Proceedings of Computer Vision and Pattern Recognition, pp. 8052–8061 (2019)
Zhang, P., Zhang, B., Chen, D., Yuan, L., Wen, F.: Cross-domain correspondence learning for exemplar-based image translation. In: Proceedings of Computer Vision and Pattern Recognition, pp. 5143–5153 (2020)
Zhang, X., Zhang, X., Xiao, Z.: Deep photographic style transfer guided by semantic correspondence. Multimedia Tools Appl. 78(24), 34649–34672 (2019). https://doi.org/10.1007/s11042-019-08099-7
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: Proceedings of Computer Vision and Pattern Recognition, pp. 633–641 (2017)
Zhou, X., et al.: CoCosNet v2: full-resolution correspondence learning for image translation. In: Proceedings of Computer Vision and Pattern Recognition, pp. 11465–11475 (2021)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of International Conference on Computer Vision, pp. 2223–2232 (2017)
Acknowledgments
This work is supported by Natural Science Foundation of China (No. 61925603).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Y., Zheng, Q., Pan, G. (2022). EasyPainter: Customizing Your Own Paintings. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13606. Springer, Cham. https://doi.org/10.1007/978-3-031-20503-3_55
Download citation
DOI: https://doi.org/10.1007/978-3-031-20503-3_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20502-6
Online ISBN: 978-3-031-20503-3
eBook Packages: Computer ScienceComputer Science (R0)