Abstract
The previous image optimization methods cannot complete the automatic refinement of semi-realistic paintings. Aiming at improving the efficiency of refinement manually, we propose an automatic refinement method for semi-realistic figure paintings guided by the line art. In order to enable the framework to adjust the draft color in the refinement process like a real painter, we design a color correction module, which automatically fixes the inappropriate color in the draft. In order to reduce artifacts and generate high-quality results, we use the line art to guide the refinement. We further devise a line art optimization module in the framework to ensure generation of high quality results by improving the quality of the line art. The experimental results and user surveys demonstrate the effectiveness of the proposed method.
This work was partly supported by the Natural Science Foundation of China under grant no. 62072328.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
AL-Mekhlafi, H., Liu, S.: Single image super-resolution: a comprehensive review and recent insight. Frontiers Comput. Sci. 17, 1–27 (2023)
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)
Fan, K., Liu, S., Lu, W.: Learning to draw semi-realistic paintings from the manga line drawings and flat shadow. In: Magnenat-Thalmann, N., et al. (eds.) Proceedings of CGI, vol. 13443, pp. 305–317. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-23473-6_24
Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1712–1722 (2019)
Hu, J., Wang, C., Zhang, Y., Liu, L., Yin, Y., Zimmermann, R.: Parallel edge-image learning for image inpainting. In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2022)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. IEEE Trans. Pattern Anal. Mach. Intell. 43(12), 4217–4228 (2021)
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1646–1654 (2016)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114 (2017)
Li, H., Sheng, B., Li, P., Ali, R.A., Chen, C.L.P.: Globally and locally semantic colorization via exemplar-based broad-GAN. IEEE Trans. Image Process. 30, 8526–8539 (2021)
Nazeri, K., Ng, E., Joseph, T., Qureshi, F., Ebrahimi, M.: EdgeConnect: structure guided image inpainting using edge prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop, pp. 3265–3274 (2019)
Qi, Y., Zhang, A., Wang, H., Li, X.: An efficient FCM-based method for image refinement segmentation. Vis. Comput. 38, 2499–2514 (2022)
Tai, Y., Yang, J., Liu, X., Xu, C.: MemNet: a persistent memory network for image restoration. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 4549–4557 (2017)
Wang, D., Hu, G., Lyu, C.: FRNet: an end-to-end feature refinement neural network for medical image segmentation. Vis. Comput. 37, 1101–1112 (2021)
Xu, S., Liu, D., Xiong, Z.: E2I: generative inpainting from edge to image. IEEE Trans. Circuits Syst. Video Technol. 31(4), 1308–1322 (2021)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Zhang, L.: Sketchkeras (2017). https://github.com/lllyasviel/sket-chKeras. https://github.com/lllyasviel/sketchKeras
Zou, Z., Shi, T., Qiu, S., Yuan, Y., Shi, Z.: Stylized neural painting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15684–15693 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fan, K., Liu, S. (2024). SemiRefiner: Learning to Refine Semi-realistic Paintings. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14496. Springer, Cham. https://doi.org/10.1007/978-3-031-50072-5_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-50072-5_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-50071-8
Online ISBN: 978-3-031-50072-5
eBook Packages: Computer ScienceComputer Science (R0)