User Guided Digital Artwork Colorization
Authors: 
Haotian Wang, Gang Liu, Xin ChenAuthors Info & Claims
Haotian Wang, Gang Liu, Xin ChenAuthors Info & ClaimsPages 93 - 97
Abstract
In this paper, we present a user-guided digital artwork colorization approach. We propose a novel network structure for this task. Our network takes grayscale artworks along with sparse user-specified color points as input and outputs color artworks using a convolutional neural network (CNN). We train our network on ten thousand digital paintings collected from the internet with simulated user inputs. The experiment result shows that our network outperforms other networks in the task of interactive digital artwork colorization and can produce artistic and convincing color artworks.
References
[1]
Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM SIGGRAPH 2004 Papers on - SIGGRAPH 04(2004).
[2]
Huang, Y. C., Tung, Y. S., Chen, J. C., Wang, S. W., and Wu, J. L. 2005 "An Adaptive Edge Detection Based Colorization Algorithm and Its Applications." Proceedings of the 13th Annual ACM International Conference on Multimedia - MULTIMEDIA 05, 2005
[3]
Qu, Y., Wong, T. T., and Heng, P. A. 2006. Manga colorization. ACM SIGGRAPH 2006 Papers on - SIGGRAPH 06(2006).
[4]
Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y. Q., and Shum, H. Y. 2007. Natural Image Colorization. Proceedings of the Eurographics Symposium on Rendering Techniques, Grenoble, France, 2007. Eurographics Association.
[5]
Morimoto, Y., Taguchi, Y., and Naemura, T. 2009. Automatic colorization of grayscale images using multiple images on the web. SIGGRAPH 09: Posters on - SIGGRAPH 09(2009).
[6]
Gupta, R. K., Chia, A. Y. S., Rajan, D., Ng, E. S., and Zhiyong, H. 2012. Image colorization using similar images. Proceedings of the 20th ACM international conference on Multimedia - MM 12(2012).
[7]
Zhang, R., Isola, P., and Efros, A. A. 2016. Colorful Image Colorization. Computer Vision - ECCV 2016 Lecture Notes in Computer Science(2016), 649--666.
[8]
Iizuka, S., Simo-Serra, E., and Ishikawa, H. 2016. Let there be color!. ACM Transactions on Graphics, 35(4), 1--11.
[9]
Charpiat, G., Hofmann, M., and Schölkopf, B. 2008. Automatic Image Colorization Via Multimodal Predictions. Lecture Notes in Computer Science Computer Vision - ECCV 2008(2008), 126--139.
[10]
Zhang, R., Zhu, J. Y., Isola, P., Geng, X., Lin, A. S., Yu, T., and Efros, A. A. 2017. Real-time user-guided image colorization with learned deep priors. ACM Transactions on Graphics36, 4 (2017), 1--11.
[11]
Yonetsuji, T. 2017. Paintschainer. https://paintschainer.preferred.tech/index_en.html.
[12]
Zhang, L., Li, C., Wong, T. T., Ji, Y., and Liu, C. 2018. Two-stage sketch colorization. ACM Transactions on Graphics37, 6 (April 2018), 1--14.
[13]
Long, J., Shelhamer, E., and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2015).
[14]
Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[15]
He, K., Zhang, X., Ren, S., and Sun, J. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016).
[16]
Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. Thirty-First AAAI Conference on Artificial Intelligence.
[17]
Ronneberger, O., Fischer, P., and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention (pp. 234--241). Springer, Cham.
[18]
Lin, G., Shen, C., Van Den Hengel, A., and Reid, I. 2016. Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016).
[19]
Yu, F., and Koltun, V. 2015. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122.
[20]
Francois Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2017).
[21]
Glorot, X., Bordes, A., and Bengio, Y. 2011. Deep sparse rectifier neural networks. Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 315--323).
[22]
Maas, A. L., Hannun, A. Y., and Ng, A. Y. 2013. Rectifier nonlinearities improve neural network acoustic models. Proc. icml (Vol. 30, No. 1, p. 3).
[23]
Ulyanov, D., Vedaldi, A., and Lempitsky, V. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022.
Index Terms
- User Guided Digital Artwork Colorization
Recommendations
User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks
MM '18: Proceedings of the 26th ACM international conference on MultimediaScribble colors based line art colorization is a challenging computer vision problem since neither greyscale values nor semantic information is presented in line arts, and the lack of authentic illustration-line art training pairs also increases ...
Real-time user-guided image colorization with learned deep priors
We propose a deep learning approach for user-guided image colorization. The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a Convolutional Neural Network (CNN). Rather than using hand-defined ...
A distance transformation based algorithm for digital image colorization
VIIP '07: The Seventh IASTED International Conference on Visualization, Imaging and Image ProcessingColorization is a term used to describe a computerized process for adding color to black and white pictures, movies or TV programs by replacing a scalar value stored at each pixel of the gray scale image by a vector in a three dimensional color space ...
Comments
Information & Contributors
Information
Published In
December 2019
370 pages
ISBN:9781450376273
DOI:10.1145/3374587
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Shenzhen University: Shenzhen University
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 March 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CSAI2019
CSAI2019: 2019 3rd International Conference on Computer Science and Artificial Intelligence
December 6 - 8, 2019
IL, Normal, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 90Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in