Haotian Wang
ABSTRACT
In this paper, we present a user-guided digital artwork colorization approach. We propose a novel network structure for this task. Our network takes grayscale artworks along with sparse user-specified color points as input and outputs color artworks using a convolutional neural network (CNN). We train our network on ten thousand digital paintings collected from the internet with simulated user inputs. The experiment result shows that our network outperforms other networks in the task of interactive digital artwork colorization and can produce artistic and convincing color artworks.
- Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM SIGGRAPH 2004 Papers on - SIGGRAPH 04(2004).Google Scholar
- Huang, Y. C., Tung, Y. S., Chen, J. C., Wang, S. W., and Wu, J. L. 2005 "An Adaptive Edge Detection Based Colorization Algorithm and Its Applications." Proceedings of the 13th Annual ACM International Conference on Multimedia - MULTIMEDIA 05, 2005Google Scholar
- Qu, Y., Wong, T. T., and Heng, P. A. 2006. Manga colorization. ACM SIGGRAPH 2006 Papers on - SIGGRAPH 06(2006).Google Scholar
- Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y. Q., and Shum, H. Y. 2007. Natural Image Colorization. Proceedings of the Eurographics Symposium on Rendering Techniques, Grenoble, France, 2007. Eurographics Association.Google Scholar
- Morimoto, Y., Taguchi, Y., and Naemura, T. 2009. Automatic colorization of grayscale images using multiple images on the web. SIGGRAPH 09: Posters on - SIGGRAPH 09(2009).Google Scholar
- Gupta, R. K., Chia, A. Y. S., Rajan, D., Ng, E. S., and Zhiyong, H. 2012. Image colorization using similar images. Proceedings of the 20th ACM international conference on Multimedia - MM 12(2012).Google Scholar
- Zhang, R., Isola, P., and Efros, A. A. 2016. Colorful Image Colorization. Computer Vision - ECCV 2016 Lecture Notes in Computer Science(2016), 649--666.Google Scholar
- Iizuka, S., Simo-Serra, E., and Ishikawa, H. 2016. Let there be color!. ACM Transactions on Graphics, 35(4), 1--11.Google ScholarDigital Library
- Charpiat, G., Hofmann, M., and Schölkopf, B. 2008. Automatic Image Colorization Via Multimodal Predictions. Lecture Notes in Computer Science Computer Vision - ECCV 2008(2008), 126--139.Google ScholarDigital Library
- Zhang, R., Zhu, J. Y., Isola, P., Geng, X., Lin, A. S., Yu, T., and Efros, A. A. 2017. Real-time user-guided image colorization with learned deep priors. ACM Transactions on Graphics36, 4 (2017), 1--11.Google Scholar
- Yonetsuji, T. 2017. Paintschainer. https://paintschainer.preferred.tech/index_en.html.Google Scholar
- Zhang, L., Li, C., Wong, T. T., Ji, Y., and Liu, C. 2018. Two-stage sketch colorization. ACM Transactions on Graphics37, 6 (April 2018), 1--14.Google Scholar
- Long, J., Shelhamer, E., and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2015).Google Scholar
- Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
- He, K., Zhang, X., Ren, S., and Sun, J. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016).Google Scholar
- Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. Thirty-First AAAI Conference on Artificial Intelligence.Google ScholarDigital Library
- Ronneberger, O., Fischer, P., and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention (pp. 234--241). Springer, Cham.Google Scholar
- Lin, G., Shen, C., Van Den Hengel, A., and Reid, I. 2016. Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016).Google Scholar
- Yu, F., and Koltun, V. 2015. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122.Google Scholar
- Francois Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2017).Google Scholar
- Glorot, X., Bordes, A., and Bengio, Y. 2011. Deep sparse rectifier neural networks. Proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 315--323).Google Scholar
- Maas, A. L., Hannun, A. Y., and Ng, A. Y. 2013. Rectifier nonlinearities improve neural network acoustic models. Proc. icml (Vol. 30, No. 1, p. 3).Google Scholar
- Ulyanov, D., Vedaldi, A., and Lempitsky, V. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022.Google Scholar
Index Terms
- User Guided Digital Artwork Colorization
Recommendations
Real-time user-guided image colorization with learned deep priors
We propose a deep learning approach for user-guided image colorization. The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a Convolutional Neural Network (CNN). Rather than using hand-defined ...
A distance transformation based algorithm for digital image colorization
VIIP '07: The Seventh IASTED International Conference on Visualization, Imaging and Image ProcessingColorization is a term used to describe a computerized process for adding color to black and white pictures, movies or TV programs by replacing a scalar value stored at each pixel of the gray scale image by a vector in a three dimensional color space ...
Application of the Extended Distance Transformation in digital image colorization
In this paper we present a novel colorization scheme that takes advantage of the modified morphological distance transform to propagate the color, scribbled by a user on the grayscale image. First, based on the scribbled image, the topological distance ...
Comments