Abstract
Previous approaches to the colorization of grayscale images rely on human manual labor and often produce desaturated results that are not likely to be true colorizations. Inspired by Matías Richart’s paper, we proposed an automatic approach based on deep neural networks to color the image in grayscale. We have studied several models, approaches and loss functions to understand the best practices for producing a plausible colorization. By noting that some loss functions work better than others, we used the VGG-16 CNN model based on the classification with the loss of cross-entropy. The experiment shows that our model can produce a plausible colorization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Burn GC (1997) Museum of broadcast communications: the Encyclopedia of television. www.museum.tv/archives/etv/index.html. Accessed Oct 2018
Carpiat G, Hofmann M, Schölkopf B (2008a) Automatic image colorization via multimodal predictions. In: Proceedins of the 10th European conference on computer vision, pp 126–139
Charpiat G, Hofmann M, Schölkopf B (2008b) Automatic image colorization via multimodal predictions. In: Computer vision–ECCV 2008, vol 5304, pp 126–139
Cheng Z, Yang Q, Sheng B (2015) Deep colorization. In: Proceedings of the IEEE international conference on computer vision, pp 415–423
Deshpande A, Rock J, Forsyth D (2015) Learning large-scale automatic image colorization. In: Proceedings of the IEEE international conference on computer vision
Di Blasi G, Recupero D Refrorgiato (2003) Fast colorization of gray images. In: Proceedings of Eurographics Italian chapter, Miliano, Italy
Gupta RK, Chia AY-S, Rajan D, Ng ES, Zhiyong H (2012) Image colorization using similar images. In: Proceedings of the 20th ACM international conference on multimedia. ACM, pp 369–378
Gustav L, Maire M, Shakhnarovich G (2016) Learning representations for automatic colorization. In: European conference on computer vision—ECCV 2016. Springer, pp 577–593
Hertzmann A, Jacobs CE, Oliver N, Curless B, Salesin DH (2001) Image analogies. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques, SIGGRAPH ‘01, pp 327–340
Horiuchi T (2004) Colorization algorithm using probabilistic relaxation. Image Vis Comput 22(3):197–202
Iizuka S, Simo-Serra E, Ishikawa H (2016) Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans Graph 35(4):110
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift, CoRR. arXiv:1502.03167
Levin A, Lischinski D, Weiss Y (2004) Colorization using optimization. In: SIGGRAPH 04, Los Angeles, Califonia. Transactions on graphics (TOG). ACM Press, 23(3):689–694
Nies D, Ma Q, Ma L, Xiao S (2007) Optimization based grayscale image colorization. Pattern Recognit Lett 28(12):1445–1451
Reinhard E, Ashikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graph Appl 21(5):34–41
Ryan D (2016) Automatic colorization. http://tinyclouds.org/colorize. Accessed Oct 2018
Sahay T, Choudhary A (2017) Automatic colorization of videos. University of Massachusetts, Amherst
Simonyan K, Zisserman A (2014a) Very deep convolutional networks for large-scale image recognition, CoRR. arXiv:1409.1556
Simonyan K, Zisserman A (2014b) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Takahama T, Horiuchi T, Kotera H (2005) Improvement on colorization accuracy by partitioning algorithm in cielab color space. In: Advances in multimedia information processing- PCM 2004, 5th Pacific rim conference on multimedia, vol 3332. Springer, pp 794–801
Welsh T, Ashikhmin M, Mueller K (2002) Transferring color to grayscale image. ACM Trans Graph 21(3):277–280
Yatziv L, Sapiro G (2006) Fast image and video colorization using chrominance blending. IEEE Trans Image Process 15:1120–1129
Zhang R, Isola P, Efros AA (2016) Colorful image colorization. In: European conference on computer vision. Springer
Funding
This study was funded by the Emergency Management Project of the National Natural Science Foundation of China (Grant Number 61741412) and the Shanxi Basic Research Project (Grant Number 201801D121143).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Author Jiancheng An declares that he has no conflict of interest. Author Kpeyiton Koffi Gagnon declares that he has no conflict of interest. Author Qingnan Shi declares that he has no conflict of interest.
Ethical approval
This article does not contain any studies with animals performed by any of the authors.
Additional information
Communicated by L. Wang.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
An, J., Kpeyiton, K.G. & Shi, Q. Grayscale images colorization with convolutional neural networks. Soft Comput 24, 4751–4758 (2020). https://doi.org/10.1007/s00500-020-04711-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-04711-3