Shamit Lal
ABSTRACT
The paper presents a fully automatic end-to-end trainable system to colorize grayscale images. Colorization is a highly under-constrained problem. In order to produce realistic outputs, the proposed approach takes advantage of the recent advances in deep learning and generative networks. To achieve plausible colorization, the paper investigates conditional Wasserstein Generative Adversarial Networks (WGAN) [3] as a solution to this problem. Additionally, a loss function consisting of two classification loss components apart from the adversarial loss learned by the WGAN is proposed. The first classification loss provides a measure of how much the predicted colored images differ from ground truth. The second classification loss component makes use of ground truth semantic classification labels in order to learn meaningful intermediate features. Finally, WGAN training procedure pushes the predictions to the manifold of natural images. The system is validated using a user study and a semantic interpretability test and achieves results comparable to [1] on Imagenet dataset [10].
- Zhang, R., Phillip, I., and Efros, A. A. 2016. Colorful image colorization. In European Conference on Computer Vision, pp. 649--666. Springer International Publishing, 2016.Google Scholar
- Larsson, G., Maire, M., and Shakhnarovich, G. 2016. Learning representations for automatic colorization. In European Conference on Computer Vision, pp. 577--593. Springer International Publishing, 2016.Google Scholar
- Arjovsky, M., Chintala, S., and Bottou, L. 2017. Wasserstein GAN. arXiv preprint arXiv:1701.07875.Google Scholar
- Mirza, M., and Osindero, S. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784.Google Scholar
- Ronneberger, O., Fischer, P., and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234--241. Springer International Publishing, 2015. Harvard.Google Scholar
- Iizuka, S., Simo-Serra, E., and Ishikawa, H. 2016. Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Transactions on Graphics (TOG) 35(4), p. 110. Google ScholarDigital Library
- Isola, P., Jhu, J., Zhou, T., and Efros, A.A. 2016. Image-to-image translation with conditional adversarial networks. arXiv preprint arXiv:1611.07004.Google Scholar
- Cheng, Z., Yang, Q., and Sheng, B. 2015. Deep colorization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 415--423. Google ScholarDigital Library
- Dahl R. Automatic colorization. 2016. http://tinyclouds.org/colorize.Google Scholar
- Deng, J., Dong, W., Socher, R., Li, L., Li, k., and Li F. 2009. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. pp. 248--255. IEEE, 2009.Google Scholar
- Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. In ACM Transactions on Graphics (ToG), vol. 23(3), pp. 689--694. ACM, 2004. Google ScholarDigital Library
- Simonyan, K., Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.Google Scholar
- Welsh, T., Ashikhmin, M., Mueller, K. 2002. Transferring color to greyscale images. ACM Transactions on Graphics (TOG) 21(3). 2002. pp. 277--280. Google ScholarDigital Library
- Ironi, R., Daniel, C., and Dani L. 2005. Colorization by Example. In Rendering Techniques, pp. 201--210. Google ScholarDigital Library
- Hariharan, B., Arbeláez, P., Girshick, R., Malik, J. 2015. Hypercolumns for object segmentation and fine-grained localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. pp. 447--456.Google ScholarCross Ref
- Kingma, D.P., and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.Google Scholar
- Janocha, K., and Czarnecki, W.M. 2017. On Loss Functions for Deep Neural Networks in Classification. arXiv preprint arXiv:1702.0565Google Scholar
Index Terms
- Automatic Image Colorization Using Adversarial Training
Recommendations
Wavelet regularization benefits adversarial training
AbstractAdversarial training methods are frequently-used empirical defense methods against adversarial examples. While many regularization techniques demonstrate effectiveness when combined with adversarial training, these methods typically ...
A hybrid adversarial training for deep learning model and denoising network resistant to adversarial examples
AbstractDeep neural networks (DNNs) are vulnerable to adversarial attacks that generate adversarial examples by adding small perturbations to the clean images. To combat adversarial attacks, the two main defense methods used are denoising and adversarial ...
LADDER: Latent boundary-guided adversarial training
AbstractDeep Neural Networks (DNNs) have recently achieved great success in many classification tasks. Unfortunately, they are vulnerable to adversarial attacks that generate adversarial examples with a small perturbation to fool DNN models, especially in ...
Comments