Zineng Tang
ABSTRACT
We propose an adversarial learning based model for image colorization in which we elaborately adapt image translation mechanism that are optimized according to the task. After developing approaches on improving the global and local quality of the image colorization by analyzing this processing made by network architecture and objective functions, we formulate a diverse map-ping from the gray scale images to colorful images by latent space variation within the model. At last, discussion on the theoretical framework for studying color information distribution and video colorization is given.
- 2019. 500px. (2019). https://500px.com/Google Scholar
- Jason Antic. 2019. Deoldify. (2019). https://github.com/jantic/DeOldifyGoogle Scholar
- Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein gan. arXiv preprint arXiv:1701.07875 (2017).Google Scholar
- Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018).Google Scholar
- R. Dahl. 2016. Automatic colorization. (2016).Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.Google ScholarCross Ref
- Emily L Denton, Soumith Chintala, Rob Fergus, et al. 2015. Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in neural information processing systems. 1486--1494.Google Scholar
- Aditya Deshpande, Jiajun Lu, Mao-Chuang Yeh, Min Jin Chong, and David A Forsyth. 2017. Learning Diverse Image Colorization.. In CVPR. 2877--2885.Google Scholar
- Aditya Deshpande, Jason Rock, and David Forsyth. 2015. Learning large-scale automatic image colorization. In Proceedings of the IEEE International Conference on Computer Vision. 567--575.Google ScholarDigital Library
- A. Dosovitskiy, P. Fischer, E. Ilg, P. Häusser, C. Hazrba, V. Golkov, P. v.d. Smagt, D. Cremers, and T. Brox. 2015. FlowNet: Learning Optical Flow with Convolutional Networks. In IEEE International Conference on Computer Vision (ICCV). http://lmb.informatik.uni-freiburg.de/Publications/2015/DFIB15Google Scholar
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google ScholarCross Ref
- Mingming He, Dongdong Chen, Jing Liao, Pedro V Sander, and Lu Yuan. 2018. Deep exemplar-based colorization. ACM Transactions on Graphics (TOG) 37, 4 (2018), 47.Google ScholarDigital Library
- Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.Google ScholarCross Ref
- E. Ilg, T. Saikia, M. Keuper, and T. Brox. 2018. Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation. In European Conference on Computer Vision (ECCV). http://lmb. informatik.uni-freiburg.de/Publications/2018/ISKB18Google Scholar
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-toimage translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125--1134.Google ScholarCross Ref
- Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for realtime style transfer and super-resolution. In European conference on computer vision. Springer, 694--711.Google ScholarCross Ref
- Taewon Kang and Kwang Hee Lee. 2019. Unsupervised Image-to-Image Translation with Self-Attention Networks. arXiv preprint arXiv:1901.08242 (2019).Google Scholar
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. 2016. Learning representations for automatic colorization. In European Conference on Computer Vision. Springer, 577--593.Google ScholarCross Ref
- Jae Hyun Lim and Jong Chul Ye. 2017. Geometric gan. arXiv preprint arXiv:1705.02894 (2017).Google Scholar
- Varun Manjunatha, Mohit Iyyer, Jordan Boyd-Graber, and Larry Davis. 2018. Learning to Color from Language. arXiv preprint arXiv:1804.06026 (2018).Google Scholar
- Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018).Google Scholar
- Takeru Miyato and Masanori Koyama. 2018. cGANs with projection discriminator. arXiv preprint arXiv:1802.05637 (2018).Google Scholar
- Kamyar Nazeri, Eric Ng, and Mehran Ebrahimi. 2018. Image Colorization Using Generative Adversarial Networks. In International Conference on Articulated Motion and Deformable Objects. Springer, 85--94.Google Scholar
- Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017. Conditional image synthesis with auxiliary classifier gans. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2642--2651.Google ScholarDigital Library
- Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015).Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.Google ScholarCross Ref
- Amelie Royer, Alexander Kolesnikov, and Christoph H Lampert. 2017. Probabilistic Image Colorization. arXiv preprint arXiv:1705.04258 (2017).Google Scholar
- Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google Scholar
- Leslie N Smith. 2017. Cyclical learning rates for training neural networks. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 464--472.Google ScholarCross Ref
- Harrish Thasarathan, Kamyar Nazeri, and Mehran Ebrahimi. 2019. Automatic Temporally Coherent Video Colorization. arXiv preprint arXiv:1904.09527 (2019).Google Scholar
- Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. Video-to-video synthesis. arXiv preprint arXiv:1808.06601 (2018).Google ScholarDigital Library
- Yi Xiao, Peiyao Zhou, and Yan Zheng. 2018. Interactive Deep Colorization With Simultaneous Global and Local Inputs. arXiv preprint arXiv:1801.09083 (2018).Google Scholar
- Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. 2018. Selfattention generative adversarial networks. arXiv preprint arXiv:1805.08318 (2018).Google Scholar
- Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas. 2017. Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 5907--5915.Google ScholarCross Ref
- Richard Zhang, Phillip Isola, and Alexei A Efros. 2016. Colorful image colorization. In European Conference on Computer Vision. Springer, 649--666.Google ScholarCross Ref
- Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman. 2017. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems. 465--476.Google Scholar
Index Terms
- Deep Colorization by Variation
Recommendations
Deep exemplar-based colorization
We propose the first deep learning approach for exemplar-based local colorization. Given a reference color image, our convolutional neural network directly maps a grayscale image to an output colorized image. Rather than using hand-crafted rules as in ...
A Novel Approach for Colorization of a Grayscale Image using Soft Computing Techniques
Colorization of grayscale image is a process to convert a grayscale image into a color one. Few research works reported in literature on this but there is hardly any generalized method that successfully colorizes all types of grayscale image. This study ...
Using assorted color spaces and pixel window sizes for colorization of grayscale images
ICWET '10: Proceedings of the International Conference and Workshop on Emerging Trends in TechnologyThere is no exact solution for colorization of grayscale images. In the initial work done [23], color traits transfer techniques to color grayscale images are proposed. The main focus of the techniques [23] is to minimise the human efforts needed in ...
Comments