Tsai-Ho Sun
ABSTRACT
We present a system to help designers create icons that are widely used in banners, signboards, billboards, homepages, and mobile apps. Designers are tasked with drawing contours, whereas our system colorizes contours in different styles. This goal is achieved by training a dual conditional generative adversarial network (GAN) on our collected icon dataset. One condition requires the generated image and the drawn contour to possess a similar contour, while the other anticipates the image and the referenced icon to be similar in color style. Accordingly, the generator takes a contour image and a man-made icon image to colorize the contour, and then the discriminators determine whether the result fulfills the two conditions. The trained network is able to colorize icons demanded by designers and greatly reduces their workload. For the evaluation, we compared our dual conditional GAN to several state-of-the-art techniques. Experiment results demonstrate that our network is over the previous networks. Finally, we will provide the source code, icon dataset, and trained network for public use.
Supplemental Material
Available for Download
zip
fp814aux.zip (9 MB)
The auxiliary material contains a demo video of our system.
- Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein Generative Adversarial Networks. Proceedings of the 34th International Conference on Machine Learning , Vol. 70 (2017), 214--223.Google Scholar
- David Berthelot, Thomas Schumm, and Luke Metz. 2017. BEGAN: boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717 (2017).Google Scholar
- Navaneeth Bodla, Gang Hua, and Rama Chellappa. 2018. Semi-supervised FusedGAN for Conditional Image Generation. In ECCV . https://doi.org/10.1145/2897824.2925974Google Scholar
- Yun Cao, Zhiming Zhou, Weinan Zhang, and Yong Yu. 2017. Unsupervised diverse colorization via generative adversarial networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 151--166.Google ScholarCross Ref
- Yuanzheng Ci, Xinzhu Ma, Zhihui Wang, Haojie Li, and Zhongxuan Luo. 2018. User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks. In Proceedings of ACM Multimedia. 1--6.Google ScholarDigital Library
- Jeff Donahue, Philipp Kr"ahenbühl, and Trevor Darrell. 2017. Adversarial feature learning. ICLR .Google Scholar
- Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, and Aaron Courville. 2017. Adversarially learned inference. In International Conference on Learning Representations .Google Scholar
- Chie Furusawa, Kazuyuki Hiroshiba, Keisuke Ogaki, and Yuri Odagiri. 2017. Comicolorization: semi-automatic manga colorization. In SIGGRAPH Asia 2017 Technical Briefs. ACM, 12.Google Scholar
- Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. 249--256.Google Scholar
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.Google Scholar
- Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. 2017. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems. 5767--5777.Google Scholar
- D Hawthorn. 2000. Possible implications of aging for interface designers. Interacting with Computers , Vol. 12, 5 (2000), 507 -- 528. https://doi.org/10.1016/S0953--5438(99)00021--1Google ScholarCross Ref
- Steven Heim. 2007. The resonant interface: HCI foundations for interaction design .Addison-Wesley Longman Publishing Co., Inc.Google Scholar
- Paulina Hensman and Kiyoharu Aizawa. 2017. cGAN-based Manga Colorization Using a Single Training Image. In Document Analysis and Recognition (ICDAR), 2017 14th IAPR International Conference on , Vol. 3. IEEE, 72--77.Google ScholarCross Ref
- William K Horton. 1994. The icon book: Visual symbols for computer systems and documentation .John Wiley & Sons, Inc.Google Scholar
- Shih-Miao Huang, Kong-King Shieh, and Chai-Fen Chi. 2002. Factors affecting the design of computer icons. International Journal of Industrial Ergonomics , Vol. 29, 4 (2002), 211 -- 218. https://doi.org/10.1016/S0169--8141(01)00064--6Google ScholarCross Ref
- Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. 2018. Multimodal unsupervised image-to-image translation. In Proceedings of the European Conference on Computer Vision (ECCV). 172--189.Google ScholarDigital Library
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 5967--5976.Google Scholar
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Shigenobu Kobayashi. 1992. Color Image Scale .Kodansha Amer Inc.Google Scholar
- J. Krause. 2002. Color Index: Over 1,100 Color Combinations, CMYK and RGB Formulas, for Print and Web Media .F&W Publications, Inc.Google Scholar
- Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, et almbox. 2017. Fader networks: Manipulating images by sliding attributes. In Advances in Neural Information Processing Systems. 5967--5976.Google Scholar
- Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. 2016. Autoencoding Beyond Pixels Using a Learned Similarity Metric. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. 1558--1566.Google Scholar
- Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew P Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et almbox. 2017. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network.. In CVPR , Vol. 2. 4.Google Scholar
- Chuan Li and Michael Wand. 2016. Precomputed real-time texture synthesis with markovian generative adversarial networks. In European Conference on Computer Vision. Springer, 702--716.Google ScholarCross Ref
- Yan-Peng Lim and Peter Charles Woods. 2010. Visual Information Communication .Springer, Chapter: Experimental Color in Computer Icons, 149--158.Google Scholar
- Jianxin Lin, Yingce Xia, Tao Qin, Zhibo Chen, and Tie-Yan Liu. 2018. Conditional Image-to-Image Translation. In CVPR. 5524--5532.Google Scholar
- Tomas Lindberg and Risto N"as"anen. 2003. The effect of icon spacing and size on the speed of icon processing in the human visual system. Displays , Vol. 24, 3 (2003), 111--120.Google Scholar
- Yifan Liu, Zengchang Qin, Tao Wan, and Zhenbo Luo. 2018. Auto-painter: Cartoon image generation from sketch by using conditional Wasserstein generative adversarial networks. Neurocomputing , Vol. 311 (2018), 78--87.Google ScholarDigital Library
- Michael Mathieu, Camille Couprie, and Yann LeCun. 2016. Deep multi-scale video prediction beyond mean square error. In ICLR .Google Scholar
- Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).Google Scholar
- Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018a. Spectral Normalization for Generative Adversarial Networks. In ICLR .Google Scholar
- Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. 2018b. Spectral normalization for generative adversarial networks. In ICLR .Google Scholar
- Yingge Qu, Tien-Tsin Wong, and Pheng-Ann Heng. 2006. Manga Colorization. ACM Trans. Graph. , Vol. 25, 3 (July 2006), 1214--1220. https://doi.org/10.1145/1141911.1142017Google ScholarDigital Library
- Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR .Google Scholar
- Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in Neural Information Processing Systems. 2234--2242.Google Scholar
- Yaniv Taigman, Adam Polyak, and Lior Wolf. 2017. Unsupervised cross-domain image generation. In ICLR .Google Scholar
- Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. 2018. Mocogan: Decomposing motion and content for video generation. In CVPR .Google Scholar
- Xiaolong Wang and Abhinav Gupta. 2016. Generative image modeling using style and structure adversarial networks. In European Conference on Computer Vision. Springer, 318--335.Google ScholarCross Ref
- Wei Xiong, Wenhan Luo, Lin Ma, Wei Liu, and Jiebo Luo. 2018. Learning to generate time-lapse videos using multi-stage dynamic generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2364--2373.Google ScholarCross Ref
- Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. 2016. Attribute2image: Conditional image generation from visual attributes. In European Conference on Computer Vision. Springer, 776--791.Google ScholarCross Ref
- Zili Yi, Hao (Richard) Zhang, Ping Tan, and Minglun Gong. 2017. DualGAN: Unsupervised Dual Learning for Image-to-Image Translation.. In ICCV . 2868--2876.Google Scholar
- Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris Metaxas. 2017b. StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks. In ICCV.Google Scholar
- Lvmin Zhang, Yi Ji, and Xin Lin. 2017a. Style transfer for anime sketches with enhanced residual U-net and auxiliary classifier gan. arXiv preprint arXiv:1706.03319 (2017).Google Scholar
- Junbo Zhao, Michael Mathieu, and Yann LeCun. 2016. Energy-based generative adversarial network. arXiv preprint arXiv:1609.03126 (2016).Google Scholar
- Jun-Yan Zhu, Philipp Kr"ahenbühl, Eli Shechtman, and Alexei A Efros. 2016. Generative visual manipulation on the natural image manifold. In European Conference on Computer Vision. Springer, 597--613.Google ScholarCross Ref
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017a. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. 2017 IEEE International Conference on Computer Vision (ICCV) (2017), 2242--2251.Google Scholar
- Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman. 2017b. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems. 465--476.Google Scholar
Index Terms
- Adversarial Colorization of Icons Based on Contour and Color Conditions
Recommendations
Research on Colorization of Qinghai Farmer Painting Image Based on Generative Adversarial Networks
CACML '23: Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine LearningAt present, deep learning method is widely used in the field of gray image colorization. Qinghai farmer painting has distinct national characteristics. The farmer painting has bright colors, high saturation, chaotic color distribution and low color ...
Color recovery of black-and-white halftoned images via categorized color-embedding look-up tables
A new method of recovering the original colors of black-and-white (B&W) halftoned images with homogeneous dot patterns is proposed. The conventional inverse halftoning method, which uses a look-up table (LUT), can establish the relation between the ...
Colorization in YCbCr color space and its application to JPEG images
This paper presents a colorization method in YCbCr color space, which is based on the maximum a posteriori estimation of a color image given a monochrome image as is our previous method in RGB color space. The presented method in YCbCr space is much ...
Comments