Abstract
In social networks, it is important to re-stylize a randomly taken photo to exhibit a unique individual character. Previous stylization methods either respect to a motivation of improving perceptual quality or artistic style transfer, are neither personalized nor photorealistic. Besides, a strong constraint on scene consistency of reference image is always required, which is not easy to meet for a customized application. In this paper, we propose a customized photorealistic re-stylization method referred to a group of user favorite images with loose scene consistency. To better express user preferred style, reference images are selected from the perspective of photographer where image content and composition are jointly considered and weighed by user preference of light and color. To achieve high perceptual quality, we map image pixels and styles based on Conditional Generative Adversarial Networks. Comprehensive experiments verify our method could improve user preferred photo re-stylization and bring in less artificiality.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Lee, J.-S.: Digital image enhancement and noise filtering by use of local statistics. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2(2), 165–168 (1980)
Zhang, R., Isola, P., Efros, A.A.: Colorful Image Colorization (2016). arXiv:160308511 Cs
Faridul, H.S., Pouli, T., Chamaret, C., Stauder, J., Tremeau, A., Reinhard, E.: A Survey of Color Mapping and its Applications (2014)
Cheng, W., Jiang, R., Chen, C.W.: Color photo makeover via crowd sourcing and recoloring. In: Proceedings of the 23rd ACM International Conference on Multimedia, New York, NY, USA, pp. 943–946 (2015)
Hristina, H., Le Meur, O., Cozot, R., Bouatouch, K.: Style-aware robust color transfer. In: Computational Aesthetics in Graphics, Visualization, and Imaging, Istambul, Turkey (2015)
Cheng, Z., Yang, Q., Sheng, B.: Deep colorization. In. Presented at the Proceedings of the IEEE International Conference on Computer Vision, pp. 415–423 (2015)
Goodfellow, I: NIPS 2016 Tutorial: Generative Adversarial Networks (2016). arXiv:170100160 Cs
Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)
HaCohen, Y., Shechtman, E., Goldman, D.B., Lischinski, D.: Non-rigid dense correspondence with applications for image enhancement. In: ACM SIGGRAPH 2011 Papers, New York, NY, USA, pp. 70:1–70:10 (2011)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414–2423 (2016)
Kang, S.B., Kapoor, A., Lischinski, D.: Personalization of image enhancement. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1799–1806 (2010)
Yao, L., Suryanarayan, P., Qiao, M., Wang, J.Z., Li, J.: OSCAR: on-site composition and aesthetics feedback through exemplars for photographers. Int. J. Comput. Vis. 96(3), 353–383 (2012)
Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM International Conference on Multimedia, New York, NY, USA, pp. 83–92 (2010)
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. Adv. Neural. Inf. Process. Syst. 19, 545–552 (2007)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-Image Translation with Conditional Adversarial Networks. arXiv:161107004 Cs (2016)
Aherne, F.J., Thacker, N.A., Rockett, P.I.: The Bhattacharyya metric as an absolute similarity measure for frequency coded data. Kybernetika 34(4), 363–368 (1998)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Bonneel, N., Sunkavalli, K., Paris, S., Pfister, H.: Example-based video color grading. ACM Trans. Graph 32(4), 39:1–39:12 (2013)
Pitié, F., Kokaram, A.C., Dahyot, R.: Automated colour grading using colour distribution transfer. Comput. Vis. Image Underst. 107(1), 123–137 (2007)
Kapoor, A., Caicedo, J.C., Lischinski, D., Kang, S.B.: Collaborative personalization of image enhancement. Int. J. Comput. Vis. 108(1–2), 148–164 (2014)
Acknowledgments
This work is supported by the National Nature Science Foundation of China (No. 61602430, No. 61702471, No. 61402428), and The Aoshan Innovation Project in Science and Technology of Qingdao National Laboratory for Marine Science and Technology (No. 2016ASKJ07).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Z., Yuan, M., Nie, J., Huang, L., Wei, Z. (2018). CGANs Based User Preferred Photorealistic Re-stylization of Social Image. In: Hong, R., Cheng, WH., Yamasaki, T., Wang, M., Ngo, CW. (eds) Advances in Multimedia Information Processing – PCM 2018. PCM 2018. Lecture Notes in Computer Science(), vol 11165. Springer, Cham. https://doi.org/10.1007/978-3-030-00767-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-00767-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00766-9
Online ISBN: 978-3-030-00767-6
eBook Packages: Computer ScienceComputer Science (R0)