skip to main content
10.1145/3338533.3366605acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Food Photo Enhancer of One Sample Generative Adversarial Network

Published: 10 January 2020 Publication History

Abstract

Image enhancement is an important branch in the field of image processing. A few existing methods leverage Generative Adversarial Networks (GANs) for this task. However, they have several defects when applied to a specific type of images, such as food photo. First, a large set of original-enhanced image pairs are required to train GANs that have millions of parameters. Such image pairs are expensive to acquire. Second, color distribution of enhanced images generated by previous methods is not consistent with the original ones, which is not expected. To alleviate the issues above, we propose a novel method for food photo enhancement. No original-enhanced image pairs are required except only original images. We investigate Food Faithful Color Semantic Rules in Enhanced Dataset Photo Enhancement (Faith-EDPE) and also carefully design a light generator which can preserve semantic relations among colors. We evaluate the proposed method on public benchmark databases to demonstrate the effectiveness of the proposed method through visual results and user studies.

References

[1]
Andrew Adams, Jongmin Baek, and Myers Abraham Davis. 2010. Fast high-dimensional filtering using the permutohedral lattice. In Computer Graphics Forum, Vol. 29. 753--762.
[2]
Andrew Adams, N. Gelfand, Jennifer Dolson, and Marc Levoy. 2009. Gaussian KD-Trees for Fast High-Dimensional Filtering. In ACM Trans. Graph.
[3]
Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. 2017. Unsupervised pixel-level domain adaptation with generative adversarial networks. (2017), 3722--3731.
[4]
Qifeng Chen, Xu Jia, and Vladlen Koltun. 2017. Fast Image Processing with Fully-Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision, 2497--2506.
[5]
Qifeng Chen and Vladlen Koltun. 2017. Photographic Image Synthesis with Cascaded Refinement Networks. In Proceedings of the IEEE International Conference on Computer Vision. 1511--1520.
[6]
Yunjin Chen and Thomas Pock. 2015. On learning optimized reaction diffusion processes for effective image restoration. In IEEE Conference on Computer Vision and Pattern Recognition. 2561--5269.
[7]
Yu-Sheng Chen, Yu-Ching Wang, Man-Hsin Kao, and Yung-Yu Chuang. 2018. Deep photo enhancer: Unpaired learning for image enhancement from photographs with gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6306--6314.
[8]
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[9]
C. Dong, C. C. Loy, K. He, and X. Tang. 2016. Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans Pattern Anal Mach Intell 38, 2, 295--307.
[10]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.
[11]
Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Kenneth Vanhoey, and Luc Van Gool. 2017. DSLR-quality photos on mobile devices with deep convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision. 3277--3285.
[12]
Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Kenneth Vanhoey, and Luc Van Gool. 2018. WESPE: weakly supervised photo enhancer for digital cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 691--700.
[13]
Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Kenneth Vanhoey, and Luc Van Gool. 2018. WESPE: weakly supervised photo enhancer for digital cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 691--700.
[14]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1125--1134.
[15]
John R Jensen and Kalmesh Lulla. 1987. Introductory digital image processing: a remote sensing perspective. In Taylor & Francis.
[16]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision. 694--711.
[17]
Takao Kakimori, Makoto Okabe, Keiji Yanai, and Rikio Onai. 2016. A System to Help Amateurs Take Pictures of Delicious Looking Food. In IEEE Second International Conference on Multimedia Big Data. 456--461.
[18]
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. 1857--1865.
[19]
Dilip Krishnan, Raanan Fattal, and Richard Szeliski. 2013. Efficient preconditioning of laplacian matrices for computer graphics. ACM Transactions on Graphics (TOG) 32, 4, 142.
[20]
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. 2018. Noise2noise: Learning image restoration without clean data. In arXiv preprint arXiv:1803.04189.
[21]
Minjun Li, Haozhi Huang, Lin Ma, Wei Liu, Tong Zhang, and Yugang Jiang. 2018. Unsupervised image-to-image translation with stacked cycle-consistent adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV). 184--199.
[22]
Ming Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised Image-to-Image Translation Networks. Advances in Neural Information Processing Systems (2017), 700--708.
[23]
Xiaojiao Mao, Chunhua Shen, and Yu-Bin Yang. 2016. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Advances in neural information processing systems. 2802--2810.
[24]
Youssef Alami Mejjati, Christian Richardt, James Tompkin, Darren Cosker, and Kwang In Kim. 2018. Unsupervised attention-guided image-to-image translation. In Advances in Neural Information Processing Systems. 3693--3703.
[25]
Weiqing Min, Bing Kun Bao, Shuhuan Mei, Yaohui Zhu, and Shuqiang Jiang. 2017. You Are What You Eat: Exploring Rich Recipe Information for Cross-Region Food Analysis. In IEEE Transactions on Multimedia, Vol. PP. 1--1.
[26]
Weiqing Min, Shuqiang Jiang, Linhu Liu, Yong Rui, and Ramesh Jain. 2019. A Survey on Food Computing. In ACM Computing Surveys, Vol. 52. 1--36.
[27]
Sarif K Naik and C A Murthy. 2003. Hue-preserving color image enhancement without gamut problem. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society 12, 12, 1591--8.
[28]
Hyojin Park, Youngjoon Yoo, and Nojun Kwak. 2018. MC-GAN: Multi-conditional Generative Adversarial Network for Image Synthesis. arXiv preprint arXiv:1805.01123.
[29]
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. 2016. Generative adversarial text to image synthesis. In arXiv preprint arXiv:1605.05396.
[30]
William Hadley Richardson. 1972. Bayesian-based iterative method of image restoration. In Optical Society of America. 55--59.
[31]
Olaf Ronneberger, Philipp Fischer, editor="Navab Nassir Brox, Thomas", Joachim Hornegger, William M. Wells, and Alejandro F. Frangi. 2015. UNet: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015". 234--241.
[32]
Kekai Sheng, Weiming Dong, Haibin Huang, Chongyang Ma, and Bao-Gang Hu. 2018. Gourmet photography dataset for aesthetic assessment of food images. In available online. 1--4. https://doi.org/10.1145/3283254.3283260
[33]
Yi-Hsuan Tsai, Xiaohui Shen, Zhe Lin, Kalyan Sunkavalli, Xin Lu, and Ming-Hsuan Yang. 2017. Deep image harmonization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3789--3797.
[34]
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. High-Resolution Image Synthesis and Semantic Manipulation With Conditional GANs. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[35]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600--612.
[36]
Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. 2017. Dualgan: Unsupervised dual learning for image-to-image translation. In Proceedings of the IEEE international conference on computer vision. 2849--2857.
[37]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. (2017), 2223--2232.

Cited By

View all
  • (2022)Research on Simulation Sample Generation Technology Based on Multiple Variable PointsMethods and Applications for Modeling and Simulation of Complex Systems10.1007/978-981-19-9195-0_43(537-548)Online publication date: 24-Dec-2022

Index Terms

  1. Food Photo Enhancer of One Sample Generative Adversarial Network

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MMAsia '19: Proceedings of the 1st ACM International Conference on Multimedia in Asia
    December 2019
    403 pages
    ISBN:9781450368414
    DOI:10.1145/3338533
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 January 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. GAN
    2. deep photo enhance
    3. food photo
    4. one sample domain

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • National Natural Science Foundation of China

    Conference

    MMAsia '19
    Sponsor:
    MMAsia '19: ACM Multimedia Asia
    December 15 - 18, 2019
    Beijing, China

    Acceptance Rates

    MMAsia '19 Paper Acceptance Rate 59 of 204 submissions, 29%;
    Overall Acceptance Rate 59 of 204 submissions, 29%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)12
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Research on Simulation Sample Generation Technology Based on Multiple Variable PointsMethods and Applications for Modeling and Simulation of Complex Systems10.1007/978-981-19-9195-0_43(537-548)Online publication date: 24-Dec-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media