Skip to main content
Springer Nature Link
Log in

Matching Low-Quality Photo to DSLR-Quality with Deep Convolutional Networks

  • Conference paper
  • First Online:
Artificial Intelligence and Mobile Services – AIMS 2018 (AIMS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10970))

Included in the following conference series:

  • 1224 Accesses

Abstract

Off-the-shelf smartphone cameras typical fail to achieve the quality results of Digital Single Lens Reflex (DSLR) cameras due to their physical limitations. In the cases of autonomous driving or surveillance systems where primitive cameras are usually employed, follow-up work may hardly proceed since the low-quality images result in strong obstacles. However, most existing photo quality enhancement methods focus on certain attributes such as super-resolution, generic photo quality enhancement has not been addressed as its entirely. In this work, we formulate this problem as an image quality matching problem under image translation framework and propose an end-to-end learning approach that translates low-quality photos captured by cameras with limited capabilities into DSLR-quailty photos. Unlike most other methods without direction of enhancement, our approach matches low-quality photos to DSLR-quailty counterparts. Qualitative and quantitative comparisons have shown that our method improves the existing state-of-art in terms of structural similarity measure, peak signal-to-noise ratio and by visual appearance, where artifacts and content changes are significantly reduced. Extensive experiments show its potential as a preprocessing module to translate image quality to target domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: Computer Vision and Pattern Recognition Workshops, pp. 1132–1140 (2017)

    Google Scholar 

  2. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  Google Scholar 

  3. Lee, J.Y., Sunkavalli, K., Lin, Z., Shen, X., Kweon, I.S.: Automatic content-aware color and tone stylization. In: Computer Vision and Pattern Recognition, pp. 2470–2478 (2016)

    Google Scholar 

  4. Koshimizu, T., Toriyama, T., Babaguchi, N.: Factors on the sense of privacy in video surveillance. In: ACM Workshop on Continuous Archival and Retrival of Personal Experences, pp. 35–44 (2006)

    Google Scholar 

  5. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: International Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  6. Zhou, T., Brown, M., Snavely, N., Lowe, D.G.: Unsupervised learning of depth and ego-motion from video. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6612–6619 (2017)

    Google Scholar 

  7. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  8. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5967–5976 (2016)

    Google Scholar 

  9. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  10. Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: Deblurgan: Blind motion deblurring using conditional adversarial networks (2017)

    Google Scholar 

  11. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein GAN (2017)

    Google Scholar 

  12. Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional GANs (2017)

    Google Scholar 

  13. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  14. Johnson, J., Alahi, A., Li, F.F.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, pp. 694–711 (2016)

    Chapter  Google Scholar 

  15. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: DSLR-quality photos on mobile devices with deep convolutional networks. In: IEEE International Conference on Computer Vision (2017)

    Google Scholar 

  16. Ledig, C., Wang, Z., Shi, W., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A.: Photo-realistic single image super-resolution using a generative adversarial network (2016)

    Google Scholar 

  17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  18. Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring, pp. 257–265 (2016)

    Google Scholar 

  19. Zhang, R., Isola, P., Efros, A.A.: Colorful image colorization. In: European Conference on Computer Vision (2016)

    Google Scholar 

  20. Larsson, G., Maire, M., Shakhnarovich, G.: Learning representations for automatic colorization. In: European Conference on Computer Vision, pp. 577–593 (2016)

    Chapter  Google Scholar 

  21. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Computer Science (2014)

    Google Scholar 

  22. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695 (2012). A Publication of the IEEE Signal Processing Society

    Google Scholar 

  23. Kang, L., Ye, P., Li, Y., Doermann, D.: Convolutional neural networks for no-reference image quality assessment. In: Computer Vision and Pattern Recognition, pp. 1733–1740 (2014)

    Google Scholar 

  24. Zhu, J.: Image gradient-based joint direct visual odometry for stereo camera. In: Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 4558–4564 (2017)

    Google Scholar 

  25. Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: Computer Vision and Pattern Recognition, pp. 5188–5196 (2015)

    Google Scholar 

  26. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)

    Google Scholar 

  27. Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph. 36(4), 1–12 (2017)

    Article  Google Scholar 

  28. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  29. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Computer Science (2014)

    Google Scholar 

  30. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2015)

    Google Scholar 

  31. Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. Computer Science (2015)

    Google Scholar 

  32. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE International Conference on Computer Vision (2017)

    Google Scholar 

  33. Lv, Y., Jiang, G., Yu, M., Xu, H., Shao, F., Liu, S. : Difference of Gaussian statistical features based blind image quality assessment: a deep learning approach. In: IEEE Conference on Image Processing (2015)

    Google Scholar 

  34. Ruderman, D.L.: The statistics of natural images. Network: computation in Neural Systems (1994)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported in part by the Special Program for HPC science application of NSFC - Guangdong Joint Fund Project and National Supercomputer Centre in Guangzhou, and the National High-Tech Research and Development Plan of China (863) (No. 2015AA015901), and Shenzhen special fund for the strategic development of emerging industries (Grant no. JCYJ20160301151844537). In addition, we would like to thank the anonymous reviewers for their valuable and constructive comments to improve the quality of the paper.

Author information

Authors and Affiliations

  1. Graduate School at Shenzhen, Tsinghua University, Shenzhen, People’s Republic of China

    Weihao Xia & Yujiu Yang

  2. SenseTime Group Limited, Hong Kong, China

    Chengxi Yang & Wenxiu Sun

Authors
  1. Weihao Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengxi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yujiu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenxiu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weihao Xia .

Editor information

Editors and Affiliations

  1. University of Stuttgart, Stuttgart, Germany

    Marco Aiello

  2. Tsinghua University, Beijing, China

    Yujiu Yang

  3. Peking University, Beijing, China

    Yuexian Zou

  4. Kingdee International Software Group Co., Ltd., Shenzhen, China

    Liang-Jie Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xia, W., Yang, C., Yang, Y., Sun, W. (2018). Matching Low-Quality Photo to DSLR-Quality with Deep Convolutional Networks. In: Aiello, M., Yang, Y., Zou, Y., Zhang, LJ. (eds) Artificial Intelligence and Mobile Services – AIMS 2018. AIMS 2018. Lecture Notes in Computer Science(), vol 10970. Springer, Cham. https://doi.org/10.1007/978-3-319-94361-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-94361-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-94360-2

  • Online ISBN: 978-3-319-94361-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics