Skip to main content
Springer Nature Link
Log in

ITM-CNN: Learning the Inverse Tone Mapping from Low Dynamic Range Video to High Dynamic Range Displays Using Convolutional Neural Networks

  • Conference paper
  • First Online:
Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11363))

Included in the following conference series:

Abstract

While inverse tone mapping (ITM) was frequently used for graphics rendering in the high dynamic range (HDR) space, the advent of HDR TVs and the consequent need for HDR multimedia contents open up new horizons for the consumption of ultra-high quality video contents. Unfortunately, previous methods are not appropriate for HDR TVs, and their inverse-tone-mapped results are not visually pleasing with noise amplification or lack of details. In this paper, we first present the ITM problem for HDR TVs and propose a CNN-based architecture, called ITM-CNN, which restores lost details and local contrast with its training strategy for enhancing the performance based on image decomposition using the guided filter. We demonstrate the benefits of decomposing the image by experimenting with various architectures and also compare the performance for different training strategies. Our ITM-CNN is a powerful means to solve the lack of HDR video contents with legacy LDR videos.

This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No. 2017-0-00419, Intelligent High Realistic Visual Processing for Smart Broadcasting Media).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Available at http://madvr.com/.

  2. 2.

    Available at https://mpc-hc.org/.

References

  1. Aurich, V., Weule, J.: Non-linear Gaussian filters performing edge preserving diffusion. In: Sagerer, G., Posch, S., Kummert, F. (eds.) Mustererkennung 1995. INFORMAT, pp. 538–545. Springer, Heidelberg (1995). https://doi.org/10.1007/978-3-642-79980-8_63

    Chapter  Google Scholar 

  2. Banterle, F., Ledda, P., Debattista, K., Chalmers, A.: Inverse tone mapping. In: Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, pp. 349–356. ACM (2006)

    Google Scholar 

  3. Banterle, F., Ledda, P., Debattista, K., Chalmers, A., Bloj, M.: A framework for inverse tone mapping. Vis. Comput. 23(7), 467–478 (2007)

    Article  Google Scholar 

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  5. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  6. Eilertsen, G., Kronander, J., Denes, G., Mantiuk, R.K., Unger, J.: HDR image reconstruction from a single exposure using deep CNNs. ACM Trans. Graph. 36(6), 178 (2017)

    Article  Google Scholar 

  7. Endo, Y., Kanamori, Y., Mitani, J.: Deep reverse tone mapping. ACM Trans. Graph. 36(6), 177 (2017)

    Article  Google Scholar 

  8. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)

    Google Scholar 

  9. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2011)

    Google Scholar 

  10. He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_1

    Chapter  Google Scholar 

  11. He, K., Sun, J., Tang, X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013)

    Article  Google Scholar 

  12. Huo, Y., Yang, F., Dong, L., Brost, V.: Physiological inverse tone mapping based on retina response. Vis. Comput. 30(5), 507–517 (2014)

    Article  Google Scholar 

  13. ITU-R: Parameter values for the HDTV standards for production and international programme exchange. ITU-R Rec. BT.709-5 (2002). http://www.itu.int/rec/R-REC-BT.709

  14. ITU-R: Reference electro-optical transfer function for flat panel displays used in HDTV studio production. ITU-R Rec. BT.1886 (2011)

    Google Scholar 

  15. ITU-R: Parameter values for ultra-high definition television systems for production and international programme exchange. Document ITU-R Rec. BT.2020-1 (2014). http://www.itu.int/rec/R-REC-BT.2020

  16. ITU-R: Image parameter values for high dynamic range television systems for use in production and international programme exchange. ITU-R Rec. BT.2100-0 (2016). http://www.itu.int/rec/R-REC-BT.2100

  17. Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph. 36(4), 144 (2017)

    Article  Google Scholar 

  18. Kovaleski, R.P., Oliveira, M.M.: High-quality reverse tone mapping for a wide range of exposures. In: 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 49–56. IEEE (2014)

    Google Scholar 

  19. Kovaleski, R.P., Oliveira, M.M.: High-quality brightness enhancement functions for real-time reverse tone mapping. Vis. Comput. 25(5–7), 539–547 (2009)

    Article  Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  21. Mantiuk, R., Kim, K.J., Rempel, A.G., Heidrich, W.: HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30(4), 40 (2011)

    Article  Google Scholar 

  22. Masia, B., Agustin, S., Fleming, R.W., Sorkine, O., Gutierrez, D.: Evaluation of reverse tone mapping through varying exposure conditions. ACM Trans. Graph. 28(5), 160 (2009)

    Article  Google Scholar 

  23. Masia, B., Serrano, A., Gutierrez, D.: Dynamic range expansion based on image statistics. Multimed. Tools Appl. 76(1), 631–648 (2017)

    Article  Google Scholar 

  24. Meylan, L., Daly, S., Süsstrunk, S.: The reproduction of specular highlights on high dynamic range displays. In: Color and Imaging Conference, vol. 1, pp. 333–338. Society for Imaging Science and Technology (2006)

    Google Scholar 

  25. Reinhard, E., Stark, M., Shirley, P., Ferwerda, J.: Photographic tone reproduction for digital images. ACM Trans. Graph. 21(3), 267–276 (2002)

    Article  Google Scholar 

  26. Rempel, A.G., et al.: Ldr2Hdr: on-the-fly reverse tone mapping of legacy video and photographs. In: ACM Transactions on Graphics, vol. 26, p. 39. ACM (2007)

    Google Scholar 

  27. SMPTE: High dynamic range electro-optical transfer function of mastering reference displays. SMPTE ST2084:2014 (2014)

    Google Scholar 

  28. Vedaldi, A., Lenc, K.: MatConvNet: convolutional neural networks for MATLAB. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 689–692. ACM (2015)

    Google Scholar 

  29. Zhang, J., Lalonde, J.F.: Learning high dynamic range from outdoor panoramas. In: Proceedings of the IEEE International Conference on Computer Vision (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Korea Advanced Institute of Science and Technology, Daejeon, Korea

    Soo Ye Kim, Dae-Eun Kim & Munchurl Kim

Authors
  1. Soo Ye Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dae-Eun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Munchurl Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Munchurl Kim .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C. V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1187 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kim, S.Y., Kim, DE., Kim, M. (2019). ITM-CNN: Learning the Inverse Tone Mapping from Low Dynamic Range Video to High Dynamic Range Displays Using Convolutional Neural Networks. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11363. Springer, Cham. https://doi.org/10.1007/978-3-030-20893-6_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20893-6_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20892-9

  • Online ISBN: 978-3-030-20893-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics