Abstract
Computational photography has become an increasingly popular technique for capturing images in high contrast scenes. Current imaging systems solve this problem by capturing a set of images with different exposure settings and then reconstructing a final image. However, this approach cannot solve the problem of revealing or predicting details in already-captured images. Convolutional neural networks (CNNs) can address this problem to some extent, but existing single image lighting enhancement methods based on deep learning suffer from CNNs’ limited receptive field and thus cannot yield the optimal results. To overcome this problem, we propose a self-attention based learning strategy inspired by high dynamic range (HDR) reconstruction process to reconstruct a properly exposed image from a single input image. Specifically, we leverage the self-attention mechanism to model the interdependencies between different locations and help reduce the local color artifacts during reconstruction. Furthermore, we adapt the idea of a generative adversarial network (GAN) and design a custom HDR loss function to achieve better image quality. We compare our method with several other recent image enhancement methods using several full-reference and non-reference image quality assessment methods. Experimental results show that our approach can produce images with better details in both over-exposed and under-exposed areas, and thereby outperform existing methods.
Supported by NSERC and UAHJIC.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
An, V.G., Lee, C.: Single-shot high dynamic range imaging via deep convolutional neural network. In: 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1768–1772, December 2017. https://doi.org/10.1109/APSIPA.2017.8282319
Eilertsen, G., Kronander, J., Denes, G., Mantiuk, R.K., Unger, J.: HDR image reconstruction from a single exposure using deep CNNs. ACM Trans. Graph. 36(6), 178 (2017). https://doi.org/10.1145/3130800.3130816
Guo, X., Li, Y., Ling, H.: LIME: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2017). https://doi.org/10.1109/TIP.2016.2639450
Jiang, Y., et al.: EnlightenGAN: Deep Light Enhancement Without Paired Supervision. arXiv:1906.06972 [cs, eess], June 2019
Kim, S.Y., Oh, J., Kim, M.: Deep SR-ITM: joint learning of super-resolution and inverse tone-mapping for 4K UHD HDR applications. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3116–3125. IEEE, Seoul, Korea (South), October 2019. https://doi.org/10.1109/ICCV.2019.00321
Kundu, D., Ghadiyaram, D., Bovik, A.C., Evans, B.L.: No-reference quality assessment of tone-mapped HDR pictures. IEEE Trans. Image Process. 26(6), 2957–2971 (2017). https://doi.org/10.1109/TIP.2017.2685941
Land, E.H.: The Retinex theory of color vision. Sci. Am. 237(6), 108–129 (1977)
Mantiuk, R., Kim, K.J., Rempel, A.G., Heidrich, W.: HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30(4), 40:1–40:14 (2011). https://doi.org/10.1145/2010324.1964935
Marnerides, D., Bashford-Rogers, T., Hatchett, J., Debattista, K.: ExpandNet: a deep convolutional neural network for high dynamic range expansion from low dynamic range content. Comput. Graph. Forum 37(2), 37–49 (2018). https://doi.org/10.1111/cgf.13340
Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Sig. Process. Lett. 20(3), 209–212 (2013). https://doi.org/10.1109/LSP.2012.2227726
Narwaria, M., Perreira Da Silva, M., Le Callet, P.: HDR-VQM: an objective quality measure for high dynamic range video. Sig. Process.: Image Commun. 35, 46–60 (2015). https://doi.org/10.1016/j.image.2015.04.009
Reinhard, E., Stark, M., Shirley, P., Ferwerda, J.: Photographic tone reproduction for digital images. In: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2002, pp. 267–276. Association for Computing Machinery, New York, July 2002. https://doi.org/10.1145/566570.566575
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs], May 2015
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Wei, C., Wang, W., Yang, W., Liu, J.: Deep Retinex Decomposition for Low-Light Enhancement. arXiv:1808.04560 [cs], August 2018
Zhang, H., Goodfellow, I., Metaxas, D., Odena, A.: Self-attention generative adversarial networks. In: International Conference on Machine Learning, pp. 7354–7363. PMLR, May 2019
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, S., Hu, K., Zhou, Z., Basu, A. (2022). Lighting Enhancement Using Self-attention Guided HDR Reconstruction. In: Berretti, S., Su, GM. (eds) Smart Multimedia. ICSM 2022. Lecture Notes in Computer Science, vol 13497. Springer, Cham. https://doi.org/10.1007/978-3-031-22061-6_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-22061-6_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22060-9
Online ISBN: 978-3-031-22061-6
eBook Packages: Computer ScienceComputer Science (R0)