Abstract
Existing image enhancement methods are typically designed to address either the over- or under-exposure problem in the input image. When the illumination of the input image contains both over- and under-exposure problems, these existing methods may not work well. We observe from the image statistics that the local color distributions (LCDs) of an image suffering from both problems tend to vary across different regions of the image, depending on the local illuminations. Based on this observation, we propose in this paper to exploit these LCDs as a prior for locating and enhancing the two types of regions (i.e., over-/under-exposed regions). First, we leverage the LCDs to represent these regions, and propose a novel local color distribution embedded (LCDE) module to formulate LCDs in multi-scales to model the correlations across different regions. Second, we propose a dual-illumination learning mechanism to enhance the two types of regions. Third, we construct a new dataset to facilitate the learning process, by following the camera image signal processing (ISP) pipeline to render standard RGB images with both under-/over-exposures from raw data. Extensive experiments demonstrate that the proposed method outperforms existing state-of-the-art methods quantitatively and qualitatively. Codes and dataset are in https://hywang99.github.io/lcdpnet/.
K. Xu and R.W.H. Lau—Joint corresponding authors. This work was led by Rynson Lau.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Afifi, M., Derpanis, K.G., Ommer, B., Brown, M.S.: Learning multi-scale photo exposure correction. In: CVPR (2021)
Bychkovsky, V., Paris, S., Chan, E., Durand, F.: Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR (2011)
Cai, B., Xu, X., Guo, K., Jia, K., Hu, B., Tao, D.: A joint intrinsic-extrinsic prior model for retinex. In: ICCV (2017)
Cai, J., Gu, S., Zhang, L.: Learning a deep single image contrast enhancer from multi-exposure images. IEEE TIP 27(4), 2049–2062 (2018)
Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: CVPR (2018)
Chen, J., Adams, A., Wadhwa, N., Hasinoff, S.: Bilateral guided upsampling. ACM TOG 35(6), 1–8 (2016)
Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM TOG 26(3), 103-es (2007)
Chen, J., Wang, X., Guo, Z., Zhang, X., Sun, J.: Dynamic region-aware convolution. In: CVPR (2021)
Chen, L., Lu, X., Zhang, J., Chu, X., Chen, C.: HINet: half instance normalization network for image restoration. In: CVPR Workshops (2021)
Chen, W., Wenjing, W., Wenhan, Y., Jiaying, L.: Deep Retinex decomposition for low-light enhancement. In: BMVC (2018)
Chen, Y., Wang, Y., Kao, M., Chuang, Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: CVPR (2018)
Fu, X., Zeng, D., Huang, Y., Zhang, X., Ding, X.: A weighted variational model for simultaneous reflectance and illumination estimation. In: CVPR (2016)
Gharbi, M., Chen, J., Barron, J.T., Hasinoff, S.W., Durand, F.: Deep bilateral learning for real-time image enhancement. In: ACM TOG (2017)
Guo, C., et al.: Zero-reference deep curve estimation for low-light image enhancement. In: CVPR (2020)
Guo, X., Li, Y., Ling, H.: Lime: low-light image enhancement via illumination map estimation. IEEE TIP 26(2), 982–993 (2017)
Hwang, S.J., Kapoor, A., Kang, S.B.: Context-based automatic local image enhancement. In: ECCV (2012)
Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: DSLR-quality photos on mobile devices with deep convolutional networks. In: ICCV (2017)
Jiang, Y., et al.: EnlightenGAN: deep light enhancement without paired supervision. IEEE TIP 30, 2340–2349 (2021)
Kaufman, L., Lischinski, D., Werman, M.: Content-aware automatic photo enhancement. In: Computer Graphics Forum (2012)
Kingma, P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
Liu, Y.L., et al.: Single-image HDR reconstruction by learning to reverse the camera pipeline. In: CVPR (2020)
Moran, S., Marza, P., McDonagh, S., Parisot, S., Slabaugh, G.G.: DeepLPF: deep local parametric filters for image enhancement. In: CVPR (2020)
Park, J., Lee, J.Y., Yoo, D., So Kweon, I.: Distort-and-recover: color enhancement using deep reinforcement learning. In: CVPR (2018)
Paszke, A., et al.: Automatic differentiation in pytorch. In: NeurIPS Workshops (2017)
Pizer, S., et al.: Adaptive histogram equalization and its variations. Graph. Image Process. Comput. Vis. 39(3), 355–368 (1987)
Ren, W., et al.: Low-light image enhancement via a deep hybrid network. IEEE TIP 28(9), 4364–4375 (2019)
Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J. VLSI Sig. Process. Syst. Sig. Image Video Technol. 38(1), 35–44 (2004). https://doi.org/10.1023/B:VLSI.0000028532.53893.82
Risheng, L., Long, M., Jiaao, Z., Xin, F., Zhongxuan, L.: Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: CVPR (2021)
Rivera, R., Ryu, B., Chae, O.: Content-aware dark image enhancement through channel division. IEEE TIP 21(9), 3967–3980 (2012)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: MICCAI (2015)
Sean Moran, Ales Leonardis, G.S.S.M.: Curl: neural curve layers for global image enhancement. arXiv:1911.13175 (2019)
Wang, R., Zhang, Q., Fu, C., Shen, X., Zheng, W., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: CVPR (2019)
Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: CVPR (2018)
Wei, H., Yifeng, Z., Rui, H.: Low light image enhancement network with attention mechanism and Retinex model. IEEE Access 8, 74306–74314 (2020)
Xu, K., Tian, X., Yang, X., Yin, B., Lau, R.W.H.: Intensity-aware single-image deraining with semantic and color regularization. IEEE TIP 30, 8497–8509 (2021)
Xu, K., Yang, X., Yin, B., Lau, R.W.: Learning to restore low-light images via decomposition-and-enhancement. In: CVPR (2020)
Xu, L., Yan, Q., Xia, Y., Jia, J.: Structure extraction from texture via natural variation measure. ACM TOG 31(6), 1–10 (2012)
Yang, W., Wang, S., Fang, Y., Wang, Y., Liu, J.: From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. In: CVPR (2020)
Yang, X., Xu, K., Song, Y., Zhang, Q., Wei, X., Lau, R.: Image correction via deep reciprocating HDR transformation. In: CVPR (2018)
Yu, R., Liu, W., Zhang, Y., Qu, Z., Zhao, D., Zhang, B.: DeepExposure: learning to expose photos with asynchronously reinforced adversarial learning. In: NeurIPS (2018)
Zamir, S.W., et al.: Multi-stage progressive image restoration. In: CVPR (2021)
Zhang, Q., Nie, Y., Zheng, W.: Dual illumination estimation for robust exposure correction. In: Computer Graphics Forum (2019)
Zhang, Q., Yuan, G., Xiao, C., Zhu, L., Zheng, W.S.: High-quality exposure correction of underexposed photos. In: ACM MM (2018)
Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: ACM MM (2019)
Acknowledgments
This project is in part supported by a General Research Fund from RGC of Hong Kong (RGC Ref.: 11205620).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, H., Xu, K., Lau, R.W.H. (2022). Local Color Distributions Prior for Image Enhancement. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13678. Springer, Cham. https://doi.org/10.1007/978-3-031-19797-0_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-19797-0_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19796-3
Online ISBN: 978-3-031-19797-0
eBook Packages: Computer ScienceComputer Science (R0)