skip to main content
research-article

A Deep Retinex-Based Low-Light Enhancement Network Fusing Rich Intrinsic Prior Information

Published: 14 November 2024 Publication History

Abstract

Images captured under low-light conditions are characterized by lower visual quality and perception levels than images obtained in better lighting scenarios. Studies focused on low-light enhancement techniques seek to address this dilemma. However, simple image brightening results in significant noise, blurring, and color distortion. In this paper, we present a low-light enhancement (LLE) solution that effectively synergizes Retinex theory with deep learning. Specifically, we construct an efficient image gradient map estimation module based on convolutional networks that can efficiently generate noise-free image gradient maps to assist with denoising. Second, to improve upon the traditional optimization model, we design a matrix-preserving optimization method (MPOM) coupled with deep learning modules, and it exhibits high speed and low memory consumption. Third, we incorporate image structure, image texture, and implicit prior information to optimize the enhancement process for low-light conditions and overcome prevailing limitations, such as oversmoothing, significant noise, and so forth. Through extensive experiments, we show that our approach has notable advantages over the existing methods and demonstrate superiority and effectiveness, surpassing the state-of-the-art methods by an average of 1.23 dB in PSNR for the LOL and VE-LOL datasets. The code for the proposed method is available in a public repository for open-source use: https://github.com/luxunL/DRNet.

References

[1]
Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, and Zhongxuan Luo. 2022. Toward fast, flexible, and robust low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5637–5646.
[2]
Francesco Rundo. 2021. Intelligent real-time deep system for robust objects tracking in low-light driving scenario. Computation 9, 11 (2021), 117.
[3]
Junjie Ye, Changhong Fu, Ziang Cao, Shan An, Guangze Zheng, and Bowen Li. 2022. Tracker meets night: A transformer enhancer for UAV tracking. IEEE Robot. Autom. Lett. 7, 2 (2022), 3866–3873.
[4]
Jinxiu Liang, Jingwen Wang, Yuhui Quan, Tianyi Chen, Jiaying Liu, Haibin Ling, and Yong Xu. 2021. Recurrent exposure generation for low-light face detection. IEEE Trans. Multimedia 24 (2021), 1609–1621.
[5]
Jun Yu, Xinlong Hao, and Peng He. 2021. Single-stage face detection under extremely low-light conditions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 3523–3532.
[6]
Xiaoning Liu, Hui Li, and Ce Zhu. 2022. Joint contrast enhancement and exposure fusion for real-world image dehazing. IEEE Trans. Multimedia 24 (2022), 3934–3946. DOI:
[7]
Jaemoon Lim, Jin-Hwan Kim, Jae-Young Sim, and Chang-Su Kim. 2015. Robust contrast enhancement of noisy low-light images: Denoising-enhancement-completion. In Proceedings of the IEEE International Conference on Image Processing (ICIP ’15), 4131–4135. DOI:
[8]
Qianting Ma, Yang Wang, and Tieyong Zeng. 2022. Retinex-based variational framework for low-light image enhancement and denoising. IEEE Trans. Multimedia 25 (2022), 1–9. DOI:
[9]
Mingliang Zhou, Xingtai Wu, Xuekai Wei, Tao Xiang, Bin Fang, and Sam Kwong. 2023. Low-light enhancement method based on a Retinex model for structure preservation. IEEE Trans. Multimedia 26 (2023), 1–13. DOI:
[10]
Xiaojie Guo, Yu Li, and Haibin Ling. 2017. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26, 2 (Feb. 2017), 982–993.
[11]
Yi-Hsien Lin and Yi-Chang Lu. 2022. Low-light enhancement using a plug-and-play Retinex model with shrinkage mapping for illumination estimation. IEEE Trans. Image Process. 31 (2022), 4897–4908.
[12]
Anqi Zhu, Lin Zhang, Ying Shen, Yong Ma, Shengjie Zhao, and Yicong Zhou. 2020. Zero-shot restoration of underexposed images via robust Retinex decomposition. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME ’20). IEEE, 1–6.
[13]
Jun Xu, Yingkun Hou, Dongwei Ren, Li Liu, Fan Zhu, Mengyang Yu, Haoqian Wang, and Ling Shao. 2020. STAR: A structure and texture aware Retinex model. IEEE Trans. Image Process. 29 (2020), 5022–5037.
[14]
Qiang Guo, Zhi Zhang, Mingliang Zhou, Hong Yue, Huayan Pu, and Jun Luo. 2023. Image defogging based on regional gradient constrained prior. ACM Trans. Multimedia Comput. Commun. Appl. 20 (2023), 1–17.
[15]
Bolun Cai, Xianming Xu, Kailing Guo, Kui Jia, Bin Hu, and Dacheng Tao. 2017. A joint intrinsic-extrinsic prior model for Retinex. In Proceedings of the IEEE International Conference on Computer Vision, 4020–4029. DOI:
[16]
Mingliang Zhou, Hongyue Leng, Bin Fang, Tao Xiang, Xuekai Wei, and Weijia Jia. 2023. Low-light Image enhancement via a frequency-based model with structure and texture decomposition. ACM Trans. Multimedia Comput. Commun. Appl. 19 (2023), 1–23.
[17]
Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. 2020. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1780–1789.
[18]
Chuanjun Zheng, Daming Shi, and Wentian Shi. 2021. Adaptive unfolding total variation network for low-light image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 4439–4448.
[19]
Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, and Zhangyang Wang. 2021. Enlightengan: Deep light enhancement without paired supervision. IEEE Trans. Image Process. 30 (2021), 2340–2349.
[20]
Qiang Guo and Mingliang Zhou. 2022. Progressive domain translation defogging network for real-world fog images. IEEE Trans. Broadcast. 68 (2022), 876–885.
[21]
Mading Li, Jiaying Liu, Wenhan Yang, Xiaoyan Sun, and Zongming Guo. 2018. Structure-revealing low-light image enhancement via robust Retinex model. IEEE Trans. Image Process. 27, 6 (2018), 2828–2841. DOI:
[22]
Xutong Ren, Wenhan Yang, Wen-Huang Cheng, and Jiaying Liu. 2020. LR3M: Robust low-light enhancement via low-rank regularized Retinex model. IEEE Trans. Image Process. 29 (2020), 5862–5876. DOI:
[23]
Zhihao Gu, Fang Li, Faming Fang, and Guixu Zhang. 2020. A novel Retinex-based fractional-order variational model for images with severely low light. IEEE Trans. Image Process. 29 (2020), 3239–3253.
[24]
Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. 2018. Deep Retinex decomposition for low-light enhancement. Brit. Mach. Vis. Conf. 1 (2018), 1–12.
[25]
Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo. 2019. Kindling the darkness: A practical low-light image enhancer. In Proceedings of the 27th ACM International Conference on Multimedia. ACM, 1632–1640.
[26]
Wenhui Wu, Jian Weng, Pingping Zhang, Xu Wang, Wenhan Yang, and Jianmin Jiang. 2022. URetinex-net: Retinex-based deep unfolding network for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5901–5910.
[27]
Xinyi Liu, Qi Xie, Qian Zhao, Hong Wang, and Deyu Meng. 2023. Low-light image enhancement by Retinex-based algorithm unrolling and adjustment. IEEE Trans. Neural Networks Learn. Syst. 1 (2023), 1–14. DOI:
[28]
Risheng Liu, Long Ma, Jiaao Zhang, Xin Fan, and Zhongxuan Luo. 2021. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 10561–10570.
[29]
Ruixing Wang, Qing Zhang, Chi-Wing Fu, Xiaoyong Shen, Wei-Shi Zheng, and Jiaya Jia. 2019. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 6849–6857.
[30]
Kui Jiang, Zhongyuan Wang, Zheng Wang, Chen Chen, Peng Yi, Tao Lu, and Chia-Wen Lin. 2022. Degrade is upgrade: Learning degradation for low-light image enhancement. In Proceedings of the AAAI Conference on Artificial Intelligence, 1078–1086.
[31]
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Zheng Wang, Xiao Wang, Junjun Jiang, and Chia-Wen Lin. 2021. Rain-Free and residue hand-in-hand: A progressive coupled network for real-time image deraining. IEEE Trans. Image Process. 30 (2021), 7404–7418. DOI:
[32]
Kui Jiang, Junjun Jiang, Xianming Liu, Xin Xu, and Xianzheng Ma. 2024. FMRNet: Image deraining via frequency mutual revision. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, 12892–12900.
[33]
Qiong Wang, Kui Jiang, Zheng Wang, Wenqi Ren, Jianhui Zhang, and Chia-Wen Lin. 2024. Multi-scale fusion and decomposition network for single image deraining. IEEE Trans. Image Process. 33 (2024), 191–204. DOI:
[34]
Yi Xiao, Qiangqiang Yuan, Kui Jiang, Jiang He, Chia-Wen Lin, and Liangpei Zhang. 2024. TTST: A top-k token selective transformer for remote sensing image super-resolution. IEEE Trans. Image Process. 33 (2024), 738–752. DOI:
[35]
Yi Xiao, Qiangqiang Yuan, Kui Jiang, Jiang He, Xianyu Jin, and Liangpei Zhang. 2024. EDiffSR: An efficient diffusion probabilistic model for remote sensing image super-resolution. IEEE Trans. Geosci. Remote Sens. 62 (2024), 1–14. DOI:
[36]
Yi Xiao, Qiangqiang Yuan, Kui Jiang, Xianyu Jin, Jiang He, Liangpei Zhang, and Chia-Wen Lin. 2024. Local-global temporal difference learning for satellite video super-resolution. IEEE Trans. Circuits Syst. Video Technol. 34, 4 (2024), 2789–2802. DOI:
[37]
Jiaying Liu, Xu Dejia, Wenhan Yang, Minhao Fan, and Haofeng Huang. 2021. Benchmarking low-light image enhancement and beyond. Int. J. Comput. Vis. 129 (2021), 1153–1184. DOI:
[38]
Vassilios Vonikakis, Rigas Kouskouridas, and Antonios Gasteratos. 2018. On the evaluation of illumination compensation algorithms. Multimed. Tools. Appl. 77, 8 (2018), 9211–9231.
[39]
Chulwoo Lee, Chul Lee, and Chang-Su Kim. 2013. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans. Image Process. 22, 12 (2013), 5372–5384. DOI:
[40]
Kede Ma, Kai Zeng, and Zhou Wang. 2015. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 24, 11 (2015), 3345–3356.
[41]
Shuhang Wang, Jin Zheng, Hai-Miao Hu, and Bo Li. 2013. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 22, 9 (2013), 3538–3548.
[42]
Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown. 2018. A high-quality denoising dataset for smartphone cameras. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’18), 1692–1700.
[43]
Yonghua Zhang, Xiaojie Guo, Jiayi Ma, Wei Liu, and Jiawan Zhang. 2021. Beyond brightening low-light images. Int. J. Comput. Vis. 129, 4 (2021), 1013–1037.
[44]
Guo-Dong Fan, Bi Fan, Min Gan, Guang-Yong Chen, and C. L. Philip Chen. 2022. Multiscale low-light image enhancement network with illumination constraint. IEEE Trans. Circuits Syst. Video Technol. 32, 11 (2022), 7403–7417. DOI:
[45]
Xiaojie Guo and Qiming Hu. 2022. Low-light image enhancement via breaking down the darkness. Int. J. Comput. Vis. 131 (2022), 1–19.
[46]
Yufei Wang, Renjie Wan, Wenhan Yang, Haoliang Li, Lap-Pui Chau, and Alex Kot. 2022. Low-light image enhancement with normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence, 2604–2612.
[47]
Chi-Wing Fu Xiaogang Xu, Ruixing Wang and Jiaya Jia. 2022. SNR-aware low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17714–17724.
[48]
Xiaogang Xu, Ruixing Wang, and Jiangbo Lu. 2023. Low-light image enhancement via structure modeling and guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9893–9903.
[49]
Jinhui Hou, Zhiyu Zhu, Junhui Hou, Hui Liu, Huanqiang Zeng, and Hui Yuan. 2024. Global structure-aware diffusion process for low-light image enhancement. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 36, 79734–79747.
[50]
Kai Zhang, Yawei Li, Wangmeng Zuo, Lei Zhang, Luc Van Gool, and Radu Timofte. 2021. Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 44, 10 (2021), 6360–6376.
[51]
Samuel Hurault, Arthur Leclaire, and Nicolas Papadakis. 2022. Gradient Step denoiser for convergent plug-and-play. In Proceedings of the International Conference on Learning Representations (ICLR ’22), 1–30.
[52]
Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12 (2012), 4695–4708.
[53]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600–612.
[54]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 586–595.

Cited By

View all
  • (2024)Noise-Resistance Learning via Multi-Granularity Consistency for Unsupervised Domain Adaptive Person Re-IdentificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3702328Online publication date: 2-Nov-2024
  • (2024)Correlation-aware Cross-modal Attention Network for Fashion Compatibility Modeling in UGC SystemsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3698772Online publication date: 5-Oct-2024
  • (2024)Efficiently Gluing Pre-Trained Language and Vision Models for Image CaptioningACM Transactions on Intelligent Systems and Technology10.1145/368206715:6(1-16)Online publication date: 29-Jul-2024
  • Show More Cited By

Index Terms

  1. A Deep Retinex-Based Low-Light Enhancement Network Fusing Rich Intrinsic Prior Information

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 20, Issue 11
    November 2024
    702 pages
    EISSN:1551-6865
    DOI:10.1145/3613730
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 14 November 2024
    Online AM: 23 August 2024
    Accepted: 13 August 2024
    Revised: 12 May 2024
    Received: 17 February 2024
    Published in TOMM Volume 20, Issue 11

    Check for updates

    Author Tags

    1. Key Words and Phrases: Retinex theory
    2. image enhancement
    3. low-light enhancement
    4. deep learning

    Qualifiers

    • Research-article

    Funding Sources

    • Natural Science Foundation of Chongqing (Innovation and Development Joint Fund)
    • National Natural Science Foundation of China
    • Fundamental Research Funds for the Central Universities
    • Natural Science Foundation of China
    • Key Projects of Basic Strengthening Plan
    • Chongqing Talent
    • Joint Equipment Pre Research and Key Fund Project of the Ministry of Education
    • Human Resources and Social Security Bureau Project of Chongqing
    • Guangdong Oppo Mobile Telecommunications Corporation Ltd.
    • Central University Operating Expenses

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)268
    • Downloads (Last 6 weeks)25
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Noise-Resistance Learning via Multi-Granularity Consistency for Unsupervised Domain Adaptive Person Re-IdentificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3702328Online publication date: 2-Nov-2024
    • (2024)Correlation-aware Cross-modal Attention Network for Fashion Compatibility Modeling in UGC SystemsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3698772Online publication date: 5-Oct-2024
    • (2024)Efficiently Gluing Pre-Trained Language and Vision Models for Image CaptioningACM Transactions on Intelligent Systems and Technology10.1145/368206715:6(1-16)Online publication date: 29-Jul-2024
    • (2024)Dual-path Collaborative Generation Network for Emotional Video CaptioningProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681603(496-505)Online publication date: 28-Oct-2024
    • (2024)Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657727(229-239)Online publication date: 10-Jul-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media