ABSTRACT
Low-light imaging task aims to approximate low-light scenes as perceived by human eyes. Existing methods usually pursue higher brightness, resulting in unrealistic exposure. Inspired by Human Vision System (HVS), where rods perceive more lights while cones perceive more colors, we propose a Low-light Degradation Rectify Model (LDRM) with color-monochrome cameras to solve this problem. First, we propose to use a low-ISO color camera and a high-ISO monochrome camera for low-light imaging under short-exposure of less than 0.1s. Short-exposure could avoid motion blurriness, while monochrome camera captures more photons than color camera. By mimicing HVS, this capture system could benefit low-light imaging. Second, we propose an LDRM model to fuse the color-monochrome image pair into a high-quality image. In this model, we separately restore UV and Y channels through chrominance and luminance branches and use monochrome image to guide the restoration of luminance. We also propose a latent code embedding method to improve the restorations of both branches. Third, we create a Low-light Color-Monochrome benchmark (LCM), including both synthetic and real-world datasets, to examine low-light imaging quality of LDRM and the state-of-the-art methods. Experimental results demonstrate the superior performance of LDRM with visually pleasing results. Codes and datasets are available at https://github.com/StephenLinn/LDRM.
- Alexandre Benoit, Alice Caplier, Barthélémy Durette, and Jeanny Hérault. 2010. Using human visual system modeling for bio-inspired low level image processing. Computer Vision and Image Understanding, Vol. 114, 7 (2010), 758--773.Google ScholarDigital Library
- Bolun Cai, Xianming Xu, Kailing Guo, Kui Jia, Bin Hu, and Dacheng Tao. 2017. A joint intrinsic-extrinsic prior model for retinex. In Proceedings of the IEEE International Conference on Computer Vision. 4000--4009.Google ScholarCross Ref
- Jiezhang Cao, Jingyun Liang, Kai Zhang, Yawei Li, Yulun Zhang, Wenguan Wang, and Luc Van Gool. 2022. Reference-Based Image Super-Resolution with Deformable Attention Transformer. In Proceedings of the European Conference on Computer Vision. 325--342.Google ScholarDigital Library
- Meng Chang, Huajun Feng, Zhihai Xu, and Qi Li. 2021. Low-light image restoration with short-and long-exposure raw pairs. IEEE Transactions on Multimedia, Vol. 24 (2021), 702--714.Google ScholarCross Ref
- Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun. 2018. Learning to see in the dark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3291--3300.Google ScholarCross Ref
- Heng-Da Cheng and X.J. Shi. 2004. A simple and effective histogram equalization approach to image enhancement. Digital Signal Processing, Vol. 14, 2 (2004), 158--170.Google ScholarCross Ref
- Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, and Baining Guo. 2022a. Cswin transformer: A general vision transformer backbone with cross-shaped windows. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12124--12134.Google ScholarCross Ref
- Xuan Dong, Chang Liu, Weixin Li, Xiaoyan Hu, Xiaojie Wang, and Yunhong Wang. 2021. Self-supervised colorization towards monochrome-color camera systems using cycle CNN. IEEE Transactions on Image Processing, Vol. 30 (2021), 6609--6622.Google ScholarDigital Library
- Xingbo Dong, Wanyan Xu, Zhihui Miao, Lan Ma, Chao Zhang, Jiewen Yang, Zhe Jin, Andrew Beng Jin Teoh, and Jiajun Shen. 2022b. Abandoning the Bayer-filter to see in the dark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17431--17440.Google ScholarCross Ref
- Hansen Feng, Lizhi Wang, Yuzhi Wang, and Hua Huang. 2022. Learnability Enhancement for Low-light Raw Denoising: Where Paired Real Data Meets Noise Modeling. In Proceedings of the 30th ACM International Conference on Multimedia. 1436--1444.Google ScholarDigital Library
- Xueyang Fu, Yinghao Liao, Delu Zeng, Yue Huang, Xiao-Ping Zhang, and Xinghao Ding. 2015. A probabilistic method for image enhancement with simultaneous illumination and reflectance estimation. IEEE Transactions on Image Processing, Vol. 24, 12 (2015), 4965--4977.Google ScholarDigital Library
- Chie Furusawa, Kazuyuki Hiroshiba, Keisuke Ogaki, and Yuri Odagiri. 2017. Comicolorization: semi-automatic manga colorization. In SIGGRAPH Asia 2017 Technical Briefs. 1--4.Google Scholar
- Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. 2020. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1780--1789.Google ScholarCross Ref
- Peiyao Guo, M. Salman Asif, and Zhan Ma. 2022. Low-Light Color Imaging via Cross-Camera Synthesis. IEEE Journal of Selected Topics in Signal Processing, Vol. 16, 4 (2022), 828--842.Google ScholarCross Ref
- Xiaojie Guo, Yu Li, and Haibin Ling. 2016. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, Vol. 26, 2 (2016), 982--993.Google ScholarDigital Library
- Shijie Hao, Xu Han, Yanrong Guo, Xin Xu, and Meng Wang. 2020. Low-light image enhancement with semi-decoupled decomposition. IEEE Transactions on Multimedia, Vol. 22, 12 (2020), 3025--3038.Google ScholarDigital Library
- Samuel W. Hasinoff, Dillon Sharlet, Ryan Geiss, Andrew Adams, Jonathan T. Barron, Florian Kainz, Jiawen Chen, and Marc Levoy. 2016. Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Transactions on Graphics, Vol. 35, 6 (2016), 1--12.Google ScholarDigital Library
- Heiko Hirschmuller and Daniel Scharstein. 2007. Evaluation of cost functions for stereo matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.Google ScholarCross Ref
- Haofeng Huang, Wenhan Yang, Yueyu Hu, Jiaying Liu, and Ling-Yu Duan. 2022. Towards low light enhancement with raw images. IEEE Transactions on Image Processing, Vol. 31 (2022), 1391--1405.Google ScholarCross Ref
- Haidi Ibrahim and Nicholas Sia Pik Kong. 2007. Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Transactions on Consumer Electronics, Vol. 53, 4 (2007), 1752--1758.Google ScholarDigital Library
- Hae-Gon Jeon, Sunghoon Im, Jaesung Choe, Minjun Kang, Joon-Young Lee, and Martial Hebert. 2022. CMSNet: Deep Color and Monochrome Stereo. International Journal of Computer Vision, Vol. 130, 3 (2022), 652--668.Google ScholarDigital Library
- Hae-Gon Jeon, Joon-Young Lee, Sunghoon Im, Hyowon Ha, and In So Kweon. 2016. Stereo matching with color and monochrome cameras in low-light conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4086--4094.Google ScholarCross Ref
- Nanfeng Jiang, Junhong Lin, Ting Zhang, Haifeng Zheng, and Tiesong Zhao. 2023. Low-Light Image Enhancement via Stage-Transformer-Guided Network. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 33, 8 (2023), 3701--3712.Google ScholarDigital Library
- Yuming Jiang, Kelvin C.K. Chan, Xintao Wang, Chen Change Loy, and Ziwei Liu. 2021a. Robust reference-based super-resolution via c2-matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2103--2112.Google ScholarCross Ref
- Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, and Zhangyang Wang. 2021b. Enlightengan: Deep light enhancement without paired supervision. IEEE Transactions on Image Processing, Vol. 30 (2021), 2340--2349.Google ScholarDigital Library
- Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In Proceedings of the European Conference on Computer Vision. Springer, 694--711.Google ScholarCross Ref
- Chongyi Li, Chunle Guo, and Chen Change Loy. 2021. Learning to enhance low-light image via zero-reference deep curve estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 8 (2021), 4225--4238.Google Scholar
- Orly Liba, Kiran Murthy, Yun-Ta Tsai, Tim Brooks, Tianfan Xue, Nikhil Karnad, Qiurui He, Jonathan T. Barron, Dillon Sharlet, Ryan Geiss, Samuel W. Hansinoff, Yael Pritch, and Marc Levoy. 2019. Handheld mobile photography in very low light. ACM Transactions on Graphics, Vol. 38, 6 (2019), 164--1.Google ScholarDigital Library
- Daniel Scharstein, Heiko Hirschmüller, York Kitajima, Greg Krathwohl, Nera Nevsić, Xi Wang, and Porter Westling. 2014. High-resolution stereo datasets with subpixel-accurate ground truth. In Pattern Recognition. Springer, 31--42.Google Scholar
- Yoav Y. Schechner, Shree K. Nayar, and Peter N. Belhumeur. 2007. Multiplexing for optimal lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 29, 8 (2007), 1339--1354.Google ScholarDigital Library
- George Seif and Dimitrios Androutsos. 2018. Edge-Based Loss Function for Single Image Super-Resolution. In IEEE International Conference on Acoustics, Speech and Signal Processing. 1468--1472.Google Scholar
- Tony Vladusich. 2007. Chromatic aberration and the roles of double-opponent and color-luminance neurons in color vision. Neural Networks, Vol. 20, 2 (2007), 153--155.Google ScholarDigital Library
- Shuhang Wang, Jin Zheng, Hai-Miao Hu, and Bo Li. 2013. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Iimage Processing, Vol. 22, 9 (2013), 3538--3548.Google ScholarDigital Library
- Tengfei Wang, Jiaxin Xie, Wenxiu Sun, Qiong Yan, and Qifeng Chen. 2021b. Dual-camera super-resolution with aligned attention modules. In Proceedings of the IEEE International Conference on Computer Vision. 2001--2010.Google ScholarCross Ref
- Yicheng Wang, Jiayong Peng, Yueyi Zhang, Shan Liu, Xiaoyan Sun, and Zhiwei Xiong. 2021a. Asymmetric stereo color transfer. In IEEE International Conference on Multimedia and Expo. 1--6.Google ScholarCross Ref
- Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, Vol. 13, 4 (2004), 600--612.Google ScholarDigital Library
- Kaixuan Wei, Ying Fu, Jiaolong Yang, and Hua Huang. 2020. A physics-based noise formation model for extreme low-light raw denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2758--2767.Google ScholarCross Ref
- Bin Xia, Yapeng Tian, Yucheng Hang, Wenming Yang, Qingmin Liao, and Jie Zhou. 2022. Coarse-to-fine embedded patchmatch and multi-scale dynamic aggregation for reference-based super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 2768--2776.Google ScholarCross Ref
- Meifang Yang, Xin Nie, and Ryan Wen Liu. 2019. Coarse-to-fine luminance estimation for low-light image enhancement in maritime video surveillance. In 2019 IEEE Intelligent Transportation Systems Conference. IEEE, 299--304.Google ScholarDigital Library
- Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2020. Learning enriched features for real image restoration and enhancement. In Proceedings of the European Conference on Computer Vision. 492--511.Google ScholarDigital Library
- Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2022a. Learning enriched features for fast image restoration and enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 2 (2022), 1934--1948.Google ScholarCross Ref
- Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Khan Shahbaz, and Ming-Hsuan Yang. 2022b. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5728--5739.Google ScholarCross Ref
- Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 586--595.Google ScholarCross Ref
- Yonghua Zhang, Xiaojie Guo, Jiayi Ma, Wei Liu, and Jiawan Zhang. 2021. Beyond brightening low-light images. International Journal of Computer Vision, Vol. 129 (2021), 1013--1037.Google ScholarDigital Library
- Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo. 2019. Kindling the darkness: A practical low-light image enhancer. In Proceedings of the 27th ACM International Conference on Multimedia. 1632--1640.Google ScholarDigital Library
- Haitian Zheng, Mengqi Ji, Haoqian Wang, Yebin Liu, and Lu Fang. 2018. Crossnet: An end-to-end reference-based super resolution network using cross-scale warping. In Proceedings of the European Conference on Computer Vision. 88--104.Google ScholarDigital Library
Index Terms
- LDRM: Degradation Rectify Model for Low-light Imaging via Color-Monochrome Cameras
Recommendations
Shoot high-quality color images using dual-lens system with monochrome and color cameras
AbstractIn the dual-lens system with monochrome and color cameras, the gray image captured by the monochrome camera has better quality than the color image from the color camera, but does not have color information. To get high-quality color ...
Low-light imaging method with visible-band and wide-band image pair
ICIP'09: Proceedings of the 16th IEEE international conference on Image processingImages captured from digital camera suffer noise due to high sensor gain under dark conditions. Furthermore, camera-shake is unavoidable when exposure time is long enough to have desirable SNR. This paper presents an image capturing method that enhances ...
Low-Light Color Imaging via Dual Camera Acquisition
Computer Vision – ACCV 2020AbstractAs existing low-light color imaging suffers from the unrealistic color representation or blurry texture with a single camera setup, we are motivated to devise a dual camera system using a high spatial resolution (HSR) monochrome camera and another ...
Comments