Zhiping Yao
ABSTRACT
High dynamic range (HDR) image shows richer scene brightness and details for better visual effects than conventional low dynamic range (LDR) images since they utilize more bits to express pixel values. With little input information, the challenge of HDR reconstruction is to recover the details lost in the under- /over-exposed regions of an image. The majority of current methods for single-frame HDR reconstruction fail to focus on image denoising and color balance. In this work, we address these difficulties by proposing the notion of extracting image luminance features and texture features separately. The method is based on a dual-input channel encoder-decoder structure and utilizes a spatial feature transform module to implement information interaction between both input branches at different feature scales. In addition, our proposed network includes a weighting network to preserve useful information about the image selectively. Through both quantitative and qualitative experiments, we demonstrate the effectiveness of the components proposed in the network. In comparison to other existing mainstream methods in the field on publicly available datasets, we have demonstrated that the proposed method enables noise reduction while recovering the lost image details. The experimental results show that our method achieves state-of-the-art performance in the task of single-frame HDR reconstruction. The code is available at https://github.com/AMSTL-PING/DEUNet-HDRI.git
- KA Akhil and CV Jiji. 2021. Single Image HDR Synthesis Using a Densely Connected Dilated ConvNet.. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 526–531.Google Scholar
Cross Ref
- Reinhard Bernstein. 1987. Adaptive nonlinear filters for simultaneous removal of different kinds of noise in images. IEEE Transactions on Circuits and Systems 34, 11 (1987), 1275–1291.Google ScholarCross Ref
- Xiangyu Chen, Yihao Liu, Zhengwen Zhang, Yu Qiao, and Chao Dong. 2021. HDRUnet: Single image hdr reconstruction with denoising and dequantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 354–363.Google ScholarCross Ref
- Paul E Debevec and Jitendra Malik. 2008. Recovering high dynamic range radiance maps from photographs. In ACM SIGGRAPH 2008 classes. 1–10.Google ScholarDigital Library
- Guanting Dong, Yueyi Zhang, and Zhiwei Xiong. 2020. Spatial hierarchy aware residual pyramid network for time-of-flight depth denoising. In European Conference on Computer Vision. Springer, 35–50.Google ScholarDigital Library
- Gabriel Eilertsen, Joel Kronander, Gyorgy Denes, Rafał K Mantiuk, and Jonas Unger. 2017. HDR image reconstruction from a single exposure using deep CNNs. ACM transactions on graphics (TOG) 36, 6 (2017), 1–15.Google Scholar
- Yuki Endo, Yoshihiro Kanamori, and Jun Mitani. 2017. Deep reverse tone mapping.ACM Transactions on Graphics (TOG) 36, 6 (2017), 177–1.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026–1034.Google ScholarDigital Library
- S-W Hong and Paul Bao. 2000. An edge-preserving subband coding model based on non-adaptive and adaptive regularization. Image and Vision Computing 18, 8 (2000), 573–582.Google ScholarCross Ref
- Nima Khademi Kalantari, Ravi Ramamoorthi, 2017. Deep high dynamic range imaging of dynamic scenes.ACM Trans. Graph. 36, 4 (2017), 144–1.Google ScholarDigital Library
- Erum Arif Khan, Ahmet Oguz Akyuz, and Erik Reinhard. 2006. Ghost removal in high dynamic range images. In 2006 International Conference on Image Processing. IEEE, 2005–2008.Google ScholarCross Ref
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Yuma Kinoshita and Hitoshi Kiya. 2019. ITM-net: Deep inverse tone mapping using novel loss function considering tone mapping operator. IEEE Access 7 (2019), 73555–73563.Google ScholarCross Ref
- Yu-Lun Liu, Wei-Sheng Lai, Yu-Sheng Chen, Yi-Lung Kao, Ming-Hsuan Yang, Yung-Yu Chuang, and Jia-Bin Huang. 2020. Single-image HDR reconstruction by learning to reverse the camera pipeline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1651–1660.Google ScholarCross Ref
- Zhen Liu, Wenjie Lin, Xinpeng Li, Qing Rao, Ting Jiang, Mingyan Han, Haoqiang Fan, Jian Sun, and Shuaicheng Liu. 2021. ADNet: Attention-guided deformable convolutional network for high dynamic range imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 463–470.Google ScholarCross Ref
- Stephen Mangiat and Jerry Gibson. 2010. High dynamic range video with ghost removal. In Applications of Digital Image Processing XXXIII, Vol. 7798. SPIE, 307–314.Google Scholar
- Rafał Mantiuk, Kil Joong Kim, Allan G Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Transactions on graphics (TOG) 30, 4 (2011), 1–14.Google ScholarDigital Library
- Xiaojiao Mao, Chunhua Shen, and Yu-Bin Yang. 2016. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Advances in neural information processing systems 29 (2016), 2810–2818.Google Scholar
- Demetris Marnerides, Thomas Bashford-Rogers, Jonathan Hatchett, and Kurt Debattista. 2018. Expandnet: A deep convolutional neural network for high dynamic range expansion from low dynamic range content. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 37–49.Google Scholar
- Manish Narwaria, Rafal Mantiuk, Mattheiu P Da Silva, and Patrick Le Callet. 2015. HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images. Journal of Electronic Imaging 24, 1 (2015), 010501.Google ScholarCross Ref
- Eduardo Pérez-Pellitero, Sibi Catley-Chandar, Ales Leonardis, and Radu Timofte. 2021. NTIRE 2021 challenge on high dynamic range imaging: Dataset, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 691–700.Google Scholar
- M. A. Picard, S. Mann, and R. W. Picard. 1995. On Being ‘undigital’ With Digital Cameras: Extending Dynamic Range By Combining Differently Exposed Pictures. In Is and ts Conference Socuety for Imaging Science and Technology.Google Scholar
- Ioannis Pitas and A Venetsanopoulos. 1986. Nonlinear mean filters in image processing. IEEE transactions on acoustics, speech, and signal processing 34, 3 (1986), 573–584.Google ScholarCross Ref
- Tobias Plötz and Stefan Roth. 2018. Neural nearest neighbors networks. Advances in Neural information processing systems 31 (2018), 1095–1106.Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234–241.Google ScholarCross Ref
- Marcel Santana Santos, Tsang Ing Ren, and Nima Khademi Kalantari. 2020. Single image HDR reconstruction using a CNN with masked features and perceptual loss. arXiv preprint arXiv:2005.07335 (2020).Google Scholar
- Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1874–1883.Google ScholarCross Ref
- Abhilash Srikantha and Désiré Sidibé. 2012. Ghost detection and removal for high dynamic range images: Recent advances. Signal Processing: Image Communication 27, 6 (2012), 650–662.Google ScholarDigital Library
- Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. 2019. Edvr: Video restoration with enhanced deformable convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1954–1963.Google ScholarCross Ref
- Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. 2018. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 606–615.Google ScholarCross Ref
- Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. In Proceedings of the European conference on computer vision (ECCV) workshops. 63–79.Google Scholar
- Yuzhi Wang, Haibin Huang, Qin Xu, Jiaming Liu, Yiqun Liu, and Jue Wang. 2020. Practical deep raw image denoising on mobile devices. In European Conference on Computer Vision. Springer, 1–16.Google ScholarDigital Library
- Xin Yang, Ke Xu, Yibing Song, Qiang Zhang, Xiaopeng Wei, and Rynson WH Lau. 2018. Image correction via deep reciprocating HDR transformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1798–1807.Google ScholarCross Ref
- Jinsong Zhang and Jean-François Lalonde. 2017. Learning high dynamic range from outdoor panoramas. In Proceedings of the IEEE International Conference on Computer Vision. 4519–4528.Google ScholarCross Ref
- Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26, 7 (2017), 3142–3155.Google ScholarDigital Library
- Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu. 2018. Residual dense network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2472–2481.Google ScholarCross Ref
Index Terms
- Dual-Encoder Unet for Single Image HDR Reconstruction with Denoising
Recommendations
HDR image reconstruction from a single exposure using deep CNNs
Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that ...
Image Denoising Based on the Dyadic Wavelet Transform
ICCIMA '03: Proceedings of the 5th International Conference on Computational Intelligence and Multimedia ApplicationsSince subsampling does not take place in image dyadic wavelet transform at each level, image representation in dyadic wavelet domain compared with wavelet series reconstruction is very redundant and part of disturbance of image dyadic wavelet ...
MergeNet: Single High Dynamic Range Image Reconstruction Method
ICMSSP '21: Proceedings of the 2021 6th International Conference on Multimedia Systems and Signal ProcessingThe high dynamic range (HDR) environment mapping could provide great dynamic range and irradiance contrast for improving the fidelity of Computer Graphics Rendering, however, the acquisition of HDR images is much harder for most scenarios. We propose ...
Comments