Shifeng Xie
ABSTRACT
The development of the display technology supports the application of High Dynamic Range (HDR) enabling devices. In order to meet the surging demand for the HDR media content, we propose a feature-transferred U-shaped network (FTUnet) to convert existing Standard Dynamic Range (SDR) images into their HDR counterparts. The proposed FTUnet is a feature transformation network that converts the encoded SDR features to the HDR features. This transformation network extracts features rich of spatial information by a self-attention mechanism, in order to improve the reconstruction of the over-exposed regions and avoid unreasonable patches. Besides, we propose an Excitation-Restoration (ER) sub-network to involve the inter-channel attention mechanism. The ER network is used to remove redundant information between channels and reserve the key features. Therefore, the proposed FTUnet can efficiently merge feature channels and contribute to the advantage in color accuracy for the generated HDR images. Experimental results show that our proposed FTUnet achieves state-of-the-art performance in both quantitative comparison and visual quality for the single HDR image reconstruction. The ablation study is also performed to demonstrate the effectiveness of each module of the proposed FTUnet.
- S M A Sharif, Rizwan Ali Naqvi, Mithun Biswas, and Sungjun Kim. 2021. A Two-Stage Deep Network for High Dynamic Range Image Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 550–559.Google Scholar
Cross Ref
- Maryam Azimi 2021. PU21: A novel perceptually uniform encoding for adapting existing quality metrics for HDR. In 2021 Picture Coding Symposium (PCS). IEEE, 1–5.Google Scholar
- Xiangyu Chen, Yihao Liu, Zhengwen Zhang, Yu Qiao, and Chao Dong. 2021. Hdrunet: Single image hdr reconstruction with denoising and dequantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 354–363.Google ScholarCross Ref
- Xiangyu Chen, Zhengwen Zhang, Jimmy S. Ren, Lynhoo Tian, Yu Qiao, and Chao Dong. 2021. A New Journey From SDRTV to HDRTV. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 4500–4509.Google ScholarCross Ref
- Consumer Technology Association. 2015. HDR10: High Dynamic Range Format. Consumer Technology Association Standard. https://www.cta.tech/Standards/Standard-Detail.aspx?Id=5434 CTA-861-G.Google Scholar
- Eilertsen Gabriel, Kronander Joel, Denes Gyorgy, Mantiuk Rafał, and Unger Jonas. 2017. HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics (TOG) 36, 6, Article 178 (2017).Google Scholar
- Michaël Gharbi, Jiawen Chen, Jonathan T Barron, Samuel W Hasinoff, and Frédo Durand. 2017. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1–12.Google ScholarDigital Library
- Param Hanji, Rafał K. Mantiuk, Gabriel Eilertsen, Saghi Hajisharif, and Jonas Unger. 2022. Comparison of single image HDR reconstruction methods — the caveats of quality assessment. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings (SIGGRAPH ’22 Conference Proceedings). https://doi.org/10.1145/3528233.3530729Google ScholarDigital Library
- Gang He, Kepeng Xu, Li Xu, Chang Wu, Ming Sun, Xing Wen, and Yu-Wing Tai. 2022. SDRTV-to-HDRTV via hierarchical dynamic context feature mapping. In Proceedings of the 30th ACM International Conference on Multimedia. 2890–2898.Google ScholarDigital Library
- Jingwen He, Yihao Liu, Yu Qiao, and Chao Dong. 2020. Conditional sequential modulation for efficient global image retouching. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16. Springer, 679–695.Google Scholar
- Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132–7141.Google ScholarCross Ref
- Yongqing Huo, Fan Yang, Le Dong, and Vincent Brost. 2014. Physiological inverse tone mapping based on retina response. The Visual Computer 30 (2014), 507–517.Google ScholarCross Ref
- International Telecommunication Union. 2020. BT.2020: Parameter values for ultra-high-definition television systems for production and international programme exchange. ITU-R Recommendation. https://www.itu.int/rec/R-REC-BT.2020 Rec. ITU-R BT.2020.Google Scholar
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125–1134.Google ScholarCross Ref
- ITU-R. 2019. Objective metric for the assessment of the potential visibility of colour differences in television. ITU-R Rec, BT.2124 (2019).Google Scholar
- Nima Khademi Kalantari, Ravi Ramamoorthi, 2017. Deep high dynamic range imaging of dynamic scenes.ACM Trans. Graph. 36, 4 (2017), 144–1.Google ScholarDigital Library
- Harpreet Kaur, Deepika Koundal, and Virender Kadyan. 2021. Image fusion techniques: a survey. Archives of computational methods in Engineering 28 (2021), 4425–4447.Google Scholar
- Soo Ye Kim, Jihyong Oh, and Munchurl Kim. 2019. Deep sr-itm: Joint learning of super-resolution and inverse tone-mapping for 4k uhd hdr applications. In Proceedings of the IEEE/CVF international conference on computer vision. 3116–3125.Google ScholarCross Ref
- Soo Ye Kim, Jihyong Oh, and Munchurl Kim. 2020. Jsi-gan: Gan-based joint super-resolution and inverse tone-mapping with pixel-wise task-specific filters for uhd hdr video. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11287–11295.Google ScholarCross Ref
- Rafael P. Kovaleski and Manuel M. Oliveira. 2014. High-Quality Reverse Tone Mapping for a Wide Range of Exposures. In 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images. 49–56. https://doi.org/10.1109/SIBGRAPI.2014.29Google ScholarDigital Library
- Rafael P Kovaleski and Manuel M Oliveira. 2014. High-quality reverse tone mapping for a wide range of exposures. In 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images. IEEE, 49–56.Google ScholarDigital Library
- Phuoc-Hieu Le, Quynh Le, Rang Nguyen, and Binh-Son Hua. 2023. Single-Image HDR Reconstruction by Multi-Exposure Generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 4063–4072.Google ScholarCross Ref
- Sang-Hoon Lee, Haesoo Chung, and Nam Ik Cho. 2020. Exposure-structure blending network for high dynamic range imaging of dynamic scenes. IEEE Access 8 (2020), 117428–117438.Google ScholarCross Ref
- Zhen Liu, Wenjie Lin, Xinpeng Li, Qing Rao, Ting Jiang, Mingyan Han, Haoqiang Fan, Jian Sun, and Shuaicheng Liu. 2021. ADNet: Attention-guided deformable convolutional network for high dynamic range imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 463–470.Google ScholarCross Ref
- Rafał Mantiuk, Kil Joong Kim, Allan G Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Transactions on graphics (TOG) 30, 4 (2011), 1–14.Google ScholarDigital Library
- Demetris Marnerides, Thomas Bashford-Rogers, Jonathan Hatchett, and Kurt Debattista. 2018. ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content. CoRR abs/1803.02266 (2018). arXiv:1803.02266http://arxiv.org/abs/1803.02266Google Scholar
- Yuzhen Niu, Jianbin Wu, Wenxi Liu, Wenzhong Guo, and Rynson WH Lau. 2021. HDR-GAN: HDR image reconstruction from multi-exposed LDR images with large motions. IEEE Transactions on Image Processing 30 (2021), 3885–3896.Google ScholarCross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).Google Scholar
- Hu Wang, Mao Ye, Xiatian Zhu, Shuai Li, Ce Zhu, and Xue Li. 2022. KUNet: Imaging Knowledge-Inspired Single HDR Image Reconstruction. In IJCAI-ECAI 2022.Google ScholarCross Ref
- Lin Wang and Kuk-Jin Yoon. 2021. Deep learning for hdr imaging: State-of-the-art and future trends. IEEE transactions on pattern analysis and machine intelligence 44, 12 (2021), 8874–8895.Google Scholar
- Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600–612.Google ScholarDigital Library
- Hui Zeng, Jianrui Cai, Lida Li, Zisheng Cao, and Lei Zhang. 2020. Learning image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 4 (2020), 2058–2073.Google Scholar
- Lin Zhang and Hongyu Li. 2012. SR-SIM: A fast and high performance IQA index based on spectral residual. In 2012 19th IEEE international conference on image processing. IEEE, 1473–1476.Google ScholarCross Ref
- Lin Zhang, Ying Shen, and Hongyu Li. 2014. VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Transactions on Image processing 23, 10 (2014), 4270–4281.Google ScholarCross Ref
- Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. 2011. FSIM: A feature similarity index for image quality assessment. IEEE transactions on Image Processing 20, 8 (2011), 2378–2386.Google ScholarDigital Library
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.Google ScholarCross Ref
Index Terms
- FTUnet: Feature Transferred U-Net For Single HDR Image Reconstruction
Recommendations
HDR image reconstruction from a single exposure using deep CNNs
Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that ...
Dual-Encoder Unet for Single Image HDR Reconstruction with Denoising
ICCAI '23: Proceedings of the 2023 9th International Conference on Computing and Artificial IntelligenceHigh dynamic range (HDR) image shows richer scene brightness and details for better visual effects than conventional low dynamic range (LDR) images since they utilize more bits to express pixel values. With little input information, the challenge of ...
Single image HDR reconstruction using a CNN with masked features and perceptual loss
Digital cameras can only capture a limited range of real-world scenes' luminance, producing images with saturated pixels. Existing single image high dynamic range (HDR) reconstruction methods attempt to expand the range of luminance, but are not able to ...
Comments