Radiance map construction based on spatial and intensity correlations between LE and SE images for HDR imaging☆
Introduction
In real scenes, large luminance changes can affect the quality of the picture captured by a digital camera because the dynamic range of a digital camera is narrower than that of the real scene. The captured images are over or under-saturated in daylight conditions. Saturated images lose color appearance information, such as hue, colorfulness, chroma, brightness, and lightness. In order to preserve the detail information of a real scene, a high dynamic-range (HDR) image and tone-mapping operators (TMOs) should be involved [1], [2], [3], [4], [5], [6], [7].
HDR image is composed of multi-exposure images and represents intensity resolution more properly than the output of 8-bit depth for each RGB channel. The conventional HDR blending algorithms mix two images using the ratio of the exposure times. A short-exposure image is compensated by that ratio to improve the intensity level of the bright regions. Then a single HDR image is generated by combining the improved bright regions of the short-exposure image and dark regions of the long-exposure image. Blending algorithms are divided into hard-switching and soft-mixing methods according to the combining forms [8], [9].
The hard-switching method combines the intensity regions that are below the threshold level in the long-exposure image and the intensity regions that are above the threshold level in the short-exposure image. The soft-mixing method mixes long and short-exposure images at the defined transition region of each input image. In general, the exposure time of a digital camera is determined according to shutter speed, lens aperture, and ISO levels; the exposure time controls the amount of light incident on an image sensor. A relatively longer exposure time can capture dim details, whereas a short exposure time can capture highlight details. In addition, the intensity level of captured images is nonlinear with respect to the intensity level of the incident light because of the inner processing steps of a digital camera, such as gamma correction, knee control, and automatic white balance (AWB). The nonlinearity causes discontinuity in the transition region of the blending algorithms. Therefore, camera characterization is required in order to predict the camera responses given the incident light [10], [11], [12]. In the characterizations, there are spectral-sensitivity-based and color-target-based methods [13], [14]. However, these methods require specialized apparatus and reference targets. Characterizing the camera in accordance with various viewing conditions is burdensome. In contrast, image-based methods use only multi-exposure images to estimate the camera response function [15], [16], [17], [18], [19]. These methods do not require expensive hardware or a complex experimental process, but several multi-exposure images to accurately characterize the transfer function of the camera.
This paper proposes an HDR blending algorithm that uses only dual-exposure images. A proposed HDR blending algorithm is based on the least squares method, which finds the accurate curve of a camera response function (CRF) by minimizing error points. New weighting functions for spatial and intensity weighting are defined to reduce the error points. The spatial weighting function removes saturated regions and selects detail regions by comparing the standard deviation of the corresponding pixels of the input images. The intensity weighting function is used to reduce the portion of the bright and dark pixels because extremely bright or dark pixels are less accurate. The intensity weighting function is designed using a detail-preserved ratio. The detail-preserved ratio is defined using the histograms of the input images and their detail regions. The remainder of this paper is organized as follows: Sections 2 presents the related works. Section 3 explains the proposed methods. Section 4 contains the simulation results. The conclusions are given in Section 5.
Section snippets
Related works
An HDR image can be constructed using at least three LDR images with different exposure times for a static scene. A long-exposure image has good details in the overall image, but bright saturations in the bright regions. A short-exposure image has good details in the bright regions, but dark saturations in the dark regions. Multi-exposure images can be combined into a single HDR image with higher luminance range than the single-exposure image. The simple combination methods are shown in Fig. 1.
HDR blending using dual LDR images
The conventional HDR blending algorithm uses multi-exposure images for a static scene, but capturing many images of the same scene is difficult because of variations in the viewing conditions, such as tilting, shifting, and moving objects. Therefore, an effective method that uses a small number of images is required. The minimum number of images that can create an HDR image is two (long and short-exposure images). Only two input images have relatively many saturated regions that must be removed
Simulation results
The conventional HDR blending algorithm uses many differently exposed images to estimate an accurate CRF. The proposed HDR blending algorithm uses only two LDR images: short and long-exposure. In order to compare the performance of the conventional and proposed algorithms, we show the CRF graphs of each algorithm and tone-mapped images using an identical tone-mapping algorithm in Fig. 11, Fig. 12. Each algorithm uses only two LDR images.
Fig. 11(a) and (b) represents the input images used to make
Conclusions
This paper proposed a novel HDR rendering process. The proposed HDR blending algorithm is based on the least squares method using dual input images (long and short-exposure images for a scene). Two input images have relatively many saturated pixels, which make it difficult to estimate an accurate CRF. As a result, color distortion and noise appear in the conventional HDR blending algorithm. To create an accurate CRF, we proposed spatial and intensity weighting functions. The spatial function is
Acknowledgement
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2015R1D1A1A01059929).
References (22)
- et al.
Compensation of de-saturation effect in HDR imaging using a real scene adaptation model
J. Vis. Commun. Image R.
(2013) - et al.
Luminance adaptation transform based on brightness functions for LDR image reproduction
Dig. Sig. Process.
(2014) - et al.
ICAM06: a refined image appearance model for HDR image rendering
J. Vis. Commun. Image R.
(2007) - et al.
High Dynamic Range Image: Acquisition, Display, and Image-Based Lighting
(2005) - et al.
Fairchild, rendering HDR images
- et al.
High dynamic range image rendering with a retinex-based adaptive filter
IEEE Trans. Image Process.
(2006) - et al.
Fast bilateral filtering for the display of high dynamic-range images
ACM Trans. Graph.
(2002) - et al.
Tone-mapping functions and multiple-exposure techniques for high dynamic-range images
IEEE Trans. Consumer Electron.
(2008) - et al.
Dynamic range compression preserving local image contrast for digital video camera
IEEE Trans. Consumer Electron.
(2005) - et al.
Photometric calibration of zoom lens system
Radiometric CCD camera calibration and noise estimation
IEEE Trans. PAMI
Cited by (10)
CAM-based HDR image reproduction using CA–TC decoupled JCh decomposition
2019, Signal Processing: Image CommunicationCitation Excerpt :An HDR image contains the lowest to highest luminance information. Conventional HDR algorithms convert the 8-bit depth of an image for each RGB channel to a radiance level using a camera response function [8–10]. The radiance level corresponds to the amount of light reaching the image sensor.
A Review of Illuminance Mapping Practices from HDR Images and Suggestions for Exterior Measurements
2023, LEUKOS - Journal of Illuminating Engineering Society of North AmericaEnhanced tone mapping using regional fused GAN training with a gamma-shift dataset
2021, Applied Sciences (Switzerland)Model for High Dynamic Range Imaging System Using Hybrid Feature Based Exposure Fusion
2021, Journal of Intelligent SystemsVisible and near-infrared image synthesis using pca fusion of multiscale layers
2020, Applied Sciences (Switzerland)Tone Mapping High Dynamic Range Images by Hessian Multiset Canonical Correlations
2020, Sensing and Imaging
- ☆
This paper has been recommended for acceptance by Dr. Zicheng Liu.