Radiance map construction based on spatial and intensity correlations between LE and SE images for HDR imaging

https://doi.org/10.1016/j.jvcir.2016.04.022Get rights and content

Highlights

  • This algorithm represents an HDR image using dual exposure images.

  • Spatial and intensity correlations estimate an accurate camera response function.

  • The camera response function constructs a radiance map for HDR tone reproduction.

Abstract

HDR image was developed for the reproduction of real scenes with an acquisition of large dynamic range. In general, HDR image consists of several different exposure images according to the exposure value of a digital camera. Before the construction of a single HDR image, each input image is calibrated using CRF to convert its image data to scene radiance. In order find CRF, conventional methods require special apparatus and reference targets, or several exposure images. This paper proposes a new HDR blending algorithm that uses only dual-exposure images. The proposed algorithm is based on the least squares method, and includes spatial and intensity weighting functions. Each weighting function is used to reduce error points and improve CRF computation accuracy. In addition, a constraint is added to correct white balance in the brightness level. The rendering results show that the proposed algorithm is superior to the conventional algorithm.

Introduction

In real scenes, large luminance changes can affect the quality of the picture captured by a digital camera because the dynamic range of a digital camera is narrower than that of the real scene. The captured images are over or under-saturated in daylight conditions. Saturated images lose color appearance information, such as hue, colorfulness, chroma, brightness, and lightness. In order to preserve the detail information of a real scene, a high dynamic-range (HDR) image and tone-mapping operators (TMOs) should be involved [1], [2], [3], [4], [5], [6], [7].

HDR image is composed of multi-exposure images and represents intensity resolution more properly than the output of 8-bit depth for each RGB channel. The conventional HDR blending algorithms mix two images using the ratio of the exposure times. A short-exposure image is compensated by that ratio to improve the intensity level of the bright regions. Then a single HDR image is generated by combining the improved bright regions of the short-exposure image and dark regions of the long-exposure image. Blending algorithms are divided into hard-switching and soft-mixing methods according to the combining forms [8], [9].

The hard-switching method combines the intensity regions that are below the threshold level in the long-exposure image and the intensity regions that are above the threshold level in the short-exposure image. The soft-mixing method mixes long and short-exposure images at the defined transition region of each input image. In general, the exposure time of a digital camera is determined according to shutter speed, lens aperture, and ISO levels; the exposure time controls the amount of light incident on an image sensor. A relatively longer exposure time can capture dim details, whereas a short exposure time can capture highlight details. In addition, the intensity level of captured images is nonlinear with respect to the intensity level of the incident light because of the inner processing steps of a digital camera, such as gamma correction, knee control, and automatic white balance (AWB). The nonlinearity causes discontinuity in the transition region of the blending algorithms. Therefore, camera characterization is required in order to predict the camera responses given the incident light [10], [11], [12]. In the characterizations, there are spectral-sensitivity-based and color-target-based methods [13], [14]. However, these methods require specialized apparatus and reference targets. Characterizing the camera in accordance with various viewing conditions is burdensome. In contrast, image-based methods use only multi-exposure images to estimate the camera response function [15], [16], [17], [18], [19]. These methods do not require expensive hardware or a complex experimental process, but several multi-exposure images to accurately characterize the transfer function of the camera.

This paper proposes an HDR blending algorithm that uses only dual-exposure images. A proposed HDR blending algorithm is based on the least squares method, which finds the accurate curve of a camera response function (CRF) by minimizing error points. New weighting functions for spatial and intensity weighting are defined to reduce the error points. The spatial weighting function removes saturated regions and selects detail regions by comparing the standard deviation of the corresponding pixels of the input images. The intensity weighting function is used to reduce the portion of the bright and dark pixels because extremely bright or dark pixels are less accurate. The intensity weighting function is designed using a detail-preserved ratio. The detail-preserved ratio is defined using the histograms of the input images and their detail regions. The remainder of this paper is organized as follows: Sections 2 presents the related works. Section 3 explains the proposed methods. Section 4 contains the simulation results. The conclusions are given in Section 5.

Section snippets

Related works

An HDR image can be constructed using at least three LDR images with different exposure times for a static scene. A long-exposure image has good details in the overall image, but bright saturations in the bright regions. A short-exposure image has good details in the bright regions, but dark saturations in the dark regions. Multi-exposure images can be combined into a single HDR image with higher luminance range than the single-exposure image. The simple combination methods are shown in Fig. 1.

HDR blending using dual LDR images

The conventional HDR blending algorithm uses multi-exposure images for a static scene, but capturing many images of the same scene is difficult because of variations in the viewing conditions, such as tilting, shifting, and moving objects. Therefore, an effective method that uses a small number of images is required. The minimum number of images that can create an HDR image is two (long and short-exposure images). Only two input images have relatively many saturated regions that must be removed

Simulation results

The conventional HDR blending algorithm uses many differently exposed images to estimate an accurate CRF. The proposed HDR blending algorithm uses only two LDR images: short and long-exposure. In order to compare the performance of the conventional and proposed algorithms, we show the CRF graphs of each algorithm and tone-mapped images using an identical tone-mapping algorithm in Fig. 11, Fig. 12. Each algorithm uses only two LDR images.

Fig. 11(a) and (b) represents the input images used to make

Conclusions

This paper proposed a novel HDR rendering process. The proposed HDR blending algorithm is based on the least squares method using dual input images (long and short-exposure images for a scene). Two input images have relatively many saturated pixels, which make it difficult to estimate an accurate CRF. As a result, color distortion and noise appear in the conventional HDR blending algorithm. To create an accurate CRF, we proposed spatial and intensity weighting functions. The spatial function is

Acknowledgement

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2015R1D1A1A01059929).

References (22)

  • G. Healey et al.

    Radiometric CCD camera calibration and noise estimation

    IEEE Trans. PAMI

    (1994)
  • Cited by (10)

    View all citing articles on Scopus

    This paper has been recommended for acceptance by Dr. Zicheng Liu.

    View full text