Keywords

1 Introduction

Outdoor images captured under foggy weather condition are often seriously degraded by the turbid medium (e.g., dust, water-droplets) in the atmosphere. And the distant objects in the degraded images lose the color fidelity and become blurred with their surrounding, as demonstrated in Fig. 1(a). This problem seriously reduces the performance of outdoor computer vision processing systems. Thus, effective and robust dehazing (or haze removal) methods are strongly desired in both computational photography and computer vision applications [1, 2], such as outdoor surveillance, intelligent driving, and satellite remote sensing system.

Fig. 1.
figure 1

Image dehazing by presented approach. (a) Input hazy image. (b) Dehazed image. (c) Estimated heterogeneous atmospheric light. (d) Estimated medium transmission.

Full size image

Available methods for image dehazing are broadly classified as image enhancement based approaches and image restoration based schemes [3]. The approaches based on image enhancement employ image processing algorithms to restore the dehazed image, enhance the visibility of a haze image, and highlight the valuable information in a scene. The advantage of this kind of method is that it is very convenient with the commonly used image enhancement algorithms. However, the recovered image usually suffers from significant halos and distorted colors. In order to recover high-quality dehazed images, the schemes based on image restoration build a atmospheric scattering model based on the scattering effect of atmospheric aerosol particles on the light. This is a special kind of haze removal method based on the physical mechanism of image degradation, thus the restored image is more photo-realistic, and the details information are also preserved well.

Early haze removal methods based on image restoration [4,5,6,7] usually require multiple images or additional information to be available. Though these methods can ameliorate the visibility of hazy images, they cannot be employed in applications where additional information or multiple images are not obtainable. Therefore, single image haze removal has been a hot spot of research given its wider application range [8]. Recently, a lot of progresses have been made since the introduction of single image haze removal. These image dehazing algorithms [1, 2, 9,10,11,12,13,14] remove the haze under certain priors or assumptions, such as dark channel prior (DCP) [2], and then recover the haze-free image with a haze model. And the advantage of them is to demand only a single input image.

Image dehazing mainly includes two tasks, estimating the atmospheric light and calculating the medium transmission, as shown in Fig. 1(c) and (d), respectively. Inaccurate estimation of the atmospheric light can incur unwanted color shift. As a result, the haze-free image cannot be obtained visually pleasing. Therefore, the estimation of the atmospheric light is the key technique for image dehazing. The homogeneous atmospheric light estimation considers the light both absorbed and scattered by turbid mediums, so it is widely applied in image dehazing. In [10], the brightest pixel of the entire hazy image was regarded as the atmospheric light. And in [2], the top 0.1% brightest pixels in the dark channel were first picked up, and then the one with the highest intensity was selected as the estimation of the atmospheric light. [12] filtered each color channel of an input image using a minimum filter, and then the maximum value of each color channel was estimated as the atmospheric light.

These methods mentioned above simply approximate the atmospheric light under an assumption that the whole scene has the same value for the atmospheric light. Namely, the atmospheric light within an entire image is homogeneous. This assumption leads to that many state-of-the-art algorithms for dehazing get much darker dehazed images, especially objects with dark color in scenes. In fact, the atmospheric light is not a valid one when there are multiple point light sources such as the sun, street lights, or vehicle headlights at night. Hence, these existing approaches often fail by mistakenly taking white objects (e.g., clouds) as the atmospheric light. Meanwhile, the amount of scattering depends on the distances of the scene points from the camera, and the degradation varies spatially [2].

In order to efficiently and robustly remove haze from a single input image, an improved dehazing scheme based on the heterogeneous atmospheric light estimation is presented in this paper. The main works are as follows. Firstly, we think that an entire image has not uniform value for the atmospheric light. In other words, the atmospheric light within an image is heterogeneous. And then, the heterogeneous atmospheric light can be obtained with max-pooling. Finally, combining the estimated heterogeneous atmospheric light and a dark channel prior dehazing method, we can recover a haze-free image with faithful colors and fine image details. As a result of our experiments, the haze-free image restored by the developed scheme achieves more satisfying image quality at an acceptable algorithm cost compared with state-of-the-art approaches.

The rest of this paper is organized as follows. The atmospheric scattering model is treated in Sect. 2. The improvements for dehazing is addressed in Sect. 3. Experimental results are shown in Sect. 4. Finally, we conclude the whole paper in Sect. 5.

2 Atmospheric Scattering Model

The atmospheric scattering model (ASM) is the basic model of image dehazing [1]. To address the formation of a hazy image, the atmospheric scattering model is first presented by McCartney in [15], and further developed by Narasimhan and Nayar [4, 5] later. When the atmospheric light is homogenous, the model can be defined as follows:

$$\begin{aligned} I(x) = J(x)t(x) + A(1 - t(x)) \end{aligned}$$
(1)

where x is the position of the pixel within the image, I(x) is the observed intensity representing the hazy image, J(x) is the scene radiance describing the haze-free image. \(\mathrm{{A}}\) denotes the atmospheric light, and t(x) indicates the medium transmission representing the portion of the light that is not scattered and reaches the camera. The goal of image dehazing is to estimate \(\mathrm{{A}}\) and t(x), and then recover J(x) from I(x) based on (1).

3 Improvements for Dehazing

To create a photo-realistic haze-free image, two improvements to the DCP approach [14] are given. One is to produce the heterogeneous atmospheric light estimation, which accurately describes how the amount of scattering from one or multiple point light sources is spatial-variant. The other is to decide how a dehazed image is recovered efficiently. Therefore, the two improvements for the addressed dehazing method are discussed in this section.

3.1 Heterogeneous Atmospheric Light Estimation

The estimation of the atmospheric light is a fundamental and challenging problem for the haze removal [16]. Under the assumption of homogeneous atmospheric light, the darker regions in these images become very dark after haze removal and some important details in the images are often lost. Therefore, the assumption of homogeneous atmospheric light does not hold in most cases. Besides, the occlusion of big objects to the light source and different albedos or absorptivities of different objects in a scene can incur the heterogeneous atmospheric light. Based on these observations, we assume that the atmospheric light is heterogeneous in whole scene. As a result, we present a novel solution which is useful for estimating heterogeneous atmospheric light from a single hazy image directly.

According to (1), the heterogeneous ASM is formulated by:

$$\begin{aligned} I(x) = J(x)t(x) + A(x)(1 - t(x)) \end{aligned}$$
(2)

where A(x) is the heterogeneous atmospheric light, and is dependent on the position x in a scene. The difference between (2) and (1) is that A(x) is a variable, not a constant. Actually, (1) can be regarded as a special case of (2) when the A(x) is a constant.

Additionally, in the ideal case, the medium transmission t(x) could be zero as the scenery objects that appear in the image could be very far from the camera, and we have:

$$\begin{aligned} I(x) = A(x), \ t(x) \rightarrow 0 \end{aligned}$$
(3)

Equation (3) shows that the intensity of the pixel can represent the value of the atmospheric light A(x) when t(x) tends to zero.

Therefore, A(x) can be calculated by I(x) approximately when the medium transmission t(x) tends to be very small. When t(x) becomes very small, all of the light radiated from objects in a scene are almost scattered and cannot reach the camera. Hence I(x) becomes nearly haze-opaque. Under this circumstance, I(x) can be taken as the approximation of the heterogeneous atmospheric light A(x).

Inspired by this idea, we propose a novel method for the estimation of heterogeneous atmospheric light. Firstly, in order to make the haze in the input image become white color, the white balance operation is conducted on the input hazy image I(x). Secondly, the max-pooling [17] is performed on the output image of white balance to obtain a small heterogeneous atmospheric light. Here the size of the window for max-pooling is \(\mathrm{{N}} \times \mathrm{{N}}\). And then, a guided image filtering [14] strategy is employed to suppress the block artifacts incurred by the max-pooling. Finally, according to the size of original hazy image, a bicubic interpolation way is used to enlarge the small heterogeneous atmospheric light, and then the heterogeneous atmospheric light A(x) can be obtained, as shown in Fig. 1(c).

3.2 Dehazing with Heterogeneous Atmospheric Light Estimation

Image dehazing is essentially under-constrained problem if the input is a single haze image. In order to handling this problem, the explore for additional priors or constraints are generally required. Dark channel prior (DCP) for single image dehazing [2] is a better solution compared with most haze removal methods. Therefore, we estimate the medium transmission t(x) based on (1). Under the DCP assumption, referring to (1), we easily derive the medium transmission t(x) as follows:

$$\begin{aligned} t(x) = 1 - \mathop {\min }\limits _{y \in \varOmega (x)} \left( {\mathop {\min }\limits _{c \in \{ R,G,B\} } \frac{{{I^c}(y)}}{{{A^c}}}} \right) \end{aligned}$$
(4)

where \({I^c}\) is a color channel of I, \(\varOmega (x)\) is a local patch center at x, \(\mathop {\min }\limits _{c \in \{ R,G,B\} }\) is conducted on each pixel, and \(\mathop {\min }\limits _{y \in \varOmega (x)} \) is a minimum filter. Hence we can directly obtain the estimation of t(x). To refine the medium transmission t(x), a guided image filtering is used to smooth the image.

The aim of image dehazing is to recover the scene radiance J(x). After finishing the estimation of the heterogeneous atmospheric light A(x) and the medium transmission t(x), referring to (2), the scene radiance can be restored by:

$$\begin{aligned} J(x) = \frac{{I(x) - A(x)}}{{\max (t(x),\varepsilon )}} + A(x) \end{aligned}$$
(5)

where \(\varepsilon \) is a small constant (typically 0.1) for avoiding division by zero. One haze-free image recovered with the developed scheme is illustrated in Fig. 1(b).

4 Experimental Results and Analysis

To confirm the performance of our proposed dehazing method, several experiments are conducted for a set of real-world hazy images. The tests are implemented with the MatlabR2014a and performed on an Intel Core4 3.1 GHz computer that has 8 GB memory. We compare the qualitative results, quantitative ones, and computing time complexity of the developed algorithm with state-of-the-art algorithms, including Tarel’s [11], He’s [14], and Meng’s methods [12].

Figures 2, 3, 4, 5, and 6 demonstrate the qualitative comparison of different results on some real-world images, including ‘Road’, ‘Swans’, ‘People’, ‘Manhattan 2’, and ‘Yosemite 1’. As depicted in Figs. 2(e), 3(e), 4(e), 5(e), and 6(e), the haze-free images recovered by the developed algorithm are much visually pleasing than the other three methods. However, there are several noticeable artifacts in the outputs using the other approaches. As shown in Figs. 2(b), 3(b), 4(b), 5(b), and 6(b), the restored images by Tarel’s method often suffer from distorted colors and significant halos. And the recovered images by He’s algorithm are too dark as depicted in Figs. 2(c), 3(c), 4(c), 5(c), and 6(c). While Meng’s scheme often tends to produce some geometric distortion and some hazes still remain in the restored images as illustrated in Figs. 2(d), 3(d), 4(d), 5(d), and 6(d).

Fig. 2.
figure 2

Qualitative comparison of different results on real-world images of ‘Road’ (a) Input hazy images. (b) Tarel’s results [11]. (c) He’s results [14]. (d) Meng’s results [12]. (e) Proposed method’s results.

Full size image
Fig. 3.
figure 3

Qualitative comparison of different results on real-world images of ‘Swans’ (a) Input hazy images. (b) Tarel’s results [11]. (c) He’s results [14]. (d) Meng’s results [12]. (e) Proposed method’s results.

Full size image
Fig. 4.
figure 4

Qualitative comparison of different results on real-world images of ‘People’ (a) Input hazy images. (b) Tarel’s results [11]. (c) He’s results [14]. (d) Meng’s results [12]. (e) Proposed method’s results.

Full size image
Fig. 5.
figure 5

Qualitative comparison of different results on real-world images of ‘Manhattan 2’ (a) Input hazy images. (b) Tarel’s results [11]. (c) He’s results [14]. (d) Meng’s results [12]. (e) Proposed method’s results.

Full size image

To compare with the other three approaches quantitatively, we use the visible edge gradient [18] metrics. The metrics include the percentage of new visible edges e, the normalized gradient mean of visible edges r, and the percentage of saturated black and white pixels \(\sigma \). The value of e evaluates the ability of the method to restore edges which were not visible in the hazy image I(x) but are in the restored image J(x). The value of \(\overline{r} \) expresses the quality of the contrast restoration. The value of \(\sigma \) measures contrast by detecting the number of saturated black and white pixels.

The metrics are separately defined as follows:

$$\begin{aligned} e = \frac{{{n_r} - {n_0}}}{{{n_0}}} \end{aligned}$$
(6)
$$\begin{aligned} \overline{r} = \exp \left( {\frac{1}{{{n_r}}}\sum \limits _{{P_i} \in {\psi _r}} {\log {r_i}} } \right) \end{aligned}$$
(7)
$$\begin{aligned} \sigma = \frac{{{n_s}}}{{w \times h}} \end{aligned}$$
(8)

where \({n_0}\) and \({n_r}\) denote the cardinal numbers of the set of visible edges in the hazy image I(x), respectively in the contrast restored image J(x). \({\psi _r}\) represents the visible edges set in J(x), \({P_i}\) is the pixel of the visible edge, and \({r_i}\) indicates the Sobel gradient ratio between \({P_i}\) and I(x). \({n_s}\) is the number of the saturated black and white pixels, w and h are separately the width and the height of I(x).

Tables 1 are some results of quantitative measurements. The notations employed to describe the metrics are the same as that used in [17]. Obviously the proposed method has shown better performance in most cases. It is reasonable that the presented scheme can properly describe that the amount of scattering from one or multiple point light sources is spatial-variant. Therefore the heterogeneous atmospheric light can be estimate accurately, and the haze-free image of high quality with faithful colors and fine edge details can be obtained.

To test the complexity of the discussed approaches, the average computing time for the algorithms is evaluated for the tested datasets. The computing time complexity is shown in Table 2. Because the He’s method has the better use of stronger priors or assumptions for image haze removal, it has the lowest computation complexity. Meng’s scheme involves time consuming for iterative operations to calculate the medium transmission map. Due to applying standard median filter to obtain a satisfactory dehazed image quality, Tarel’s method has the highest computation complexity. The proposed algorithm has to employ extra operations to obtain the heterogeneous atmospheric light map, and hence demands more run-time. Comparing with the other three methods, the computing time complexity of the treated way is moderate.

According to these results, the developed methods can work efficiently with only minor increase of the computational complexity.

Fig. 6.
figure 6

Qualitative comparison of different results on real-world images of ‘Yosemite 1’ (a) Input hazy images. (b) Tarel’s results [11]. (c) He’s results [14]. (d) Meng’s results [12]. (e) Proposed method’s results.

Full size image
Table 1. Quantitative comparisons
Full size table
Table 2. Computing time
Full size table

5 Conclusions

In this paper, an improved single image dehazing scheme based on heterogeneous atmospheric light estimation is developed to get a better visual quality of haze-free images. The improvements include two things. One is the heterogeneous atmospheric light can be estimated by max-pooling. The other is a haze-free image is recovered with the obtained atmospheric light. Because the derived atmospheric light can be adjusted adaptively according to the input image brightness, some encouraged results are obtained. Experiment results confirm the efficiency of the presented algorithms with only minor increase of the computing time complexity. In the future, more efficient medium transmission estimation is a very important research topic.