Nonuniformity Correction Method of Thermal Radiation Effects in Infrared Images

Abstract

This paper proposes a correction method based on dark channel prior. The method takes full advantage of the sparseness of dark channels of latent images. It applies an L0 norm constraint of dark channels to the latent images, and an L2 norm constraint with smoothing gradient to the intensity bias field caused by the aero-optic thermal radiation effects. And finally it adopts split Bregman method to solve the nonconvex and nonlinear optimization problem. The experimental results show that compared with the existing methods, this method greatly reduces aero-optic thermal radiation effects in infrared imaging detection system.

1 Introduction

When aircraft with optical imaging detection system flies at high speed, the air density around its cap has changed drastically because of the intense interaction between the cap and the air flow. Meanwhile, the gradient of atmospheric refractive index in the mixing layer has also changed. The fluctuation of atmospheric refractive index and high temperature will cause distortion and heat of the optical window, and make the target image produce the phenomena such as pixel bias, phase jitter, shake and blur, a phenomenon known as aero-optical effects [1, 2]. Because of the strong thermal radiation effect, the details of the over-saturated image become unintelligible and the detection performance is seriously affected. There are several measures that could be taken to reduce the aero-optic thermal radiation effects: (1) selecting appropriate detector angle, spectral bandwidth, detector integration time and other detector parameters [3], (2) limiting the window temperature to a lower range [4]. All these measures can physically reduce the aero-optic thermal radiation effects. However, these correction and processing measures are only partial and insufficient, since, on the one hand, they are complex in construction, expensive in cost, and inconvenient in maintenance; and, on the other hand, they can only be applied to the correction of thermal radiation effects under limited conditions. Therefore, due to the urgent need of economy, applicability and detection, it is necessary to further correct the degraded image with thermal radiation effects. In this paper, a post correction method is proposed to improve the infrared image quality against the aero-optic thermal radiation effects.

A few researchers have studied the principles and methods of correcting aero-optic thermal radiation effects. In literature [5], the authors analyzed and modeled the aero-optic thermal radiation in air according to the experimental data. Cao and Tisse construct a correction model by fitting the derivatives of bivariate polynomial to the gradient information of infrared image, and correct the aero-optic thermal radiation effects of an uncooled long wave infrared image acquisition camera from a single image [6]. Liu modeled the low-frequency intensity bias field as a representation of the bivariate polynomial representation and estimated it by using an isotropic total variation model [7]. In literature [8], the authors noted that infrared images usually have smaller targets, and proposed a variational model based on L₀ regularization, using prior knowledge, to complete the nonuniformity correction depending on optical temperature. And yet, for these correction methods, intensity bias field is considered to be the representation of K degree bivariate polynomials, which makes their calculation more than necessarily complex. In addition, the correction of aero-optic thermal radiation effects is not significant enough, so there is still much room for improvement.

In this paper, by comparing clear image and infrared image with aero-optic thermal radiation effects, it is found that the dark channel of infrared images with thermal radiation effects is globally bright, while the dark channel of clear images is generally dark. This means that value zero is in the majority of the dark channel of the infrared image without aero-optic thermal radiation effects.

2 Intensity Nonuniformity Correction Method Based on Dark Channel Prior

Dark channel was first introduced by He in image dehazing. He concluded that in most local non-sky areas, some pixels always have at least one color channel with lower values based on statistics of a large number of hazy images and haze-free images [9]. In this section, dark channel prior is introduced into the correction model of aero-optic thermal radiation effects for the first time, and then split Bregman method [10] is used to solve the nonconvex nonlinear minimization problem and the intensity bias field estimation is obtained. The intensity bias field is substracted from the degraded image to eliminate the aero-optic thermal radiation effects.

Since the intensity bias field caused by aero-optic thermal radiation effects is additive and varies smoothly, the general model of aero-optic thermal radiation effects degradation can be expressed as:

$$ z = f + b + n $$

(1)

Where z denotes an observed infrared image with aero-optic thermal radiation effects, f denotes a latent clear image (without aero-optic thermal radiation effects), b denotes intensity bias field induced by aero-optic thermal radiation effects, and n denotes the system noise.

For a color image f, the dark channel is defined as follow

$$ D\left( f \right)\left( x \right) = \mathop {\hbox{min} }\limits_{{y \in {\rm N}\left( x \right)}} \left( {\mathop {\hbox{min} }\limits_{{c \in \left\{ {r,g,b} \right\}}} f^{c} \left( y \right)} \right) $$

(2)

Where x and y denote the position of pixels, $N(x)$ denotes an image block centered on pixel x, $f^{c}$ denotes c color channels, and the dark channel is the minimum pixel value of the three channels. The clear image and degraded image with aero-optic thermal radiation effects both are gray-scale images, we have $\mathop {\hbox{min} }\limits_{{c \in \left\{ {r,g,b} \right\}}} f^{c} \left( y \right) = f\left( y \right)$. The dark channel prior is used to describe the minimum values in neighbourhood. It is found that dark channel of degraded image with aero-optic thermal radiation effects is less sparse. The elements of dark channel for clear image are almost zero and dark channel for degraded image with aero-optic thermal radiation effects has fewer zero elements.

2.1 Intensity Nonuniformity Corrected Model

Adding dark channels prior, we present a correction model of aero-optic thermal radiation effects:

$$ \mathop { \hbox{min} }\limits_{f,b} \left\| {z - f - b} \right\|_{2}^{2} + \alpha \left\| {\nabla f} \right\|_{0} + \beta \left\| {D\left( f \right)} \right\|_{0} + \gamma \left\| {\nabla b} \right\|_{2}^{2} $$

(3)

where regularization parameters α, β, γ are positive values. The degraded images are often contaminated by Gaussian noise. Suppose that n is Gaussian noise, then we can fit the aero-optic thermal radiation effects correction model by an L2 norm shown in the first term of functional (3). We use the second term to constrain the gradient of latent clear image, where $\nabla f = \left( {f_{x} ,f_{y} } \right)^{\prime }$. The nonzero values of ∇f in real infrared aero-optic thermal radiation effects images are denser than in clear images. Therefore, the gradient of a clear image and a degraded image can be differentiated by an L₀ norm $\left\| {\nabla f} \right\|_{0}$ which counts the number of nonzero values of ∇f. We use the fourth term to constrain the gradient of the intensity bias field, where $\nabla b = \left( {b_{x} ,b_{y} } \right)^{\prime }$ is the first-order spatial derivative, and we use the gradient prior to penalize high-frequency components of b. Since parameter γ controls the gradient regularization strength, the noise will be not well suppressed if γ is too small; however, if γ is too large, the edge and detailed information will be covered by aero-optic thermal radiation effects. The third term we use in this model is the dark channel prior term with the L₀ norm representation. Because the infrared image is a gray image, and the three color channels of the image are the same, the dark channel representation can be obtained just by calculating the minimum value of a single channel. The dark channel corresponding to clear images without aero-optic thermal radiation effects is usually dark, with almost no information, and 0 values are the majority. The dark channel corresponding to images with aero-optic thermal radiation effects is usually bright and the non-zero values are the majority. Therefore, the dark channel corresponding to the clear image without aero-optic thermal radiation effects is sparse, and the norm L₀ is used to represent its sparsity.

2.2 Intensity Bias Field Estimation

We use the split Bregman method to facilitate the solution. The variables d₁, d₂ and the auxiliary variables b₁, b₂ are introduced to rewrite problem (3) as an unconstrained optimization problem:

$$ \mathop { \hbox{min} }\limits_{{f,b,d_{1} ,d_{2} ,b_{1} ,b_{2} }} \left\| {z - f - b} \right\|_{2}^{2} + \alpha \left\| {d_{1} } \right\|_{ 0} + \beta \left\| {d_{2} } \right\|_{0} + \gamma \left\| {\nabla b} \right\|_{2}^{2} + \gamma_{1} \left\| {d_{1} { - }\nabla f{ - }b_{1} } \right\|_{2}^{2} + \gamma_{2} \left\| {d_{2} { - }D\left( f \right){ - }b_{2} } \right\|_{2}^{2} $$

(4)

where regularization parameters $\gamma_{ 1}$, $\gamma_{ 2}$ are positive values. Since D(f) is a nonlinear operator, it is difficult to solve the minimization of f in formula (4). We therefore use a linear operator M to transform it [11]. Let $y = \arg \min_{{q \in {\rm N}\left( x \right)}} f\left( q \right)$, then the linear operator M is defined as:

$$ M\left( {x,q} \right) = \left\{ {\begin{array}{*{20}l} {1,} \hfill & {q = y,} \hfill \\ {0,} \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right. $$

(5)

For a true clear image, Mf = D(f) strictly holds. The approximation of M is computed using the intermediate clear image at each iteration. As the intermediate clear image becomes closer to the true clear image, the linear operator M approximates to the desired nonlinear operator D. Given the linear operator M, the minimization problem in regard to variable f can be rewritten as:

$$ \mathop {\hbox{min} }\limits_{f} \left\| {z - f - b} \right\|_{2}^{2} + \gamma_{1} \left\| {d_{1} { - }\nabla f{ - }b_{1} } \right\|_{2}^{2} + \gamma_{2} \left\| {d_{2} { - }Mf{ - }b_{2} } \right\|_{2}^{2} $$

(6)

Problem (6) is an $L_{2} - L_{2}$ norm form minimization problem. It is a quadratic function. Let the partial derivative of the energy function f equal to 0, and we get a closed form solution of problem (6). Then, according to the Parseval’s theorem, the solution of f can be easily obtained in frequency domain:

$$ f = \frac{{F^{ - 1} \left( {F(z{ - }b) + \gamma_{1} F\left( {\nabla^{T} \left( {d_{1} { - }b_{1} } \right)} \right) + \gamma_{2} F\left( {M^{T} \left( {d_{2} { - }b_{2} } \right)} \right)} \right)}}{{1 + \gamma_{1} \nabla^{T} \nabla + \gamma_{2} M^{T} M}} $$

(7)

where $F\left( \cdot \right)$ and $F^{ - 1} \left( \cdot \right)$ denote FFT and inverse FFT respectively. Given f, the minimization problem of variable d₁ becomes:

$$ \mathop { \hbox{min} }\limits_{{d_{1} }} \alpha \left\| {d_{1} } \right\|_{0} + \gamma_{1} \left\| {d_{1} { - }\nabla f{ - }b_{1} } \right\|_{2}^{2} $$

(8)

The solution of the variable d₁ is obtained based on [12].

Similarly, given f, the minimization problem of variable $d_{ 2}$ becomes:

$$ \mathop { \hbox{min} }\limits_{{d_{2} }} \beta \left\| {d_{2} } \right\|_{0} + \gamma_{2} \left\| {d_{2} { - }D\left( f \right){ - }b_{2} } \right\|_{2}^{2} $$

(9)

The auxiliary variables b₁ and b₂ are updated to be:

$$ \begin{aligned} b_{1} = b_{1} + \nabla f - d_{1} \hfill \\ b_{2} = b_{2} + D\left( f \right) - d_{2} \hfill \\ \end{aligned} $$

(10)

Given f, intensity bias field b is updated as follows:

$$ \mathop {\hbox{min} }\limits_{b} \left\| {z - f - b} \right\|_{2}^{2} + \gamma \left\| {\nabla b} \right\|_{2}^{2} $$

(11)

This is an $L_{2} - L_{2}$ norm form minimization problem. Intensity bias field b can be obtained by Euler-Lagrange linear equation:

$$ b = \frac{z - f}{{1 + \gamma \nabla^{T} \nabla }} $$

(12)

In the previous section, the intermediate latent sharp image f is estimated, which is in turn used to estimate exactly the intensity bias field b. And the final clear image after correction is $z - b$.

3 Experimental Results and Analysis

To illustrate the efficiency of the proposed method for the correction of simulating aero-optic thermal radiation effects, the proposed method is compared with the current state-of-art correction methods, Cao’s method [6] and Liu’s method [7]. We take the simulated degraded images with aero-optic thermal radiation effects from the Liu’s reference. The three images in the first row at the top of Fig. 1 are the aero-optic thermal radiation effects images obtained during flight. Images in the second, third, and fourth rows in Fig. 1 are the corresponding images obtained with Cao’s method and Liu’s method, and our correction methods for the aero-optic thermal radiation effects respectively. By comparison, although Cao’s method and Liu’s method can reduce aero-optic thermal radiation effects and both of them have a good correction effect, whereas theirs still have residual aero-optic thermal radiation effects. The results of our method seem to be more homogeneous and better, because our method completely eliminates the aero-optic thermal radiation effects and has better image details.

Fig. 1.

Comparison of simulation methods for correction of aero-optic thermal radiation effects.

The first line in Fig. 2 is the intensity bias field map obtained by our proposed method. The second line in Fig. 2 is 3D display. It can be seen that it is a good indication of the aero-optic thermal radiation effects. In addition, variance coefficient [13] (cv(f) = variance(f)/mean(f)) is used to evaluate the correction performance of three methods quantitatively, where variance(f) is the variance of image f, and mean(f) is the mean of image f. The lower the variance coefficient value, the better the image quality. As shown in Table 1, our method has a lower variance coefficient.

Fig. 2.

Intensity bias field map and 3D display.

Table 1. Comparison of CV values of three correction methods.

Figure 3(a) is a real infrared aero-optic thermal radiation effects images obtained by infrared window heating test, wherein the experimental temperature of Fig. 3(a) is 599 K, Fig. 3(b) is the correction result of the Cao’s method, Fig. 3(c) is the correction result of Liu’s method, and Fig. 3(d) is the correction result of our proposed method. By contrast, the background of our method’s corrected image is smoother and the target area is more clearly visible.

Fig. 3.

Aero-optic thermal radiation correction experiment under window heating (window temperature is 599 K).

In the window heating experiment, the pixel gray value distribution of the real aero-optic image with aero-optic thermal radiation effects, correction result of Cao’s method, correction result of Liu’s method, correction result of our proposed method are shown in Fig. 4. Figure 4(a) and (c) are pixel gray value distribution of the 160th column and 200th row from Fig. 3. Figure 4(a) shows that the pixel gray value distribution has three peaks, the regions with larger pixel gray values are located at three target points, and the background pixel gray values are low, which is consistent with the actual situation.

Fig. 4.

Pixel gray value distribution (Fig. 3).

4 Conclusion

In this paper, a correction method based on the dark channel prior is proposed to reduce the aero-optic thermal radiation effects. By comparing the degraded images with aero-optic thermal effects and clear images, the dark channel prior constraints of latent images is added to the additive correction model, the L₂ norm constraint is applied to the gradient of the intensity bias field, and a correction model of aero-optic thermal radiation effects based on dark channel is established. In order to solve the nonconvex and nonlinear optimization problem, the split Bregman method is introduced into the proposed model. The minimum problem of multi-variables is split into a series of single variable minimum problems, which accelerates the iterative convergence. The experimental results show that, compared with the state-of-art methods, this method can obtain better aero-optic thermal radiation correction results.