Keywords

1 Introduction

In recent years, more and more dust weather has seriously affected images quality. With the rapid development of artificial intelligence, the identification of dust images still constraint the increasing of self-driving cars. So how to enhance dust images is an important problem need to be solved.

Dust weather means wind raise a large number of dust into the air. Massive solid particles make the air turbid that reduced the visibility. Because of increasing dust particles in the air, the scattering effect of light is enhanced. So the color of dust images is yellowing, brightness and clarity is obviously reduced.

Currently, dust image enhancement algorithms can be divided into two types: one is improved defog algorithm [1,2,3] because dust images and fog images also caused by light scattering and reflection from various particles in the air [4,5,6]. These algorithms tend to ignore the difference between dust and fog that dust is a solid particle with color. Hence, this kind of methods cannot resolve the problem of hue yellowing and low brightness in dust images. The other type is using different ways to work out different questions in dust images [7,8,9]. We can correct color cast, increase light and improve definition separately. These algorithms solved the basic problem of dust image and get a good result. But the implementation process of these methods always bring in a large number of parameters, thus the application range of dust image enhancement algorithms is limited.

Due to shortcomings of traditional algorithms, we proposed a dust image enhancement algorithm based on color transfer. Firstly, applying scene gist feature [10] to select the target image which has the highest similarity scene with the input image; then, the color transfer algorithm [11] is used to transfer the color information in target image to input image; finally, utilizing the contrast limited adaptive histgram equalization algorithm (CLAHE) [12] to restore the definition in dust image. We also design a index to test the effect of our result. Important intermediate results of this algorithm are show in Fig. 1.

Fig. 1.
figure 1

Important intermediate result images of this algorithm. (a) input image, (b) target image, (c) image after color transfer, (d) result image.

Full size image

2 Related Works

Bertalmío [4] proposed a Retinex algorithm based on kernel function which has better reinforcing effect for dust images. However, this method cannot effectively solve the problem about hue yellowing in dust images. Yan [5] use partially overlapped sub-block histogram equalization to improve dust images. This algorithm effectively get a better contrast in the enhancement image. But this method result will bring hue distortion. Li [8] using dark channel prior proposed a single image restoration algorithm. This algorithm improves dust image clarity, but the brightness of result image is dim. Above algorithms are improving defog algorithm to enhance dust images. However, these ways only resolve the question of contrast in dust images, while ignoring the color correction of image enhancement.

In order to obtain better image enhancement effect on dust images, many scholars began to solve the hue yellowing, low brightness and low-quality definition separately. Jiang [7] proposed a dust image enhancement algorithm base on gray world and dark channel. This method effectively improve dust images contrast and clarity but it also has more serious color problems. Ning et al. [13] proposed a new algorithm based on hue adjustment and contrast enhancement. Though this method use Gauss model to adjust the color in dust images and singular value decomposition to enhance the dust image quality. It also bring too many parameters, which leads a bad effect on images that have heavy dust.

Some of the above mentioned algorithms just pay attention to the effect of dust image clarity enhancement, while ignoring color problems in dust image. Although other algorithms solve color and clarity questions at the same time. They bring many parameters in calculation process that have a bad influence on the applicability of this algorithm. Therefore, we proposed a dust image enhancement algorithm based on color transfer, which obtaining a better result and reducing the number of parameters.

3 Proposed Approach

3.1 Image Matching Based on Scene Gist Feature

How to choose a clearer image which has similar scene consistent with the input image scene is a key problem of the proposed algorithm. A good target image not only proved the effectiveness of scene matching algorithm, but also a similar scene can get better result in process of color transfer. In order to solve this problem, the scene gist feature [10] is used to select the target image. The image matching process based on the scene gist feature [10] is shown in Fig. 2.

Fig. 2.
figure 2

Image matching based on scene gist

Full size image

Scene gist [10] is a feature that is commonly used in scene classification. In our method, we use it to select target image have the same scene with input image. The feature can be used to determine the semantic attributes of the scene according to low-level features and rough spatial structure of the scene.

Firstly, extract gist feature from input image and database images. Because scene gist is extracted by multiple low-level feature in scene images (such as spatial frequency, color and texture informations) to create structure about whole scene. Before we select target image which has similar scene with dust image, we should remove the effect of color feature on image scenes. Therefore, when we extracting scene gist feature in image, the multi-scale and multi-direction gabor filter is adopted to extract the texture features in images. This way can reduce the influence of color features on matching process. Secondly, the filtered image is divided into \(4*4\) blocks. The global feature information of the image is extracted from each block to get scene gist. Next, the euclidean distance about scene gist between input image and database images is calculated. Finally, the image has the smallest euclidean distance in database is the target image.

Target image is selected in a clear images database. So if the database has large number of clear, normal and different scene images are important. So far, we do not find any dust images matching database. According requirement above, we designed a clear images database. Using this database, our method get a good result. In this paper, database design process will be described in Sect. 4.1.

3.2 Color Transfer Model

In 2001, Reinhard et al. [11] proposed a method about color transfer between two different images. The method effectively passes the color of the target image to the input image in CIELab color space. In this way, we descend parameters number in processing procedure and the application scope of our algorithm is extended. This method is decomposed into three steps:

Step 1: In order to removing correlation between different color channels, we convert the target image from RGB color space to CIELab color space.

Step 2: We calculate the mean and variance of each color channel in the target image. Continue doing this work for input image. In this step, mean represents images detail features and variance represents the hue information of the image.

Step 3: According (1), we restore the input image;

$$\begin{aligned} I_{k}=\frac{\delta ^{k}_{t}}{\delta ^{k}_{s}}(S^{k}-mean(S^{k}))+mean(T^{k}),\qquad k\in (l,a,b) \end{aligned}$$
(1)
Fig. 3.
figure 3

Color transfer process. (a) input image, (b) target image, (c) image after color transfer. Second line images are the corresponding images of RGB. (Color figure online)

Full size image

In this model, S is the input image, T is the target image, \(\delta _{s}\) and \(\delta _{t}\) is the variance of the input image and target image; mean(S) and mean(T) is the mean of the input image and target image; \(k\in (l,a,b)\) represents three color channels. The main process are shown in Fig. 3.

From the second line in Fig. 3, we can see that the input image RGB is not uniform. After color transfer operation, the result image RGB channel are consistent distribution. It means no color cast in image. In summary, through color transfer we could effectively solve the problem about hue yellowing and low brightness in dust image.

3.3 CLAHE Algorithm

We can find from Fig. 3 that the image after color transfer can solve the color problems in dust images. But problem still exist in result images about low-quality in definition. In order to solve this problem, we utilizing CLAHE algorithm [12] to enhance contrast of the image and finally get a clear result image. CLAHE algorithm [12] has two stages, first we calculate the local histogram of the image. Second, we change the image contrast by redistributing the brightness after set the contrast range. This algorithm can improve the local contrast of the image. Through this way, we can get more image details and raise the image definition.

4 Experimental Results and Analysis

4.1 Database

The purpose of this algorithm is to enhance the dust image to get a clear image. In order to ensure the practical application of our algorithm, we should restrict database images. In the database, images which have reflection of light, shadow, color cast and a large area have same color should remove to avoid the bad affect about dust image enhancement. Because the dust image mainly exist in cities, streets and other artificial scene images, we select these kinds of clear images to get a good match result.

According to above principles, we choose artificial scene images in Oliva and Torralba’s database in MIT [14] and Li’s database [9] first. Then, remove ineligible images. Finally, we get a clear images database which have 1898 images in total. The color image size in this database are \(256*256\) (Fig. 4).

Fig. 4.
figure 4

Database image and ineligible image examples. (a) examples of database image, (b) examples of ineligible image. (Color figure online)

Full size image

4.2 Subjective Experiment Results of Dust Images

All experiment results that we shown in this paper are obtained under the same computer environment (Intel (R), Core (TM), i5-4590CPU@3.30 GHz, 8.00 GB, RAM).

The subjective visual effects that we compared with the Retinex algorithm [4], Jiang algorithm [7], and original image, are shown in Fig. 5. It can be seen from this comparison about the dust image enhancement result, not only our method effectively resolve the question about color cast in dust images, but also retains more image details. It proves that our method is beneficial to dust image further processing. From the first image and the fourth image, compared with other two algorithms, it can be seen that our algorithm can improve the brightness of the image. At the same time, as shown in the eighth image, Retinex algorithm and Jiang algorithm dust image enhancement effect is not very ideal, but our algorithm get a better result on enhance severe dust images.

Fig. 5.
figure 5

Subjective results of dust images enhancement. (a) input image, (b) Retinex [4], (c) Jiang [7], (d) our method.

Full size image

4.3 Objective Experimental Results of Dust Images

In this paper, we use three indexes that include local information entropy, average gradient and contrast to evaluate results of dust images enhancement. The specific data are shown in Tables 1, 2 and 3.

Table 1. Local entropy of our method compared with others.
Full size table
Table 2. Average gradient of our method compared with others.
Full size table
Table 3. Contrast of our method compared with others.
Full size table

Image information entropy is usually used to measure the richness of image informations. The average gradient can be used to represent the relative clarity of images. The high contrast is very helpful for image clarity, detail performance and gray level representation. Therefore, these objective quality evaluation indexs can give a comprehensive evaluation to measure the quality of images.

As shown in Table 1, the local information entropy of the proposed algorithm is higher than other two algorithms, which shows that the proposed algorithm preserves more details in dust image. In Table 2, the average gradient of our algorithm is better than other algorithms, it represents our results are more clear than others. In contrast of Table 3, our method result is higher than other algorithms. It shows that our algorithm not only removes the hue deviation of dust images, but also improves the contrast of images greatly and keep images color undistorted. The results of our algorithm are more conducive to the subsequent processing of dust images.

4.4 Simulative Dust Image Experimental Results

In order to further verify the effectiveness of our algorithm, we find four clear images and then use photoshop software to add a mask to adjust image color, contrast and brightness. After these steps, we convert a clear image into a dust image. Then, dust images are processed by Retinex algorithm [4], Jiang algorithm [7], and our algorithm respectively. The final experimental results are shown in Fig. 6.

Fig. 6.
figure 6

Subjective results of simulative dust images enhancement. (a) original image, (b) simulative dust image, (c) Retinex [4], (d) Jiang [7], (e) our method. (Color figure online)

Full size image

As shown in Fig. 6, the Retinex algorithm [4] results are slant in red, Jiang algorithm [7] overall slant in blue, while our algorithm does not exist these problems. To better describe the result about our method, we will evaluate the quality of result images and original images respectively.

In this part, we use an integrated local natural image quality evaluation (IL-NIQE) [15] was proposed by Zhang in 2015. This algorithm which calculate image informations like color, texture, contrast and so on in different databases to train an image quality evaluation model. We use IL-NIQE to evaluate the quality of the original images and result images are shown in Fig. 6 through IL-NIQE. The experimental results are show in Table 4. In this image evaluation model measure result, we can find our algorithm is significantly similar with original images than other algorithms. It also objectively confirmed the effectiveness of the proposed algorithm.

Table 4. IL-NIQE values of our method compared with others.
Full size table

Although we verified the effectiveness of the proposed algorithm from this evaluation model. But this method requires a lot of training data to construction the evaluation model and also expend more time. Because of the restriction of the database, this method cannot generated objective evaluation results for any picture. Therefore, in order to get a simple and effective method to evaluation dust image enhancement result, we proposed a color similarity evaluation index named absolute color difference (ACD). First, we calculate the difference of two images between each color channel. Then, we get absolute value from these difference and finally calculating the mean of the absolute value. The calculation equation of the algorithm is shown in (2).

$$\begin{aligned} M=\frac{1}{3} \sum _{c=1}^{3}mean(|I^{c}_1-I^{c}_2|) \end{aligned}$$
(2)

M is the color absolute difference, c represents R, G, B three channels of images, \(I_{1}\) is the original image, \(I_{2}\) is the result image. It can be seen from (2) that smaller absolute color difference represents smaller color difference between two images. It also means better effect of dust image enhancement.

Fig. 7.
figure 7

Images with different levels of mask. (a) original image. (b) 30% opacity mask image. (c) 60% opacity mask image. (d) 90% opacity mask image. (Color figure online)

Full size image

To verifying the objective of the evaluation criteria, we use photoshop software to add a yellow mask to Fig. 7(a). The opacity of mask are 30%, 60%, and 90%, respectively. Then we use absolute color difference to calculate the color similarity between these pictures with original picture. The results of ACD are 4.89, 9.71 and 14.60, respectively. Hence, we can concluded that with the increase of mask opacity, the absolute color difference is improving, the greater difference with original color. We illustrate the absolute color difference can effectively evaluate image color similarity.

Table 5. ACD of our method compared with others.
Full size table

After verifying the objective validity of the absolute color difference on two images color similarity. We use this index assess images in Fig. 6. The experimental results are shown in Table 5. From the comparison of the absolute color difference between different algorithms, we can find that the algorithm is proposed in this paper is superior to Retinex algorithm [4] and Jiang algorithm [7]. These results further proves the superiority of our algorithm.

5 Conclusions

Aim at the problems of dust image about hue yellowing, low-quality brightness and definition, we put forward a dust image enhancement algorithm based on color transfer. Firstly, we through the scene gist feature select a target image in clear images database which has the highest similarity with input image. Then, we transfer the target image color information to the input image that solve color questions in dust image. Next, we use CLAHE algorithm to increase image definition. Finally, we achieve the purpose of image enhancement for dust. The subjective and objective experimental results show that the proposed algorithm is superior to traditional algorithms. In the future, we will study how to apply this algorithm in more vision tasks such as dust image text detection and recognition.