Elsevier

Neurocomputing

Volume 400, 4 August 2020, Pages 294-308
Neurocomputing

Single image dehazing by approximating and eliminating the additional airlight component

https://doi.org/10.1016/j.neucom.2020.03.027Get rights and content

Abstract

This paper proposes a novel technique for single image dehazing using adaptive nearest neighbor regularization to obtain a haze-free transmission map and then approximating the additional airlight component present in the hazy image. The proposed method relies on the intensity distribution in the small image patches of the image, exhibited from the Y channel of the YCbCr representation of the image, in order to preserve the texture information of the image. We substitute the commonly used soft matting technique in assessing the refined transmission map for haze removal, by adaptive nearest neighbor classifier. We assume that the actual color of the haze-free pixels in the image is approximated by a set of discrete colors. We discover the haze-free pixels using the Nearest-Neighbor (NN) regularization. Finally, unlike the state-of-the-art methods, we approximate the additional airlight present in the image patch and eliminate that to clear the haze, instead of estimating the transmission of the medium. The proposed nearest neighbor regularization technique automatically changes the patch size, which helps in dealing with the high depth region (e.g., sky region) of the image. We experimented on standard synthetic and real hazy image datasets and observed that, the proposed method outperforms the state-of-the-art, especially for images with sky regions.

Introduction

Images captured by the camera, are often affected by haze due to several atmospheric conditions such as fog, smoke, multiple light sources, the scattering of light, etc. Presence of haze in an image obscures the clarity and hence needs to be restored. The flux of light per unit area received by the camera from the scene is attenuated along the line of sight. This redirection of light lessens the immediate scene transmission and replaces with a layer of scattered light known as airlight. Such scattering of light due to rigid particles floating in the air reduces the visibility of the scene. To enhance the quality of images is essential in various applications of computer vision, for instance, object recognition, scene analysis and traffic observation. Notwithstanding, outdoor images inevitably encounter the ill effects of awful climate conditions that straightforwardly cause the degradation of the image quality. The typical effects that result from the haze, fog, mist and smoke incredibly decay the visibility of the image. Haze/mist can massively reduce the color contrast, as appeared in Fig. 1(a). Fig. 1(b) shows the eventual outcome of image dehazing on Fig. 1(a).

Presence of haze in an image obscure the visibility of the image and hence needs to be restored. The flux of light per unit area received by the camera from the scene is attenuated along the line of sight. This redirection of light lessens the immediate scene transmission and replaces with a layer of scattered light known as airlight. Such scattering of light due to rigid particles floating in the air reduces the visibility of the scene. Since the local non-uniform density of haze relies upon unknown scene depth data, dehazing remain an under-constrained issue for a hazy/foggy image. The effect of the scattering at any pixel depends on the depth of the pixel, and the degradation of the image due to haze is spatial-variant.

Traditional strategies for haze removal primarily depend on additional depth information or various multiple observations of a similar scene. The conventional methods exploited multiple images to overcome the problem. For example, the method of Narasimhan and Nayar [1] requires multiple pictures of a similar scene taken under various climate conditions. The method of Shwartz et al. [2] requires images with the various level of polarization. However, images taken in various climatic conditions may be unavailable in reality. Recently researches are attempting for single image dehazing, where reference images for the given hazy images are not required.

Under the assumption that haze-free images contain a considerably higher contrast compared to hazy images, Tan [3] proposed a dehazing approach that maximizes local contrast primarily based on a Markov random field model. Fattal proposed another dehazing technique based on unbiased component analysis under the assumption that surface shading and medium transmission are locally uncorrelated [4]. He et al. [5] proposed a straightforward and effective dehazing technique that depends on the dark channel prior (DCP). The DCP-based approaches produce good result, but cannot handle locales where shading segment does not shift fundamentally contrasted with noise. Another approach was introduced by Fattal in [6], where the color line was introduced for dehazing. The color line based methods approximate transmission map based on the shift of the direction of atmospheric light from the origin. However, both the approaches fail to retain the original color of the objects in the hazy image.

Recently, some deep learning based methods [7], [8], [9], [10], [11], [12] have been proposed, producing much better results compared to the handcrafted features like DCP and color-line. However, in case of images where sky background is present, the deep learning based methods fail to dehaze properly. The amount of haze at high depth area (where distance of the object from the camera is very high, such as sky area) is naturally higher compared to the low depth area of the image. Number of images with a significantly large sky region is very less in the datasets we use. Moreover, the number of patches in the images including sky regions is even less. Hence, being data-greedy techniques, deep learning based techniques fail to learn the amount of haze at high depth regions, due to the lack of adequate patches including high depth regions.

In this paper, we propose a novel approach to address the problem of single image dehazing, preserving the texture information. The proposed method relies on the pixels with moderate intensities of small image patches and exhibit concordant distributions in RGB space. The distribution of color is captured from the Y channel of YCbCr color map to consider the texture. Our second contribution is to substitute the commonly used soft matting technique [13] in assessing the transmission map for haze removal, by assuming that color of the haze-free pixels in the image is approximated by a hundred discrete colors. Based on this assumption we discover the haze-free pixels and apply the Nearest-Neighbor (NN) regularization to predict the value of hazy pixels. The proposed method replaces the traditional softmatting technique for smoothening the transmission map of the image, by NN regularization technique which reduces the execution time drastically. Finally, unlike the state-of-the-art methods, we do not consider and estimate the transmission of the medium. Alternatively, we approximate the additional airlight present in the image patch and eliminate that to clear the haze.

Unlike Ancuti et al. [14], where the concept of estimating the airlight in local image patches was introduced, the proposed method applies the approximation of airlight in local patches of YCbCr representation of images, to preserve the original texture of the image. Moreover, unlike the state-of-the-art methods, we apply NN regularization on the DCP image, to smoothen the DCP. We interject and regularize the fractional evaluations of pixel intensity values into an entire transmission map, and outline an Adaptive Naive Bayes Classification approach to deal with pixels with a significant depth (such as sky region). At the sky regions of the hazy image, the amount of haze becomes uneven due to the atmospheric scattering. The uneven haze leads to estimate an uneven color at the flat sky region. In adaptive filtering scheme, the patch size is automatically increased at the flat regions of the image in order to get information from more pixels in the neighborhood, which results in estimation of flat intensity values at the flat sky regions. Contrary to the conventional field models which comprise of normal coupling between adjacent pixels, we settle the transmission in isolated locales by fluctuating the number of pixels. Specifically, the proposed adaptive filter uses smaller patches at textured regions of the image, whereas, we take the likelihood of more number of pixels in a bigger patch, to get the suitable pixel intensity value.

In this paper, we have extended the idea of our arxived paper [15]. In [15] we combine two different approaches. We initially compute the DCP and then apply an NN regularization technique to obtain a smooth transmission map of the hazy image. We consider the effect of airlight on the image by using the color line model to assess the commitment of airlight in each patch of the image and interpolate at the local neighborhood where the estimate is unreliable. In addition to [15], the proposed method relies on the Y channel of the YCbCr representation of the image, in order to preserve the texture information of the image. Second, the noise and compression artifact that is imperceptible in the hazy image will be intensified, particularly in the area of low transmission esteems (sky regions). Hence, we implement the regularization not to damage the image content in the nearby locale yet strengthen the smoothing impact in the outlying area.

The rest of the paper is organized as follows. Section 2 describes the problem and a survey on the existing literature. Section 3 displays the mathematical and numerical foundation for the proposed strategy for dehazing. Section 4 details the experiments conducted with the proposed algorithm and results obtained by the proposed method compared to the state-of-the-art. Concluding comments are given in Section 5.

Section snippets

Background and related work

Single image dehazing is a challenging task which draws significant attention from researchers of the fields of Computer Graphics and Image Processing [16]. The physical model often used to characterize the haze formation that causes degradation of an image, is known as Koschmieder’s atmospheric scattering model [17]:I(x)=t(x)J(x)+(1t(x))A,where I(x) is the observed intensity value of pixel x in image I; J(x) is the scene radiance of a haze-free image at x and t(x) is the medium transmission

Proposed approach

In several restoration-based approaches, strong visual effect regions (i.e., regions with high contrast and high reflectance) in fog images can unavoidably be selected to assess atmospheric light A, which prompts reflection issues. In the regions with a heavy fog, the pixels in the image will have higher color intensity. Hence, we can induce that the intensity channel of an image invariably contains the ambiguous luminance information.

In this section, the proposed method is described as a

Results

We have performed extensive experiments on a variety of images and the results are compared with the state-of-the-art image defogging algorithms. Here, we utilize both subjective comparisons and objective quality assessments to examine the effectiveness of the proposed dehazing algorithm.

Conclusions

We have proposed a novel mathematical technique for haze removal emphasizing the generic regularity across pixels representing the same regions of the image. The proposed technique can minimize the haze, without affecting the original color of the image. The experiments on benchmark real and synthetic image datasets show that the proposed method performs much better than the state-of-the-art non-learning based techniques. Moreover, the proposed method shows comparable results with the recent

CRediT authorship contribution statement

Kushal Borkar: Conceptualization, Methodology, Data curation, Software. Snehasis Mukherjee: Writing - original draft, Visualization, Investigation, Supervision, Writing - review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Kushal Borkar completed his Undergraduate course in Electronics and Communication Engineering from the Indian Institute of Information Technology SriCity (IIIT SriCity) with Honours in 2019. His research interest includes Computer Vision, Image Processing and Graphics.

References (53)

  • K. He et al.

    Single image haze removal using dark channel prior

    Proceedings of the 2009 CVPR

    (2009)
  • R. Fattal

    Dehazing using color-lines

    ACM Trans. Graph.

    (2014)
  • W. Ren et al.

    Single image dehazing via multi-scale convolutional neural networks

    Proceedings of the 2016 ECCV

    (2016)
  • B. Cai et al.

    DehazeNet: an end-to-end system for single image haze removal

    IEEE Trans. Image Process.

    (2016)
  • B. Li et al.

    Aod-net: all-in-one dehazing network

    Proceedings of the 2017 ICCV

    (2017)
  • X. Ding, Y. Wang, Y. Yan, Z. Liang, Z. Mi, X. Fu, Jointly Adversarial Network to Wavelength Compensation and Dehazing...
  • K. Jiang et al.

    Edge-enhanced GAN for remote sensing image superresolution

    IEEE Trans. Geosci. Remote Sens.

    (2019)
  • K. Jiang et al.

    Deep distillation recursive network for remote sensing imagery super-resolution

    Remote Sens.

    (2018)
  • K. He et al.

    Single image haze removal using dark channel prior

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • C. Ancuti et al.

    Night-time dehazing by fusion

    Proceedings of the IEEE International Conference on Image Processing (ICIP)

    (2016)
  • K. Borkar, S. Mukherjee, Single Image Dehazing Based on Generic Regularity, arxive:1808.08610...
  • B. Li et al.

    Benchmarking single-image dehazing and beyond

    IEEE Trans. Image Process.

    (2019)
  • H. Koschmieder

    Theorie der Horizontalen Sichtweite: Kontrast und Sichtweite

    (1925)
  • J.-P. Tarel et al.

    Fast visibility restoration from a single color or gray level image

    Proceedings of the 2009 ICCV

    (2009)
  • K. Nishino et al.

    Bayesian defogging

    Int. J. Comput. Vis.

    (2012)
  • D. Singh et al.

    Modified gain intervention filter based dehazing technique

    J. Mod. Opt.

    (2017)
  • Cited by (0)

    Kushal Borkar completed his Undergraduate course in Electronics and Communication Engineering from the Indian Institute of Information Technology SriCity (IIIT SriCity) with Honours in 2019. His research interest includes Computer Vision, Image Processing and Graphics.

    Snehasis Mukherjee has obtained his Ph.D. in Computer Science from the Indian Statistical Institute in 2012. Before doctoral study, he has completed his Bachelors degree in Mathematics from the University of Calcutta and Masters degree in Computer Applications from the Vidyasagar University. He did his Post Doctoral Research works at the National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, USA. Currently he is working as an Assistant Professor in the Computer Vision Group of the Indian Institute of Information Technology SriCity (IIIT SriCity). He has written several peer-reviewed research papers (in reputed journals and conferences). His research area includes Computer Vision, Machine Learning, Image and Video Processing.

    View full text