Variational optimization based single image dehazing

Highlights

• We propose a structure-aware transmission map for single image dehazing.

• We formulated a new variational optimization problem with regularization terms.

• The proposed method performs image dehazing with better contrast, color, and details.

Abstract

In this paper, we present a new approach for single image dehazing based on the proposed variational optimization. A hazy image captures the information about haze in terms of the transmission map and object details present in it. We propose to estimate the initial transmission map by performing the structure-aware smoothing of the hazy image. Further, we formulated a variational optimization for the estimation of final transmission, which refines the initial transmission of a hazy image. Atmospheric light can be considered to be constant throughout the scene for practical purposes. The uniform atmospheric light is computed from the dark channel of a hazy image. The exhaustive experimentation shows that the performance of the proposed method is comparable or better.

Introduction

Computer vision tasks require images with appropriate visibility of the scene. However, sometimes the environment is not favorable to capture an image with good visibility, resulting in hazy images, rainy images, many other types of degraded images. In a real-world scenario, the atmosphere contains tiny particles that cause reflection, refraction, and light scattering due to bad weather conditions. Thus, outdoor images captured in lousy weather suffer from degraded visibility. While capturing an image in such conditions, the blending of atmospheric light and the transmission leads to a hazy image. Most vision-based tasks like object detection [1], fruit picking robot [2], detection of surface deformation and strain [3], 3D reconstruction [4], [5], and others [6], [7], [8], [9], [10], [11] depend significantly upon the image quality. The hazy images contain faded colors, low contrast, and fewer details. Due to haze in an image, feature extraction becomes challenging that degrades the performance of the vision-based tasks. To improve the performance of vision-based tasks in a hazy environment, we must remove the effect of haze from the image. Image dehazing enhances the colors and contrast of an image by removing or reducing the effect of haze thus, improves the performance of the vision-based tasks. An inferior image dehazing method may cause undesirable artifacts and loss of crucial details. Therefore, developing a compelling image dehazing algorithm is a challenging task.

Researchers proposed various algorithms [12], [13], [14], [15] to perform image dehazing. Image dehazing algorithms can be broadly categorized as: traditional methods, prior-based methods, fusion-based methods, and learning-based methods.

Traditional methods: Researchers [16], [17], [18], [19] developed multiple-input images based image dehazing algorithms. Nayar and Narasimhan [16], [17] proposed a binary scattering model for image dehazing. The authors analyze the scattering of atmospheric light to develop a method for recovering true scene radiance. Schechner et al. proposed a polarization-based multi-image algorithm [18] for image dehazing. This algorithm uses a polarization filter to estimate the range map, which helps in the estimation of true scene radiance. Shwartz et al. proposed a blind method [19] that separates airlight and recovers contrast. It estimates the degree of polarization using a probabilistic model to estimate the transmission map. These algorithms require multiple images to estimate true scene radiance. However, sometimes it is challenging to collect multiple images as per the requirement, which restricts the performance of these algorithms.

Another set of Researchers attempted single image dehazing. Tan [20] used Markov random field to develop a cost function. The cost function focuses on the contrast of the images and the smoothness of the airlight. Tan performed maximization of local contrast for image dehazing using the cost function. Fattal [21] proposed a mathematical model for image dehazing. It uses contrast and gradient of an image to estimate the enhanced image. Further, Fattal proposed color lines model [22] and automatic optical vector calculation [23]. Fattal used the property of pixels in small patches to develop color lines and estimate the atmospheric light. All of these algorithms result in the over-saturation of colors and the generation of halo effects.

Prior-based methods: He et al. [24] proposed Dark channel prior (DCP) for single image dehazing. DCP estimates atmospheric light and transmission map from the dark channel. DCP uses the image matte of dark channel as refined transmission map. Image matte uses fine details to estimate the refined transmission map. The algorithm generates artifacts while dealing with the object having the same color as the airlight. Gibson and Nguyen [25] presented a new dark channel prior. The algorithm uses minimum volume ellipsoid approximation, which is inaccurate with pixels corresponding to bright objects. Zhu et al. [26] proposed color attenuation prior that focuses on depth map based estimation of transmission map. The estimation of the depth map from an image is ill-posed, which also affects the performance of CAP. Singh et al. proposed gradient channel prior [27], [28] that uses gradient for estimation of transmission map from a hazy image. Nair and Sankaran proposed an algorithm [29] that estimates transmission using surround filter and DCP. It is computationally simple; however, it fails to provide adequate lightness in dehazed images. The estimation of transmission map and atmospheric light is ill-posed problem. These algorithms sometimes lead to improper estimation of the transmission map, which causes the generation of artifacts and low-visible results.

Fusion-based: Ancuti et al. [30], [31] proposed an algorithm that achieves image dehazing by fusing multiple versions of the input image. However, estimation of multiple versions from an input image for fusion is challenging task. Zhu et al. [32] proposed a multi-exposure image fusion method for image dehazing. It uses gamma correction to estimate multiple components of an image for fusion. Sometimes, multiple gamma-corrected images fail to capture the haze properties. Thus, the algorithm achieves limited enhancement while dealing with dense haze. Galdran et al. [33] presented a method that combines fusion and variational optimization for image dehazing. The algorithm estimates two components using variational optimization and then performs fusion to combine them for image dehazing.

Learning-based: The application of deep learning provided significant improvement in image dehazing. Cai et al. [34] developed DehazeNet, a learning-based approach that estimates transmission map. DehazeNet works on the assumption that global airlight is constant, due to which it fails to deal with both real outdoor and indoor images. Li et al. [35] proposed All-in-One Dehazing Network, a lightweight image dehazing network capable of producing haze-free image directly from the hazy image. Li et al. reformulated the atmospheric scattering model (2) to accommodate a K-estimation module. AOD-Net provides results with low brightness while dealing with images that are not from the dataset and fails to eradicate haze in some cases. Qin et al. [36] developed FFA-Net that treats different pixels and channels unequally by combining pixel attention and channel attention into a Feature Attention module. Liu et al. [37] presented a multi-scale dehazing model using the attention-based CNNs. GCA [38], and DHS [39] methods uses generative adversarial network (GAN) for image dehazing. GAN [38], [40] based methods are difficult to optimize and have high chances of yielding an over-enhanced or under-enhanced image. Although learning-based strategies are effective, a bias can be observed with the images similar to the training dataset. These techniques sometimes fail to deal with images that are distinct from the training images.

Most of the image dehazing algorithms suffer from one or more of the following limitations:

• limited generalization in case of learning-based dehazing methods.

• Over or under enhancement in dehazed image.

• Undesired artifacts and halo effects.

• Insignificant haze removal i.e. restricted image dehazing.

• Distortion of colors in dehazed images.

• Loss of finer details in the dehazed images.

In this paper, we propose a new single image dehazing algorithm using variational optimization-based transmission estimation to overcome these issues. The estimation of transmission map, atmospheric light, and scene radiance from an image without prior knowledge is ill-posed. The transmission map can be estimated from structural details (dominant edges of a scene, objects, and haze) of an image. To estimate the structure-aware transmission map, we leverage the notion of adaptive bilateral filtering [41]. Further, We formulate and solve a new variational optimization to estimate final transmission map. The objective function of variational optimization function helps in achieving the textual suppression and structural preservation. The significant contributions are summarized as follows:

• We propose a new method for the estimation of structure-aware initial transmission. We leverage the concept of adaptive bilateral filtering to achieve the structure-aware initial transmission.

• We formulated a new variational optimization problem with regularization terms to preserve the structural details in the final transmission while smoothing the textural details. We used the Alternative Direction Minimization (ADM) algorithm [42] to solve the formulated variational optimization.

• We performed an exhaustive analysis of the proposed approach with state-of-the-art algorithms on various datasets [24], [26], [43], [44], [45] using qualitative and visual analysis.

The rest of the paper is organized as follows. The related work is discussed in Section 2. The proposed method is described in Section 3; it contains transmission estimation and image dehazing. Section 4 includes the experimental results and analysis. Finally, the conclusion of the paper is in Section 5.