Underwater image enhancement based on conditional generative adversarial network

Abstract

Underwater images play an essential role in acquiring and understanding underwater information. High-quality underwater images can guarantee the reliability of underwater intelligent systems. Unfortunately, underwater images are characterized by low contrast, color casts, blurring, low light, and uneven illumination, which severely affects the perception and processing of underwater information. To improve the quality of acquired underwater images, numerous methods have been proposed, particularly with the emergence of deep learning technologies. However, the performance of underwater image enhancement methods is still unsatisfactory due to lacking sufficient training data and effective network structures. In this paper, we solve this problem based on a conditional generative adversarial network (cGAN), where the clear underwater image is achieved by a multi-scale generator. Besides, we employ a dual discriminator to grab local and global semantic information, which enforces the generated results by the multi-scale generator realistic and natural. Experiments on real-world and synthetic underwater images demonstrate that the proposed method performs favorable against the state-of-the-art underwater image enhancement methods.

Introduction

Underwater images carry significant information of the underwater environment and are widely-used for exploring, recognizing, and monitoring underwater world. High-quality underwater images are desired in practical applications, which can guarantee the reliability of analysis and processing. However, suffering from the effects of light absorption and scattering, underwater images have obvious quality degradation issues, such as color distortion, low contrast, blurring, low light, uneven illumination, etc [1]. Concretely, the wavelength related attenuation causes color casts while the scattering brings a distance-dependent component into a clear image, which causes low-contrast of underwater images. Besides, underwater images also suffer from the noise and blurring induced by the cameras and motion. Several typical quality degraded underwater images are presented in Fig. 1. Such degraded underwater images limit their further applications in underwater object detection, underwater scene understanding, underwater robot inspection, underwater 3D reconstruction, to name a few [2], [3].

To obtain high-quality underwater images, the existing methods range from polarization filters, reversing physical models to deep learning-based technologies [4]. Due to the diversity of underwater image degradation [5], the performance of current methods has room to be improved. In recent years, much attention has been paid to deep learning-based visual tasks. Moreover, the deep learning-based methods have achieved encouraging results in many visual tasks [6], [7], [8], [9], [10], [11], [12]. Nevertheless, the deep learning-based underwater image enhancement methods usually fall behind the traditional underwater image enhancement methods. The main reasons focus on the limited and monotonous training data as well as ineffective network structures [1]. More recently, the success of cGANs [13] inspires us to explore the performance of cGANs in underwater image enhancement. Additionally, we also propose to solve the issues of limited underwater image training data based on a kind of random sampling strategy and the blending of synthetic and real data. The promising performance of recent low-level visual tasks [14], [15], [16], [17] also encourages our method. Experimental results on synthetic and real-world underwater image datasets demonstrate the robust and decent performance of our proposed method.

The contributions of this paper are summarized as follows.

• We propose a conditional generative adversarial network (cGAN) for underwater image enhancement. To the best of our knowledge, this is the first cGAN with dual discriminator for underwater image enhancement problem.

• To produce more realistic and natural results, we develop a dual discriminator which determines the reliability of generated results from different views.

• Our proposed method can improve the contrast and remove color casts of underwater images, and achieves state-of-the-art performance on diverse scenes.

The remainder of this paper is scheduled as follows. In Section 2, we briefly review the related works, including single image dehazing methods, underwater image enhancement methods, and cGANs in low-level visual tasks. In Section 3, the proposed cGAN for underwater image enhancement is presented and analyzed. In Section 4, extensive experiments are conducted to demonstrate the effectiveness of the proposed method and the advantages over current methods. In Section 5, we discuss and conclude this paper.