Attention-guided dynamic multi-branch neural network for underwater image enhancement

Abstract

Underwater images often suffer from quality deteriorations such as color deviation, reduced contrast, and blurred details due to wavelength-dependent light absorption and scattering in water media. Recently, convolutional neural networks (CNNs) have achieved impressive success in underwater image enhancement (UIE). However, there is still room for improvement in terms of representational capability and receptive field (RF)/channel variant ability in almost all CNN-based UIE networks. To treat these problems, we propose an attention-guided dynamic multibranch neural network (ADMNNet) to obtain high-quality underwater images. Different from existing CNN-based UIE networks that generally share a fixed RF size of artificial neurons in one feature layer, we propose an attention-guided dynamic multibranch block (ADMB) to boost the diversity of feature representations by merging the properties of different RFs into a single-stream structure. Concretely, ADMB includes two main components, namely, a dynamic feature selection module (DFSM) and a multiscale channel attention module (MCAM). Inspired by the selective kernel mechanism in the visual cortex, we incorporate a nonlinear strategy into the DFSM that allows the neurons to adjust their RF sizes reasonably by using soft attention to achieve dynamic fusion of multiscale features. In the MCAM, channel attention is designed to exploit the interdependencies among the channelwise features extracted from different branches. Our ADMNNet can obtain better visual quality of underwater images captured under diverse scenarios and achieves superior qualitative and quantitative performance compared to state-of-the-art UIE methods.

Introduction

Underwater image enhancement (UIE) is a fundamental operation in the computer vision community that has become a hot spot in the field of image processing in recent years. The main purpose of UIE is to recover a clean image by eliminating degradations (e.g., color deviation and low contrast caused by wavelength-dependent attenuation) from its corresponding degraded version [1], [2], [3]. Research on this problem can be used in latent applications, such as underwater detection [4] and marine environmental surveillance [5]. However, it is an extremely challenging and ill-posed task, owing to medium attenuation properties or the diversity of underwater image distributions.

Studies show that earlier techniques expressly applied image priors handcrafted with empirical observations. These prior-based techniques often perform worse than expected because constructing strong image priors is difficult and usually fails to generalize. Recently, the rapid development of convolutional neural networks (CNNs) has promoted new strategies and perspectives for problem solving in the field of image processing. Therefore, CNNs have emerged as a dominant UIE approach, which can achieve state-of-the-art results with extraordinary feature representation capabilities.

Natural scenarios possess different areas, which should be taken into account at distinct scales. For instance, smooth areas correspond to features at larger scales, while the features in textured areas correspond to smaller ones [6]. However, most CNN-based UIE methods generally possess a finite receptive field (RF), which makes it difficult to estimate multiscale features. This inevitably limits their applicability to the diversity of water types and complex degradation levels.

To ameliorate the aforementioned problems, we present an effective CNN-based UIE solution. Following this intuition, to accomplish this challenging task, we build on two main facts/ observations. First, a few UIE techniques [7], [8] based on multiscale CNNs have been introduced, in which a linear manner is used to merge multiscale features. However, numerous studies [9], [10], [11] have demonstrated that the RF sizes of neurons in the same region (e.g., the primary visual cortex) are different but naturally adjusted via the stimulus. Therefore, these techniques cannot truly imitate the astonishing capability of neurons to adaptively integrate visual inputs, thereby restricting their performance. Second, enhancing underwater images is a wavelength-sensitive task due to variable levels of light attenuation for different wavelengths [12], [13], [14]. Each channelwise feature in an image embodies distinct types of input information. Some features may help to cope with the issues of color deviation, and others may contribute to improving the low contrast. If the interdependencies across channels are not taken into consideration, this will weaken the representational power of the network.

Based on the above motivation, a building block called the attention-guided dynamic multibranch block (ADMB) is introduced, which extracts the desired dynamic information from inputs at different branches. ADMB is mainly composed of a dynamic feature selection module (DFSM) and a multiscale channel attention-guided module (MCAM). To effectively advance the UIE task, multiple ADMB units are stacked in an end-to-end architecture, which is termed the attention-guided dynamic multibranch neural network (ADMNNet). Concretely, DFSM is proposed to exploit the gain of RF properties, which is responsible for receiving multiple branches with different RF sizes. To produce a final and global representation for the selection weights, the information from multiple branches is merged. An attention mechanism is applied to estimate selection weights, which can adaptively underline the most representative features. MCAM is proposed to improve the channel variant ability of the network. The proposed ADMNNet can effectively remove color deviation and improve detailed information, as shown in Fig. 1. Briefly, the significant contributions of this work are highlighted as follows.

$•$ We propose an attention-guided dynamic multibranch neural network (ADMNNet) to achieve superiority and adaptability for complex and numerous underwater images. Extensive experiments show that the proposed model introduces excellent robustness and flexibility when compared with state-of-the-art methods.

$•$ We develop a dynamic feature selection module (DFSM), in which the RF size of neurons can be adaptively modified by stimulation. More importantly, soft attention is used to fulfill the selective kernel mechanism between multiscale features. Put differently, our network has a self-adaptive adjustment function based on the contextual information of the input.

$•$ We design a multiscale channel attention-guided module (MCAM), which can be leveraged simultaneously to explore channelwise information through a more effective approach.

The rest of this paper is organized as follows: Section 2 introduces related work. The proposed method is presented in Section 3. Section 4 evaluates and compares the experimental results. Section 5 concludes the paper.