Optimization and regularization of complex task decomposition for blind removal of multi-factor degradation

Highlights

• A general regularization decomposition strategy for complex tasks is proposed.

• An optimization strategy based on regularization decomposition for complex tasks is designed.

• A regularization strategy based on task decomposition is used to image restoration.

Abstract

Most existing image restoration methods based on deep neural networks are developed for images which only degraded by a single degradation mode and imaging under an ideal condition. They cannot be directly used to restore the images degraded by multi-factor coupling. A complex task decomposition regularization optimization strategy (TDROS) is proposed to solve the problem. The restoration of images degraded by multi-factor coupling is a complex task that can be solved by separating these multiple factors, that is, breaking the complex task into numbers of simpler tasks to make the entire complex problem be overcome more easily. Motivated by this idea, the TDROS decomposes the complex task of image restoration into two sub-task: the potential task constrained by regularization and the main task for reconstructing high-definition images. In TDROS, the front of the neural network is focused on the restoration of images degraded by additive noise, while the other part of the network is focused mainly on the restoration of images degraded by blur. We applied the TDROS to an 11-layer convolutional neural network (CNN) and compared it with initial CNNs from the aspects of restoration accuracy and generalization ability. Based on these results, we used TDROS to design a novel network model for the restoration of atmospheric turbulence-degraded images. The experimental results demonstrate that the proposed TDROS can improve the generalization ability of the existing network more effectively than current popular methods, offering a better solution for the problem of severely degraded image restoration. Moreover, the TDROS concept provides a flexible framework for low-level visual complex tasks and can be easily incorporated into existing CNNs.

Introduction

Restoration of degraded images is a low-level vision task. It can help sensors obtain high quality images and has wide applications, such as image deblurring [1], [2], [3], [4], [5], image denoising [6], [7], image dehazing [8], image rain removal [9], and image super-resolution [10], [11], [12].

For these low-level computer vision tasks, deep learning has achieved remarkable success [13]. However, these successes mainly depend on the continuous innovation of deep neural network structure [13]. Xu et al. proposed a deep convolutional neural network (CNN) to capture the characteristics of degradation [14], which can recognize different types of the blurred images, thereby achieving excellent restoration results under certain conditions. Zhang et al. used a learning gradient method to guide image deconvolution [15]. Hradis et al. restored highly structured text through CNNs and achieved competitive results in the blind restoration of text [16], Zhang et al. used an end-to-end recurrent neural network (RNN) network to learn the spatial distribution involved in motion blur [5], while Guo et al. blindly removed noise from degraded images by using an estimated noise map as the input [7]. Nah et al. developed a multi-scale CNN that can restore sharp images in an end-to-end manner when blur is caused by various sources [1]. Mao et al. achieved good results using a very deep skip-link CNN to reconstruct images [17]. Traditional methods such as these based on network structure innovation are devised mainly to restore degraded images with a single mode, however, these methods are often ineffective in the restoration of degraded images with complex patterns.

Some exploratory work has been carried out in the field of deep neural networks for complex tasks in low-level vision. Among these methods, representative examples are CNN denoising prior [18] and DualCNN [19]. Both the CNN denoising prior and DualCNN methods have achieved good results in low-level visual tasks, such as image denoising, deblurring and super-resolution. However, since the degraded data for network training are independent of each other, the trained network can only solve a single task at a time. In actual image restoration such as spatial target recognition, degradation is often caused by a variety of complex factors, especially those based on ground-based observations. Degraded images are not only affected by atmospheric turbulence during imaging but are also susceptible to interference from ground gas, stray light, high-energy particles, nebulae, thermal noise and electronic noise. Thus, images ground-based spatial targets are not only susceptible to severe degradation but also can have very low peak signal-to-noise ratios (PSNR) [20], [21], [22].

The traditional solution to such complex low-level vision tasks is to improve the model’s presentative ability by increasing the complexity of the model. However, more complex models often need to be provided with more parameters and are, therefore, more prone to neural network overfitting, a challenge that several researchers have worked hard to resolve. Many functions [23] have been used to characterize the generalization boundary. Many of these bounds are obtained either through some forms of regularization (typically L2 regularization [24], [25]) or by restricting the complexity of the function class. Mainstream strategies for preventing overfitting during neural network training include drop-out, early stoppage and data augmentation [23]. Recently, the network pre-training method [26], the network layered-training method [27] and the method of using auxiliary variables for hidden layers [28] have also been proposed.

The reconstruction of complex low-level visual tasks, especially the restoration of degraded images caused by multi-factor coupling, is very difficult. Compared to image degradation caused by a single factor (such as motion blur), the patterns in image degradation caused by multi-factor coupling are more difficult to learn, and the degraded spaces tend to be larger, thus making restoration more problematic. The end-to-end deep neural network is not conducive to learning the inverse transformation mode of image degradation when training data is limited, resulting in a poor quality of restoration. Therefore, this study proposes an optimization strategy based on the regularization and decomposition of complex tasks for the blind removal of turbulence blur, based on the characteristics of degraded images caused by multi-factor coupling, together with the concept of complex task decomposition. The results of this strategy show it to be more effective than other existing methods, and the main contributions of this paper are threefold, namely:

(1) A general regularization decomposition strategy for complex tasks is proposed, in which different parts of the deep neural network can focus on distinct aspects of the entire problem (the different parts of the network correspond to different parts of the problem), thus achieving task decomposition and reducing the complexity of the problem.

(2) Based on the characteristics of image degradation caused by multi-factor coupling in the actual imaging process, an optimization strategy based on regularization decomposition for complex tasks is designed for use in the restoration of atmospheric turbulence-blurred images. The results of the image restoration effect of the proposed strategy are superior to those of other existing methods.

(3) This method is a regularization strategy based on task decomposition, designed to reduce the complexity of the problem, enhance the generalization of the network, and reduce overfitting.

The remainder of this paper is organized as follows: Section 2 provides an overview of related research work; Section 3 analyzes the motivation and rationale behind the proposed method in a physical sense; Section 4 details the design and validity analysis of the TDROS; Section 5 combines the TDROS with a restoration model for turbulence-blurred images; Section 6 describes the experiments performed on the proposed model and compares the results with those of some existing models; and, finally, Section 7 summarizes and provides conclusions to the work.