A weighted nuclear norm (WNN)-based retinex DIP framework for restoring aerial and satellite images corrupted by gamma distributed speckle noise

Abstract

Restoration and enhancement are crucial preprocessing steps in the satellite domain. Mainly in active remote sensing such as Synthetic Aperture Radar (SAR), the images are more prone to speckle distortions and their reduction is not so trivial. Traditional deep learning models require large training datasets, limiting their applicability. This paper introduces a novel approach that combines the Deep Image Prior (DIP) model with a weighted nuclear norm (WNN) within a variational retinex framework to address these challenges. DIP leverages prior knowledge about noise distribution and works effectively with a single noisy image, eliminating the need for a large number of training images or ground truth. The WNN assigns non-negative weights to singular values, capturing the significance of each value and preserving crucial information during restoration. This approach offers a promising solution for satellite image restoration without relying on huge training data. The proposed method is evaluated through extensive experiments using various image quality metrics, including PSNR, SSIM, ENL, CNR, Entropy, and GCF. The comparative studies provide compelling evidence that the proposed method surpasses existing techniques in effectively restoring and enhancing speckled input images. Furthermore, statistical analysis performed using the Friedman test demonstrates the superior denoising performance of the model. Additionally, an ablation study is conducted to empirically determine the optimal regularization parameters, ensuring the optimal performance of the model. However, the theoretical selection of parameters for achieving optimal results remains an area that requires further exploration.

Appendices

Appendix

A bayesian MAP estimator for gamma noise

In this section, the MAP estimation of Gamma noise is provided in detail. The authors in [33] derived the fidelity term for the gamma noise by maximizing the posterior probability of obtaining a clean estimate given noisy data. The extract of the same is provided here for completeness. According to Bayes’ theorem, the posterior probability P(X|Y) is defined as given in (A1),

$$\begin{aligned} P(X|Y) = \frac{P(Y|X)P(X)}{P(Y)}, \end{aligned}$$

(A1)

where, X and Y represent random variables and P(X|Y) represents the conditional probability i.e. the probability of X given Y. In the imaging domain, we model the observed data as $P(u_0|u)$, where u and $u_0$ are the original and distorted images, respectively. MAP estimator maximizes the posterior probability $P(u|u_0)$ and the (A1) takes the form as in (A2),

$$\begin{aligned} {arg}\max _u \{P(u|u_0)\} = \underset{u}{arg}\ \max \frac{\{P(u_0|u)P(u)\}}{P(u_0)}. \end{aligned}$$

(A2)

P(u) is the prior probability. $P(u_0)$ is a constant that represents total probability. This term is omitted in the further steps as it doesn’t affect the minimization process. Minimizing the (A2) is equivalent to maximizing its negative $\log$ likelihood ((A3), as log is a monotonic function),

$$\begin{aligned} {arg} \min _u \{-log\{P(u|u_0)\}\} = {arg} \min _u \{-\log P(u_0|u)-\log P(u)+\log P(u_0)\}. \end{aligned}$$

(A3)

Under the assumption that the noise samples are mutually independent of each other, the likelihood estimate is given by the product of sum terms denoted as in (A4),

$$\begin{aligned} P(u_0|u) = \prod _{i}P(u_0(i)|u(i)). \end{aligned}$$

(A4)

For Gamma distribution, $P(u_0|u) = \prod _{i}\frac{L^L}{u(i)^L\Gamma (L)} u_0(i)^{L-1} e^{-\frac{Lu_0(i)}{u(i)}}$ where L stands for the number of looks. Assuming the Gibb’s prior, $P(u) = e^{-\lambda _k\phi (u)}$, the minimization function in (A3) is defined as shown in (A5),

$$\begin{aligned} {arg}\min _u (-\log (P(u|u_0)))= arg\min _u{\sum _{i} \left[ \log (u(i))+\frac{u_0(i)}{u(i)}+\lambda _k(\phi (u(i))) \right] }. \end{aligned}$$

(A5)

For the sake of simplicity, we drop the index term "i". For a continuous function u (after dropping the index i), the minimization term in the image domain $\Omega$ is written as shown in (A6),

$$\begin{aligned} {arg}\min _u \left\{ \int _{\Omega }\left( \left( \log (u)+\frac{u_0}{u}\right) +\lambda _k(\phi (u))\right) d\Omega \right\} . \end{aligned}$$

(A6)

B Image quality metrices

Various reference and non-reference-based matrices used for the quantitative evaluation are detailed below.

1.1 B.1 Reference based IQM

Reference-based measures rely on ground truth data for the evaluation of the restored image quality.

1.1.1 B.1.1 peak signal to noise ratio

PSNR measures the signal-to-noise ratio. It quantifies the extent of denoising in an image. Using the same notations as in (11), PSNR is defined as in (B1),

$$\begin{aligned} PSNR = 10\log \left( \frac{{\mathcal {M}_u}^2}{MSE}\right) , \end{aligned}$$

(B1)

where, $\mathcal {M}_u$ is the maximum possible pixel value in the image u and MSE represents the corresponding mean square error defined as (B2),

$$\begin{aligned} MSE = \frac{1}{width \times height}\sum _{p=0}^{width-1}\sum _{q=0}^{height-1}|u(p,q)-u_0(p,q)|^{2}. \end{aligned}$$

(B2)

If the noisy or the restored image varies significantly from the clean image, then MSE increases, leading to a lesser PSNR. As the noise decreases in the image, the PSNR value increases conveying better denoising and restoration.

1.1.2 B.1.2 structural similarity measure

SSIM analyses the noise reduction, structure and contrast preservation. It signifies the closeness or similarity of two images under consideration. SSIM is evaluated as as given in (B3),

$$\begin{aligned} SSIM(u,u_0)= \frac{(2\mu _u{\mu _u}_0+t_1)(2{s_u}_0+t_2)}{(\mu _u^2{\mu _u}_0^2+t_1)(s_u+{s_u}_0^2+t_2)}, \end{aligned}$$

(B3)

where, where $\mu _u$, ${\mu _u}_0$ denotes the mean and ${s_u}$,${s_u}_0$ denote the variance of u, $u_0,$ respectively.

1.2 B.2 Non-Reference based IQM

These are the blind quality metrics that rely only on noisy data to evaluate the quality of the restored image.

1.2.1 B.2.1 Equivalent Number of Looks

ENL is calculated on the homogeneous patches in the images where the intensity variation is minimum (only variation can be because of noise). The smaller the variance is, the larger is the ENL value, which indicates a better despeckled image. ENL is defined as in (B4),

$$\begin{aligned} \frac{\mu _p^2}{s_p^2}, \end{aligned}$$

(B4)

where $\mu _p$ and $s_p^2$ represent patch mean and variance respectively.

1.2.2 B.2.2 contrast to noise ratio

It quantifies the contrast factor along with the denoising, similar to the PSNR. It is computed as mentioned in (B5),

$$\begin{aligned} \frac{|(\mu _p-\mu _n)|}{\sqrt{s_p^2+s_n^2}}, \end{aligned}$$

(B5)

where $\mu _p$, $\mu _n$, and $s_p$, $s_n$ represent the mean and variance of the reconstructed and noisy samples, respectively.

1.2.3 B.2.3 Entropy

The entropy measure assesses the information gain. The value of the entropy depends on the intensity distribution of the pixels in an image. It is defined as specified in (B6),

$$\begin{aligned} Entropy = -\sum _{i=0}^{MAX}P(i)\log P(i). \end{aligned}$$

(B6)

The entropy can get its minimum value of 0 for the homogeneous region(ideally no variation in the pixels) because for such regions, $P(i) = 1$ and $\log 1 = 0$). The higher entropy is achieved when the image is heterogeneous.

1.3 B.3 global contrast factor

GCF corresponds to the human perception of contrast. GCF is evaluated as given in (B7),

$$\begin{aligned} GCF = \sum _{i=1}^N\alpha _i\mathcal {C}_i,\nonumber \\ where, \mathcal {C} = \frac{1}{width\times height}\sum _{j=1}^{width\times height}lc_{j}, \end{aligned}$$

(B7)

where, $\alpha _i$ is the weight associated with the local average contrast $\mathcal {C}_i$. $lc_j$ is the average local contrast of pixels in the image of size $width\times height$. N is the total number of iterations for which the local average is measured.