Estimation of the parameters in regularized simultaneous super-resolution
Introduction
Many applications are benefited by acquisition systems that provides images with the best possible resolution, while introducing minimum distortions due to imperfections of the image sensor and the optical system. However, the cost of image acquisition systems, like digital cameras, camcorders and scanners, increases with the resolution of the sensor and with the quality of the optical system. An interesting alternative to improve the resolution and the quality of captured images, without increasing the cost of the system, is to employ digital processing techniques to achieve super-resolution (SR).
Research on super-resolution algorithms began in the 1990s with pioneer works, as Tsai and Huang (1984), which employed Fourier domain methods. Since then, different approaches have been developed, including projections onto convex sets (POCS) (Stark and Oskoui, 1989, Tekalp et al., 1992), non-uniform interpolation (Aizawa et al., 1992) and iterative back-projection (Irani and Peleg, 1991, Irani and Peleg, 1993). Regularized SR approaches based on maximum a posteriori (MAP) and regularized least squares appeared in (Schultz and Stevenson, 1996, Hong et al., 1997). Traditionally, regularized approaches minimize a cost function composed by the residual associated with the estimated high-resolution (HR) frame plus another term, called the prior term, used to regularize the problem (Park et al., 2003). In these approaches, the regularization parameter scales the influence of the prior term in the resulting solution. In most SR methods, the parameter is assumed to be known by some way. In the majority of practical cases, the parameter as well as the HR images must be estimated from the data.
In simultaneous SR methods, proposed in (Borman and Stevenson, 1999, Zibetti and Mayer, 2005, Zibetti and Mayer, 2006, Zibetti and Mayer, 2007), all frames of an image sequence are estimated in a single process. Two different kinds of priors are employed, one to achieve spatial smoothness and other to achieve higher similarity of the HR frames in the motion trajectory. In these problems at least two parameters are necessary. Interesting multiple parameter selection methods for general inverse problems have been studied in (Belge et al., 2002, Hansen, 1998). These methods, however, have not been applied for simultaneous super-resolution. We have addressed this problem in (Zibetti et al., 2008c). In this work, which is an extended version of Zibetti et al. (2008c), the following novelties are provided: (a) more details in the derivation of the method proposed in (Zibetti et al., 2008c); (b) two possible implementations of the method, explained in details and with a pseudocode; and (c) more comprehensive experiments to evaluate the performance of the proposed method, contrasting to other competitive multiple parameter methods such as Belge et al. (2002).
In this paper, we address the problem with two parameters in the simultaneous SR. Section 2 provides a detailed description of system models used in this work. The simultaneous SR algorithm with fixed regularization parameters is reviewed in Section 2.2. In Section 3, the new method with automatic determination of the parameters is proposed using the joint maximum a posteriori (JMAP) estimation technique (Mohammad-Djafari, 1996). The classical JMAP approach, which assumes uniform density for the hyperparameters is, in general, unstable (Mohammad-Djafari, 1996). To circumvent this, we assume a gamma probability density for the hyperparameters which results in a stable algorithm with a unique global solution. In Section 3.3, two detailed implementations of the method are proposed. One is based on non-linear conjugated gradient (Vogel, 2002), the other proposed implementation is based on alternated minimization of updating the parameters and re-solving a linear system with conjugated gradient (Vogel, 2002, Golub and Loan, 1996). Section 4 presents experiments, comparisons and discussions to illustrate the performance. Some new experiments, which provide some insights about the proposed algorithm are shown in this section. Section 5 concludes this paper.
Section snippets
Review of the simultaneous SR methods with fixed parameters
This section describes the models adopted in the super-resolution algorithms and presents the simultaneous super-resolution algorithm with fixed parameters.
Proposed automatic determination of the regularization parameters
This section describes the proposed approach to estimate the parameters based on the joint maximum a posteriori (JMAP) estimation. JMAP is a Bayesian estimator that focus on the estimation of the HR images and the parameters together (Mohammad-Djafari, 1996).
Experiments
The following experiments evaluate the performance of the simultaneous SR algorithms with known fixed parameters and with automatic determination of the parameters. Given a HR image sequence, with known or previously estimated motion, the simulated acquisition process was performed by employing the average of a squared area of pixels using two subsampling factors , and an additive white Gaussian noise with variance adjusted to achieve a fixed SNR
Conclusions
In this paper, an automatic determination of regularization parameters method is proposed for the class of regularized simultaneous super-resolution techniques. The problem of parameters and image sequence estimation has been addressed with the Bayesian theory, using joint maximum a posteriori (JMAP) estimation. A gamma density is proposed for the hyperparameters in order to provide a globally convex cost function, resulting in a unique solution. The proposed method provides a computational
Acknowledgements
This work was supported by CNPq under grants number 140543/2003-1, 300487/94-0(NV), 306273/2008-0 and 472536/2007-9.
References (44)
- et al.
Determining optical flow
Artif. Intell.
(1981) - et al.
Improving resolution by image registration
CVGIP: Graph. Model Image Process.
(1991) - et al.
Motion analysis for image enhancement: Resolution, occlusion, and transparency
J. Vis. Comm. Image Represent.
(1993) - et al.
Subpixel motion estimation for super-resolution image sequence enhancement
J. Vis. Comm. Image Represent.
(1998) - et al.
Determining the regularization parameters for super-resolution problems
Signal Process.
(2008) - Aizawa, K., Komatsu, T., Saito, T., 1992. A scheme for acquiring very high resolution images using multiplecameras. In:...
- et al.
Foundations of Image Science
(2004) - et al.
An improved fixed-point algorithm for determining a Tikhonov regularization parameter
Inverse Probl.
(2009) - et al.
Efficient determination of multiple regularization parameters in a generalized L-curve framework
Inverse Probl.
(2002) - Borman, S., Stevenson, R., 2004. Linear models for multi-frame super-resolution restoration under non-affine...
A generalized Gaussian image model for edge-preserving MAP estimation
IEEE Trans. Image Process.
Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information
IEEE Trans. Inform. Theory
An introduction to compressive sampling
IEEE Signal Process. Mag.
Fast and robust multi-frame super-resolution
IEEE Trans. Image Process.
CMOS image sensors
IEEE Circuit Device Mag.
Matrix Computations
Outlier modeling in image matching
IEEE Trans. Pattern Anal. Machine Intell.
Cited by (17)
Zoom based image super-resolution using DCT with LBP as characteristic model
2022, Journal of King Saud University - Computer and Information SciencesAn efficient and robust multi-frame image super-resolution reconstruction using orthogonal Fourier-Mellin moments
2017, DisplaysCitation Excerpt :Various types of prior models are used in reconstruction-based SR approaches to solve the SR problem. These include maximum a posteriori (MAP) [16], Tikhonov regularization (TR) [17], prior models based on gradient profile [18], edge profile [19], total variation (TV) [20], etc. The edge directed priors such as gradient profile [18], TV prior [20], etc., are specifically designed to enhance edge structures and to sharpen the output HR image.
Image super-resolution employing a spatial adaptive prior model
2015, NeurocomputingCitation Excerpt :Image SR refers to a process that not only increase image pixels, but gets more details by considering the degradation process of the LR images. As a consequence, the SR technique has drawn more and more attention of researchers in the last two decades [4–10]. Multi-frame image SR reconstruction techniques aim at estimating a HR image by utilizing complimentary information of a sequence of LR images captured from one or more cameras, which cannot only increase the image pixels but also get more details by consider the degradation process.
A non-stationary image prior combination in super-resolution
2014, Digital Signal Processing: A Review JournalCitation Excerpt :The limited accuracy inherent to HR registration from LR images is a shortcoming of this first approach. The second approach is to alternate between HR image registration and HR image estimation (see [15–20,7,9,21–23,8,10]). In this paper, we extend the work in our previous conference paper [24], in which the application of a spatially adaptive general linear filter combination prior model to the SR problem was proposed.
Bayesian combination of sparse and non-sparse priors in image super resolution
2013, Digital Signal Processing: A Review JournalInfrared super-resolution imaging method based on retina micro-motion
2013, Infrared Physics and TechnologyCitation Excerpt :Continuously improving infrared image resolution has become more and more pressing needs in infrared imaging field. It has great significance in military, aerospace, public security, civil and other fields [3]. Infrared super-resolution imaging method is an effective technology of improving infrared image spatial resolution [4].