Abstract
To overcome the limitations of the traditional fields of experts (FoE) model, which will blur image edges and texture during the denoising processing, a spatial information weighted FoE (WFoE) model has been presented to introduce the image spatial structure information into the FoE model. A monotone decreasing function is based on the curvature difference to control the filter weight in the edge and smooth region. The proposed WFoE model can better remove noise while preserving edges. Additionally, the proposed WFoE model is designed as a regularization term in the maximum a posteriori-based multi-frame image super-resolution (SR) reconstruction algorithm, enabling the development of a new SR method. Since the WFoE model is more inclined to keep image edges, the proposed WFoE-based SR reconstruction method can obtain better results than traditional FoE model with respect to preserving image edges. Experimental results demonstrate that our method has better peak signal-to-noise ratio and visual verisimilitude compared with some existing SR methods.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
P. Milanfar. MDSP Super-Resolution and Demosaicing Datasets Online: http://users.soe.ucsc.edu/∼milanfar/software/srdatasets.html.
P. Vandewalle and S. Süsstrunk Super-Resolution Data Sets Online: http://users.soe.ucsc.edu/∼milanfar/software/srdatasets.html.
References
Bareja, M. N., & Modi, C. K. (2012). An effective iterative back projection based single image super resolution approach. IEEE Computer Society,14(4), 95–99.
Chen, P., Nelson, J., & Tourneret, J. Y. (2017). Toward a sparse Bayesian Markov random field approach to hyperspectral unmixing and classification. IEEE Transactions on Image Processing,26(1), 426–438.
Chen, J., Nunez-Yanez, J., & Achim, A. (2011). Video super-resolution using generalized Gaussian Markov random fields. IEEE Signal Processing Letters,19(2), 63–66.
Cheng, P., Qiu, Y., Wang, X., & Zhao, K. (2017). A new single image super-resolution method based on the infinite mixture model. IEEE Access,5(9), 2228–2240.
Demirel, H., Izadpanahi, S., & Anbarjafari, G. (2009). Improved motion-based localized super resolution technique using discrete wavelet transform for low resolution video enhancement. In Proceedings of European signal processing conference (pp. 1097–1101).
Deng, L. J., Guo, W., & Huang, T. Z. (2016). Single image super-resolution by approximated Heaviside functions. Information Sciences,348, 107–123.
Elad, M., & Feuer, A. (1997). Restoration of a single super-resolution image from several blurred, noisy, and under-sampled measured images. IEEE Transactions on Image Processing,6(12), 1646–1658.
Farsiu, S., Robinson, M. D., Elad, M., & Milanfar, P. (2004). Fast and robust multi-frame super resolution. IEEE Transactions on Image Processing,13(10), 1327–1344.
Geman, S., & Geman, D. (1984). Stochastic relaxation Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence,6, 721–741.
Huang, S., Sun, J., Yang, Y., Fang, Y., & Lin, P. (2017). Multi-frame super-resolution reconstruction based on gradient vector flow hybrid field. IEEE Access,5, 21669–21683.
Huang, S., & Yang, Y. (2013). Super-resolution reconstruction sensor using adaptively combined partial differential equations. Sensor Letters,11(11), 2126–2130.
Kanemura, A., Maeda, S. I., & Ishii, S. (2009). Super-resolution with compound Markov random fields via the variational EM algorithm. IEEE Transactions on Neural Networks,22(7), 1025–1034.
Li, F., Xin, L., Guo, Y., Gao, J., & Jia, X. (2017). A framework of mixed sparse representations for remote sensing images. IEEE Geoscience and Remote Sensing Letters,55(2), 1210–1221.
Lukeš, T., Křížek, P., Švindrych, Z., Benda, J., Ovesný, M., Fliegel, K., et al. (2014). Three-dimensional super-resolution structured illumination microscopy with maximum a posteriori probability image estimation. Optics Express,22(24), 29805–29817.
Marquina, A., & Osher, S. J. (2008). Image super-resolution by TV-regularization and bregman iteration. Journal of Scientific Computing,37(3), 367–382.
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer and vision (Vol. 2(11), pp. 416–423).
Pan, R., & Reeves, S. J. (2006). Efficient Huber–Markov edge-preserving image restoration. IEEE Transactions on Image Processing,15(12), 3728–3735.
Portilla, J., Strela, V., Wainwright, M. J., & Simoncelli, E. P. (2001). Adaptive Wiener denoising using a Gaussian scale mixture model in the wavelet domain. International Conference on Image Processing,2(2), 37–40.
Ren, C., He, X., & Nguyen, T. (2017). Single image super-resolution via adaptive high-dimensional non-local total variation and adaptive geometric feature. IEEE Transactions on Image Processing,26(1), 90–106.
Rhee, S., & Kang, M. G. (1999). Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Optical Engineering,38(8), 1348–1356.
Robinson, M. D., Toth, C. A., Lo, J. Y., & Farsiu, S. (2010). Efficient fourier-wavelet super-resolution. IEEE Transactions on Image Processing,19(10), 2669–2681.
Roth, S., & Black, M. J. (2005). Fields of experts: A frame work for learning image priors. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2(2), pp. 860–867).
Roth, S., & Black, M. J. (2007). High-order markov random fields for low-level vision. Providence: Brown University.
Sharmin, N., & Brad, R. (2012). Optimal filter estimation for Lucas–Kanade optical flow. Sensors,12(9), 12694–12709.
Soccorsi, M., Gleich, D., & Datcu, M. (2010). Huber–Markov model for complex SAR image restoration. IEEE Geoscience and Remote Sensing Letters,7(1), 63–67.
Stark, H., & Oskoui, P. (1989). High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America. A, Optics, Image Science,6(11), 1715.
Tom, B. C., Galatsanos, N. P., & Katsaggelos, A. K. (1994). Reconstruction of a high resolution image from multiple low resolution images. Visual Communications and Image Processing’,2308, 971–981.
Tsai, R. Y., & Huang, T. S. (1984). Multi-frame image restoration and registration. Advance Computer Visual and Image Processing,1(2), 317–339.
Wang, Q., & Shi, W. (2014). Utilizing multiple sub-pixel shifted images in sub-pixel mapping with image interpolation. IEEE Geoscience and Remote Sensing Letters,11(4), 798–802.
Wang, W., & Yuan, X. (2017). Recent advances in image dehazing. IEEE/CAA Journal of Automatica, Sinica,4(3), 410–436.
Xiao, J., Pang, G., Zhang, Y., Kuang, Y., Yan, Y., & Wang, Y. (2016). Adaptive shock filter for image super-resolution and enhancement. Journal of Visual Communication and Image Representation,40, 168–177.
Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016). Image super-resolution: The techniques, applications, and future. Signal Processing,128, 389–408.
Zhang, X., Lam, E. Y., Wu, E. X., & Wong, K. K. Y. (2008). Application of tikhonov regularization to super-resolution reconstruction of brain MRI images. Lecture Notes in Computer Science,4987(23), 51–56.
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Nos. 61862030, 61662026 and 61462031), by the Natural Science Foundation of Jiangxi Province (Nos. 20182BCB22006, 20181BAB202010), and by the Project of the Education Department of Jiangxi Province (Nos. GJJ170318, GJJ170312 and KJLD14031).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, S., Wu, J., Yang, Y. et al. Multi-frame image super-resolution reconstruction based on spatial information weighted fields of experts. Multidim Syst Sign Process 31, 1–20 (2020). https://doi.org/10.1007/s11045-019-00648-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-019-00648-5