Skip to main content
SpringerLink
Log in

A Blind Super-Resolution Reconstruction Method Considering Image Registration Errors

  • Published:
International Journal of Fuzzy Systems Aims and scope Submit manuscript

Abstract

Super-resolution (SR) image reconstruction refers to a process that produces a high-resolution (HR) image from a sequence of low-resolution images that are noisy, blurred, and downsampled. Blind SR is often necessary when the blurring function is unknown. In this paper, to reduce registration errors, we present a new joint maximum a posteriori (MAP) formulation to integrate image registration into blind image SR reconstruction. The formulation is built upon the MAP framework, which judiciously combines image registration, blur identification, and super-resolution. A cyclic coordinate descent optimization procedure is developed to solve the MAP formulation, in which the registration parameters, blurring function, and HR image are estimated in an alternative manner, given the two others, respectively. The proposed algorithm is tested using simulated as well as real-life images. The experimental results indicate that the proposed algorithm has considerable effectiveness in terms of both quantitative measurements and visual evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)

    Article  Google Scholar 

  2. Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14(2), 47–57 (2004)

    Article  Google Scholar 

  3. Zhang, H., Yang, Z., Zhang, L., Shen, H.: Super-resolution reconstruction for multi-angle remote sensing images considering resolution differences. Remote Sens. 6(1), 637–657 (2014)

    Article  Google Scholar 

  4. Yang, J., Yu, P., Kuo, B., Su, M.: Nonparametric fuzzy feature extraction for hyperspectral image classification. Int. J. Fuzzy Syst. 12(3), 208–217 (2010)

    Google Scholar 

  5. Lakdashti, A., Moin, M.S., Badie, K.: Reducing the semantic gap of the MRI image retrieval systems using a fuzzy rule based technique. Int. J. Fuzzy Syst. 11(4), 232–249 (2009)

    Google Scholar 

  6. Tsai, R.Y., Huang, T.S.: Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process. 1, 317–339 (1984)

    Google Scholar 

  7. Kim, S.P., Bose, N.K., Valenzuela, H.M.: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans. Acoust. Speech Signal Process. 38(2), 1013–1027 (1990)

    Article  Google Scholar 

  8. Kim, S.P., Su, W.Y.: Recursive high-resolution reconstruction of blurred multiframe images. IEEE Trans. Image Process. 2(4), 534–539 (1993)

    Article  Google Scholar 

  9. Sanchez-Beato, A., Pajares, G.: Noniterative interpolation-based super-resolution minimizing aliasing in the reconstructed image. IEEE Trans. Image Process. 17, 1817–1826 (2008)

    Article  MathSciNet  Google Scholar 

  10. Zhou, F., Yang, W., Liao, Q.: Interpolation-based image super-resolution using multi-surface fitting. IEEE Trans. Image Process. 21(6), 3312–3318 (2012)

    Article  MathSciNet  Google Scholar 

  11. Irani, M., Peleg, S.: Improving resolution by image registration. CVGIP: Graph Models Image Process. 53, 231–239 (1991)

    Google Scholar 

  12. Stark, H., Oskoui, P.: High-resolution image recovery from image plane arrays, using convex projections. J. Opt. Soc. Am. A 6, 1715–1726 (1989)

    Article  Google Scholar 

  13. Patti, A. J., Sezan, M. I., Tekalp, A. M.: High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur. Presented at IEEE international conference on image processing, Austin, TX, USA (1994)

  14. Patti, A.J., Sezan, M.I., Tekalp, A.M.: Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6(8), 1064–1076 (1997)

    Article  Google Scholar 

  15. Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5(6), 996–1011 (1996)

    Article  Google Scholar 

  16. Zhang, L., Zhang, H., Shen, H., Li, P.: A Super-resolution reconstruction algorithm for surveillance images. Signal Process. 90(3), 848–859 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  17. Pickup, L. C., Capel, D. P., Roberts, S. J., Zisserman, A.: Bayesian image super-resolution, continued. Adv. Neural Inf. Process. Syst. pp. 1089–1096 (2006)

  18. Zhang, H., Shen, H., Zhang, L.: A super-resolution reconstruction algorithm for hyperspectral images. Signal Process. 92(9), 2082–2096 (2012)

    Article  MathSciNet  Google Scholar 

  19. Hardie, R.: A fast image super-resolution algorithm using an adaptive Wiener filter. IEEE Trans. Image Process. 16(12), 2953–2964 (2007)

    Article  MathSciNet  Google Scholar 

  20. Barzigar, N., Roozgard, A., Verna, P., Cheng, S.: A video super resolution framework using SCoBeP. IEEE Trans. Circuits Syst. Video Technol. (2014). doi:10.1109/TCSVT.2013.2283108

    Google Scholar 

  21. He, H., Kondi, L.P.: An image super-resolution algorithm for different error levels per frame. IEEE Trans. Image Process. 15(3), 592–603 (2006)

    Article  Google Scholar 

  22. He, Y., Yap, K.H., Chen, L., Chau, L.P.: A soft MAP framework for blind super-resolution image reconstruction. Image Vis. Comput. 27, 364–373 (2009)

    Article  Google Scholar 

  23. Nguyen, N., Milanfar, P., Golub, G.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Process. 10(9), 1299–1308 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  24. Woods, N., Galatsanos, N., Katsaggelos, A.: EM-based simultaneous registration, restoration, and interpolation of super-resolved images. Presented at IEEE international conference on image processing, vol. 2, pp. 303–306 (2003)

  25. Yang, H., Gao, J., Wu, Z.: Blur identification and image super-resolution reconstruction using an approach similar to variable projection. IEEE Signal Process. Lett. 15, 289–292 (2008)

    Article  Google Scholar 

  26. Yagle, A.: Blind superresolution from undersampled blurred measurements. Presented at advanced signal processing algorithms, architectures, and implementations XIII, vol. 5205, pp. 299–309 (2003)

  27. He, H., Kondi, L. P.: A regularization framework for joint blur estimation and super-resolution of video sequences. Presented at IEEE international conference on image processing, Genova (2005)

  28. Sroubek, F., Cristóbal, G., Flusser, J.: A unified approach to super-resolution and multichannel blind deconvolution. IEEE Trans. Image Process. 16(9), 2322–2332 (2007)

    Article  MathSciNet  Google Scholar 

  29. Shen, H., Zhang, L., Huang, B., Li, P.: A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Trans. Image Process. 16(2), 479–490 (2007)

    Article  MathSciNet  Google Scholar 

  30. Tian, Y., Yap, K.: Joint image registration and super-resolution from low-resolution images with zooming motion. IEEE Trans. Circuits Syst. Video Technol. 23(7), 1224–1234 (2013)

    Article  Google Scholar 

  31. Vandewalle, P., Susstrunk, S., Vetterli, A.: A frequency domain approach to registration of aliased images with application to super-resolution. EURASIP J. Appl. Signal Process. (Article ID 71459) (2006)

  32. Ng, M. K., Shen, H., Lam, E., Zhang, L.: A total variation regularization based Super-Resolution Reconstruction Algorithm for Digital Video. EURASIP J. Appl. Signal Process. (Article ID 74585) (2007)

  33. Hu, T., Zhang, H., Shen, H., Zhang, L.: Robust registration by rank minimization for multi-angle multi/hyper-spectral remote sensing imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(6), 2443–2457 (2014)

    Article  Google Scholar 

  34. He, Y., Yap, K.H., Chen, L., Chau, L.P.: A nonlinear least square technique for simultaneous image registration and super-resolution. IEEE Trans. Image Process. 16(11), 2830–2840 (2007)

    Article  MathSciNet  Google Scholar 

  35. Tom, B. C., Katsaggelos, A. K.: Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images. Presented at IEEE international conference on image processing, Washington, DC (1995)

  36. Hardie, R.C., Barnard, K.J., Armstrong, E.E.: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6(12), 1621–1633 (1997)

    Article  Google Scholar 

  37. Segall, C.A., Molina, R., Katsaggelos, A.K.: High-resolution images from low-resolution compressed video. IEEE Signal Process. Mag. 20(3), 37–48 (2003)

    Article  Google Scholar 

  38. Segall, C.A., Katsaggelos, A.K., Molina, R., Mateos, J.: Bayesian resolution enhancement of compressed video. IEEE Trans. Image Process. 13(7), 898–910 (2004)

    Article  Google Scholar 

  39. Woods, N.A., Galatsanos, N.P., Katsaggelos, A.K.: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Trans. Image Process. 15(1), 201–213 (2006)

    Article  MathSciNet  Google Scholar 

  40. Chung, J., Haber, E., Nagy, J.: Numerical methods for coupled superresolution. Inverse Prob. 22, 1261–1272 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  41. Elad, M., Hel-Or, Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Image Process. 10(8), 1187–1193 (2001)

    Article  MATH  Google Scholar 

  42. Zhang, H., He, W., Zhang, L., Shen, H., Yuan, Q.: Hyperspectral image restoration using low-rank matrix recovery. IEEE Trans. Geosci. Remote Sens. 52(8), 4729–4743 (2014)

    Article  Google Scholar 

  43. Jiang, C., Zhang, H., Shen, H., Zhang, L.: Two-step sparse coding for the pan-sharpening of remote sensing images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7(5), 1792–1805 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Basic Research Program of China (973 Program) under Grant 2011CB707105, by the 863 Program under Grant 2013AA12A301, and in part by the National Natural Science Foundation of China under Grant 61201342 and Grant 41431175.

Author information

Authors and Affiliations

  1. State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, and Collaborative Innovation Center for Geospatial Technology, Wuhan University, Wuhan, People’s Republic of China

    Hongyan Zhang & Liangpei Zhang

  2. School of Resource and Environmental Sciences, Wuhan University, Wuhan, People’s Republic of China

    Huanfeng Shen

Authors
  1. Hongyan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liangpei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huanfeng Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongyan Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Zhang, L. & Shen, H. A Blind Super-Resolution Reconstruction Method Considering Image Registration Errors. Int. J. Fuzzy Syst. 17, 353–364 (2015). https://doi.org/10.1007/s40815-015-0039-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40815-015-0039-y

Keywords

Search

Navigation