Skip to main content
Log in

Integrating the Missing Information Estimation into Multi-frame Super-Resolution

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Multi-frame super-resolution image reconstruction aims to restore a high-resolution image by fusing a set of low-resolution images. The low-resolution images are usually subject to some degradation, such as warping, blurring, down-sampling, or noising, which causes substantial information loss in the low-resolution images, especially in the texture regions. The missing information is not well estimated using existing traditional methods. In this paper, having analyzed the observation model describing the degradation process starting with a high-resolution image and moving to the low-resolution images, we propose a more reasonable observation model that integrates the missing information into the super-resolution reconstruction. Our approach is fully formulated in a Bayesian framework using the Kullback–Leibler divergence. In this way, the missing information is estimated simultaneously with the high-resolution image, motion parameters, and hyper-parameters. Our proposed estimation of the missing information improves the quality of the reconstructed image. Experimental results presented in this paper show improved performance compared with that of existing traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Y. Anagun, E. Seke, Super Resolution Using Variable Size Block-Matching Motion Estimation with Rotation, in Proceedings of the 2012 International Symposium on Innovations in Intelligent Systems and Applications (INISTA) (IEEE, Trabzon, Turkey, 2012), pp. 1–5

  2. S. Babacan, R. Molina, A. Katsaggelos, Variational Bayesian super resolution. IEEE Trans. Image Process. 20(4), 984–999 (2011)

    Article  MathSciNet  Google Scholar 

  3. M. Camponez, E. Salles, M. Sarcinelli-Filho, Super resolution image reconstruction using nonparametric Bayesian INLA approximation. IEEE Trans. Image Process. 21(8), 3491–3501 (2012)

    Article  MathSciNet  Google Scholar 

  4. V. Capasso, J. Periaux, Multidisciplinary Methods for Analysis Optimization and Control of Complex Systems. Mathematics in Industry, vol. 6, in Nonlinear Inverse Problems: Theoretical Aspects and Some Industrial Applications (Springer, Berlin, Heidelberg, 2005), pp. 3–47

  5. J. Chen, J. Nunez-Yanez, A. Achim, Video super-resolution using generalized Gaussian Markov random fields. IEEE Signal Process. Lett. 19(2), 63–66 (2012)

    Article  Google Scholar 

  6. T. Cho, C. Zitnick, N. Joshi, S. Kang, R. Szeliski, W. Freeman, Image restoration by matching gradient distributions. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 683–694 (2012)

    Article  Google Scholar 

  7. N. El-Yamany, P. Papamichalis, An Adaptive m-Estimation Framework for Robust Image Super Resolution Without Regularization, in Proceedings of SPIE 6822, Visual Communications and Image Processing (SPIE, San Jose, CA, 2008), p. D8221

  8. N. El-Yamany, P. Papamichalis, Robust color image superresolution: an adaptive M-estimation framework. EURASIP J. Image Video Process. 2008, 763,254 (2008)

    Article  Google Scholar 

  9. M. Elad, A. Feuer, Restoration of a single super resolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)

    Article  Google Scholar 

  10. C. Estatico, M. Pastorino, A. Randazzo, A novel microwave imaging approach based on regularization in \(l^p\) banach spaces. IEEE Trans. Antennas Propag. 60(7), 3373–3381 (2012)

    Article  MathSciNet  Google Scholar 

  11. S. Farsiu, M. Robinson, M. Elad, P. Milanfar, Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Article  Google Scholar 

  12. W. Freeman, C. Liu, Advances in Markov Random Fields for Vision and Image Processing, in Chapter 10: Markov Random Fields for Super-resolution and Texture Synthesis (MIT Press, 2011)

  13. S. Izadpanahi, H. Demirel, Motion block based video super resolution. Digit. Signal Process. 23(5), 1451–1462 (2013)

    Article  MathSciNet  Google Scholar 

  14. R. Molina, M. Vega, J. Abad, A. Katsaggelos, Parameter estimation in Bayesian high-resolution image reconstruction with multisensors. IEEE Trans. Image Process. 12(12), 1655–1667 (2003)

    Article  Google Scholar 

  15. W. Nam, S. Kong, Least-squares-based iterative multipath super-resolution technique. IEEE Trans. Signal Process. 61(3), 519–529 (2013)

    Article  MathSciNet  Google Scholar 

  16. H. Rue, S. Martino, N. Chopin, Approximate Bayesian inference for latent Gaussian models by using integrated nested laplace approximations. J. R. Stat. Soc. Ser. B (Statistical Methodology) 71(2), 319–392 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  17. H. Su, L. Tang, Y. Wu, D. Tretter, J. Zhou, Spatially adaptive block-based super-resolution. IEEE Trans. Image Process. 21(3), 1031–1045 (2012)

    Article  MathSciNet  Google Scholar 

  18. H. Su, Y. Wu, J. Zhou, Super-resolution without dense flow. IEEE Trans. Image Process. 21(4), 1782–1795 (2012)

    Article  MathSciNet  Google Scholar 

  19. J. Tian, K. Ma, Stochastic super-resolution image reconstruction. J. Vis. Commun. Image Represent. 21(3), 232–244 (2010)

    Article  Google Scholar 

  20. S. Villena, M. Vega, S. Babacan, R. Molina, A. Katsaggelos, Bayesian combination of sparse and non-sparse priors in image super resolution. Digit. Signal Process. 23(2), 530–541 (2013)

    Article  MathSciNet  Google Scholar 

  21. S. Villena, M. Vega, S. Babacan, R. Molina, A. Katsaggelos, Using the Kullback–Leibler Divergence to Combine Image Priors in Super-Resolution Image Reconstruction, in Proceedings of 17th IEEE International Conference on Image Processing (ICIP 2010) (IEEE, Hong Kong, China, 2010), pp. 893–896

  22. S. Villena, M. Vega, R. Molina, A. Katsaggelos, Bayesian Super-Resolution Image Reconstruction Using an L1 Prior, in Proceedings of 6th International Symposium on Image and Signal Processing and Analysis (ISPA) (IEEE, Salzburg, 2009), pp. 152–157

  23. C. Wang, N. Komodakis, N. Paragios, Markov random field modeling, inference, learning in computer vision, image understanding: a survey. Comput. Vis. Image Underst. 117(11), 1610–1627 (2013)

    Article  Google Scholar 

  24. Z. Wang, F. Qi, On ambiguities in super-resolution modeling. IEEE Signal Process. Lett. 11(8), 678–681 (2004)

    Article  Google Scholar 

  25. M. Watanabe, Y. Sakuta, T. Goto, S. Hirano, M. Sakurai, Super-Resolution Image Processing with Total Variation Regularization and Shock Filters, in Proceedings of 1st IEEE Global Conference on Consumer Electronics (GCCE) (IEEE, Tokyo, Japan, 2012), pp. 570–571

  26. W. Zeng, X. Lu, A generalized DAMRF image modeling for superresolution of license plates. IEEE Trans. Intell. Transp. Syst. 13(2), 828–837 (2012)

    Article  Google Scholar 

  27. X. Zeng, L. Yang, A robust multiframe super-resolution algorithm based on half-quadratic estimation with modified BTV regularization. Digit. Signal Process 23(1), 98–109 (2013)

    Article  MathSciNet  Google Scholar 

  28. T. Zhang, X. Yu, A Contourlet-Based Image Super-Resolution Approach, in Proceedings of 2012 International Conference on Audio, Language and Image Processing (ICALIP 2012) (IEEE, Shanghai, China, 2012), pp. 375–378

  29. W. Zhou, A. Bovik, H. Sheikh, E. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 1–14 (2004)

    Article  Google Scholar 

Download references

Acknowledgments

We are thankful to the anonymous reviewers for their constructive suggestions which helped us improving our manuscript. This work is partly supported by the Hubei Province Natural Science Foundation (No. 2013CFB152) and partly supported by the Ph.D. Programs Foundation of Ministry of Education of China (No. 20120142120110).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shengrong Zhao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, C., Liang, H., Zhao, S. et al. Integrating the Missing Information Estimation into Multi-frame Super-Resolution. Circuits Syst Signal Process 35, 1213–1238 (2016). https://doi.org/10.1007/s00034-015-0114-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-015-0114-5

Keywords

Navigation