Skip to main content
SpringerLink
Log in

A structural low rank regularization method for single image super-resolution

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Example-learning-based algorithms such as those based on sparse coding or neighbor embedding have been popular for single image super-resolution in recent years. However, affected by several critical factors on the training data and example representation, their reconstructions are usually plagued by kinds of artifacts. The removing of these artifacts is one of the major tasks for these methods. Unlike most existing methods that employ more complicated training methods, in this paper we would like to recover a clear reconstruction by fusing several “dirty” coarse reconstructions which are outputs of one or several simple training methods with small training set. One underlying key observation is that although coarse reconstructions are corrupted by different artifacts, they refer to the same high-resolution image. This global structure information is captured by an image structure-based low rank regularization method. The advantage of our method is that it can remove not only small noises but also gross artifacts. Except sparsity and randomness of the large artifacts, no other knowledge about them is required. Experimental results show that the proposed method can not only dramatically improve coarse reconstructions but also achieve competitive results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Aharon, M., Elad, M., Bruckstein, A.: K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  2. Bertsekas, D.: Constrained Optimization and Lagrange Multiplier Method. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  3. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  4. Buades, A., Coll, B., Morel, J.M.: Nonlocal image and movie denoising. Int. J. Comput. Vis. 76(2), 123–139 (2008)

    Article  Google Scholar 

  5. Cai, J.F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2008)

    Article  Google Scholar 

  6. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? IEEE Trans. Signal Process. 58(1), 1–37 (2009)

    Google Scholar 

  7. Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., pp. 275–282 (2004)

  8. Chen, X., Qi, C.: Low-rank neighbor embedding for single image super-resolution. IEEE Signal Process. Lett. 21(1), 79–82 (2014)

    Article  Google Scholar 

  9. Dong, W., Zhang, L., Shi, G.: Centralized sparse representation for image restoration. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1259–1266. IEEE (2011)

  10. Dong, W., Zhang, L., Shi, G., Wu, X.: Image deblurring and supper-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. Image Process. 20(7), 1838–1857 (2011)

    Article  MathSciNet  Google Scholar 

  11. Elad, M., Figueiredo, M.A.T., Ma, Y.: On the role of sparse and redundant representations in image processing. IEEE Proc. Spec. Issue Appl. Sparse Represent. Compress. Sens. 98, 972–982 (2010)

  12. Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based superresolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  13. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Article  MATH  Google Scholar 

  14. Gao, X.B., Zhang, K.B., Tao, D.C., Li, X.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21(7), 3194–3205 (2012)

  15. Hale, E.T., Yin W., Zhang, Y.: A fixed-point continuation method for l1-regularized minimization with applications to compressed sensing. CAAM TR07-07, Rice University, 43:44 (2007)

  16. Hale, E.T., Yin, W., Zhang, Y.: Fixed-point continuation for l1-minimization: methodology and convergence. SIAM J. Optim. 19(3), 1107–1130 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  17. He, L., Qi, H., Zaretzki, R.: Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. Comput. Vis. Pattern Recogn. 1, 345–353 (2013)

    Google Scholar 

  18. Huang, T.S., Tsai, R.Y.: Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process. 1(2), 317–339 (1984)

    Google Scholar 

  19. Irani, M., Peleg, S.: Motion analysis for image enhancement: resolution, occlusion and transparency. J. Vis. Commun. Image Represent. 4(4), 324–335 (1993)

    Article  Google Scholar 

  20. Izenman, A.J.: Modern Multivariate Statistical Techniques: Regression, Classiffication, and Manifold Learning. Springer, New York (2008)

    Book  Google Scholar 

  21. Kim, C., Choi, K., Ra, J.B.: Example-based super-resolution via structure analysis of patches. IEEE Signal Process. Lett. 20(4), 407–410 (2013)

    Article  Google Scholar 

  22. Kim, K.I., Kwon, Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1127–1133 (2010)

    Article  MathSciNet  Google Scholar 

  23. Larsen, R.M.: Propack-software for large and sparse svd calculations (2005). http://sun.stanford.edu/~rmunk/PROPACK/. Accessed 12 APR 2014

  24. Li, X., Orchard, M.T.: New edge-directed interpolation. IEEE Trans. Image Process. 10(10), 1521–1527 (2001)

    Article  Google Scholar 

  25. Lin, Z., He, J., Tang, X., Tang, C.K.: Limits of learning-based super resolution algorithms. Int. J. Comput. Vis. 80(3), 406–420 (2008)

    Article  Google Scholar 

  26. Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. Technical report, University of Illinois at Urbana-Champaign, Urbana-Champaign, IL (2010)

  27. Lin, Z., Liu, R., Su, Z.: Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in Neural Information Processing Systems, pp. 612–620 (2011)

  28. Lu, X., Yuan, H., Yan, P., Yuan, Y., Li, X.: Geometry constrained sparse coding for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1648–1655 (2012)

  29. Marquina, A., Osher, S.J.: Image super-resolution by tv-regularization and bregman iteration. J. Sci. Comput. 37(3), 367–382 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  30. Park, S., Park, M., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)

    Article  Google Scholar 

  31. Pati, Y.C., Rezaifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: Proc. 27th Asilomar Conf. Signals, Syst. Comput., pp. 40–44 (1993)

  32. Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: Proc. the 10th European Conference on Computer Vision, vol. III, pp. 57–68 (2008)

  33. Protter, M., Elad, M., Takeda, H., Milanfar, P.: Generalizing the nonlocal-means to super-resolution reconstruction. IEEE Trans. Image Process. 18(1), 36–51 (2009)

    Article  MathSciNet  Google Scholar 

  34. Recht, B., Fazel, M., Parrilo, P.: Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52(3), 471–501 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  35. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)

    Article  Google Scholar 

  36. Sun, J., Sun, J., Xu, Z., Shum, H.: Image super-resolution using gradient profile prior. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8 (2008)

  37. Takeda, H., Farsiu, S., Milanfar, P.: Kernel regression for image processing and reconstruction. IEEE Trans. Image Process. 16(2), 349–366 (2007)

    Article  MathSciNet  Google Scholar 

  38. Wang, Z., Bovik, A.C., Sheikh, H.R.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  39. Xu, H., Caramanis, C., Mannor, S.: Sparse algorithms are not stable: a no-free-lunch theorem. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 187–193 (2012)

    Article  MathSciNet  Google Scholar 

  40. Xu, H., Zhai, G., Yang, X.: Single image super-resolution with detail enhancement based on local fractal analysis of gradient. IEEE Trans. Circuits Syst. Video Technol. 23(10), 1740–1754 (2013)

    Article  Google Scholar 

  41. Yang, J., Lin, Z., Cohen, S.: Fast image super-resolution based on in-place example regression. In: Computer Vision and Pattern Recognition (CVPR), pp. 1059–1066. IEEE (2013)

  42. Yang, J.C., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8 (2008)

  43. Yang, J.C., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  44. Yang, J.C., Wang, Z.W., Lin, Z., Cohen, S., Huang, T.: Coupled dictionary training for image super-resolution. IEEE Trans. Image Process. 21(8), 3467–3478 (2011)

    Article  MathSciNet  Google Scholar 

  45. Yang, S., Wang, M., Chen, Y., Sun, Y.: Single-image super-resolution reconstruction via learned geometric dictionaries and clustered sparse coding. IEEE Trans. Image Process. 21(9), 4016–4028 (2012)

    Article  MathSciNet  Google Scholar 

  46. Yang, M., Wang, Y.: A self-learning approach to single image super-resolution. IEEE Trans. Multimed. 15(3), 498–508 (2013)

    Article  Google Scholar 

  47. Yu, J., Gao, X., Tao, D., Li, X., Zhang, K.: A unified learning framework for single image super-resolution. IEEE Trans. Neural Netw. Learn. Syst. 20(4), 407–410 (2013)

    Google Scholar 

  48. Zhang, H., Yang, J., Zhang, Y., Huang, T.S.: Non-local kernel regression for image and video restoration. In: Computer Vision-ECCV 2010, pp. 566–579. Springer, Berlin (2010)

  49. Zhang, L., Wu, X.: An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans. Image Process. 15(8), 2226–2238 (2006)

    Article  Google Scholar 

Download references

Acknowledgments

The work described in this paper is supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 11401231, 91330105), Science Foundation of Fujian Province (No. 2015J01254) and Science Technology Foundation for Middle-aged and Young Teacher of Fujian Province (No. JA14021).

Author information

Authors and Affiliations

  1. The School of Computer Science and Technology, Huaqiao University, Xiamen, 361021, China

    Jialin Peng

  2. Department of Mathematics, City University of Hong Kong, Hong Kong, SAR, China

    Benny Y. C. Hon

  3. Department of mathematics, Zhejiang University, Hangzhou, China

    Dexing Kong

Authors
  1. Jialin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benny Y. C. Hon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dexing Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jialin Peng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, J., Hon, B.Y.C. & Kong, D. A structural low rank regularization method for single image super-resolution. Machine Vision and Applications 26, 991–1005 (2015). https://doi.org/10.1007/s00138-015-0712-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-015-0712-z

Keywords

Search

Navigation