Skip to main content

Advertisement

Springer Nature Link
Log in

Texture Repairing by Unified Low Rank Optimization

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, we show how to harness both low-rank and sparse structures in regular or near-regular textures for image completion. Our method is based on a unified formulation for both random and contiguous corruption. In addition to the low rank property of texture, the algorithm also uses the sparse assumption of the natural image: because the natural image is piecewise smooth, it is sparse in certain transformed domain (such as Fourier or wavelet transform). We combine low-rank and sparsity properties of the texture image together in the proposed algorithm. Our algorithm based on convex optimization can automatically and correctly repair the global structure of a corrupted texture, even without precise information about the regions to be completed. This algorithm integrates texture rectification and repairing into one optimization problem. Through extensive simulations, we show our method can complete and repair textures corrupted by errors with both random and contiguous supports better than existing low-rank matrix recovery methods. Our method demonstrates significant advantage over local patch based texture synthesis techniques in dealing with large corruption, non-uniform texture, and large perspective deformation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Efros A A, Leung T K. Texture synthesis by non-parametric sampling. In Proc. the 7th IEEE International Conference on Computer Vision (ICCV), September 1999, pp.1033-1038.

  2. Efros A A, Freeman W T. Image quilting for texture synthesis and transfer. In Proc. the 28th SIGGRAPH, Aug. 2001, pp.341-346.

  3. Liang L, Liu C, Xu Y, Guo B, Shum H. Real-time texture synthesis by patchbased sampling. ACM Transactions on Graphics (TOG), 2001, 20(3): 127–150.

    Article  Google Scholar 

  4. Komodakis N, Tziritas G. Image completion using global optimization. In Proc. the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June 2006, pp.442-452.

  5. Sun J, Yuan L, Jia J, Shum H Y. Image completion with structure propagation. ACM Transactions on Graphics (TOG), 2005, 24(3): 861–868.

    Article  Google Scholar 

  6. Criminisi A, Pérez P, Toyama K. Object removal by exemplar-based inpainting. In Proc. the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), June 2003, pp.721-728.

  7. Liu Y, Lin W C, Hays J. Near-regular texture analysis and manipulation. ACM Transactions on Graphics (TOG), 2004, 23(3): 368–376.

    Article  Google Scholar 

  8. Bertalmio M, Sapiro G, Caselles V, Ballester C. Image inpainting. In Proc. the 27th SIGGRAPH, July 2000, pp.417-424.

  9. Bertalmio M, Bertozzi A L, Sapiro G. Navier–Stokes, fluid dynamics, and image and video inpainting. In Proc. the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Dec. 2001, pp.355-362.

  10. Oliveira M M, Bowen B, Mckenna R, Chang Y. Fast digital image inpainting. In Proc. the International Conference on Visualization, Imaging and Image Processing (VIIP), September 2001, pp.261-266.

  11. Levin A, Zomet A, Weiss Y. Learning how to inpaint from global image statistics. In Proc. the 9th IEEE International Conference on Computer Vision (ICCV), October 2003, p.305.

  12. Mairal J, Elad M, Sapiro G. Sparse representation for color image restoration. IEEE Trans. Image Processing (TIP), 2007, 17(1): 53–69.

  13. Fadili M J, Starck J, Murtagh F. Inpainting and zooming using sparse representations. The Computer Journal, 2009, 52(1): 64–79.

    Article  Google Scholar 

  14. Elad M, Starck J, Querre P, Donoho D. Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Applied Computational Harmonic Analysis, 2005, 19(3): 340–358.

    Article  MathSciNet  MATH  Google Scholar 

  15. Bugeau A, Bertalmío M, Caselles V, Sapiro G. A comprehensive framework for image inpainting. IEEE Trans. Image Processing (TIP), 2010, 19(10): 2634–2645.

  16. Kwatra V, Schödl A, Essa I, Turk G, Bobick A. Graphcut textures: Image and video synthesis using graph cuts. ACM Transactions on Graphics (TOG), 2003, 22(3): 277–286.

  17. Hays J, Efros A A. Scene completion using millions of photographs. ACM Transactions on Graphics (TOG), 2007, 26(3): Article No. 4.

  18. Barnes C, Shechtman E, Finkelstein A, Goldman D B. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (TOG), 2009, 28(3): Article No. 24.

  19. Pritch Y, Kav-Venaki E, Peleg S. Shift-map image editing. In Proc. the 12th Internetional Conference on Computer Vision (ICCV), October 2009, pp.151-158.

  20. Candès E, Li X, Ma Y, Wright J. Robust principal component analysis? Journal of the ACM, 2011, 58(3): Article No. 11.

  21. Zhou T Y, Tao D C. GoDec: Randomized lowrank and sparse matrix decomposition in noisy case. In Proc. the 28th International Conference on Machine Learning (ICML), Jun. 28-Jul. 2, 2011, pp.33-40.

  22. Bao B, Liu G, Xu C, Yan S. Inductive robust principal component analysis. IEEE Trans. Image Processing (TIP), 2012, 21(8): 3794–3800.

  23. Zhang Z, Yan S, Zhao M. Similarity preserving low-rank representation for enhanced data representation and effective subspace learning. Neural Networks, 2014, 53: 81–94.

    Article  MATH  Google Scholar 

  24. Peng Y, Ganesh A, Wright J, Xu W, Ma Y. RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Analysis and Machine Intelligence, 2012, 34(11): 2233–2246.

    Article  Google Scholar 

  25. Zhang Z, Yan S, Zhao M, Li F. Bilinear low-rank coding framework and extension for robust image recovery and feature representation. Knowledge-Based Systems, 2015, 86: 143–157.

    Article  Google Scholar 

  26. Zhang Z, Liu C, Zhao M. A sparse projection and lowrank recovery framework for handwriting representation and salient stroke feature extraction. ACM Trans. Intelligent Systems and Technology, 2015, 6(1): 9:1–9:26.

  27. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Analysis and Machine Intelligence, 2013, 35(1): 171–184.

    Article  Google Scholar 

  28. Gonzalez R C, Woods R E, Eddins S L. Digital Image Processing Using MATLAB (2nd edition). Gatesmark Publishing, 2009.

  29. Cai J, Chan R H, Nikolova M. Fast two-phase image deblurring under impulse noise. Journal of Mathematical Imaging and Vision, 2010, 36: 46–53.

    Article  MathSciNet  Google Scholar 

  30. Wang Y, Szlam A, Lerman G. Robust locally linear analysis with applications to image denoising and blind inpainting. SIAM Journal on Imaging Sciences, 2013, 6(1): 526–562.

    Article  MathSciNet  MATH  Google Scholar 

  31. Yan M. Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting. SIAM Journal on Imaging Sciences, 2013, 6(3): 1227–1245.

    Article  MathSciNet  MATH  Google Scholar 

  32. Liang X, Ren X, Zhang Z, Ma Y. Repairing sparse low-rank texture. In Proc. ECCV, October 2012, pp.482-495.

  33. Candès E J, Recht B. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 2009, 9(6): 717–772.

    Article  MathSciNet  MATH  Google Scholar 

  34. Chandrasekaran V, Sanghavi S, Parrilo P A, Willsky A S. Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization, 2011, 21(2): 572–596.

    Article  MathSciNet  MATH  Google Scholar 

  35. Cai J, Candès E, Shen Z. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 2010, 20(4): 1956–1982.

    Article  MathSciNet  MATH  Google Scholar 

  36. Toh K, Yun S. An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pacific Journal of Optimization, 2010, 6(3): 615–640.

    MathSciNet  MATH  Google Scholar 

  37. Ganesh A, Lin Z, Wright J, Wu L, Chen M, Ma Y. Fast algorithms or recovering a corrupted low-rank matrix. In Proc. the 3rd International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, December 2009, pp.203-216.

  38. Lin Z, Chen M, Wu L, Ma Y. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical Report, UILU-ENG-09-2215. arxiv: 1009.5055, 2009. http://arvix.org/abs/1109.0367, Apr. 2016.

  39. Bertsekas D P. Nonlinear Programming (2nd edition). Athena Scientific, 2004.

  40. Yang J, Yuan X. Linearized augmented lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 2013, 82(281): 301–329.

    Article  MathSciNet  MATH  Google Scholar 

  41. Lin Z, Liu R, Su Z. Linearized alternating direction method with adaptive penalty for low-rank representation. In Advances in Neural Information Processing Systems, Shawe-Taylor J, Zemel R S, Bartlett P L (eds.), 2011, pp.612-620.

  42. Liu J, Musialski P, Wonka P, Ye J. Tensor completion for estimating missing values in visual data. In Proc. the 12th IEEE Internetional Conference on Computer Vision (ICCV), Sept. 27-Oct. 4, 2009, pp.2114-2121.

  43. Zhou Z, Wagner A, Mobahi H, Wright J, Ma Y. Face recognition with contiguous occlusion using Markov random fields. In Proc. the 12th IEEE International Conference on Computer Vision (ICCV), Sept. 29-Oct. 2, 2009, pp.1050-1057.

  44. Zhang Z, Liang X, Ganesh A, Ma Y. TILT: Transforminvariant low-rank textures. International Journal of Computer Vision (IJCV), 2012, 99(1): 1–24.

    Article  MathSciNet  MATH  Google Scholar 

  45. Zhang Z, Liang X, Ma Y. Unwrapping low-rank textures on generalized cylindrical surfaces. In Proc. the International Conference on Computer Vision (ICCV), November 2011, pp.1347-1354.

  46. Barnes C, Shechtman E, Goldman D B, Finkelstein A. The generalized Patch-Match correspondence algorithm. In Proc. the 11th European Conference on Computer Vision (ECCV), September 2010, pp.29-43.

Download references

Author information

Authors and Affiliations

  1. Institute for Advanced Study, Tsinghua University, Beijing, 100086, China

    Xiao Liang

  2. Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, 61801, U.S.A.

    Xiang Ren

  3. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, 02139, U.S.A.

    Zhengdong Zhang

  4. School of Information Science and Technology, ShanghaiTech University, Shanghai, 200031, China

    Yi Ma

Authors
  1. Xiao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengdong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao Liang.

Additional information

A preliminary version of the paper was published in the Proceedings of ECCV 2012.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, X., Ren, X., Zhang, Z. et al. Texture Repairing by Unified Low Rank Optimization. J. Comput. Sci. Technol. 31, 525–546 (2016). https://doi.org/10.1007/s11390-016-1645-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-016-1645-3

Keywords