Skip to main content
SpringerLink
Log in

Region covariance based total variation optimization for structure-texture decomposition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With conventional structure-preserving filters, it is not always easy to remove texture details and preserve important structures from images of high complexity. To enhance the performance of structure-texture decomposition, a new method based on region covariance of simple image features and total variation optimization is proposed in this paper. We first identify texture from structure by patch-based covariance, which shows highly discriminative power for textures. Then, a total variation model built on the joint covariance and gradient is used for structure-preserving smoothing. To overcome the inherent limitation of covariance descriptor in locating main structures, patch shifting based on the variation of the region covariance is introduced. We compare our approach with state-of-the-art structure-preserving decomposition methods and the results show that our approach outperforms them in removing unimportant texture details while preserving main structures. Even for images containing the mixture of high-contrast textures with obscure boarders between them, our approach still can improve the decomposition at few extra cost of computation. Besides better decomposition results and robustness for various types of images, the simplicity of our approach make it easy to implement and adaptable to other applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Aujol JF, Chambolle A (2005) Dual norms and image decomposition models. Int J Comput Vis 63(1):85–104

    Article  MATH  Google Scholar 

  2. Aujol JF, Gilboa G, Chan T, Osher S (2006) Structure-texture image decomposition - Modeling, algorithms, and parameter selection. Int J Comput Vis 67(1):111–136

    Article  MATH  Google Scholar 

  3. Bae SM, Paris S, Durand FE (2006) Two-scale tone management for photographic look. ACM T Graphic 25(3):637–645

    Article  Google Scholar 

  4. Bao L, Song Y, Yang Q, Yuan H, Wang G (2013) Tree Filtering: Efficient Structure-Preserving Smoothing With a Minimum Spanning Tree. IEEE Trans Image Process Publ IEEE Signal Process Soc 23(2):555–569

    Article  MathSciNet  MATH  Google Scholar 

  5. Bi S, Han XG, Yu YZ (2015) An L-1 Image Transform for Edge-Preserving Smoothing and Scene-Level Intrinsic Decomposition. ACM T Graphic 34(4):Article No. 78

  6. Bolz J, Farmer I, Grinspun E, Schroder P (2003) Sparse matrix solvers on the GPU: Conjugate gradients and multigrid. ACM T Graphic 22(3):917–924

    Article  Google Scholar 

  7. Chaudhury KN (2015) Fast and Accurate Bilateral Filtering Using Gauss-Polynomial Decomposition. Ieee Image Proc, pp 2005–2009

  8. Cho H, Lee H, Kang H, Lee S (2014) Bilateral Texture Filtering. ACM T Graphic 33(4):Article No. 128

  9. Durand F, Dorsey J (2002) Fast bilateral filtering for the display of high-dynamic-range images. ACM T Graphic 21(3):257–266

    Article  Google Scholar 

  10. Fadili MJ, Starck JL, Bobin J, Moudden Y (2010) Image Decomposition and Separation Using Sparse Representations: An Overview. Proc IEEE 98(6):983–994

    Article  Google Scholar 

  11. Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM T Graphic 27(3):Article No. 67

  12. Fattal R (2009) Edge-Avoiding Wavelets and their Applications. ACM T Graphic 28(3):Article No. 22

  13. Gastal ESL, Oliveira MM (2011) Domain Transform for Edge-Aware Image and Video Processing. ACM T Graphic 30(4):Article No. 69

  14. Gilles J, Meyer Y (2010) Properties of BV - G Structures plus Textures Decomposition Models. Application to Road Detection in Satellite Images. IEEE T Image Process 19(11):2793–2800

    Article  MATH  Google Scholar 

  15. Gunturk BK (2011) Fast Bilateral Filter With Arbitrary Range and Domain Kernels. IEEE T Image Process 20(9):2690–2696

    Article  MathSciNet  MATH  Google Scholar 

  16. Guo D, Cheng YA, Zhuo SJ, Sim T (2010) Correcting Over-Exposure in Photographs. 2010 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr). pp 515–521

  17. Ham B, Cho M, Ponce J (2015) Robust image filtering using joint static and dynamic guidance. In: Computer Vision and Pattern Recognition. pp 4823–4831

  18. Hays J, Leordeanu M, Efros AA, Liu Y (2006) Discovering Texture Regularity as a Higher-Order Correspondence Problem. Springer, Berlin Heidelberg

    Book  Google Scholar 

  19. He KM, Sun J, Tang XO (2013) Guided Image Filtering. IEEE T Pattern Anal 35(6):1397–1409

    Article  Google Scholar 

  20. Hong XP, Chang H, Shan SG, Chen XL, Gao W (2009) Sigma Set: A Small Second Order Statistical Region Descriptor. Proc Cvpr IEEE. pp 1802–1809

  21. Jeon J, Lee H, Kang H, Lee S (2016) Scale-aware Structure-Preserving Texture Filtering. Comput Graph Forum 35(7):77–86

    Article  Google Scholar 

  22. Jia JY, Sun J, Tang CK, Shum HY (2006) Drag-and-drop pasting. ACM T Graphic 25(3):631–636

    Article  Google Scholar 

  23. Karacan L, Erdem E, Erdem A (2013) Structure-Preserving Image Smoothing via Region Covariances. ACM T Graphic 32(6):Article No. 176

  24. Kass M, Solomon J (2010) Smoothed Local Histogram Filters. ACM T Graphic 29(4):Article No. 100

  25. Kopf J, Cohen MF, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. ACM T Graphic 26(3):Article No. 96

  26. Krishnan D, Fattal R, Szeliski R (2013) Efficient Preconditioning of Laplacian Matrices for Computer Graphics. ACM T Graphic 32(4)

  27. Kyprianidis JE, Kang H (2011) Image and Video Abstraction by Coherence-Enhancing Filtering. Comput Graph Forum 30(2):593–602

    Article  Google Scholar 

  28. Lischinski D, Farbman Z, Uyttendaele M, Szeliski R (2006) Interactive local adjustment of tonal values. ACM T Graphic 25(3):646–653

    Article  Google Scholar 

  29. Liu Y, Lin WC, Hays J (2004) Near-regular texture analysis and manipulation. ACM T Graphic 23(3):368–376

    Article  Google Scholar 

  30. Liu QG, Liu JB, Dong P, Liang D (2013) SGTD: Structure Gradient and Texture Decorrelating Regularization for Image Decomposition. IEEE I Conf Comp Vis. pp 1081–1088

  31. Ojala T, Pietikäinen M, Mäenpää T (2000) Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns. Springer, Berlin Heidelberg

    Book  MATH  Google Scholar 

  32. Ono S, Miyata T, Yamada I (2014) Cartoon-Texture Image Decomposition Using Blockwise Low-Rank Texture Characterization. IEEE T Image Process 23(3):1128–1142

    Article  MathSciNet  MATH  Google Scholar 

  33. Osher S, Sole A, Vese L (2003) Image decomposition, image restoration, and texture modeling using total variation minimization and the H-1 norm. 2003 International Conference on Image Processing, Vol 1, Proceedings. pp 689–692

  34. Paris S, Durand F (2009) A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach. Int J Comput Vis 81(1):24–52

    Article  Google Scholar 

  35. Paris S, Hasinoff SW, Kautz J (2011) Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid. ACM T Graphic 30(4)

  36. Rudin LI, Osher S, Fatemi E (1992) Nonlinear Total Variation Based Noise Removal Algorithms. Physica D 60(1–4):259–268

    Article  MathSciNet  MATH  Google Scholar 

  37. Schaeffer H, Osher S (2013) A Low Patch-Rank Interpretation of Texture. Siam J Imaging Sci 6(1):226–262

    Article  MathSciNet  MATH  Google Scholar 

  38. Su Z, Luo X, Deng Z, Liang Y, Ji Z (2013) Edge-Preserving Texture Suppression Filter Based on Joint Filtering Schemes. IEEE Trans Multimedia 15(3):535–548

    Article  Google Scholar 

  39. Subr K, Soler C, Durand F (2009) Edge-preserving Multiscale Image Decomposition based on Local Extrema. ACM T Graphic 28(5):Article No. 147

  40. Szeliski R (2006) Locally adapted hierarchical basis preconditioning. ACM T Graphic 25(3):1135–1143

    Article  Google Scholar 

  41. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Computer Vision, 1998. Sixth International Conference on, 4–7 Jan 1998. pp 839–846

  42. Tschumperle D (2006) Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE's. Int J Comput Vis 68(1):65–82

    Article  Google Scholar 

  43. Winnemoller H, Olsen SC, Gooch B (2006) Real-time video abstraction. ACM T Graphic 25(3):1221–1226

    Article  Google Scholar 

  44. Xu L, Lu CW, Xu Y, Jia JY (2011) Image Smoothing via L-0 Gradient Minimization. ACM T Graphic 30(6)

  45. Xu L, Yan Q, Xia Y, Jia JY (2012) Structure Extraction from Texture via Relative Total Variation. ACM T Graphic 31(6):Article No. 139

  46. Xu L, Ren JSJ, Yan Q, Liao R, Jia J (2015) Deep edge-aware filters. In: 32nd International Conference on Machine Learning, ICML 2015, July 6, 2015 - July 11, 2015, Lile, France. 32nd International Conference on Machine Learning, ICML 2015. International Machine Learning Society (IMLS), pp 1669–1678

  47. Yang Q (2016) Semantic Filtering. In: IEEE Conference on Computer Vision and Pattern Recognition. pp 4517–4526

  48. Yang Q, Tan KH, Ahuja N (2009) Real-Time O(1) Bilateral Filtering. In: IEEE Conference on Computer Vision and Pattern Recognition. pp 557–564

  49. Yin WT, Goldfarb D, Osher S (2005) Image cartoon-texture decomposition and feature selection using the total variation regularized L-1 functional. Lect Notes Comput Sci 3752:73–84

    Article  MATH  Google Scholar 

  50. Zhang B, Allebach JP (2008) Adaptive bilateral filter for sharpness enhancement and noise removal. IEEE T Image Process 17(5):664–678

    Article  MathSciNet  Google Scholar 

  51. Zhang Q, Xu L, Jia J (2014) 100+ Times Faster Weighted Median Filter (WMF). In: Computer Vision and Pattern Recognition. pp 2830–2837

  52. Zhang Q, Shen X, Xu L, Jia J (2014) Rolling Guidance Filter. In: European Conference on Comput. Vis. (ECCV), LNCS, vol. 8691:815–830

  53. Zhano M, Gunturk BK (2008) Multiresolution Bilateral Filtering for Image Denoising. IEEE T Image Process 17(12):2324–2333

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This research was partially supported by the National Natural Science Foundation of China (Grant No. 61540062) and the Science Foundation of Yunnan Provincial Department of Education (Grant No. 2015Y018).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Yunnan University, Kunming, China

    Hao Wu, Dan Xu & Guowu Yuan

Authors
  1. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guowu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Wu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, H., Xu, D. & Yuan, G. Region covariance based total variation optimization for structure-texture decomposition. Multimed Tools Appl 77, 16985–17005 (2018). https://doi.org/10.1007/s11042-017-5266-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5266-9

Keywords

Search

Navigation