Elsevier

Digital Signal Processing

Volume 49, February 2016, Pages 162-181
Digital Signal Processing

An edge-weighted second order variational model for image decomposition

https://doi.org/10.1016/j.dsp.2015.10.010Get rights and content

Abstract

Decomposing an image into structure and texture is an important procedure for image understanding and analysis. Structure retains object hues and sharp edges whilst texture contains oscillating patterns of an observed image. The classical Vese–Osher model has been used for image decomposition, but its resulting structure image tends to show the undesirable staircase effect. Second order variational models that use a bounded Hessian regulariser have been proposed to remedy this side effect, but they tend to blur edges of objects in structure components. In this paper, we propose an edge-weighted second order variational model for image decomposition, which is able to eliminate staircase effects and preserve object edges. To avoid directly calculating the high order nonlinear partial differential equations of the proposed model, a fast split Bregman algorithm is developed, which uses the fast Fourier transform and analytical generalised soft thresholding equations. Extensive experiments demonstrate that the proposed variational image decomposition model outperforms state-of-the-art first and second order image decomposition models. By removing the texture component from the original noisy image, the effectiveness of the proposed model for image denoising has also been validated.

Introduction

Image decomposition into structure and texture has been used, for instance, for image similarity analysis [1], texture synthesis [2], texture image segmentation [3], texture decomposition and feature selection [4], [5], [6], and structure and texture image inpainting [7], etc.

A popular method for image decomposition is the variational approach. Vese and Osher proposed a first order variational model (FOVO) [8] for image decomposition. However, the FOVO model suffers from the undesirable staircase effect, that is, the resulting structure image has a jagged appearance. This is because the energy minimisation in FOVO is in the bounded variation (BV) space [9], which results in a piecewise constant function leading to the staircase side effect. Recently, a second order regulariser defined in bounded Hessian (BH) space, the BH regulariser, has been employed to remedy this side effect [10], [11], [12], [13], [14]. Compared with other non-convex high order regularisers, such as the mean curvature [15], [16], [17] and Euler-elastica [18], [19] etc., the BH regulariser is a convex high order extension of the total variation (TV) regulariser [9] which is less dependent on initialisation. Compared with the convex total generalised variation (TGV) regulariser [20], [21], [22], the BH regulariser is also more efficient to implement. However, the edges of objects in the resulting structure image are often blurred due to the fact that the model imposes too much regularity on the image. In addition, the Euler-equations of the BH regulariser are fourth order nonlinear partial differential equations (PDEs), which are very difficult to discretise to solve computationally.

In this paper, we propose an edge-weighted second order (EWSO) variational model for image decomposition to overcome the problems with the existing first and second order models mentioned above. Preliminary results of this work have been presented in a conference [24]. Instead of using the TV regulariser as in the FOVO model, the proposed EWSO model uses the BH regulariser together with an edge diffusivity function. The former aims to remove staircase effects, whilst the latter aims to preserve object edges. The split Bregman algorithm [23] is adapted to improve the computational speed by transforming the energy minimisation problem of the proposed EWSO model into four subproblems, which are then efficiently solved using the fast Fourier transform (FFT) and analytical soft thresholding equations without any iterations. The new model is validated through extensive experiments. Experimental results show that the proposed new model outperforms the state-of-the-art variational methods for both image decomposition and image denoising. The contributions of the paper are threefold: 1) A new second order variational image decomposition model is proposed; 2) A fast split Bregman algorithm is developed for image decomposition based on a finite difference scheme; 3) The new image decomposition model is used for image denoising applications.

The paper is organised as follows: Section 2 sets the background for the paper by introducing some existing variational models and their drawbacks for image decomposition. Section 3 introduces the proposed second order variational model for image decomposition. Section 4 describes the proposed split Bregman algorithm for solving the variational model efficiently. Section 5 gives details of the experiments using the proposed model for image decomposition and denoising. Section 6 concludes the paper. Implementation details of the proposed split Bregman algorithm are given in Appendix.

Section snippets

Background

In image decomposition, an image f is split into two components f=u+v where u represents structure containing object edges and hues, and v represents texture (noise or oscillating patterns).

The proposed edge-weighted second order model

From Section 2.1, the function u in the FOVO model defined in the BV(Ω) space can lead to the staircase effect in the structure image. Though high order models such as the INFCON, CFS and TGV models introduced in Section 2.2 can eliminate this side effect, they become less efficient due to the complicated coupling of the first and second order derivatives and more penalty parameters used in these models. Second order models that directly use the BH regulariser can remedy the staircase effect

The proposed split Bregman for image decomposition

In Goldstein–Osher [23], a fast iterative algorithm was proposed for the TV model. It is one of the most efficient numerical schemes for solving the L1-based variational models [30], [31], [38]. In addition, it has been recently rediscovered as an equivalent form of the alternating direction method of multipliers (ADMM). In this paper, the algorithm is adapted for solving the proposed EWSO model. The basic idea is to first split the original minimisation problem into several subproblems by

Experimental results

In this section, experiments are conducted to select the best parameters for the proposed model, as well as to illustrate the efficiency and effectiveness of the proposed model and to compare it with the existing models for image decomposition and denoising. All the experiments are performed using Matlab 2014b on a Windows 7 platform with an Intel Xeon CPU E5-1620 with 3.70 GHz and 32 GB memory.

Conclusion

In this paper, a new variational model is introduced for image decomposition, which can decompose an image into structure and texture without causing staircase effect or blurring object edges. The model is used for image denoising. By separating the optimisation problem into subproblems and minimising them sequentially using a split Bregman algorithm, the proposed model does not need to solve the high order PDEs and thus it is computationally efficient. Experimental comparison of the proposed

Acknowledgements

We thank the Editor and anonymous reviewers for their valuable comments and suggestions on our manuscript. We thank Prof. Xue-Cheng Tai of the University of Bergen, Norway for providing the Euler's elastica code. We thank Prof. Irene Gottlob and Dr. Frank Proudlock of the University of Leicester, UK for providing the OCT image for this research. The copyright of the image Fig. 11(a) is owned by Dr. Konstantinos Papaftsoros of the University of Cambridge.

Jinming Duan received the BSc degree from the Nanjing University of information science and technology, PR China, in 2011, and the MSc degree from the Qingdao University, PR China, in 2014. He is currently pursuing the PhD degree in computer science in the University of Nottingham, UK. His research interests include variational image restoration, multiphase image segmentation, implicit surface reconstruction, medical image analysis.

References (38)

  • M. Bertalmio et al.

    Simultaneous structure and texture image inpainting

    IEEE Trans. Image Process.

    (2003)
  • L. Vese et al.

    Modeling textures with total variation minimization and oscillating patterns in image processing

    J. Sci. Comput.

    (2003)
  • M. Lysaker et al.

    Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time

    IEEE Trans. Image Process.

    (2003)
  • M. Bergounioux et al.

    A second-order model for image denoising

    Set-Valued Var. Anal.

    (2010)
  • R. Lai et al.

    A ridge and corner preserving model for surface restoration

    SIAM J. Sci. Comput.

    (2013)
  • J. Duan et al.

    Second order Mumford–Shah model for image denoising

  • W. Zhu et al.

    Image denoising using mean curvature of image surface

    SIAM J. Imaging Sci.

    (2012)
  • W. Zhu et al.

    Augmented Lagrangian method for a mean curvature based image denoising model

    Inverse Probl. Imaging

    (2013)
  • X. Tai

    Fast numerical schemes related to curvature minimization: a brief and elementary review

    (May 2014)
  • Cited by (49)

    • Generalized multi-scale image decomposition for new tone manipulation

      2023, Digital Signal Processing: A Review Journal
    • Cartoon-Texture decomposition with patch-wise decorrelation

      2023, Journal of Visual Communication and Image Representation
    • Unsupervised learning of multi-task deep variational model

      2022, Journal of Visual Communication and Image Representation
    • Learned prior-guided algorithm for flow field visualization in electrical capacitance tomography

      2022, Digital Signal Processing: A Review Journal
      Citation Excerpt :

      By using variable substitution and alternating optimization methods, this approach enables a complex optimization problem to be separated into several simpler sub-problems that can be solved flexibly. The performance, convergence, computational complexity and application of the algorithm have been extensively discussed and studied, and more details can be found in [17,80–82]. Our model uses deep learning techniques to fit low-fidelity samples because it excels at processing high-dimensional image data and eliminating the need to use handcrafted feature engineering.

    View all citing articles on Scopus

    Jinming Duan received the BSc degree from the Nanjing University of information science and technology, PR China, in 2011, and the MSc degree from the Qingdao University, PR China, in 2014. He is currently pursuing the PhD degree in computer science in the University of Nottingham, UK. His research interests include variational image restoration, multiphase image segmentation, implicit surface reconstruction, medical image analysis.

    Zhaowen Qiu received the BSc degree from the Heilongjiang University, PR China, in 1997, and the MSc and PhD degrees from Harbin Institute of Technology, in 2003 and 2009, respectively. He was a senior visiting scholar with the Carnegie Mellon University, USA from 2009 to 2010. He is currently an associate professor in the Northeast Forestry University, PR China. His research interests include information retrieval, machine learning, medical image analysis.

    Wenqi Lu received the BSc in the Department of Information Science and Engineering from the Shandong University of Science and Technology, PR China, in 2013, and she is currently pursuing the MSc degree in the Qingdao University, PR China. Her research interest includes three-dimensional image processing.

    Guodong Wang received the BSc and MSc degrees in control technology and control engineering from the Qingdao Science and Technology University, PR China, in 2001 and 2004, respectively, and the PhD degree in pattern recognition and intelligent systems from the Huazhong University of Science and Technology, in 2008. He is an Associate Professor of the College of Information Engineering at Qingdao University, China. His research interests include biometrics, image processing, intelligent video monitoring and analysis.

    Zhenkuan Pan received the BSc degree in engineering mechanics from the Northwestern Polytechnical University, PR China, in 1987, and the PhD degree in engineering mechanics from Shanghai Jiao Tong University, PR China, in 1992. He is currently a Professor of the College of Information Engineering at Qingdao University, China. From 2005 to 2006, he was a senior visiting scholar with the Department of Mathematics, the University of California, Los Angeles. His research interests include image processing, variational image science, and three-dimensional reconstruction.

    Li Bai received the PhD degree in computer science from the University of Nottingham, UK, in 1994. She was a mathematician in China before she joined the University of Nottingham, in 1989. Her research interests include pattern recognition, computer vision, advanced interfaces, and medical imaging.

    View full text