An edge-weighted second order variational model for image decomposition
Introduction
Image decomposition into structure and texture has been used, for instance, for image similarity analysis [1], texture synthesis [2], texture image segmentation [3], texture decomposition and feature selection [4], [5], [6], and structure and texture image inpainting [7], etc.
A popular method for image decomposition is the variational approach. Vese and Osher proposed a first order variational model (FOVO) [8] for image decomposition. However, the FOVO model suffers from the undesirable staircase effect, that is, the resulting structure image has a jagged appearance. This is because the energy minimisation in FOVO is in the bounded variation (BV) space [9], which results in a piecewise constant function leading to the staircase side effect. Recently, a second order regulariser defined in bounded Hessian (BH) space, the BH regulariser, has been employed to remedy this side effect [10], [11], [12], [13], [14]. Compared with other non-convex high order regularisers, such as the mean curvature [15], [16], [17] and Euler-elastica [18], [19] etc., the BH regulariser is a convex high order extension of the total variation (TV) regulariser [9] which is less dependent on initialisation. Compared with the convex total generalised variation (TGV) regulariser [20], [21], [22], the BH regulariser is also more efficient to implement. However, the edges of objects in the resulting structure image are often blurred due to the fact that the model imposes too much regularity on the image. In addition, the Euler-equations of the BH regulariser are fourth order nonlinear partial differential equations (PDEs), which are very difficult to discretise to solve computationally.
In this paper, we propose an edge-weighted second order (EWSO) variational model for image decomposition to overcome the problems with the existing first and second order models mentioned above. Preliminary results of this work have been presented in a conference [24]. Instead of using the TV regulariser as in the FOVO model, the proposed EWSO model uses the BH regulariser together with an edge diffusivity function. The former aims to remove staircase effects, whilst the latter aims to preserve object edges. The split Bregman algorithm [23] is adapted to improve the computational speed by transforming the energy minimisation problem of the proposed EWSO model into four subproblems, which are then efficiently solved using the fast Fourier transform (FFT) and analytical soft thresholding equations without any iterations. The new model is validated through extensive experiments. Experimental results show that the proposed new model outperforms the state-of-the-art variational methods for both image decomposition and image denoising. The contributions of the paper are threefold: 1) A new second order variational image decomposition model is proposed; 2) A fast split Bregman algorithm is developed for image decomposition based on a finite difference scheme; 3) The new image decomposition model is used for image denoising applications.
The paper is organised as follows: Section 2 sets the background for the paper by introducing some existing variational models and their drawbacks for image decomposition. Section 3 introduces the proposed second order variational model for image decomposition. Section 4 describes the proposed split Bregman algorithm for solving the variational model efficiently. Section 5 gives details of the experiments using the proposed model for image decomposition and denoising. Section 6 concludes the paper. Implementation details of the proposed split Bregman algorithm are given in Appendix.
Section snippets
Background
In image decomposition, an image f is split into two components where u represents structure containing object edges and hues, and v represents texture (noise or oscillating patterns).
The proposed edge-weighted second order model
From Section 2.1, the function u in the FOVO model defined in the space can lead to the staircase effect in the structure image. Though high order models such as the INFCON, CFS and TGV models introduced in Section 2.2 can eliminate this side effect, they become less efficient due to the complicated coupling of the first and second order derivatives and more penalty parameters used in these models. Second order models that directly use the BH regulariser can remedy the staircase effect
The proposed split Bregman for image decomposition
In Goldstein–Osher [23], a fast iterative algorithm was proposed for the TV model. It is one of the most efficient numerical schemes for solving the L1-based variational models [30], [31], [38]. In addition, it has been recently rediscovered as an equivalent form of the alternating direction method of multipliers (ADMM). In this paper, the algorithm is adapted for solving the proposed EWSO model. The basic idea is to first split the original minimisation problem into several subproblems by
Experimental results
In this section, experiments are conducted to select the best parameters for the proposed model, as well as to illustrate the efficiency and effectiveness of the proposed model and to compare it with the existing models for image decomposition and denoising. All the experiments are performed using Matlab 2014b on a Windows 7 platform with an Intel Xeon CPU E5-1620 with 3.70 GHz and 32 GB memory.
Conclusion
In this paper, a new variational model is introduced for image decomposition, which can decompose an image into structure and texture without causing staircase effect or blurring object edges. The model is used for image denoising. By separating the optimisation problem into subproblems and minimising them sequentially using a split Bregman algorithm, the proposed model does not need to solve the high order PDEs and thus it is computationally efficient. Experimental comparison of the proposed
Acknowledgements
We thank the Editor and anonymous reviewers for their valuable comments and suggestions on our manuscript. We thank Prof. Xue-Cheng Tai of the University of Bergen, Norway for providing the Euler's elastica code. We thank Prof. Irene Gottlob and Dr. Frank Proudlock of the University of Leicester, UK for providing the OCT image for this research. The copyright of the image Fig. 11(a) is owned by Dr. Konstantinos Papaftsoros of the University of Cambridge.
Jinming Duan received the BSc degree from the Nanjing University of information science and technology, PR China, in 2011, and the MSc degree from the Qingdao University, PR China, in 2014. He is currently pursuing the PhD degree in computer science in the University of Nottingham, UK. His research interests include variational image restoration, multiphase image segmentation, implicit surface reconstruction, medical image analysis.
References (38)
- et al.
Nonlinear total variation based noise removal algorithms
Physica D
(1992) - et al.
Image decomposition combining staircase reduction and texture extraction
J. Vis. Commun. Image Represent.
(2007) - et al.
Denoising optical coherence tomography using second order total generalized variation decomposition
Biomed. Signal Process. Control
(2016) - et al.
Quantization analysis and enhancement of a VLSI gradient-based motion estimation architecture
Digit. Signal Process.
(2012) - et al.
Similar symmetries: the role of wallpaper groups in perceptual texture similarity
Symmetry
(2011) - et al.
Locally parallel texture modeling
SIAM J. Imaging Sci.
(2011) - et al.
Unsupervised texture segmentation using active contour model and oscillating information
J. Appl. Math.
(2014) - et al.
Image cartoon-texture decomposition and feature selection using the total variation regularized L1 functional
- et al.
The total variation regularized L1 model for multiscale decomposition
Multiscale Model. Simul.
(2007) Mathematical analysis of an inf-convolution model for image processing
J. Optim. Theory Appl.
(2015)
Simultaneous structure and texture image inpainting
IEEE Trans. Image Process.
Modeling textures with total variation minimization and oscillating patterns in image processing
J. Sci. Comput.
Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time
IEEE Trans. Image Process.
A second-order model for image denoising
Set-Valued Var. Anal.
A ridge and corner preserving model for surface restoration
SIAM J. Sci. Comput.
Second order Mumford–Shah model for image denoising
Image denoising using mean curvature of image surface
SIAM J. Imaging Sci.
Augmented Lagrangian method for a mean curvature based image denoising model
Inverse Probl. Imaging
Fast numerical schemes related to curvature minimization: a brief and elementary review
Cited by (49)
Generalized multi-scale image decomposition for new tone manipulation
2023, Digital Signal Processing: A Review JournalCartoon-Texture decomposition with patch-wise decorrelation
2023, Journal of Visual Communication and Image RepresentationUnsupervised learning of multi-task deep variational model
2022, Journal of Visual Communication and Image RepresentationLearned prior-guided algorithm for flow field visualization in electrical capacitance tomography
2022, Digital Signal Processing: A Review JournalCitation Excerpt :By using variable substitution and alternating optimization methods, this approach enables a complex optimization problem to be separated into several simpler sub-problems that can be solved flexibly. The performance, convergence, computational complexity and application of the algorithm have been extensively discussed and studied, and more details can be found in [17,80–82]. Our model uses deep learning techniques to fit low-fidelity samples because it excels at processing high-dimensional image data and eliminating the need to use handcrafted feature engineering.
A weighted bounded Hessian variational model for image labeling and segmentation
2020, Signal Processing
Jinming Duan received the BSc degree from the Nanjing University of information science and technology, PR China, in 2011, and the MSc degree from the Qingdao University, PR China, in 2014. He is currently pursuing the PhD degree in computer science in the University of Nottingham, UK. His research interests include variational image restoration, multiphase image segmentation, implicit surface reconstruction, medical image analysis.
Zhaowen Qiu received the BSc degree from the Heilongjiang University, PR China, in 1997, and the MSc and PhD degrees from Harbin Institute of Technology, in 2003 and 2009, respectively. He was a senior visiting scholar with the Carnegie Mellon University, USA from 2009 to 2010. He is currently an associate professor in the Northeast Forestry University, PR China. His research interests include information retrieval, machine learning, medical image analysis.
Wenqi Lu received the BSc in the Department of Information Science and Engineering from the Shandong University of Science and Technology, PR China, in 2013, and she is currently pursuing the MSc degree in the Qingdao University, PR China. Her research interest includes three-dimensional image processing.
Guodong Wang received the BSc and MSc degrees in control technology and control engineering from the Qingdao Science and Technology University, PR China, in 2001 and 2004, respectively, and the PhD degree in pattern recognition and intelligent systems from the Huazhong University of Science and Technology, in 2008. He is an Associate Professor of the College of Information Engineering at Qingdao University, China. His research interests include biometrics, image processing, intelligent video monitoring and analysis.
Zhenkuan Pan received the BSc degree in engineering mechanics from the Northwestern Polytechnical University, PR China, in 1987, and the PhD degree in engineering mechanics from Shanghai Jiao Tong University, PR China, in 1992. He is currently a Professor of the College of Information Engineering at Qingdao University, China. From 2005 to 2006, he was a senior visiting scholar with the Department of Mathematics, the University of California, Los Angeles. His research interests include image processing, variational image science, and three-dimensional reconstruction.
Li Bai received the PhD degree in computer science from the University of Nottingham, UK, in 1994. She was a mathematician in China before she joined the University of Nottingham, in 1989. Her research interests include pattern recognition, computer vision, advanced interfaces, and medical imaging.