Smooth Chan–Vese segmentation via graph cuts
Highlights
► We study the graph cut based minimization of the Chan–Vese segmentation model. ► Sparse neighbourhoods cause geometric artefacts and jagged boundaries, dense neighbourhoods are inefficient. ► Two-stage algorithm for smooth and memory efficient segmentation is proposed.
Introduction
Modern image segmentation approaches based on mathematically well-founded energy minimization techniques date back to the 1980s. One of the most referenced is the work of Mumford and Shah (1989) who introduced a suitable energy functional aiming for reconstruction of the input image by piecewise smooth functions. Unfortunately, the associated optimization task is complex and not easily accomplished using the standard calculus of variations. Hence, a simpler piecewise constant modification for two-phase (i.e., binary or foreground/background) segmentation has been proposed by Chan and Vese (2001). The devised algorithm quickly gained popularity mainly for its ability to cope with objects that are not necessarily defined by gradient.
Several tools are utilized to determine the minimum of the Chan–Vese functional. In the original paper (Chan and Vese, 2001), the variational problem is solved in the level set framework (Osher and Sethian, 1988). In (Zeng et al., 2006, El-Zehiry et al., 2007), the minimization exploits the graph cut framework (Boykov and Funka-Lea, 2006, Kolmogorov and Zabih, 2004, Boykov and Veksler, 2006). Recently, a convex relaxation approach has been employed (Chan et al., 2006, Brown et al., 2011). In terms of speed and efficiency, the graph cut based minimization is probably the most suitable (Zeng et al., 2006, El-Zehiry et al., 2007). Promising algorithms are emerging also for convex optimization (Goldstein et al., 2009) offering similar performance and rotational invariance but lacking a natural stopping criterion and the flexibility of graph cuts.
Major objection against discrete graph cuts is that they suffer from geometric artefacts and jagged boundaries. To obtain smooth results challenging the continuous methods, a dense neighbourhood has to be considered when constructing the graph (Klodt et al., 2008, Boykov and Kolmogorov, 2003). However, large connectivity leads to a substantial increase in memory footprint and burdens the efficiency of the method. In fact, processing of large volumetric images on common workstations may become impossible due to memory constraints.
Various methods were proposed to cope with large graphs that do not fit in the memory. In hierarchical approaches (Lombaert et al., 2005, Sinop and Grady, 2006), the problem is addressed by computing a coarse segmentation on a reduced computational domain which is then refined in a narrow band around the boundary. In region based techniques (Li et al., 2004, Cigla and Alatan, 2008), pre-segmentation is calculated (e.g., using a watershed (Meyer and Beucher, 1990)) to reduce the size of the graph. However, the former group of algorithms often suffers from incorrect segmentation of narrow structures and low contrast data while the latter group requires cautious selection of the low-level segmentation algorithm.
In this paper, we specifically focus on the Chan–Vese model and efficient computation of dense neighbourhoods that are essential for smooth segmentation boundaries. We design a two-stage hierarchical technique that does not reduce the computational domain initially but employs connectivity scaling instead. First, coarse segmentation is calculated using a sparse neighbourhood over the whole image. In the second stage, the segmentation is refined by employing a dense neighbourhood in a narrow band around the boundary from the first stage. The ability of the algorithm to produce smooth Chan–Vese segmentations for only a minor increase in memory consumption and execution time is demonstrated on several examples. Moreover, it does not have some of the aforementioned limitations. In particular, it can handle segmentation of small and narrow structures and does not require any pre-segmentation. Under specific conditions, the same solution is obtained as if the dense neighbourhood was used on a whole image.
The paper is organized as follows. In Section 2, we give a brief overview of the two-phase Chan–Vese image segmentation model and outline the graph cut based minimization of the functional. In Section 3, the motivation of the paper and the advantages of the proposed method over existing solutions are covered in more detail. The developed two-stage algorithm is described in Section 4 and experimental results on generated and real biomedical data are presented in Section 5. We conclude the paper in Section 6.
Section snippets
Preliminaries
In this section, we first recall the two-phase segmentation model by Chan and Vese and then review the graph cut based minimization.
Boundary smoothness and memory efficiency
Graph cuts tend to produce geometric artefacts depending primarily on the density of the neighbourhood system (Klodt et al., 2008, Boykov and Kolmogorov, 2003). Consider the artificial example in Fig. 1a and its smooth Chan–Vese segmentation produced by a rotationally invariant method shown in Fig. 1b. Graph cut based results on the same data using various neighbourhood systems are presented in Fig. 1c–e. For a 3D analogy of the image, two graph cut based segmentations are depicted in Fig. 1h
The two-stage algorithm
Assume a coarse segmentation using a sparse neighbourhood determined by a cut . Consider a fixed and the following sets:where U is the set of graph nodes corresponding to pixels and d measures the distance of a node from a set of nodes using a chosen metric (e.g., the city-block metric). The sets H(S1) and H(T1) contain nodes in S1 and T1, respectively, whose distance from a boundary is greater than the
Experimental results and discussion
We evaluated on two sample images to verify that our assumption about the closeness of the boundaries for a sparse and dense neighbourhood are realistic. The graphs in Fig. 4 show that the distance depends on the input image as well as the parameters and behaves unpredictably in general. However, most of the time it is around 5 pixels which confirms our assumption. Also note that the high peaks are sometimes caused by outliers (e.g., an isolated thin line) which are hardly noticeable in
Conclusion
In this paper, a new hierarchical method for efficient graph cut based Chan–Vese segmentation was presented. It employs a two-stage connectivity scaling approach leading to a significant reduction of the memory consumption and execution time and enabling the use of dense neighbourhoods that are essential for smooth segmentation boundaries. Promising results were presented on both generated and real data showing apparent improvement in segmentation quality and decline in computational demands. A
Acknowledgements
This work has been supported by the Ministry of Education of the Czech Republic (Projects No. MSM-0021622419 and No. LC535) and Grant Agency of the Czech Republic (Grant No. P302/12/G157).
References (25)
- et al.
Morphological segmentation
J. Visual Comm. Image Represent.
(1990) - et al.
Fronts propagating with curvature dependent speed: Algorithms based on Hamilton–Jacobi formulation
J. Comput. Phys.
(1988) - et al.
Topology cuts: A novel min-cut/max-flow algorithm for topology preserving segmentation in N-D images
Comput. Vision Image Understand.
(2008) - et al.
Graph cuts and efficient N–D image segmentation (review)
Internat. J. Comput. Vision
(2006) - Boykov, Y., Kolmogorov, V., 2003. Computing geodesics and minimal surfaces via graph cuts. In: ICCV 2003: 9th IEEE...
- et al.
An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision
IEEE Trans. Pattern Anal. Machine Intell.
(2004) - et al.(2006)
- et al.
Completely convex formulation of the Chan–Vese image segmentation model
Internat. J. Comput. Vision
(2011) - et al.
Algorithms for finding global minimizers of image segmentation and denoising models
SIAM J. Appl. Math.
(2006) - et al.
Active contours without edges
IEEE Trans. Image Process.
(2001)
Cited by (17)
A scalable approach to T2-MRI colon segmentation
2020, Medical Image AnalysisDense subgraph mining with a mixed graph model
2013, Pattern Recognition LettersCitation Excerpt :Frequently applied methods using the standard model are graph partitioning and dense subgraph mining methods. Graph cuts (Boykov and Kolmogorov, 2004; Danek et al., 2012; Cousty et al., 2010), spectral partitioning, several MST-based clustering methods such as Zhou et al., 2011 belong to the partitioning methods. On the other hand, clique mining (Feige, 2004) is an example of density based methods.
Automated Iris Segmentation from Anterior Segment OCT Images with Occludable Angles via Local Phase Tensor
2019, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBSComparison of Deep Learning-Based Recognition Techniques for Medical and Biomedical Images
2019, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)Intensity and Compactness Enabled Saliency Estimation for Leakage Detection in Diabetic and Malarial Retinopathy
2017, IEEE Transactions on Medical Imaging