Topology cuts: A novel min-cut/max-flow algorithm for topology preserving segmentation in N–D images
Introduction
Many computer vision problems such as segmentation, stereo reconstruction and image restoration, can be formulated as a minimization of an energy function [26]. These energy functions are naturally divided into two groups: continuous and discrete. For the problem of image segmentation, the level sets method [25] is a representative model in the continuous community, while the graph-based Markov Random Fields (MRFs) [13] is a very popular model in the discrete group. One very efficient algorithm for solving a subclass of the MRF energy function is the graph cuts algorithm [8]. In recent years, there has been a number of works showing the close relationship between level sets and graph cuts ([4], [6], [19], etc.), and how shape priors can be incorporated into the graph cuts framework ([12], [22]). In this work, we show how the idea of topology preserving segmentation from the level sets literature [14] can be transposed to the graph-based algorithms. We propose the first min-cut/max-flow algorithm that is designed to explicitly incorporate topology as a global constraint in the segmentation. We call our new algorithm Topology Cuts, in analogy to the popular graph cuts algorithm [8].
Topology as a prior is available in many applications. For example, the anatomy of human tissues provides important topological constraints that ensure the correctness in biomedical image segmentation. Existing techniques that enforce topology constraints into the graph cuts algorithm, do so by simply tuning the parameters of the energy function [3], [7]. This scheme usually requires intense user interactions and is not applicable in cases where user manipulation is difficult. In contrast, we propose to embed the topological constraint into the discrete min-cut/max-flow algorithm, which leads to a new and efficient way of considering global topology information for the general problem of topology preservation.
Our work is inspired by the topology preserving level set method of [14]. This algorithm makes use of digital topology theory for N–D images [2] to detect topology changes during the evolution of level sets. Taking advantage of the fact that the level set functions are solved in a gradient descent manner, and assuming that the change of sign for the pixels only occurs one pixel at a time on the boundary of the evolving objects, the topology of the object can be easily controlled. The advantage of our method over [14] is the speed-up and numerical stability inherent to discrete max-flow methods. In addition, our method has a guaranteed convergence property.
However, transferring the idea of topology preserving evolution from the continuous level sets algorithm to the discrete graph-based algorithm is not straightforward. The main difficulty lies in the fact that previous graph-cut implementations [1], [5], [15] are inherently topology-free and thus not conductive to topology considerations during the search of max-flow. To make these consideration possible for the discrete graph-based algorithm, we introduce the following elements.
- (1)
An F/B label attribute is introduced to explicitly handle the topology property in the image. This resolves an ambiguity in the existing graph cuts algorithms, i.e., it is possible that the labels for a subset of the graph’s nodes can be changed without changing the optimal solution (multiple solutions for the energy minimization problem). Existing algorithms set these nodes’ labels to a default label, which unavoidably leads to topological errors.
- (2)
An initialization step is used to provide the graph with initial topology information.
- (3)
The computation of max-flow is divided into inter-label and intra-label stages, to facilitate the propagation of topology information during the search for the minimum of the energy function.
- (4)
A distance map (function) which keeps track of the nodes that are closest to the current boundary between the different label sets is set in the beginning and is updated during the computation.
- (5)
To efficiently insert and extract nodes on the current evolving boundary (the level set of the distance map), we use the bucket priority queue data structure [9], [11], which only requires time of O(1) complexity for each insertion and extraction operation. Hence, there is no loss of efficiency compared to the previous graph-cut algorithms. Our algorithm shares the same complexity with the widely used graph-cut implementation [5], and in practice it runs in comparable speed.
The contributions of this paper can be summarized as follows:
- •
To the best of our knowledge, this is the first work that incorporates a global topology prior into the design of the discrete graph-based min-cut/max-flow algorithms.
- •
We prove that enforcing global optimality of the solution while considering topology constraints is NP-hard. That means any algorithm that enforces topology constraints either interactively or automatically can not obtain the global optimum.
- •
In the design of our algorithm, we combine concepts from the level-set literature (such as distance maps and level-set evolution [24], [25]) into an efficient discrete graph-based algorithm. The techniques we use here are general and define a new way of incorporating geometric prior knowledge into the existing graph-cut models/algorithms such as curvature or shape priors.
Additionally, our new algorithm is suitable for the concept of multilevel banded graph cuts [23] to fairly speedup the computation. In experiments, we show that our algorithm achieves more meaningful and visually better results compared with graph cuts for problems where topology information is available, e.g., image segmentation and object tracking.
In Section 2, we review the essential background for describing our new algorithm. Section 3 discusses the primal–dual schema in the min-cut/max-flow algorithm and the overall principles of our algorithm. Section 4 gives the formulation of the topology-cut problem and proves its NP-hardness. Section 5 explains the design of our new algorithm. Section 6 discusses the detailed implementation. Section 7 analyzes our algorithm with respect to convergence, topology preservation and time complexity. The experimental results are presented in Section 8. Finally, we conclude and outline future work in Section 9.
Section snippets
Digital topology
Here, we discuss two key concepts in the topology of digital images[2]: connectivity and simple point.
The connectivity of a digital image specifies the condition of adjacency that two points must fulfill, in order for the foreground F and the background B to be considered connected respectively.
To ensure this (Fig. 1), different connectivity for the object and the background must be specified. For 2D images, valid (foreground, background) connectivity pairs are (4, 8) and (8, 4); for 3D images,
Tree membership and the primal–dual solution to the s/t cut
If we relax the variables of the discrete optimization problem (1) to be continuous, its duality can be formulated as in [30]It can be shown that for any feasible solution of Eq. (2), fts ⩽ copt where copt is the optimal solution of the primal Eq. (1). Thus finding the max-flow of the graph corresponds to finding a lower bound of the primal problem. In the ideal situation this lower bound reaches the optimal solution of the
The topology cuts problem
The energy function that combines the MRF formulation and digital topology on the image grid is
Here, denotes the topology of the 0/1 labeled image as defined in [2]. is the initial topology information that is assigned to the image either interactively, or automatically as shown in our tracking example. The meanings of the other notations are the same as in Eq. (1). Note that here we only consider the hard-constraints on
Design of the topology cuts algorithm
In this section, we discuss the novel aspects of our algorithm. The whole algorithm can be found in Table 1.
Implementation details
The overall guideline in implementing the topology cuts algorithm is to reduce the energy function while maintaining the topology constraint. Observe that if after computing the maximum flow, all nodes with label F only belong to the source tree or be free and all nodes with label B only belong to the sink tree or be free, then the energy function is minimized. Thus, during the maximum flow computation, we need to update the label of each node according to which tree it belongs to (favors).
Convergence
Convergence of our algorithm is ensured by (1) all augmenting paths between s and t are guaranteed to be saturated, and (2) the algorithm will stop within two iterations.
To verify the first claim, observe that in the first iteration, all augmenting paths between the underlying four subtrees with two different label sets are saturated after finding the inter-label maximum flow. Likewise, between the stages of searching the intra-label maximum flow and changing labels, all augmenting paths with
Experimental results
We apply our topology cuts algorithm to two problems: image segmentation and object tracking.2 All results were run on a PC equipped with an Intel Pentium M 2.0 GHz processor and 1.5 G memory.
Conclusions and future work
We proposed a new algorithm for solving a subset of MRF functions that can be addressed by graph cuts while respecting topological constraints. It combines certain advantages of level sets and graph cuts. The idea of boundary evolution is introduced into the graph cuts framework by using the explicit F/B label attribute. Rather than evolving the boundary in a gradient descent manner to update the distance function as level sets do, the boundary evolution of the F/B label set is driven by the
Acknowledgements
We are grateful to Prof. Y. Boykov for enlightening discussions on this work. This work was partially supported by NIDA Grant: 1 R01 DA020949-01, NSF Grants: CNS-0627645, CNS-0627645, IIS-0527585 and 863 Program of China (No. 2006AA01Z314).
References (32)
Simple points, topological numbers and geodesic neighborhoods in cubic grids
Pattern Recognition Letters
(1994)- et al.
Fronts propagating with curvature-dependent speed: Algorithms based on the Hamilton-Jacobi formulation
J. Comput. Phys.
(1988) - et al.
Network Flows: Theory, Algorithms, and Applications
(1993) - et al.
Graph cuts and efficient N–D image segmentation
IJCV
(2006) - Y. Boykov, V. Kolmogorov, Computing geodesics and minimal surfaces via graph cuts, in: ICCV 03, 2003, pp....
- et al.
An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision
PAMI
(2004) - Y. Boykov, V. Kolmogorov, D. Cremers, A. Delong, An integral solution to surface evolution PDEs via geo-cuts, in: ECCV...
- et al.
Graph cuts in vision and graphics: theories and applications
- et al.
Fast approximate energy minimization via graph cuts
PAMI
(2001) Calendar queues: a fast 0(1) priority queue implementation for the simulation event set problem
Commun. ACM
(1998)
Active contour model without edges
IEEE Transactions on Image Processing
On implementing the push-relabel method for the maximum flow problem
Algorithmica
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
A topology preserving level set method for geometric deformable models
PAMI
Cited by (76)
Topology-sensitive weighting model for myocardial segmentation
2023, Computers in Biology and MedicineCardiac function review by machine learning approaches
2021, Image Processing for Automated Diagnosis of Cardiac DiseasesUsing the Split Bregman Algorithm to Solve the Self-repelling Snakes Model
2022, Journal of Mathematical Imaging and VisionStructure-Aware Image Segmentation with Homotopy Warping
2022, Advances in Neural Information Processing Systems