Topology cuts: A novel min-cut/max-flow algorithm for topology preserving segmentation in N–D images

https://doi.org/10.1016/j.cviu.2008.07.008Get rights and content

Abstract

Topology is an important prior in many image segmentation tasks. In this paper, we design and implement a novel graph-based min-cut/max-flow algorithm that incorporates topology priors as global constraints. We show that the optimization of the energy function we consider here is NP-hard. However, our algorithm is guaranteed to find an approximate solution that conforms to the initialization, which is a desirable property in many applications since the globally optimum solution does not consider any initialization information. The key innovation of our algorithm is the organization of the search for maximum flow in a way that allows consideration of topology constraints. In order to achieve this, we introduce a label attribute for each node to explicitly handle the topology constraints, and we use a distance map to keep track of those nodes that are closest to the boundary. We employ the bucket priority queue data structure that records nodes of equal distance and we efficiently extract the node with minimal distance value. Our methodology of embedding distance functions in a graph-based algorithm is general and can also account for other geometric priors. Experimental results show that our algorithm can efficiently handle segmentation cases that are challenging for graph-cut algorithms. Furthermore, our algorithm is a natural choice for problems with rich topology priors such as object tracking.

Introduction

Many computer vision problems such as segmentation, stereo reconstruction and image restoration, can be formulated as a minimization of an energy function [26]. These energy functions are naturally divided into two groups: continuous and discrete. For the problem of image segmentation, the level sets method [25] is a representative model in the continuous community, while the graph-based Markov Random Fields (MRFs) [13] is a very popular model in the discrete group. One very efficient algorithm for solving a subclass of the MRF energy function is the graph cuts algorithm [8]. In recent years, there has been a number of works showing the close relationship between level sets and graph cuts ([4], [6], [19], etc.), and how shape priors can be incorporated into the graph cuts framework ([12], [22]). In this work, we show how the idea of topology preserving segmentation from the level sets literature [14] can be transposed to the graph-based algorithms. We propose the first min-cut/max-flow algorithm that is designed to explicitly incorporate topology as a global constraint in the segmentation. We call our new algorithm Topology Cuts, in analogy to the popular graph cuts algorithm [8].

Topology as a prior is available in many applications. For example, the anatomy of human tissues provides important topological constraints that ensure the correctness in biomedical image segmentation. Existing techniques that enforce topology constraints into the graph cuts algorithm, do so by simply tuning the parameters of the energy function [3], [7]. This scheme usually requires intense user interactions and is not applicable in cases where user manipulation is difficult. In contrast, we propose to embed the topological constraint into the discrete min-cut/max-flow algorithm, which leads to a new and efficient way of considering global topology information for the general problem of topology preservation.

Our work is inspired by the topology preserving level set method of [14]. This algorithm makes use of digital topology theory for N–D images [2] to detect topology changes during the evolution of level sets. Taking advantage of the fact that the level set functions are solved in a gradient descent manner, and assuming that the change of sign for the pixels only occurs one pixel at a time on the boundary of the evolving objects, the topology of the object can be easily controlled. The advantage of our method over [14] is the speed-up and numerical stability inherent to discrete max-flow methods. In addition, our method has a guaranteed convergence property.

However, transferring the idea of topology preserving evolution from the continuous level sets algorithm to the discrete graph-based algorithm is not straightforward. The main difficulty lies in the fact that previous graph-cut implementations [1], [5], [15] are inherently topology-free and thus not conductive to topology considerations during the search of max-flow. To make these consideration possible for the discrete graph-based algorithm, we introduce the following elements.

  • (1)

    An F/B label attribute is introduced to explicitly handle the topology property in the image. This resolves an ambiguity in the existing graph cuts algorithms, i.e., it is possible that the labels for a subset of the graph’s nodes can be changed without changing the optimal solution (multiple solutions for the energy minimization problem). Existing algorithms set these nodes’ labels to a default label, which unavoidably leads to topological errors.

  • (2)

    An initialization step is used to provide the graph with initial topology information.

  • (3)

    The computation of max-flow is divided into inter-label and intra-label stages, to facilitate the propagation of topology information during the search for the minimum of the energy function.

  • (4)

    A distance map (function) which keeps track of the nodes that are closest to the current boundary between the different label sets is set in the beginning and is updated during the computation.

  • (5)

    To efficiently insert and extract nodes on the current evolving boundary (the level set of the distance map), we use the bucket priority queue data structure [9], [11], which only requires time of O(1) complexity for each insertion and extraction operation. Hence, there is no loss of efficiency compared to the previous graph-cut algorithms. Our algorithm shares the same complexity with the widely used graph-cut implementation [5], and in practice it runs in comparable speed.

The contributions of this paper can be summarized as follows:

  • To the best of our knowledge, this is the first work that incorporates a global topology prior into the design of the discrete graph-based min-cut/max-flow algorithms.

  • We prove that enforcing global optimality of the solution while considering topology constraints is NP-hard. That means any algorithm that enforces topology constraints either interactively or automatically can not obtain the global optimum.

  • In the design of our algorithm, we combine concepts from the level-set literature (such as distance maps and level-set evolution [24], [25]) into an efficient discrete graph-based algorithm. The techniques we use here are general and define a new way of incorporating geometric prior knowledge into the existing graph-cut models/algorithms such as curvature or shape priors.

Additionally, our new algorithm is suitable for the concept of multilevel banded graph cuts [23] to fairly speedup the computation. In experiments, we show that our algorithm achieves more meaningful and visually better results compared with graph cuts for problems where topology information is available, e.g., image segmentation and object tracking.

In Section 2, we review the essential background for describing our new algorithm. Section 3 discusses the primal–dual schema in the min-cut/max-flow algorithm and the overall principles of our algorithm. Section 4 gives the formulation of the topology-cut problem and proves its NP-hardness. Section 5 explains the design of our new algorithm. Section 6 discusses the detailed implementation. Section 7 analyzes our algorithm with respect to convergence, topology preservation and time complexity. The experimental results are presented in Section 8. Finally, we conclude and outline future work in Section 9.

Section snippets

Digital topology

Here, we discuss two key concepts in the topology of digital images[2]: connectivity and simple point.

The connectivity of a digital image specifies the condition of adjacency that two points must fulfill, in order for the foreground F and the background B to be considered connected respectively.

To ensure this (Fig. 1), different connectivity for the object and the background must be specified. For 2D images, valid (foreground, background) connectivity pairs are (4, 8) and (8, 4); for 3D images,

Tree membership and the primal–dual solution to the s/t cut

If we relax the variables of the discrete optimization problem (1) to be continuous, its duality can be formulated as in [30]maxftss.t.fpqwpq,(p,q)Ep:(p,q)Efpq-p:(q,p)Efqp0,qVfpq0(p,q)E.It can be shown that for any feasible solution of Eq. (2), fts  copt where copt is the optimal solution of the primal Eq. (1). Thus finding the max-flow of the graph corresponds to finding a lower bound of the primal problem. In the ideal situation this lower bound reaches the optimal solution of the

The topology cuts problem

The energy function that combines the MRF formulation and digital topology on the image grid isinfxp,pVpVD(xp)+(p,q)EVpq(xp,xq),s.t.T=Tinitwithxp{0,1},pV.

Here, T denotes the topology of the 0/1 labeled image as defined in [2]. Tinit is the initial topology information that is assigned to the image either interactively, or automatically as shown in our tracking example. The meanings of the other notations are the same as in Eq. (1). Note that here we only consider the hard-constraints on

Design of the topology cuts algorithm

In this section, we discuss the novel aspects of our algorithm. The whole algorithm can be found in Table 1.

Implementation details

The overall guideline in implementing the topology cuts algorithm is to reduce the energy function while maintaining the topology constraint. Observe that if after computing the maximum flow, all nodes with label F only belong to the source tree or be free and all nodes with label B only belong to the sink tree or be free, then the energy function is minimized. Thus, during the maximum flow computation, we need to update the label of each node according to which tree it belongs to (favors).

Convergence

Convergence of our algorithm is ensured by (1) all augmenting paths between s and t are guaranteed to be saturated, and (2) the algorithm will stop within two iterations.

To verify the first claim, observe that in the first iteration, all augmenting paths between the underlying four subtrees with two different label sets are saturated after finding the inter-label maximum flow. Likewise, between the stages of searching the intra-label maximum flow and changing labels, all augmenting paths with

Experimental results

We apply our topology cuts algorithm to two problems: image segmentation and object tracking.2 All results were run on a PC equipped with an Intel Pentium M 2.0 GHz processor and 1.5 G memory.

Conclusions and future work

We proposed a new algorithm for solving a subset of MRF functions that can be addressed by graph cuts while respecting topological constraints. It combines certain advantages of level sets and graph cuts. The idea of boundary evolution is introduced into the graph cuts framework by using the explicit F/B label attribute. Rather than evolving the boundary in a gradient descent manner to update the distance function as level sets do, the boundary evolution of the F/B label set is driven by the

Acknowledgements

We are grateful to Prof. Y. Boykov for enlightening discussions on this work. This work was partially supported by NIDA Grant: 1 R01 DA020949-01, NSF Grants: CNS-0627645, CNS-0627645, IIS-0527585 and 863 Program of China (No. 2006AA01Z314).

References (32)

  • G. Bertrand

    Simple points, topological numbers and geodesic neighborhoods in cubic grids

    Pattern Recognition Letters

    (1994)
  • S. Osher et al.

    Fronts propagating with curvature-dependent speed: Algorithms based on the Hamilton-Jacobi formulation

    J. Comput. Phys.

    (1988)
  • R.K. Ahuja et al.

    Network Flows: Theory, Algorithms, and Applications

    (1993)
  • Y. Boykov et al.

    Graph cuts and efficient N–D image segmentation

    IJCV

    (2006)
  • Y. Boykov, V. Kolmogorov, Computing geodesics and minimal surfaces via graph cuts, in: ICCV 03, 2003, pp....
  • Y. Boykov et al.

    An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision

    PAMI

    (2004)
  • Y. Boykov, V. Kolmogorov, D. Cremers, A. Delong, An integral solution to surface evolution PDEs via geo-cuts, in: ECCV...
  • Y. Boykov et al.

    Graph cuts in vision and graphics: theories and applications

  • Y. Boykov et al.

    Fast approximate energy minimization via graph cuts

    PAMI

    (2001)
  • R. Brown

    Calendar queues: a fast 0(1) priority queue implementation for the simulation event set problem

    Commun. ACM

    (1998)
  • T. Chan et al.

    Active contour model without edges

    IEEE Transactions on Image Processing

    (2001)
  • B.V. Cherkassky et al.

    On implementing the push-relabel method for the maximum flow problem

    Algorithmica

    (1997)
  • D. Freedman, T. Zhang, Interactive graph cut based segmentation with shape priors. in: CVPR 05, 2005, pp....
  • S. German et al.

    Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

    (1988)
  • X. Han et al.

    A topology preserving level set method for geometric deformable models

    PAMI

    (2003)
  • O. Juan, Y. Boykov, Active graph cuts, in: CVPR 06, 2006, pp....
  • Cited by (76)

    • Cardiac function review by machine learning approaches

      2021, Image Processing for Automated Diagnosis of Cardiac Diseases
    • Structure-Aware Image Segmentation with Homotopy Warping

      2022, Advances in Neural Information Processing Systems
    View all citing articles on Scopus
    View full text