Narrow band region-based active contours and surfaces for 2D and 3D segmentation

https://doi.org/10.1016/j.cviu.2009.05.002Get rights and content

Abstract

We describe a narrow band region approach for deformable curves and surfaces in the perspective of 2D and 3D image segmentation. Basically, we develop a region energy involving a fixed-width band around the curve or surface. Classical region-based methods, like the Chan–Vese model, often make strong assumptions on the intensity distributions of the searched object and background. In order to be less restrictive, our energy achieves a trade-off between local features of gradient-like terms and global region features. Relying on the theory of parallel curves and surfaces, we perform a mathematical derivation to express the region energy in a curvature-based form allowing efficient computation on explicit models. We introduce two different region terms, each one being dedicated to a particular configuration of the target object. Evolution of deformable models is performed by means of energy minimization using gradient descent. We provide both explicit and implicit implementations. The explicit models are a parametric snake in 2D and a triangular mesh in 3D, whereas the implicit models are based on the level set framework, regardless of the dimension. Experiments are carried out on MRI and CT medical images, in 2D and 3D, as well as 2D color photographs.

Introduction

Segmentation by means of deformable models has been a widely studied aspect of computer vision over the last two decades. Since their introduction by Kass et al. [1], deformable models have found many applications in image segmentation and tracking. From an initial location, which may be manually or automatically provided, these models deform according to an iterative evolution algorithm until they fit one or more structures of interest. The evolution method is usually derived from the minimization of some energy functional, including regularizing terms for geometrical smoothness and external terms relating the model to the data. They are powerful tools thanks to their ability to adapt their geometry and incorporate prior knowledge about the structure of interest.

Several implementations of these active models were developed. Explicit deformable models represent the evolving boundary as a set of interconnected control points or vertices. Among these, the original 2D parametric contour and the 3D triangular mesh [2], [3] are intuitive implementations, in which the boundary is deformed by direct modifications of vertices coordinates. The main drawback is that polygon and meshes do not modify their topology naturally, i.e. techniques for detection of topological changes must be implemented beside the evolution algorithm. Conversely, implicit implementations, based on the level set framework [4], handle the evolving boundary as the zero level of a hypersurface, defined on the same domain as the image. They are often chosen for their natural handling of topological changes and intuitive extensibility to higher dimensions. Their algorithmic complexity is a function of the image resolution, making them time-consuming. Despite the development of accelerating methods, like the narrow band technique [4] or the fast marching method [5], their computational cost remains higher than their explicit counterparts.

Deformable models, whether they are explicit or implicit, are attached to the image by means of a local edge-based energy or force. Since they consider only local boundaries, classical snakes are relatively blind, in the sense they are unable to reach boundaries if their initial location is far from them. The increasing use of region terms inspired by the Mumford–Shah functional [6], [7] has proven to overcome the limitations of uniquely gradient-based models, especially when dealing with data sets suffering from noise and lack of contrast. Indeed, many anatomical structures encountered in medical imaging lend themselves to region-based segmentation. Global statistical data computed over the entire region of interest is a well established technique to improve the behavior of snakes. Early work, including the anticipating snake by Ronfard [8] and the active region model by Ivins and Porrill [9], introduced the use of region terms in the evolution of parametric snakes. The region competition method by Zhu and Yuille [10] was developed later, combining aspects of snakes and region growing techniques. Many papers have dealt with region-based approaches using the level set framework, including the Chan–Vese model [11], the deformable regions by Jehan-Besson et al. [12] and the geodesic active regions by Paragios and Deriche [13]. These implementations have the advantage of adaptive topology at the expense of computational cost. In the context of 3D segmentation, a deformable mesh endowed with a Chan–Vese region energy was presented in [14], whereas Dufour et al. [15] used an implicit active surface to perform segmentation and tracking of cells, where computations are particularly time-expensive.

Most existing region-based deformable models segment images according to statistical data computed over the entire regions, i.e. the object of interest and the background. These approaches have an underlying notion of homogeneity, in the sense that image partitions should be uniform in terms of intensity, whether prior knowledge on the distribution of pixel intensities is available [16] or not. Instead of raw pixel intensity, higher level features like texture descriptors may also be considered [17]. We now focus on the region energy of the Chan–Vese model [11]. Let Rin be the region enclosed by deformable curve Γ, and Rout its complement. The energy penalizes the curve splitting the image into heterogeneous regions, using intensity deviations. In addition to length and area terms, the Chan–Vese model has the following global data term:EregionC–V[Γ]=λinRin(I(x)-kin)2dx+λoutRout(I(x)kout)2dxwhere kin and kout are intensity descriptors inside and outside the curve, respectively. By gradient descent, these descriptors are assigned to average intensity values [11]. At the end of the segmentation process, region Rin is expected to coincide with the target object. Hence, although constraints on intensity deviations can be adjusted by tuning parameters λin and λout, the global region term is by definition devoted to segment uniform objects and backgrounds. Let us consider the images depicted in Fig. 1, in which the object of interest is the white cup. Ignoring the influence of illumination changes, case (a) is the typical configuration which the Chan–Vese model aims at, since the cup and the floor areas are nearly constant with respect to color.

Uniformity of intensity over regions is a rather strong assumption. However, strict homogeneity is not necessarily a desirable property, especially for the background. The ideal case (a) is rarely encountered in most of computer vision applications. For instance, when one wants to isolate a single structure from the rest of the image in medical data, the background contains various anatomical structures, which differ in their overall intensities and textures. In this context, the use of local features was already addressed in the literature. For other work dealing with local statistics in region-based segmentation, the reader may refer to [18], [19], [20], [21]. For the same purpose, active contours embedded with both edge and region terms were studied in [22], [23], [24] and extended to textured region segmentation [25]. In cases (b) and (c), the background, made up of the floor and the plate, is now piecewise uniform. Case (b) depicts a particular situation where the background is uniform in a small band around the cup boundaries. We believe that many objects can be discriminated from the background according to intensity features only in the vicinity of their boundaries, which leads to the development of our first narrow band region energy. Extending the work in [26], we formulate our energy as the intensity variance over an inner and an outer band around the evolving boundary. Case (c) represents an even more general case, where the outer band around the target object is piecewise constant. Indeed, the cup is surrounded by the floor in the upper half and the plate in the bottom half. The role of our second narrow band region energy is to handle configurations in which the outer neighborhood of the target object presents several distinct areas.

In the paper, we first describe the theoretical framework of the narrow band energy. This includes mathematical derivations to yield a suitable form for the region term, i.e. a formulation enabling natural implementation. Our mathematical development is based on the theory of parallel curves and surfaces [27], [28]. We endeavor to develop a framework which is applicable both to 2D and 3D segmentation. Indeed, after describing our region terms on a planar curve, we extend them to a deformable surface model. Then, in order to allow gradient descent afterwards, we determine the variational derivatives of the region energies with respect to the curve, thanks to calculus of variations, and extend them to the surface model as well. Then, we deal with numerical implementation issues, including model structure and energy minimization. We first present the explicit implementation, which lies in a 2D polygonal contour and a 3D triangular mesh. These models are able to perform resampling, in order to overcome the lack of geometrical flexibility of traditional snakes and meshes. We also provide a level-set implementation, which offers the advantage of a common mathematical description in 2D and 3D, in addition to the topological adaptability. Finally, experiments are carried out on medical data and natural color images. For both explicit and implicit implementations, the tests discuss the advantages of our narrow band terms over other data terms including edge energies and global region energies.

Section snippets

Energies

The continuous active contour model is represented as a parameterized curve Γ with position vector c:Γ:ΩR2uc(u)=[x(u)y(u)]Twhere x and y are continuously differentiable with respect to parameter u. The parameter domain is normalized: Ω=[0,1]. We assume that the curve is simple, i.e. non-intersecting, and closed: c(0)=c(1). Segmentation of an object of interest is performed by finding the curve Γ minimizing the following energy functional:E[Γ]=ωEsmooth[Γ]+(1-ω)Eregion[Γ]where Esmooth and E

Energies

The active contour method approach naturally extends to a three dimensional segmentation problem. In a continuous space, a deformable model is represented by a parameterized surface Γ.Γ:Ω2R3(u,v)s(u,v)=[x(u,v)y(u,v)z(u,v)]T

In all subsequent derivations, we will assume a closed surface with a parameterization homeomorphic to a torus:s(0,v)=s(1,v)vΩs(u,0)=s(u,1)uΩor a sphere:s(0,v)=s(1,v)vΩs(u1,0)=s(u2,0)(u1,u2)Ω2s(u1,1)=s(u2,1)(u1,u2)Ω2

Note that these parameterizations are given only

Calculus of variations

Image segmentation is performed through numerical minimization of the energy functional using gradient descent. The negative discretized variational derivative of the energy term is usually considered for the descent direction. In this section, we express the variational derivatives of the energies, especially focusing on the region terms, for both contour and surface.

Polygon and mesh

To describe the discrete forms of active 2D contour and 3D surface models simultaneously, we introduce a general framework. The contour is a discrete closed curve, whereas the surface model is a triangular mesh built by subdividing an icosahedron [37]. The models have a constant global topology, their initial shape being circular and spherical, respectively. Both are made up of a set of n vertices, denoted pi=[xiyi]T in 2D and pi=[xiyizi]T in 3D. Each vertex pi has a set of neighboring

Level-set implementation

We provide an implicit implementation of our narrow band energies as well. In addition to topological flexibility, the level-set formulation presents the advantage of a common formulation for both 2D and 3D models. We consider the level set function ψ:RdR, where d is the image dimension. The contour or surface is the zero level set of ψ. We define the region enclosed by the contour or surface by Rin={x|ψ(x)0}. Instead of forces applied on vertices, we now deal with speeds applied to function

Results and discussion

Regarding the results, we should first point out that the goal of our experiments is not to compare explicit and implicit implementations, since it is well accepted that both exhibit their own advantages. These ones are typically topological and geometrical freedom for level sets. On the other hand, explicit approaches with polygonal snakes and triangular meshes yield less computational cost than level set and allow more control. The purpose of our tests is to compare the behavior of active

Conclusions

We have presented in this paper a narrow band region approach for deformable contours and surfaces driven by energy minimization. The approach is based on two novel region terms, formulating a homogeneity criterion in inner and outer bands neighboring the evolving curve or surface. The first and second term relies on the asumption of a, respectively, uniform and piecewise uniform background in the vicinity of the target object. Based on the theory of parallel curves and surfaces, a mathematical

References (51)

  • Z. Zhang, F. Braun, Fully 3D active surface models with self-inflation and self-deflation forces, in: IEEE Computer...
  • R. Malladi et al.

    Shape modeling with front propagation: a level set approach

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1995)
  • D. Mumford et al.

    Optimal approximation by piecewise smooth functions and associated variational problems

    Communications on Pure and Applied Mathematics

    (1989)
  • T. Brox, D. Cremers, On the statistical interpretation of the piecewise smooth Mumford–Shah functional, in:...
  • R. Ronfard

    Region based strategies for active contour models

    International Journal of Computer Vision

    (1994)
  • S. Zhu et al.

    Region competition: unifying snakes, region growing, Bayes/MDL for multiband image segmentation

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1996)
  • T. Chan et al.

    Active contours without edges

    IEEE Transactions on Image Processing

    (2001)
  • S. Jehan-Besson et al.

    DREAM2S: deformable regions driven by an Eulerian accurate minimization method for image and video segmentation

    International Journal of Computer Vision

    (2003)
  • G. Slabaugh, G. Unal, Active polyhedron: surface evolution theory applied to deformable meshes, in: IEEE Computer...
  • A. Dufour et al.

    Segmenting and tracking fluorescent cells in dynamic 3-D microscopy with coupled active surfaces

    IEEE Transactions on Image Processing

    (2005)
  • M. Taron et al.

    Registration with uncertainties and statistical modeling of shapes with variable metric kernels

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2009)
  • N. Paragios et al.

    Geodesic active regions and level set methods for supervised texture segmentation

    International Journal of Computer Vision

    (2002)
  • J. Piovano et al.

    Local statistic based region segmentation with automatic scale selection

  • M. Alemán-Flores et al.

    Texture-oriented anisotropic filtering and geodesic active contours in breast tumor ultrasound segmentation

    Journal of Mathematical Imaging and Vision

    (2007)
  • C. Li, C. Kao, J. Gore, Z. Ding, Implicit active contours driven by local binary fitting energy, in: IEEE Computer...
  • Cited by (69)

    • Segmentation by deformable models

      2023, Medical Image Analysis
    • A semi-automated method for measuring the evolution of both lumen area and blood flow in carotid from Phase Contrast MRI

      2015, Computers in Biology and Medicine
      Citation Excerpt :

      This is not a limitation in our context because PC-MRI sequences are not acquired at carotid bifurcations, therefore excluding possible changes of topology (this being not the case in previously mentioned applications based on CTA or MRA images [4–6,16]). As underlined in [16], the use of a narrow band enables to be less sensitive to anatomical structures surrounding carotid (e.g. jugular vein indicated in Fig. 1A). Fig. 2C reports deformed contours obtained using bandwidths of 2 and 8 pixels (ω is tuned, in each case, in order to obtain similar deformation result on the upper right part of the carotid).

    View all citing articles on Scopus
    View full text