Elsevier

Pattern Recognition

Volume 45, Issue 4, April 2012, Pages 1578-1590
Pattern Recognition

An improved region-based model with local statistical features for image segmentation

https://doi.org/10.1016/j.patcog.2011.09.008Get rights and content

Abstract

In this paper, we propose a new region-based active contour model (ACM) for image segmentation. In particular, this model utilizes an improved region fitting term to partition the regions of interests in images depending on the local statistics regarding the intensity and the magnitude of gradient in the neighborhood of a contour. By this improved region fitting term, images with noise, intensity non-uniformity, and low-contrast boundaries can be well segmented. Integrated with the duality theory and the anisotropic diffusion process based on structure tensor, a new regularization term is defined through the duality formulation to penalize the length of active contour. By this new regularization term, the structural information of images is utilized to improve the ability of capturing the geometric features such as corners and cusps. From a numerical point of view, we minimize the energy function of our model by an efficient dual algorithm, which avoids the instability and the non-differentiability of traditional numerical solutions, e.g. the gradient descent method. Experiments on medical and natural images demonstrate the advantages of the proposed model over other segmentation models in terms of both efficiency and accuracy.

Highlights

► We improve the regularization term by the structure tensor. ► Improved regularization is represented in a dual formulation. ► Statistics of intensity and the magnitude of gradient are extracted for evolution. ► Local region is the neighborhood of the contour. ► Our model is solved by the dual algorithm for efficiency.

Introduction

Image segmentation plays an important role in the field of image processing and computing vision. The active contour models (ACM), which are based on the theory of contour evolution and geometric flows, have been extensively studied and used in image segmentation. The basic idea of the active contour model is to evolve a contour under some constraints to extract desired object. According to the nature of constraints, the existing active contour models can be categorized into two types: the edge-based models [1], [2], [3], [4] and the region-based models [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [27], [28]. Each of them has its pros and cons, and the choice between them in applications depends on different characteristics of images.

The edge-based models use image gradient as an additional constraint to control the motion of the contour. Usually the edge-based active contour models have an edge-based stopping term to stop the contour on the desired object boundary. The edge-based model [2] introduces a balloon force term to shrink or expand the contour, yet it is difficult to design the balloon force. If the balloon force is not large enough, the contour may not be able to pass through the narrow parts of an object. If the balloon force is too large, the active contour is likely to pass through weak object boundary. In addition, the edge-based models are prone to local minimum, failing to detect the object boundaries when the initial contour is far from the object boundaries.

Region-based active contour models have many advantages over edge-based ones. First, region-based models use the statistical information inside and outside a contour to control the evolution, which is less sensitive to noise and have better performance with weak boundaries. Second, they are less sensitive to the location of the initial contour and then can efficiently detect the exterior and interior boundaries simultaneously. Based on the Mumford–Shah functional [5] for segmentation, Chan and Vese proposed an easily handle model, called the Chan–Vese (CV) model [6], to detect objects whose boundaries were not necessarily detected by the gradient of images. This model has been successfully used in binary phase segmentation with the assumption that each image region is statistically homogeneous and is implemented by the level set method [16]. Vese and Chan extended their work in [17] to utilize multiphase level set functions to represent multiple regions. These models are called piecewise constant (PC) models. However, the CV model and the PC models often lead to poor segmentation results for images with intensity non-uniformity since they assume that the intensities in each region always maintain constant.

In order to segment images with intensity non-uniformity, Vese and Chan [17] and Tsai et al. [18] proposed two similar models, which are called the piecewise smooth (PS) models. However, the PS models are computationally expensive. The local statistical information of intensity is applied in the region-based active contour models to approximate the images for improving the segmentation accuracy, e.g. [8], [9], [10], [12], [15]. For example, Li et al. [15] proposed the local binary fitting (LBF) model, which embeds the local statistical information of intensity into a region-based active contour model, can well segment the images with intensity non-uniformity. These models usually are improved by local statistical information, which is computed by assuming the intensity based on some particular uniform distribution. Nevertheless, the intensities of images are not necessarily described by one kind of specific distribution, i.e. the intensities vary in different positions and so does the intensity uniformity. Hence it will not be accurate to partition each region by the statistical information of intensity computed by a fixed-scale estimation method because the intensity non-uniformity varies in different positions. For solving this problem, the authors [12], [13] proposed the region-based contour models using the histogram of the intensity to drive the evolution of the contour. However, as were shown in the experiments in [12], [13], the precision of segmentation of these models with respect to the natural images sometimes is limited.

In this paper, we introduce a new region fitting term, which enables the extraction of accurate image statistic information. In general, the distribution of the intensity in the local region is more likely to be uniform. In our work, we utilize the statistical information of image in the local region surrounding a contour to construct this new region fitting term. Thus there is no need to extract the local image information with respect to all the pixels in the image. The intensity non-uniformity is induced by the variation of intensity, which can be measured by the magnitude of gradient. Motivated by this idea, we compute the statistical information of the magnitude of gradient to approximate the variation of the intensity in each region. The statistical information of both intensity and the magnitude of gradient in the neighborhood of a contour are computed to construct this new region fitting term. Consequently, more accurate local image information can be used to drive the active contour propagation.

The energy functional of region-based active contour model, such as the Mumford–Shah model [5] and the Chan–Vese model [6], usually includes a regularization term implemented by a total variation (TV) norm for capturing boundaries between sub-regions and keeping the contour smooth. However, this regularization term often introduces undesired over-smoothing to the sharp features of the boundaries, especially corners. The anisotropic TV norm is introduced to be the regularization term of the variational model for image denoising. It is validated to be able to reduce the stair-casing effect [19], [20], [21], [22]. However, as pointed in [8], the contour will need a correct approximation for the finer grids if the anisotropic TV norm is used as the regularization term for image segmentation.

The gradient decent flow of TV norm is identical with curvature, which plays an important role for shock calculations. The similar role is played by the nonlinear diffusion processes. Two types of nonlinear diffusion processes are studied: an isotropic one with a scalar-valued diffusivity, and an anisotropic counterpart with a diffusion tensor, leading to anisotropic diffusion filters. One of the anisotropic diffusion filters based on the structure tensor, introduced by Weikert [23], has been proven its usefulness in many application fields such as corner detection, texture analysis, and diffusion filtering. The structure tensor offers three advantages. Firstly, the matrix representation of the image gradient allows the integration of information from a local neighborhood without cancellation effects. The cancellation effects would appear if two gradients with opposite orientations were integrated with each other. Secondly, smoothing the resulting matrix field yields robustness under noise by introducing an integration scale. This scale determines the local neighborhood over which the orientation estimation at a certain pixel is performed. Thirdly, the integration of local orientation creates additional information, as it becomes possible to distinguish regions where structures are oriented uniformly, like in regions with edges, from areas where structures have different orientations, like in corner regions. The classical structure tensor applies a linear technique such as Gaussian convolution for averaging information within a neighborhood. Gaussian smoothing not only improves the orientation information with regard to noise, but also creates a scale-space as the size of the neighborhood considered for the structure analysis. In principle, it is worthy to use the classic structure tensor in the application of improving the TV norm working as the regularization term of the energy functional.

In our work, we propose an improved regularization term coupling with the anisotropic diffusion process [23] based on the classic structure tensor. The anisotropic diffusion process is implemented on the dual variable of the level set function, which belongs to a particular set. Different from the TV norm and the anisotropic TV norm, this regularization term detects structural image information by the anisotropic diffusion process with respect to the dual variable. By this improved regularization term, the geometric feature of object boundary is preserved effectively.

It is difficult to minimize the energy functional including the TV-regularization term by the traditional gradient descent method, since the TV-regularization term is nonlinear and non-differential. Inspired by the application of dual algorithms for image restoration [24], [25], Chambolle proposed an efficient dual algorithm in [26] to overcome the drawbacks of the traditional numerical approaches. Based on the splitting approach proposed by Aujol et al. [29] and the research of Chan et al. [30], Bresson et al. [28] extend the Chambolle dual algorithm to TVL1 minimization problem by introducing an auxiliary variable to split the L1 data fidelity term into a quadratic one and an L1-term of the new variable. With this treatment, the main sub-problem (a TVL2 minimization problem) can be solved efficiently by the Chambolle dual algorithm. Following this idea, we solve our model through a dual formulation of the minimization problem.

This paper is organized as follows. In the next section, we review the previous work on the Chan–Vese model, the global convex segmentation model (GCS) [14], [28], the local binary fitting (LBF) model [15]. In Section 3, we propose our model and illustrate the advantages of our model. The numerical method of the proposed model is also summarized in this section. In Section 4, we validate our model by experiments on medical images compared with the modern methods such as the LBF model [15], the GCS model [14], [28], the local image fitting (LIF) model [10], and the global intensity fitting model (GIF) [11]. We show the experiments on natural images compared with the GCS model and the Grabcut method [33]. In Section 5, we end the paper by a brief conclusion.

Section snippets

The Chan–Vese model

Chan and Vese [6] proposed an active contour model based on the Mumford–Shah [5]. Let I:ΩR be an input image and C be a close contour. The energy functional is defined byECV(C,c1,c2)=μlength(C)+λ1inside(C)|I(x)c1|2dx+λ2outside(C)|I(x)c2|2dx,xΩWhere μ, λ1, and λ2 are positive parameters. The Euclidean length term is used to regularize the contour. c1 and c2 are the intensity averages of I inside and outside the contour C, respectively.

To minimizing the above energy functional, the level

Improved region-based active contour model

In this section, we shall present and discuss the details of our proposed improved region-based active contour model and its numerical implementation. The energy functional EIRACM of the proposed model consists of two parts: the new local region fitting term EF and the improved regularization term ER. Thus the overall energy functional can be described asEIRACM=EF+ER

Experiments

In this section, we will present the experimental results from our model on some medical images and the natural images. All the experiments are run with Matlab code on the work station of CPU 2.67 GHz, RAM 4.00 G. The comparison of the accuracy will be shown in Fig. 15, Fig. 16.

In Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7, Fig. 8, Fig. 9, Fig. 10, Fig. 11, we show the experiments results for medical image segmentation of the local binary fitting model (LBF) [15], the GCS model [24], the local image

Conclusions

In this paper, we propose a new region-based active contour model for image segmentation. The proposed model is different from other general region-based models in two ways. First, the new regularization term proposed in the proposed model is capable of extracting the complete local structural information from an image. It is represented by the dual formulation and implemented by the efficient dual algorithm. Second, the statistics of the intensity and the magnitude of gradient are extracted in

Qi Ge received the B.Sc. degree in College of Math & Physics, Nanjing University of Information Science & Technology, Nanjing, China, in 2006, M.Sc. degree in Applied Mathematics from College of Math & Physics, Nanjing University of Information and Technology, Nanjing, China, in 2009. She is now in pursuit for Ph.D. degree in Pattern Recognition and Intelligent System in Nanjing University of Science and Technology. Her research interests include pattern recognition, image processing, and image

References (34)

  • X. Bresson, T. Chan, Non-local Unsupervised Variational Image Segmentation Model, Technical Report 67, Mathematical...
  • T. Brox et al.

    On the statistical interpretation of the piecewise smooth Mumford–Shah functional

    Scale Space and Variational methods in Computer Vision

    (2010)
  • Y. Tian, M. Zhou, Z. Wu, X. Wang, A region-based active contour model for image segmentation, in: 2009 IEEE...
  • K. Ni et al.

    Local histogram based segmentation using the Wasserstein distance

    International Journal of Computer Vision

    (2009)
  • J. Kim et al.

    A nonparametric statistical method for image segmentation using information theory and curve evolution

    IEEE Transactions on Image Processing

    (2005)
  • T. Goldstein, X. Bresson, S. Osher, Geometric Applications of the Split Bregman Method: Segmentation and Surface...
  • C. Li et al.

    Minimization of region-scalable fitting energy for image segmentation

    IEEE Transactions on Image Processing

    (2008)
  • Cited by (27)

    • Segmentation of bias field induced brain MR images using rough sets and stomped-t distribution

      2019, Information Sciences
      Citation Excerpt :

      But, the intrinsic nature of the brain makes this procedure much difficult and challenging. Hence, in medical imaging, accurate segmentation of the brain structure into its three main tissue types, namely, white matter (WM), gray matter (GM), and cerebro-spinal fluid (CSF), has gained significant amount of research attention in past few decades [10,15,19]. Brain image segmentation is essential for many diagnostic studies.

    • Robust noise region-based active contour model via local similarity factor for image segmentation

      2017, Pattern Recognition
      Citation Excerpt :

      The principle of active contour model is based on the theory of contour evolution under given constraints to more accurately detect the object boundaries. As a consequence, depending on the kind of information used, existing active contour models are normally categorized into several types: edge-based models [1,3,4,6–11], global region-based models [5,12–16], edge/region-based active contour models [17], local region-based active contour models [18–29] and global/local region-based active contour models [30]. Edge-based models design a gradient stop function to accurately segment object boundaries in high quality images, while they are more sensitive to image noise and boundaries presenting weak gradient magnitude.

    • Pixel classification based color image segmentation using quaternion exponent moments

      2016, Neural Networks
      Citation Excerpt :

      In region-based image segmentation, the intensity inside the region is assumed to be approximately constant, to vary only slowly or to be generated by a suitable probability model. Ge, Xiao, and Zhang (2012) proposed a new region-based active contour model for image segmentation. In particular, this model utilizes an improved region fitting term to partition the regions of interests in images depending on the local statistics regarding the intensity and the magnitude of gradient in the neighborhood of a contour.

    • Quantification of the internal structure and automatic generation of voxel models of textile composites from X-ray computed tomography data

      2015, Composites Part A: Applied Science and Manufacturing
      Citation Excerpt :

      This paper investigates the application of a so-called structure tensor for the determination of the orientations of the fibres in textile composites based on micro-CT images. The concept of the structure tensor is known from the field of image processing, where it is used for the detection of edges in images [27,28], image segmentation [29], to visualize and quantify tissue microstructure [30], to quantify structural anisotropy in tissue [31] and for analysis of textures in images [32]. Here the structure tensor is also used for the segmentation of the micro-CT image into matrix, yarns and voids (pores) for the purpose of automatic generation of a voxel-based description or voxel model of a composite material.

    • Saliency Snake: A unified framework with adaptive initial curve

      2014, Optik
      Citation Excerpt :

      Over the past decades, researches about active contour model have been a hot topic and various approaches have been proposed up to now. The latest literature about active contour models have mainly focused on increasing the capture range [8,9], improving energy terms [10,11] and extending the application area [12–14]. These research results have improved the performance of active contour model from different aspects and in various domains.

    View all citing articles on Scopus

    Qi Ge received the B.Sc. degree in College of Math & Physics, Nanjing University of Information Science & Technology, Nanjing, China, in 2006, M.Sc. degree in Applied Mathematics from College of Math & Physics, Nanjing University of Information and Technology, Nanjing, China, in 2009. She is now in pursuit for Ph.D. degree in Pattern Recognition and Intelligent System in Nanjing University of Science and Technology. Her research interests include pattern recognition, image processing, and image segmentation.

    Zhi Hui Wei (Professor, Ph.D.) received the B.Sc. and M.Sc. degrees in Mathematics from Southeast University, Nanjing, China, in 1983 and 1986, respectively. He received the doctor degree in 2003 in Radio Electronics Department from Southeast University, Nanjing, China, in 2003. His research interests cover image processing, image modeling, wavelet analysis, multi-scale analysis, digital watermark, and image coding and compressing.

    Liang Xiao (Ph.D.) is an Associate Professor and his research interest covers variational partial differential equation (PDE) application in image processing, image modeling, pattern recognition, motion estimation and tracking, virtual reality, and systemsimulation.

    Jun Zhang received B.Sc. and M.Sc. degrees in Computational Mathematics from Wuhan University, Wuhan, Hubei, China, in 1999 and 2002, respectively, and the Ph.D. degree in Pattern Recognition and Intelligent Systems from Nanjing University of Science and Technology, Nanjing, Jiangsu, China, in 2010. He is currently a lecturer of Nanjing University of Science and Technology. His research interests are in mathematical image processing. His current research is focused on the fractional-order PDE based image modeling and algorithms in image denoising, inpainting, segmentation, and super-resolution image processing, image denoising, and pattern recognition.

    View full text