Elsevier

Information Sciences

Volume 493, August 2019, Pages 152-175
Information Sciences

A variational level set model for multiscale image segmentation

https://doi.org/10.1016/j.ins.2019.04.048Get rights and content

Abstract

In this paper, we propose a variational level set model for multiscale image segmentation that can extract desired object from the background under various scales. The new model consists of three terms: A boundary extraction term, a regularization term and a multiscale representation term. The boundary extraction term is based on the Chan–Vese (CV) functional, which drives the active contour toward the boundaries of the objects. The regularization term penalizes the length of the active contour and keeps the level set function close to a signed distance function, which enables the active contour to maintain its stability during the evolution process. The multiscale representation term is based on the total variation (TV) energy that can provide a multiscale representation of the input image by tuning the scale parameter. We use an alternating iterative algorithm that combines the gradient descent algorithm and the alternating direction method of multipliers (ADMM) to numerically solve the model. The experimental results for both synthetic and real images demonstrate the robustness and effectiveness of the proposed model for multiscale image segmentation. In addition, the proposed model is a promising model for image denoising applications.

Introduction

Image segmentation is the partitioning of an image domain into parts and the extraction of objects of interest from a complex background, which can be applied in computer vision [1], [2], image analysis [2], [17], and image understanding [16], [18]. In the past few decades, image segmentation has received wide attention in both theoretical and applied research and many segmentation methods have been proposed, such as threshold methods [26], [31], edge detection [7], [24], region growing [22], [48], snakes model [11], geometric active contour (GAC) model [4], and image clustering segmentation [27], [33]. Among these methods, the GAC model that is based on the level set method [25], [30], has received substantial attention from scholar due to its sound theoretical basis and satisfactory experimental performance.

The traditional GAC model is based on the level set method and curve evolution [4], [28], [30]. This model implicitly represents a two-dimensional curve C by the zero level set of a three-dimensional Lipschitz function z=ϕ(x,y), namely, C={(x,y):ϕ(x,y)=0}, where the function ϕ(x, y) is called the level set function, which is typically defined as the signed distance function of the curve C. In this case, the evolution curve C(x(t), y(t)) is represented by the zero level set of the family of functions ϕ(x, y, t) at time t. Then, the solution of the curve C can be obtained by evolving the level set and solving a partial differential equation (PDE) with respect to the level set function ϕ(x, y, t). Compared to the parametric active contour method, which evolves the contour by solving the PDE with respect to the parameters of curve C(x(t), y(t)), GAC allows for cusps, corners, and automatic topological changes.

Another GAC model is the variational level set model, which is based on variational theory and the level set method [5], [14]. Similar to the traditional GAC model, the variational level set model embeds the curve C into a high-dimensional function and represents it by the zero level set of this function. The difference is that to obtain the evolving PDE with respect to the level set function, traditional GAC converts the parametric active contour into the geometric active contour via the differential geometry method directly, while the variational level set model is based on energy minimization (or variational calculus) and the gradient descent method. Compared to the traditional GAC model, the variational level set model can solve for the evolving curve more flexibly by incorporating constraints into the energy.

Models that utilize the variational level set method can be divided into two main categories: edge-based models [14], [19] and region-based models [8], [21], [23], [35], [36], [37], [38]. Edge-based models utilize the image gradient to define an edge stopping function that stops the contour evolution on the object boundaries and detect the boundaries accurately by detecting sharp intensity changes. However, this function often fails to detect the objects with weak or narrow boundaries. In addition, the edge-based models can easily become trapped in a local minimum and fail to detect boundaries that are far away from the initial contour [5].

Region-based models incorporate the regional statistical information into the energy. They are less sensitive to noise and perform better for images with weak object boundaries. In addition, they are not sensitive to the location of the initial contour and can detect boundaries that are far away from the initial contour. One of the most popular region-based models is the CV model [5], which is based on the Mumford–Shah functional [28]. The CV model is a piecewise-constant and binary segmentation model, which performs well only for images with two constant regions (namely, images with only foreground and background regions that are each of homogeneous intensity). The CV model fails to segment images with intensity inhomogeneity, strong noise or multiple-objects. Therefore, many improved region-based models have been proposed for various applications, such as [20], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45].

For segmenting images with intensity inhomogeneity, many local region models [12], [13], [42], [43], [44], [45] have been proposed. The local binary fitting (LBF) model [13] and the local image fitting (LIF) model [44] are two famous local region models, which use the local fitting energy to identify objects with intensity inhomogeneity. However, the two models only update the local intensity means to handle intensity inhomogeneity but ignore the local intensity variance. Hence, they fail to segment images with nonuniform local intensity [38]. In [38], Wang et al. proposed a local Gaussian distribution fitting (LGDF) energy for modeling images with nonuniform local variances. Li et al. [12] utilized the local intensity clustering (LIC) criterion to obtain the segmentation results and bias correction simultaneously. Similarly, the LIC model ignores the cluster variance. Zhang et al. [43], [45] proposed a local statistical active contour model (LSACM) that is based on the maximum a posterior (MAP) criterion for improving the LIC model. Subsequently, Wu et al. [40] proposed a retinex-modulated piecewise-constant variational model that is based on Retinx theory for segmenting images and correcting bias field. Duan et al. [8] proposed a two-stage segmentation method that obtains the true intensity and the bias field in the first stage and calculates the segmentation results via thresholding in the second stage.

To segment images that are contaminated by strong noise, robust variational level set models have been proposed. Wu et al. [41] proposed a region-based fuzzy active contour model with a kernel metric (KFAC), where the fuzzy measurement and kernel metric in the energy render the updating of region centers more robust to noise. Liu et al. [21] proposed a data fidelity term that utilizes a kernel metric that is based on the Gaussian radial basis function kernel to reduce the influence of noise. Tang [33] proposed a variational level set model that is combined with the fuzzy C means clustering algorithm with a spatial (FCMS) constraint, which renders the model more robust to noise. Another approach is to regularize the level set function to improve the robustness of the model to noise. For example, Zhang et al. [44] utilized a Gaussian kernel to smooth the level set function at each iteration. Zhang et al. [46] proposed a reaction-diffusion model for evolving the curve, in which the diffusion term, which is equivalent to a Gaussian filter, enables the contour to remain stable during the evolution. Recently, Wu et al. [39] proposed an indirectly regularized method for restraining the auxiliary function of the level set function.

To segment images with multiple objects, multiphase segmentation models have been proposed. For example, Vese and Chan [34] proposed a multiphase level set framework that uses more than one level set function. They showed that K level set functions can be used to construct up to N=2K distinct indicator functions and, therefore, to represent up to N regions. To improve the segmentation quality in [34], Bertelli et al. [3] proposed new multiphase length and area regularization terms, which enable the contours to sense the presence of the other nearby contours and evolve accordingly. Similarly, Zhang et al. [47] incorporated local information into the energy and proposed a variational multiphase level set approach for simultaneous segmentation and bias correction that utilizes multiple level set functions. In contrast to this approach, Chung and Vese [6] proposed a multilayer variational level set model that uses m distinct levels of a single level set function to partition the image domain into m+1 disjoint regions.

The motivation of this work is stated as follows: The variational level set models that are described above are examples of a large class of image segmentation approaches that are under a fixed scale. If they are used for object extraction, one can only extract the object on a fixed scale. However, a multiscale approach is appropriate for image segmentation because: (i) Humans visualize a scene under multiple scales and single-scale segmentation may not accurately represent the process that is utilized by the human visual system (HVS) for target identification; (ii) multiscale image segmentation can provide fine object extraction results that contain many image details and coarse extraction results that only contain object contours according to the requirements; (iii) multiscale image segmentation can provide abundant resource material for image consequent processing, such as pattern recognition, image retrieval, and computer-aided diagnosis. To realize reliable segmentation on multiple scales, both large-scale and small-scale behaviors should be investigated and incorporated appropriately. Thus, a natural approach for addressing this problem is multiscale analysis.

In this paper, we propose a multiscale image segmentation model for extracting a desired object from the background under multiple scales. The new model is based on the variation method and level set evolution and consists of three terms. The first term is the CV energy, which drives the active contour toward edges of the object; the second term is regularization term, which enables the contour to maintain stability during the evolution; and the last term is the total variation (TV) energy, which provides a multiscale representation of the input image. By minimizing the energy that is defined as the sum of these three terms, we can obtain a multiscale segmentation of the input image. In addition, we use an alternating iterative algorithm in combination with the gradient descent algorithm and the alternating direction method of multipliers (ADMM) to numerically solve the proposed model. The main contributions of this paper are summarized as follows:

(1) We propose a new variational level set model for image multiscale segmentation, which consists of three components: the boundary extraction term, the regularization term and the multiscale representation term. Our model can also be applied for image denoising.

(2) We describe properties of the scaling parameter and introduce an alternating iterative algorithm that combines the gradient descent algorithm and the alternating direction method of multipliers (ADMM) to numerically solve the model.

(3) We conduct experiments on both synthetic and real images to evaluate the robustness and accuracy of the proposed model for multiscale image segmentation and image denoising applications.

The remainder of the paper is organized as follows: In Section 2, we propose the multiscale segmentation model. In Section 3, we apply the alternating iterative algorithm that combines the gradient descent algorithm with the ADMM algorithm to numerically solve the proposed model.The experimental results are presented in Section 4. Our work is summarized in Section 5.

Section snippets

Proposed model

Let Ω be an image domain that is a bounded open subset of R2 and f(x):ΩR be an input digital image. The multiscale segmentation model is defined as follows:minC,uλ{E(uλ,C)+R(C)+M(uλ,f)},where λ > 0 is a scale parameter that controls the scale of the image representation; uλ is a multiscale representation of the image f, C is the evolving contour for capturing the desired object, E(uλ, C) is the boundary extraction energy, which drives the contour C toward the boundaries of the object; R(C) is

Numerical implementation of the proposed model

In this paper, the alternating iterative algorithm is employed to minimize the energy E(c1, c2, uλ(x), ϕ(x)) with respect to the functions ϕ(x) and uλ(x) and constants c1 and c2. The minimizing solutions of ϕ(x), uλ(x) and (c1, c2) are obtained alternatively while the other variables are fixed, which leads to{(c1k+1,c2k+1)=argminc1,c2E(c1,c2,uλk(x),ϕk(x)),ϕk+1(x)=argminϕ(x)E(c1k,c2k,uλk(x),ϕ(x)),uλk+1(x)=argminuλ(x)E(c1k,c2k,uλ(x),ϕk(x)).

In the following, we solve each subproblem in the

Numerical results

In this section, the proposed model is evaluated in terms of image multiscale segmentation performance on both synthetic and real images. In addition, we apply it to object extraction and image denoising. Comparison experiments with several state-of-the-art variational models in object extraction and image denoising are also performed to evaluate the performance of the proposed model. All the programs are coded in Matlab and implemented on a personal computer with an Inter Pentium CPU 2.50 GHz,

Conclusions

In this paper, by simulating the HVS to visualize the scenes, we proposed a variational level set model for multiscale image segmentation, which can extract the objects from the background on various scales. The new model consists of three terms: A boundary extraction term, a regularization term and a multiscale representation term. We used an alternating iterative algorithm that combines the gradient descent algorithm and the ADMM algorithm to numerically solve the model. Experimental results

Acknowledgment

This work was supported in part by the Natural Science Foundation of China under Grant No. 61561019, and the Doctoral Scientific Fund Project of Hubei University for Nationalities under Grant No. MY2015B001.

References (48)

  • S. Patra et al.

    A novel context sensitive multilevel thresholding for image segmentation

    Appl. Soft Comput.

    (2014)
  • L. Rudin et al.

    Nonlinear total variation based noise removal algorithms

    Physica D.

    (1992)
  • H. Wang et al.

    An active contour model and its algorithms with local and global Gaussian distribution fitting energies

    Inf. Sci.

    (2014)
  • L. Wang et al.

    An active contour model based on local fitted images for image segmentation

    Inf. Sci.

    (2017)
  • L. Wang et al.

    Active contours driven by edge entropy fitting energy for image segmentation

    Signal Process.

    (2018)
  • L. Wang et al.

    Active contours driven by local Gaussian distribution fitting energy

    Signal Process.

    (2009)
  • Y. Wu et al.

    Indirectly regularized variational level set model for image segmentation

    Neurocomputing

    (2016)
  • Y. Wu et al.

    A retinex modulated piecewise constant variational model for image segmentation and bias correction

    Appl. Math. Model.

    (2018)
  • Y. Wu et al.

    Novel fuzzy active contour model with kernel metric for image segmentation

    Appl. Soft Comput.

    (2015)
  • Y. Yang et al.

    Split Bregman method based level set formulations for segmentation and correction with application to MR images and color images

    Magn. Reson. Imaging

    (2019)
  • K. Zhang et al.

    Active contours driven by local image fitting energy

    Pattern Recognit.

    (2010)
  • D. Barik et al.

    Object identification for computer vision using image segmentation

    IEEE International Conference on Education Technology and Computer

    (2010)
  • L. Bertelli et al.

    On the length and area regularization for multiphase level set segmentation

    Int.J. Comput. Vision

    (2010)
  • V. Caselles et al.

    Geodesic active contours

    Int. J. Comput. Vision

    (1997)
  • Cited by (22)

    • A level-set method with the 3D multiplicative-additive model for CT volume data segmentation

      2024, Measurement: Journal of the International Measurement Confederation
    • Unsupervised learning of multi-task deep variational model

      2022, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      In the last experiment, our DFV model (Fig. 4) is applied for multiscale and object segmentation, with the additional effects of structure–texture extraction and restoration. The variational multiscale segmentation model [45] and double-DIP networks [13] are used for comparison. A segmentation comparison with double-DIP networks is shown in Fig. 13.

    • RVLSM: Robust variational level set method for image segmentation with intensity inhomogeneity and high noise

      2022, Information Sciences
      Citation Excerpt :

      In the following, we compare the proposed RVLSM with three classical level set models (CV [20], LBF [22], and LIC [32]), four state-of-the-art level set models (CLSM [34], LHIF [26], LaGII [23], and LATE [28]), and two popular anti-noise level set models (SLSA [33] and RLSF [38]). Other state-of-the-art models such as [40–43] are not compared with our method since their source codes are currently unavailable. The experiments are first conducted to compare our RVLSM with three classical models, involving CV [20], LBF [22], and LIC [32], on 60 natural images that randomly chosen from the Weizmann database [49].

    • Make complex CAPTCHAs simple: A fast text captcha solver based on a small number of samples

      2021, Information Sciences
      Citation Excerpt :

      This paper proposes a fast and general text-based captcha solver based on a small number of samples to overcome the above-mentioned difficulties. It is worth noting that the attack accuracy of captcha recognition heavily depends on character segmentation, and moreover, the complex security features such as occluding line, character collapsing, and noisy background are the main restriction factors for segmentation [23,42]. Therefore, the core idea of this study was to design a model that can transform complex captchas into simple ones with security features removed, thereby reducing the number of labeled samples required for recognition network.

    View all citing articles on Scopus
    View full text