Elsevier

Medical Image Analysis

Volume 17, Issue 7, October 2013, Pages 746-765
Medical Image Analysis

Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features

https://doi.org/10.1016/j.media.2013.04.004Get rights and content

Abstract

Phase-contrast microscopy is one of the most common and convenient imaging modalities to observe long-term multi-cellular processes, which generates images by the interference of lights passing through transparent specimens and background medium with different retarded phases. Despite many years of study, computer-aided phase contrast microscopy analysis on cell behavior is challenged by image qualities and artifacts caused by phase contrast optics. Addressing the unsolved challenges, the authors propose (1) a phase contrast microscopy image restoration method that produces phase retardation features, which are intrinsic features of phase contrast microscopy, and (2) a semi-supervised learning based algorithm for cell segmentation, which is a fundamental task for various cell behavior analysis. Specifically, the image formation process of phase contrast microscopy images is first computationally modeled with a dictionary of diffraction patterns; as a result, each pixel of a phase contrast microscopy image is represented by a linear combination of the bases, which we call phase retardation features. Images are then partitioned into phase-homogeneous atoms by clustering neighboring pixels with similar phase retardation features. Consequently, cell segmentation is performed via a semi-supervised classification technique over the phase-homogeneous atoms. Experiments demonstrate that the proposed approach produces quality segmentation of individual cells and outperforms previous approaches.

Graphical abstract

Highlights

  • Construct a dictionary based on diffraction patterns of phase contrast images.

  • Develop a sparse representation approach to restore phase retardation features.

  • Partition phase contrast microscopy images into phase-homogeneous atoms.

  • Segment cells via semi-supervised classification over the phase-homogeneous atoms.

  • Provide useful information for cell classification and potential applications.

Introduction

Phase contrast microscopy (Murphy, 2001) is a widely used optical microscopy technique, especially for the examination of transparent and colorless specimens. It produces images by converting the phase difference between waves traversing the biological material and those passing through the surrounding medium to a visible difference in image intensity. Therefore, it allows cell observation without exogenous fixing or staining, and thus enables a long-term monitoring of proliferation processes of live cells by recording time-lapse microscopy images, such as cell migration, cell cycle, and cell differentiation. Although analysis of such images can be conducted manually, large volumes of image data captured from long-time high-throughput biological experiments make manual analysis extremely time-consuming, labor-intensive and prone to human error. Therefore, it is imperative to develop a computer-aided method to identify the individual cells and measure relevant cell characteristics automatically and accurately. Among the tasks of automatic microscopy cell image analysis, cell segmentation is recognized as one of the most fundamental components, because lots of subsequent analysis is performed based on it, e.g., cell tracking and cell event detection, etc.

Different challenges arise for cell segmentation in phase contrast microscopy images. Firstly, phase contrast microscopy images are often of low contrast between cells and background. Therefore, the single grayscale thresholding (e.g., Otsu, 1979) may fail to separate dark cells out of the background due to the low contrast (see Fig. 1.2). A multi-level thresholding method, which segments images into bright cells, dark cells and background, improves the results, but cells are still not well segmented due to the intensity similarities between some cells and background (see Fig. 1.3). Deformable models, a popular category of approaches including active contours (Yezzi et al., 1999) and level sets (Vese et al., 2002), are also challenged by low contrast of phase contrast microscopy images. Active contour (Grimm et al., 2003, Li et al., 2009) detects positions of cells’ edges in phase contrast microscopy images, but it presents poor results if the boundaries are fuzzy. Level-set based approaches (Xiong et al., 2006, Padfield et al., 2009, Ambuhl et al., 2012), which compute energy of an object using intensity variance inside and outside the contours, are sensitive to initializations (see Fig. 1.4).

Secondly, cell segmentation is challenged by the dissimilarity among subcellular structures. Watershed (Kachouie et al., 2008), which assumes intensity gradients are small inside cells while they are large around cell membranes, often results in over segmentation due to regional minimums (see Fig. 1.5). Ta et al., 2009, Lesko et al., 2010 implemented cell segmentation via the graph cut framework (Felzenszwalb and Huttenlocher, 2004), which cuts a weighted graph into exclusive sub-graphs according to morphological features of cells, but likewise, it renders noisy over-segmented clips (see Fig. 1.6).

Thirdly, cell segmentation process is also challenged by the presence of imaging artifacts caused by phase contrast optics, such as bright halos and shade-off. Segmentation algorithms that treat phase contrast microscopy images as general natural images do not always yield good results, because imaging mechanisms of microscopy and natural images are completely different (Murphy, 2001). Recently, Yin et al. (2012) studied the optical properties of phase contrast microscopes, and developed a linear imaging model to approximate its image formation process. As a result, artifacts are removed based on this imaging model, and cells can be segmented with effortless thresholding. However, this linear imaging model fails to segment bright cells in phase contrast microscopy images, e.g., mitotic or apoptotic cells, as shown in Fig. 1.7, because the model assumes that the phase retardation caused by cells is small, which is not valid when cells become thick during their mitotic and apoptotic stages and thus appear bright in phase contrast microscope.

In summary, cell segmentation still remains a challenging problem because of low contrast in microscopy images, unidentifiable textures between different cell identities along wih dissimilar image intensities between subcellular structures within the same cells, and inherent artifacts such as bright halos and shade-off.

In this paper, we analyze the particular formation process of phase contrast microscopy images, and propose a novel cell segmentation scheme via semi-supervised classification over the optics-related image features. The general framework of our proposed approach for cell segmentation is illustrated in Fig. 2 with four major modules.

  • 1.

    Construction of a dictionary based on diffraction patterns. Phase contrast microscopy images are generated by the interference of waves diffracted by specimens and those passing through background medium. Considering this particular mechanism, we construct a dictionary based on diffraction patterns with different phase retardations to approximate the imaging process.

  • 2.

    Discrimination-oriented sparse representation of phase contrast microscopy images. We model a phase contrast microscopy image by sparse representation of diffraction patterns, i.e., a linear combination of a few bases corresponding to different diffraction patterns selected from the dictionary. Since cell segmentation is a discriminative task (i.e., differentiating cell pixels from background pixels), we formulate an optimization problem that is both reconstructive and discriminative to derive phase retardation features for each pixel.

  • 3.

    Phase-homogeneous atom generation based on phase retardation features. We partition phase contrast microscopy images into phase-homogeneous atoms by clustering neighboring pixels with similar phase retardation features, thereby eliminating the local redundancy of images. The attributes of an atom include both its unary features and characteristics of its neighboring atoms. Cell segmentation is then performed over the atoms.

  • 4.

    Cell segmentation based on label propagation. Addressing the inherent complexity and ambiguity of cell images, we advocate a semi-supervised scheme based on label propagation to implement cell segmentation, which utilizes a small number of labeled atoms together with large amounts of unlabeled atoms to build a better classifier. The proposed scheme requires less human effort, and renders a better result.

Compared with the previous approaches for cell segmentation based on image intensity, we study the particular imaging mechanism of phase contrast microscopy and develop a sparse representation approach to approximate its imaging process. By analyzing the phase retardation features of different cells, each pixel is restored into an optics-related feature vector – phase retardation feature. Subsequently, cell segmentation is achieved via semi-supervised classification over the phase retardation features. Experimental results validate that cells with different optical natures, either dark or bright, are segmented out of background with a high quality. As an additional advantage, the segmentation results are of great relevance to biophysical information and useful for cell category classification (e.g., intermitosis, mitosis).

The remainder of this paper is organized as follows. In Section 2, we study the imaging mechanism of phase contrast microscopy, and model the imaging process with discrimination-oriented sparse representation. In Section 3, we introduce phase-homogeneous atom generation and semi-supervised classification method to realize cell segmentation. The experimental setup and results with discussions are presented in Section 4. Finally, the paper is summarized in Section 5.

Section snippets

Phase contrast imaging model based on discrimination-oriented sparse representation

In this section, we develop a discrimination-oriented sparse representation method to restore phase contrast microscopy images. Firstly, we construct a dictionary based on different diffraction patterns by understanding the optics of phase contrast microscopy. Furthermore, we select a small number of bases based on their discrimination power. The feature corresponding to the bases with top discrimination power, so-called phase retardation features are computed for each pixel, which will be used

Cell segmentation based on semi-supervised classification

The imaging model maps a microscopy image to a feature space, and provides a facilitative precondition for subsequent analysis. In this section, we first present how to partition a phase contrast microscopy image into phase-homogeneous atoms by clustering neighboring pixels’ phase retardation feature vectors. Then, we introduce a semi-supervised classification technique to segment cells by classifying the phase-homogeneous atoms.

Experiments

In this section, we first describe the datasets in Section 4.1, and then we present the parameter setting in Section 4.2. We demonstrate and evaluate our discrimination-oriented sparse representation in Section 4.3. Cell segmentation results are presented and compared with other approaches in Section 4.4, and the quantitative evaluation on cell segmentation is summarized in Section 4.5. Finally, we show how our segmentation result can facilitate cell category classification in Section 4.6.

Conclusion

In this paper, we propose a cell segmentation algorithm via a semi-supervised classification technique over phase retardation features , which are introduced based on the understanding of phase contrast microscopy image formulation. Specifically, a phase contrast microscopy image is modeled with a dictionary of different diffraction patterns caused by specimens, and based on the model phase retardation features for each pixel in the image are obtained by solving a discriminative min  1

Acknowledgments

This research is supported by funds from Cell Image Analysis Consortium of Carnegie Mellon University and University of Missouri Research Board. The work of Hang Su is supported by grants of NFSC. 61171172 and NSFC. 61102099.

References (30)

  • H.P. Grimm et al.

    Analysis of actin dynamics at the leading edge of crawling cells: implications for the shape of keratocyte lamellipodia

    European Biophysics Journal

    (2003)
  • K. Huang et al.

    Sparse representation for signal classification

  • S. Huh et al.

    Automated mitosis detection of stem cell populations in phase contrast microscopy images

    IEEE Transactions on Medical Imaging

    (2011)
  • S. Huh et al.

    Apoptosis detection for adherent cell populations in time-lapse phase-contrast microscopy images

  • Kachouie, N.N., Fieguth, P., Jervis, E., 2008. Watershed deconvolution for cell segmentation. In: 30th Annual...
  • Cited by (79)

    • Segmentation of patchy areas in biomedical images based on local edge density estimation

      2023, Biomedical Signal Processing and Control
      Citation Excerpt :

      Based on the above approach, we have proposed and implemented a simple algorithm for the (semi-)automated segmentation of biomedical images according to their local patchiness. We have shown explicitly that the proposed algorithm achieves 95 to 99% median accuracy in the segmentation of image areas covered by the cell monolayer in an in vitro scratch assay, that is competitive with state-of-the-art approaches used in microscopic image segmentation under certain conditions (see, e.g., [25,39–42] and references therein). Cross-validation with the state-of-the-art machine learning based approaches indicate that only slight enhancement could be achieved, mainly for strongly imbalanced image sets, at the cost of significant computation times that are several orders of magnitude above those required by the proposed algorithm.

    • Automatic quantitative analysis of metabolism inactivation concentration in single bacterium using stimulated Raman scattering microscopy with deep learning image segmentation

      2022, Medicine in Novel Technology and Devices
      Citation Excerpt :

      Therefore, an automated analysis algorithm for rapid bacterial segmentation and SC-MIC calculation is urgently needed in clinics. Machine learning algorithms [14] have been used for cell segmentation in images. However, these methods are inaccurate when cells and bacteria are in touch and overlapping.

    • GANs for medical image analysis

      2020, Artificial Intelligence in Medicine
    • DeepDistance: A multi-task deep regression model for cell detection in inverted microscopy images

      2020, Medical Image Analysis
      Citation Excerpt :

      These studies directly use the gradients to define their feature maps (Koyuncu et al., 2016) or alternatively get pixels voted along their gradient directions and use the votes the pixels take to define their maps (Xing et al., 2014). The subregion-wise studies first partition an image into over-segmented subregions (e.g., superpixels), extract handcrafted color, gradient, and shape features from these subregions, and merge them based on their extracted features to obtain cell locations (Genctav et al., 2012; Su et al., 2013; Koyuncu et al., 2018). To reduce the required efforts for manual feature definition, deep learning based methods learn high-level features from image data.

    • Microscopy image formation, restoration, and segmentation

      2020, Computer Vision for Microscopy Image Analysis
    View all citing articles on Scopus
    View full text