Segmentation of cytoplasm and nuclei of abnormal cells in cervical cytology using global and local graph cuts

Abstract

Automation-assisted reading (AAR) techniques have the potential to reduce errors and increase productivity in cervical cancer screening. The sensitivity of AAR relies heavily on automated segmentation of abnormal cervical cells, which is handled poorly by current segmentation algorithms. In this paper, a global and local scheme based on graph cut approach is proposed to segment cervical cells in images with a mix of healthy and abnormal cells. For cytoplasm segmentation, the multi-way graph cut is performed globally on the a* channel enhanced image, which can be effective when the image histogram presents a non-bimodal distribution. For segmentation of nuclei, especially when they are abnormal, we propose to use graph cut adaptively and locally, which allows the combination of intensity, texture, boundary and region information. Two concave points-based approaches are integrated to split the touching-nuclei. As part of an ongoing clinical trial, preliminary validation results obtained from 21 cervical cell images with non-ideal imaging condition and pathology show that our segmentation method achieved 93% accuracy for cytoplasm, and 88.4% F-measure for abnormal nuclei, outperforming state of the art methods in terms of accuracy. Our method has the potential to improve the sensitivity of AAR in screening for cervical cancer.

Introduction

Screening by cytology is the commonest approach to prevent cervical cancer at a pre-cancerous stage [1]. However, it is known that screening of cervical cytology slides is labor intensive and mentally demanding on the cytotechnologist [2]. Automation-assisted reading (AAR) techniques have the potential to increase productivity and reduce screening errors. AAR uses automated microscopy to collect images of cervical cells, and then performs cell segmentation to quantitate cellular structures and morphology in order to extract candidate cells for targeted reading by pathologists. A large, prospective randomized trial found that although AAR increased the productivity by 60–80%, the sensitivity was reduced [3]. To improve the sensitivity, it is important to segment the abnormal cells reliably.

A variety of segmentation methods for cervical cytology have been proposed in recent years. The majority of cytoplasm segmentation used one or multiple of the following techniques: K-means [4], [5], edge detection [6], thresholding [7], [8] and active contours [5], [9]. Most of these works are designed for images of isolated cells, especially for those in the Herlev data set [11]. For segmentation in images containing multiple cells, thresholding [7], [8], [12] and level set [9], [10] techniques have been used. However, thresholding may lead to inaccuracy due to non-ideal imaging conditions, such as inconsistent staining and/or illumination and overlapping cells. The level set method is also suboptimal, due to computation cost and its tendency to locate some local extrema (e.g., inhomogeneous illumination).

For nucleus segmentation, related works can be divided into three groups: single-nucleus segmentation, multiple-nuclei segmentation, and touching-nuclei splitting. Existing single-nucleus segmentation approaches utilize contour, shape and color information by using active contour model [5], [13], parametric fitting [14] and difference maximization [4]. Nucleus segmentation for images containing multiple cells employ thresholding [9], [15], Hough transform [7], morphology (watershed) [8], [16], [17], and level set [10] techniques. Some of these methods [7], [8], [9] assume all nuclei located within the cytoplasm, as a result in that bare nuclei, which are strongly indicative of early cervical cancer, are missed. For splitting of touching-nuclei, morphological erosion [15] is a commonly used approach. An unsupervised method [18] for splitting touching nuclei is proposed by a combination of distance transform, expectation–maximization algorithm and ellipse fitting techniques. Through representing the nuclear shape by the vibrations of a spring-mass system, and learning the vibration models by active shape model, the nuclear boundary in the overlapping areas can be obtained accurately [19].

Generally, there are many cervical cells with mutual overlaps in a field-of-view (FOV) (see Fig. 1(a) as an example). Hence, a segmentation algorithm should capture the cytoplasm, multiple nuclei and touching-nuclei. Three of the aforementioned methods [7], [8], [10] can achieve all of these three tasks. However, these methods were developed to segment images, which only contain healthy cells rather than the mixture of healthy and abnormal cells. Since the size, shape and chromatin distributions of abnormal nuclei vary significantly (Fig. 1(b)), further development is needed to address typical variations encountered in a clinical setting. So far we have found only one previous study of automated segmentation of abnormal nuclei [20] and only very limited data was reported.

Segmentation of nuclei in histological images employ adaptive thresholding combined with active contour models [21]. Such automated methods are comparable with the manual delineation in segmentation accuracy. Similarly, graph cut (GC) approaches [22] are highly attractive in nucleus segmentation. The binarization of nuclei based on GC is addressed in [23] with results being more accurate than global thresholding. Prior knowledge like nucleus shape [24], manual annotation and local image features [25] can be incorporated in the GC framework to allow more robust segmentation.

In this paper, we propose to segment cervical cytoplasm globally and segment cervical nuclei locally based on GC approaches, respectively. The overall method, the outline of which is shown in Fig. 2, follows two tracks. The first track applies cytoplasm segmentation method to separate the cytoplasm from the background (upper part in Fig. 2). The second track applies nucleus binarization and touching-nuclei splitting methods to segment nuclei (lower part in Fig. 2). Specific contributions of the presented work consist of: (1) cytoplasm segmentation by using the multi-way GC [22] on the a* channel (CIE L*a*b* model in Ref. [26]) enhanced images (Section 2.1); (2) nucleus segmentation using a proposed “Local Adaptive GC” (LAGC) method which results in robust binarization of nuclei with a variety of morphologies, chromatin distributions and low contrasts (Section 2.2); (3) a splitting method for touching-nuclei by combining two concave points-based methods [27], [28] (Section 2.3).