Segmentation of leukocyte by semantic segmentation model: A deep learning approach

https://doi.org/10.1016/j.bspc.2020.102385Get rights and content

Highlights

  • DeepLab architecture is used to segment leukocytes from microscopic blood images.

  • It has a higher mean accuracy of 96.1% and a mean intersectionover-union of 92.1%.

  • The suggested algorithm modeled segmentation as binary classification.

  • It attains good accuracy with limited data sets available in biomedical applications.

Abstract

In diagnostic research, analysis of blood micrographs has emerged as one of the relevant techniques for identifying various blood-related diseases. Analysis of white blood cells using computer-aided techniques aids the pathologist to promote accurate diagnosis and early detection of blood diseases. An automated white blood cell analysis system involves cell segmentation, feature extraction, and classification, and its performance depends upon the accuracy of cell segmentation. Accurate and automatic segmentation of leukocyte remains a difficult task because of the complex nature of cell images, staining techniques, and imaging conditions. Here, we employ a semantic segmentation technique that uses a deep learning network to segment leukocyte from microscopic blood images accurately. The proposed model uses DeepLabv3+ architecture with ResNet-50 as a feature extractor network. The experiments have been carried out on three different public datasets consisting of five categories of white blood cells, and 10-fold cross-validation is performed to assert the model's effectiveness. The average segmentation accuracy achieved throughout the suggested network is 96.1% and 92.1% intersection- over-union, which is more than different approaches to supervised learning. Experimental results reveal that the suggested model performs better than other techniques and is appropriate for hematological analysis.

Introduction

Traditionally, in the laboratory, blood microscopy is used to diagnose various blood cell disorders and its accuracy depends on the pathologist's/hematologist's skill. Blood cell detection from micrographs using computer-aided techniques helps medical practitioners to detect blood diseases such as blood cancer, anemia, and malaria. Blood is mainly composed of white blood cells (WBCs) or leukocytes, red blood cells (RBCs), and platelets. Leukocytes are part of the immune system and are composed of granulocytes, lymphocytes, and monocytes. Granulocytes are further divided into neutrophils, basophils, and eosinophils. The type and count of leukocytes are the two important parameters that detect white blood cell disorders such as leukemia, lymphoma, etc. Counting of WBC by traditional methods are time-consuming and require medical experts. Hence, an automated white blood cell analysis system that involves cell segmentation, feature extraction, and classification is needed. White blood cell segmentation intends to extract leukocyte from blood smear images which are used for finding the necessary features that are essential for further processing. The reliability of the system depends upon the accuracy of segmentation.

From an image processing perspective, segmentation is the technique of separating a group of pixels from an image called objects. Segmentation techniques are mainly categorized into two types: region-based and boundary-based techniques. Region-based techniques divide the image into regions having similar properties, whereas boundary-based techniques divide an image depending upon the sudden intensity changes at edges. We have used a region-based segmentation technique to separate WBC from the background.

Segmentation of leukocyte is a tedious work due to the wide variation in cell shape and imaging environment. As an example, Fig. 1 shows the various types of WBC along with their ground truth images. According to literature, a few automated systems are available that can analyze leukocytes from blood smear images. Researchers are still working on building a system that can segment leukocyte automatically with a hundred percent accuracy in minimal time. Some papers use algorithms that segment WBC alone, whereas some researchers publish algorithms to segment nucleus and cytoplasm [1]. The models that segment WBC can be categorized mainly into two groups: unsupervised learning model and supervised learning model. The unsupervised techniques include thresholding [2], [3], clustering-based technique [4], [5], [6], edge-based [7], region-based [8] and fuzzy set-based models [9], [10], [11], [12], [13]. Soltanian-Zadeh et al. developed an active contour-based model for the segmentation of WBC [14], but it requires good contour initialization to get better results. Cao et al. developed an algorithm that uses the stepwise averaging method for nucleus segmentation and interval-valued fuzzy sets for cytoplasm segmentation from blood smear images [15]. All the supervised methods consider segmentation as a multi-class classification task where each pixel is categorized into a region of interest and non-region of interest. Saidi et al. developed a pixel-based classification for the segmentation of the nucleus and cytoplasm using random forest classifier [16] and achieved an accuracy of more than 95%. Zheng et al. described a segmentation method that uses a combination of unsupervised initial segmentation with supervised segmentation refinement [17] and achieves a low error rate.

In addition to all of the above traditional methods, deep learning networks were also used in the segmentation of objects in biomedical images to improve accuracy [18], [19]. Deep learning has attained interest in the segmentation of medical images because of its self-learning and generalization ability [20]. Deep learning-based models demand significant memory and computational time for training and testing the data. The use of GPU and GPU computing libraries make the system trained several times faster than on CPUs [20]. Segmentation using deep learning is mainly categorized as instance segmentation and semantic segmentation. Instance segmentation uses different masks for each region of interest, whereas semantic segmentation carries out pixel-based classification using a single mask for the entire image [21]. Instance segmentation can be done by using an extra object detection task that separates each class in an image. For example, instead of classifying four cells as one instance, it will identify the individual cell. Our research focuses on white blood cell segmentation using a convolutional neural network-based semantic segmentation model. The various types of convolutional neural networks devised for semantic image segmentation include fully convolutional networks (FCN), SegNet, U-Net, and DeepLab [22]. The FCN supports an input image of any size and it uses skip architecture to produce the final output [23], whereas U-Net [24] and SegNet [25] uses Encoder-Decoder architecture. DeepLab uses a powerful FCN architecture composed of three components viz. atrous convolution, atrous spatial pyramid pooling, and fully connected conditional random fields [26]. The introduction of the attention mechanism for scene segmentation in deep learning has improved the success of various models in recent years. This includes a dual attention network that uses the self-attention mechanism to adapt local semantic features [27] and attention complementary network that uses depth information to achieve better segmentation performance [28]. However, such models are not necessary for WBC segmentation as they are used for scene segmentation, which consists of many artifacts.

The major contribution of our research is to segment leukocytes from microscopic blood images with different staining techniques and imaging conditions using a deep learning approach that uses the DeepLabv3+ network based on ResNet-50. The proposed approach yields better results compared to existing supervised and unsupervised techniques in WBC segmentation. The remaining part of this article is structured in the following manner. Section 2 describes the methods. The result is described in Section 3, the discussion is given in Section 4, and the conclusion is presented in Section 5.

Section snippets

Methods

The proposed work uses semantic segmentation using DeepLabv3+ network for the segmentation of white blood cells from microscopic blood images. It is similar to a binary classification task where the pixels in WBC correspond to positive classes and pixels in the background corresponds to negative classes. Fig. 2 reflects the pipeline of the suggested work. At first, the datasets are resized to 224 × 224 to make it compatible with the input size of the pre-trained network called ResNet. In

Experimental evaluation

Experiments are done using MATLAB R2019b on a PC having an Intel CORE i5 processor running at 1.85 GHz, with 16 GB RAM and Intel UHD Graphics 620 GPU. Within each dataset, the entire image is divided into ten groups. We have used one set as test images and the other nine set as training data and tested different parameters to get the best outcomes. The proposed model performance is assessed using cross-validation.

Quantitative assessment

In our proposed method, WBC corresponds to one class and background corresponds to another class. As the size of the cell varies, one class dominates the image and creates a class imbalance issue. This means that our classes are imbalanced; hence in addition to accuracy, we have used IoU and BF score to show how well our proposed system works. The functioning of the suggested model with an unknown dataset is evaluated and the results obtained are given in Table 1.

Comparison with existing works

Our approach is compared to

Conclusion

This paper presents a deep learning-based technique for the segmentation of leukocyte from the microscopic blood images. Segmentation of WBC is difficult due to the intensity variation and imaging condition. Our approach is based on semantic segmentation using the DeepLabv3+ network based on ResNet 50. The proposed system performance is compared with the existing systems, both supervised and unsupervised, and shown to be better in terms of overall error rate. The experimental outcome shows that

Declaration of Competing Interest

The authors report no declarations of interest.

Acknowledgment

The authors are grateful to Jiangxi Tecom Science Corporation in China and CellaVision Company in Sweden for availing their datasets of images.

References (52)

  • F. Qin et al.

    Fine-grained leukocyte classification with deep residual learning for microscopic images

    Comput. Methods Programs Biomed.

    (2018)
  • D. McNeely-White et al.

    Inception and resnet features are (almost) equivalent

    Cogn. Syst. Res.

    (2020)
  • X. Zheng et al.

    A novel algorithm based on visual saliency attention for localization and segmentation in rapidly-stained leukocyte images

    Micron

    (2014)
  • C. Pan et al.

    Leukocyte image segmentation using simulated visual attention

    Expert Syst. Appl.

    (2012)
  • B.C. Ko et al.

    Automatic white blood cell segmentation using stepwise merging rules and gradient vector flow snake

    Micron

    (2011)
  • L. Yang et al.

    Unsupervised segmentation based on robust estimation and color active contour models

    IEEE Trans. Inform. Technol. Biomed.

    (2005)
  • J. Prinyakupt et al.

    Segmentation of white blood cells and comparison of cell morphology by linear and naïve bayes classifiers

    Biomed. Eng. Online

    (2015)
  • S. Mohapatra et al.

    Blood microscopic image segmentation using rough sets

    2011 International Conference on Image Information Processing

    (2011)
  • N.M. Salem

    Segmentation of white blood cells from microscopic images using k-means clustering

    2014 31st National Radio Science Conference (NRSC)

    (2014)
  • Z. Liu et al.

    Segmentation of white blood cells through nucleus mark watershed operations and mean shift clustering

    sensors

    (2015)
  • F. Sholeh

    White blood cell segmentation for fresh blood smear images

    2013 International Conference on Advanced Computer Science and Information Systems (ICACSIS)

    (2013)
  • S. Arslan et al.

    A color and shape based algorithm for segmentation of white blood cells in peripheral blood and bone marrow images

    Cytometry Part A

    (2014)
  • M. Ghosh et al.

    Yager’s measure based fuzzy divergence for microscopic color image segmentation

    2013 Indian Conference on Medical Informatics and Telemedicine (ICMIT)

    (2013)
  • H. Danyali et al.

    Robust leukocyte segmentation in blood microscopic images based on intuitionistic fuzzy divergence

    2015 22nd Iranian Conference on Biomedical Engineering (ICBME)

    (2015)
  • M. Saidi et al.

    Application of pixel selection in pixel-based classification for automatic white blood cell segmentation

    Proceedings of the Mediterranean Conference on Pattern Recognition and Artificial Intelligence

    (2016)
  • H. Yang et al.

    Clci-net: Cross-level fusion and context inference networks for lesion segmentation of chronic stroke

    International Conference on Medical Image Computing and Computer-Assisted Intervention

    (2019)
  • Cited by (42)

    • A combination of simple and dilated convolution with attention mechanism in a feature pyramid network to segment leukocytes from blood smear images

      2023, Biomedical Signal Processing and Control
      Citation Excerpt :

      An average IoU score of above 90% was obtained to segregate WBCs from the rest of the background. In the same year, Roy et al. [48] used a combination of DeepLabV3+, which applies depthwise separable convolutions and a pre-trained ResNet50 model to segment WBCs from blood smear images obtained from three datasets. Similarly, under deformable methods, active contour models [36–38] and level-set methods [39,40] have been used for precise segmentation based on prior knowledge of the curves and topology of cells.

    • Semantic segmentation of low magnification effusion cytology images: A semi-supervised approach

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      The time it takes for cytopathologists to screen the entire slide for malignancies is reduced by automation. Typically automation algorithms use only high magnification images [1–7]. On the other hand, cytopathologists take multiple magnifications of cytology images into account when predicting malignancy.

    View all citing articles on Scopus
    View full text