Segmentation of leukocyte by semantic segmentation model: A deep learning approach
Introduction
Traditionally, in the laboratory, blood microscopy is used to diagnose various blood cell disorders and its accuracy depends on the pathologist's/hematologist's skill. Blood cell detection from micrographs using computer-aided techniques helps medical practitioners to detect blood diseases such as blood cancer, anemia, and malaria. Blood is mainly composed of white blood cells (WBCs) or leukocytes, red blood cells (RBCs), and platelets. Leukocytes are part of the immune system and are composed of granulocytes, lymphocytes, and monocytes. Granulocytes are further divided into neutrophils, basophils, and eosinophils. The type and count of leukocytes are the two important parameters that detect white blood cell disorders such as leukemia, lymphoma, etc. Counting of WBC by traditional methods are time-consuming and require medical experts. Hence, an automated white blood cell analysis system that involves cell segmentation, feature extraction, and classification is needed. White blood cell segmentation intends to extract leukocyte from blood smear images which are used for finding the necessary features that are essential for further processing. The reliability of the system depends upon the accuracy of segmentation.
From an image processing perspective, segmentation is the technique of separating a group of pixels from an image called objects. Segmentation techniques are mainly categorized into two types: region-based and boundary-based techniques. Region-based techniques divide the image into regions having similar properties, whereas boundary-based techniques divide an image depending upon the sudden intensity changes at edges. We have used a region-based segmentation technique to separate WBC from the background.
Segmentation of leukocyte is a tedious work due to the wide variation in cell shape and imaging environment. As an example, Fig. 1 shows the various types of WBC along with their ground truth images. According to literature, a few automated systems are available that can analyze leukocytes from blood smear images. Researchers are still working on building a system that can segment leukocyte automatically with a hundred percent accuracy in minimal time. Some papers use algorithms that segment WBC alone, whereas some researchers publish algorithms to segment nucleus and cytoplasm [1]. The models that segment WBC can be categorized mainly into two groups: unsupervised learning model and supervised learning model. The unsupervised techniques include thresholding [2], [3], clustering-based technique [4], [5], [6], edge-based [7], region-based [8] and fuzzy set-based models [9], [10], [11], [12], [13]. Soltanian-Zadeh et al. developed an active contour-based model for the segmentation of WBC [14], but it requires good contour initialization to get better results. Cao et al. developed an algorithm that uses the stepwise averaging method for nucleus segmentation and interval-valued fuzzy sets for cytoplasm segmentation from blood smear images [15]. All the supervised methods consider segmentation as a multi-class classification task where each pixel is categorized into a region of interest and non-region of interest. Saidi et al. developed a pixel-based classification for the segmentation of the nucleus and cytoplasm using random forest classifier [16] and achieved an accuracy of more than 95%. Zheng et al. described a segmentation method that uses a combination of unsupervised initial segmentation with supervised segmentation refinement [17] and achieves a low error rate.
In addition to all of the above traditional methods, deep learning networks were also used in the segmentation of objects in biomedical images to improve accuracy [18], [19]. Deep learning has attained interest in the segmentation of medical images because of its self-learning and generalization ability [20]. Deep learning-based models demand significant memory and computational time for training and testing the data. The use of GPU and GPU computing libraries make the system trained several times faster than on CPUs [20]. Segmentation using deep learning is mainly categorized as instance segmentation and semantic segmentation. Instance segmentation uses different masks for each region of interest, whereas semantic segmentation carries out pixel-based classification using a single mask for the entire image [21]. Instance segmentation can be done by using an extra object detection task that separates each class in an image. For example, instead of classifying four cells as one instance, it will identify the individual cell. Our research focuses on white blood cell segmentation using a convolutional neural network-based semantic segmentation model. The various types of convolutional neural networks devised for semantic image segmentation include fully convolutional networks (FCN), SegNet, U-Net, and DeepLab [22]. The FCN supports an input image of any size and it uses skip architecture to produce the final output [23], whereas U-Net [24] and SegNet [25] uses Encoder-Decoder architecture. DeepLab uses a powerful FCN architecture composed of three components viz. atrous convolution, atrous spatial pyramid pooling, and fully connected conditional random fields [26]. The introduction of the attention mechanism for scene segmentation in deep learning has improved the success of various models in recent years. This includes a dual attention network that uses the self-attention mechanism to adapt local semantic features [27] and attention complementary network that uses depth information to achieve better segmentation performance [28]. However, such models are not necessary for WBC segmentation as they are used for scene segmentation, which consists of many artifacts.
The major contribution of our research is to segment leukocytes from microscopic blood images with different staining techniques and imaging conditions using a deep learning approach that uses the DeepLabv3+ network based on ResNet-50. The proposed approach yields better results compared to existing supervised and unsupervised techniques in WBC segmentation. The remaining part of this article is structured in the following manner. Section 2 describes the methods. The result is described in Section 3, the discussion is given in Section 4, and the conclusion is presented in Section 5.
Section snippets
Methods
The proposed work uses semantic segmentation using DeepLabv3+ network for the segmentation of white blood cells from microscopic blood images. It is similar to a binary classification task where the pixels in WBC correspond to positive classes and pixels in the background corresponds to negative classes. Fig. 2 reflects the pipeline of the suggested work. At first, the datasets are resized to 224 × 224 to make it compatible with the input size of the pre-trained network called ResNet. In
Experimental evaluation
Experiments are done using MATLAB R2019b on a PC having an Intel CORE i5 processor running at 1.85 GHz, with 16 GB RAM and Intel UHD Graphics 620 GPU. Within each dataset, the entire image is divided into ten groups. We have used one set as test images and the other nine set as training data and tested different parameters to get the best outcomes. The proposed model performance is assessed using cross-validation.
Quantitative assessment
In our proposed method, WBC corresponds to one class and background corresponds to another class. As the size of the cell varies, one class dominates the image and creates a class imbalance issue. This means that our classes are imbalanced; hence in addition to accuracy, we have used IoU and BF score to show how well our proposed system works. The functioning of the suggested model with an unknown dataset is evaluated and the results obtained are given in Table 1.
Comparison with existing works
Our approach is compared to
Conclusion
This paper presents a deep learning-based technique for the segmentation of leukocyte from the microscopic blood images. Segmentation of WBC is difficult due to the intensity variation and imaging condition. Our approach is based on semantic segmentation using the DeepLabv3+ network based on ResNet 50. The proposed system performance is compared with the existing systems, both supervised and unsupervised, and shown to be better in terms of overall error rate. The experimental outcome shows that
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgment
The authors are grateful to Jiangxi Tecom Science Corporation in China and CellaVision Company in Sweden for availing their datasets of images.
References (52)
- et al.
Automatic segmentation, counting, size determination and classification of white blood cells
Measurement
(2014) - et al.
Automated leukocyte recognition using fuzzy divergence
Micron
(2010) Accurate segmentation of leukocyte in blood cell images using atanassov's intuitionistic fuzzy and interval type ii fuzzy set theory
Micron
(2014)Fuzzy c means detection of leukemia based on morphological contour segmentation
Proc. Comput. Sci.
(2015)- et al.
Automatic recognition of five types of white blood cells in peripheral blood
Comput. Med. Imaging Graph.
(2011) - et al.
A novel algorithm for segmentation of leukocytes in peripheral blood
Biomed. Signal Process. Control
(2018) - et al.
Fast and robust segmentation of white blood cell images by self-supervised learning
Micron
(2018) - et al.
Rescuenet: an unpaired gan for brain tumor segmentation
Biomed. Signal Process. Control
(2020) - et al.
A review: deep learning for medical image segmentation using multi-modality fusion
Array
(2019) - et al.
Semantic versus instance segmentation in microscopic algae detection
Eng. Appl. Artif. Intell.
(2020)
Fine-grained leukocyte classification with deep residual learning for microscopic images
Comput. Methods Programs Biomed.
Inception and resnet features are (almost) equivalent
Cogn. Syst. Res.
A novel algorithm based on visual saliency attention for localization and segmentation in rapidly-stained leukocyte images
Micron
Leukocyte image segmentation using simulated visual attention
Expert Syst. Appl.
Automatic white blood cell segmentation using stepwise merging rules and gradient vector flow snake
Micron
Unsupervised segmentation based on robust estimation and color active contour models
IEEE Trans. Inform. Technol. Biomed.
Segmentation of white blood cells and comparison of cell morphology by linear and naïve bayes classifiers
Biomed. Eng. Online
Blood microscopic image segmentation using rough sets
2011 International Conference on Image Information Processing
Segmentation of white blood cells from microscopic images using k-means clustering
2014 31st National Radio Science Conference (NRSC)
Segmentation of white blood cells through nucleus mark watershed operations and mean shift clustering
sensors
White blood cell segmentation for fresh blood smear images
2013 International Conference on Advanced Computer Science and Information Systems (ICACSIS)
A color and shape based algorithm for segmentation of white blood cells in peripheral blood and bone marrow images
Cytometry Part A
Yager’s measure based fuzzy divergence for microscopic color image segmentation
2013 Indian Conference on Medical Informatics and Telemedicine (ICMIT)
Robust leukocyte segmentation in blood microscopic images based on intuitionistic fuzzy divergence
2015 22nd Iranian Conference on Biomedical Engineering (ICBME)
Application of pixel selection in pixel-based classification for automatic white blood cell segmentation
Proceedings of the Mediterranean Conference on Pattern Recognition and Artificial Intelligence
Clci-net: Cross-level fusion and context inference networks for lesion segmentation of chronic stroke
International Conference on Medical Image Computing and Computer-Assisted Intervention
Cited by (42)
An indeterminacy fusion of encoder-decoder network based on neutrosophic set for white blood cells segmentation
2024, Expert Systems with ApplicationsBiomass characterization with semantic segmentation models and point cloud analysis for precision viticulture
2024, Computers and Electronics in AgricultureAutomatic myeloblast segmentation in acute myeloid leukemia images based on adversarial feature learning
2024, Computer Methods and Programs in BiomedicineData augmentation guided breast tumor segmentation based on generative adversarial neural networks
2023, Engineering Applications of Artificial IntelligenceA combination of simple and dilated convolution with attention mechanism in a feature pyramid network to segment leukocytes from blood smear images
2023, Biomedical Signal Processing and ControlCitation Excerpt :An average IoU score of above 90% was obtained to segregate WBCs from the rest of the background. In the same year, Roy et al. [48] used a combination of DeepLabV3+, which applies depthwise separable convolutions and a pre-trained ResNet50 model to segment WBCs from blood smear images obtained from three datasets. Similarly, under deformable methods, active contour models [36–38] and level-set methods [39,40] have been used for precise segmentation based on prior knowledge of the curves and topology of cells.
Semantic segmentation of low magnification effusion cytology images: A semi-supervised approach
2022, Computers in Biology and MedicineCitation Excerpt :The time it takes for cytopathologists to screen the entire slide for malignancies is reduced by automation. Typically automation algorithms use only high magnification images [1–7]. On the other hand, cytopathologists take multiple magnifications of cytology images into account when predicting malignancy.