Deeply supervised full convolution network for HEp-2 specimen image segmentation

Abstract

Human Epithelial-2 (HEp-2) cell images play an important role for the detection of antinuclear autoantibodies (ANA) in autoimmune diseases. Segmentation is the primary step for classification, further treatment and diagnosis. However, the staining patterns and scales of HEp-2 specimen images have different variances, which still make segmentation quite a challenging task. To solve it, we propose a novel deeply supervised full convolutional network (DSFCN) for robust segmentation of different HEp-2 cell images dataset. DSFCN is based on a very deep network, which integrates the dense deconvolution layer (DDL) and hierarchical supervision structure (HS). Specifically, The DDL uses the up-sampling to restore the high resolution of the original input images to replace the traditional deconvolution layer, and the hierarchical supervision is added to capture feature information of the shallow layers. The high-resolution predictive output is obtained by capturing local and global information between layers. Without relying on the prior knowledge and complex post-processing, DSFCN can be effectively trained in an end-to-end manner. The proposed method is trained and tested on the I3A-2014 public dataset, and the segmentation result demonstrates that the performance of our model outperforms other state-of-the-art methods.

Introduction

HEp-2 cells with indirect immunofluorescence (IIF) is a commonly-used technique for detecting anti-nuclear anti-bodies (ANA), which can be visualized via a fluorescence microscope. Segmenting HEp-2 specimen images is indispensable due to its importance in daily clinical practice to improve the efficiency of computer-aided diagnosis and detection. However, manual analysis from a large number of IIF images still has the limitations (e.g., high clinical experiences, time-consuming and inter-variability among doctors’ knowledge). As a result, the subjective results and inter-laboratories diversity restrict the true expression of the reading results [1]. To address these limitations, a number of automatic and robust HEp-2 cell classification models have been proposed in recent years [2], [3], [4]. In these methods, segmentation is the first step for HEp-2 cell images classification since the accurate segmentation results are beneficial for the subsequent classification processing [3], [4].

Intensity thresholding is one of the most popular and preliminary approaches for cell segmentation. Petra et al. proposed a HEp-2 cell image segmentation method using Otsu by utilizing the first thresholding [5]. Jiang et al. proposed a novel approach based on the framework of verification-based multi-threshold probing for HEp-2 cell image segmentation [6]. Many studies aimed at the segmentation of HEp-2 cells [7], [8] due to the large variances of appearances among different HEp-2 cell categories. However, most of the previous works achieved accurate segmentation for images containing a certain pattern of cells while failing to achieve good results when different staining patterns were provided. Examples of the staining patterns of HEp-2 specimen images are illustrated in Fig. 1. However, there is still room for robustness improvement of the HEp-2 specimen images segmentation method.

To improve the segmentation performance of HEp-2 specimen images, a method with an impressive feature is highly desirable. In recent years, the deep convolutional neural networks (CNNs) have attracted wide attention due to their impressive performance in various image processing tasks [9], [10], [11]. The fully convolutional network (FCN) extends the traditional CNN, which is one of the most representative models [12], [13] for segmentation. The main idea under FCN model is to apply the classification networks (AlexNet [14], VGG net [15], GoogLeNet [16], and ResNet [17]) to the segmentation task by transforming the last classifier layers to the deconvolutional layers. In fact, the deeper level of network layer information and fusion can further improve the segmentation performance. Hence, the full convolution residual network (FCRN) with a deeper residual network (ResNet) was proposed [18]. However, when the fully convolutional connection is applied to the fully convolutional layer in the deep network, the resolution of the feature map of the output layer will be reduced. This results in loss of information which is highly undesirable for the segmentation of medical images.

In order to tackle this problem, a lightweight neural network called U-Net was proposed [19]. The U-Net architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. However, the network is inefficient to capture the edge information for some HEp-2 specimen patterns. To better express the image information, Isola et al. explored generative adversarial networks (GANs) in the conditional setting and proposed pix2pix network framework based on U-Net [20]. This architecture makes conditional GANs suitable for image-to-image translation tasks, where an input image is fed into the network and a corresponding output image is generated. With the adversarial learning, the network can learn rich edge information. Nevertheless, the feature information is easily decreased in the processing of skip connection. To overcome this limitation, the Dense Deconvolutional Layer (DDL) structure is fetched in this paper. This idea has been proposed in recent years and achieved considerable segmentation performance [21], [22], [23], [24], [25].

The DDL consists of a series of skip connection layers between the previous and later layer instead of performing a summation operation. This architecture also resolves the gradient vanishing problem effectively. Inspired by the previous works, we propose a novel end-to-end Deep Supervised Fully Convolutional Network (DSFCN), which utilizes DDLs without requiring prior knowledge and post-processing. The improved loss functions are introduced in the two lateral output layers to optimize the output feature maps so that the hierarchical supervision (HS) depth is fully exploited. Our proposed DSFCN framework is able to learn rich hierarchical features and captures the local and global contextual information effectively. Due to the perfect performance, the proposed approach can be regarded as a general technology for image segmentation. In summary, the main contributions of this paper are three-fold:

• We propose a novel end-to-end deep learning framework based on FCN. Due to the DDL structure, the network can learn rich boundary information for HEp-2 specimen images.

• An improved HS mechanism is added to the network, which can optimize the output feature maps.

• Experimental results demonstrate that the proposed method achieves the state-of-the-art segmentation performance on the I3A-2014 dataset.

The rest of this paper is organized as follows. The related work is presented in Section 2. Section 3 introduces the proposed network framework in detail. The experiment settings and comparison results are illustrated in Sections 4 and 5. Sections 6 and 7 are dedicated to discussions and conclusions, respectively.