A dense connection encoding–decoding convolutional neural network structure for semantic segmentation of thymoma

Highlights

• Three-channel pseudo-color images preprocessing method is designed by concatenating different CT windows.

• A dense skip connection encoding–decoding model (DSC-Net) is proposed to perform automatic segmentation of thymoma base on a deep convolutional neural network.

• Dense connections are introduced into the architecture of the encoding path across different level feature maps in the DSC-Net.

• Different level skip connections are designed between the encoding and decoding path in the DSC-Net.

Abstract

Accurately positioning and segmenting thymoma from computed tomography (CT) images is of great importance for an image-driven thymoma analysis. In clinical practice, the diagnosis and segmentation of thymomas for radiologists are time-consuming and inefficient tasks. Thus, it is necessary to develop a method to accurately and efficiently realize automatic segmentation of thymoma. Here, a dense skip connection encoding–decoding model (DSC-Net), which is a deep convolutional neural network, was proposed to perform automatic segmentation of thymoma with the ability to fuse feature maps under receptive fields of different scales. An image preprocessing method was also proposed to provide much more texture information and enhance the contrast between thymoma and its surrounding tissues. A total of 310 subjects who underwent contrast-enhanced CT scanning were included in this ethically-approved retrospective study. All of the CT slices were manually labeled by four experienced radiologists, and 80% of images were included in the training set and the rest were included in the testing set. The performance of segmentation was evaluated by calculating the accuracy, intersection over union (IoU), and Boundary F1 contour matching score (BFScore) between the predicted segmentation and the manual labels. For segmentation of thymoma in the testing set, the accuracy, IoU and BFScore were 92.96%, 87.86% and 0.9087 respectively. Compared to the U-Net method, the DSC-Net model improved IoU by 3.94%. In addition, the efficacy and robustness of DSC-Net in segmentation of different patients and different types of thymoma classified by the WHO histological classification criteria were verified. The proposed preprocessing method and DSC-Net demonstrated improved performance in segmentation of thymomas, suggesting the ability to provide consistent delineation and assist radiologists in their clinical applications.

Introduction

Mediastinal tumors are common clinical chest diseases, of which thymoma is the most common primary anterior mediastinal tumor, accounting for 20% to 40% of all mediastinal tumors in adults [1]. However, thymoma has not been thoroughly studied [1], [2]. Whether the cancerous part can be accurately identified and segmented during the diagnostic process is of great significance for doctors to determine the subsequent treatment strategy [3]. Current technologies often require manual inspections and segmentation by radiologists, which are time-consuming, labor intensive, and prone to error. Therefore, it is urgent to realize the automatic segmentation and diagnosis of thymoma. Computer-aided diagnosis, particularly with the artificial intelligence technique using deep learning, is increasingly being employed in clinical diagnosis [4], [5], [6].

Automatic segmentation methods based on medical images mainly include morphological methods, statistical methods and convolutional neural network (CNN)-based methods. Tang et al. [7] proposed an image segmentation algorithm by improving the traditional threshold method. Hui-Yan et al. [8] combined the regional growth algorithm and OSTU algorithm to effectively segment abdominal MRI images with fuzzy edges. Joan et al. [9] used an iterative regional growth algorithm to effectively segment the chromatin in ovarian cells. Angelina et al. [10] proposed a new region growing and merging algorithm of the medical image segmentation algorithm combined with the genetic algorithm. At the same time, image segmentation methods based on fuzzy clustering and their improved algorithms have been widely used in the medical field, such as the automatic intuitive fuzzy clustering method (FCM) for brain MRI image segmentation [11], and the FCM clustering method with bilateral filtering for medical image segmentation [12].

In recent years, with the improvement of the computing power and the increase in the amount of medical imaging data, the development of automatic segmentation method based on CNN has become a hot research field. U-Net [13] was proposed for the cell segmentation task of electron microscopy images. The structures of encoding–decoding path and skip connection between the encoding and decoding paths have excellent performance in the segmentation of medical images. Thus, U-Net based image segmentation has been further applied to the segmentation of various medical images such as retinal blood vessels [14], lung parenchyma [15] and pancreas [16]. One of the research contents is the improvement of the encoding–decoding structure. One study replaced convolutional blocks in U-Net with a ResNet block to perform segmentation of liver tumors [17]. Li et al. proposed H-Dense U-Net [18], in which dense blocks were used to replace the convolution operation in the encoding path. It solves the difficulty of calculation by training a network with a deep structure. The network can retain low-level spatial features to better explore the contextual information between slices. Another research direction is the skip connections, U-Net++ [19], which replaces skip connections with a series of long and short connections, captures features of different levels, and integrates them through feature concatenation. MultiResUNet [20] added the residual blocks into the encoding path, decoding path and skip connection respectively, and replaced skip connections with respaths, the features of the encoder that perform additional convolution operations before concatenating with the corresponding features in the decoder. These studies aimed to retain the effective features to the greatest extent in the down-sampling process and locate the active area as accurately as possible in the up-sampling process. Among them, residual modules and dense connections play important roles. However, adding residual connections only to the encoding path would not compensate for the lost information during upsampling, and adding dense connections only to skip connections would not retain low-level features during downsampling. Therefore, it is important to retain and recover effective features in both processes of subsampling and upsampling.

Thus, in order to realize fast and accurate segmentation of thymoma, we proposed a method of dense feature connections, designed an encoding–decoding structure based on the U-Net model, and applied the dense feature connection method to both the encoding path and decoding path. Dense connections were introduced into the architecture of the encoding path across different level feature maps, which could compensate for the loss during the down-sampling process. Meanwhile, different level skip connections were designed between the encoding and decoding paths to introduce more abundant feature information from the encoding path to the decoding path by building more effective connections.

In addition, unlike 8-bit natural images, medical images, including CT images, usually have 10–12 bit depth. Direct mapping of the medical images to 8-bit images would result in information loss; thus, we proposed an image preprocessing method. Using different CT windows, three-channel pseudo-color images were designed to provide more texture information and enhance the contrast between thymoma and its surrounding tissues.