CGBO-Net: Cruciform structure guided and boundary-optimized lymphoma segmentation network

https://doi.org/10.1016/j.compbiomed.2022.106534Get rights and content

Highlights

  • A method to extract cruciform structure from PET images is proposed.

  • A semi-automatic segmentation network using cruciform structure is proposed.

  • A boundary gradient loss function is proposed to optimize boundary.

  • We are the first to incorporate cruciform structure into PET/CT lymphoma segmentation.

Abstract

Lymphoma segmentation plays an important role in the diagnosis and treatment of lymphocytic tumor. Most current existing automatic segmentation methods are difficult to give precise tumor boundary and location. Semi-automatic methods are usually combined with manually added features such as bounding box or points to locate the tumor. Inspired by this, we propose a cruciform structure guided and boundary-optimized lymphoma segmentation network(CGBS-Net). The method uses a cruciform structure extracted based on PET images as an additional input to the network, while using a boundary gradient loss function to optimize the boundary of the tumor. Our method is divided into two main stages: In the first stage, we use the proposed axial context-based cruciform structure extraction (CCE) method to extract the cruciform structures of all tumor slices. In the second stage, we use PET/CT and the corresponding cruciform structure as input in the designed network (CGBO-Net) to extract tumor structure and boundary information. The Dice, Precision, Recall, IOU and RVD are 90.7%, 89.4%, 92.5%, 83.1% and 4.5%, respectively. Validate on the lymphoma dataset and publicly available head and neck data, our proposed approach is better than the other state-of-the-art semi-segmentation methods, which produces promising segmentation results.

Introduction

Lymphoma is a tumor originating from the lymphatic hematopoietic system and one of the most common tumors throughout the body. 18F-fluorodeoxyglucose FDG captured in cells visualized by positron emission tomography can highlight the tumor. The standardized uptake value (SUV) output by PET can reflect the metabolism of different regions of the human body, which is of great value for tumor detection [1]. Because of its low resolution and high signal-to-noise ratio [2], it is usually studied with CT images to better show the anatomical features. Precise segmentation of lymphoma is a key step in the diagnosis and treatment of lymphoma. Because the medical image is noisy, the boundary of lymphoma is blurred and the shape is irregular [1], the method of automatic segmentation of lymphoma is usually difficult to get the satisfied results. Therefore, it is necessary to use user-provided information (such as key points and borders) to guide tumor feature extraction to obtain precise segmentation.

Traditional interaction methods usually require manual creation of features to segment tumors, such as Graph cut [3], GrabCut [4] and Intelligent Scissors [5]. Eric N. Mortensen and William A. Barrett [6] proposed the Intelligent Scissors which is an interactive algorithm for image segmentation and be used for 2D image segmentation. This algorithm can assist users to outline the area of interest and quickly locate objects boundary. These methods are limited by hand-extracted features. As deep learning methods have achieved promise performance, many interactive methods are proposed to fusion with it. For example: Maninis et al. [7] proposed Extreme Cut (DEXTR) to segment objects from four extreme points (left-most, right-most, top and bottom pixels) [8] of the labeled image as the bounding box. Wang et al. [9] used a method similar to GrabCut for image segmentation. The user first drew a bounding box, and the region inside the bounding box was used as the input of the Convolutional Neural Network(CNN) [10] to obtain an initial result, and then the CNN was fine-tuned image-specifically. Wang et al. [11] used two CNNs, in which the first CNN obtained an automatic segmentation result, and the user provided interaction points or short lines to mark the wrongly segmented regions on this basis, and then used it as the input of the second CNN to obtain Corrected result.

The Response Evaluation Criteria in Solid Tumors (RECIST) [12] is often used clinically for the detection of cancer patients, marking the long and vertical diameters of the tumor at a significant level of the tumor (the axial section with the largest tumor area). The commonly used RECIST diameters is marked on the CT image. Because lymphoma is not as obvious on the CT image, the cruciform structure we used was obtained on the PET image and its long and vertical diameters were extracted using hypermetabolic features similar to RECIST, as shown in Fig. 1. Because we are a semi-automatic network, we use the features provided by humans, which in our approach are the cruciform structure features. Use it to indicate the location and size of the tumor.

Many hospitals have a large number of RECIST diameters that can be directly used to guide tumor segmentation [13]. Doctors only draw on the single largest tumor slice, but the segmentation task needs to perform segmentation for each slice. In addition, there is no RECIST diameters in open source PET/CT datasets, only the label of tumor segmentation. In order to obtain the cruciform structure of each slice to assist the segmentation, we first proposed a cruciform structure extraction method based on axial context, which was approved by doctors. In order to leverage PET images, CT images, cruciform structure, we propose the cruciform structure guided and boundary-optimized lymphoma segmentation network(CGBO-Net) from the traditional encoder–decoder structure model, which has three encoders and two decoders. The three encoders include Extra encoder, PET encoder, and CT encoder. The two decoders include the multi-level optimized decoder(MOD), the cruciform structure guided and multi-level optimized decoder(CMD). Our proposed methods include the following steps:

(1) Cruciform structure extraction: we first obtain the cruciform structure from the ground truth of the largest tumor slice. Then we use the cruciform structure and the corresponding PET image to obtain an initial segmentation threshold. Finally, the cruciform structure of the contextual tumor slice is obtained through this threshold.

(2) Lymphoma segmentation network: After obtaining guidance marks, we develop a cruciform structure guided and boundary- optimized lymphoma segmentation network(CGBO-Net) to apply cruciform structure in tumor segmentation. In the encoder architecture, the corresponding three features (Extra feature is called EF, PET feature is called PF, CT feature is called CF) are extracted by taking the cruciform structure gaussian heat map, PET image, and CT image as input respectively. In the decoder architecture, we distinguish the three features extracted from the encoder according to the input of different combinations to the decoder. They are the MOD which takes PF, CF as input and the CMD which takes EF, PF, CF as input. Each decoder is also divided into two channels: pixel segmentation channel and boundary segmentation channel. In the boundary segmentation branch we use the ground truth boundary map and the proposed loss function for supervision. We proposed a new definition method of boundary gradient loss(LBG), and combines pixel cross-entropy loss and fusion boundary cross-entropy loss as the overall loss of segmentation network.

Compared to the previous Semi-automatic methods, our work makes the following contributions.

(1) We propose the cruciform structure and used it as an additional feature to guide lymphoma segmentation. We achieve this by employing the cruciform structure as an additional input-channel of the segmentation network. Compared with the point feature, cruciform structure contains more specific information for lymphoma. To our best knowledge, we are the first to use cruciform structure in lymphoma segmentation.

(2) We develop a cruciform structure guided and boundary- optimized lymphoma segmentation network(CGBO-Net). In our CGBO-Net, The cruciform structure is used to guide tumor segmentation. In order for the network not to rely too much on its characteristics, we set up two kinds of decoders. In the decoder, a boundary gradient loss is devised to enhance the network’s ability to learn lymphoma boundary.

(3) We validate our method on two PET/CT datasets. The cruciform structure of each tumor slice was obtained by our proposed method. Experiments on the two datasets demonstrate the effectiveness and superiority of our proposed method.

Section snippets

Semi-automatic object segmentation methods

Individual medical problems vary greatly from case to case. Compared with the automatic segmentation method, the semi-automatic segmentation method has a wider range of applications. Additional features can be selected in a variety of ways. The characteristics selected take many forms. Region growing [14], Marker-Controlled Watershed Segmentation, and other methods [3], [7], [15] use point as additional features. The limited information that a point can express as a feature. The level set

Methods

The current common fully automatic segmentation methods for lymphoma cannot precisely segment the tumor boundary, so we further process the additional features and use a semi-automatic network for lymphoma segmentation.

Our method mainly consists of two stages: the first stage is the cruciform structure extraction, which first extracts cruciform structure based on the ground truth of the largest tumor slice. We extract all cruciform structure containing tumor slices based on the axial context.

Data and preprocessing

We performed the experiment on two datasets, a lymphoma dataset and a public HECKTOR challenge dataset [26]. Details are as follows:

Our lymphoma dataset contains PET/CT scans of 54 patients (30 male and 24 female) with lymphoma at the General Hospital of Shenyang Military Region. 3 of the 54 patients had performed 18F-FDG PET/CT scan twice. 48 subjects are with histologically confirmed non-Hodgkin’s lymphoma (NHL) and 6 subjects are with histologically confirmed Hodgkin’s lymphoma (HL). We

Discussion

In this study, we propose a new deep learning-based semi-automatic method for lymphoma segmentation. The cruciform structure of all tumor slices are roughly generated by our method and used as additional input to our CGBO-Net network to obtain the corresponding tumor probability maps.

In Section 4, we evaluate by quantitative analysis. As seen in Table 1, GrabCut and Deep GrabCut use bounding box and get lower scores in evaluation metrics when compare to other networks. This can be easily

Conclusion

In conclusion, we propose a deep learning-based semi-automated lymphoma segmentation method that uses cruciform structure as an additional input to the network. In our method, the proposed axial context-based cruciform structure extraction method is used to extract cruciform structure for all tumor sections. We propose a cruciform structure guided and boundary-optimized lymphoma segmentation network. PET/CT and the corresponding cruciform structure are used to extract tumor information, and

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (28)

  • RotherCarsten et al.

    GrabCut: interactive foreground extraction using iterated graph cuts

    ACM Trans. Graph.

    (2004)
  • Eric N. Mortensen, William A. Barrett, Intelligent scissors for image composition, in: Proceedings of the 22nd Annual...
  • K.-K. Maninis, S. Caelles, J. Pont-Tuset, L. Van Gool, Deep Extreme Cut: From Extreme Points to Object Segmentation,...
  • Dim P. Papadopoulos, Jasper R.R. Uijlings, Frank Keller, Vittorio Ferrari, Extreme Clicking for Efficient Object...
  • Cited by (1)

    This work was supported by the National Natural Science Foundation of China (No. 61872075) and Natural Science Foundation of Liaoning Province, China (No. 2021-YGJC-07).

    View full text