Generating CT images in delayed PET scans using a multi-resolution registration convolutional neural network

Highlight

• We proposed a novel registration network to generate DL-based delayed CT images.

Abstract

A Delayed scan refers to the reacquisition of CT and PET after a regular scan to improve the sensitivity and specificity of PET/CT examination. However, an additional CT scan will lead much more X-ray radiation to the patient. Therefore, developing a method to generate delayed CT (T2CT) images to avoid additional CT scans is particularly important in clinic. This paper aims to generate T2CT images from delayed PET (T2PET), regular PET (T1PET) and regular CT (T1CT) images using deep learning methods. However, it may encounter difficulties such as intrinsic differences between multi-modal images and large deformations caused by two scans. To address these issues, a multi-resolution registration convolutional neural network (MRR-CNN) is introduced to improve the accuracy of generating CT images. MRR-CNN employs three models to separately predict deformation vector field (DVF) in different resolution levels. In this method, the global deformation is evaluated firstly, and then local deformations are gradually fused to generate accurate T2CT images. We selected a recently published deep learning-based method (VoxelMorph) to compare the effectiveness of our method on 10 clinical patient data, using mean absolute error (MAE) and root mean square error (RMSE) as evaluation metrics. Compared with VoxelMorph, the proposed MRR-CNN achieves lower MAE (61.26 vs. 67.24) and lower RMSE (118.74 vs. 126.13). The experimental results indicate that our proposed method outperforms VoxelMorph in generating T2CT images.

Introduction

The positron emission tomography/computed tomography (PET/CT) system provides crucial information for radiation treatment, making critical decisions when diagnosing tumour, prognosis and staging [1]. A delayed PET/CT scan refers to scanning the selected bed positions again within a certain period of time after the regular PET/CT scan to make a more accurate diagnosis of the disease [2]. At the same time, precise PET images demand the anatomical information provided by CT images, but this will increase the overall X-ray radiation to the patient [3]. Therefore, finding a method to generate delayed CT (T2CT) to avoid additional CT scans is particularly important.

The development of deep learning (DL) has shown tremendous potential in the field of medical imaging processing and its application [4]. Various DL techniques such as segmentation and reconstruction are being used to build computer-aided diagnosis and treatment systems [5], [6]. Recently, several convolutional neural network (CNN) models have shown great results in the field of image registration [7], [8], [9], [10], [11], [12], [13], [14]. For generating CT, these methods can not only relieve the burden of experts but also decrease the exposure time of patients to X-ray radiation, especially for those people who need repeated scans [15], [16]. However, there are still some problems to be resolved.

Firstly, the grayscale differences of images from different modalities are too large, which leads to the poor performance of traditional CNN models when dealing with multi-modal image registration. Secondly, many mainstream methods cannot effectively deal with the deformation of abdominal organs affected by the human body's digestive state and breathing motion. Finally, the performance of DL-based models is highly correlated with the quantity and quality of training dataset. However, it’s a great challenge to collect a large number of paired PET/CT images with high-quality. Therefore, these existing problems motivate us to conduct further research in this area.

In this paper, a multi-resolution registration convolutional neural network (MRR-CNN) is proposed to alleviate these problems. MRR-CNN combined the information of PET and CT in the encoder to learn the features of multi-model images. We also used a mixed loss function including PET and CT to further improve the adaptation performance of the model. Moreover, the proposed multi-resolution structure enables the proposed model to learn large, medium and small deformation of images of different resolutions to predict deformation vector field (DVF) more precisely than the traditional CNN architecture networks. To address the lack of datasets, we proposed a DVF generation strategy for generating new training data.

The overall contributions of this paper are listed as follows:

• We proposed a novel registration network to generate DL-based delayed CT (DL-T2CT) images by taking full advantage of existing delayed PET (T2PET), regular PET (T1PET) and regular CT (T1CT) images.

• In order to accurately generate DL-T2CT images in delayed scans, the framework of proposed model is designed as MRR-CNN to obtain DVF with multi-scale deformations.

• Group normalization (GN) is introduced instead of batch normalization (BN) to reduce the large bias when given a small batch size [17]. GN divides the channels into groups and normalizes within each group to keep its accuracy be stable even for small batch sizes.

• A module for simulating delayed scans generation is introduced for training data augmentation.

The rest parts of this paper is organized as follows: Section 2 discusses the related work. Section 3 presents the details of how we built the MRR-CNN. In Section 4, our proposed method is extensively evaluated and provided a performance comparison with other DL-based method. A final conclusion and the potential of the proposed method are presented in Section 5.