Dual attention mechanism network for lung cancer images super-resolution
Introduction
The manifestations of early-stage lung cancer are various, the most common ones include small parenchymal nodules and ground glass nodules. Pulmonary nodules vary in size and density, and their pathological types are also different. The increase in diameter increases the probability of malignancy, especially for ground-glass nodules, adenocarcinoma in situ and microinvasive adenocarcinoma often present as ground-glass lesions, and mixed ground-glass nodules often contain invasive adenocarcinoma components. Ground-glass nodules have appeared more and more in recent years, which is also closely related to the popularization of dose spiral Computed Tomography (CT) technology. Ordinary chest X-rays cannot display Ground-glass nodules, so screening thin-slice CT is recommended, and it is the most critical measure to detect early lung cancer [1]. Some external and human factors lead to the low resolution of CT images, and artifacts and noise often accompany the images. In addition, the residual air containing alveoli and dilated lobular terminal bronchioles in the focus of micro lung cancer (diameter < 10 mm) leads to the slight density difference between tumor tissues and adjacent tissues, which brings some difficulties in early detection and early diagnosis. However, deep learning medical image super-resolution reconstruction methods can provide powerful technical assistance for the early diagnosis of pulmonary nodules.
Single-image super-resolution reconstruction (SISR) is a technique for recovering high-resolution (HR) images from low-resolution (LR) images. The HR images are widely used in remote sensing mapping, medical imaging, video surveillance and image generation [2], [3]. Due to current technological development limitations and cost considerations, software processing methods to obtain higher resolution images have become a research hotspot in image processing [4].
Compared with the traditional algorithm, the method based on deep learning has significantly improved performance [5]. Dong et al. [6] first introduced a three-layer convolutional neural network (CNN) in the image super-resolution (SR) task. Kim et al. [7], [8] increased the network depth to 20 layers in VDSR, and DRCN, and significantly improved visual effects and indicators compared to SRCNN. Shi et al. [9] proposed a sub-pixel convolution method that does not require preprocessing LR images and directly serves as the network's input for feature extraction. Furthermore, the feature maps are arranged in the last layer to realize the upsampling operation, which reduces the destruction of LR image context information. With the proposal of the residual network, Lim et al. [10] designed an enhanced deep SR reconstruction network (EDSR), constructed a deeper convolution network by stacking more network layers, and took more features from each layer to reconstruct the image, which significantly improved the network performance. In addition, inspired by the residual dense network, Zhang et al. [11] designed residual dense blocks, which can more effectively extract feature information and improve reconstruction quality through the interconnection and fusion of multiple residual dense blocks [12].
Our main contributions are as follows:
- 1)
Propose the dual attention mechanism network structure to integrate channel and spatial attention effectively. The network can focus on more valuable channels, enhance the discrimination learning ability, and improve the algorithm's accuracy.
- 2)
Design a hybrid attention mechanism that can learn the relationship between the spatial area of a feature map and the channel pixels, distinguish between essential features and unimportant features, and strengthen the Reconstruction of high-frequency information.
- 3)
Design a hybrid loss function, using the L1 loss function and multi-scale structure similarity loss function, which can better maintain the color and brightness of the image during training, and retain high-frequency information such as image edges and texture details.
Section snippets
Residual learning
Research shows [13], [14], [15], [16] that the deeper the neural network, the more sufficient information can be extracted, and the more beneficial it is for subsequent processing. But in practice, it is found that simply increasing the depth will cause the gradient dispersion problem in the network [17].
Although regularization can avoid this problem, it can also lead to network degradation problems. For this reason, He et al. [18] developed a residual network to maintain the stability of the
Methodology
The DAMN network directly reconstructs the HR image from the original LR image, and its basic network structure is shown in Fig. 3. According to the characteristics of pulmonary nodules, the DAMN overall network structure includes three parts, namely the shallow feature extraction module, six residual attention mechanism modules (RAM), and the reconstruction module.
Dataset and training details
Following previous works [13,16,24,25], we use the DIVerse 2 K resolution image dataset (DIV2K) as the training dataset [32]. DIV2K dataset includes 800 training images, 100 verification images, and 100 test images. At the same time, through data enhancement, the training data is mirrored left and right and rotated at different angles (0°, 90°, 180°, 270°) to obtain training data with 8 times the amount of original data.
The test dataset uses the ILSVRC 2014 Imagenet and Kaggle lung cancer
Conclusion
In this paper, we propose a Dual Attention Mechanism Network (DAMN) for single image super-resolution. In DAMN, we designed a dual residual attention mechanism that allows the network to re-check the extracted feature maps, enhance the network's discrimination learning ability, and improve the algorithm's accuracy. Among them, the long and short skip connection effectively enhances the data characteristics and solves the problem of deep network gradient explosion. In addition, a hybrid loss
Ethical approval
No ethics approval was required.
Conflict of interest
The authors declare that they have no conflicts of interest.
Acknowledgement
This work was supported by the National Natural Science Foundation of China (Grant Nos. 71673143 and 18ZDA327), the 2022 Dezhou municipal R & D plan project.
References (38)
- et al.
Residual dense network for medical magnetic resonance images super-resolution
Comput. Methods Programs Biomed.
(2021) - et al.
Multiple improved residual networks for medical image super-resolution
Future Gen. Comput. Syst
(2021) - et al.
gradual back-projection residual attention network for magnetic resonance image super-resolution
Comput. Methods Programs Biomed.
(2021) - et al.
Single image super-resolution based on global dense feature fusion convolutional network
Sensors
(2019) - et al.
Dual U-net residual networks for cardiac magnetic resonance images super-resolution
Comput. Methods Programs Biomed.
(2022) - et al.
Multiple resolution residually connected feature streams for automatic lung tumor segmentation from CT images
IEEE Trans. Med. Image.
(2019) - et al.
Deeply-recursive convolutional network for image super-resolution
Cubic convolution interpolation for digital image processing
IEEE Trans. Acoust. Speech, Signal Process.
(1981)- et al.
Photo realistic single image super-resolution using a generative adversarial network
- et al.
Accelerating the super-resolution convolutional neural network
Enhanced deep residual networks for single image super-resolution
Image super-resolution using very deep residual channel attention networks
Deep residual learning for image recognition
Accurate image super-resolution using very deep convolutional networks
On single image scale-up using sparse-representations
residual dense network for image super-resolution
Enhanced single image super resolution method using lightweight multi-scale channel dense network
Sensors
Sex and smoking status effects on the early detection of early lung cancer in high-risk smokers using an electronic nose
IEEE Trans. Bio. Eng.
Image quality assessment: from error visibility to structural similarity
IEEE Trans. Image Process.
Cited by (2)
Transformaer-based model for lung adenocarcinoma subtypes
2024, Medical PhysicsAccurate segmentation algorithm of acoustic neuroma in the cerebellopontine angle based on ACP-TransUNet
2023, Frontiers in Neuroscience