Feedback attention network for cardiac magnetic resonance imaging super-resolution
Introduction
Atrial fibrillation (AF) is the most common persistent heart rate disorder caused by a fast and irregular beating of the heart. The odds of developing AF increase significantly with age. Approximately two percent of people under 65 years of age have AF, and nine percent of those 65 years or older have AF [1,2]. The AF predisposes the body to blood clots, which block blood vessels and significantly increase heart failure and strokes, resulting in higher morbidity and mortality. According to the survey, among people over 60 years old, strokes due to AF accounts for one-fifth [3].
Recurrent episodes of AF also lead to further changes in structural features, namely structural remodeling of the atrium (dilation, muscle fiber changes, and fibrosis). Therefore, the direct study of the atrial structure and its changes in patients with AF is crucial for understanding and treating AF [4,5]. To date, gadolinium-based contrast agents are used in one-third of cardiac magnetic resonance imaging (CMRI) scans to enhance the clarity of images of the anatomy of a patient's internal organs by improving the visibility of common disease-related structures (e.g., fibrosis, inflammation), such as tumors and blood vessels [5]. Clinical studies of AF patients have shown that atrial fibrosis's extent and distribution are reliable predictors of catheter ablation success and can be used for patient stratification in medical management. However, experts cannot make an accurate diagnosis due to the low-resolution (LR) of CMRI caused by medical equipment and improper operation. The use of deep learning technology to improve the resolution is of great significance to assist experts in the medical treatment of AF [6].
In recent years, the problem of image super-resolution (SR) reconstruction has received extensive attention from scholars at home and abroad. Currently, the most studied is single-image super-resolution reconstruction (SISR). The task of SISR is to reconstruct corresponding high-resolution (HR) images from degraded LR images, but this is an ill-conditioned inverse problem because one LR image corresponds to multiple HR images, and different methods rarely construct the HR images are also different [7,8].
With the development of deep learning, more and more image SR methods based on deep convolutional neural networks have been proposed. In 2014, Dong et al. [9] introduced depthwise convolution to the field of SISR and proposed a deep neural network with three layers of convolutional neural network (CNN), namely the SRCNN method. Compared with traditional sparse coding methods, SRCNN has more advanced performance, which has triggered the upsurge of deep learning in the field of SR. On this basis, Dong et al. [10] proposed the FSRCNN method. Shi et al. [11] proposed the ESPCN method, which directly inputs the LR image without interpolation and amplification to accelerate the network's performance.
Existing studies have shown that more profound and broader networks can usually bring better performance, but deeper and broader networks often get difficulties in network training [12]. He et al. [13] proposed a deep residual network (ResNet). The proposal of ResNet not only enables the movement of deeper networks but also improves network performance to a certain extent. Therefore, Kim et al. [14] proposed the VDSR method based on previous research scholars, further increasing the depth of the network to twenty layers and introducing global residual learning to ease the difficulty of training so that the network performance has been dramatically improved. Furthermore, Kim et al. [15] proposed the DRCN method, which uses the recursive idea to simplify the network and achieves a similar effect to VDSR. Inspired by VDSR and DRCN, Tai et al. [16] proposed the DRRN method, which increased the network depth and improved performance by combining local and global residuals.
Although the above algorithm achieved good results, it gave the same weight to all channels with different amounts of information in network training. Therefore, Zhang et al. [17] used the channel attention mechanism in the SR field to treat other channels differently and proposed residual channel attention networks (RCAN). However, the RCAN method only performs upsampling once and cannot fully use high-frequency information. Later, Harris et al. [18] proposed the DBPN method based on the back-projection network. In an iterative process, the error feedback mechanism is used to adjust the error, so that the neural network can better learn the mapping relationship between LR and HR.
Section snippets
Residual learning
The classic convolutional or fully connected layers will have problems such as information loss and loss during information transmission. Residual networks solve this problem to some extent, protecting the integrity of the information by directly detouring the input information to the output. The entire network only needs to learn the art of the difference between input and output, simplifying the learning objective and difficulty [19]. The residual block is constructed based on a feedforward
Feedback residual attention network
The network structure of our proposed Feedback Attention Network (FBAN) method is shown in Fig. 3, which consists of a shallow initial feature layer, a recurrent feedback attention module, and a reconstruction module. Where represents the input LR image, represents the original SR image after cyclic the FBAB module reconstruction, and represents the final SR image.
Dataset and training details
The DIV2K dataset [23] is used as a training and validation dataset, a high-quality image dataset containing 800 training images and 100 validation images. The training data is rotated, scaled, and flipped to enhance. In the training dataset, LR images and HR images exist in pairs. To obtain the corresponding LR images in the DIV2K training dataset, the HR images are scaled down using Bicubic interpolation with scaling factors in Matlab R2021b. The test dataset is Set5 [24], Set14 [25],
Conclusion
In this paper, we propose the Feedback Attention Network (FBAN) super-resolution method for the rich texture of CMRI. The FBAN method realizes the multiplexing of network parameters by iterating the feedback attention module many times, improves the utilization of high-frequency information, recognizes the multiple utilization of LR image, and then obtains the final HR image through the sub-pixel convolution layer. To a certain extent, the problem of high-frequency information loss is
Ethical approval
No ethics approval was required.
Data availability statement
The datasets [GENERATED/ANALYZED] for this study can be found in the [Cardiac MRI dataset] [https://digital-heart.org/], [AMRG Cardiac Atlas] [http://www.cardiacatlas.org/studies/amrg-cardiac-atlas/].
Declaration of Competing Interest
The authors declare that they have no conflicts of interest.
Acknowledgment
This work was supported by the National Natural Science Foundation of China (Grant Nos. 71673143 and 18ZDA327).
References (32)
- et al.
Dual U-Net residual networks for cardiac magnetic resonance images super-resolution
Comput. Methods Progr. Biomed.
(2022) - et al.
Residual dense network for medical magnetic resonance images super-resolution
Comput. Methods Progr. Biomed.
(2021) - et al.
Multiple improved residual networks for medical image super-resolution
Future Gen. Comput. Syst
(2021) - et al.
Gradual back-projection residual attention network for magnetic resonance image super-resolution
Comput. Methods Progr. Biomed.
(2021) - et al.
Residual dense network for intensity-guided depth map enhancement
Inf. Sci.
(2019) - et al.
A super-resolution guided network for improving automated thyroid nodule segmentation
Comput. Methods Programs Biomed.
(2022) - et al.
Early rhythm-control therapy in patients with atrial fibrillation
N. Engl. J. Med.
(2020) - et al.
Left atrial remodeling mechanisms associated with atrial fibrillation
Cardiovasc. Eng. Technol.
(2021) - et al.
Compressed sensing dynamic cardiac cine MRI using learned spatiotemporal dictionary
IEEE Trans. Biomed. Eng.
(2014) - et al.
Advances in multimodality cardiovascular imaging in the diagnosis of heart failure with preserved ejection fraction
Front. Cardiovasc. Med.
(2022)
Image super-resolution via sparse representation
IEEE Trans. Image Process.
Learning a deep convolutional network for image super-resolution
Accelerating the super-resolution convolutional neural network
Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network
Deep residual learning for image recognition
Accurate image super-resolution using very deep convolutional networks
Cited by (4)
Learn from orientation prior for radiograph super-resolution: Orientation operator transformer
2024, Computer Methods and Programs in BiomedicinePET Imaging Super-Resolution Using Attention-Enhanced Global Residual Dense Network
2023, 2023 IEEE 3rd International Conference on Computer Systems, ICCS 2023