Super-resolution reconstruction of pneumocystis carinii pneumonia images based on generative confrontation network☆
Introduction
At present, the diagnosis of pneumocystis carinii pneumonia (PCP) is mainly based on the observation of lung changes on HRCT images, including lobular interval, lobular morphology, intralobular structure, pleura, the interface between the lung and the pleura, and the interface between the lung and the bronchi and blood vessels, etc. [1]; presence or absence of consolidation, ground glass shadow, interstitial changes, mosaic appearance, lithotripsy, lung air sacs, mediastinal lymph node enlargement, etc. The common HRCT manifestations of PCP patients are: ground glass shadow, interlobular septal thickening, thickening of interstitial around the bronchial vascular bundle, intralobular interstitial thickening, lung air sacs, mosaic manifestations, lithotripsy signs, consolidation, subpleural space Thickness, enlarged mediastinal lymph nodes, traction bronchiectasis [2]. HRCT scans can show interstitial changes, symmetrical ground glass shadows, new lung air sacs and mosaic manifestations. We should be highly vigilant about PCP to diagnose PCP early and improve prognosis.
Single Image Super-Resolution (SISR) reconstruction as a low-level computer vision processing task is widely used in military, remote sensing, medical, and video surveillance fields. The goal of SISR is to recover a high resolution (HR) image from a single low resolution (LR) image [3]. There are mainly interpolation-based algorithms, reconstruction-based algorithms and learning-based algorithms. The interpolation-based algorithm uses the information of neighboring pixels to estimate the pixel value of the HR image. The calculation is relatively simple, and the super-resolution (SR) of the image can be generated in linear time. However, it does not consider the semantics of the entire image, resulting in the lack of high-frequency detail information of the original image and the inability to achieve image sharpening. Even if the pixels of the image are increased, the visual effect is blurry and there is serious distortion. The reconstruction-based algorithm introduces image priors or constraints between LR images and HR images, and uses sample information to calculate the distribution of real data. Algorithms based on reconstruction include convex set projection method, iterative back projection method and maximum posterior probability estimation method. Due to the constraints of computing resources and prior conditions, this algorithm cannot generate high-quality images.
In recent years, algorithms based on deep learning have appeared one after another. Convolutional Neural Networks (CNN) is applied to SISR reconstruction using the SRCNN method. Subsequently, a variety of CNN-based network architecture designs and training strategies have been proposed. However, these methods tend to output excessively smooth results with missing high-frequency details. In response to this problem, the perceptual loss optimized super-resolution model calculated in the feature space instead of the pixel space can effectively avoid the problems of excessive smooth output and lack of high-frequency details. Introducing Generative Adversarial Networks (GAN) into super-resolution reconstruction tasks can prompt the network to generate more realistic and natural images.
On the basis of the Super-Resolution Generative Adversarial Networks (SRGAN) structure, the residual block (RB) that removes the Batch Normalization (BN) layer is used and an enhanced depth residual is proposed. Poor network, you can get richer high-frequency detailed information [4]. If the Residual Dense Block (RDB) is used as the main body of the generation network, although the image reconstruction effect can be improved, the image artifacts are more [5]. In order to improve the quality of image reconstruction, this paper proposes a texture enhanced super-resolution generative adversarial networks (TESRGAN) construction algorithm. Use the residual dense blocks removed from the batch normalization layer to form a generation network, and use the VGG19 network as the basic framework of the discriminant network [6].
While strengthening forward feature reuse and reducing the number of parameters, the training direction of the generated image is controlled. The texture loss function, perceptual loss function, confrontation loss function, and content loss function are combined to form the objective function of the generator, and reconstruct the image on the Set5, Set14 and BSD100 data sets.
Section snippets
GAN principle
GAN is a new framework for estimating the generative model in deep learning through the confrontation process. Its structure is shown in the Fig. 1. It is mainly composed of a generator G (Generator) and a discriminator D (Discriminator).
The basic idea of GAN is zero-sum game theory. G is used to learn the distribution of training data to generate new sample data that conforms to the real data distribution [7]. D is used to distinguish the true and false of the new sample generated by G, and
Network structure
The TESRGAN algorithm is based on a generative confrontation network, which mainly includes a generative network and a discriminant network. The structure of the TESRGAN algorithm is shown in Fig. 5.
The input of the generation network is the LR image. After convolution to extract the features, the residual model is input for non-linear mapping, and then the image is reconstructed through the upsampling layer and the convolution layer, and the generated HR image is output [11]. Then, weinput it
Experiment
Through image reconstruction experiments, various algorithms and the focus on TESRGAN algorithm are verified. The experimental environment is NVIDIA graphics card GeForceMX150, Intel®CoreTM i7–8550 U CPU @ 2.00 GHz,8 GB RAM, and the compilation software is Pycharm 2017 and MATLAB 2018a [12]. The experiment uses the DIV2K data set to train the network. The data set has 1000 RGB images. Note that 800, 100, and 100 images are taken as the training set, validation set, and test set. The images are
Objective effects
Super-resolution experiments were conducted on the Set5 dataset and the Set14 dataset to compare and analyze the effects of the introduction of RDB structure, texture loss function Ltex, improved combat loss function Ladv, and perceptual loss function Lper on the performance of image super-resolution reconstruction algorithms [14]. Table 2 shows the PSNR of the image super-resolution reconstruction algorithm on the Set5 and Set14 datasets under different conditions. It can be seen that the
Conclusion
The reconstruction results are more in line with the requirements of visual sensory evaluation. In the diagnosis of PCP, it is possible to observe the lobule interval, lobule shape, pleura more clearly and accurately, the interface between the lung and the pleura, and the interface between the lung and the bronchi and blood vessels; It can provide more accurate image support for the presence or absence of ground glass shadows, interstitial changes, mosaic performance, gravel road signs, lung
Declaration of Competing Interest
There are no conflicts of interest to disclose for publication of this paper.
Acknowledgements
This research is supported by Science and Technology Program of Quanzhou (No.2021CT0010). The authors also acknowledge the support by Fujian Provincial Key Laboratory of Data-Intensive Computing, Fujian University Laboratory of Intelligent Computing and Information Processing, and Fujian Provincial Big Data Research Institute of Intelligent Manufacturing.
References (22)
- et al.
Two-phase deep learning model for short-term wind direction forecasting
Renew. Energy
(2021) - et al.
Training very deep networks [EB/OL]
Training Very Deep Networks
(2021) - et al.
J. Appl. Radiol.
(2020) Early diagnosis and early treatment of interstitial pulmonary disease is still very challenging – Visiting Professor Dai Hauling of Department of Respiratory and Critical Medicine in China-Japan Friendship Hospital
Contemp. Chin. Med.
(2019)- et al.
Video prediction based on spatial adaptive convolution LSTM
Comput. Appl. Softw.
(2020) - et al.
Multi-modal medical image fusion based on dual residual superdense network
Comput. Sci.
(2021) - et al.
Q transplants. A superresolution method of CCD image based on wavelet and bicubic interpolation
Appl. Comput. Study Pollinat.
(2009) - et al.
New advances in derivative models and applications of generative adversarial network theory framework
J. Small Microcomput. Syst.
(2018) Diagnostic criteria of diffuse interstitial pulmonary disease in traditional Chinese medicine (2012 edition)
J. Tradition. Chin. Med. Chin. J. Contemp. Med.
(2019)- et al.
MSCT study on the relationship between solitary pulmonary nodules and bronchial and vascular
J. Med. Imaging
(2019)
Alveolar injury in renal transplant recipients with pneumocystis carinii pneumonia
Chin. J. Tubercul. Resp.
Cited by (4)
An Adaptive Offset Activation Function for CNN Image Classification Tasks
2022, Electronics (Switzerland)
- ☆
The authors declare that there is no conflict of interests.
- 1
Authors contributed equally to this paper.