1 Introduction

Corona Viruses (CV) is a family of viruses that initially created infection in birds and mammals. Alpha, beta, gamma, and delta (Paules et al. 2020). CV was identified first in the late 1930s from domestic chickens. First human coronaviruses (HCV) were identified in 1960, causing respiratory infections such as Severe Acute Respiratory Syndrome (SARS) and the Middle East respiratory syndrome (MERS). Later, HCV was identified in humans in Wuhan, China (World Health Organization 2020) in December 2019. It is recognized as SARS and increased rapidly in major cities of China. World Health Organization (WHO) announced it as an epidemic in January 2020 and then declared it a pandemic in March 2020. Nearly half of the world’s population, such as 3.3 billion people affected by HCV in 2020 (World Health Organization 2020). Covid pandemic affected the economic and social well-being of tens of millions of people at the end of 2020. The rapid increase in infection rate and death rate in many countries such as the USA, Brazil, Spain, and India leads to a hazardous situation in human lives. The primary reason behind this disaster is the lack of early diagnosis of covid in humans. The lack of prognosis tools and techniques for HCV leads to high infection rates and death rates around the globe. A high densely populated country like India is affected more due to a lack of prognosis tools for HCV. One standard technique for HCV diagnosis was Reverse Transcription Polymerase Chain Reaction (RT-PCR) (Altan and Karasu 2020). However, it lacks accuracy, sensitivity, and the slow process of diagnosis.

In this digital era, deep learning (DL) techniques play a vital role in health care sectors to diagnose various diseases from medical images. Radiology images are used for the diagnosis of HCV. Chest X-ray (CXR) images are used for prognosis, whereas CT scan images are used for severity analysis in the lungs. Khan et al. (2020) proposed the DL method to diagnose HCV from X-ray images. He used a pre-trained deep neural network called Xception, and the proposed method is trained and tested by benchmark datasets. He claimed an accuracy of 89.6. Sethy et al. (2020) proposed a DL model for HCV diagnosis. Deep learning methods extract in-depth features, and these features are trained by a support vector machine (SVM). The authors claimed an accuracy of 95.38. Alakus and Turkoglu (2020) proposed a diagnosis method by combining a convolutional neural network (CNN) with a recurrent neural network (RNN). The authors claimed an accuracy of 86.66. Yoo et al. (2020) proposed a DL approach for HCV diagnosis based on CXR images with different classifiers and claimed an accuracy of 98%. Sedik et al. (2020) proposed a DL model by combining CNN with long short-term memory (LSTM) for HCV diagnosis. Nour et al. (2020) have developed a method of HCV diagnosis by combining CNN and SVM. CNN extracts in-depth features, and these features are classified by SVM. Jamshidi et al. (2020) proposed various DL approaches such as generative adversarial networks (GAN), extreme learning machines (ELM), and LSTM. Waheed et al. (2020) developed a method based on auxiliary GAN to produce CXR images. Then, the generated CXR images are used to balance the dataset. The authors used the deep learning model VGG16 for classification. Based on these synthetic images, they achieved an accuracy of 95—similar DL-based techniques for diagnosing HCV are tabulated in Table 1.

Table 1 HCV prognosis methods based on CXR
Full size table

All the methods mentioned above focus on deep learning architecture for better diagnosis. In order to attain the highest level of precision and accuracy, there is a demand for image enhancement techniques (Zimmerman et al. 1988). The image enhancement technology improves the visual features of the image without distracting the truth features. Covid CXR images are collected from different countries in different timelines. There is a variation in the orientation and radiation. In order to design the intelligent global system for covid prognosis, there is a demand to make the uniform labeled samples with a reduced intra-class variation. This paper proposes the Delaunay triangulation-based image enhancement method to reduce intra-class variation. Hence, it provides the ideal accuracy in the training and validation phase with reduced converge time.

2 Proposed Delaunay triangulation based covid prognosis system

Delaunay triangulation (DT) represents the surface of the image into triangular regions with common properties. The triangulation is performed based on the region of interest (ROI). A set of points are constructed for CXR images which form the ROI. DT is applied to this set of points. It maximizes the minimal vertex angles of the triangles. Hence it leads to an equiangular triangle to avoid sharp and stretched triangles. It interpolates the pixel values within the region of the enhanced images. The proposed diagnosis system, dtXpert, has three primary stages: data augmentation, deeply fused Delaunay triangulation (DT) of X-ray images, and a deep transfer learning model with an Adam optimizer. Figure 1 shows the framework of dtXpert. All this processing was implemented in MATLAB 2020b with supported toolboxes.

Fig. 1
figure 1

Proposed dtXpert framework

Full size image

2.1 Data augmentation

The proposed system is based on the “Covid-19 Radiography Database” (https://www.kaggle.com/tawsifurrahman/covid19-radiography-database). This database is constructed by a team of researchers from Qatar University, Doha, Qatar, and the University of Dhaka, Bangladesh, Pakistan, and Malaysia with collaborative efforts from clinical experts. It includes 3616 covid-19 CXR images and 1341 normal images. In order to prevent unbalance dataset issues, the 3500 normal CXR images are obtained from the Radiological Society of North America (RSNA) Kaggle database pneumonia dataset. Original X-ray images are available in png and Dicom format and grayscale. Then the images are resized into the size of 299*299. This dataset is divided into different partitions for training and validation. To perform hold-out validation, a set of images is to be kept aside for training. Those unseen data are to be given to the model for perfect validation. Hence, the dataset is divided into 70% (4981 images) for training and 30%(2136 images) for validation for case 1 and 80% (5693 images) for training data, and 20%(1423 images) for validation data for case 2.

2.2 High-resolution image formation from low radiation CXR based on DT

As the proposed project dtXpert is built for global use, it is trained and validated by versatile global datasets. The primary challenge with using multiple datasets is the variation in quality, contrast, intensity, and radiological parameters. Varying radiological settings lead to interclass similarity and intraclass variations. Figure 2 shows the intraclass variation in the CXR images. To achieve better accuracy for multi dataset diagnosis systems, there is a need to reduce the intraclass variation.

Fig. 2
figure 2

Intra-class variation of TB CXR images

Full size image

To reduce the intra-class variation and interclass similarity of low radiation CXR images, the Delaunay triangulation (DT) method (Amidror 2002; Lertrattanapanich and Bose 2002) is employed for the creation of scattered interpolation points based on Region of Interest (ROI). And then, the scattered interpolated points are fused with the original CXR images for the enhanced quality image with a focus on ROI.

2.2.1 Linear triangular interpolation based on Delaunay triangulation

The covid diagnosis in the CXR was decided based on the reference point, as shown in Fig. 3. Scattered points are located in the 2D (x, y) plane, and their values are represented as altitudes \({Z}_{i}\) on the plane. Triangulation brings a piecewise triangular surface over this 2D (x, y) plane, where points are referenced \(({x}_{i},{y}_{i},{z}_{i})\). Those triangular pieces are connected by edges which form a triangular irregular network.

Fig. 3
figure 3

Reference points for CXR

Full size image

A bivariate Linear Interpolation (BLI) scheme is used for each triangle to form piecewise interpolation. Consider the points of the triangle \({P}_{1}=\left({x}_{1},{y}_{1},{z}_{1}\right), {P}_{2}=\left({x}_{2},{y}_{2},{z}_{2}\right)\) and \({P}_{3}=\left({x}_{3},{y}_{3},{z}_{3}\right).\)

Obtain linear equations such as,

$$\begin{aligned}{z}_{1}& =a{x}_{1}=b{y}_{1}+c\\ {z}_{2}& =a{x}_{2}=b{y}_{2}+c \\ {z}_{3}&=a{x}_{3}=b{y}_{3}+c\end{aligned}$$

Unknown values \((a,b,c)\) are computed by solving this linear equation. Then, we can compute \(z\) for any arbitrary point with the coordinate as \(\left(x,y\right)\) within the triangle (Figs. 4, 5).

Fig. 4
figure 4

Interpolated Point P from P1, P2 and P3

Full size image
Fig. 5
figure 5

Mapping of points P1, P2, and P3 from the (x, y) plane into the (u, v) plane

Full size image

For the interpolation of image pixels, the points from the (x, y) plane are to be mapped onto the (u, v) plane. Consider estimating the interpolated point P inside the triangle based on three points, \({P}_{1}, {P}_{2}\), and \({P}_{3}\), in the affine coordinate system (x, y). The affine coordinates of these points are represented as \({X}_{1}=({x}_{1}, {y}_{1})\), \({X}_{2}=({x}_{2}, {y}_{2})\), and \({X}_{3}=({x}_{3}, {y}_{3})\).

$$X={X}_{1}+\left({X}_{2}-{X}_{1}\right){a}_{2}+\left({X}_{3}-{X}_{1}\right){a}_{3}$$

The interpolated point coordinates in the affine system is represented as,

$$\begin{aligned}x & ={x}_{1}+\left({x}_{2}-{x}_{1}\right){a}_{2}+\left({x}_{3}-{x}_{1}\right){a}_{3} \text {and}\\ y& ={y}_{1}+\left({y}_{2}-{y}_{1}\right){a}_{2}+\left({y}_{3}-{y}_{1}\right){a}_{3}\end{aligned}$$

Hence, the matrix form,

$$\begin{aligned}\left(\genfrac{}{}{0pt}{}{x}{y}\right)& =\left(\genfrac{}{}{0pt}{}{{x}_{1}}{{y}_{1}}\right)+\left(\begin{array}{cc}{x}_{2}-{x}_{1}& {x}_{3}-{x}_{1}\\ {y}_{2}-{y}_{1}& {y}_{3}-{y}_{1}\end{array}\right)\left(\genfrac{}{}{0pt}{}{{a}_{2}}{{a}_{3}}\right)\\ \left(\genfrac{}{}{0pt}{}{{a}_{2}}{{a}_{3}}\right) & ={\left(\begin{array}{cc}{x}_{2}-{x}_{1}& {x}_{3}-{x}_{1}\\ {y}_{2}-{y}_{1}& {y}_{3}-{y}_{1}\end{array}\right)}^{-1}\left(\genfrac{}{}{0pt}{}{x-{x}_{1}}{y-{y}_{1}}\right)\end{aligned}$$

Consider (u, v) coordinates mapping of three points \({P}_{1}, {P}_{2}\), and \({P}_{3}\). The coordinates of these points are represented as \({U}_{1}=({x}_{1}, {y}_{1})\), \({U}_{2}=({x}_{2}, {y}_{2})\), and \({U}_{3}=({x}_{3}, {y}_{3})\). By incorporating linear DT interpolation, the point \(X=(x,y)\) is mapped onto the point \(U=(u,v)\)

$$U={U}_{1}+\left({U}_{2}-{U}_{1}\right){a}_{2}+\left({U}_{3}-{U}_{1}\right){a}_{3}$$

The interpolated point coordinates in the affine system is represented as,

$$u={u}_{1}+\left({u}_{2}-{u}_{1}\right){a}_{2}+\left({u}_{3}-{u}_{1}\right){a}_{3}\mathrm{ and }v={v}_{1}+\left({v}_{2}-{v}_{1}\right){a}_{2}+\left({v}_{3}-{v}_{1}\right){a}_{3}$$

Hence, the matrix form,

$$\left(\genfrac{}{}{0pt}{}{u}{v}\right)=\left(\genfrac{}{}{0pt}{}{{u}_{1}}{{v}_{1}}\right)+\left(\begin{array}{cc}{u}_{2}-{u}_{1}& {u}_{3}-{u}_{1}\\ {v}_{2}-{v}_{1}& {v}_{3}-{v}_{1}\end{array}\right)\left(\genfrac{}{}{0pt}{}{{a}_{2}}{{a}_{3}}\right)$$

By substituting (a2, a3) from (1) the new interpolated image pixel is computed in the (u, v) as,

$$\left(\genfrac{}{}{0pt}{}{u}{v}\right)=\left(\genfrac{}{}{0pt}{}{{u}_{1}}{{v}_{1}}\right)+\left(\begin{array}{cc}{u}_{2}-{u}_{1}& {u}_{3}-{u}_{1}\\ {v}_{2}-{v}_{1}& {v}_{3}-{v}_{1}\end{array}\right){\left(\begin{array}{cc}{x}_{2}-{x}_{1}& {x}_{3}-{x}_{1}\\ {y}_{2}-{y}_{1}& {y}_{3}-{y}_{1}\end{array}\right)}^{-1}\left(\genfrac{}{}{0pt}{}{x-{x}_{1}}{y-{y}_{1}}\right)$$

Figure 6 shows the overlay image formulated based on the DT interpolated point from the reference points in Fig. 3.

Fig. 6
figure 6

Overlay image based on DT Interpolated points

Full size image

The deep fused image is shown in Fig. 7. The main aim of DT interpolation is to reduce the intra-class variation from CXR images from different data sources. Figure 8 shows the histogram representation of the fused images with minor intra-class variation compared to Fig. 2.

Fig. 7
figure 7

Deep fused CXR image

Full size image
Fig. 8
figure 8

Intra-class variation of CXR images after DT

Full size image

In order to prove the intra-class similarity and interclass variation, the correlation coefficient is computed between each pair of images based on the formula given,

$$r=\frac{\sum_{m}\sum_{n}({A}_{mn}-\vec{A})({B}_{mn}-\overline{B })}{\sqrt{\sum_{m}\sum_{n}{({A}_{mn}-\overline{A })}^{2}}\sum_{m}\sum_{n}{({B}_{mn}-\overline{B })}^{2}}.$$

A and B are two images with the size m*n. To verify intra-similarity, the correlation coefficient is computed for all possible pairs of covid images and all possible pairs of normal images.

Figure 9 shows the intra-class similarity of covid images with the original dataset and DT interpolated (DTI) dataset. It is shown in the plot that the peak of the DTI covid dataset is 0.8 and the peak of the original covid dataset is 0.48, which implies that the DTI method increases the intra-class similarity in the covid dataset. DT removes another challenging issue with CXR images, such as inter-class similarity. It is also proved based on the correlation coefficient. Figure 10 shows the correlation coefficient between each pair of the original covid –normal dataset and the DTI covid-normal dataset. Even though the correlation is less for both datasets, the DTI dataset yields a much lesser correlation than the original dataset.

Fig. 9
figure 9

Correlation coefficient of DTI covid dataset vs. original covid dataset

Full size image
Fig. 10
figure 10

Correlation coefficient of Covid vs. Normal images

Full size image

Figure 11 depicts that the DTI increases the intra-class similarity and reduces the interclass similarity compared to the original dataset.

Fig. 11
figure 11

Interclass similarity and intraclass variation analysis

Full size image

2.3 Transfer learning for covid diagnosis

The transfer learning (TL) task utilizes the pre-trained model (PTM) with a sufficiently large amount of data and then transfers that knowledge to a relatively complex task with fewer data sources. The ImageNet dataset trains PTM with 1000 categories of images.

The source domain knowledge \(S\) is represented as a triplet,

$$S=\left\{{I}_{s},{G}_{s},{O}_{s}\right\},$$

where \({I}_{s}\) is the Image net dataset, \({G}_{s}\) is the ground truth or label of the dataset, and \({O}_{s}\) is the objective predictive function of PTM.

The target domain knowledge is represented as a triplet for the X-ray images,

$${T}_{PA}=\left\{{I}_{PA},{G}_{PA},{O}_{PA}\right\},$$

\({I}_{PA}\) is the binary class source x-ray images with PA view, \({G}_{PA}\) is the ground truths of the dataset, and \({O}_{PA}\) is the classifiers.

With TL, the classifiers can be defined as,

$${O}_{PA}= \left\{\begin{array}{c}{O}_{PA}\left({I}_{PA},{G}_{PA }|S\right)= {O}_{PA}\left({I}_{PA},{G}_{PA }|{I}_{s},{G}_{s},{O}_{s}\right) ,with Covid \\ {O}_{PA}\left({I}_{PA},{G}_{PA }\right) , without Covid\end{array}\right..$$

TL classifier should reduce the error in the prediction,

$$\mathrm{err}\left[{O}_{PA}\left({I}_{PA},{G}_{PA }|S\right)\left(I\right),G\right]<\left[{O}_{PA}\left({I}_{PA},{G}_{PA }\right)\left(I\right), G\right].$$

This TL is based on the fact that the initial layers of PTM are to be used to extract the low-level features, and the target model is to be retrained only in the last layers to perform the complex prediction task.

The learnable layers are identified and removed in this proposed model, and fully convolutional layers are added for better prediction.

Algorithm 1: Transfer learning (TL) network.

Procedure

Step 1: Install the chosen PTM and store it into the variable \({N}_{0}\) and its number of learnable layers is \({L}^{n}\)

Step 2: Remove the last \({L}^{n}\) number of learnable layers and store it in the variable \({N}_{1}\)

\({N}_{1}= {F}_{r} ({N}_{0}, {L}^{n})\), Where \({F}_{r}\) is the remove layer function, and \({L}^{s}\) is the number of learnable layers.

Step 3: Add two fully connected layers and store them in the variable \({N}_{2}\)

\({N}_{2}= {F}_{A} ({N}_{1}, 2)\), Where \({F}_{A}\) is the function to add fully connected layers.

Step 4: Assign the learning rate as 0 to freeze those layers,

\(\overrightarrow{{l}_{r}}[{N}_{2}(1:L-{L}^{n}]\leftarrow 0\), Where \({l}_{r}\) is the learning rate, and \(L\) is the total number of \({L}^{n}\) learnable layers.

Step 5: Assign learning rate 1 for the newly added two fully connected layers

$$\overrightarrow{{l}_{r}}[{N}_{2}({L}_{2}-1: {L}_{2})]\leftarrow 1$$

3 Results and discussion

This section focuses on the experimental setup for validating the proposed model, followed by a performance analysis. Finally, the proposed dtXpert is compared with other state-of-the-art deep learning models to diagnose covid.

3.1 Experimental setup and evaluation

The proposed transfer learning model is implemented in MATLAB 2020b with GPU NVIDIA and 16 GB RAM Workstation computer. NLM, Belarus, and NIAID datasets are accepted as the benchmark data sets by the researchers. The proposed model is trained and tested by the same dataset. The evaluation of the proposed model is based on two phases training phase and the validation phase. The acceptable tuned hyperparameters train the proposed transfer learning model, and the overall performance is validated. The first standard metric for evaluating the proposed model is confusion matric, including sensitivity, specificity, precision, and accuracy.

3.2 Performance of DTI CXR in transfer learning with shallow neural networks

SqeezeNet is based on two convolutional layers and eight fire modules. It performs max-pooling. It has the advantage that it includes smaller CNNs. Hence, it is easy to deploy with the power-constraint and low-memory devices.

Figure 12 shows the performance of SqeezeNet with the original dataset and DTI dataset. DTI dataset attains an accuracy of 100% and AUC of 100%.

Fig. 12
figure 12

Performance analysis of DTI with SqeenzeNet

Full size image

3.3 Performance of DTI CXR in transfer learning with dense neural networks

Inception-Resnet V2 combines the two recent ideas, such as Residual connections and revised Inception architecture, which are incorporated. Residual connections are needed for training the deep neural architectures. It is an optimum idea to replace the filter concatenation stage of the Inception network with residual connections. Hence, it leads to achieving the advantages of residual architecture by retaining the computational efficiency of Inception architecture. Each of the Inception blocks is suffixed by a filter layer used for scaling up the dimensionality of the filter bank to compensate for the dimensionality reduction of the Inception block.

Figure 13 shows the performance of Inception-ResNet v2 with the original dataset and DTI dataset. DTI dataset attains an accuracy of 100% and AUC of 100%. The proposed model is to diagnose infectious Covid. Hence, this tool can be used as a screening test. It shows that the proposed model provides an accuracy of 100%, a sensitivity of 100%, specificity of 100%, and precision/PPV of 100%. Then, the proposed model is further validated based on the Receiver Operating Characteristic Curve (ROC), a graphical representation that illustrates the diagnostic capacity of the system. This curve is constructed by plotting True Positive Rate (TRP) against False Positive Rate (FPR). Another metric derived from the ROC plot is the Area Under the Curve (AUC), which measures the model's overall quality. In this model for the infectious disease, the fraction of infected cases to be detected correctly who need immediate antibiotics. It yields an AUC of 100. The proposed model is also based on DTI-enriched CXR images. The impact of incorporating the DTI is better training performance and validation performance. Figure 14a shows that the proposed model reaches the maximum training accuracy of 100 in the 25th iteration. Figure 14b shows the equivalent training loss of the proposed model. The drop in the Fig. 14b shows the instability of the initial phases of training which is expected to be common in most of the deep learning models.

Fig. 13
figure 13

Performance analysis of DTI with Inception-ResNet V2

Full size image
Fig. 14
figure 14

Training performance analysis of DtXpert

Full size image

The proposed dtXpert provides the highest accuracy of 100%, highest sensitivity of 100%, and highest specificity of 100% precision value of 100% with all types of deep learning networks. The proposed model outperforms all the existing state-of-the-art methods to diagnose covid.

3.4 Statistical analysis of dtXpert

The proposed dtXpert provides the highest accuracy of 100%. The claimed accuracy is validated by chi-square test.

Ho – Accuracy is greater than 99.99%

H1 – Accuracy is less than 99.99%

Degrees of freedom – 1 No. of attribute – 1 and the individual \(\chi^{2}\) test is applied at 0.01% level of significance. \(\chi^{2}\) table value is 6.635. Table 2 gives the value as 0.0019 which is less than \(\chi^{2}\) table value. So null hypothesis is accepted and the accuracy for dtXpert can be more than 99.99%.

Table 2 Statistical analysis—dtXpert
Full size table

4 Conclusion

A deep learning-based robust early diagnosis method for covid is proposed. To balance the interclass variation and interclass similarity in CXR images, the Delaunay Triangulation is incorporated as the CXR images are low radiation and unbalanced quality images. The image quality is enhanced to increase the robustness of the diagnosis method. Without segmentation, the proposed DT algorithm achieves the accurate visualization of the suspicious region in the CXR. The Deep Convolutional Neural Networks train deep fused images with residual connections. The proposed model is trained and tested by enormous benchmark datasets. The performance of the proposed system is analyzed in terms of accuracy, sensitivity, specificity, and AUC. The proposed system yields a validation accuracy of 100%, a sensitivity of 100%, specificity of 100%, a precision of 100%, and an AUC of 100%, which is ideal compared to other state-of-the-art deep learning methods for diagnosing covid. The proposed dtXpert can be used as a computer-aided prognosis method for covid to reduce the manual time, effort, and dependency on the specialist's expertise level.