Deep and hand-crafted features based on Weierstrass elliptic function for MRI brain tumor classification

Terminology

AVG

average pooling layer

BRATS

brain tumor segmentation

CNNs

convolutional neural networks

DB

dense blocks

DL

deep learning

DWT

discrete wavelet transform

FC

fully connected layer

LSTM

long short-term memory

MP

max pooling layer

MRI

magnetic resonance imaging

PCA

principal component analysis

SVM

support vector machines

WEF

Weierstrass elliptic function

1 Introduction

The brain is a crucial human body part that controls the functions of all other organs and assists in decision-making. Brain tumors are one of the deadliest diseases in the world, affecting people physically, intellectually, and psychologically. Most of these diseases begin in other parts of the body and subsequently progress to the brain. The ability to correctly diagnose and treat brain tumors is made possible by the classification of brain tumor images. Brain tumors are a serious medical condition that can cause a wide range of symptoms and, if untreated, can be fatal. The best treatment strategy and increased likelihood of a patient’s recovery depend on an early and accurate diagnosis [1]. Uncontrolled proliferation leads to the formation of malignant cells, which are pathologic cells. Brain tumors are made up of heterogeneous cells that proliferate at an uncontrollable rate and differ greatly in terms of morphological traits and genetic variants. Some of the common types of brain tumors that can be classified using imaging include:

• Gliomas: Are tumors that arise from the brain’s supporting glial cells.

• Meningiomas: These tumors grow from the meninges, the membranes that cover the brain and spinal cord.

• Pituitary tumors: The pituitary gland, which is found at the base of the brain, is where these tumors grow.

• Metastatic tumors: These tumors are secondaries that have metastasized from other parts of the body to the brain.

• Medulloblastomas: These are rapidly expanding tumors that most frequently affect children and form in the cerebellum, the region of the brain responsible for balance and coordination.

Early identification of a brain tumor is essential for lowering the patient’s mortality rate. However, due to tumor size, form, location, and kind, it is a difficult to process [2]. The result of a magnetic resonance imaging (MRI) examination is a collection of tissue-specific images with various contrast visualizations. These pulse sequences offer useful anatomical details that assist clinicians in precisely diagnosing the pathological conditions. The two types of MRI technologies are T1-weighted (T1-w) and T2-weighted (T2-w). Because they have a high resolution and fewer artifacts, T1-w images are used as anatomical references. T2-weighted (T2-w) images, on the other hand, are significant MRI sequences that are useful for identifying the limits of pathological structures [3,4]. Research on brain cancer has advanced significantly because of the accurate classification of brain tumors. Researchers can create new therapies that target pathways and mutations that fuel tumor growth with a better understanding of the genetic and molecular features of brain tumors. In addition, precise brain tumor classification enables medical professionals to customize a treatment plan based on the tumor. Early tumor identification is critical in the therapy procedure. The radiologist classifies the MR image as normal or abnormal using classification algorithms. Manual classification is a time-consuming task. Furthermore, they may obtain different results from different observers or different findings from the same observer when distinguishing tumor. As a result, automatic classification algorithms are required. Many algorithms based on image processing and machine learning have been developed by computer vision researchers [5,6]. Characterizing regions in images that are measured using texture analysis is the goal of any diagnostic imaging technique. In addition, texture analysis is thought to be more accurate than human visual examination of the brain in MR images.

The feature extraction methods are categorized as handcrafted (classical) [7] or deep learning (DL)-based [8]. Handcrafted techniques in the traditional approach include features such as texture features, and shape features. These features are then fed into a machine learning algorithm as input. The handcrafted feature extraction methods are based on using texture analysis to characterize tissues in brain images. Texture analysis is a quick and easy approach to get high-level data. Furthermore, texture analysis has been found to be more effective than human in distinguishing between tissues.

DL approaches extract features from raw images using multiple layers of DL [9,10]. Numerous layers of convolution, pooling, and activation functions are typically used during the feature extraction process in Convolutional neural networks (CNNs). Many techniques based on DL have been introduced in the literature, and few of them are described here. The benefit of handcrafted feature extraction combined with a CNN is that it enables the extraction of both significant, clinically relevant features. This can improve the accuracy and interpretability of the results. Medical science has made significant advances in recent years as a result of the machine learning and the DL [11]. Deep transfer learning employs an existing model that has already been shown to be effective rather than developing a new one [12]. Existing literature predominantly focuses on either handcrafted or DL-based methods in isolation. As mentioned, the handcrafted features capture domain-specific knowledge, while DL features discern complex, abstract patterns within the images. Therefore, the aim of this study is to use both the handcrafted and DL features that are extracted from brain MRI to provide additional diagnostic information for classifying brain as normal or abnormal (with tumor). The primary contribution of this study is the developmental model for extracting features from both Weierstrass elliptic function (WEF) and DenseNet-201 and combining them into a single feature set to achieve superior diagnostic accuracy and interpretability. High frequency image details of the brain image are captured using the texture features of the proposed WEF model. While the overall classification of MRI brain images has been greatly enhanced by the deep features of the DenseNet-201 model.

2 Related works

Radiologists can examine medical data with a second perspective thanks to the MRI classification. To achieve exact diagnosis, it is necessary to design an excellent diagnostic tool for image classification from MRI scans. The MRI brain images can be diagnosed using supervised techniques such as artificial neural networks, which classify images into one of two categories: normal and abnormal [13,14,15]. The term abnormal (pathological) brain MRI refers to the scans that do not show a healthy brain, whereas normal refers to the appearance and intensity of brain that are normal [16]. Several approaches for MRI brain tumor classification have been proposed.

Chaddad [17] proposed a unique approach for feature extraction utilizing T1 and T2 weighted MRI based on Gaussian mixture model features. The diseased region was identified utilizing multi-threshold segmentation with morphological MR image processing. The comparative investigation found that principal component analysis and wavelet-based features achieved the best results. This method, however, is dependent on the accuracy of the segmentation stage. Fayaz et al. [16] proposed an effective binary classification model for MRI images of the brain. The suggested model used a blended neural network for brain MRI classification and a discrete wavelet transform for feature extraction with statistical feature reduction. According to the results, the proposed approach produces good outcomes and detection accuracy. However, the approach used for feature selection was the reason for the good results.

In computer vision, the DL has become a popular family of machine learning algorithms that has been effectively applied to a variety of problems. Due to its hierarchical technique of learning high-level from input data, the DL employing the CNN is quickly increasing in neuro imaging. Several research used DL algorithms for MRI image classification, utilizing different CNN architectures. Khan et al. [18] proposed a CNN approach for classifying brain cancers on MRI images. Although the experiment was conducted on a limited dataset, the results reveal that the model accuracy was effective. In comparison to other pre-trained models, this methodology consumed less processing power and produced significantly superior accuracy outcomes. Rehman et al. [19] classified the brain tumors using three distinct pre-trained CNN models (GoogleNet, VGG16, and AlexNet). Most of the employed CNN representations are modeled by deep extraction of image features, which works well with images. Aziz et al. [9] proposed employing an array of optimum DL features to classify multimodal brain tumors. After pre-processing the image database, two pre-trained DL models are used to select the best feature from both deep models. The experiment was carried out using the BraTs2019 dataset and yielded accuracies of 87.8%. However, because most extracted features are irrelevant to the classification phase, feature selection is utilized to improve classification accuracy. Aamir et al. [20] developed an automated method for identifying brain cancers on MRI scans first, a pre-processing stage is employed to improve visual quality, followed by the application of two separate pre-trained DL models to extract strong characteristics from photos. Then, a cluster reveals the top tumor sites. The suggested method outperformed previous approaches with the classification accuracy. This method relies on clustering to segment the tumor region, which enhanced the classification accuracy. Wahlang et al. [15] proposed DL architectures for classifying brain MRI images as normal or abnormal, while gender and age are introduced as higher characteristics to improve classification accuracy and meaning. A DL CNN-based approach was used with other deep architectures such as ResNet, LeNet, and AlexNet. In comparison to support vector machines (SVM) and AlexNet, the overall accuracy attained is 88%. However, in this approach, the gender and age features were used to reduce the effect of irrelevant features, and high similarity between tumor types. In order to identify early brain tumors, Noreen et al. [21] developed a multi-level feature extraction method using two DenseNet-201 and Inception-v3. In contrast to the most advanced DL and machine learning-based techniques available today for classifying brain tumors, Inception-v3 and DenseNet-201 outperformed previous approaches with the classification accuracy. The lack of labeled data is one of the most critical challenges to DL in medical applications. As recent breakthroughs, DL models have demonstrated that the more data there are, the better the result. The primary goal of this work is to develop an automated method for identifying the brain as normal (without tumor) or abnormal (with tumor) using a combination of handcrafted and DL features from brain MRI. In addition to the DenseNet-201deep features, the handcrafted feature extraction using the mathematical model of WEF is what makes this study novel.

3 Materials and methods

Handcrafted techniques in the traditional approach include features such as texture features, and shape features. These features are then fed into a machine learning algorithm as input. The handcrafted feature extraction methods are based on using texture analysis to characterize tissues in brain images. Texture analysis is a quick and easy approach to get high-level data. Furthermore, texture analysis has been found to be more effective than human in distinguishing between tissues.

The existing literature predominantly focuses on either handcrafted or DL-based methods in isolation, while the brain sequences cannot be accurately classified as normal or abnormal brain tissue using just one method of MRI extracted features. In this case, the proposed study involves extracting features using two different methods: the new handcrafted feature extraction method based on proposed WEF and the DL feature extraction method. The proposed feature models are extracted using MATLAB 2023b. The number of extracted features recovered by one approach was insufficient to reliably identify the MRI sequences as normal or abnormal brain tissue. The proposed study includes three stages. The first stage uses a newly created, handcrafted feature extraction using the WEF model, while the deep feature extraction model based on DenseNet-201 is used in the second stage. The third step involves combining several feature extraction techniques to create a feature vector for the classification stage in which SVM is utilized. The flow process for MRI classification is illustrated in Figure 1.

Figure 1

The flow process for MRI classification [24].

3.2 Method details

In this section, the pixel energy based on WEF, which is the exact solution of the complex differential equations of type complex Ginzburg–Landau equation, is defined. Finally, the formula for obtaining the image features is defined using the values of the solution’s Weierstrass coefficients.

3.2.1 Complex differential equations (CDEs)

A differential equation which incorporates complex-valued functions, and their derivatives is referred to as a CDE. Usually, it looks as follows:

(1)

where is a complex variable, represents the first derivative of in regard to an unknown variable (commonly written as t), represents the second derivative, and so on through the nth derivative.

According to their characteristics, CDEs can be divided into distinct types, such as linear or nonlinear, ordinary or partial, and homogeneous or nonhomogeneous. Complex-valued functions that satisfied the equation for a specific domain are the solutions to CDEs. Similar methods to those used to solve real-valued differential equations, such as variable separation, integrating factors, power series solutions, and Laplace transforms, are frequently employed to solve CDEs. Nevertheless, the existence of fictitious and complex numbers in complex differential equations can lead to extra complications. It is significant to highlight that CDEs have uses in many disciplines, including physics, engineering, and mathematics, where the description of physical processes and the development of theoretical frameworks inherently involve the use of complex numbers.

3.2.2 Image processing by CDEs

CDEs are frequently used to express and evaluate a variety of image-related events in the area of image processing. Complex-valued functions, with real and imaginary portions that depict various facets of the image, are used in these equations. The complex Ginzburg–Landau equation (CGLE) is a typical example of a complex differential equation utilized in image processing. A partial differential equation known as the CGLE can be used to model complex-valued fields like the amplitudes of optical waves in some systems. It is frequently employed in the fields of texture analysis, image segmentation, and pattern creation research. The following provides the CGLE’s general form:

(2)

At this juncture, I embodies the complex-valued field (image matrix), denotes time, ∇² denotes the Laplacian operator, |I|² indicates the squared magnitude of I, and κ, γ, and μ are parameters that control the undercurrents of the equation. The CGLE captures the interaction between linear diffusion, nonlinearity, and coupling in the development of complex patterns.

One can examine the creation of different patterns in images, such as spirals, waves, and horizontal stripes, by resolving the CGLE. Investigators are able to comprehend the fundamental mechanics of image formations by using it to analyze the stability and bifurcations of these patterns. A robust mathematical foundation for describing and interpreting intricate patterns and structures in image processing is provided by CDEs. They make it possible for experts to learn more about how images behave and to create cutting-edge methods for image creation, segmentation, and enhancement. The solution of the equation is given by using the WEF [25].

(3)

where is defined in the puncture unit disk. Moreover, the coefficients (ρ) are defined as follows:

(4)

where ( ) = (0,1) for the equianharmonic case and ( ) = (1,0) for the lemniscate case.

Proposition 2.1

The upper bound in the open unit disk of is

(5)

Moreover, if it is convex, then

(6)

Proof

Let = r < 1. Then,

□

where 0 < r < 1 is the probability of the pixel, and ρ denotes the Weierstrass coefficients.

For the second part, since the functional is convex in the open unit disk, it satisfies the subordination inequality [26].

But,

Hence,

By Proposition 2.1, the measurable amount of an image pixel's energy is determined by its textural uniformity, image similarity, and degree of pixel pair Repetitions, which is given by:

(7)

where G(i,j) is the input image pixels, r denotes the pixel probability, and an experimental fix of 1.5 has been made for the value of the Weierstrass coefficients ρ. It is important to note that the domains of image feature extraction in image processing and computer vision frequently use ideas from convex optimization to address specific issues. The input image G(i,j) is first divided into non-overlapping blocks for the feature extraction. The non-overlapping blocks are set to size (16 × 16). For each block, compute the block feature using equation (7). Figure 2 shows the behavior of the differences between the two classes using a scatter plot (tumor and non-tumor).

Figure 2

The distribution of the two classes of features. Non-Tumor (blue dots) and tumor (red dots).

The texture feature of Weierstrass coefficients of the WEF are used to capture high frequency image details of the brain image. These extracted features are unique and can be used to detect brain tumor.

3.3 CNN models

There have been numerous attempts to classify patients’ brain images from MRI images using various techniques and image datasets, which inspired this study to investigate the CNN DL feature extraction method. The CNN-based learning algorithms are more appropriate for medical image classification. Numerous images are required for the CNN medical image training, which is difficult to accomplish in standard testing circumstances. Rather than beginning the learning process from scratch, transfer learning builds on past knowledge. The principal benefits of transfer learning include reduced training time and enhanced performance on small datasets. There are two ways to employ pre-trained CNN models traditionally: first, the ready-made pre-trained CNN models are used for feature extraction and training. Second, fine-tune pre-trained models in selected layers to achieve the desired results. The pre-trained DenseNet-201 has been used in this study to extract features from a dataset of brain tumors. Multiple dense blocks (DBs) with direct connections between layers are present in DenseNet-201. Every layer is connected to every other layer in the dense convolutional network (CONV). The vanishing-gradient problem is eliminated, feature propagation is improved, features are reused, and there is a significant decrease in the number of parameters, among other benefits. After several convolutional layers, it can be challenging to distinguish between some brain MRI slices because they share many characteristics. High-level neural networks suffer from accuracy declines due to vanishing gradients, which is why DenseNet was developed. Due to this, DenseNet appears to be a viable option for this study’s classification of brain tumors using MRI. One convolutional layer (CONV), one max pooling layer (MP), three transition layers (TL), one average pooling layer (AVG), one fully connected layer (FC), and one Softmax layer make up DenseNet121. There are four DB layers in total. The AVG layer accepts all the feature maps of the network to perform classification. There are four DBs with various numbers of convolutional layers in DenseNet-201 as illustrated in Figure 3 and in Table 1 [27].

Figure 3

Framework of DenseNet-201.

Table 1

DenseNet-201 architectures

Layers	Size
Convolution (Conv)	112 × 112	7 × 7, stride 2
Pooling (MP)	56 × 56	3 × 3 Maxpool (MP), Stride 2
Dense block 1 (DB)	56 × 56	6 × [(1 × 1 conv), (3 × 3 conv)], 1 × 1 conv
Transitional layer 1 (TL)	56 × 56	2 × 2 average pool (AVGP), stride 2
Dense block 2	28 × 28	12 × ([1 × 1 conv], [3 × 3 conv]), 1 × 1 conv
Transitional layer 2	28 × 28	2 × 2 average pool, stride 2
Dense block 3	14 × 14	48 × ([1 × 1 conv), [3 × 3 conv]), 1 × 1 conv
Transitional layer 3	7 × 7	2 × 2 average pool, stride 2
Dense block 4	7 × 7	32 × ([1 × 1 conv], [3 × 3 conv]), 7 × 7 global average pool
Classification layer	1,000	Fully connected (FC), Softmax

During the feature extraction step, the transfer learning approach is used to extract the DL features from the pre-training network. The network’s last fully linked layer is used as a feature vector through feature extraction.

4 Results and discussion

In this study, MATLAB R2021b on Windows 10 with processor Intel(R) Core i7-7700HQ CPU @ 2.80 GHz 16 GB was used to produce the test results. The complete dataset has been split into three sections: testing, validation, and training. This has helped to prevent overfitting and improve the model’s ability to generalize. In addition, the MRI images are scaled to fit the pre-trained network’s input size. The proposed classification model investigates the effects of various image characteristics on an image classification scheme’s performance. The classifiers go through three different phases. The WEF feature model, the deep DenseNet-201 features model, and the feature combination of proposed WEF feature with the deep DenseNet-201 features using SVM classifier.

The hyperplane in an SVM classifier is chosen based on how well it divides the data into distinct classes. This hyperplane seeks to minimize the classification error while maximizing the margin between the two classes for binary classifications such as tumor vs non-tumor. Regularization parameter (C) and kernel hyperparameters for the radial basis function were employed in the applied SVM. These settings are adjusted for best results.

4.3 Results of classification using proposed WEF with the deep DenseNet-201 features

The classification result of proposed WEF features is illustrated in Figure 4. The combined features extraction of proposed WEF handcrafted feature extraction model with pre-trained DenseNet-201 features succeeded to obtain good classification accuracy which reflects the benefits of applying the combined features extraction for detection in brain MRI images. The accuracy on BRATS MRI brain dataset, using the handcrafted WEF and the DenseNet-201 features was 94.65 and 95.85%, respectively. On the BRATS2019 MRI brain dataset, the feature combination of WEF handcrafted with DenseNet-201 was able to achieve 98.55% accuracy, which produced the best classification result. With a classification accuracy of 95.85%, pre-trained DenseNet-201 comes second. The proposed WEF handcrafted comes third with a classification accuracy of 94.65%. Figure 4 presents each model’s classification outcomes. When compared to single feature extraction techniques, the accuracy yielded by the suggested feature combination showed the best performance.

Figure 4

The classification results of WEF handcrafted with pre-trained DenseNet-201 features on MRI dataset.

5 Conclusion

This study presented a novel mathematical model for classifying brain medical images using the WEF, and deep feature extraction. The brain image’s high frequency details are captured by the proposed Weierstrass coefficients of WEF, which calculates each pixel’s energy. While the DL feature extraction DenseNet-201 model was used to extract deep features. The novelty of this study is the WEF feature extraction mathematical model in addition to DL features. The BRATS dataset with a five-fold cross-validation scheme was used for the validation. The proposed brain medical image classification approach outperforms the comparator methods in experiments using brain MRI medical image sets. In this study, the classification success achieved was 98.55% accuracy for WEF with pre-trained DenseNet-201. More refined trained CNNs with parameter optimization at the same level of efficacy could be added in the future as a potential improvement. A limitation of the proposed feature extraction method is that the hand-crafted features are susceptible to noise and artifacts in the data, which can impair classification performance.