A Feature Extraction Using Probabilistic Neural Network and BTFSC-Net Model with Deep Learning for Brain Tumor Classification

Abstract

Background and Objectives: Brain Tumor Fusion-based Segments and Classification-Non-enhancing tumor (BTFSC-Net) is a hybrid system for classifying brain tumors that combine medical image fusion, segmentation, feature extraction, and classification procedures. Materials and Methods: to reduce noise from medical images, the hybrid probabilistic wiener filter (HPWF) is first applied as a preprocessing step. Then, to combine robust edge analysis (REA) properties in magnetic resonance imaging (MRI) and computed tomography (CT) medical images, a fusion network based on deep learning convolutional neural networks (DLCNN) is developed. Here, the brain images’ slopes and borders are detected using REA. To separate the sick region from the color image, adaptive fuzzy c-means integrated k-means (HFCMIK) clustering is then implemented. To extract hybrid features from the fused image, low-level features based on the redundant discrete wavelet transform (RDWT), empirical color features, and texture characteristics based on the gray-level cooccurrence matrix (GLCM) are also used. Finally, to distinguish between benign and malignant tumors, a deep learning probabilistic neural network (DLPNN) is deployed. Results: according to the findings, the suggested BTFSC-Net model performed better than more traditional preprocessing, fusion, segmentation, and classification techniques. Additionally, 99.21% segmentation accuracy and 99.46% classification accuracy were reached using the proposed BTFSC-Net model. Conclusions: earlier approaches have not performed as well as our presented method for image fusion, segmentation, feature extraction, classification operations, and brain tumor classification. These results illustrate that the designed approach performed more effectively in terms of enhanced quantitative evaluation with better accuracy as well as visual performance.

1. Introduction

Brain tumor (BT) is the tenth leading cause of death and disability worldwide and has been recognized by organizations such as the World Health Organization (WHO), National Brain Tumor Society (NBTS), and the Indian Society of Neuro-Oncology (ISNO) as one of the most important primary neoplasms causing morbidity and mortality across the world [1]. According to the international agency for cancer research, over 97,000 people in the United States and nearly 126,000 people worldwide suffer disability and related consequences of brain tumors each year [2]. If appropriate diagnosis, intervention, and management are received at an early stage, an increase in survival rates can be achieved.

Medical image fusion (MIF) techniques such as digital imaging, pattern recognition, and machine learning ML) by fusion of images now have various applications in clinical medicine and are widely used to diagnose neoplasms as well [3]. These fusion methods overcome the constraints of traditional imaging techniques and are described to be more effective than magnetic resonance imaging (MRI) as well as computed tomography (CT) [4]. Various images of organs have been created by using different sensor strategies [5]. It is crucial to differentiate a neoplastic lesion and also to derive lesion characteristics as opposed to the surrounding normal tissues to help in diagnosis. Since lesion size, texture, shape, and placement vary based on different individual characteristics, images must be evaluated as well as categorized using tissue segmentation, which has been detailed in different forms of automation, to produce a diagnosis. Curvelet domain coefficients providing substantial structural features and functional mapping are necessary [6]. To distinguish the boundaries between normal and abnormal brain tissue, expert segmentation requires several upgraded datasets and pixel profiles. It is ideal in this case to use fully automated machine learning techniques. Multimodal medical image fusion was thus designed utilizing the particle swarm optimization technique (PSO) to improve the efficiency of multimodal mapping [7].

The fusion-based techniques help to analyze and evaluate the disease by expanding the visual data and lucidity. Direct fusion techniques frequently create unwanted impacts prompting distortion and low contrast. Multi-scale decomposition (MSD) strategies have made progress in different image fusion issues and the limitations are overcome through the total variation (TV-L1) method [8]. MIF is well-known terminology in this area of satellite imaging and diagnostic medical imaging, which has now expanded its presence in clinical diagnosis and has proven to be beneficial in multimodal medical image fusion (MMIF) [9]. The spatial domain is where multimodal image fusion and pixel-to-pixel fusion take place. Additionally, a max/min for the fusion approaches using weighted pixels is available [10].

Utilizing phase congruency and local laplacian energy, pixel activity determines the weights to select the most active pixels. [11]. The image features are first scaled by a designed method with average weighting. Yin et al. [12] in their study utilised the maximum average mutual information, this image fusion technique. MIF has widely used optimization techniques to resolve deficiency issues such as the whale optimization algorithm (WOA), which is one of the most used optimal algorithms [13].

For image fusion, the authors of [14] designed a method and different features are extracted by individually multiplying each input image’s first convolutional sparsity component. A few techniques are also discussed, including the principle of feature measurement [15] and graph filter and sparse representation (GFSR) [16]. Dual-branch CNN (DB-CNN) [17] as well as separable dictionary learning based on Gabor filtering [18] and local difference in the non-subsampled domain (LDNSD) [19] are also considered. Edge-based artifacts are instances of blocking objects in these methods [7].

To match the high-frequency images, the Laplacian re-decomposition (LRD) method offered a way to inverse LRD fusion-based rule by producing pixels surrounding the overlapped domains [20]. Deep learning and MMIF are also created for disease diagnostics in medicine. Deep learning is combined with MDT and actual analysis to characterize the MMIF. It can successfully overcome the difficulty of only one-page processing and make up for the lack of image fusion. It can also be created to address many types of MMIF challenges in batch processing mode [21]. To improve the execution of various methods of image processing, algorithms perform optimization and give an optimal solution for image fusion; various global optimization techniques are imposing procedures that can convey better clarifications for various issues [22].

Positron Emission Tomography (PET) images are also enhanced in terms of spatial resolution and given acceptable color [23]. MRI images also have improved spatial resolution and provide decent color. Using a CNN-based technique, weight maps are created from input medical images. Following that, the weight maps formed are subjected to gaussian pyramid decomposition, and a multi-scale high-resolution image is obtained using the contrast pyramid decomposition approach [24]. To break down the image into sub-band categories, the Non-Subsampled Contourlet Transform (NSCT) is utilized [25]. The presentation of an image fusion framework based on two CNP models using nonsubsampled shearlet transforms (NSST) is made. These two CNP models were used to constrain the fusion of the NSST domain, which operates at lower frequencies. Applying the encoder network to the encoded image, its features are extracted and fused using Lmax norms [10].

Additional segmentation methods include a fast level set based CNN [26], bayesian fuzzy clustering with hybrid deep auto-encoder (BFC-HAD) [27], and symmetric-driven adversarial network (SDAN) [28]. However, the precision and accuracy of these techniques are compromised. The categorization of brain tumor segmentation, synthetic data augmentation using multiscale CNN [29] as well as deep learning [30] are also designed. Meningioma from non-meningioma brain images is distinguished using a learning-based fine-tuning transfer [31] and dice coefficient index [32]. Three forms of brain tumors were distinguished by the authors of [33] using the transfer learning CNN model. An already-built pre-trained network was modified by expanding the tumor, ring-dividing it, and using T1-weighted contrast-enhanced MRI [34]. Hybridization of two methods entropy-based controlling and Multiclass Vector machine (M-SVM) is used for optimal feature extraction [35,36,37]. The differential deep-CNN model for detecting brain cancers in MRI images was put to the test by the authors in [38]. Here, CNN-multi-scale analysis was used to create 3D-CNN [39] utilizing MRI images of pituitary tumors. The classification procedure was made to perform better in [36,40,41] by using variational auto-encoders with generative adversarial networks and a hybrid model. These conventions still need to be refined because they do not produce adequate classification and segmentation results.

As dataset sizes grow, the high computational complexity has a negative impact on traditional models. Before their implementation in hardware, neural networks (NNs) must first be evaluated for computational complexity, which uses the majority of the CPU and GPU resources [42,43,44]. The difficulty, however, is dependent on the specific deep learning-probabilistic neural network (DLPNN) design and can be decreased if we can accept a more accurate trade-off: changing the values of the hyper-parameters can have an impact on the accuracy while also working to lower the computational complexity. In addition, the performance of conventional approaches for segmentation, classification, and fusion needs to be improved. The main contributions of the paper are listed below to help solve these issues:

• Preprocessing, fusion, segmentation, and classification steps were combined to create a brand-new BTFSC-Net model that no other authors have yet created.
• The original purpose of HPWF was to improve the contrast, brightness, and color qualities of MRI and CT medical images by removing various noises from them.
• The REA analysis is combined with the input data MRI and CT images using DLCNN-based fusion network, which identified the tumor region.
• For the separation of a tumor region from a fused image, the HFCMIK method is used to further characterize the major region of the brain tumor.
• Using the gray-level cooccurrence matrix (GLCM), redundant discrete wavelet transform (RDWT) trained features, the classification of benign and malignant tumors is achieved using DLPNN.
• Data from simulations illustrate that the designed approach outperformed state-of-the-art methodology.

2. Proposed Methodology

The classification techniques for brain tumors are thoroughly examined in this section. Both Figure 1 and

Table 1. Fusion of a proposed segmentation and classification system for brain tumors.

Input: MRI and CT brain images Output: Classified outcome Intermediate outcomes: Fused and Segmented outcomes Objective Evaluation-Set 1: Entropy, MI, UQI, STD, PSNR. Objective Evaluation-Set 2: SACC, SSEN, SPEC, SPR, SNPV, SFPR, SFDR, SFNR, SF1, SMCC. Objective Evaluation-Set 3: CACC, CSEN, CPEC, CPR, CNPV, CFPR, CFDR, CFNR, CF1, CMCC

Step 1:  Perform preprocessing operations using HPWF for the removal of various noises from MRI and CT medical images, which also enhances the contrast, brightness, and color properties. Step 2:  Then, DLCNN-based Fusion-Net is used to fuse the preprocessed MRI, and CT images with the REA analysis, which improves the region of the tumor. Step 3:  In addition, HFCMIK is used to segment the tumor region from the fused outcome, so an accurate area of the brain tumor is detected. Step 4:  Finally, DLPNN is used to classify the benign and malignant tumors from the GLCM, RDWT-trained features. Step 5:  Calculate objective and subjective performance. The objective evaluation of set-1 is measured from Fusion-Net. The objective evaluation of set-2 is measured from HFCMIK. The objective evaluation of set-3 is measured from DLPNN.

present the proposed algorithm for the BTFSC-Net technique. Medical images are put through an HPWF before being processed further to remove any noise. To fuse MRI and CT medical images while retaining REA capabilities a new method is designed. Since it is essential to obtain the slopes and boundaries of the image, REA is utilized.

HFCMIK clustering is utilized to separate the diseased region from the fused image. Furthermore, the fused image is utilized to integrate empirical color features, low-level features based on the RDWT, and texture characteristics based on the GLCM to form hybrid features [37,45]. The distinction between benign and malignant tumors is made using a DLPNN.

2.1. Hybrid Probabilistic Wiener Filter Method (HPWF)

With the use of Gaussian mask kernels, the HPWF successfully improves and eliminates noise from images. The HPWF algorithm may be found in

Table 2. HPWF Approach.

Input: Noisy image Xij Output: Denoised image Yij

Step 1: Consider Xij image with M × N sizes.

Step 2: Estimate σGN of $X_{ij}$ as follows: $${\sigma }_{GN}=\frac{1}{MN}{{\displaystyle \sum }}_{i=1}^M{{\displaystyle \sum }}_{j=1}^N{\left|X\ast MASK\right|}_{i,j}$$ (2)     Here, ∗ denotes the convolution operation, σGN is noise deviation.

Step 3: Change the size of the mask based on σGN factor as follows: $$w=\left\{\begin{array}{c}3\times 3,{\sigma }_{GN}<20\\ 5\times 5,{\sigma }_{GN}\ge 20\end{array}\right.$$ (3)

Step 4: Apply Xij input to the mean filtering for low-level noise removal. $${\tilde{X}}_{i,j}=\frac{1}{R}{{\displaystyle \sum }}_{i=1}^R{{\displaystyle \sum }}_{j=1}^R{X}_{i,j}\ast w_{i,j}$$ (4)     Here, R is a range of pixels in the mask, which is either 9 or 25.

Step 5: Calculate the absolute difference between the mean filtered outcome to Xij. $$D_{i,j}=\left|X_{i,j}-{\tilde{X}}_{i,j}\right|$$ (5)

Step 6:  Eliminate pixels more than the mean value (µ). Choose the pixels in ${\tilde{X}}_{i,j}$ or Dij as follows: $$V=\left\{\begin{array}{c}\left\{D_{\left(1\right)},D_{\left(2\right)},D_{\left(3\right)}\dots D_{\left(n\right)}\right\},D_{ij}<\mu \hfill \\ \left\{{\tilde{X}}_1,{\tilde{X}}_2,{\tilde{X}}_3,\dots ,{\tilde{X}}_n\right\},D_{ij}\ge \mu \hfill \end{array}\right.$$ (6)

Step 7: Apply the weighted average on V, and it generates the denoised outcome (Yij). $$Y_{ij}=mean\left(V\right)$$ (7)

, and pixels comprise images. Many groups of the image have been divided. After the pixel cluster is implemented to HPWF in one of these ways, the resultant pixel is an improved version of the primary pixel. Each image pixel is contaminated by a variety of noise sources. Consider the original image O_ij and G_ij is the noise term with ith row, jth column pixel values. Here is the produced image with noise X_ij:

$$X_{ij}=O_{ij}+G_{ij}$$

(1)

To estimate the gaussian noise variance, the Gaussian mask is convolved with a noisy image X_ij of size M × N. We describe the use of eigenvalues as a threshold parameter for denoising Gaussian noise in images. Gaussian noise is removed through thresholding in the transform domain rather than the spatial domain. Using spatial coefficients, thresholding is performed to estimate great pixels. The noisy image of size M × N is then mean filtered and stored in ${\tilde{X}}_{i,j}$. By subtracting the noisy image from the mean filtered image, the difference mask (D_i,j) is created. This difference mask is then used to keep only uniformly distributed pixels (V) of an image based on whether (D_i,j) is less or greater than the mean (µ). The denoised (Y_i,j) image is created by adjusting the mean filtered image based on the threshold setting.

The noisy image of size M × N is then mean filtered. By subtracting the noisy image from the mean filtered image, the difference mask is created. The denoised image is created by adjusting the mean filtered image based on the threshold setting.

2.2. Proposed Fusion Strategy

The new technique may combine images from a variety of imaging methods, e.g., MRI-SPECT, MRI-CT, and MRI-PET combinations, by using two distinct structures. The hybrid DLCNN for fusing MRI-CT/PET/SPECT images is shown in Figure 2. To enhance slope analysis in the event of misalignment, image decomposition is performed using the REA technique. This removes the edges and slopes from the primary input image. Medical images typically have piecewise smooth slopes, with Analysis indicating the edges. The edge positions on these images should line up with the CT image when they are aligned. This highlights how the analytical feature is dynamic and changing using the given medical images. Analysis literature has not looked at this property. Active slope analysis images and signals are the names given to this type of image or signal today.

By subtracting the blurred input photos from the represented input images using HPWF, crisp images are produced. The fused output is then produced by combining the results from the HPWF and REA. The operational technique of the recommended method is shown in

Table 3. Proposed Fusion algorithm.

INPUT OUTPUT	MRI Medical Images, PET/SPECT/CT Image Types Fused Image
Step 1	XCT    ← MRI CT Images
	XPET  ← MRI PET Images
	XSPECT ← MRI SPECT Images
	//Input MRI Greyscale medical images
Step 2	if (XCT) then      //compare image type  EedgeSlopeImg ← REA (XCT)
	//apply Robust Edge Analysis (REA) to generate edge-slope-analysed images Else if (XPETORXSPECT) then  Cb, Y, Cr ← RGB2YCbCrColorCon (XPET, XSPECT)
	//the above function converts   EedgeSlopeImg ← REA (XCT)
	//Convert all the images I into data vectors
Step 3	Ffeatures ← HPWF (EedgeSlopeImg)
	//calculate weighted data points of the image vector I
Step 4	if (XCT) then  XFO ← FusionNetFeatureFusion (Ffeatures)
	Else if (XPET ORXSPECT) then   FOfusedOutcome ← FusionNetFeatureFusion (Ffeatures)
	XF ← YCbCr2RGBColorCon (FOfusedOutcome)

.

2.2.1. Robust Edge Analysis

During the fusion process, contemporaneous registration of the raw medical images is accomplished using an REA approach shown in

Table 4. REA algorithm.

INPUT: OUTPUT:	Medical Images (I) Edge Based slope analyzed images
Phase 1:	X  ← Medical Images
	//input of medical images   XTest  ← TrainTest (X)
	//extract the test images from the medical images X
Phase 2:	YNoiseFree  ← GaussianFiltering (XTest)
	//Noise-free medical images using the gaussian approach
Phase 3:	DDecomposedImages   ← CannyEdgeDetection (YNoiseFree)
	//decomposed images, extracted with perfect shapes, edges, textures, and spatial regions
Phase 4:	IFinedImages   ← EdgeRemovalAverageThresholdWeight (DDecomposedImages)
	//This removes unnecessary edges & detailed layers of each image are generated using the above method
Phase 5:	EEnergyDetails  ← LayerWiseEnergyCal (IFinedImages)
	//The above function will calculate & generate energy details with Equation (1), & variations present in the energy level of an image then subproblems of test images will be developed
Phase 6:	EPerfectEnergyLevels  ← $\xi$(PerfectEnergyCal (EEnergyDetails))
	//The above function develops the perfect energy levels using slope parameter ε
Phase 7:	IEnergyOptimizedImage  ← EnergyOptimizedImages (EPerfectEnergyLevels)
	//Apply ${\sigma }_{GN}=\frac{1}{MN}\left({{\displaystyle \sum }}_{i=1}^m{{\displaystyle \sum }}_{j=1}^n{\left|X\ast MASK\right|}_i{}_j\right)$ formula to remove irrelevant energy levels for smooth and non-smooth regions
Phase 7:	if (IenergyOptimizedImage = smooth) then   IOptimizedImage = EdgeBasedSlopeImage (IenergyOptimizedImage) else if (IenergyOptimizedImage = non-smooth) then   IOptimizedImage = EnergyMap (IenergyOptimizedImage)
Phase 8:	Output the IoptimizedImage for further image analysis

. Because misalignment is challenging to correct, accurate registration is necessary when combining MR and CT images. To increase resolution and enable precise image registration, the MR image is first focalized.

Similarly, a gradual mismatch reduction enables precise image fusion. Until convergence is reached, these two processes are repeated. Additionally, it takes into account the hitherto ignored intrinsic correlation of various bands. A novel active slope methodology has been devised to optimize the total energy value in these bands. Backtracking is employed to extract the slope while quick REA iterations are used to effectively solve the subproblems. The REA contains just one non-sensitive argument, unlike conventional variation techniques. Medical image fusion begins with input medical image registration. Without increasing processing complexity, large medical images require trustworthy similarity evaluation. Data is therefore kept in space via REA. A mismatch in the medical imaging would also make slopes analysis worse. As a result, REA is employed in nature to establish similarity [44]. Energy has a role in the simultaneous registration and fusion of molecules.

$$E=\frac{1}{2}{\left|\psi R-\overline{M}-\overline{C}\right|}^2$$

(8)

The terms R, down-sampling process, and E stand for the image energy and energy function optimization of the subproblem image, respectively. The low resolution would cause a significant shift and image blurring because Equation (8) obtains the per-pixel energy cost utilization [46]. The slope extraction method is employed to prevent such a simple fix. Iteratively moving a image in the wrong direction could be effectively avoided by doing this. Equation (9) demonstrates that the objective of this method is to optimize energy. To begin with, the issue is resolved for R as follows:

$$E=\xi \left({\left|\psi R-{\left(M-C\right)}^{’}\right|}^2+{\left|\psi R-{\left(M+C\right)}^{’}\right|}^2\right)$$

(9)

Here, the term in Equation (2)’s first part is smooth, however, the second part is not; $\xi$ is an ideal slope parameter.

2.2.2. DLCNN-based Fusion Network

Applications for segmenting, classifying, and fusing images typically use deep learning. A Fusion-Net architecture based on DLCNN is shown in Figure 3 for fusing several image modalities according to characteristics. The proposed Fusion-Net architecture makes use of convolutional layers: max layering concatenation and two-way SoftMax. Convolution layers are used to extract fine details, and their main job is to perform convolution between the kernel-based weight fusion and the input image patch. Convolution layers applied to MR and CT images are shown in Equations (10) and (11), respectively.

$$F_1=max\left(0,W_1\ast \left(X_{MR}\right)+B_1\right)$$

(10)

$$F_2=max\left(0,W_1\ast \left(X_{CT}\right)+B_1\right)$$

(11)

Here, the Bias function is denoted by B₁ which depends on the rectifier linear unit (ReLU), and W₁ is a matrix based on the kernel containing the most recent weight values. Primarily, the neural network includes 64 feature maps of 16 × 16 size, whereas the convolutional layer kernel consists of 3 × 3 size features. Equation (12) depicts how the ReLU works.

$$ReLU=max\left(0,x\right)$$

(12)

Additionally, the MaxPooling layer receives the information from the convolutional layers as input and precisely extracts each type of feature. The primary objective of the MaxPooling layer is to extract the inter and intra dependencies between MR and CT features. By using both inter and intra dependencies, it is possible to determine how CT affects MRI and vice-versa [46]. Equations (19) and (20) illustrate how the MaxPooling layer operates.

$$F_3=max\left(0,W_2\ast \left(F_1+F_2\right)+B_2\right)$$

(13)

$$F_4=max\left(0,W_2\ast \left(F_2+F_1\right)+B_2\right)$$

(14)

Here, W₂ denotes the matrix based on the kernel containing the recent weight parameters and B₂ denotes a bias function built using a ReLU, respectively. The combined MR and CT features are produced as MaxPooling layer outputs with the letters F₃ and F₄, respectively. Additionally, the kernel in the MaxPooling layer in the second stage has 128 feature maps with a 16 × 16 size, and vice versa.

$$F_R=max\left(0,W_3\ast \left\{F_3,F_4\right\}+B_3\right)$$

(15)

Here, fused feature maps are represented with F_R created by the neural network, The kernel matrix with the most recent weight values is known as W₃, B₃ stands for the bias function based on ReLU, and W₃ stands for the kernel matrix.

In Figure 3a, Convolution layers are used in the developed model to analyse the two-channel image and produce feature maps with change information. The final feature vector for these feature maps is derived using the global average pooling, which helps to lessen the fitting problem. The fully connected layer (FC) then outputs the change category. In Figure 3b, Within the designed model, each branch employs several convolutional layers and global average pooling. The top network, which is made up of FC, receives the two branch outputs after being concatenated. With the top network acting as a classifier and the two branches acting as two feature extractors.

2.3. Proposed Hybrid Fuzzy Segmentation Model

The analysis and segmentation of tumors depend heavily on brain imaging. The intended method for segmenting tumors using HFCMIK is shown in Figure 4. Adaptive cluster index localization, which also introduces similarity matching and the mean characteristic of cluster centers, addresses the problem and maximizes the limitations of conventional k-means clustering. Adaptive k-means clustering (AKMC) is a new technique for adaptive segmentation and cluster will be addressed by fuzzy kernel c means (FKCM) clustering approach.

Two stages make up the HFCMIK algorithm. In the first step, initial centroids are selected using the AKMC approach. Therefore, initial centroids are set for the duration of the process. It lowers the number of iterations required to combine similar things. By choosing a distinct starting centroid, the AKMC algorithm offers local optimum results in the end. This results in the global optimum solution of the AKMC algorithm. For the second phase, On employs the weighted FKCM approach based on Euclidean distance and segments deal with the weights related to each feature value range in the collection of data [46]. A weighted ranking method is used by the upgraded proposed algorithm. Each piece of data’s distance from the origin to the weighted attribute is calculated using the weighted ranking approach. Equation (16) calculates weighted data points.

$${{\displaystyle \mathsf{\cup }}}_{j=1}^n={{\displaystyle \sum }}_{i=1}^n{W}_i{X}_i$$

(16)

where the weightage of input is indicated by $W_i$. Equation (18) shows the format for calculating n numbers of U values for n data points.

$${{\displaystyle \mathsf{\cup }}}_{j=1}^n\left(\begin{array}{c}{U}_1\\ U_2\\ U_n\end{array}\right)$$

(17)

The weighted data points (U_i) are subsequently sorted by their distances. This data is then separated into pixel block sets, where k is the whole quantity of clusters. The center points or the mean value of each group serve as the initial centroids. Consistency among cluster members is caused by the initial centroids of this technique. The proposed weighted ranking method selects beginning centroids using the distance formula since HFCMIK selects group members using distance measurements. The recommended weighted ranking notion is therefore helpful for maximizing the choice of initial centroids. The method’s first phase includes two key components: the distinct initial centroid selection and the processing weights based on attribute values. An image is divided into clusters via the clustering method known as the HFCMIK and the proposed hybrid fuzzy segmentation model is shown in

Table 5. Proposed hybrid fuzzy segmentation model.

INPUT OUTPUT	Fused Medical Images(I) Segmented Medical Image(S)
Step 1	I  ←  FusedImages
	//Initialize the I variable with fused medical images
Step 2	X[I]  ← ImgToVectorConvert (I)
	//Convert all the images I into data vectors
Step 3	U[i]  ←  W[i]*X[i]
	//calculate weighted data points of the image vector I
Step 4	Repeat step 2,3 until all the images are converted to vectors and weighted data points calculated
Step 5	Ccentroid  ←  AKMC (U,K)
	//calculated and identify the initial centroids(cluster centers) with k data points by//Adaptive k-Means Clustering (AKMC)
Step 6	Ssort  ←  WeightedSorting (U)
	//perform the sorting operation on weighted data points U
Step 7	D  ← FKMC (U, Ccentroid)
	//calculate the Euclidian distance from weighted data points (U) to the centroids Ccentroid using fuzzy Kernel c-Means (FKMC) & initialize to distance D
Step 8	CSClusterSegment  ←  FindOptimalCentroid (MIN (D))
	//fetches the optimal centroid with minimum distance and assigned as cluster segment CS
Step 9	Repeat steps 5 to 7 until all clusters traversed
Step 10	S  ← CombineAllSegments (CS)
	//combine all the cluster segments and produce the calculated segment output S

. It re-estimates the segmented output while using centroids to represent its artificially created cluster.

2.4. Proposed Hybrid Feature Extraction

The different lesions can be categorized using a number of the brain tumor’s features that can be retrieved. Several crucial traits, such as low-level features based on RDWT and texture characteristics based on GLCM, features of matrix color, and others, that support the differentiating of brain tumors have been extracted. The spatial relationship between the pixels representing the brain tumors is taken into account using the texture analysis method known as GLCM. The GLCM method computes the texture of the tumor and uses that data to characterize the texture using repeated frequently occurring image combinations with measured value and frequency characteristics that exist in the brain tumor. The approach outlined above can be used to get statistical texture features from the GLCM matrix once it has been created. This probability measure describes the probability that a particular grey level is present near another grey level [46].

$$\mathrm{Contrast}={{\displaystyle \sum }}_{a,b=0}^{N-1}{S}_{a,b}{\left(a-b\right)}^2$$

(18)

$$\mathrm{Homogeneity}={{\displaystyle \sum }}_{a,b=0}^{N-1}\frac{S_{a,b}}{1+{\left(a-b\right)}^2}$$

(19)

$$\mathrm{Correlation}={{\displaystyle \sum }}_{a,b=0}^{N-1}{S}_{a,b}\left[\frac{\left(a-{\mu }_a\right)\left(b-{\mu }_b\right)}{\sqrt{\left({\sigma }_a{}^2\right)\left({\sigma }_b{}^2\right)}}\right]$$

(20)

$$\mathrm{Angular}\mathrm{Second}\mathrm{Moment}\left(\mathrm{ASM}\right)={{\displaystyle \sum }}_{a,b=0}^{N-1}{S}^2{}_{a,b}$$

(21)

$$\mathrm{Energy}=\sqrt{ASM}$$

(22)

Following that, a two-level RDWT is used to extract the low-level features.

The LL1, LH1, HL1, and HH1 bands will be represented by the segmented result when RDWT is initially applied to the outcome. The entropy, energy, as well as correlation parameters, are calculated using the LL band. The result is then effectively obtained as LL2, LH2, HL2, and HH2 while applying RDWT one more time to the LL output band. As illustrated in Figure 5, the LL2 band used to be once more utilized to evaluate the entropy, energy, and correlation characteristics.

The segmented image is then used to obtain statistical color data based on the mean and standard deviation.

$$\mathrm{Mean}\left(\mu \right)=\frac{1}{N^2}{{\displaystyle \sum }}_{i,j=1}^{N}I\left(i,j\right)$$

(23)

$$\mathrm{Standard}\mathrm{Deviation}\left(\sigma \right)=\sqrt{\frac{{{\displaystyle \sum }}_{i,j=1}^N{\left[I\left(i,j\right)-\mu \right]}^2}{N^2}}$$

(24)

Following that, all of these features are integrated using array concatenation, producing a hybrid feature matrix as the output.

2.5. Proposed DLPNN Classification

Deep learning models are playing an increasingly significant role in feature extraction and classification operation. Deep learning-based probabilistic neural network (DLPNN) models may be utilized to extract highly correlated detailed features from the segmented images. The segmented image links between various segments can be identified by this model, and the links can then be extracted as features. Finally, the DLPNN models that used these features during training perform the classification task. Using distinguishing features from upper-level inputs, DLPNN derives more complex features at earlier stages. The DLPNN method for feature classification and extraction is shown in Figure 6. Each layer is closely examined, and statistics on the layer’s size, number of iterations (or kernel size), number of filters, and characteristics are as follows: training samples are 369, the batch size is 90, the number of epochs: 300, number of iterations: 25,000, learning rate: 0.02, Conv2D layer: 2, activation Layer: ReLU activation with frame size: Layer1 {62, 3 × 3, 32}, Layer2 {29, 3 × 3, 64}, MaxPooling 2D layer: 2 with frame size: Layer1 {31, 2 × 2, 32}, Layer2 {14, 2 × 2, 64}, flatten layer: 1 with frame size: Layer1 {1 × 12,544}, dense layer: 2 for activation layer: ReLU activation with frame size: Layer1 {1 × 128}, Layer2 {1 × 21}.

Each layer is closely examined, and statistics on the layer’s size, number of iterations (or kernel size), number of filters, and characteristics are presented. The DLPNN model, which executes the joint classification and feature extraction of brain tumors, is constructed by integrating all of the layers.

3. Results and Discussion

The extensive simulation model of the Fusion-Net deep learning model used for image fusion, which is performed by using the simulation environment of MATLAB R2019a, is presented in this section. The simulations also make use of datasets, and objective and statistical efficiency is evaluated.

3.1. Data Set

The BraTS 2020 dataset (https://www.kaggle.com/datasets/awsaf49/brats20-dataset-training-validation (accessed on 1 July 2022) and the CBICA Image Processing Portal (https://ipp.cbica.upenn.edu/ (accessed on 1 July 2022) is applied to study the efficiency of the designed model. Multimodal brain MR analyses include 369 training, 125 validations, 169 tests, and T1-weighted (T1), T1ce-weighted (T1ce), T2-weighted (T2), and flair sequences. All MR images are 240, 240, 155 pixels. Results showed evaluated each study’s enhancing tumor (ET), peritumoral edema (ED), necrotic, and non-enhancing tumor core (NET). The training sets are annotated for online evaluation and the best classification challenge, but the validation, as well as test sets, are not.

3.2. Fusion Process’s Performance Measurement

MR-CT Fusion approach: the effectiveness of the proposed method for integrating MR and CT medical images is shown, both objectively and as well as subjectively. Additionally, the performance of other standard techniques was compared shown in

Table 6. MR-CT image fusion approaches performance analysis.

	MCA-CS [14]	LTEM [15]	DB-CNN [17]	GF-SDL [18]	LDNSD [19]	Proposed
Entropy	5.394	5.847	6.7983	7.397	7.0488	11.4936
MI	0.4747	0.7973	0.8875	0.936	1.283	1.704
PSNR	24.377	34.394	37.894	40.458	43.569	47.390
SSIM	0.837	0.874	0.896	0.942	0.968	0.997
STD	0.2873	0.138	0.10327	0.0837	0.0675	0.0437

and

Table 7. MR-PET image fusion approaches performance analysis.

Metric	MCA-CS [14]	LTEM [15]	DB-CNN [17]	GF-SDL [18]	LDNSD [19]	Proposed
Entropy	8.191	7.976	8.884	10.365	10.0489	14.4187
MI	1.5672	1.6821	1.9563	2.66	3.28	6.04
PSNR	25.377	31.394	34.894	43.458	46.569	51.39
SSIM	0.87	0.884	0.916	0.942	0.968	1.097
STD	0.3902	0.4188	0.5109	0.683	0.775	0.8437

. Entropy, standard deviation (STD), structural similarity index metric (SSIM), peak signal-to-noise ratio (PSNR), and mutual information (MI) metrics are used to compare performance.

Figure 7a,b show the source CT and MR images, and Figure 7c–h shows the results of various fusion techniques. Figure 7 demonstrates how the Fusion-Net performed better than more traditional methods. The usual approaches struggle to produce superior visual outcomes in this situation due to issues with brightness and saturation. For all quality measures, it can be seen from Table 6 that the proposed approach produced superior quantitative analysis than the conventional approaches, as shown in Figure 8.

MR-PET Fusion approach: the efficiency of the designed approach for fusing MR and PET medical images is shown in this section, both subjectively and objectively. Additionally, the performance of many standard techniques was compared, including MCA-CS [14], LTEM [15], DB-CNN [17], GF-SDL [18], and LDNSD [19], respectively.

Figure 9a,b show the original MR and PET images, and Figure 9c–h show the results of various fusion techniques. Figure 9 demonstrates that when compared to traditional methods the proposed fused output produced the best subjective performance. The various sorts of noise in this situation prevent the standard procedures from producing better visual outcomes. Table 7 shows that for all of the quality measures for color PET images, the proposed approach produced better objective analysis when compared to conventional approaches. MR-PET image fusion techniques for all metrics are shown in Figure 10.

3.3. Proposed Segmentation Method

Figure 11 illustrates how the suggested HFCMIK segmentation strategy enhanced the localization of the tumor tissue in MR, CT, and fused images analyzed with current methodologies. All of the traditional methods localize the tumor region incorrectly. To evaluate different techniques’ performance and analyzed with a proposed method as well as compared in

Table 8. Performance evaluation of various methodologies for segmentation in percentage.

Method	SACC	SSEN	SPEC	SPR	SNPV	SFPR	SFDR	SFNR	SF1	SMCC
U-NET [20]	90.17	90.39	90.53	93.03	90.81	90.64	90.51	94.19	91.38	94.17
CMDFL [22]	91.2	90.67	91.94	93.41	92.53	92.62	90.86	94.82	93.12	95.51
ERV-NET [23]	93.33	94.12	92.59	94.04	93.33	95.93	92.53	97.11	94.97	97.56
BFC-HDA [27]	96.45	98.32	96.65	95.15	95.88	98.78	96.14	98.13	97.9	97.99
HFCMIK	99.76	99.11	98.19	99.59	98.88	98.89	98.99	99.92	99.43	99.93

to cutting-edge. The suggested HFCMIK technique produced higher objective performance, as seen in Table 8. Figure 12 displays a graphic depiction of Table 8.

3.4. Proposed Classification Methodology

Comparison is made with the designed DLPNN classification strategy with existing techniques. Several metrics evaluation parameters are used in this article to compare the performance of various methods.

Table 9. Comparison of different methods’ classification performance in percentage.

Method	CACC	CSEN	CPEC	CPR	CNPV	CFPR	CFDR	CFNR	CF1	CMCC
TL-CNN [33]	93.54	91.45	91.23	92.81	93.67	94.98	90.78	94.87	93.31	91.22
TL-CNN [38]	95.64	95.14	94.36	95.76	96.79	96.42	94.72	95.48	94.45	93.62
FTTL [31]	96.84	96.73	95.44	96.86	97.89	97.84	96.71	96.59	95.26	96.38
GAN-VE [36]	98.96	97.94	97.55	97.73	98.88	98.83	96.99	97.48	97.74	98.81
VGG19 + GRU [46]	97.08	97.77	96.55	95.63	98.25	96.35	97.35	96.89	96.69	98.38
DLPNN	99.91	99.95	98.91	99.57	99.77	99.94	98.86	98.47	98.92	99.91

compares the proposed DLPNN approach’s classification performance to cutting-edge approaches. Table 9 illustrates that the designed DLPNN method exceeded state-of-the-art techniques with there better results. Figure 13 shows a graphical representation of tables.

4. Discussion

Traditionally used models experience considerable computational complexity as dataset sizes increase. It is also necessary to improve the efficiency of traditional techniques in terms of fusion, segmentation, and classification. In the related work, various deep learning models are designed with a great level of complexity. Therefore, in the present study specific models with a distinctive selection of filters, filter widths, stride factor, and layers is proposed. Each of these arrangements results in a distinct framework in the proposed fusion classification model.

Additionally, at present, there is no standard approach for feature extraction, segmentation, fusion, or classification. Additionally, none of these hybrid method combinations are documented in the literature. As a result, a unique BTFSC-Net model including steps for pre-processing, image fusion, image segmentation, and image classification has been designed in this presented research. Regarding the limitations of the proposed method, small medical datasets could be partially or fully overcome by MRI and CT images, which would also aid deep learning models in producing passable results.

5. Conclusions

A segmentation and classification model was constructed in this study along with a hybrid fusion. This model efficiently helps radiologists and medical professionals locate brain tumors more accurately. This technique is effective for computer-assisted brain tumor categorization. This study first used the HPWF filtering method to preprocess the original photos and remove noise. Additionally, a Fusion-Net based on DLCNN was used to combine the two source images with various modalities. The location of the brain tumor was then determined using enhanced segmentation based on HFCMIK. Additionally, using GLCM and RDWT techniques, hybrid features were recovered from the segmented image. The trained characteristics were then utilized to classify benign and malignant tumors using DLPNN. The simulation-based research results showed that the proposed fusion, segmentation, and classification approaches showed an improved performance. The research results also demonstrated that the suggested strategy can be used in real-time applications to classify brain tumors. The outcomes of the present study in comparison to the findings from previous studies demonstrated the potential capability of the proposed framework. The designed approach may be utilized to produce medical images for a variety of medical conditions in addition to brain tumors. Additionally, this study can be expanded to build a categorization system for brain tumors using bio-optimization techniques.