Tetra-ConvBiNet: MRI-based brain tumor classification using distributed tetra head attention-enabled convolutional bidirectional network

The limitations that occurred during the classification of BT in earlier approaches are discussed below.

• In the suggested model [1], the classification accuracy for detecting the BT is improved, but the processing time required is longer.

• The EDN-SVM classifier method [11] considers only the gray-scale images to accurately detect the normal and abnormal tissues present in the brain cells, which limits its applicability across diverse types.

• In [6], the PDCNN method captures both high- and low-level image features, but expanding the dataset is necessary for improved BT cell classification accuracy.

• The EfficientNetB2 method shows better performance with good efficiency, but the training time required is more, which diminishes the speed of detection [2].

• In the developed method [6], only the selected features are utilized by the model, so the classification process depends on the selected region and suppresses the other tissues in the MRI image.

BT causes disability, dreadful illness, and even death to people at an early stage. The DL model with advanced techniques in the medical field enables to diagnosis of BT at an early stage to save human life. But still, the researchers face many challenges in detecting the BT and diagnosing it, because of its complex nature of the brain. The main barriers in the existing models involved lower accuracy, overfitting problems, low classification performance was low, and higher time complexity. Therefore, to detect the disease accurately with more precision and effectively enhance the tumor grading performance, a SEnO-DTCBi Net model is proposed. The input images collected from the datasets is denoted as: (1) D=s1,y,s2,y,…,si,y,…,sm,y D = \left\{ {\left( {{s_1},y} \right),\left( {{s_2},y} \right), \ldots ,\left( {{s_i},y} \right), \ldots ,\left( {{s_m},y} \right)} \right\} where D represents the dataset, s_i implies the input samples of i^th image, s_m implies the total amount of input samples, m denotes the total number of samples, and y expresses the tumor types for the input samples. The samples are passed through the ROI extraction stage, in which specific areas are isolated and forwarded to the segmentation module. This step, executed using OFA-based DW-Net, enhances model efficiency and reduces computational load. The segmented samples are combined with the fused attention mechanism and capture the multiple-scale features. This provides an enhanced feature refinement as well as the localization effect. Furthermore, the segmentation model extracts the segmented regions from the samples and categorizes the different BT cells from the scanned image samples, and addresses the variations correspondingly. Therefore, from the segmented output G, the segmented results are accurately obtained. Furthermore, the extracted feature map Q is given as input to the proposed model for tumor classification. Following feature extraction, the resulting feature maps serve as input to the tumor classification model. The SEnO-DTCBiNet architecture is assessed using the categorical cross-entropy (CCE) loss function throughout the training process, which is mathematically defined as: (2) Loss fun=−∑a=1nbalogUa Loss\;fun = - \sum\limits_{a = 1}^n {{b_a}\log \left( {{U_a}} \right)} where the predicted probability is denoted as U_a, the actual label as b_a, and n indicates the total number of classes. Additionally, the real class labels from the BT dataset [17] are defined as: (3) ba=1,meningioma2,glioma3,pituitary tumor {b_a} = \left\{ {\matrix{ {1,meningioma} \hfill \cr {2,glioma} \hfill \cr {3,pituitary\;tumor} \hfill \cr } } \right.

The actual labels of the Brats2020 dataset [18] and Brats2018 dataset [19] are specified as: (4) ba=1,Necrotic and non−enhancing tumor2,GD enhancing tumor3,peritumoral edema {b_a} = \left\{ {\matrix{ {1,Necrotic\;and\;non - enhancing\;tumor} \hfill \cr {2,GD\;enhancing\;tumor} \hfill \cr {3,peritumoral\;edema} \hfill \cr } } \right.

The input images are collected from the Brats2020 dataset [18], the BT dataset [17], and the Brats2018 dataset [19]. Let us assume the dataset is D, and it is given by, (5) D=si;1≤i≤m D = \left\{ {{s_i}} \right\};1 \le i \le m where s_i denotes the i^th input image sample with a dimension of [240,240,1], and the total number of samples present in an input image is denoted as m.

The collected input image s_i is fed to an ROI extraction phase, which extracts the selected ROI from the images for further processing through a binary thresholding mechanism. The original image is changed into a grayscale image with pixel values ranging from [0,255]. The threshold value T_r is calculated by taking the average value of the grayscale image. For each pixel s_i, the intensity is compared with the threshold value T_r. During comparison, when the pixel intensity value is higher than T_r, then the binary value is denoted as “1,” and that is taken as the interested regions, and if the pixel intensity value is less than T_r, then the binary value will be represented as “0,” and those regions will be eliminated. So, to obtain a clear image, a binary mask is applied that retains the required region. The major purpose of the mechanism is to reduce errors and to enhance classification accuracy. In the ROI extraction phase, the output dimension of the images varies for each slice, and hence, the resized images are formed with a size [512,512,1] that helps the segmentation process to be carried out more smoothly. Thus, the resultant output H is obtained from the ROI extraction, which is further transferred to the segmentation phase.

Segmentation is employed to enhance the precision of BT classification and support more accurate diagnosis. By segmenting the images, the detection process becomes more efficient, and the overall analysis of the medical images is significantly streamlined. Hence, the segmentation process is performed by an OFA-based DW-Net model for identifying the accurate position of the BT. The image obtained from the ROI extraction step serves as the input for the OFA-based DW-Net method [20]. In general, the W-Net structure is considered, where four attention mechanisms are fused, namely, BAM, SE, ULSAM, and CC attention, in a distributed manner to derive the OFA-based DW-Net segmentation model. Additionally, the model’s segmentation component is fine-tuned through an optimization algorithm that operates at the fifth layer of the framework, with detailed insights provided in Section 4.6.

The OFA-based DW-Net segmentation depends on the W-Net model, which reconstructs the input image and predicts the output map without degrading the information. It is composed of two parts encoder and a decoder, with 47 convolutional layers with 18 modules. In the encoder path, the down-sampling process is carried out with the fused Attention mechanisms of BAM, SE, ULSAM, and CC. These fused mechanisms are arranged in a distributed manner, and following this, the up-sampling phase is employed. The context is captured through the utilization of a contacting path, which is presented as Z_enc encoder, the expansive path that is useful for precise localization is known as the decoder Z_dec. The encoder part of the segmentation model takes the input as H having the dimension of dimension (N,512,512,1). Moreover, the encoder module is composed of 2 × 2 convolutional layers followed by a dropout layer and a max pooling layer, such that the dimension at each stage of the down-sampling layer in the encoder phase increases. On the contrary, the dimension is reduced at each layer of the up-sampling layer that lies in the encoder module. The OFA-based DW-Net architecture is depicted in Figure 2.

Figure 2:

OFA-based DW-Net model architecture.

The resultant output obtained at the final dropout layer of the down (N,64,64,512)-sampling process at the encoder side is represented as R, having the dimension of [N,32,32,1024], which is further passed to the fused attention framework in a distributed manner, and the concatenated output derived through the attention model has the size of [N,32,32,4096], respectively. In the up-sampling process, with the transposed H(N,512,512,1) 2D convolutional layer, the feature maps are expanded and combined with the equivalent sizes of the feature map in the down-sampling process by a layer called the skip layers. The skip layers are utilized in up-sampled feature maps for improving the region localization. A SoftMax layer is applied after the final convolutional layer in the encoder path to minimize the normalized cut (Ncut) value.

The Ncut loss function is used to eliminate the segmentation noise and smooth the layer. Therefore, the Ncut value is calculated by a mathematical expression as, (6) NcutjN=∑j=1JcutEj,N−EjassoEj,N Ncu{t_j}\left( N \right) = \sum\limits_{j = 1}^J {{{cut\left( {{E_j},N - {E_j}} \right)} \over {asso\left( {{E_j},N} \right)}}} where N represents the all-pixel groups, and E_j describes the group of pixels in j^th the image. Furthermore, the OFA-based DW-Net model is ready to minimize reconstruction loss for implementing a contracted path and consists of numerous collections of information about the original image. Thus, the reconstruction loss is expressed by a mathematical equation of, (7) ψreconst=si−ZdecZencsi;Benc;Bdec22 {\psi _{reconst}} = \left\| {{s_i} - {Z_{dec}}\left( {{Z_{enc}}\left( {{s_i};{B_{enc}}} \right);{B_{dec}}} \right)} \right\|_2^2 where the parameters of the encoder and decoder are represented by B_enc, and B_dec, respectively, and the original input image is depicted as s_i. Hence, the pixel set values in the selected region are maximized and acquire the regenerated image at the same time, to boost the segmented precision of the image. The main advantage of using the OFA-based DW-Net method is that it enhances the accurate performance of the image segmentation and captures fine details of the features. Along with this computational cost is mitigated with better generalization capability, robustness, and good flexibility. So, these fused attention mechanisms are utilized in this research to provide better performance in disease classification. Based on the segmentation process, nontumor cells are removed, and G represents the segmented tumor cell. Hence, the segmented image’s size is [N,512,512,1], which is further used for the feature extraction process. The brief explanation of BAM, SE, ULSAM, and CC fused attention mechanisms is elaborated below.

c.i

Bottleneck attention mechanism

BAM [21] is widely used with the convolutional network model because of its simple and efficient size. The major purpose of this attention is to increase the representation capability of the model by considering spatial and channel attention. The Channel attention and the spatial attention are the two different branches exist in the channels of the feature map. The input feature is taken as R ∈ ℝ^c×h×w, and a 3D attention map for BAM is noted as A(R) ∈ ℝ^c×h×w. Figure 3 represents the structure of BAM.

Figure 3:

Bottleneck attention mechanism.

In the channel attention branch, to calculate a feature map, average pooling layer is taken, which encodes the information present in each channel. For determining the attention, a multilayer perceptron is used in a hidden layer, and a batch normalization layer is also used for adjusting scale parameters. Hence, the channel attention is mathematically expressed as: (8) AcR=BaMIApR {A_c}\left( R \right) = Ba\left( {MI\left( {Ap\left( R \right)} \right)} \right) where Ap(R) implies average pooling, Ml is the multilayer perceptron, Ba denotes Batch normalization, and A_c(R) denotes channel attention.

Spatial attention is employed to filter out features distributed across different spatial positions. This module utilizes two key hyperparameters: the dilation rate, which helps in capturing contextual information, and the reduction ratio, which manages channel capacity and controls computational overhead. The feature is first reduced using (1× 1) convolution, and then (3 × 3) dilated convolutions are used, and finally, the reduced features are again convoluted using (1× 1) batch normalization in the spatial branch output for scale adjustment. Therefore, spatial attention is represented as, (9) AsR=Baf31×1f23×3f13×3f01×1R {A_s}\left( R \right) = Ba\left( {f_3^{1 \times 1}\left( {f_2^{3 \times 3}\left( {f_1^{3 \times 3}\left( {f_0^{1 \times 1}\left( R \right)} \right)} \right)} \right)} \right) where f represents a convolution operation, Ba implying the batch normalization, A_s(R) and denotes the spatial attention. To obtain an efficient module, channel attention is added with the spatial attention along with the sigmoid function, and then the attention map is calculated as: (10) AR=σAcR+AsR A\left( R \right) = \sigma \left( {{A_c}\left( R \right) + {A_s}\left( R \right)} \right)

The sigmoid function is denoted by σ, spatial attention is expressed by A_s(R), and channel attention is denoted by A_c(R). Thus, the BAM output R₁ is calculated by adding the original input feature with the attention module, and it is mathematically expressed as, (11) R1=R+R⊗AR {R_1} = R + R \otimes A\left( R \right) where ⊗ express the element-wise multiplication. Thus, the BAM model effectively enhances the model performance and mitigates the overhead and computational complexity problems.

c.ii

SE attention

The working principle of SE attention [22] depends on the convolution network model, and the spatial dependencies are figured out by global average pooling. The SE method in image segmentation provides more pixel-wise information about the spatial blocks, which is more accurate. Assume the input of SE as R = {r₁, r₂ ,..., r_n} with the number of channels, such that r_k = ℝ^h×w, where spatial height and spatial height are indicated as h and w, respectively. The working mechanism of SE is based on two different frameworks, namely the spatial framework and channel framework. Hence, the spatial model employs global average pooling as a squeeze operation. This generates a vector by summarizing the spatial information of each feature map q, and it is represented by: (12) q=1h×wrki,j q = {1 \over {h \times w}}{r_k}\left( {i,j} \right)

Furthermore, the vector q is processed through two fully connected (FC) layers with the ReLU operator. Finally, the resultant factor of the FC layer is given to the sigmoid layer to generate the output σq^ \sigma \left( {\hat q} \right) . (13) R^cSE=σq1^r1,σq2^r2,…,σq^rn {\hat R_{cSE}} = \left[ {\sigma \left( {\widehat {{q_1}}} \right){r_1},\sigma \left( {\widehat {{q_2}}} \right){r_2}, \ldots ,\sigma \left( {\hat q} \right){r_n}} \right]

The channel model performs channel squeeze operations using a convolutional layer, its output is denoted as c, and it can be represented by c = R × w_t, where w_t denotes the weight factor, and c implies the output of the projection sensor. Each projection tensor is further fed to the sigmoid layer to retrieve the output as σ(c), and the channel model output is represented by: (14) R^sSE=σc1r1,σc2r2,…,σcnrn {\hat R_{sSE}} = \left[ {\sigma \left( {{c_1}} \right){r_1},\sigma \left( {{c_2}} \right){r_2}, \ldots ,\sigma \left( {{c_n}} \right){r_n}} \right]

Also, the irrelevant features are reduced, and important spatial locations are delivered by the recalibration. The spatial, channel, squeeze, and excitation components are integrated into a single block by concatenating the outputs from both channels. (15) R2=conRcSE^,RsSE^ {R_2} = con\left( {\widehat {{R_{cSE}}},\widehat {{R_{sSE}}}} \right)

This combined network of the feature map provides both channel-wise and spatial information, segments the important features, reduces the complexity problems, and the resultant output will be denoted as R₂. Figure 4 represents the SE structure.

Figure 4:

SE attention. SE, squeeze and excitation.

c.iii

Ultralightweight subspace attention mechanism

The structure of ULSAM [23] is designed by considering a convolution and pooling layer such that it learns an attention map in each feature subspace, which minimizes the channel and spatial redundancy. Since the attention map was learned by the model, it enhances the image segmentation and efficiently learns the cross-channel feature map interdependencies. Let us consider a feature map R ∈ ℜ^b×h×w from the dropout layer, b be the number of input channels, and h w represents the dimension of the spatial map. The major goal is to capture features of cross-channel interdependencies effectively in the feature map. The feature map R is divided into d groups, and each group has a feature map D [R₁,R₂ ,...,R_n ,...,R_d]. In ULSAM, the features are extracted through a max pooling layer, a depthwise convolution layer, and a point-wise convolution layer with a feature map R_n. The structure of ULSAM is depicted in Figure 5.

Figure 5:

Structure of ULSAM.

An attention map Att_m captures each group’s nonlinear dependencies to capture the cross-channel information in a feature map. To verify weighing attention, a gating function is utilized with the SoftMax activation function and refines the feature map. The output R₃ obtained in the feature map is obtained by joining the feature maps of every group. (16) Attnn=softmaxPCMPDCR Att{n_n}\; = soft\max \left( {PC\left( {MP\left( {DC\left( R \right)} \right)} \right)} \right) (17) R^n=Attnn⊗Rn⊕Rn {\hat R_n} = \left( {Att{n_n} \otimes {R_n}} \right) \oplus {R_n} (18) R3=concatR^1,R^2,…,R^n,…R^d {R_3} = concat\left( {{{\hat R}_1},{{\hat R}_2}, \ldots ,{{\hat R}_n}, \ldots {{\hat R}_d}} \right) where soft max denotes the SoftMax activation, PC signifies point-wise convolution, MP represents a max pooling layer, DC shows depth-wise convolution, Attn_n is the attention map in each group, the feature map at each layer is shown as R^n {\hat R_n} , defines the refined feature map, and ⊕ implies the element-wise addition.

c.iv

CC attention

The CC attention [24] model is mainly used to capture the contextual information from the images in an optimal way. Because of its lightweight computation and more memory capability to define the local feature representations. This attention mechanism collects the vertical and horizontal contextual information to improve the pixel-wise explanation. The structure of CC attention is presented in Figure 6.

Figure 6:

Structural diagram of CC. CC, criss-cross.

Consider a local feature map R ∈ ℝ^c×w×h, which is fed into convolution layers to generate the feature maps as, J, L, and X. {J, L} ∈ ℝ^c′×w×h, c′ denotes the number of channels that exist. An attention map is generated after acquiring the feature maps J, L through an affinity operation. At each position τ in the spatial dimension, feature vectors generated from J and L are indicated as Jτ and Lτ, respectively. Hence, the affinity operation is expressed as A_τ = J_τ L_τ, which refers to the degree of correlation among feature vectors J_τ and L_τ. Furthermore, a SoftMax operation is applied to A_τ compute the attention map AM. Moreover, a convolution layer with [1× 1] a filter is applied over the input feature to create X for feature adaptation process, and the respective feature vector is denoted as X_τ. (19) R4=∑AMτxτ+R {R_4} = \sum {A{M_\tau }{x_\tau } + R} where AM_τ is the attention map’s output, and R₄ represents the output of the CC attention model. Thus, the CC attention captures the contextual information pixel-wise. Therefore, the total fused attention mechanism output is concatenated, and the resultant output Q is passed as an input to the feature extraction phase. (20) Q=R1‖R2‖R3‖R4 Q = \left\{ {{R_1}||{R_2}||{R_3}||{R_4}} \right\}

The MSMSD-based feature extraction strategy incorporates multiple feature types, including statistical deep flow features, GLCM-based features, and 3D structural pattern features. This approach helps minimize computational complexity and enhances the model’s accuracy.

Hybrid structural pattern features encompass a range of descriptors, including local binary pattern (LBP), local ternary pattern (LTP), and local gradient pattern (LGP).

The effective features from the images are acquired using the statistical approach known as LBP [25]. Therefore, it considers only the local features of the ROI extracted images, and it is mathematically expressed as: (21) LBPQ=∑αPz−Px2z LBP\left( Q \right) = \sum {\alpha \left( {{P_z} - {P_x}} \right){2^z}} where LBP(Q) denotes the features of LBP with the dimension [N,120,120,1], P_x represents the center pixel values, and P_z implies the z^th neighbor pixels. The LTP is a three-valued texture feature in determining the pixel values α denotes the constant. The extension of LBP is called the LTP and assigns the values from −1 to 1. Therefore, the expression of LTP is denoted by: (22) LTPQ=1 if Pz−Px>η0 ifPz−Px≤η−1 if Pz−Px<−η LTP\left( Q \right) = \left\{ {\matrix{ {1\;\;\;if\;{P_z} - {P_x} > \eta } \hfill \cr {0\;\;\;if\left| {{P_z} - {P_x}} \right| \le \eta } \hfill \cr { - 1\;\;\;if\;{P_z} - {P_x} < - \eta } \hfill \cr } } \right. where the features of LTP with the dimension [N,120,120,1] is represented by LTP(Q), and η is defined as the threshold value. LTP feature extraction method is utilized because of its classification accuracy and for extracting the texture features of an image. The LGP uses the adjacent pixel gradient in the quantified pixel values. The LGP method [26] enhances the interpretability of the model when extracting the features for classification. The expression for LGP is mathematically denoted by: (23) LGPQ=∑αvn−v¯2n LGP\left( Q \right) = \sum {\alpha \left( {{v_n} - \bar v} \right){2^n}} where LGP(Q) implies the features of LGP with the dimension [N,120,120,1], v¯ \bar v denotes the mean gradient, and v_n expresses the central pixels and the adjacent pixels’ average value. Therefore, the hybrid 3D structural pattern-based features are represented by concatenating all the extracted features. (24) β=LGPQLBPQLTPQ \beta = \left\{ {LGP\left( Q \right)\left\| {LBP\left( Q \right)} \right\|LTP\left( Q \right)} \right\}

β represents the extracted hybrid 3D structural pattern-based features with dimension [N × 120 × 120 × 3].

Statistical deep flow-based features are used to isolate the ROI from the extracted image, allowing the generation of a deep flow feature map. The statistical features [27] improve the performance of classification accuracy and make the model more reliable. The statistical deep flow feature uses the “cv2.calcOpticalFlowPyrLK” package to acquire the deep flow feature, and it is mathematically denoted as φ_i. As a result, the statistical features derived include mean, standard deviation, variance, skewness, and kurtosis. Table 1 demonstrates the characteristics obtained from the deep flow feature representation.

Table 1:

Features	Description	Mathematical notation	Output size
Mean	It is the ratio between the total intensity of all pixels and the total number of pixels within the deep flow feature image.	T1=1r∑i=1rφi {T_1} = {1 \over r}\sum\limits_{i = 1}^r {{\varphi _i}}	[N,120,120,1]
		where φi is the feature and r the total images.
Kurtosis	Kurtosis is defined as the shape of the selected images taken for the statistical measurement.	T2=∑T1−T1¯4rT3 {T_2} = {{\sum {{{\left( {{T_1} - \overline {{T_1}} } \right)}^4}} } \over {r\left( {{T_3}} \right)}}	[N,120,120,1]
		T2 is the kurtosis
Standard deviation	The standard deviation is defined as the square root of the variance and represents the average deviation of each pixel intensity from the mean.	T3=1r∑i=1rφi−T12 {T_3} = \sqrt {{1 \over r}\sum\limits_{i = 1}^r {{{\left( {{\varphi _i} - {T_1}} \right)}^2}} }	[N,120,120,1]
		standard deviation is denoted as T3
Skew	Skewness is defined as a measure of symmetry of the image.	T4=1r∑i=1rφi−T1T3 {T_4} = {1 \over r}\sum\limits_{i = 1}^r {\left[ {{{{\varphi _i} - {T_1}} \over {{T_3}}}} \right]}	[N,120,120,1]
		T4 Is the skew
Variance	Variance is defined as the square of the standard deviation.	T5=T32=1r∑i=1rφi−T12 {T_5} = {\left( {{T_3}} \right)^2} = {1 \over r}\sum\limits_{i = 1}^r {{{\left( {{\varphi _i} - {T_1}} \right)}^2}}	[N,120,120,1]
		T5 Is the variance

Eq. (25) demonstrates the statistical deep-flow-based features with the dimensionality [N,120,120,5]. (25) θ=T1‖T2‖T3‖T4‖T5 \theta = \left\{ {{T_1}||{T_2}||{T_3}||{T_4}||{T_5}} \right\}

GLCM [28] method is preferred because of its accuracy, and the computing time for extracting the images is shorter. It consists of entropy, energy, contrast, homogeneity, and dissimilarity. In Table 2, the features of GLCM are tabulated below.

Table 2:

Features	Overview	Formula	Dimensions of outputs
Homogeneity	Homogeneity calculates the similarity of the texture in the distributed gray-level object pairs.	E3=∑kln−1Mkl1+k−l {E_3} = \sum\limits_{kl}^{n - 1} {{{{M_{kl}}} \over {1 + \left| {k - l} \right|}}}	[N,120,120,1]
		homogeneity feature is represented as E3.
Energy	Energy is used to calculate the uniformity of an image.	E1=∑kln−1Mkl2 {E_1} = \sum\limits_{kl}^{n - 1} {{{\left( {{M_{kl}}} \right)}^2}}	[N,120,120,1]
		E1 It is the energy feature, and GLCM of the image Q is denoted as Mkl.
Entropy	Entropy reproduces the complexity of an image present in the GLCM features.	E4=∑kln−1Mkllog2Mkl {E_4} = \sum\limits_{kl}^{n - 1} {{M_{kl}}{{\log }_2}\left( {{M_{kl}}} \right)}	[N,120,120,1]
		E4 Represents the entropy.
Contrast	Contrast calculates the local variation amounts present in the image.	E5=∑kln−1M(k−l)2 {E_5} = \sum\limits_{kl}^{n - 1} {M{{(k - l)}^2}}	[N,120,120,1]
		E5 Depicts the contrast.
Dissimilarity	It measures the gaps between the mean variances and ROI in the gray-scale image	E2=∑kln−1Mklk−l {E_2} = \sum\limits_{kl}^{n - 1} {{M_{kl}}\left| {k - l} \right|}	[N,120,120,1]
		E2 Denotes the dissimilarity feature of GLCM.

ROI, region of interest.

The equation below represents the extracted GLCM features: (26) μ=E1‖E2‖E3‖E4‖E5 \mu = \left\{ {{E_1}||{E_2}||{E_3}||{E_4}||{E_5}} \right\} where μ denotes the GLCM features by containing a dimension of [N,120,120,5]. Thus, the MSMSD-based features are obtained by concatenating all the features, and it is established as: (27) U=β‖θ‖μ U = \left\{ {\beta ||\theta ||\mu } \right\}

The extracted MSMSD-based features are specified as U with dimension [N,120,120,5]. These are transferred to the distributed tetra head attention-based convolutional bidirectional network (DTCBiNet) model to classify the BTs.

The DTCBiNet model is the integration of a convolutional and BiLSTM network, and it is used for the classification of BT. The DTCBiNet model comprises different layers, such as an input layer, a convolutional layer, a Leaky ReLU layer, a max pooling layer, a Dropout layer, a Reshape layer, an attention mechanism layer, a concatenation layer, a BiLSTM layer, and a Dense layer, respectively. This DTCBiNet model helps to extract the complex patterns in the feature and boosts the performance of the model with better classification accuracy. Along with this, the DTCBiNet model extracts both the spatial and the temporal relationships of the features and enhances the model’s performance. The introduction of an attention mechanism in the model mitigates redundancy factors and effectively maximizes the learning process.

In this DTCBiNet model, the initially extracted image U is passed as input to the model, with [N,120,120,13] dimension, the image fed to the convolutional 2D layer. This layer extracts more features, and it acts as the main component in the DTCBiNet model. In the first convolutional layer, low-level features are first extracted, and high-level features are transferred to further convolutional layers. The mathematical equation of a convolutional layer is represented by, (28) J1=∑L⊗fi+bi {J_1} = \sum {L \otimes fi + bi} where the feature map is indicated by J, the bias is denoted by bi, and the convolutional filter is depicted by fi. The feature map J output is passed to the max pooling layer, and if the feature map gained more output from the convolutional layer, then it cannot be applied to the training directly. This causes more issues in overfitting problems and leads to more consumption of time, so down-sampling is used to minimize the parameters to reduce the size of the max pooling to produce an effective outcome. The size is reduced by the dimension of [N,60,60,16], after the maxpooling convolutional operation, and it is transferred to a second convolutional 2D network to extract the features, and it is represented by J₂. The output from another convolutional layer is passed to an activation function, which is called Leaky ReLU. This activation function is utilized because of its low computation time and simplicity. The Leaky ReLU is represented by, (29) LRLJ2=max0,J2+υ×min0,J2 LRL\left( {{J_2}} \right) = \max \left\{ {0,{J_2}} \right\} + \upsilon \times \min \left\{ {0,{J_2}} \right\} where LRL(J₂) is the Leaky ReLU activation function, υ implies the parameter of the Leaky ReLU function, and has a dimension of [N,60,60,64]. Next, output from the Leaky ReLU is transferred to a next max pooling layer, followed by a Dropout layer with a dimension of [N,30,30,64]. Consequently, the output from the dropout layer with size [N,30,30,64] gets reshaped to the size of [N,900,64], and it can be depicted as d, which is transferred to the multi-head attention mechanisms. This MA mechanism is a combination of various attention mechanisms and gains a corresponding input feature. The combination of BAM, SE, ULSAM, and the CC mechanism is obtained by concatenating the features of all mechanisms, and they are elaborated in Section 4.3 with a detailed explanation and diagram. And these concatenated mechanisms produce an output with a dimension of [N,900,256]. The BiLSTM consists of an input layer, a bidirectional layer, and an output layer, which boosts the model’s performance and captures both the bidirectional features of the model. The bidirectional layer included both forward and backward layers, in which the forward layer extracts forward features from the input and the backward layer extracts backward features. Then, both layers are concatenated to produce the output. The mathematical expression for representing these BiLSTM layers is: (30) fl→=σwtsV,wx−1→+bi \overrightarrow {fl} = \sigma \left[ {wts\left( {V,\overrightarrow {{w_{x - 1}}} } \right) + bi} \right] (31) bf←=σwtsV,wx−1←+bi \overleftarrow {bf} = \sigma \left[ {wts\left( {V,\overleftarrow {{w_{x - 1}}} } \right) + bi} \right] where fl→,bf← \overrightarrow {fl} ,\overleftarrow {bf} denotes the forward and backward layer outputs, σ implies the sigmoid function, wts and bi denotes the model’s weight and bias, wx−1,→wx−1← \overrightarrow {{w_{x - 1}},} \overleftarrow {{w_{x - 1}}} denotes the forward hidden and the backward hidden layer, and the output is represented as: (32) K=fl→⊕bf← K = \left[ {\overrightarrow {fl} \oplus \overleftarrow {bf} } \right] where BiLSTM layer output is represented as K, and ⊕ denotes the addition operation, and then the output is transferred to the next BiLSTM layer, with the size of [N,128] is passed to a dense layer. The dense layer has a dimension [N × 64] and is then transferred to the next dense layer with a reduced dimension [N,32]. Finally dense layer produces the output with the size of [N,3]. Finally, output is categorized into three labels as glioma, pituitary tumor, and meningioma, and the output is mathematically expressed as U_a. The biases and weights of the DTCBiNet model were optimized by the SEnO algorithm, and it is discussed in Section 4.6 section, the optimization is applied after the result of the BiLSTM layer. The DTCBiNet model architecture is depicted in Figure 7.

Figure 7:

Architecture of DTCBiNet. DTCBiNet, distributed tetra head attention-based convolutional bidirectional network.

The SEnO algorithm is obtained by integrating the bat optimization [29], the echolocation-based Dolphin optimization [30] algorithm, and the Sparrow Search Algorithm (SSA) [31]. The SEnO algorithm is employed to optimize the bias and weight of the DTCBiNet model by applying the optimization at the end of the BiLSTM layer. On the contrary, for training the segmentation approach, on the fifth layer, the optimization is integrated to improve the performance of the model.

f.i

Motivation

For finding prey and to navigate by listening to the echoes in the surroundings, the bat uses the echolocation behavior. In dark situations, it avoids the hindrance problem. Likewise, the dolphin utilizes echolocation by generating the sound, which is in the form of clicks, to locate the obstacles and its prey. The SEnO algorithm rapidly improves the model performance and reduces the overfitting challenges. The development of the SEnO algorithm in the model executes with better escaping, grouping, and alarming strategies and enhances the overall model mechanisms.

a)

Initialization

In the initialization phase, first consider the population with a number of solutions h, and the total number of solutions t. Algorithm’s initial population is mathematically expressed as: (33) F=F1⋮Fi⋮Fh=F1,1,⋯F1,q⋯F1,t⋮⋮⋮Fi,1⋯Fi,q⋯Fi,t⋮⋮⋮Fh,1⋯Fh,q⋯Fh,th×t F = \left[ {\matrix{ {{F_1}} \hfill \cr \vdots \hfill \cr {{F_i}} \hfill \cr \vdots \hfill \cr {{F_h}} \hfill \cr } } \right] = {\left[ {\matrix{ {{F_{1,1}},} & \cdots & {{F_{1,q}}} & \cdots & {{F_{1,t}}} \cr \vdots & {} & \vdots & {} & \vdots \cr {{F_{i,1}}} & \cdots & {{F_{i,q}}} & \cdots & {{F_{i,t}}} \cr \vdots & {} & \vdots & {} & \vdots \cr {{F_{h,1}}} & \cdots & {{F_{h,q}}} & \cdots & {{F_{h,t}}} \cr } } \right]_{h \times t}} where F_i,q implies the location of the i^th solution in the q^th dimension. (34) Fi,q=Upi,q+e1Lwi,q−Upi,q i=1,2,…,hq=1,2,…,t {F_{i,q}} = U{p_{i,q}} + {e_1}\left( {L{w_{i,q}} - U{p_{i,q}}} \right)\;\;\;\left\{ {\matrix{ {i = 1,2, \ldots ,h} \hfill \cr {q = 1,2, \ldots ,t} \hfill \cr } } \right. where the number of dimensions is denoted as t, the number of solutions is expressed as i in the search space, h refers total number of solutions, and F_i,q denotes the first position of the i^th solution in q^th the dimension. e₁ Refers to a random number within the interval of (0,1). The q^th search space upper bound is denoted as Up_i,q and the lower bound is represented as Lw_i,q.

b)

Calculation of fitness function

Fitness function of the SEnO algorithm is computed depending on the accuracy measure, such that the model’s performance is observed by attaining maximum accuracy, which is represented by: (35) Fitn=f1⋮fi⋮fhh×1=fF1⋮fFi⋮fFhh×1 Fitn = {\left[ {\matrix{ {{f_1}} \hfill \cr \vdots \hfill \cr {{f_i}} \hfill \cr \vdots \hfill \cr {{f_h}} \hfill \cr } } \right]_{h \times 1}} = {\left[ {\matrix{ {f\left( {{F_1}} \right)} \hfill \cr \vdots \hfill \cr {f\left( {{F_i}} \right)} \hfill \cr \vdots \hfill \cr {f\left( {{F_h}} \right)} \hfill \cr } } \right]_{h \times 1}} where Fitn denotes the vector obtained from the objective function, and f_i represents the objective value obtained from the i^th solution.

c)

Enhancement parameter consideration

The SEnO algorithm moves toward the optimal solution for obtaining a better solution with two decision factors, and it can be expressed as: (36) Mit+1=κMit M_i^{\left( {t + 1} \right)} = \kappa M_i^{\left( t \right)} (37) Vit+1=Vi01−exp−λt V_i^{\left( {t + 1} \right)} = V_i^{\left( 0 \right)}\left[ {1 - \exp \left| { - \lambda t} \right|} \right] where Mit+1 Mit M_i^{\left( {t + 1} \right)}\;M_i^{\left( t \right)} represents the learning rate factor at (t + 1)^th t^th iterations, Vit+1 V_i^{t + 1} shows pulse emission at (t + 1)^th iteration, Vi0 V_i^{\left( 0 \right)} represents the first pulse emission, κ,λ are the parameters that are used to improve the decision, and they fall in the range of [0 < κ < 1],[0 < λ < 1].

• Case (i): Exploration Phase: When Ra<Vit \left( {Ra < V_i^t} \right) Whenever the random variable ranges among [0,1], this criterion occurs, the exploration phase will be implemented.

• Subcase (i): Wide search based on Velocity: If (S < 0.5), when the random values fall in the middle of [0,1] and <0.5, this case is considered. The wide region searching capability is improved to the extended global search and making the solution wider to find the best optimal solution. This uses a velocity factor to speed up the exploration phase to find the best regions correspondingly. Therefore, the characteristics of a positional update can be expressed as: (38) Fit+1=veit+1+Fit F_i^{\left( {t + 1} \right)} = ve_i^{\left( {t + 1} \right)} + F_i^{\left( t \right)} where the previous iteration solution’s position is denoted as Fit+1 F_i^{\left( {t + 1} \right)} , the current iteration solution position is denoted as Fit F_i^t and the wide search parameter is denoted by: (39) Figt=Fi,qtexp−iκ⋅tmax F_{ig}^t = F_{i,q}^t\exp \left( {{{ - i} \over {\kappa \cdot t\max }}} \right) where κ is the constant parameter, and the velocity parameter can be expressed as: (40) veit+1=veit+Fit+1−Fbestt⋅fr ve_i^{\left( {t + 1} \right)} = ve_i^{\left( t \right)} + \left( {F_i^{\left( {t + 1} \right)} - F_{best}^{\left( t \right)}} \right) \cdot fr where fr represents the frequency parameter, and it can be expressed as: (41) fr=frmin+frmax−frminρ fr = f{r_{\min }} + \left( {f{r_{\max }} - f{r_{\min }}} \right)\rho where ρ ∈ (0,1) it is a random vector which is taken from the uniform distribution. fr_min,fr_max Is the minimum and maximum frequency of pulse emissions. Subcase (ii): Random Search: If Ra ≥ 0.5 Here, search variables fall among the values (0, 1), where it is higher than or the same value as 0.5, then this case will be implemented. The updated position will be mathematically expressed as: (42) Fit+1=Frandt−Dis⋅w F_i^{\left( {t + 1} \right)} = F_{rand}^{\left( t \right)} - Dis \cdot w where Dis refers to the distance between the target and solution, and w denotes the iteration period. Therefore, the iterations and the distance are represented as: (43) Dis=2Φ⋅Frandt−Fit Dis = \left| {2\Phi \cdot F_{rand}^{\left( t \right)} - F_i^{\left( t \right)}} \right| (44) w=21−yymax w = 2\left( {1 - {y \over {{y_{\max }}}}} \right) where the random integer with the value of [0,1] is expressed as Φ, and the maximum number of iterations and the current iteration are represented as y, y_max. Furthermore, the updated new position equation is denoted by: (45) Fit+1=κ1Re*Re−zFitFi+Fit+1−κFrandt⋅Dis w F_i^{t + 1} = \kappa \left[ {{1 \over {{\mathop{\rm Re}\nolimits} }}*\left( {{\mathop{\rm Re}\nolimits} \; - \left| z \right|} \right)Fit\left( {{F_i}} \right) + F_i^{\left( t \right)}} \right] + \left( {1 - \kappa } \right)\left[ {F_{rand}^{\left( t \right)} \cdot Dis\;w} \right] where κ act as a hybridization factor, which is in the range of (0, 1), Re is the alternative index. By the above Eq. (43), the positions are updated, and it traverses the current search space, which improves the chance of getting Persistent local optima are reduced.

  explores the new search space, and the chances of getting stuck in the local optima are minimized.

• Case (ii)-Exploitation Phase: If Pa≥Fit \left( {Pa \ge F_i^t} \right) , this phase will be introduced where the randomly created value is superior to or equal to pulse emissions. The updation of the solution depends on the average loudness and the randomness of the solution. The slower convergence rate and the exploitation phase issues are overcome by generating a new solution and which is represented by: (46) Fit+1=∂Fit+ϖ⋅Avgt + 1−∂Foptt−Fit⋅cos2πR5+Foptt \matrix{ {F_i^{\left( {t + 1} \right)} = \partial \left[ {F_i^{\left( t \right)} + \varpi \cdot Av{g^{\left( t \right)}}} \right]} \hfill \cr {\;\;\;\;\;\;\;\;\;\; + \;\left( {1 - \partial } \right)\left[ {\left| {F_{opt}^{\left( t \right)} - F_i^{\left( t \right)}} \right| \cdot {{\cos}}\left( {2\pi {R_5}} \right) + F_{opt}^{\left( t \right)}} \right]} \hfill \cr } where Avg^(t) implies the average of all learning rate parameters, ∂ denotes the hybridization parameter in the range of [0,1], ϖ denotes the random integer in between the value of [−1, 1], Foptt F_{opt}^{\left( t \right)} is the current iteration optimal solution, and R₅ denotes the arbitrary integer with the value of −1 and 1. Eq. (44) can be updated through the quick movement by considering the random variable that is greater than the threshold value, which enables the solution to quickly reach the optimal region of the exploitation phase. Therefore, the updated solution can be mathematically represented by, (47) Fit+1=Fit+A⋅B+∂Fit+ϖAvgt + 1−∂Foptt−Fit⋅cos2πR5+Foptt \matrix{ {F_i^{\left( {t + 1} \right)} = F_i^t + {\rm{A}} \cdot {\rm{B}} + \partial \left[ {F_i^t + \varpi Av{g^{\left( t \right)}}} \right]} \hfill \cr {\;\;\;\;\;\;\;\;\;\; + \;\left( {1 - \partial } \right)\left[ {\left| {F_{opt}^{\left( t \right)} - F_i^{\left( t \right)}} \right|} \right] \cdot \cos \left( {2\pi {R_5}} \right) + F_{opt}^{\left( t \right)}} \hfill \cr } where A denotes a random number that obeys the normal distribution, and B represents a matrix, and each element inside is 1. Therefore, the introduction of the quick movement factor within the exploitation phase enables the solution to find a better optimal region. Hence, the SEnO algorithm finds the best solution by searching and exploring the location. Thus, the computational complexity and the best solution delivered within the minimum time by the algorithm.

d)

Termination

The SEnO algorithm halts execution once a predefined condition is met, indicating convergence, and it then returns the most optimal solution. The flowchart outlining the SEnO algorithm’s steps is shown in Figure 8.

Figure 8:

The flowchart of the SEnO algorithm.

The SEnO-DTCBiNet framework was developed using Python 3.7 on a Windows 11 platform. The system setup includes a 13th-generation Intel Core i7-13770K processor, 16 GB of RAM, and an Nvidia GeForce RTX 3080 Ti GPU with 12 GB of dedicated memory. Development was carried out using the PyCharm Community Edition, with storage handled by a 128 GB ROM. The hyperparameters used by the SEnO-DTCBiNet model for classification are displayed in Table 3.

Datasets included in the research are described below:

• BT dataset [17]: It consists of three different kinds of BT with 3064 images. Moreover, the collected images are split into four subgroups, and they are stored in 4.zip files, respectively.

• BraTS 2020 dataset [18]: It comprises MR images from 369 patients, and four types of MR images were gathered from each patient for the training set. There are four kinds of images present in this dataset.

• Moreover, the validation set consists of MR images of 125 patients, and these images are collected from the various 19 institutes with several clinical protocols.

BraTS 2018 dataset [19], the training set comprises 285 patients; among them, 210 patients have high-grade glioma and 75 patients have low-grade glioma. Likewise, the validation set includes 66 patients with unknown-grade BTs. The dataset visualization is illustrated in Figure 9.

Figure 9:

Dataset visualization. (A) BraTS 2020 dataset, (B) BT dataset, and (C) BraTS 2018 dataset. BT, brain tumor.

Precision: It measures the percentage of the SEnO-DTCBiNet model’s all positive classifications, and it is denoted as: (48) pre=tepsteps+flps pre = {{t{e_{ps}}} \over {t{e_{ps}} + f{l_{ps}}}} where pre indicates the precision measure, the true positive is represented as te_ps, and the false positive is denoted as fl_ps.

Recall: It measures the percentage of all positives, which are classified correctly by the SEnO-DTCBiNet model as positive, and it is represented as: (49) re=tepsteps+flng re = {{t{e_{ps}}} \over {t{e_{ps}} + f{l_{ng}}}} where fl_ng is the false negative, and re indicates the recall measure.

Accuracy: It measures the SEnO-DTCBiNet model’s overall correctness in classifying data, and it is represented as: (50) acc=teps+tengteps+teng+flps+flng acc = {{t{e_{ps}} + t{e_{ng}}} \over {t{e_{ps}} + t{e_{ng}} + f{l_{ps}} + f{l_{ng}}}} where accuracy is denoted as acc, and the true negative is presented as te_ng.

Fi-score: It estimates the performance of the SEnO-DTCBiNet model by the harmonic mean of recall and precision, which is denoted as: (51) F1=2×pre×repre+re F1 = 2 \times {{pre \times re} \over {pre + re}} where F1-measure is signified as F1.

The image results of the SEnO-DTCBiNet model across the BraTS 2018 dataset, BraTS 2020 dataset, and the BT dataset are represented in Figures 10–12, respectively.

Figure 10:

Image results based on the BraTS 2020 dataset. ROI, region of interest.

Figure 11:

BT dataset-based image results. BT, brain tumor; ROI, region of interest.

Figure 12:

BraTS 2018 dataset-based image results. ROI, region of interest.

The model performance across Brats2020, BT dataset, and the Brats2018 for different amounts of training data and epochs is demonstrated in this section.

e.i

BraTS 2020 dataset-based performance analysis

Figure 13 represents the SEnO-DTCBiNet model performance on the BraTS 2020 dataset, assessed using training data. The precision value of the SEnO-DTCBiNet model at epoch 100 with the training percentage of 50%, 70% and 90% is 94.21%, 95.87%, and 97.30%, respectively. The SEnO-DTCBiNet model’s recall value at epoch 60 is 94.45% and at epoch 100 is 97.88%. Likewise, the accuracy and F1-score value achieved by the SEnO-DTCBiNet model at epochs 20 to 100 with 90% of training data are 88.59%, 90.00%, 92.49%, 93.59%, 97.17% and 87.06%, 87.36%, 88.03%, 88.27% and 97.25%, respectively.

Figure 13:

SEnO-DTCBiNet model performance on the BraTS 2020 dataset. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

e.ii

BT dataset-based performance analysis

Figure 14 represents the proposed SEnO-DTCBiNet model’s effectiveness utilizing the BT dataset. Precision value obtained by the SEnO-DTCBiNet model at epochs 100, 60, and 20 for 90% of the training data is 96.86%, 92.58%, and 91.10%, respectively. Similarly, the recall of the SEnO-DTCBiNet model at epoch 40 is 92.31% and then it increases to 93.84% at epoch 80, and lastly it reaches 98.72% at epoch 100, respectively. The SEnO-DTCBiNet model’s accuracy at epochs 20–100 is 92.53%, 93.15%, 94.33%, 95.50%, and 98.48%. The SEnO-DTCBiNet model achieved an F1-score of 96.54%, 97.78%, and 97.78% at epoch 100, corresponding to training percentages of 60%, 80%, and 90%, respectively.

Figure 14:

Performance of the SEnO-DTCBiNet model using the BT dataset. BT, brain tumor. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

e.iii

BraTS 2018 dataset-based performance analysis

The SEnO-DTCBiNet model performance is evaluated under varying training percentages is presented in Figure 15. The precision of the SEnO-DTCBiNet model at epochs 20–100 is 87.45%, 91.38%, 93.50%, 93.75%, and 94.16% for the training percentage of 90%. Similarly, recall and accuracy values obtained by the SEnO-DTCBiNet model at epochs 60 and 100 with 90% of training data are 90.46%, 97.42% and 92.29% and 97.76%, respectively. The SEnO-DTCBiNet model F1-score at epoch 20 is 86.18% and then it rises to 89.83% at epoch 60, and finally it reaches 95.76%, respectively.

Figure 15:

Performance of SEnO-DTCBiNet model using the BraTS 2018 dataset. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

The comparative approaches are CJHBA-DRN [4], EDN-SVM [11], 2D-CNN-CAE [15], PatchResNet [7], Bat-Algorithm-based multihead convolutional bidirectional memory (BA-MCBM) [29], Dolphin swarm algorithm-based multihead bidirectional convolutional memory (DE-MCBM) [30], multihead sonar prey optimized convolutional bidirectional memory (SPO-MCBM), and SSA-based distributed tetra head attention-based convolutional bidirectional network (SSA-DTCBiNet) [31].

f.i

Comparative analysis on the BraTS 2020 dataset in terms of training percentage

The SEnO-DTCBiNet model performance against several traditional approaches on the BraTS 2020 dataset is presented in Figure 16. The precision of the SEnO-DTCBiNet model is 97.30%, which shows that it is higher than the EDN-SVM by 12.31%, 2D-CNN-CAE by 8.16%, CJHBA-DRN by 10.25%, BA-MCBM by 7.28%, Dense-CNN by 14.27%, PatchResNet by 9.66%, DE-MCBM by 6.36%, SSA-DTCBiNet by 6.82%, and SPO-MCBM by 1.86% for 90% of the training data. The SEnO-DTCBiNet model achieved a recall value of 97.88%, which indicates an enhancement by 5.06%, 2.45%, 4.61%, 1.85%, 6.11%, 4.24%, 1.68%, 1.76%, and 0.11 for the EDN-SVM, 2D-CNN-CAE, CJHBA-DRN, BA-MCBM, Dense-CNN, PatchResNet, DE-MCBM, SSA-DTCBiNet, and SPO-MCBM, respectively. The accuracy of the SEnO-DTCBiNet model is 97.17%, and it is improved by 10.05% with EDN-SVM, 6.63% with 2D-CNN-CAE, 8.59% with CJHBA-DRN, 6.10% with BA-MCBM, 11.29% with Dense-CNN, 7.49% with PatchResNet, 3.59% with DE-MCBM, 4.85% with SSA-DTCBiNet, and 0.65 with SPO-MCBM for 90% of the training data. Likewise, the SEnO-DTCBiNet model attained an F1-score of 97.25%, which is an improved value over other comparative approaches.

Figure 16:

SEnO-DTCBiNet model performance comparison on the BraTS 2020 dataset. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

f.ii

K-fold cross-validation using the BraTS 2020 dataset

The SEnO-DTCBiNet model performance against the other conventional models based on K-fold l using the BraTS dataset is shown in Figure 17. K-fold cross-validation is performed for KF-4 to KF-10 against the baseline models, with the proposed SEnO-DTCBiNet. The SEnO-DTCBiNet model achieved 96.27% of accuracy, which is an improved value than Dense-CNN and EDN-SVM, by 8.92% and 6.31% at a fold of 10. Likewise, in the precision, the proposed model attains a higher value of 95.53%, showing improvement over 5.99% by CJHBA-DRN and 3.44% by PatchResNet. The recall value achieved by the SEnO-DTCBiNet framework is 97.74% showing an enhancement of 3.69% over 2D-CNN-CAE and 2.52% over BA-MCBM, respectively. The F1-score of the SEnO-DTCBiNet model is 96.62%, which is greater than DE-MCBM by 1.18%, SSA-DTCBiNet by 0.32%.

Figure 17:

K-fold analysis on the BraTS 2020 dataset.

f.iii

Training percentage-based comparative analysis using the BT dataset

The comparative results of the SEnO-DTCBiNet model against the baseline model based concerning training Percentage on the BT dataset is presented in Figure 18. The SEnO-DTCBiNet model obtained a precision of 96.86%, which is also higher than other comparative EDN-SVM, 2D-CNN-CAE, CJHBA-DRN, BA-MCBM, Dense-CNN, PatchResNet, DE-MCBM, SSA-DTCBiNet, and SPO-MCBM approaches. The recall of the SEnO-DTCBiNet model is 98.72%, and it is improved over 6.29% with EDN-SVM, 4.03% with 2D-CNN-CAE, 5.83% with CJHBA-DRN, 2.93% with BA-MCBM, 7.06% with Dense-CNN, 5.65% with PatchResNet, 2.61% with DE-MCBM, 1.40% by SSA-DTCBiNet, and 0.20 by SPO-MCBM. The SEnO-DTCBiNet model achieved an accuracy of 98.48%, which shows that it is higher than EDN-SVM by 9.57%, 2D-CNN-CAE by 6.80%, CJHBA-DRN by 7.46%, BA-MCBM by 6.76%, Dense-CNN by 10.06%, PatchResNet by 7.24%, DE-MCBM by 4.23%, SSA-DTCBiNet by 2.40%, and SPO-MCBM by 0.57 for 90% of the training data. The F1-score of SEnO-DTCBiNet model is 97.985, which displays the enhancement by 8.62%, 5.72%, 8.40%, 4.96%, 9.89%, 7.08%, 2.78%, 1.58%, 0.38 for the EDN-SVM, 2D-CNN-CAE, CJHBA-DRN, BA-MCBM, Dense-CNN, PatchResNet, DE-MCBM, SSA-DTCBiNet, and SPO-MCBM, respectively.

Figure 18:

Comparison of the SEnO-DTCBiNet model based on the training percentage using the BT dataset. BT, brain tumor. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

f.iv

K-fold cross-validation using BT dataset

The proposed SEnO-DTCBiNet model’s performance against the other traditional models, based on the K-fold using the BT dataset, is depicted in Figure 19. The proposed SEnO-DTCBiNet model achieved 96.23% of accuracy, showing improved performance over Dense-CNN by 7.18% and EDN-SVM by 4.92%. Additionally, its precision reached a high value of 95.49%, outperforming CJHBA-DRN by 4.65% and PatchResNet by 4.31%. The SEnO-DTCBiNet framework also demonstrated a superior recall value of 97.71% representing an enhancement of 0.66% and 0.33% over 2D-CNN-CAE and BA-MCBM. Furthermore, the model’s F1-score reached 0.81% at fold-10, exceeding DE-MCBM by 1.06% and SSA-DTCBiNet by 0.81%. The proposed SEnO-DTCBiNet framework significantly advances BT segmentation by achieving improved performance. This makes it a more effective tool for analysis compared to the conventional methods.

Figure 19:

K-fold analysis on the BT dataset. BT, brain tumor.

f.v

Comparative analysis based on training percentage using BraTS 2018 dataset

Figure 20 presents the performance comparison of the proposed SEnO-DTCBiNet model which attained of 94.16%, which shows a development by 4.47%, 1.48%, 4.32%, 1.15%, 5.12%, 2.66%, 1.08%, 0.70, and 0.28 for the EDN-SVM, 2D-CNN-CAE, CJHBA-DRN, BA-MCBM, Dense-CNN, PatchResNet, DE-MCBM, SSA-DTCBiNet, and SPO-MCBM. The SEnO-DTCBiNet model recall and accuracy with 90% of training data are 97.42% and 97.76% respectively. The F1-score of SEnO-DTCBiNet model is 95.76%, which is greater than the EDN-SVM by 5.73%, 2D-CNN-CAE by 3.00%, CJHBA-DRN by 4.57%, BA-MCBM by 2.29%, Dense-CNN by 6.29%, PatchResNet by 3.68%, DE-MCBM by 1.90%, SSA-DTCBiNet by 1.35%, and SPO-MCBM by 0.34.

Figure 20:

Comparison of the SEnO-DTCBiNet model based on training percentage using the BraTS 2018 dataset.

f.vi

K-fold cross-validation using the BraTS 2018 dataset

The model evaluation on the K-fold cross-validation against the baseline models across the BraTS 218 dataset is depicted in Figure 21. Contrasting it with conventional approaches. With a K-fold 10, the SEnO-DTCBiNet model achieved an accuracy of 95.51%, exceeding that of Dense-CNN and EDN-SVM by 6.70% and 5.96% respectively. The precision also registered a higher value of 95.37%, showing an improvement of 3.68% over CJHBA-DRN and 2.45% over PatchResNet. The SEnO-DTCBiNet framework’s recall reached 95.79%, outperforming 2D-CNN-CAE by 3.77% and BA-MCBM by 2.75%. Finally, at fold-10, the model secured an F21-score of 96.62%, which is 1.18% greater than DE-MCBM and 0.32% greater than SSA-DTCBiNet. By consistently beating multiple baseline models across different K-folds, the SEnO-DTCBiNet model demonstrates superior performance and robustness.

Figure 21:

K-fold based on the BraTS 2018 dataset.

f.vii

Comparative analysis based on training percentage using the real-time dataset

The SEnO-DTCBiNet model performance evaluation across the other baseline models using the real-time dataset is presented in Figure 22. The SEnO-DTCBiNet model achieved 96.35% accuracy, performing better than the Dense-CNN by 5.40%, EDN-SVM by 4.46%, and CJHBA-DRN by 4.06% when trained on 90% of the data. The model attains a precision rate of 95.98%, showing improvements over other models such as 2D-CNN-CAE, BA-MCBM, and PatchResNet by 2.10%, 1.81% and 2.82% respectively. The SEnO-DTCBiNet model achieved a recall value of 97.10%, which is 1.25% better than DE-MCBM and 0.21% better than SSA-DTCBiNet. Additionally, the F1-score of the model is 96.54%, with a gain over SPO-MCBM of 0.38%. Overall, these results indicate that the SEnO-DTCBiNet model offers an effective performance enhancement, particularly in the context of BT classification, when compared to conventional models.

Figure 22:

Comparison of the SEnO-DTCBiNet model on the real-time dataset concerning training percentage. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

f.viii

K-fold cross validation using the real-time dataset

Figure 23 represents the SEnO-DTCBiNet model outcomes across the traditional methods based on the training Percentage of the real-time dataset. the SEnO-DTCBiNet model achieved an accuracy of 96.47%, surpassing Dense-CNN by 8.78%, and EDN-SVM by 6.23%, on K-fold 10. Furthermore, the model demonstrated a high precision of 95.78%, outperforming approaches such as 2D-CNN-CAE, BA-MCBM, and PatchResNet by 2.38%, 2.33% and 3.52%. In recall, the SEnO-DTCBiNet reached 97.87%, which is 0.54% higher than DE-MCBM and 0.43% above SSA-DTCBiNet. Its F1-score also shows a higher value, 96.81% presenting an improvement over SPO-MCBM by 0.80%. These results highlight the SEnO-DTCBiNet framework as a significant advancement in performance, particularly in BT classification compared to other conventional models.

Figure 23:

K-fold analysis on the real-time dataset.

Figure 24 portrays a Receiver Operating Curve of the SEnO-DTCBiNet framework against the other comparative models, including CJHBA-DRN, BA-MCBM, 2D-CNN-CAE, Dense-CNN, EDN-SVM, PatchResNet, DE-MCBM, SSA-DTCBiNet, and SPO-MCBM methods based on three datasets. Based on this ROC analysis, it indicates that the SEnO-DTCBiNet model performs significantly to the other comparative methods.

Figure 24:

ROC analysis. (A) BraTS 2020 dataset, (B) BT dataset, and (C) BraTS 2018 dataset. BT, brain tumor.

The Time Complexity of the SEnO-DTCBiNet model against other existing methods using the BraTS2019, BT, and the BraTS2020 datasets is depicted in Figure 25. The analysis reveals that the SEnO-DTCBiNet model requires less time for tumor classification as the number of iterations increases, outperforming the comparative approaches.

Figure 25:

Time complexity analysis. (A) BraTS 2020 dataset, (B) BT dataset, and (C) BraTS 2018 dataset. BT, brain tumor.

The convergence analysis of the SEnO optimization compared to approaches such as RMsProp, Adam, BA, SSA, DEA, and SPO is depicted in Figure 26. The analysis indicates that SEnO’s loss value decreases more effectively than its counterparts as the number of epochs increases. At epoch 50, the SSA and DEA methods reach a loss of 6.73 × 10⁻⁰⁷, 3.13 × 10⁻⁰⁶ whereas the proposed optimization reaches a lower loss of 1.00 × 10⁻⁰⁶, respectively. Similarly, at epoch 99, the proposed optimization remains the lower loss value of 1.00 × 10^−06, whereas the existing approaches, RMsProp, Adam, and BA reach a loss of 0.630, 0.002, and 0.001. These findings therefore demonstrate the effectiveness of the SEnO optimization in achieving an optimal solution for classifying the BT.

Figure 26:

Convergence analysis. SSA, Sparrow Search Algorithm.

Figure 27 presents the computational complexity of the proposed brain tumor detection model and various other methods, using FLOPS as the single arithmetic operation, such as addition or multiplication, performed on two floating-point numbers. At epoch 50, existing methods exhibit high FLOPS requirements, such as the Dense-CNN report 1.59 × 10⁰⁸ and EDN-SVM report 1.45 × 10⁰⁸, whereas the proposed model utilizes fewer FLOPs of 1.42 × 10⁰⁸. Similarly, at epoch 99, the SSA-DTCBiNet, SPO-MCBM uses fewer FLOPs of 2.08 × 10⁰⁸; in comparison, the proposed SEnO-DTCBiNet model significantly reduces the computational complexity to 2.05 × 10⁰⁸ FLOPS through the SEnO algorithm, which significantly configures the model to enhance classification performance.

Figure 27:

Computational complexity analysis in terms of FLOPS. FLOPS, floating-point operations per second.

As described in Figure 28, ablation is performed to evaluate the individual contributions of various architectural components in the proposed model. Starting with a BiLSTM, the model achieved 93.68% accuracy, effectively capturing temporal dependencies. Incorporating the SEnO optimizer with CNN shows improved performance to 93.96%, showing the benefit of optimized weight updates in the convolutional network. The CBiNet model, individually reached 94.64%, while the combination of SEnO + BiLSTM yielded 95.24%, demonstrating enhanced training stability. Adding the Tetra Head Attention to CBiNet further increases the performance to 95.74%, and SEnO + CBiNet reached 96.35% by unifying optimization and hybrid architecture. Finally, the complete SEnO-DTCBiNet model achieved an accuracy of 97.18%, validating the effectiveness of combining CNN, BiLSTM, SEnO optimization, and multihead attention in one unified framework. These results confirm that each component contributes to a layered improvement in classification, with the fully integrated model delivering the most robust performance.

Figure 28:

Ablation study for SEnO-DTCBiNet model components. SEnO-DTCBiNet, sonar energy optimized distributed tetra head attention-based convolutional bidirectional network.

An ablation study focused on the impact of individual attention mechanisms and their combinations is depicted in Figure 29. When used independently, BAM, SE, ULSAM, and CC achieved an accuracy of 87.57%, 85.68%, 91.55%, and 92.54% respectively, each excelling in different aspects, such as channel recalibration and spatial attention. Combining BAM + SE improved accuracy to 95.55%, and CC + BAM + SE achieved 94.55%, reflecting the complementary effect of combining spatial and channel attention. The trio of ULSAM + SE + CC achieved 96.46% of accuracy, showcasing the synergy between cross-channel, lightweight, and contextual attention mechanisms. Ultimately, the full DTHA configuration combining BAM, SE, ULSAM, and CC achieved the peak accuracy of 97.18%, proving that fusing diverse attention modules significantly boosts feature representation. These attention mechanisms, when integrated into the proposed model, enhance fine-grained tumor localization and classification, making SEnO-DTCBiNet a highly effective tool in classifying the BT.

Figure 29:

Ablation study for attention mechanisms. CC, criss-cross; DTHA, distributed tetra head attention; SE, squeeze and excitation.

The SEnO-DTCBiNet model confusion matrix, across the BraTS 2020, BT, and the BraTS 2018 datasets is represented in Figure 30. This matrix serves to validate the model’s performance by comparing its classification against the actual labels.

Figure 30:

Confusion matrix.

The statistical Analysis for the proposed SEnO-DTCBiNet model across the BraTS 2020 is presented in Table 4, across the BT is presented in Table 5, for the BraTS 2018 is shown in Table 6, and the real-time dataset is presented in Table 7.

A T-test is performed to compare the means of different methods. A p-value of <0.05 indicates a statistically significant difference between the results of the compared models. Table 8 represents the T-test analysis on the BraTS 2020 dataset, Table 9 presents the T-test analysis on the BT dataset, Table 10 depicts the BraTS 2018 dataset based on the T-test analysis, and Table 11 represents the T-test analysis results on the real-time dataset. Thus, the results ensure that the model enhanced BT classification performance across the baseline models under different datasets.

The increase in mortality rate due to BT has led to the development of an effective system that detects and classifies the type of BT based on their origin. However, conventional approaches used in the context provide promising outcomes; their performance is hindered by increased complexities and abnormal performance. The Dense-CNN [1], due to its sophisticated architecture, leads to increased processing time and inherited feature detection capability. While the EDN-SVM [11] improvised by utilizing a machine-learning architecture leveraging its simple structure, the inability to handle structural patterns caused overfitting and computational inconsistencies. The CJHBA-DRN [4] limited its application to recognizing diverse features that are essential for discriminating among the complex brain structures, thus leading to misclassification or diminished performance. The PatchResNet [7] involved different layered patches along with the pretrained models to detect BT; however, the approach lacked optimization of the learning parameters, which led to reduced classification accuracy. The 2D-CNN-CAE [15] leads to overfitting and underfitting issues due to the single-domain data for classification. The BA-MCBM [29] and DE-MCBM [30], due to the individual characteristics of optimization, lead to premature convergence or getting stuck in local optima. The above-mentioned drawbacks are effectively tackled by the proposed SEnO-DTCBiNet by combining the OFA-based DW-Net for precise segmentation of tumor regions, deep BiLSTM layers for contextual understanding of complex structures. Additionally, the SEnO algorithm is used for hyperparameter tuning. These strategies allow the proposed SEnO-DTCBiNet to obtain improved results in classification. The comparison performance of the SEnO-DTCBiNet with other approaches based on Training Percentage and K-fold under the BraTS 2018, BraTS 2020 dataset, BT dataset, and real-time dataset is depicted in Tables 12 and 13.