ECG Decision Support System based on feedforward Neural Networks

Table 1

Feed forward artificial neural networks

The ANN is inspired from the biological neural networks functioning. They are distinguished by their ability to identify nonlinear relationships between data, without the need to understand the nature of the phenomena (Dalvi et al., 2016). Regarding the feed forward ANN, they are familiar by the aptitude of their neurons in the input layer to feed their output, through the hidden layer, until the output layer. In fact, whatever the feed forward ANN, the number of neurons in the input layers is adjusted to the number of variables in the input vector. The number of neurons in the output layer is appropriate for the number of classes. The number of neurons in hidden layer depends on the classification problem.

The following equation defines how the neurons compute their outputs:

(5)

$O=\sum _{i=1}^N{w}_i{x}_i$

where O is the neuron’s output, N is the number of inputs, x_i, w_i are the input variable and its corresponding weight, respectively.

In this work, we have focused on three feedforward neural networks, which are the MLP, RBF, and PNN. In the following sub sections, each ANN is explained in details.

MLP network

The MLP consists of an input layer, one or more hidden layers followed by an output layer. MLP uses the backpropagation algorithm for training data. In fact, it begins the training with random weight values and calculates the output value from a set of records for which the expected output value is known. The weights are transferred to the next level via the activation functions. The below equation shows how the output of the MLP (Yi) can be computed (Khan and Jabbar, 2009):

(6)

$Y_i=\sum _{j=1}^{m}f\left(w_{ij}{x}_j+b_i\right)$

where w_ij is the corresponding weight for the input j in the layer I, m is the number of inputs in the previous layer, f is the activation function, and Y_i is the output of the layer. Figure 4 illustrates a MLP network structure.

Figure 4

RBF

Radial basis function network consists also of three layers: an input layer, one hidden radial basis layer, and an output linear layer. It is known by its hidden layer which contains a set of radial basic functions, which are symmetrical and centered at each input feature. They are similar to normal distribution curves. Each neuron in the hidden layer computes the distance between the input vector and its own center. The calculated distance is transformed using the basis function and the result is the output from the neuron. Then, this later is multiplied by a weighting value and fed into the output layer. Finally, the output layer with its linear activation function acts to sum the outputs of the previous layer and yields a final output value for each class (Seshagiri and Khalil, 2000). Figure 5 illustrates a RBF network structure.

Figure 5

The output neuron y_k(x) is as follows:

(7)

$y_k\left(x\right)=\sum _{j=1}^l{w}_{kj}\phi \left(\left(\Vert x-{\mu }_j\Vert \right),{\delta }_j\right),$

where y_k is the output for the class k, w_kj is the associated weigh for the result of the hidden neurons j corresponding to the class k, φ is the radial function, (μ, δ) are its center and its width, respectively, l is the number of hidden neurons, x is a vector of input data.

Accordingly, to configure the RBF network, the choice of the radial basis function as well as the estimation of its centers and its widths is the main challenge.

PNN

PNN consists of four layers: an input layer, one hidden layer, a summation layer, and an output layer. The input layer contains N neurons corresponding to N features. The second layer consists of Gaussian functions for each exemplar feature vector. The third layer consists of the summation layer which sums the output of the hidden layer neurons belonging to the same class. The output of each summation unit is proportional to the Probability Density Function (PDF) of the corresponding class. The fourth layer consists of a competitive output layer that picks the maximum value of its inputs and returns the corresponding class associated to this maximum.

The PNN network for K classes is defined as it is shown in the below equation (Farhidzadeh, 2015):

(8)

$y_j\left(x\right)=\frac{1}{n_j}\sum _{i=1}^{n_j}\exp \left(-\frac{{\left(\Vert x_{j,i}-x\Vert \right)}^2}{2{\sigma }^2}\right)$

The PNN classifies x into k class when the following condition is true:

(9)

$y_k\left(x\right)>y_j\left(x\right)$

where n_j represents the number of features in class j, j ∈ 6 [1, .., M], M is the number of classes, ǁx_j,i − xǁ² is calculated as the sum of squares, x is any input feature vector, x_j,i is the i feature of a training feature vector belonging to class j, σ is the smoothing parameter (Spread) which has to be selected in order to minimize the estimated PDF error. Figure 6 illustrates a PNN network structure.

Figure 6

Similarities and difference between networks

Table 2 presents a comparison study which illustrates the similarities and the differences between the three networks (MLP, RBF, and PNN), regarding structural and parametric criteria.

Table 2

MLP, RBF, and PNN Feed_Forward neural networks.

First, by using the MLP with a single hidden layer, the highest ACC reached an ACC of 86.4%. Whereas, by adding a second hidden layer, the most accurate MLP achieves 81.8%. Therefore, it is not necessary to use an MLP with two hidden layers in this classification task. Second, when the number of neurons in the single hidden layer is small, for example 5, the ACC (72.7%) is the lowest and the MSE (0.026) is the highest. Thus, the ACC increases with the increasing number to some extent. Indeed, when the number reaches 15 neurons in the single hidden layer, the ACC is stacked in 81.8% and does not improve anymore.

Third, we have concluded that as long as the network has less number of hidden neurons, the network is considered fast. In fact, by adding neurons in the hidden layer, the learning time increases (0.221 s up to 0.406 s) and the test time decreases (0.026 s up to 0.013 s). As a result, we have kept the MLP structure (10, 10, 1) which has 10 neurons in the input layer, 10 hidden neurons, and a neurons in the output layer. By using this structure, we have obtained 86.4% as ACC, a MSE equals to 0.094, a Tr_time equals to (0.222 s), and a test _time equals to (0.024 s).

In this section, we have used two categories of learning algorithms based on the principle of backpropagation to evaluate the performances of the saved MLP structure (10, 10, 1). Indeed, the first category uses Jacobian derivatives algorithms which include Levenberg–Marquardt backpropagation (trainlm) and Bayesian regularization backpropagation (trainbr) algorithms. While the second category uses gradient derivatives which contains several algorithms. However, in this study, we have chosen some of them which are the Scaled Conjugate Gradient backpropagation (trainscg), Gradient Descent with Adaptive learning rate backpropagation (traingda), Resilient Backpropagation (trainrp), Gradient Descent with momentum and Adaptive learning rate backpropagation (traingdx), BFGS quasi-Newton backpropagation (trainbfg), Conjugate gradient backpropagation with Powell-Beale restarts (traincgb). The obtained results are shown in Table 4.

Regarding ACC, the MLP network achieves the highest ACC of 86.4% by applying the following algorithms: “trainlm,” “trainscg,” “trainrp,” and “trainbfg.” However, concerning time response, the Jacobian derivatives algorithms are faster than those of the gradient derivatives. In fact, the slowest learning time is given by the algorithm trainLM (0.222 s). In addition, relating to efficiency, the gradient derivatives algorithms generate more efficient results in terms of MSE (0.073) compared to the results obtained using the Jacobian derivatives algorithms (0.094). As a result, we have chosen the trainrp algorithm since it generates a maximum ACC (86.4%), a minimum MSE value (0.073), a Tr_time (0.294 s), and a Test _time (0.096 s).

Referring to Table 5, whatever the Lr, we have obtained the same ACC and approximately the same time response. Therefore, we have chosen the Lr value equals to 0.1, as we have obtained the lowest MSE (0.064). Hence, the MLP_opt is the structure which has a single hidden layer with 10 neurons and applies the trainrp learning algorithm. Its parameters, Lr, Δ_Inc, and Δ_Dec are maintained to 0.1, 1.2, and 0.5, respectively.

Referring to Table 6, it can be seen that the ACC increases with the increasing number of neurons in the single hidden layer. Therefore, ACC improves from 54.5% up to 99.9%. Moreover, the MSE is becoming closer to zero (2.411e-30 to 0.204). However, the Tr_time increases from 0.380 s to 0.699. Besides, the Test time is almost the same (on average 0.015) regardless of the number of neurons in the hidden layer. Hence, we have chosen the RBF network which achieves the highest ACC of 99.9% and the lowest MSE (1.266e-30) by using 20 hidden neurons.

In this section, we have evaluated the saved RBF structure (10, 20, 1) by using several types of radial basic functions, such as the Bihararmonic, Multiquadric, Inverse Multiquadric, Thin Spline, and Gaussian (Buhmann, 2000). Table 7 summarizes all the results for the different basic functions.

Based on the results obtained in Table 7, since we have obtained the same ACC as well as the Time_response (on averge 0.661 s), we have kept the radial function which accords the lowest MSE (0.250e-30). Thus, the Inverse_multiquadratic function is chosen. This basic function is presented in Equation (10) (Farhidzadeh, 2015). Where ∅ is the basic function, μ is the center, δ is the width (or the spread), and x is the input vector.

This function computes the Euclidean distance (||.||) between the interpolation centers and the input vectors:

We have obtained the same ACC (99.9%), Time_response (0.662 s on average), and the same MSE (0.250 e-30), whatever the value of the spread parameter. However, we have obtained different Test_time results. So, by referring to the fastest one (0.121 s), we have chosen the spread value 10. As a result, the optimum structure RBF_opt (10, 20, and 1) uses the inverse quadratic radial functions which its center is calculated randomly, its spread value is 10 and its weights are computed using the backpropagation algorithm. This network achieves the highest ACC (99.9%), a MSE close to zero, and a significant test time (0.121 s).

Referring to the results obtained in Table 10, we have remarked that whatever the spread value, we have obtained same ACC, MSE, and Test_time. Accordingly, it is clear that the PNN network is not influenced by the choice of the spread parameter. We have chosen the value 1 of the spread parameter, since we have obtained the shortest learning time (0.070 s). As a result, the optimum structure PNN_opt (10, 22, and 1) with a spread value is equal to 1. This network achieves the highest ACC (79.5%), a MSE (0.162), and a significant Tr_time (0.070 s).

Thus, referring to Table 11, all the optimum ANNs have good performances on classifying ECG records into normal and arrhythmia class. However, each network has its advantages and its drawbacks. In the first hand, concerning ACC, it is revealed that RBF network achieves the highest ACC of 99.9% followed by MLP_opt network (86.7%) and PNN (79.5%). In the second hand, regarding the number of hidden nodes, we have obtained 10, 20, and 22, for the MLP, RBF, and PNN networks, respectively. Consequently, it is noting that the MLP_opt network is the least complex network among the different classifiers. In the third hand, linking to classification time response, we have analyzed the Tr_time and Test_time for each optimum ANN. Therefore, first, PNN_opt achieves the lowest Tr_time (0.070 s) and the highest Test_time (0.218 s). Then, by using PNN_opt, nearly no time consumed for training but Test_time is long. Second, MLP_opt consumes the shortest Test_time (0.096 s) than the RBF_opt (0.121 s) and the PNN_opt (0.162 s). As a result, the comparison between the three optimum ANNs proves that the most relevant ACC (99.9%) and the lowest MSE (0.250 e-30) are provided by the RBF_opt network. However, PNN_opt applies the rapidest training phase (0.070 s) and MLP generates the shortest testing time (0.096 s).

Hence, according to the users’ criteria, the DSS will choose one of the optimum structures. In fact, it will select RBF_opt, if experts want an accurate response regardless of the classification time_response. However, if users want a fast response, DSS will select MLP_opt (Test_time = 0.007 s). Whereas, when users have a huge dataset, it is required to choose PNN_opt since it consumes the lowest learning time (Tr_time = 0.06 s).

Based on the results in Table 12, the classification accuracies are effective to the tune of about 99.9 to 82.5% ACC. Therefore, the obtained accuracies (99.9, 86.5, and 79.5%) are considered acceptable. However, comparing only the accuracies of the feed forward ANN classifiers is not sufficient to make a fair comparison. In fact, it is necessary to highlight several factors, such as the choice of the database and the number of recording analyzed.

Indeed, according to the Table 12, the ECG arrhythmia database MIT_BIH was adopted by all the studies, except (Rai et al., 2013; Savalia et al., 2017) who have added to MIT_BIH recordings the ECG signals from NSR database. It is also found that each study analyzed the different number and recording time of ECG signals. Besides, each study has its own methodology. For example, for the pre-processing stage, many approaches are approved for the feature extraction, such as the use of DWT coefficient (Abhinav-Vishwa et al., 2011), morphological features (Buhmann, 2000; Rai et al., 2013; Dalvi et al., 2016; Savalia et al., 2017) and many others. Similarly, for feature selection, many methods are employed like the Principal component analysis (PCA) (Dalvi et al., 2016), the correlation method (our proposed work) and others. However, there are works which do not use any method for feature selection (Buhmann, 2000) and also for normalization (Savalia et al., 2017). Other point, each study has divided the input dataset into two sets, for training and testing, by using different approaches. For our proposed work, we have respected the AAMI recommendations, while (Abhinav-Vishwa et al., 2011) has chosen its own datasets and (Savalia et al., 2017) has applied the cross validation division.