Abstract
Measuring brain activity through Electroencephalogram (EEG) analysis for eye state prediction has attracted attention from machine learning researchers. There have been many methods for EEG analysis using supervised and unsupervised machine learning techniques. The tradeoff between the accuracy and computation time of these methods in performing the analysis is an important issue that is rarely investigated in the previous research. This paper accordingly proposes a new method for EEG signal analysis through Self-Organizing Map (SOM) clustering and Deep Belief Network (DBN) approaches to efficiently improve the computation and accuracy of the previous methods. The method is developed using SOM clustering and DBN, which is a deep layer neural network with multiple layers of Restricted Boltzmann Machines (RBMs). The results on a dataset with 14980 instances and 15 attributes representing the values of the electrodes demonstrated that the method is efficient for EEG analysis. In addition, compared with the other supervised methods, the proposed method was able to significantly improve the accuracy of the EEG prediction.
1. Introduction
There is a field of study in human-computer interaction (HCI) [1–3], namely, brain computer interface (BCI) [4], and it has wide ranges, such as security supervision, critical safety, and applications in industry and medicine as well [5]. It is quite important in the applications of medicine; for example, the disabled people are able to interact better by utilizing this way, and this is achieved by founding a method that acts like a bridge between computer and human brain. Electroencephalogram (EEG) is related to the signals that are received electrically from the brain activity. It is considered as an outcome of the data flow during exchanging information of numerous neurons and it can be recorded for long time [6]. The main issue in the EEG signal inspection is the data amount that can be obtained from the recorded EGG signals [7]. Moreover, the signals with artifact interferences are the important issues in EEG systems. In fact, the signals of the artificial objects are in high-frequency domain and they can intensely change the signals that are recorded. Additionally, they belong to two categories, that is, external and internal groups [8]. The former group is created from the power or environment supplies and the latter one is related to the movements of the eyes, artificial respiratory objects, and muscle [9].
There are numerous scientific applications that have used Self-Organizing Map (SOM) in their work because this neural network is designed based on unsupervised learning by Kohonen [10] and it is useful for the tasks like data clustering and reduction in the high-dimensional data [11]. Furthermore, SOM can be investigated to recognize the existing outliers [12]. The distribution of the Quantization Errors (QEs) on the map can be helpful to detect the outlier. The QE term is related to the measure of representing the unit that is associated with each empirical vector. Therefore, there would be possibility for observing the outlier, if the experimental vector of QE value is high.
In today’s life, the field of Deep Learning (DL) has been becoming more popular among scientists and scholars and many novel scientific papers have been publishing with this state-of-the-art method. It has been achieving an immense number of workable applications, succeeded in multifarious matches in Machine Learning (ML) and Pattern Recognition (PR) methods, and most significantly it can obtain highly accurate classification far superior than human [13–15]. From a statistical point of view, DL algorithms are considered as recursive linear models that are regression based [16]. In shallow learning, which is one of the statistical models, there is one or no hidden layers [17], while DL approaches utilize multihidden-layer architectures, such as Stacked Denoising Autoencoders (SDA) [18], Deep Belief Network (DBN) [19], and Convolutional Neural Network (CNN) [20].
This paper investigates eye state identification using EEG signals. In the current study, we applied machine learning techniques to examine the EEG signals and state of the eye and numerical analysis. In fact, a hybrid intelligent method is developed through Self-Organizing Map (SOM) clustering and Deep Belief Network (DBN) approaches. First, SOM is used to cluster the data in different sections prior to the classification step by DBN. Actually, it is worth nothing to say that the act of clustering by SOM plays a crucial role in enhancing the classification accuracy of the EEG by DBN. Furthermore, in EEG classification, in order to overcome the curse of dimensionality and complexities, we carried out one-class-one-network. The goal of the global network is to arrange the whole subnetworks outputs in the problems of classification. It has been a perfect selection to have a balance between the classification complexity and accuracy rate issues. Then, two-layer BP network is utilized for one-class-one-network and DBNs are applied for subnetworks in the global network. In the EEG classification step, we used the data that is located in every cluster. For our experimental results, we utilize the EEG eye state corpus from the UCI machine learning repository that includes 14980 instances with 15 features.
We present the acronyms list that is applied in the current paper in Table 1. This work has the following order. In Section 2, we discuss some of the problems and provide our contributions of the present paper. In Section 3, we review the related and background studies. In Section 4, the preliminaries are presented. Section 5 gives the experimental results and assessment of our study in more detail. In Section 6, we discuss our work limitations and future work. In the end, in Section 6.1, we give the concluding remarks of this paper.
2. The Problems and Our Contributions
The use of machine learning methods is essential in developing the identification of the status of the human eye. In case of EEG signals processing to identify the eye status, applying the supervised and unsupervised machine learning approaches has been eminent and their contributions in the EEG analysis have been encouraging. As in the eye identification status systems, the prediction of eye status is significant, choosing the proper and applicable supervised and unsupervised machine learning methods can play a prominent role in boosting both the accuracy and speed of the systems. In prior studies, the techniques developed for eye status identification based on EEG signals mostly have relied on one learning approach, which believes that their accuracy can be improved through the combination of learning techniques, such as supervised and unsupervised learning techniques. In addition, deep learning approaches have demonstrated their effectiveness in EEG analysis [21–24] and presented higher accuracy in relation to the conventional machine learning methods.
From the background and literature review on EEG signals for identifying the human eye status systems, we observe that most methodologies in improving these kinds of systems utilize single learning approaches and there is no implementation of clustering methods in this context. Additionally, there is no research by utilizing DBN classification and SOM clustering techniques for predicting the status of the eye by EEG signals. The previous study only performs DBN and Stacked Autoencoders (SAE) methods in EEG signals context [25]; however, a few researches take the benefits of DBN methods for identification of the eye status based on EEG signals. Moreover, there is no research to investigate the pros of the SOM clustering technique and DBN learning methods for identifying the eye status systems based on EEG signals. For EEG signal analysis, it is found that clustering may impact on the accuracy of prediction [26]. Accordingly, in the current work, we attempt to improve the eye status identification system based on EEG signals using the SOM clustering ML method. We apply the SOM clustering method to cluster the data and the DBN method for classifying tasks. Overall, our contributions to this research in developing the eye status identification based on EEG signals are as follows:(i)In order to predict the eye state through EEG signal analysis, a novel approach is presented based on SOM clustering and DBN methodologies. We developed our model by utilizing a hybrid of SOM clustering and DBN techniques. The aim is to improve the accuracy of EEG classification. We use the SOM clustering approach to cluster the data into different dissimilar groups. It has an important role in improving the EEG classification accuracy. In addition, as this study aims to analyze relatively a big EEG dataset, clustering of data could be an effective way to reduce the time complexity of data analysis, which is ignored in the previous studies.(ii)We apply DBN methodology on the data in each cluster for EEG classification. DBN method is a Deep Neural Network that includes several graphical model layers that comprise the directed and undirected edges. It compounds various hidden layer units and every layer has a connection to each other. Furthermore, it is based on two phases which are Belief Net (BN) and Restricted Boltzmann Machine (RBM). In the BN part, which is a directed acyclic graph, there are layers that are stochastic binary; every joined layer has some weight and there are 0 and 1 statuses for stochastic binary unit.(iii)In order to conquer the curse of dimensionality, we adopt one-class-one-network to develop the ability of the neural network and the goal of using that is to coordinate the whole subnetworks output in the classification issues. Pretraining and fine-tuning are applied in DBN in order to prevent Gradient Diffusion (GD). BP algorithm is utilized in the fine-tuning step. Wake-Sleep algorithm is performed to avoid GD and speed up the feature extraction convergence.(iv)In this research, for our experimental result, we use the EEG eye state corpus from the UCI machine learning repository and all the data were obtained from single successive EEG measurements with the EMOTIV EEG neuroheadset. For experimental results, we utilize 14980 instances with 15 attributes that show the value of the electrodes. Accuracy and Area Under Curve (AUC) are used to evaluate the performance and efficiency of our proposed method. True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) are performed to compute the classification metrics for forming the confusion matrix.
3. Related and Background Work
In [27], the authors provide a novel study based on developed EEG bands using regularized linear regression (RLR) classification method, which is structured by the time-frequency disintegration. Then, to recognize the state of the brain, the regression is utilized for obtaining a proper frequency distribution framework. Similarly, in the work presented by [28], EEG signals for detecting the changes in the eye states in the real-time system were investigated. In this study, when they acquired EEG signals for determining the available alteration of the state of the eye, one of the methods of decomposition, which is a multivariate empirical model, is applied to extract the proper features. Then, they used Logistic Regression (LR) and Artificial Neural Network (ANN) [14] to classify the obtained features perfectly. The rate of accuracy that they achieved with their proposed method was 88.2% in less than two seconds, which was an important advancement. A classification approach based on Radial Basis Function Neural Network (RBFNN) [29] was developed by [30] for identifying epilepsy in EEG signal, and they performed Discrete Wavelet Transform (DWT) as a preprocessing step before classification step. Furthermore, Particle Swarm Optimization (PSO) [31] was utilized for optimizing the Mean Square Error (MSE) of the trained network and they achieved 99% accuracy rate in their work. The authors in [32] found a combined mechanized seizure identification framework, in which they used two models of classifiers (Support Vector Machine (SVM) and Weighted Permutation Entropy (WPE) for improving the identification accuracy and sensitivity. Furthermore, they showed that during the epileptic process, the entropy of the EEG segments that are based on measures is low in comparison to the regular EEG. From the proficiency point of view, it is clear that their work has sufficient performance as it can recognize the unexpected alternations in the signal more accurately and allots less intricacy to the segments that show normalcy to noise impacts. A conceptual system based on ML was proposed by [33] to represent an automatic identification of EEG artifacts in three subjects: canine, rodent, and people. In other words, they used the SVM method for classifying the epochs of artificial EEG in the subjects and it leads to the determination of the individual’s eye states.
The authors in [34] determined a method based on EEG signals for the identification of human by utilizing eye blinking electrooculogram (EOG) signals. In this study, they tested the feature and score level fusion methods for their multiple frameworks and they used density and canonical correlation approaches for that purpose. Then, to extract the features, they applied time delineation and EEG signal based on autoregressive modeling. In the end, linear discrimination approach is used for the classification step. In the other study, the authors detect the sleepiness and wakefulness from EEG signals. They used nonlinear wavelet decomposition for extracting raw signal features as nonlinear theory provides this opportunity to assess the physiological signals from complexity and nonlinear characteristics point of views more accurately. Then, for developing the identification performance generally, they performed a feature-level fusion approach. Finally, Extremely Learning Machine (ELM) was utilized because of its quick learning speed as the factor related to the calculation speed is quite crucial in the smart and machine-based frameworks [35]. The authors in [36] carried out research based on Binary Flower Pollination (BFP) method for human EEG-based detection. Also, they solved the problem of diminishing the number of necessary sensors, which they faced during the similar execution. Furthermore, they assessed BFP method with multifarious transfer functions in order to choose the proper channels that improve the accuracy rate and it was calculated by Optimum-Path Forest (OPF) classifier. The researchers in [37] proposed a new framework based on a combined Outlier Detection (OD) and Independent Component Analysis (ICA) approaches to eliminate the Ocular Artifacts (OA) from EEG signals automatically by ignoring the EOG reference signal. This method is robust because the hybrid OD-ICA approach looking for OA templates among the whole items; furthermore, this approach eliminates just OA templates and keeps the other significant EEG signals.
In [38], the authors presented a study regarding the classification of rock penetrability based on the SOM and Artificial Bee Colony (ABC) method. The authors in [39] provided a novel study based on SOM for adaptive learning. According to this method, the weights were updated regularly and then, the appropriate weights were achieved shortly. Furthermore, lower QEs were obtained due to the optimal weight vectors. In order to classify the hyperspectral images obtained from satellite and to optimize SVM method, SOM approach was used. They used Indian Pines and Pavia University databases for assessing the performance of their proposed method [40]. In the other work by [41], an unsupervised classification of the images by Deep SOM (DSOM) approach was presented. In order to optimize the accuracy, training time, and ability of the system, they expand DSOM as E-DSOM. In the recent paper presented by [42], SOM architecture was used to detect an unsupervised intrusion. They contrasted their work with two models using an SOM intrusion recognition approach. The author in [11] developed a smoothed SOM (S-SOM) method. The benefit of this method regarding the outliers is to boost the SOM’s clustering and vector quantization characteristics. Moreover, the authors have published an article based on SOM regarding the projections of the diminished rainfall in South Florida. They applied their method to every three months season and they created a group of 1500 autonomous day-by-day precipitation replacements for every framework simulation. Furthermore, this method performed to the CMIP5 from 1976 to 2005 and upcoming CMIP5 (RCP 4.5) projections for 30 years’ day-by-day precipitation [43].
The authors in [44] demonstrated a study based on DL approach in order to classify the EEG signals for motor imagery. Then, DBN was performed for categorizing both sides of motor imagery. Next, in order to boost the power of their classifier, they used Ada-Boost method. Finally, Contrastive Divergence (CD) method was applied in order to train DBN more efficiently. Correspondingly, the authors of this work applied DBN to categorize two emotional groups from EEG signals. Moreover, in order to capture the most trustworthy and accurate emotional step changing, Hidden Markov Model (HMM) was utilized. From their outcome, it is clear that they achieved a high rate of accuracy due to use of combined DBN and HMM methods [45]. In the work presented by [46], a new method based on DBN approach was presented for diagnosing the fault of analog circuits. In other words, they developed a smart recognition solution that was not dependent on extracting and selecting the features manually and preparing complete contrast research on the two understandable empirical circuits with various levels of intricacies under smoothed fault statuses. The authors in [47] developed a study based on hybrid DBN, experimental mode disintegration, and ensemble method to solve the problem of air-conditioning framework related to cooling load foreseeing. First, the decomposition approach was applied in order to disintegrate the series of cooling load data to few elements with the goal of achieving high quality in terms of behaviors as well as outliers. Then, DBN approach was performed to extract perfectly the nonlinear features that were hidden and also high standard constant frameworks in data. Finally, to diminish the effect of ambiguities such as noise, accuracy, and model doubtfulness, ensemble method was utilized. Additionally, in order to forecast the power of the wind, the authors of this paper proposed a novel methodology that has carried out DBN and k-means clustering approaches. According to this study, in the first step, the data (Numerical Weather Prediction (NWP)) was chosen as an input for their model because it impacts the accuracy highly. Next, k-means clustering approach was used to deal with these kinds of data. During this process, plenty of NWP data were selected as an input of DBN framework to boost the effectiveness of this model. Furthermore, in order to assess the model, they used Sotavento wind farm, which was located in Spain. In the end, in order to contrast their model, they made a comparison with two different Morlet Wavelet Neural Network (MWNN) and Back-Propagation Neural Network (BPNN) [48]. The authors in [49] developed a method based on deep integrated neural network and transfer learning for eye state recognition. They used Closed Eyes in the Wild (CEW) and Zhejiang University Eyeblink datasets for their experiments. They applied transfer learning strategy to extract effective abstract eye features. In addition, they could improve the classification capability of the proposed model on small sample datasets through the use of transfer learning strategy. The authors in [50] used Optimized Deep Belief Network for Internet of vehicles to predict self-organizing traffic flow. They used back-propagation technique for fine-tuning the weight parameters among the visible and hidden layers of Restricted Boltzmann Machine (RBM). The authors in [51] conducted a study on the diagnosis and prognosis of mental disorders through the use of EEG and deep learning approach. The authors in [52] investigated classification of fine hand movements from low-frequency EEG through the use of Convolutional Neural Network (CNN). They evaluated the method on self-paced fine hand movements (touch, grasp, palmar, and lateral grasp) and compared the results with the shrinkage-linear discriminant analysis (LDA) and Random Forest (RF). A comparative study was conducted by [53] to track mental depression from EEG data. They used four neural network-based deep learning architectures, namely, CNN, Multilayer Perceptron (MLP), Recurrent Neural Network (RNN), RNN with Long Short-Term Memory (LSTM), and two supervised machine learning techniques, namely, Support Vector Machine (SVM) and Logistic Regression (LR), to track the mental depression from EEG data.
4. Preliminaries
4.1. Self-Organizing Map
Self-Organizing Map (SOM) has been widely used as an effective tool for many purposes, such as data classifying, analyzing, and visualizing [54]. neurons are the 2D distribution that are used for presenting SOM. is the weight vector related to the dimension D and is defined as an input vector, where D is related to the size of that is defined as follows:
Learning and recall are the two essential phases that SOM operation demands in order to perform. To train the network, the neuron weights of the outputs that are generated by the map operate as input vectors’ function throughout the phase of learning. Best Matching Unit (BMU) or winner neuron is the name of the map’s neuron(s) that every input vector is associated with this/these neuron(s). After the phase of learning, the trained map can be utilized for making a decision in the phase of recall. Then, the distance which is Euclidean distance (L2) is calculated between the two weights and input vector as below:
Thus, is the weight vector that is near to (input vector) and it is used to recognize the neuron that is winner and according to this operation, this makes the competition phase, which is defined as follows:
The Euclidean distance and competition phase are applied on the both learning and recall phases to discover the neuron that is winner. Furthermore, while the learning phase is processing, the updating operation is carried out to update the winner neuron and the nearest likelihood neurons. The following equation shows this adaptation process:
The learning degree of the neuron is defined as the neighborhood function (). Additionally, it quite depends on the neuron position than the winner neuron and to show the learning iteration numbers, the epoch number is considered. Therefore, the neighborhood function () is demonstrated as follows:where is defined as the rate of learning, is defined for the neighborhood rate, the winner neuron position is considered as , and the neuron position with index is shown as .
4.2. Deep Belief Network
Many scholars have demonstrated that Deep Belief Network (DBN) method can create the perfect models with higher efficiency and higher throughputs. In the current research, DBN is utilized to model a novel eye identification based on EEG signals. DBN is an effective DL approach based on a combination of Restricted Boltzmann Machine (RBM) [55]. Generative Stochastic Neural Network (GSNN) [56] is used for RBM such that there is a link for every hidden layer () to the visible one () related to the next RBM. Furthermore, the RBM higher layer’s input is obtained from the RBM lower layer’s output. Gibbs sampling [57] is performed because of the lack of links in the similar layer between units and nodes. Moreover, there is a deep framework in DBN, which produces a significant model with RBM’s multilayers. In order to boost our DBN method, both the first and second hidden layers of the two and hidden links are considered. The feature extraction and selection from the input data is one of the advantages of utilizing DBN. A joint distribution of RBM is defined as follows:where the hidden units and binary visible are defined as and , respectively, and the balanced weights between them is , corresponding biases are considered as c and b parameters, and finally normalization constant is . The following equation shows the function of energy that is between the and parameters:
The logistic sigmoid function is defined as an activator for the binary hidden unit and is demonstrated as below:
Log-likelihood gradient regarding the visible units is computed utilizing an opposing convergence after k iterations to seek the proper parameters and it is defined as the following equation:where after m iterations regarding the opposing divergence, is considered as the mean value.
5. Experimental Results
In this research, we used the EEG eye state corpus which is available in the UCI machine learning repository [58]. The dataset was donated by Rösler and Suendermann [59]. Using EMOTIV EEG neuroheadset, the data were derived from one continuous EEG measurement. In EMOTIV EEG neuroheadset, “FC5,” “F8,” “T7,” “P7,” “O1,” “AF3,” and “AF4” correspond to a maximum increase and “F7,” “T8,” “P8,” “O2” “F3,” “F4,” and “FC6” to a minimum decrease in eye states. The dataset includes 14,980 instances, which are based on 14 attributes with a class for eye states (open/closed). The 14 attributes are for the signals recorded from different locations on the scalp: “FC5,” “F8,” “T7,” “P7,” “O1,” “AF3,” “AF4,” “F7,” “T8,” “P8,” “O2” “F3,” “F4,” and “FC6.” From 14,980, 8,255 records in the corpus are for the eye open and 6,722 samples for the eye closed, as shown in Table 2. 117 seconds were considered for EEG measurement through a camera. The samples 899, 10,387, 11,510, and 13,180 were detected as outliers in the dataset and accordingly were removed. The eye state sensor values for eye-closed samples and eye-open samples are presented in Figures 1(a) and 1(b), respectively.
(a)
(b)
We use SOM clustering to cluster the data in dissimilar groups before the classification task by DBN. In fact, it is believed that clustering by SOM can play an important role in increasing the accuracy of EEG classification by DBN. The results for SOM are presented in Figure 2. The eight clusters presented in this figure are visualized through different principal components generated by PCA. The eight clusters centroids are presented in Table 3. We also present the clusters information in Table 4. According to the results, the numbers of the sample are 771, 2143 1109, 487, 2774, 3160, 3133, and 1399, respectively, in clusters 1–8. In Figures 3 and 4, we present the eye state sensors values versus eye-open and eye-closed samples in the first four clusters of SOM.
(a)
(b)
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
We adopt one-class-one-network to improve the neural network able to overcome the problems of complexity in data processing and the curse of dimensionality in EEG classification (see Figure 5). This strategy has been mainly applied for speech recognition [60], automatic handwriting recognition systems [61], and face recognition [62]. The aim of the global network is to coordinate the outputs of all subnetworks in classification problems. According to its nature, one-class-one-network includes one global network and several subnetworks. It has been an appropriate choice to have a tradeoff between accuracy and complexity of classification issues. In this research, the global network in one-class-one-network is a two-layer back-propagation network and the subnetworks are Deep Belief Networks.
(a)
(b)
The data in each cluster is then used for EEG classification. Accordingly, we divided the data of each cluster into two sets, training and test sets. The Deep Belief Network is applied on the training set to construct the classification models. To avoid Gradient Diffusion (GD), pretraining and fine-tuning are used in Deep Belief Network. The hidden layers in Deep Belief Network are trained layer by layer in the first stage of learning. The hidden layers of the pretraining stage in Deep Belief Network provide the data reconstruction and feature extraction models, which include generative weights and detection weights. In the fine-tuning stage, which is the stage for training the model for features extraction and data reconstruction, the back-propagation algorithm is mainly used. In this stage, squared error as the error signal is calculated. This error is regarding the error between reconstructed visible data and visible data. We use the Wake-Sleep algorithm [63] to avoid GD in this stage and to accelerate the feature extraction convergence in Deep Belief Network.
This study used deep belief network with 2, 4, and 6 RBM layers (see Table 5) with different number of neurons in each layer: [50, 50], [500, 500, 500, 500], and [2000, 1000, 500, 250, 125, 62]. To train the DBN, we set the number of input layer’s neurons equal to the number of input features in the dataset (14 features) and the number of output layer’s neurons equal to the number of classes (two classes). The fine-tuning algorithms and unsupervised training were accomplished for 100 epochs in each run and the batch size was set to 50. For each cluster, these network architectures were considered and the DBN classification models were constructed.
We adopted two performance evaluation metrics to assess the effectiveness of the proposed method, which are accuracy and Area Under Curve (AUC) as aggregate measure of performance [64, 65]. Classification metrics are calculated from True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN), which will form the confusion matrix as shown in Figure 6.
The ROC, which presents a single scalar value and plots the sensitivity versus 1 − specificity, is a widely accepted performance measure for classifiers in machine learning [66, 67]. AUC value for classifiers ranges from 0 to 1, indicating that AUC = 0 for 100% wrong classification and AUC = 1 for 100% correct classification. The curve in ROC plots two parameters: True Positive Rate (TPR) and False Positive Rate (FPR).
The results for the average accuracy of the clusters are presented in Table 6. The ROC results are visualized in Figure 7. All features were considered in our experiments. The results are performed without performing the dimensionality reduction or feature selection. We also compare the results of the proposed method with other classifiers. We selected Gaussian Naïve Bayes, Linear SVM, and Decision Trees as the selected methods for comparison aim. The results of our analysis showed that the method which combines SOM with DBN in DBN A (), DBN B , and has outperformed other classifiers: Linear SVM (), Gaussian Naïve Bayes (), and Decision Trees ().
6. Conclusion
In real-time applications, the development of the individual machine interface for detecting the eye state is considered as an important issue. It is better to consider a few cheap software and hardware that can support complex EEG signals analysis in an automated fashion. This research investigates the impact of supervised and unsupervised learning methods to recognize the eye state by utilizing EEG signal analysis. First, prior to the DBN task, we utilize SOM clustering in order to cluster the experimental data into different groups, where each group includes the similar EEG signals. More interestingly, by performing the clustering method by SOM algorithm, we can see its significant impact on the EEG classification accuracy. The SOM obtained eight clusters with samples number 771, 2143 1109, 487, 2774, 3160, 3133, and 1399 in, respectively, clusters 1–8 to be used in DBN technique. We apply one-class-one-network to develop the ability of the neural network to conquer the complexities and dimensionality curse in the EEG classification. In fact, the aim of using global network was to make a balance regarding the outputs in all the subnetworks in the problems of the classification step. It also has global and other subnetworks and it is a significant choice to use as making a tradeoff between complexity and accuracy of the classification matters. We consider a two-layer BP network for the global network which is in one-class-one-network and subnetworks that are DBNs. After that, the data that is in each cluster was utilized for EEG classification. These data were then separated into two groups, training and test, to, respectively, construct and evaluate the DBN classification models. In order to prevent GD, we utilize pretraining and fine-tuning in DBN. In the step of pretraining, the hidden layers were trained layer by layer and these trained layers build the data and the models of feature extraction. In the next step, that is fine-tuning, our model was trained for extracting the features and reconstructing the data by utilizing BP algorithm. Then, we compute the error, which exists between the visible and reconstructed visible data. Finally, we apply Wake-Sleep method to reduce GD and speed up the feature extraction convergence process.
In this research, we utilize DBN with 2, 4, and 6 layers, which included various neurons numbers in every layer. In order to train this network, the number of neurons in the input layer were quite equal to the input features and the number of neurons in the output layer was equal to the number of classes. Furthermore, we consider 100 epochs in every run for unsupervised and fine-tuning algorithm and 50 is considered for the batch size. These architectures of the network are considered for every cluster and then the models for DBN classification are built. For assessing and evaluating our proposed approach, we applied two performance evaluation factors, which are accuracy and AUC. The results showed that the use of DNB combined with the SOM clustering technique can be effective in classification of EEG for eye state detection. The results further showed that SOM + DBN can outperform other classification techniques, Linear SVM, Gaussian Naïve Bayes, and Decision Trees, in EEG classification.
6.1. Limitations and Future Work
We must acknowledge some limitations. First, this study has used SOM for EEG signals clustering. Hence, other clustering approaches such as Expectation-Maximization (EM) algorithm could be implemented and compared with the SOM results. In addition, SOM was not implemented for ensemble learning. Ensemble approaches in data clustering have produced better results than those of the individual clustering [68]. Hence, different ensemble approaches in SOM or other clustering techniques could be evaluated for recognizing the eye state by utilizing EEG signal analysis. Second, DBN is implemented for nonincremental learning. Hence, it is possible to develop DBN for online learning as incremental learning has demonstrated to be more efficient in data analysis than conventional learning [69, 70]. Thus, in our future work, the impact of DBN with ensemble learning techniques will be investigated on EEG classification accuracy. Specifically, we will try to find an effective solution to speed up the DBN learning process through incremental learning approaches. Additionally, there is room to still improve the accuracy of nonincremental DBN through the use of optimization techniques such as Grasshopper Optimization Algorithm (GOA) [71] for EEG signal analysis. This may be effective to reduce the training error in DBN.
Data Availability
The data are freely available at UCI Machine Learning Repository.
Conflicts of Interest
The authors declare that they have no conflicts of interest regarding the publication of this paper.