Brain-Computer Interface for Control of Wheelchair Using Fuzzy Neural Networks
The design of brain-computer interface for the wheelchair for physically disabled people is presented. The design of the proposed system is based on receiving, processing, and classification of the electroencephalographic (EEG) signals and then performing the control of the wheelchair. The number of experimental measurements of brain activity has been done using human control commands of the wheelchair. Based on the mental activity of the user and the control commands of the wheelchair, the design of classification system based on fuzzy neural networks (FNN) is considered. The design of FNN based algorithm is used for brain-actuated control. The training data is used to design the system and then test data is applied to measure the performance of the control system. The control of the wheelchair is performed under real conditions using direction and speed control commands of the wheelchair. The approach used in the paper allows reducing the probability of misclassification and improving the control accuracy of the wheelchair.
Human brain control of wheelchairs for physically disabled people has attracted great attention due to their convenience and relatively low cost, high mobility, and quick setup. The measurement of human brain signals and converting them into control signals require the development of an interface between the brain and the computer. A brain-computer interface (BCI) system provides communication between computer and mind of pupils. This interface can be based on brain activity during muscular movements or the changes of the rhythms of brain signals . These brain activities can be detected using electroencephalographic (EEG) signals. BCI transforms the EEG signals produced by brain activity into control signals which can be later used for controlling the wheelchair without using any physical controls. Since the brain signals are very weak, we need to apply amplifiers and some spatial and spectral filters to the EEG signals in order to extract the features of these signals. The detected EEG signals are based on the change of frequencies and change of amplitudes. For example, during voluntary thoughts, the frequencies of signals change, and during movement, synchronisation/desynchronisation of brain activity which involves μ rhythm amplitude change happens. This relevant characteristic makes rhythm based BCI suitable to be used.
Recently, some research works have been done to develop many applications of BCI for wheelchairs. The main function of BCI is to convert and transmit human intentions into appropriate motion commands for the wheelchairs, robots, devices, and so forth. BCI allows improving the quality of life of disabled patients and letting them interact with their environment. Reference  presents the application of BCI and control of wheelchair in an experimental situation. The research considers the driving of a simulated wheelchair in a virtual environment before using BCI in a real situation. The virtual reality (VR) decreases the number of dangerous situations by using training and testing applications. Reference  describes a BCI system which controls the wheelchair that moves in only one direction: move forward. In [4, 5], BCI is designed for control of wheelchair using three possible commands: turn left, turn right, and move forward. In , BCI is designed using EEG signal captured by eight electrodes. Wavelet transform was used for feature extraction and the radial basis networks were used to classify the predefined movements. In , controller based on the brain-emotional-learning algorithm is used to control the omnidirectional robot. Reference  presents the design of an asynchronous BCI based control system for humanoid robot navigation using an EEG. Reference  considers a noninvasive EEG-based brain-computer interface system to achieve stable control of a low speed unmanned aerial vehicle for indoor target searching. References [10–15] consider the design of brain-controlled wheelchair. The construction of viable brain-actuated wheelchair that combines BCI with a commercial wheelchair, via a control layer, is considered. Combining the BCI with shared control architecture  allows for dynamically producing intuitive and smooth trajectories. The processes of feature extraction and classification are very important in BCI design and they have a great effect on the performance of the BCI system. Set of research works has been done for improvement of the feature extraction and classification algorithms [12–19]. References [16, 17] consider feature extraction algorithms for BCI. Reference  uses adaptive common spatial patterns for feature extraction.
Different clustering algorithms based on support vector machines, linear discriminant analysis, and neural networks are applied for classification of brain signals. Reference  uses features, optimised in the sense of statistically significant and potentially discriminative coherences at a specific frequency, and applies linear discriminant analysis (LDA) for classification purpose. SVM  and LDA  are used for classification purpose of brain signals.
Recently, several soft computing techniques are used for recognition of brain activity [21–23]. Reference  uses fuzzy logic and  uses neural networks with fuzzy particle swarm optimisation for BCI design. In , continuous wavelet transform is used to extract highly representative features and then an Adaptive Neuron-Fuzzy Inference System (ANFIS) is applied for classification. The systems based on fuzzy logic can make classifications using vague, imprecise, noisy, or missing input information. On given problem, human perception process can be efficiently modelled using fuzzy logic.
As shown, feature extraction and classification play an important role in the design of brain-based control for obtaining high classification accuracy. In the BCI design, high classification rate is very important. Otherwise, the presence of errors can cause initiation of a wrong command that can lead to dangerous situations. Therefore, achieving low error rates keeps the users safe. Different clustering algorithms based on support vector machines, linear discriminant analysis, and neural networks are applied for classification of brain signals . Fuzzy classification represents knowledge more naturally to the way of human thinking and is more robust in tolerating imprecision, conflict, and missing information. In this paper, the fuzzy neural network structure is proposed for the design of BCI in order to achieve efficient brain-based control of wheelchair. In the literature, different neural and fuzzy networks are proposed for solving various problems [23–29]. In [23–26], FNN structure is designed for control of dynamic plants. In [27–29], neurofuzzy inference systems are designed for classification and control purposes. The systems designed in these papers are used for special purposes. In the neurofuzzy structures, the rules are constructed using all possible combinations of inputs and cluster centres. The problems that are characterised by multiple inputs and multiple outputs will have a huge number of fuzzy rules. The constructions of such systems are not efficient and these systems have a huge number of parameters. In this paper, the number of rules is selected using the clustering results which is equal to the number of cluster centres. In this paper, in order to improve the performance of classification system, a multi-input and multioutput fuzzy neural system (FNS) based on TSK rules is proposed for classification of the ECG signals.
The paper is organised as follows. Section 2 presents the architecture of BCI system based on FNN. Signal processing and feature extraction stages have been described. Section 3 presents classification algorithm based on FNN. Section 4 presents parameter updates rule used for FNN. The fuzzy means classification and gradient descent algorithms are applied for updating parameters of FNN. Section 5 gives experimental results obtained for FNN based BCI system. Section 6 presents conclusions of the paper.
2. BCI System Architecture
Figure 1 depicts BCI based control of the wheelchair. BCI system consists of an Emotiv headset connected to a computer. Emotive sensors supply information to the computer. The computer runs the signal processing and classification algorithms and is connected to a microcontroller that controls the movement of the wheelchair. The wheelchair can move in four directions. The speed of the wheelchair is taken as constant and the wheelchair can be switched on and off in the case of necessity. Taking into account the abovementioned functionality, the BCI system uses the following commands: move forward, move backward, turn left, turn right, and turn on and turn off the switch.
A BCI based control system is usually composed of five main units: signal acquisition unit, signal preprocessing unit, feature extraction unit, classification unit, and action unit that controls motors of the wheelchair. The main units of the decision system are represented in Figure 1. In signal acquisition block, the EEG signals are captured using the Emotiv headset. Emotiv EPOC is an EEG headset which supplies 14-channel EEG data (Figure 2) and 2 gyros for 2-dimensional controls. Its features are adequate for a useful BCI (resolution and bandwidth). Our system uses upper face gestures for actuation commands; since most Emotiv sensors are located in the frontal cortex, they are the most reliable signals to detect. The EEG input signals are sent to the signal preprocessing unit for filtering and scaling and sent to the feature extraction block. In this block, the basic features are extracted and sent to the classification system. The classification block processes the input signals and outputs the control instructions. Later, these control instructions are sent to the motors of the wheelchair.
The EEG signals measured by Emotiv headset are first processed by signal preprocessing and feature extraction blocks. Signal preprocessing block filters the noises and scales the signals in a certain interval. These signals are very long and need certain time for processing. Therefore, the feature extraction technique is applied in order to decrease the signal size and extract more important features for classification. In the paper, we used fast Fourier transform (FFT) for extraction of the features of the input EEG signals. Figure 3 presents the operations used in the feature extraction stage. The input signal received from the headset is divided into windows having 2 sec time interval with 50% overlap. The use of overlapping windows allows us to increase the accuracy of the classification. Each two-second window corresponds to 256 samples of data. Each second headset returns 128 data samples. The obtained signals from the channels, stored as windows, are then sent to normalisation block. Each channel is normalised in order to centre each channel on zero by calculating the mean value of each channel for the window and then subtracting it from each of the data points in the channel. After normalisation, Hamming window is applied to each channel in the window. EEG signals do not generally repeat exactly, over any given time interval, but the math of the Fourier transform assumes that the signal is periodic over the time interval. This mismatch leads to errors in the transform called spectral leakage. Hamming window is used to mitigate this problem. Then, fast Fourier transform (FFT) is applied to each channel in the window to find out the frequency components of the signal. Each frequency component is used as a feature, which results in features. In order to increase the performance of the classification, the features are ranked by evaluating the worth of a frequency by measuring the information gain with respect to the class. The expected information gain is the change in information entropy () from a prior state to a state that takes some information as given.
Information gain selects a subset of the original representation attributes according to Information Theory quality metric. This method computes the value of the metric for each attribute and ranks the attributes. Then, it simply decides a threshold in the metric and keeps the attributes with a value over it.
After frequency representation, all channels in the window are combined into a single unit so as to apply classification on all channels at once. The filtering operation is applied in order to select important features of the brain signals. These features are used for classification purpose.
Besides the above-described approach, we can use also another approach for signal processing. In the second approach, the acquired brain signal after windowing, normalisation, and combining operations is used for classification purpose:
In the paper, we use frequency representation of signals for classification. These signals are processed and classified. The output of classification system is used to control the wheelchair. Even though during training system reports 100% success rate in real-world conditions, it does misclassify, a state machine is used to further increase safety and reduce misclassification. As an example, the system will not transition from forward motion to backward motion without stopping in neutral. The output of the state machine drives the microcontroller which controls the motors on the wheelchair. The number of classes is equal to the number of control actions.
3. FNN Based Classification
The features extracted from the EEG signals are used for classification and determining control action. In this paper, we propose a novel approach which is based on FNN for the classification of brain signals. The extracted features are input signals of the FNN based classifier. The classifier based on the extracted features classifies the signals into the following six classes: move forward, move backward, turn left, turn right, and turn on and turn off the switch. The design of FNN includes the development of the fuzzy rules that have IF-THEN form. This is implemented by dint of optimal definition of the premise and consequent parts of fuzzy IF-THEN rules for the classification system through training of fuzzy neural networks. In the paper, the Takagi-Sugeno-Kang (TSK) types of IF-THEN rules that have a fuzzy antecedent and crisp consequent parts are used. The TSK-type fuzzy system approximates nonlinear system with linear systems and has the following form:Here, and are input and output signals of the system, respectively, is the number of input signals, and is the number of rules. are input fuzzy sets; and are coefficients.
The structure of fuzzy neural networks used for the classification of EEG signal is based on TSK-type fuzzy rules and is given in Figure 4. The FNN consists of six layers. The first layer is used to distribute the input signals. The second layer includes membership functions. Here, each node represents one linguistic term. Here, for each input signal entering the system, the membership degree where input value belongs to a fuzzy set is calculated. In the paper, the Gaussian membership function is used to describe linguistic terms. where are centre and width of the Gaussian membership functions, respectively. is membership function of the th input variable for the jth term. m is a number of input signals; r is a number of fuzzy rules (hidden neurons in the third layer).
The third layer is a rule layer. Here, the number of nodes is equal to the number of rules. Here, represents the rules. The output signals of this layer are calculated using t-norm min (AND) operation:where is the min operation.
These signals are input signals for the fifth layer. The fourth layer is a consequent layer. It includes linear systems. Here, the values of rules output are determined as
In the next fifth layer, the output signals of the third layer are multiplied by the output signals of the fourth layer. The output of th node is calculated as
In the sixth layer, the output signals of FNN are determined asHere, are the output signals of FNN (). After calculating the output signal, the training of the network starts.
The design of FNN (Figure 4) includes determination of the unknown parameters that are the parameters of the antecedents and () and the consequents () parts of the fuzzy IF-THEN rules (2). In the next section, the training of the parameters of FNN is presented.
4. Parameter Updates
In the fuzzy IF-THEN rules (2), the antecedent part represents the input space by dividing the space into a set of fuzzy regions and the consequent part describes the system behaviour in those regions. In the design of FNN model, the basic problem is the determination of the unknown parameters of antecedent and consequent parts. Recently, a set of different approaches has been applied for designing fuzzy IF-THEN rules. These are clustering [30–35], gradient algorithms [24–27, 34–36], the least-squares method (LSM) [27, 33], and genetic algorithms [27, 34].
In this paper, the fuzzy clustering and gradient descent algorithms are applied for determining the parameters of FNN. The basic parameters of the antecedent part are the centres and widths of the membership functions. Learning of FNN starts with the update of parameters of antecedent part of IF-THEN rules, that is, the parameters of the second layer of FNN (Figure 4). For this purpose, FCM is applied in order to partition input space and construct antecedent part of fuzzy IF-THEN rules. In the result of partitioning the cluster centres are determined. These centres correspond to the centres of the membership functions used in the input layer of FNN. Using the distances between the cluster centres, the widths of the membership functions are determined.
After finding the parameters of the antecedent’s parts, the design of the consequent part of the fuzzy rules starts. For this purpose, the gradient descent algorithm is applied for the parameter update of the consequent part, that is, the parameters of the fourth layer of FNN. In learning of FNN, 10-fold cross validation is applied for separation of the data into training and testing set.
The initial values of the parameters of consequent parts are generated randomly. The training of the parameters has been carried out using errors calculated on the output of the network. For generality, we have given the learning procedure of all parameters of FNN using gradient descent algorithm with adaptive learning rate. The adaptive learning rate used guarantees the convergence and speeds up the learning process. In addition, the momentum is also used to speed up the learning processes.
The error in the output of the network is calculated asHere, is the number of output signals of the network; are desired and current output values of the network (), respectively. The parameters () in consequent part of network and the parameters of membership functions and () in the premise part of FNN structure are adjusted aswhere m is the number of input signals of the network (input neurons) and is the number of fuzzy rules (hidden neurons). is the learning rate; is the momentum.
The derivatives in (9) are computed as
In (10), the derivatives are determined as Here, .
5. Experiments and Results
The BCI system is simulated and used in real life applications. The EEG signals are measured with signal acquisition unit, the Emotiv EPOC headset. In the experiments, we have utilised 14 channels for measuring EEG signals. The measured EEG signals have different rhythms within the frequency band. The experiments show that measuring brain signals is difficult so we have tested our system using brain muscle signals. As an example, the signals obtained from 5 sample channels are shown in Figure 5. Figure 5(a) depicts a neutral pose, patient relaxing and not doing anything. Figure 5(b) depicts a positive gesture. As shown in the figures, the EEG signals with positive gesture pose are changing more frequently than a neutral pose. In the paper, the FFT is applied to extract important features of the signal. After the preprocessing stage, given in Section 2, the important features of these signals are extracted and used for classification purpose. The number of extracted features was determined as 100. These signals are inputs for FNN system. Outputs of FNN model are clusters. The following clusters are used in the experiment: move forward, move backward, turn left, turn right, and turn on and turn off the switch. For each cluster, the system recorded 10 seconds of data.
In this paper, the classification of the EEG signals is performed using FNN model. For this purpose, the FNN structure with hundred input and six output neurons is generated. In the papers [27, 29], the neurofuzzy systems have been efficiently applied for different classification problems. If we use these structures for 100 inputs and 2 cluster centres, 2100 rules should be generated. The rules are constructed using all possible combinations of inputs and cluster centres. This is a very large number. In this paper, the number of rules is selected according to the clustering results, equal to cluster centres.
Fuzzy means classification is used in order to design the premise parts of (2) and to determine the parameters of Gaussian membership functions used in the second layer of FNN. In experiments, different cluster numbers, 5, 6, 9, and 16, are used to design FNN structure. These experiments have been done in order to increase the performance of classification system. At first, FCM clustering is used for the input space with 6 clusters for each input. Six fuzzy rules are designed using a different combination of these clusters for 100 inputs. After clustering input space gradient descent algorithm is applied for learning of consequent parts of the fuzzy rules, that is, parameters of the 4th layer of FNN. In learning of FNN, 10-fold cross validation is used for separation of the data into training and testing set.
The initial values of the parameters of FNN are randomly generated in the interval and, using the gradient algorithm derived above, they are updated for the given input-output training pairs. As a performance criterion, RMSE is used. Figure 6 depicts the evolution of the RMSE values over 1000 epochs.
For training of the FNN, 1000 epochs are used. As a result of training, the values of the parameters of the FNN system were determined. Once the FNN has been successfully trained, it is then used for the classification of the EEG signals. During learning, the value of RMSE was obtained as 0.223264 for training data and 0.241625 for evaluation. After learning, for the test data, the value of RMSE was obtained as 0.257986 with 100% accuracy of classification. Figure 6 depicts RMSE values obtained during training. The design of FNN model is performed using a different number of rules. Table 1 includes results of simulations with 5, 6, 9, and 16 rules, respectively. As shown, accuracy of FNN classification model is 100%.
For comparison purpose, we test the system using different classification techniques. As a result of the classification, the following results are obtained (Table 2). In the table, the classification results of FNN model are compared against linear logistic regression model , SVM with various kernels, multilayer perceptron (MLP) with various hidden layers, Naïve Bayes classifier , Random Tree, and Random Forest . As shown, the simulation results demonstrate the efficiency of application of FNN model in the classification of EEG signals. These clusters activate the corresponding control signal which is then used to actuate the motors of the wheelchair.
The paper presents the design of BCI based on FNN for a wheelchair. The emotional and muscular states of the user are evaluated for control purposes. The design of BCI has been done to actuate a brain-controlled wheelchair using six mental activities of the user: move backward, move forward, turn left, turn right, turn on, start, and stop. For classification of EEG signals, the FNN with 10-fold cross validation data set is used. The design of the FNN system is implemented using fuzzy means classification and gradient descent algorithm. The obtained 100% classification results prove that the used techniques are a potential candidate for the classification of the EEG signals in the design of brain-based control system. In the future, we are going to improve the number of commands for control of wheelchair and decrease detection time of the EEG signal used for measuring brain activities and design efficient brain-controlled wheelchair.
The authors declare that they have no competing interests.
F. Velasco-Álvarez and R. Ron-Angevin, “Asynchronous brain-computer interface to navigate in virtual environments using one motor imagery,” in Bio-Inspired Systems: Computational and Ambient Intelligence, J. Cabestany, F. Sandoval, A. Prieto, and J. M. Corchado, Eds., vol. 5517 of Lecture Notes in Computer Science, pp. 698–705, Springer, Berlin, Germany, 2009.View at: Publisher Site | Google Scholar
F. Galán, M. Nuttin, D. Vanhooydonck et al., “Continuous brain-actuated control of an intelligent wheelchair by human EEG,” in Proceedings of the 4th International Brain-Computer Interface Workshop and Training Course, pp. 315–320, TU Graz/Büroservice, Graz, Austria, 2008.View at: Google Scholar
R. Leeb, D. Friedman, G. R. Müller-Putz, R. Scherer, M. Slater, and G. Pfurtscheller, “Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegic,” Computational Intelligence and Neuroscience, vol. 2007, Article ID 79642, 8 pages, 2007.View at: Publisher Site | Google Scholar
A. Fattouh, O. Horn, and G. Bourhis, “Emotional BCI control of a smart wheelchair,” International Journal of Computer Science Issues, vol. 10, no. 3, pp. 32–36, 2013.View at: Google Scholar
V. Khare, J. Santhosh, S. Anand, and M. Bhatia, “Brain-computer interface based real time control of wheelchair using electroencephalogram,” International Journal of Soft Computing and Engineering (IJSCE), vol. 1, no. 5, 2011.View at: Google Scholar
M. A. Sharbafi, C. Lucas, and R. Daneshvar, “Motion control of omni-directional three-wheel robots by brain-emotional-learning-based intelligent controller,” IEEE Transactions on Systems, Man and Cybernetics—Part C: Applications and Reviews, vol. 40, no. 6, pp. 630–638, 2010.View at: Publisher Site | Google Scholar
T. Carlson, R. Leeb, R. Chavarriaga, and J. del R Millan, “The birth of the brain-controlled wheelchair,” in Proceedings of the 25th IEEE/RSJ International Conference on Robotics and Intelligent Systems (IROS '12), pp. 5444–5445, Vilamoura, Portugal, October 2012.View at: Publisher Site | Google Scholar
R. H. Abiyev, N. Akkaya, E. Aytac, I. Günsel, and A. Çağma, “Brain based control of wheelchair,” in Proceedings of the World Congress in Computer Science, Computer Engineering, and Applied Computing (ICAI '15), Las Vegas, Nev, USA, July 2015.View at: Google Scholar
M. Kaur and P. Tanwar, “Developing brain-computer interface using fuzzy logic,” International Journal of Information Technology and Knowledge Management, vol. 2, no. 2, pp. 429–434, 2010.View at: Google Scholar
R. Chai, S. H. Ling, G. P. Hunter, Y. Tran, and H. T. Nguyen, “Brain-computer interface classifier for wheelchair commands using neural network with fuzzy particle swarm optimization,” IEEE Journal of Biomedical and Health Informatics, vol. 18, no. 5, pp. 1614–1624, 2014.View at: Publisher Site | Google Scholar
S. Darvishi and A. Al-Ani, “Brain-computer interface analysis using continuous wavelet transform and adaptive neuro-fuzzy classifier,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, vol. 2007, pp. 3220–3223, Lyon, France, August 2007.View at: Google Scholar
R. H. Abiyev, R. A. Aliev, and R. R. Aliev, “The synthesis of fuzzy control system with neural networks based tuner,” News of Academy of Sciences, Tech. Cybernetics, no. 2, pp. 192–197, 1994.View at: Google Scholar
R. A. Aliev, R. H. Abiyev, and R. R. Aliev, “Synthesis of industrial neural controllers,” in Proceedings of the IEEE International Conference on Neural Networks, 1994. IEEE World Congress on Computational Intelligence, vol. 1–7, pp. 1644–1649, San Diego, Colif, USA, July 1994.View at: Google Scholar
R. Abiyev, “Neuro-Fuzzy system for technological processes control,” in Proceedings of the 6th World Multi-Conference on Systemics, Cybernetics and Informatics, vol. 12, SCI-20002, pp. 249–253, Orlando, Florida, USA, July 2002.View at: Google Scholar
J.-S. R. Jang, C.-T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing, chapter 17, Prentice-Hall, New Jersey, NJ, USA, 1997.
J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algoritms, Plenum Press, New York, NY, USA, 1981.
S. L. Chiu, “Fuzzy model identification based on cluster estimation,” Journal on Intelligent Fuzzy Systems, vol. 2, pp. 267–278, 1994.View at: Google Scholar
K. Demirli and P. Muthukumaran, “Higher order fuzzy system identification using subtractive clustering,” Journal of Intelligent and Fuzzy Systems, vol. 9, no. 3-4, pp. 129–158, 2000.View at: Google Scholar
G. H. John and P. Langley, “Estimating continuous distributions in bayesian classifiers,” in Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, pp. 338–345, San Mateo, Calif, USA, 1995.View at: Google Scholar