Abstract
While the world is stepping into the aging society, rehabilitation robots play a more and more important role in terms of both rehabilitation treatment and nursing of the patients with neurological diseases. Benefiting from the abundant contents of movement information, electroencephalography (EEG) has become a promising information source for rehabilitation robots control. Although the multiple linear regression model was used as the decoding model of EEG signals in some researches, it has been considered that it cannot reflect the nonlinear components of EEG signals. In order to overcome this shortcoming, we propose a nonlinear decoding model, the particle filter model. Two- and three-dimensional decoding experiments were performed to test the validity of this model. In decoding accuracy, the results are comparable to those of the multiple linear regression model and previous EEG studies. In addition, the particle filter model uses less training data and more frequency information than the multiple linear regression model, which shows the potential of nonlinear decoding models. Overall, the findings hold promise for the furtherance of EEG-based rehabilitation robots.
1. Introduction
With the growth of the aging population, treatment needs for nervous system diseases (e.g., spinal cord injury or stroke) have become bigger and bigger. As an advanced form of medical technology, rehabilitation robots have great potential in terms of both rehabilitation treatment and nursing. In recent years, the rehabilitation robot has become a research hotspot in the fields of brain science, biomedical and rehabilitation engineering, intelligent information processing, bionics, and so on. The control strategies of rehabilitation robots include force control, force field control, and bioelectrical signal (EMG, EEG, etc.) control. Brain signals are recorded by electrodes, reflecting the summation of the synchronous and rhythmic activity of neurons. Compared with EMG, brain signals contain more motion patterns and are available for more people; they are thus more suitable as an information source for rehabilitation robots. Brain signals can be obtained by invasive or noninvasive methods, though signals acquired by invasive methods have high signal-to-noise ratio and spatial resolution, and invasive method-based rehabilitation robot research has made great strides in animal experiments [1–4].
Due to the inherent risks of surgery and the gradual degradation of signal integrity, the invasive method is difficult to promote in clinical application. Although neuronal data acquired noninvasively from the scalp via electroencephalography (EEG) has comparatively low signal-to-noise ratio and spatial resolution, it is more suitable to be used in rehabilitation robots and in clinical applications because of its directness, its security, the simplicity of its acquisition equipment, its easy operation, its lower cost, and its fewer environmental restrictions.
In recent years, researchers have tried to read human mind from the EEG to determine the movement intent of people to achieve a noninvasive intelligent prosthetic control using, for example, the motor imagery-based rehabilitation system [5], the MindWalker of the Twente University [6], and the lower exoskeleton control system studied by the University of Houston [7]. Researchers, who study the EEG signals, concentrated more on pattern classification and feature extraction based on event-related potentials [8], visual evoked potential [9], and mental tasks [10].
In order to better control prostheses using EEG, researchers have been trying to get more movement information from the EEG signal besides pattern recognition or time-frequency analysis to identify the intent to achieve movement. For instance, Zhao et al. used the duration of a specific image to define the intensity of the task control commands, thus providing continuous control of an additional parameter [11]. Logar controlled the clamping force according to the phase demodulation method based on EEG [12]. The literature [13] analyzed the relationship between the actual movement velocity and the image. But, this movement information for intelligent prosthesis control is also far less.
Researchers have conducted studies on motion information extraction from EEG based on research on motion information extraction methods from an invasive signal (mainly decoding method includes linear filtering [14–16], Kalman filtering [17, 18], linear equation of state [19], and support vector machines [20]). In 2009, Bradberry et al. proposed a method to continuously decode hand position, velocity, and acceleration from 55-channel EEG signals and established a mapping model between EEG and motion information [14]. On this basis, in 2010, Bradberry continuously decoded 2D center-out movements, unconstrained 3D center-out movements, and 3D finger gestures, showing that EEG signals also contain rich motion information in the macroscale movement [15]. Lv et al. used a Kalman filter to predict the hand velocity in a “self-routed” movement based on features such as amplitude and power spectrum [21]. Antelis et al. established a mapping model between EEG and motion information by training the recorded trajectory from the initial point to the fixed point or any target point and the features of EEG in the spatial and time-frequency domain through the support vector machine (SVM) [22]. However, the researches above are limited to the decoding of specific limb trajectory. In 2011, Presacco et al. proposed a method that decoded unconstrained treadmill walking from EEG [16] and successfully obtained the linear and angular kinematics of the ankle, knee, and hip joints during walking.
During research on motion information extraction from EEG, most researchers have used a multiple linear regression decoding model based on a neural decoding method from invasive signals. Antelis et al. pointed out that the use of a linear regression model implies that the relevant EEG component has to be in the same frequency range as the signal to be decoded and suggested that a nonlinear model should be used to relate the limb kinematics to EEG temporal sequences [23]. Because EEG is a recording of electrical activity along the scalp, which comes from the neurons via the skull, the conductivity of the skull is nonlinear. The multiple linear regression model cannot reflect the nonlinear component of EEG.
Some nonlinear invasive neural decoding methods such as neural networks, support vector machines, and particle filter [24] for the movement decoding from EEG provide a reference. Particle filter is a technique for implementing a recursive Bayesian filter by Monte Carlo simulations. The key idea is to represent the required density function by set of random samples (particles) with associated weights. Particle filtering algorithm has been successfully applied in invasive neural decoding. For example, Wood et al. used particle filtering to recursively infer hand kinematics and attentional state conditioned on neural firing rates with a monkey [25]. Kelly and Lee decoded the V1 neuronal activity using particle filtering with Volterra kernels [26]. Gao et al. described the Bayesian decoding of hand motion from firing activity using a particle filter [27].
This paper proposes a nonlinear decoding model based on a particle filter. The multiple linear regression model and particle filter model are evaluated and compared using two- and three-dimensional hand motion decoding experiments.
2. Decoding Methods
2.1. Decoding Model by Multiple Linear Regression
The multiple linear regression model assumes that the kinematics of the hand are related to the EEG signals at present and the EEG signals at the previous moment and assumes that the relationship between the EEG signals of each channel is linear. The corresponding weight of each EEG channel can be obtained through multiple linear regression, and the decoding model is given by where , , and are, respectively, the horizontal, vertical, and depth position of the hand at time sample , is the number of EEG channels, is the number of time lags, is the voltage measured at EEG channel at time lag , and the and variables are weights obtained through multiple linear regression.
2.2. Decoding Model by Particle Filter
We have implemented a particle filtering method for reconstructing hand movement information from EEG signals. And, in particle filter decoding (or Bayesian decoding), the object is to find, for each time , the distribution of the unobserved signal (where has been used to represent the position of hand at time step , with two- or three-dimensional vector) that is the key position information, given observations , while the observation represented the vector of EEG. Hence, we view the decoding problem as a statistical inference problem in which we could get a Bayesian estimate of the posterior at every time step. Making certain independence and first-order Markov assumptions leads to a recursive estimate of the posterior: where is a normalizing constant.
Particle filter decoding consists of two statistical models: a state motion model (or temporal prior),, for a process describing the evolution of the state we are trying to predict (here, position of hand) and (2) a measurement model (or an observation model, or likelihood), , specifying the probability distribution of the data given the underlying state .
2.2.1. The State Motion Model
The state motion model describes the distribution of the unobserved signal one step in the future, , given the current value of the signal . Here, we use a second-order model, and the state motion model is as follows: where is the key position information, is the transfer matrix, and is the Gaussian noise.
2.2.2. The Measurement Model
The measurement model specifies the relationship between the unobserved signal and the observation . We assume the conditional independence of the EEG channels where the likelihood for the EEG signals is taken to be a Gaussian distribution. So, with some other usual assumption [23], the measurement model can be expressed as where the -dimensional vector is the mean and the covariance is the positive definite matrix . The main steps of the particle filter decoding model are as follows.(1)Initialization. Produce the particle swarm by the prior probability, and the weight of each particle is .(2)Update. Update the particle weights at time : Then, normalize The least mean-square estimate of the unknown parameter at time can be obtained from (3)Resampling. Get a new particle collection .(4)Prediction. Use the state equation to predict the unknown parameter .(5)At time , return to step (2).
We use one set of experimental data to train the particle filter model and another experimental data set to verify the model. The Pearson correlation coefficient between the measured and reconstructed hand positions was computed.
3. Experimental Design
After giving informed consent, six healthy, right-handed subjects, aged 20–25 (4 men and 2 women) and with no history of neurological disease, participated in the experiments. The experiments included two- and three-dimensional hand motions. To verify the applicability of the two decoding methods, we first tested their validity in two-dimensional hand motion, and then we further tested their validity in three-dimensional hand motion.
3.1. Two-Dimensional Hand Motion Experiment
As shown in Figure 1, participants sat upright in a chair in front of the computer screen; the chair could be adjusted to obtain an appropriate height. During the experiment, participants were instructed to move their right arm/finger to track a computer-controlled cursor that moved along a spiral line in two dimensions on the computer screen. Meanwhile, EEG of participants was acquired. Participants were asked to keep other parts of their body except the right arm still and not to blink to reduce EMG and ocular artifacts. The most frontal electrodes (FP1, FP2) were removed offline from participants, as they are usually contaminated by eye blinks.
(a)
(b)
3.2. Three-Dimensional Hand Motion Experiment
As Figure 2 shows, while wearing an EEG cap, participants held a target object whose three-dimensional motion was tracked by the optical tracking system. Participants swung their hands within a certain range in space slowly and freely. The body of participants remained still except for hand motions to minimize interference from EMG during the experiment, avoiding blinks to reduce ocular artifacts. The EEG signals from the FP1 and FP2 channels were also removed in the three-dimensional motion decoding. The optical tracking system was a PST IRIS motion capture device (PS-tech, Amsterdam, The Netherlands) with acquisition frequency of 120 Hz.
(a)
(b)
A Neuroscan NuAmps Express system (Compumedics Ltd., VIC, Australia) was used to acquire EEG signals with the reference on the right mastoid process behind the right ear. The number of EEG channels, collection frequency, notch frequency, and low-pass cut-off frequency were 30, 500 Hz, 50 Hz, and 100 Hz, respectively. The location of 30 electrodes according to the extended international 10–20 system is shown in Figure 3.
In the two experiments, the hand movement and EEG signals were both acquired with timestamps. And they can be synchronized according to the timestamps.
3.3. Preprocessing
Vertical electroocular signals (VEOG) were measured with two electrodes attached superior and inferior to the orbital fossa of the left eye. And horizontal electroocular signals (HEOG) were measured with two electrodes attached to external canthi. In EEG signals recorded by a DC amplifier, baseline drift can occur artificially and simultaneously. First, the effects of baseline drift were removed. And the eye movements were removed from the EEG using a regression analysis. Then, the EEG signals were filtered with a 5th-order, low-pass Butterworth filter with a cut-off frequency of 2 Hz. The movement time of the small ball along the spiral line was 48 s in the two-dimensional experiment, and the number of coordinate positions of the small ball was 3079, so the ball movement sampling frequency (i.e., the sampling frequency of the hand motion) was about 64 Hz. Subsequently, the signals from each EEG channel were resampled from 500 Hz to 64 Hz, to ensure the same sampling frequency as that of the hand motion. The sampling frequency of the hand motion was 120 Hz in the three-dimensional experiment, so the EEG signals needed to be resampled from 500 Hz to 120 Hz. Then, the signals from each EEG channel were standardized according to the following: where is the normalized EEG signals, is the mean of EEG signals, and is the standard deviation of EEG signals.
The , , and coordinates of the hand position during the hand movement were also standardized using the same equation. Finally, the hand position was decoded from EEG signals using the multiple linear regression and particle filter models. The entire process is shown in Figure 4.
4. Results
4.1. Multiple Linear Regression Model
For each subject, data of 10 trials was collected under each experimental condition (2D, 3D (data length: 8 s, 15 s, and 30 s)). And 6 trials were selected from all the 10 trials according to the state of the subjects (concentration, movement of the body, etc.) during the trial. A 6 × 6-fold cross-validation procedure was employed to assess the reconstruction accuracy of the hand position from the EEG signals. In this procedure, the data of 5 trials was used for training; the remaining 1 trial data was used for testing. Figures 5(a) and 5(b) show, respectively, examples of the measured (red) and reconstructed (blue) hand positions in two and three dimensions in terms of decoding accuracy. The lengths of the data in the three-dimensional experiment include 8 s, 15 s, and 30 s. The 6 × 6-fold cross-validation procedure was used for every data length. In Figure 5(b), Test Data 1 and Test Data 4 are the results for data lengths of 8 s, Test Data 2 and Test Data 5 are the results for data lengths of 15 s, and Test Data 3 and Test Data 6 are the results for data length of 30 s. Table 1 reports the Pearson correlation coefficients in the two- and three-dimensional experiments for Subject 1.
(a) The results of the two-dimensional experiment
(b) The results of the three-dimensional experiment
4.2. Particle Filter Model
As a nonlinear decoding model, the particle filter model can reflect the nonlinear components of EEG signals and only needs one set of experimental data to train the model. For the two-dimensional experiment, we used Test Data 1 to train the particle filter model and the other five sets of experimental data to verify the model. The measured (red) and reconstructed (blue) hand positions in two dimensions are shown in Figure 6(a). For the three-dimensional experiment, we used the particle filter model for data of lengths 8 s, 15 s, and 30 s; the measured (red) and reconstructed (blue) hand positions in three dimensions are shown in Figure 6(b). The Pearson correlation coefficients of Subject 1 in the two-dimensional and three-dimensional experiments are reported in Table 2.
(a) The results of the two-dimensional experiment
(b) The results of the three-dimensional experiment
The measured positions and decoded positions of hand movement in 2- and 3-dimensional spaces are shown in Figure 7. In order to avoid mess, we only selected a part of the hand trajectory in 3-dimensional space. As can be seen from Figure 7, the reconstructed position curve fits the measured position curve well.
(a)
(b)
With the multiple linear regression model, the mean and SD of the correlation coefficients across cross-validation procedure for all subjects are reported in Table 3, while the results of the particle filter model are shown in Table 4.
In the two-dimensional experiment, multiple linear regression model and particle filter model have similar decoding accuracy (Figure 8). However, the three-dimensional decoding results of the multiple linear regression model are a little bit better than those of the particle filter model. The standard deviation of from particle filter model is smaller, which means that it is more stable. It is evident that the decoding accuracy among subjects is variable. The results of Subject 1 are the best because he has conducted the experiments many times and became more adaptive.
The topographies of the contribution of each electrode in the multiple linear regression model (at the best lag) and in the particle filter model were plotted in Figure 9. For the particle filter model, we used EEG signals of only one electrode to decode hand movement each time. And the correlation coefficient got from one electrode decoding was used to represent the contribution of this electrode. After the one electrode decoding procedure was used in all 30 electrodes, we can get the contribution of each electrode in the particle filter model. The topography of multiple linear regression model shows contributions from primary motor sensory area and occipital region. For the particle filter model, electrode locations at F4, Fc4, P3, P6, and TP8 are relevant for decoding right hand movement. It can be inferred that neural information about right hand movement is distributed across both hemispheres and the areas contributing to decoding are different in two decoding models.
(a)
(b)
5. Discussion
As can be seen from Figure 5, the reconstructed curves in the and directions in the two-dimensional experiment and the reconstructed curves in the , , and directions in the three-dimensional experiment fit the measured curves well, showing the validity of the multiple linear regression model in the extraction of motion information from the EEG. However, there are high-frequency fluctuations in the reconstructed curves. One reason may be that the cut-off frequency of the low-pass filter during preprocessing is too high. As can be seen from Figure 10, the reconstructed curve with the cut-off frequency of 4 Hz has more frequent fluctuations than the one with the cut-off frequency of 2 Hz. It indicates that the cut-off frequency has an impact on the reconstructed curve. Reducing the cut-off frequency of the low-pass filter during preprocessing, or smoothing the reconstructed curve with a low-pass Butterworth filter, can effectively reduce these high-frequency fluctuations. Compared with the results in the two-dimensional experiment, the decoding accuracy in the three-dimensional experiment is low. The increased movement complexity may increase the difficulty of decoding.
In Figure 6(a), the reconstructed curves from the five sets of test data using the particle filter model fit the measured curves well in the trend. However, the decoding accuracies for Test Data 2 and Test Data 4 were relatively low, and the corresponding Pearson correlation coefficients are small. This may have been caused by inattention, eye blinks, or the movements of other parts of the body that introduced noise in the EEG data acquisition process for Test Data 2 and Test Data 4. The decoding accuracy for the three-dimensional experiment is smaller than that of the two-dimensional experiment, and Table 2 shows that the Pearson correlation coefficients for the data of lengths 8 s and 30 s in the three-dimensional experiment are smaller than that of the 15 s data. The same phenomenon exists when we analyze a large amount of data offline. Thus, the particle filter model may have an optimal data length. When the data length is shorter than the optimal data length, the number of iterations is small and leads to a low decoding accuracy. However, when the data length is longer than the optimal data length, the difference between the training data and test data becomes bigger, which also leads to a low decoding accuracy. Compared with the multiple linear regression model, the reconstructed curves from the particle filter model are smoother and have no high-frequency fluctuations. The Pearson correlation coefficients of the results—except Test Data 2 and Test Data 4 in the two-dimensional experiment—are above 0.7, showing a strong positive correlation. The reconstructed curves in the three-dimensional experiments, except Test Data 3, fit the measured curves well in the trend, showing the validity of the particle filter model.
Compared with the multiple linear regression model, the particle filter model uses less training data. Figure 11 shows the decoding results using the multiple linear regression model when the number of the training data sets increases from 1 to 5. Table 5 reports the corresponding Pearson correlation coefficients. Figure 12 shows how the Pearson correlation coefficient changes with the number of training data sets in the and directions, respectively.
(a)
(b)
As can be seen from Figure 12, the decoding accuracy of the multiple linear regression model in the and directions increases as the number of training data sets increases. When the number of training data sets is small, the decoding accuracy increases rapidly with the increase in training data sets, and when the number of training data sets is large, the decoding accuracy increases slowly with the increase in training data sets. This is because when the number of training data sets is small, the influence of random interference factors, such as noise, in the training data is big. The influence of the random interference factors decreases with the increase of training data sets, as they can cancel each other out. As a result, a large training data set leads to a high decoding accuracy. When training data sets increase to a certain number, the impact of increasing the training data sets on the improvement of decoding accuracy becomes weaker. Thus, the decoding accuracy tends to stabilize. The decoding results in two dimensions in Table 2 show that the particle filter model can achieve high decoding accuracy using only one training data set. So when the number of experimental data is small, it is easier for the particle filter model to achieve high decoding accuracy.
The high-frequency EEG components will cause high-frequency fluctuations in the reconstructed curves in the multiple linear regression model and lower its decoding accuracy. However, the particle filter model can process the high-frequency components of the EEG directly and use the information in a wide range of frequencies. Both the training data and test data are not low-pass filtered and are then used to decode the EEG using the multiple linear regression model and particle filter model. The decoding results are shown in Figure 13.
(a) Multiple linear regression decoding
(b) Particle filter decoding
Figure 13 shows the two different methods’ processing capacities for high-frequency components with the same EEG signal as an input. In Figure 13(a), for multiple linear regression, the Pearson correlation coefficients for the and directions are 0.7228 and 0.6927, respectively, lower than the decoding accuracy of the results after low-pass filtering in Table 1, and have a bad denoising ability. Only when the EEG signal frequency is consistent with the hand-moving frequency can a high decoding accuracy be achieved. In experiments, the hand-moving frequency is below 2 Hz; hence, the EEG signals need to be processed by a low-pass filter with a cut-off frequency of 2 Hz. In Figure 13(b), for the particle filter decoding results, the Pearson correlation coefficients for the and directions are 0.9044 and 0.8421, respectively, little different from the results after low-pass filtering in Table 2. Furthermore, it has a good effect on the processing of the high-frequency components. This is because the particle filter can process the high-frequency components of EEG in its algorithm, so the low-pass filter is not necessary. Moreover, the high-frequency components of the EEG may contain motion information; hence, the particle filter model can make use of more extensive frequency information, which may improve the decoding accuracy.
6. Conclusion
This paper used the multiple linear regression model and the particle filter model to decode the hand motion information in two and three dimensions from EEG signals and analyzed the decoding results and the factors that influenced it to compare the decoding features of the two methods and their usability. The experimental results showed that the multiple linear regression model needed multiple sets of training data to train and only worked well for low-frequency decoding. Conversely, for the particle filter model, only one set of training data was necessary, and the model could process EEGs containing high-frequency components, which means a more extensive utilization of frequency information. Nevertheless, the length of the EEG data affected the decoding accuracy. With this in mind, the decoding model should be carefully chosen in accordance with the model features and its application scenario.
Our future work is to research the influence of other factors on the decoding, optimize the particle filter method to improve its decoding accuracy, and try to perform online decoding and manipulator control.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors thank the anonymous referees for their helpful comments. The support of the Overseas Returnees Foundation of the Ministry of Education of China and the Special Program of the Postdoctoral Science Foundation of China (Approval no. XJ2013045) is gratefully acknowledged.