Abstract
This study presents a new steady-state visual evoked potential (SSVEP) paradigm for brain computer interface (BCI) systems. The new paradigm is High-Frequency Combination Coding-Based SSVEP (HFCC-SSVEP). The goal of this study is to increase the number of targets using fewer stimulation frequencies, with diminishing subject’s fatigue and reducing the risk of photosensitive epileptic seizures. This paper investigated the HFCC-SSVEP high-frequency response (beyond 25 Hz) for 3 frequencies (25 Hz, 33.33 Hz, and 40 Hz). HFCC-SSVEP produces with high stimulation frequencies through Time Series Combination Code. Furthermore, The Improved Hilbert-Huang Transform (IHHT) is adopted to extract time-frequency feature of the proposed SSVEP response. Lastly, the differentiation combination (DC) method is proposed to select the combination coding sequence in order to increase the recognition rate; as a result, IHHT algorithm and DC method for the proposed SSVEP paradigm in this study increase recognition efficiency so as to improve ITR and increase the stability of the BCI system. Furthermore, SSVEPs evoked by high-frequency stimuli (beyond 25 Hz) minimally diminish subject’s fatigue and prevent safety hazards linked to photo-induced epileptic seizures. This study tests five subjects in order to verify the feasibility of the proposed method.
1. Introduction
A brain computer interface (BCI) is a direct communication pathway between a human or animal brain and an external device. Nowadays, noninvasive scalp electroencephalogram (EEG) measurements have become a popular solution in BCI research. The most commonly used signals in EEG-based BCI systems are event-related synchronization of mu and beta bands, event-related potentials, and steady-state visual evoked potential (SSVEP) [1–3].
The SSVEP is usually elicited by flickering stimulation frequency higher than 6 Hz, while the frequency for inducing FVEP (Flash Visual Evoked Potential) should be lower than 2 Hz [4]. The most common measure to assess the performance of a BCI system is Shannon’s information transfer rate (ITR) [5]. Steady-state visual evoked potential (SSVEP) has been regarded as an efficient approach to design BCI with high information transfer rate (ITR). SSVEP-based systems lead to transfer rates of 100 bits/min and beyond [2, 6], compared to about 10–25 bits/min of other BCI systems [5]. Furthermore, SSVEP-based BCI systems have the advantages of short responding time, minimum training, and high ITR [4, 7, 8].
The visual stimulator plays an important role in an SSVEP BCI. Several visual stimulators have been used for evoking SSVEP, such as a cathode ray tube (CRT) monitor [9], liquid crystal display (LCD) monitor, and light-emitting diode (LED) array [10]. Considering stimulation parameters such as size, color, and position, LED arrays are not flexible and increase the overall system cost; presenting flickers on a computer monitor is more flexible than using stand-alone lights/LEDs [9].
Current SSVEP-based BCI system utilizes single frequency to encode each target. Hence, a large number of targets require a large number of frequencies. Increasing the number of targets then decreases the frequency resolution which in turn makes classification more difficult. This is especially problematic on computer screens. Recently, some researchers began to study how to increase the number of targets with fewer frequencies, such as stimuli flickered at the same frequency and differed only in relative phase [11], right-and-left field stimulation with two frequencies [12], and using dual frequency stimulation [4]. However, SSVEPs are usually evoked by low-mid frequencies in the 7–30 Hz, but the low-mid frequency band has some disadvantages [8]. First, the low frequency band covers the alpha band (8–13 Hz) which can cause a considerable amount of false positives. Second, subjective evaluations showed that frequencies between 5 and 25 Hz are more annoying than higher ones; visual fatigue would easily occur. Third, flash and pattern reversal stimuli can provoke epileptic seizures especially in the 15–25 Hz range [13]. All of these disadvantages can be avoided by using the high frequency band [14–16].
This study proposes High-Frequency Combination Coding-Based flickers for evoking SSVEP (HFCC-SSVEP). The HFCC-SSVEP is essentially based on frequency modulation. Each flicker comprises different high flickering frequencies through High-Frequency Combination Code. The High-Frequency Combination Coding-Based SSVEP (beyond 25 Hz) is induced by CRT (Cathode Ray Tube) and LED. This system uses only one Oz-A1 EEG channel for SSVEP recording. The proposed new paradigm produces with high stimulation frequencies through Time Series Combination Code. Furthermore, conventional spectral methods are not always suitable for detecting high-frequency SSVEPs. In this paper, an improved HHT- (Hilbert-Huang Transform-) based high-frequency-modulated SSVEP feature extraction method is proposed to extract time-frequency feature of the proposed SSVEP response. Furthermore, SSVEPs evoked by high-frequency stimuli (beyond 25 Hz) minimally diminish subject’s fatigue and prevent safety hazards linked to photo-induced epileptic seizures.
2. Subject and Experimental Condition
Three healthy volunteers (three males randomly selected from the students in the research institution), aged from 21 to 25 years old, participated in this study from Xi’an Jiaotong University. They were seated in a comfortable armchair in a dimly illuminated EEG signals testing lab, which is quiet without any distractions. All participants were 50 cm away from the stimulation unit (CRT display OR LED). EEG signals were measured from three EEG electrodes (g.USBamp, g.tec Guger Technologies, Austria) placed at Oz-A1 (Oz-unilateral earlobe) and Fpz (ground) in compliance with the international EEG 10–20 system. The unilateral (left or right) earlobe was used as the recording reference, and all electrode impedances were kept below 5 kOhm. The experimental data sampling frequency is 1200 Hz.
3. Experiment Paradigm
Figure 1 shows the block diagram of a HFCC-SSVEP for BCI. HFCC-SSVEPs were induced by 27 stimulus series; these stimuli were displayed by cycle flickers at the center of CRT. The 27 stimulus series were generated from 3 fundamental frequency elements through Time Series Combination Code. The used fundamental frequency elements were only three high frequencies, that is, 25, 33.33, and 40 Hz; the three frequencies were labeled as 1, 2, and 3. The selection of fundamental frequency elements depended on the screen refresh rate and sampling rate. Each fundamental frequency element was generated from 20 pulses. The data lengths of 27 stimulus series were variably between 1.5 s and 2.4 s. The subjects gazed each stimulus sequence and the stimulus sequence was displayed by cycle flickers at the center of CRT.
4. Data Processing and Result
Figure 2 shows EEG signals of SSVEP response presented through LED with 25 Hz, 30 Hz, 40 Hz, and 75 Hz. The length of EEG data is 8 s. The frequency spectrum of EEG signals was determined using the fast Fourier transform (FFT) technique in MATLAB software. Figure 3 shows the FFT spectrum of SSVEP EEG signals of LED. The spectrum frequency detection is usually the appropriate for frequency analysis of SSVEPs in BCI applications. Figure 4 shows FFT’s amplitude of EEG signals for LED. Figure 3 shows that SSVEP response of 25 Hz, 30 Hz, and 40 Hz is sensitive, but the 75 Hz spectral line cannot be distinguished in the FFT spectrum from Figure 3. Figure 4 shows the fitted curve of FFT spectrum energy.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
In this study, each Oz-A1 EEG signal was segmented into ten segments. Figure 10 shows that SCCBH-SSVEP EEG signals 6 of 27 selections. The length of EEG data is unequal between 1.5 s and 2.4 s. The frequency spectrum of averaged EEG signals was determined using the fast Fourier transform (FFT) technique in MATLAB software. The spectrum frequency detection is usually the appropriate for frequency analysis of SSVEPs in BCI applications; but the FFT spectrum cannot contain temporal information. The 27 selections represented by three high frequencies elements cannot be distinguished from each other in the FFT spectrum.
Spectrum analysis is usually used to extract the frequency information in traditional evoked SSVEPs responses. The underlying idea is always the same: a blinking or moving visual stimulus at a constant frequency (the stimulus frequency) elicits a response in the brain at the same frequency and its even harmonics (SSVEP frequency is equal to stimulus frequency plus its even harmonics). Note that this behavior denotes a nonlinearity of the visual system [5].
The evoked SSVEP response of High-Frequency Combination Coding-Based stimulus not only contains frequency information but also contains temporal information. The frequency information consists of three frequencies through Time Series Combination Code; the length of evoked SSVEP response data varies between 1.5 s and 2.4 s (see Figure 2); as a result, the evoked SSVEP responses are nonstationary and nonlinear. The traditional spectrum detection method is not suited for analyzing the time series combination coding-based high-frequency evoked SSVEP response. More sophisticated nonstationary and nonlinear signal processing techniques have recently been used to analyze the evoked SSVEP response [17–19].
Hilbert-Huang transforms [20], consisting of empirical mode decomposition (EMD) and Hilbert spectral analysis, is a newly developed adaptive data analysis method. The HHT is designed specifically for analyzing nonlinear and nonstationary data. The key part of HHT is EMD with which any complicated data set can be decomposed into a finite and often small number of intrinsic mode functions (IMFs). The instantaneous frequency defined using the Hilbert transform denotes the physical meaning of local phase change better for IMFs than for any other non-IMF time series. As the decomposition is based on the local characteristics of the data, it has been proved quite versatile in a broad range of applications for extracting signals from data generated in noisy nonlinear and nonstationary processes [21]. It has been widely used to analyze EEG signals [22–26]. In order to extract the time-frequency characteristics of high-frequency time series combination coding-based SSVEPs, we proposed IHHT-based high-frequency time-modulated SSVEP method.
Figure 9 shows the steps of HHT-based high-frequency time-modulated SSVEP feature extraction, a local spectrum extreme target identification algorithm and differentiation combination method. The extraction method consists of synchronous averaging, band pass filtering, EMD, selection of IMF, instantaneous frequency, and Hilbert spectrum. In order to ensure that the time-frequency characteristics of high-frequency time series combination coding-based SSVEPs are efficiently extracted, we must choose and optimize the key algorithm. The specific method is as follows. The end effect and stopping criterion are the core problems of empirical mode decomposition (EMD); two methods are selected and optimized in order to overcome the shortage of end effect and stopping criterion of empirical mode decomposition (EMD) in the processing of variable frequency EEG data.
(i) Optimization Selection of the EMD Endpoint Prediction. Upper and lower envelope averaging is one of the cores of the EMD algorithm; the upper and lower envelopes are attained through the extreme value point of the spline curve fitting. The EMD end effect is how to find the proper fitting curve between the last extreme value point and endpoint in the curve fitting. The boundary prediction method and traditional method are compared in order to optimize the problem for the end effect in the high-frequency time-modulated SSVEP feature extraction.
Figure 5 shows the boundary prediction method. The boundary prediction method is to use the 1 order approximate point as the endpoint. Through the theoretical and real data analysis [27], it is not only to conform to the conditions of the cubic spline curve fitting but also to ensure that the fitting curve fluctuation is minimal. In Figure 6, E-Natural curve is the upper envelope curve fitting based on cubic spline curve through the boundary prediction method, Natural curve is the upper envelope curve fitting based on cubic spline curve through the traditional method. Figure 6 shows that the fluctuation of the fitting curve based on boundary prediction in the end is minimal. So the boundary predictions are used to overcome the shortage of end effect of empirical mode decomposition (EMD) in the processing of variable frequency EEG data.
(a)
(b)
(c)
(d)
(ii) The Parameter Optimization of EMD Stopping Criterion. The EMD algorithm is essentially a process of screening; the screening mathematical formula is as follows:In formula, is the target signal, is the IMF that is decomposed through the EMD algorithm, is the number of IMF, , is the average of the upper and lower envelope curve in the process of screening, and , is the number of screening.
The core problem of EMD is how to choose the end conditions in every process of screening, which is the problem of the EMD stopping criterion.
The EMD algorithm is essentially binary half-band filter, which is actually binary wavelet. In order to ensure the screening results of IMF in the amplitude and that frequency has enough physical meaning, which essentially is to ensure that scale ratio of the adjacent two IMF values is close to 2, the number of screening () must be limited in the process of screening.
The fixed sifting (iterating) 10 times are used to overcome the shortage of stopping criterion of empirical mode decomposition (EMD) in the processing of variable frequency EEG data, That is , which can ensure that scale ratio of the adjacent two IMF values is close to 2. The advantage of this selection is as follows [28]:(a)Ensure that the EMD algorithm is essentially binary half-band filter bank.(b)Ensure that the result has physical meaning.(c)The decomposition results number is limited <= .
(iii) The Optimization of Calculation Method for Instantaneous Frequency. In order to ensure that the instantaneous frequency based on Hilbert Transform (HT) has physical meaning, the conditions are as follows:(a)The result must be IMF.(b)The signal amplitude change is not too large (due to the Bedrosian theoretical limitation).(c)The phase of signal cannot be too complicated (due to the Nuttall theoretical limitation).
The GZC (generalized zero-crossing) algorithm is proposed to calculate the instantaneous frequency of variable frequency EEG data, as shown in Figure 7 and formula (2).
The computational formula of instantaneous frequency is as follows:In the formula, (2) is the time frame between zero and peak time.
The GZC algorithm is based on the average concept in nature, so the GZC algorithm has the advantage of high stability and high precision of calculation. Therefore the GZC algorithm is very suitable for the fact that the frequency fluctuation of signal is not big. The variable frequency characteristics of high-frequency time series combination coding-based SSVEPs in this paper is segment-wise stationary; as a result, the generalized zero-crossing (GZC) is used to compute the instantaneous frequency of the proposed SSVEP respondent signals.
(iv) Local Spectrum Extreme Target Identification Algorithm. The relevant IMFs are evenly divided into 3 sections, then to respectively calculate their spectrum extreme . Then according to the following rules, is encoded.(a), is 1.(b), is 2.(c), is 3.
Finally, compare the coding result to the element in the encoded library; if the coding result is in the library, it is marked as nonzero; otherwise it is marked as zero. The of nonzero number is the correct identification number; as a result, the recognition rate is quantitatively calculated through the local spectrum extreme target identification algorithm. The recognition accuracy of computational formula is as follows:
(v) Differentiation Combination Method. In the experiment, the used fundamental frequency elements were only three high frequencies, that is, 25, 33.33, and 40 Hz, which are labeled as 1, 2, and 3. The data lengths of 27 stimulus series were variably between 1.5 s and 2.4 s. The 27 stimulus series are averaged by superposition of 1~10 times. The recognition accuracy of the different superposition time’s average results is calculated through the local spectrum extreme target identification algorithm. Figure 8 shows the statistical result between the different superposition time’s average results, , and the recognition accuracy, .
(a) Data 123
(b) Data 132
(c) Data 213
(d) Data 231
(e) Data 312
(f) Data 321
From statistics results in Figure 8, we can see that the recognition accuracy is only 77.78% through 10 times superposition average. This is obviously not what we expected. The statistical result between the different superposition time’s average results and the recognition accuracy in Figure 8 shows that the recognition accuracy is different for all of the combinations of three stimulation units. Different combinations make a difference in recognition rate. Through the comparison and analysis, the recognition accuracy is larger through differentiation combination method of three stimulation units. The reason is that differentiation combination method can break the adaptability of the vision system [29], compared with single frequency, so the different frequency encoding (differentiation combination method) can improve the recognition rate. As a result, on the basis of the fundamental frequency elements, that is, 25, 33.33, and 40 Hz, which are labeled as 1, 2, and 3, the differentiation combination encoding of 27 combination encoding mode is 123, 132, 213, 231, 312, and 321. Figure 10 shows SCCBH-SSVEP EEG single response signals of differentiation combination method. Figure 12 shows that the Hilbert spectra of the selections signals are presented by using HHT-based high-frequency time-modulated SSVEP method.
Each Oz-A1 EEG signal is firstly segmented into ten segments for synchronous averaging, which are to decrease the noise of EEG data. Next, the data in the frequency range of 25–45 Hz are chosen for EMD analysis, owing to the frequency range of three selected fundamental frequency elements, 25 Hz, 33.33 Hz, and 40 Hz. According to EMD results, the relevant IMF is selected empirically for analysis. Then, the instantaneous frequency of the selected relevant IMF is calculated by using generalized zero-crossing algorithm [30]. At last, according to the calculated instantaneous frequency, the Hilbert spectrum is presented. A local spectrum extreme target identification algorithm is proposed to calculate recognition rate; the experimental result shows that the recognition rate is not high for all combination coding sequences. So the differentiation combination method is proposed to select the combination coding sequence in order to increase the recognition rate; as a result, 6 combination coding sequences are selected; they are 123, 132, 213, 231, 312, and 321 (fundamental frequency: 25, 33.33, and 40 Hz, number resp.: 1, 2, and 3).
Figure 11 shows that the spectra of selected EEG signals, the frequency spectra of the selected EEG signals, were determined using the fast Fourier transform (FFT) technique in MATLAB software. In the spectrum in Figure 11, the lime lines represent 25 Hz, the red lines represent 33.33 Hz, and the springs green lines represent 40 Hz. The spectrum frequency detection is usually the appropriate for frequency analysis of SSVEPs in BCI applications; but the FFT spectrum cannot contain temporal information. The 6 selections represented by three high frequencies elements cannot be distinguished from each other in the FFT spectrum. Figure 12 shows that the Hilbert spectra of the selected signals are presented by using HHT-based high-frequency time-modulated SSVEP method. Obviously, the Hilbert spectrum presents the time-frequency characteristics of flashing sequences based on high-frequency time series combination code using three high frequencies. Compared to FFT spectrum, it not only provides the frequency information but also presents frequency change with time, which extracts the features of Time Series Combination Code of three high frequencies. According to the combination code time-frequency characteristics, the Hilbert spectra (see Figure 12) obviously identify the selections.
(a) Data 123
(b) Data 132
(c) Data 213
(d) Data 231
(e) Data 312
(f) Data 321
(a) Hilbert spectrum 123
(b) Hilbert spectrum 132
(c) Hilbert spectrum 213
(d) Hilbert spectrum 231
(e) Hilbert spectrum 312
(f) Hilbert spectrum 321
Finally, six stimulus targets are presented with three high frequencies through HFCC-SSVEP; in contrast, three stimulus targets are presented with three low frequencies through traditional SSVEP; the above two kinds of different contrast experiments in Figure 13 are applied to intelligent wheelchair navigation control in order to verify the technical advantage of the proposed method and ensure the HFCC-SSVEP-based intelligent wheelchair navigation system efficiency and undamaging. Figure 13(a) shows the experimental paradigm of HFCC-SSVEP with 6 targets: 123, 132, 213, 231, 312, and 321 (fundamental frequency: 25, 33.33, and 40 Hz, number resp.: 1, 2, and 3); they are presented on the screen. Figure 13(b) shows the experimental paradigm of the traditional SSVEP; the frequencies of 3 targets are, respectively, 12, 12.5, and 15 Hz. Table 1 is the comparison of HFCC-SSVEP and traditional SSVEP; it is average experimental results of five subjects. In Table 1, is stimulation time, is the recognition rate, df is the degree of fatigue (fatigue time), and ITR is the information transmission rate of BCI system. Table 1 shows that HFCC-SSVEP has the advantage of shorter time, higher accuracy, not easy fatigue, and higher ITR, compared with the traditional SSVEP (Figure 14).
(a) HFCC-SSVEP
(b) Traditional SSVEP
5. Discussion and Conclusion
We studied SSVEP high frequency (beyond 25 Hz) response of SSVEP whose paradigm is on the LED. Figure 3 shows that the 75 Hz spectral line cannot be distinguished in the FFT spectrum. The SNR (signal to noise ratio) of high frequency (beyond 40 Hz) response is very low, which has been unable to be distinguished through the traditional analysis method. So we must study the weak signal feature extraction method for SSVEP high frequency (beyond 40) response. This part of the research will be conducted in the future research work. As a result, 25 Hz, 33.33 Hz, and 40 Hz in the high frequency area are selected as the fundamental frequency for HFCC-SSVEP paradigm.
The proposed HFCC-SSVEP paradigm for a BCI system not only can increase the number of targets through Time Series Combination Code of fewer frequencies for stimulation but also can shorten the recognition time. Furthermore, it can diminish user’s fatigue and risk of photosensitive epileptic seizures. This study specifies twenty-seven (33) selections using only 3 frequencies with about 2 s EEG data. However, the targets are not distinguished from each other using traditional spectrum method. Consequently, in this case, IHT-based high-frequency time-modulated SSVEP feature extraction method, which is a nonlinear and nonstationary signal processing method, is proposed to extract the time-frequency characteristics of targets. As a result, the HFCC-SSVEP targets can be obviously distinguished from each other in the Hilbert spectrum. The proposed method helps increase the recognition efficiency of SSVEP for BCI systems. Results show that this High-Frequency Combination Code paradigm is suitable for the SSVEP BCI system.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by a grant from the National Natural Science Foundation of PR China (Approval no. 51175412).