Effects of Visual Attention on Tactile P300 BCI

Chen, Zongmei; Jin, Jing; Daly, Ian; Zuo, Cili; Wang, Xingyu; Cichocki, Andrzej

doi:https://doi.org/10.1155/2020/6549189

Computational Intelligence and Neuroscience

On this page

Abstract Introduction Materials and Methods Results Discussion Conclusions Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 6549189 | https://doi.org/10.1155/2020/6549189

Effects of Visual Attention on Tactile P300 BCI

Zongmei Chen,¹Jing Jin,¹Ian Daly,²Cili Zuo,¹Xingyu Wang,¹and Andrzej Cichocki^3,4,5,6

Academic Editor: Ivan Volosyak

Received22 Oct 2019

Revised17 Jan 2020

Accepted01 Feb 2020

Published19 Feb 2020

Abstract

Objective. Tactile P300 brain-computer interfaces (BCIs) can be manipulated by users who only need to focus their attention on a single-target stimulus within a stream of tactile stimuli. To date, a multitude of tactile P300 BCIs have been proposed. In this study, our main purpose is to explore and investigate the effects of visual attention on a tactile P300 BCI. Approach. We designed a conventional tactile P300 BCI where vibration stimuli were provided by five stimulators and two of them were fixed on target locations on the participant’s left and right wrists. Two conditions (one condition with visual attention and the other condition without visual attention) were tested by eleven healthy participants. Main Results. Our results showed that, when participants visually attended to the location of target stimulus, significantly higher classification accuracies and information transfer rates were obtained (both for ). Furthermore, participants reported that visually attending to the stimulus made it easier to identify the target stimulus in random sequences of vibration stimuli. Significance. These findings suggest that visual attention has positive effects on both tactile P300 BCI performance and user-evaluation.

1. Introduction

A brain-computer interface (BCI) provides a new pathway between the brain and an external device to achieve direct control and communication [1]. The first BCI system was developed by Vidal in the 1970s [2]. In the decades since, BCIs based on electroencephalography (EEG) recordings have been increasingly frequently explored as they are safe and relatively cheaper than BCIs based on other neuroimaging technologies. The EEG is recorded from sets of electrodes placed on the scalp and comprises a time series of electropotentials generated in the cerebral cortex [3]. The selection of electrode positions and their quantity generally depends on the aims of the study, the ultimate aim of which is, typically, to achieve optimal system performance. The acquired brain signals (e.g., the EEG data) from the selected channels are usually processed through the following steps: preprocessing, feature extraction, feature selection, and classification. These processes seek to identify the intention of the user in order to generate corresponding commands. Finally, these commands can be used for practical applications including, but not limited to, wheelchair navigation [4, 5], character speller [6, 7], and robotic arm manipulation [8, 9].

The brain activities that are most frequently used to control BCI systems include event-related potentials (ERP) [10], steady-state evoked potentials [11], and event-related desynchronization (ERD) [12] and event-related synchronization (ERS) [13]. In an ERP-based BCI, the induction of the ERP is achieved by presenting a predictable sequence of stimuli with one or more rarely, randomly occurring (unpredictable) stimuli interleaved amongst the predictable stimuli. The users are instructed to effectively discriminate the stimuli by means of counting the number of rare stimuli occurrences (presenting at a low frequency and referred to as the “target stimuli”), while ignoring other nontarget stimuli. The P300 is one of the positive components of the ERP and occurs around 300 ms after a target stimulus presentation [14].

Early BCI systems were primarily based on stimuli that were presented visually. For example, the first visual P300 BCI was reported by Farwell and Donchin in 1988 [15] and used a 6 × 6 letters matrix, which was displayed to participants as stimuli on a computer screen. Following on from this work, some researchers took measures to pursue better system performance, an effort which led to, amongst other work, an influential study in which traditional visual letters were replaced with faces [16].

However, the standard visual P300 BCIs depend on gaze control and are not suitable for visually impaired individuals. Consequently, the auditory and tactile P300 BCIs were gradually explored as alternative solutions. Hill et al. [17] first proposed an auditory P300 BCI in which the auditory stimuli were composed of deviant and standard tones. The feasibility of primary tactile P300 BCI was demonstrated by Brouwer and Van Erp [14]. In their study, motors providing vibration stimuli were situated at different locations around the participant’s waist. The effects of the number of motors and the stimulus onset asynchrony (SOA) on classification performance were also investigated.

Our study focuses on the tactile P300 BCI. Researchers have attempted to apply tactile stimulation to various parts of body, such as chest [18], fingers [19], back [20], and head [21]. In addition, in order to improve the performance of tactile P300 BCIs, multisensory BCI systems have also been proposed. For example, Brouwer et al. combined tactile stimuli with visual stimuli to construct a visual-tactile bimodal P300 BCI [22], and Yin et al. proposed an auditory-tactile bimodal P300 BCI [23]. Both of them found that BCI with bimodal stimuli obtained higher classification performance compared to that with unimodal stimuli.

In this study, we investigate whether visual attention by the BCI user has any effects on the tactile P300 BCI performance and on the usability of the BCI (as assessed by user-evaluation). A conventional tactile P300 BCI was designed in which vibration stimuli were delivered respectively to participant’s left wrist, right wrist, abdomen, left ankle, and right ankle. The participant was asked to distinguish the stimulus on the left wrist or right wrist from other stimuli. Two conditions were tested by participants: one condition used visual attention (called the VA condition) and the other did not use visual attention (called the NVA condition). Notably, in the NVA condition, the participants were required to silently count the number of target vibration appearances only by spatial attention. While in the VA condition, in addition to the counting tasks, the participants also had to pay visual attention to the target vibration location all the time from the short target vibration cue until the target cue moved to another location.

2. Materials and Methods

2.1. Participants

A total of eleven healthy adults from East China University of Science and Technology in Shanghai, China (including 4 females and 7 males, aged from 22 to 26) participated in this study; they were designated P1, P2, …, P11. All participants had normal or corrected-to-normal vision and intact tactile sensation (self-reported). Importantly, none of them were trained before. In order to achieve the aim of the study, the experimental procedure and the required tasks were explained in detail before any of the individuals participated. Moreover, each participant signed a written consent form prior to the study, which was approved by the local ethics committee.

2.2. Stimuli and Procedure

The vibrotactile stimuli were provided by g.VIBROstims, the main unit of which was DC motors that produced the vibrations. As shown in Figure 1, the motor was hidden in a cylindric casing, which was placed on the participant’s body by adhesive plaster. The g.VIBROstims were driven by a g.STIMbox (g.tec medical engineering GmbH, Schiedlberg, Austria), which was connected to the computer via USB and was controlled by a Simulink block (Matlab 2015b). Based on a previous research, the stimulus duration was set to 200 ms and the interstimulus interval (ISI) was set to 400 ms [22].

For both conditions, each participant sat in a chair in front of a monitor and the vibration stimulators were placed on the left wrist, right wrist, abdomen, left ankle, and right ankle, which ensures sufficient spatial distance to achieve distinguishability between individual stimuli. Figure 2 shows the placements of vibration stimulators on each participant’s body. Compared to the abdomen, left ankle, and right ankle, the left and right wrists are easier for visual attention. In addition, if participants pay visual attention to the abdomen, left ankle, or right ankle, it will bring larger movements of the head or eyeball. So only the left and right wrists were selected as the target stimulus positions where the stimulators were marked in red. The rest of the stimulators were never selected as target stimulus positions and only were used as standard stimulus positions for reducing the probability of the target stimulus presentation, in which the stimulators were marked in black. Each participant’s task was to silently count the number of times the target vibration was presented and avoid unnecessary body movements. In particular, for the VA condition, besides the counting tasks, the participants were also asked to give visual attention to the target stimulus positions. The positional conversion of visual attention and the stimulus location that the participant needed to attend to both depended on a particular target vibration cue. To prevent head or eye movement caused by positional shift of visual attention, before carrying out each condition, the participants were asked to place their left and right hands on a desk and their left and right wrists were simultaneously shown in the field of view. Furthermore, the participants were told to immediately switch visual attention in accordance with the target vibration cue. Conversely, for all participants, there was no visual attention to the target stimulus positions during presentation of the NVA condition.

Each condition required participants to complete a corresponding experiment, and they should be done on the same day. The order of two experiments was random. In our study, six participants chose to do the VA condition experiment first. In order for the participant to maintain sufficient energy to complete each experiment, there would be an interval between the two experiments, the length of which depended on the individual. Each experiment contained an offline phase and an online phase (see Figure 3). In the offline phase, three runs were included and each run consisted of five blocks (i.e., five target selections). Prior to each block, the target vibration cue was presented for 1.5 s. There were 10 trials per block and all trials within a block had the same target. In each trial, five vibrations occurred randomly. To mitigate for fatigue, each participant could take a short break between offline runs. Furthermore, a long break was used to allow participants to prepare for the following online phase. The length of time of both breaks depended on the individuals. In the online phase, only one run was involved, but there were 20 blocks (i.e., 20 target selections). The number of trials per block (n) was variable, which was automatically determined based on an adaptive strategy [24], and each trial also was composed of five vibrations.

2.3. EEG Acquisition

For each participant, EEG data was recorded at a sampling rate of 256 Hz with a g.USBamp (high-pass and low-pass filters set at 0.1 Hz and 30 Hz; a notch filter set at 50 Hz) and a g.EEGcap (Guger Technologies, Graz, Austria). EEG electrodes were positioned according to the international 10-20 system. In our study, fourteen wet active Ag-AgCl electrodes (Fz, FC1, FC2, C3, Cz, C4, CP3, CP1, CP2, CP4, P3, Pz, P4, and Oz) were selected. In addition, FPz was selected as the ground electrode and the right mastoid (A) was selected as the reference electrode. As shown in Figure 4, the black circles mark the 14 EEG recording electrodes, while the gray circles mark the ground electrode (FPz) and reference electrode (A). The impedances of these electrodes were below 10 kΩ and EEG waveforms from all channels remained relatively stable at the start of each experiment.

2.4. Feature Extraction and Classification

In each experiment, an 800 ms data segment was extracted after each vibration stimulus presentation. This resulted in a total of 750 data segments, including 150 targets and 600 nontargets, extracted from the offline phase of the experiment. Each EEG data segment was filtered into the frequency range 0.1–30 Hz by a 3^rd order Butterworth band-pass filter and then downsampled from 256 Hz to 36.6 Hz by selecting every seventh sample. Therefore, a spatiotemporal feature vector was formed with a dimensionality of 14 × 29 (14 channels and 29 sample points). In this case, 750 such feature vectors were collected as calibration data for each condition. Moreover, winsorizing was adopted to remove interference signals resulting from muscle activity, eye blinks, or eye movement. Firstly, the 10th and 90th percentiles for each sample were computed; secondly, the values of each sample lying less than the 10th percentile or more than the 90th percentile were replaced with the 10th or the 90th percentile, respectively [25, 26].

Bayesian linear discriminant analysis (BLDA) was chosen to build the classifier model for online validation. This approach has been widely employed in an increasing number of P300 BCI systems due to its superior classification performance [27, 28]. The classification rule can be defined aswhere denotes the discriminant vector, two hyper parameters α and are the inverse variance of prior distribution and noise, denotes a matrix containing feature vectors, and denotes the regression targets, which is regulated for class 1 in and for class 2 in (where is the number of features from class 1, is the number of features from class 2, and is the total number of features from both classes). The variable denotes the output of the classifier, and denotes the new input feature vector.

For online classification and recognition in our study, five spatiotemporal feature vectors were obtained from five vibrations (i.e., five stimulus positions) during each single trial. These were then input into the classifier to calculate whether their probability distributions belong to the target class. Finally, the stimulus position with the maximal probability distribution was identified and reported as the classification result.

2.5. Performed Analysis

In this paper, in order to investigate whether visual attention had any effects on the tactile P300 BCI performance, we analyzed both the offline and online data recorded during presentation of the VA and NVA conditions. For the offline data recorded during presentation of the VA and NVA conditions, the ERP amplitudes and the r-squared values were used to show how ERPs differed between the two conditions. The definition of r-squared values is as follows:where denotes the features of each class and denotes the number of samples ( = 1, 2).

In addition, we explored the amplitude (i.e., the peak value) and latency (i.e., the peak time) of the N200, P300, and N400 ERPs at different electrode sites averaged across 11 participants for each condition. Apart from these, the mean amplitude of the P300 ERP at electrode Cz for each participant was also analyzed. For the purpose of comparing the offline performance differences between the two stimulation conditions, the offline classification accuracy and raw bit rate (i.e., information transfer rate) were both averaged across the 11 participants across 1–10 trials [29], and the offline classification accuracies, based on single trials, for the 11 participants were calculated. We also analyzed the contributions of the N200, P300, and N400 ERPs to the classification accuracy, as well as the single-target classification accuracy for the 11 participants with each stimulation condition. To further make a comparison of the online performance differences between the NVA and VA conditions, based on online data, the online classification accuracy, raw bit rate, and required average number of trials used to classify each position were calculated.

Before carrying out any statistical comparison between data obtained from these two conditions, we first tested the normality of the data (one-sample Kolmogorov–Smirnov tests). For the data that were observed to be normally distributed, we used paired-samples t-tests to estimate the significance of the differences, while for the data that was not observed to be normally distributed, a nonparametric test was needed. Therefore, we chose a Wilcoxon signed-rank test to make a comparison [26, 30]. The significance level was set to .

2.6. Subjective Feedback

The feedback from participants can provide further information that allows us to investigate the effects of visual attention on user-evaluation when using a conventional tactile P300 BCI. Consequently, we conducted a questionnaire survey after each participant completed the corresponding experiments for the two conditions. The questions were delivered in Chinese (the first language of all 11 participants). The English translations of the questions are as follows:(1)Which condition did you feel was more difficult to use? Please give scores to both conditions on a scale of one to five. The higher the score, the more difficult you feel the condition was to use.(2)Which condition made you feel more tired? Please give scores to both conditions on a scale of one to five. The higher the score, the more tired you felt as a result of using this condition.

3. Results

3.1. ERPs

Figure 5 shows the grand averaged ERPs when attending to the targets and the nontargets over all 11 participants, for each of the 14 electrode sites. Figure 6 shows the r-squared values of the ERPs from 0 to 1000 ms, averaged over all 11 participants. It can be observed that the VA and NVA conditions had similar ERP components (see Figure 5), but the feature difference between targets and nontargets in the VA condition was larger than that in the NVA condition (see Figure 6).

Table 1 shows the mean peak values and peak times of the N200, P300, and N400 ERPs at different electrode sites averaged over all 11 participants. For the N200 and P300 ERPs, the most negative peak and the most positive peak were, respectively, observed to occur between the 100–250 ms and 250–400 ms. The most negative peak of the N400 ERP was observed to occur from 400 ms to 650 ms after the stimulus [31]. This result shows that the N200 ERP, recorded from electrode Pz and evoked by the VA stimulation condition, had a higher absolute mean peak value and a shorter peak duration than the ERP evoked by the NVA condition. The same result was observed in the case of the N400 ERP. Similarly, the P300 ERP had a higher absolute mean peak value and a shorter peak duration when recorded from both electrodes Pz and Cz during the VA stimulation condition, compared to the NVA condition.

Figure 7 shows the mean amplitude of the P300 ERP, for each participant, recorded from electrode Cz. The mean amplitude was averaged from each ERP peak point ±25 ms [27, 32]. The result of paired-samples t-tests showed that the mean amplitude of the P300 ERP at electrode Cz during presentation of the VA condition was significantly larger than during presentation of the NVA condition (t = 2.736, )).

3.2. Offline Performance

Figure 8 shows the mean offline performance averaged over all 11 participants across 1–10 trials. The offline classification accuracy (see Figure 8(a)) and raw bit rate (see Figure 8(b)) were calculated from 15-fold cross-validation. The offline classification accuracy and raw bit rate of the VA condition were both significantly higher than those of the NVA condition. Figure 9 shows the single-trial classification accuracy of each participant using the offline data for each of the two stimulation conditions. The results of paired-samples t-tests showed that the VA condition achieved significantly higher single-trial classification accuracy than that achieved with the NVA condition (t = 4.641, ).

(a)

(b)

Figure 10 shows the contributions of the N200 (peaking between 150 ms and 300 ms), the P300 (300 ms and 450 ms), and the N400 (450 am and 700 ms) ERPs to offline classification accuracy for each participant. It can be seen that all the time windows were crucial in achieving the classification results. The results of paired-samples t-tests showed that the contributions of these three ERPs to offline classification accuracy in the VA condition were all significantly higher than those in the NVA condition (N200: t = 3.472, ; P300: t = 4.539, ; N400: t = 2.380, ). Figure 11 shows the offline single-target classification accuracy for each participant. Most participants achieved higher classification accuracy with the left wrist than that with the right wrist for each stimulation condition (see the left panel of Figure 11, 7 out of 11 participants for the VA condition; see the right panel of Figure 11, 8 out of 11 participants for the NVA condition). The results of paired-samples t-tests showed that the VA condition achieved significantly higher single-target classification accuracy than that achieved with the NVA condition (target at left wrist: t = 4.993, ; target at right wrist: t = 5.418, ).

(a)

(b)

(a)

(b)

3.3. Online Performance

Table 2 shows the online classification accuracy, average number of trials, and raw bit rate for each participant in the two stimulation conditions. The classification accuracy and raw bit rate of the VA condition were significantly higher than those of the NVA condition (t = 8.484, for classification accuracy; t = 7.667, for raw bit rate). Moreover, the average number of trials of the VA condition was significantly less than that of the NVA condition (t = −3.688, ).

3.4. Participant Evaluation

Table 3 shows the scores given by the 11 participants to the two questions for each condition. Compared to the NVA condition, the VA condition obtained lower scores in terms of both the degree of difficulty and the tiredness resulting from using the stimulation condition for all the participants. This demonstrated that all 11 participants felt the NVA condition to be more difficult and tiring than the VA condition. The result of a nonparametric Wilcoxon signed-rank test showed that there were significant differences between the two conditions in both the degree of difficulty () and the degree of tiredness ().

4. Discussion

In the current study, we designed a conventional tactile P300 BCI, in which five tactile stimulators were spatially distributed over a participant’s left wrist, right wrist, abdomen, left ankle, and right ankle. Only the left and right wrists were selected as target stimulus positions and the rest were used as standard stimulus positions. Junichi Hori et al. have reported that the frequency of each stimulus should be consistent to prevent the P300 ERP occurring in response to the nontarget stimuli with some participants [33]. Therefore, the standard stimuli in our study were placed on three different body positions as a solution to this problem. In order to explore whether visual attention had effects on this tactile P300 BCI, the VA and NVA conditions were setup and tested by 11 participants. In each trial of the two conditions, five stimulators vibrated randomly and the participant performed a counting task to count the target stimulus onsets. At this time, the targets and nontargets could be endogenously discriminated based on their location. In contrast to the NVA condition, the participants using the VA condition were also instructed to pay visual attention to the position of the target vibration. It is worth noting that there are only tactile stimuli without visual stimuli in the VA condition, so it is still categorized as a unimodal BCI. It is different from the visual-tactile bimodal BCI designed by Brouwer et al., in which both tactile and visual-tactile stimuli are provided and the visual stimulus reflects the same tap pattern as presented by the tactor [22]. The presentation of visual stimuli requires certain equipment provided externally, and there will be visual potentials in the subject’s EEG. However, these will not happen when paying attention to the position of the tactor on the body.

Researches have shown that spatial attention can be used to modulate ERP components [34, 35]. This corresponds to our findings that the N200, P300, and N400 ERPs were evoked during both the VA and NVA conditions (see Figure 5). Specifically, the VA condition yielded more discriminative features between targets and nontargets compared to the NVA condition (see Figure 6). The mean peak values of the N200 ERP at electrode Pz, the P300 ERP at electrodes Pz and Cz, and the N400 ERP at electrode Cz during the VA condition were higher than those observed during the NVA condition. However, the mean peak times of the N200 ERP at electrode Pz, the P300 ERP at electrodes Pz and Cz, and the N400 ERP at electrode Cz during the VA condition were lower than those observed during the NVA condition (see Table 1).

As for the mean amplitude of the P300 ERP at electrode Cz for each participant, a significant difference was observed between the two conditions (see Figure 7). Additionally, the mean offline classification accuracy and raw bit rate over the 11 participants, when different numbers of trials were used to construct the ERP (1–10 trials) during the VA condition, were higher than those observed during the NVA condition in the first trail. Subsequently, the classification accuracy of both conditions improved gradually as the number of trials increased. Finally, both conditions achieved a classification accuracy higher than 70% (see Figure 8(a), 96.36% was achieved for the VA condition; 70.30% for the NVA condition), and this is considered as the minimum accuracy percentage necessary for effective BCI control [36].

When single-trial classification was used, the offline classification accuracy of the VA condition was significantly higher than that of the NVA condition (see Figure 9). For each condition, the contributions of the N200, P300, and N400 ERPs to the offline classification accuracy for each participant were different (see Figure 10), but the contributions of all the time windows to offline classification accuracy in the VA condition were all significantly higher than those in the NVA condition. We found that the late ERPs contributed more to the classification accuracy than the early ERPs for most participants in the VA condition, while the NVA condition happened to be the opposite. In this study, the left and right wrists were used for delivering target vibration stimuli, which allowed for visual attention and discrimination between targets due to their spatial distribution. The resulting single-target classification accuracies showed that the mean classification accuracy of the left target was higher than that of the right target for both of the conditions (see Figure 11). However, the single-target classification accuracy showed that there was no significant difference between two target locations in both paradigms. This phenomenon can be explained by the description in the somatosensory homunculus that the left and right sides of the wrists have similar tactile sensitivity [37]. Significantly higher single-target classification accuracies were achieved with the VA condition than those achieved with the NVA condition.

The online results showed that the classification accuracy and raw bit rate of the VA condition were both significantly higher than those of the NVA condition (see Table 2), which proved that the VA stimulation condition was feasible and effective. In particular, in the VA condition, three participants obtained a peak online classification accuracy of 100% and 8 out of the 11 participants achieved online classification accuracies higher than 90%. Moreover, the lowest online classification accuracy (75%) in the VA condition is equivalent to the highest online classification accuracy in the NVA condition. As in all cases, the VA condition could obtain superior performance compared to the NVA condition.

According to the feedback provided by the 11 participants who attempted to control the BCI using both stimulation conditions, it was easier to clearly count the number of times the target stimuli was presented in the VA condition. Most importantly, all participants hold the view that the NVA condition made them more tired compared to the VA condition (see Table 3). On one hand, these phenomena indicated that visual attention could help the participants pay more attention to the targets and avoid forgetting the position of the target stimuli. On the other hand, in order to accurately count the number of times the target appears, the participants needed to spatially concentrate on the target, which could cause fatigue and discomfort over a prolonged period of time. Conversely, visual attention would deal with these problems and make the participants feel relaxed.

5. Conclusions

The main goal of this study was to assess the influence of using visual attention during attempted control of a conventional tactile P300 BCI. Two stimulation conditions were explored and compared. The test results of eleven participants showed that the VA condition could obtain superior performance and was preferred by the participants over the NVA condition. Thus, the involvement of visual attention can have positive effects on both tactile P300 BCI performance and user-evaluation. Future work will concentrate on further optimization such tactile BCI stimulation conditions and on further validation by more participants and BCI end user groups.

Data Availability

The data used to support the findings of this study are included within the article.

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgments

This work was supported by the National key Research and Development Program 2017YFB13003002. This work was also supported in part by the Grant National Natural Science Foundation of China, under Grant nos. 61573142, 61773164, and 91420302, the Programme of Introducing Talents of Discipline to Universities (the 111 Project) under Grant B17017, and the “ShuGuang” project supported by Shanghai Municipal Education Commission and Shanghai Education Development Foundation under Grant 19SG25.

References

U. Chaudhary, N. Birbaumer, and A. Ramos-Murguialday, “Brain-computer interfaces for communication and rehabilitation,” Nature Reviews Neurology, vol. 12, no. 9, pp. 513–525, 2016.
View at: Publisher Site | Google Scholar
J. J. Vidal, “Toward direct brain-computer communication,” Annual Review of Biophysics and Bioengineering, vol. 2, no. 1, pp. 157–180, 1973.
View at: Publisher Site | Google Scholar
M. F. Mason, M. I. Norton, J. D. Van Horn, D. M. Wegner, S. T. Grafton, and C. N. Macrae, “Wandering minds: the default network and stimulus-independent thought,” Science, vol. 315, no. 5810, pp. 393–395, 2007.
View at: Publisher Site | Google Scholar
A. Herweg, J. Gutzeit, S. Kleih, and A. Kübler, “Wheelchair control by elderly participants in a virtual environment with a brain-computer interface (BCI) and tactile stimulation,” Biological Psychology, vol. 121, pp. 117–124, 2016.
View at: Publisher Site | Google Scholar
H. Wang and A. Bezerianos, “Brain-controlled wheelchair controlled by sustained and brief motor imagery BCIs,” Electronics Letters, vol. 53, no. 17, pp. 1178–1180, 2017.
View at: Publisher Site | Google Scholar
X. An, J. Tang, S. Liu et al., “Effects of temporal congruity between auditory and visual stimuli using rapid audio-visual serial presentation,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 10, pp. 2125–2132, 2016.
View at: Google Scholar
M. Xu, X. Xiao, Y. Wang et al., “A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli,” IEEE Transactions on Biomedical Engineering, vol. 65, no. 5, pp. 1166–1175, 2018.
View at: Publisher Site | Google Scholar
F. Achic, J. Montero, C. Penaloza et al., “Hybrid BCI system to operate an electric wheelchair and a robotic arm for navigation and manipulation tasks,” in Proceedings of the IEEE Workshop on Advanced Robotics and its Social Impacts, pp. 249–254, Shanghai, China, July 2016.
View at: Publisher Site | Google Scholar
Q. Gao, L. Dou, A. N. Belkacem et al., “Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system,” BioMed Research International, vol. 2017, Article ID 8316485, 8 pages, 2017.
View at: Publisher Site | Google Scholar
B. Abibullaev and A. Zollanvari, “Learning discriminative spatiospectral features of ERPs for accurate brain-computer interfaces,” IEEE Journal of Biomedical and Health Informatics, vol. 149, pp. 93–99, 2019.
View at: Google Scholar
J. N. Cruz, F. Wan, C. M. Wong et al., “Adaptive time-window length based on online performance measurement in SSVEP-based BCIs,” Neurocomputing, vol. 23, no. 5, pp. 2009–2020, 2015.
View at: Publisher Site | Google Scholar
G. Pfurtscheller, “Graphical display and statistical evaluation of event-related desynchronization (ERD),” Electroencephalography and Clinical Neurophysiology, vol. 43, no. 5, pp. 757–760, 1977.
View at: Publisher Site | Google Scholar
G. Pfurtscheller, “Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest,” Electroencephalography and Clinical Neurophysiology, vol. 83, no. 1, pp. 62–69, 1992.
View at: Publisher Site | Google Scholar
A. M. Brouwer and J. B. Van Erp, “A tactile P300 brain-computer interface,” Frontiers in Neuroscience, vol. 4, no. 4, p. 19, 2010.
View at: Publisher Site | Google Scholar
L. A. Farwell and E. Donchin, “Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials,” Electroencephalography and Clinical Neurophysiology, vol. 70, no. 6, pp. 510–523, 1988.
View at: Publisher Site | Google Scholar
T. Kaufmann, S. Schulz, C. Grünzinger et al., “Flashing characters with famous faces improves ERP-based brain-computer interface performance,” Journal of Neural Engineering, vol. 8, no. 5, Article ID 056016, 2011.
View at: Publisher Site | Google Scholar
N. J. Hill, T. N. Lal, K. Bierig et al., “An auditory paradigm for brain-computer interfaces,” Advances in Neural Information Processing Systems, vol. 17, pp. 569–576, 2005.
View at: Google Scholar
H. Mori, S. Makino, and T. M. Rutkowski, “Multi-command chest tactile brain computer interface for small vehicle robot navigation,” in Lecture Notes in Computer Science, pp. 469–478, Springer, Cham, Switzerland, 2013.
View at: Publisher Site | Google Scholar
H. Mori, Y. Matsumoto, V. Kryssanov et al., “Multi-command tactile brain computer interface: a feasibility study,” in Proceeding of the International Workshop on Haptic and Audio Interaction Design, pp. 50–59, Daejeon, Korea, April 2013.
View at: Publisher Site | Google Scholar
T. Kodama, S. Makino, and T. M. Rutkowski, “Spatial tactile brain-computer interface paradigm applying vibration stimuli to large areas of user’s back,” 2014, https://arxiv.org/abs/1404.4226.
View at: Google Scholar
T. M. Rutkowski and H. Mori, “Tactile and bone-conduction auditory brain computer interface for vision and hearing impaired users,” Journal of Neuroscience Methods, vol. 244, pp. 45–51, 2015.
View at: Publisher Site | Google Scholar
A. M. Brouwer, J. B. Van Erp, F. Aloise et al., “Tactile, visual, and bimodal P300s: could bimodal P300s boost BCI performance?” SRX Neuroscience, vol. 2010, Article ID 967027, 9 pages, 2010.
View at: Publisher Site | Google Scholar
E. Yin, T. Zeyl, R. Saab, D. Hu, Z. Zhou, and T. Chau, “An auditory-tactile visual saccade-independent P300 brain-computer interface,” International Journal of Neural Systems, vol. 26, no. 1, Article ID 1650001, 2016.
View at: Publisher Site | Google Scholar
J. Jin, B. Z. Allison, E. W. Sellers et al., “An adaptive P300-based control system,” Journal of Neural Engineering, vol. 8, no. 3, Article ID 8036006, 2011.
View at: Publisher Site | Google Scholar
U. Hoffmann, J.-M. Vesin, T. Ebrahimi, and K. Diserens, “An efficient P300-based brain-computer interface for disabled subjects,” Journal of Neuroscience Methods, vol. 167, no. 1, pp. 115–125, 2008.
View at: Publisher Site | Google Scholar
J. Cheng, J. Jin, and X. Wang, “Comparison of the BCI Performance between the Semitransparent face pattern and the traditional face pattern,” Computational Intelligence and Neuroscience, vol. 2017, Article ID 1323985, 9 pages, 2017.
View at: Publisher Site | Google Scholar
S. Zhou, J. Jin, I. Daly et al., “Optimizing the face paradigm of BCI system by modified mismatch negative paradigm,” Frontiers in Neuroscience, vol. 10, p. 444, 2016.
View at: Publisher Site | Google Scholar
J. Jin, H. Zhang, I. Daly et al., “An improved P300 pattern in BCI to catch user’s attention,” Journal of Neural Engineering, vol. 14, no. 3, Article ID 036001, 2017.
View at: Publisher Site | Google Scholar
J. R. Wolpaw, N. Birbaumer, W. J. Heetderks et al., “Brain-computer interface technology: a review of the first international meeting,” IEEE Transactions on Rehabilitation Engineering, vol. 8, no. 2, pp. 164–173, 2000.
View at: Publisher Site | Google Scholar
E. Maby, R. Le Bouquin Jeannes, C. Liegeok-Chauvel, B. Gourevitch, and G. Faucon, “Analysis of auditory evoked potential parameters in the presence of radiofrequency fields using a support vector machines method,” Medical and Biological Engineering and Computing, vol. 42, no. 4, pp. 562–568, 2004.
View at: Publisher Site | Google Scholar
J. Jin, B. Z. Allison, Y. Zhang, X. Wang, and A. Cichocki, “An ERP-based BCI using an oddball paradigm with different faces and reduced errors in critical functions,” International Journal of Neural Systems, vol. 24, no. 8, Article ID 1450027, 2014.
View at: Publisher Site | Google Scholar
P. E. Clayson, S. A. Baldwin, and M. J. Larson, “How does noise affect amplitude and latency measurement of event-related potentials (ERPs)? A methodological critique and simulation study,” Psychophysiology, vol. 50, no. 2, pp. 174–186, 2013.
View at: Publisher Site | Google Scholar
J. Hori and N. Okada, “Classification of tactile event-related potential elicited by braille display for brain-computer interface,” Biocybernetics and Biomedical Engineering, vol. 37, no. 1, pp. 135–142, 2017.
View at: Publisher Site | Google Scholar
G. R. Mangun and S. A. Hillyard, “Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming,” Journal of Experimental Psychology: Human Perception and Performance, vol. 17, no. 4, pp. 1057–1074, 1991.
View at: Publisher Site | Google Scholar
Y. Wang, J. Wu, S. Fu, Y. Luo et al., “Orienting and focusing in voluntary and involuntary visuospatial attention conditions,” Journal of Psychophysiology, vol. 24, no. 3, pp. 198–209, 2010.
View at: Publisher Site | Google Scholar
M. Schreuder, B. Blankertz, M. Tangermann et al., “A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue,” PLoS One, vol. 5, no. 4, Article ID e9813, 2010.
View at: Publisher Site | Google Scholar
W. Penfield and E. Boldrey, “Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation,” Brain, vol. 60, no. 4, pp. 389–443, 1937.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Zongmei Chen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1137

Downloads

1771

Citations