A Computationally Efficient Method for Hybrid EEG-fNIRS BCI Based on the Pearson Correlation

Hasan, Mustafa A. H.; Khan, Muhammad U.; Mishra, Deepti

doi:https://doi.org/10.1155/2020/1838140

BioMed Research International

On this page

Abstract Introduction Materials and Methods Results Discussion Conclusions Data Availability Conflicts of Interest References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 1838140 | https://doi.org/10.1155/2020/1838140

A Computationally Efficient Method for Hybrid EEG-fNIRS BCI Based on the Pearson Correlation

Mustafa A. H. Hasan,¹Muhammad U. Khan,¹and Deepti Mishra²

Academic Editor: Camillo Porcaro

Received11 Feb 2020

Revised29 Jun 2020

Accepted31 Jul 2020

Published19 Aug 2020

Abstract

A hybrid brain computer interface (BCI) system considered here is a combination of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). EEG-fNIRS signals are simultaneously recorded to achieve high motor imagery task classification. This integration helps to achieve better system performance, but at the cost of an increase in system complexity and computational time. In hybrid BCI studies, channel selection is recognized as the key element that directly affects the system’s performance. In this paper, we propose a novel channel selection approach using the Pearson product-moment correlation coefficient, where only highly correlated channels are selected from each hemisphere. Then, four different statistical features are extracted, and their different combinations are used for the classification through KNN and Tree classifiers. As far as we know, there is no report available that explored the Pearson product-moment correlation coefficient for hybrid EEG-fNIRS BCI channel selection. The results demonstrate that our hybrid system significantly reduces computational burden while achieving a classification accuracy with high reliability comparable to the existing literature.

1. Introduction

In 1924, Hans Berger, a neurologist, recorded human brain signals through EEG for the first time. This actually encouraged other researchers to further investigate the human brain and record its activity using brain computer interface (BCI). This system provides the means for disabled patients to control and communicate with the surroundings, namely, quadcopters, manipulators, and other mechanisms, solely using brain activities. There are different invasive and noninvasive modalities used for BCI applications such as near-infrared spectroscopy (NIRS) [1], functional magnetic resonance imaging (fMRI) [2], and magnetoencephalography (MEG) [3]. In the last decade, most BCI studies have mainly concentrated on electroencephalography (EEG).

EEG, a noninvasive method, registers the electrical activity in the scalp, generated by the brain to provide high temporal resolution with low cost, and portability. Yet, it lacks spatial resolution [4], not to mention poor signal-to-noise ratio, relying on physical or mental tasks, and subject to contamination with various artefacts, such as external electromagnetic waves, e.g., from an electromyogram and an electrooculogram [5]. As a result, the classification accuracy obtained through EEG suffers heavily and often leads to misclassification and false-command generation. Therefore, in order to enhance EEG’s performance and make it more reliable, several studies have suggested supporting it with a second modality, such as NIRS [6–9].

Functional near-infrared spectroscopy (fNIRS), another noninvasive method, measures the concentration change of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) by picking two distinct near-infrared (NIR) wavelengths (600 nm to 1000 nm). fNIRS offers subsecond temporal resolution and spatial resolution in 1 cm² [10] and has shown its potential in localizing task activations in the same way as fMRI. However, fNIRS has some serious concerns that constrains the overall performance. Contrary to EEG, NIRS is considered to have strong immunity against electrical noises and motion artefacts; however, it suffers from long delay in the hemodynamic response. The response time in order to generate the execution command for NIRS is approximately nine times that of EEG [7]. Over the past few years, researchers have been striving to increase the information transfer rates (ITR) and to overcome the limitations of unimodal systems, leading to multimodal ones, formally known as a hybrid BCI.

In [11], the authors made the first attempt to investigate a hybrid EEG-fNIRS-based BCI system, and to integrate the features of EEG and fNIRS, they designed a metaclassifier for the hybrid BCI. Since then, many other studies have addressed the benefits of the complementary properties of integrated EEG-fNIRS [6, 7, 9, 12]. This combination enhanced the signal strength, improved the classification accuracy, and overcame most of the limitations of single modalities [6, 11, 13]. The main challenge in a hybrid BCI is to reduce system complexity, improve response time, and maintain high classification accuracy [1, 14, 15]. Though, the multimodal BCI system has enhanced the classification accuracy, methodological constraints still exist, such as heavy reliance on the principle component analysis (PCA) and common spatial pattern (CSP) [16, 17], empirical wavelet transform (EWT) [18], and multivariate empirical wavelet transform (MEWT) [19] to reduce the dimensionality of the original dataset. Another motivation behind the integration of EEG-fNIRS is to retain and highlight the favourable properties of each unimodal system [20].

In hybrid BCI studies, three major steps define the overall performance of the system: channel selection, feature extraction, and classification. This study focuses on the identification and selection of most optimal channels with the intention to reduce the computational cost while maintaining high classification accuracy. In previous studies [1, 7], all available channels from both hemispheres were considered for feature extraction and classification. This not only increased system complexity but also computational cost.

To confront these issues, various methods have been proposed in the near past. It is worth mentioning that most of the literature has merely focused on crude analysis to select the fewer channels, manually [21–23]. This approach may work in some cases, but the generalization of such methods is limited by self-analysis as well as by the excessive time required in order to analyse each individual channel. In contrast, some researchers endeavoured for the sophisticated approaches as how to determine the optimal channels. In [9], the authors proposed to select a singular channel of EEG-fNIRS from both hemispheres using a general linear model (GLM); however, the obtained average classification accuracy was moderate. Also, the performance of GLM might be subjected to artefacts that infect the data, such as motion artefacts, low frequency inclines, and serial correlations [24]. Common spatial patterns (CSPs) have also been utilized to determine the most effective channels [25]. Based upon the average energy of each channel, the ones with the highest energy are selected; whereas, those with low energy are assumed to carry noise and, hence, set aside.

In this paper, we present a novel approach for a hybrid EEG-fNIRS BCI channel selection using the Pearson product-moment correlation coefficient (PPMCC). In recent years, researchers have tried to explore the benefits of correlation in BCI for the window selection [26], channel selection [27], and classification [28]. The determination of the correlation coefficient is a statistical approach that allows us to quantify the strength of association. The calculation yields the linear association between two channels which is then used to select the most dominant ones. Based upon the correlation coefficient, a rank matrix is developed. The highest ranked channels are classified as active channels or true motor imagery signals, whereas the lowest ranked channels are regarded as nonrepresentative channels of motor imagery signals. Those with lower correlation coefficients can also be interpreted as artefacts, noise, or other signals that are not correlated with motor imagery signals. To the best knowledge of the authors, PPMCC has never been utilized for channel selection in a BCI paradigm before. Therefore, this study is expected to introduce a completely new perspective for channel selection in hybrid BCI systems. Through results, it is demonstrated that the proposed approach can play its role effectively, when used in conjunction with two different classifiers, to attain reduced computational time and high classification accuracy. The following sections present the signal processing, feature extraction, and classification approaches. In Results, the outcomes are shown with focus on the performance. Discussion and Conclusion conclude the work by summarizing the findings and proposed future work.

2. Materials and Methods

2.1. Data Source and Data Processing

The dataset for this study is taken from an online repository, provided by [6]. It contains raw data of simultaneously recorded EEG-fNIRS motor tasks of the left arm, the left hand, the right arm, and the right hand. For motor imagery tasks, fifteen healthy subjects took part in the experiment, which lasted an hour. During the experiment, the subjects were instructed to perform four movements—right–left-hand gripping, right–left-arm raising—by following visual instructions displayed on a laptop screen placed 1 m away from where they were sitting. For each subject, the trials started with a rest task of 6 seconds, followed by another 6 seconds of movements according to the screen instructions. The raw EEG signals were obtained at 250 Hz through twenty-one channels, all of which were baseline corrected by subtracting the mean value of channels and then filtered at 1-25 Hz by a 4th-order IIR Butterworth filter [9]. The fNIRS signals were acquired at a sampling frequency of 10.42 Hz for two wavelengths—760 nm and 850 nm—equipped with twelve sources and twelve electrodes on thirty-four channels. The sources and detectors of the fNIRS are at maximum 3.4 cm apart from each other, in order to ascertain high-quality signals. The raw data is later decomposed into oxyhemoglobin and deoxyhemoglobin concentration changes (HbO and HbR, respectively) through the Modified Beer-Lambert law [29]. In this study, we used HbO, referred to as fNIRS from here on. The fNIRS data is filtered by a 4th-order band pass IIR Butterworth filter (between 0.01 and 0.2 Hz) to remove artefacts [6, 9]. Initialization trial prior to the motor tasks was segmented out. The filtered EEG and fNIRS data are then normalized by subtracting the mean and dividing by the standard deviation. Figure 1 shows the hybrid BCI system that highlights all the major steps involved.

2.2. Channel Selection for Hybrid BCI

In hybrid BCI studies, channel selection is considered as the key element that directly affects the system’s performance. The Pearson correlation coefficient has been applied by a number of researchers [27, 30, 31] in order to solve practical problems in medical research. Although it has given some promising results, it has never been tested for hybrid BCI systems before. Hence, we propose a novel approach to ensure optimal performance by involving only the most optimized channels using the Pearson product-moment correlation coefficient. The approach is based upon statistical analysis that allows to quantify the strength of a linear association between two channels, denoted by , having a value between . The basic idea is to associate the data of two channels through the best fit line. The Pearson correlation coefficient, , is an indicator of the placement of these data points in reference to the line of best fit. A higher positive value indicates a stronger association, whereas a more negative value is an indicator of a stronger negative association. The third possibility is absolutely no association between the variables, i.e., .

EEG-fNIRS channels are distributed into two groups based upon their placement in the right and left hemispheres, as shown in Figure 2. The correlation coefficient between the two intervals in each group is a measure of how close a linear relationship these two intervals possess. Given a pair of two channels , the correlation coefficient is defined as follows: where represents the mean value, denotes the standard deviation, and represents the expectation operator.

The reference guideline for interpreting the strength of association based upon the correlation coefficient is given in Table 1. For the purpose of selecting the most representative channels (see Algorithm 1) with the most meaningful performance, and to eliminate signals with unnecessary information and noise, highly correlated EEG-fNIRS channels from each hemisphere are chosen based upon their ranking. To reduce dimensions and improve the parity of the two systems, the EEG-fNIRS dataset is downsized through a 0.096 sec averaged moving window.

2.3. Feature Extraction

Once the channels are selected, the next task is to prepare a feature set for classification. Four different statistical features are extracted using spatial averaging of the selected channels for both EEG and fNIRS features.

Ranking of channels.
Input: Number of channels
Output: The ranking of channels
for each channel to do

for each channel to do
ifthen
Compute correlation between channels and using equation (1)

End if
End for

Based upon the , classify channels as {high, mid, low} according to Table 1
End for.

2.3.1. Signal Mean ()

For channel acquired through EEG-fNIRS, the signal is contained in through , then the mean value for the discrete signal is evaluated as follows: where is the total number of observations.

2.3.2. Signal Skewness (SK)

The skewness of a channel is the third standardized moment, represented as follows:

The distribution of is said to be positively skewed (), negatively skewed (), or unskewed ().

2.3.3. Signal Kurtosis (KR)

The kurtosis of is the fourth standardized moment, defined as follows:

Kurtosis is an indicator of the peak of the distribution. For positive it is peaked more, and for negative it is peaked less than the normal distribution.

2.3.4. Signal Peak ()

The peak of a channel is defined as .

Once the feature set is defined, the next process is to normalize the feature set of both EEG-fNIRS between 0 and 1 using

From here on, it is assumed that all the features are normalized, hence new feature vectors are obtained as , , , and . But, in order to avoid the ambiguity and for the sake of easiness, the normalized features are represented as , , , and .

2.4. Classification

Prior to classification, three different sets of features are constructed: the EEG-only set, the fNIRS-only set, and the hybrid EEG-fNIRS set. Two different classifiers, -nearest neighbor (KNN) and Decision Tree classifier, are used to perform classifications among five different classes: four motor tasks and one rest task. Both classifiers belong to the supervised learning and are widely used due to their simplicity and ease of implementation. A 10-fold crossvalidation paradigm is implemented to split each feature set into ten subsets, where nine subsets are used to train the classifiers, and the remaining one subset is used to test the classification accuracy. Through the results, it is observed that both classifiers have performed well in order to attain high classification accuracy.

3. Results

The goal of this study is to perform a comparison among three different sets of features: EEG-only, fNIRS-only, and hybrid EEG-fNIRS for the selected channels based on a correlation coefficient. To achieve this, we performed two-trial classifications of the four motor execution tasks versus rest. A classification accuracy of 100% refers to perfectly separated motor tasks, whereas 50% signifies poor performance. The movements of the subjects are recorded through twenty-one EEG and thirty-four fNIRS channels in total.

To analyse the reduction in the computational time, the temporal distance between the filtration step and feature extraction is recorded as an evaluation time. Comparison is made for 10 sec of the sampled data obtained through EEG and fNIRS. Figure 3 shows the computational time required in order to process a hundred and twenty-five-hundred samples of fNIRS and EEG, respectively, against a varying number of channels. It can be observed that the relation between the number of channels and computational time is almost linear. Thus, by utilizing a lesser number of channels, time reduction is expected. Through the proposed approach, only highly correlated channels are obtained and processed. For EEG, only six channels are selected out of twenty-one, and for fNIRS, only ten channels are selected out of thirty-four. The processing time is reduced by more than 40% for EEG and around 20% for fNIRS. In this study, it has been observed that for the experiments considered, if we further try to reduce the number of channels, the accuracy will start to deteriorate with not much improvement in time. In the later part of this section, it is revealed that only those channels that are the carrier of less significant information compared to the rest are dropped. This helps to get rid of the outliers and noise variation that may have been introduced in some channels at the time of data acquisition.

With the help of a novel correlation coefficient-based method, only those channels that contain the most relevant motor imagery information are picked. The probability of the channels being selected based upon the correlation results obtained for subjects 1, 4, and 15 is depicted in Figure 4. The highest correlation coefficients are obtained in the motor cortex region in the right and left hemispheres. According to Figure 4, this is true because the movements considered are of the left arm, left hand, right arm, right hand, and rest. The highest correlation coefficient helps to separate the optimized channels from the rest. Once the channels are selected, the next task is to train the classifiers on the given dataset from different sources in order to obtain accurate results.

(a)

(b)

(c)

3.1. EEG

The average classification accuracies for five different classes (four movements, one rest) obtained using KNN and Tree classifiers are shown in Table 2. Mean (), peak (), kurtosis (KR), and skewness (SK) are defined as statistical features. The reduced number of channels from eight subjects are processed to generate a feature set that contains all one-to-one possible combinations of the features. The Tree classifier has produced acceptable results when evaluated against KNN for the given feature set. The average classification accuracy obtained through KNN is well below the defined acceptable threshold of 50%. This phenomenon has occurred, according to our observation, as some of the subjects were not presumably fully engaged in the task and lost interest at some stage during the process. This caused an overall drop in the accuracy, thus the motivation to eliminate the channels that directly affects the accuracy. In order to observe the performance of eight individual subjects, the classification results obtained through both classifiers using the optimized channels are plotted in Figure 5. In accordance with the results obtained in Table 2, it is reaffirmed that the KNN classifier has not been able to produce any satisfactory results, whereas the Tree classifier hardly satisfies the defined acceptable threshold. The average classification accuracies of the EEG-only feature set achieved through the Butterworth filter are and . The classification accuracies obtained through the Tree classifier for the selected subjects against a varying number of channels are plotted in Figure 6. The feature set is based upon the data obtained from five selected subjects including randomly selected (). It is observed that an increase in the number of channels has not produced any significant effect on the accuracy but certainly increased the computational cost.

(a)

(b)

3.2. fNIRS

Table 2 shows the KNN and Tree classification results—using the most optimized channels from eight subjects—for the fNIRS-only dataset. In this study, it is revealed that, for fNIRS, the combination of peak and skewness has produced the highest accuracy for both classifiers. The accuracy obtained through fNIRS is much higher as compared to EEG. To highlight the performance variation among subjects, classification accuracies based upon eight individuals—along with their average—are depicted in Figure 7. The fNIRS-only feature set filtered by the Butterworth filter achieved an average accuracy of and .

(a)

(b)

It is observed that some subjects have really outperformed others; e.g., subject nine can be regarded as the healthiest among the sample set, whose accuracy is well above the average accuracy for fNIRS. Thus, it can be concluded that for the overall high accuracy, every subject is supposed to perform at their best. Figure 8 shows the average classification accuracies using the Tree classifier for five selected subjects including randomly selected (sr), when evaluated against a varying number of channels. Again, subject nine has been able to perform the best as compared to the others. The increase in the number of channels did not play a big role in improving the accuracy; in fact, if we select less than 30% of the total number of channels, then the accuracy drops significantly.

3.3. Hybrid EEG-fNIRS

For the hybrid EEG-fNIRS, the EEG feature set contains six feature vectors that are combined with the fNIRS feature set, which also contain the same number of feature vectors, in order to generate a total of thirty-six feature vectors. Table 3 shows the performance in terms of the average classification accuracy for eight subjects obtained through KNN and Tree classifiers. No absolute judgement can be made regarding the most desired feature set; however, has performed well as compared to the rest at multiple instances. The highest and the lowest average classification accuracies of 78.2% and 66.7% are obtained through the Tree classifier, whereas the highest and lowest average classification accuracies obtained through the KNN classifier are 75.28% and 60.76%.

Figure 9 shows the classification accuracies for eight subjects obtained through the KNN and Tree classifiers against different features; moreover, the average accuracies are also depicted through filled circles. It is observed that against some feature vectors, the Tree classifier has been able to get as high as 90% classification accuracy. Overall, the Tree classifier has proved to be more effective when compared against KNN for the hybrid EEG-fNIRS BCI system.

To evaluate the performance of the proposed approach, it is compared against [6], who provided the actual dataset. The classification accuracy is obtained through a linear discriminant analysis (LDA) classifier for the rest vs. the right-hand movement. For true analysis, the type of classifier and the movement is kept the same as [6]. For EEG-only, fNIRS-only, and hybrid EEG-fNIRS, our approach achieves the classification accuracy of 61.6%, 98%, and 98.6%, respectively. The classification results obtained by [6] for EEG-only, fNIRS-only, and hybrid EEG-fNIRS are 85.4%, 92.4%, and 94.2%, respectively. It can be observed that at two out of three instances, the proposed approach performed better against Buccino et al. [6], who considered all channels.

4. Discussion

The hybrid BCI using EEG-fNIRS has proved itself capable of improving classification accuracy as compared to a single modality [11]. In this paper, we attempted to improve the classification accuracy of motor tasks as well as to reduce the computational cost. The results demonstrate that the EEG-fNIRS combination based upon the selection of the most optimized channels has performed better compared to the unimodal approaches. It is widely accepted that the design and application of a BCI system is strongly influenced by the selection of channels, their number, and their placement [9].

To generate execution commands based upon motor imagery tasks, previous studies have recommended picking channels from the C3 and C4 areas [6, 11]. Yet, this approach cannot be generalized for all due to variations from subject to subject, as identical channels may exist in different brain regions. Although some researchers investigated the efficiency of different channel selection approaches [7, 17, 33], little effort has been made in the selection of the most optimized channels. In this study, we use a set of channels from each hemisphere with high similarities based on the correlation coefficient to ensure that the most effective channels are chosen for feature extraction and classification. The obtained results show that not only did the classification accuracy increase but the computational complexity also reduced. As Table 3 summarizes the classification accuracy of the merged features of EEG-fNIRS, it is noticeable that although few channels are used for the classification, the accuracy tended to be decent as compared to previous studies.

The classification performance of a full channel set does not necessarily yield an optimal performance as witnessed in Figures 6 and 8. Thus, it is desired that only those sets of channels that actually contain a significant amount of information must be considered. Through the selection of the most optimized channels, the processing time has improved by 40% for EEG and around 20% for fNIRS. The channel selection approach based on the ranking of the correlation coefficient can identify the optimal channel combination and enhance the classification performance. The classification accuracy of the system can further be increased by selecting optimal features for the system. Signal mean, peak, skewness, and kurtosis are used as the feature set for the classification. It has been shown that the use of these sets of features achieved high accuracy in fNIRS classification, yet the accuracy in EEG is low due to the dimensionality issue and, as mentioned by [6], the experimental design and procedure strongly affect the performance.

5. Conclusions

In this study, a hybrid EEG-fNIRS configuration is proposed for motor task classification. The primary goal of this work is to reduce the computational cost by not compromising the classification accuracy. In order to realize such a system, for the first time we propose to utilize a correlation coefficient for the selection of the most optimized channels. To validate the effectiveness of the proposed approach, eight different subjects are considered and multiple trials are performed. As evident from the results, our hybrid system significantly reduces the computational burden while achieving the classification accuracy with high reliability comparable to the existing literature.

Despite this accomplishment, we observed two major limitations that hold back the overall accuracy of the hybrid BCI system. The motor imagery task data was collected with an EEG visual feedback system. Through the visual information displayed on the screen, the subjects were instructed to perform certain movements. The motor classification accuracy of is achieved through EEG only, which is not even comparable to the accuracy of obtained through fNIRS using the Tree classifier. This phenomenon has also been experienced by some researchers when dealing with EEG [34]. This is perhaps because the subjects’ involved in the experiments were exposed to the motor imagery tasks for the first time. Therefore, in order to improve the results in motor imagery tasks, it has been suggested in [35] that all the subjects be trained for 1-4 hours with visual feedback informing the user whether his/her imagery strategy is correctly classified. Secondly, as EEG suffers from low spatial resolution, it directly affects the overall performance of the hybrid system. The use of high-resolution EEG can possibly overcome this problem. As to future work, we focus on the validation of the results from the practical perspective [36–38] and the investigation of the performance of the entire system using online data. Furthermore, different feature sets will be explored in our studies to further improve the classification of the system.

Data Availability

The raw data used to support the findings of this study are deposited in EEG-fNIRS hybrid SMR BCI data S1-S8 and EEG-fNIRS hybrid SMR BCI data S9-S15.

Conflicts of Interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

References

N. Naseer and K. S. Hong, “fNIRS-based brain-computer interfaces: a review,” Frontiers in Human Neuroscience, vol. 9, no. 3, 2015.
View at: Publisher Site | Google Scholar
M. O. Sokunbi, V. B. Gradin, G. D. Waiter et al., “Nonlinear complexity analysis of brain FMRI signals in schizophrenia,” PLoS One, vol. 9, no. 5, article e95146, 2014.
View at: Publisher Site | Google Scholar
S. K. L. Lal, A. Craig, P. Boord, L. Kirkup, and H. Nguyen, “Development of an algorithm for an EEG-based driver fatigue countermeasure,” Journal of Safety Research, vol. 34, no. 3, pp. 321–328, 2003.
View at: Publisher Site | Google Scholar
B.-K. Min, M. J. Marzelli, and S.-S. Yoo, “Neuroimaging-based approaches in the brain-computer interface,” Trends in Biotechnology, vol. 28, no. 11, pp. 552–560, 2010.
View at: Publisher Site | Google Scholar
O. Aydemir and T. Kayikcioglu, “Decision tree structure based classification of EEG signals recorded during two dimensional cursor movement imagery,” Journal of Neuroscience, vol. 229, pp. 68–75, 2014.
View at: Publisher Site | Google Scholar
A. P. Buccino, H. O. Keles, and A. Omurtag, “Hybrid EEG-fNIRS asynchronous brain-computer interface for multiple motor tasks,” PLoS One, vol. 11, no. 1, article e0146610, 2016.
View at: Publisher Site | Google Scholar
M. J. Khan and K.-S. Hong, “Hybrid EEG-fNIRS-based eight-command decoding for BCI: application to quadcopter control,” Frontiers in Neurorobotics, vol. 11, p. 6, 2017.
View at: Publisher Site | Google Scholar
B. Koo, H. G. Lee, Y. Nam et al., “A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery,” Journal of Neuroscience Methods, vol. 244, pp. 26–32, 2015.
View at: Publisher Site | Google Scholar
R. Li, T. Potter, W. Huang, and Y. Zhang, “Enhancing performance of a hybrid EEG-fNIRS system using channel selection and early temporal features,” Frontiers in Human Neuroscience, vol. 11, p. 462, 2017.
View at: Publisher Site | Google Scholar
M. L. Schroeter, T. Kupka, T. Mildner, K. Uludağ, and D. Y. von Cramon, “Investigating the post-stimulus undershoot of the BOLD signal—a simultaneous fMRI and fNIRS study,” NeuroImage, vol. 30, no. 2, pp. 349–358, 2006.
View at: Publisher Site | Google Scholar
S. Fazli, J. Mehnert, J. Steinbrink et al., “Enhanced performance by a hybrid NIRS-EEG brain computer interface,” NeuroImage, vol. 59, no. 1, pp. 519–529, 2012.
View at: Publisher Site | Google Scholar
X. Yin, B. Xu, C. Jiang et al., “A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching,” Journal of Neural Engineering, vol. 12, no. 3, article 036004, 2015.
View at: Publisher Site | Google Scholar
F. Putze, S. Hesslinger, C. Y. Tse et al., “Hybrid fNIRS-EEG based classification of auditory and visual perception processes,” Frontiers in Neuroscience, vol. 8, p. 373, 2014.
View at: Publisher Site | Google Scholar
J. Shin, K. R. Müller, C. H. Schmitz, D. W. Kim, and H. J. Hwang, “Evaluation of a compact hybrid brain-computer interface system,” BioMed Research International, vol. 2017, Article ID 6820482, 11 pages, 2017.
View at: Publisher Site | Google Scholar
A. Zafar, K.-S. Hong, and M. J. Khan, “Initial dip detection based on both HbO and HbR vector-based phase analysis,” in 2016 55th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 543–548, Tsukuba, Japan, 2016.
View at: Publisher Site | Google Scholar
Z. Li, Y. Wang, W. Quan, T. Wu, and B. Lv, “Evaluation of different classification methods for the diagnosis of schizophrenia based on functional near-infrared spectroscopy,” Journal of Neuroscience Methods, vol. 241, pp. 101–110, 2015.
View at: Publisher Site | Google Scholar
B. Blankertz, G. Dornhege, M. Krauledat, K. R. Müller, and G. Curio, “The non-invasive Berlin brain-computer interface: fast acquisition of effective performance in untrained subjects,” NeuroImage, vol. 37, no. 2, pp. 539–550, 2007.
View at: Publisher Site | Google Scholar
M. T. Sadiq, X. Yu, Z. Yuan et al., “Motor imagery EEG signals classification based on mode amplitude and frequency components using empirical wavelet transform,” IEEE Access, vol. 7, pp. 127678–127692, 2019.
View at: Publisher Site | Google Scholar
M. T. Sadiq, X. Yu, Z. Yuan et al., “Motor imagery EEG signals decoding by multivariate empirical wavelet transform-based framework for robust brain-computer interfaces,” IEEE Access, vol. 7, pp. 171431–171451, 2019.
View at: Publisher Site | Google Scholar
F. Al-Shargie, M. Kiguchi, N. Badruddin, S. C. Dass, A. F. M. Hani, and T. B. Tang, “Mental stress assessment using simultaneous measurement of EEG and fNIRS,” Biomedical Optics Express, vol. 7, no. 10, pp. 3882–3898, 2016.
View at: Publisher Site | Google Scholar
S. Ge, Q. Yang, R. Wang et al., “A brain-computer interface based on a few-channel EEG-fNIRS bimodal system,” IEEE Access, vol. 5, pp. 208–218, 2017.
View at: Publisher Site | Google Scholar
Y. Xie, Q. Yang, P. Lin et al., “Application of phase space reconstruction in a few-channel EEG-fNIRS bimodal brain-computer interface system,” in 2017 International Conference on Computing Intelligence and Information System (CIIS), pp. 66–71, Nanjing, China, 2017.
View at: Publisher Site | Google Scholar
J. Shin, J. Kwon, and C. H. Im, “A ternary hybrid EEG-fNIRS brain-computer interface for the classification of brain activation patterns during mental arithmetic, motor imagery, and idle state,” Frontiers in Neuroinformatics, vol. 12, p. 5, 2018.
View at: Publisher Site | Google Scholar
X.-S. Hu, K. S. Hong, S. S. Ge, and M. Y. Jeong, “Kalman estimator-and general linear model-based on-line brain activation mapping by near-infrared spectroscopy,” Biomedical Engineering Online, vol. 9, no. 1, p. 82, 2010.
View at: Publisher Site | Google Scholar
H. Fauzi, M. I. Shapiai, N. A. Setiawan, J. Jaafar, and M. Mustafa, “Channel selection for common spatial pattern based on energy calculation of motor imagery EEG signal,” in 2017 International Conference on Control, Electronics, Renewable Energy and Communications (ICCREC), pp. 33–39, Yogyakarta, Indonesia, 2017.
View at: Publisher Site | Google Scholar
J. Feng, E. Yin, J. Jin et al., “Towards correlation-based time window selection method for motor imagery BCIs,” Neural Networks, vol. 102, pp. 87–95, 2018.
View at: Publisher Site | Google Scholar
J. Jin, Y. Miao, I. Daly, C. Zuo, D. Hu, and A. Cichocki, “Correlation-based channel selection and regularized feature optimization for MI-based BCI,” Neural Networks, vol. 118, pp. 262–270, 2019.
View at: Publisher Site | Google Scholar
M. Rahman and S. A. Fattah, “Mental task classification scheme utilizing correlation coefficient extracted from interchannel intrinsic mode function,” BioMed Research International, vol. 2017, Article ID 3720589, 11 pages, 2017.
View at: Publisher Site | Google Scholar
R. A. Khan, N. Naseer, N. K. Qureshi, F. M. Noori, H. Nazeer, and M. U. Khan, “fNIRS-based neurorobotic interface for gait rehabilitation,” Journal of Neuroengineering and Rehabilitation, vol. 15, no. 1, p. 7, 2018.
View at: Publisher Site | Google Scholar
H. Akoglu, “User’s guide to correlation coefficients,” Turkish Journal of Emergency Medicine, vol. 18, no. 3, pp. 91–93, 2018.
View at: Publisher Site | Google Scholar
M. M. Mukaka, “A guide to appropriate use of correlation coefficient in medical research,” Malawi Medical Journal, vol. 24, no. 3, pp. 69–71, 2012.
View at: Google Scholar
B. Ratner, “The correlation coefficient: its values range between+1/−1, or do they?” Journal of Targeting, Measurement and Analysis for Marketing, vol. 17, no. 2, pp. 139–142, 2009.
View at: Publisher Site | Google Scholar
H.-D. Nguyen and K.-S. Hong, “Bundled-optode implementation for 3D imaging in functional near-infrared spectroscopy,” Biomedical Optics Express, vol. 7, no. 9, pp. 3491–3507, 2016.
View at: Publisher Site | Google Scholar
B. Blankertz, M. Tangermann, C. Vidaurre et al., “The Berlin brain-computer interface: non-medical uses of BCI technology,” Frontiers in Neuroscience, vol. 4, p. 198, 2010.
View at: Publisher Site | Google Scholar
B. Graimann, B. Allison, and G. Pfurtscheller, Brain-computer interfaces: a gentle introduction, Springer, 2009.
View at: Publisher Site
G. Di Pino, C. Porcaro, M. Tombini et al., “A neurally-interfaced hand prosthesis tuned inter-hemispheric communication,” Restorative Neurology and Neuroscience, vol. 30, no. 5, pp. 407–418, 2012.
View at: Publisher Site | Google Scholar
M. Tombini, J. Rigosa, F. Zappasodi et al., “Combined analysis of cortical (EEG) and nerve stump signals improves robotic hand control,” Neurorehabilitation and Neural Repair, vol. 26, no. 3, pp. 275–281, 2012.
View at: Publisher Site | Google Scholar
F. Ferracuti, V. Casadei, I. Marcantoni et al., “A functional source separation algorithm to enhance error-related potentials monitoring in noninvasive brain-computer interface,” Computer Methods and Programs in Biomedicine, vol. 191, article 105419, 2020.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Mustafa A. H. Hasan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1808

Downloads

1429

Citations