Table of Contents Author Guidelines Submit a Manuscript
Computational Intelligence and Neuroscience
Volume 2017, Article ID 3524208, 24 pages
https://doi.org/10.1155/2017/3524208
Research Article

Mental Task Evaluation for Hybrid NIRS-EEG Brain-Computer Interfaces

Energy, Materials, and Telecommunications, Institut National de la Recherche Scientifique, University of Quebec, Montreal, QC, Canada

Correspondence should be addressed to Hubert Banville; ac.srni.tme@ellivnab.trebuh

Received 6 April 2017; Revised 27 July 2017; Accepted 29 August 2017; Published 18 October 2017

Academic Editor: Manuel Rosa-Zurera

Copyright © 2017 Hubert Banville et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, no. 6, pp. 767–791, 2002. View at Publisher · View at Google Scholar · View at Scopus
  2. J. R. Wolpaw and E. W. Wolpaw, “Brain-computer interfaces: something new under the sun,” in Brain-Computer Interfaces: Principles and Practice, pp. 3–12, Oxford University Press, 2012. View at Publisher · View at Google Scholar · View at Scopus
  3. J. B. F. Van Erp, F. Lotte, and M. Tangermann, “Brain-computer interfaces: beyond medical applications,” Computer, vol. 45, no. 4, pp. 26–34, 2012. View at Publisher · View at Google Scholar · View at Scopus
  4. S. M. Coyle, T. E. Ward, C. M. Markham, and G. McDarby, “On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces,” Physiological Measurement, vol. 25, no. 4, article 815, 2004. View at Publisher · View at Google Scholar · View at Scopus
  5. H.-J. Hwang, S. Kim, S. Choi, and C.-H. Im, “EEG-based brain-computer interfaces: a thorough literature survey,” International Journal of Human-Computer Interaction, vol. 29, no. 12, pp. 814–826, 2013. View at Publisher · View at Google Scholar · View at Scopus
  6. E. C. Leuthardt, G. Schalk, J. R. Wolpaw, J. G. Ojemann, and D. W. Moran, “A brain-computer interface using electrocorticographic signals in humans,” Journal of Neural Engineering, vol. 1, no. 2, pp. 63–71, 2004. View at Publisher · View at Google Scholar · View at Scopus
  7. J. Mellinger, G. Schalk, C. Braun et al., “An MEG-based brain-computer interface (BCI),” NeuroImage, vol. 36, no. 3, pp. 581–593, 2007. View at Publisher · View at Google Scholar · View at Scopus
  8. R. Sitaram, A. Caria, and N. Birbaumer, “Hemodynamic brain-computer interfaces for communication and rehabilitation,” Neural Networks, vol. 22, no. 9, pp. 1320–1328, 2009. View at Publisher · View at Google Scholar · View at Scopus
  9. R. Sitaram, A. Caria, R. Veit et al., “FMRI brain-computer interface: A tool for neuroscientific research and treatment,” Computational Intelligence and Neuroscience, vol. 2007, Article ID 25487, 2007. View at Publisher · View at Google Scholar · View at Scopus
  10. N. Naseer and K. Hong, “fNIRS-based brain-computer interfaces: a review,” Frontiers in Human Neuroscience, vol. 9, article 3, 2015. View at Publisher · View at Google Scholar
  11. H. Banville and T. Falk, “Recent advances and open challenges in hybrid brain-computer interfacing: a technological review of non-invasive human research,” Brain-Computer Interfaces, vol. 3, no. 1, pp. 9–46, 2016. View at Publisher · View at Google Scholar
  12. G. Pfurtscheller, B. Z. Allison, G. Bauernfeind et al., “The hybrid BCI,” Frontiers in Neuroscience, vol. 4, no. 3, 3 pages, 2010. View at Google Scholar
  13. G. R. Müller-Putz, R. Leeb, J. d. Millán et al., “Principles of hybrid brain–computer interfaces,” in Towards Practical Brain-Computer Interfaces, Biological and Medical Physics, Biomedical Engineering, pp. 355–373, Springer, Berlin, Germany, 2013. View at Publisher · View at Google Scholar
  14. T. Cao, F. Wan, C. M. Wong, J. N. da Cruz, and Y. Hu, “Objective evaluation of fatigue by EEG spectral analysis in steady-state visual evoked potential-based brain-computer interfaces,” BioMedical Engineering Online, vol. 13, no. 1, article no. 28, 2014. View at Publisher · View at Google Scholar · View at Scopus
  15. B. Obermaier, C. Neuper, C. Guger, and G. Pfurtscheller, “Information transfer rate in a five-classes brain-computer interface,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 9, no. 3, pp. 283–288, 2001. View at Publisher · View at Google Scholar · View at Scopus
  16. E. Curran, P. Sykacek, M. Stokes et al., “Cognitive Tasks for Driving a Brain-Computer Interfacing System: A Pilot Study,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 12, no. 1, pp. 48–54, 2004. View at Publisher · View at Google Scholar · View at Scopus
  17. F. Sepulveda, M. Dyson, J. Q. Gan, and C. L. Tsui, “A comparison of mental task combinations for asynchronous EEG-based BCIs,” in Proceedings of the 29th Annual International Conference of the IEEE, pp. 5055–5058, Engineering in Medicine and Biology Society, 2007. View at Scopus
  18. F. Faradji, R. K. Ward, and G. E. Birch, “A brain-computer interface based on mental tasks with a zero false activation rate,” in Proceedings of the 2009 4th International IEEE/EMBS Conference on Neural Engineering, NER '09, pp. 355–358, May 2009. View at Publisher · View at Google Scholar · View at Scopus
  19. M.-C. Dobrea and D. M. Dobrea, “The selection of proper discriminative cognitive tasks - A necessary prerequisite in high-quality BCI applications,” in Proceedings of the 2nd International Symposium on Applied Sciences in Biomedical and Communication Technologies, ISABEL 2009, November 2009. View at Publisher · View at Google Scholar · View at Scopus
  20. E. V. C. Friedrich, R. Scherer, K. Sonnleitner, and C. Neuper, “Impact of auditory distraction on user performance in a brain-computer interface driven by different mental tasks,” Clinical Neurophysiology, vol. 122, no. 10, pp. 2003–2009, 2011. View at Publisher · View at Google Scholar · View at Scopus
  21. E. V. C. Friedrich, R. Scherer, and C. Neuper, “The effect of distinct mental strategies on classification performance for brain-computer interfaces,” International Journal of Psychophysiology, vol. 84, no. 1, pp. 86–94, 2012. View at Publisher · View at Google Scholar · View at Scopus
  22. R. Scherer, J. Faller, D. Balderas et al., “Brain-computer interfacing: more than the sum of its parts,” Soft Computing, vol. 17, no. 2, pp. 317–331, 2013. View at Publisher · View at Google Scholar · View at Scopus
  23. E. V. C. Friedrich, R. Scherer, and C. Neuper, “Long-term evaluation of a 4-class imagery-based brain-computer interface,” Clinical Neurophysiology, vol. 124, no. 5, pp. 916–927, 2013. View at Publisher · View at Google Scholar · View at Scopus
  24. E. V. C. Friedrich, R. Scherer, and C. Neuper, “Stability of event-related (de-) synchronization during brain-computer interface-relevant mental tasks,” Clinical Neurophysiology, vol. 124, no. 1, pp. 61–69, 2013. View at Publisher · View at Google Scholar · View at Scopus
  25. E. V. C. Friedrich, C. Neuper, and R. Scherer, “Whatever works: a systematic user-centered training protocol to optimize brain-computer interfacing individually,” PLoS ONE, vol. 8, no. 9, Article ID e76214, 2013. View at Publisher · View at Google Scholar · View at Scopus
  26. Y. Hoshi and M. Tamura, “Near-infrared optical detection of sequential brain activation in the prefrontal cortex during mental tasks,” NeuroImage, vol. 5, no. 4, pp. 292–297, 1997. View at Publisher · View at Google Scholar · View at Scopus
  27. B. Abibullaev, J. An, and J.-I. Moon, “Neural network classification of brain hemodynamic responses from four mental tasks,” International Journal of Optomechatronics, vol. 5, no. 4, pp. 340–359, 2011. View at Publisher · View at Google Scholar · View at Scopus
  28. S. D. Power, A. Kushki, and T. Chau, “Towards a system-paced near-infrared spectroscopy brain-computer interface: differentiating prefrontal activity due to mental arithmetic and mental singing from the no-control state,” Journal of Neural Engineering, vol. 8, no. 6, Article ID 066004, 2011. View at Publisher · View at Google Scholar · View at Scopus
  29. T. H. Falk, M. Guirgis, S. Power, and T. T. Chau, “Taking NIRS-BCIs outside the lab: towards achieving robustness against environment noise,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 19, no. 2, pp. 136–146, 2011. View at Publisher · View at Google Scholar · View at Scopus
  30. S. D. Power, A. Kushki, and T. Chau, “Intersession consistency of single-trial classification of the prefrontal response to mental arithmetic and the no-control state by NIRS,” PLoS ONE, vol. 7, no. 7, Article ID e37791, 2012. View at Publisher · View at Google Scholar · View at Scopus
  31. C. Herff, D. Heger, F. Putze, J. Hennrich, O. Fortmann, and T. Schultz, “Classification of mental tasks in the prefrontal cortex using fNIRS,” in Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, (EMBC '13), pp. 2160–2163, July 2013. View at Publisher · View at Google Scholar · View at Scopus
  32. L. C. Schudlo, S. D. Power, and T. Chau, “Dynamic topographical pattern classification of multichannel prefrontal NIRS signals,” Journal of Neural Engineering, vol. 10, no. 4, Article ID 046018, 2013. View at Publisher · View at Google Scholar · View at Scopus
  33. H.-J. Hwang, J.-H. Lim, D.-W. Kim, and C.-H. Im, “Evaluation of various mental task combinations for near-infrared spectroscopy-based brain-computer interfaces,” Journal of Biomedical Optics, vol. 19, no. 7, Article ID 077005, 2014. View at Publisher · View at Google Scholar · View at Scopus
  34. L. C. Schudlo and T. Chau, “Single-trial classification of near-infrared spectroscopy signals arising from multiple cortical regions,” Behavioural Brain Research, vol. 290, pp. 131–142, 2015. View at Publisher · View at Google Scholar · View at Scopus
  35. A. Myrden, A. Kushki, E. Sejdić, and T. Chau, “Towards increased data transmission rate for a three-class metabolic brain-computer interface based on transcranial Doppler ultrasound,” Neuroscience Letters, vol. 528, no. 2, pp. 99–103, 2012. View at Publisher · View at Google Scholar · View at Scopus
  36. I. Aleem and T. Chau, “Towards a hemodynamic BCI using transcranial Doppler without user-specific training data,” Journal of Neural Engineering, vol. 10, no. 1, Article ID 016005, 2013. View at Publisher · View at Google Scholar · View at Scopus
  37. A. Faress and T. Chau, “Towards a multimodal brain-computer interface: combining fNIRS and fTCD measurements to enable higher classification accuracy,” NeuroImage, vol. 77, pp. 186–194, 2013. View at Publisher · View at Google Scholar · View at Scopus
  38. N. E. Nawa and H. Ando, “Classification of self-driven mental tasks from whole-brain activity patterns,” PLoS ONE, vol. 9, no. 5, Article ID e97296, 2014. View at Publisher · View at Google Scholar · View at Scopus
  39. E. Hortal, D. Planelles, A. Costa et al., “SVM-based brain-machine interface for controlling a robot arm through four mental tasks,” Neurocomputing, vol. 151, no. 1, pp. 116–121, 2015. View at Publisher · View at Google Scholar · View at Scopus
  40. S. Fazli, J. Mehnert, J. Steinbrink et al., “Enhanced performance by a hybrid NIRS-EEG brain computer interface,” NeuroImage, vol. 59, no. 1, pp. 519–529, 2012. View at Publisher · View at Google Scholar · View at Scopus
  41. M. J. Khan, M. J. Hong, and K.-S. Hong, “Decoding of four movement directions using hybrid NIRS-EEG brain-computer interface,” Frontiers in Human Neuroscience, vol. 8, no. 1, article 244, 2014. View at Publisher · View at Google Scholar · View at Scopus
  42. B. Koo, H.-G. Lee, Y. Nam et al., “A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery,” Journal of Neuroscience Methods, vol. 244, no. 1, pp. 26–32, 2015. View at Publisher · View at Google Scholar · View at Scopus
  43. R. Zimmermann, L. Marchal-Crespo, J. Edelmann et al., “Detection of motor execution using a hybrid fNIRS-biosignal BCI: a feasibility study,” Journal of NeuroEngineering and Rehabilitation, vol. 10, no. 1, article 4, 2013. View at Publisher · View at Google Scholar · View at Scopus
  44. H. Banville, HyBrid Brain-Computer Interfaces: Improving Mental Task Classification Performance through Fusion of Neurophysiological Modalities [Master, thesis], Hybrid Brain-Computer Interfaces, 2015.
  45. S. G. Hart and L. E. Staveland, “Development of NASA-TLX (Task Load Index): results of empirical and theoretical research,” Advances in Psychology, vol. 52, pp. 139–183, 1988. View at Publisher · View at Google Scholar · View at Scopus
  46. J. Cegarra and N. Morgado, “Étude des propriétés de la version francophone du NASATLX,” Communication présentée à la cinquième édition du colloque de psychologie ergonomique (Epique) , 2009.
  47. A. Delorme and S. Makeig, “EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis,” Journal of Neuroscience Methods, vol. 134, no. 1, pp. 9–21, 2004. View at Publisher · View at Google Scholar · View at Scopus
  48. A. J. Bell and T. J. Sejnowski, “An information-maximization approach to blind separation and blind deconvolution,” Neural Computation, vol. 7, no. 6, pp. 1129–1159, 1995. View at Publisher · View at Google Scholar · View at Scopus
  49. A. Mognon, J. Jovicich, L. Bruzzone, and M. Buiatti, “ADJUST: an automatic EEG artifact detector based on the joint use of spatial and temporal features,” Psychophysiology, vol. 48, no. 2, pp. 229–240, 2011. View at Publisher · View at Google Scholar · View at Scopus
  50. G. Pfurtscheller and F. H. L. da Silva, “Event-related EEG/MEG synchronization and desynchronization: basic principles,” Clinical Neurophysiology, vol. 110, no. 11, pp. 1842–1857, 1999. View at Publisher · View at Google Scholar · View at Scopus
  51. J. Kalcher and G. Pfurtscheller, “Discrimination between phase-locked and non-phase-locked event-related EEG activity,” Electroencephalography and Clinical Neurophysiology, vol. 94, no. 5, pp. 381–384, 1995. View at Publisher · View at Google Scholar · View at Scopus
  52. G. Pfurtscheller, “Functional brain imaging based on ERD/ERS,” Vision Research, vol. 41, no. 10-11, pp. 1257–1260, 2001. View at Publisher · View at Google Scholar · View at Scopus
  53. T. J. Huppert, S. G. Diamond, M. A. Franceschini, and D. A. Boas, “HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain,” Applied Optics, vol. 48, no. 10, pp. D280–D298, 2009. View at Publisher · View at Google Scholar · View at Scopus
  54. M. Cope, D. T. Delpy, E. O. Reynolds, S. Wray, J. Wyatt, and P. van der Zee, “Methods of Quantitating Cerebral Near Infrared Spectroscopy Data,” in Oxygen Transport to Tissue X, vol. 215 of Advances in Experimental Medicine and Biology, pp. 183–189, Springer, 1988. View at Publisher · View at Google Scholar
  55. I. Guyon and A. Elisseeff, “An introduction to variable and feature selection,” Journal of Machine Learning Research, vol. 3, pp. 1157–1182, 2003. View at Google Scholar · View at Scopus
  56. M. L. Braun, J. M. Buhmann, and K.-R. Müller, “On relevant dimensions in kernel feature spaces,” Journal of Machine Learning Research, vol. 9, pp. 1875–1908, 2008. View at Google Scholar · View at Scopus
  57. S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, pp. 65–70, 1979. View at Google Scholar
  58. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, “Gene selection for cancer classification using support vector machines,” Machine Learning, vol. 46, no. 1–3, pp. 389–422, 2002. View at Publisher · View at Google Scholar · View at Scopus
  59. R. Gupta, H. J. Banville, and T. H. Falk, “PhySyQX: a database for physiological evaluation of synthesised speech quality-of-experience,” in Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2015, October 2015. View at Publisher · View at Google Scholar · View at Scopus
  60. J. M. Zacks, “Neuroimaging studies of mental rotation: A meta-analysis and review,” Journal of Cognitive Neuroscience, vol. 20, no. 1, pp. 1–19, 2008. View at Publisher · View at Google Scholar · View at Scopus
  61. C. J. Price, “The anatomy of language: a review of 100 fMRI studies published in 2009,” Annals of the New York Academy of Sciences, vol. 1191, pp. 62–88, 2010. View at Publisher · View at Google Scholar · View at Scopus
  62. A. Ischebeck, L. Zamarian, C. Siedentopf et al., “How specifically do we learn? Imaging the learning of multiplication and subtraction,” NeuroImage, vol. 30, no. 4, pp. 1365–1375, 2006. View at Publisher · View at Google Scholar · View at Scopus
  63. T. Fehr, C. Code, and M. Herrmann, “Common brain regions underlying different arithmetic operations as revealed by conjunct fMRI-BOLD activation,” Brain Research, vol. 1172, no. 1, pp. 93–102, 2007. View at Publisher · View at Google Scholar · View at Scopus
  64. C. E. Curtis and M. D'Esposito, “Persistent activity in the prefrontal cortex during working memory,” Trends in Cognitive Sciences, vol. 7, no. 9, pp. 415–423, 2003. View at Publisher · View at Google Scholar · View at Scopus
  65. B. Kleber, N. Birbaumer, R. Veit, T. Trevorrow, and M. Lotze, “Overt and imagined singing of an Italian aria,” NeuroImage, vol. 36, no. 3, pp. 889–900, 2007. View at Publisher · View at Google Scholar · View at Scopus
  66. A. Gunji, R. Ishii, W. Chau, R. Kakigi, and C. Pantev, “Rhythmic brain activities related to singing in humans,” NeuroImage, vol. 34, no. 1, pp. 426–434, 2007. View at Publisher · View at Google Scholar · View at Scopus
  67. T. Ino, Y. Inoue, M. Kage, S. Hirose, T. Kimura, and H. Fukuyama, “Mental navigation in humans is processed in the anterior bank of the parieto-occipital sulcus,” Neuroscience Letters, vol. 322, no. 3, pp. 182–186, 2002. View at Publisher · View at Google Scholar · View at Scopus
  68. G. Pfurtscheller and C. Neuper, “Motor imagery activates primary sensorimotor area in humans,” Neuroscience Letters, vol. 239, no. 2-3, pp. 65–68, 1997. View at Publisher · View at Google Scholar · View at Scopus
  69. G. Pfurtscheller, C. Brunner, A. Schlögl, and F. H. Lopes da Silva, “Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks,” NeuroImage, vol. 31, no. 1, pp. 153–159, 2006. View at Publisher · View at Google Scholar · View at Scopus
  70. A. Ishai, J. V. Haxby, and L. G. Ungerleider, “Visual imagery of famous faces: effects of memory and attention revealed by fMRI,” NeuroImage, vol. 17, no. 4, pp. 1729–1741, 2002. View at Publisher · View at Google Scholar · View at Scopus