Computational Intelligence and Neuroscience

Volume 2016, Article ID 6734720, 12 pages

http://dx.doi.org/10.1155/2016/6734720

## A Novel Fixed Low-Rank Constrained EEG Spatial Filter Estimation with Application to Movie-Induced Emotion Recognition

Department of Dynamic Brain Imaging, Cognitive Mechanisms Laboratories, Advanced Telecommunications Research Institute International, 2-2-2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0288, Japan

Received 28 April 2016; Revised 15 June 2016; Accepted 19 June 2016

Academic Editor: Victor H. C. de Albuquerque

Copyright © 2016 Ken Yano and Takayuki Suyama. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This paper proposes a novel fixed low-rank spatial filter estimation for brain computer interface (BCI) systems with an application that recognizes emotions elicited by movies. The proposed approach unifies such tasks as feature extraction, feature selection, and classification, which are often independently tackled in a “bottom-up” manner, under a regularized loss minimization problem. The loss function is explicitly derived from the conventional BCI approach and solves its minimization by optimization with a nonconvex fixed low-rank constraint. For evaluation, an experiment was conducted to induce emotions by movies for dozens of young adult subjects and estimated the emotional states using the proposed method. The advantage of the proposed method is that it combines feature selection, feature extraction, and classification into a monolithic optimization problem with a fixed low-rank regularization, which implicitly estimates optimal spatial filters. The proposed method shows competitive performance against the best CSP-based alternatives.

#### 1. Introduction

Brain computer interfaces (BCIs) are a rapidly growing field of research that combines neurophysiological insights, statistical signal analysis, and machine learning. BCIs are generally designed based on a pattern recognition approach, that is, extracting features from EEG signals and using a classifier to identify the user’s mental state from such features [1]. Those sequential approaches are called “bottom-up” schemes; given a large collection of single-trial EEG data, better representations of the data are extracted to finally obtain the classification output at the top. In contrast, discriminative or “top-down” approaches focus on predicting user intentions and are based on two criteria: the empirical prediction performance and the regularizer. Suitably chosen regularizers automatically induce sparse decomposition of the signal, which corresponds to conventional feature extraction [2].

This paper proposes a discriminative method using a low-rank regularizer to estimate spatial filters for extracting effective features under a study. The advantage of the proposed method is that it combines feature selection, feature extraction, and classification into a monolithic optimization problem with a low-rank regularization, because this approach includes spatial filter estimation in the optimization framework of statistical inference model. Under a suitable chosen regularizer, it induces the best inference model, which implicitly estimates optimal spatial filters under the regularization assumption.

Emotion classification from EEG data has attracted much attention recently [3, 4]. Emotion also plays an important role in human-human communication and interaction. The ability to recognize the emotional states of people is an important part of natural communication. This field of research is still relatively new, and there is still much to be done to improve on existing elements in BCI but also to discover new possibilities.

For evaluation of the proposed methods, experiments were conducted to induce emotions by movies for dozens of young adult subjects and estimated the emotional states using the proposed method. The results were compared with conventional methods using a common spatial pattern (CSP).

This paper’s contribution is the proposal and the explicit derivation of the fixed low-rank constrained discriminative approach and its application to emotion recognition with comparative analysis with conventional methods. This paper is organized as follows. Section 2 describes the background of emotion recognition from EEGs, and Section 3 describes the proposed method. Section 4 presents the data acquisition and experimental protocol. Section 5 describes the results and discussion. Section 6 concludes the paper.

#### 2. Background

##### 2.1. Emotion in the Brain

The limbic system which is like a cortical ring around the brain stem is responsible for initial emotional interpretation of the signals from the autonomic nervous system. The hypothalamus is responsible for processing the incoming signals and triggering the corresponding visceral physiological effects like a raised heart rate or galvanic skin response.

From the hypothalamus the stimuli information is passed on to the amygdala, which is important for learning to connect stimuli to emotional reactions (reward/fear) and for evaluating new stimuli by comparing them to past experience.

The amygdala is considered vital for emotion processing. However, since it is an underlying structure like the rest of the limbic system, it cannot be detected directly in recording from the scalp. The amygdala is connected to the temporal and prefrontal cortices, which is thought to be the way visceral sensations are interpreted cognitively, resulting in a consciously experienced feeling of an emotion [5].

The temporal lobe is essential for hearing, language, and emotion and also plays an important role in memory. The prefrontal lobe is involved in the so-called highest level of functioning. It is responsible for cognitive, emotional, and motivational processes. The prefrontal lobe is part of the frontal cortex, which is said to be the emotional control center and to even determine personality. It is involved in, among others, judgment and social behavior. These functions are very much based on the experience of emotions.

##### 2.2. Valence: Hemispherical Asymmetry

Psychophysiological research has shown the importance of the difference in activation between the two cortical hemispheres in the reaction that subjects show toward stimuli. Left frontal inactivation is an indicator of a withdrawal response, which is often linked to a negative emotion. On the other hand, right frontal inactivation is a sign of an approach response, or positive emotion.

Harmon-Jones [6] suggests that the hemispherical differences are not an indication of affective valence, but of motivational direction (approach or withdrawal behavior to the stimulus). Affective valence does seem tightly linked to motivational direction. Therefore, the hemispherical asymmetry patterns do indicate the affective valence.

Davidson and Fox [7] found that 10-month-old infants exhibited increased left frontal activation in response to a film clip of an actress generating a happy facial expression as compared to a sad facial expression. Frontal cortical activity has been found to relate to facial expressions of positive and negative emotions as well.

#### 3. Method

##### 3.1. General Framework

Given a short high-pass filtered EEG segment, , where is the number of channels and is the number of time points, the data are first band-pass filtered at a band range being studied. A commonly used form of a second-order or power oscillation-based linear model can be written as follows:

Here, is the spatial filters, are the weighting coefficients of the features, and is a bias term. The classifier first projects the signal by spatial filters. Next, it takes a logarithm of the power of the projected signal. Finally it linearly combines these dimensional features and adds bias.

To determine spatial filters , CSP is often used [1]. Many variants of the original CSP have been proposed. [8]. Coefficients and are determined statistically from the training examples, that is, the pairs of trials and labels collected in the calibration phase. Label corresponds to the binary classes being studied.

To briefly summarize CSP to compute spatial filter , it is obtained by extremizing the following function:where is the spatial covariance matrix of the EEG signals from class as follows:where we assume a zero mean for the EEG signal.

Since remains unchanged if is rescaled, extremizing is equivalent to extremizing subject to the constraint . Using the Lagrange multiplier method, this constrained optimization problem amounts to extremizing the following function:The spatial filter extremizing is such that the derivative of with respect to equals 0:

The spatial filters are the eigenvectors of which correspond to its largest and lowest eigenvalues.

##### 3.2. Proposed Model Calibration

If we ignore the logarithm in (1), it can be reformulated as follows:where and is the covariance matrix of . Finally we obtain

Note that is the elementwise inner product of the two matrices. To determine parameters , logistic regression was employed with low-rank regularization of . This amounts to solving the following optimization problem with training examples:where is the th singular value of and is the rank constraint of . The first term is convex. But since the low-rank constraint term is nonconvex, it is not guaranteed to find the optimal point. To solve this problem, the alternating direction method of multipliers (ADMM) [9] is employed with a hope that it has better convergence properties than other local optimization methods. For nonconvex problems, depending on the initial values, the solution can converge to different points.

The optimization problem is rephrased as follows:where is the set of matrices with rank . To solve it by ADMM, it can be rewritten as follows:where is the indicator function of . The augmented Lagrangian (using the scaled dual variable) iswhere is called the penalty parameter. So the iterative optimization of ADMM for this problem iswhere is the projection onto . Hence, is determined by carrying out a singular value decomposition, , and keeping the top singular values; that is,

Here we can initialize and as zero w.l.o.g. The primal and the dual residuals at iteration are defined as follows:These residuals converge to zero as ADMM proceeds.

##### 3.3. Multiple Frequency Bands

The proposed method can be extended for estimating the spatial filters for multiple frequency bands. Let be the band-pass filtered data by filtering operator . The covariance matrix of the signal denoted as is obtained separately for each frequency pass band. Then align them as a large block diagonal matrix (14). To obtain the spatial filters for multiple bands, this block diagonal matrix is substituted for in (7). The solution is expected to effectively select the optimal spatial features from multiple frequency bands:

##### 3.4. Merits of the Proposed Method

CSP estimates spatial filters based on a criterion that corresponding components produce minimum variance for one condition and maximum variance for the other and thus increase discriminative ability. However because the spatial filter estimation is decoupled from the inference model, such as logistic regression, optimal filters can only be predicted by using cross-validation of the inference model and select the one which produces the best empirical inference performance.

On the other hand, our proposed model derived from “top-down” approach incorporates spatial filter estimation in the predictive model. Hence by focusing on the prediction performance with suitably chosen regularizer, such as fixed low-rank in our model, it induces sparse decomposition of the signal which corresponds to conventional feature extraction. Hence, it implicitly estimates optimal spatial filters of the best inference model under the assumption.

#### 4. Emotion Recognition

To predict the state of emotion experienced by the participants from single EEG segments, a predictive model was employed that estimates from a given short EEG segment (here 5 sec) the probability that the participant experienced positive or negative emotions during that period. For the evaluation, fivefold cross-validation is performed by holding out one-session dataset for the test and the remaining four-session datasets with labels were used to estimate parameters . For each round, the held-out dataset was used for tests to evaluate the classification error rate. In each round, the classification error rate is computed as the ratio of the number of correctly classified EEG segments divided by the total number of EEG segments in the trial.

##### 4.1. Data Acquisition

Twenty-three healthy adult volunteers participated under the informed consent that was approved by the ethical committee of ATR. Among them, ten subjects (males = 3, females = 7, age = ) were selected for analysis. The EEGs were recorded from 32 gel-based scalp electrodes, as shown in Figure 1, and four EOG placements around the eyes using an eego amplifier (ANT Neuro, Enschede, Netherlands) with 24-bit resolution. The EEGs were sampled at 512 Hz. The protocol of the EEG experiment is described in Figure 2. To elicit emotions, a set of movie clips that were used in Samson et al. [10] was used. The movie clip set includes four classes of different target emotional states: positive, negative, neutral, and mixed. The average length of each clip was about 20 seconds. For each trial, to elicit emotions, four randomly selected movie clips of the same emotional class were played continuously without intervals and followed by self-assessment questions. One session consisted of four trials of four different movie classes. Before each trial, a random color grating pattern was displayed for 90 seconds to wash out the emotional states of the participant. The entire experiment consisted of seven sessions. For the analysis, however, only the first five sessions were used because, during the last two sessions, most participants appeared fatigued or drowsy.