Abstract
Emotion plays a crucial role in understanding each other under natural communication in daily life. Electroencephalogram (EEG), based on emotion classification, has been widely utilized in the fields of interdisciplinary studies because of emotion representation’s objectiveness. In this paper, it aimed to introduce the Korean continuous emotional database and investigate brain activity during emotional processing. Moreover, we selected emotion-related channels for verifying the generated database using the Support Vector Machine (SVM). First, we recorded EEG signals, collected from 28 subjects, to investigate the brain activity across brain areas while watching movie clips by five emotions (anger, excitement, fear, sadness, and happiness) and a neutral state. We analyzed EEG raw signals to investigate the emotion-related brain area and select suitable emotion-related channels using spectral power across frequency bands, i.e., alpha and beta bands. As a result, we select the eight-channel set, namely, AF3-AF4, F3-F4, F7-F8, and P7-P8, from statistical and brain topography analysis. We perform the classification using SVM and achieve the best accuracy of 94.27% when utilizing the selected channels set with five emotions. In conclusion, we provide a fundamental emotional database reflecting Korean feelings and the evidence of different emotions for application to broaden area.
1. Introduction
Emotion is the most important human factor and plays a vital role in daily life. Meanwhile, it provides diverse information about the human experience. Similarly, emotional interaction between humans and machines has been increasingly focused on one of the critical issues accompanied by the recent development of artificial intelligence systems. For a successful emotion classification of humans, it is the most critical step to develop an emotion classification system that guarantees classification accuracy, robustness against artifacts, and practical application scenarios [1].
In the last decades, many researchers have attempted to investigate physiological signals such as electroencephalogram (EEG), electrocardiogram (ECG), and skin temperature (SKT) for emotion classification. The physiological signals provide more objective and appropriate information for representing the emotional states than behavioral responses such as the face and voice [2]. Among these physiological signals, EEG has more various benefits, for example, easy use, a high time resolution, and direct measurement compared to other signals [3, 4]. For this reason, many researchers prefer EEG signals to the emotion classification usage to achieve the reliable outputs in responses to emotional states [5].
However, since multichannels or even more channels generally record EEG raw signals and include many noises (i.e., body movement, muscle activity, and electrical power line), it is required to conduct the more sophisticated preprocessing and artifact removals. If raw EEG signals included many artifacts or are of low quality (e.g., noisy), we cannot achieve more stable results. To overcome these issues, it needs to implement good quality EEG signals throughout the sophisticated preprocessing steps ahead of emotion-related EEG feature extraction. As final outputs, we achieve reliable signals from raw EEG signals.
There are several EEG emotion databases, i.e., Database for Emotion Analysis (DEAP) [6] and SJTU Emotion EEG Dataset (SEED) [7], are publicly available for researchers. They are used to be applied for emotion categorizations based on emotional theory, such as Ekman’s discrete model [8] and Russell’s circumplex model [9]. Also, these databases have used emotional stimuli consisting of their mother tongue, English and Chinese. The social factors, i.e., languages and cultures, assisted in understanding the situated emotion and inducing more proper emotions when inducing emotional stimuli. In other words, when the emotion is elicited by humans using different languages, it is not enough to understand the appropriate feelings from humans in case of misunderstanding about the emotional stimuli from individuals. To collect the higher-quality emotion database, they are given to subjects to properly understand emotional situations reflected by specific social and cultural characteristics.
In this paper, we introduce the EEG emotion database using Korean movie clips and verify the generated database. Toward this purpose, we record EEG signals by subjects under the emotional stimulus, which is selected by watching Korean movie clips. The generated database has under a series of preprocessing steps to feature extraction for verification database. The preprocessed EEG data extracts the features corresponding to emotions and investigates the emotion-related channels set, examining where the brain activates related to emotional processing with statistical analysis. Finally, we conduct the emotion classification using a Support Vector Machine (SVM) and evaluate the classification accuracy for verification of the generated database. The main contributions are as follows:(i)We collect the continuous EEG-based emotional database, reproduced using Korean movie clips.(ii)We conduct the verification of generated database using the machine learning approach, Support Vector Machine (SVM).
2. Related Works
2.1. Emotion Theory
Many different emotional states have been defined, ranging from basic emotions to combining social emotions. Such categorization is generally divided into two approaches, i.e., discrete and dimensional approaches [8, 9].
One approach is to organize primary emotions (e.g., fear, sadness, and happiness) to match the environment’s unique parameters, physiology, and behavior. Ekman’s theory has been very influential with his studies of facial expressions as a discrete model [8]. He suggested emotion theory and defined six basic emotions: sadness, happiness, disgust, anger, fear, and surprise.
Another approach, the dimension model, described adequate space with a limited number of underlying dimensions designed by Russell [9]. He introduced the circumplex model that is divided according to two-dimensional space. In other words, he discovered that emotional state could be represented by valence and arousal level. Valence ranged from negative (or unpleasant) to positive (or pleasant) to explain emotions quantitatively. If the valence is directed to a high level, it means a pleasant score, defined by positive characteristics (e.g., pleasantness, happiness, and elation). But, the low level states a reverse score (e.g., unpleasantness, sadness, and stress) defined by negative characteristics. Arousal ranged from calm (or inactive) to active (or excited) to describe emotion measured as an intensity. For example, the activation (excitement, anger) is directed to high-level arousal; on the contrary, a low score means inactivity (boredom, sadness).
2.2. Electroencephalogram (EEG)
Electroencephalogram (EEG) is a measurement of the electrical activity generated by the brain. The EEG signals are recorded simultaneously by placing multiple channels on the scalp sites. EEG signals are beneficial tools, have high temporal resolution, are easy to use, and are not invasive. When EEG is recording, the international channel 10–20 system is usually recommended as a standard layout [10].
EEG signals can be generally represented as a distribution of signal power across the different frequency spectrum. Brain neural activity has been linked to various physiological states and psychological functions in different frequency bands [11]. The frequency bands are generally subdivided into several bands as delta, theta, alpha, beta, and gamma based on the neural activity. Table 1 shows EEG signals in terms of frequency bands. Delta (1–4 Hz), slow wave, is associated with deep sleep, unconscious condition. Theta (4–8 Hz) has represented the sleeping and dreaming states, alpha (8–13 Hz) is associated with relaxation and not yet awareness, beta (13–30 Hz) is related to the alert, thinking, and active state of mind, and gamma (above 30 Hz) shows the rhythms for hyper brain activity [11].
According to researchers, it is different from a few Hertz, the beginning and the end of the bands. Also, most studies are focused on the alpha and beta frequency bands concerning emotion processing. It is well known that the alpha band reflects attention processing, and the beta band reflects emotional and cognitive processing in the brain corresponding to the previous studies [12, 13].
2.3. Emotion-Related EEG Channels
When it comes to emotions, it turns out that there are regions in the brain that are associated with each of the main emotions. For example, happiness activates several areas of the brain, including the right frontal cortex. Fear activates the areas of the left frontal cortex. Sadness is associated with increased activity of the right occipital lobe [14]. This means that emotions are actually experiences that are associated with activation of certain regions in the brain. In addition, when recording EEG signals, the multiple channels are generally recorded as two or more over 256 channels. The channels are located throughout the brain region; especially most of the channels are placed at the frontal lobe, which is related to emotional processing [15]. Since human emotional activities are mainly concentrated in the frontal areas, many researchers are focused on the frontal lobe to find the critical channels for EEG-based emotion classification [16–20]. They founded that frontal EEG channels reveal the most prominent emotional processing in brain signals. It usually demonstrates the balance of brain activation in the left and right frontal areas in emotion processing and is considered a reliable indicator of affective, emotional representation [17].
In previous studies, most of the studies were adopted emotional processing following Davidson’s model [16]. Davidson’s model described that emotional processing is associated with the asymmetry between the left and right frontal or prefrontal lobes expressed in an alpha frequency band, i.e., 8–12 Hz. Some studies also revealed that the left frontal brain areas are involved in positive emotions, whereas the right frontal regions are involved in negative emotions [18–20]. Coan et al. [18] proposed that alpha (8–13 Hz) asymmetry in the frontal lobe is one of the well-documented measures of emotional responses. They also revealed that the increase in the frontal lobe activity was higher than that in other brain regions.
Similarly, they reported higher frequency bands (alpha, beta) to significantly affect the lower frequency bands (delta and theta) in emotion classification. Zhao et al. [19] investigated brain activities in four emotions (amusement, tenderness, anger, and fear) while watching emotional films. They represented F3-F4 pairs in the frontal lobe that detected alpha (8–13 Hz) between the left and right areas, which is informative in valence predictions. The theta (4–8 Hz) is related to arousal prediction in F3-F4 channels. Furthermore, Bos [20] proposed that F3-F4 pairs are the most suitable for emotion recognition regarding valence and arousal level. In other words, The Fpz is the best channel for detecting valence level at an alpha band (8–12 Hz), and F3 and F4 channels are the best channels for detecting arousal level at a beta band (12–30 Hz). In summary, the frontal lobe played an essential role in emotion classification and primarily demonstrated alpha and beta frequency bands during emotional processing.
2.4. Emotion-Related EEG Classification
Recently, many researchers have tried to classify emotion using machine learning approaches such as K-Nearest Neighbor (KNN) Algorithm, Support Vector Machine (SVM), and Artificial Neural Network (ANN). Among them, the SVM is the more preferable than other classifiers for its effectiveness and better accuracy, and then, it is widespread, utilized for emotion classification [21–25].
Zheng et al. [21] conducted emotion recognition using differential entropy (DE) features with three emotions (positive, neutral, and negative). They introduced four types with 4 channels (FT7, FT8, T7, and T8), 6 channels (FT7, FT8, T7, T8, TP7, and TP8), 9 channels (FP1, FPZ, FP2, FT7, FT8, T7, T8, TP7, and TP8), and 12 channels (FT7, FT8, T7, T8, C5, C6, TP7, TP8, CP5, CP6, P7, and P8), which achieved classification accuracies of 82.88%, 85.03%, 84.02%, and 86.65%, respectively, using an SVM classifier. As a result, the best accuracy is 86.65% using 12 channels, but they proposed the four channels because of a more stable result of 82.88%.
According to Mohammadi et al. [22], they presented that F1 and F2 channels in the frontal lobe are related to emotion classification for valence and arousal states. They conducted the classification using four emotional categories (low arousal/low valence, low arousal/high valence, high arousal/low valence, and high arousal/high valence) with extracted discrete wavelet transformation (DWT). As a result, they achieved a maximum accuracy of 80.68% from valence, using only two frontal channels (Fp1 and Fp2). Wang et al. [24] proposed the optimal EEG channels using valence and arousal to evaluate emotion classification performance. They used a spectrogram with normalized mutual information (NMI) as feature selection. They achieved an average accuracy of 74.41% for valence using 10 channels (AF3, F7, FC5, P7, Pz, O2, P4, FP2, FC6, and P3) and 73.64% for arousal using 7 channels (FC1, P3, Pz, Oz, CP2, C4, F4, and Fz) using SVM classifier.
In addition, some studies presented emotion classification using more subdivided emotional categories, i.e., anger, happiness, sadness, and others. Yousaf et al. [25] performed emotion classification with four emotions (pleasant, sad, happy, and frustrated). They utilized Higuchi’s Fractal Dimension as features extraction with SVM. They compared the classification accuracies using 3 (AF3, F4, and FC6) and 8 (Fp1, Fp3, F3, F4, T3, T4, P3, and P4) channels. As a result, the accuracy using 3 channels is 59.17%, and that using 8 channels is 87.62% in the arousal level. For the valence level, the accuracy is 68.39% for 3 channels and 83.28% for 8 channels. Also, Valenzi et al. [23] performed emotion classification with four emotions (amused, disgusted, sad, and neutral). As the spectral power from five frequency bands, i.e., delta, theta, alpha, beta, and gamma, the feature is extracted by the method of Short-Time Fourier Transform (STFT) from 8 channels (AF3, AF4, F3, F4, F7, F8, T7, and T8). They achieved the best accuracy of 87.5%. However, most of the studies mentioned above selected the channels as being randomly chosen or equally used to follow the previous studies [26, 27].
Thus, the researchers performed EEG-based emotion classification with machine learning methods using all or partly selected channels. Also, they did not consider brain activity and then select channels randomly. For example, the occipital lobe is related to the brain’s visual role, not related to emotional processing. Moreover, the emotion categories were used from two to four emotions in the previous studies. Since emotions have different characteristics under valence and arousal level, the categories should be more subdivided, corresponding to valence and arousal. For example, anger and sadness have equal to the arousal level, but valence is different, and behavioral expression and feeling are also different responses by humans. Therefore, it needs emotional categories to be divided into more detailed categories for more proper emotion classification based on Russell [9] and Ekman et al. [8] emotion model.
3. The Generated Korean EEG Emotional Database
3.1. The Selection of Emotional Stimuli
We selected 60 movies from the Korean movie list as the emotional stimuli released for the last decade by the Korea Film Council (KOFIC). All movies are also the grade as G-rated (general rated) and excluded as R-rated. Also, the movies are excluded from the specific scenes containing many hateful scenes, i.e., stabbing a person with a knife and a set suddenly surprised. The survey is conducted two times using the paper questionnaire. The two hundred participants conducted the survey (first survey: one hundred sixty participants, second survey: forty participants, age range from 15 to 50). The detailed information about the survey is shown in Table 2.
In the first survey, 160 participants were asked to evaluate movie’s feelings, which are filled out only when they have never seen it before. The evaluation criteria are used by the self-assessment manikin (SAM) [28, 29] (see Figure 1).
(a)
(b)
In the second survey, 40 participants who did not participate in the first survey are requested to fill out a questionnaire after watching the movie clips. The 24 movie clips are prepared for 4 minutes for rating using the dimension model according to the first survey. For eliciting the participant’s emotional states, the movie clips are selected based on the following criteria; (1) the movie clip should be avoided to inducing multiple emotions, (2) the movie clip should be understood by participants without any explanation, and (3) the movie clip should be elicited a single targeted emotion.
3.2. Analysis of Self-Assessment Manikin (SAM)
All subjects are requested to evaluate emotional stimulus that match to target emotion or not at the end of each trial in the experiment using the basic emotion category and SAM.
Table 3 presented the mean values (standard deviation) of the self-rating by dimension model (the scale ranges from 1 to 9). According to self-rating results, happiness is the highest valence level (M = 6.9, SD = 1.1), which was higher than excitement (M = 5.5, SD = 1.17). In the neutral, valence level is nearly the middle of the scale (M = 4.7, SD = 1.15). The valence level increased more positive characteristics of emotion, i.e., happiness, excitement. Moreover, fear is the highest arousal level (M = 7.6, SD = 0.88), but the valence is the lowest among emotions. The emotional stimuli were successfully elicited by matched with the dimension model.
As illustrated in Figure 2, the self-rating in response to emotional stimulus is evaluated by all subjects. The average value of correct identification is achieved at 84.92% concerning emotional stimulus. The highest percentage for correct identification was a neutral state at 96.43%. The rate of correct emotional responses was 87.16% for sadness, 82.14% for excitement, 83.10% for happiness, and 84.0% for fear. The lowest emotion is the accuracy of 79.79% for anger. This result matched the emotion model by Ekman model [8]. Finally, 10 movie clips are determined as emotional stimuli as follows (Table 4);(1)Anger: The Attorney (2013), Veteran (2016)(2)Excitement: Roaring currents (2014), The Thieves (2012)(3)Fear: The train to Busan (2016, 2 clips)(4)Happiness: Sunny (2011), Speed Scandal (2008)(5)Sadness: Miracle in Cell No.7 (2012), Secretly Greatly (2013)
We also added the neutral state as a control state [30, 31]. The neutral state is shown on the black screen with a cross-mark in the center of the monitor. Finally, the ten movie clips and neutral states are utilized to record EEG for eliciting the emotion. The next section describes the collection of the EEG emotion database.
4. Data Acquisition
4.1. Ethics Statement
This study was carried out under the guidelines for the use of human subjects approved by the Institutional Review Board (IRB) at Yonsei university (Approval NO. 7001988-201807-HR-424-03). All subjects signed written informed consent before the experiment and received KRW 30,000 ($26).
4.2. Subjects
Twenty-eight healthy subjects (13 males and 15 females; age range from 20 to 35; Mean = 24.46; SD = 3.74), who were right-handed, participated in the experiments. All subjects were either graduate or undergraduate students at Yonsei University. They also have no neurological or psychiatric illness before and had normal vision or corrected to normal vision. Similarly, each participant was requested to abstain from caffeine, tobacco, and alcohol use 24 hours before the experiment.
4.3. Experimental Procedure
The experiment is performed in a quiet room at Yonsei University. We controlled the sound and light conditions in the place. The participants were requested to sit in comfortable chairs in front of the monitor. The experiment introduction is presented at the monitor. After the participants correctly understand the instructions, they pressed any key on the keyboard to move to the next step. Next, the dark screen was presented for 5 seconds, and successively the emotional movie clips were displayed on the monitor.
The experimental procedure is shown in Figure 3. It consists of two sessions. Each session contains six trials, including five emotional movie clips and one neutral state. One trial consists of the following. The first dark screen was presented for 5 seconds, followed by the emotional movie clip for 240 seconds. After watching the movie clip, the subjects are requested to complete the questionnaire about their feelings using SAM. Before the subsequent trial started, if the participant could not do relaxation or need to take a break, they had spent more time stable their conditions. It is repeatedly processed six times until the first session is finished. At the end of the first session, they take a short break and check the experimental equipment. The second session is also conducted as equal to the first session, only changing the emotional movie clips. The total experiment is around 90 minutes with break times. The emotional movie clips were presented in a random order using a useful Psycho toolbox in the behavioral experiment [32]. When the experiment finished, subjects were requested to complete the wrap-up questionnaire about writing the emotional feeling or overall experiments.
4.4. EEG Recording and Preprocessing
EEG recordings were performed on the Emotiv EPOC wireless headset (Neuro-sky Inc.) with 14 channels: AF3, AF4, F7, F8, F3, F4, FC5, FC6, T7, T8, P7, P8, O1, and O2. The channels were placed according to the international 10–20 system and used as the common ground reference (left and right mastoid). The sampling rate was 128 Hz, and the impendence of each channel was kept less than .
The recorded EEG data performed the preprocessing steps as follows: channel location, segmentation, re-reference, and filtering. First, raw EEG signals are redefined as the device channel location. The raw data is divided into 12 trials under emotional categories separately. After segmentation, each data is extracted from 60 seconds to 180 seconds for using more elicited emotional responses. The raw EEG signals changed to the average reference to normalization, which applies to mean values throughout all channels. The band-pass filter was extracted with a lower cutoff frequency of 0.1 Hz and a higher cutoff frequency of 45 Hz to eliminate artifacts, including blinks, eye movements, muscle activity, and cardiac signals.
5. EEG Data Analysis
5.1. Statistical Data Analysis and Results
For Verifying generated EEG database, we extracted the emotion-related EEG features. We conducted statistical analysis (repeated-measures ANOVA (analysis of variance)) in accordance with brain mapping topography analysis. We conducted the repeated-measures (mixed design) ANOVA using the variables: emotional states and channel sites and frequency bands. The significance level in all statistical analysis was considered as 0.05 (). It was performed using the statistical software package SPSS (25.0 version). We extracted the spectral power obtained as follows:(i)Frequency bands: alpha/beta (8–30 Hz) bands vs. all (1–45 Hz) bands(ii)EEG channels: prefrontal (AF3, AF4), frontal (F3, F4, FC5, FC6, F7, and F8), temporal (T7, T8), parietal (P7, P8), and occipital (O1, O2)
As a result, we confirmed that the correlation among the emotions × frequency bands × channels interaction effect was highly significant . These results showed that the effect reveals the differences between frequency bands and channels corresponding to different emotional states. Moreover, there were significant main effects among the frequency bands × channels interaction effect (Table 5). As a result, it was shown that frequency bands have different spectral power values from each channel by emotions.
We also confirmed each emotion with the frequency bands and channel pairs to explain frequency bands × channels interaction using the post hoc analysis. The paired t-test, as post hoc analysis, is performed to analyze the significance of channel pairs in alpha and beta frequency bands from brain areas with different emotions based on the statistical results (Table 6).
5.2. Brain Mapping Topography Analysis
In this paper, we generated EEG topo maps, and they are compared with statistical results to examine brain activity. It is presented that the brain areas are divided into different regions: Frontal (F), Temporal (T), Parietal (P), and Occipital (O) for analysis [33, 34]. After topography is generated, it needs to examine based on brain areas locating each channel on topography to track the brain activation area across different emotional processing.
As shown in Figures 4 and 5, it is the spectral power value of each frequency band corresponding to different emotional states, where the black points are the channel positions. The isobar color expresses the strength of the correlation in a given region (from strongly positive in red to strongly negative in blue).
As shown in Figure 4, the beta band is also more widely activated in the AF4 and F4 in the right frontal lobe in the anger state. The temporal lobe is activated in both frequency bands, in which P7 is more activated than P8. In an excitement state, the frontal lobe also exists in a more widely distributed brain area than the temporal lobe. It reveals the activation in the frontal lobe that is concentrated in AF3 and AF4 in the alpha frequency band, while the beta band shows activation in the only AF3. In the fear state, the frontal lobe is widely distributed when activated in the beta band. It can be explainable that the beta band is not only activated in the frontal lobe, but also more intensively activated. The alpha band shows significance in P7-P8, O1-O2, whereas the beta band shows AF3-AF4 and F7-F8 as statistical analysis.
As shown in Figure 5, both frequency bands are more activated in the left frontal, and the occipital lobe, AF3, compared with AF4 (alpha: , beta: ); especially the beta band is more activated than the alpha band in the happiness state. It is found that P7 is more activated than P8 in the temporal lobe. It is significant at alpha () and beta () bands. In the sadness state, the frontal lobe is activated in the alpha band, especially in the AF4 in the right frontal lobe. The beta band is also activated at the same site. It can be confirmed that both frequency bands are more activated in the right frontal lobe. However, the beta band is more reduced in the activation area than the alpha frequency band. However, the statistical analysis is only significant in the beta band, the AF3-AF4 is significant (), and AF4 is higher than the AF3 and P7-P8 in the alpha band.
6. Emotion-Related Channels Selection
We conducted two analysis methods for the selection of emotion-related channels. Most of the emotions, except the neutral state, are activated in the prefrontal and frontal lobe. The negative emotions, anger, fear, and sadness, are related to activation in the right frontal lobe, whereas the positive emotions, happiness and excitement, are activated in the left frontal lobe, especially the prefrontal lobe. As a result, we propose the emotion-related channels: AF3-AF4, F3-F4, F7-F8, P7-P8, F7-F8, and O1-O2. In the case of O1-O2, they are placed on the occipital lobe, which is responsible for analyzing the brain’s vision-related functions. Therefore, we excluded O1-O2 channels from further analysis. However, we adopted the P7-P8 channels located in the parietal lobe, because the parietal lobe plays a role in controlling the brain area, especially attention and cognition. Finally, we used the eight channels as the emotion indicators. In the next section, we conducted the Support Vector Machine (SVM) for verifying the generated database.
7. Emotion Classification using Support Vector Machine (SVM)
7.1. EEG Feature Extraction
For verifying the generated EEG database, we adopted spectral power: the total energy intensity of channels on a specific region at each frequency band. Spectral power is one of the most common features in defining and extracting the features of the signals [35]. When subjects watch movie clips, the spectral power changed according to emotional states and times-sequence. The spectral power is one of the representative EEG features to confirm the brain activation area during emotional processing. The higher values indicate a specific emotion that is most vigorous.
We extract the spectral power of the EEG signals from each channel ranging from 1 to 45 Hz. We conduct a Fast Fourier transform (FFT) with a half-overlapped 1 − s Hanning window. It is applied to each of the 14 channels of the EEG signals to compute the spectral time series, which were utilized the frequency bands of alpha (8–13 Hz), beta (14–30 Hz), and all (1–45 Hz) bands. Power density values were calculated, averaging spectral power within each participant’s frequency bands at each channel site. Then, a Fast Fourier Transform (FFT) was applied using the name Darbeliai toolbox.
7.2. The Generated EEG Feature Vector
The EEG signals as a feature vector are constructed into two types of frequency bands (alpha and beta vs. all bands) using spectral power. The experimental conditions are organized as follows:(i)Frequency band: alpha/beta (8–30 Hz) bands vs. all (1–45 Hz) bands(ii)Emotion category: six emotions (anger, excitement, fear, happiness, neutral, and sadness) vs. five emotions (anger, excitement, fear, happiness, and sadness)(iii)Channels: All channels (14 channels) vs. Selected channels (8 channels).
EEG signals as a feature vector have used one sample with a length of 512 samples (4 seconds) generating a step of 128 samples (1 second). Since EEG signals conducted preprocessing for artifact removal, the data size is different from 426 to 642 samples from emotions. The total samples are 3,195. The details of the sample data size are described in Table 7.
Input feature vectors are transformed into a linearly separable dimensional feature space by a mapping function to solve nonlinear problems. In this study, 5-fold cross-validation is adopted. The dataset is randomly divided into five subsets (equal or approximately).
7.3. Hyperparameter Tuning
We use the SVM classifier to classify the emotional state for proposed channels and employ the Radial Basis Function (RBF) kernel. SVM classifier also implements using sklearn with python [36].
We also implement optimal hyperparameter using a grid search. Grid search is performed to search for the best hyperparameter values and optimize the classification, which is a proper parameter setting that can improve classification accuracy rate [37]. With the RBF kernel, there are two parameters to be determined in the SVM model: (1) penalty parameter C and (2) the size of the kernel parameter gamma (default values C = 1, = 1/(number of features x variance of data, as ) as the value of gamma) [36]. In the grid search approach, pairs of are tried, and the one with the best cross-validation accuracy is chosen.
Parameter C indicates the penalty for misclassifying a data point. It is related to decision boundaries with different margins. It determines how many data samples are allowed to be placed in different classes. For smaller C values, the classifier is more allowed with misclassified data points (high bias, low variance). For greater C values, the classifier is heavily penalized for misclassified data (low bias, high variance).
Another parameter gamma determines the kernel parameter that is related to the variance of data. This parameter can be thought of as the “spread” of the kernel and, therefore, the decision region. When gamma is low, the “curve” of the decision boundary is very low, and thus, the decision region is very broad. When gamma is high, the “curve” of the decision boundary is high, creating islands of decision boundaries around data points.
In the prior study, they adjusted the parameter to improve the emotion classification; all related parameters are computed by a grid search method. Here, the parameters used a series of ranges by data. They are dependent on raw data, so, the parameter value is different than the prior study. We finally selected the optimal hyperparameter using grid search adjusted two parameters C and . Experimental results are presented in the following section.
7.4. Experimental Results
Figure 6 shows the results of the search for hyperparameter using a grid search. To improve the emotion classification, we changed the hyperparameter as the following range:(1)Parameter C: 0, 1000(2)Parameter : 0.00001, 1,
(a)
(b)
In both emotions, the best hyperparameter is using C = 100, and . In addition, the classification accuracy is similar to default value accuracy when used 0.0001.
Table 8 shows the classification performance obtained by SVM classifiers using the grid search method. The optimal hyperparameter is calculated C = 100 and using grid search.
The best accuracy is 94.72% using five emotions with alpha and beta bands, which is 3.07% higher than using all channels at 91.20%. In all conditions, using the proposed channels have shown higher emotion classification accuracy than using all channels. Also, alpha and beta frequency bands (8–30 Hz) achieved higher accuracy than all frequency bands (1–45 Hz).
Figure 7 summarizes the average confusion matrix obtained by SVM applied to different channels across frequency bands with six emotions. The best average accuracy for five emotional states was obtained for anger (99.56%), followed by excitement, happiness, sadness, fear, and neutral with an accuracy of 95.46%, 94.98%, 82.85%, 75.99%, and 67.96%, respectively. According to the results, fear and neutral have lower accuracy than other emotions relatively. The results are similar in the confusion matrix using five emotions.
(a)
(b)
8. Conclusions
In this paper, we introduce the EEG-based Korean emotion database and verify the generated database using emotion classification. We describe a novel continuous EEG-based emotional database that includes 20 subjects after watching emotional stimuli presented to elicit specific emotions. We obtained the eight channels, AF3-AF4, F3-F4, F7-F8, and P7-P8, reflected by emotion activation.
In addition, we performed Support Vector Machine (SVM) to verify the effectiveness of proposed emotion-related channels. The classification performance consists of diverse configurations using channels, frequency bands, and emotions. As a result, the best accuracy achieved was 94.27% using five emotions with alpha and beta bands. In summary, this study verifies the effectiveness of the proposed EEG emotion-related channels and introduces the Korean EEG emotion database for EEG-based emotion recognition.
Although we use only power spectrum feature in this paper, we achieve a high accuracy. We propose further research. Firstly, it needs to extend and subdivide emotions into more specific emotion categories with dimension space to improve EEG-based emotion classification. Moreover, diverse machine learning techniques and feature extraction methods on the selected emotion-related channels are expected to achieve higher accuracy. Given that EEG signals have extracted the emotion-related novelty features, deep learning methods present better representation and classification on many time-series problems when configured and trained correctly. In future directions, the accuracy might be further improved by sequential models such as Recurrent Neural Networks (RNNs) and Long-short term memory (LSTM), since EEG signals are time-series data.
In conclusion, these findings will be available to provide a benchmark database to utilize in interdisciplinary research for behavioral and emotional analyses. It also will be supporting emotion classification based on our proposed emotion-related channels using machine learning algorithms. It also provides them with the usage of diverse comparative research such as cross-language and cross-culture.
Data Availability
Since the data are included personal information, it cannot be made publicly available. The interested parties may request the data from the first author. Informed consent was obtained from all the subjects involved in the study. Written informed consent has been obtained from the subjects to publish this paper.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This research was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2021S1A5A8070305).