AI Powered Service Optimization for Edge/Fog ComputingView this Special Issue
[Retracted] Personalized Emotion Recognition and Emotion Prediction System Based on Cloud Computing
Promoting economic development and improving people’s quality of life have a lot to do with the continuous improvement of cloud computing technology and the rapid expansion of applications. Emotions play an important role in all aspects of human life. It is difficult to avoid the influence of inner emotions in people’s behavior and deduction. This article mainly studies the personalized emotion recognition and emotion prediction system based on cloud computing. This paper proposes a method of intelligently identifying users’ emotional states through the use of cloud computing. First, an emotional induction experiment is designed to induce the testers’ positive, neutral, and negative three basic emotional states and collect cloud data and EEG under different emotional states. Then, the cloud data is processed and analyzed to extract emotional features. After that, this paper constructs a facial emotion prediction system based on cloud computing data model, which consists of face detection and facial emotion recognition. The system uses the SVM algorithm for face detection, uses the temporal feature algorithm for facial emotion analysis, and finally uses the classification method of machine learning to classify emotions, so as to realize the purpose of identifying the user’s emotional state through cloud computing technology. Experimental data shows that the EEG signal emotion recognition method based on time domain features performs best has better generalization ability and is improved by 6.3% on the basis of traditional methods. The experimental results show that the personalized emotion recognition method based on cloud computing is more effective than traditional methods.
Emotion recognition has become the research field of artificial intelligence . With the improvement of cloud computing technology’s ability to perceive human emotions, the interaction between humans and computers has also been improved, especially in human-computer interaction, virtual reality, and computer-assisted application in education. In the research of emotion recognition, many aspects are included, such as the recognition of facial emotions, the recognition of sound emotions, the recognition of body emotions, and the recognition of physiological signal emotions. The communication between people is to convey information through language, and the emotional information conveyed through sound signals is an important source of information and an indispensable part of people’s perception and judgment of things. Information transmission in the spatial dimension is to convey richer emotional information through the understanding of cloud computing .
The purpose of emotion recognition is to extract, analyze, and understand people’s emotional behavior characteristics and use pattern recognition to realize the process of emotion recognition. Emotion recognition has many practical meanings [3, 4]. First of all, understanding emotions can help improve people’s health. In the medical field, research on identifying the emotional state of patients through physiological signals and improving human health through specific emotional states is ongoing. Second, understanding emotion recognition helps improve the interaction between humans and machines. Machine equipment can identify the emotional state of users and provide users with better quality and humanized services [5, 6]. Finally, by determining the emotional state of the user, it is also possible to provide the user with individual recommendations that match the emotional state.
At present, many scholars have deepened their research on emotion recognition. Jenke R uses EEG signals for emotion recognition, which can directly assess the user’s “internal” state, which is considered an important factor in human-computer interaction . He has studied many feature extraction methods and usually selects suitable features and electrode positions based on neuroscience findings. He used a small number of different feature sets and tested them on different (usually smaller) data sets to test their suitability for emotion recognition. He reviewed the feature extraction methods for emotion recognition in EEG signals based on 33 studies. An experiment was conducted to compare these features using machine learning techniques to perform feature selection on a self-recorded data set, giving results about the performance of different feature selection methods, the usage of selected feature types, and electrode position selection. The elements selected by the multivariate method are slightly better than the univariate method. However, his conclusion is not supported by corresponding data, so it is not authoritative enough . Atkinson J’s research concluded that the current emotion recognition computing technology has successfully correlated emotion changes with EEG signals. He believes that if appropriate stimuli are applied, they can be identified and classified from EEG signals. However, due to signal characteristics and noise, EEG constraints, and topic-related issues, automatic recognition is usually limited to a few types of emotions. In order to solve these problems, he proposed a novel feature-based emotion recognition model for the brain-computer interface based on EEG . Kaya H uses an extreme learning machine (ELM) to model modal features and combine the scores to make final predictions. Use deep neural network (DNN) or support vector machine (SVM) to obtain the latest results of auditory and visual emotion recognition. He proposed that the ELM paradigm is a fast and accurate alternative to these two popular machine learning methods. Thanks to the rapid learning advantage of ELM, he used moderate computing resources to conduct extensive tests on the data. In the video modality, the combination of regional visual features obtained from the inner face is tested. In audio mode, tests are performed to enhance training with other emotional corpora. The applicability of several recently proposed feature selection methods to tailored acoustic features is further investigated . Zheng W L introduced the Deep Belief Network (DBN) to build an EEG-based emotion recognition model for three emotions: positive, neutral, and negative. He developed an EEG data set obtained from 15 subjects. Each subject conducted two experiments every few days. The DBN is trained using differential entropy features extracted from multichannel EEG data. Check the weights of the well-trained DBN and study the key frequency bands and channels. Choose from four different profiles for 4, 6, 9, and 12 channels. The key frequency bands and channels determined by using the well-trained DBN weights are consistent with existing observations .
The main innovations of this paper include the following aspects. (1) We analyze and investigate the movement characteristics of expressions in various emotional states, analyze the existence time and expression differences of expressionlessness under positive, neutral, and negative emotions, and analyze gender differences. (2) This paper uses two feature extraction methods, SVM multiclassification algorithm and temporal feature algorithm. After training and testing, the correct emotion recognition rate in different states can be calculated.
2. Personalized Emotion Recognition Based on Cloud Computing
2.1. Emotion Recognition
Emotion recognition is the use of computers to detect human faces and analyze the characteristics of the performance information [12, 13]. The machine realizes the purpose of human beings’ recognition and understanding of emotional expression. From the point of view of the expression recognition process, emotion recognition can be divided into three main steps, namely, the detection and position of the face, the feature extraction of the expression, and the classification of the expression .
2.1.1. Face Detection and Positioning
The detection and localization of facial images is the first step in facial expression recognition. The content of this step is to find the correct position of the face from the acquired image or image sequence . In face detection, a statistical method is used to model the face, and the detected face area is compared with the face model to obtain the possible face area.
2.1.2. Extraction of Facial Features
An important part of the facial expression recognition process is facial expression feature extraction, and its main function is to extract information features that can characterize human facial expressions. Expression feature extraction methods can be divided into deformation features and motion features [16, 17]. This paper uses geometric feature-based methods to extract expression features and introduces a method of extracting expression features based on still images. When a person’s facial expression changes, important information features will be extracted from the deformation process of the face. Methods of obtaining facial features: that is, as an expression feature vector, the main distance between facial features is obtained, the shape changes, and relative positions of various organs are analyzed to achieve the purpose of obtaining facial features. Geometric deformation is an obvious response to changes in human facial expressions and can handle still pictures and animated expressions . However, the geometric feature method ignores other subtle changes when extracting feature information of multiple expressions, so the overall recognition rate is not high.
2.1.3. Facial Expression Classification
Expression classification analyzes the relationship between expression functions and assigns them to corresponding categories. Next, the method based on neural network will be explained. The neural network is composed of various parallel units. The change of expression drives the change of the neural network. The output node of the neural network corresponds to 10 general basic corresponding points. The output node connects multiple processing neurons to form the entire neural network structure [19, 20]. Artificial neural networks can learn repeatedly and obtain invisible effects from the corresponding point rules. After feature extraction, according to the performance classification method of artificial neural network, an extremely obvious expression classification effect can be obtained .
2.2. Cloud Computing Features
Cloud computing mainly has the following characteristics:(1)Dynamic resource allocation: Cloud computing can dynamically allocate or release some physical and virtual resources according to user needs. As user needs increase, available resources can be allocated and released after reducing user needs . Provide users with surplus resources of flexible resources. Cloud computing can provide unlimited services through the expansion of resources.(2)Self-service provision of services: Cloud computing can automatically provide users with resource services, and users can obtain the resources they need without having to talk to suppliers. There are mainly service descriptions and catalogs, and services can be selected based on this information.(3)Services can be measured on general-purpose computers: Cloud computing provides services through the Internet. As long as users have computers and Internet, they can get the services provided by cloud computing. Therefore, cloud computing is universal. When cloud computing provides services, meters can be used to configure resources according to the services required by users. In other words, because cloud computing resources can be monitored and controlled, they can be used immediately if the service is charged.(4)Resource pool and transparency: For suppliers, cloud computing can protect the differences in basic resources such as computing, storage, and business logic. You can perform comprehensive scheduling and management of cloud computing resources across resource boundaries [23, 24]. This is a “resource pool” that can provide services to users on demand. For users, cloud computing is transparent, as long as they care about whether they can meet their needs, and they do not have to care about its structure.(5)High-performance price ratio: The high-performance computing function provided by cloud computing is the integration of multiple computing resources, and the hardware requirements are inappropriate. Users do not need to purchase a large amount of hardware and software resources, which can greatly save consumption costs [25, 26].(6)Flexibility: According to virtualized computing, cloud computing can quickly build infrastructure and dynamically increase or release resources as needed. Cloud computing provides users with flexible purchase time (time, day, month, etc.).(7)Reliability: Cloud computing is a service provided by multiple nodes. Data storage and data calculation are scattered in different nodes. Even if a node fails, new nodes can be dynamically allocated to provide services. Cloud computing also uses multiple technologies such as data fault tolerance technology to ensure service reliability.
2.3. Brain Operating Mechanism of Emotions
In the human brain, the prefrontal lobe accounts for about 40% of the cerebral cortex. The cerebral cortex is mainly composed of the motor cortex, the premotor cortex, the forebrain cortex (forebrain cortex, PFC), and the full medial part of the forebrain . This is the operating center of the brain function. It is connected to other parts of the brain to process and integrate information while selecting appropriate emotional and motor responses. It will not only change the disease function, action, and decision-making ability of the scene in front, but also affect feelings and mood. The cerebral cortex is an important area of emotion induction and regulation. There are three opinions on the role of the front desk in emotional processing.(1)The orbitofrontal area of the prefrontal lobe is related to reward processing and reinforcement learning. In particular, nerve cells in this field can perceive changes in stimuli, reverse the compensation of stimuli, and change this response . The cortex plays an important role in the connection between external stimuli and reward enhancers.(2)The ventromedial prefrontal lobe can be used as a communication platform between visceral responses and high-level cognitive functions, that is, the “physical identification hypothesis.” Somatic cell markers are peripheral responses to stimuli. The ventromedial prefrontal lobe will be processed as part of the guided advanced cognitive system.(3)The “asymmetry of power hypothesis” in the prefrontal cortex, that is, the tendency to form biological motivation, can be defined from the perspective of approaching the avoidance level. If the motivation of this method is activated, organisms will have a strong motivation to pursue compensation goals. On the contrary, emphasizing the activation of avoidance motivation is not to get compensation, but to avoid harmful situations. The central proposition of the asymmetric value hypothesis is that the right anterior lobe activates the evasive motive, and the left anterior lobe activates the approach motive to form adaptive actions.
2.4. SVM Multiclassification Algorithm
SVM is a two-type model, which is a linear classifier with a maximum interval defined by a feature space. The goal is to find the largest gap. The kernel method of support vector machine is the main manifestation of its advantages. Linear inseparable data is mapped to high-dimensional feature space through kernel technology and can be classified into high-order element space. The principle is that, in the case of a brief introduction to the binary classification problem, the following format records the training data set on the feature space.
Among them, means that the total number of samples is . The characteristic parameter of each sample data is a column vector, and the vector is n-dimensional. In the case of sample , the distance to the separation hyperplane when the distance is less than 1 is represented by . Therefore, the restriction conditions are appropriately relaxed, can be manually assigned, and its value represents the final SVM classification, and the result is the tolerance for misclassification. When the value of is larger, it means that the sample data allows higher misclassification. In general, when the amounts of positive data and negative data of the sample data set are extremely unbalanced, by changing the value of this variable, the classification result of the support vector machine will be more strict for the sample type with less data.
2.5. Time Domain Feature Algorithm Extraction
This article starts with the cloud computing analysis method and extracts the statistical parameters of the expression signal from the time domain as the analysis feature. Extract expression features from the following 6 statistical methods:(1)The mean value of the original signal.(2)The standard deviation of the original signal.(3)The first-order difference average absolute value of the original signal.(4)Standardize the signal first, and then find the average absolute value of the first-order difference. After the original signal is normalized,(5)Take the average absolute value of the second-order difference to the original signal.(6)Normalize the signal to find the average absolute value of the second-order difference.
The above-mentioned time features are classified as single-featured emotions based on cloud computing analysis and calculation methods and are used as standard quantities for emotion recognition research. Next, the six extracted time-domain features are combined into a fusion feature vector. After emotion classification, the classification performance is compared with the standard quantity to draw experimental conclusions.
3. Facial Emotion Recognition Experiment Based on Cloud Computing
3.1. Experimental Environment and Configuration
This article will be based on cloud computing, the programming experiment of facial expression recognition experiment will be carried out in MATLAB 2016 software. At the same time, there is a Java language programming that supports VBA applications, as shown in Table 1.
3.2. Data Collection
The data set used in the experiment is a facial emotion recognition data set composed of 4679 facial expression photos. Among them, 953 people have angry faces, 547 people are bored, 512 people are scary, 969 people are happy, 657 people are sad, 412 people are surprised, and 629 people are expressionless. The data set consists of three parts. The first part is a training set containing 2154 images. The second part is the verification set containing 1536 images. The third part is a test group containing 989 images.
3.3. Experimental Procedure
This article next mainly verifies how the server side of the emotion recognition system realizes the communication connection with the cloud. After the connection is successful, the server starts to receive and save EEG data. When the received data reaches a certain time (60 seconds), the program will automatically perform the emotion recognition process. As shown in Figure 1. The communication process between the server and the cloud experimental data set will be summarized into the following three steps.(1)At the beginning of the emotion recognition experiment, first execute the MATLAB program to detect the specific port stored by cloud computing on the expression server side. If there is data sent from the data set on the port, the server starts to receive and save the data. Like the API included in the Android operating system, the program also comes with a communication package for the expression prediction system. Since the related functions can be used directly after import, the development process is extremely simple. The first step in using the program communication function is to create an object that needs expression detection. The program udpc = dsp.UDP receiver (local IP port, 9999) creates a receiving object. The local port used to receive and transmit data is 9999, and the udpc object is used to save the received data. To prevent the port from being occupied, the object must be released after the program ends. The end function to be used is udpc.release().(2)The data received by the program is of string type. To convert to a directly processed number type, type conversion is required. The function used is str2 double(). According to the judgment condition, after the received data reaches 30,000 (60 seconds), the emotion recognition algorithm program is called, and the result is executed. Then, use the program res = uint10(num2str(res)) to convert the value of the result, and use UDP to return the result to the cloud.(3)Finally, use udpe = dsp.UDP Sender (‘Remote IP address’, Android IP,’Remote IP port’, 9999) to create an experimental emoticon sending object. The IP address of the sending device is Android IP, and the sending remote port is 9999. We need to send this. The data is stored in the udpe object. Similarly, the object must be released after the program ends, and the function used is udpe.release().
The above process is a complete cycle of the EEG data collected by the EEG device and the analysis result, and the result is displayed on the cloud as the final symbol. After continuing the above process, the cloud can continue to display the analysis results in the background until the EEG device no longer collects EEG data.
4. Analysis of Facial Emotion Recognition Data
4.1. Mixed Data Feature Fusion Emotion Recognition Analysis
In the experimental verification part, the data is divided into two types for comparison experiments. One is a mixed data set composed of all volunteer data, and the other is a personal data set composed of volunteer data with more complete personal data. Two different types of data will be used to recognize user emotions based on feature fusion, and the recognition results will be compared and analyzed. The seven classifiers of mixed data fusion are GTB, Random Forest, Ada Boost, Decision Tree, KNN, and SVM. Using mixed data for classification and recognition, the experimental results are shown in Figure 2. It can be seen from the results in the figure that when the basic emotion model is used as the emotion classification standard, the KNN classifier has the highest recognition accuracy, reaching 66.24%; when the ring emotion model is the recognition rate of the GTB classifier, it can reach 69.63%.
By comparing the accuracy of the two classification models, it is found that, in the mixed data experiment, the six classifiers use the ring emotion model to identify the results that are higher than the basic emotion model. According to the experimental results, the reason why the circular emotion model has a higher recognition rate than the basic emotion model is that continuous emotion is a vague measurement method, which is more humane than the specific classification of emotions into a certain category, which is convenient for users to measure and select.
4.2. Analysis of SVM Emotion Recognition Classification Results
SVM classifier is used for these 11 single features. This article is designed on the SVM toolkit. Perform optimization of penalty parameter C and kernel function r to get the classification result, and perform 5 times of 5-fold crossover operation. Take the average classification accuracy obtained as the final classification result. The average accuracy rate of the subjects’ two emotional valence classifications is shown in Figure 3.
When using SVM as a classifier, the study found that 1st, 2nd, std, FD, and SE performed extremely well. Since the two features of 1st and 2st have high similarity, this also confirms the effectiveness of the two time-domain signals, the mean of the absolute value of the first-order and second-order differences, in the two-classification task of emotional valence. At the same time, it can be concluded that the nonlinear characteristics are superior in EEG signal processing. Combined with the single feature classification results of ELM, 2st, FD, and SE are selected as the best three features for analysis. But at the same time, it is found that the differential entropy of ADE, BDE, CDE, and GDE in the four different frequency bands has little difference in the classification task. The classification accuracy difference between the best-performing GDE and the worst-performing ADE is 5.012%.
4.3. Comparison of Sentiment Prediction Results
The time-domain feature algorithm can obtain appropriate data by training and learning EEG data samples, distinguishing the characteristics of EEG data samples that contribute differently to emotions, then more accurately measure the similarity of EEG data samples, and finally achieve improved emotions, which is the purpose of forecast accuracy. The classification algorithm still uses the SVM algorithm for comparison, and a five-fold cross-validation experiment is performed on the training samples and test samples of the EEG data of 8 subjects. The comparison of the accuracy of emotion prediction based on time domain features is shown in Table 2, and the analysis of emotion prediction based on time domain features is shown in Figure 4.
Table 2 and Figure 4 show the prediction accuracy rates of the positive, neutral, and negative emotions of 8 subjects. It can be seen that, compared with the traditional method, after adding the time domain feature to the EEG signal emotion recognition, it can improve the accuracy of emotion recognition to a certain extent. Among them, the EEG signal emotion recognition method based on time domain features performs best and has better generalization ability. It is improved by 6.3% on the basis of traditional methods. This shows that EEG data samples will be in the new feature space. After the different features that contribute to emotion prediction are treated differently, the similarity between EEG data samples can be measured more accurately; that is, it can be highly classifiable. In addition, the SVM algorithm in Chapter 2 of this article is used to process abnormal samples on the original training set, detect and remove samples with wrong emotion labels, and improve the accuracy of emotion prediction by 6.5% on the basis of traditional methods, further verifying the effectiveness of the method.
4.4. Analysis of Emotional Classification Model
The model training of emotional classification in this paper adopts the experimental part of the low score database. The training method is as follows: first, input the data stored in cloud computing, extract the expression features transmitted by brain waves, and then reduce the dimension of the extracted expression features and input them into the SVM. Train the emotional classification model in the classifier, and finally obtain a model with higher accuracy by minimizing the error method. The test set uses the data corresponding to the image sequence in the self-built database, and the experimental results on the test set are shown in Table 3.
Because the emotional samples of each category in the data set are not uniform, the amount of learning data of various samples in the model is inconsistent. Therefore, the classification effect of each category is different. It can be concluded from Table 3 that the classification accuracy rate of happy and sad emotions is low, because emotions are more noisy in emotions when they are more excited and are easily misidentified as emotions such as anger and surprise. When sad, they are easily misidentified as neutral, disgust, and fear. In future experiments, some methods of removing noise interference are needed to improve the recognition rate of these two emotions. The data obtained in the above experiment is processed with software. The processing method for the prediction results of image sequences and audio data in the same time period in the same video is the use of weighted fusion image sequence expression recognition results and emotional classification results. The experimental results are shown in Figure 5.
From Figure 5 it can be concluded that since the expression prediction result of the image sequence in the video and the emotional prediction result of the audio are two independent results, the weighted fusion image sequence expression recognition result and the audio emotional classification result significantly improve the expression recognition accuracy. Compared with a single mode of facial expression recognition, this method of judging facial expressions fully considers various influencing factors of facial expressions and assigns different weights to different determinants. This method is more robust and has wider applicability.
This paper uses the expression characteristic signal parameters as the basis of emotion recognition and uses cloud computing network algorithms to realize emotion recognition. Similarly, the expression file is divided into a training set and a test sample set. The input uses the selected training sample set to determine the input, the number of hidden layers and the number of neurons contained in each layer, the learning rate, and other parameters to train the deep belief network, so that the deep belief network can respond to the input training sample set and learn the characteristics of each emotion. We use the constructed emotional model to recognize the picture and text file of the test and obtain the recognition rate corresponding to each sample of the test sample set. This paper uses the final test data of the SVM multiclassification algorithm to investigate which of the SVM multiclassification algorithms has the highest emotion recognition rate. It mainly compares the emotion recognition algorithms based on cloud computing from the recognition rate, the number of optimization parameters, the optimization algorithm, and the energy consumption of the two algorithms.
According to the facial emotion recognition model proposed in this paper, a facial emotion prediction system is developed. The system uses cloud computing to capture the video stream, uses the SVM algorithm to detect the facial images in the video of the read video, and delivers the intercepted facial images to the emotion model for recognition and analysis, and the graphical user controller displays it on the scroll interface. The system can analyze facial information in real time, and the model can perform excellently. In addition, based on cloud computing, a lightweight model is used. This model uses a small amount of memory and has a small amount of calculation, and its application prospects are excellent.
For the channel selection problem in emotion recognition based on cloud computing, this paper introduces the SVM method with high spatial resolution as an auxiliary method and proposes a channel selection method based on cloud computing emotion model. First, we learn from the problem-solving method, establish the emotional model of cloud computing, and obtain the transmission matrix between the signal source and the electrodes on the head surface of the cerebral cortex. As a result, the activation result of the emotion experiment can be mapped to the surface of the head, and an EEG pattern reflecting the degree of emotional correlation can be obtained. The analysis of experimental data shows that the emotion correlation recognition map obtained from the activation status reflects the relationship between emotion signals and emotions of different electrodes to a certain extent and can provide a specific theoretical basis for cloud computing emotion recognition methods.
No data were used to support this study.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Z. Lv and W. Xiu, “Interaction of edge-cloud computing based on SDN and NFV for next generation IoT,” IEEE Internet of Things Journal, vol. 7, no. 7, pp. 5706–5712, 2019.View at: Google Scholar
Z. Lv, L. Qiao, Q. Wang, and F. Piccialli, “Advanced machine-learning methods for brain-computer interfacing,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 16, 2020.View at: Google Scholar
Y. Zhang, X. Xiao, L. X. Yang, Y. Xiang, and S. Zhong, “Secure and efficient outsourcing of PCA-based face recognition,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 1683–1695, 2019.View at: Google Scholar
Z. Li, “Application of a resource-sharing platform based on cloud computing technology,” Agro Food Industry Hi Tech, vol. 28, no. 1, pp. 2205–2209, 2017.View at: Google Scholar
N. Liouane, “Recursive identification based on OS-ELM for emotion recognition and prediction of difficulties in video games,” Studies in Informatics and Control, vol. 29, no. 3, pp. 337–351, 2020.View at: Google Scholar
W. Li, B. Jiang, and W. Zhao, “Obstetric imaging diagnostic platform based on cloud computing technology under the background of smart medical big data and deep learning,” IEEE Access, vol. 8, pp. 78265–78278, 2020.View at: Google Scholar