Abstract
The partial differential equation learning model is applied to another high-level visual-processing problem: face recognition. A novel feature selection method based on partial differential equation learning model is proposed. The extracted features are invariant to rotation and translation and more robust to illumination changes. In the evaluation of students’ concentration in class, this paper firstly uses the face detection algorithm in face recognition technology to detect the face and intercept the expression data, and calculates the rise rate. Then, the improved model of concentration analysis and evaluation of a college Chinese class is used to recognize facial expression, and the corresponding weight is given to calculate the expression score. Finally, the head-up rate calculated at the same time is multiplied by the expression score as the final concentration score. Through the experiment and analysis of the experimental results in the actual classroom, the corresponding conclusions are drawn and teaching suggestions are provided for teachers. For each face, a large neighborhood set is firstly selected by the -nearest neighbor method, and then, the sparse representation of sample points in the neighborhood is obtained, which effectively combines the locality of -nearest neighbor and the robustness of sparse representation. In the sparse preserving nonnegative block alignment algorithm, a discriminant partial optimization model is constructed by using sparse reconstruction coefficients to describe local geometry and weighted distance to describe class separability. The two algorithms obtain good clustering and recognition results in various cases of real and simulated occlusion, which shows the effectiveness and robustness of the algorithm. In order to verify the reliability of the model, this paper verified the model through in-class practice tests, teachers’ questions, and interviews with students and teachers. The results show that the proposed joint evaluation method based on expression and head-up rate has high accuracy and reliability.
1. Introduction
In the previous studies on the evaluation of students’ concentration in college Chinese classes, most of them adopted homework test, questionnaire scale, and instrument measurement. In recent years, some studies have tried to use students’ emotions and expressions as indicators to evaluate students’ concentration in a college Chinese class. However, through literature investigation and research, it is found that the existing evaluation methods of students’ concentration in a college Chinese class still have the following problems: first, in the one-to-many classroom environment, it is easy to cause incomplete and untimely evaluation; second, the existing evaluation method of students’ concentration in a college Chinese classroom is too single, which cannot take into full consideration the factors affecting students’ concentration in a college Chinese classroom; third, there is no reasonable partial differential equation analysis and evaluation model of focus for face recognition between different evaluation indicators, which is greatly affected by a certain indicator and cannot effectively reduce the error; and fourthly, the recognition effect of the facial expression recognition algorithm is not ideal, leading to a certain error in the evaluation of concentration. Therefore, how to establish a comprehensive, efficient, and accurate evaluation model of students’ concentration in a college Chinese class is a problem that needs to be solved by the educational circle at present.
Concentration allocation refers to the ability of an individual to allocate attention to two or more kinds of activities or information at the same time. For example, in class, teachers need to write on the blackboard, give lectures, and pay attention to students’ dynamics. Students need to see on the blackboard, listen to the teacher, take notes by hand, and think in the brain, so that the concentration can be allocated to various ongoing activities. In recent years, many researchers have realized the importance of concentration on students’ academic performance through many experimental studies, and the method to evaluate students’ concentration in a college Chinese class has emerged. At first, the researchers mostly used the teacher interview method, direct observation method, and parent interview method. However, the researchers found that these methods were too subjective and could not rely on the research experience of observers. With the deepening of researchers’ research on the measurement of concentration, the commonly used methods of concentration measurement include homework test, questionnaire, and instrument measurement [1, 2].
This paper proposes a learning model of partial differential equation for feature extraction and applies it to face recognition in high-level visual-processing problems. Feature learning is an important step in pattern recognition (such as image classification). However, most feature-learning methods are not able to recognize some transformations without changing features. This limits the improvement of identification (classification) results, especially for small sample sizes. In order to solve this problem, a new feature selection method based on the partial differential equation learning model was proposed, which kept the rotation and translation invariance and was more robust to the illumination change. After the learning features are obtained, a simple linear classifier is used to obtain the final recognition results. Compared with the most advanced methods on four public standard face recognition databases, good experimental results are obtained. In order to verify the reliability of the model, the model was tested by in-class practice tests and teachers’ questions. Through many experiments, it has been shown that the better the concentration of the college Chinese class is, the better the corresponding performance of in-class practice tests and the passing rate of teachers’ questions are. It verifies the reliability of the evaluation of students’ concentration in the college Chinese class based on expression and head-up rate.
2. Related Work
Traditional PDE image-processing methods are dependent on specific problems. When people are faced with a problem, they need to dig deep into the characteristics of the problem, understand the key to solve the problem, and formalize and calculate it. But the human eye or brain has only one system, and its structure does not change with the problem, but it can effectively solve a variety of problems. This prompts people to consider whether there is a unified model to solve different image-processing problems. Recently, the partial differential equation learning model (LPDE) [3] has been proposed, which has been proven to be able to solve a series of image-processing problems with the same form of partial differential equation, including deblurring, denoising, edge detection, image segmentation, object detection, and color image interpolation. Good results have been achieved in these problems. In this framework, users who want to obtain partial differential equations only need to provide input and output training image pairs, thus saving the trouble of in-depth digging and research on specific problems [4]. LPDE introduces the idea of “learning” in machine learning to the field of differential equation image processing and proposes a theoretical framework to solve problems in image processing by learning specific partial differential equations through training data [5]. Firstly, a unified intelligent differential equation system is constructed by using rotation and translational basic differential invariants as basic functions, and then, the specific partial differential equation form is obtained by using optimal control theory training of differential equation constraints. These basic differential invariants are derived from some empirical and mathematical aspects of image processing. Kitahata et al. provided 17 basic differential invariants for many problems [6].
Firstly, it is assumed that the image-processing task to be done can be described by evolution equations, that is, the input image evolves according to a specified evolution equation, and the result of evolution is the desired processing node. So the first thing we need to do is to build a unified intelligent system of partial differential equations to describe evolution. The established partial differential equation system is composed of two coupled evolutionary partial differential equations [7]. As for the evolution function of the output image, one is an indicator function used to control the evolution and collect the overall scale information of the image. The established differential equation system is only about the image evolution function , and the control function is removed [8]. At present, most scholars focus on using differential invariants to solve the following problems, such as constructing differential invariants, solving variable problems, and determining their exact solutions and the basis of conservation law, equivalent transformation, and symmetry, and some of its strict properties are determined by differential invariants [9, 10]. And finding differential invariants in differential equation theory is regarded as a classical study. However, this paper is committed to applying differential invariants to the actual problems of image processing, that is, constructing partial differential equations of image processing with them [11]. For general image processing and computer vision problems, the invariant properties of translation and rotation are very important, that is, when the input image is translated or rotated, the output image is also translated or rotated accordingly. Therefore, the invariance of translation and rotation should be taken into account when constructing partial differential equations. In fact, the space constructed from translation and rotation differential invariants is infinitely dimensional, because any function constructed from translation and rotation invariants is still translation and rotation invariant. When spatial variables increase, it is also difficult to derive differential invariants [12]. Based on the previous experience and analysis of image processing, some basic differential invariants under translation and rotation are given, and a partial differential equation is constructed by using translation and rotation differential invariants. Since the functions formed by translation and rotation invariants are also translation and rotation invariants, the simplest constituent functions are linear combinations [13].
The accuracy of facial expression recognition can be influenced by various factors such as the change of illumination intensity, occlusion of the face, and change of facial posture. In the research on the existing FER technology, 3D face modeling is usually used to solve the impact of two variable factors, face pose change and light intensity change, on the accuracy of facial expression recognition [14]. However, the current 3D face modeling technology can not only achieve automatic modeling, and the accuracy is not up to the standard applied in various fields. However, the feature decomposition method that can solve the problem of automatic modeling is still in the preliminary research stage and cannot meet the requirements of application. Therefore, the feature decomposition method may become the focus of FER technology research in the future [15]. In recent years, many researchers have realized the importance of concentration on students’ academic performance through many experimental studies, and the method to evaluate students’ concentration in a college Chinese class has emerged. At first, the researchers mostly used the teacher interview method, direct observation method, and parent interview method. However, the researchers found that these methods were too subjective and could not rely on the research experience of observers. With the deepening of researchers’ research on the measurement of concentration, the commonly used methods of concentration measurement include homework test, questionnaire, and instrument measurement [16, 17]. At present, in the existing studies on the evaluation of attentiveness, advanced technologies such as face recognition based on deep learning are more inclined to analyze the attentiveness of the observed. For example, Wang and Liu [18] proposed a study on the concentration of the college Chinese class based on face detection, Xu et al. [19] proposed a study on the attention of the college Chinese class based on face detection, and Guo Xiaoxu proposed a systematic study on the concentration analysis of the college Chinese class based on microexpression recognition. The former two are used to analyze the concentration of the college Chinese class by calculating the head-up rate after detecting the face through a convolutional neural network, while the latter uses facial expression recognition technology to recognize the facial expression of students and obtains the concentration evaluation through facial expression. On the study of students’ concentration, clickstream data also as a kind of commonly used research data, especially in the online learning platform, such as MOOC clickstream data analysis which is often used to predict the student dropout rate; also, from the side, it reflects the students in the process of online learning focus and persistent concentration [20]. Considering the weekly learning history of student data, the change of student behavior can be noticed over time. Based on this, a machine learning algorithm is proposed to process clickstream data. In the study [21], the hidden Markov model (HMM) [22] was used to simulate the time-varying sequence of student behavior, and a simple cross-product method was used to encode several consecutive features into a discrete observable state, so as to obtain a high accuracy prediction of the student dropout rate. Research [23] explores accurate early identification of students at risk of failing to complete the course. On the basis of logistic regression, two transfer learning algorithms are proposed, and their effectiveness is proved on Coursera and MOOC platforms.
With the help of the development of face recognition, posture recognition, and other related technologies, there are also many methods based on the analysis of facial expression, eye gaze, posture, and other local features of students’ attention recognition based on pictures and videos. Although there are many methods based on local features, the main process can be summarized as follows: extracting a single local feature or extracting multiple local features and combining them for concentration analysis. In the studies [24, 25], students’ concentration was automatically recognized according to facial expressions. Researchers [26] proposed to obtain clues from the eye movement pattern of head movement to infer the information of students’ concentration in an e-learning environment. Research [27] used an open source tool library to extract multimodal features including eye fixation, head posture, and body posture to predict students’ concentration. In the study [28], the machine learning model was used to analyze the extracted features. After extracting local features in the study, LSTM and GRU were used to analyze the time series in the video.
3. Study on Attentiveness of Partial Differential Equation Learning Face Recognition Model
3.1. Face Recognition Framework
The face detection and localization part will decompose the classroom video by frame- and extract-effective pictures. The workflow of face detection model based on partial differential equation is shown in Figure 1. After get effective graphics, entered the stage of face detection and location, in order to ensure the accuracy of face detection, as shown in Figure 1, determine the face position and segmentation, this method is mainly from the scale-invariant, image resolution, and context (hair, clothes, and other related information) to process three aspects, first will detect zooming different proportion. This model is a multitask-trained partial differential equation model with different sizes of templates. Finally, we apply nonmaximum suppression fusion to the original resolution image to obtain the final detection results. The method has seen great results on some people’s faces, as well as on Face Detection Data Set and Benchmark (FDDB) and Wider Face Dataset.
When features are extracted, a linear classifier is used for face recognition. The training image set needs to be prepared before the training process, is the face image input and is the label. If is class , then the -th element of is equal to 1; is the number of training samples. For each face image, features can be obtained by learning the evolution equation and used for the following classification. Therefore, the complete partial differential equation learning model of face recognition is to determine the function and learning classifier parameters, and then, the loss function with smooth regular constraints is obtained as follows:
For most evolution equations, they can be written in the following form:
Variation is a huge challenge in most image classification tasks (such as face recognition). In order to make the extracted features meet the invariance of illumination, a nonlinear mapping is added to each basic differential invariant, which can greatly weaken the influence of illumination variation.
In order to demonstrate the superiority of the feature-learning pDES model, the general linear classifier is simply adopted. Hinge loss function shows its significant superiority in many aspects and has achieved good results in face recognition. Therefore, the multivariate ridge regression model is used for classification. The specific form is as follows:
When the parameter is fixed, the gradient descent method is used to update parameter . It can be calculated by the chain derivative rule:
The specific steps of the algorithm are shown in Table 1.
3.2. Partial Differential Equation Alternate Optimization of Face Recognition Algorithm
It is the process of identification to classify and recognize the selected face images to determine whether the face belongs to the known sample database of the experiment and to determine which category the face image belongs to. This module compares the face features to be recognized with the sample features in the sample database and then carries out comparative experiments according to certain algorithms. The criterion is to group together features that have the same or similar structure. In the classification, the classification matching algorithm has a close relationship with the features extracted; only selecting the appropriate matching strategy can quickly retrieve from the database the face features to recognize matching and give the recognition results. K-L change is an optimal orthogonal change in image compression; high-dimensional images through K-L change can obtain a set of orthogonal basis; choosing to retain part of the orthogonal basis can generate low-dimensional face space.
For any face image , its coefficient vector can be obtained through the above subspace projection:
Thus, the overall dispersion matrix is
Its flow chart is shown in Figure 2.
3.3. Face Recognition Technology of College Chinese Classroom Concentration Evaluation Model
In this experiment, the detection of students’ head up is accomplished by face detection; that is, when a student’s face is detected at a certain moment, it is considered that the student belongs to the head-up state. Specifically, it uses MTCNN (multitask convolutional neural network) to conduct face detection every 2 seconds in the classroom, and statistics is done on the face data detected at every moment. In this experiment, errors caused by complex situations in an actual class should be reduced as far as possible; for example, students in the front row turn their heads to cover the faces of students in the back row, but these inevitable errors should be allowed. By calculating the proportion of the number of faces detected in the total number of students in the class, we can get the rise rate of the class as a whole at the moment:
The proposed improved concentration evaluation network model outputs the probability of each expression after classification by Softmax, so the probability of each dimension is multiplied by their respective weights and then accumulated as the concentration score based on expression:
Further, in order to facilitate the subsequent calculation and highlight the intuitive meaning, the concentration score based on expression is normalized to the range [0-1]:
Therefore, the expression concentration score of a single student can be obtained after normalization, while the expression concentration score of the class as a whole is the concentration score of each student and then divided by the total number of students in the class.
Therefore, after normalization, the expression concentration score of a single student is obtained, while the expression concentration score of the class as a whole is every student’s concentration score summed and divided by the total number of students in the class:
In the experiment, the program detected every 2 seconds and saved the obtained images to the corresponding folder, so in 2t seconds, we could obtain the images of students’ facial expressions t times. Further, if you want to obtain the concentration of a certain student in class or the whole class, you can obtain the concentration of a certain student in a class by summing up the concentration score of each test of the same student and averaging it. Similarly, the overall concentration of a class can be obtained by summing up and averaging the overall concentration scores of the whole class in each test.
In order to evaluate students’ concentration in class, this paper divides the concentration in a college Chinese class into three grades: low, medium, and high. The specific scores are divided as shown in Table 2.
Usually, teachers cannot pay attention to the concentration of each student because they need to take into account both blackboard writing and lectures in class. They can only judge the concentration of the class students roughly by the number of heads. But this method of assessment ignores the change of students’ mood, which greatly reduces the accuracy of the assessment. In order to evaluate the class’s overall concentration more comprehensively and scientifically, this section will further analyze the class’s concentration by establishing a joint model based on expression and head-up rate. The structure of the concentration analysis and evaluation model of partial differential equation face recognition is shown in Figure 3.
As shown in Figure 3, compared with the pure expression concentration evaluation model, the partial differential equation face recognition concentration analysis evaluation model based on expression and head-up rate combines expression and students’ head up in class. In the figure above, facial expression data is obtained after MTCNN face detection, and the number of faces is also detected and the head-up rate is calculated. Then, the improved network based on the concentration analysis and evaluation model was used to recognize the expression and then multiplied by their weight coefficients to calculate the expression score. Finally, the joint evaluation concentration score was obtained by multiplying the head-up rate detected at the same time and the calculated expression score.
The expression score of each student can be obtained by recognizing each face detected at the same time using the improved network based on concentration analysis and evaluation model and multiplying the corresponding weight . The individual expression score was adjusted to include only those detected.
4. Example Verification
In this section, face expression recognition is carried out by using the concentration analysis and evaluation model network. A student is randomly selected to identify the expressions detected 1200 times in one class and calculate the expression concentration score, as shown in Figure 4.
Figure 4 is the line chart of concentration detection of students in a certain class. Although the fluctuation in the chart is relatively large, it can be seen from the first 400 tests that student’s concentration has been improving. From the test of 400-1000, it can be seen that the concentration of the student in this stage fluctuated, but the average remained above 0.4, indicating that the student’s overall concentration in class was medium concentration.
Through the experiments of three classes, we combined the head-up rate and facial expression to calculate the class’s overall concentration score of the concentration analysis and evaluation model based on facial expression and facial expression head-up rate. Figure 5 is the scatter diagram of the overall class head-up rate in the first class, and Figure 6 is the broken line diagram of the concentration analysis and evaluation model of partial differential equation face recognition.
As can be seen from the scatter chart of 5, more than half of the class’s overall head-up rate is above 0.5, and most of the time, the head-up rate of the class is greater than 0.6, indicating that the overall head-up rate is not bad.
As can be seen from Figure 6, the overall concentration of the class is above 0.3, which is generally a good level. The overall concentration of the class is above medium. The concentration of the class fluctuates greatly in the first ten minutes of class and also fluctuates greatly in about five minutes before class.
The application and test are carried out in the video samples of classroom scenes to detect the face fatigue state of each frame, and the face concentration can be expressed by calculating the proportion of the fatigue frames to the total frames. There are 100 faces in the class data set; the fatigue discrimination accuracy of the concentration analysis and evaluation model of partial differential equation face recognition and other face fatigue detection algorithms are shown in Table 3.
The class comprehensive evaluation module includes two parts: class comprehensive performance and class interaction. The class comprehensive performance will record the data of students’ concentration in the whole class and draw charts according to the concentration scores at different times. As shown in Figure 7, students can check their own concentration curve and compare it with the class comprehensive concentration analysis curve to check their own performance and the average class level. Teachers can also check the overall performance of students and dynamically adjust their teaching content according to the performance of students, for example, theory explanation in the morning when students are full of energy and review and homework explanation in the afternoon, making it easier for students to absorb and understand. In the classroom comprehensive interaction module, the number of hands raised and the number of students standing to answer questions are counted, and the interaction between teachers and students is counted.
5. Conclusion
At present, most feature-learning methods are not able to recognize some transformations without changing features. This limits the improvement of identification (classification) results, especially for small sample sizes. In order to solve this problem, a new feature selection method based on partial differential equation learning was proposed, which kept the rotation and translation invariance and was more robust to illumination changes. After the learning features are obtained, a simple linear classifier is applied to obtain the final recognition results. Experimental results on four publicly available standard facial recognition databases are all better than current state-of-the-art methods. By giving certain weight to the seven dimensions of expression, the weighted sum is used as the concentration score of students. Through the experiment of a college Chinese classroom, this paper analyzes the students’ concentration in class and gives corresponding teaching suggestions to the teachers to help them improve their concentration in class. This paper not only uses facial expression recognition to calculate the concentration of college Chinese class but also explores the analysis and evaluation model of the concentration of the college Chinese class using partial differential equation face recognition of facial expression and student head-up rate. A validation analysis experiment was designed to verify the reliability of the expression concentration experiment by using in-class practice tests and teachers’ questions. The results of the verification experiment prove the reliability of the evaluation method based on expression concentration and the reliability of the evaluation model of partial differential equation face recognition concentration analysis.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.