Abstract
In recent years, online and offline teaching activities have been combined by the Small Private Online Course (SPOC) teaching activities, which can achieve a better teaching result. Therefore, colleges around the world have widely carried out SPOC-based blending teaching. Particularly in this year’s epidemic, the online education platform has accumulated lots of education data. In this paper, we collected the student behavior log data during the blending teaching process of the “College Information Technology Fundamentals” course of three colleges to conduct student learning behavior analysis and learning outcome prediction. Firstly, data collection and preprocessing are carried out; cluster analysis is performed by using k-means algorithms. Four typical learning behavior patterns have been obtained from previous research, and these patterns were analyzed in terms of teaching videos, quizzes, and platform visits. Secondly, a multiclass classification framework, which combines a feature selection method based on genetic algorithm (GA) with the error correcting output code (ECOC) method, is designed for training the classification model to achieve the prediction of grade levels of students. The experimental results show that the multiclass classification method proposed in this paper can effectively predict the grade of performance, with an average accuracy rate of over 75%. The research results help to implement personalized teaching for students with different grades and learning patterns.
1. Introduction
In recent years, with the rapid development of Massive Open Online Course (MOOC), millions of students can have a more convenient and efficient learning experience [1]. However, MOOC always serves tens of thousands of students at the same time, which cannot satisfy the individual needs of the specific student group because they have different levels of learning skills. To address the problem, Small Private Online Course (SPOC) was developed as an alternative which can provide the learning resource in MOOC and perform the interaction between instructors and students in offline class [2]. Therefore, it is an important academic and educational problem to find the relationship between the learning behavior and learning outcome by machining learning methods, which analyze the log data of student learning behaviors on both MOOC platform and offline class [3, 4].
In this paper, we study how students engage in SPOCs through clustering and multiclassification algorithms and the relationship between their learning pattern and learning outcome. The main contributions of this paper are as follows:(1)Previous researches mainly focus on log analysis of MOOC. But the SPOC teaching mode is the important teaching method implemented by colleges in recent years because it combines online MOOC with offline traditional class and can achieve a better learning outcome than traditional MOOC. The study of SPOC-based learning behavior analysis is more conducive to the improvement of teaching effects.(2)The clustering algorithm is used to find out the learning behavior patterns of students; combined with years of teaching experience, four typical learning behavior patterns are summarized; the characteristics of different learning patterns are analyzed to provide the basis for individual teaching.(3)Previous studies focus on predicting students’ dropout, obtaining certificates, and other binary classification problems. This paper studies four classification problems (excellent, good, pass, and fail), which are helpful to distinguish the learning levels of students and lay the foundation for personalized teaching.(4)The multiclass classification framework that combines the error correcting output code (ECOC) method and genetic algorithm (GA) obtains good prediction performance.
The rest of the paper is organized as follows. Section 2 describes the related work on education data mining and learning analytics. Section 3 briefly introduces data sources and the preprocessing. In Section 4, four typical learning behaviors are presented in detail. Section 5 presents the proposed multiclassification approach. Section 6 compares the proposed approach with others. Finally, Section 7 draws conclusions and provides the future research trends.
2. Literature Review
There are several successful cases of how MOOC can be combined with flipped classroom [5–7]. A lecturer at the University of Regina tested the idea in one of his Electronic Systems Engineering courses. Students watched videos of lecture at their own time and own pace before the class time. Then they were assigned into groups to complete group and individual assignments in the classroom. The lecturer kept on monitoring the group performance and interacting with his students. The performance of students in this preliminary research was found to be positive. The lecturer summed up his experience that the nature of in-class assignment was critical to success. Assignments relying heavily on group interaction usually lead to an outcome of higher ability students dominating less able students. He suggested that individual assessments with group interactions may have an overall involvement [6].
In another case, Ng et al. studied the student readiness and their learning achievements after adopting flipped classroom teaching in a database class [7]. They analyzed the command types of each clickstream and the relationship between the activity rate in MOOC and midterm test results. The results showed a positive relationship between students’ activity rate in MOOC and their academic outcome.
On the other hand, a study conducted by the Stanford and Cornell University showed a positive relationship between the student engagement and the exam results on MOOC [8]. However, the linear relationships only hold up to a certain point. When the activity level increased around 80% score, the examination results showed nondirect association and the trend of diminishing marginal returns was applied with respect to the examination grade. In 2015, Hughes et al. conducted a research to analyze the attendance on MOOC and the impact on the final grades. The research treated the overall module attendance as the students’ engagement and found a generally positive trend of students’ attendance and examination outcome [9].
Recently, Qiu et al. analyzed the students’ demographics and learning activities including forum activities, time spent on videos, and assignments, which influenced students’ engagement in MOOC through the ordinary least squares (OLS) models [10]. A latent dynamic factor graph (LadFG) model was also proposed to predict the students’ assignment grade as well as certificate earners. The proposed approach performed better than several previous alternative predictive approaches, that is, logistic regression classifier (LR) [11], support vector machine (SVM), and factorization machines. Similarly, Xu and Yang [12] presented a two-step approach to find the surrogate exam-takers on MOOC, which, in nature, is a certificate earner prediction approach. They developed a function to divide all learners into three groups including certificate earning, video watching, and course sampling. Then an SVM-based algorithm was proposed to predict the certification earners in the certificate earning group. Bailey et al. [13] developed a logistic regression-based approach to predict the probabilities of assignments grade weekly to intervene the at-risk students, who were nearby the pass/fail border. In order to trade-off smoothness and accuracy, two proposed LR-based transfer learning algorithms were added a regularization term to minimize the difference of failure probability between consecutive weeks. Ezen-Can et al. [14] proposed an algorithm that combined a k-medoids clustering with the greedy seed selection method to understand the discussion forums automatically on MOOC, which allows automated discourse analysis and mining to better support students’ learning. Zhang et al. collected multisource heterogeneous data from two MOOC courses, “Data Structures and Algorithms” and “Introduction to Computers.” Then they analyzed students’ learning content by identifying important concepts in the courses, students’ knowledge was evaluated through students’ test data, and algorithms were adopted to predict whether students dropout. By doing so, personalized learning guidance is available by analyzing multisource data in MOOCs [15]. Yu et al. identified seven cognitive participation models of students based on their video clickstream logs and designed a classification algorithm based on k-nearest neighbor (KNN), SVM, and artificial neural network (ANN) to predict whether students can pass the exam [16].
In order to improve the teaching efficiency of traditional offline courses and MOOC courses, Meier et al. used course historical teaching data, including homework, quizzes, and midterm exams, to predict the performance of students in subsequent learning output (good/poor), to gain time for intervention in the early stage of teaching for poor students in UCLA [17]. Xu et al. designed an ensemble classification system with a two-layer structure to predict the changing learning status of students dynamically. And they proposed a data-driven method based on the latent factor model and probability matrix decomposition and found the relevance of the curriculum. By doing so, the forecast accuracy was improved [18]. Ulloa-Cazarez et al. [19] proposed a genetic programming (GP) algorithm to predict whether students can pass the final exam. In order to identify students who have difficulty in learning the “Digital Design” course, Hussain et al. developed the ANN and SVM by using the learning behavior data on the learning system to predict students with a learning difficulty, which, in turn, facilitates early teaching intervention [20].
3. The Data Set and the Preprocessing
Three online courses were involved in this study. We try to acquire learning behavior data from these students, such as video watching times, access parameter in MOOC (i.e., frequency and time slot), homework/test score, and the final course grades as well. Our work does not only provide an effective experimental review for the use of SPOC in different schools but also help to find out effective learning patterns of student in common.
The selected course was the “College Information Technology Foundation,” which is a compulsory subject for the undergraduate students in three schools. The course contents included computer hardware and software, database, multimedia, information security, and network. The course adopted the SPOC teaching mode which combines with the MOOC and offline traditional classroom. Students studied the basic knowledge by watching several short videos on the MOOC platform before traditional offline class. Then flipped class was executed. Furthermore, students were required to complete eight assignments and eight tests on the MOOC provided by the Chaoxing Company. There were 2,439 students from school A, 3,119 students from school B, and 3612 students from school C who participated in the course.
In the data set, the data of school A contains 43 fields, school B contains 89 fields, and school C contains 200 fields. The fields include basic information about students and teachers, viewing of teaching videos, completion of students’ tests, and final grades. Firstly, the unrelated features and redundant features are eliminated by feature selection, including information such as student number, professional class number, teacher ID, and discussion number. Secondly, the fields of data set are processed and converted, and then all features are normalized, including the watching time of each video, score of the homework/test, and the submission time of each test. Finally, the samples of students who had not watched the lecture video or completed the test on the MOOC platform are defined as invalid samples. The invalid samples, in turn, are removed during the data preprocessing. After data preprocessing, school A retains 2430 records containing 35 fields, school B retains 3107 records containing 82 fields, and school C retains 3612 records containing 153 fields. Finally, the data is classified into four classes according to the final grade. Class 4 is the students whose final score is between 0 (including 0) and 60, Class 3 is the students whose final score is between 60 (including 60) and 75, and Class 2 is the students whose final score is between 75 (including 75) and 85. And the students whose final score is between 85 (85) and 100 are classified as Class 1. Table 1 shows the number of students for different classes in three schools.
4. Learning Behavior Analysis by Clustering Algorithms
Video watching time and tests completed are two essential aspects of student learning behavior which can affect the learning performance. Data related to video viewing behaviors includes total video viewing time and the percentage of viewing time of video 1 to video 11. The data related to tests’ completed behaviors include test progress, scores from test 1 to test 8, and submission time from test 1 to test 8. In addition, the number of visits to the student’s MOOC platform also reflected the student’s learning behavior in some way. Clustering algorithm has been succeeding application in bibliometric data analysis, text classification, and so on. In order to identify the learning behavior patterns, the k-means algorithm is adopted to cluster students with different learning behavior patterns from school A; combining the experience of years in blended teaching, four typical learning behavior patterns were found, as shown in Table 2. The similar learning behavior patterns can also be found in schools B and C, but the proportion of the four types of students is different from that of school A due to different data distribution. For the sake of simplicity, we only introduce the detailed learning behavior of four categories of students in school A.
The engagement of “MOOC-disinterested” students in online learning tasks is not high, and their completion of video watching is not so satisfied. The viewing time of video 1 to video 11 of these students does not reach the average viewing time level of all the schools; many of the students did not complete the test; they were not enthusiastic to complete the test; this became worse along with the time passed; their persistence of conscious learning was not long. However, the average final grade score of “MOOC-disinterested” students ranked second among the four learning patterns. According to manual analysis afterwards, we found that this type of students mainly came from science and engineering disciplines, and they had an advantage in studying the computer course of “College Information Technology Foundation.” Therefore, even if this kind of students completed the online learning tasks casually, they can still get good grades.
“Self-motivated” students had longer video watching time and higher test scores. They were self-driven in learning from the beginning of the course and persisted until the end of the course. This learning pattern is the most ideal. These students can finish watching the course videos as requested by them. After learning the content of each part, they can also complete the after-class quizzes in time, to review what they had learned. The highest average final score of this type of student is 70.1 points.
“Video-preferred” students had a good performance in watching videos, and almost all the 11 teaching videos can be watched completely. Most of the students submitted the test in December 2016, and some of them submitted the test in January 2017 (the deadline of MOOC course). Although the number of people who did not submit the test is lower in the beginning, the number increased along with the MOOC proceed, which shows that their motivation to complete the test had decreased over time. In addition, the average test score of these students is failed. It can be seen from these phenomena that this type of student had better learning habits, are able to take the initiative to watch the course videos, and can complete the test in time. But they may not pay attention to the after-class quizzes and regard completing the test as a mandatory task. As a result, their score is the third in four patterns. Therefore, the average final grade score of such students is just over the passing line, which is 62.6 points.
The video viewing time, the number of tasks completed, and the average test score of the “task-oriented” students ranked first. The test submission time of this type of student was mainly limited in January 2017, which was the deadline for submission. Most of these students did not complete the test in time to consolidate their knowledge. They completed the test to meet the teacher’s request only. The average final exam score was 53.3. This indicates that this type of student completes online learning tasks in a “task-oriented” style, so the learning effect was poor, and the average final grade of them failed to pass.
The majority of “task-oriented” students cannot pass the final exam, so teachers needed to remind and supervise these students so that they can watch the teaching videos in time and complete homework to help them pass the final exam. Furthermore, in the teaching of the new semester, the current result can be used to carry out corresponding teaching interventions to various types of students, thereby improving the quality of teaching and learning.
4.1. Comparison of Video Watching Behavior of Four Classes
The average percentage of each video viewing time of the four types of students is shown in Figure 1. Figure 1 shows that the viewing time of the “MOOC-disinterested” students is different from the other three types significantly, while the viewing time of the three clusters of “self-motivated,” “video-preferred,” and “task-oriented” is similar. Ranking all clusters by descending order of video viewing time, “task-oriented” is the first, while “self-motivated” and “video-preferred” follow, and “MOOC-disinterested” is at the bottom. Most of the “MOOC-disinterested” students did not finish watching all videos, while the viewing time of other types basically exceeded 1 in each video, which means these three types of students watched all the videos, and some of them watched the videos repeatedly. However the students in the cluster of “MOOC-disinterested” were not willing to watch the videos, and nearly all the course videos were not finished.
The up and down trend of the four kinds of students were relatively consistent in video viewing curves. All students watched videos 3, 8, and 9 for a long time. This may due to the fact that the content of these three teaching videos involves algorithms, database technology, and basic computer structure. It is difficult to learn and students need to be watched repeatedly.
4.2. Comparison of Quiz Behavior of Four Classes
The average scores of the four kinds of students in tests 1–8 are shown in Figure 2. Figure 2 shows that the gap among the four groups in test scores is more obvious than in the video viewing behavior. The test scores of “self-motivated” and “task-oriented” are higher, and the scores of each test are relatively close, while the test scores of the “MOOC-disinterested” and “video-preferred” are significantly different from the “self-motivated” and “task-oriented.” The ranking of test scores from top to bottom is “task-oriented,” “self-motivated,” “video-preferred,” and “MOOC-disinterested.”
The scores of “MOOC-disinterested” students are the lowest in each test, and the scores show a decreasing trend. This shows that the “MOOC-disinterested” students were less and less motivated to complete the test over time, and their continuity of self-driven learning is not long. The test scores of “video-preferred” students are higher than those of “MOOC-disinterested” students and rank third place. The fluctuation of each test is small, but they failed in each test. This shows that although “video-preferred” students are able to persist in completing the test, the quality of completion was not high. The scores of the “self-motivated” and “task-oriented” students in each test are relatively close, reaching above 80 points. The “task-oriented” group is slightly higher than the “self-motivated” group, and the test scores of these two types of students show an increasing trend. It indicates that the two kinds of students can persist in completing the test, and the quality of completing the test is high.
Figures 3–6 further compared the time taken by four students to submit tests.
Among the “MOOC-disinterested” students, the number of students who submitted the tests (test 1 to test 8) early is quite low and it decreases in later tests, while the number of students who did not submit tests is large and gradually increases. This verifies the assumption above that the enthusiasm of “MOOC-disinterested” students to complete the test is lower and lower over time, and the continuity of self-driven learning was not long.
It can be seen from Figure 4 that, in the “self-motivated,” a large number of students submitted the test in December 2016. These students reviewed and tested the knowledge they learn in time after completing the corresponding course. This shows that they have a good learning attitude and high self-consciousness. From test 1 to test 8, the number of students submitted in each period is basically the same (the number of people who submitted the test in November 2016 decreases slightly, which is related to the curriculum arrangement of course, which means the dates of the test). In addition, “self-motivated” students completed all tests, which shows that such students are more self-driven and their learning enthusiasm did not fade over time.
Figure 5 shows the test submission time of “video-preferred” students. Among the “video-preferred” students, the majority of them submitted the test in December 2016, but not as many as the “self-motivated.” Besides, the number of students who submitted in January 2017 is quite large, and the proportion is greater than the “self-motivated” significantly. It also can be seen that the number of students who had not completed the test in the “video-preferred” is more than that of the “self-motivated,” but much less than the “MOOC-disinterested.” This means that although the “video-preferred” students are not as motivated as the “self-motivated” students, most of them had completed the tests and their study habits are better.
Figure 6 reflects the submission time of the “task-oriented” students’ tests. It can be seen that the vast majority of students had submitted the tests, and only a few had not submitted the tests. The completion of the online tests is relatively high, but in the latter part of the course, the number of students who submitted in January 2017 accounts for the largest proportion, and from test 1 to test 8, the number of students who submitted tests gradually decreases in December 2016, and the number of students who submitted tests gradually increases in January 2017. This shows that the “task-oriented” students can basically complete the test but not in time. Most of them finished the test before the deadline for submitting the tests on the MOOC platform. The enthusiasm and consciousness of learning need to be improved. Although the average score of the “task-oriented” students’ test ranks the top of the four types, this is not a learning habit worth to be promoted. It can be assumed that “task-oriented” students complete the tests just to meet the teacher’s request.
By the analysis of Figures 4–6, it can be found that the learning habit of “self-motivated” students’ is the best, and almost all of these students had submitted the tests. The learning habit of “video-preferred” students is the second, and a small part of them had not completed the tests. Most of these two types of students can review and test their knowledge in time after completing the corresponding courses. The students of the “task-oriented” had a corresponding high degree of completion of the online test, and almost all of these students had submitted all the tests. However, most of them submitted the tests in a rush when the deadline approached, and the self-driven learning of them is not as good as the “self-motivated.” Their main purpose of completing the tests may be meeting the teacher’s request only. The completion of “MOOC-disinterested” students is the worst in tests. Many students had not completed all the tests, the enthusiasm for completing the tests got lower and lower over time, and the continuity of self-driven learning does not last long. These students may not care whether they can pass the course of “College Information Technology Fundamentals.” Also, they were not very enthusiastic in online learning.
4.3. Comparison of Visiting Behavior of Four Classes
Figure 7 shows the average visit of four kinds of students. The horizontal axis indicates four types of students ranked by the average visits. And the vertical axis represents the number of average visits. The number of the “task-oriented” is significantly higher than that of the others. Among the other three types, the total number of visits is ranked as follows: “video-preferred,” “self-motivated,” and “MOOC-disinterested.”
5. Grade Prediction by ECOC-Based Classification Algorithms
5.1. ECOC Algorithms
There are two common categories of machine learning algorithms which have been widely used: classification and clustering. Classification can be divided into binary-class and multiclass classification problems. For binary-class problems, many excellent machine learning algorithms have been developed, while for multiclass classification problems, most of the data set cannot be fit effectively by a single classification algorithm; therefore, “partition” is adopted. The “partition” decomposes the multiclass classification problem into several binary-class problems and then uses the base classifiers to perform binary-class classification. This helps to solve the multiclass classification problem effectively.
ECOC algorithm was developed for solving multiclass classification problems, which has been successfully applied in different fields, such as face recognition [21] and bioinformatics [22–24]. It comes from the signal transmission problem in communication. The process of the ECOC algorithm can be divided into three basic steps: encoding, training, and decoding [25]. The key process of the ECOC algorithm to deal with multiclass classification problems lies in the selection of matrix encoding method and decoding method. Encoding matrix algorithms can be mainly divided into two categories: data-independent and data-dependent. Data-independent encoding algorithms do not rely on data samples in data encoding. There are mainly One VS One (OVO), One VS All (OVA) [26], Dense Random (DR) [27], and Sparse Random (SR) [28] encoding matrix. Data-dependent encoding algorithms have different encoding matrices for different data sample distribution. Such encoding methods include DECOC, Forest-ECOC, and ECOC-ONE [29]. The ternary coding strategy is widely used in ECOC, where each element can take a value in the set {0, +1, −1}. At this time, +1 or −1 denotes that the original class is assigned to the positive class group or the negative class group, respectively; 0 denotes that there is no corresponding class in the training process, which means the class is ignored.
5.2. GA-Based Feature Selection Methods
In general, there are some redundant or irrelevant attributes (features) in the original data set. In order to avoid the curse of the dimensionality and improve the classification accuracy of the model as well, feature selection is considered as an effective method to pick up most representative subset of features from the data set. In this paper, GA is employed to realize the feature selection method because it has powerful searching ability [30] and been successfully applied in feature selection [31, 32]. Thus, GA is developed to construct an efficient ECOC ensemble system with feature selection.
5.3. The Design of GA-ECOC Classification Method
This paper proposes a multiclass classification algorithm framework based on GA and ECOC for student grade prediction, namely, GA-ECOC. In the framework, GA is used as a feature selection method. The feature of data set is encoded by binary chromosome. Each bit of the chromosome represents a feature: 1: the feature is selected, and 0: the feature is not selected. The school A data set has 35 fields, school B has 82 fields, and school C has 153 fields. For the samples from 3 schools, the remaining fields are used as the characteristics except the final grade of each sample. The GA has two operations: crossover operation and mutation operation. The crossover operation is selecting two individuals randomly from the population and randomly selecting one of the chromosomal positions as the crossover position and crossover to obtain two new individuals. The mutation operation is selecting a bit randomly from each individual to mutate according to a certain probability. The average classification accuracy of ECOC is used as the fitness function of GA. And the method sorts the population individuals according to their fitness; the top 20% of individuals with high fitness are selected firstly. Based on the random selection probability, the remaining individuals having a 50% probability are selected to produce the next generation. In order to verify the effectiveness of the proposed approach, tenfold cross-validation is employed for all experiments; the data set is divided into 10 folds; one of the folds is used for testing and others are used as training data in each interaction. In the experiment, SVM classifier is used as the base classifier in ECOC, and the average accuracy of all interactions is used as the individual fitness. The framework of the methodology is shown in Figure 8.
6. Experiments and Discussions
This paper uses the learning behavior data of three schools to predict students’ grade. The parameters of the experiment are set as follows: the initial population of the GA is 100, the number of iterations is 50 generations, the mutation rate is 0.01, the optimize selection rate is 0.2, and the random selection rate of the remaining individuals is 0.5. SVM is selected as the base classifier of ECOC, according to the average accuracy of 10-fold cross-validation. In the individual chromosomes, the length of the individual chromosomes algorithm of school A is 34 bits corresponding to the algorithm, while the length of school B is 81 bits and the length of school C is 152 bits. And the ensemble learning algorithm, Random Forest [33], is used for comparison.
6.1. Performance Comparison of Methods
Firstly, the prediction result of the ECOC and Random Forest algorithm is shown in Table 3. It can be seen from that, for the data set of school A, the average accuracy obtained by the ECOC algorithm was about 79%. For school B, the average accuracy obtained by the ECOC algorithm was approximately 70%, and for school C, the average accuracy is approximately 68%. What is more, all the ECOC algorithms can perform better than the Random Forest in three data sets.
In further experiments, the result of the multiclass classification algorithm (GA-ECOC) is shown in Table 4. The result can be analyzed in two aspects, accuracy and encoding methods. Firstly, for accuracy, the average accuracy of school A is about 80%, while the average accuracy of school B is about 75% and the average accuracy of school C is about 74%. Secondly, for the encoding methods used in three data sets, the SR encoding method achieves the highest accuracy, followed by the DR encoding. OVA encoding and OVO encoding have poor performance in accuracy, compared with the two methods above. Finally, the result of the data set from school B and school C shows that the accuracy rate of school B is about 5% lower than that of school A, and the accuracy rate of school C is about 6% lower than that of school A. The reason may be that the feature dimension of school B data is twice as the dimension of school A, the feature dimension of school C is even higher, schools B and C have more space for optimization than school A, and therefore the classification accuracy of the model has a significant decrease.
The results of the two algorithms can be compared in three data sets. It can be seen from Figure 9 that, for school A data set, the accuracy rate obtained by the GA-ECOC algorithm is about 2% higher than those obtained by the ECOC algorithm. Figure 10 indicates that, for the school B data set, the accuracy of the GA-ECOC algorithm is improved by about 5%, compared with the ECOC algorithm. Figure 11 indicates that, for school C data set, the accuracy of the GA-ECOC algorithm is improved by about 6% on average, compared to the ECOC algorithm.
Based on the experiments above, the classification performance of GA-ECOC algorithms is significantly better than the ECOC algorithm in all data sets. The main reason may be that some features in the SPOC data are not related to student performance or redundant, and some features have a strong correlation with student performance. GA is able to select an effective feature subset, which helps to improve ECOC Classification accuracy. Therefore, feature selection can not only improve the classification accuracy effectively with four different ECOC matrix encoding methods (OVO, OVA, DR, and SR) but also improve the classification accuracy significantly in different school data set.
6.2. Analysis of Computational Complexity of Methods
Table 5 shows the comparison of the time complexity of different algorithms. The training time of the GA-ECOC algorithm is much longer than the ECOC algorithm. In the data sets of the three schools, the ECOC algorithm is used with different codes. They only use less than 90 seconds to complete the training process, which means that the algorithm complexity is lower. After introducing GA as a feature selection method, multi-iteration is conducted, which is computationally expensive, to find the optimal feature subset. Most of the algorithms required more than 4 hours to complete model training. What is more, the complexity of GA calculation is different from data sets with different feature numbers. School C has 152 features, the individual chromosome is the longest among three schools, the feature dimension to be searched is quite large, and the computing time is the longest; school B has 81 features, which are fewer than those in school C, the feature dimension to be searched is small, so the computing time is less. The data of school A has only 34 features. With the fewest features, the computing time is the shortest. It takes around 4 hours only to complete feature selection and model training. On the other hand, different ECOC encoding methods have different calculation costs. OVA and OVO matrix coding are basic relatively, and the corresponding matrix generation method is simple; while DR and SR are the method with random coding, the matrix generation method is complicated, and it is easy to cause coding problems, which have to debug and rerun until a satisfactory result is acquired, so more time is required. Therefore, the calculation time of GA-ECOC based on DR and SR is longer than that based on OVA and OVO.
7. Conclusion
With the rise of SPOC blended teaching, using student online/offline learning behavior log data to analyze and predict students’ learning performance is an important factor for personalized education. This paper analyzes the learning behavior of students in SPOC blended teaching in three schools by the clustering algorithm, ECOC algorithm, and GA-ECOC algorithm. The main findings are shown as follows:(1)By the clustering algorithm, combining with the teaching experience accumulated by staff, typical learning patterns of students can be classified into four clusters. The learning behavior patterns among the four clusters are significantly different. Learning the patterns is quite helpful in curriculum design and providing personalized guidance for students.(2)A multiclass classification algorithm based on GA and ECOC is designed, in order to predict student performance. The powerful searching capability of GA in feature selection and then ECOC algorithm makes a good performance in multiclass prediction. Compared with the Random Forest and single ECOC algorithm, the classification prediction of GA-ECOC algorithms is more precise and effective.
The next-stage work can be considered from the following aspects:(1)For data collection, more teaching-related data should be collected, such as students’ previous academic performance and students’ offline learning behavior in order to have a full picture of students’ learning process.(2)The algorithm should be designed to identify different learning behavior patterns and predict students with different learning behavior patterns in advance. This can help to provide different suggestions to the staff, according to different students; therefore, the teaching quality and student learning experience can be improved.(3)For the design of the multiclass classification algorithm, the types of the coding matrix generated are limited by the method of the traditional coding matrix, and the best classification performance cannot be achieved. Therefore, the searching method of feature subset and method of coding matrix generation should be optimized, and this will become an essential factor in future research.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (no. 61772023), Key Research and Development Program of China (no. 2019QY1803), Natural Science Foundation of Fujian Province of China (nos. 2020J01697, 2020J01707, 2020R0066, and 2018J01538), and the Opening Fund of Digital Fujian Big Data Modeling and Intelligent Computing Institute.