Abstract
Hybrid online and offline teaching is becoming the mainstream teaching method in the postepidemic era. However, research on assessing its teaching quality is still limited. This article thus develops a teaching quality evaluation model based on the BP neural network. A three-dimensional indicator system involving 19 indicators is set in the model. The established model was demonstrated and validated by a case study. The results show that the developed model can accurately assess the teaching quality of hybrid online and offline teaching. Findings from this study can provide valuable references for improving the quality of hybrid online and offline teaching.
1. Introduction
In the digital era, digital technologies represented by the new generation of Internet technology have significantly reduced the cost of information dissemination. Students can, therefore, easily access a vast amount of high-quality teaching resources through the Internet. In this context, both “teaching” and “learning” are given a new connotation of the times [1]. A lot of online teaching platforms like Coursera, Udacity, and edX have emerged. More and more schools, including some top universities like Harvard and MIT, have opened their public courses. Online teaching is beginning to be accepted by the public and is increasing [2]. Especially in the wake of the 2020 New Crown Pneumonia outbreak in countries around the world, online teaching has become a choice many universities have to make in the context of epidemic prevention and control [3]. Even though the epidemic has been initially controlled in some countries and regions, the situation of epidemic prevention and control is still very serious [4]. At present, more and more universities select hybrid online and offline teaching for daily teaching tasks, such as Hong Kong Polytechnic University, Sydney university, and so on. The Ministry of Education has issued policy documents such as “Action Plan of Education Informatization 2.0” and “Opinions on Strengthening the Application and Management of Online Open Courses in Higher Education Institutions.” And at the same time, the Ministry of Education has held a series of meetings such as the “National Conference on Higher Undergraduate Education in the New Era” to promote the construction of online resources of various courses [5]. It encourages universities to actively build online resources of multiple classes and promote the deep integration of modern information technology and education [6]. In November 2020, the first batch of 5118 national first-class undergraduate courses, including 868 online and offline hybrid first-class courses, was launched. Facing the new challenges in the postepidemic era, the large-scale hybrid online and offline teaching tends to become the mainstream mode of teaching in the future [7].
Even though the mode of hybrid online and offline teaching has obtained certain development and achievements, there are still many teachers, students, and experts who doubt that the online and offline hybrid teaching mode cannot achieve the expected teaching effect and thus negatively affect the personnel training of the universities. Prospective research on the evaluation of teaching quality of hybrid online and offline teaching is apparent [8]. However, at present, systematic and comprehensive assessment indicators and approaches are unavailable.
To address the research gap, this study proposes a process model and develops an indicator system for assessing the teaching quality of hybrid online and offline teaching from the perspective of the whole process of curriculum construction, teaching implementation, and teaching effects. An evaluation model based on BP neural network is also developed. A lecture called Introduction to Machine Learning in a university in Shanxi, China, is selected for a case study to demonstrate and validate the established model. On the one hand, findings from this study can make up for the deficiency of emphasizing teaching process evaluation rather than course evaluation in current research on hybrid teaching quality evaluation. On the other hand, it provides suggestions and improvement direction for similar colleges to perfect the construction of related curriculums and improve the quality of hybrid online and offline teaching.
2. Related Works
From the late 1990s to the present, scholars have experienced three stages of technology-centered, teacher-centered, and student-centered cognition of hybrid teaching, which is a teaching mode combining online and offline teaching. Most scholars believe that the online part of blended teaching includes MOOC teaching, SPOC teaching, APP teaching, and other forms [9, 10], and the offline part is mainly face-to-face teaching in classrooms, but the application of modern digital technology should be emphasized. Baragash and Al-Samarraie conducted a large-scale survey on perceptions of hybrid teaching and learning in US colleges and universities [11]. They found that faculty perceptions of hybrid teaching and learning have moved beyond technology integration to a higher level of content and effectiveness improvement. Laura et al. suggest that teachers and students derive greater psychological satisfaction from hybrid online and offline teaching than from traditional instructional programs based on cognitive theory [12]. Furnes et al. found that a 2-year follow-up study of a hybrid online and offline teaching program for undergraduate students at the University of Miami [13]. They also found that the hybrid online and offline teaching program significantly improved students’ communication and innovation skills and was a highly effective teaching model.
Teaching quality is a direct reflection of the effect of educational activities [13, 14], and global scholars have conducted a lot of research on teaching quality evaluation, covering various aspects, such as influencing factors, dimension classification, evaluation indexes, and methods [15–19]. In terms of factors influencing online teaching quality, Xu et al. identified interaction (including the interaction between students, the interaction between students and teachers, and the interaction between students and contents), online self-efficacy, and self-regulated learning habit as essential factors influencing the quality of hybrid online and offline teaching for university students [20]. In terms of hybrid teaching quality dimension division, Huang et al. pointed out that during the new crown pneumonia epidemic, the construction of an online teaching quality assurance system in national universities showed comprehensive, diversified features from three dimensions of supervisors, teachers, and students [21]. Scholars have also proposed using big data methods and machine learning methods to convert macroscopic, qualitative evaluation to microscopic, quantitative evaluation [19, 22, 23].
In summary, in the face of the rapid advancement of digital technologies and the impact of the new coronary pneumonia epidemic, worldwide scholars continue to be enthusiastic about hybrid online and offline teaching and learning and have achieved specific achievements in the research of hybrid online and offline teaching quality assessment. However, there are still the following deficiencies:(1)The hybrid online and offline teaching mode is becoming more affluent, especially with the development of mobile Internet, which further promotes the diversification of hybrid online and offline teaching. But the current research on the evaluation of hybrid online and offline teaching quality has not yet paid attention to this new change.(2)The existing research on the quality of hybrid online and offline teaching mainly starts from the perspective of the general process of teaching activities, ignoring the impact of hybrid online and offline teaching courses as the basic unit and new mode teaching mode on educational effects.(3)The hybrid online and offline teaching quality evaluation indicator system established by the existing research is relatively brief, ignoring the key index points that affect the quality of hybrid teaching, such as course objectives, course philosophy, and teaching team, which affects the completeness of the hybrid teaching quality evaluation system.(4)The current research on the implementation strategy of hybrid teaching has not yet formed a good echo with the research on teaching quality evaluation, and the suggestions on the implementation strategy of hybrid teaching quality evaluation based on the expansion of practice level will help the application and promotion of the hybrid teaching.
3. Development of Evaluation Indicator System
3.1. Principles
Hybrid online and offline teaching is based on constructivist learning theory and emphasizes student-centeredness. Currently, the development of 5G technology and the popularity of mobile Internet have expanded hybrid teaching to new forms, such as mobile Internet APP, and the basic elements involved in hybrid teaching, such as teaching resources, are also expanding. This paper proposes the principles that should be followed in constructing a hybrid teaching quality evaluation indicator system from the following aspects.
3.1.1. Integration of Course Evaluation and Implementation Evaluation
The quality evaluation system of hybrid online and offline teaching must start from the curriculum as the high level of curriculum is the prerequisite and fundamental to ensure the quality of blended teaching. The prerequisite and foundation for the quality of hybrid online and offline teaching is a high-level curriculum. The evaluation of the implementation and feedback of hybrid teaching is carried out to grasp the fundamental elements of the curriculum in hybrid teaching. The quality evaluation system of hybrid online and offline teaching should be based on the whole teaching process, and the coverage of teaching quality evaluation index points should be improved.
3.1.2. Combination of Process Evaluation and Outcome Evaluation
The student-centered teaching concept requires that the evaluation of hybrid online and offline teaching quality should focus on process evaluation on the one hand and outcome evaluation on the other. The process evaluation needs to focus on the dimensions of teacher-student interaction, student-student interaction, and teaching contents, and its forms include learning behavior evaluation on the desktop and mobile end of the teaching platform, offline learning behavior evaluation, stage assignments, and so on; the form of outcome evaluation includes test papers, essays, defenses, and so on.
3.1.3. Diversification of Evaluation Subjects and Evaluation Indicators
Hybrid online and offline teaching includes both flexible online desktop teaching and mobile APP teaching and traditional offline, face-to-face teaching. Both teachers and students are deeply involved in the whole process of teaching activities, especially in the construction of hybrid teaching courses; the teachers’ overall design of the course has an important impact on the teaching quality. Therefore, in the selection of evaluation subjects, teachers, students, and experts inside and outside the university and the teaching platform should be included in the evaluation subjects, and the application of learning behavior data from the desktop and mobile ends of the teaching platform should be emphasized. As for the evaluation indicators, the diversified design of combining qualitative and quantitative indexes, objective data, and subjective judgment should be insisted on.
3.1.4. Comprehensive Evaluation Content and Evaluation Method
As the course construction and organization implementation process in hybrid teaching are relatively complex, the quality evaluation needs to be more comprehensive and detailed, focusing not only on the all-round examination of the course construction but also on the evaluation of questions, tests, and assignments in the traditional offline teaching process, as well as the evaluation of video viewing, chapter tests, topic discussions, and other contents in the online teaching process. In terms of evaluation methods, evaluation can be carried out with the data from the online desktop and mobile APP teaching platforms, and phase evaluation and rolling dynamic evaluation can also be carried out according to the course teaching progress.
3.2. Indicator System
The key to constructing a hybrid teaching quality evaluation system is the selection and determination of evaluation indicators. This paper develops a hybrid teaching quality evaluation system containing three first-level indicators: course construction evaluation, teaching implementation process evaluation, and teaching effect evaluation.
Firstly, in the evaluation of hybrid teaching course construction, combined with the specific elements contained in the course, we focus on whether the system meets the basic construction standards and, at the same time, has the essential characteristics of hybrid teaching. Specifically, seven secondary indicators such as course objectives, teaching content, and teaching design are evaluated comprehensively and should be considered by different evaluation subjects because of the difference in the connotation of secondary indicators of different dimensions. The indicator system is shown in Table 1.
Secondly, in evaluating the teaching implementation process of hybrid teaching, we aim to improve the interactivity and build a teaching process evaluation system based on the interaction between “teaching” and “learning.” Specifically, it includes four secondary indicators, namely, students’ online session, teachers’ online session, students’ offline session, and teachers’ offline session, as shown in Table 2.
At last, in the evaluation of hybrid teaching effect, the teaching effectiveness of both online teaching and offline classroom is combined to make a comprehensive judgment. Specifically, from two dimensions of process assessment and outcome assessment, eight secondary indicators are designed, including assessment objectives, assessment forms, assessment contents, and assessment quality, as shown in Table 3.
4. Assessment Methods
4.1. Model Development
Traditional assessment methods such as the single-factor evaluation method, the comprehensive evaluation method, fuzzy mathematics, operation research, multivariate statistical analysis, multidimensional scalar analysis, and so on have been used to evaluate the quality of hybrid teaching. Even though these methods could achieve certain results to different degrees, they were imperfect, mainly in the following aspects: (1) It is difficult to determine the weights of each secondary index, and the evaluation is usually subjective and arbitrary by virtue of experts’ experience. There is thus a specific error with the actual value. (2) It is difficult to make an accurate evaluation of the results of specific indicators by traditional methods. (3) The calculation is complicated, and the solution is tedious. (4) The algorithm lacks self-learning ability. Therefore, this study tries to obtain a fast, effective, and accurate method of hybrid teaching quality evaluation.
Artificial Neural Network, in the field of machine learning and cognitive science, is a mathematical or computational model that mimics the structure and function of biological neural networks (the central nervous system of animals, especially the brain) for estimation or approximation of functions [24]. A neural network consists of a large number of artificial neurons linked for computation. In most cases, artificial neural networks are able to change their internal structure based on external information and are adaptive systems, which are commonly known as having a learning function. Modern neural networks are a nonlinear statistical data modeling tool, and they are usually optimized by a learning method based on a mathematical, statistical type, so they are also a practical application of mathematical, statistical methods, which allow us to obtain a large number of local structural spaces that can be expressed as functions [25, 26]. On the other hand, in the field of artificial perception in artificial intelligence, we can do decision problems in artificial perception through the application of mathematical statistics (i.e., through statistical methods, artificial neural networks can have the ability to make simple decisions and superficial judgments like human beings), and this method has advantages over formal logical reasoning [27]. Like other machine learning methods, neural networks have been used to solve a wide variety of problems, such as machine vision and speech recognition. These are problems that are difficult to be solved by traditional rule-based programming.
BP neural network, also called “error backpropagation neural network,” provides a novel technique for teaching quality evaluation, which can effectively overcome the shortcomings of traditional evaluation methods. In essence, this is a category of dynamic information processing systems composed of a large number of information processing units through a wide range of linkages, and this system is unique in processing various kinds of paradoxical, ambiguous, random, large volumes dynamic, and low-precision information. It has the functions of learning, remembering, associating, summarizing, generalizing, extracting, guest error, and self-adaptive capabilities and is a system that can handle nonlinear problems.
A typical BP neural network is a three-layer feed-forward hierarchical network consisting of an input layer, an implicit layer, and an output layer. The learning process of a BP neural network consists of forwarding propagation of information and backward propagation of errors. When given a set of input patterns to the network, the BP network will sequentially learn this set of input patterns in turn in the following manner: first, the input pattern is transmitted from the input layer to the implicit layer unit, and after being processed by the implicit layer unit layer by layer, an input pattern is generated and transmitted to the output layer, which is called forward propagation [28]. The output result is then compared with the expected value [29]. If it does not meet the anticipated expectation, it is transformed into a backpropagation of the error, which returns the error along the original path and makes the error signal smaller by modifying the connection weights of the neurons in each layer. These forward and backward propagation alternate with each other and are seen as a “memory training” process. The system repeats these two processes until the error between the output value, and the expected value is reduced to within a specified range [30]. At this moment, the new samples are fed into the already trained network, and the corresponding output values are obtained.
The BP neural network model for hybrid teaching quality evaluation is established in the following steps.
4.1.1. Determination of the Number of Neurons in the Transmitter Layer
According to the evaluation indicator system of hybrid online and offline teaching quality, there are 19 subindicators in the system. These 19 subindicators can be used as the input neurons of the model. Therefore, the number of neurons in the input layer n is determined as n = 19.
4.1.2. Determination of the Number of Neurons in the Output Layer
This study uses the evaluation results as the output of the developed network and the number of output layers m = 1. The desired outcomes are grade A for excellent; grade B for good; grade C for pass; and grade D for fail.
4.1.3. Determination of the Number of Implied Layers of the Network
The indicated layer can be one or more layers. According to the theory of Kolmogorov, the following has been proved: for any given continuous function, a three-layer neural network can then accurately implement the predefined model functions. Therefore, in the hybrid online and offline teaching quality evaluation model, we choose the hidden layer as one layer.
4.1.4. Determination of the Number of Neurons in the Hidden Layer
In general, the number of neurons in the hidden layer is determined based on the convergence performance of the network. Too few neurons in the hidden layer may not train the network, or the obtained network may not be “strong” enough to recognize previously unseen samples, and the error tolerance is poor. However, too many neurons in the hidden layer may make the learning time too long, and the error may not be optimal either. Therefore, there is a problem with how to determine the appropriate number of neurons in the hidden layer of the network. Generally, the “trial-and-error method” can be used to compare the network output error with the desired error and select the simulation effect. The number of hidden layer nodes used for the best simulation results is chosen by comparing the fit between the output and desired errors, but this approach is tedious and time-consuming. The number of neurons in the hidden layer can also be determined by referring to some empirical formulas to determine. The following equations are two widely used empirical formulas.where a is a constant between 1 and 10, n is the number of neurons in the input layer, and m is the number of neurons in the output layer. In this paper, we initialize the number of neurons in the hidden layer based on the relevant empirical formula as s = 12.
4.1.5. Determination of Neuronal Transition Function
BP neural network neuron conversion function is generally chosen as sigmoid function:in which the coefficient determines the degree of compression of the sigmoid function.
4.1.6. Determination of the Model Structure
From the above results, it can be determined that the structure of the hybrid online and offline teaching quality evaluation model based on the BP neural network is shown in Figure 1.
4.2. Model Training
A lecture called Introduction to Machine Learning in a university in Shanxi, China, is selected for a case study to demonstrate and validate the established model. We conducted a questionnaire survey among 60 students of this lecture. Surveyed subjects are asked to mark the 19 indicators from 0 to 9. The three-layer BP neural network shown in Figure 1 was used for identifying the above hybrid online and offline teaching quality assessment system. The numbers of neurons in the input layer, implicit layer, and output layer are 38, 12, and 1, respectively, and the learning rate . 40 randomly selected samples of data were considered as the training set of the neural network. The target error is 0.001. The specific training process is shown in Figure 2.
4.3. Model Test
After the training of the network was completed, it was tested using the remaining 20 sets of data. Then, the error between the evaluation target value and the actual evaluation target value output by the neural network is checked and compared, and the comparison results are shown in Table 4. It can be learnt that the output values of the hybrid online and offline teaching quality evaluation model built based on the BP neural network are very close to the real values. In other words, the model can accurately determine the teaching effectiveness based on each evaluation indicator.
5. Conclusions
The hybrid online and offline teaching quality evaluation system is a complex nonlinear system with many uncertainties between the input and output. The BP neural network model can effectively overcome the defects of traditional evaluation methods and weaken the human influence factors of index weight determination in traditional evaluation methods because of its highly nonlinear function mapping function and self-adaptive and self-learning ability. It is not only feasible but also has high accuracy. After the empirical study, we found that the error between the output value of the BP neural network model and the real value is relatively small. The performance can fully meet the requirements of practical applications. In addition, the output accuracy of the network depends on the number of input training samples, and the more the number of training samples, the closer the output teaching effect evaluation value is to the actual evaluation value. In conclusion, the developed hybrid online and offline teaching quality evaluation model established based on the BP neural network is expected to provide helpful reference for universities and teaching management departments to seek scientific solutions for teaching quality evaluation and improvements. This research still has two main limitations: (1) the evaluation model is still superficial; (2) only a case is applied to validate the model. In the future, the evaluation model can be further improved by introducing big data and deep learning. In addition, more cases should be used to further validate the developed indicator system and evaluation model.
Data Availability
The labeled datasets used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This study was sponsored by Xinjiang Normal University undergraduate teaching project, research, and practice of online and offline mixed teaching mode of minority preparatory Chinese language (no. SDJG2020-15).