Abstract
Higher vocational education has developed rapidly and attracted wide attention. Teaching quality evaluation is an important part of teaching management in classroom teaching and is an important part of teaching management. Based on the theory of teaching quality, 16 evaluation indicators are identified from the four aspects of teaching attitude, teaching content, teaching process, and teaching results and an HVE quality evaluation index system is constructed. The evaluation model is established by BP neural network algorithm. Finally, five higher vocational colleges in Henan Province are selected for empirical research. The results show that the evaluation model constructed in this paper has certain applicability and can provide a reference for HVE quality evaluation and improvement. Although the evaluation models herein have certain applicable values, further improvements are still needed in terms of indicator identification and correction.
1. Introduction
With the continuous development and progress of society, Higher Vocational Education (HVE) obviously has become an important part of the development of higher education. Improving the quality and development connotation has become a new focus, and higher education is increasingly becoming a concern. HVE is vocational education and technical education belonging to the level of education. Includes professional technical education before employment and postemployment-related continuing education. For example, some teaching programs of the American Institute of Technology and the Community College [1], Japan's short-term university partial teaching plan and specialized courses [2], French technical colleges and advanced technicians [3], Chinese early higher industry schools, specialized schools, specialists Schools [4], etc., as well as the education provided by some teaching plans such as adult colleges and universities. China began new development in the 1980s mainly cultivating liberal arts, science, engineering, agricultural and forestry, pharmaceutical, political and law, and finance and managerial professional assistant talents. Teaching quality assessment can provide a basis for improving the quality of running schools in higher vocational colleges, and establishing a foundation for the teaching quality assurance system.
Different subjects of HVE have different views on teaching quality. In the eyes of students, students are the main body of education, and all teaching behaviors are aimed at students’ participation in teaching activities and are process-oriented. Teachers provide students with knowledge and skills through classroom teaching and develop students’ values. For teachers, teaching ability and teaching level are the direct factors affecting the quality of teaching. Teaching behavior plays a guiding role in students, and it is the teacher’s control to cultivate students’ quality. The school level believes that the school’s concept of talent training and the theme of running a school affect the quality of teaching, and the school’s control of education quality is mostly reflected in the two levels of teachers’ teaching and student participation. The social level believes that teaching quality is the satisfaction and adaptability of the public to the development of students. The teaching process should meet the development and needs of the society, and cultivate different talents in the society as the primary goal while ignoring the individual development of students. In summary, the quality of teaching should be a comprehensive consideration of the feelings of both students and teachers, and the quality of students directly reflects the level of teaching quality.
Teaching quality evaluation is to monitor teaching through a series of evaluation standards and control methods, obtain relevant data according to the evaluation standards, and build a reasonable evaluation index system. Guided by quality evaluation, teachers can form inertial thinking, optimize their own teaching behavior, and improve teaching effects. The evaluation of teaching quality is an activity that takes students and teachers as the main body and is guided by teaching goals. It is necessary to obtain correct evaluation results through technical analysis [5]. Therefore, it is very necessary to use relevant mathematical models to evaluate the quality of HVE.
HVE is in a period of economic transformation and development, due to the adjustment of industrial structure, it needs to be gradually perfected and improved. At present, HVE still has problems such as unclear training objectives, an imperfect education system, and differences in quality evaluation systems [6]. Lei and Zhong pointed out that in the teaching process of HVE, it is necessary to pay attention to the rational application of teaching evaluation and to use the multiple intelligence education quality evaluation systems to ensure the scientific and rational evaluation [7]. Li et al. and others analyzed the particularity of higher vocational education through research. The improvement of students’ application ability is an important pursuit in the teaching process. At present, HVE needs to be improved in terms of teaching content, methods, teachers, and evaluation system. To meet the needs of higher vocational colleges [8]. Zhou explores students’ ability development and its influencing factors in practical curriculum-oriented courses in higher vocational colleges through a fuzzy comprehensive evaluation method [9]. Miao started from the problems faced by the teaching reform of art courses in higher vocational colleges, established an evaluation index system from the evaluation of teaching reform ability, and used gray theory to build a teaching ability evaluation model [10].
In terms of research on constructing the evaluation model of HVE teaching quality, Liu and Lin constructed an evaluation index system from three aspects of teachers’ technical ability, practical ability, and social ability under the background of the integration of production and education [11]. Kalkan and Cosguner have shown that according to the calculation results of the artificial neural network, academic self-efficacy is the most critical variable that affects students’ achievements [12]. Based on improving BP neural network, the high vocational university performance innovation education model has been built by Zhang an evaluation index system from school supervision, teachers, and student education effects. Based on this, Fourier technology is used to improve the algorithm of the BP neural network to simplify the model [13]. In order to measure the quality of college physical education, Feng (2021) has built a teaching quality index system with big data technology and gives artificial intelligent quality data to construct a teaching quality assessment model [14]. The basic characteristics of teaching attitude, teaching content, teaching methods, and teachers, build a postgraduate teaching quality assessment index system and use BP neural network algorithm to build evaluation models, application sensitivity test identification key indicators [15]. Wei et al. has built a graduate nursing professional degree education quality assessment index system from the four aspects of input quality, process quality, output quality, and development quality, and empowering weights through Delphi law and level analysis [16]. Liu and Yin proposed a hybrid intelligence algorithm based on a genetic algorithm and reverse propagation neural network to assess the quality level of teaching, and play an intelligent algorithm in the evaluation model construction [17]. Considering the advantages of BP neural networks in evaluation [18], this study uses BP neural networks to analyze HVE evaluation indexes to establish a better evaluation index system and provide more accurate results.
This paper takes students and teachers as the main body of HVE, and constructs the quality evaluation index system of HVE from four aspects: teaching attitude, teaching content, teaching process, and teaching achievement. The HVE evaluation model is constructed based on the BP neural network algorithm, and empirical research is carried out with five universities in Henan province as the research object to verify the rationality and scientificity of the model constructed in this paper.
2. Evaluation Index System of HVE
The quality evaluation of HVE is guided by educational theory and teaching quality evaluation theory. Based on literature identification and brainstorming, this paper adopts the Delphi method to correct the evaluation index, and finally forms the quality evaluation index system of HVE. The construction of the evaluation index system is the core content of HVE teaching quality evaluation, and the scientific nature of the evaluation index system is the key to HVE teaching quality evaluation. In the process of teaching quality evaluation work, it is often necessary to pass the evaluation of peer teachers, students, and experts. In the evaluation process, teachers pay more attention to teachers’ classroom performance, whether they can design teaching content based on the characteristics of the discipline, and whether they can combine theory with practice to achieve the purpose of higher vocational education. Expert evaluation mainly starts from the characteristics of the discipline, and examines whether teachers’ teaching thinking and teaching content are closely related to the needs of social development. Expert evaluation is usually carried out in the form of lectures and courseware reviews. The evaluation from the student’s point of view pays more attention to the content of the class and the conclusion after class. Students can understand the quality of teaching more intuitively by participating in classroom practice, and the teaching level of teachers is reflected in whether the knowledge imparted can be better absorbed and transformed by students. Teachers can also use evaluation indicators and evaluation standards as a reference to continuously adjust their teaching thinking, teaching content, and teaching methods to promote their own ability. It is the mission of higher vocational colleges to cultivate graduates with vocational skills. Therefore, in the construction of the quality evaluation index system of higher vocational education, it is necessary to focus on the interface between the teaching content and the needs of social talents.
The HVE evaluation index system is an evaluation of the teacher as a higher vocational education, from teaching preparation, teaching content, teaching process, teaching results, etc. As a multi-dimensional work, teaching quality evaluation should be a result of interacting with multiple factors. After preliminary identification and screening of the indicator, this article determines the first level indicator in the HVE evaluation index system as teaching attitude, teaching content, teaching processing, and teaching achievement, as shown in Figure 1. Teaching Attitude expressed the teacher’s teaching work and attitude toward students. Teaching Content and Processing reflects the quality and ability of teachers to show in higher vocational education. Teaching achievement refers to the effects of students who have rendered after the education activities. The above four aspects can be more comprehensive to evaluate HVE.
Teaching attitude is an important factor in measuring HVE teaching value and teaching effect. The intuitive manifestation of teaching attitudes is whether the teacher can reach the teaching work on time, which reflects the self-discipline and integrity of teachers. The effect of the preparation determines the performance of teachers during the teaching process, and it is necessary to have a failure and low quality. In addition, teachers have determined students’ attitude towards courses, and teachers should have strict requirements for students and maintain good communication with them. Based on the above discussion, the evaluation index of teaching attitude includes Punctuality in class , Adequacy of preparation , Attitude toward students , Teacher-student relationship .
The classroom teaching of HVE is different from undergraduate education, and its teaching content considers the cultivation of students’ practical ability. When evaluating the quality of the teaching content, it is necessary to consider the cultivation of students’ theoretical learning ability and hands-on ability. Based on the general teaching quality evaluation index, this paper considers two aspects of the combination of theory and practice in higher vocational education and theoretical teaching depth. Therefore, the evaluation indicators on the teaching content mainly include Well organized content , Connection between theory and practice , Clear content , Cultivate comprehensive quality , Moderate in depth , etc.
The teaching process is the focus of HVE evaluation. In the process of higher vocational education, students should be student as the main body, considering the career development needs of students. HVE should jointly cultivate professional quality and overall quality, and the teaching method of practice classroom has a great difference in theory and theoretical learning. Therefore, in the teaching process, students should be highlighted in the cultivation of students, and teaching methods should be in both theory and practice. In terms of classroom organization design, teachers should pay attention to teaching logic and orderly, so that students can clearly understand the teaching intentions and teaching objectives, and complete their studies under the guidance of teachers to achieve learning goals. In addition, the proficiency of teaching content determines the authority of teaching, and inspirational artificiality reflected in the lecture process can effectively improve students’ interest. In summary, the indicators in the quality assessment of the teaching process mainly include highlighting professional quality , Applicability methods , Logical and orderly , Inspiring artistry , etc.
Teaching achievements are the direct performance of HVE quality. The teaching achievements are mainly reflected in the quality of students’ training, and the purpose of education is to cultivate people. HVE is the professional and technical personnel of society and workplace. Therefore, when evaluating the quality of the teaching results, teachers should consider the training and knowledge of students ‘cultivation and students’ learning and knowledge. Students’ knowledge masters refer to whether students can fully master knowledge and apply the performance after they enter the workplace. In addition, the expansion of students’ vision is also an important part of higher vocational education. Therefore, the evaluation index of teaching results mainly includes Students’ level of knowledge , Professional ability development , Broadening of students’ knowledge , etc.
The evaluation index system can summarize the actual situation of HVE comprehensively. And these indexes can realize the evaluation of HVE as the input samples of the evaluation model, as shown in Table 1.
3. Evaluation Method of HVE
In this paper, a combination of fuzzy analytic hierarchy and BP neural network method is used to evaluate HVE. This can make full use of the advantages of both qualitative and quantitative methods. Fuzzy mathematics can better solve the fuzzy and difficult-to-quantify problems. The neural network has a strong non-linear mapping capability. In turn, the comprehensive method can effectively avoid the influence of too many subjective factors in the HVE assessment process and ensure the objectivity of the assessment results.
3.1. Fuzzy Analytic Hierarchy Process (FAHP)
FAHP is a comprehensive evaluation method based on the theory of fuzzy mathematical affiliation, which has the characteristics of strong systematic and clear results, and is suitable for solving various nondeterministic problems. The core idea of this method is to convert qualitative evaluation problems into quantitative evaluation problems. It uses fuzzy mathematics to make an overall evaluation of the object which is subject to multiple factors. FAHP as a combined quantitative and qualitative evaluation method has the characteristics of strong systematic, clear thinking, and simple method. However, it still has certain shortcomings, for example, there is a certain bias in judging the consistency of the matrix, and there is still no scientific basis for the consistency test criteria. Therefore, FAHP makes use of the advantages of fuzzy mathematics to make up for the shortcomings of the hierarchical analysis method. By introducing the fuzzy judgment matrix, the influence of subjective factors of experts is reduced.
Suppose the set of evaluation indicators is , where ( = 1, 2, …, n) is the evaluation factor. is the number of individual factors at the same level. The set of evaluation results is assumed to be . where ( = 1, 2, …, m) is the evaluation result. is the number of elements in the set, which is also the number of levels. This set specifies the range of results selection for a given evaluation element. The elements in the result set can be either qualitative or quantitative scores. Let the weight vector be , where ( = 1, 2, …, n) indicates the importance of factor . The sum of the weights is 1, as shown in the following equation:
The relevant personnel in the education industry are organized to conduct fuzzy evaluation of each evaluation factor. According to the distribution of statistical evaluation levels of each indicator, assuming that the ratio of the number of evaluators who evaluate the indicator item as level to the number of all evaluators is , then the judgment matrix is . Obviously, each row of this matrix is the evaluation result of every single factor, and the whole matrix contains all the information obtained from the evaluation of the set of evaluation factors by the set of evaluation results . The final evaluation result is obtained by synthesizing the weight vector with the judgment matrix . That is, , as shown in the following equation:
The evaluation result of the first level evaluation factor can be obtained by normalizing . After that, the previous steps can be repeated to solve the evaluation results of the upper-level indicators in turn, and the overall fuzzy evaluation can be finally obtained.
3.2. Back-Propagation Neural Network (BPNN)
BPNN is the most widely used neural network, which utilizes the error backpropagation algorithm to train multilayer feedforward neural networks. The learning memory capability and self-adaptive ability of BPNN are powerful, thanks to its non-linear information processing system consisting of a large number of interconnected processing units. An important advantage of BPNN is that it eliminates the step of pre-training a fixed model. To obtain more desirable prediction results, it is only necessary to learn the training samples and then accumulate empirical knowledge.
As shown in Figure 2, the standard BP neural network model contains three layers, which are the input layer, the hidden layer, and the output layer. In the forward propagation process, the input samples are passed in from the input layer, processed through the hidden layer, and passed to the output layer. If the actual output of the output layer is not within the error range from the desired output, the backward propagation of the error is performed. After calculating the total error between the desired output and the actual value of all neurons in the output layer, the network weights and thresholds are adjusted. The error signals of the units in each layer are used to correct the weights of each unit so that the actual output of the neural network continuously approximates the desired output.
Where is the input layer node, is the hidden layer node, is the output layer node, and is the weight value between the layers. BPNN is a multilayer feedforward neural network with unidirectional propagation, and the standard BP network algorithm includes both forward propagation and backward transmission. For forward propagation, the data are corrected for weights through the input layer, and the resulting weighted sum is passed to the hidden layer, which is then activated by an activation function and passed to the output layer, as shown in the following equation:where is the number of nodes in the input layer, and is the weighted sum of the j-th node in the hidden layer. Through the action of the activation function and transmitted to the output layer, as shown in the following equation: is the result of the weighted sum of the inputs applied to the hidden layer by the activation function. For the evaluation of HVE, using the Sigmoid function as the activation function, the categories can be predicted and also approximate probability predictions can be obtained. The logarithmic odds function is a convex function of arbitrary order derivable, which has good mathematical properties and can be used directly by many numerical optimization algorithms to find the optimal solution.
For backpropagation, the purpose is to adjust the network weights and thresholds so that the error between the actual output value and the desired value is as close as possible to the predetermined value, and therefore a loss function needs to be designed. The mean square error (MSE) selected for this experiment, which is the mean value of the sum of squares of the errors at the corresponding points of the predicted and original data. It reflects the degree of difference between the estimated quantity and the estimated quantity, and the closer its value is to 0, the better the model selection and fitting, and the more accurate the data prediction, as shown in the following equation: is the expected value, is the output value, and is the number of nodes in the output layer. The connection weights are adjusted more accurately along the decreasing direction of the gradient of the mean square error , as shown in the following equation: is the weight value between the hidden layer and the output layer, and is the learning rate. The training is stopped when the actual output value of the output layer is almost indistinguishable from the desired value by continuously forward and backward propagation.
3.3. Comprehensive Evaluation Method
In this paper, an evaluation method based on the combination of FAHP and BPNN is proposed. First, we obtain a priori samples through FAHP and use these data to train BPNN. then, we further optimize the index weights through the advantage of the nonlinear calculation of BPNN. On the basis of combining quantitative and qualitative aspects, the final weights of each indicator are obtained, and then the purpose of evaluating HVE is achieved. The comprehensive evaluation process is shown in Figure 3.
4. Results and Discussion
In this paper, we take HVE assessment in Henan Province as an example. FAHP is used to realize the initial determination of index weights, construct the dataset, train BPNN, and obtain the final HVE assessment model. The experimental data in this paper are obtained from various higher vocational institutions in Henan Province. The big data are integrated and processed to obtain the HVE assessment index data.
4.1. Initial Determination of Index Weights
In the initial weighting period using FAHP, the relative importance of each index factor at each level is compared in the form of questionnaire. A total of 50 questionnaires are distributed to relevant experts in the field of education, and 46 valid questionnaires are recovered. The “0.1–0.9” scale shown in Table 2 is used to express their relative importance, so as to construct the fuzzy complementary judgment matrix of the corresponding indicators.
According to the results of the questionnaire, the scoring results of each index are averaged and five fuzzy complementary judgment matrices are calculated, which are , , , and . Based on the fuzzy complementary discriminant matrix , the corresponding fuzzy consistency matrix is calculated by using equation (7).
Finally, based on the fuzzy consistency matrix, the initial weight vector of each layer of indicators is obtained by equation (8). The final calculation result is shown in equation (9).
4.2. Indicator Weights Optimization
The linear weighted values of the sample indicators are calculated using the initial weights as the sample output labels, and the data set required for BPNN is constructed. The training set is divided into training and validation sets in the ratio of 8 : 2, where the training set contains 386 data and the validation set contains 97 data respectively. The evaluation grade of HVE is divided into four categories: A, B, C, and D according to the total score. Class A represents the best, i.e., the score range [4, 5], and class D represents the least, i.e. [1, 2].
A three-layer neural network structure is used with 16 nodes in the input layer, 3 nodes in the hidden layer, and 1 node in the output layer. The maximum number of training rounds is set to 1000 and the learning rate is 0.05. If the average error of the model on the validation set is less than 0.001, the training is ended early to prevent the overfitting phenomenon in further training. To verify the applicability of BPNN with HVE evaluation, the dataset was divided into 6 data sets and compared using the random forest (RF) and support vector machine (SVM) algorithms. The comparison results of the accuracy are shown in Figure 4.
The average accuracy of different machine learning methods is over 95%, which shows that the data in this paper is very reliable. And the applicability of BPNN with this paper is the strongest among these three methods. The role of BPNN in the combined model proposed in this paper is to reduce the influence of subjective factors of experts on the determination of weights, so the indicator weights of the neural network model after passing the training should be close to the initial weights, which will also increase the interpretability and reasonableness of BPNN. After several trials, the final model optimized indicator weights are shown in Table 3.
4.3. Comprehensive Evaluation Based on FAHP-BPNN
In order to prove the practical value of the model and make the model truly serve HVE, this paper invited relevant experts from higher vocational institutions in Henan Province to conduct interviews. The FAHP-BPNN model in this paper is used to evaluate the current educational status of five typical institutions so that the schools can have a better knowledge of today’s teaching and the learning situation, and then make targeted improvements. The final score of HVE of each institution is shown in Table 4.
HZ pays attention to machine building and achieved the highest score of 4.0122, which belongs to grade A. While the comprehensive scores of TZ, KF, and SL are 3.5877, 3.3372, and 3.1264, respectively. Although all three are in the B grade, the LY has a much lower rating. SZ is the lowest score of 2.6773, a C grade.
Respondents generally give high ratings to the HVE effectiveness assessment index system. The effectiveness of teaching inside and outside the classroom is reflected by the indicators, and the indicators such as Connection between theory and practice not only reflect the technical but also the academic nature of higher vocational schools, which helps students to improve their overall quality. In addition, the weights in this paper are generally agreed upon. HVE values practical skills and therefore places the highest weight on teaching achievements. Teaching attitude, teaching content and teaching process complement each other and all obtain a weighting of 0.2 or more. And teaching content, as the core aspect of the classroom, achieves the second most important ranking.
In the final results, the indicators of Teacher-student relationship, Connection between theory and practice, and Students’ level of knowledge receive the highest weights. This also points to the direction for the improvement of HVE. Testing and examinations are important initiatives that need to be used in vocational education in order to ensure the quality of education. Each program must establish an industry testing team to provide sound assessment and testing feedback on the results of vocational education. In order to improve the teacher-student relationship and to ensure the teaching of students according to their abilities, a new educational philosophy is needed. The ultimate goal of vocational education is to socialize and professionalize human development. To concretize and document the responsibilities and roles of vocational institutions, cooperative teams, and other relevant stakeholders to improve management and create a better environment for students’ learning.
HZ received an A rating in the evaluation, and its teaching philosophy of serving development and promoting employment is worthy of consideration. HVE has the essential characteristics of applicability and technicality. Students should be instructed to use their strongest abilities in different environments to improve their adaptability and ability to solve problems effectively. While re-emphasizing skill cultivation, the education of the humanistic spirit should not be neglected. With economic globalization and social informatization, higher education institutions should also focus on students’ personalized development and cultivate them into higher technical application talents with innovative abilities.
5. Conclusion
HVE is the advanced stage of vocational education, and from the viewpoint of vocational education itself, employment orientation requires schools to cultivate technically skilled talents who can meet job requirements. HVE evaluation needs to strengthen the internal construction, which is also the way to improve the quality of education in higher vocational institutions. This paper follows the basic principles of combining theoretical analysis and empirical research, unifying qualitative and quantitative research, and integrates research methods such as literature research, questionnaire survey, interview, comparative study, and case study.
According to the actual situation of HVE development, 16 HVE evaluation indexes are proposed in four dimensions: teaching attitude, teaching content, teaching procedure, and teaching achievement. The optimal weights of each index were obtained based on the FAHP-BPNN method. the BPNN also achieved optimal results in the comparison of accuracy rates of different machine learning. Among the first-level indicators, the teaching achievement received the highest weight of 0.275. This fits with the practicality of higher education institutions. After the comprehensive evaluation, SZ school obtained a C grade, which indicates that its overall teaching quality is poor and needs to be enhanced according to the evaluation index system and the actual situation. the practical application of FAHP-BPNN comprehensive evaluation model in five typical higher vocational schools in Henan Province shows that the model can be widely used to guide the teaching quality improvement of higher vocational schools.
Due to the limitations of the research conditions, there are some aspects of this study that need to be improved. First, there is a need to improve the HVE evaluation indicators around the core of quality. HVE to change the type of education involving the enterprise society, there is a need to establish and improve the indicator dimensions related to the integration of industry-education and school-enterprise cooperation. Data collection and analysis can also be further improved to expand the range of respondents.
Data Availability
The labeled dataset used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.