Big Data Modelling of Engineering and Management 2021View this Special Issue
Research Article | Open Access
Zhen-Hui Li, Jia-Jia Yang, Hai-Qing Qin, Yi-Wei Xia, Mei-er Zhuang, "A Study of Efficiency Evaluation of National Quality Online Courses during the Epidemic: Based on Fuzzy Logic Calculation and Bootstrap-DEA", Mathematical Problems in Engineering, vol. 2021, Article ID 5534583, 7 pages, 2021. https://doi.org/10.1155/2021/5534583
A Study of Efficiency Evaluation of National Quality Online Courses during the Epidemic: Based on Fuzzy Logic Calculation and Bootstrap-DEA
The study combines fuzzy logic calculation method and bootstrap-DEA to explore the efficiency of National Quality Online Courses (NQOOCs) during the new coronavirus epidemic. We choose Project 985 universities in China as our sample considering their academic influence. The results show that the efficiency value of NQOOCs calculated by bootstrap-DEA differs from the efficiency value measured by traditional DEA. The difference of universities` ranking is more pronounced in bootstrap-DEA. The amount of input resources (including human resources, science and technology resources, and financial resources) may not be directly reflected on the efficiency of NQOOCs. There is still much room for improvement in the efficiency of NQOOCs in most universities.
Under the influence of the new coronavirus epidemic, most of universities have actively responded to the Ministry of Education’s initiative of “no suspension of classes, no suspension of classes” and launched online courses. During this period, “National Quality Online Open Courses (NQOOCs)” played a leading role in demonstration. Many universities have opened high-quality online courses and virtual simulation experimental teaching resources free of charge to students across the country through online education websites such as “Chinese Universities MOOC.” However, due to the lack of online teaching experience and the different input levels and methods of online courses, the effectiveness of online education construction in various universities is uneven. In order to improve the quality of online courses education in universities, it is particularly important to explore the relationship between the investment and effectiveness of online courses in universities.
Although there have been studies on the advantages and disadvantages of online courses, the development model, and the problems faced by online teaching during the epidemic, most of them are qualitative analysis, and there is a lack of quantitative evaluation of the construction methods and effects of online education in different universities. As the pioneer of online education in China, universities sponsored by Project 985 have accumulated relative experience. The National Quality Online Open Courses represent the best quality in online courses. Therefore, this article takes universities sponsored by Project 985 as the representative and uses the DEA-SBM model to analyze the relationship between the input elements and output elements of the national quality online open courses during the epidemic in 2020 and obtains the optimal benchmarking model by comparison. It is hoped that this research can provide reference suggestions for improving the quality of online education courses in universities.
2. Literature Review
E-learning, also known as distance education and online learning, is a learning method that can spread content across time and space through the use of information technology and Internet technology, bringing great convenience to our lives, learning, and work [1, 2]. Its forms include micro-classes, MOOCs, SPOC, and so on. Among them, MOOCs are the main form of China’s online education designated quality online courses. In 2015, the “Opinions of the Ministry of Education on Strengthening the Application and Management of Online Open Curriculum Construction in Colleges and Universities” pointed out that China’s universities should strengthen the construction of an online open curriculum system and platform with Chinese characteristics, and actively promote the update of college education concepts and the optimization of teaching models . Major universities in China actively respond to the call of the Ministry of Education, and many famous universities led by Peking University and Tsinghua University actively participate in the construction of online open courses. In 2019, the Ministry of Education identified 801 courses as 2018 national quality online open courses, including 690 undergraduate education courses and 111 vocational higher education courses.
Online education in colleges and universities has emerged as a teaching model in recent years, and many scholars have analyzed its advantages, disadvantages, and development models. Online education can increase the interactivity of courses through the information database management technology and two-way interactive function of the computer network, and the asynchronous or synchronous learning network model reduces the time and space constraints [4, 5], and the learners of the course have no background and experience restrictions , which provides participants with convenient learning opportunities. However, the existing online courses have the problems of insufficient wisdom, low teachers’ new media literacy, and lack of innovation in instructional design . Zhang and Wang found that there is still a lot of room for improvement in the quantity, technology, and quality of the MOOC and MOOC platform through the observation of relevant data of the national quality online open courses . In response to the above problems, a lot of research has begun to focus on the factors that affect the efficiency of online education and the ways and means to improve online education . The study mainly explores the influence of learner factors, teacher factors, and technical factors on the satisfaction of online education courses from the TAM model [10, 11]. The exploration of the efficiency of online courses in colleges and universities under the background of new coronavirus is also very important. For example, Zou et al. combined with the situation in the Guangdong-Hong Kong-Macao Greater Bay Area to build an online teaching model for universities and implemented it in emergency to solve the urgent need for online teaching organization and management during the outbreak .
However, the relevant research on online education, especially MOOCs, mainly focuses on the influencing factors of course efficiency, such as course design factors. However, such factors mainly affect learners’ satisfaction and the willingness to learn. Besides, the measurement of the outcome of MOOCs is mostly based on singular variable, and the influence of the learner factor is exaggerated . For the national quality online courses, the input of universities has a more significant impact on their quantity and quality, so it is of great practical significance to explore the relationship between university inputs and the output of quality online courses. In addition, the existing literature has proposed many strategies for online education in colleges and universities, but it has not obtained the optimal model by comparing the efficiency of online education construction in many colleges.
Therefore, this article takes universities sponsored by Project 985 as the representative and combines the fuzzy logic calculation method and bootstrap-DEA model to analyze the relationship between the input elements and output elements of the national excellent courses from January to May 2020 to find the optimal benchmarking model. This article will also provide reference suggestions for universities to improve the quality of online education courses.
3. Data Sources and Variables
Data for the input of universities for education are derived from panoramic data platform of universities, the platform comprehensively covers more than 150 high-interest development related data indicators, and the collected data cover double first-class construction, faculty, scientific research, courses construction, education and teaching, and so on. The data are accurate and provide a good data source for researchers.
The data of Project 985 national high-quality online courses are mainly from the Chinese University MOOC website. This website is a domestic high-quality Chinese MOOC learning platform, which was created by the Love Course Network and Netease Cloud Classroom. The platform has more than a thousand courses including Project 985 universities, among which are the first batch of certified national quality online open courses data, including the number of courses, course ratings, and student participation. It can provide data support for our analysis of the efficiency of national quality online courses (https://www.icourse163.org/).
Combining the data from the two data platforms and the list of 985 national universities, a total of 39 universities were identified. After that, we excluded three schools that did not open online courses on the MOOC platform of Chinese universities during the epidemic, namely, Tsinghua University, Northwest A&F University, Lanzhou University, South China University of Technology, and National University of Defense Technology that lacked input data. Finally, 34 Chinese Project 985 universities were identified in our research sample.
The input variables we choose are human resources, science, and technology resources and financial resources universities invested in 2019, which are essential elements for universities` courses design and construction. We use high-end talent (HET), tech talent (TT), and course professionals (CP) to measure human resources. And to measure science and technology resources, we use scientific research projects (SRP), research scale (RS), and discipline construction (DC) as variables. Financial resources are measured by project investment (PI), social donation (SD), and fund investment (FI). Besides, to measure the output of NQOOC, we use total courses (TC), teaching quality (TP), and total number of participants (TNP) during the epidemic situation (from Jan 2020 to May 2020). Definitions and statistic descriptions of the above variables are shown in Table 1.
HET represents scholars who have made outstanding contributions to the development of related fields. TT are scholars who have a high world influence. CP are members of the College Teaching Steering Committee of the Ministry of Education. These experts have strong teaching ability and high academic attainments and can provide consultation for the discipline construction of universities.
SRP are projects supported by the National Natural Science Foundation of China usually have high research value and significance. RS refers to number of papers included in the Scopus database, which is used as an important data source in various domestic and foreign university rankings. The selection of “double first-class” DC is a great affirmation of the effectiveness of college discipline construction.
Funding for scientific research projects is an important material basis for the smooth development of scientific research in colleges and universities. So, we choose PI as the representor of scientific and technological research funds. SD is one of the important sources of funding for colleges and universities, and it can play a role in helping colleges and universities. Through FI, we can see the overall scale and overall scientific research strength of basic research and scientific research workers in each school.
4. Mathematical Model and Data Analysis
4.1. Fuzzy Logic Calculation
Based on multi-valued logic, fuzzy logic is a science using fuzzy sets to study uncertainty concept and ambiguity phenomena as well as their laws. Fuzzy logic can imitate the way of thinking of human brain, express qualitative knowledge and experience with unclear boundaries, and judge and reason about uncertain concepts. It uses membership function to distinguish fuzzy sets and handle fuzzy relationships. For description systems where the model is unknown or uncertain, fuzzy sets and fuzzy rules are used for reasoning and expressing transitional boundaries or qualitative knowledge experience. Meanwhile, fuzzy comprehensive judgment is carried out and solves regular fuzzy information problems that conventional methods are difficult to deal with.
In this paper, by using fuzzy logic, the second-level indicators of input and output can be reasoned and judged comprehensively, thus scientifically and effectively merged into the first-level indicators of input and output, which lays the foundation for the efficiency evaluation below. Specific steps are as follows.
First, each column of sample data of each secondary index is subjected to min-max normalization processing. The min-max normalization method is to linearly transform the original data. The formula is new data = (original data-minimum value)/(maximum value-minimum value).
Second, define the variables in the fuzzy logic and determine the degree of membership. Convert the input value of fuzzy logic into the degree of membership of each set.
Third, conduct the process of fuzzification. Determine the relationship between the input value and membership, so that the input value can find the membership of the corresponding set at any point. Define Gaussian, trapezoidal, and triangular membership function parameters, and add input and output variable membership functions to the fuzzy inference system.
Fourth, design the fuzzy logic judgment operation. Each membership degree decomposed, imitating human judgment using the concept of fuzzy, the fuzzy rule base and fuzzy inference of the previous step are used to obtain fuzzy logic control result signal. In this study, we use the minimum membership method (MIN implication) for processing.
Fifth, add fuzzy logic decision rules. After the input is fuzzified, rules need to be set and recombined with the operation of fuzzy logic.
Finally, conduct the process of de-fuzzification. This step is to convert the fuzzy value of the inference result into a clear control signal value. After the fuzzy logic converts the input value into the membership of each set through fuzzy, then several outputs can be obtained through rules and operations (see Table 2), and the membership function graphs are drawn (see Figure 1).
As can be seen from Figure 1 and Table 2, after fuzzy logic processing, the number of variables is effectively reduced to three inputs and one output. Since the set data output interval is between 0 and 20, the index results after fuzzy logic control are all within 20, which provides good data conditions for further empirical analysis.
4.2. NQOOCs Efficiency Analysis of Universities Based on the Bootstrap-DEA Method
Bootstrap-DEA is an extension of the traditional DEA method. Although the DEA method for some technical parameters estimation method has several advantages, the estimation results are susceptible to random interference factors, with sample sensitivity , and the bootstrap-DEA is very essential to overcome the efficiency value of the inner dependency. It makes statistical inferences from the raw data, without making any assumptions about the unknown population. And it generates pseudorandom numbers by taking back samples from existing samples, thus inferring the characteristics of the population . Efficiency value calculation with bootstrap-DEA includes the following steps: Firstly, each DMU , , uses the traditional DEA method to calculate the efficiency value of the sample data. Secondly, based on the efficiency , the bootstrap method produces n efficiency value , where b represents the b-th iteration using the bootstrap method Thirdly, calculate bootstrap simulation sample , , where , (k = 1, …,n) Fourthly, the traditional DEA method is used to simulate each bootstrap method sample, and the efficiency value , , is calculated again Finally, repeat steps 2–4 B times to generate the efficiency value ,
Generally speaking, the more iterations of bootstrap-DEA, the more accurate the calculation results of efficiency value will be. The greater the confidence, the greater the distance between the upper and lower limits of the confidence interval . According to existing studies, without loss of generality, this paper sets the number of bootstrap-DEA analysis iterations as 2000 and the confidence as 95%.
In the previous paper, the fuzzy logic calculation method is used to obtain the fuzzy values of input variables and output variables. On this basis, bootstrap-DEA was used to analyze the operational efficiency of online education in 34 Project 985 universities. The results are shown in Table 3.
From the perspective of ranking, the ranking of universities before and after rectifying deviation has changed greatly. From a comprehensive perspective, except for Hunan University and Northwestern Polytechnical University, the ranking of other universities has decreased by different degrees compared with the traditional DEA results. Except for Hunan University, the former top universities all dropped in different degrees. Beihang University, Beijing Normal University, and Jilin University ranked third, fourth, and fifth, respectively. The ranking of Beijing Institute of Technology dropped to 12, and the efficiency value after rectifying deviation was 0.845, which was the university with the largest decline. In addition, Northwestern Polytechnical University ranked second both before and after the correction.
The NQOOCs of Hunan University was 0.932 after rectifying deviation, and the lowest value was 0.342 after rectifying deviation of Peking University. There is still much room for improvement in the efficiency of NQOOCs in most universities.
Among the top 10 ranking universities, there are three polytechnic universities, namely, Northwestern Polytechnical University, Beihang University, and Dalian University of Technology, which indicates that the inputs of these universities are well utilized to courses construction. Three comprehensive universities, namely, Hunan University, Jilin University, and Nankai university, also perform well. Besides, the only two normal universities of Project 985 universities, namely, Beijing Normal University and East China Normal University, are both on the top 10 list. The possible reason is that normal universities have more teaching techniques and experience to improve the courses effect, and can better promote the conversion of invested resources in teaching performance. Moreover, Ocean University of China and China Agricultural University achieved the 8th and 9th rankings, which suggests that course characteristics and professionalism are also important factors in ensuring course quality.
Based on the fuzzy logic calculation analysis and bootstrap-DEA analysis, this study evaluated the efficiency of national quality online open courses of Chinese Project 985 universities. The results have some implications.
The combination of fuzzy logic calculation analysis and bootstrap-DEA analysis may provide some reference for the process of dealing with variables that are of similarities. And the use of bootstrap-DEA can overcome several disadvantages of traditional DEA, especially that there are significant difference in the ranking results of bootstrap-DEA, which is essential for us to probe into the possible causes.
There are some practical implications of this paper. Firstly, universities with abundant resources should pay more attention to the efficiency of curriculum construction, because there are a large number of participants that rely on the NQOOCs to gain knowledge. Secondly, ensuring the distinguishing feature and professional level of NQOOCs is a vital way to promote teaching efficiency. What is more, the focus of this research is online education, which can provide reference value for scholars in this field in the future, enriching literature materials.
Additionally, this study still has some limitations that may offer future research direction. Although we have evaluated the efficiency of NQOOCs during the new coronavirus epidemic, the previous invested resources may be for a whole year of output. Future research can investigate the efficiency of NQOOCs measured by the whole year outputs. Besides, the results of bootstrap-DEA only offer the efficiency of NQOOCs, but it is not clear how different input elements produce a marked effect. Using other data-mining methods is necessary to test the mechanism of input elements and outcomes.
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare there are no conflicts of interest regarding the publication of this paper.
This research was financially supported by Asian Media Research Center Project, “Research on the Improvement PATH of China’s International Communication Ability in the New Media Era” (Grant no. AMRC2018-3).
- W. Bhuasiri, O. Xaymoungkhoun, H. Zo, J. J. Rho, and A. P. Ciganek, “Critical success factors for e-learning in developing countries: a comparative analysis between ICT experts and faculty,” Computers & Education, vol. 58, no. 2, pp. 843–855, 2012.
- P.-C. Sun, R. J. Tsai, G. Finger, Y.-Y. Chen, and D. Yeh, “What drives a successful e-learning? an empirical investigation of the critical factors influencing learner satisfaction,” Computers & Education, vol. 50, no. 4, pp. 1183–1202, 2008.
- L. Zhao, C. Yun, and Bo Yu, “Research on online teaching mode of ordinary colleges and universities from the perspective of smart education,” Computer Knowledge and Technology, vol. 15, no. 30, pp. 158-159, 2019, in Chinese.
- G. Piccoli, R. Ahmad, and B. Ives, “Web-based virtual learning environments: a research framework and a preliminary assessment of effectiveness in basic IT skills training,” MIS Quarterly, vol. 25, no. 4, pp. 401–426, 2001.
- Y. J. Katz, “Attitudes affecting college students’ preferences for distance learning,” Journal of Computer Assisted Learning, vol. 18, no. 1, pp. 2–9, 2002.
- A. McAuley, “Massive open online courses: digital ways of knowing and learning,” The MOOC Model for Digital Practice, vol. 5, pp. 3–6, 2010.
- A. Margaryan, M. Bianco, and A. Littlejohn, “Instructional quality of massive open online courses (MOOCs),” Computers & Education, vol. 80, pp. 77–83, 2015.
- L. Zhang and J. Wang, “Data analysis of the national excellent online open courses of the Mu class platform,” Digital Education, vol. 5, no. 5, pp. 36–41, 2019.
- A. H. Mousa, “Advance acceptance status model for E-learning based on university academics and students,” IOP Conference Series: Materials Science and Engineering, vol. 671, no. 1, 2020.
- L. L. Martins and F. W. Kellermanns, “A model of business school students’ acceptance of a web-based course management system,” Academy of Management Learning & Education, vol. 3, no. 1, 2011.
- R. Edmunds, M. Thorpe, and G. Conole, “Student attitudes towards and use of ICT in course study, work and social activity: a technology acceptance model approach,” British Journal of Educational Technology, vol. 43, no. 1, pp. 77–84, 2012.
- Y. Zou, C. Li, and Y. Xie, “Construction and implementation of online teaching “Bay Area Model” in colleges and universities during the epidemic situation,” China Audio-Visual Education, vol. 4, pp. 22–28, 2020, in Chinese.
- M.-L. Song, L.-L. Zhang, W. Liu, and R. Fisher, “Bootstrap-DEA analysis of BRICS’ energy efficiency based on small sample data,” Applied Energy, vol. 112, pp. 1049–1055, 2013.
- Qi. Zhang, “Efficiency evaluation of Chinese commercial banks based on DEA and bootstrap method,” Economic Research Guide, vol. 32, pp. 123–125, 2009.
- W. Yin and W. Yuan, “Evaluation of scientific research efficiency of universities directly under the ministry of education of China based on Bootstrap-DEA method,” Statistics and Information Forum, vol. 28, no. 6, pp. 61–69, 2013.
Copyright © 2021 Zhen-Hui Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.