Abstract
Virtual reality technology (VR) has been widely used in education and teaching, but the results of a large number of experimental studies on the impact of VR on students’ technical skill learning are not consistent. In this study, a meta-analysis method was used to conduct in-depth quantitative analysis of 32 experimental or quasi-experimental research literatures based on virtual reality in the past 20 years. The results showed that: (1) virtual reality technology has a moderately positive effect on the learning of technical skills of science and engineering college students (); (2) the effect of VR on cognitive strategies () was significantly greater than that of mental skills () for technical skills under different measurement dimensions. (3) Different test periods, previous experience levels, and auxiliary methods have different degrees of positive influence on improving students’ technical skills, among which the 40-60 minutes experiment period () has the greatest influence on the improvement of technical skills. According to the research conclusions, we suggest to make an overall planning for the construction of VR enabling teaching application scenarios and create a human-machine collaborative interactive teaching model of “theory+VR simulation+feedback” based on flow theory.
1. Introduction
Virtual reality technology has been more widely used in education and teaching, and the integration of technology and teaching practice has injected new vitality into the cultivation of skilled talents. As an important part of the education system and human resource development, vocational education has become more and more important in recent years as it is responsible for cultivating diversified talents, passing on technical skills, and promoting employment and entrepreneurship. VR technology, as a new and powerful educational technology, provides a new platform and means for the design and implementation of educational teaching scenarios, and interactive experimental teaching and skill training can be conducted based on VR technology, especially for flammable and explosive chemical synthesis experiments or human anatomy experiments that are not possible to be carried out realistically in real life etc. [1]. Virtual reality in a broad sense encompasses immersive, augmented, and hybrid virtual reality technologies, while the virtual reality discussed in this paper is immersive virtual reality (VR), in a narrow sense, which refers to computer technology as the core and combined with other science and technology to generate a digital environment that is highly similar to the real or imaginary environment, with immersion, interactivity, conceptualization, and intelligence features [2]. The virtual simulation experiment section of the National Smart Education Platform has been officially launched in September this year, which will have a profound impact on the future experimental teaching and the cultivation of students’ technical skills.
2. Literature Review
VR teaching is a new technology and a new means of educational technology, which is widely used in teaching various disciplines. Virtual reality learning environments play a huge role in facilitating students’ acquisition of knowledge and skills [3]. The virtual reality learning environment is a great aid and facilitator for students’ learning, especially for their knowledge and skill acquisition. A large number of teaching studies based on virtual reality technology have been done by domestic and foreign scholars, but there is no consensus on its application to science and technology subjects and its teaching effect on students’ technical skills development.
Marks and Thomas conducted a five-semester study with a virtual reality lab designed at the University of Sydney and found that 71.5% of students showed significant improvements in learning outcomes [4]. Sultan et al. conducted an experimental and control group study at a health sciences university medical school and found that the VR group has significantly higher skill scores than the conventional group, and that virtual reality technology provided a richer, more interactive educational environment for students, increasing their interest and motivation to learn and effectively supporting knowledge retention and skill acquisition [5]. Panxinou et al. integrated three different teaching scenarios of optical microscopy experiments in a biology laboratory course and found that the teaching effectiveness and student skill gains were significantly higher in the virtual reality educational tools group than in the conventional teaching practices group, and students with virtual laboratory experience were more competent than other students in terms of operational sensitivity [6].
Another view is that virtual simulation laboratory instruction based on virtual reality technology is as effective as traditional physical laboratory instruction. By evaluating a comprehensive set of virtual laboratory college physics courses and comparing students’ physical laboratory versus virtual laboratory experiences, Darrah et al. found that the results of the two university studies showed that the teaching and learning outcomes were nearly identical whether the virtual lab was used in a laboratory setting, as a supplement to a hands-on experiment, or as a traditional hands-on experiment [7]. Abeer et al. compared the differences in phlebotomy between the control group taught by traditional methods and the experimental group using virtual reality simulation through a quasi-experimental study, which found that no performance metrics were significant between the two groups in terms of number of instrument reinsertions, tourniquet application time, and time to successful completion of the procedure [8]. Trent Wells and Miller conducted a study at Iowa State University on whether the use of VR technology affects welding skills and found no statistically significant differences in total welding scores between the four training groups randomized to 100% physical welding, 100% VR welding, 50% physical welding/50% VR welding, and 50% VR welding/50% physical welding [9].
Although scholars have conducted more meta-analytic studies on the teaching effect of virtual reality technology, there is no research on its effect on students’ technical skills development in polytechnics, and after sorting and summarizing the literature, we found that the teaching effect of VR application in polytechnics has not yet formed the same conclusion. In order to summarize the experience of VR teaching in polytechnics and provide a reference for using VR to teach technical skills, this paper uses meta-analysis to quantify the findings of existing empirical studies and tries to answer the following research questions: (1) does VR significantly improve students’ technical skill learning compared with traditional physical laboratory teaching? (2) Does the difference in disciplines affect the effectiveness of teaching virtual reality technology? (3) Which teaching effectiveness is better from the experimental cycle level? (4) From the point of view of VR applied to teaching methods, which one is more effective in assisting teaching? (5) Which is more effective in terms of improving technical skills? (6) Does the presence or absence of VR experience affect the effectiveness of learning technical skills?
3. Study Design
3.1. Research Methods and Tools
Meta-analysis refers to the quantitative analysis method of extracting and organizing multiple results of experimental or quasi-experimental studies on the same research problem and producing average effect values by weighting the sample size, mean, standard deviation, and other data of the existing research results and analyzing the effect values to obtain the results [10]. The meta-analysis method has been widely used in education, and this study compares and combines literature on the same research topic but with different research results by extracting data such as pre- and postmeasurement means, sample sizes, and standardized mean differences from relevant literature, while using the standard deviation (SMD), which can correct for small sample bias, as the effect value [11] to indicate the degree of impact of VR instruction on student learning of technical skills. The study entered the relevant data into CMA meta-analysis software (Comprehensive Meta-Analysis 3.0) for data analysis.
3.2. Research Process
To ensure the quality of the study, this study strictly followed the meta-analysis criteria proposed by Glass [12], which was mainly divided into four assessment procedures: literature collection, literature coding, effect size calculation, and moderating variable analysis, and finally a comprehensive effect size exploration and study results.
3.2.1. Literature Search
In order to ensure the timeliness of the study, this study mainly searched the relevant virtual reality topic research since 2002 to 2022, mainly in China Knowledge Network, ERIC, Springer Link, Web of Science, Science Direct, and other major databases at home and abroad and searched the literature by “AND”. The keywords of virtual reality included: Virtual Reality, VR, and Immersive/Virtual learning Environment; the keywords of technology skill learning included: Technology, Skill, and Learning, and the selected articles were all from SSCI or SCI authoritative journals and CSSCI Chinese core journals of article literature type. To avoid omissions, this study also supplemented the search with the references of relevant articles.
3.2.2. Literature Selection and Inclusion Criteria
In order to find articles that meet the subject matter requirements, this study used the PRISMA (2020) process for literature processing [13]; the literature search, screening, and inclusion process is shown in Figure 1. Combining the needs of the meta-analysis method itself and ensuring the accuracy and rigor of the research results, the following selection and inclusion criteria were used: (1) duplicate literature must be removed; (2) must contain the effect of VR-based instruction on the learning effect of technical skills; (3) must be an empirical research type article; (4) must be the content of a science and technology course; (5) must be based on university students as experimental subjects; and (6) complete data for which effect values could be calculated must be available. A total of 72 articles were screened by two researchers in the inclusion phase, and those with inconsistent screening were discussed, and the final decision was made to include 32 articles in the meta-analysis, which met the inclusion criteria for the number of articles in the meta-analysis method.
3.2.3. Literature Code
To ensure the objectivity of the coding process, the coding of the 32 empirical research articles included in the meta-analysis was done independently by two researchers in this study, and the coding results were tested for consistency using SPSS 24.0, and the Kappa value was 0.884, which was greater than 0.7, indicating that the coding effect was valid and the results were credible. The coding content mainly included the following five items: subject category, experimental period, assistance mode, learning environment, and prior experience, and the specific coding content is shown in Table 1.
Since this study is aimed at exploring the impact of virtual reality technology on the learning effect of technical skills of students in science and engineering colleges and universities and to explore other factors that may moderate this impact, according to the results of literature coding, refer to Ling et al.’s meta-analysis theoretical framework [14], it draws the theoretical framework of meta-analysis as shown in Figure 2.
4. Results
4.1. Publication Bias Test
Since 71 separate effect sizes were included in this study and the number of experiments was large, the std diff in mean value was selected as the unbiased effect value (SMD value for short). In order to ensure the possibility of no deviation between the reported results and the real results, the funnel plot method was used to qualitatively analyze the publication bias. Publication bias was quantitatively analyzed by Begg’s rank test, Trim and Fill, and Fail-safe N. Publication bias is crucial to the results of meta-analysis. If the research literature fails to systematically represent all existing studies in the field as a whole, publication bias may exist [15]. As shown in Figure 3, most of the study effect values were clustered in the funnel plot, and a small number of effect values were relatively skewed to the right. Begg’s rank test , , indicating that publication bias may exist on the surface. Therefore, the fail-safe factor and the scissor-supplement method were further used to identify the severity of publication bias. The combined effect value based on the random effect model did not change after the shear supplement, while the combined effect value based on the fixed effect model was 0.304 and 0.214, indicating that the possibility of publication bias in this study was small [16], and the fail-safe factor showed that . It is much larger than “” (), suggesting that an additional 3090 unpublished studies would be needed to reverse the results [17]. Therefore, it can be considered that there is no significant publication bias in this study.
4.2. Heterogeneity Test
To ensure that the effect values of the independent samples in this study were combinable, and values were used to define heterogeneity. Higgins et al. classified heterogeneity as low, medium, or high, as measured by the magnitude of the statistic, which was 25%, 50%, and 75%, respectively [18]. In addition, if the statistic is significant, then the hypothesis that there is no heterogeneity among the sample data should be rejected. Based on the forest plot of and (), the results indicate that there is a moderately high degree of heterogeneity between the samples; therefore, this study used a random effects model for correlation analysis to eliminate some of the effects of heterogeneity. It also further suggests the need to conduct moderated effects to test the impact of virtual reality technology on technical skills.
4.3. The Overall Impact of Virtual Reality Technology on Technical Skill Learning of Students in Science and Technology Colleges and Universities
Cohen proposed the effect value analysis theory in 1988; he believed that effect criteria measure the effect determined by the effect value (ES); when the ES is less than 0.2, it indicates a small effect impact; when the ES is between 0.2-0.8, it indicates a moderate effect; and when , it indicates a significant effect impact [19]. This study included 71 experimental data from 32 empirical research papers, and as shown in Table 2, the combined effect value of the effect of virtual reality technology on technical skill learning of students in polytechnics was 0.363 and , indicating that the overall effect was a significant and moderately positive facilitating effect.
Anderson, a famous American cognitive psychologist, divided knowledge into two categories: declarative knowledge and procedural knowledge, and according to this classification criterion, technical skills belong to procedural knowledge. In this paper, the literature included in the meta-analysis was further divided into two subcategories of mental skills and cognitive strategies for analysis according to Gagne’s information processing theory, as shown in the data in Table 3; the virtual reality technology had a moderate positive impact () and a greater impact on cognitive strategies () than on mental skills, indicating that VR technology has a greater impact on students’ methods and techniques for self-regulation and control of information processing.
4.4. Test of Adjustment Effect
4.4.1. Differences in the way VR Assists in Influencing Students’ Technical Skills
The application of VR in teaching is divided into two main approaches: stand-alone adjunct teaching and traditional face-to-face classroom adjunct teaching, and the results of their different adjunct teaching approaches on students’ technical skills are shown in Table 4. The results show that both have a positive impact on students’ learning of technical skills, and the teaching effect of the VR+ approach () is significantly higher than that of the VR approach (), which may be related to the traditional classroom teacher’s lecture style.
4.4.2. Disciplinary Category Differences in the Impact of VR on Students’ Technical Skills
The results of the impact of virtual reality technology in different subject content on students’ technical skills are shown in Table 5. Overall subject content has a significant moderate effect on technical skill learning (, ), with a positive enhancement effect on different subjects. In terms of the effect values for each discipline, , due to the small literature on the disciplines of agronomy in this experiment, the analysis focused on the effect of VR on the technical skills of students in science, engineering, and medicine. The results showed that VR had the most significant impact on technical skill learning of students in engineering disciplines, followed by science and medicine, and that there were significant differences in the impact on different disciplines.
4.4.3. Experimental Cycle Differences in the Impact of VR on Students’ Technical Skills
The results of the effect of virtual reality technology on students’ technical skills at different experimental cycles are shown in Table 6. The results in descending order of effect values were 40-60 minutes (), more than 60 minutes (), 0-20 minutes (), and 20-40 minutes (), and all reached a statistically significant level (). This suggests that there are differences in the duration of VR use on the learning of technical skills, and that not the longer the duration of use is better, but instead the 40-60 minutes experimental cycle, roughly one class period, is the best for learning, reaching a moderate to positive impact.
4.4.4. Prior Experience Differences in the Impact of VR on Students’ Technical Skills
The results of the effect of students’ experience with or without the application of virtual reality technology on students’ technical skills are shown in Table 7. The overall effect value () for the moderating effect of different learning bases on technical skill learning indicates a moderate positive effect, with those without VR experience () outperforming those with VR experience () and the between-group effect value , indicating that the effect of having or not having VR experience on technical skill learning is significant.
5. Discussion
This study used meta-analysis to systematically sort and quantitatively analyze 32 experimental or quasi-experimental research papers published between 2002 and 2022 on the impact of virtual reality technology on technical skill learning and whose subjects were students in science and technology colleges and universities and dissected the differences brought about by different moderating variables. The results show that: (1) compared with traditional physical laboratory teaching, VR can significantly improve students’ technical skill learning; (2) the degree of impact of virtual reality technology on different disciplines varies; (3) from the experimental cycle level, not the longer the experimental time, the better the technical skill learning effect, 40-60 minutes is the best effect; (4) from the VR application to teaching methods, as the traditional face-to-face teaching; (5) from the perspective of technical skills, the impact on cognitive strategies is greater than mental skills; (6) the effect of improving technical skill learning for students without VR experience is better. Specific analysis and discussion are as follows.
5.1. Virtual Reality Technology Positively Contributes to Technical Skill Learning for Students in Science and Technology Colleges and Universities
First, the combined effect value of () for the impact of virtual reality technology on technical skill learning indicates that the integration of virtual reality technology with technical skills education has a moderately positive contribution for students in science and technology colleges and universities compared to traditional physical laboratory instruction. Second, VR had a significant positive impact on students’ procedural knowledge processing using concepts and rules for internal regulation, suggesting that the impact of VR on students’ technical skill learning is not only on external information processing methods and techniques but also on self-cognitive strategies to achieve self-regulation and control over information processing.
This finding is consistent with the results of an empirical study of 30 virtual experiments using meta-analytic methods by Liu et al. Virtual experiments have a moderate positive effect on students’ learning outcomes. This may be related to the fact that VR can create a richer and more diverse learning scenario that combines reality and reality, giving students more opportunities to operate and experience, which enhances students’ subjective experience and thus is more conducive to students’ construction and practice of knowledge, which in turn enhances cognitive strategies and improves technical skill learning. Although the results of the meta-analysis indicate that VR has a positive impact on students’ technical skill learning, there are different degrees of differences in its four types of moderating variables, suggesting that more in-depth research is needed to promote and popularize the application of VR in polytechnics.
5.2. Discussion of Moderating Factors
In terms of disciplinary categories, VR had the greatest impact on students’ technical skill learning in engineering (), followed by science (), with a relatively small effect on medicine (). Consistent with the findings of Cui and Zhao, virtual reality technology has a positive effect on teaching and learning in all disciplines, especially in science and technology, and a slightly weaker effect on disciplines such as life medicine [20] VR has the “3I” characteristics of imagination, interaction, and immersion, which are more conducive to the improvement of technical skills in operational subjects. In science and engineering courses, a large number of experimental operations are required. Virtual reality technology brings abstract theoretical knowledge to concrete practical operations, breaking the limitation of time and space so that students can practice repeatedly, thus improving the effect of technical skill learning. Compared with real medical experiments, the realistic effect of virtual experiments is relatively poor, and thus the impact is weak.
From the experimental cycle, the virtual experiment time of 40-60 minutes has the best effect on students’ technical skill improvement (), and the improvement effect of the experimental cycle above 60 minutes decreases instead (). The reason for this may be that students need a certain amount of adaptation time to shift their attention from technology to knowledge itself at the beginning of participating in virtual experiments, and as the learning time increases, students’ curiosity gradually decreases, and the learning effect naturally decreases [21]. Although VR can visualize abstract knowledge and provide a rich immersive learning environment, as the experimental period continues to grow, learners repeatedly operate the same experiment, as the scenario is the same each time, the brain will be in a state of inhibition, and the receptivity to information will decline, and natural learning motivation will be at a lower level, thus affecting the learning effect. Zhou and Li said such as the teaching model based on flow can better help students in VR scene situation of abstract knowledge [22], and feedback is an important way to help learners to build knowledge and cognition, is the precondition of speculative and creation, is the bridge of the students deep learning and deep learning machine, and can stimulate the learners’ learning in the improvement. Therefore, this study proposes a teaching model of “VR+simulation+feedback” based on flow theory. When students conduct simulation experiments with the help of VR, teachers pay attention to the performance of students in the virtual experiment and give appropriate feedback, such as classroom observation and process evaluation; simulation experiment, pay attention to the evaluation of operational skills; pen and paper test, focus on thinking ability evaluation.
In terms of the mode of assistance, virtual reality technology aided teaching in various ways had positive teaching effects, but the effects differed significantly, with VR facilitating more significantly as a form of teacher-assisted teaching () than VR applied independently to the whole process of teaching (). The research results show that VR teaching with the participation of teachers has a better effect on the improvement of students’ technical skills. The research results of Salomoni et al. also show that no artificial system can cooperate well with human beings [23]; that is, VR cannot completely replace traditional teaching methods. Indicates when the student into the visual and auditory-triggered immersive virtual environment, teachers should give full play to the role of leading people, class led the students feel the technology at the same time, effectively guide their performance will be technical skills in practice, should pay attention to the communication between students and teachers at the same time, and guarantee better interaction experience. In the process of education, even in the era of artificial intelligence, teachers are indispensable [24]. In the era of intelligent education, teachers should take the initiative to master information technology, give play to the innovation of educational means, and promote the normalization and diversified development of virtual simulation teaching. It provides a reference for colleges and universities to carry out a new model of personalized, intelligent, and ubiquitous experimental teaching combining online and offline under the epidemic. As existed in the work of experimental teaching in colleges and universities in science and engineering problems such as high cost, high consumption, or high-risk environment, the teacher fully using VR had advantages to create simulation environment, increase exploration depth fusion and innovation of VR and technical skills teaching application, build multivariate, flexible, and effective system of ecological education, and broaden the breadth and depth of the experimental teaching content.
In terms of learning base, the effect of VR on technical skill enhancement of learners with different levels of prior experience differed significantly, with a greater effect on learners with no prior experience level () than those with prior experience level (). The same conclusion was reached by Xuesong et al. Learners without prior knowledge experience would save irrelevant cognitive loss, would not occupy cognitive space beyond the knowledge itself, and would induce a keen interest in exploring the unknown [25]. The same experience reversal effect exists in the design of prerequisite questions and feedback in instructional videos, with low-experience learners maintaining the best performance by watching instructional videos with prerequisite questions without feedback, but not valid for high-experience learners [26]. The reason for this is the presence of prior knowledge experience. The reason for this is that learners with prior knowledge and experience focus more on making up for the shortcomings of prior knowledge, focusing on the differences between traditional physical lab learning and VR virtual experiments, while learners without prior experience focus more on the knowledge itself, establishing knowledge associations and learning skills, which are more conducive to technical skill enhancement.
6. Conclusion
This study conducted a meta-analysis of 32 empirical research literatures on the use of virtual reality technology to intervene in technical skill learning. The results show that technical skill learning based on virtual reality technology has a moderate positive impact on students in science and engineering colleges and universities. Evidence is provided for the applied advantages of cultivating high-quality technically skilled personnel. Through the analysis of different moderating variables, the study found that 40-60 minutes of VR intervention has the best effect on improving technical skills, and the promotion effect is more obvious as a teacher-assisted teaching method. Although our findings have important implications for educators, there are some limitations. For example, some studies using VR for teaching lacked sufficient statistical information to be included in the analysis, and most studies did not categorize technical skills, limiting our ability to conduct a more detailed analysis of the effects of technical skills enhancement. Despite these limitations, front-line educators can still draw on our research results to carry out VR teaching and explore more effective ways to use VR to promote the learning of technical skills.
Data Availability
The survey data are available upon request through email contact.
Conflicts of Interest
No conflict of interest exists in this study.
Acknowledgments
This work was supported by the 14th Five-Year Educational Science Planning Project of Chongqing (No. 2021-GX-014), the Postgraduate Research Innovation Project of Chongqing in 2022 (No. CYS22571), and the Special Project of Chongqing Normal University Institute of Smart Education in 2022 (No. YZH22019).