Abstract
The research is aimed at verifying the application effect of the online automatic evaluation system in English translation teaching and at understanding the satisfaction of students with different feedback methods. The research uses three classes of human resource management majors in Xi’an Technological University as the research object and uses questionnaire survey and comparative experiment methods to compare and analyse the three feedback methods: teacher feedback, online automatic feedback, and teacher feedback combined with online automatic feedback. The research answers the following three questions: (1) will the three feedback methods affect students’ English translation performance? (2) Which of the three feedback methods will improve students’ English performance better? (3) What about students’ satisfaction with current feedback methods? The research results show that the significance value between the control group (CG) and the experimental group 1 (EG1) is 0.029, that between CG and the experimental group 2 (EG2) is 0.432, and that between EG1 and EG2 is 0.001. There are obvious differences in the posttest scores of the three groups of students. EG2 has the largest average posttest score, which is 9.8182; there is no obvious difference in posttest translation scores between CG and EG2. It indicates that “teacher feedback + online automatic feedback” and teacher feedback have the equivalent effect on improving students’ translation. The results of the questionnaire survey show that students have the highest degree of recognition of “teacher feedback + online automatic feedback.” The research is helpful for teachers to better understand the shortcomings in the translation teaching process, so that they can take effective measures against these problems in the follow-up teaching process to improve their teaching effect.
1. Introduction
In today’s environment, the technological revolution is booming. As an important driving force of the current industrial revolution, artificial intelligence (AI) is constantly changing people’s lifestyles. Human society will further develop into a new era of human-machine integration, cocreation and sharing, and intelligence [1, 2]. AI is the core technology of the current scientific and technological revolution, and it is a general trend to apply it to the field of education and teaching. With the rapid development of AI technology, the development of intelligent evaluation system (IES) such as http://www.pigai.org is also very rapid. When students’ translations and English compositions are corrected, the use of an IES greatly reduces the time required for review and reduces the pressure of reviewing teachers. Teaching feedback has gradually changed from previous teacher feedback to student mutual evaluation, system automatic feedback, etc., and teaching feedback methods have become more diversified [3–5].
In the 1960s, an IES appeared in the United States. At the end of the last century, on the development of artificial intelligence technology, IESs have also been developed rapidly, and some mainstream IESs have emerged, such as Intellimetri and E-rater. At the beginning, IES was mainly used to score the composition and translation in large-scale examinations and did not have functions such as revision and review [6]. At the beginning of the 21st century, IESs were used, such as Writing Roadmap and My Access, with a variety of functions such as personalized reviews, writing assistance, and automatic scoring. After the emergence of IES, many researchers have conducted related researches on its effectiveness. Some scholars have found through experiments that when reviewing a large number of students’ compositions, using IES can not only save teachers and students’ time but also evaluate its effectiveness, with high accuracy [7–9]. Some scholars are opposed to IES. Research has found that IES can only identify errors in grammar and vocabulary in students’ compositions but has inexhaustible evaluation effects on subjective issues [10].
In recent years, China has made some progress in the theoretical and empirical research on IESs. But online English translation and automatic feedback is far from enough, and traditional intelligent teaching systems are lacking in emotion. Therefore, the research takes http://www.pigai.org as the research object and compares the three feedback methods of intelligent feedback, teacher feedback, and teacher feedback combined with intelligent feedback and explores which teaching mode is conducive to improving students’ effect of learning. The research can enable teachers to understand the insufficiency of teacher feedback and intelligent feedback. Teachers can conduct targeted education on current problems, which is conducive to improving students’ translation ability and improving the efficiency of college English teaching. Because http://www.pigai.org adds an emotion recognition module to the traditional teaching system, it is constructed using text recognition and other technologies. Thus, it can recognize the emotions of students in learning and propose corresponding emotional incentives on the emotions of the students’ strategies to achieve emotional teaching.
2. Materials and Methods
To study the application effect of http://www.pigai.org, an automatic evaluation system, in college English translation teaching, the research compares the effects of online automatic feedback, teacher feedback, and the combination of the two feedback methods on students’ translation performance and answers the questions shown in Figure 1.
2.1. Research Object
Three classes of human resource management at Xi’an Technological University are used as the research object, with a total of 95 students. All three classes are taught by the same teacher. In order to ensure the effect of the experiment, the English scores, teaching progress, teaching methods, and other variables of the three classes studied are kept consistent. As the school stipulates that sophomore students must take the CETB-4 and the school will take the CETB-4 scores as part of the students’ final grades, compared with students in the other three grades, sophomore students pay more attention to English courses. Therefore, the research selects sophomore students as the research object. Before the experiment, the students in the three classes have already used http://www.pigai.org for composition correction, so no additional training is required. Set the three classes as CG, EG1, and EG2. The feedback method used by CG is direct teacher feedback, the feedback method used by EG1 is online intelligent feedback, and the two feedback methods used by EG2 are combined. The number of people in the three groups is shown in Figure 2.
The experimental group used http://www.pigai.org throughout the experiment to correct the composition and translation. In order to understand the specific situation of http://www.pigai.org in the process of students’ use and students’ satisfaction with http://www.pigai.org, the research conducts a questionnaire survey on 61 students in two experimental groups.
2.2. Questionnaire
The research uses a questionnaire method to understand students’ recognition of the current teaching feedback methods. The questionnaire used contains three dimensions and a total of 25 multiple-choice questions. The questionnaire is designed using a five-level scale, with 5 being very agreeable, with a decreasing degree of approval from 5-1, and with 1 being completely disapproved [11–13]. The main content and distribution of the questionnaire are shown in Figure 3.
2.3. Research Process
The http://www.pigai.org used introduces the text recognition function on the basis of the traditional intelligent teaching system. The text recognition function can adjust the learner’s emotion when the learner’s state is not good, so that the learner is always in a good state during the learning process. In the learning process, the learner is in a positive emotion, such a state is helpful for the student to continue learning. When the learner is in a negative emotion, make him stop learning, judge the type of his negative emotion, and carry out the corresponding incentive strategy. The whole experiment process can be divided into three phases: preparation phase, experiment phase, and data collection and analysis, which lasted for three months.
2.3.1. Preparation Stage
Before the experiment, the English translation level of the students in the three classes is tested to understand the English translation level of the students in the three classes and to ensure the fairness of the experiment. Before the test, in order to enable students to take the test seriously, the teacher told the students in advance that the test scores will be included in the final grade. The test questions used in this test are the real translation questions from the CETB-4 in December 2019. The full score of this test question is 15 points, and the scoring standard is consistent with the scoring standard of CETB-4 translation [14–16].
After the grading, a one-way analysis of variance is performed on the test scores of the three classes of students. The results showed that there is no big difference in the average English translation scores of the three classes of students, and the next experiment can be carried out [17, 18]. Set Human Resource Management Class 1, Class 2, and Class 3 as CG, EG1, and EG2, respectively. In order to control the experimental variables, the teacher in these three classes is the same person.
2.3.2. Experimental Stage
Students in the three classes will collectively organize a translation test every week, and they completed 10 translation tests during the experiment. After the test is completed, the feedback method of CG is teacher feedback, the feedback method of EG1 is online software intelligent feedback, and EG2 combines the above two methods for feedback [19]. The practice questions used in the translation test are all real translation questions from CET-4 over the years.
2.3.3. Data Collection and Analysis
After the above experiment process, in order to compare the learning results of the three classes over the past three months, the final test is conducted on the students with the CETB-4 translation questions in June 2020, and the scoring standards are consistent with the CETB-4 grading standards [20]. After the test is completed, a questionnaire is issued to all students in experimental class 1 and experimental class 2 to understand the satisfaction of the two classes with the current feedback method and the online correction system [21]. In order to ensure the quality of answering questionnaires, this questionnaire survey is conducted in the classroom, and the questionnaires are returned after class. A total of 61 questionnaires are distributed in this questionnaire survey, and 61 valid questionnaires are finally recovered.
The data collected in the experiment mainly include four parts, as shown in Figure 4.
The research uses SPSS25.0 to analyse the collected experimental data and uses Microsoft Excel to tabulate the acquired experimental data for subsequent analysis [22]. The whole experiment process is shown in Figure 5.
3. Results
3.1. Comparison of Translation Results before and after
3.1.1. Test of Translation Level before the Experiment
Before the experiment, in order to understand whether there are differences in the English proficiency of the three groups of students, the English translation level of the three groups of students is tested. The total score of this test is 15 points. After the test, two experienced English teachers will review the papers. Due to the subjective nature of translation scoring, the average score given by two teachers is used as the student’s final score. The final scores of the three groups of students in the pretest English are shown in Figure 6.
(a)
(b)
(c)
The results of a statistical analysis of the English scores of the above three groups of students are shown in Figure 7.
(a)
(b)
(c)
Figure 7 shows that the average preexperiment test scores of CG, EG1, and EG2 are 8.1765, 8.2581, and 8.3636, respectively, and the difference in the average preexperiment test scores of the three groups of students is very small. A one-way variance test is performed on the preexperiment test scores of the three groups of students, the value is 0.060, and the significance is 0.942, which far exceeded the significance level of 0.05. This result shows that the preexperiment test translation scores of CG and the experimental groups are not much different. Multiple comparisons of the preexperiment test scores of the three groups of students are performed, and the results are shown in Figure 8.
In Figure 8, I and J of 1-6 represent EG2, EG1; EG2, CG; EG1, EG2; EG1, CG; CG, EG1; and CG, EG2. Figure 8 shows that the significance value between CG and EG2 is 0.942, which is greater than 0.05 and does not reach the significant level, and the confidence interval of the two includes 0. The preexperiment test score difference between CG and EG2 is not obvious. The significance value between EG1 and EG2 is 0.982, which does not reach the significance level, and the confidence interval of the two includes 0. It indicates that there are no significant differences in the level of English translation between the two experimental groups. The preexperiment test scores of the three groups of students are not significantly different, and the next experiment can be carried out.
3.1.2. Posttest Translation Level
After the three-month experiment, the translation results of the three groups of students are shown in Figure 9.
(a)
(b)
(c)
Figure 10 presents the results of a statistical analysis of the English scores of the above three groups of students after the experiment.
(a)
(b)
(c)
Figure 10 shows that the average English translation scores of CG, EG1, and EG2 after the experiment are 9.2353, 8, and 9.8182, respectively, and the standard deviations are 1.79323, 1.65328, and 2.02260, respectively. The average score is significantly higher than the average score of the two experimental groups after the experiment. A one-way variance test is performed on the English translation scores of the three groups of students. The value is 8.161, and the significance is 0.001, which is far less than the significance level of 0.05, indicating that at least the difference between the average scores of the two groups is more obvious. In order to further explore the differences of students’ English translation scores, the research makes multiple comparisons of the English translation scores of the three groups of students after the experiment. The results are shown in Figure 11.
In Figure 11, I and J of 1-6 represent EG2, EG1; EG2, CG; EG1, EG2; EG1, CG; CG, EG1; and CG, EG2. Figure 11 shows that the significance value between CG and EG1 is 0.029, which is less than 0.05, indicating that there is a higher level of significance between CG and EG1. The confidence interval between CG and EG1 does not contain 0, indicating that the differences in the English translation scores of the two groups after the test are more significant. The significance value between CG and EG2 is 0.432, which is greater than 0.05 and did not reach the significant level. The confidence interval of CG and EG2 contains 0, which can indicate that the difference in English translation scores after the test between CG and EG2 is not obvious. The significance value between EG1 and EG2 is 0.001, which is much smaller than the significance index of 0.05, indicating that the English translation results of the two experimental groups after the test have a higher level of significance. The confidence interval of EG1 and EG2 does not contain 0, indicating that the difference between the English translation scores of the two experimental groups after the test is more significant.
3.2. Errors in the Translated Translation after the Test
According to the grading standard of the CETB-4 translation, the errors in the translation can be divided into the following aspects, as shown in Figure 12.
The research uses SPSS25.0 software to conduct a single-factor analysis of the problems such as “misuse of vocabulary” and “other minor errors” in the translations of the students after the test and discusses the above three types of problems in the translations of the three groups of students after the test. Which kind of feedback method used is beneficial to improve the accuracy of students’ translations that is further analysed?
3.2.1. Comparison of “Misuse of Vocabulary” in the Translations of CG and the Experimental Groups after the Test
Figure 13 displays the statistic about the “misuse of vocabulary” in the three groups of students after the experiment.
(a)
(b)
(c)
Figure 13 shows that the students in EG1 have the most “vocabulary use errors,” with an average value of 1.3677. In CG, the mean value of “misuse of vocabulary” is 1.1059. The students in EG2 had the least “misuse of vocabulary,” with an average value of 0.8303. The standard deviations of the students in CG, EG1, and EG2 are 0.26622, 0.56118, and 0.38119, respectively. Figure 14 shows the multiple comparison of the “misuse of vocabulary” among the three groups of students.
In Figure 14, I and J of 1-6 represent EG2, EG1; EG2, CG; EG1, EG2; EG1, CG; CG, EG1; and CG, EG2. Figure 14 shows that the significance value between CG and EG1 is 0.045, which is less than 0.05, reaching a significant level. The confidence interval between CG and EG1 does not contain 0, indicating that there is a certain difference in the scores of “word use error” between the two groups. The significance value between CG and EG2 is 0.029, which is less than 0.05, reaching a significant level. But the confidence interval between CG and EG2 does not contain 0, indicating that there are certain differences in the scores between CG and EG2 in “word use error”. The significance value between EG1 and EG2 is 0.000, which is much smaller than the significance index of 0.05, indicating that the English translation scores of the two experimental groups after the test have a higher level of significance. The confidence interval of EG1 and EG2 does not contain 0, indicating that the two groups have significant differences in the average scores of “word use error.”
3.2.2. Comparison of “Other Minor Errors” in the Translation between CG and the Experimental Groups after the Test
Count the “other minor errors” that occurred in the three groups of students after the experiment, and the results are shown in Figure 15.
(a)
(b)
(c)
Figure 15 shows that students in EG2 have the most “other minor errors” in their English translations, with an average value of 0.4839. In EG1, the mean value of “other minor errors” is 0.4839. The students in CG had the least “other minor errors,” with an average of 0.3775. The standard deviations of students in CG, EG1, and EG2 for “other minor errors” are 0.33158, 0.36857, and 0.3168, respectively. Figure 16 describes the multiple comparison of the “other minor errors” of the three groups of students.
In Figure 16, I and J of 1-6 represent EG2, EG1, EG2, CG; EG1, EG2; EG1, CG; CG, EG1; and CG, EG2. Figure 16 shows that the significance value is 0.453 between CG and EG1, 0.980 between CG and EG2, and 0.572 between EG1 and EG2. The significance value between the two groups is greater than 0.05, which does not reach the significant level. The three groups of students in CG, EG1, and EG2 have basically the same “other minor mistakes,” and there is no significant difference.
3.3. Data Analysis and Questionnaire Survey Results during the Experiment
3.3.1. The Completion of 10 Translation Tasks
The task completion status of CG, EG1, and EG2 during the experiment is shown in Figure 17.
Figure 17 shows that all students in CG, EG1, and EG2 have completed the 10 translation tasks assigned by the teacher during the experiment. The overall revision rate of CG is relatively high, but some students still did not revise the teacher’s feedback after the teacher’s feedback. The overall revision rate of CG is 89.31%. Most of the students in EG1 completed the feedback modification using http://www.pigai.org, and the overall modification rate is as high as 91.46%. Most students in EG2 can modify the comments on http://www.pigai.org and then submit them to the teacher. After submission, a small number of students did not modify the teacher’s comments. The overall modification rate of EG2 is 89.68%. Therefore, the completion rate and revision rate of the three groups of students for the 10 translation tasks during the experiment are relatively good.
A one-way analysis of variance is performed on the modification rate of the three groups of students for 10 translation tasks, and the results are shown in Figure 18.
In Figure 18, I and J of 1-6 represent CG, EG1; CG, EG2; EG1, CG; EG1, EG2; EG2, CG; and EG2, EG1. Figure 18 shows that the significance is 0.42 between the average of the translation task modification rate of CG and EG1. The significance is 0.988 between the average of the translation task modification rate of CG and EG2. The significance of the average modification rate is 0.347 between EG1 and EG2, the significance of the three groups is greater than 0.05, and no significant difference is reached. The experimental data show that there is no big difference in the revision rate of the translation tasks among the three groups of students.
3.3.2. Comparative Analysis of the Results of 10 Translation Tasks
Analyse the average scores of three groups of students on http://www.pigai.org for 10 translation tasks, as shown in Figure 19.
Figure 19 shows that students in EG2 have relatively better translation test scores and students in EG1 have the lowest average score. The scores of the students in CG, EG1, and EG2 all showed an upward trend, indicating whether it is teachers’ feedback, http://www.pigai.org’s feedback, or a combination of the two can effectively improve the student’s academic performance. In comparison, accepting feedback from http://www.pigai.org and teachers’ feedback is more conducive to improving students’ English translation performance, and accepting only http://www.pigai.org changes has the least significant improvement in student performance.
3.3.3. Questionnaire Survey Results
To answer the three questions in Figure 1, the research uses SPSS25.0 to test the results of the questionnaire. The results are shown in Figure 20.
(a)
(b)
In Figure 20, E1 and E2 represent EG1 and EG2, respectively. The students in EG1 and EG2 scored more than 3 points for http://www.pigai.org, and their expectations for feedback methods are the highest, 3.9821 and 3.8822, respectively. The two groups of students rated the current feedback method of http://www.pigai.org relatively high, 3.5253 and 3.8528, respectively, indicating that the students in the experimental groups are relatively satisfied with the feedback method of http://www.pigai.org. From the overall score of the experimental groups, the overall score of experimental group 2 is significantly higher than that of EG1, indicating that students are more satisfied with the feedback method that combines teacher feedback with http://www.pigai.org feedback.
4. Conclusions
All the data and analysis obtained answer the following three questions: (1)The impact of three feedback methods on students’ translation performance: through one-way analysis of variance on the translation scores of the three groups of students, the significance of the scores of CG and EG1 after the experiment is 0.029, which is less than 0.05, indicating a higher difference between CG and EG1. The significance of the results of CG and EG2 after the test is 0.432, which is greater than 0.05, which did not reach the significance level. The significance of the results after the test between EG1 and EG2 is 0.001, much less than 0.05, indicating that there is a higher level of significance between them. There are obvious differences in the English translation results of the three groups of students after the test, indicating that different feedback methods will cause more obvious differences in the students’ academic performance.(2)The most effective feedback method to improve translation performance: from the data obtained in (1), the English translation score of experimental group 2 is the highest after the test, which is 9.8182; the English translation score of experimental group 1 is the lowest, which is 8. There is no obvious difference in the English translation scores between CG and EG2 after the test. It indicates that the teacher feedback and the combination of teacher feedback and online feedback have no significant difference in the improvement of students’ academic performance. Compared with online automatic feedback, it has obvious advantages.(3)Students’ attitudes towards current feedback methods: from the results of the questionnaire survey, the average value given by the students in EG2 is 3.9114, which is significantly higher than the average score of experimental group 1. The significance value between EG1 and EG2 is 0.046, which is less than 0.05, reaching a significant level, indicating that students are more satisfied with the feedback method that combines teacher feedback with http://www.pigai.org feedback.
Both teacher feedback and http://www.pigai.org feedback have their own advantages. Teacher feedback can effectively improve students’ academic performance, is more accurate in finding errors in students’ translations, is targeted, and can effectively reduce students’ translation process the error occurred in. Due to the limited time and energy of teachers, it takes a long time to correct the translation, so it is difficult for teachers to provide timely and effective feedback to each student. The feedback method of http://www.pigai.org can effectively compensate for some problems in teacher feedback. http://www.pigai.org is fast, and the feedback is very detailed, which can effectively improve the learning efficiency of students and can make up for the lack of emotion in the traditional intelligent teaching system. Using text recognition and other technologies, students’ emotions during learning can be recognized, and corresponding emotional motivation strategies can be proposed to achieve emotional teaching. However, the intelligent feedback system still has the problems of correcting errors and not correcting errors and other problems that need to be solved urgently, and it is still necessary to keep pace with the times and continue to improve.
Data Availability
The data that support the findings of this study are available upon request.
Conflicts of Interest
The authors declare that they have no competing interests.