Abstract
Technological progress has brought about significant changes in education, and the development of intelligent speech systems has provided a new technical support and a broader resource platform for teaching and evaluating college English speaking. Evaluation is the basis for optimizing college English-speaking instruction (ESI); this paper explores the influencing factors of ESI and their relationship through PLS-SEM. Students’ course cognition, learning behavior, knowledge mastery, and teacher-student communication have a significant positive relationship with course satisfaction and classroom effect. On this basis, we make suggestions for optimizing college ESI from three aspects: resource acquisition, platform utilization, and teaching evaluation in combination with an intelligent speech system.
1. Introduction
With the trend of globalization and the reform and opening-up policy in the world, the role of English, as the most widely used language in the world, has become more and more important. The positioning of university English courses is to meet the national strategic needs on the one hand and to meet the needs of students’ professional learning and international exchange on the other. Speaking ability has become the biggest weak point and the most urgent part of English learners’ ability in colleges and universities, and optimizing the teaching of college English speaking is the aspect of college English course teaching that needs to be improved. With the progress of technology, the way of teaching and learning spoken English has undergone changes. In the 1950s, computer technology and listening method brought the speech room to English listening courses; in the 1960s, computer-assisted teaching began to enter the English speaking learning classroom; in the 1990s, network technology flourished and network-assisted language learning began to enter the English teaching classroom. The 21st century is the era of digital information and artificial intelligence, and intelligent speech systems are sprouting and developing. The future reform of college English course teaching will focus on speaking ability, assisted by modern technology, combined with modern information digital and artificial intelligence technology to assist ESI.
The design of the curriculum for teaching spoken English is the basis for the development of teaching activities. Yang et al. [1] believe that the innovation of ESI design mainly lies in the real experience and interaction. Nation and Macalister [1] specifically established a language course design model that emphasizes the process of designing a college ESI. Course objectives should be determined based on three components: course principles, course links, and demand elements, and course content, course implementation, and course evaluation systems should be reasonably constructed. Y. Liu and M. Liu [2] point out that the college English teaching mode under the background of artificial intelligence should be subject to listening teaching, speaking teaching, writing teaching, and translation teaching. Richards [3] divides English courses into three separate phases—before, during, and after class—based on language input, process, and output, focusing on a circular approach to curriculum design, in-class learning, and learning-oriented instruction.
Research on teaching and learning English courses with the assistance of online resources and mobile devices began in the 1990s and has continued to integrate with the ESI. Li [4] fully combined ESI with information technology and proposed an innovative multimedia interactive teaching mode based on the intelligent speech system. Han and Niu [5] point out that the goal of learning the English language is oral communication; virtual scenario can enhance the students’ ability of language expression and stimulates their learning interest. Nguyen et al. [6] show that listening, speaking, reading, and writing are important abilities in ESI. Through research, Yee et al. [7] found that students prefer to use technical tools such as PowerPoint to improve the effect of English speeches, which also reveals that teachers should use more information technology methods in ESI.
The evaluation of college ESI depends on the comparison between the expectations of educational service and the actual level of educational service perceived by the subjects of educational needs. It is both related to the process of education service and closely related to the effect of education service, and the specific influencing factors include teaching plan, teaching organization, teaching level, professional ethics, etc. Based on the research of a customer satisfaction model, this paper establishes a corresponding evaluation model of college ESI based on the actual situation of current college ESI. PLS-SVM is used to solve the problem of bias caused by small survey data samples, multicollinearity, and nonnormal distribution factors. On this basis, the results of the survey were used to conduct an empirical analysis of the evaluation of college ESI.
2. Evaluation Model of ESI Based on PLS-SEM
Like other disciplines, the teaching of college English as a foreign language has been under constant reform and innovation. In order to meet the needs of students, change the education management mode, base the teaching decision on the cultivation of students, refer to SCSB, ACSI, ECSI, and the National Student Satisfaction Report of the United States, establish the evaluation index system of college ESI according to the actual situation in China, and build the SEM causality concept model.
2.1. Conceptual Model
Through literature analysis, research interviews, and other methods, this paper constructs a conceptual model of causality from five dimensions, including course cognition, learning behavior, knowledge mastery, teacher-student interaction, and course satisfaction, with 20 items, as shown in Figure 1.
The degree of course cognition reflects the subjective value of students’ cognition of oral English courses. Cybinski and Selvanathan [8] conducted a survey on the learning methods and learning effects of college students and found that subjective feelings will affect students’ learning effects. In addition, research by Lai et al. [9] also showed that high self-perceived value has a positive impact on the behavioral intention of the courses studied. The research of Jiang et al. [10] has proven that students’ perception of oral complexity will greatly affect their oral performance, and automatic speech recognition technology can improve the speaking mastery of students. Therefore, this paper puts forward four measures of necessity awareness , practical awareness , importance awareness , and expandability awareness in terms of curriculum awareness.
Learning behavior refers to the habitual behavior of students in the process of oral language learning, and an evaluation index system is formed through constructivist learning theory and cognitive learning theory. Zhao et al. [11] and others [12–15] believed that the environment can influence the results of formative assessments. According to the above description, this paper proposes four measures of learning interest , course difficulty , in-class learning initiative , and extracurricular expansion learning initiative in the dimension of learning behavior.
The degree of knowledge mastery refers to Peter’s learning level classification and learning effect evaluation [16]; combined with the actual situation of oral English learning, the knowledge mastery degree is divided into theoretical knowledge mastery degree , learning thinking formation , oral proficiency level, and knowledge system construction degree .
Teacher-student interaction is a requirement at all stages of the teaching process, and the interaction between teachers and students can establish the connection between teachers, students, learning content, and learning methods [17]. Nguyen et al. [18] believe that teacher-student interaction plays a very important role in students’ learning. Through teacher-student interaction, the quality of teaching can be guaranteed and the learning effect can be improved. In addition, the oral learning experience of interactive communication has a positive effect on improving student performance and contributing to student satisfaction [19]. Based on this, in this paper, the degree of participation in theoretical learning is , the degree of participation in oral practice is , the degree of participation in communication with classmates after class is , and the degree of participation in communication with teachers after class is .
The measurement of course satisfaction refers to the theory of customer satisfaction and evaluates it from the perspective of students. The quality of the course is evaluated by the difference between the students’ subjective perception of teaching quality and the expected level before and after participating in college oral English teaching activities. The relevant measures proposed in this paper include the satisfaction of teaching content , the satisfaction of teaching method , the satisfaction of teaching logic , and the satisfaction of teacher level .
Based on the above analysis, this paper constructs an evaluation model of college oral English teaching through three independent variables, one dependent variable, and one moderating variable from students’ perception. Meanwhile, the following six hypotheses are proposed according to the relationship between variables.
H1: positive degree of course cognition has a positive impact on course satisfaction.
H2: positive student learning behavior profile has a positive impact on course satisfaction.
H3: positive student knowledge acquisition has a positive effect on course satisfaction.
H4: a positive correlation between faculty-student interaction and student course awareness and course satisfaction.
H5: teacher-student interaction is positively correlated with student learning behavioral profiles and course satisfaction.
H6: a positive correlation between teacher-student interaction and student knowledge acquisition and course satisfaction.
The relationship between the independent and dependent variables is reflected in Figure 1. Each scale entry was measured on a 5-point Richter scale (1 for “strongly disagree” and 5 for “strongly agree”). Missing data in the entries were filled by linear interpolation. The specific scales are shown in Table 1.
At present, most of the academics use partial least squares (PLS) to study teaching satisfaction in colleges and universities, and PSL is also the more appropriate method in studies in which the prediction of the relationship between model variables is the content of analysis and theoretical confirmation is the goal of analysis. The latent variables in the conceptual model of causality need to be assessed by constructing multidimensional measurable variables. The measurables established in this paper include the following steps.
Step 1. Establish the initial indicator system by referring to literature relevant to the study and consulting with relevant experts.
Step 2. Classification of indicators using the maximum mathematical fuzzy clustering method.
Step 3. Nonparametric tests for each subcategory to test for significant differences.
Step 4. For the group of indicators without significant differences, the indicator with the largest sum of squares of biased rank correlation coefficients for other indicators was selected by the rank correlation coefficient method.
There is no limit to the number of observed variables of potential variables, but the identification principle of SEM should be satisfied.
2.2. Method of ESI Evaluation
(1)Structural equation modeling (SEM)
SEM is a technique that includes a series of multivariate analysis methods such as regression analysis, factor analysis, and variance analysis. It is actually a statistical model and method for multivariate analysis by means of hypothesis testing. In order to explore the relationship between oral language teaching and latent variables, we can build a causal relationship model and confirm whether the model holds through statistical tests [20]. SEM can discover unrecognized conceptual relationships and reflect the element information. The main difference between SEM and path analysis is that the complete SEM includes measurement relationships, and the analysis process of SEM is shown in Figure 2.
The relationship between latent variables in our SEM will be defined as
In equation (1), . Recursive relations are represented by partial least squares (PLS), as shown in
and are the coefficients linking predicted endogenous variables with exogenous latent variables, while is the endogenous residual variable.
Combining equation (2), we can get
The relationship between the observed variable and the latent variable is defined as
The weight relationship is shown in
Based on the basic idea of SEM, we can draw the specific calculation process as shown in Figure 3. (2)Model parameter estimation method based on PLS
PLS is a multivariate statistical data analysis method, which projects the high-dimensional data space of the independent variable and the dependent variable into the corresponding low-dimensional space and establishes the univariate linear regression relationship between the eigenvectors of the independent variable and the dependent variable. Not only does it overcome the collinearity problem but also it removes the effect of unhelpful noise on the regression, allowing the model to contain a minimal number of variables.
PLS is often used to compensate for the limitations of SEM to better estimate model parameters. The external estimation is that the latent variables and are estimated by the linear combination of the and groups of measurable variables and , respectively, denoted as , . Let , be the external weights; then, the external estimation of the latent variable can be expressed as
The calculation process of internal estimation and is shown in
In equation (8), means to compress the data. , are internal weights; they represent the sign function value of the latent variable and its internal estimated correlation coefficient: , .
The weight coefficient and are very important in the PLS algorithm, and there are currently two methods that can be used to estimate them. and are covariance coefficients of the measurable variables with respect to the standardized quantities , respectively, as shown in
In the second method, the weights are the regression coefficient vectors of () with respect to the measurable variables, as shown in
The iterative algorithm of PLS is summarized as follows.
Step 1. Assign the initial weights arbitrarily; for example, one of them can be set to , and the other weights can be set to 0.
Step 2. Calculate new weights .
Step 3. Determine whether the new weights satisfy the convergence formula: , . If the condition is met, go to step 4; if not, go to step 1.
Step 4. , , , and are estimated values.
Step 5. Substitute the estimated values for the original values of the latent variables, and then, use PLS regression to estimate the model parameters.
3. Experimental Results and Analysis
In order to obtain the accuracy of the research conclusions and be closer to the actual situation of ESI, this paper distributes questionnaires in the School of Foreign Languages of Tianjin University to obtain sample data in the form of research. A total of 290 questionnaires were distributed this time, 243 questionnaires were recovered, 235 questionnaires were valid, and the effective rate was 81%. In this paper, descriptive statistical analysis, reliability and validity tests, and structural equation modeling are used to verify and analyze the sample data. Descriptive analysis is to test sample data through indicators such as mean and frequency. Reliability and validity analysis can further verify the scientific validity of the scale.
3.1. Descriptive Analysis
Descriptive analysis of the items of the scale is shown in Table 2.
It can be seen from Table 2 that the maximum, minimum, average, standard deviation, skewness, and kurtosis of the 20 items basically satisfy the normal distribution.
3.2. Reliability Analysis
Reliability refers to the consistency, stability, and reliability of questionnaire results. There are many methods of reliability evaluation, and Cronbach’s coefficient is more suitable in this paper, which is also known as comprehensive reliability. The observed variables corresponding to the same latent variable shall be consistent in theory, and Cronbach’s is used to measure the consistency between the observed variables corresponding to the same latent variable. The coefficients of this paper are greater than 0.7, indicating that the statistical data in this paper have good reliability, and the selection of items meets the requirements. The specific calculation results are shown in Table 3.
3.3. Validity Analysis
Validity is used to evaluate whether the observation variables designed in the questionnaire measure the latent variables well. This paper ensures the content validity of measurement variables by reading literature and consulting experts and scholars. Construct validity is used to measure the measurement of multiple indicators, including aggregate validity and discriminant validity.
According to Table 3, the compositional reliability of the five variables ranged from 0.68 to 0.864, and the average amount of variation extraction ranged from 0.442 to 0.577, indicating good aggregation validity.
In this study, the AVE method was used to test the discriminant validity. If the open root of AVE of each factor is greater than the correlation coefficient, it indicates that the discriminant validity is good. After calculation, the detailed data are shown in Table 4. According to the data in Table 4, the open square root values of AVE of the five variables are greater than the correlation coefficients of rows and columns. The coefficient between the degree of course cognition and satisfaction is 0.531, the coefficient between learning behavior and satisfaction is 0.442, the correlation coefficient between the degree of knowledge mastery and satisfaction is 0.479, and the correlation coefficient between teacher-student interaction and satisfaction is 0.496, indicating that the five variables are significantly related to satisfaction.
3.4. Model Test
(1)Main effect test
Based on the previous research, the initial model of structural equation is established. Studying the relationship between latent variables is the key to building the initial model, as shown in Figure 4.
Then, we use AMOS23.0 to test, and Table 5 includes details. The calculation results show that each fitness index meets the standard and has good fitting.
According to the statistical table of the model path coefficient in Table 6, the standardized coefficients of the degree of course cognition, learning behavior, and the degree of knowledge mastery and course satisfaction are 0.321, 0.197, and 0.252, respectively, and , indicating that these three endogenous variables have a direct positive impact on curriculum satisfaction and classroom effect, and the hypothesis is valid. (2)Adjusting effect test: hierarchical regression
It can be seen from Table 7 that the of the three endogenous variables are 0.422, 0.290, and 0.575, respectively, indicating that the fitting degree of the model is high. The statistical results such as value reveal that the overall regression effect of variables on curriculum satisfaction is significant. The regression coefficients of variables on curriculum satisfaction in the context of teacher-student interaction are 0.293, 0.362, and 0.257, respectively, showing that the significant positive regulatory effect of is established.
Figure 5 summarizes the analysis results of the PLS-SEM method in this paper. According to the above analysis, the research hypotheses of this paper are valid. Students’ classroom perception, learning habits, mastery, and classroom satisfaction are very important to the improvement of ESI. Positive teacher-student communication will play a significant positive regulatory role between these three variables and classroom satisfaction and play a prominent role in the development of ESI.
4. Improvement Measures of ESI
The development of technology and the Internet has brought about significant changes in education, and the emergence of artificial intelligence has provided a new technical support and resource platform for teaching and evaluating spoken English in college. An intelligent speech system is a product of AI background, which provides technical support and learning platform for English speaking learning through AI technology. Through the questionnaire survey and the previous discussion, the teaching of spoken English in college should be based on strengthening the communication and interaction between educators and educated people, and the intelligent teaching of spoken English in college should be realized through the intelligent speech system. Therefore, this paper proposes the improvement and optimization of college ESI from three perspectives: learning resource acquisition, intelligent speech system interaction, and intelligent evaluation of speaking teaching.
Learning resources determine students’ motivation and self-motivation. Learning is a long-term, complex cognitive process, and traditional fill-in-the-blank education is gradually being eliminated with the development of information technology, making it difficult for students to construct a knowledge system efficiently and to have high quality in a single solidified learning. If we can organically integrate learning contents, activities, difficulties, question types and evaluation systems, and design learning resources that are novel and close to life and current events, we can promote students’ interest in independent learning. The development of technology makes it more convenient for students to obtain resources from the Internet, and teachers can also obtain oral learning resources in rich and varied forms. Learning methods include dubbing, reading along, and human-computer dialogue. By building a reasonable personalized learning platform through the intelligent speech system, a variety of high-quality speaking practice resources from all over the world can be efficiently accessed. By fragmenting English learning resources, a great movie clip, a celebrity speech, or an English class program can be used to help students expand their knowledge and learn as much as possible about the customs of native English-speaking countries in non-English speaking countries.
The intelligent speech system platform can provide a new platform for teaching spoken English. The traditional speaking classroom is full of situations where the teacher instills knowledge in the classroom and students bury their heads in notes, which in the long run will demotivate students and defeat the fundamental purpose of speaking teaching. With the support of the intelligent speech system, students have more diverse channels to acquire knowledge, and the teacher’s role changes from being the authoritative instructor and the only source of knowledge to being the guide of learning and the organizer of classroom activities. In the speaking practice session, the teacher no longer just lectures but encourages students to complete the corresponding tasks through cooperation and debate by setting tasks. By highlighting the students’ main position, the teaching objectives are achieved. In the intelligent speech system platform, the teacher needs to assign preclass pretraining tasks for students, and students download the materials through the platform and study according to the task list. Any questions can be asked on the platform at any time, and the teacher and other students can answer for them. The integrated construction before, during, and after class continues the speaking practice time and reduces the difficulty of teachers’ speaking teaching. With speaking practice as an output language, teachers need to assign diverse tasks according to students’ ground base, interest range, and topical questions and integrate movie dubbing, role play, English songs, speech debates, etc. into the teaching activities. Students can actively experience pronunciation skills in topics of interest, including alliteration, swallowing, and flashing. Teachers can also rely on the digital teaching platform to assign review tasks and assignments as a way to urge students to consolidate and improve their knowledge, thus achieving the purpose of teaching speaking.
The evaluation of learning effect is an effective driving force to promote students’ active learning. With the help of the intelligent speech system, students can get rid of the limitation of classroom and textbook, and the online and offline teaching has greatly changed the way of oral teaching evaluation. With the help of big data technology, students’ individual differences are highlighted, and the results are made more objective and scientific by teacher evaluation, intelligent speech system scoring, and student self-evaluation. The evaluation content also changed from a single voice intonation to learning attitude, pronunciation skills, emotional engagement, and knowledge expansion. At the same time, with the assistance of the intelligent speech system, prereading and review, independent learning, and online oral expression are recorded on the digital online platform. The teacher is able to make a comprehensive score based on the formative assessment recorded by the system and the summative assessment in the final exam, which can effectively improve the students’ oral learning performance.
It provides all-round assistance for college English speaking learning from three perspectives: resource acquisition, application of the intelligent speech system platform, and evaluation of speaking ability, integrates speaking learning into daily life, gives full play to the utility of the intelligent speech system, and then gives feedback into course teaching through the teacher’s end to effectively optimize college ESI.
5. Conclusion
The factors affecting ESI are complex and numerous, and they interact with each other, ultimately acting on students’ course satisfaction and teaching effectiveness. In order to ensure the rationality and scientificity of teaching evaluation and improvement, this paper adopts the PLS-SEM method to deeply analyze the influencing factors of ESI and finally find out the improvement path based on the intelligent speech system. We have obtained three endogenous variables: the degree of course cognition, learning behavior, and the degree of knowledge master, as well as an important regulatory variable teacher-student interaction, and the evaluation of ESI is reflected by the measurement of course satisfaction. The results of the study showed that endogenous variables were strongly and positively correlated with course satisfaction and that faculty-student communication played a nonnegligible positive moderating role. Combined with the statistical analysis results, taking improving students’ classroom satisfaction as the main line, on the basis of actively promoting the communication between teachers and students, focusing on improving positive classroom perception, learning habits, and students’ mastery, and relying on the intelligent speech system, builds a multidimensional ESI promotion system, which is significant for improving students’ speaking skills.
Due to the limitations of the research conditions, the evaluation method in this paper can be improved. The evaluation data in this paper uses the traditional questionnaire method, but nowadays, intelligent speech systems have a variety of functions that can be combined with methods such as deep learning, and indicators such as classroom communication can then be judged by expression recognition methods. We hope to include more objective methods in subsequent studies to enhance the scientific nature of the study.
Data Availability
The labeled dataset used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no competing interests.
Acknowledgments
This work was supported by the project of Hebei Province National Social Science: “Research on overseas communication between Hebei culture and one belt and one road from cross-culture perspective” (no. HB19YY024).