Abstract
The study’s aim was to develop an intervention program and to evaluate its contribution to students’ self-regulated learning (SRL) and text analysis skills. In a student-focused training approach, the students themselves acquired the training strategies, whereas in the teacher-focused training, the teachers were enabled to explicitly impart these strategies to their students. In order to investigate the effectiveness of the intervention in terms of transfer benefits on SRL and text analysis skills, 274 lower secondary students were examined in a pretest-training-posttest design. Based on two different training approaches, a distinction was made between four groups: student training (singleST), teacher training (singleTT), combination of student and teacher training (ComT), and control group (CG). Substantially more transfer was revealed in all training conditions as compared to the control group. Specifically, the singleST group showed the highest learning gains for all variables. Conversely, a combination of both approaches (ComT) did not result in synergetic effects, but rather in reciprocal interferences.
1. Introduction
In the last few decades, the continuous technological revolution resulted in an efficient processing of information at any time and place as well as in a more flexible learning management. The omnipresent availability and complexity of information, however, necessitate the acquisition of new abilities in order to achieve the level of skills that is necessary to manage the demands of today’s society. For this reason, getting used to the continuous and lifelong need to adapt one’s own behavior and knowledge through self-evident and conscious application of strategies requires starting early with strategy instruction and promotion. Different training programs in all areas of life, whether at preschool [1], primary school [2], secondary school [3], or in the workplace [4], have already shown to be effective in enhancing learning processes and encouraging learners to acquire more self-regulatory learning abilities. Self-regulated learning (SRL) is defined as an “active, constructive process whereby learners (the term “learner” refers to students throughout the document, as the study focuses on an academic setting) set goals for their learning and then attempt to monitor, regulate, and control their cognition, motivation, and behavior, guided and constrained by their goals and the contextual features in the environment” [5]. For the purpose of the present study, we aimed at refining guidelines for a training of self-regulatory skills. The important new insight we like to gain is how this training can be implemented in a modern learning management system and how it may help to also foster general text structuring skills as a part of the analysis of Latin texts.
One possibility which has shown to be effective in training students’ learning [6] is the use of learning management systems (LMS) such as Moodle (modular object-oriented dynamic learning environment). Such platforms present the challenge of dealing with open learning scenarios and applying self-regulation processes while studying, which has turned out to be difficult for students [7]. Generally, students today are quite familiar with Internet-based tools but are not specifically used to employ them for learning purposes [8].
To introduce a new way of integrating SRL into classroom instruction, this study adopted an innovative approach by employing the online learning platform Moodle as a medium of intervention for promoting students’ self-regulatory abilities and strategic competencies when dealing with Latin texts.
In terms of its conceptual development, the present study formed a close connection to previous intervention studies that dealt with the support of SRL [9]. However, the realization and implementation of the training program was novel and applied to the domain-specific field of text analysis using the largely uninvestigated subject of Latin as example. This proceeding benefits from the findings of previous studies which showed that the impartment of cross-curricular strategies should always be combined with the teaching of domain-specific strategies [3]. The domain-specific strategies in this study will refer to text analysis strategies which will be considered as conscious activities undertaken by the students to analyze complex texts in Latin.
At this point, the question arises as to why Latin was our domain of choice. The answer is that, first, we were driven by our interest in how far even so-called “dead languages” leave considerable scope for integrating the support of SRL. Second and more importantly, we wanted to offer the prospect that this combination of SRL and Latin text analysis leaves potential to improve general skills in analyzing texts which can be helpful also with regard to dealing with texts in other modern languages. This might help to derive new insights into the findings of Haag and Stern [10] who disenchanted the myth of the usefulness of Latin.
A decisive assumption is that dealing with complex Latin texts can be regarded as linguistic problem solving [11] because the unusual structure of original Latin sentences can challenge today’s young people to successfully analyze the structure and meaning of the texts. That is why we wanted to establish a common ground from which the students will be equipped with strategies that support the text analysis process.
The practical implications of this paper are in sum to present an approach to promoting SRL and text analysis skills using the example of Latin, which can be integrated into a regular class setting by means of a LMS.
However, a critical question in the context of fostering SRL still is which role the teachers might play in such a classroom intervention on SRL. Teachers should take into consideration that students might need support to develop and to optimize their SRL skills when they are confronted with complex learning situations like the domain of text analysis. Although results of constantly expanding research on SRL have revealed its relationship with academic achievement [12], teachers seem to lack knowledge and have little confidence in how to enhance SRL [13] and how to adequately realize an implementation in the classroom [14].
In the present study, we therefore also varied different training approaches in order to not only train students but also to encourage teachers to take part or to quicken the formative process of SRL abilities in classrooms. We thus differentiated between a student-focused and a teacher-focused intervention approach.
2. The Transfer Hypothesis
The effectiveness of a training intervention is expressed by positive transfer to the general competency level [15]. Such transfer can be measured by indirect gains of training-induced benefits on related but new assessments or tasks after the actual intervention phase (i.e., on transfer tasks). In this regard, task performance is considered to be a function of an underlying general ability and a task-specific component. If the training and the transfer tasks (partly) overlap on the general ability level, a basis for the transfer benefit is provided. In the case that different abilities are trained together in the course of a multicomponential intervention (present study: SRL plus text analysis strategies), gains moreover reflect the transfer projections onto the intersection of those competencies, that is, their common latent ground or shared variance [15]. Two indicators of transfer will be distinguished in the present study: self-reported transfer means the generalization of training-induced improvements that is subjectively perceived and reported by the intervention’s addressees. In contrast to that, actual transfer refers to the measurable competency change in objective task performance. In the context of SRL, competency changes in SRL and domain-specific abilities are often assessed by means of both self-reports [16] and objective achievement markers (e.g., standardized test scores [17]).
In the present study, a link between SRL and text analysis skills was postulated. For that purpose, we measured (a) self-reported transfer on general SRL competency, (b) self-reported transfer on text analysis competency, and (c) actual transfer on the combination of SRL and text analysis competency by means of an objective text-structuring task. As can be seen from this, the objective transfer measure used in this study refers to the joint latent share of both the acquired SRL and domain-specific skills. However, a joint transfer presumes some common ground between the trained competencies on the latent level for which a basis was prepared in this study.
2.1. The Common Ground of SRL and Latin Text Analysis
The present study was founded on SRL theory and on considerations regarding text analysis, aiming for a conflation of these two concepts. Our study’s model (cf. Figure 1) is based on the framework of self-regulation [5] that emphasizes a procedural character of SRL comprising phases (planning, monitoring, control, and reflection) during which different areas (cognition, motivation, behavior, and context) are regulated. In the present study, the learners’ engagement in all phases of the learning process was analyzed whereby not all regulation areas were taken into account for each phase. During the planning phase, the learner approaches the task at hand and employs cognitive activities, such as goal setting strategies [5] that are particularly useful for structuring the completion of the task and serve as guiding principles that help to measure individual progress. Additionally, behavioral factors concern the management of time which includes the control of the amount of time the learner can spend on certain activities with the aim to make the learning process more efficient and productive (e.g., by reserving fixed time slots for homework). In the monitoring and control phase, the learners increase the awareness related to their own learning behavior. Monitoring and control processes [18] come into use including metacognitive judgments and awareness of the learning behavior which can result in an adjustment (e.g., change of strategy application or effort). Motivational control integrates the use of self-motivation and reward strategies (e.g., meeting friends after studying [17]), whereas contextual control focuses on the creation of optimal learning conditions (e.g., elimination of distractions). As there is little empirical evidence for a definite separation of monitoring and control processes [19], we combined both phases, which makes our model comparable to the cyclical model of SRL by Zimmerman [20].
During the reaction and reflection phase, the learners’ performance and adequate strategy application are evaluated on the basis of their attribution style [21]. Depending on the result, future planning and goal setting are influenced so that a new process of learning is initiated. In this context, the adoption of a positive attitude toward mistakes helps to regard failure as an opportunity to sustain the learning process [22].
In parallel to the SRL process, three phases were distinguished [23]: decoding (text preparation), recoding (strategies for transposing the text’s content into the target language), and restructuring (strategy evaluation, correcting). A well-decoded text provides an excellent foundation for breaking up the complex structures of the Latin language and launching comprehensive text processing. Hence, the training program focused on the instruction of the following decoding strategies [24].(a)Organization strategies are targeted toward text structuring and help to organize the information in the first reading. They include analyzing the grammatical elements and syntactic functions [25].(b)Elaboration strategies are useful to develop a first understanding of the basic messages of the text (e.g., thinking about the headline, taking notes while reading the text). In this connection, the activation of prior knowledge (e.g., “What do I already know about this?”) plays an important role because it incites a linkage of concepts.
While recoding, word choice and expression have to adhere to target language norms in order to recode the original text accurately. Linked to this is a profound semantic analysis. It sensitizes students to uncover specific meanings of words and sentences and to ask themselves questions and formulate expectations with respect to the thematic progression, to make sure that the text is understood.
After completion of the restructuring phase, the learners are requested to evaluate their strategy application and to make corrections aiming at a close conformation to conventions of the target language.
The final model (cf. Figure 1) implies that SRL skills help to track whether the use of text analysis strategies has led to a satisfactory processing and understanding and therefore provides a framework appropriate for conceptualizing a translation oriented text analysis.
This model obviously points to the overlap between SRL and text analysis in Latin with regard to their inherent procedural character and their close strategic link. We assumed that this overlap would canalize the transfer of trained skills towards the structuring of language-learning tasks that are different from the trained ones.
2.2. Trainability of Self-Regulated Learning and Latin Text Analysis Skills
To date, many intervention programs which aim at fostering SRL have been proven to result in training-induced transfer to SRL competency and academic achievement in different domains [27]. The evaluation of these interventions showed that SRL can be regarded as a valuable supplement of domain-specific strategy instruction because it increases the effects on the competencies that are in the focus of the domain concerned (e.g., reading comprehension, mathematical problem solving). Generally, studies have shown that the connection to domain-specific competencies is most effective with regard to the promotion of both cross-curricular and subject-related abilities [3, 28].
Most of the existing studies have operated in the fields of mathematics [27], reading [24], and writing [29]. No study so far addressed the question of investigating the beneficial effects of text analysis skills in Latin, although this might provide more general tools for language strategy processing due to its particular demands on strategic decryption. Even if deemed to be a “dead” spoken language, Latin carries unique, valuable clues for general text processing and analysis [30] as well as for dealing with (written) language [31, 32] which can be of huge value for students working with texts. For the purpose of the present study, we therefore aimed at extending former research by investigating the transfer benefit to the uninvestigated field of text analysis in Latin class.
2.2.1. Design Principles
As far as dealing with information in web-based or hypermedia learning systems is concerned, research has revealed that learners have deficient skills [33]. In general, the investigation of SRL processes in connection with virtual learning environments is still quite rare but should increase in view of the growing practical implementation of LMS (e.g., Blackboard, Moodle). First results of recent studies have indicated a positive impact of open learning contexts in virtual format on students’ learning development, particularly showing that Moodle as an LMS is an efficient medium to promote SRL and to raise students’ motivation and knowledge [34]. These findings prompted us to create a web-based learning scenario via Moodle in order to provide a learning environment that is beneficial to the learners’ self-regulatory abilities and text analysis competencies as possible.
As far as the form of intervention is concerned, different variations have been shown to be effective for the promotion of SRL [35]. For the purpose of the present study, we will briefly introduce the distinction between implicit and explicit teaching of learning strategies. On the one hand, SRL can be promoted by teaching strategies without informing the students about their certain importance (implicit) or by particularly demonstrating the students the meaning of a certain strategy (explicit). Since empirical studies have revealed the importance of explicit strategy trainings [36–38], the idea of the present study was to explicitly demonstrate the usefulness of SRL strategies by offering a learning environment which is favorable of the unfolding of self-regulatory skills. The variation that was made concerned the addressees of the intervention program. In one condition, the intervention program was targeted at the students themselves whose skills were intended to be improved in an explicit way. In the other condition, the teachers were addressed and trained in explicitly assisting the development of their students’ SRL abilities and in creating a learning environment that favours the direct promotion of those abilities. Additionally, there was a third condition, in which both the students and their teachers participated in a training program so that mutual spill-over effects were possible. Both intervention programs (for students and for teachers) were implemented into the learning management system Moodle because it makes the impartment of explicit strategies possible (e.g., by a constant exchange between trainers and learners, by the possibility of giving the learners immediate feedback) and helps to provide a learning environment that is conducive to the promotion of SRL.
3. Research Goals
Our research interest focused on investigating whether the conclusions from the framework of SRL and text analysis would hold in a web-based setting implemented in Latin class. To test the effectiveness of an intervention program in terms of increasing students’ general SRL and text analysis abilities (on the level of self-reported as well as actual transfer), different forms of training were applied (student-focused/singleST, teacher-focused/singleTT, and combination/ComT). Our major assumption was that each training form would have a significant positive impact on students’ SRL abilities and text analysis skills and thus that all training groups (put together called experimental groups) would outperform a control group (CG) which was not involved in the training procedure. We intended to explore whether different effects between the single training groups could be identified and assumed that the singleST group would outperform the singleTT group. This assumption was in line with previous findings [39, 40] and aimed to test existing effects in a new context. An open question was whether a combination of both training approaches would lead to higher learning gains in students (synergisms) than would the training forms separated from one another [15].
4. Method
4.1. Research Design
The field study employed a factorial quasiexperimental design to investigate the effects of student training/ST and teacher training/TT on self-reported (students’ SRL competency, student’s text analysis skills) and actual transfer (intersection of SRL and text analysis skills). For the intervention, we applied two factors that were embedded in a longitudinal design, with two measuring points (time 1: pretest/time 2: posttest) and a time interval of three weeks. Interaction effects between type of training and time fell under the scope of investigation. The participants were assigned to one of four conditions: singleST (single student training), singleTT (single teacher training), ComT (a combined training), and CG (control group).
4.2. Participants and Procedure
Participants included 336 students altogether from secondary school (174 girls, 158 boys, and 4 not specified) who were in their third or fourth year of learning Latin, with an overall mean age of 15.87 (SD = 7.98) years. They were recruited from two German federal states of Western Germany. The recruiting process was partially randomized: any grammar school in the catchment areas that offered Latin as a subject was contacted and asked for its interest in participation. After confirmation, the teachers were given encompassing information sheets for students and parents, which contained the study’s details, the central principles of data privacy, and a reference to the voluntary nature of participation.
The sample was matched by the propensity score matching procedure [41] in order to correct for sample selection bias and baseline differences between the experimental groups (singleST, singleTT, and ComT) and the control group, arriving at a reduced sample of 274 students (143 females, 127 males, and 4 not specified) with a mean age of 15.12 years (SD = 1.13). The data of this more homogenous sample were applied to the analyses, whereby the numeric distribution to the single conditions was the following: singleST (), singleTT (), ComT (), and CG ().
4.3. Intervention
Between pre- and posttest, the intervention was conducted differentially for the various training groups. The contents of singleST and singleTT were parallelized, but the teaching techniques differed from each other. In the singleST, the students autonomously dealt with the contents that were provided via Moodle, whereby strategies were explicitly taught and discussed with external trainers. In contrast to that, the singleTT focused on giving teachers instructions on how to explicitly impart the given strategies to their students.
4.3.1. Student-Focused Training (singleST)
The complete student-focused training program was divided into three succeeding thematic units in line with the phases postulated in the framework the study is based on (cf. Figure 1). Both self-regulatory and domain-specific Latin text analysis strategies were dealt with. Two external trainers were especially skilled in the training contents and conditions. They gave feedback within a tight time frame regarding the worksheets that the students had completed and uploaded. It was warranted that the students received help around the clock and that the web-material was permanently updated.
Self-Regulatory Content. In an introductory section, the students read up on some knowledge-units about general principles of SRL and its significance for the learning process. In the subsequent units, the strategies were specifically trained and discussed.
In unit 1, principles of goal setting (the setting of realistic goals) and time-planning (creating time-schedules) were taught, before units 2 and 3 went on further to introduce the students to principles of self-motivation (strategies for overcoming procrastination), dealing with distraction (strategies for avoiding negative thoughts and distracting working conditions), self-evaluation (reflection of one’s own learning), and favorable attribution styles (attributing success to internally stable factors, for example, one’s abilities).
Worksheets were used to raise students’ awareness of the meaning and significance of self-regulatory strategies and to encourage them to have a try at applying these strategies. Consequently, the students were asked to create time-schedules for their learning both at school and at home for the period of the training and to follow realistic goals. In order to practice self-motivation strategies, the students had to analyze a cartoon that depicted a totally unmotivated person that was paralyzed with indecision and needed directions for how to motivate himself. In the discussion forum they exchanged suggestions for the effective dealing with distractions and practiced the method of cognitive reframing by which the students learned to stop negative thought processes by substituting them with more positive self-talk [42]. After each unit, the students were constantly invited to monitor their progress by means of a short questionnaire. In the last unit, the students were introduced to self-evaluation strategies and were finally asked to judge their progress and validate it according to their goals they set at the beginning. In this context, favorable attribution styles and dealing with failure were important topics. The aim was to help the students to accept failure as part of the learning process [22] and to attribute it more to external, uncontrollable factors.
Text Analysis Content. The students practiced the text analysis ability by means of Latin prose texts whose focus was on Cicero’s orations against Verres. This type of text was chosen because, due to their structure and thematic issue, prose texts are particularly suitable for the training of text analysis skills. Besides, the thematic focus chosen had the largest intersection with the requirements of most curricula of the participating schools. In the introductory part of the training program, a handout introduced the focal points of the trial of Verres as well as the historical background. In order to activate their prior and to consolidate their new knowledge, the students were asked to participate in a quiz. Furthermore, the meaning and importance of text analysis strategies was highlighted in a short handout and a strategic map was provided as download in order to give the students an idea of how to strategically proceed when working with Latin texts.
The strategies that were explicitly taught in unit 1 were elaboration and organization strategies. Elaboration strategies involved thinking about the headline, taking notes while reading the text, clarifying unknown vocabulary, and formulating a main idea. In this connection, the activation of prior knowledge played an important role because the students had to recall what they had already learned about the topic. The usage of organization strategies required the structuring of the text by underlining key words, highlighting syntactic functions (e.g., subject, predicator) and relations between words (e.g., agreement between noun and adjective in gender, number, and case) or sentences (e.g., existence of conjunctions). The students uploaded their first text structuring via Moodle and received feedback in terms of the quality of the text preparation from the external trainers. Based on the feedback, the students proceeded with unit 2 which focused on the recoding of the texts. The aim was that extracting the meaning of the sentences would result in the semantic recognition of the whole text. The recoding process was supported by integrated questions that asked the students for their expectations in terms of the thematic progression of the text (e.g., what do you think will happen next?). Again, the students uploaded their texts they had produced in their target language (German) and received immediate feedback in terms of their accuracy and consistency. In unit 3, finally, the students were asked to evaluate their strategy application, to analyze common mistakes, and to discuss them with the other students in a forum. In order to experience the whole process of text analysis once more and to evaluate one’s own strategic behavior, a text puzzle was provided. The idea of that puzzle was to reconstruct the meaning of a Latin text that was presented in mixed-up parts. For that purpose, the students had to orient themselves both on syntactic and semantic cues in order to find the logic behind the text.
4.3.2. Teacher-Focused Training (singleTT)
The teacher training was a thematically adapted version of the student training (cf. Table 1) which was also designed as a web-based program via Moodle. The teachers were trained in acting as multipliers who explicitly pass strategies they acquired on to their students. The aim was the creation of a conducive environment which allows for the enhancement of self-regulatory behavior and the application of text analysis strategies.
Just like the student-focused training program, the teachers’ training was divided into three succeeding thematic units in line with the model the study is based on (cf. Figure 1). Both self-regulatory and domain-specific text analysis strategies were dealt with. The teachers also received feedback and support within a tight time frame regarding the worksheets they had completed and uploaded.
Self-Regulatory Content. The introduction was formed by a face-to-face kick-off seminar during which the general proceedings were presented. In addition, the teachers were introduced to the thematic concept by offering a basic input on the importance of self-regulatory skills and raising awareness with regard to their own state of knowledge. The three subsequent units concentrated on providing guidance and suggestions with regard to the impartment of SRL strategies. For that purpose, the teachers were trained in how to teach realistic goal setting and in how to encourage students to create time-schedules for better time-management. A central component of units 2 and 3 was the promotion of students’ motivation (e.g., by providing positive and individual feedback and by supporting students’ autonomy and self-determination) in order to positively influence the students’ efforts, self-evaluation, and attribution styles. The participants were asked both to upload worksheets that they had to process and to discuss issues among each other in order to exchange experiences.
While working on the different units, the teachers were requested to implement the training contents in class. All units were complemented with download files that consisted of detailed suggestions and prepared material for the realization in the classroom.
Text Analysis Content. The participating teachers all taught Latin and were trained in introducing the topic of Cicero’s orations against Verres according to the suggestions of the training program. In the introductory part of the domain-specific contents, the teachers were asked to discuss possibilities of how to activate the students’ prior knowledge and to try to use the handout and quiz for that purpose. Furthermore, the strategic map was suggested as a valuable tool to illustrate the structural process when working with texts. In unit 1, the teachers were guided in the teaching of elaboration and organization strategies when preparing a text for the semantic analysis. In this context, the importance of generating a first idea based on relevant information was addressed. Additionally, the use of highlighters to mark key words, syntactic functions, and relations between words or sentences was taken into consideration. Unit 2 dealt with the recoding of texts and with suggestions of how to induce the semantic analysis (e.g., to integrate questions that encourage the students to think about the thematic progression of the text). Unit 3 finally stimulated the teachers to find ways of teaching evaluation strategies that focus on the accuracy and logic of the text production and strategy application. For that purpose, the teachers were animated to employ text puzzles in order to check whether the students are able to apply the strategies that they had taught them.
4.3.3. Combination (ComT) and Control Group (CG)
In the ComT, the student-focused and teacher-focused interventions were assembled. Hence, the teachers and their students were trained simultaneously in the manners outlined above. In this condition a very intensive exchange and explicit discussion of the training contents was intended because both students and teachers were involved and familiarized with the training strategies. The CG, however, did not receive any form of intervention. Thus, teachers and students underwent their habitual lessons without any added information about the self-regulated translation process. Notwithstanding, the teachers of the CG were asked to assure that comparable prose texts were handled in the classroom so that a similar base activity to the training groups was warranted.
4.4. Instruments
In general, the effectiveness of our training program was evaluated through the triangulated assessment of students’ self-reported data combined with objective data, such as test scores, and the extent to which students actually applied the trained strategies. Additionally, in line with Leard and Hadwin [43], log-file data viewable in Moodle were taken into consideration, with which the offline data could be validated. The instruments will be described on the students’ level only.
4.4.1. Self-Reported Transfer
General SRL Competency. A standardized questionnaire was used to measure students’ self-reported transfer on SRL competencies. The responses were rated on a scale ranging from 1 (definitely not true) to 4 (definitely true). The content validity of the questionnaire was ascertained employing items from established instruments [44]. Some items were newly developed or logically modified, if necessary. The internal consistencies for the overall SRL skills and the various subordinated scales can be seen in Table 2 and can be rated as satisfactory.
In order to assure the convergent validity, we interrelated the ascertained transfer score on SRL competency in the posttest with two training indicators.(a)Trace data: Moodle allows the collection of digital traces of students’ learning behavior in order to construct profiles of the frequency of SRL activity across participants. Thereby, a connection between actual and self-reported learning behavior can be identified which is increasingly demanded in the field of research [45]. The correlation between online and offline assessment is generally reported to be low [46]. In our study, the extent of SRL (overall) judged by the questionnaire correlated significantly .33 () with the corresponding trace data. Thus, we could presume an acceptable validity of the used self-assessment questionnaire referred to the training sensitivity.(b)Content analysis of students’ submitted worksheets: in order to gain insight not only into the frequency of using Moodle but also into the quality of use, we examined the quality of student’s work on the Moodle training tasks as a second validity criterion. We rated the quality of content of the student’s submitted worksheets on a grading system from 0 to 100%. We collapsed the separate task-performance scores across training time resulting in a total assessment of the quality of training performance. This quality score was then correlated with the self-reported SRL skills and, as a result, a substantial correlation of () between content quality and SRL was attained. Thus, this training marker indicated a satisfactory criterion-related validity of the questionnaire scores at the posttest.
General Text Analysis Skills. The quantification of the self-reported transfer on general text analysis skills (scaled from 1 to 4) referred to the students’ subjective strategic self-assessment by means of questionnaire items that asked them to estimate their decoding, recoding, and restructuring abilities. Satisfactory reliabilities were revealed for all scales (cf. Table 3).
4.4.2. Actual Transfer
Intersection of SRL and Text Analysis Skills. The actual strategy use which was influenced by the strategic components of both cross-curricular SRL and domain-specific textual analysis skills was assessed in the context of a standardized experimenter-designed Latin translation test for two points of measurement. The tests each consisted of a text that was kept in prose form and was oriented toward Cicero’s rhetorical work De oratore (text passagepretest 2, 217–290; text passageposttest 2, 18, 75). Both versions of the text were conceptualized comparably regarding length and the amount of difficulties (e.g., grammatical constructions). In order to ensure content validity, the texts were designed in consultation with experienced Latin teachers. For appraisal of the convergent criterion-related validity, the students’ pre- and posttest translation achievement scores were correlated with the Latin marks of a recent classroom exam (Time 1: , ; Time 2: , ) and with the last report marks (Time 1: , ; Time 2: , ). The German point system for the 2nd secondary school was used with 15 points being the best grade and 0 being the worst.
The discriminatory power was satisfactory for all sections of both achievement tests ( each > .60). In addition, the texts showed a medium level of difficulty (; ). A certain aspiration level was indispensable in that the students had to be challenged to first decode the text strategically rather than directly start translating it word by word. The texts were subdivided into nineteen parts. The test evaluation was standardized: two independent raters used a prepared schema and were blind to each other’s scores, arriving at adequate interrater reliabilities (Cohen’s ).
The quantification of the actual transfer on the intersection of SRL and text analysis skills referred to visible structuring and meaningful marking of the text and was rated from 0 (no marking) to 3 (each syntactic function highlighted [e.g., subject, predicator, and object] and all relations between words [e.g., noun-adjective agreement] or sentences underlined [e.g., conjunctions]). This arrangement of the texts’ structure specifically challenged a synergistic coaction of both SRL and text analysis skills.
4.5. Information with regard to Data Analysis: Propensity Score Matching
Based on the propensity score matching method [41], the subsamples of the aggregated experimental groups and the CG were matched to control for baseline differences regarding a circumscribed set of relevant variables. Thereby, we aimed at creating a more homogenous pool of control participants. We checked some relevant sociodemographic variables (sex, actual year of learning Latin) and initial competency markers (recent Latin test and report mark, baseline-competencies of SRL, and Latin text analysis) for pretest discrepancies. There were no significant level differences for any of the listed variables except for the initial Latin text analysis skills, , . When gathering this control variable, an auxiliary matrix program matched each experimental subject with a statistical twin from the CG. Thereby, we chose a threshold value of medium height () (we conducted various matching trials starting with a very high -value of .50 that was successively diminished (iterating matching procedure). -values higher than .1 failed to level the baseline translation differences. -values lower than .1 resulted in considerable sample reductions. Therefore, we stated with the .1-solution a -value of medium height and, respectively, reached a medium matching quality arriving at a more homogenous reference group than in the starting solution) in order to warrant a justifiable balance between dyad resemblance and sample size. In the resulting homogenized sample, the baseline differences were leveled, , .
5. Results
5.1. Analysis of Pretest Differences
For the purpose of further controlling the data, we also examined potential pretest differences between the separate training groups that had previously been collapsed for the propensity score matching procedure (singleST, singleTT, and ComT). On the level of the sample’s demographics, we found significant discrepancies concerning the students’ sex, , , and their recent Latin test mark, Kruskal Wallis , [47]. Therefore, these two variables were controlled as covariates in all subsequent group analyses.
With regard to the baseline competency in respect of the study variables, no pretest differences were found.
5.2. Data Analysis
Table 4 summarizes the adjusted mean scores and standard deviations of the groups for the different following analyses. For convenience, contrast analyses and trend analyses for the separate groups will be reported on the overall scales only. Results are differentiated by significance at the .05 level and those at the adjusted -level using Bonferroni corrections for the number of comparisons made (; ; ). To examine the range of transfer effects in the different training groups, we calculated effect sizes (ES) by Cohen’s or the standardized mean difference in performance between pretest and posttest.
5.3. Self-Reported Transfer
Self-Regulated Learning Competency. To investigate self-reported transfer of the differential training approaches on SRL competency, data were subjected to a three-way MANCOVA with the between-subjects factors ST(+/−) and TT(+/−) and the within-subjects factor time (pretest/posttest) on the subscales of planning, monitoring/control, and reflection. The multivariate analysis revealed a significant three-way interaction (cf. Figure 2) for the overall scale SRL, , , , and so did the univariate analysis for the scales planning , , and reflection , , . To disentangle the latter interaction, we specified three group contrasts based on the combined factor levels. First, we collapsed all training groups and compared them with the control condition (singleST, singleTT, ComT versus CG). Second, we compared the two single training groups against each other to reveal the superior training approach in between (singleST versus singleTT). Third, we also contrasted the combination of both approaches against the single training groups (ComT versus singleST, singleTT).
As proposed, the training groups reported together higher improvements on self-regulatory abilities from pre- to posttest than the CG, , , . In line with our assumptions, we found that singleST significantly outperformed singleTT, , , . A substantial difference between the single groups and ComT was not revealed, , .
Regarding group-wise effects, there was a significant increase of singleST, , , with a large ES of , while singleTT and ComT stagnated in their competency. CG, however, decreased its SRL skills significantly from pre- to posttest, , , reaching a medium ES of .
Text Analysis Skills. To examine training-induced benefits on self-rated transfer to text analysis skills, data were again subjected to a three-way MANCOVA with the between-subjects factors ST(+/−) and TT(+/−) and the within-subjects factor time (1/2) on the subscales of decoding, recoding, and restructuring. The expected three-way interaction could not be found to be significant, , . Nevertheless, there was a significant two-way interaction for the overall scale, , , , as well as for the scales of decoding, , , , recoding, , , , and restructuring, , , . This result indicated that the effect of ST did not depend on the levels of the TT factor (i.e., the gains of ST could be pronounced irrespective of the actual manifestation of TT). The groups with student training grouped together (singleST, ComT) estimated their strategy use to be increased from pre- to posttest, , , with a common ES of , whereas the groups without student training (singleTT, CG) recorded no significant change, , .
5.4. Actual Transfer
Intersection of SRL and Text Analysis Skills. Looking finally at actual transfer benefits on both SRL and text analysis skills, data were subjected to a three-way ANCOVA with the between-subjects factors ST(+/−) and TT(+/−) and the within-subjects factor time (1/2). For the actual application of strategies, a significant three-way interaction (cf. Figure 3) was revealed, , , . A subsequent contrast analysis based on the same group contrasts as in the previous section resulted in a significant difference between all training groups and CG regarding their increase of strategy application, , , . Beyond that, singleST and singleTT showed nearly identical change patterns, , , and so did ComT as compared to the single training approaches, ,
Regarding the transfer scope, all groups recorded a uniform improvement of their actual strategy application, , , , , , , , , with ES for the training groups ranging from to , and a much smaller ES for the control condition () which failed to meet the criterion requiring that ES should be larger than to reach practical relevance [15].
6. Discussion
The primary aim of this study was to investigate the effectiveness of different training approaches implemented in an LMS for enhancing SRL and text analysis skills. We were successful in identifying important results that contribute to both the field of research and methodology for teaching.
In general, we found evidence for the effectiveness of our intervention program in terms of self-reported transfer to SRL and self-rated transfer to the knowledge of text analysis strategies as all training groups taken together differed significantly in their learning gradient from the control condition. Looking specifically at the development of the separate conditions (singleST, singleTT, ComT, and CG), we identified singleST as being superior.
6.1. Self-Reported Transfer on SRL Competency
Regarding transfer on reported general SRL skills, the significant three-way interaction between ST, TT, and time indicated that the difference in terms of an increase of SRL abilities between the groups with student training (singleST, ComT) and those without student training (singleTT, CG) over time was moderated by the presence or absence of the teacher training (i.e., the effect was more obvious in case there was no teacher training). In fact, the intervention led only in the singleST group to a substantial increase in abilities from pre- to posttest; that is, self-reported transfer on SRL competency was only ascertainable in the separate student training group, yielding high practical relevance (cf. ). A similar pattern of results arose for the specific subscales of the SRL process whereby the three-way interaction was only revealed for planning and reflective skills rather than for monitoring and control skills. To disentangle this interaction as well on the univariate level, pairwise comparisons yielded once again that the singleST outperformed the singleTT, which was an additional indicator of the predominance of the student training over the teacher training.
In this context, several previous studies demonstrated that interventions are more effective when external trainers are involved in the intervention [39] rather than leaving the implementation to the teachers alone. This finding might also apply to our results, even though beforehand we had been hopeful to make a success of encouraging teachers to take the role of autonomous multipliers and to impart SRL strategies to their students by themselves. Such a success would have been insightful from an applied perspective as a teacher-directed setting is easier to implement in the everyday classroom context. However, we failed to obtain this expected benefit despite our innovative teacher intervention approach via Moodle in the context of Latin teaching. Presumably, the teachers still could not adequately apply or communicate with the SRL strategies they were taught in the training to the classroom. This confirms evidence of a strong lack of general knowledge in teachers about how to enhance SRL [48] and of an insufficient proficiency in implementing SRL content into the classroom. Even if the teachers were explicitly trained in the impartment of the training contents and were familiarized with them, the results point to these deficits in teaching behavior being very resistant to change [24].
Beyond that, the lack of synergetic effects of singleST and singleTT in the ComT could even be due to singleTT counteracting the positive effects of singleST. The singleTT might have interfered with students’ processing of new strategies [49] because the teachers’ style of teaching deviated from their conventional practice and from the way strategies were provided in the singleST. Probably, the teachers were not able to support the open and self-determined learning environment that was offered in the singleST group. Because of that, interferences between self- and other-determined influences on the development of SRL skills might have occurred. Following the self-determination theory [50], encouraging students’ willingness to adopt new strategies autonomously (such as in the student training condition) might have been undermined when strategies were additionally imposed by the teachers (such as in the additional teacher training condition) even if they had been instructed to support an autonomous learning environment for their students and to impart new self-regulatory contents in a more subtle, though explicit, way. Yet as the teachers themselves were possibly unsure of their own self-regulatory skills, they might have struggled to realize such recommendations and instead reverted to old directive teaching patterns. Additionally, the students’ as well as the teachers’ intrinsic attitude toward new cross-curricular strategies they were confronted with might have moderated this interplay and might have led to problems of acceptance. As a result a mathematantic effect [51] might have occurred both on the student and on the teacher level. Thus, the moderating function of intrinsic interest in the learning of new strategies should be controlled in future investigations.
Against this background, we call for an intensive partnership between researchers and practitioners in order for teachers to rethink their practice and feel supported in the management of substantial changes in the classroom. As aforementioned, we believe in line with other studies [24] that the teachers of the singleTT approach—which aimed at providing teachers with usable didactical suggestions within a fully developed program—did not identify with the study’s concepts. As a consequence, to reduce barriers for the realization of interventions in general, the teachers’ awareness of new concepts has to be created as early as possible in their teaching careers [52] and their beliefs in enhancing SRL in particular have to be influenced before they are developed incorrectly based on their daily teaching practice [13]. In this context, studies have shown that teachers who are self-regulated with regard to their own learning and who believe in developing their students’ SRL skills are more likely to promote these competencies [53].
An explanation for the significant decline of SRL in the CG over time (albeit with a medium ES, cf. ) is provided by the fact that the individuals developed an intensified self-attention [54] and critical evaluation of themselves [54], triggered by the first employment of the questionnaire at the first measurement point. As SRL was assessed by means of a self-report questionnaire, it is assumed that the self-judgment of the individual’s own SRL abilities turned out to be more negative in the posttest.
The fact that the significant three-way interaction between ST, TT, and time was on the univariate level specifically found for planning and reflective skills rather than for monitoring and control skills is striking. In terms of these framing aspects of the SRL process, the student training approach appears to be more productive compared to additional teacher training. Indeed, distinctive planning skills that include the setting of realistic goals pave the way for a successful SRL process and are a precondition for being motivated and focused on learning [55]. Likewise, reflective skills allow for the evaluation of the learning outcome realistically compared to the self-set goals on the one hand and on the other hand, for the realization that strategic adjustments are needed when learning was not effective.
A combination of both student and teacher training approaches seems to lower transfer on these subcompetencies, though. This notwithstanding a rather synergetic effect could be found for the monitoring and control scale. When regarding the descriptive statistics (cf. Table 4), it is obvious that ComT could offset its lack of general transfer effects showing a remarkable increase specifically for monitoring and control abilities. Probably the synergy of singleST and singleTT intensified the awareness in terms of the importance of strategy use and therefore led to a conscious self-monitoring of learning behavior. To back up this assumption, singleTT in isolation was inhibitory to the unfolding of monitoring and control skills. This interpretation should be ensured by process analyses that could provide continuous data concerning the development of monitoring and control abilities.
6.2. Self-Reported Transfer on Text Analysis Skills
In terms of text analysis skills, reported improvement from pre- to posttest in the ST groups did not depend on the levels of the TT factor as revealed by a nonsignificant three-way interaction between ST, TT, and time. Conversely, only the ST factor seemed to moderate the transfer outcomes as suggested by the substantial respective two-way interaction. The conditions that included student-focused training (singleST, ComT) showed grouped together a large increase in their self-rated text analysis abilities (cf. ) as compared to the groups without student-focused intervention (singleTT, CG) which stagnated in their self-assessment. This result is an indicator of the transfer effect of the domain-specific training on text analysis skills within a student training intervention, independently of an involvement of the teachers [24]. In the absence of the student training condition, the training measure could therefore once again not affect the participants’ self-estimated knowledge of text analysis strategies, which could be related to the teachers’ rigid thinking of whether and how strategies are important and should be applied. This pattern of results was further reproducible on the univariate level for all phase-specific skills (decoding, recoding, and restructuring) of the text analysis process.
The effectiveness of the combined approach for the development of self-assessed knowledge of text analysis strategies implies that domain-specific strategies could be implemented and put into practice more successfully than the SRL strategies, which supports findings of an interconnection of domain-specific and cross-curricular strategies [3]. In this subject-related context, no reciprocal interferences between the student- and teacher-guided learning environments emerged, as the addressees of the training both were a priori more competent in domain-specific skills than in SRL strategies. Moreover, the conventional teaching habits in this domain should have been more familiar for students and thus leaving more room for the acquisition of the respective strategies than in a fully new context, where autonomous processes or self-determinative feelings are seriously threatened.
6.3. Actual Transfer on the Intersection of Both SRL and Text Analysis Skills
The results for the actual application of training-induced benefits on the competency levels of SRL as well as text strategy knowledge on the objective text structuring task challenging both abilities showed a slightly different result pattern. All groups made significant progress up to , whereas this increase was not substantiated in the CG (cf. ). Furthermore, a difference between the treatment conditions grouped together (singleST, singleTT, and ComT) and the CG emphasized the latter result, indicating that the experimental groups yielded significantly more improvement. Remarkably, there were much smaller differences between the training groups on actual joint SRL and text processing transfer as compared to self-reported gains on each competency by itself. The teacher-focused training in particular seemed to be more beneficial on the actual transfer level than suggested by self-report results. However, the actual-transfer task resembled the most the everyday school tasks of students when dealing with Latin texts. Thus, teachers might have been indeed successful in imparting strategic contents to their students, particularly when referred to domain-specific skills, and these gains could be expressed by students on the application level. On the other hand, teachers failed to sharpen the students’ more abstract self-concept on each competency by itself (SRL, text analysis skills), while in the student-focused condition, students became more aware of their gains in strategy knowledge and thus could better incorporate them into their self-identity, reflected also in greater self-report of transfer.
7. Conclusion
To conclude, the results supported the assumption that Moodle served as a successful platform for enhancing both students’ cross-curricular and domain-specific strategies, whereas the additional teacher training approach seemed at least effective for the impartment of subject-related competencies. In terms of SRL competencies, more guidance as well as a closer cooperation must be offered to the teachers in order to familiarize them with the importance of SRL. The involvement of teachers in the implementation of a training program by means of an LMS was innovative and helped to recognize deficits, but also potential. The contrast of student and teacher training interventions and combination of both have not yet been empirically examined very intensively, let alone being realized in a design. Therefore, the present study was a further decisive step in raising the teachers’ awareness for the importance of promoting cross-curricular competencies in their domain and also to sensitize students to the facilitating role of having a broad spectrum of strategies. Moreover, the successful use of Moodle as a means of teaching could encourage and inspire teachers to deploy learning platforms to create a more open and self-determined classroom.
Nonetheless, there are limitations to the study that should be pointed out with regard to future studies. First, we expect larger effects in terms of the internalization and automation of strategy report and actual usage [56], particularly for ComT in cases where the intervention duration is longer than three weeks. This expectation is substantiated by the findings of other studies that yielded the effect sizes of intervention programs increasing with the number of training sessions [57].
Furthermore, it has to be taken into account that the analyses of transfer effects were to a large extent based on students’ self-report data. In particular, in terms of measuring SRL processes, self-report questionnaires are restrictive because the extent to which the students can actually regulate their learning is difficult to reconstruct [58]. Maybe, this could have even lowered the comparability of our two different transfer layers (self-reported/actual) and contributed to the differing results in the teacher training condition. Hence, there is a call for more online measurements, such as think-aloud protocols or direct observation, because they correspond to the learners’ actual behavior and do not depend on their precise self-estimation [45]. To take this into account, we examined the relations among students’ trace data, their worksheets’ content, and self-reports, whereby the significant correlations were moderate and should be validated against other online measures [59]. The problem of trace data is that the researcher can register behavior only at the object-level and has to infer the self-regulatory nature underlying that behavior. Nevertheless, even though online measures also have their weaknesses [60] and self-report measurements are at least applicative for the assessment of a general aptitude in using self-regulatory processes [61], future research should strive for multimethod designs employing different assessment instruments in order to support construct validity [62]. The present study took a step in this direction by employing an LMS as a platform for directly training SRL and domain-specific strategies. As training effects were achieved with the training program, it should be expanded into a robust concept, setting higher priority on the multimethod approach.
Finally, even though our self-reported and actual transfer results had been very promising, the present study merely focused on near-transfer gains, that is, transfer to tasks or assessments that were structurally similar to the trained ones (here measuring SRL and Latin text analysis skills). A next step could be to extend our findings to the level of far transfer, that is, transfer to structurally dissimilar tasks that are also tapped by the trained competencies, but further relate to other domains. In our example, it would be undoubtedly valuable to investigate such far transfer from beneficial Latin text analysis training to modern language text analysis skills. These gains would highlight how Latin can indeed provide general and broadly applicable tools for text processing and analysis and should then no longer be devalued as a “dead” but rather be valued as a “reviving” pioneer language.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.