About this Journal Submit a Manuscript Table of Contents
Education Research International
Volume 2013 (2013), Article ID 309894, 12 pages
http://dx.doi.org/10.1155/2013/309894
Research Article

Socioscientific Decision Making in the Science Classroom: The Effect of Embedded Metacognitive Instructions on Students' Learning Outcomes

1Department for Biology Education, Faculty of Biology and Psychology, Georg-August-University Göttingen, Waldweg 26, 37073 Göttingen, Germany
2Center for Research on Education and Human Development, German Institute for International Educational Research (DIPF), Goethe-University Frankfurt, Schloßstraße 29, 60486 Frankfurt, Germany

Received 2 October 2012; Revised 23 November 2012; Accepted 29 November 2012

Academic Editor: Bracha Kramarski

Copyright © 2013 Sabina Eggert et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The purpose of the present study was to examine the effects of cooperative training strategies to enhance students' socioscientific decision making as well as their metacognitive skills in the science classroom. Socioscientific decision making refers to both “describing socioscientific issues” as well as “developing and evaluating solutions” to socioscientific issues. We investigated two cooperative training strategies which differed with respect to embedded metacognitive instructions that were developed on the basis of the IMPROVE method. Participants were 360 senior high school students who studied either in a cooperative learning setting (COOP), a cooperative learning setting with embedded metacognitive questions (COOP+META), or a nontreatment control group. Results indicate that students in the two training conditions outperformed students in the control group on both processes of socioscientific decision making. However, students in the COOP+META condition did not outperform students in the COOP condition. With respect to students' learning outcomes on the regulation facet of metacognition, results indicate that all conditions improved over time. Students in the COOP+META condition exhibited highest mean scores at posttest measures, but again, results were not significant. Implications for integrating metacognitive instructions into science classrooms are discussed.

1. Introduction

Over the past decades curriculum authorities as well as science educators and researchers worldwide have called for changes in the way science is taught at schools (e.g., [14]). Modern science education should not only foster the acquisition of scientific content knowledge but engage students in scientific inquiry, in lifelong learning and in discussions about modern science problems, their technological applications as well as their personal and societal implications [15]. In a similar vein, the implementation of socioscientific issues into the science classroom has been proposed for more than two decades (e.g., [610]). Socioscientific issues represent modern science problems, such as global climate change or the loss of worldwide biodiversity, that are tightly linked to social, political, and economical concerns (e.g., [11]). They are complex, real-world scenarios at the interplay between science and society and thus, can no longer be solved by relying on scientific knowledge only [8, 10, 11]. Consequently, they fundamentally challenge the aims and scope of traditional science instruction.

A growing body of research within the area of science education highlights the notion that the implementation of socioscientific issues into science classrooms can enhance students’ learning outcomes with respect to conceptual scientific knowledge as well as reasoning and argumentation skills (e.g., [8, 1215]). Ottander and Ekborg found that interest in socioscientific issues correlates with self-reported learning outcomes in science education [16]. In addition, they have the potential to prepare students’ becoming literate citizens (e.g., [8, 10]). However, working with complex socioscientific issues also poses high cognitive demands on students, because students need to engage in various information search and evaluation processes as well as argumentation, reasoning, and problem solving processes. This also involves the ability to take perspectives and to integrate multiple perspectives into the development of solution strategies. Thus, implementation of learning settings that enable students to engage in peer interactions and motivate them to argue, to reason, and to negotiate how to solve these problems and hereby participate in discourse on modern science problems is crucial (e.g., [7, 11, 17, 18]).

Moreover, learning about complex issues needs to be carefully structured, as prior research also showed that students can easily be distracted when working on socioscientific issues where the outcome is uncertain [16, 19]. Embedded metacognitive guidance or self-regulated scaffolds have widely been regarded as one means to meet these ends. Among the most prominent approaches that use cooperative-metacognitive settings are Palinscar and Browns’ reciprocal teaching method to enhance reading literacy, King and Kitcheners’ reflective judgment model, or Kings’ Guided Peer Questioning [2022]. Based on the seminal work of Polya, Mevarech and Kramarski developed the IMPROVE method to activate students’ metacognitive skills during mathematical problem solving to enhance students’ mathematical achievement [23, 24]. Within science education, Mevarech and colleagues also used this method to enhance students’ scientific inquiry skills [25]. Azevedo and colleagues could show that facilitation of self-regulated learning can improve student achievement on complex science topics [26]. However, still only few studies exist that analyse the effects of such metacognitive or self-regulated learning settings on students’ socioscientific decision making and reasoning (e.g., [27, 28]). The present study aims to contribute to this research need. It analyses the effects of embedded cooperative-metacognitive trainings on senior high school students’ reasoning and decision making about socioscientific issues.

1.1. Effects of Cooperative-Metacognitive Learning Settings on Student Achievement

Cooperative learning has been on the international agenda for more than half a century by now both in educational research and in educational practice (e.g., [2932]). Research on the effects of cooperative learning is traditionally rooted either in social or cognitive psychology. While social psychologists take on a motivational or social cohesion perspective on cooperative learning, cognitive psychologists often refer to mental information processes that are stimulated by cooperative learning (e.g., [32]). From a constructivist point of view, new knowledge can only be attained if it is connected to and integrated into prior knowledge (e.g., [33]). While learners interact with each other, they provide explanations, engage in discussions, develop arguments about complex problems, and reflect upon the topic and tasks at hand. These peer-to-peer interactions can lead to deeper processing of information, facilitation of higher-order thinking skills, and construction of profound knowledge. Thus, they are likely to enhance individual achievement (e.g., [34, 35]).

Numerous research studies could actually show that cooperative learning has beneficial effects not only on student achievement, but also on student interest as well as social skills [3032, 36, 37]. As a consequence, cooperative learning has often been euphorically advocated as the optimal learning strategy [34, 38]. However, empirical research also highlights the notion that cooperative learning is not per se more beneficial than other learning settings [34, 38]. Merely putting learners into small groups will not lead to interactive group work and meaningful learning (for an overview see [34]). Referring back to the works of Johnson and Johnson as well as Slavin, cooperative learning settings need to account for positive interdependence and individual accountability, promote face-to-face interaction, and foster interpersonal and social skills to be successful (e.g., [29, 32]). Moreover, cooperative groups need to be able to monitor and reflect upon their learning processes [29, 32]. Especially this last aspect has been identified as one crucial factor for successful collaboration (e.g., [20, 22, 23, 35]) Typically, these studies provide support measures in the form of metacognitive guidance or self-regulated learning trainings to support students’ elaboration and learning processes [20, 22, 23, 35, 39].

Metacognitive guidance has been extensively used and analysed in the area of mathematics education (e.g., [40]). Mevarech and Kramarski developed the IMPROVE method to enhance students’ mathematical reasoning [23]. Central to IMPROVE are metacognitive questions that can be differentiated into comprehension, connection, strategic and reflection questions [23]. Comprehension questions address the main idea of the problem or the task to be solved. Connection questions support students in analysing similarities and differences between the current task and tasks that were solved in the past. Strategic questions ask students to reflect on the specific strategy that might be appropriate to solve the task. Finally, reflection questions ask students to either monitor their learning or problem solving process during or at the end of the process. Mevarech, and colleagues showed in a series of studies that students who studied under the IMPROVE method outperformed students who studied under traditional, more individual instruction or under cooperative instruction that was not additionally structured by metacognitive guidance (e.g., [23, 24]). In addition, they could show that a metacognitive instruction using IMPROVE did not only have immediate but also delayed effects [41]. Furthermore, Mevarech and Fridkin showed that an intervention using IMPROVE did not only foster students’ mathematical knowledge and reasoning but also their metacognitive skills [42].

Within the area of science education, fewer studies explicitly implement cooperative-metacognitive trainings or self-regulated learning in science classrooms. Zion and colleagues transferred the use of IMPROVE to an intervention study on scientific inquiry in microbiology [25]. They showed that students who studied under IMPROVE in a network technology environment outperformed those groups that had no metacognitive guidance. Moreover, Azevedo and colleagues showed that students who studied about complex scientific issues in self-regulated learning settings with embedded scaffolding outperformed students who studied in self-regulated learning settings without any additional scaffolding [26, 43].

Metacognition and self-regulation are often treated as two separate concepts in the literature [44, page 223]. However, this is not due to the fact that they are different concepts but that they originally stem from two research fields: developmental psychology and educational psychology [44]. There are still ongoing discussions about defining the relations between metacognition and self-regulation, but a common ground seems to be that metacognition can be seen as a part of self-regulation in that self-regulation can be described as the dynamic interaction of cognitive, metacognitive and motivational aspects of learning [4446]. Metacognition is typically defined by two components: knowledge of cognition and regulation of cognition (e.g., [44, 47]). The former is described as knowledge about one’s own cognitive functions and is often differentiated into declarative, procedural, and conditional knowledge (e.g., [44, 47]). The latter is typically regarded as control of one’s own cognitive activities and typically refers to processes such as planning, monitoring and evaluation [44, 47, 48]. As the present study aims to enhance students’ learning outcomes by using the IMPROVE method, the theoretical basis of the present study refers more to the concept of metacognition than to the concept of self-regulation.

1.2. Enhancing Students’ Socioscientific Reasoning and Decision Making

Socioscientific issues represent controversial issues of modern science that involve social, political, economical, and ethical considerations [8, 10, 49, 50]. Examples for socioscientific issues are loss of worldwide biodiversity, but also bioethical dilemmas or biotechnology issues such as genetic engineering. They often represent issues of first frontier science or “science in the making” [49, page 294]. They have their basis in science, but can no longer be solved by relying on scientific evidence only [8, 11]. Instead, they are factually and ethically complex and do not have a clear-cut solution (e.g., [8, 10, 15, 51, 52]). Moreover, multiple solutions exist that all have their drawbacks [8, 10, 15, 52, 53]. New solution strategies have to be developed by integrating multiple, often competing, perspectives. In addition, socioscientific issues and solution strategies are subject to ongoing inquiry and are often based on uncertain, fragile and conflicting evidence [8, 10, 50, 53].

Working with socioscientific issues in the science classroom poses high processing demands on students because they are engaged in various information search and evaluation processes as well as argumentation and reasoning processes (e.g., [12, 15, 52, 54]). As socioscientific issues cannot be solved on the basis of “simple cause and effect reasoning” [10, page 375], students first need to understand and describe a socioscientific issue in its complexity. Second, they need to be able to generate solutions that account for multiple perspectives on the issue, and third they have to be able to critically evaluate developed or existing solutions (e.g., [55]).

There is empirical evidence that students can be promoted with respect to socioscientific decision making and reasoning. Several studies focused on the quality of argumentation and reasoning processes while dealing with socioscientific issues (e.g., [7, 1215, 27, 54, 56]). Results showed that students can be trained in developing pro and contra arguments, in using trade-offs to compare possible solutions and in weighing arguments or decision criteria to reach an informed decision [7, 1215, 27, 54, 56].

Few studies exist that analyzed the effect of embedded metacognitive or self-regulated trainings on students’ socioscientific decision making and reasoning. Gresch and colleagues showed in a pre-post-follow up control-group design that a web-based training program with additional metacognitive prompts to support task analysis enhances students’ socioscientific decision making with respect to “evaluating solutions” [28]. Labuhn and colleagues showed again in a pre-post-follow up control-group design that self-regulated learning elements can be successfully integrated into science classrooms. In addition, they showed that students who studied in a self-regulated learning condition outperformed students who studied under traditional instruction on a knowledge test about decision-making processes [57]. Eggert and colleagues used the IMPROVE method in an intervention study among seventh graders to enhance socioscientific decision making (“evaluating solutions”) with respect to the issue of river assessment and renaturation [27]. Results showed positive effects in both training groups. Students in the IMPROVE condition performed better at posttest measures, but the effect was not statistically significant. However, results from this study are promising that metacognitive guidance can have a positive impact on students’ socioscientific reasoning and decision making.

1.3. Objectives of the Current Study

On the basis of existing research, we aimed to investigate the effect of two cooperative training strategies on students’ socioscientific reasoning and decision making. As described above, working on socioscientific issues is a complex endeavor. We assume that cooperative learning settings will provide learners with multiple opportunities to engage in peer-to-peer interactions that are needed to reason and argue about complex socioscientific problems. This may than lead to deeper information processing as well as elaboration processes and eventually to better individual performance. Referring to Kirschner and colleagues [34] who postulate that cooperative groups are most successful in terms of effective learning when task complexity is high, we assume that cooperative learning settings are especially adequate for working on socioscientific issues. As socioscientific issues are not only complex, and solutions need to be developed by integrating multiple perspectives, individuals might benefit from the advantage to distribute information processing and thus, to reduce cognitive load (cf. [34]).

In more detail, we hypothesize that students who study in cooperative learning settings will produce better learning outcomes with respect to socioscientific reasoning and decision making than students who study under more traditional, individual instruction.

Similar to existing research that highlights the importance of metacognitive guidance to support group processing, we also assume that individual student achievement will be enhanced through an additional metacognitive training that explicitly supports students in formulating and answering questions. Referring to Mevarech and Kramarski’s work on mathematical problem solving [19, 23], we assume that students who work on metacognitive questions will gain a deeper understanding of the problems they work on. In more detail, we assume that students who learn in a cooperative-metacognitive setting will produce better learning outcomes with respect to socioscientific reasoning and decision making than students who study in a cooperative learning setting.

2. Method

2.1. Participants

Participants included 360 senior high school students (151 males and 209 females, mean age: 17.35 years; ) from nine high schools in Germany. All participants were from grades 11–13 (last three years of senior high school in Germany). Students studied in three different conditions: cooperative learning (COOP), cooperative learning with embedded metacognitive instruction (COOP+META), and a nontreatment control group with traditional instruction. Due to restrictions concerning school and classroom settings, participants could not be randomly assigned to the different conditions, but assignment took place at the class level. In total, 112 students from 7 classes were assigned to the COOP condition, 129 students from 8 classes to the COOP+META condition, and 119 students from 8 classes to the control group. 21 teachers (12 females) participated in the study (mean age = 43 years; age range from 29 to 63 years; mean teaching experience = 13.2 years).

2.2. Training Conditions and Learning Material

Both training conditions (COOP and COOP+META) were identical in terms of lesson structure and time as well as context and tasks. They only differed with respect to the presence or absence of metacognitive instruction. While students in the COOP+META condition spent time on the metacognitive guidance, students in the COOP condition had time to elaborate on the socioscientific issue of palm oil production in Indonesia (see below). Both, the COOP as well as the COOP+META condition used the same set of cooperative learning methods such as the jigsaw and the fishbowl method. In addition, think-pair-share processes were included in all of the lessons [29].

The COOP+META condition was developed using the IMPROVE method [23, 24]. On the basis of IMPROVE we integrated comprehension, connection, strategic, and reflection questions into the learning material. These questions were given to students prior to and during learning activities as well as after having finished learning activities. Appendix A shows an example for the implementation of these metacognitive questions into one of students’ group work.

The socioscientific issue addressed in both training conditions was the issue of palm oil production in Indonesia. There is an increasing demand on palm oil worldwide as an ingredient for cosmetics and food, but especially with respect to its potential as a biofuel. Palm oil is typically produced on monocultures within the Indonesian rainforest. Due to the increasing demand, more and more plantations emerge. Many people on Sumatra, one of the main islands in Indonesia, work on these plantations to earn their living. As a consequence, the Indonesian rainforest decreases. In addition, indigenous people who traditionally live in and subsist on the rainforest are negatively affected. The described problem represents a typical socioscientific issue. It is factually and ethically complex and needs to be addressed by incorporating ecological, economical, and social aspects and perspectives [51]. Various social groups play a role within this problem situation such as workers on the plantations (representing nonsustainable users), indigenous people (representing sustainable users) who live in the rainforest, but also external stakeholders such as governments and the consumer in general. With respect to the problem, students need to understand the situation in its complexity. They need to understand the interdependence between the nonsustainable users and the decrease of the rainforest as a natural resource. The indigenous people in turn suffer from the overuse of the rainforest. Thus, both social groups are interrelated. Often, such problem situations are described as socioecological dilemmas [58].

Students in the nontreatment control group received traditional, individual instruction. They studied according to their regular school curriculum, which did not include the specific socioscientific issue of palm oil production in Indonesia. However, working on socioscientific issues is mandatory according to the national educational standards and all training conditions were obliged to teach to these standards [4].

Teachers in both training groups received an introductory training on the learning material. All teachers were familiar with cooperative learning and implemented it regularly in their classrooms. Teachers in the COOP+META condition received a one day introductory training on the IMPROVE method. The training was designed in the biology education research group and administered to the teachers by the researchers themselves. Teachers were first introduced to the theoretical construct behind the IMPROVE method and then worked on exemplary student tasks that included the four different metacognitive questions. Teachers in the control group received no specific training.

2.3. Measures
2.3.1. Socioscientific Decision Making

Students’ learning outcomes were measured using two 45 min paper-and-pencil tests on socioscientific decision making prior to and after the intervention. The pre- as well as the posttest consisted of three socioscientific issues (SSI) that were identical in structure but used different contexts in order to keep students motivated at the posttest. In addition, different contexts were used to counteract increases in students’ learning outcomes at the posttest that are only due to training effects on the questionnaire (Appendix B shows two example socioscientific issues from the pretest questionnaire). Table 1 shows the distribution of the different contexts in the pre-and posttest.

tab1
Table 1: Contexts for the different SSIs used in the pre- and posttest questionnaire on socioscientific decision making.

All test items to these socioscientific issues were presented in an open-ended format. With respect to the first two socioscientific issues, students had to describe the problem as well as to develop sustainable solutions to the problem. With respect to the third socioscientific issue, students were asked to evaluate presented solutions in terms of their sustainability and to suggest improvements to these solutions. Students’ responses to the open-ended questions were coded independently by two of the researchers. The final scoring guide consisted of 10 items (Table 2). Interrater-reliability was found to be sufficient (Cohen’s Kappa: ). In case of disagreement, discussions took place until agreement on the score could be reached.

tab2
Table 2: Scoring guide for the assessment of students’ socioscientific decision making (pre- and posttest).

The socioscientific decision making questionnaire (pre-and posttest version) includes two scales. Scale 1 consists of four items that represent the description of SSIs (items 1-2 for issue no. 1, items 3-4 for issue no. 2; Table 2). Scale 2 consists of six items that represent the development and evaluation of sustainable solutions to SSIs (item 5 for issue no. 1, item 6 for issue no. 2, items 7–10 for issue no. 3; Table 2). Both scales for the pre- and posttest were analysed in terms of reliability in previous studies. Reliability indices were found to be acceptable . In addition, item difficulties were checked to allow comparisons between pre- and posttest scores. With respect to the present study reliability indices for the pretest were (scale 1) and (scale 2) and for the posttest (scale 1) and (scale 2).

2.3.2. Metacognition

To assess general metacognition a questionnaire developed by A. Kaiser and R. Kaiser [59] was used. The original questionnaire consists of 19 items. Seven of these items refer to the regulation of cognition (planning, monitoring, and debugging), the facet of general metacognition that is relevant for the instructional approach addressed in the present study. Exemplary items were “I check my knowledge in detail that can be helpful to work on the assigned task” or “If I realise that I’m stuck, I will check whether another strategy will be more successful”. Each item was scored on a four-point Likert type scale ranging from “I completely agree” to “I completely disagree”. Cronbach’s Alpha was found to be for the pretest and for the posttest.

3. Results

3.1. Socioscientific Decision Making

With respect to the socioscientific decision making scales, data were analysed as follows. Concerning “Describing Socioscientific Issues”, we conducted a one-way ANOVA to examine group differences between the control group and the two experimental groups at the pretest. Results indicated no significant differences prior to the beginning of the intervention with respect to scale 1 ( , ). This legitimated us to conduct a (treatment × measurement points) repeated-measures ANOVA with “Describing Socioscientific Issues” (scale 1) as the dependent variable. Table 3 presents the mean scores and standard deviations with respect to scale 1 by time and treatment.

tab3
Table 3: Mean scores and standard deviations on “describing socioscientific issues” (scale 1) by time and treatment.

The repeated measures ANOVA indicated a significant main effect for time ( , , ) and a significant main effect for treatment ( , , ). The interaction effect between treatment and time was also significant ( , , . Post hoc Tukey tests revealed that students in both experimental groups (COOP and COOP+META) performed significantly better than the control group (both Ps <.001). However, the two experimental groups did not differ significantly, thus, indicating that the COOP+META group did not benefit from the embedded metacognitive instruction.

With respect to “Developing and Evaluating Solutions” we also conducted a one-way ANOVA to check group differences on the pretest scores. Results indicated that there was a significant difference between groups ( , ). Post hoc Tukey tests revealed that the control group differed significantly from the COOP group on pretest scores at the 5% level of significance. Consequently, we used a multiple regression analysis with prior knowledge (pretest score) and treatment conditions as independent variables and the posttest score of scale 2 as dependent variable. Concerning treatment conditions, two contrast variables were coded. Contrast one examined the difference between the control group and both experimental groups (Control versus COOP and COOP+META). Contrast two examined the difference between the two experimental groups (COOP versus COOP+META). Predictor variables were entered blockwise into the regression analysis. Table 4 shows the mean and standard deviations on “Developing and Evaluating Solutions” (scale 2) by time and treatment.

tab4
Table 4: Mean scores and standards deviations on “developing and evaluating solutions” (scale 2) by time and treatment.

Results from regression analyses showed that prior knowledge as well as both contrasts predict students’ learning outcomes at posttest measures. Table 5 shows the unstandardized beta values and their standard errors as well as standardized beta values with respect to the different regression models. The final statistical model accounted for 27% of the variance with prior knowledge accounting for 19%, the first contrast variable accounting for 7% percent of the variance and the second contrast variable accounting for 1%. Interestingly, the second contrast variable, which represented the difference between the COOP and the COOP+META condition, exhibited a negative relationship with posttest performance.

tab5
Table 5: Multiple regression predicting posttest performance on “developing and evaluating solutions” (scale 2) by prior knowledge and treatment condition.

3.2. Metacognition

With respect to the regulation facet of general metacognition, a one-way ANOVA was conducted to examine possible differences between groups at the pretest. Results indicated no significant differences prior to the beginning of the intervention ( , ). Thus, data were analysed using a (treatment × measurement points) repeated-measures ANOVA. Table 6 presents the mean scores and standard deviations by time and treatment.

tab6
Table 6: Mean scores and standards deviations on metacognition (regulation facet) by time and treatment.

The repeated-measures ANOVA indicated a significant main effect for time ( , , ), but no significant main effect for the treatment ( , ). The interaction effect between treatment and time was significant ( , , ). Post hoc Tukey tests revealed no significant differences between the groups.

4. Discussion

The major purpose of the present study was to examine the effects of two cooperative training settings (COOP and COOP+META) on students’ socioscientific decision making and metacognition. Socioscientific decision making refers to the description of socioscientific issues as well as to the development and evaluation of solutions to socioscientific issues. Findings show that both training groups outperformed the nontreatment control group on both scales. This is in line with a large body of research that identified beneficial effects of structured cooperative learning settings on students’ learning outcomes (for an overview, e.g., [30]). It also reflects findings from studies that used the IMPROVE method in mathematics education [23, 24, 35, 41].

However, with respect to the COOP+META treatment condition, findings did not meet our expectations. Students who studied in the COOP+META condition did not benefit from the embedded metacognitive training, as there were no differences on “Describing Socioscientific Issues” (scale 1) between the COOP and the COOP+META condition. With respect to “Developing and Evaluating Solutions” (scale 2), findings even exhibited a negative relationship between the corresponding contrast variable and students’ performance on the scale at the posttest. This, at first side astonishing, negative impact raises several questions. Why did students not benefit from the additional embedded metacognitive training and in more detail, which factors can be identified that lead to the decline in posttest measures when compared to the COOP condition? What can we deduce with respect to future research?

Empirical research on the effects of cooperative learning settings on student achievement suggests that students benefit most from collaboration if task complexity is high because individuals are more willing to distribute information processing among group members to reduce cognitive load [34]. These beneficial effects were found mostly in highly structured cooperative groups (e.g., [34, 60]). On the basis of these findings, we assumed that the COOP+META condition would outperform the COOP condition. However, this was not the case. One possible explanation might be that cooperation between group members in the COOP+META condition was overly structured so that natural cooperation was disturbed or even disrupted. Students were not able to cooperate naturally but were forced into a script that they felt was artificial or too detailed (cf. [60]). While students worked on complex socioscientific issues, overly structuring their group processes may even have hindered them from employing higher-order thinking skills and being creative (cf. [36]).

When confronted with socioscientific issues students have to perform a variety of information search and evaluation processes as well as reasoning and argumentation processes. Students may likely have experienced cognitive overload during group work as they had to solve a complex socioscientific issue, collaborate with their peers, and understand and work with the metacognitive instructions (cf. [60]). Thus, they may have concentrated more on solving the socioscientific issues or on working with the metacognitive instruction. As posttest measures on “Developing and Evaluating Solutions” were lower compared to the COOP condition, metacognitive guidance may even have hindered students from dealing with the socioscientific issue. Consequently, for future research, we need to carefully reconsider the design of the metacognitive guidance to ensure an adequate balance between group autonomy and provision of additional support measures.

Another important aspect addresses the issue of successful implementation of metacognitive trainings into the science classroom. Referring to existing research three fundamental principles have to be acknowledged: ensuring connectivity, being explicit about the function of metacognitive guidance, and extensive and prolonged metacognitive training [61, page 9], [62, page 85]. Possible explanations to the unexpected results refer to a combination of these three principles. Although the metacognitive questions were integrated into teaching materials at various stages in the teaching unit and contextualised with respect to the issue taught, students did probably not make use of these questions to their full extent. Although teachers in the COOP+META condition explained the metacognitive questions and their function, students may not have acknowledged their usefulness. Thus, they did not invest the extra effort that is needed for successful metacognitive instruction [61, page 9]. This is in line with findings from Hogan, who argues that “simple immersion” of metacognitive guidance in the task is not sufficient to build students’ metacognitive knowledge [56, page 1101]. Instead, an intervention that explicitly focuses on the use of metacognitive guidance and its functions seems to be more successful [56].

Moreover, written comments on lesson plans from teachers who taught in the COOP+META condition revealed that the use of reflection questions especially at the end of lessons often fell short. This has two possible reasons. On the one hand, lesson plans were quite packed with respect to learning goals on socioscientific decision making, the socioscientific issues taught as well as the extra metacognitive guidance. Although teachers aimed faithfully to implement the lessons according to our instructions, it is only reasonable that they considered the curricular requirements with respect to socioscientific decision making first.

On the other hand, teachers in the COOP+META condition had no prior experience with metacognitive instruction, which might have led to difficulties during the intervention. Although they were trained in using the metacognitive questions, a shortcoming of the present study is that teachers were not additionally supported during the intervention in their classrooms.

With respect to posttest measures for metacognition, results indicate that all groups, including the control group, improved on the metacognition scale at posttest measures. Although the COOP+META condition had the highest mean score, differences between groups were not statistically significant. Thus, one has to be cautious about interpreting these results. However, they might still contribute to the discussion described above that students either concentrated more on solving the socioscientific issues or on working with the metacognitive questions. Given the highest mean scores on the posttest measures, one might argue that students who studied under the COOP+META condition focused more on working with the metacognitive guidance.

Another explanation to this finding is that the intervention itself aims to enhance students’ critical thinking and reflection on socioscientific issues and possible solutions strategies. While developing and evaluating solution strategies, students need to engage in critical thinking to be able to identify nonsustainable solutions, to incorporate multiple perspectives and to monitor and regulate their own problem-solving processes, especially during group work. Developing solutions to a socioscientific issue can be described as a special problem solving process, which can be divided into three main aspects. First, students need to understand and describe the problem situation, second they need to develop possible solutions on the basis of relevant information, and third, they need to evaluate possible solutions in order to reach an informed decision [8, 52, 55]. Especially with respect to the second and third aspect students need to monitor their information search as well as their decision making process as a whole. The described phases were also taught and discussed with students in both training groups. In line with existing research, this may likely have enhanced performance on the metacognition scale at posttest measures (e.g., for an overview see [45]).

Implications for future research are diverse, but one major aspect refers to the improvement of teacher support during the intervention. Apparently, the introductory training to the metacognitive guidance was not sufficient to enable teachers to implement both socioscientific decision making and metacognitive instruction into their science classrooms as we had wished. In future research, we need to be even more aware of the “teachers’ dilemma” [56, page 1104] to teach according to curricular requirements as well as to focus on the metacognitive instruction and especially become aware of the potential benefits of such instruction for students’ learning processes. In terms of biology education, it is highly important that the teachers themselves conduct the training and not the researchers, although the latter might lead to better results with respect to student achievement [63].

With respect to the methodological limitations of the present study, a mixed-method approach should be applied in future research that converges both product and process data (e.g., [43]). The analysis of process data would have given deeper insights into possible difficulties with respect to the instruction of metacognitive guidance on the teachers’ side as well as the actual learning outcomes on the students’ side.

In addition, research needs to be done to shed more light on the relationship between the two concepts of socioscientific decision making and metacognition. As described above, dealing with socioscientific issues can be described as a problem solving process. Consequently, students are already engaged in reflection and monitoring processes. Thus, it would be extremely important to analyse in depth which processes with respect to the regulation of cognition are being promoted by socioscientific decision making in the science classroom. Therefore, process data are absolutely vital (cf. [64]).

Appendices

A. Metacognitive Guidance

Note. The following questions were given to students in the COOP+META condition during one of their group works. The overall task was to develop a solution to the problem of palm oil production in Indonesia and its side effects with respect to the rainforest and the indigenous people who live in the forest. Each group took up the perspective of one of the groups that are part of the problem. A panel discussion, which succeeded this group work, aimed to integrate all the different perspectives and solutions developed.

Here are some questions that can help you before you actually start your task.(i)What are the goals of our task?(ii)Can we describe the current situation of the person we are dealing with? Take some notes.(iii)Which aspects are essential to develop a good solution from the perspective of …? Take some notes.(iv)How should we proceed to develop a solution and in which way can we apply the strategies from previous lessons? Quote some.

Here are some questions that can help you while working on your task.(i)Are we still on task or are we running off the track?(ii)Are we incorporating all essential aspects?

Here are some questions that you should consider just before completing your task.(i)From the perspective of …, did we consider all important aspects for our solution?(ii)If our solution to the problem was implemented, how would the situation improve from our perspective? Take some notes.(iii)Anticipate the consequences that our solution would have for the other social groups! Take some notes.

B. Examples for Socioscientific Issues from the Pretest Questionnaire

B.1. Abbreviated Version of Issue No. 1: Uncontrolled Collection of Rattan in the Indonesian Rainforest

Note. The description of this socioscientific issue is based on research and findings from Koch and colleagues [65].

B.1.1. Rattan from Indonesia

Rattan is a very popular material for the production of chairs, armchairs, or outdoor furniture. In the 1980s furniture that was made out of Rattan became popular in Europe and North America and has been popular ever since. 90% of the Rattan that is used in the furniture industry stems from Indonesian Rattan. […].

In Sulawesi, one of the Indonesian islands, indigenous people, who live in and subsist on the rainforest, collect Rattan. They use Rattan to make ropes for fishing or for farming. They also use Rattan for building their houses. […]. When collecting Rattan, they take care that some of the shoots will not be harvested, so that the Rattan plants are able to resprout.

Other Sulawesian people also collect Rattan. They do not only need it for their own supply but they collect and sell it to agents who in turn sell it to the furniture industry. These people also depend on collecting Rattan to assure their livelihood. The money that they get depends on the amount as well as the weight and diameter of the harvested Rattan stems. They harvest Rattan in large groups so that they can collect a large amount of Rattan per day. Often, they collect all of the Rattan shoots in one area. The harvest is being collected and then transported out of the rainforest.

Rattan is a palm that climbs through and over other vegetation. Depending on the Rattan species, it takes about 5–25 years to harvest Rattan for the first time. It grows 0.2–1.5 meters per year. For resprouting, it is important that enough Rattan shoots remain in the forest. Otherwise, Rattan species will likely decline within the area.

Although it is forbidden to collect Rattan within the national park on Sulawesi, illegal harvesting still takes place as park rangers often cannot control the whole area. As a consequence, Rattan species are also under threat in the national park.

Tasks are mentioned below.(i)Describe the problem situation and explain the interrelations of central aspects.(ii)Develop a possible solution to this problem that acknowledges these interrelations.

B.2. Abbreviated Version of Issue No. 3: Shrimp Aquaculture in Mangrove Areas in Indonesia
B.2.1. Shrimp Aquaculture in Southeast Asia
B.2.2. Introductory text to the problem

Due to the high demand for shrimps in Germany and worldwide, shrimp farming in mangrove areas of Southeast Asia is steadily increasing. Shrimp farms provide jobs for many people but also have negative side effects on mangrove areas as well as on the people who live in these areas and subsist on the mangroves.

Solution A: Shrimp production in Europe. Shrimp production will be moved to shrimp farms in Europe. Existing shrimp farms in Southeast Asia will be closed and no new aquacultures will be built. Thus, mangrove areas in Southeast Asia and the people living in mangrove areas won’t be affected any longer.

In Europe, indoor shrimp aquacultures will be built that simulate conditions of mangrove areas, in particular marine water conditions. In such aquacultures, shrimps can be raised up to their requested size and then be sold to the food industry.

Solution B: Installing sustainable shrimp production in Southeast Asia. Shrimp farming in Southeast Asia will be shifted towards sustainable production. The overall aim is to receive a certified label for shrimp production in these shrimp farms. To receive such a label, shrimp farms have to meet a variety of requirements: At least half of the farming area needs to be covered with mangroves. Existing mangroves must not be cut down, otherwise new mangroves need to be planted. To assure sustainable production permanent controls need to be put through. Due to these new-less intensive-working conditions, less workers will be needed on shrimp aquacultures.

Tasks are mentioned below.(i)Evaluate both solutions with respect to their sustainable development. Consider positive and negative outcomes in case these solutions would be considered for implementation.(ii)Develop suggestions for improvement for both solutions. Explain!

Acknowledgments

The intervention study was conducted with support from the German Research Foundation (DFG) and its graduate research program 1195 “Understanding and Enhancing Educational Congruence in Schools.” The development of the test instruments for socioscientific decision making were supported by Grant BO 1730/3-2 from the German Research Foundation (DFG) in the Priority Program “Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes” (SPP 1293).

References

  1. American Association for the Advancement of Science, Science for All Americans, Project 2061, Oxford University Press, Washington, DC, USA, 1989.
  2. National Research Council, National Science Education Standards, National Academy Press, Washington, DC, USA, 1996.
  3. Qualifications and Curriculum Authority, The National Curriculum for England, Key Stages 1–4, Crown, London, UK, 1999.
  4. Kultusministerkonferenz (KMK), Bildungsstandards im Fach Biologie für den mittleren Schulabschluss, Luchterhand, München, Germany, 2005.
  5. Organisation for Economic Co-Operation and Development (OECD), Learning for Tomorrow’s World, First Results from PISA, 2003, OECD, Paris, France, 2004.
  6. R. Driver, P. Newton, and J. Osborne, “Establishing the norms of scientific argumentation in classrooms,” Science Education, vol. 84, no. 3, pp. 1–312, 2000. View at Scopus
  7. M. Ratcliffe, “Pupil decision-making about socio-scientific issues within the science curriculum,” International Journal of Science Education, vol. 19, no. 2, pp. 167–182, 1997. View at Scopus
  8. M. Ratcliffe and M. Grace, Science Education for Citizenship, Oxford University Press, Maidenhead, UK, 2003.
  9. L. Roberts, M. Wilson, and K. Draney, “The SEPUP assessment system: an overview,” BEAR Report Series SA-97-1, University of California, Berkeley, Calif, USA, 1997.
  10. T. D. Sadler, S. A. Barab, and B. Scott, “What do students gain by engaging in socioscientific inquiry?” Research in Science Education, vol. 37, no. 4, pp. 371–391, 2007. View at Publisher · View at Google Scholar · View at Scopus
  11. C. Oulton, J. Dillon, and M. Grace, “Reconceptualizing the teaching of controversial issues,” International Journal of Science Education, vol. 26, no. 4, pp. 411–423, 2004. View at Publisher · View at Google Scholar · View at Scopus
  12. M. Grace, “Developing high quality decision-making discussions about biological conservation in a normal classroom setting,” International Journal of Science Education, vol. 31, no. 4, pp. 551–570, 2009. View at Publisher · View at Google Scholar · View at Scopus
  13. S. Seethaler and M. Linn, “Genetically modified food in perspective: an inquiry-based curriculum to help middle school students make sense of tradeoffs,” International Journal of Science Education, vol. 26, no. 14, pp. 1765–1785, 2004. View at Publisher · View at Google Scholar · View at Scopus
  14. M. Siegel, “High school students’ decision making about sustainability,” Environmental Education Research, vol. 12, no. 2, pp. 201–215, 2006. View at Publisher · View at Google Scholar
  15. A. Zohar and F. Nemet, “Fostering students' knowledge and argumentation skills through dilemmas in human genetics,” Journal of Research in Science Teaching, vol. 39, no. 1, pp. 35–62, 2002. View at Publisher · View at Google Scholar · View at Scopus
  16. C. Ottander and M. Ekborg, “Students' experience of working with socioscientific issues—a quantitative study in secondary school,” Research in Science Education, vol. 42, no. 6, pp. 1147–1163, 2011. View at Publisher · View at Google Scholar · View at Scopus
  17. L. Simonneaux, “Argumentation in socio-scientific contexts,” in Argumentation in Science Education: Perspectives from Classroom-Based Research, S. Erduran and M. P. Jiménez-Aleixandre, Eds., Science & Technology Education Library, pp. 179–199, Springer, Berlin, Germany, 2008.
  18. L. Colucci-Gray, E. Camino, G. Barbiero, and D. Gray, “From scientific literacy to sustainability literacy: an ecological framework for education,” Science Education, vol. 90, no. 2, pp. 227–252, 2006. View at Publisher · View at Google Scholar · View at Scopus
  19. D. L. Zeidler, T. D. Sadler, M. L. Simmons, and E. V. Howes, “Beyond STS: a research-based framework for socioscientific issues education,” Science Education, vol. 89, no. 3, pp. 357–377, 2005. View at Publisher · View at Google Scholar · View at Scopus
  20. A. Palinscar and A. Brown, “Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities,” Cognition and Instruction, vol. 1, no. 2, pp. 117–175, 1984. View at Publisher · View at Google Scholar
  21. P. M. King and K. Strohm Kitchener, “Developing reflective judgment,” in Understanding and Promoting Intellectual Growth and Critical Thinking in Adolescents and Adults, Jossey-Bass Publishers, San Francisco, Calif, USA, 1994.
  22. A. King, “Discourse patterns for mediating peer learning,” in Cognitive Perspectives on Peer Learning, A. M. Donnell and A. King, Eds., pp. 87–115, Erlbaum Associates, Mahwah, NJ, USA, 1999.
  23. Z. R. Mevarech and B. Kramarski, “IMPROVE: a multidimensional method for teaching mathematics in heterogeneous classrooms,” American Educational Research Journal, vol. 34, no. 2, pp. 365–394, 1997. View at Scopus
  24. B. Kramarski, Z. R. Mevarech, and M. Arami, “The effects of metacognitive instruction on solving mathematical authentic tasks,” Educational Studies in Mathematics, vol. 49, no. 2, pp. 225–250, 2002. View at Scopus
  25. M. Zion, T. Michalsky, and Z. R. Mevarech, “The effects of metacognitive instruction embedded within an asynchronous learning network on scientific inquiry skills,” International Journal of Science Education, vol. 27, no. 8, pp. 957–983, 2005. View at Publisher · View at Google Scholar · View at Scopus
  26. R. Azevedo, D. C. Moos, J. A. Greene, F. I. Winters, and J. G. Cromley, “Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia?” Educational Technology Research and Development, vol. 56, no. 1, pp. 45–72, 2008. View at Publisher · View at Google Scholar · View at Scopus
  27. S. Eggert, S. Bögeholz, R. Watermann, and M. Hasselhorn, “Förderung von Bewertungskompetenz im Biologieunterricht durch zusätzliche metacognitive Strukturierungshilfen—Ein Beispiel für Veränderungsmessung,” Zeitschrift für Didaktik der Naturwissenschaften, vol. 16, pp. 299–314, 2010.
  28. H. Gresch, M. Hasselhorn, and S. Bögeholz, “Training in decision-making strategies: an approach to enhance students’ competence to deal with socioscientific issues,” International Journal of Science Education. In press. View at Publisher · View at Google Scholar
  29. D. W. Johnson and R. T. Johnson, Learning Together and Alone: Cooperative, Competitive, and Individualistic Learning, Allyn & Bacon, Boston, Mass, USA, 4th edition, 1994.
  30. D. W. Johnson, R. T. Johnson, and M. B. Stanne, “Cooperative learning methods: a meta-analysis,” 2000, http://www.tablelearning.com/uploads/File/EXHIBIT-B.pdf.
  31. R. E. Slavin, “Cooperative learning,” Review of Educational Research, vol. 50, pp. 315–342, 1980.
  32. R. E. Slavin, “Research on cooperative learning and achievement: what we know, what we need to know,” Contemporary Educational Psychology, vol. 21, no. 1, pp. 43–69, 1996. View at Publisher · View at Google Scholar · View at Scopus
  33. M. C. Wittrock, “Students’ thought processes,” in Handbook of Research on Teaching, M. Wittrock, Ed., Macmillan, New York, NY, USA, 1986.
  34. F. Kirschner, F. Paas, and P. A. Kirschner, “A cognitive load approach to collaborative learning: united brains for complex tasks,” Educational Psychology Review, vol. 21, no. 1, pp. 31–42, 2009. View at Publisher · View at Google Scholar · View at Scopus
  35. Z. R. Mevarech, “Effects of metacognitive training embedded in cooperative settings on mathematical problem solving,” Journal of Educational Research, vol. 92, no. 4, pp. 195–205, 1999. View at Scopus
  36. E. G. Cohen, “Restructuring the classroom: conditions for productive small groups,” Review of Educational Research, vol. 64, no. 1, pp. 1–35, 1994.
  37. R. M. Gillies, “The effects of cooperative learning on junior high school students during small group learning,” Learning and Instruction, vol. 14, no. 2, pp. 197–213, 2004. View at Publisher · View at Google Scholar · View at Scopus
  38. A. Renkl and H. Mandl, “Kooperatives Lernen: Die Frage nach dem Notwendigen und dem Ersetzbaren,” Unterrichtswissenschaft, vol. 23, pp. 292–300, 1995.
  39. B. Rosenshine, C. Meister, and S. Chapman, “Teaching students to generate questions: a review of the intervention studies,” Review of Educational Research, vol. 66, no. 2, pp. 181–221, 1996. View at Scopus
  40. A. H. Schoenfeld, “Learning to think mathematically: problem solving, metacognition, and sense making in mathematics,” in Handbook of Research on Mathematics Teaching and Learning, D. A. Grouws, Ed., pp. 165–197, MacMillan, New York, NY, USA, 1992.
  41. B. Kramarski and Z. R. Mevarech, “Enhancing mathematical reasoning in the classroom: the effects of cooperative learning and metacognitive training,” American Educational Research Journal, vol. 40, no. 1, pp. 281–310, 2003. View at Scopus
  42. Z. R. Mevarech and S. Fridkin, “The effects of IMPROVE on mathematical knowledge, mathematical reasoning and meta-cognition,” Metacognition and Learning, vol. 1, no. 1, pp. 85–97, 2006. View at Publisher · View at Google Scholar · View at Scopus
  43. R. Azevedo, J. G. Cromley, D. C. Moos, J. A. Greene, and F. I. Winters, “Adaptive content and process scaffolding: a key to facilitating students’ self-regulated learning with hypermedia,” Psychological Test and Assessment Modelling, vol. 53, pp. 106–140, 2011.
  44. M. Hasselhorn and A. S. Labuhn, “Metacognition and self-regulated learning,” in Encyclopedia of Adolescence, Normative Processes in Development, B. B. Brown and M. J. Prinstein, Eds., vol. 1, pp. 230–233, 2011.
  45. G. Schraw, K. J. Crippen, and K. Hartley, “Promoting self-regulation in science education: metacognition as part of a broader perspective on learning,” Research in Science Education, vol. 36, no. 1-2, pp. 111–139, 2006. View at Publisher · View at Google Scholar · View at Scopus
  46. B. J. Zimmerman, “Attaining self-regulation: a social cognitive perspective,” in Handbook of Self-Regulation, M. Boekaerts, Ed., pp. 13–39, Elsevier, Academic Press, Burlington, Mass, USA, 2000.
  47. G. Schraw and R. S. Dennison, “Assessing metacognitive awareness,” Contemporary Educational Psychology, vol. 19, no. 4, pp. 460–475, 1994. View at Publisher · View at Google Scholar · View at Scopus
  48. G. Schraw and D. Moshman, “Metacognitive theories,” Educational Psychology Review, vol. 7, no. 4, pp. 351–371, 1995. View at Publisher · View at Google Scholar · View at Scopus
  49. S. D. Kolstø, “Scientific literacy for citizenship: tools for dealing with the science dimension of controversial socioscientific issues,” Science Education, vol. 85, no. 3, pp. 291–310, 2001. View at Publisher · View at Google Scholar · View at Scopus
  50. T. D. Sadler and D. L. Zeidler, “Patterns of informal reasoning in the context of socioscientific decision making,” Journal of Research in Science Teaching, vol. 42, no. 1, pp. 112–138, 2005. View at Publisher · View at Google Scholar · View at Scopus
  51. S. Bögeholz and J. Barkmann, “Rational choice and beyond: Handlungsorientierende Kompetenzen für den Umgang mit faktischer und ethischer Komplexität,” in Lehr-und Lernforschung in der Biologiedidaktik, R. Klee, A. Sandmann, and H. Vogt, Eds., vol. 2, pp. 211–224, Studien, Innsbruck, Austria, 2005.
  52. S. Eggert and S. Bögeholz, “Students' use of decision-making strategies with regard to socioscientific issues: an application of the rasch partial credit model,” Science Education, vol. 94, no. 2, pp. 230–258, 2010. View at Publisher · View at Google Scholar · View at Scopus
  53. S. D. Kolstø, “Patterns in students' argumentation confronted with a risk-focused socio-scientific issue,” International Journal of Science Education, vol. 28, no. 14, pp. 1689–1716, 2006. View at Publisher · View at Google Scholar · View at Scopus
  54. M. P. Jiménez-Aleixandre and C. Pereiro-Muñoz, “Knowledge producers or knowledge consumers? Argumentation and decision making about environmental management,” International Journal of Science Education, vol. 24, no. 11, pp. 1171–1190, 2002. View at Publisher · View at Google Scholar · View at Scopus
  55. S. Bernholt, S. Eggert, and C. Kulgemeyer, “Capturing the diversity of students' competences in science classrooms. Differences and commonalities of three complementary approaches,” in Making It Tangible—Learning Outcomes in Science Education, S. Bernholt, K. Neumann, and P. Nentwig, Eds., pp. 187–217, Waxmann, Münster, Germany, 2012.
  56. K. Hogan, “Thinking aloud together: a test of an intervention to foster students' collaborative scientific reasoning,” Journal of Research in Science Teaching, vol. 36, no. 10, pp. 1085–1109, 1999. View at Scopus
  57. A. S. Labuhn, S. Bögeholz, and M. Hasselhorn, “Lernförderung durch Anregung der Selbstregulation im naturwissenschaftlichen Unterricht,” Zeitschrift für Pädagogische Psychologie, vol. 22, no. 1, pp. 13–24, 2008.
  58. A. Ernst, Ökologisch-Soziale Dilemmata, Psychologie Verlags Union, Weinheim, Germany, 1997.
  59. A. Kaiser and R. Kaiser, Metakognition. Denken und Problemlösen optimieren, Luchterhand, Neuwied, Germany, 1999.
  60. P. Dillenbourg, “Over-scripting CSCL: the risks of blending collaborative learning with instructional design,” in Three Worlds of CSCL: Can We Support CSCL? P. A. Kirschner, Ed., pp. 61–91, Open University of the Netherlands, Heerlen, The Netherlands, 2002.
  61. M. V. J. Veenman, B. H. A. M. van Hout-Wolters, and P. Afflerbach, “Metacognition and learning: conceptual and methodological considerations,” Metacognition and Learning, vol. 1, no. 1, pp. 3–14, 2006. View at Publisher · View at Google Scholar · View at Scopus
  62. B. Kramarski, “Promoting teachers' algebraic reasoning and self-regulation with metacognitive guidance,” Metacognition and Learning, vol. 3, no. 2, pp. 83–99, 2008. View at Publisher · View at Google Scholar · View at Scopus
  63. C. Dignath, G. Buettner, and H. P. Langfeldt, “How can primary school students learn self-regulated learning strategies most effectively? A meta-analysis on self-regulation training programmes,” Educational Research Review, vol. 3, no. 2, pp. 101–129, 2008. View at Publisher · View at Google Scholar · View at Scopus
  64. H. Gresch and S. Bögeholz, “Identifying non-sustainable causes of action: a prerequisite for decision making in education for sustainable development,” Research in Science Education. In press. View at Publisher · View at Google Scholar
  65. S. Koch, J. Barkmann, L. Sundawati, and S. Bögeholz, “Subjective theories of Indonesian agronomy and biology teacher students on environmental commons dilemmas,” International Research in Geographical and Environmental Education. In press.