Abstract
This article encompasses a systematic review of the research on ICT-based learning environments for metacognitively oriented K-12 mathematics education. This review begins with a brief overview of the research on metacognition and mathematics education and on ICT and mathematics education. Based on a systematic screening of the databases Web of Science and ERIC wherein three elements—ICT-based learning environments, metacognitive pedagogies, and mathematics—are combined, 22 articles/studies were retrieved, situated at various educational levels (kindergarten, elementary school, and secondary school). This review revealed a variety of studies, particularly intervention studies, situated in elementary and secondary schools. Most studies involved drill-and-practice software, intelligent tutoring systems, serious games, multimedia environments, and computer-supported collaborative learning environments, with metacognitive pedagogies either integrated into the ICT software itself or provided externally by the teacher, mainly for arithmetic or algebraic word problem-solving but also related to other mathematical topics. All studies reported positive effects on mathematical and/or metacognitive learning outcomes. This review ends with a discussion of issues for further theoretical reflection and empirical research.
1. Introduction
In How People Learn, Bransford et al. [1] propose the support and exploitation of metacognition as one of the three core instructional design principles. This proposal is based on the vast amount of research showing that (a) learners with better metacognitive knowledge and skills will acquire domain-specific knowledge and skills more effectively and efficiently and will transfer their knowledge and skills more easily to other domains and contexts and (b) well-designed metacognitively oriented instruction embedded in a specific content domain has a positive impact on learners’ metacognitive behavior as well as their learning within that domain [2–7].
This article encompasses a systematic review of the research on learning mathematics in metacognitively oriented ICT-based learning environments: research describing and evaluating attempts to increase the power of ICT-based learning environments for K-12 mathematics education by enriching them with a metacognitive pedagogy, or the reverse, that is, attempts to increase the power of metacognitive pedagogies for K-12 mathematics education by embedding them in ICT-based learning environments. We begin with a brief overview, first, of the research on metacognition and mathematics education. Afterwards, we briefly review the research on ICT and mathematics education. Finally, we explicate the aims and scope of our literature review wherein these three elements—metacognition, ICT, and mathematics education—are combined.
1.1. Metacognition and Mathematics Education
Since the 1980s, it is commonly accepted that the cognitive system depends on higher-order processes that enable it to work efficiently. Examples of metacognitive components are planning, monitoring, control, and reflection (e.g., [8]). From the very beginning, researchers ([8]; see also [9, 10]) distinguished between two closely interrelated components of metacognition: (1) knowledge of cognition (e.g., knowledge about the task, strategies appropriate for solving the task, and personal characteristics relevant to the task) and (2) regulation of cognition (e.g., monitoring, control, and reflection). (The term “metacognition” is closely related to the term “self-regulation.” According to some researchers, self-regulation refers to the second component of the metacognitive system, namely, one’s ability to control, evaluate, and modify his or her cognitive processes in the pursuit of a particular goal [11]. Others conceive self-regulation more broadly, referring not only to the ways that learners systematically activate and sustain their cognitions towards the attainment of their goals but also to their behaviors, motivations, and affects [6]. In this article, we take the former perspective, implying that we conceive metacognition as the most general concept and self-regulation as the second, regulatory component of metacognition.)
While the early research on metacognition assumed that children younger than 10 years have little or no metacognition, from the late nineties on, researchers started to report evidence demonstrating that kindergarten children can already spontaneously plan, monitor, control, and reflect on their mathematics activities [12–15].
The relationship between metacognition and mathematical reasoning and problem-solving is well documented in the research literature (e.g., [16, 17]). Research has indicated that learners at all ages, from kindergartners to adult learners, who plan, monitor, evaluate, and reflect on their problem-solving processes, perform better on mathematics reasoning tasks and solve mathematical problems better than those who do not use these activities, or use them less frequently (e.g., [18]). This phenomenon has been documented by applying a large variety of off- and online measurements, including questionnaires, observations, think-aloud and retrospective techniques, (semi)structured interviews, various types of brain coding techniques, and analysis of peers explaining the solutions to one another or working in small groups. In general, these studies reported high positive correlations between metacognition and mathematical reasoning or problem-solving, even after controlling for IQ [13, 19].
Given the well-documented positive association between metacognition and mathematical reasoning or problem-solving [1], researchers started to look for ways of improving learners’ mathematical reasoning and problem-solving, based on the teaching of metacognitive knowledge and/or skills [6, 16]. Over the years, several such “metacognitive pedagogies” have been designed for the area of mathematics learning, for various age levels and embedded in different parts of the mathematics curriculum.
Most of these methods were routed in the seminal work of Schoenfeld [5, 20]. In his classic studies from the 1980s, Schoenfeld [20] worked with undergraduates on improving their problem-solving by regularly prompting them to consider metacognitive questions such as “What are you doing?” “How are you doing it?” and “How does it help you?” He found that, by asking students these questions throughout a one-semester course, the students themselves began asking and answering these questions, which in turn yielded greater metacognitive awareness of their problem-solving as well as more efficient problem-solving.
These results about the possibility of improving mathematical competence through metacognitively oriented instruction were confirmed in a number of subsequent studies throughout the nineties and into the 2000s not only with university students but also with learners as young as kindergartners (e.g., [16, 21–28]).
In particular, Kramarski, Mevarech, and colleagues engaged in a robust program of research related to metacognition instruction in mathematics for more than 20 years (e.g., [16, 25, 26]). They designed and investigated an instructional method that they refer to as IMPROVE, an acronym representing the series of following teaching steps:(i)Introducing the new materials, concepts, problems, or procedures using metacognitive scaffolding(ii)Metacognitive self-directed questioning in small groups or individually(iii)Practicing by employing the metacognitive questioning(iv)Reviewing the new materials by teacher and learners, using the metacognitive questioning(v)Obtaining mastery on higher-order and lower-order cognitive processes(vi)Verifying the acquisition of cognitive and metacognitive skills based on feedback-corrective processes(vii)Enrichment and remedial activities
The core component of the IMPROVE program consists of training the learners to use four kinds of metacognitive self-directed questions:(i)Comprehension: What is the problem all about?(ii)Connection: How is the problem at hand similar to or different from problems you have solved in the past?(iii)Strategies: What strategies are appropriate for solving the problem and why?(iv)Review: Does the solution make sense? Can you solve the problem differently, how? Are you stuck, why?
However, in many cases, the positive effects of all these interventions were moderate rather than strong, implying that improving learners’ competence in mathematical reasoning and problem-solving by means of metacognitively oriented interventions is a complex and demanding matter, which comes with a cost, in terms of effort, time consumption, and reduction of the curriculum in the traditional sense [13].
1.2. Metacognition and ICT
Similar to the research literature on metacognition and mathematics education, the literature on metacognition and ICT is immense. In the early days of ICT, a computer’s potential was mainly seen as a machine for drill-and-practice in basic arithmetic and algebraic skills [11]. But as a result of emergence of new theoretical approaches to learning and instruction (e.g., constructivism), on the one hand, and new technological advancements, on the other hand, researchers started to explore its potential for developing learners’ higher-order cognitive and metacognitive competencies by means of more advanced learning technologies such as intelligent tutoring systems, simulations, programming, serious games, hypermedia, computer-supported collaborative learning environments, and virtual reality [11, 29–31].
So, already for several decades, ICT is being considered a powerful tool for teaching higher-order processes such as metacognition and self-regulation, in various domains including mathematics. As argued by Mayer (2010, in [16]; p. 148), “ICT is particularly useful for approaching such complex and unfamiliar problem-solving tasks because it enables the learner to search for information on the web, look for similar problems and sub-problems on-line, and use various computerized tools that can carry out the tedious work that is sometimes associated with solving mathematics problems (such as plotting graphs or doing computations) and hence release cognitive energy for higher-order cognitive processes [32]. Furthermore, search tools and online information sources (…) may lead users to reflect on the given information, decide which piece of information is most applicable to the given problem (…). Computer supported collaborative learning (e.g., asynchronous learning networks, forums, even emails) can become a powerful reflection tool, enabling students to be aware of how and why a solution path was chosen (…).”
This claim has been reiterated by scholars working in research projects that are aimed at realizing learning outcomes in various curricular domains, including mathematics, through advanced learning technologies such as simulations, serious games, hypermedia, computer-supported collaborative learning environments, and virtual reality. According to these researchers, several design characteristics of these advanced learning technologies (such as learners’ access to nonlinear information, to multiple representations, or to input or ideas from other learners) may play a critical role in the development of their metacognitive or self-regulatory skills and, consequently, lead to improved learning outcomes in the curricular domain at stake [30, 33–35].
However, the above-mentioned design features of these advanced learning technologies also require learners to actively, deliberately, and critically consider and monitor the logical relations between various pieces of information, different kinds of representation, or the varied input from co-learners, in relation to the progression in their own learning process. Clearly, all these processes can be characterized as metacognitive or self-regulatory [6]. So, learners’ ability to monitor, control, and reflect on their learning in these advanced technological learning environments will also have a significant impact on their learning outcomes [16, 33]. Indeed, successfully navigating nonlinear information, integrating information from various representations of the same concept, or integrating input from a peer learner requires learners to actively regulate their own learning, and learners’ inability to do so will undermine their learning success with these technologies. In response to this challenge, researchers have developed various kinds of scaffolds or prompts designed to stimulate, guide, and support learners’ metacognition or self-regulation while working with these advanced learning technologies [30, 33]. While initially these metacognitive scaffolds or prompts embedded in available ICT environments were typically general and static in nature, researchers are increasingly striving for more individually tailored pathways involving adaptive scaffolding, prompts, assessments, and navigation, by making use of techniques of artificial intelligence, learning analytics, etc. [33, 36].
Learners’ inability to monitor, control, and reflect on their learning with advanced learning technologies is not the only reason why the potential of advanced technologies in (mathematics) teaching and learning may not be fulfilled. Other reasons for that failure given in the literature are (1) the very nature of ICT, which calls for “trial-and-error” and does not encourage learners to reflect on their problem-solving processes, and (2) the cognitive overload in learners that these ICT-enhanced learning environments might be induced by the multitude of information and extra tools they offer [16].
1.3. Aims and Scope of This Review
This article reviews research on enriching ICT-enhanced environments by metacognitive scaffolds or prompts, or the reverse, that is, attempts to increase the power of metacognitive pedagogies by embedding them in ICT-based learning environments in the domain of K-12 mathematics education. So, our review covers research on how to embed metacognitive pedagogies in available ICT environments to increase the quality and efficiency of mathematics learning, as well as research on how to apply ICT to increase the quality and efficiency of metacognitively oriented mathematics learning environments.
With this systematic review, we hope to answer the following questions concerning the available research on mathematics learning in metacognitively oriented ICT-based learning environments:(i)What is the educational level of the learners (ranging from kindergarten to secondary education)?(ii)What are the characteristics of the learners (e.g., regular learners, low- or high-achieving learners, or learners with special needs)?(iii)What kind of ICT environment is being used (e.g., computer-supported practice, educational e-books, intelligent tutoring systems, serious games, multimedia, simulations, hypermedia, computer-supported collaborative learning environments, and virtual environments)?(iv)What is the nature of the mathematical subdomain being addressed (e.g., early number sense, arithmetic, geometry, algebra, graphing, and word problem-solving)?(v)What research methodology is being used (e.g., observational study, case study, correlational study, (quasi-)experimental intervention study, and design research)? and in case of intervention studies, What is the nature of its conditions?(vi)What kind of metacognitive training, guidance, or support is provided (e.g., utilizing software features that help the learner to navigate within a math learning environment, providing assistance in solving problems or processing information by offering metacognitive scaffolds, and increasing the awareness of the problem-solving or learning process with mirroring tools [13])? And how is the metacognitive support delivered? Is it embedded in the ICT environment itself, provided outside by the teacher or other instructional means (e.g., cue cards), or both?(vii)What are the targeted learning outcomes of the learning environment (i.e., cognitive knowledge and skills in a given content domain, metacognitive knowledge and self-regulatory skills, improved performance in the specific mathematical subdomain being addressed, or affective or motivational learning outcomes)?(viii)What are the main findings?
2. Method
We conducted a systematic review of the research literature regarding technology-mediated K-12 mathematical learning environments implementing a metacognitive pedagogy, making use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses [37]) guidelines. These guidelines consist of a checklist and a flow diagram and help improve the reporting of this review. A summary of the search and selection process, which consists of four consecutive phases, is presented in Figure 1 and is based on the PRISMA statement.
First, a systematic search was done through the consultation of the databases ERIC and Web of Science. The databases were browsed for peer-reviewed English articles, based on four combinations—accounting for synonyms in search terms—that is, “metacognition AND mathematics AND online learning” (n = 33), “metacognition AND mathematics AND computer-based learning” (n = 12), “self-regulation AND mathematics AND online learning” (n = 36), and “self-regulation AND mathematics AND computer-based learning” (n = 28). The total number of articles found by these search terms was 109 (78 in ERIC and 31 in Web of Science).
Second, the literature findings were narrowed down by excluding duplicates and conference proceedings, resulting in 46 articles.
Third, the dataset was further reduced—by reading the abstracts, and if necessary the full texts—based on four inclusion criteria that were implied by the focus of this article. More particularly, the study has to involve (1) an online learning environment, (2) K-12 learners in regular education, (3) relation to math content, and (4) implementation of a metacognitive pedagogy. After application of these four criteria, 10 articles remained in the dataset.
Fourth, backward and forward reference search was applied to these 10 articles. In case of backward reference searching, reference lists of these 10 articles were systematically screened to identify relevant studies for the scope of this article. Forward reference search was applied to identify more recent articles that cited one of the initial 10 articles in our dataset. This backward and forward reference search resulted in another 12 articles that also met the above-mentioned inclusion criteria. This means that a total number of 22 articles formed the basis of this review study; these articles are marked with an asterisk in the reference list.
To answer the above-mentioned research questions, each article was summarized in terms of the following aspects: (a) the educational level of the learners, (b) other characteristics of the learners, (c) the kind of ICT-based environment being used, (d) the mathematical subdomain being addressed, (e) the metacognitive pedagogy being included, (f) the overall research method and the nature of its conditions, (g) the dependent variables being targeted, and (h) the obtained results.
3. Overview of the Available Studies
An overview of the retrieved studies on metacognitively oriented ICT-based learning in K-12 mathematics education is provided in Table 1. In this overview, we use the learners’ educational levels—namely, kindergarten, elementary education, or secondary education—as the main ordering principle. In addition, each article was summarized in terms of the following aspects: other characteristics of the learners, the kind of ICT-based environment being used, the mathematical subdomain being addressed, the metacognitive pedagogy being included, the overall research method and its conditions, the learning outcomes being targeted, and the main result(s).
3.1. Kindergarten
We retrieved three studies on metacognitively oriented ICT-based learning environments for mathematics education in kindergarten [12, 38, 39]. The main features of these three studies are, respectively, summarized in nos. 1, 2, and 3 in Table 1.
Children in these studies had the mean age between 4 and 6 years. Two of the three studies (2 and 3) involved children at risk for (mathematical) learning disabilities.
In two studies, the ICT-based environment involved drill-and-practice (1 and 2), while the other one was built around an educational e-book (3).
In all three cases (1, 2, and 3), the mathematical subdomain was early numerical skills such as counting, numerical magnitude comparison, ordinal number relations, and simple addition.
All three studies were intervention studies with (quasi-)experimental designs comparing an ICT-based environment with and without a metacognitive pedagogy, and in two studies (1 and 2), a non-ICT control condition was added.
The metacognitive training was embedded twice in the ICT environment (1 and 3) and once provided by the teacher as a supplement to the existing ICT environment (2). The metacognitive strategies taught to the children were basic forms of planning, monitoring, and reflection, which were provided by means of instructional techniques such as providing cues and questions, demonstrations, and explanations.
As far as targeted learning outcomes are concerned, all three studies assessed the impact on the early mathematics skills that were addressed in the learning environment, and in one study (3), transfer to the emergent literacy skill of rhyming was assessed too. None of the studies involved a metacognitive measure.
In all three studies, the extra metacognitive pedagogy resulted in a significant positive learning effect compared to the other conditions. Remarkably, in the study of Shamir and Lifshitz [39] (i.e., study 3 in Table 1), this positive learning effect only holds for the literacy and not for the mathematics measure. Furthermore, taking into account the results of Shamir and Lifshitz [39], it seems important that the metacognitive support does not overload these young children’s working memory, especially when these children are at risk for learning disabilities.
In sum, even though only few studies have explored the potential of metacognitively oriented ICT-supported mathematics education to preschoolers, the available evidence suggests that it is possible to pay instructional attention to basic metacognitive skills in ICT-based environments from a very early age on, before the start of elementary school. Not surprisingly, at-risk children seem to be seen as a particularly relevant group for this kind of learning environments, and in line with the educational technology for early mathematical education in general, the privileged application domain is drill-and-practice software for basic numerical skills, although one study involved e-book reading. While these kindergarten studies suggest that this instructional attention to metacognitive pedagogy in ICT-based environments may lead to better learning outcomes in emergent mathematical and/or literacy skills, absence of metacognitive pre- and posttest measures and/or process data about the development of children’s metacognitive behavior during the intervention forces to be cautious by making causal claims about the role of metacognition in children’s improved early mathematical and/or literacy skills. Furthermore, the results indicate that these metacognitive pedagogies may either be provided by the ICT itself or be supplemented by the teacher. Finally, reflection on the disappointing results for early mathematics from one study (3) led to the important warning that the added metacognitive support runs the risk of overloading young children’s working memory, especially when these children are at risk for learning difficulties.
3.2. Elementary School
We retrieved seven studies on metacognitively oriented ICT-based learning environments for mathematics education in elementary school [40–46]. Their main features are, respectively, summarized in nos. 4–10 in Table 1.
These seven studies involving elementary school children showed a strong tendency to focus on regular children from the upper grades of elementary school. Remarkably, none of the studies involved children in the age group between 6 and 9, and in only one study (10), participants were low achievers in mathematics.
The studies applied a variety of ICT uses. In five studies (5, 6, 7, 9, and 10), the ICT-based learning environment essentially involved individual tutoring in and practicing of the targeted content and skills, and the two remaining ones (4 and 8) were essentially about computer-supported collaborative learning.
As far as the mathematical content is concerned, more than half of the studies (5, 7, 8, and 10) were specifically about mathematical word problem-solving, one about arithmetic computation (9), one about geometry (4), and one covered the whole mathematical curriculum (6).
As for the kindergarten level, the majority of the studies (4, 5, 6, 7, 9, and 10) were (quasi-)experimental intervention studies comparing various experimental and control groups. One of these studies (10) also had a case-study component (10b), and one other study (8) was an independent case study. As far as the experimental studies are concerned, four of them (4, 6, 9, and 10a) involved a comparison between an ICT-based learning environment with and without a metacognitive pedagogy, whereas the two other intervention studies (5 and 7) compared an ICT-based version with a metacognitive pedagogy with a “business as usual” mathematics education control condition. Interestingly, one pair of studies (5 and 7) consisted of an evaluation of the effectiveness of a metacognitively oriented computer-based word problem-solving training under rather strictly controlled experimental conditions and realized by the researchers themselves (7), followed by an examination of its effectiveness in a more naturalistic setting (5).
Most of the time (4, 5, 7, and 9), the metacognitive pedagogy was implemented in the ICT-based learning environment, whereas in two other studies (6 and 10), it was the teacher who was responsible for the extra metacognitive instruction and support through the organization of the lessons and/or additional curricular materials (e.g., worksheets), and in one other study, the metacognitive pedagogy was jointly realized by the computer and the teacher. The nature of the metacognitive pedagogy differed according to the mathematical content. For instance, the studies focusing on word problem-solving consisted of a number of general and/or specific metacognitive hints or questions—provided by the computer program or the teacher—that helped the learner to approach and solve each word problem in a thoughtful and systematic way. For instance, in Teong’s study [46] (no. 10 in Table 1), within the computer program were various types of general and specific metacognitive hints that students could request. The general hints were displayed as a staircase containing the following steps (and accompanying hints): read and analyze (“I read carefully”), explore (“I know the type of problem”), plan (“I make a plan”), verify (“I check my answer”), and reflect (“What did I learn?”). The specific hints were about looking within the word problem for what is asked, what numbers are given, and how the problem can be represented.
While in the three kindergarten studies, the assessment was restricted to cognitive learning outcomes related to the content and skills addressed in the training, in all studies with elementary school children, the assessment also involved measurements of a metacognitive (4, 5, 6, 7, 8, 9, and 10) and motivational (6) nature. These metacognitive measures involved (a) general questionnaires about metacognition, such as the “Motivated Strategies for Learning Questionnaire” (MSLQ) [62], being administered at (pretest and) posttest (4 and 6), (b) specifically designed task-specific instruments that are aimed at revealing the use of the intended metacognitive knowledge and/or skills while performing the task during (pretest and) posttest (5, 6, 7, 8, 9, and 10a), and/or (c) log-file analyses or more qualitative evaluations of learners’ metacognitive behavior while working on the mathematical tasks in different experimental conditions (7 and 10b).
In all studies, positive results were reported concerning the superiority of the condition that combined ICT and metacognitive pedagogy, even though the design of several of these studies did not allow straightforward conclusions about the added value of this combination compared to other conditions involving only ICT, only metacognitive pedagogy, or neither of the two. Interestingly, in all cases wherein both cognitive and metacognitive measures were involved, learners in the ICT-based environment with metacognitive guidance demonstrated positive learning effects both for the cognitive outcomes related to the mathematical content and skills addressed in the training and for metacognitive outcomes (4, 5, 6, 7, 8, 9, and 10). Some additional findings obtained in studies that made extra attempts to understand the underlying mechanisms were that learners who made more use of the metacognitive guidance benefitted more from the intervention (7), that extended training provided longer-lasting learning effects (9), and that learning effects were not restricted to immediate effects but lasted at least until five months after the end of the intervention (6).
In sum, various scholars have tried to enhance children’s cognitive and metacognitive development in the context of elementary school mathematics learning, particularly in the context of word problem-solving. In the studies reviewed above, all of which involved regularly developing children from the upper grades of elementary school, the metacognitive pedagogy was either embedded within the ICT environment or provided by the teacher or extracurriculum materials. Taken as a whole, these studies provide ample evidence for the positive role that a metacognitively oriented training combined with an ICT-based learning environment can play in enhancing (upper) elementary school children’s mathematical performance, their metacognitive skills, and—at least in one study—also their motivation. However, flaws in the design of the studies and/or in the analyses of the learning processes during the intervention do not always allow a clear and convincing picture of the contribution of the ICT and the metacognitive pedagogy aspects to the success of the intervention.
3.3. Secondary School
Also with samples of secondary school students, researchers have examined the instructional interventions designed to influence and exploit students’ metacognition during mathematical reasoning and problem-solving in an ICT environment or to make their metacognitively oriented mathematical learning environment more powerful by means of ICT [47–56, 59–61]. The main characteristics of these 12 studies are summarized in nos. 11 to 22 in Table 1.
Except for two studies (20 and 21), which (also) involved students from the upper grades of secondary school, all studies with secondary school students took place in the lower and middle grades of this school level. Moreover, all studies involved regular learners, except for one study with low-achieving students (11) and one with students with autism (19).
Compared to the two lower educational levels, the studies with secondary school students involve a somewhat broader variety of ICT uses. Most of the studies used traditional ICT for individual tutoring in and/or practicing of certain mathematics knowledge and skills (11, 12, 15, 16, and 19), while some others (20, 21, and 22) used intelligent tutoring systems. In another substantive set of studies (13, 14 17, and 18), students were put in the complex (multimedia) learning environment wherein they could explore, in pairs or small groups, certain mathematical topics and/or solve mathematical problems by making use of various search, representation, and communication tools.
The studies covered various parts of the secondary school curriculum. As for secondary school mathematics, (algebra) word problem-solving was the most frequent topic (11, 12, 13, 17, and 18). Other topics were geometry (20, 21, and 22), algebra (15, 16, and 20), and graphing (14).
All studies were (quasi-)experimental studies comparing the efficacy of two or more learning environments with or without certain ICT-based instructional elements and/or metacognitive pedagogies. Several studies had basic experimental designs, such as the comparison of an ICT-based instructional program with or without the provision of extra metacognitive cue cards (15, 16, 15, 17, 19, 20, 21, and 22), whereas others had more complex designs involving three or even four conditions combining the absence or presence of ICT-based and metacognitively oriented instructional elements (11, 12, 13, 14, and 18).
The majority of the studies involving secondary school students had metacognitive hints or guidance tools that were actually embedded in the ICT software (11, 12, 14, 15, 19, 20, and 21). In only one study, the metacognitive pedagogy was provided by the teacher (22), and in the remaining studies (13, 16, 17, and 18), the computer and teacher were jointly responsible for the metacognitive training component. The concrete nature of the (computer-based) metacognitive guidance and support differed according to the type of ICT environment and its mathematical content, but essentially, it consisted of providing cues and questions before, during, and/or after task execution. In the studies of Kramarski and associates, the nature of the metacognitive guidance and support was based on the IMPROVE approach towards metacognitively oriented instruction explained in Introduction. For instance, in their study about e-learning, Kramarski and Gutman [51] (see no. 15 in Table 1) compared the impact of e-learning supported with self-metacognitive questioning based on the IMPROVE approach (EL + IMP) with e-learning without such an explicit support of self-regulation (EL). Students in both environments practiced 10 interactive mathematics problems presented electronically to them in Excel. Students were supposed to interact in pairs about the problems and their solutions before sending their common solution to the teacher via e-mail. Students from both groups were encouraged to work with e-learning, but only the EL + IMP group received the extra metacognitive supports (based on IMPROVE) that appeared in pop-up screens and provided the students with differential metacognitive cues for solving the problems. While the research program of Kramarski et al. was essentially aimed at evaluating various forms of e-learning supported by various kinds of metacognitive instructions built in into the ICT environment or provided from outside, Roll et al. ([58], see nos. 20 and 21 in Table 1) investigated the additive effects of implementing metacognitive scaffolding on an existing intelligent tutoring system for teaching algebra and geometry to secondary school students. More specifically, they designed and implemented a metacognitive tutor for teaching help-seeking students starting from the available algebra or geometry ITS. Based on clear instructional goals with respect to metacognition on the one hand and a good model of the student’s actual metacognitive behavior on the other hand, the auxiliary metacognitive tutor provides help with respect to various aspects of the student’s help-seeking behavior while (s)he is working through the learning materials. Importantly, this adaptive tutor is designed in such a way that it does not merely compensate for the student’s failing metacognitive c.q. help-seeking behavior, but rather it tries to make the student increasingly competent in the intended metacognitive behavior, on top of increasing his/her mathematical knowledge and skills.
As for the investigations with elementary school children, except for one study that only measured the impact on students’ mathematical learning outcomes (11), all studies at the secondary school level involved both cognitive and metacognitive measures (12, 13, 14, 15, 16, 17, 18, 19, 20, 21, and 22), while, except for study no. 14, none included affective/motivational measures as well. The cognitive measures are aimed at assessing content-related learning, which was mostly closely related to the mathematical content being taught but sometimes also involved more distant measures such as specially designed transfer tests or general mathematics achievement tests (e.g., 12, 13, and 15) or assessments aimed at assessing possible changes in students’ cognitive processes and strategies (e.g., 14). The metacognitive measures involved (a) general questionnaires about metacognition or self-regulation being administered at (pretest and) posttest (e.g., 13, 14, 15, 17, 18, and 19), (b) specifically designed task-specific instruments that are aimed at revealing the use of the intended metacognitive knowledge and/or skills while performing the task during (pretest and) posttest (e.g., 20 and 21), and/or (c) log-file analyses or other types of qualitative evaluations of learners’ metacognitive behavior while working individually, in pairs or groupwise on the mathematical tasks in some or all experimental conditions (e.g., 12, 14, 16, and 22).
As far as the results are concerned, generally speaking, the experimental condition wherein the combination of ICT use and metacognitive pedagogy was implemented in the most intensive or most sophisticated way led to better cognitive learning outcomes, at least for some of the cognitive measures, in terms of learning outcomes as revealed either by pre- and posttest comparisons or by evaluation of the quality of the mathematical work during the instructional program (11, 12, 13, 14, 15, 16, 17, 18, 19, 20, and 21b). Moreover, most studies report positive outcomes for at least some of their metacognitive measures (13, 14, 15, 16, 17, 18, 20, 21, and 22). Interestingly, the few studies that performed separate analyses for different levels of mathematical achievement showed that the learning environment was particularly helpful for the low-achieving students (11 and 22).
However, the size and nature of the effects differed depending on the way in which the metacognitive guidance and training were provided. For instance, the studies suggest that group feedback guidance works better than self-explanation guidance (13), that consistently and regularly exposing students to on-screen metacognitive prompts (without learner control) is more efficient than giving students free access to these prompts (14), and that embedding the metacognitive prompts in the online learning environment leads to better learning effects for mathematics than presenting them face-to-face (18). However, not all studies yielded significant positive outcomes for mathematics learning through metacognitive pedagogies. For instance, in studies by Roll et al. ([59, 60], see nos. 20 and 21 in Table 1), improved metacognitive behavior in students working with a metacognitive tutor was not accompanied by enhanced mathematical performance.
4. Conclusion and Discussion
In this article, we have systematically synthesized the research that integrates ICT and metacognitive pedagogies in mathematics learning from preschool until the end of secondary education. This multidimensional research topic has been approached from two different angles. On the one hand, researchers working with new ICT-based environments for mathematics learning became aware of the pitfalls and risks involved with the increased use of ICT for mathematics learning when viewed from a metacognitive perspective. On the other hand, based on the finding that learners of different ages trained in learning to monitor and control their own cognitive processes for mathematics problems and learning mathematics do better than untrained learners, researchers have started to design metacognitive pedagogies for mathematics learning. Initially, these intervention studies were carried out in traditional learning environments that did not make intentional and systematic use of ICT (e.g., [26, 28]). Yet, the very nature of ICT and its widespread use in the educational systems raise the question to what extent such computer-based environments have the potential to enhance the power of metacognitive pedagogies in the domain of mathematics and, in doing so, enhance learners’ mathematics learning. Over the years, various authors have provided arguments in favor of this claim. While some refer to arguments related to the motivational impact of computers and other ICT tools (e.g., [38]), most scholars ground ICT’s potential impact primarily on their capacity to provide metacognitive cues, questions, and other forms of support and to follow and guide learners’ metacognitive thinking and development in a more fine-grained, systematic, adaptive—and therefore more efficient—way. Although it is not always easy to categorize a study into either of these perspectives, it seems clear that, either explicitly or implicitly, the vast majority of studies involved in this review took the former approach, namely, enriching existing ICT mathematics learning environments with metacognitive pedagogies with a view to make the mathematics learning more successful and more efficient.
As a result of these two distinct but complementary developments within research, the past decades have witnessed a productive line of research on the development and use of various ICT-based tools to stimulate and support different aspects of metacognitively guided mathematics problem-solving and learning.
Given the recency of researchers’ interest in young children’s metacognitive development in general [13], it should, first of all, not come as a surprise that most studies were about elementary and secondary school learners rather than kindergartners. For reasons of feasibility, we did not include other educational levels such as higher education and teacher education into our review, but we do know of studies that are situated at these other levels of mathematics education too (see [63] for example of a study with preservice teachers). It would be interesting to elaborate the present systematic review towards these other educational levels. In contrast, the relatively small number of studies focusing on specific subgroups of mathematically weaker learners or learners with special learning needs is notable, particularly since the potential of both ICT and metacognitive pedagogies for these subgroups has been acknowledged widely. More research on these groups of learners is therefore recommendable.
Second, our review yielded great variation in the kind of ICT-based learning environments that have been developed and used in these studies, including computer-supported practice, educational e-books, intelligent tutoring systems, serious games, multimedia, and computer-supported collaborative learning environments, and this variation tends to become larger with increasing educational levels. While the relation with metacognitive pedagogies has been explored and investigated for many kinds of ICT applications in mathematics education, our literature search has not identified studies that have examined the effects of learning mathematics with newer information and communication technologies such as Facebook and Twitter, or virtual and augmented reality. Given the increasing use of these newer ICT applications and technologies, also in (math) educational contexts [64, 65], it would be interesting to explore their potential for learning mathematics in metacognitively oriented ICT-based learning environments too.
Third, our review revealed that the studies that investigated the interrelation between ICT, metacognition, and mathematics have focused mainly on certain mathematical topics, with an emphasis on early numerical skills in kindergarten and arithmetic and algebra word problem-solving in, respectively, elementary and secondary schools. The strong emphasis on word problem-solving in this review is not surprising, given that mathematical word problem-solving has always been conceived as a privileged topic for assessing and stimulating metacognition and self-regulation [13, 66]. Anyhow, taking into account that each subdomain of mathematics (arithmetic, algebra, geometry, calculus, statistics, etc.) involves different problem types, reasoning styles, and solution methods [11], it is recommendable that future research on learning mathematics in metacognitively oriented ICT-based learning environments would cover a broader spectrum of mathematical subdomains. To give just one example, the metacognitive aspects of learning about mathematical argumentation and proof may differ substantially from those that play a role in learning to become an accurate and quick arithmetic calculator or a better solver of authentic mathematical modeling and application tasks.
Fourth, our review also yielded a wide variety of forms of metacognitive support at all educational levels. Overviewing the studies, a common characteristic seems to be the explicit teaching of the metacognitive skills within the content area through one or more powerful instructional techniques such as modeling and articulation of the intended metacognitive skills and the application of various forms of metacognitive scaffolding (including pop-up screens with metacognitive questions, cues, or scripts) or coaching during intensive and deliberate practice, either individually or in small groups. In contrast, only few learning environments make systematic use of the instructional technique of fading, i.e., the gradual removal of the metacognitive support as the learner progresses, and even fewer do so in an adaptive or intelligent way based on an individualized analysis of the level of metacognitive functioning of the learner. Also, explicit, informative, and individualized feedback on the learners’ metacognitive functioning during the mathematical task completion is typically lacking, although the intelligent tutoring system for learning secondary school algebra and geometry learning in [56, 60] is a prominent exception. Taking into account the well-documented research evidence for the importance of adaptive feedback and support in ICT-based learning environments in general [36, 67], more attention to the design and evaluation of advanced forms of adaptivity in metacognitive pedagogies is warranted. Finally, our review revealed that these powerful instructional techniques are either embedded in the ICT itself or provided by the teacher or otherwise. Except for the study in [54], direct comparisons of ICT-based or human-teacher-based delivery of the (same) metacognitive pedagogy are lacking. So, a remaining question for further research is whether it is better to implement the metacognitive pedagogy into the ICT, to leave it in the hands of the teacher, or to look for blended forms of metacognitive support.
Fifth, while our search terms allowed for a wide variety of research methods, including case studies, observational studies, correlational studies, design experiments, and (quasi-)experimental intervention studies, the vast majority of investigations were of the latter type. Many of those intervention studies were characterized by good internal validity, given that they involved both pre- and posttests and an experimental and control group or several experimental groups, allowing for valid conclusions about the contribution of the ICT and/or metacognitive element to the learning outcomes. However, in a significant number of studies, the research design did not allow to derive grounded conclusions as to what elements or aspects of the computer-based environment were exactly responsible for the obtained (positive) effects. Moreover, many studies involved only pre- and posttest measures, without additional attempts to assess the mathematical and metacognitive learning processes during the intervention, which further jeopardizes a deep understanding of the instructional factors and learning mechanisms that caused the obtained learning effects. Echoing Schunk and Greene’s [6] plea for more research aimed at the exploration of “the operation of self-regulation processes as learners employ them and relate moment-to-moment changes in self-regulation to changes in outcome measures” (p. 6), there is a need for studies with methodologies that also allow to capture this dynamic and reciprocal nature of metacognitive thinking and learning, such as think-aloud protocols, looking for observable traces of metacognition, and microanalytic methods examining learners’ behavior and cognitions in real time as they engage in the learning tasks. Similarly, the additional use of unobtrusive physiological measures during math learning in an ICT-based learning environment might provide a more appropriate picture of learners’ metacognitive thinking and learning. Wrist-worn wearables tracking a.o. skin conductance, electrodermal activity, and heart rate might be promising in this regard [68]. As far as external/ecological validity is concerned, many studies involved rather short metacognitively oriented ICT-based interventions on a specific mathematical content, organized and realized by the researchers outside the context of the regular mathematics lessons. This raises the question if such learning environments would also improve learners’ mathematical and/or metacognitive competence in (more) natural class settings. The scarcely available findings (e.g., [41]) suggest that this is the case but probably with a smaller impact. This is not surprising. Indeed, the effective and sustained implementation of innovative learning environments places high demands on the teachers, requires substantial changes in their traditional roles and practices, and pragmatic adaptations to the concrete demands of the local conditions. This is in line with the more general well-documented findings from meta-analyses of the effects of metacognitive studies on elementary and secondary education, which have made clear that the general impact becomes much smaller once teachers instead of the researchers start to work with the program [2]. Given the complexity of metacognitive development and its complex relation to mathematics learning, on the one hand, and given that ICT is being increasingly widely used in education, on the other hand, longitudinal intervention studies covering large parts of the mathematics curriculum and taking place in regular mathematics classes are needed.
Sixth, the reviewed studies also differ in terms of their dependent variables. While, as a direct result of the scope of our systematic review, all studies involved at least some kind of measure of mathematics achievement, there was great variety in the nature of this measure: some studies used a general measure (i.e., a standardized achievement test or an available overall grade score for mathematics), others made use of a specific measure that was particularly tailored to the mathematical content (e.g., arithmetic or algebra word problem-solving, and geometry) being addressed in the learning environment, and still others also included specifically selected transfer measures. Furthermore, in most studies, particularly at the elementary and secondary school levels, these mathematical assessments were combined with metacognitive measures. These measures were either metacognitive questionnaires or tests administered before and after the intervention or observations, self-reports, and log files collected during the intervention. While all authors explicitly or implicitly assume a positive mediating role of metacognitive activity in mathematics learning, quite remarkably, we did not encounter studies that explicitly tested this mediating effect. Given the assumed positive role of metacognition in ICT-supported mathematics learning, it is highly recommended that this mediating hypothesis is tested in an explicit and appropriate way. Finally, studies investigating the effect of ICT or metacognitive pedagogy on motivational-affective outcomes, such as students’ interest or self-esteem in mathematics, were very rare. Given the well-documented longstanding problems with the affective and motivational aspects of mathematics learning, from the elementary school level onwards [69], more research attention to these aspects is needed.
Finally, as far as the research findings are concerned, taken together, the available research evidence indicates that ICT-based mathematics learning environments supported by metacognitive pedagogies enhance both mathematics learning and achievement and metacognitive thinking and learning, more than environments with no ICT and/or metacognitive support. However, this does not imply that all interventions were successful on all dependent variables being assessed and that all effect sizes were large. But given the literature search method being applied in our systematic review, we cannot exclude a publication bias towards successful studies with significant positive results in favor of the condition(s) with ICT and/or metacognitive support.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.