Abstract
This paper considers the status of educational research that looks to replicate previous findings in a novel educational context, taking as its focus an active area of research in a range of national contexts: studies into students’ ideas about scientific topics. The paper considers the circumstances under which a “replication” study should be considered to offer original new knowledge worthy of publication in international research journals. It is argued here that there are sound principled reasons to expect studies undertaken in different educational contexts to be able to contribute to a progressive research programme, and so researchers should be encouraged to undertake such work. However, technically competent papers submitted to prestigious journals will be rejected if they are considered to merely replicate previous work without offering novel empirical or theoretical content that is considered to make an original contribution. This paper explores the basis for welcoming research “testing-out” published findings in new contexts and considers the place of such studies within a progressive research programme. This analysis can inform research design for those looking to explore learners’ ideas in local educational contexts, by offering clear guidance on the forms of research likely to offer significant contributions to public knowledge.
1. Introduction
This paper will discuss the nature and value of “replication” studies in the science education literature, with a particular focus on research into learners’ scientific ideas and thinking undertaken in diverse educational contexts. The motivation for preparing this paper comes from reflection upon experience as a “user” (reader) of research literature, as an author reporting research, and as someone asked to review research reports submitted for publication to research journals. This varied experience suggests that there may be a lack of clarity—or at least consensus—among the educational research community about what should count as an original contribution to the literature, and so be suitable for publication in international research journals. This certainly seems to be the case in the context of studies of students’ ideas’ and thinking about science topics undertaken in different educational and cultural contexts [1, 2].
The purpose of the present paper then is to explore this issue, and to provide an analysis that may offer the basis for discussion (and perhaps adoption) within the community. In particular, the paper will be based on (i) a consideration of the case for encouraging the “testing-out” of research results reporting learners’ ideas and thinking in science in diverse educational contexts (and especially different cultural contexts); (ii) an exploration of the key notions of “sample,” “population,” and “context” in relation to studies of learners’ ideas. The analysis presented in this paper is intended to inform researchers when designing studies that they hope will lead to international publication in due course.
A theoretical position underpinning the argument made here is that for research within education to be productive, it should be conceptualised within a coherent “research programme” which offers guidance on “live” research questions and fertile and timely directions for research, as well as appropriate methodology [2–4]. This perspective will inform the arguments both about (a) the potential value of research into learners’ scientific ideas and thinking undertaken in diverse educational contexts, and (b) the criteria such studies should meet to be considered to offer significant original new knowledge.
The views presented here are, of course, purely those of one researcher, but may encourage debate and discussion around this issue so that the community may move to a broad consensus to inform researchers in this field. The general principles proposed here may well also be considered useful in other research topics in, and beyond, science education.
1.1. The Structure of This Paper
The paper begins with an explanation of the issue to be addressed, when designing or evaluating studies of student ideas and thinking that may be submitted for consideration by research journals. It is then suggested that studies of this type may be considered to fall within a well-established tradition within science education, that may usefully be understood as a “scientific research programme” [4]. Such a research programme offers heuristic guidance to researchers in the field about the research that is indicated, and how it should be conceptualised (theoretically) and executed (methodologically).
This perspective will then be used to highlight significant features of the research programme into learning in science that will provide the basis for:(i)making the case for the potential significance of further research into students’ ideas about scientific topics across diverse educational contexts;(ii)indicating appropriate methodological approaches for such studies;(iii)offering criteria for deciding whether studies seeking to “test out” (or “replicate”) published findings in new educational contexts make contributions that are significant for progressing the research programme, and so worthy of reporting in international research journals.
2. The Issue: When Does a Study from a New Educational Context Become More Than “Replication”?
There is a vast literature on aspects of students’ ideas and thinking in science [1, 2]. Some of this literature consists basically of studies that describe, characterise, and label the ideas and thinking of groups of students about certain scientific topics. This corpus of work has been accruing for several decades and includes studies across educational levels and from many national contexts.
We might wish to explore learners’ conceptions of scientific topics, or aspects of their scientific thinking, for their intrinsic interest. Alternatively, we might select such foci as a means of investigating conceptual development in its widest sense. The Piagetian research programme [5, 6] was of this form. However, within science education there has been a strong tradition of arguing that such research will inform science teaching [7, 8].
This argument is often based on the basic “constructivist” principle [9–11], that a learner’s current conceptions and beliefs will be strong determinants of learning that is likely to occur in the future. For learning to be “meaningful” [12], it has to link to existing “conceptual structure” [13], and so existing knowledge is the starting point for new learning. So a key assumption of much research into student ideas and thinking is that learning is constrained and channeled by prior knowledge, and that teaching can therefore be informed by an understanding of the learner’s prior knowledge [2].
Clearly another implicit assumption here is that learning is also channeled by at least one factor external to current levels of knowledge: teaching. If we believe that teaching can be made more effective when informed by research into learners’ ideas, then we clearly also believe that changing teaching potentially leads to different learning outcomes, and so that the nature of teaching is one determining factor in school learning.
A further consideration is that research into learners’ ideas and thinking in science has not simply sought to identify whether certain prerequisite knowledge is present among learners: rather learners’ ideas have been compared to the curricular models that act as target knowledge in formal education [14], and variations as well as deficits have been characterized. Some of these “variations,” the so-called alternative conceptions and frameworks reported in the literature, have been judged to be common among (populations of) learners [15, 16], and have often been considered potentially very significant for the course of school learning.
Studies exploring aspects of student ideas and thinking in science continue, although many recent studies offer something beyond description of learners’ ideas (as discussed below). However, some researchers (particularly in countries without long-standing traditions of research in science education) are still undertaking studies that primarily look to elicit and characterise student thinking about curriculum topics in the national context. Often these studies focus on topics and educational levels that have already been explored and reported in the literature [1], but it is not known if the findings from previous research will apply in the local context.
Such studies tend to be framed in terms of a broad constructivist perspective and may be seen as usefully informing local teachers about the thinking of students in that educational context, for example, testing out whether the common alternative conceptions reported in the literature from other contexts have a high incidence locally. Such research is undoubtedly of value in the context where it is undertaken, but the researchers may wish to publish their work in the international literature. This is a worthy aspiration, but such studies, even if considered technically competent, will be rejected from leading journals if they seem to offer nothing of originality and significance. This may seem unfair to the authors: if previous studies on the topic from other countries have been considered to be publishable, then authors may wonder why a comparable study from their country is not considered worthy of international attention.
This issue has the potential to be divisive for the international research community, as inevitably the majority of previously published studies were undertaken by researchers in contexts where there are strong traditions of science education work (principally Europe, Australasia, and North America), whereas the apparently similar studies rejected by journals often originate in countries still engaged in establishing such traditions. This does not imply that there is any kind of “intellectual imperialism” at work, but it is clearly important that all concerned appreciate the objective criteria that justify such editorial decisions [17].
In view of what we understand about how learners’ ideas in science develop (discussed in a later section) it is argued here that systematically treating such papers as mere replications is not a position which is in principle supportable (although this may well be an appropriate judgment on many specific studies that are submitted for publication), and that there may be very good reasons for welcoming such studies (where they meet particular criteria) as significant new contributions to knowledge.
That said, the rationale for the significance of new studies exploring the incidence of established thinking in “new” populations needs to be made. It is the basis for such a programme of studies that the present paper sets out to explore and hopefully clarify: in other words, the circumstances under which a paper exploring the presence and incidence of previously reported findings (e.g., alternative conceptions) should be considered as replication, and the conditions under which such a study should be seen as offering significant new knowledge. In this paper, the notion of significant research contributing to a progressive research programme is used to explore such a position.
3. Judging Studies in terms of Contribution to a Research Programme
In any field of research, individual studies seldom have significance out of the context of the wider field. Research builds upon previous work (i.e., cited in published reports), and almost inevitably offers findings that are provisional, partial, tentative, or at least limited to specific contexts—but nonetheless providing leads for what further research is indicated. Given this, it is useful to have a productive way of modelling a research field.
3.1. A Field of Research
The body of research into students’ ideas in science derives from a range of theoretical perspectives [18], and studies have been considered as significant to the extent that they are viewed as potentially able to inform teaching [19]. Science education research exploring students’ ideas and thinking are then here considered part of a wider field which explores learning in science primarily to support science teaching through informing curriculum design, pedagogy, assessment techniques, and so forth.
3.2. Scientific Research Programmes
The approach to conceptualising research into learning in science taken in this paper follows Gilbert and Swift [20] and Erickson [3] in using Lakatos’ notion of scientific research programmes [4]. “Scientific” here signifies educational enquiry that is based on a broad postpositivist view of science [2], such as that espoused by the National Academy of Sciences in the US [21].
Research into learning science informed by constructivism has often been considered as a paradigm [22, 23] in the sense of Kuhn [24]. However, Lakatos’ model [4] is more useful here, as rather than being primarily descriptive (Kuhn describes “normal science,” but does not provide specific guidance on how researchers should develop the disciplinary matrix within any paradigm), it is a prescriptive model that offers criteria for considering how a research programme remains “scientific” or “progressive.” A Lakatosian analysis of this research field has been developed [2] to both defend constructivism as a progressive influence in science education [8] and to indicate fruitful future directions for the research programme [7].
4. The Progressive Research Programme into Learning in Science
The science education literature already includes a great many studies of learners’ ideas and beliefs in science topics [1]. A multitude of terms [25] has been used to describe the outcomes of research into learners’ ideas in science (intuitive theories, misconceptions, alternative frameworks) with no clear consensus or agreed definitions within the research community [2]. In the present paper I will mainly use the terms “thinking” and “ideas” to refer to the results reported in such studies.
This literature includes studies across a wide range of science topics; collected before, during, and after instruction; from learners of different ages; from a range of national educational contexts. As much of this work was undertaken in contexts and topics of particular interest to individual researchers or research groups, rather than following any coordinated programme, it initially became characterised as being akin to “fishing expeditions” or “butterfly collecting” [26, 27]. The criticism here is that when such studies fail to link substantially with theoretical models (e.g., just identifying “misconceptions”), they do not significantly contribute to a better understanding of learning in science.
However, such a “natural history” phase might be expected in any new area of enquiry, providing the basis upon which a more “scientific” (programmatic) phase of research can build [2, 8]. The early “naturalists” catalogue and start to form typologies of phenomena providing the database to initiate the theorising necessary to proceed to a fully “scientific” development of the field.
4.1. Characterising the Research Programme
From this perspective, much of the research into learners’ ideas in science may—despite considerable variation in methodology, characterisation of outputs, and so forth—be understood to form part of a developing research programme into learning science [2, 7]. This claim derives from the identification, a common “hard core” of assumptions (a key characteristic of any Lakatosian research programme), underpinning a good deal of the research into learning in science.
This hard core comprises of principles that are well established in science education [19, 28–31], such as the following.(i)Learners come to science learning with existing ideas about many natural phenomena.(ii)The learners’ existing ideas have consequences for the learning of science.(iii)It is possible to teach science more effectively if account is taken of the learners’ existing ideas.Such principles provide the tenets for developing a “research programme” into learning in science, suggesting general research questions such as “what ideas do learners’ bring to science classes,” “what is the nature of these ideas?” and “how do learners’ ideas interact with teaching?” [2, 7].
In a “progressive” research programme [4] such questions stimulate studies that lead to theory development and so further refining of research questions. For example, claims of alternative conceptions, alternative conceptual frameworks, mini-theories, multiple frameworks, knowledge-in-pieces and so forth, proposed to describe and characterise student thinking, act as “refutable variants” within the research programme—that is, the theoretical elaborations of the core principles that act as the focus for debate and development in the field [2, 7].
4.2. Progressing the Research Programme
Several decades after the “hard core” of the “constructivist” research programme was established and initially characterised [20], there is undoubtedly a much stronger research base to support pedagogy into teaching science as a result of the vast amount of work that has been done looking into aspects of student thinking and learning in science subjects, despite the considerable amount of fragmentation in the corpus (both in fundamental perspectives and preferred ways of describing phenomena). Although few of the original central questions raised by the programme can be considered to be fully answered yet, there are now many useful models and concepts that have allowed considerable refinement of the questions [2, 7]. The current state of knowledge offers heuristic indications of what needs doing next [4].
For example, studies looking at the complexity of processes of student learning in classroom contexts are more viable for being informed by accounts of the kinds of thinking and the types of ideas found among learners at different levels. Research exploring student thinking has moved beyond simple questions of what students might think or believe in relation to particular topics; or which previously identified conceptions are widely represented; for example to explore how ideas develop over time, within the context of other learning experiences and in interaction with teaching. The foci of these studies call for nuanced conceptualisation, and require sophisticated idiographic approaches to data collection and analysis: developments supported by earlier work within the programme.
4.3. A Focus on Educational Context
The present paper is concerned with one particular aspect of the research programme, that is the value of exploring learners’ ideas across different educational contexts. Such work has certainly appeared in the literature,
“Studies that compare populations across advanced Western countries seem to find few differences of statistical significance… In contrast, studies that search for [sic] differences in substantially divergent cultures often find an “overlay” of traditional views that are quite distinct from explanations offered by contemporary science.” [32, page 186].
However, studies that seek to elicit student thinking on a topic and at an age that has previously been well described, but in a novel educational context, may now be judged as purely replication studies unless they report significant new examples of student thinking [33], or survey conceptions as part of a more ambitious study, such as being the initial phase of an intervention project [34].
From the perspective of the research programme, such studies are only considered significant if they are directed towards progressing the programme by offering something that is theoretically or empirically novel [4]. So, for researchers wishing to describe students’ thinking about topics in a local population, there is a good chance that the results may well appear to journal editors and referees as just “more of the same” unless findings can be linked to specified aspects of the educational context in ways that offer more general significance.
The argument made here is that researchers interested in students’ ideas in science should be encouraged to explore the extent to which findings reported in the literature can be “replicated” among learners in different educational contexts, providing this can be done in ways that further the research programme. In the next section, this position is developed through considering how the current state of the research programme suggests that [2]:(i)learners’ ideas have a range of characteristics, so that merely identifying conceptions is insufficient for effectively informing teaching;(ii)that learners’ ideas are contingent upon a range of influences, which interact in the development of scientific thinking;(iii)studying learners in diverse educational contexts can in principle help identify which classes of influences are particularly significant in the formation of particular ideas;(iv)being able to identify where different classes of influence are significant can inform judgments about the types of educational response most likely to help learners acquire target knowledge in different science topics. In terms of Lakatos’ model of research programmes, research into the effects of different educational contexts on student thinking is directed by the “positive heuristic” of the programme (i.e., the areas of research indicated as likely to be fertile by the current state of knowledge). It is clear that student learning is highly complex, and so optimising pedagogy is at best a long-term goal. Whilst individual studies will only offer partial and tentative insights, those that are clearly guided by the heuristic of an established research programme will be those able to offer original and significant knowledge.
5. The Case for “Replication” Studies
5.1. Factors Influencing the Development of Learners’ Ideas
There has been a considerable debate in the scholarly literature about the nature and status of learners’ ideas (with the choice of terminology often reflecting the views taken by different authors). It is not productive to rehearse all the arguments [2] in detail here, but it seems that learners’ ideas may be tenacious, or labile; may be consistently held, or not; may be extensive and theory like, or relatively discrete and (conceptually) isolated; may have a clear simple structure, or may be multilayer or multifaceted.
Where some authors have argued that learners’ ideas in science fall at one pole of these constructs; the view taken here is that there is likely to be considerable variation in the character of specific conceptions elicited in research. Learners of science themselves vary in many ways: age and maturity, interest in a topic, motivation for school learning, and so on. Therefore rather than make an ad hoc assumption that their conceptions of science topics will all be (e.g.) romanced notions that are soon dismissed—or all be strongly held beliefs to which they are highly committed—a sensible default assumption may be that elicited ideas are likely to fall upon a continuum stretching between such extremes, and this would seem to be supported by the variety found in research findings [2]. Students’ ideas may be better considered as being located somewhere in a complex phase space (e.g., see Figure 1), reflecting how they may be contingent on a range of influences.
5.2. Intuitive Ideas
One possible origin of learners’ ideas is that they are instinctive: that is that they are programmed into us as part of our genetic heritage. It would be easy to dismiss such a source, as it seems highly unlikely that human DNA can directly code for (say) a belief that the compound ATP has an “energy rich” bond or that circular motion is “natural” (i.e., two commonly reported alternative conceptions from science topics). However, important aspects of the human perceptual-cognitive apparatus would seem to be part of our common genetic heritage, and that such “biases” in perception and cognition certainly constrain and channel our thinking [13]. In other words, this may well be a contributing factor, if not the causal factor, in the development of many alternative conceptions [13]. Research suggests that some common conceptions may be due in significant part to our instincts [36, 37].
Even if the role of instinct may seem fanciful, there seems little doubt that intuition plays an important part in learning [38]. Intuition is a process where we come to understanding or judgment without consciously following a logical argument [39]. This certainly does not need to be seen as anything mystical, as intuition is clearly the output of cognitive functioning, even if that functioning is not available to our conscious minds, and does not seem to follow a linear logical process. Intuition can certainly be reliable in some circumstances, even if when well honed we may prefer to label it as “expertise” [40]. The important point here is that intuition is based upon experience, and using that experience to in some way model the world, for example, to allow us to visualize hypothetical situations [41], as in thought experimentation [42].
It would seem likely that intuition, in this sense, is largely responsible for one of the best-established and common alternative conceptions: the belief that force causes motion (rather than acceleration), and that without a force acting objects will soon lose their “impetus” [43, 44]. Whilst this is seen as an alternative conception in science education, it is clearly based on what is common experience to us all, and so might be considered as “common sense” to those who do not see the advantages of the scientific perspective [45]. The intuitive understanding certainly works in modeling the world in most “everyday” circumstances, and it could be argued that it is the school science formalism (taking the absence of resistive forces and gravitational field as a starting point for analysis) that is better considered an alternative conceptual framework.
5.3. Cultural Effects in Learning Science
Considering “common-sense” brings us to another possible source of our conceptions: other people [46]. All other things being equal, it is probably an adaptive trait (i.e., a useful natural tendency that has evolved) to take on board the ideas and beliefs common among your peers, even when you do not have direct evidence yourself. Learners learn from when they are told by others (albeit filtered through their “instincts” and intuitions). These others may be parents, siblings, friends, and in this day and age, various media (newspapers, television, film, novels, comics, the internet, etc.). Sometimes, of course, these others may also be their teachers.
So some common ideas elicited from children are spread, at least in part, through informal learning in everyday “life-world” contexts [47]. Through such processes youngsters are inducted into the beliefs of their culture. Ideas that are common in a culture will not usually contradict everyday experience, but clearly beliefs may develop and be disseminated without matching formal scientific knowledge. Ideas that become “common knowledge” [48] may be adaptive within the society [49], for all sorts of reasons (relating to social cohesion, etc.), as is clear from the widespread occurrence of various superstitions and folk-beliefs. When such beliefs become associated with important cultural rituals, taboos, claims for social status and so forth, there may well be robust mechanisms maintaining them within the culture [50].
Where life-world beliefs are relevant to school science—perhaps contradicting scientific principles, perhaps apparently offering an explanation of some science taught in school; perhaps appearing to provide familiar examples of taught principles—then it is quite possible, indeed likely, that such prior beliefs will interfere with the learning of school science. There are clearly several possible outcomes in this situation (as was pointed out long ago by Gilbert et al. [30]), and sometimes school knowledge may well be compartmentalized as a special domain for use in schoolwork and examinations [51]. However, if new meaningful learning occurs by building upon existing knowledge structures, then this will commonly mean making sense of school science in terms of wider beliefs systems [29].
Different common beliefs will be found among different cultural groups, and therefore it is likely that the same scientific concepts will be interpreted differently among different cultural groups as they will be interpreted through different existing conceptual frameworks. All school level learners in science be can be considered to be “crossing borders” when moving from the familiar life-world ways of thinking and knowing to the formal knowledge structures and epistemological rules of science [52], but in some populations, where the prevalent worldview is most at odds with that common in technologically advanced society [53], this cultural shift takes them to a place that must seem very “anthropologically strange” indeed [54].
5.4. Linguistic Environment and Learning Science
Much of the communication of ideas within a culture, both informally and in the classroom, takes place through verbal language, and so the particulars of a learner’s language will enable, channel and constrain learning [55]. Science is sometimes considered to be a universal language, and certainly the scientific community puts stress on internationally agreed definitions and systems of nomenclature (SI units, IUPAC rules for naming compounds, etc.). However, individual scientists tend to work in a particular language community. Whilst English has become established as the main language for many international journals, research from many countries is commonly only translated into English at the point of dissemination. Scientific terms may themselves have different nuances in different languages.
For example, the notion of an element (a fairly basic scientific concept) appears to carry different nuances in France to in Anglophone countries [56]. A student in an English school who suggested that the elements that make up [sic] a compound are still present in the compound would probably be considered to have missed an abstract, but significant, feature of how substances are defined in chemistry. So sodium chloride has new unique properties distinct from its “component” [sic] elements, as it does not actually “contain” any sodium or chlorine—these substances cease to exist on reacting to give a new product. Yet in Francophone countries “element” has a somewhat different meaning (perhaps implying more something of the “essence” of the Anglophone element), and the element is understood to be conserved on forming compounds. A student in a French school making the same statement (in French!) would not be considered to have formed an alternative conception, as the French language is not simply a different lexicon for the same basic meanings that are signified by English words, but rather a distinct system for organising and communicating meanings such that translation inevitably modifies meaning. As has been well recognized, our thinking is channeled by the language we have available to express our ideas, and the nuances that attach to the words we use [57].
5.5. Teaching as an Input, Influencing Learning
Research into learners’ understanding of, and ideas about, science topics is expected to inform features of science teaching. These features will potentially include such matters as the sequencing and “spacing” of related topics, and the teaching models (analogies, etc.) that are effective.
Clearly then, these are factors which we expect to influence future learning, and so are variables that may well have partly determined the ideas that learners presently exhibit. This should imply that groups of learners taught science topics in significantly different ways (in these terms) might be expected to have developed different sets of ideas.
Science is represented in the curriculum as a set of curriculum models that are (more or less intentionally designed) simplifications of formal scientific knowledge considered appropriate for students at a particular stage of their scientific education. These curriculum models are then themselves represented through the teaching models used in the classroom [14, 58]. Different curriculum authorities, for example in different countries, will make different decisions about what is important to teach; what topics are suitable at different grade levels; the optimum level of simplification of material that can be accessible to learners whilst retaining the essential aspects of the scientific knowledge [59]. These decisions will clearly influence what is learnt, and what is understood, and so the nature and frequency of common alternative conceptions.
It therefore follows that a full description of an educational context, in terms of relevance for studies into learners’ ideas, would need to specify such matters as curricular content and sequence, and the ways scientific knowledge is modeled in teaching. Therefore groups of learners who had experienced teaching that varied in terms of, for example, sequencing of topics, can be considered to occupy different educational contexts and could well be considered to be distinct populations of learners. This would obviously mean that students of the same age, and apparently studying “the same topic,” in different countries were likely to be different populations, and so studies into their ideas about the science topic should not be considered as simple replications.
Of course, there may well be cases where different groups of learners in the same country would need to be considered as different populations by this criterion. Indeed, in the UK context, it is common that different classes in the same year group in the same school rotate around science topics in a different order for pragmatic and organisational reasons. In principle there are different educational contexts at work here, offering opportunities for “natural experiments” (or quasi-experiments) [60] into the effects of different topic sequences.
5.6. Researching Learning in Babel
Of course, it is unrealistic to consider that researchers could ever provide fully detailed accounts of the educational contexts of particular studies—for example individual teachers may use idiosyncratic metaphors, models, and explanations that could be significant but which are seldom likely to be documented [61]. This is especially the case when we consider the iterative nature of learning, as a significant teacher input may occur years before a period of data collection.
Yet there is an important principle here. We consider issues such as sequencing of materials, levels of simplification of concepts, and use of models and metaphors, to be important in teaching, and so they are part of the learning context that contributes to the development of student thinking probed in educational research. This is certainly recognized in some studies that explore teaching and learning at the level of classroom processes [62] and/or individual’s learning [63], although it may seem totally unrealistic to expect research with large (cross-teacher, cross-institution) samples to be able to offer documentation of context at such a level. This raises the issue of the methodologies that are indicated by the current state of the research programme.
6. Methodological Approaches Indicated by the Research Programme
The position derived from above is not based on any “bold conjectures” [64], but relies upon ideas that are largely well established in the thinking and literature of science education: the ideas that learners hold will surely depend, to various degrees, upon the inherent biases of the human perceptual-cognitive apparatus (which is at least partly under genetic control); the intuitions developed from interpreting common experiences through that apparatus; from opportunities to learn from others, either through everyday life-world contexts, or through formal educational settings, where symbolic communication, and especially verbal language will act as a medium.
Learning will be contingent upon all these factors, and as new learning builds upon previous learning iteratively these factors will interact. Different personal experience, different cultural beliefs, different languages, and different curricular contexts, can all potentially lead to different conceptions of scientific topics.
At one level this analysis suggests that (a) it is clearly impossible to ever offer a full analysis of how any learner came to a particular understanding of a scientific topic; (b) individual learners will have unique “learning histories” that make generalisation in this area difficult: as even within a single teaching group, there will be significant variations in how individuals understand a topic that has been taught (neither of these conclusions will be surprising to anyone who has worked in education as a teacher or researcher.)
Although the use of controlled experiments has sometimes been seen as an ideal in science education [65], this is clearly not a realistic approach (leaving aside the ethics of treating human learners as experimental subjects) when dealing with learners studying in very different educational contexts where many “variables” are likely to be pertinent, interacting, and even shifting. This complicates, but does not negate the value of exploring students’ ideas to inform teaching. It is clearly the case that we are dealing with phenomena that are too complex to ever describe and understand fully, but there are—nonetheless—practical ways of developing useful knowledge, that fall within a generally postpositivist (i.e., “scientific,” [2]) approach to producing new knowledge [21]. In particular, there are two distinct and complementary types of research that can contribute to a progressive research programme [66].
One approach is to undertake in-depth case studies of particular learners’ developing ideas, or of the teaching and learning in specific contexts. Case studies are indicated when the phenomena to be studied are complex, subtle, and somewhat idiosyncratic [67–69]. Research into aspects of student thinking that probe the nature and evolution of learners’ ideas may be productively studied in this way.
As such idiographic research recognises the unique nature of individual learners, informants are not expected to be representative of a wider population, and “generalisation” as understood in more traditional research is not expected [70]. Indeed, the selection of cases may involve identifying atypical cases [60] either for principled reasons (the case is considered to be of particular interest) or due to pragmatic considerations such as access and extent of available data [63]. Even if an informant is considered to be typical of a wider group when selected for a study, the very process of contributing to an in-depth study is likely to itself be an influence on the very phenomenon being studied [71]. Generally, studies that report in-depth accounts of individuals’ thinking are expected to offer detail of the research context [72], but as they deal with small numbers of atypical cases, they cannot be generalised to wider populations.
The second approach involves attempts to survey identifiable populations of learners, by using techniques that allow data to be collected from larger sample sizes. This approach clearly requires some conception of the population being sampled. If each learner is an individual with a unique learning history, then clearly all populations are heterogeneous in many ways. Yet, it is clearly possible in principle to identify populations in terms of certain relevant characteristics that allow them to be compared with other populations with different characteristics. Such an approach can never “control” for all possibly relevant variables, but it can—nonetheless—offer comparisons that may suggest very useful insights.
6.1. Surveying Populations in Diverse Educational Contexts
It follows from the earlier account of the different influences on student thinking that we should expect similarities and differences in the frequencies of, for example, alternative conceptions in different educational contexts to reflect the relative significance of such contextual variables as, interalia, contingent factors in the development of those conceptions.
If similar frequencies of common alternative conceptions are found in different contexts, then it may be that the conceptions tend to develop regardless of these contextual factors (language, cultural beliefs, and educational practices) or that although cultural factors play a significant role, the two educational contexts are similar in terms of the particular significant contextual factors at work. Where very different frequencies of conceptions are found among “comparable” populations in different contexts (e.g., students of the same age, having been exposed to teaching that seems to be directed to similar target knowledge) then there would seem to be a prima facie case for considering contextual factors as a significant influence, and for looking to identify likely factors (see Table 1).
Cross-cultural studies that use common methodology to explore student thinking (rather than focusing on attainment or attitudes to science [73]) in different contexts [74–76], and which find similar frequencies of common alternative conceptions in a range of contexts (e.g., different cultural groups, different languages of instruction, different school systems) begin to suggest that the development of these conceptions may be strongly influenced by factors that we might class as “intuitive” and so forth (see above).
Studies that find features of student thinking that seem to be significantly more common in specific contexts offer hints at how cultural, linguistic, or institutional factors may influence the development of scientific thinking [75]. All of this information, however tentative, may be useful in building up an understanding of how science is learnt, that can inform teaching. Studies that are able to identify specific institutional features relating to the curriculum models used, the sequencing of instruction (within science, and in terms of other school subjects and age-related development), common teaching models and analogies, and so forth, may lead to more specific hypotheses that can be more directly tested (see Figure 2, [2])—for example in curriculum projects [77], lesson study [78], or “design experiments” [79].
However, differences between populations can only be considered significant where surveys are based upon(i)methodology that is comparable; (ii)sampling methods that can be considered to give representative findings.These are not trivial concerns. For example, Kuiper [80] failed to replicate Watts’ [81] reported alternative conceptions for force when surveying populations from different cultural contexts to that where Watts undertook his original research. However, the methodology used in the two studies was incommensurable, so that very little can be read into the “lack of replication” of the original findings (this example is explored in more detail elsewhere [60]). Even when attempts are made to carefully replicate the original methods, if research instruments have to be translated into a local language, there is immediately a problem as translation inevitably modifies meaning and emphasis to some extent.
6.2. Sampling a Population
A study that explicitly discusses a sample implies there is some particular (defined) population being sampled, so we would expect any such study to offer a clear account of what is meant by the population being sampled in that study. For research that replicates previous studies in distinct populations can only make significant original contributions to the field, when research reports offer a clear description of the population being studied.
Yet few educational researchers are in a position to undertake large-scale random (or stratified) sampling of populations, so compromises usually have to be made (e.g., in working with a small number of schools who are considered to be reasonably typical of those in the region or country). Consequently, most educational research studies (there certainly are exceptions [82]) do not use very rigorous methods to sample populations of learners.
Despite these limitations, it is clear that if surveys are to contribute to a developing understanding of the how features of educational context influence student thinking then researchers must be able to clearly define the population sampled; provide “thick description” [83] of the educational context in which the population studies; provide assurance that the sampling methods used if not technically rigorous are at least suitably robust.
There is some degree of iterative work needed to meet these requirements: decisions about which aspects of an educational context are salient and worth reporting, and so how to demarcate what is to be considered an identifiable population for these purposes are necessarily underdetermined. To design effective studies, we need to be able to take into account the very things the studies are meant to be finding out. Initially a good deal of informed guesswork may be needed. However, over time the indications from studies designed to best meet these criteria should offer increasingly sophisticated guidance on how to operationalise these requirements.
What does seem clear is that descriptions of survey samples limited to very general information about student ages and geographical location will be inadequate to support the production of original knowledge that can be considered to make significant contributions to the research programme.
7. Conclusion: Criteria for Designing Significant “Replication” Studies in Diverse Educational Contexts
The argument made in this paper is that in principle it is useful and informative for the findings from research into learners’ ideas to be tested out among different populations from diverse educational contexts, even though making direct comparisons is difficult both for pragmatic and principled reasons.
The range of types of “factors” that we might currently expect to be significant in determining the ideas that learners acquire and develop prior to and through science education leads to an expectation that educational context is often likely to be significant in determining the extent to which particular ideas develop, are committed to, and may be avoided or challenged by changes in teaching approaches.
This suggests that studies from different contexts (e.g., different countries, different cultures, different languages of instruction, and different curriculum organisations) should be encouraged for what they can tell us about the relative importance of educational variables in encouraging, avoiding, overcoming, or redirecting various types of ideas students are known to develop. Whilst these factors only operate among, and in interaction with, others, they are the ones over which educators potentially have some control. Where learners can be initially guided to the target knowledge through pedagogy, the challenges of responding to alternative ways of thinking may be avoided. Research that surveys the incidence of particular ideas in well defined populations of learners, has potential to make a significant contribution to developing the field and informing teaching, but only when published reports offer sufficient background information on the educational contexts studied to begin the process of teasing out significant contextual factors from among the myriad interacting variables.“Education does not take place in a cultural vacuum. All teaching and learning has a geographical, a historical and a social context; it happens in a particular place, at a particular time, and it involves particular people. It makes use of a particular language, and it takes place against the background of a view of the world and of man’s place in it characteristic of a particular society.” [84, page vii].
Clearly then, the value of such research depends upon authors documenting their work in sufficient detail for findings to contribute to the on-going research programme. So a study that reproduces a survey with learners of the same age and educational level in another country without engaging with the issues raised here is likely to be considered to be replicating existing work, and not suitable for reporting in a top journal. This is a fair judgment for reviewers and editors to make when readers are not offered a basis for considering how the different contexts might have influenced the similarities and differences between the aspects of student thinking reported in the different contexts.
This analysis leads to the following general guidelines (which need to be refined further as this strand of the research programme proceeds) for researchers wishing to contribute to this research programme by testing-out previously reported findings in new contexts.(1)The study should give a clear indication of the bounds and the context of the study population, and how it compares with the context of the original research. This should, as far as possible, highlight relevant similarities and differences that might be expected to act as contingent factors influencing the development of student ideas: specific linguistic factors, relevant cultural beliefs or traditions, curriculum factors, significant features of pedagogic practice, and so forth. (2)The study should make it clear how the sample of informants in the study can be considered representative of the population claimed as the focus of the study. Whilst large-scale random sampling of regional or national populations is seldom possible, it should be clear what safeguards have been taken to provide a typical sample, and what, if any, caveats need to be considered.(3)Researchers making quantitative comparisons with the findings of previous research, should offer a convincing case that the instrumentation used is suitable for this purpose (as between population comparisons are of limited value when changes to the data-collection techniques used would be likely to modify the frequencies of different ideas elicited or identified within a population).In practice these guidelines are challenging. Clearly there are a great many potential features of a research context that could be relevant—and it is not possible to identify and detail them all. The original data-collection instrument may be unavailable or culturally inappropriate or impractical in the new context, and so forth.
Samples are seldom fully representative of larger populations, and there is a temptation to assume that samples that are only representative of narrower populations (high-schools in one city; undergraduates in a more prestigious university) may seem to be of more limited intrinsic interest. However, the logic of the argument made here is that a representative survey of a modest but tightly bound population with a well-described context is more informative than research that claims to report on a large population, but inevitably targets a heterogeneous educational context with inadequate sampling.
In the final analysis, few individual studies in education offer definitive findings, as we usually have to simultaneously deal with too many variables beyond our direct control. However, a study of students’ ideas in a new context rises above simple “replication” to provide significant new knowledge when it moves the research programme forward by offering clear indications of the extent to which aspects of student thinking seems to be contingent upon particular cultural, linguistic, or educational factors. Careful studies offer hints that provide testable conjectures for further research—that may ultimately lead to findings that can directly inform teaching (see Figure 2). Progress will be incremental, but studies designed to incorporate the features recommended here will at least have the potential to contribute to this challenging but ultimately important work.