Abstract

Using a comparative case study design, this paper explores the impacts of a technology-related professional development (TTPD) design aimed at helping science and mathematics teachers design classroom activities using the wealth of resources available on the Internet. Using the lens of curricular adaption and the notion of teachers’ varying pedagogical design capacity, we analyzed the experiences of four teachers in terms of the kinds of instructional activities teachers designed, how these were supported with online resources, and teachers’ perceptions of impacts on student learning. Findings suggested that participants used a variety of personally relevant design strategies when applying TTPD concepts to their contexts. In particular, the teachers discussed how they tailored instruction to fit their students’ needs and interests, and how they incorporated instructional games, simulations, and interactive resources to enhance motivation and provide self-paced instruction.

1. Introduction

The past decade has seen enormous growth in the Internet-based network of free, online resources available for teaching and learning. These resources, variously called learning objects, open educational resources, or online learning resources, include innovative and interactive curricula, teacher-created lesson plans, as well as tools such as visualizations and simulations that support the manipulation of real-world phenomena and datasets [13]. These resources are often aggregated, curated, and made available in resource collections, such as the Digital Library for Earth System Education, and portals that facilitate educator access, such as the National Science Digital Library, Teachers’ Domain, and the recentlyannounced US Department of Education’s Learning Registry. The vision is that, supported by this increasingly available infrastructure, teachers and students can access, create, connect, and share knowledge in ways that fundamentally transform practice [1].

Yet little is known about what kind of instructional practices best supports student learning with online resources [1, 4]. Moreover, despite educators’ documented beliefs that online resources can enrich their classrooms and improve student learning [5], many barriers remain. These include insufficient technology knowledge, inadequate technology access, and the overall inherent complexity of classroom technology integration [69].

Many studies have documented that teacher professional development can be an effective way to improve teacher knowledge [10]. As such, to help teachers enhance their knowledge and skills in learning to design activities for students using online learning resources, we developed a technology-related teacher professional development (TTPD) design. As part of the TTPD, teachers learned to use a free web-based tool designed to help teachers find and design with online resources, called the Instructional Architect (http://ia.usu.edu/).

The impacts of the TTPD design were evaluated in a quantitative study involving 36 science and mathematics teachers and over 1,200 students [11]. While the results of this quantitative study revealed significant positive proximal and self-reported impacts, less was known about the experience of individual teachers when attempting to apply TTPD concepts in designing and implementing classroom activities using online resources. As such, the purpose of this paper is to present findings from a comparative, multiple-case study and provide in-depth analysis of four teachers’ experiences and activities. Using the lens of curricular adaption, we examined the kinds of instructional activities teachers designed, the pedagogical strategies they used, how these were supported with online resources, and teachers’ perceptions of their impact on student learning. We also examined the barriers teachers encountered during design and implementation.

In the next section, we describe the theoretical framework underlying our study, followed by the study context. We then present findings from the case studies and conclude with a discussion of implications and limitations.

2. Theoretical Framework

While less is known about teacher use of online learning resources, prior research has examined teacher adoption (and nonadoption) of curricular material [12, 13]. This work has critically examined the assumption that curriculum materials are implemented unchanged by teachers. In a review of the literature, Remillard [13] proposed a framework for describing teacher use of curriculum, in which teachers’ beliefs, knowledge, and identity interact with curriculum features (e.g., representations, structures, and voice) to influence resulting design and enactment. This perspective fits with a more contemporary view of teaching as a kind of design task, in which teacher adaptation and use of materials is seen as a critical step in curriculum design.

Empirical studies have indeed found that teachers do adapt curriculum to fit their teaching context [14]. This adaptation process can support both the development of instruction tailored for individual students as well as helping the teacher learn new content and skills [15]. Moreover, Brown and Edelson [16] suggested that teachers vary with respect to their ability and skills to engage in principled design and adaptation of curriculum, a skill dubbed pedagogical design capacity. In this view, curricular materials afford and constrain design, interacting with teachers’ unique knowledge, skills, and experience. As part of this work, Brown and Edelson [16] defined a continuum of teachers’ curriculum use, which ranged from offloads to adaptations to improvisations. This continuum describes the degree to which the design of instructional activity is differentially divided between the instructional resources and the teacher. In an offload, the curriculum resource is implemented essentially unchanged, and the majority of instructional decisions are scripted within the resource. At the other extreme, improvisation, a teacher may flexibly mix and match aspects of the curriculum while playing a large role in instructional decision-making. Adaptation, then, represents the midpoint of the continuum. Brown and Edelson [16] also noted that the continuum is neutral with regards to quality or effectiveness of the resulting designed activity.

To more fully explain how teachers’ lesson plans and classroom activities can be characterized as an offload, adaptation, or an improvisation, we propose an analogy with three styles of music composition and performance. Recall that offload designs are instances in which teachers use the curricular resources unchanged. An offload, then, can be compared to classical music score and performance. As expected by the composer, a classical piece of music is practiced and performed as the score is exactly written. The performers do not deviate from the original composer’s score.

On the opposite side of the spectrum, an improvisation is comparable to jazz improvisation and performance in that the musicians may have a composition they intend to follow, but will frequently improvise by adding extra elements into the performance, which were unplanned during rehearsal. Like jazz musicians, teachers may have designed an activity or lesson plan based upon learning objectives and pedagogical principles, but implementation in the classroom often mirrors an unplanned rehearsal, using an outline or idea as a starting point, but where the material is presented in an impromptu manner to the class.

An adaptation is comparable to a familiar melody already composed, rewritten, or adapted by a different composer, thus creating a new composition. Design adaptations, then, are lesson plans or activities in which the teacher has modified the existing material in a small manner, but the original design would be readily observable. Once implemented in the classroom, an adaption is similar. A teacher would present the activity, with some elements of the original curriculum design but will have modifications within that curriculum.

Supporting this view of design, some scholars argue that teacher professional development should explicitly focus on supporting teachers in productively designing with curricular materials [1618]. In this way, teachers may increase their pedagogical design capacity in order to make principled adaptations of high-quality curriculum materials that are responsive to the needs and interests of their students [19].

In this work, we do not mean to imply that curriculum usage and adaptation necessarily entails the same processes as using online resources. For example, we do not propose that online resources play the same role as, for example, a school district-mandated curriculum. In the latter, organizational factors clearly play a large role. Instead, we use this lens to investigate how teachers choose to use online learning resources in their own design and implementation of classroom activities. We also examine how teachers’ perceptions of the needs and interests of their students may also play a role.

Finally, we note that Brown and Edelson [16] were examining instructional planning and classroom implementation together. However, others have proposed that planning and implementation should be considered separately [13, 20]. Thus, in our work, we separated these temporal events by examining the activities designed by teachers as well as their descriptions of classroom implementation experiences and impacts on students.

3. Case Study Research Design and Methods

The multiple, comparative case study was bounded by a teacher, his/her students, and classroom planning and implementation activities. The following research questions were investigated.(1)What kinds of activities did participants design and how were these supported with online resources? What barriers and enablers did participants encounter?(2)What were teachers’ perceptions of impacts on students when learning with online resources?

3.1. Case Study Context

This case study was conducted in conjunction with a larger study investigating the impact of a TTPD intervention on teachers’ practice and student learning. The study took place within a large, suburban school district in western USA. Thirty-six junior high school mathematics and science teachers and 1,247 students participated in the study.

The TTPD focused on enhancing participating teachers’ technology skills for finding and selecting online resources from the wealth of information available on the Internet, and designing classroom activities around these resources using web-based software called the Instructional Architect. In the next sections, we describe the technology context (the Instructional Architect), the TTPD design, and the case study methods and participants.

3.2. Technology Context

The technology context for the technology-related teacher professional development (TTPD) is a free, web-based tool, called the Instructional Architect (http://ia.usu.edu/). The IA supports teachers in authoring instructional activities for students using online resources increasingly available on the Internet and in specialized educational repositories, such as the National Science Digital Library (http://nsdl.org/) [21].

Teachers can use the IA in several ways. Once logged in, the “My Resources” area allows teachers to search for and save links to online resources, including interactive and multimedia resources. In the “My Projects” area, teachers can select online resources and annotate them with text to create learning activities (called IA projects). Finally, teachers can “Publish” these IA projects for their own students, or anyone on the Internet. In this way, the IA supports a community of teacher users by allowing teachers to share and copy each others’ IA projects.

Since 2005, the IA has over 7,500 registered users who have gathered over 72,000 online resources and created over 17,000 IA projects. Since August 2006, public IA projects have been viewed over 2.5 million times. IA projects created by each of the case study participants are shown in the following.

3.3. Technology-Related Teacher Professional Development Workshops

The TTPD workshop activities focused on the following technology skills: (1) finding online resources, (2) designing learning activities around these online resources using the IA, and (3) implementing these IA projects in the classroom. The TTPD was structured as a series of three workshops with in-between activities (see Table 1) and was conducted face-to-face over three months.

Following design-oriented approaches in technology-related professional development [18], the participating teachers engaged in authentic and complex problems within their own teaching, designed solutions using the IA, implemented these in their classrooms, and reflected with their peers on their experiences.

Table 1 shows a sketch of the key activities in the technology-related teacher professional development, as well as all data sources and collection points.

3.4. Participants

From the 36 participating teachers, four participants were purposively selected from the TTPD implementations using the following criteria: (1) they completed all the components of the TTPD (including attending TTPD workshops, completing pre- and postteacher surveys, submitting reflection papers, designing at least two IA projects, and collecting student data); (2) they represented different levels of technology knowledge (based on teacher self-reported presurvey data); (3) both math and science teachers were represented, and (4) both male and female teachers were represented (see Table 2).

3.4.1. Case 1, Mrs. B.: Low Technology Knowledge Teacher

Mrs. B. is a female junior high school science teacher with seven years of teaching experience. She has taught “all over the world” and considers herself an “anxious teacher” when using technology. Her first IA project was on the topic of biology classification and her second was on “states of matter” (see Figure 1). She created the largest number of IA projects (13) and collected the most resources (63) over the 9-month time period.

3.4.2. Case 2, Mr. O.: Low Technology Knowledge Teacher

Mr. O. is a male junior high school mathematics teacher with eight years of teaching experience in his current position, having also previously taught elementary school. Mr. O. viewed himself as a moderately capable Internet user, frequenting the National Council of Teachers for Mathematics (NTMC) website for math information. His first IA project was on the topic of “interpreting graphs and tables” while his second covered scientific notation (see Figure 2). Compared to the other three participants, Mr. O., however, was a lukewarm IA user, recording some of the lowest numbers of logins to the IA (37 times), as well as number of resources collected (15), IA projects created (5), and views (332).

3.4.3. Case 3, Mr. W.: High Technology Knowledge Teacher

Mr. W. is a male junior high school science teacher, who has taught for three years. His first IA project addressed the topic of ecology while his second was on physical and chemical changes (see Figure 3). Mr. W. explained that he “was never scared of technology and used it frequently in his teaching.” In terms of the IA usage data, Mr. W. had a low number of logins to the IA (37 times) and yet collected a large number of resources (42), created 11 IA projects, and had the largest number of views of his IA projects (1932) over the 9-month period.

3.4.4. Case 4, Mrs. R.: High Technology Knowledge Teacher

Mrs. R. is a female junior high school science teacher with six years of teaching experience. Her first IA project was on the topics of “solids, liquids, and gases,” and her second taught the concept of density (see Figure 4). She logged in to the IA the most frequently (50 times) and yet only created six IA projects and used 21 online resources over the 9-month time period.

3.5. Data Sources

Table 3 describes the primary data sources used for the case study. Online pre- and postsurvey questions regarding teachers' experiences in the TTPD were administered before and at the end. Details of the survey design are described in [11]. IA usage data, automatically collected by the IA system, was analyzed to determine teacher and student use of the IA. Reflection papers provided by each of the participants were collected after the second and third workshops. In these papers, teachers were asked to respond to prompts shown in Table 3.

The four teachers were interviewed by one of the authors approximately two months after their participation in the TTPD. Each interview was approximately 45 minutes long and was framed by a set of open-ended questions that enabled teachers to discuss their experiences (see Table 3). The interviews were recorded and transcribed.

3.6. Data Analyses

The interviews and reflection papers were analyzed using the constant comparative method [22]. Four researchers first independently read one of four case narratives and identified the emerging themes. Then, themes were compared and categorized to form a new coding scheme. All cases were then read again and recoded according to the new coding scheme. Through this iterative process, data were reduced, consistent themes were identified, and findings were triangulated as they addressed the research questions.

The content of each teacher’s IA projects, designed and implemented after the second workshop, was examined for their overall design. Our analysis of IA project design followed Brown and Edelson [16] in that we categorized the IA projects’ design as an offload, an adaptation, or an improvisation. We then referred to other data sources, such as interview transcriptions and reflection papers to determine how the IA projects were implemented with students, and the teachers’ perceptions of the implementations.

4. Case Study Findings

4.1. Research Question 1

What kinds of activities did participants design and how were these supported with online resources? What barriers and enablers did participants encounter?

4.1.1. Case 1, Mrs. B.

Mrs. B.’s second IA project on the topic of “States of Matter” consisted of a large collection of links with direction on how to access them (see Figure 1). Students had a worksheet of assessment items, and the IA project directed them to respond to specific items after interacting with the content on each link. In this way, the IA project’s design can be characterized as an offload:

“I had a matter video. I had the students answer 12 questions about States of Matter and then they learned a new state of matter called the BOSE Einstein condensate and then they had a worksheet. I’m not as technologically young as some teachers so some of them had Google docs set up and I didn’t understand how to do that. So my worksheet was typed on Microsoft word and run off and given to my students. And so they answered (questions) through the guiding worksheet describing matter” (Mrs. B. Interview).

Mrs. B. also used the IA project as an offload in her classroom, in which students followed her directions while she projected the IA project and then had students to answer questions on a worksheet.

In terms of technology barriers, Mrs. B. reported “slow video as everyone tried to access at once, difficulties in reserving the computer lab, lack of computers in classrooms, lack of headphones in lab, and crashing program” (Mrs. B. Reflection paper 2).

In terms of enablers, Mrs. B. considered the IA to be a time saver because it saved her grading time:

“some handwriting is so illegible, so having students go through the IA was less reading for me to have to go through, less paper work for me, less time for me to have to review and grade papers. For me it was a time saver” (Mrs. B. Interview).

4.1.2. Case 2, Mr. O.

Mr. O.’s second IA project on the topic of “Scientific Notation” was very short, with only a small amount of text and a few links to resources containing examples. The project design was classified as an offload (see Figure 2). However, as he explained in his interview, Mr. O. wanted students to deduce rules (a more inquiry approach) by looking at examples:

“that’s not a true discovery lesson that I came up with but it was closer than I did the first time. Because they were looking at examples, correct and incorrect ways in writing scientific notation and they were trying to come up with the rules on their own. So they were doing deductive thinking rather than just being told the rules” (Mr. O. Interview).

In this way, while the design of his second IA project can be characterized as an offload, his interview suggested that the classroom implementation was an improvisational learning experience in that the resources were a catalyst for deductive thinking. Thus, the classroom implementation of the IA project included extra elements, while the IA project itself appeared to be a straightforward set of links to be used in an offload manner.

In terms of barriers, Mr. O. mentioned that a key barrier was not having enough time to fully develop his lesson, as well as not having ready access to the school computer lab.

4.1.3. Case 3, Mr. W.

Mr. W.’s second IA project on the topic of “Physical and Chemical Change” (see Figure 3) can be characterized as an adaptation. It consisted of instructions of what to do with links to various resources, some of which had been previously authored by Mr. W. Unlike the other teachers, Mr. W. also used the IA in combination with other software and online resources. For example, students were asked to complete an online assignment by entering answers in a Google Doc that uploaded it to a school district website. The other online resources were links to examples demonstrating chemical processes and to an online quiz. In this manner, Mr. W. created a design adaptation through the use of previously created instructional material and new online resources.

In his interview, Mr. W. mentioned that he liked to teach with technology. He described his IA project as “student centered” in which online resources were purposefully chosen to enhance student interactivity. We therefore classified the classroom implementation as an improvisation:

“we had IA … (and) a Google doc as well as a downloaded program. They played a game. So I have multiple resources that I can integrate together on the IA … It made it more student centered, which is good. I want them to manipulate more variables and (then) to see demonstration. They … take something interactive, and they could see what the changes do, because they were manipulating and instead of watching me do a demo” (Mr. W. Interview).

In terms of barriers, Mr. W. noted that several resources he found while searching at home were later blocked by school district Internet filters, causing considerable frustration.

4.1.4. Case 4, Mrs. R.

Mrs. R.’s second IA project, on the topic of “Density,” demonstrated design improvisation, in which online resources were combined with an open-ended problem, guided learning sequences, and reflection prompts. In this IA project, students were presented with a real-life problem “building a raft to cross a lake”, and provided with links to resources to help them understand density (see Figure 4). One resource was a game that allowed the user to manipulate properties of a block in order to visualize how it floats.

Mrs. R. created all aspects of her IA project, illustrating the full potential of her design capacity. She wrote the problem story, assembled online resources for student exploration, and incorporated reflection strategies to guide student learning.

Additionally, Mrs. R.’s interview suggested that the classroom implementation of her IA project exhibited improvisation characteristics. It required more student involvement, such as critical thinking and experiential learning:

“for some students—they didn’t like learning that way, but you have all types of learners and a lot of them like learning. It goes through a little story about them hiking and building a raft based on density … It got them thinking about how density is a part of true life. I liked that. I liked that they were thinking about something other than being in a classroom playing with the toys (digital devices) in front of them. They could figure out exactly real life concepts. A few of them complained because they wanted the information instead of having to think about it” (Mrs. R. interview).

In terms of barriers, Mrs. R. noted several technical barriers to fully implement her IA projects. These related to poor network bandwidth and restricted access to computer labs. These limitations, however, did not temper Mrs. R.’s enthusiasm about the IA and online resource use. She also noted that she intended to reuse the IA project and had shared the project with another teacher:

“I will keep this website that I created because students had a lot of fun with it. I can also use it as a review at the end of the year again right before we take the end of level tests” (Mrs. R. Reflection paper 2):

“I shared this website with another teacher in my building and she used it for her students as well. She used it for more of a review, but said that her students had fun with it as well” (Mrs. R. Reflection paper 2).

In general, it appears that the IA and the TTPD provided Mrs. R. with a different methodological perspective, enabling her to more fully use online resources. This allowed her to meet her instructional goals in a more fluid manner than in previous years:

“would it have been different if they had not used technology? Yeah, I’ve never introduced density in this way before. I’ve never used technology to introduce density, which is why I chose density, it’s a difficult concept for 7th graders to learn” (Mrs. R. Interview).

4.2. Research Question 2

What were teachers’ perceptions of impacts on students when learning with online resources?

The purpose of this research question was to examine participants’ perceptions of how their design of activities using online resources impacted student learning and motivation. Case study analyses revealed two themes.

4.2.1. Theme 1: Teachers Perceive That Students Are Engaged and Motivated to Learn with Technology and Online Resources

In their interviews, both Mrs. R. and Mrs. B. noted the motivating power of using online resources comprised of educational games and simulations:

“density is so hard to teach and to show them and to have those visual little games that were included on the project. They were just playing them the whole time. I told them to move on. So I think they liked the idea of playing around and trying to figure out a concept rather than me telling them this is what it is and why you do it” (Mrs. R. Interview).

In her interview, Mrs. R. specifically noted how students liked “playing” with online simulations to learn challenging concepts and engage in discovery. Similarly, Mr. W. stated that learning with technology and online resources helped engage students and support students to be more active learners:

“they didn’t learn different material, they learned how to access different material . they are active participants in their learning with the IA” (Mr. W. Interview).

4.2.2. Theme 2: Teachers Perceive That Students Learn Better with Technology and Online Resources

Mr. W. commented on his students’ ability to learn better when he taught with the IA and online learning resources, especially visualizations:

“this (first IA project) is much more involved and interactive. I think (the students) learned how organisms’ attributes help them survive in the environment. They saw it visually … so, I think it was a lot better for them to be able to take those traits and apply them” (Mr. W. Interview).

Mrs. B. talked about the importance of visualizations for student learning as well: “having students see the position and simulated motion of particles in different types of matter made it easier for them to learn, rather than seeing a diagram in a book” (Mrs. B. Reflection paper 1).

Mrs. B. mentioned that different-ability students could use the materials differently when used in a self-paced manner:

“they (students) could replay it over and over … The honors students only need to see it once. Other students can replay the same video over and over and then it makes the connection. This was another way of presenting the material rather than having the same teacher talking in the same voice, so I think they learned it because it was enjoyable” (Mrs. B. Interview).

Mr. W. also expressed that a student-directed approach allowed content to be covered more quickly as well as enabled students to work in a self-paced manner, thus freeing up the instructor to work one-on-one with his students:

“(I conducted) the informal assessment in the classroom with them, because I’m not directly instructing them, so I can spend more one-on-one time” (Mr. W. Interview).

He also commented on how attentive his students were while using the IA:

“you put them in this setting (IA), and they are focused. I think it helps it be accessible to more students, just because they could learn in different ways” (Mr. W. Interview).

In this way, Mr. W. articulated the efficiency of this instructional approach, in addition to its effectiveness. He was able to add to his repertoire of pedagogical practices by using the IA’s ability to support independent student learning so that he could individually help students.

5. Discussion and Conclusion

This paper presented four case studies of teachers’ experiences designing classroom activities using the IA and online resources after participating in a 3-month long TTPD. Using the lens of curriculum adaption and the notion of teachers’ varying pedagogical design capacity [16], we examined teachers’ second IA project designs and their reports of impacts on students. We noted that two projects were categorized as offload, one demonstrated adaptation, and one project showed elements of improvisation (see Table 4 for a summary). In this way, teachers appeared to use a variety of personally relevant strategies when applying the TTPD concepts to their contexts.

As described above, Mr. W., a science teacher with high self-report technology knowledge scores on the pre-survey, talked enthusiastically about how he integrated his IA projects with other technology he regularly uses, notably Google Docs. In this way, his approach to technology was eclectic, mixing and matching tools to best meet his needs. Conversely, Mrs. B., a low technology knowledge science teacher, recognized her poor technology background and the importance of professional development opportunities in increasing these critical skills.

While Mrs. B. primarily used the IA as a means to collect resources and present these to students, Mr. W. spoke about the value of interactivity and used Google Docs to administer student assessments. Mrs. R., a high technology knowledge science teacher, spoke about the value of online resources in supporting exploration and the ease of sharing online content with colleagues. More generally, the teachers discussed how they tailored their IA project designs to address their students’ needs and interests, and how they incorporated instructional games, activities, and interactive resources to enhance motivation and provide self-paced instruction.

In terms of student learning, some participants strongly stated their belief that students prefer to learn with technology, are adept at it, and that using technology could make learning more fun and motivating. This preference was not seen in a negative way, but rather as an increasingly critical factor to consider when designing classroom activities. Participants also expressed, albeit less frequently, a related belief that using technology can help improve student learning. These participants described the way interactive simulations allow students of different abilities to “play with” and “see” difficult concepts such as density, motion, and heredity. It also allowed for differentiated learning.

Despite great strides in recent decades in computing access in US schools, all teachers identified several barriers due to technology infrastructure in their schools (see Table 4). Barriers mentioned included school district Internet filters, limited access to computer labs, and slow download times. In terms of enablers, three participants also liked the simplicity of the IA, perceiving it as a time saver, valuable for collecting and organizing online resources, and easy to combine with other tools they already knew about for instructional purposes.

In addition, differences were seen between teachers’ IA project designs and the resulting classroom implementations. For example, while some teachers appeared to value inquiry learning as an instructional strategy, their IA projects differed significantly in presentation. On the surface, Mr. O.’s second IA project appeared to use direct instruction in that it presented a series of links to online resources. However, in the interview, Mr. O. noted that he wanted students to “discover” rules about scientific notation by having them interact with examples. This finding underscores the importance of not assuming that the designed artifacts reflect subsequent classroom implementations.

Limitations of this study include that findings are descriptive and suggestive due to the case study research design. The nature of case study design also leads to generalization issues within research studies [23]. In addition, researchers might be biased due to their role as TTPD designers. However, multiple data sources were triangulated in this study and the research findings resonate with previous findings, suggesting the trustworthiness of the interpretations. In terms of practical implications for teacher professional development providers, our findings support the view that teachers need explicit support in order to design productively and enhance their pedagogical design capacity, and that this capability should not be assumed.

Acknowledgments

This paper is based upon work supported by the National Science Foundation (0937360). Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors thank the participating teachers in the study, the district science coordinator, Victor Lee, Hui Qiao, and members of the research group.