Abstract
A partnership including 11 school districts, a university, service agency, and private nonprofit education organization formed a collaborative partnership to improve teaching and learning in elementary school science and mathematics. The partnership designed research-based professional development for 150 teachers of grades 3–5. The professional development resulted in statistically significant increases for those elementary school teachers on math and science competency tests over a two-year period. The professional development was the vehicle for providing teachers with professional development so that they could (a) increase their content background in science and mathematics and (b) apply newly learned inquiry practices in their math and science instruction.
1. Introduction
The design of professional development in elementary school science and mathematics is a key to its effectiveness. Researchers [1–4] identified factors that contribute to professional development that is more likely to improve teacher practice and, therefore, more likely to improve student achievement. Loucks-Horsley et al. [3] and Quick et al. [5] determined that teachers need (a) specific instruction on both theory and on strategies, (b) to see examples of the use of those strategies, and (c) to practice those strategies themselves.
Professional development should also address the broad needs of the content areas being served and content needs have been well documented [6–8]. Alberts [9] argued that, in too many classrooms, the teaching of science has become an exercise in knowing specific vocabulary at the expense of seeking understanding of the underlying scientific principles. He stated, “The broad goal for science education must be to provide students with the skills of problem solving, communication, and general thinking required to be effective workers and educated citizens in the 21st century” (page 80). Science professional development should, therefore, move participants toward planning and conducting lessons that provide those kinds of experiences for students. Bybee and Van Scotter [7] discussed the rigor, focus, and coherence necessary for science education to be effective, which supports the need for teachers to be strongly grounded in both content and pedagogy. Lewthwaite [8] noted the complexity of providing science education that is related to the “knowledge base, beliefs, and attitudes of teachers” (page 181). In providing professional development, there is an inherent need to address each of those areas.
Specific research related to mathematics professional development has supported the improvement of teacher content knowledge and work with strategies as effective practice. Bell et al. [10] conducted professional development with kindergarten through middle grades teachers. That professional development, titled Developing Mathematical Ideas, was designed to help teachers learn mathematics content and to improve their abilities to work with students’ thinking processes. Teachers who participated in the professional development scored significantly higher on content-related items than teachers who did not participate. That project used as its content assessment measure the tests designed by the Learning Mathematics for Teaching (LMT) project, and that assessment was chosen for assessment of the project reported in this paper.
For the professional development described here, the focus on content knowledge with accompanying pedagogy is supported by the literature. Seidel and Shavelson [11] defined pedagogical content knowledge as “an in-depth understanding of the content and the nature of the domain that is pedagogically useful” (page 459). Content knowledge was necessary, but not sufficient, criterion for pedagogical content knowledge; that is, teachers’ use of pedagogical strategies and processes appropriate to that content [12–14]. Current research demonstrates that content knowledge and pedagogical knowledge are interdependent.
Teacher professional development can be designed to successfully address the needs of school systems, their teachers, and the students in those systems. The professional development that is the focus of this report was carefully crafted to avoid the flaws of ineffective programs and to clearly focus on the power of effective characteristics. This paper describes the implementation of that program and gives results of measures of teacher content knowledge. The theoretical framework and research basis for this professional development are provided and specific elements of the design are described. The purpose of this research was to gather data to answer the following question: what were the results and impacts of the professional development?
2. Theoretical Framework
The framework for this program design was supported by a well-established and successful teacher professional development project titled, Learning through Inquiry Science and Technology (LIST) [15, 16]. LIST was a series of projects, initially supported through Title II Eisenhower funds (later called Teacher Quality Grants), which was designed to pilot research-based professional development activities for middle school and high school science teachers. Lessons learned from developing, implementing, and evaluating LIST programs provided the foundation for the project reported in this paper.
LIST was a professional development program initially designed for science teachers. It was begun in 1998 as an effort to improve elementary school students’ science achievement, because science standardized test scores in the region served by the university partner in this project were the lowest content scores for most area schools. Each LIST program involved 20–25 teachers from one district. This focused approach to professional development achieved immediate systemic change in the way science was taught in the participating schools [15, 16]. The primary objective of LIST programs was to transform the science teaching culture within a school and/or district from conventional, teacher-centered programs to an inquiry design, placing students at the center of their own learning.
The inquiry design for LIST and for the professional development in this project was the learning cycle: exploration, explanation, and expansion [17]. Learning cycles are a curriculum design and teaching practice with a long history of success in science education [18]. Learning cycles begin with an exploration phase designed to guide students to collect good data, which is then used in the next phase to develop understanding of concepts. The exploration phase is student-centered, materials-rich and allows students to begin to assimilate the concept. The explanation phase follows the exploration, and students are guided in interpreting their data in order to construct and understand concepts. As students are developing concept understanding, terminology is provided. This step is crucial in learning new concepts, and the explanation phase is designed for students to cognitively accommodate the new concept.
Following the explanation phase is the expansion phase, which is designed to provide additional activities. During expansion, students apply their newly learned concepts to other related concepts and to real world applications. In other words, students organize concepts in relation to what they already know from past learning cycles and from their daily lives. The expansion phase of the learning cycle may include, but is not limited to, additional activities, experiments, and/or readings. From the teachers’ perspectives, the learning cycle is an instructional procedure that is designed to allow for a variety of teaching methods, for example, cooperative learning, demonstrations, class discussions, student presentations, and field trips [18].
The most significant finding from five LIST projects was that nearly 100% of the participating science teachers continued to use the learning cycle teaching procedure in their science instruction [15]. Student results were equally dramatic. For example, representative results from a participating district included increases in science pass rate on the state’s high school graduation test from 63% to 70%. This is an increase in average scores from 508 to 530, highest in school history, on the physical science portion of the graduation test one year after implementing the project [16]. Another participating district recorded significant increases on the state’s high school graduation test science pass rates from 40% to 60% within 2 years of beginning their project [16].
3. Elements of the Design
3.1. National Standards
Complementing the theoretical foundation of this project was congruence with the National Science Education Standards [19] and the National Council of Teachers of Mathematics Principles and Standards [20], which included placing students at the center of their learning experiences, encouraging them to engage in explorations, form new understandings, and relate those understandings to other concepts. Other important dimensions, incorporated into the professional development and supported by research on effective professional development, included (a) extensive use of technology in the learning cycle sessions, (b) use of experienced practicing teachers to model inquiry-based science and mathematics lessons, (c) establishment of a project management team, and (d) extensive followup and teacher support activities [2, 3].
3.2. Partnership and Participants
A special partnership was the vehicle for providing teachers with professional development so that they could increase their content background in science and mathematics and apply newly learned inquiry practices in their math and science instruction. The partnership included all of the elementary schools in the 11 districts (a total of 35 schools), a mid-sized state university, regional educational service agency, and a nonprofit educational organization from the private sector, all located in a southeastern state. A total of 150 third, fourth, and fifth grade teachers from the 11 school systems engaged in intensive content and pedagogy workshops based upon the state’s standards. Only 4 of the 11 systems were previously meeting state and national Annual Yearly Progress requirements.
The regional university brought to the partnership project directors for mathematics and science from the College of Education, engaged some Arts and Sciences faculty for workshop content sessions, secured master teachers to provide other workshop instruction and support, and served as a liaison to the private nonprofit educational agency that partnered to provide additional master teachers. In addition, the university provided planning and coordination of all activities, materials, faculty, and facilities. The state’s regional service agency, which was designated by the state to serve the 11 school districts involved in the study, was the fiscal agent for the project. Two primary goals were central to the project: (a) to increase the effectiveness of 3rd–5th grade teachers of science and mathematics and (b) to enhance the science and mathematics curricula within each district to reflect the state’s standards.
3.3. Program Schedule
Four intensive, 1-week summer workshops and follow-up activities were designed and delivered by collaborating staff, which included faculty from the university’s College of Arts and Sciences and College of Education, master teachers of science and mathematics from the region, and the educational service agency’s support personnel. Workshops were conducted during the summers of 2007 (1 week in July), 2008 (1 week in June and 1 week in July), and 2009 (1 week in June) and involved inquiry-based instruction via the learning cycle; thus the sessions modeled the content delivery in the way that teachers were to teach in their classrooms.
3.4. Instructional Teams
Dyads led the workshop sessions designed to deliver content and pedagogy. Scientists or mathematicians from the local university were paired with master teachers, often elementary school teachers. Scientists, mathematicians, and master teachers were screened and selected based upon their understanding and experience with learning cycle inquiry. Many of the teachers were graduates of the university’s teacher education program where they began their in-depth studies of and experiences with the learning cycle. Each pair was responsible for planning and delivering workshops aligned with the state’s process and content standards for 3rd–5th grades mathematics and science. Each dyad also prepared materials related to content and pedagogy to help the teacher participants gain in-depth understandings of natural sciences, mathematics, and the nature of science and the methodology of learning cycle science for elementary school students. Again, dyads designed and delivered the workshop sessions in learning cycles; in other words, they conducted the workshops as their teacher participants were ultimately to teach their students.
Lessons learned from research and from previous professional development projects led to both similar and divergent paths in development of this project. The similar paths included using master teachers, scientists or mathematicians, and education faculty in all sessions (high degree of collaboration), delivering content in the same format that the teachers are to teach it (providing exceptional modeling for inquiry lessons), and incorporating technology whenever possible. Additionally, the structure of summer workshops with followup was important. However, the design of these workshops differed from past projects in that the design included a multifaceted approach to the followup to keep participants actively engaged in learning activities and in communication with project staff and colleagues throughout the life of the project.
3.5. Workshop Sessions
The time for teachers in the summer workshops was equally divided between science and mathematics. In each content area, teachers were divided into small groups, of about 15 to 18 teachers, and by grade-level to provide for active participation and access to materials and instructor time. Concurrent sessions were offered to accommodate approximately 150 elementary school teachers during each institute. The workshops took place in an elementary school next to the university campus, and the instructional dyads made sure their classrooms were full of math and science materials and equipment. Central to each session were content, pedagogy, and the state’s standards for math and science [21, 22]. Perhaps the best way to understand the nature of the workshop experiences is to read a description of a representative session.
Professor Roberta and Master Teacher Robert (pseudonyms) began their session titled, “The Rock Cycle” with handouts and graphics of the rock cycle. Professor Roberta spent about 15 minutes reviewing the teachers’ understandings or misunderstandings, of the earth science concept named the rock cycle. Then the master teacher led an exploration into the concept with an activity designed for the teacher participants to make rock types using white, tan, and dark brown chocolate chips (not an uncommon experiment). Teacher participants recorded their observations, which were qualitative data describing the three different “rock types” (chocolates) and the processes to create each rock type. The second phase of the learning cycle, explanation, had the master teacher leading a discussion to interpret the teacher participants’ data and develop the concept named rock cycle from the participants’ data and discussion. The investigation continued through the third phase of the learning cycle, led by the master teacher and assisted by the professor. Teacher participants expanded their understanding of the concept (rock cycle) by applying it to every day experiences and identifying rocks. Throughout the session both instructors interacted with the teacher participants; the master teacher addressed the process of teaching the inquiry lesson and the professor addressed the content in depth. Both instructors commented, throughout the session, on specific state standards as applicable to process and content of this learning cycle.
Math sessions followed a plan similar to the science session delineated above. A mathematics professor and a master teacher served as the instructional dyad. The math content was standards-based, and each teacher attended sessions that focused on the National Council of Teachers of Mathematics (NCTM) content standards for their grade level. Sessions were designed as learning cycle investigations based upon the NCTM process standards.
Examples of content and topics for the workshops in the natural sciences and technology and the general areas addressed in mathematics can be seen in Table 1 of representative session titles.
4. Data Collection
Evaluation instruments measured teacher participant content knowledge, attitudes, and pedagogy (inquiry practices and technology usage). Teacher participants’ content knowledge in each subject area was quantitatively assessed through the use of reliable and valid instruments specifically designed for assessment of professional development programs. Science content knowledge was measured using the Misconceptions-Oriented Science Assessment Resource for Teachers (MOSART) assessment [23]. Creation of the MOSART instruments was funded by the National Science Foundation in order to provide support for assessing science content knowledge of teachers participating in Math Science Partnership grant workshops. The items on the MOSART were field tested and analyzed and provide evidence of conceptual shifts in science content.
The Learning Mathematics for Teachers (LMT) assessment [24] was used to measure content knowledge in mathematics. On the homepage of the LMT Project, designers describe the purpose of the LMT, which is composed of three separate tests in three math content strands, as follows:
Our project investigates the mathematical knowledge needed for teaching, and how such knowledge develops as a result of experience and professional learning. We do so through the writing, piloting, and analysis of problems that reflect real mathematics tasks teachers face in classrooms—for instance, assessing student work, representing numbers and operations, and explaining common mathematical rules or procedures. Assessments composed of these problems are often used to measure the effectiveness of professional development intended to improve teachers' mathematical knowledge. (“What is LMT?”, paragraph 1).
The MOSART and the LMT were chosen because they were designed to measure results of professional development activities in science and mathematics. Pretests of the MOSART and the LMT were administered to teacher participants at the beginning of the partnership’s program, which was summer 2007. Posttests were administered upon completion of the program in summer 2009.
In addition to the content assessments, all participants completed qualitative surveys, which asked teacher participants to evaluate components of the project and to self-report changes in their classrooms and/or their content knowledge acquisition as a result of participation in the partnership’s professional development. Teacher participants were also provided space on the surveys for comments on any aspect of the workshops.
5. Results and Impacts of the Professional Development
5.1. Science Content Results
The Misconceptions-Oriented Science Assessment Resource for Teachers [23] pretest was given during the first day of the first summer workshop and the posttest was given 2 years later. Though many more teachers participated in the workshops, attrition and addition of new participants resulted in an N of 102 teachers who completed both sets of science content tests, and only those scores were included in the statistical analysis. Science tests address concepts of physical science, astronomy, and space science. The mean score of the MOSART pretests in physical science was 12.56, and the mean posttest score was 13.18. A -test was used to compare the means, and the mean posttest score was significantly higher (, ) than the mean pretest score. Teachers who participated in the three summers of science professional development scored significantly higher on physical science concepts after completing the program.
Breaking down the results into gains and losses, 50% of the participants showed gains in physical science from pre- to posttest. Gains ranged from an increase of one or two questions () to increases of 6–8 items (). About one-third () of the 102 participants showed decreases in physical science scores and 16 remained the same.
In the areas of astronomy and space science, the mean score of the pretests was 8.3 and the mean posttest score was 8.8. MOSART pretest (13 items) to posttest (13 items) gains in astronomy and space science knowledge were similar to the collective changes in physical science. Fifty percent of the participants showed gains in astronomy and space science scores, about one-third showed decreases, and 18 remained the same. Gains ranged from one or two points () to increases of 10-11 points (). In the area of astronomy and space science, at least 50% of teachers had test evidence that their knowledge of those concepts had improved.
5.2. Mathematics Content Results
The Learning Mathematics for Teachers assessment [24] was also administered as a pretest during the first day of the first summer workshop, and then again 2 years later to serve as the mathematics posttest. Only those teacher participants (95) who completed both the pretests and posttests of the LMT assessments were included in the summarized data. The mean score of the pretests was 10.48 and mean posttest was 14.24. A -test was calculated and showed significant gains (, ) from pretests to posttests. Teachers who participated in the 2 years of mathematics professional development provided in this project had significant gains on tests of mathematical content knowledge.
Improvements in mathematics results were slightly higher than those for science scores. Ninety percent of the participants showed gains in elementary geometry from pre- to posttest. Increased numbers of correct answers were evident: 49 participants increased their number correctly by one to four questions, and 33 participants answered between five and ten additional items correctly on the posttest. Only 4 of the 95 participants showed decreases in elementary geometry scores and 5% (5) remained the same. Mean score of the geometry pretests was 10.48 and mean posttest was 14.24. A -test was calculated for the geometry assessment and showed that teachers made significant gains (, ) for 52% of the 95 teachers completing the pretests and posttests on geometry.
Approximately 57% of the teacher participants showed gains in the area of patterns, functions, and algebra; 20.6% () showed decreases, and 22.6% of workshop participants () earned the same score as previously. Gains ranged from 1–4 items (45 participants) to increases of 5-6 items correct (10 participants). The mean score of the pretests was 6.29 and the mean posttest score was 7.63.
Considering results from both the mathematics and the science content tests, workshop participants generally gained content knowledge in mathematics and had fewer science misconceptions after the workshops than they did prior to participation.
5.3. Survey Results
Workshop surveys produced qualitative results that indicated strong positive successes of the partnership’s professional development (see Tables 2 and 3). Teacher participants responded to the Math Workshop Evaluation Survey () and to the Science Workshop Evaluation Survey (). Years of teaching experience for the respondents ranged from 1 to 37 years with a median of 11 years and a mode of 3 years. Survey results from the math workshop are presented in Table 2.
Workshop participants were asked to rate each content area in which they received instruction in terms of both the content and the presenters. There were 112 respondents, but not all answered each item. Of those responding, 95% or more rated the content and the presenters for every area as Very Good or Excellent, with the exception of the content of the Algebra segment. For the Algebra area, 93% of respondents rated the content as Very Good or Excellent, though that strand actually had more respondents rating both the content and the presenters as Excellent than any other topic. In mathematics, workshop participants gave high ratings to both the content they were studying and to the university faculty and master teachers who were instructing the workshop sessions.
Comments made by respondents related to the mathematics sessions were gathered from surveys and analyzed for themes and general tone. The most positive aspects of the math workshops, as identified by the participants, were the hands-on activities, attitudes of the instructors, and the resources the teacher participants received. Workshop instructors’ attitudes were described by the participants in all positive terms. Negative comments were relatively minor and, for the most part inconsequential, focusing on lunch, length of the instructional day, and other nonacademic items.
Responses from additional questions on the final survey required self-reporting and some items related to implementation of what was learned during the workshops. Generally (see Table 3), two-thirds of teachers reported positively on items such as perception of increased ability to design inquiry-related instruction, ability to use a variety of instructional strategies, increased knowledge in math and in science, and the use of technology.
One of the greatest challenges mathematics teachers face in their classrooms is the lack of sufficient time and resources for planning and teaching an effective curriculum. Participants were also concerned that they may not be motivating all students or reaching all levels of learners. Some respondents reported that there is a scripted curriculum that they must use, which leaves little opportunity for creativity that could increase the effectiveness of the curriculum.
Results from the science workshop surveys were similar to the math workshop results; teacher participants rated the program as excellent. In describing the science workshop, respondents used powerful positive words (awesome, wonderful, great, best, superb, relevant, informative, applicable, enthusiastic, and fantastic). Furthermore, all of the teacher participants stated that they would recommend this workshop to other teachers. Strengths included the enthusiastic and knowledgeable instructors. Teacher participants were complimentary of the hands-on activities, materials, and instruction. The resources were described as awesome, fantastic, and free!
5.4. Open-Ended Questions
Four questions were posed to the teacher participants on the final day of the final workshop, which was concluded during the third summer, or two years after the professional development began. Their anonymous responses describe well how the partnership made a difference in elementary school science and mathematics. Over 120 teacher participants completed the questionnaire. A summary of their comments (italicized) follows each question.
(1) What is the single greatest, most significant thing you take from the math-science program of the past two years? Please address your top choice only.
Over three quarters of the teacher participants (77%) listed one or more of these phrases in their answers: make-and-take activities, hands-on activities, and/or inquiry activities that match the state standards. The remaining responses were spread evenly among content, friendly presenters, technology, confidence, or passion gained in the workshops.
(2) After question one is answered completely, answer this item. Now describe another significant outcome from the math-science program of the past two years.
Four categories garnered most of the responses. The largest percent (25) listed the hands-on activities; apparently 2% felt so strongly about the activities that they listed it for item 1 and 2 (77% and 25%, resp.). Twenty four percent listed relationships with other teachers as another significant outcome, 15% described the technology, and 12% listed resources. The remaining 24% of the responses were spread among content knowledge, instructional strategies, presenters, and self-confidence.
(3) What weakness or shortcoming could you identify about the math-science program of the past two years?
Two-thirds of the responses were spread among a variety of items while 34% listed not enough time as a shortcoming of the program. Sixteen different categories emerged from other “teacher participants” (66%). Key categories included such things as commuting distance, lunch issues, no internet for teacher participants, some of the presenters, some of the activities, crowded media center, lack of advertisement, or too much material.
(4) How do you incorporate the information/experiences from the MSP program into your classroom? Please elaborate.
Ninety-three teacher participants (76%) described how they used the ideas, materials, and hands-on activities in their classrooms. Eleven teachers (9%) listed technology usage and 7% listed sharing with other teachers. The remaining 8% were spread evenly among 4 categories: school visits by university faculty, cross-curricular information, better able to explain curriculum and more student-centered teaching.
Most of the teacher participants indicated that they would incorporate the lessons, experiments, and activities from the professional development into their curriculum. Some would use them as is, while others would modify the lessons. There were differing levels of commitment to inquiry learning but after the workshop, many would change their programs to include hands-on science learned at the workshop and others would incorporate the technology in their classrooms.
Survey and qualitative data provided evidence that teacher participants were positive about the workshops, that the content of the workshops was strong and useful, that the workshop leaders modeled the strategies of effective teaching, and that teacher participants had improved their own content knowledge. Teachers expressed a commitment to applying what they had learned in their classrooms and schools.
6. Conclusions and Discussion
Based on evidence from the content tests in science (MOSART) and in mathematics (LMT), grades 3–5 teachers, who participated in 4 weeks of professional development over three summers, provided evidence that elementary teachers who participate in professional development can significantly improve their content knowledge in both science and mathematics. These findings agree with Bell et al. [10], who found that carefully designed professional development did improve teachers’ content knowledge, and with Bybee and Van Scotter [7] who implemented a higher level science curriculum and found that preparing teachers for implementing that curriculum required improving their content knowledge.
The significant improvement in content knowledge may have been directly linked to the design of the professional development, which was based on the findings of Loucks-Horsley et al. [3] and Garet et al. [2]. The workshops were grounded in theory, addressed effective strategies, were taught utilizing best practices in the content areas, and provided for active participation by teachers in the workshops.
The problem identified in this report was how to design professional development that would significantly impact the participants, and thereby impact achievement of those students taught by workshop participants. Through use of design elements that have been shown to improve teacher effectiveness, this professional development project did significantly impact the content knowledge of the participants, and exposure over two calendar years of workshops provided extensive experience with effective strategies.
Systemic change is difficult at best, but the involvement of a large number of the teachers in grades 3–5 in the 11-system area served by this project makes it possible to impact the curriculum and the practice of many elementary schools. In cases where a smaller percentage of the teachers from any given school attended the workshops, it is possible that the influence of the professional development was not as pervasive as in some other cases where a majority of teachers at a given grade level attended the workshops. However, it is also possible that networks of teachers, created at a workshop that crossed the boundary lines of systems in the area, may have developed and may continue to provide, a system of professional support.
The design of this professional development project could provide a model for other such projects by encouraging providers of professional development to plan for strong instruction, both in content and pedagogy, active participation by workshop attendees, a partnership with several groups of stakeholders, and workshop leaders who have a strong track record in professional development.
It is clear that the findings of this research are limited by a lack of data specifically linking student achievement to the teacher participants and the workshop. It is also acknowledged that survey results rely on self-reporting, which may or not may not give an accurate measure of the true behavior of an individual. However, results are positive enough to suggest that those providing professional development for mathematics and science teachers in grades 3–5 should use the findings to guide preparation of their professional development programs.
The limitations of this study naturally lead to opportunities for further research. The impact of No Child Left Behind legislation has been to produce data on all levels, including state, system, school, and teacher levels. Teacher-linked data could be used to determine whether teachers who attend professional development, designed with the characteristics described in this paper, have students with greater achievement than the students of teachers who do not attend the professional development. Research will always be needed to investigate other elements that could significantly increase positive impacts of professional development programs designed using this model. Finally, because the participants in this study were limited to teachers in 11 counties in a southern state, additional research with a wider sample could yield more generalizable results.
Acknowledgments
The Math Science Project (GA070647) was funded through the US Department of Education and administered through the State Department of Education. The views and opinions of authors expressed in this paper do not necessarily reflect those of the state or the U.S. Government.