Teaching quality publications

It is hard to find many teachers or principals who believe that traditional teacher evaluation systems are of much value. Typically, they neither identify struggling teachers nor contribute to improving practice for any teachers. While there may be some exceptions, most teacher evaluation systems are simply a series of sporadic events designed to gauge practice in the moment. On-going professional growth and improvement rarely enter into the equation. This report draws lessons from three California school districts and their teachers’ unions that have charted a different course and determined that the purpose of evaluation should be to improve teaching in order to advance student learning.
California policy on teacher evaluation has not changed since 1971 despite some legislative appetite for change. In anticipation of future legislative efforts to revise California’s teacher evaluation system, the Stuart Foundation made a modest grant to J. Koppich & Associates and SRI International to document the work of the San Juan, Poway, and San José school districts as they developed and implemented professional growth systems to replace their old teacher evaluation systems.
In the process, we learned a few basic lessons. First, there is no one perfect teacher evaluation system. Policy makers seeking a uniform system that will improve teaching and learning in all local contexts are certain to be disappointed. Second, labor-management collaboration is essential to developing new and innovative teacher evaluation systems. Third, “evaluation” is a loaded word. Educators (both teachers and principals) view evaluation as a tool to remove ineffective teachers, but not to improve all teachers’ professional practice. Fourth, successful implementation does not happen all at once. The work of developing a new system is bound to encounter problems that will test teachers’ and administrators’ dedication and commitment to a set of values. Re-invention takes time as old attitudes, limited expertise, and other simultaneous initiatives make implementing a new teacher evaluation system a challenge. Beginning with small scale experimentation and successive refinements, rather than instituting a totally new system throughout the entire school district, is likely to have a greater chance of realizing meaningful improvement.
The San Juan, Poway, and San José school districts and their teachers’ unions have, each in their own way, demonstrated new thinking about how best to re-invent teacher evaluation. (At the end of this paper, we briefly describe each of the new systems). Each variation on a professional improvement system provides important lessons for state policy makers and districts across California. But, as they will tell you, there is no quick fix. These new approaches are very much works-in-progress and likely will continue to face implementation challenges as they mature. However, the examples from these three districts clearly point to the wisdom of allowing local districts and unions to tackle the difficult work of improving teaching quality through new forms of evaluation. While this hard work needs to be done at the local level, the state has an important role to play in providing guidance by developing an operating framework that emphasizes the goal of the system as professional growth and improvement for all.

SRI Education recruited three education policy experts to contribute concept papers that explore the research and policy implications of the STEM indicators system. Together, these papers can guide policymakers, researchers, and educators through the potential implications and implementation of the STEM education indicator system. In this paper, Measuring the Quantity & Quality of the K-12 STEM Teacher Pipeline, Suzanne Wilson of University of Connecticut discusses how traditional measures of teacher preparation are only weakly related to student outcomes. Teacher preparation affects student achievement, as well as teachers’ evaluations and persistence in teaching. Wilson highlights the NRC’s enhanced indicators of teacher preparation: science and mathematics content knowledge for teaching, and participation in STEM-specific professional development activities. The paper reviews state efforts and other projects designed to collect data on these indicators. An executive summary of the paper is also available.
For more on the STEM Education indicators, refer to the Developing a K-12 STEM Education Indicator System page.

This research brief examines Springfield Public Schools’ Effective Educator Coach (EEC) program. The EEC program is a comprehensive mentoring program in which experienced teachers support new or struggling teachers. The EEC program was studied as part of a larger evaluation of a Massachusetts’ Teacher Incentive Fund (TIF) grant implemented in the Springfield Public Schools (SPS); SRI International and Abt Associates conducted the study on behalf of the Massachusetts Department of Elementary and Secondary Education.
The brief describes the following key implementation findings for the EEC program:

The autonomy to determine content, structure, and timing of coaching activities was important to both EECs and mentees.
Expanding the program to serve all schools districtwide meant changes to the program’s structure, which in turn led to challenges with coaches’ capacity.
Principal involvement in the program varied across schools, from schools whose principals’ roles were primarily to identify potential EECs to schools whose principals were actively engaged in EEC-related activities and decisions throughout the school year.

The brief then presents key considerations for districts interested in learning about a coaching program.

In 2012, the Massachusetts Department of Elementary and Secondary Education contracted with SRI International and its research partners, Abt Associates, Nancy Brigham Associates, and J Koppich and Associates, to conduct an independent study of the implementation of Massachusetts’ Educator Evaluation Framework. During the 2014–15 school year, the research team administered a statewide survey of principals and school staff and conducted educator interviews and focus groups in seven Massachusetts case study districts.
This research brief summarizes the following key findings from the independent evaluation:

Educators were divided about whether the EEF is focused on professional growth and development. Educators’ experiences with and perceptions of the Educator Evaluation Framework appeared strongly linked to districts’ communication efforts about its purpose and uses.
Educators had mixed views of whether individual components of the evaluation improved instruction. Staff and principals valued the components of the Educator Evaluation Framework that provided opportunities to collaborate and reflect on instructional practices, such as goal-setting, observations, and feedback.
Districts have made significant progress toward developing and implementing district-determined measures (DDMs) of student learning, though educators had mixed perceptions about their utility and fairness. Interviewed staff and principals were more likely to report positive perceptions of the implementation of DDMs when they had been involved in the identification and development process and when their districts had emphasized the purpose and uses of the DDMs for informing practice.
Surveyed principals and staff were confident that evaluators had the appropriate knowledge and skills to effectively conduct evaluations. Nevertheless, evaluators indicated that they struggled to balance their responsibilities and ensure consistency. Perceived inconsistencies have led educators to question the system’s fairness.

In 2012, the Massachusetts Department of Elementary and Secondary Education contracted with SRI International and its research partners, Abt Associates, Nancy Brigham Associates, and J Koppich and Associates, to conduct an independent study of the implementation of Massachusetts’ Educator Evaluation Framework. During the 2014-15 school year, the research team administered a statewide survey of principals and school staff and conducted educator interviews and focus groups in seven case study districts. In the case study districts, the team focused on exploring promising practices related to use of evaluation data for human resources decisions, implementation of district-determined measures of student learning, and capacity of evaluators to conduct fair and thorough evaluations. This brief is the first in a three-part series dedicated to sharing these promising practices with other districts in Massachusetts.
This case study brief focuses on Shrewsbury Public Schools’ use of evaluation data to inform human resources practices and policies. In 2014–15, the team found that administrators are using data from the new evaluation system to better inform decisions about renewal for beginning or struggling teachers. Moreover, both new and experienced educators have leveraged evaluation data to inform instructional improvement.
Among Shrewsbury’s accomplishments are several practices that may be instructive for other districts:

A large team of district and school leaders developed a robust messaging and communications plan that eased the district’s transition into using evaluation data for human resource decision-making.
The district incorporated evaluation data into preexisting structures to streamline and strengthen key human resource decisions.
The district leveraged teacher leadership by identifying and training teacher leaders at each school site to facilitate questions about the evaluation system.
District and school leaders emphasized the positive features of the evaluation framework, focusing on its value for data-driven instructional improvement.

The Rio Grande Valley Center for Teaching and Leading Excellence (the Center) was a partnership between IDEA Public Schools, a charter management organization, and Pharr-San Juan-Alamo Independent School District (PSJA). The Center aimed to strengthen support for new teachers, teacher leaders, and school leaders and to raise the quality of teaching for students in one of the poorest regions in the country.
SRI conducted the independent evaluation of this i3 development grant. The evaluation tracked implementation fidelity of new teacher training and teacher leader training over the 3 years of implementation under the grant. The evaluation also studied the impact of new teacher training on student achievement in core subjects using a randomized controlled trial for students in grades 4 through 8 in PSJA. Teacher outcomes included instructional efficacy, job satisfaction, and retention. Impact of the teacher leader training on leadership-related efficacy measures, job satisfaction, and retention derived from randomized controlled trials of teachers meeting the teacher leader selection criteria in each district. The evaluation included three cohorts of participants from each district for both the new teacher training and teacher leader training.
New teacher training participants in PSJA ISD reported slightly higher job satisfaction ratings than comparison teachers. No statistically significant differences were found between the students of new teacher training participants in PSJA and those of comparison teachers. Because the comparison teachers were more experienced on average, similar performance among students of new teachers and experienced teachers suggests that the novices received enough supports to promote student learning in their first year in the classroom.

The purpose of this study was to examine the impact of the Florida Master Teacher Initiative (FMTI)–an i3-funded early learning program aimed at improving the quality of teaching and student outcomes in grades PreK through third grade in high need schools. The FMTI schools participated in four program components: (1) a job-embedded graduate degree program with an early childhood specialization, (2) a Teacher Fellows program through which teachers engage in yearlong inquiry projects around their practice, (3) a Principal Fellows program during which principals work together to strengthen their facilitative leadership skills, and (4) Summer Leadership Institutes to review data and engage in action planning. The impact evaluation had two primary goals: (1) to assess the school-level impact of FMTI on teachers and students; and (2) to assess the impact of FMTI on teachers enrolled in the job-embedded early childhood graduate degree program and their students. To achieve the first goal, the evaluation used a cluster random assignment design, in which 40 Miami-Dade County Title I public elementary schools were randomly assigned to the FMTI program or a status-quo control condition. To achieve the second goal, the evaluation used an embedded quasi-experimental design using propensity score matching and difference-in-differences approaches. SRI International administered schoolwide surveys at baseline and in the final year of the grant in both intervention and control schools; conducted classroom observations of job-embedded graduate program teachers and a matched comparison group using the Classroom Assessment Scoring System (CLASS) early in the teachers’ first year of the graduate program and a follow-up observation occurred after or near the end of the program, with program teachers and sample of comparison teachers; and gathered student reading and math achievement data on children in kindergarten through fifth grade who were at the 40 study schools at the time of random assignment for a total of more than 10,000 students in the FMTI schools and a similar number of students in the control schools. The study did not find school-level impacts on student achievement or on the majority of outcomes measured through the teacher survey. Analysis of the impact of the job-embedded graduate degree program found a positive difference of 1.7 points for participating teachers in the instructional quality domain of the CLASS compared to matched comparison teachers. The evaluation also found positive and statistically significant results for the graduate program teachers compared to comparison teachers on the teacher survey in the areas of engagement in leadership activities, engagement in governance activities, engagement in outreach activities, self-reported early childhood knowledge, and self-reported general instructional knowledge. No significant differences in math or reading achievement were found for students of the graduate program teachers compared to students of a matched sample of teachers in control schools. The implementation of the FMTI program was not sufficiently robust to definitively determine its effectiveness. FMTI treatment schools that achieved medium or high fidelity of implementation across the three years experienced more positive outcomes. The evaluation has illuminated lessons about how to effectively provide job-embedded professional development to support teacher quality improvement.