Case Study Brief: Promising Practices to Build Evaluator Capacity

Abstract

In 2012, the Massachusetts Department of Elementary and Secondary Education contracted with SRI International and its research partners, Abt Associates, Nancy Brigham Associates, and J Koppich and Associates, to conduct an independent study of the implementation of Massachusetts’ Educator Evaluation Framework. During the 2014-15 school year, the research team administered a statewide survey of principals and school staff and conducted educator interviews and focus groups in seven case study districts. In the case study districts, the team focused on exploring promising practices related to use of evaluation data for human resources decisions, implementation of district-determined measures (DDMs) of student learning, and capacity of evaluators to conduct fair and thorough evaluations. This brief is the third in a three-part series dedicated to sharing these promising practices with other districts in Massachusetts.

The case study brief highlights three districts’ (Revere, West Springfield, and Northbridge Public Schools) efforts to calibrate evaluators’ feedback and ratings for school staff and to reduce evaluators’ workloads associated with the new evaluation system. These districts have implemented promising strategies to promote evaluator consistency within and across schools and to relieve burden on evaluators, including analyzing anonymized feedback and teacher videos and revisiting district guidelines on observations, though these areas remain a challenge. Findings from these case studies may be useful for other districts:

1. Districts provided varied and ongoing opportunities for evaluators across schools to become consistent in their understanding of the performance rating levels and in their manner of providing feedback. They supplemented these efforts with formal and informal collaboration among evaluators at the same schools. Implementing this holistic approach helped ensure that all educators receive clear communications about evaluator expectations for feedback and timing of observations and ultimately heightens educators’ perceptions of the system’s fairness overall.
2. Districts used technology in a variety of ways to increase consistency across evaluators in the feedback and ratings that they provided to educators.
3. District administrators served as additional evaluators, and the district adjusted guidelines for the quantity of observations and evidence—strategies that helped to ease evaluator burden.