Evaluation of iCoachTEAM: Final Report

Abstract

Nationally, students’ math achievement has continued a downward trend since the COVID-19 pandemic. Digital Promise sought to address this challenge through Instructional Coaching for Tech-Enhanced Approaches to Mathematics (iCoachTEAM), a program that trained coaches to support middle school math teachers to use technology to promote student conceptual understanding and conduct formative assessment. Digital Promise received an Education Innovation and Research (EIR) Early-phase grant to pilot and test the efficacy of iCoachTEAM at improving student math achievement, with the 5-year grant period beginning in 2021 and extending through 2025. It partnered with SRI to examine the implementation and impact of the program.

This technical report describes a quasi-experimental study of iCoachTEAM, implemented in two pilot districts and four implementation districts. SRI matched schools within iCoachTEAM districts (treatment) to other schools in the same state implementing business-as-usual math coaching practices (comparison). The study compared math achievement on state standardized tests in grades 6–8 in the treatment and comparison schools after two years of iCoachTEAM implementation. SRI measured teacher practice outcomes using an observation tool developed by Digital Promise to capture technology use and the Danielson Framework for Teaching.

Despite medium to high implementation fidelity of the intervention, the study found no effects of iCoachTEAM on student or teacher outcomes. Under an intent-to-treat design, the study included all teachers in a study school, many of whom did not receive coaching, resulting in limited contrast between the treatment and comparison conditions. Moreover, the observation sample was too small to detect significant differences. However, the implementation study uncovered several implications for supporting teachers to move beyond the use of technology for procedural tasks and toward approaches that build conceptual understanding.