Roschelle, J., Murphy, R., Feng, M., & Bakia, M. (2017). How big is that? Reporting the Effect Size and Cost of ASSISTments in the Maine Homework Efficacy Study. Menlo Park, CA: SRI International.
Abstract
In a rigorous evaluation of ASSISTments as an online homework support conducted in the state of Maine, SRI International reported that “the intervention significantly increased student scores on an end-of-the-year standardized mathematics assessment as compared with a control group that continued with existing homework practices.” (Roschelle, Feng, Murphy & Mason, 2016). We reported an effect size of g = 0.18 of a standard deviation ( t (20) = 2.992, p = 0.007) based on a two-level hierarchical linear model (Roschelle et al., 2016). Naturally, education stakeholders want to know how big the improvement was practically. In this technical report, we present alternatives for explaining the effect size, building on the guidance of Lipsey et al. (2012), a leading researcher who developed broad recommendations for effect size reporting.