Citation
Smerdon, B., Garet, M., O’Day, J., Miller, L., Means, B., & Mitchell, K. (2003, April). Charting a course: Evaluation design of the National School District and Network Grants Program. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
Introduction
Over the past decade, a number of reform efforts have aimed at reducing the size of the learning communities in the nationās schools. Most recently, the Bill & Melinda Gates Foundation has committed more than $375 million to improving Americaās high schools through its National School District and Network Grants Program. Likely to be one of āthe most publicly scrutinized educational initiativesā in recent history, the foundationās program is providing a catalyst to educators and policy makers to radically reshape secondary education in the United States. The foundation is working with school districts and network organizations to create education systems that work for all students, including those students who have been characterized in the past as most underserved by the nationās education system. At the secondary level, the foundation is supporting small schools as a corrective to the large, impersonal āshopping mallā high schools that dominate today. To this end, the foundation supports: 1) districts and network organizations in converting large high schools into smaller learning communities, and 2) network organizationsā efforts to replicate successful model schools or to implement small, effective schools from the ground up.
Through a competitive award process, the foundation selected our organizationsāthe American Institutes for Research (AIR) and SRI Internationalāto evaluate its initiative. We are currently beginning our third year of data collection. The purpose of this paper is to describe the framework underlying our evaluation design of a large-scale, multifaceted school reform initiative and to identify some of the methodological challenges inherent in work of this kind. Throughout this paper, we use travel as a metaphor for our evaluation design process. Specifically, we begin by describing the earliest stages of our journeyāour starting point. Here we outline the foundationās theory of changeātheir goals and vision for the initiative. We then turn to a discussion of defining the destinationāthe research questions we seek to answerāand finding the coordinatesāthe conceptual framework that will guide our evaluation. Next, we explore the terrain, providing an overview of the grantees and schools in the population, and determine our route (e.g., the design basics of our evaluation). We conclude with a discussion section that we have entitled, Reading the Map While Driving. Here we discuss a few of the challenges of our evaluation and our need for reflection and refinement as the initiative grows and changes and as we learn through our research activities.