Evaluation of the Next Generation Courseware Challenge | SRI International

Toggle Menu
woman looking at a computer in a computer lab

Evaluation of the Next Generation Courseware Challenge

SRI is evaluating NGC projects funded by the Bill & Melinda Gates Foundation to understand the technology features and implementation practices that achieve positive educational impacts at scale.

The Next Generation Courseware Challenge (NGC) is a $20 million grant program of the Bill & Melinda Gates Foundation to develop personalized courseware that improves learning outcomes and post-secondary success, particularly for low-income and disadvantaged students. To meet the ambitious social mission of improving outcomes for these student populations, colleges and universities must improve undergraduate teaching and learning using evidence-driven practice.

The foundation has funded seven organizations developing and disseminating courseware for high-enrollment, lower-division undergraduate courses: Acrobatiq, Cerego, CogBooks, Lumen Learning, Rice University OpenStax, SmartSparrow, and Stanford University.

The foundation is supporting its investment in these NGC projects with research and evaluation geared toward improving and documenting the courseware’s educational impacts at scale. SRI is conducting research and evaluation activities to support the seven NGC projects and the foundation in addressing the key challenge for the NGC initiative: Obtaining consistently positive impacts from courseware use at scale.

The SRI project team is working closely with each of the seven NGC projects to assess its need for research and evaluation technical assistance, document its plans for implementation and demonstration of effectiveness, and provide coaching on how to capture valid, reliable data on its courseware’s effectiveness and scale of implementation. In addition, the SRI project team will collect independent data from a sample of participating institutions and perform its own analysis of the student-level data generated by the seven NGC grantees’ impact studies.

Key Evaluation Questions

For each of the seven NGC projects, the SRI evaluation team will collect data to answer four questions:

  1. How many low income and non-low income students are experiencing the courseware products, and in what contexts? 
  2. Are the new courseware products working as intended? What are barriers and facilitators for adoption?
  3. How does students’ performance with the courseware vary in different settings, and how does it compare to business-as-usual?
  4. How does use of the courseware impact student and institutional costs, and what factors are associated with cost-effective implementation?

A synthesis of data across all seven projects will address three broader issues around the role of digital learning systems in postsecondary education:  

  • What courseware design features are associated with more consistently positive blended learning student outcomes? To what extent do these promising features vary for different kinds of learning goals, subject areas, students, and contexts?
  • What blended learning implementation practices are associated with consistently positive student outcomes? Can these be provided at low enough cost to make high-quality courseware implementations cost effective?
  • What factors affect the scalability of Next-Generation courseware? Can wide scale be achieved without compromising effectiveness?

Related News

Gates Foundation Announces Finalists for $20 Million in Digital Courseware Investments (press release)