Evaluating Transformative Research Programmes: A Case Study of the NSF Small Grants for Exploratory Research Programme

Abstract

This article describes an evaluation of the National Science Foundation’s Small Grants for Exploratory Research (SGER) programme conducted for NSF by SRI International (a non-profit research company). SGER was a 16-year programme sponsored by NSF and operating across the agency from 1990 until 2006 to encourage programme directors to invest in high-risk, high-reward research that might not pass the traditional peer review process. This article provides a detailed background of SGER; a description of the outcomes of the programme; details about the methodology used to evaluate the SGER programme; and the findings of the evaluation. The analysis shows that SGER was highly successful in supporting research projects that produced transformative results as measured by citations and as reported through expert interviews and a survey. However, the NSF programme directors as a whole underutilized the tool for most of the years it was in operation spending far less than the allowable funds allocated to exploratory research; this suggests that internal actions to take risks may not have been rewarded. Moreover, the programme itself was successful beyond expectations. A high-risk programme would be expected to have transformative results in just a few cases. SGER had transformative research results tied to more than 10% of projects. This suggests that programme managers remained risk averse and continued to support projects that were likely to produce positive outcomes.