Center for Science, Technology, and Economic Development (CSTED) > Selected Reports

The Role of NSF's Support of Engineering in Enabling Technological Innovation

V. CONCLUDING OBSERVATIONS

OBSERVATIONS ON THE RESULTS

We introduced four foci of analysis as a provisional set of categories to use in examining each case and, in the future, in conducting comparative, cross-case analyses:

• Industry, government, and university roles and relationships.
• Relationships between fundamental research and technology development.
• Intellectual property rights.
• The NSF role.

These have been useful, at least on a case-by-case basis, but the diversity of the first three cases and the relative weakness of different kinds of data in each case (see below) mean that cross-case analysis at this stage is premature. Still, reading across the three cases in light of these categories, one is struck by the tremendous variations in institutional relationships and the relative timing and significance of different elements that shape innovation. A second feature of the cases is the apparent influence that interinstitutional networking had on the evolution of these innovations. In some instances, this influence was manifested historically in the career paths of key contributors to the innovation; in others, knowledge and technology crossed multiple institutional boundaries as the innovation evolved. Finally, NSF's role varied widely across the three cases: in RIM, direct research support appeared to be the primary influence; in MRI, it was through support of research on NMR spectroscopy, the technical and analytical core of the innovation, and education of graduate students who built on their NMR experiences in subsequent MRI work; and in the Internet, it was organizational leadership. Other modes of support were important, yet often subtle or indirect, so that assigning relative influence to each mode would be difficult. The following tables present a possible structure in which such support might be analyzed and described after more cases have been completed. With more experience studying the results of additional cases, it may be possible to complete the tables with ratings of the relative importance of each mode of support in each of the first three cases.

OBSERVATIONS ABOUT METHODOLOGY

The cases were more diverse than we had anticipated. As a consequence, the kinds of data that proved most useful and reliable varied widely. This variety necessitated very different approaches to the three cases, and sometimes required shifts in strategy later in the study process than was desirable. Lessons were learned about approaches to future cases, and this concluding section is devoted to describing them.

We expected that defining the boundaries of each innovation would be problematic, and it was. But it was problematic in some ways we did not anticipate. The Internet, diffuse and global, was supposed to be difficult to define and bound. Advanced polymer matrix composites was a narrow and specific topic; it was supposed to be easy to define. In each case, the opposite was true.

Once we began to explore the literature on the Internet, it quickly became clear that the major contributions were well documented, the contributors were well known, and their relative importance noncontroversial. Further, the technologies that "defined" the Internet-the unique, intrinsic elements, in contrast to supporting technologies-were unambiguous and few in number. A similar situation existed in the MRI case. In both the Internet and MRI cases, the significance of the innovations was such that an extensive historical literature already existed, a literature that described the basic chronology and identified major contributors. Thus a framework for our specific inquiry already existed. This was not the case for RIM and, we suspect, would not have been the case if we had studied, as originally intended, a product made of an advanced polymer matrix composite. The problem may have been exacerbated because RIM is a process technology. Not only was it difficult to define the innovation, but almost no guidance existed concerning the chronology or the relative significance of human or organizational contributors. The structure for this case, such as one emerged, had to be pieced together from diverse and fragmented sources: patents, bibliographic databases, technical summaries, trade literature, and NSF awards data. All of this material came late in the study's timetable-too late for it to guide the interviews. (In the other two cases, which were conducted first, the interviews were more productive and enlightening because they were conducted with a framework already in place.) We suspect that future cases will be more like RIM than not, so we must begin early to explore patent, bibliographic, and research support data, and the trade literature.

For a number of different reasons, it proved difficult to apply the interview protocol (see Appendix A) in many interviews. Strong-willed individuals often have a story to tell, and they want to tell it in their own way. In the Internet and MRI cases, substantial consensus existed about who the key contributors were and what they contributed. The interviews reinforced this consensus, so the interviews could focus more specifically on possible influences that NSF support might have had. In the RIM case, little prior evidence about the relative importance of contributors (individual and institutional) and their contributions was available, and, for reasons that are unclear, our respondents were unable to provide much insight on the question. This experience also led to the same conclusion that interviews should be used to fill in gaps in the main "story," provide nuances and insights, and help resolve apparent contradictions-not, as was the situation in RIM, to help lay out the structure of the case itself. Interviews should come later in the course of each case and serve a less demanding purpose than we had initially assumed.[78]

Another lesson learned relates to the value and limitations of the NSF awards database. We had assumed, incorrectly, that the database contained all major awards. But some center awards do not appear (omissions seem idiosyncratic rather than systematic), major facility awards sometimes do not appear (e.g., the National Center for Supercomputer Applications), and key programs such as NSFNET do not necessarily appear in a recognizable form. Award titles do not always accurately describe the nature of the award or can be misleading. Thus the awards data should be gathered early in future cases and supplemented with internal NSF budget documents and program review materials. These should be used to prepare a preliminary record of NSF funding for the research area to be studied and checked by knowledgeable program managers at NSF.

Our study design called for interview material and more objective data such as bibliometric and patent citations to complement each other and, more importantly, to serve as independent sources that could be used for mutual verification. As it turned out, the quality of and ease of access to these two different types of data varied widely and unpredictably in the pilot cases. Rather than complementing each other, in each case one type of data proved far more rich, reliable, and appropriate to the innovation than the other. In RIM, interview data and the literature proved to be weaker sources of information than we expected, compelling us to rely more heavily on bibliometric and patent data for the basic structure of the case than in the other two cases. In the MRI and Internet cases, existing literature turned out to be a significant and reliable source of information, leading to a lesser role for bibliometric and patent data. Intellectual property rights played a negligible role in the Internet case and a subtle one in MRI, a role not easily revealed by standard patent analysis. In sum, in future cases, we cannot expect several different sources of data to be comparably available, relevant, and reliable. Early, careful examination of future cases will be necessary to anticipate the strengths and weaknesses of each type of data, perhaps leading to revision of each case study strategy.

Our Technical Review Panel acted wisely when they advised us to consider the first three cases as pilot studies. The experience of this first year will greatly enhance our ability to approach future cases in ways that will yield the kinds of results we all seek.