The Role of NSF's Support of Engineering in Enabling Technological Innovation

INTRODUCTION TO THE CASES

PURPOSE OF THE STUDY

The 3 cases presented in this report are the first of 12 to be conducted by SRI International as the central component of what was designed as a 4-year project. The project is examining how National Science Foundation support for research, particularly engineering research, has contributed to the development and commercialization of recent, significant engineering innovations. The project is, to some extent, analogous to the several studies carried out in the late 1960s and early 1970s (Hindsight, TRACES) that sought to identify the origins in science of significant innovations. According to NSF, the project's purpose is

to conduct a systematic examination of the antecedent discoveries, events, people, interactions, and conditions that lead to the evolution of the 12 most significant engineering innovations to have emerged in the preceding decade to: (1) document NSF's involvement in bringing about the innovations; and (2) evaluate the significance of NSF's role in the broader context of the innovations' development.

The innovations to be studied should have "emerged as significant in the last decade in broad technical areas that NSF has supported for decades." They should meet the following criteria:

Considerable engineering content.
A significant research component.
An outcome that causes major changes in the quality of life, how tasks are performed, and/or the cost or efficiency of production.
The likelihood of at least some NSF engineering involvement at some point in the innovation's evolution.

A Technical Review Panel was assembled whose responsibilities are to help select the 12 innovations, provide background information on those selected, and review the 3 cases completed in each year. The members of the Panel are:

Robert Clagett, former Dean of the Business School, University of Rhode Island
Mary Good, Undersecretary of Commerce
William Howard, former Senior Vice President and Director of Corporate Research, Motorola
Arthur Humphrey, Professor of Chemical Engineering, Penn State
David Mowery, Professor of Business Administration, University of California, Berkeley
Karl Pister, Chancellor, University of California, Santa Cruz
William Raub, Office of the Assistant Secretary, Planning and Evaluation, Department of Health and Human Services
Roland Schmitt, former President, Rensselaer Polytechnic Institute
Roy Weston, Chairman Emeritus, Roy F. Weston, Inc.
Bert Westwood, former Vice President, Lockheed-Martin.

The initial meeting of the Technical Review Panel took place on November 16, 1995. At this meeting, the three innovations to be studied the first year were chosen:

Magnetic resonance imaging
High-performance polymer matrix composites
The Internet.

METHODOLOGICAL AND CONCEPTUAL ISSUES IN CONDUCTING RETROSPECTIVE CASE STUDIES

In this section, we summarize the shortcomings, limitations, and criticisms that have been associated with past studies of technological change using the retrospective case approach.[5] We then describe the SRI research strategies that are intended to address these shortcomings. One criticism of past studies is that the cases selected were not chosen statistically to be representative of a larger population, and thus were subject to charges of case selection bias. Second, one of the key units of analysis in these studies was the "event," and since identification of events and judgments of whether they were "significant" or "critical" for the innovation were made by the researchers, there were reliability problems. Third, the cases tended to have a deterministic flavor to them, because the uncertainty inherent in the innovation process was not captured in the historical traces. In particular, failures or "dead ends" were not identified, even though these might have yielded knowledge that eventually proved important to realizing the innovation. Finally, there was a "hardware" bias in defining events; managerial and organizational innovations important to the innovation generally were not recognized or acknowledged. Our research strategy was designed explicitly to address these issues.

There are several categories of possible bias in the selection of innovations for this study:

Chronological (the innovation was more or less likely to be influenced by NSF support, which began in 1954).
Technical and/or knowledge field (NSF does not support engineering research uniformly across fields, support of fields shifts over time, and definition of fields shifts over time).
Complexity (the more technically complex, the more inputs from knowledge and technology, thus the greater the likelihood of NSF influence).

Problems of bias were addressed by using the independent Technical Review Panel, jointly with SRI, to select innovations, and by choosing innovations with relatively recent impact. It was appropriate to select innovations known to have some relevance to fields supported by NSF, because the purpose of the study was not to compare NSF's contribution with that of some other source of support, or to generalize to some population. It was also appropriate to choose technically complex (but researchable) innovations so that opportunities for potential NSF influence of different types and timing would be maximized.

In the more than 20 years since the early "traces" studies were done, much has been learned about processes of technological innovation. The "pipeline" model, in which fundamental research precedes applied research and problem solving, which in turn lead to product development, has been replaced by far more complex and accurate models. The key features of these models are feedback loops between and among stages, and recognition of continuous exchange between technology development and the existing knowledge base and between knowledge-producing institutions and all phases of development, including production. The implication of the more complex model for data collection is that our initial interviews must include all phases of innovation, including the final site(s) at which innovation is introduced. All significant inputs from existing knowledge and technology bases, as well as research activity generated because of downstream problems with the innovation process, had to be identified. Interviews included, to the best of our knowledge, all major contributors to the innovation at all stages up to and including commercial introduction, if appropriate. All cases employed basic searches of citations and patents for principal investigators, contributors, institutions, and sources of support. It was intended that at least one of the first three cases should test sophisticated citation and cluster analysis, but delays in the availability of funding made this impossible.

Once the three innovations were selected, library work was undertaken to identify the major players, timeline, technological changes, and other features of each innovation. SRI then interviewed NSF staff and SRI scientists and engineers to obtain more detail about individual and institutional contributors, milestones, patents or copyrights, and related advances. NSF award data files were available to identify principal investigators, award institutions, doctoral grants, travel awards, workshops, and other types of awards associated with particular innovations from the beginning of NSF through FY 1995. SRI conducted interviews with key individuals involved in the research, development, and introduction of the innovation into the marketplace, using interview protocols based on a model of the innovation process that incorporates current understanding of its complexities and feedback elements.[6] Once the major contributing streams of knowledge and technology were identified via personal interviews and associated site visits, a variety of explorations filled out the innovation's history, including alternative paths avoided and dead ends. Throughout the tracing of the innovation's history, the type and influence of NSF support and other sources of support were identified. The first three cases were viewed as pilot tests of the SRI approach and of the value of bibliometric methods as a complement to interviews and more traditional archival data.

To identify points at which particular technical solutions to problems or knowledge inputs entered the flow of information, we asked interview respondents (in person or by telephone) to rate the importance of the input to the innovation (e.g., were alternatives available? Was this input unique, a breakthrough in its own right?). Reliability of the data could then be checked through multiple respondents and independent review of draft cases, and initial assessments of impact could be compared with bibliometric and/or patent citation analysis as independent measures of impact. Similarly, we planned to assess influence of participants in the innovation on some ordinal scale (from minimal to crucial). We asked major contributors to each innovation for background on knowledge and technical inputs that they used in working on the innovation or its antecedents, and explicitly for the basis for a particular choice of technology or information: for example, was it known to be a likely solution because of a known failure of an alternative solution? These "dead ends" could then be identified and scored as significant knowledge/technical inputs.

SPECIFIC RESEARCH TASKS

Decomposition

It was essential first to identify technologies that underlie each innovation, as distinguished from the sociotechnical system that contributes significantly to the innovation's socioeconomic or other consequences. Among the technologies that constitute the innovation, it was next important to distinguish "intrinsic" from "supporting" technologies. Intrinsic technologies are those that are unique to the innovation studied; that is, they were developed as an integral part of the innovation. Supporting technologies are essential to the functioning of the subject innovation, but they already existed in the "environment" and thus could be incorporated largely "as is" in the innovation. Only intrinsic technologies were studied in detail, but the importance of existing technologies and the possible role that NSF may have played in their realization were acknowledged.

Library Search

This involved search of online databases, using keywords associated with the intrinsic technologies. All major works published that describe the development of these technologies were identified.

Bibliometrics

Two very small and tentative experiments were performed with bibliometric data concerning the initial innovation cases. The data used were the 1988, 1989, and 1990 Research Front Databases of the Institute for Scientific Information (ISI). These databases use a clustering technique to group publications in an annual file of ISI-indexed journal papers into interlinked research areas that resemble the specialties of working scientists. Sometimes referred to as "maps" or "descriptive models" of research activity for a given year, the data were explored to show the close relationship of NMR imaging to NMR spectroscopy research, and to seek NSF-funded researchers among the major contributors in the field of composite materials. The results of these experiments are described in the appropriate case study context.

Institutional Analysis

This involved identifying the major companies, federal labs, federal agencies, universities, and other organizations that played a significant role in the development of enabling technologies. The process began with each intrinsic technology and used basic library search strategies to identify contributing institutions. This step was followed by discussions with NSF program managers, interviews with key contributors to the innovation, and searches of NSF's awards database.

Patent Analysis

Patent analysis involved searches of the standard patent databases available at federal government repository libraries. Of interest are the names of the inventors, their institutional affiliations, coinventors, and citations of key research literature.

Personal Interviews

A small number of focused personal interviews with the people identified as key contributors to each major intrinsic technology were conducted. Respondents were those closest to the intrinsic technologies when these were brought together (with existing technologies) to create the system that had the socioeconomic impact associated with the innovation. They were the people most likely to be able to describe the contributing knowledge and technology streams that led to the intrinsic technologies' final realization.

Phone Interviews

Respondents were those identified in institutional searches, patent searches, informal discussions, and personal interviews as having some knowledge of the development and intellectual past of each intrinsic technology. The interviews, the bibliometric/patent data, and the NSF awards data together formed the data from which NSF's role and impact were identified and detailed.

REFERENCES

Kreilkamp, Karl. "Hindsight and the Real World of Science Policy," Science Studies,
1 (1971): 43-66.

Mowery, David, and Rosenberg, Nathan. "The Influence of Market Demand upon Innovation: A Critical Review of Some Recent Empirical Studies," Research Policy, 8 (1979): 103-153.