Center for Science, Technology, and Economic Development (CSTED) > Selected Reports

Study Design

Preparation of the study design recognized that identifying the full range of potential or existing university-industry interactions would not be an easy task. By 1993, ERC program staff had identified ten mechanisms of industrial interaction in the ERCs:[4]

• Joint strategic planning of the ERC's research agenda;
• Engineers on loan from industry working at the ERC;
• Joint projects including experimental testbeds;
• Technical advisory groups;
• Patents, especially licenses of patents and software;
• Placement of ERC graduates in industry;
• Startup companies based on ERC advances;
• Consortia formed with industry and the ERC to work on specialized problems;
• Students: work with engineers from industry on joint projects, mentors, thesis committee members from industry, summer jobs, visits to companies, and seminar speakers from industry;
• Industrial support: cash for membership fees and research contracts, equipment donations, personnel exchange.

Recent conceptual, empirical, and historical studies on the industrial transferability of academic research results and the contributions of universities to economic growth provided a fruitful starting point. For example, one study of interaction between industry and the national laboratories identified a range of at least ten ways, most similar to those listed above, in which this could occur, and noted that each could have a variety of impacts that ripple through the firm.[5]

The variety of potential impacts highlighted the difficulties in fulfilling the second objective of the study: to "determine which types of interaction are most useful to industry." This meant seeking information concerning the value of useful interactions to industry. While some of these, such as the contribution to corporate earnings of a profitable new product that results from the licensing of an ERC patent, can be given a monetary value, it is far more difficult to establish the value of the reciprocal flow of influences resulting in industry interaction with the training of an ERC student, the impact of the perspective brought by hiring such a student, or the access to the desired cross-disciplinary approaches and new configurations of knowledge that results from university-industry interaction in the ERC context. Moreover, where interaction occurs, there are likely to be a multitude of impacts that flow through an industrial firm. A variety of proxy and intermediate measures would be needed to fill out the conceptual model that would serve as the basis of the overall project. Finally, the process of university-industry interaction is a dynamic one, and the study needed to consider this dynamic rather than being limited to a retrospective consideration of the results of a program that remains relatively young and could be expected to continue to evolve.

Methodology

In order to deal with these challenges in the initial research design, we first conducted a series of case studies within a small number of firms to provide details on intra-firm activities related to ERC participation. Drawing upon the case studies, a focus group consisting of representatives from companies that participate in ERCs was held, and two senior members of the SRI study team prepared a paper describing a preliminary model of university-industry collaboration. This "process model" helped to guide the major research effort of the study, both in the design of a survey instrument and in selection of the targeted respondents in all 707 industrial partnerships with ERCs in the 1993-94 ERC reporting year.

The results of the five case studies, each involving one ERC and a member company, and of the focus group, which included seven member companies, produced insights and results that raised serious questions about the original research design.[6]  In particular, these results required that "typical" approaches to survey design (e.g., how respondents are identified; the unit of analysis) be rethought, and that prevailing ideas about how firms value their participation in ERCs (and other investments in external R&D, for that matter) are incomplete and simplistic.

Results of the case studies and focus group had implications for any effort to measure impacts or benefits from industry participation in collaborative R&D relationships with universities (or with other R&D suppliers, for that matter). First, the business unit, not the firm or even the division, is the active response category. Thus, the survey instrument must be directed to the unit that is the primary direct beneficiary of the company's participation in an ERC. Second, surveys must distinguish carefully between at least two key roles within the unit: the "champion," or individual most extensively involved in the firm's interactions with the ERC, as distinguished from the decision-maker who approves the budget for ERC membership. Data must be obtained from both. Third, the survey must establish the nature of each respondent's involvement with the ERC. Our preliminary research showed that involvement has a large number of possible dimensions, and valid analysis of any impact or benefit data must control for the nature of each respondent's interaction. Finally, it is important to separate "results" or "outcomes" from "benefits," because different respondents within the same unit may agree on results or impacts but impute quite different levels of benefit to them. These factors were taken into account in the survey design, and the resulting eleven-page instrument, shown in Appendix A, sent to the Office of Management and Budget for approval.

At the beginning of the study, each ERC was requested to furnish the name and address of one contact for each company that they had listed in the latest complete NSF ERC database (1994) as providing "support", whether through membership fees, contracts, unrestricted grants, or in-kind donations. To increase the possibility that the primary survey recipient would be the ERC "champion" in the firm, ERCs were requested to identify specific individuals for each company in the following order of preference:

1) A member of the ERC's Advisory Committee;

2) If the company did not have a member on the Advisory Committee,

then a member of one of the ERC's Technical Committees;

3) If the company had neither a member on the ERC's Advisory Committee

nor any of its Technical Committees, then the primary point of contact

for the ERC with that firm.

An initial mailing of the questionnaire, a postcard reminder, a second mailing of the questionnaire, follow-on telephone reminders, and, in some cases, a third mailing of the questionnaire resulted in 355 valid returns. After correcting for erroneous addresses, retirements, or individuals who claimed not to have been involved with the ERC, the resulting survey population size was 497, for which the 355 valid returns represented a 71% rate of response.

In order to assess differences in perspective within individual firms on the nature and extent of the impact of ERC participation, respondents to the primary participant questionnaire were asked to identify two additional persons within their company to whom a somewhat shorter version of the questionnaire could subsequently be mailed: the person who reviews or approves the company unit's participation in the ERC (if other than the respondent to the primary participant questionnaire) and another individual who had interacted a considerable amount with the ERC. This resulted in a list of 266 individuals to whom the secondary participant questionnaire was mailed. (A copy of the secondary participant survey instrument, also approved by OMB, is shown in Appendix B.) A 52% response rate was achieved after an initial mailing of the questionnaire, a reminder postcard, and a second mailing of the questionnaire. Because the response rate was relatively unimportant for the back-up use this secondary participant questionnaire was designed to serve, no follow-on telephone calls were made.

The result is a rich data base that permits many analyses of outcomes and benefits of ERC/industry interaction by various aspects of the nature of the respondents' interaction with an ERC, the role of each respondent in the company's relationship with the ERC, the length of company involvement with an ERC, and numerous other factors, not all of which could be explored in detail in this report. The principal results of the survey are discussed in the remainder of this report.

Results of the survey brought to the fore certain questions and themes for which more detailed information was desired. Accordingly, telephone interviews were conducted with about twenty survey respondents selected because of either the depth of the benefits they reported or the richness of their open-ended responses, to discuss these issues in greater depth. Examples from these interviews, the protocol for which is shown in Appendix C, are described along with those elements of the survey findings that they principally elaborate. Seven interviews, selected to highlight a range of issues as well as certain common themes that were explored in all twenty, are summarized in a more narrative fashion in Chapter 9.