Zalles, D., & Haertel, G. (2005). Expert panel review of OERL’s instructional utility. Paper presented at the American Educational Research Association (AERA) Annual Meeting, Montreal, Canada.
Introduction
As part of the site’s internal formative evaluation, the OERL team has adapted and integrated several evaluation methodologies to examine patterns of usage, customer satisfaction, and the instructional utility of the site. These methods include surveys of
target audiences, usability testing, and an expert panel review. This chapter presents the results from the expert review.
The Online Evaluation Resource Library is a Web site of evaluation resources (http://oerl.sri.com). SRI International developed OERL under contract with the National Science Foundation. It is intended to provide evaluators of NSF projects with resources they can use to develop sound evaluation plans, reports, and instruments. The site includes a collection of plans, instruments, and reports (selected from artifacts of NSFfunded projects) and professional development modules on topics such as methodology, sampling, and questionnaire development. To date, the site has been visited by thousands of users from a variety of organizations including institutions of higher education, government, military, and industries.
As a library of evaluation resources, OERL has many potential uses, in addition to its primary role as a support to evaluators of NSF-funded projects. In this expert panel review, we chose to focus on the potential for OERL to be used in graduate evaluation courses as a tool for building the capacity of new and potential evaluators. In addition to being a possible teaching tool, introducing OERL to graduate level evaluation students is a form of dissemination. Before we began our expert panel review, we knew that OERL had been used by some professors, but we wanted to better understand its promise for use in graduate level evaluation courses. An expert panel of five university faculty members responsible for graduate training in evaluation were invited to review the site. The review began in mid-December of 2002 and the phone call conferences took place in January and February of 2003.