• Skip to primary navigation
  • Skip to main content
SRI logo
  • About
    • Press room
    • Our history
  • Expertise
    • Advanced imaging systems
    • Artificial intelligence
    • Biomedical R&D services
    • Biomedical sciences
    • Computer vision
    • Cyber & formal methods
    • Education and learning
    • Innovation strategy and policy
    • National security
    • Ocean & space
    • Quantum
    • QED-C
    • Robotics, sensors & devices
    • Speech & natural language
    • Video test & measurement
  • Ventures
  • NSIC
  • Careers
  • Contact
  • 日本支社
Search
Close
Education & learning publications April 1, 2006

Reverse Engineering the NAEP Floating Pencil Task Using the PADI Design System (PADI Technical Report 16)

Citation

Copy to clipboard


Haynie, K., Haertel, G., Lash, A., Quellmalz, E., & DeBarger, A. H. (2006). Reverse Engineering the NAEP Floating Pencil Task Using the PADI Design System (PADI Technical Report 16). Menlo Park, CA: SRI International.

Abstract

Large-scale science assessments have been criticized for not tapping rich, authentic scientific problems. The use of evidence-centered design (ECD) principles in large-scale assessment design can potentially improve the quality of performance assessment tasks for scientific inquiry. The Principled Assessment Designs for Inquiry (PADI) design system has been used by the PADI team to analyze several well-known science assessments. This report will discuss the reverse engineering of the National Assessment of Educational Progress (NAEP) Floating Pencil task. The PADI design system was used as an analytical tool for understanding the characteristics and underlying assessment argument of this chosen task. Carrying out this work, we grappled with task complexity and came to understand how science performance assessment tasks might be constructed in the future.

In reverse engineering the Floating Pencil task into an assessment blueprint, our team created a task specification on the PADI design system. In designing the Floating Pencil task specification, we specified the Task Model, Student Model, and Evidence Model. In defining the Task Model, we considered what family of tasks Floating Pencil might be a member of—helping to define the fixed characteristics and Task Model Variables for Floating Pencil. Our team considered a variety of potential Student Models and chose one based on the NAEP content-by-process framework (Allen, Carlson, & Zelenak, 1999). The chosen Evidence Model included an evaluative submodel that mirrored NAEP’s rubric for Floating Pencil (publicly released, see https://nces.ed.gov/nationsreportcard/itmrlsx/sampleq/96sci8.pdf. We defined unidimensional Rasch Measurement Models for each Activity within Floating Pencil.

The use of the PADI design system to reverse engineer the Floating Pencil task resulted in the creation of new assessment knowledge, general and specific to Floating Pencil, for our team and the PADI project. We considered the coherence and linkages among the Task Model, Student Model, and Evidence Model and grappled with the underlying assessm ent argument for Floating Pencil. Reverse engineering the Floating Pencil task not only contributed to our knowledge of the characteristics of one particular large-scale performance assessment task, but shed light on how new science performance assessments might be forward engineered.

↓ Download

Share this

How can we help?

Once you hit send…

We’ll match your inquiry to the person who can best help you.

Expect a response within 48 hours.

Career call to action image

Make your own mark.

Search jobs

Our work

Case studies

Publications

Timeline of innovation

Areas of expertise

Institute

Leadership

Press room

Media inquiries

Compliance

Careers

Job listings

Contact

SRI Ventures

Our locations

Headquarters

333 Ravenswood Ave
Menlo Park, CA 94025 USA

+1 (650) 859-2000

Subscribe to our newsletter


日本支社
SRI International
  • Contact us
  • Privacy Policy
  • Cookies
  • DMCA
  • Copyright © 2022 SRI International