Establishing the Validity and Diagnostic Capacity of Facet-Based Science Assessments | SRI International

Toggle Menu
teacher and two students working on an electronics test

Establishing the Diagnostic Capacity of Facet-based Assessments

SRI is validating a web-based diagnostic physics classroom assessment tool.

Formative use of diagnostic classroom assessment has the potential to provide critical information to students and teachers about whether students understand the targeted concepts, and if not, what problematic or partial understandings are present instead. Teachers can then use the evidence about student understanding to decide what to do to help students progress from partial or incorrect understandings toward targeted learning goals.

Diagnostic assessments can be particularly informative when they are based on a detailed conceptual foundation that incorporates what is known about learning and cognition in the domain. The resulting assessment framework is a clear and transparent articulation of the linkage between the assumptions about cognition and the nature of the observations to be made of student performance.

The design of facet-based diagnostic assessments has been based on an impressive foundation rooted in instructional practice and incorporating knowledge about student cognition and learning in STEM areas, in concordance with national science standards. Facet-based assessments are designed to reveal not just whether a student falls short of mastery and needs more instruction, but the particular nature of a student’s problematic understanding and hence the type of additional learning experiences needed. Despite their foundation, existing facet-based assessments were not developed with explicit cognizance of particular psychometric models and related issues of measurement reliability and score consistency.

The objective of this project is to investigate and strengthen the cognitive, instructional, and empirical validity of facet-based physics assessments available through Diagnoser Tools that are intended to help teachers diagnose student understanding in science. The project is doing so by incorporating state-of-the-art psychometric methods within an integrated validity framework to contribute to the improvement of the diagnostic capabilities of science assessments for formative uses.