Measuring learning progressions using bayesian modeling in complex assessments

SRI Authors: Daisy Wise Rutstein

Abstract

This research examines issues regarding model estimation and robustness in the use of Bayesian Inference Networks (BINs) for measuring Learning Progressions (LPs). It provides background information on LPs and how they might be used in practice. Two simulation studies are performed, along with real data examples. The first study examines the case of using a BIN to measure one LP, while the items in the second study are designed to measure two LPs. For each study, data are generated under four alternative models, and each of the models is fit to the data. The results are compared in  terms of fit, parameter recovery, and classification accuracy for individuals. In the case where one LP was used, two models provided high correct classification rates. When two LPs are being measured the classification rates were not found to be high, although an unconstrained model with freely-estimated conditional probabilities had slightly higher rates than a constrained model in which the conditional probabilities were given by lower-dimensional functions. Overall, while BIN show promise in modeling LPs, further research is needed to determine the conditions under which this modeling approach is appropriate