Project

Research on Assessment in Games (GlassLab-Research)

SRI is researching the inferential validity, reliability, and effectiveness of formative assessments embedded within games.

young students in a classroom using computers

In summer 2012, the Bill & Melinda Gates Foundation, in cooperation with the MacArthur Foundation, made a significant investment to establish the Games Learning and Assessment Lab (GlassLab), which includes top game developers, assessment experts, and researchers from multiple fields and disciplines. The program was divided into two teams to mitigate conflict of interest and guarantee independent validation of assessments developed by the program.

The programming and development group (GlassLab) was tasked to design and develop state-of-the-art, game-based formative assessments. These assessments are being developed in response to the climate of student disengagement that currently exists in many classrooms. By leveraging the popularity of digital video games and by applying Evidence Centered Design (ECD), the game-based formative assessments address the needs of both students and teachers for reliable and valid real-time actionable data within a motivating learning environment. This work is being conducted by the Institute of Play, the Educational Testing Service (ETS), Pearson, Inc., Electronic Arts (EA), and the Entertainment Software Association (ESA).

Concurrently, the Foundation tasked the SRI-led research team (GlassLab-Research) to independently conduct research on the qualities, features, inferential validity, reliability, and effectiveness of the assessments that were embedded within the GlassLab game products. The GlassLab-Research work is being conducted by experts in assessment, learning sciences, science education, and learning technology at SRI with the support of external consultants.

GlassLab-Research's agenda has four strands:

  • In Strand 1, GlassLab-Research conducted two meta-analyses on the effects of using video games and computer-based simulations in educational settings. The games meta-analysis focused on the effects (cognitive, interpersonal, and intrapersonal) of video games in K – 16 classrooms and the meta-analysis of computer-based simulations focused on the cognitive effects of simulations in K-12 STEM education. Executive summaries and at-a-glance briefs for each meta-analysis are available at the links below.

Simulations for STEM Learning: A Systematic Review and Meta-Analysis (Report Overview)

Simulations for STEM Learning: A Systematic Review and Meta-Analysis (Executive Summary)

Simulations for STEM Learning: A Systematic Review and Meta-Analysis (Full Report)

Digital Games for Learning: A Systematic Review and Meta-Analysis (Brief)

Digital Games for Learning: A Systematic Review and Meta-Analysis (Executive Summary)

  • Strand 2 of the GlassLab-Research verifies the use of ECD in the design and development of the GlassLab game-based assessments. This qualitative study uses observation of game development activities and playtesting, structured interviews of game and assessment designers, and the collection of design artifacts created during the GlassLab design processes. Forthcoming reports describing the use of ECD in the GlassLab game design will be available from this website.
  • Strand 3 involves the design and conduct of a series of construct validity, enacted validity, and learning studies to examine how the games and their associated assessments developed by GlassLab influence and measure: (1) learning, (2) the transfer of learning, (3) problem-solving, (4) student engagement in formal education settings, and 5) classroom implementation. Forthcoming reports describing findings of validity studies will also be available from this website.
  • Strand 4 of the research applies advanced data analytics to student data captured during gameplay to measure directly the activities completed by the student. This provides a valuable data source for assessing student engagement and learning within a variety of game-based instructional experiences and complements other more traditional measures of learning, engagement and participation. Forthcoming reports on the use of data analytics will also be available from this website.

For more information, please contact GlassLab-Research@sri.com.