Los Angeles City College launched the STEM Pathways program in 2016 to improve students’ STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students. With funding from the U.S. Department of Education, the program offered a variety of supports including Supplemental Instruction (SI), peer tutoring in STEM, a book and technology loan program, specialized counseling, a math boot camp, and an undergraduate research experience. This report presents findings from three quasi-experimental studies estimating the impact of program participation overall, as well as STEM Learning Center participation and SI participation, specifically on STEM course success and continuation in STEM. SRI found positive and statistically significant results for continuation in STEM and STEM course success.
STEM and computer science education publications
Evaluation of Los Angeles City College’s STEM Pathways Program: Impacts of STEM Pathways Program Participation on Student Outcomes
Los Angeles City College launched the STEM Pathways program in 2016 with funding from the U.S. Department of Education. The intent of the STEM Pathways program was to improve STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students. The program offered a variety of supports including Supplemental Instruction (SI), peer tutoring in STEM, a book and technology loan program, specialized counseling, a math boot camp, and an undergraduate research experience. This report presents findings from a quasi-experimental study estimating the impact of STEM Pathways program participation on STEM course success and continuation in STEM, as well as degree attainment. SRI found positive and statistically significant results for continuation in STEM and STEM credits earned.
Evaluation of Los Angeles City College’s STEM Pathways Program: Impacts of the Supplemental Instruction Program on Student Outcomes
Los Angeles City College launched the STEM Pathways program in 2016 with funding from the U.S. Department of Education. The intent of the STEM Pathways program was to improve students’ STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students. The program provided a variety of supports including Supplemental Instruction (SI) for key STEM courses. This report presents findings from a quasi-experimental study estimating the impact of SI participation on STEM course success and continuation in STEM. SRI found positive and statistically significant results for focal course passing, focal course grade, and continuation in STEM.
Evaluation of Los Angeles City College’s STEM Pathways Program: Impacts of the STEM Learning Center on Student Outcomes
Los Angeles City College launched the STEM Pathways program in 2016 with funding from the U.S. Department of Education. The intent of the STEM Pathways program was to improve students’ STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students, through a variety of supports. The STEM Learning Center, a drop-in peer tutoring program offering students assistance in STEM courses, was one of the grant’s most-used supports. This report presents findings from a quasi-experimental study to estimate the impact of the STEM Learning Center on STEM course success and continuation in STEM. SRI found positive effects of STEM Learning Center participation on students’ STEM outcomes, in terms of increased STEM credits and STEM continuation.
ST Math: Nonregulatory ESSA Standards Evidence Review & What Works Clearinghouse Standards Review
MIND Research Institute contracted with SRI Education for an independent review of a quasi-experimental study WestEd conducted in partnership with MIND: “A Cross-State Evaluation of MIND Research Institute’s ST Math Program and Math Performance.” The study authors conducted multiple analyses to investigate the extent to which use of Spatial-Temporal (ST) Math in grades 3–5 was associated with increased school mathematics achievement.
SRI determined that this study provides moderate evidence for ST Math efficacy in grades 3–5 according to Every Student Success Act (ESSA) levels of evidence provided by the U.S. Department of Education guidelines. SRI determined that study fulfills the design, analytic, and technical requirements of WWC v4.0 Group Design Standards for Meets Evidence Standards with Reservations.
The authors selected two outcomes of interest for study: average scale scores and percent proficient rates on state standardized mathematics assessments. The authors applied multiple comparison corrections for their main findings. On both outcomes, grade-level clusters using ST Math performed significantly better in mathematics than similar grade-level clusters not using the program. As measured by average scale scores on state standardized mathematics assessments, the authors found a positive effect size of .13. As measured by percent proficient rates on state standardized mathematics assessments, the authors found a positive effect size of .17. Considerably larger positive effects were found in reduced analytic samples of grade-level clusters implementing ST Math according to specific fidelity metrics.
Early science and engineering: The impact of The Cat in the Hat Knows a Lot About That! on Learning
Over 450 children living in low-income households across the U.S. participated in this study, which explores whether providing families with access to PBS KIDS The Cat in the Hat Knows a Lot About That! ™ videos, digital games, and hands-on activities can help 4- and 5-year old children learn physical science and engineering concepts and practices.
Evaluating Digital Learning for Adult Basic Literacy and Numeracy
More than 36 million adults in the United States do not have the basic literacy and math skills needed for many entry-level jobs and even less so for the types of jobs expected to dominate in the future. And current federal- and state-funded adult basic education (ABE) programs, the main providers of skill development and training programs for this population, do not have the resources, facilities, or trained staff to serve all those adults in need of further education to improve their basic skills and job prospects. In 2014, The Joyce Foundation asked SRI Education’s Center for Technology in Learning to investigate the role and efficacy of five different digital learning products in (1) improving the basic reading, writing, and math outcomes for low-skilled adults in ABE programs and (2) helping programs increase their capacity to serve a greater number of students. Through this research, we set out to understand how ABE programs might use these technologies to improve the instruction they offer, whether such technologies are effective with low-skilled adults (performing at fourth- to ninth-grade levels in reading and/or math) and which practices and product features might be associated with better outcomes for students and programs. In this report, we present the findings from this three-year study.
Developing protocols to support collaborative teacher reflection and professional learning for science argument writing
This white paper describes the development of a suite of teacher reflection protocols that structure generative conversations among teachers about instruction in support of students’ written scientific arguments. Guided by the principles of design-based implementation research, a team of researchers from SRI and The Lawrence Hall of Science collaborated with the National Writing Project to develop three distinct protocols, each geared to a different stage of an instructional cycle: a planning protocol for assignment and supporting lesson design, an assignment protocol to support collaborative reflection on a writing task and its instructional surrounds, and a consultancy protocol to help teachers guide a larger group examination of student work.
The protocols are designed to be used by teachers—either individually or as part of a professional learning community—or by professional development providers to guide the creation of an assignment or to analyze the assignment and instructional supports. All the protocols are intended to focus teachers’ discussions on specific features of scientific argument prompts and instructional supports for students to write an argument. By explicitly grounding the research in components of the NGSS Science and Engineering Practices, the protocol suite supports teachers with the complex process of developing students’ emergent science argument writing practice.
Constructing Assessment Tasks that Blend Disciplinary Core Ideas, Crosscutting Concepts, and Science Practices for Classroom Formative Applications
How do we measure knowledge in use? In this paper we describe how we use principles of evidence-centered design to develop classroom-based science assessments that integrate three dimensions of science proficiency—disciplinary core ideas, science practices, and crosscutting concepts. In our design process, we first elaborate on, or “unpack”, the assessable components of the three dimensions. We then use these elaborations to specify a set of claims calledlearning performances that describe what students need to be able to know and do in order to meet knowledge-in-use learning goals, such as the performance expectations articulated in the U.S. Next Generation Science Standards. Learning performances are crafted as knowledge-in-use statements that integrate aspects of the three dimensions, but are smaller in scope than end-of-grade-band performance expectations. Next, we define task features to elicit from students the desired evidence of proficiency. Our final step entails using design patterns derived from specifying learning performances, specifying evidence, and defining task features to construct tasks that measure science proficiency. In this paper, we present our design approach, provide examples of tasks, and consider implications of this work for next generation science assessment.
