Tech Report

IXL contracted with SRI Education for an independent review of a quasi-experimental study they conducted: “Assessing the Impact of IXL Math over Three Years: A Quasi-Experimental Study.” The study author conducted a retrospective analysis to investigate the extent to which school use of IXL Math during a three-year period was associated with higher rates of proficiency on statewide assessments. SRI determined that this study provides moderate evidence for IXL Math efficacy in grades 3-8 over a three-year period according to the ESSA levels of evidence provided by the U.S. Department of Education guidelines.

Los Angeles City College launched the STEM Pathways program in 2016 with funding from the U.S. Department of Education. The intent of the STEM Pathways program was to improve STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students. The program offered a variety of supports including Supplemental Instruction (SI), peer tutoring in STEM, a book and technology loan program, specialized counseling, a math boot camp, and an undergraduate research experience. This report presents findings from a quasi-experimental study estimating the impact of STEM Pathways program participation on STEM course success and continuation in STEM, as well as degree attainment. SRI found positive and statistically significant results for continuation in STEM and STEM credits earned.

In spring 2020, at the outset of the pandemic, some intrepid employers quickly converted internships to virtual experiences rather than canceling them. SRI’s STEM Core Expansion Alliance evaluation team took advantage of this unprecedented growth in remote internships in STEM fields to capture innovations and lessons learned from these improvised programs.
SRI investigated employers’ strategies for recruiting, mentoring, and supporting remote interns. The team examined whether and how students used employer supports and what they gained from networking and professional learning opportunities in the virtual context. SRI paid particular attention to strategies for making virtual internships more accessible and equitable for all students and for diversifying intern cohorts.
The team used results from surveys of and interviews with employers and interns to develop guidance for employers. The resulting report offers a rich, cross-sector look at what STEM employers learned from their experiences and clearly conveys how they can improve their virtual internship programs in the future.

These case stories intend to give an in-depth look at each school’s setting and the details of how ConnectED unfolded in each of the following places: Compton, California; Tuskegee, Alabama; and Orondo, Washington.

Even when innovations have rigorous evidence of impact, they often are not widely adopted by the field, or their use is not sustained. To support more successful transitions of educational research to the field, SRI researchers modified its Invent-Apply-Transition (I-A-T) framework, that has been successfully used to scale research to practice in healthcare, enterprise software, and robotics, to educational contexts based on the experiences of an Advisory Council and Expert Panel Members, all with direct experience successfully scaling educational innovations. SRI began this project by conducting a brief literature scan to identify common themes that education researchers employed in effective scale-up strategies. SRI researchers conducted semi-structured interviews with the four Advisory Council and nine Expert Panel members to gather feedback from experts in the field who have successfully scaled educational innovations on the I-A-T framework, and then used their feedback to make relevant modifications. This report introduces the I-A-T framework and how it has been modified to educational contexts. The report also describes an Embedded Entrepreneur Program model that could be used to support Institute of Education Sciences (IES) grantees to increase their probability of scaling and transitioning their innovations to the educational market by helping them analyze their needs hypotheses, product-user fit, stakeholder mapping, market differentiation, product market fit, and scaling pathway hypotheses. Our initial findings suggest that education researchers and innovators could especially benefit from attending more to three factors: (1) the feasibility/usability of the innovation (the “Convenience” component of the Performance, Reliability, Convenience, Cost [PRCC] approach), and how it would be incorporated into the operating environment of the school and district; (2) an understanding of the scope of the innovation (i.e., whether it is a product or a feature); and (3) early identification of a commercialization strategy that is aligned with the scope of the innovation. The report explains how educational researchers can benefit from following the steps outlined in the modified I-A-T framework and collaborating with entrepreneurs with demonstrated skills and dedicated time for scaling innovations. The following appendices are included: a detailed description of the I-A-T framework, a description of the Embedded Entrepreneur Pilot Program, and a literature scan.

Los Angeles City College launched the STEM Pathways program in 2016 with funding from the U.S. Department of Education. The intent of the STEM Pathways program was to improve students’ STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students, through a variety of supports. The STEM Learning Center, a drop-in peer tutoring program offering students assistance in STEM courses, was one of the grant’s most-used supports. This report presents findings from a quasi-experimental study to estimate the impact of the STEM Learning Center on STEM course success and continuation in STEM. SRI found positive effects of STEM Learning Center participation on students’ STEM outcomes, in terms of increased STEM credits and STEM continuation.