In spring 2020, at the outset of the pandemic, some intrepid employers quickly converted internships to virtual experiences rather than canceling them. SRI’s STEM Core Expansion Alliance evaluation team took advantage of this unprecedented growth in remote internships in STEM fields to capture innovations and lessons learned from these improvised programs.
SRI investigated employers’ strategies for recruiting, mentoring, and supporting remote interns. The team examined whether and how students used employer supports and what they gained from networking and professional learning opportunities in the virtual context. SRI paid particular attention to strategies for making virtual internships more accessible and equitable for all students and for diversifying intern cohorts.
The team used results from surveys of and interviews with employers and interns to develop guidance for employers. The resulting report offers a rich, cross-sector look at what STEM employers learned from their experiences and clearly conveys how they can improve their virtual internship programs in the future.
Publications
The Apple and ConnectED Initiative: Three Case Stories
These case stories intend to give an in-depth look at each school’s setting and the details of how ConnectED unfolded in each of the following places: Compton, California; Tuskegee, Alabama; and Orondo, Washington.
How a Researcher Learns from Practice
SRI Authors: Michelle W. Woodbridge
What, Again? Automatic Deductive Synthesis of the Unification Algorithm
We describe work in progress towards deriving a unification algorithm automatically from a declarative specification using deductive methods. The specification is phrased as a logical theorem and the program is extracted from the proof. The theorem is proved using Stickel’s system SNARK, operating over a theory of expressions and substitutions. The theory has been formulated to allow a simpler specification of the algorithm, and the theorem prover has discovered novelties in the implementation. It is hoped that the same techniques may enable the discovery of previously unknown unification algorithms for specific theories.
Animal Models of Narcolepsy and the Hypocretin/orexin System: Past, Present, and Future
Animal models have advanced not only our understanding of the etiology and phenotype of the sleep disorder narcolepsy but have also informed sleep/wake regulation more generally. The identification of an inheritable narcolepsy phenotype in dogs in the 1970s allowed the establishment of a breeding colony at Stanford University, resulting in studies that provided the first insights into the genetics and neurotransmitter systems that underlie cataplexy and rapid-eye movement sleep atonia. Although the discovery of the hypocretin/orexin neuropeptides in 1998 initially seemed unrelated to sleep/wake control, the description of the phenotype of the prepro-orexin knockout (KO) mouse as strongly resembling cataplexy, the pathognomonic symptom of narcolepsy, along with identification of a mutation in hypocretin receptor-2 gene as the source of canine narcolepsy, unequivocally established the relationship between this system and narcolepsy. The subsequent discovery of hypocretin neuron degeneration in human narcolepsy demystified a disorder whose etiology had been unknown since its initial description 120 years earlier. These breakthroughs prompted the development of numerous other animal models that have allowed manipulation of the hypocretin/orexin system, thereby advancing our understanding of sleep/wake circuitry. While animal models have greatly informed understanding of this fascinating disorder and the role of the hypocretin/orexin system in sleep/wake control, the question of why these neurons degenerate in human narcolepsy is only beginning to be understood. The development of new immune-mediated narcolepsy models are likely to further inform the etiology of this sleep disorder and animal models will undoubtedly play a critical role in the development of novel narcolepsy therapeutics.
Evaluation of the College, Career, and Community Writers Program: Findings from the i3 Scale-up Grant
The National Writing Project’s (NWP) College, Career, and Community Writers Program (C3WP) is designed to improve students’ argument writing through intensive teacher professional development, instructional resources, and formative assessment. In 2016, based on evidence of C3WP’s prior success in improving student achievement, NWP received a federal Investing in Innovation (i3) Scaleup grant to test C3WP in new contexts.
As part of this grant, SRI International (SRI) conducted a 1-year random assignment evaluation of C3WP in Grades 7–9 that found consistent program implementation and positive, statistically significant impacts on student writing achievement.
This is the third evaluation of C3WP in secondary grades, all of which have found positive and statistically significant effects on student achievement.
This technical report provides a detailed description of C3WP and the findings from the evaluation of the i3 Scale-up grant.
From Research to Market: Development of a Transition Process to Integrate Sustainable Scaling Methodologies into Education Innovation Research Design and Development
Even when innovations have rigorous evidence of impact, they often are not widely adopted by the field, or their use is not sustained. To support more successful transitions of educational research to the field, SRI researchers modified its Invent-Apply-Transition (I-A-T) framework, that has been successfully used to scale research to practice in healthcare, enterprise software, and robotics, to educational contexts based on the experiences of an Advisory Council and Expert Panel Members, all with direct experience successfully scaling educational innovations. SRI began this project by conducting a brief literature scan to identify common themes that education researchers employed in effective scale-up strategies. SRI researchers conducted semi-structured interviews with the four Advisory Council and nine Expert Panel members to gather feedback from experts in the field who have successfully scaled educational innovations on the I-A-T framework, and then used their feedback to make relevant modifications. This report introduces the I-A-T framework and how it has been modified to educational contexts. The report also describes an Embedded Entrepreneur Program model that could be used to support Institute of Education Sciences (IES) grantees to increase their probability of scaling and transitioning their innovations to the educational market by helping them analyze their needs hypotheses, product-user fit, stakeholder mapping, market differentiation, product market fit, and scaling pathway hypotheses. Our initial findings suggest that education researchers and innovators could especially benefit from attending more to three factors: (1) the feasibility/usability of the innovation (the “Convenience” component of the Performance, Reliability, Convenience, Cost [PRCC] approach), and how it would be incorporated into the operating environment of the school and district; (2) an understanding of the scope of the innovation (i.e., whether it is a product or a feature); and (3) early identification of a commercialization strategy that is aligned with the scope of the innovation. The report explains how educational researchers can benefit from following the steps outlined in the modified I-A-T framework and collaborating with entrepreneurs with demonstrated skills and dedicated time for scaling innovations. The following appendices are included: a detailed description of the I-A-T framework, a description of the Embedded Entrepreneur Pilot Program, and a literature scan.
Evaluation of Los Angeles City College’s STEM Pathways Program: Impacts of the STEM Learning Center on Student Outcomes
Los Angeles City College launched the STEM Pathways program in 2016 with funding from the U.S. Department of Education. The intent of the STEM Pathways program was to improve students’ STEM degree completion and transfer to 4-year colleges, particularly for low-income and Latinx students, through a variety of supports. The STEM Learning Center, a drop-in peer tutoring program offering students assistance in STEM courses, was one of the grant’s most-used supports. This report presents findings from a quasi-experimental study to estimate the impact of the STEM Learning Center on STEM course success and continuation in STEM. SRI found positive effects of STEM Learning Center participation on students’ STEM outcomes, in terms of increased STEM credits and STEM continuation.
MaAST: Map Attention with Semantic Transformers for Efficient Visual Navigation
Visual navigation for autonomous agents is a core task in the fields of computer vision and robotics. Learning based methods, such as deep reinforcement learning, have the potential to outperform the classical solutions developed for this task; however, they come at a significantly increased computational load. Through this work, we design a novel approach that focuses on performing better or comparable to the existing learning-based solutions but under a clear time/computational budget. To this end, we propose a method to encode vital scene semantics such as traversable paths, unexplored areas, and observed scene objects–alongside raw visual streams such as RGB, depth, and semantic segmentation masks—into a semantically informed, top-down egocentric map representation. Further, to enable the effective use of this information, we introduce a novel 2-D map attention mechanism, based on the successful multi-layer Transformer networks. We conduct experiments on 3-D reconstructed indoor PointGoal visual navigation and demonstrate the effectiveness of our approach. We show that by using our novel attention schema and auxiliary rewards to better utilize scene semantics, we outperform multiple baselines trained with only raw inputs or implicit semantic information while operating with an 80% decrease in the agent’s experience.
Hypocretin/Orexin Receptor Pharmacology and Sleep Phases
The hypocretins/orexins are two excitatory neuropeptides, alternately called HCRT1 or orexin-A and HCRT2 or orexin-B, that are the endogenous ligands for two G-protein-coupled receptors, HCRTR1/OX1R and HCRTR2/OX2R. Shortly after the discovery of this system, degeneration of hypocretin/orexin-producing neurons was implicated in the etiology of the sleep disorder narcolepsy. The involvement of this system in a disorder characterized by the loss of control over arousal state boundaries also suggested its role as a critical component of endogenous sleep-wake regulatory circuitry. The broad projections of the hypocretin/orexin-producing neurons, along with differential expression of the two receptors in the projection fields of these neurons, suggest distinct roles for these receptors. While HCRTR1/OX1R is associated with regulation of motivation, reward, and autonomic functions, HCRTR2/OX2R is strongly linked to sleep-wake control. The association of hypocretin/orexin with these physiological processes has led to intense interest in the therapeutic potential of compounds targeting these receptors. Agonists and antagonists for the hypocretin/orexin receptors have shown potential for the treatment of disorders of excessive daytime somnolence and nocturnal hyperarousal, respectively, with the first antagonists approved by the US Food and Drug Administration (FDA) in 2014 and 2019 for the treatment of insomnia. These and related compounds have also been useful tools to advance hypocretin/orexin neurobiology.
Measuring the Impact of Trauma-Focused, Cognitive Behavioral Group Therapy with Middle School Students
This study examines differential effects of the Cognitive Behavioral Intervention for Trauma in Schools (CBITS) program on behavioral and academic outcomes of middle school students. Researchers administered screenings to grade 6 students to assess traumatic stress and then randomized those with elevated levels to the CBITS treatment ( n = 150) or comparison group ( n = 143). Analyses examined the overall impact of CBITS and differential effects among subpopulations of students who reported clinically significant externalizing ( n = 73) or internalizing behavior ( n = 181) at baseline. Results demonstrated that relative to counterparts in the comparison group, externalizers in CBITS reported significantly reduced post-traumatic stress, anxiety, anger, internalizing and externalizing problems, and total behavior problems, and they also significantly improved scores on a standardized literacy assessment at posttest. Students with internalizing behavior problems showed differential academic effects at 1-year follow-up; those in CBITS did significantly better on standardized math tests but significantly worse on a reading subtest than their counterparts.
Early Childhood Throughout the COVID-19 Pandemic: The Experiences of Arkansas Educators
The COVID-19 pandemic has transformed the experiences of young children and their caregivers over the past year. SRI Education and the National Center on Children in Poverty partnered with the Arkansas Division of Child Care and Early Childhood Education (DCCECE) to examine early care and education programs throughout the COVID-19 pandemic. This brief, which represents the second of two reports, highlights findings from a second sample of Arkansas (AR) educators who completed surveys and focus groups in spring 2021. It includes information similar to the first report and provides additional information related to vaccination, supports for students with disabilities, and educators’ plans for moving forward.
Our representative survey of Arkansas early educators found that:
The AR early educators who participated in the fall 2020 and spring 2021 studies were consistent in reporting high compliance and agreement with COVID-19 pandemic procedures.
Nearly all AR early childhood educators reported that they changed practices in response to COVID & will continue to implement one or more of these changes even after restrictions are lifted.
Some ECE teachers reported concerns related to children’s ability to focus their attention during group activities and engage in cooperative play.
ECE program directors reported the greatest interest in receiving additional information about vaccines to share with children’s families; ECE teachers reported the greatest interest in whether ECE staff will be required to get the COVID-19 vaccine.
Both ECE directors and teachers also indicated that having someone with expert knowledge to discuss their individual concerns would motivate early educators to become vaccinated.
Scaling Up Coolthink@JC Implementation Study Baseline Report
CoolThink@JC aims to nurture students’ proactive use of technologies for social good from a young age, preparing them for a fast-changing digital future through hands-on, minds-on, and joyful learning experiences. After a successful pilot in 32 schools, CoolThink’s co-creators, led by the Hong Kong Jockey Club Charities Trust (HKJC), have undertaken an ambitious initiative to take CoolThink to scale within Hong Kong, supporting high-quality adoption in 200 primary schools and laying a foundation throughout the system for more widespread adoption. By demonstrating success at scale, CoolThink partners hope to create a new paradigm for CTE at the upper primary level that will serve as an international model for other cities and states.
To capture the lessons learned from this effort, HKJC has engaged SRI International to study the implementation of CoolThink at scale. This implementation study will:
Assess the extent to which schools’ adoption of CoolThink is consistent with the initiative’s design principles and sustained over time,
Identify the conditions that support or impede successful adoption at the classroom and school levels, and
Validate an implementation model that will help interested stakeholders to learn from CoolThink’s scaling experience.
CoolThink partners began scaling CoolThink in summer 2020, when a third cohort of 47 schools joined the first two pilot cohorts in teaching CoolThink lessons. Drawing on data from teacher and school leader surveys administered between November 2020 and January 2021, this baseline report sets the context for the rollout of CoolThink in Cohort 3 schools.
This report is the first in a series from an implementation study being conducted by SRI International (SRI). The purpose of the study is to help stakeholders understand how CoolThink is taking shape in classrooms, schools, and systems, and to offer models for other initiatives as they seek to go to scale. This baseline report, based on surveys of school leaders and teachers prior to their implementation of CoolThink, focuses on conditions for success.
Mid-Scale Research Infrastructure for STEM Education
SRI convened diverse stakeholders in workshops about midscale research infrastructure for STEM education. Participants discussed challenges and opportunities for transforming our STEM education system and what infrastructure is needed to support necessary research.
SRI International convened a diverse group of researchers, practitioners, administrators, developers and non-profit leaders in a series of workshops about midscale research infrastructure for STEM education. Participants discussed major challenges of research and practice, future opportunities for strengthening our STEM education system and pipeline, and what infrastructure is needed to support the research needed to realize these opportunities. Many opportunities involved transformative uses of information technology to create collaborative, immersive learning opportunities and more effective approaches to data collection. New data infrastructure and analysis tools are needed to conduct research across data siloes. This effort was supported by a grant from the National Science Foundation (NSF) in order to generate ideas and inform priorities for the new midscale research infrastructure (MSRI) funding streams.
Maintaining Momentum of 1:1 Programs: Observations from the Apple and ConnectED Initiative
The Apple and ConnectED Initiative provides valuable lessons for practitioners, policymakers, and researchers about what it takes to sustain 1:1 programs in schools serving high concentrations of students facing socioeconomic barriers. SRI’s 6-year study of the initiative illuminates how key factors and conditions came together in some schools to keep their programs going and reveals some important lessons for future endeavors.
Experience has shown that sustaining the progress of 1:1 programs in schools serving under-resourced communities is hard, as technology requires upkeep and the initial burst of focused energy sparked by new initiatives can dissipate. The Apple and ConnectED Initiative was designed with an intentional approach to building a foundation for continued use of technology and to create conditions that would set school communities on a new learning trajectory, leading to continued deepening and expansion of technology use in classrooms. This vision of sustainability involved ramping up the provision of technology and integration support as schools were ready and then removing these scaffolds gradually to allow schools to assume local ownership of their 1:1 programs.
Through SRI International’s (SRI) 6-year study of the initiative, the Apple and ConnectED Initiative provided a unique opportunity to observe how sustainability played out across many schools over an extended time period. This report describes findings from the research about the strategies that schools used to address inevitable challenges to sustainability and what factors and conditions appeared to make a difference. It further addresses the dynamic relationship among these factors and conditions, which can lead to positive reinforcement.
Schools that appeared to maintain momentum of their 1:1 initiatives demonstrated a broad commitment to the program and shared vision for how technology could support instructional goals, strong leadership (often but not exclusively from the principal’s office), and community support. This shared commitment and leadership made it possible to put plans in place for mobilizing resources and devising ways to keep the program going. The continued use of technology and, in some cases, continued growth in practices and community engagement using iPad devices, produced visible benefits for key stakeholders which in turn helped to reinforce commitment.
