SRI International

SRI International - American Nonprofit Research Institute

Home » Archives for Haiwen Wang

Haiwen Wang

SRI Author

Education & learning publications

Scaling Up Coolthink@JC Implementation Study Baseline Report

, , ,

CoolThink@JC aims to nurture students’ proactive use of technologies for social good from a young age, preparing them for a fast-changing digital future through hands-on, minds-on, and joyful learning experiences. After a successful pilot in 32 schools, CoolThink’s co-creators, led by the Hong Kong Jockey Club Charities Trust (HKJC), have undertaken an ambitious initiative to take CoolThink to scale within Hong Kong, supporting high-quality adoption in 200 primary schools and laying a foundation throughout the system for more widespread adoption. By demonstrating success at scale, CoolThink partners hope to create a new paradigm for CTE at the upper primary level that will serve as an international model for other cities and states.
To capture the lessons learned from this effort, HKJC has engaged SRI International to study the implementation of CoolThink at scale. This implementation study will:

Assess the extent to which schools’ adoption of CoolThink is consistent with the initiative’s design principles and sustained over time,
Identify the conditions that support or impede successful adoption at the classroom and school levels, and
Validate an implementation model that will help interested stakeholders to learn from CoolThink’s scaling experience.

CoolThink partners began scaling CoolThink in summer 2020, when a third cohort of 47 schools joined the first two pilot cohorts in teaching CoolThink lessons. Drawing on data from teacher and school leader surveys administered between November 2020 and January 2021, this baseline report sets the context for the rollout of CoolThink in Cohort 3 schools.
This report is the first in a series from an implementation study being conducted by SRI International (SRI). The purpose of the study is to help stakeholders understand how CoolThink is taking shape in classrooms, schools, and systems, and to offer models for other initiatives as they seek to go to scale. This baseline report, based on surveys of school leaders and teachers prior to their implementation of CoolThink, focuses on conditions for success.

Education & learning publications

The Apple and ConnectED Initiative: Research Study Methods

, , , , , ,

This report describes the methodology of Apple and ConnectED Research, a six-year study of the Apple and ConnectED Initiative that uses a variety of qualitative and quantitative methods to tell a comprehensive story of implementation and outcomes.
This report describes the research design of the Apple and ConnectED Initiative, beginning with discussions of the theory of change and the conceptual framework that ground the research and an introduction to the Apple and ConnectED Initiative on which the research focuses. The report then describes the research design and methodology, which employ complementary substudies to create research that is both broad and deep. The substudies include:

Surveys of teachers, principals, and students
Case studies of selected schools
Analysis of lessons and student work to determine opportunities for deeper learning
A student achievement study that uses state test data to investigate student learning outcomes

This report is intended for researchers and other technical audiences interested in understanding the methodological details that underly the Apple and ConnectED Research study. It complements a series of reports that present study results and discuss findings.

Education & learning publications

1:1 Teaching and Learning in the Apple and ConnectED Initiative: Lessons from Early Implementation

, , , , , ,

Through the lens of the Apple and ConnectED Initiative, this report asks the questions, what does a promising start look like when you add technology to education and what types of support can enable conditions for success?
Technology, such as iPad™ devices for students and teachers, has the potential to energize classrooms and bring substantially new types of learning opportunities to children of all ages. These changes are not an automatic result of adding technology to education, and they often take place over a long period of time. This report asks the questions, what does a promising start look like and what types of support can enable conditions for success?
We ask these questions through the lens of the Apple and ConnectED Initiative, which has been the focus of a rigorous 6-year research study. Launched in 2014, the Apple and ConnectED Initiative has supported 114 participating schools across the country with an iPad for every student. Schools received a host of programmatic supports including extensive professional learning opportunities for teachers and leaders, technology infrastructure upgrades, and process management.
The initiative and this research are explicitly situated in a diversity of traditionally under-resourced communities, with schools ranging from pre-K to secondary and from the inner city to rural migrant communities to Native American villages. This report focuses on the first year of iPad use across these schools to describe the initial changes that might be expected to appear when sufficient support is in place to lower common barriers to its adoption.
The report describes early implementation in three successive stages:

Access: Many of the ConnectED schools saw daily iPad use across multiple subjects, even early in implementation. This level of use was facilitated by strategic and technical preparation prior to the introduction of the iPad devices, coupled with initial strategies for their instructional application. Daily widespread use demonstrated how universal technology access has the potential to “level the playing field” and broaden students’ horizons.
Integration: In classrooms where iPad use had become the norm, the learning environments looked different from those in typical classroom settings. In particular, iPad classrooms leveraged immediate access to rich information, offered new opportunities for expression, used technology to increase student engagement, and benefited from more organized and efficient workflows.
Innovation: In addition to more active and engaging learning environments, technology is often seen as holding promise to facilitate meaningful changes to students’ opportunities for learning. This study uses a framework for “deeper learning” to describe emerging opportunities for teamwork, critical thinking, and other skills that prepare students for success beyond the classroom. Teacher surveys and a review of lesson plans reveal some initial steps toward deeper learning for a broad range of teachers, particularly in the dimensions of personalization and communication/creation that were most directly enabled by the affordances of the new iPad devices. More advanced opportunities require careful and creative lesson planning, and were most likely to be seen in the classrooms of teachers who held deeper learning as an explicit goal.

Education & learning publications

Scaling Up Teacher Induction: Implementation and Impact on Teachers and Students

, , ,

Beginning teachers enter a profession that places particularly challenging demands on novice practitioners. The New Teacher Center’s (NTC) induction model provides intensive, instructionally focused coaching to teachers during their first two years in the classroom, in-depth training for induction mentors, a suite of tools to guide coaching cycles, and capacity-building for district leaders to sustain induction mentoring programs after NTC’s direct involvement in the district ends.
With funding from a U.S. Department of Education Investing in Innovation (i3) scale-up grant, NTC tested strategies for scaling its validated induction model to 301 schools in five school districts serving high proportions of students of color and students from low-income households. NTC adapted its model to support district adoption at scale, including an option for deploying part-time, school-based mentors, reduced requirements for mentor training, and online training and video-sharing tools.
SRI’s evaluation of the implementation and impact of NTC’s i3 scale-up grant employed a cluster-randomized controlled trial design with schools as the unit of randomization. All first-year teachers in randomized schools were included in the study. Treatment teachers received induction supports from NTC-trained mentors, while control teachers received the supports provided by their districts under business-as-usual conditions.
The evaluation examined fidelity of implementation to the model as designed, the contrast between the induction supports in the treatment and control conditions, and impacts on three key outcomes: 1) teachers’ classroom practice as measured by the Danielson Framework for Teaching, 2) student achievement on state standardized assessments in mathematics and English Language Arts (ELA) in grades 4 through 8, and 3) teacher retention within district.
NTC’s induction model was not implemented with adequate fidelity in any of the five sites according to thresholds set by NTC, and the mentoring received by NTC treatment teachers was not substantially different in key respects from the mentoring received by control teachers. There were no statistically significant impacts of the model as implemented on overall teacher practice, student achievement, or teacher retention.
Exploratory findings suggest conditions under which NTC might see a greater impact. There was a positive correlation between students’ mathematics achievement and mentoring that met NTC’s fidelity thresholds for frequency and duration, as well as between mathematics achievement and mentoring that met NTC’s expectations for instructional focus. NTC induction supports also had a positive impact on student ELA achievement in schools with higher proportions of historically underserved students.
These findings indicate the importance of ensuring high-quality implementation of a program. Under the i3 scale-up grant, NTC attempted to adapt its model for scaling, but the partner districts failed to fully implement key components and mediators as intended. There is evidence that the model has promise when fully implemented, particularly in schools with higher proportions of historically underserved students, but without further research this evidence is simply suggestive.

Education & learning publications

Elementary English learner classroom composition and academic achievement: The role of classroom-level segregation, number of english proficiency levels, and opportunity to learn

Using mixed methods, we investigated (a) the association of the extent of English learner (EL) classroom-level segregation (proportion EL) and number of EL English proficiency levels with elementary EL academic achievement, using 2 years of administrative data, and (b) school staff–reported opportunity to learn–related advantages and disadvantages in segregated versus integrated compositions, using 3 years of interviews. Findings were corroborative across methods. After accounting for student-, classroom-, and school-level covariates, we found that ELs in more segregated classrooms exhibited lower performance, on average, on state tests of English language arts, mathematics, and English proficiency, and little evidence that classroom number of EL English proficiency levels was related to achievement. School staff consistently detailed the instructional, academic, and socio-emotional opportunities to learn afforded by the diversity/heterogeneity of integrated classrooms.

Education & learning publications

Broadening participation in STEM college majors: Effects of attending a STEM-focused high school

,

To increase participation in science, technology, engineering, and mathematics (STEM) studies and careers, some states have promoted inclusive STEM high schools. This study addressed the question of whether these high schools improve the odds that their graduates will pursue a STEM major in college. State higher education records were obtained for students surveyed as seniors in 23 inclusive STEM high schools and 19 comparison schools without a STEM focus. Propensity score weighting was used to ensure that students in the comparison school sample were very similar to those in the inclusive STEM school sample in terms of demographic characteristics and Grade 8 achievement. Students overall and from under-represented groups who had attended inclusive STEM high schools were significantly more likely to be in a STEM bachelor’s degree program two years after high school graduation. For students who entered two-year colleges, on the other hand, attending an inclusive STEM high school was not associated with entry into STEM majors.

Education & learning publications

A longitudinal study of the impact of attending an inclusive STEM high school: The case for using two comparison groups

Policymakers argue that only by enlarging the science, technology, engineering, and mathematics (STEM) pipeline in a way that attracts, supports, and sustains the participation of students from all kinds of backgrounds can the United States meet its needs for science and technology innovation, economic prosperity, and social well-being (National Academies, 2005). To meet this need, inclusive STEM high schools (ISHSs) combine rich STEM course offerings and experiences with an explicit mission to serve students from under-represented groups accepted on the basis of interest rather than competitive examination. One of the distinctive features of the present ISHS study is that it provides a comprehensive picture of the impact of ISHSs by using two sets of comparison groups: schools in the same districts as the ISHSs to control for local context; and comparable schools in districts with no access to STEM schools to alleviate potential bias caused by student self-selection into ISHSs. The two comparisons validate each other in providing solid evidence regarding the impact of ISHSs. This study addresses the following research questions: (1) Do students attending ISHSs differ from students in other same-district high schools in terms of demographic characteristics and middle school achievement? and (2) Is there evidence of an impact of ISHS attendance on students’ persistence to 12th grade, high school graduation, and college readiness and aspirations? The data indicates that North Carolina ISHSs served a diverse set of students. Compared with students in the same districts, ISHS students had slightly lower incoming academic achievement and were more likely to be African American and to come from low-income households. Within- and out-of-district comparisons provide consistent findings on the impact of ISHS attendance. ISHS attendance appears to have a positive impact on students’ persistence to 12th grade, high school graduation, and college readiness and aspirations. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]

Teaching quality publications

A comprehensive model of teacher induction: Implementation and impact on teachers and students evaluation of the New Teacher Center’s i3 validation grant, final report

, , ,

SRI Education’s evaluation of the New Teacher Center’s (NTC’s) Investing in Innovation (i3) Validation grant examined the impact of the NTC induction model on teacher practice and student achievement. To account for different local contexts and needs, the study used randomized controlled trials (RCTs) in two districts—Broward County Public Schools and Chicago Public Schools—and a quasi-experimental design in Grant Wood Area Education Agency, a consortium of districts in Iowa. In the RCT districts, the study found positive impacts of the induction model on student achievement in English language arts and mathematics. The study found no statistically significant differences between NTC-supported teachers and comparison teachers on teacher practice measures or on teacher retention into their third year of teaching.
However, the high implementation fidelity levels and contrasts in induction experiences between treatment and comparison teachers indicate that the NTC induction model can be implemented well in a range of district contexts.

Education & learning publications

Making english learner reclassification to fluent english proficient attainable or elusive: When meeting criteria is and is not enough

Employing longitudinal, multisite comparative mixed-methods, we describe patterns of reclassifying and not reclassifying eligible English learner (EL) students to fluent English proficient, and we identify factors impeding and facilitating reclassification. Analyses of administrative data for 7 cohorts of students over 3 years in one district and 2 years in another revealed a considerable number of students meeting all criteria, and a nonoverlapping group meeting at least standardized-test criteria, were not reclassified, implicating this practice in the production of long-term EL status. However, the rate of not reclassifying was more than 2 to 5 times higher in one district than the other. Analyses of policy documents, reclassification decision forms, staff interviews, and surveys revealed that undergirding these differences were distinct reclassification policies and practices. We discuss policy, practice, and research implications, including issues of opportunity to learn, equity, and the need for a common definition of ELs involving a common set of specific criteria and standardized processes for reclassification statewide.