Jessica L. Mislevy

This report describes the methodology of Apple and ConnectED Research, a six-year study of the Apple and ConnectED Initiative that uses a variety of qualitative and quantitative methods to tell a comprehensive story of implementation and outcomes.
This report describes the research design of the Apple and ConnectED Initiative, beginning with discussions of the theory of change and the conceptual framework that ground the research and an introduction to the Apple and ConnectED Initiative on which the research focuses. The report then describes the research design and methodology, which employ complementary substudies to create research that is both broad and deep. The substudies include:

Surveys of teachers, principals, and students
Case studies of selected schools
Analysis of lessons and student work to determine opportunities for deeper learning
A student achievement study that uses state test data to investigate student learning outcomes

This report is intended for researchers and other technical audiences interested in understanding the methodological details that underly the Apple and ConnectED Research study. It complements a series of reports that present study results and discuss findings.

Through the lens of the Apple and ConnectED Initiative, this report asks the questions, what does a promising start look like when you add technology to education and what types of support can enable conditions for success?
Technology, such as iPad™ devices for students and teachers, has the potential to energize classrooms and bring substantially new types of learning opportunities to children of all ages. These changes are not an automatic result of adding technology to education, and they often take place over a long period of time. This report asks the questions, what does a promising start look like and what types of support can enable conditions for success?
We ask these questions through the lens of the Apple and ConnectED Initiative, which has been the focus of a rigorous 6-year research study. Launched in 2014, the Apple and ConnectED Initiative has supported 114 participating schools across the country with an iPad for every student. Schools received a host of programmatic supports including extensive professional learning opportunities for teachers and leaders, technology infrastructure upgrades, and process management.
The initiative and this research are explicitly situated in a diversity of traditionally under-resourced communities, with schools ranging from pre-K to secondary and from the inner city to rural migrant communities to Native American villages. This report focuses on the first year of iPad use across these schools to describe the initial changes that might be expected to appear when sufficient support is in place to lower common barriers to its adoption.
The report describes early implementation in three successive stages:

Access: Many of the ConnectED schools saw daily iPad use across multiple subjects, even early in implementation. This level of use was facilitated by strategic and technical preparation prior to the introduction of the iPad devices, coupled with initial strategies for their instructional application. Daily widespread use demonstrated how universal technology access has the potential to “level the playing field” and broaden students’ horizons.
Integration: In classrooms where iPad use had become the norm, the learning environments looked different from those in typical classroom settings. In particular, iPad classrooms leveraged immediate access to rich information, offered new opportunities for expression, used technology to increase student engagement, and benefited from more organized and efficient workflows.
Innovation: In addition to more active and engaging learning environments, technology is often seen as holding promise to facilitate meaningful changes to students’ opportunities for learning. This study uses a framework for “deeper learning” to describe emerging opportunities for teamwork, critical thinking, and other skills that prepare students for success beyond the classroom. Teacher surveys and a review of lesson plans reveal some initial steps toward deeper learning for a broad range of teachers, particularly in the dimensions of personalization and communication/creation that were most directly enabled by the affordances of the new iPad devices. More advanced opportunities require careful and creative lesson planning, and were most likely to be seen in the classrooms of teachers who held deeper learning as an explicit goal.

The Virginia Department of Education’s Virtual Virginia (VVA) program delivers more than 90 supplemental online courses to more than 10,000 middle and high school students each year. The department partnered with Regional Educational Laboratory (REL) Appalachia to examine the relationship between students’ participation in a subset of 31 VVA online courses and their performance on end-of-course Virginia Standards of Learning (SOL) and Advanced Placement (AP) exams during the 2015/16 and 2016/17 school years. Relying on administrative records available through the Virginia Longitudinal Data System, REL Appalachia researchers conducted descriptive analyses and regression analyses with statistical controls. After controlling for demographic and prior achievement differences, students in face-to-face courses were more likely to score higher and demonstrate proficiency on SOL and AP exams than students taking the same courses through VVA across most subject areas. Because the study design was correlational, additional research is needed to determine whether the difference in performance is due to the characteristics of the students in online courses, aspects of the VVA program itself, or some other factor such as students’ reasons for enrolling in the course or the supports available in their local schools.