SRI Authors: Rebecca Griffiths, Claire Christensen Abstract There are few published studies investigating the effectiveness of hybrid formats at the program level in graduate legal education. A hybrid Juris Doctorate (J.D.) program launched by a Midwestern institution was the first ABA-accredited law degree program with a substantial online learning component. This study takes a mixed […]
Claire Christensen
SRI Author
Sparking Connections: Evaluations of Mobile Messaging on Responsive Caregiving
SRI Authors Elisa Garcia, Sarah Nixon Gerard, Claire Christensen, Tejaswini Tiruke, Carolina Zamora, Todd A. Grindal Abstract Mobile messaging programs are a low-cost, scalable approach to building parents’ knowledge and capacity to support their children’s development. These programs directly deliver simple and straightforward information, tips, and activities that parents can incorporate into daily routines. Yet […]
The Efficacy of Digital Media Resources in Improving Children’s Ability to Use Informational Text: An Evaluation of Molly of Denali from PBS KIDS
Informational text—oral or written text designed to inform—is essential to daily life and fundamental to literacy. Unfortunately, children typically have limited exposure to informational text. Two nine-week randomized controlled trials with a national sample of 263 first-graders examined whether free educational videos and digital games supported children’s ability to use informational text to answer real-world questions. Participants received data-enabled tablets and were randomly assigned to condition. Study 1 found significant positive intervention impacts on child outcomes; Study 2 replicated these findings. Combined analyses demonstrated primary impact on children’s ability to identify and use structural and graphical features of informational text. Results are discussed in the context of the scalability of educational media to support informational text learning.
A pre-print version of the manuscript submitted to the American Educational Research Journal is available on ResearchGate.
Measuring Chinese middle school students’ motivation using the Reduced Instructional Materials Motivation Survey (RIMMS): A validation study in the adaptive learning setting
Valid measures of student motivation can inform the design of learning environments to engage students and maximize learning gains. This study validates a measure of student motivation, the Reduced Instructional Materials Motivation Survey (RIMMS), with a sample of Chinese middle school students using an adaptive learning system in math. Participants were 429 students from 21 provinces in China. Their ages ranged from 14 to 17 years old, and most were in 9th grade. A confirmatory factor analysis (CFA) validated the RIMMS in this context by demonstrating that RIMMS responses retained the intended four-factor structure: attention, relevance, confidence, and satisfaction. To illustrate the utility of measuring student motivation, this study identifies factors of motivation that are strongest for specific student subgroups. Students who expected to attend elite high schools rated the adaptive learning system higher on all four RIMMS motivation factors compared to students who did not expect to attend elite high schools. Lower parental education levels were associated with higher ratings on three RIMMS factors. This study contributes to the field’s understanding of student motivation in adaptive learning settings.
Digital media can help preschoolers learn real-world science skills
Young children spend about two hours each day using screen-based media, about half of which is spent on educational media, according to their parents. Many studies report that children can learn a range of skills from well-designed educational media. Yet we know relatively little about whether and how well children are able to apply skills they’ve learned from digital media in the real world. This question is particularly important for subjects that involve learning about the physical world, like science. There is a small amount of evidence that children can learn science from media. At the same time, digital media differ from the real world in ways that may be challenging for children to reconcile: digital science media are two-dimensional, are often cartoon-like or anthropomorphized, and frequently focus on refuting misconceptions rather than teaching science facts. Yet there may be ways to design science media to support children in connecting their learning to the real world. A recent study conducted by Education Development Center (EDC) and SRI Education found that four- to five-year-old children can apply science skills that they learned from digital media in the real world. The study was commissioned by the U.S. Department of Education’s Ready To Learn initiative, led by the Corporation for Public Broadcasting and PBS.
Photo: Burt Granofsky, EDC. A mother and child play with household objects inspired by the Cat in the Hat digital app.
n this randomized-control trial, researchers assigned 454 four- and five-year olds from low-income households to either a treatment or control group. Both groups received tablet computers with internet access. The treatment group’s tablets were loaded with resources from the third season of The Cat in the Hat Knows A Lot About That! including videos, games, and hands-on/real world activities. Researchers asked the control group to use other educational media of their own choosing. Researchers asked families in both groups to have their children use the resources for one hour per week over eight weeks.
At the beginning and the end of the study, researchers assessed the children’s science and engineering knowledge and practices using several measures, including hands-on performance-based tasks to assess their understanding in three areas:
structural stability,
forces and friction, and
how objects can be sorted based on material properties and uses.
The structural stability task provided children with a group of objects of different strengths and lengths, and asked them to choose the most suitable object for building a bridge that can support weight. The forces and friction task provided children with three differently textured slides and asked them to choose the slide that would enable a toy figure to slide down fastest. The sorting task asked children to sort items based on material properties such as color, size, shape, and use.
Photo: Burt Granofsky, EDC
Researchers found that access to Cat in the Hat resources had medium to large impacts on children’s understanding of the role of strength and length in structural stability and the influence of friction on movement down an incline. This finding suggests that young children’s experiences with manipulating objects in digital games and watching characters manipulate objects in digital videos can, in fact, transfer to off-screen contexts. Meanwhile, the study found no significant effect on the sorting task, which suggests that some skills may be more difficult for preschoolers to transfer from digital to hands-on contexts.
What helped preschoolers transfer their learning from digital videos and games to two of the hands-on tasks used in this study? The study did not attempt to disentangle which aspects of the resources were most effective, but we speculate a few reasons based on prior research.
First, the Cat in the Hat resources used in this study—videos, games, and hands-on activities—were intentionally developed to support young children’s understanding of physical science and engineering. The resources are aligned with the K-12 Next Generation Science Standards and the Head Start Early Learning Outcomes Framework. They focus on physical science and engineering concepts, which lend themselves to visual depiction. Each resource focuses on one or two focal concepts. Characters model the associated practices and vocabulary realistically, even though they are often in fanciful contexts. This focused, deliberate approach may support children’s deeper learning.
In addition, children found the Cat in the Hat materials to be engaging and fun, evidenced by the fact that over half of the children in the treatment group (59%) engaged with the Cat in the Hat games and videos for more than 20 hours over the eight week study. Parents also reported that treatment children watched the Cat in the Hat videos on other platforms, such as Netflix and YouTube, and a few parents said their children mimicked the activities they saw in the media with their toys. Children who are more engaged may learn more: researchers found some evidence that the more time children watched the “Bridge-a-rama” videos and played the “Slidea-ma-zoo” games, the more they outperformed our predictions on the related assessments, although this relationship was not linear and weakened at very high levels of use. Why did families find the materials engaging? It could be the show’s entertaining narratives. Studies show that children are more likely to retain educational concepts if the digital media follow a plot or a narrative and are therefore able to keep them engaged. The Cat in the Hat himself may also have engaged children—children tend to learn more from characters they know and like.
Photo: Burt Granofsky, EDC. Children play with household objects inspired by the Cat in the Hat digital app.
Further, children may have benefitted from the inclusion of both videos and digital games in this study as both have strengths for supporting learning. Studies have found that young children can transfer skills from game-like digital activities to the real world. Videos may be even more effective than games for supporting transfer of hard-to-learn skills, perhaps because playing games demands more working memory capacity. Games may be more effective for helping preschoolers transfer their learning to similar tasks, whereas videos may be more effective for helping preschoolers transfer their learning to different tasks. The mix of both videos and games in this study may have exposed children to the same ideas in multiple ways, supporting their ability to think flexibly about science content.
This study suggests that children may be able to apply skills learned from high-quality digital STEM media in off-screen, hands-on situations. Based on prior research, we speculate that media can help children connect digital to real-world learning by providing focused educational content in the context of engaging narratives that span digital videos and games.
PBS KIDS Play & Learn Science | ready to learn-funded evaluation
This study explored the impact of the PBS KIDS Play & Learn Science app, when used in a supportive context, on children’s understanding of science concepts and use of science and engineering practices; children’s use of science vocabulary; child and parent-child engagement in science and engineering; and parent confidence supporting their child’s science learning.
This study was commissioned by the U.S. Department of Education’s Ready To Learn Initiative, led by the Corporation for Public Broadcasting and PBS. It was conducted by EDC and SRI International, long-standing evaluation partners for the Ready To Learn Initiative.
Executive Summary | PBS Kids Play & Learn Science Evaluation
This evaluation of the PBS KIDS Play & Learn Science app and activities, summarizes impacts on children and their parents, including children’s understanding of science content and practices, use and understanding of science vocabulary, and excitement about science.
Evaluation Flyer | PBS Kids Play & Learn Science
This infographic shows key facts & figures from an evaluation of the PBS KIDS Play & Learn Science app and activities, including impacts on children’s understanding of science content and practices and use of science vocabulary.
Full Evaluation Report | PBS Kids Play & Learn Science
This comprehensive report includes in-depth analysis and graphs of impacts on children and their parents, including children’s understanding of science content and practices, use and understanding of science vocabulary, and excitement about science.
Getting Ready to Learn: Creating Effective, Educational Children’s Media
Getting Ready to Learn describes how educational media have and are continuing to play a role in meeting the learning needs of children, parents, and teachers. Based on years of meaningful data from the CPB-PBS Ready To Learn Initiative, chapters explore how to develop engaging, playful, and developmentally appropriate content. From Emmy-Award-winning series to randomized controlled trials, this book covers the media production, scholarly research and technological advances surrounding some of the country’s most beloved programming.
Why does the rain fall down instead of up? How parents support science learning, and how media can help
In Jackson, Mississippi, researchers visited a family of seven (including a niece and nephew), headed by a stay-at-home mom in her 20s. During the visit, the children sat around the mom as she described their latest learning activities, including 1-on-1 homework time and making “slime.” One activity—planting a seed in the yard in front of their apartment building—garnered a lot of excitement. Here’s how the mom described her family’s experience.
“[My son] wanted to know, ‘How did that tree get in the ground?’ … What I did was, like, when I was in school with my teacher, we grew a plant. So that’s what we’re doing in my house now, growing a plant, so he can see. I told him it wouldn’t get as big as the trees are, but he could see it grow.
I Googled ‘How to plant a plant’… to make sure that I was doing it right. I hadn’t done it in a while, so I wanted to make sure it would sprout. … Google always leads you to YouTube. So we looked at a video showing us how different people started planting different types of things, and how to take care of them. … I wanted to make sure that it was getting everything that it needed.”
Photo: Burt Granofsky, EDC
Science-related media have the potential to engage and support families in exploring science together, but are families using media for this purpose? A new study examined the ways that families think about and engage young children in science, and how they use science-related media to support these efforts. The study was conducted by EDC and SRI International and commissioned by the U.S. Department of Education’s Ready To Learn initiative, led by the Corporation for Public Broadcasting and PBS. It included a nationally-representative survey of over 1,400 parents and caregivers of 3- to 6-year-old children, and interviews with 65 parents.
Findings from the survey suggest that most parents feel responsible for helping their children learn, and that they do so daily. Most parents feel very confident helping their young children learn social and behavioral skills, math, and literacy. But only about half of parents—and even fewer low-income parents—feel very confident helping their children learn science.
Not only are parents less likely to be very confident in doing science with their children, they are also less likely to do science with their children daily, compared to other skills. When asked about learning activities they do with their children, parents were most likely to report that they read, involve their children in chores, and work on math concepts with their children every day; far fewer parents report doing science learning activities daily. Many parents, especially low-income parents, say that more ideas and resources would help them to do more science with their children.
EDC photo/Burt Granofsky.
Media could fill that need. Close to two-thirds of parents reported that their child had watched science-related TV shows or videos weekly or more often in the past month, and about half of parents said their children played science-related apps or video games about science weekly or more often.
Based on these findings, we offer some suggestions for media producers—and others who develop resources for families—to help families with young children explore science together.
Finding: Fewer parents are confident in their ability to help children learn science, compared with literacy, math, and social skills.
If families think science is hard, they may not do it. We know from prior research that media can play a role in helping parents support their young children’s learning —especially for parents who are less confident about their abilities to help their children. Here are three ways media can help overcome this confidence barrier.
Show that science is fun. For example, playfully pushing a cart down a hill or joyfully banging on pots and pans to see which ones are louder when hit with a wooden spoon.
Explore, don’t answer. Children ask a lot of science questions, and adults may feel intimidated when they can’t answer them. For example, parents in our study described their children asking questions such as, “Why does the rain fall down instead of up?” and “Do trees breathe?” Media can flip the script by showing adults and children exploring together—observing, collecting data, and sharing their ideas about the how and why of things. It is important for parents to see that science learning is not about providing answers; it’s about discovering together.
Provide relatable family science role models. Families engaged in science activities in TV shows and games should represent the racial, ethnic, cultural, and geographic diversity of the target audience. For example, not all families may be able to collect insects in their backyards, but nature can be found in a single tree, or a patch of dirt by the sidewalk.
EDC photo/Burt Granofsky.
Findings: Nine out of ten parents report doing learning activities with their children daily. About half of parents report doing science-related activities with their children daily. Parents indicated that easy-to-do ideas for science activities would help them do more science.
Media can model or encourage everyday ways to engage children in science. Media can also remind families that they probably already do science without realizing it.
Show that science is everywhere. Science doesn’t require special supplies or set-up. Media can show characters listening for birds while running errands, building with empty cartons and containers, or exploring rolling and sliding with a piece of cardboard and different-shaped household objects.
Finding: Most parents do not feel that their child is learning science from media.
Our interview data suggest that parents may not always recognize the educational value of science media. Clear science content may help parents recognize and support what their children are learning. Here are three research-based strategies.
Begin with a preview of the science activities and concepts featured in the episode (an advance organizer). For example, “Today we’re building things. We will see what makes some things stable, and some things wobbly.” This introduction can help children and parents focus on the educational content. Similarly, end with a wrap-up of what the characters did and found out related to the science content.
Integrate science content into the narrative, rather than saving it for interstitials. This approach helps parents and children to engage in the science content throughout the episode. It also helps families realize that they naturally experience science as part of life, much like the characters do.
Raise the same science concepts in different ways, using multiple examples throughout the episode or game. Use a variety of visuals, sound, spoken language, print text, and/or other symbolic representations to reinforce concepts. For example, an episode about habitats could include pictures of a habitat, language that describes the habitat, and text or symbols to represent different animals or habitats.
Finding: About 30% of parents do not talk to their child about connections between science media and science in everyday life.
Media can support these conversations by providing discussion prompts for parents and children to use during or after the episode or game. Here are three tips for writing discussion prompts.
Keep conversations open-ended. Focus on topics that are interesting to children, rather than on specific facts. For example, identify each family member’s favorite colors on objects around you.
Encourage children to relate content to something else they have done or observed, since prior knowledge improves children’s comprehension. For example, “That episode/game was about plants. What plants do we see around where we live?”
Launch related explorations in the real world. For example, a prompt might state, “This episode/game was about how the parts of a plant help it grow. Let’s find (or draw) a plant and look for its different parts.”
Media have a strong potential to support science learning at home. Families are already using science media, although they may not consider its educational value. If crafted intentionally, science media can engage parents in co-learning with their children, boost parents’ confidence in doing science, and suggest ways to continue learning off-screen.
Early Experiences with EdReady— A Tool for Improving Math College Readiness
The Bill & Melinda Gates Foundation’s Postsecondary Success initiative is concerned with understanding how online technology applications can assess and improve students’ math readiness to better prepare them for college-level math courses and postsecondary success. As part of this initiative, this first report of SRI’s EdReady evaluation summarizes emerging lessons in EdReady adoption and implementation, with particular focus on prevalent uses of EdReady and on statewide adoptions. The report identifies six instructional use cases and examines four statewide adoptions in progress: Hawaii, Montana, Nevada, and North Carolina. Through interviews, observations of EdReady use, and review of pilot reports, SRI identified five indicators of potential readiness for EdReady adoption and five lessons learned about strong implementations.
