This report offers a framework for reviewing CoolThink@JC’s accomplishments to date and computational thinking education (CTE) initiatives in other jurisdictions.
Satabdi Basu
SRI Author
Scaling Up Coolthink@JC Implementation Study Baseline Report
CoolThink@JC aims to nurture students’ proactive use of technologies for social good from a young age, preparing them for a fast-changing digital future through hands-on, minds-on, and joyful learning experiences. After a successful pilot in 32 schools, CoolThink’s co-creators, led by the Hong Kong Jockey Club Charities Trust (HKJC), have undertaken an ambitious initiative to take CoolThink to scale within Hong Kong, supporting high-quality adoption in 200 primary schools and laying a foundation throughout the system for more widespread adoption. By demonstrating success at scale, CoolThink partners hope to create a new paradigm for CTE at the upper primary level that will serve as an international model for other cities and states.
To capture the lessons learned from this effort, HKJC has engaged SRI International to study the implementation of CoolThink at scale. This implementation study will:
Assess the extent to which schools’ adoption of CoolThink is consistent with the initiative’s design principles and sustained over time,
Identify the conditions that support or impede successful adoption at the classroom and school levels, and
Validate an implementation model that will help interested stakeholders to learn from CoolThink’s scaling experience.
CoolThink partners began scaling CoolThink in summer 2020, when a third cohort of 47 schools joined the first two pilot cohorts in teaching CoolThink lessons. Drawing on data from teacher and school leader surveys administered between November 2020 and January 2021, this baseline report sets the context for the rollout of CoolThink in Cohort 3 schools.
This report is the first in a series from an implementation study being conducted by SRI International (SRI). The purpose of the study is to help stakeholders understand how CoolThink is taking shape in classrooms, schools, and systems, and to offer models for other initiatives as they seek to go to scale. This baseline report, based on surveys of school leaders and teachers prior to their implementation of CoolThink, focuses on conditions for success.
Building Teacher Capacity in K–12 Computer Science by Promoting Formative Assessment Literacy
The rapid expansion of computer science (CS) instruction in primary and secondary education has highlighted the shortage of teachers qualified to teach the subject. A key strategy for building CS teaching capacity has been preparing teachers of other subjects (e.g., math, technology applications, business) to teach introductory CS through short-term professional development (PD) workshops or online training modules. However, these professional learning experiences tend to be brief in duration and focused on short-term adoption of specific CS curricula, or technology-enhanced tools at the expense of developing teachers’ conceptual understanding of CS standards, and pedagogy for monitoring and supporting students’ progress toward the standards. This white paper highlights some of the challenges faced by current CS teachers and presents a call to action for states and school districts to support CS teacher capacity building through standards-aligned, sustained, scalable, and reusable teacher PD that focuses on promoting teachers’ CS formative assessment literacy as a way to improve teachers’ ability to effectively teach CS. The corresponding practice guide provides concrete steps to systematically develop or select formative assessment tasks and use them to inform instruction.
Practice Guide: Applying a Principled Approach to Develop and Use K–12 Computer Science Formative Assessments
Formative assessment can be a powerful tool to support effective K-12 computer science (CS) instruction to increase student engagement and improve learning outcomes. In this practice guide, we show how to apply the five-step process outlined in the corresponding white paper to systematically develop or select formative assessment tasks and use them to inform instruction. This guide illustrates formative assessment literacy in practice and can be used in CS professional development workshops, teacher communities of practice, policy guidelines, and other avenues. The guide is not based on any specific curriculum; it can be used by anyone tasked with teaching CS. It is designed to enhance teachers’ understanding of CS standards, determine how to select and implement appropriate formative assessment tasks, and learn how to modify instruction to address student challenges identified from the formative assessments. This process can increase teachers’ knowledge of the CS content and how to teach it, as well as improve student engagement and learning.
Using Computational Modeling to Integrate Science and Engineering Curricular Activities
We articulate a framework for using computational modeling to coherently integrate the design of science and engineering curricular experiences. We describe how this framework informs the design of the Water Runoff Challenge (WRC), a multi-week curriculum unit and modeling environment that integrates Earth science, engineering, and computational modeling for upper elementary and lower middle school students. In the WRC, students develop conceptual and computational models of surface water runoff, then use simulations incorporating their models to develop, test, and optimize solutions to the runoff problem. We conducted a classroom pilot study where we collected students’ learning artifacts and data logged from their use of the computational environment. We illustrate opportunities students had to integrate science, engineering, and computational thinking during the unit in a pair of contrasting vignettes.
A principled approach to designing a computational thinking practices assessment for early grades
In today’s increasingly digital world, it is critical that all students learn to think computationally from an early age. Assessments of Computational Thinking (CT) are essential for capturing information about student learning and challenges. Several existing K-12 CT assessments focus on concepts like variables, iterations and conditionals without emphasizing practices like algorithmic thinking, reusing and remixing, and debugging. In this paper, we discuss the development of and results from a validated CT Practices assessment for 4th-6th grade students. The assessment tasks are multilingual, shifting the focus to CT practices, and making the assessment useful for students using different CS curricula and different programming languages. Results from an implementation of the assessment with about 15000 upper elementary students in Hong Kong indicate challenges with algorithm comparison given constraints, deciding when code can be reused, and choosing debugging test cases. These results point to the utility of our assessment as a curricular tool and the need for emphasizing CT practices in future curricular initiatives and teacher professional development.
Leveraging Evidence-Centered design to develop assessments of Computational Thinking practices
Computational thinking is a core skill in computer science that has become a focus of instruction in primary and secondary education worldwide. Since 2010, researchers have leveraged Evidence-Centered Design (ECD) methods to develop measures of students’ Computational Thinking (CT) practices. This article describes how ECD was used to develop CT assessments for primary students in Hong Kong and secondary students in the United States. We demonstrate how leveraging ECD yields a principled design for developing assessments of hard-to-assess constructs and, as part of the process, creates reusable artifacts—design patterns and task templates—that inform the design of other, related assessments. Leveraging ECD, as described in this article, represents a principled approach to measuring students’ computational thinking practices, and situates the approach in emerging computational thinking curricula and programs to emphasize the links between curricula and assessment design.
A Principled Approach to Designing Assessments That Integrate Science and Computational Thinking
There is increasing interest in broadening participation in computational thinking (CT) by integrating CT into precollege STEM curricula and instruction. Science, in particular, is emerging as an important discipline to support integrated learning. This highlights the need for carefully designed assessments targeting the integration of science and CT to help teachers and researchers gauge students’ proficiency with integrating the disciplines. We describe a principled design process to develop assessment tasks and rubrics that integrate concepts and practices across science, CT, and computational modeling. We conducted a pilot study with 10 high school students who responded to integrative assessment tasks as part of a physics-based computational modeling unit. Our findings indicate that the tasks and rubrics successfully elicit both Physics and CT constructs while distinguishing important aspects of proficiency related to the two disciplines. This work illustrates the promise of using such assessments formatively in integrated STEM and computing learning contexts.
Measuring Student Learning in Introductory Block-Based Programming: Examining Misconceptions of Loops, Variables, and Boolean Logic
Programming in block-based environments is a key element of introductory computer science (CS) curricula in K-12 settings. Past research conducted in the context of text-based programming points to several challenges related to novice learners’ understanding of foundational programming constructs such as variables, loops, and expressions. This research aims to develop assessment items for measuring student understanding in introductory CS classrooms in middle school using a principled approach for assessment design. This paper describes the design of assessments items that were piloted with 100 6th, 7th, 8th graders who had completed an introductory programming course using Scratch. The results and follow-up cognitive thinkalouds indicate that students are generally unfamiliar with the use of variables, and harbor misconceptions about them. They also have trouble with other aspects of introductory programming such as how loops work, and how the Boolean operators work. These findings point to the need for pedagogy that combines popular constructionist activities with those that target conceptual learning, along with better professional development to support teachers’ conceptual learning of these foundational constructs.
