Despite recent increases in enrollment, women and minorities remain underrepresented in those attaining computer science (CS) degrees and entering CS careers. The low number of students taking advanced placement CS exams in high school, which is often used as a rough measure of student interest in pursuing CS in college, shows that we have a long way to go to broaden participation in the expanding workforce for computer science.
To address this challenge, the National Science Foundation (NSF)-funded Into the Loop project developed Exploring Computer Science (ECS), a new high school curriculum that focuses on developing students’ problem-solving and computational thinking skills. ECS was designed to support the creation a more diverse pipeline to CS education and careers by targeting urban high schools and expanding the participation of traditionally underrepresented students in introductory computer science.
ECS is spreading rapidly in response to demand from school districts nationwide, from industry and professional organizations, and from CS education leaders. Teachers implementing the curriculum need high-quality assessments of student learning so they can understand their students’ computational thinking skills and help student’s learn those skills. Assessments support the adoption of a curriculum because, without high-quality assessments, teachers can’t make accurate claims about student learning or identify where students need to improve, and school and district leaders cannot make accurate judgments about the value and place of the curriculum in their course offerings.
Principled Assessment of Computational Thinking (PACT)
As part of the NSF-funded Principled Assessment of Computational Thinking ( PACT) suite of projects, SRI Education has been working with curriculum authors and teachers, assessment experts, and computer scientists to develop assessments for ECS.
ECS emphasizes inquiry-based teaching to develop students’ problem solving skills, as well as their abilities to explain, elaborate, and evaluate what they are learning, often using multiple representations of particular solutions. These skills go well beyond recalling facts or giving inputs to a program and predicting its outputs. As a result, the SRI PACT team had to design and develop assessment tasks that elicited students’ problem solving and inquiry skills in authentic contexts and gave them opportunities to represent their skills in their own words and ways.
Applying a principled design method, the team first developed generalized design templates for computational thinking practices. These practices refer to how students design and implement creative solutions and artifacts, how they design and apply abstractions and models, and how they analyze their computational work and the work of others (among other practices). We then used these templates to guide the development of assessment tasks and scoring rubrics aligned with the skills related to the learning goals of the ECS curriculum.
The assessments were piloted in the 2013-2014 and 2014-2015 school years. The results of each pilot, along with continual feedback from our panels of expert advisors, were used to further refine the assessments prior to their release.
Exploring Computer Science Assessments
The PACT team is making the assessments and scoring rubrics for Units 1-4 of the ECS curriculum available online, as well as a cumulative assessment and scoring rubric covering core knowledge and skills across the foundational ECS units (ECS Assessments). The assessments and rubrics are being released to ECS teachers throughout fall 2015 and winter 2016 as a resource via the new and improved CS10k community of practice web site. We designed the ECS assessments to support ECS teachers in making evidence-supported claims about the proficiency of students in the skills related to the learning goals of the curriculum. Teachers can use the assessment results to inform students’ unit or overall course grades.
Next Steps
In our new work, we are using the assessment tasks and other instruments to study factors that influence implementation of ECS as it scales. Some of our partners include the ECS group, Computer Science Teacher’s Association (CSTA), Code.org, The Learning Partnership, and Education Development Center, Inc.
In a separate project with American Institutes for Research (who operates the CS10K teacher online community) we are porting the tasks to an online format for delivery via the Edfinity platform. We are also validating the online assessments, developing supports for scoring, and prototyping a crowdsourced item bank for future ECS assessment tasks. This will allow us to reach a much larger population of students more quickly, and it opens up the opportunity to further our research with more extensive data reporting.
This material is based upon work supported by the National Science Foundation under Grant Numbers CNS-1240625, CNS-1433065 and DRL-1418149.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
