Yvonne Kao
David Weintrop
ABSTRACT
The expansion of computer science (CS) into K-12 contexts has resulted in a diverse ecosystem of curricula designed for various grade levels, teaching a variety of concepts, and using a wide array of different programming languages and environments. Many students will learn more than one programming language over the course of their studies. There is a growing need for computer science assessment that can measure student learning over time, but the multilingual learning pathways create two challenges for assessment in computer science. First, there are not validated assessments for all of the programming languages used in CS classrooms. Second, it is difficult to measure growth in student understanding over time when students move between programming languages as they progress in their CS education. In this position paper, we argue that the field of computing education research needs to develop methods and tools to better measure students' learning over time and across the different programming languages they learn along the way. In presenting this position, we share data that shows students approach assessment problems differently depending on the programming language, even when the problems are conceptually isomorphic, and discuss some approaches for developing multilingual assessments of student learning over time.
- John R. Anderson. 1993. Rules of the Mind .Lawrence Erlbaum Associates, Hillsdale, NJ.Google Scholar
- Michal Armoni, Orni Meerbaum-Salant, and Mordechai Ben-Ari. 2015a. From Scratch to 'Real' Programming. ACM Transactions on Computing Education , Vol. 14, 4, Article 25 (2015), bibinfonumpages15 pages. https://doi.org/10.1145/2677087Google ScholarDigital Library
- Michal Armoni, Orni Meerbaum-Salant, and Mordechai Ben-Ari. 2015b. From Scratch to "Real" Programming. ACM Trans. Comput. Educ. , Vol. 14, 4, Article 25 (Feb. 2015), bibinfonumpages15 pages. https://doi.org/10.1145/2677087Google ScholarDigital Library
- American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. 2014. Standards for Educational and Psychological Testing .AERA, Washington, DC.Google Scholar
- Guanhua Chen, Ji Shen, Lauren Barth-Cohen, Shiyan Jiang, Xiaoting Huang, and Moataz Eltoukhy. 2017. Assessing elementary students' computational thinking in everyday reasoning and robotics programming. Computers & Education , Vol. 109 (2017), 162--175.Google ScholarDigital Library
- D. C. Cliburn. 2008. Student opinions of Alice in CS1. In 2008 38th Annual Frontiers in Education Conference. IEEE, T3B--1--T3B--6. https://doi.org/10.1109/FIE.2008.4720254Google ScholarCross Ref
- Smarter Balanced Assessment Consortium. 2019. Interpretive Guide for English Language Arts/Literacy and Mathematics Assessments . https://portal.smarterbalanced.org/library/en/reporting-system-interpretive-guide.pdfGoogle Scholar
- Holger Danielsiek, Wolfgang Paul, and Jan Vahrenhold. 2012. Detecting and Understanding Students' Misconceptions Related to Algorithms and Data Structures. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (Raleigh, North Carolina, USA) (SIGCSE '12). Association for Computing Machinery, New York, NY, USA, 21--26. https://doi.org/10.1145/2157136.2157148Google ScholarDigital Library
- Wanda Dann, Dennis Cosgrove, Don Slater, Dave Culyba, and Steve Cooper. 2012. Mediated transfer: Alice 3 to java. In Proceedings of the 43rd ACM technical symposium on Computer Science Education. 141--146.Google ScholarDigital Library
- Adrienne Decker and Monica M. McGill. 2019. A Topical Review of Evaluation Instruments for Computing Education. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE '19). Association for Computing Machinery, New York, NY, USA, 558--564. https://doi.org/10.1145/3287324.3287393Google ScholarDigital Library
- Allison Elliott Tew. 2010. Assessing fundamental introductory computing concept knowledge in a language independent manner . Ph.D. Dissertation. Atlanta, GA. http://hdl.handle.net/1853/37090Google Scholar
- Allison Elliott Tew and Mark Guzdial. 2011. The FCS1: A Language Independent Assessment of CS1 Knowledge. In Proceedings of the 42nd ACM Technical Symposium on Computer Science Education (Dallas, TX, USA) (SIGCSE '11). Association for Computing Machinery, New York, NY, USA, 111--116. https://doi.org/10.1145/1953163.1953200Google ScholarDigital Library
- Mohammed F. Farghally, Kyu Han Koh, Jeremy V. Ernst, and Clifford A. Shaffer. 2017. Towards a Concept Inventory for Algorithm Analysis Topics. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (Seattle, Washington, USA) (SIGCSE '17). Association for Computing Machinery, New York, NY, USA, 207--212. https://doi.org/10.1145/3017680.3017756Google ScholarDigital Library
- Diana Franklin, David Weintrop, Jennifer Palmer, Merijke Coenraad, Melissa Cobian, Kristan Beck, Andrew Rasmussen, Sue Krause, Max White, Marco Anaya, et almbox. 2020. Scratch Encore: The Design and Pilot of a Culturally-Relevant Intermediate Scratch Curriculum. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 794--800.Google ScholarDigital Library
- Dan Garcia, Brian Harvey, and Tiffany Barnes. 2015. The beauty and joy of computing. ACM Inroads , Vol. 6, 4 (2015), 71--79.Google ScholarDigital Library
- Mark Guzdial. 2019. We Should Stop Saying 'Language Independent.' We Don't Know How To Do That . https://cacm.acm.org/blogs/blog-cacm/238782-we-should-stop-saying-language-independent-we-dont-know-how-to-do-that/fulltextGoogle Scholar
- Brian Harvey and Jens Mönig. 2010. Bringing "no ceiling" to scratch: Can one language serve kids and computer scientists. Proc. Constructionism (2010), 1--10.Google Scholar
- Charles E. Lance, Marcus M. Butts, and Lawrence C. Michels. 2006. The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? Organizational Research Methods , Vol. 9, 2 (2006), 202--220. https://doi.org/10.1177/1094428105284919Google ScholarCross Ref
- Robert J. Mislevy, Russell G. Almond, and Janice F. Lukas. 2003. A brief introduction to evidence-centered design . Technical Report RR-03--16. Educational Testing Service, Princeton, NJ. https://www.ets.org/Media/Research/pdf/RR-03--16.pdfGoogle Scholar
- Mark Noone and Aidan Mooney. 2018. Visual and textual programming languages: a systematic review of the literature. Journal of Computers in Education , Vol. 5 (2018), 149--174. https://doi.org/10.1007/s40692-018-0101--5Google ScholarCross Ref
- Jum Nunnally and Ira H. Bernstein. 1994. Psychometric Theory: 3rd Edition .McGraw-Hill Education.Google Scholar
- Miranda C. Parker, Mark Guzdial, and Shelly Engleman. 2016. Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment. In Proceedings of the 2016 ACM Conference on International Computing Education Research (Melbourne, VIC, Australia) (ICER '16). Association for Computing Machinery, New York, NY, USA, 93--101. https://doi.org/10.1145/2960310.2960316Google ScholarDigital Library
- Miranda C. Parker, Yvonne S. Kao, Dana Saito-Stehberger, Diana Franklin, Susan Krause, Debra Richardson, and Mark Warschauer. 2021. Development and Preliminary Validation of the Assessment of Computing for Elementary Students (ACES). In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (Virtual Event, USA) (SIGCSE '21). Association for Computing Machinery, New York, NY, USA, 10--16. https://doi.org/10.1145/3408877.3432376Google ScholarDigital Library
- Leo Porter, Daniel Zingaro, Soohyun Nam Liao, Cynthia Taylor, Kevin C. Webb, Cynthia Lee, and Michael Clancy. 2019. BDSI: A Validated Concept Inventory for Basic Data Structures. In Proceedings of the 2019 ACM Conference on International Computing Education Research (Toronto ON, Canada) (ICER '19). Association for Computing Machinery, New York, NY, USA, 111--119. https://doi.org/10.1145/3291279.3339404Google ScholarDigital Library
- Arif Rachmatullah, Bita Akram, Danielle Boulden, Bradford Mott, Kristy Boyer, James Lester, and Eric Wiebe. 2020. Development and validation of the middle grades computer science concept inventory (MG-CSCI) assessment. Eurasia Journal of Mathematics, Science and Technology Education , Vol. 16, 5 (2020). https://doi.org/10.29333/ejmste/116600Google Scholar
- Alper Sahin and Duygu Anil. 2017. The effects of test length and sample size on item parameters in item response theory. Educational Sciences: Theory & Practice , Vol. 17, 1 (2017). https://doi.org/10.12738/estp.2017.1.0270Google Scholar
- Elizabeth Schofield, Michael Erlinger, and Zachary Dodds. 2014. MyCS: CS for Middle-Years Students and Their Teachers. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (Atlanta, Georgia, USA) (SIGCSE '14). Association for Computing Machinery, New York, NY, USA, 337--342. https://doi.org/10.1145/2538862.2538901Google ScholarDigital Library
- Jean Scholtz and Susan Wiedenbeck. 1990. Learning second and subsequent programming languages: A problem of transfer. International Journal of Human-Computer Interaction , Vol. 2, 1 (1990), 51--72.Google ScholarCross Ref
- Exploring Computer Science. [n.d.]. What is ECS? http://www.exploringcs.org/Google Scholar
- Juha Sorva. 2012. Visual program simulation in introductory programming education; Visuaalinen ohjelmasimulaatio ohjelmoinnin alkeisopetuksessa . G4 Monografiaväitöskirja. http://urn.fi/URN:ISBN:978--952--60--4626--6Google Scholar
- Rebecca Vivian, Diana Franklin, Dave Frye, Alan Peterfreund, Jason Ravitz, Florence Sullivan, Melissa Zeitz, and Monica M. McGill. 2020. Evaluation and Assessment Needs of Computing Education in Primary Grades. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (Trondheim, Norway) (ITiCSE '20). Association for Computing Machinery, New York, NY, USA, 124--130. https://doi.org/10.1145/3341525.3387371Google ScholarDigital Library
- David Weintrop, Heather Killen, and Baker E Franke. 2018. Blocks or Text? How programming language modality makes a difference in assessing underrepresented populations. In Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS) 2018. International Society of the Learning Sciences. https://doi.org/10.22318/cscl2018.328Google Scholar
- David Weintrop and Uri Wilensky. 2015. Using Commutative Assessments to Compare Conceptual Understanding in Blocks-Based and Text-Based Programs. In Proceedings of the Eleventh Annual International Conference on International Computing Education Research (Omaha, Nebraska, USA) (ICER '15). Association for Computing Machinery, New York, NY, USA, 101--110. https://doi.org/10.1145/2787622.2787721Google ScholarDigital Library
- David Weintrop and Uri Wilensky. 2019. Transitioning from introductory block-based and text-based environments to professional programming languages in high school computer science classrooms. Computers & Education , Vol. 142 (2019), 103646. https://doi.org/10.1016/j.compedu.2019.103646Google ScholarDigital Library
- Linda Werner, Jill Denner, Shannon Campe, and Damon Chizuru Kawamoto. 2012. The Fairy Performance Assessment: Measuring Computational Thinking in Middle School. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (Raleigh, North Carolina, USA) (SIGCSE '12). Association for Computing Machinery, New York, NY, USA, 215--220. https://doi.org/10.1145/2157136.2157200Google ScholarDigital Library
- Eric Wiebe, Jennifer London, Osman Aksit, Bradford W. Mott, Kristy Elizabeth Boyer, and James C. Lester. 2019. Development of a Lean Computational Thinking Abilities Assessment for Middle Grades Students. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE '19). Association for Computing Machinery, New York, NY, USA, 456--461. https://doi.org/10.1145/3287324.3287390Google ScholarDigital Library
Index Terms
- Multilingual CS Education Pathways: Implications for Vertically-Scaled Assessment
Recommendations
A survey of Australian teachers' self-efficacy and assessment approaches for the K-12 digital technologies curriculum
WiPSCE '18: Proceedings of the 13th Workshop in Primary and Secondary Computing EducationAs K-12 computer science (CS) education has been introduced to a number of countries around the world, the CS education community has been busy working to understand the learning and teaching of computing at these year levels, as well as how to build ...
CS Principles Higher Education Pathways
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science EducationWith approximately 37,000 students entering college with an Advanced Placement CS Principles credit, students, parents, and teachers are wondering how that AP credit "counts" in college. In this panel, we will share perspectives from the College Board ...
Teaching how to teach computational thinking
ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science EducationComputational Thinking is argued to be an essential skill for the workforce of the 21st century. As a skill, Computational Thinking should be taught in all schools, employing computational ideas integrated into other disciplines. Up until now, questions ...
Comments