skip to main content
10.1145/3287324.3287384acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article

Software Testing in Introductory Programming Courses: A Systematic Mapping Study

Published: 22 February 2019 Publication History

Abstract

Traditionally, students learn about software testing during intermediate or advanced computing courses. However, it is widely advocated that testing should be addressed beginning in introductory programming courses. In this context, testing practices can help students think more critically while working on programming assignments. At the same time, students can develop testing skills throughout the computing curriculum. Considering this scenario, we conducted a systematic mapping of the literature about software testing in introductory programming courses, resulting in 293 selected papers. We mapped the papers to categories with respect to their investigated topic (curriculum, teaching methods, programming assignments, programming process, tools, program/test quality, concept understanding, and students' perceptions and behaviors) and evaluation method (literature review, exploratory study, descriptive/persuasive study, survey, qualitative study, experimental and experience report). We also identified the benefits and drawbacks of this teaching approach, as pointed out in the selected papers. The goal is to provide an overview of research performed in the area, highlighting gaps that should be further investigated.

References

[1]
ACM/IEEE-CS. 2013. Computer Science Curricula 2013. (December 2013). Joint Task Force on Computing Curricula, available at www.acm.org/education/CS2013-final-report.pdf.
[2]
Ahmed Al-Zubidy, Jeffrey C. Carver, Sarah Heckman, and Mark Sherriff. 2016. A (Updated) Review of Empiricism at the SIGCSE Technical Symposium. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE'16) . ACM, New York, NY, USA, 120--125.
[3]
J. C. Carver and N. A. Kraft. 2011. Evaluating the testing ability of senior-level computer science students. In 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T) . 169--178.
[4]
Stephen H. Edwards. 2003. Rethinking Computer Science Education from a Test-first Perspective. In Companion of the 18th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA '03). ACM, New York, NY, USA, 148--155.
[5]
Stephen H. Edwards. 2004. Using Software Testing to Move Students from Trial-and-error to Reflection-in-action. In Proceedings of the 35th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '04). ACM, New York, NY, USA, 26--30.
[6]
Vahid Garousi and Junji Zhi. 2013. A Survey of Software Testing Practices in Canada . Journal of Systems and Software, Vol. 86, 5 (May 2013), 1354--1376.
[7]
Mark Guzdial. 2013. Exploring Hypotheses About Media Computation. In Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research (ICER '13) . ACM, New York, NY, USA, 19--26.
[8]
David Janzen and Hossein Saiedian. 2008. Test-driven Learning in Early Programming Courses. In Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '08). ACM, New York, NY, USA, 532--536.
[9]
Edward L. Jones. 2001. Integrating testing into the curriculum -- arsenic in small doses. In Proceedings of the Thirty-second SIGCSE Technical Symposium on Computer Science Education (SIGCSE '01). ACM, New York, NY, USA, 337--341.
[10]
Arnold Pears, Stephen Seidman, Lauri Malmi, Linda Mannila, Elizabeth Adams, Jens Bennedsen, Marie Devlin, and James Paterson. 2007. A Survey of Literature on the Teaching of Introductory Programming. In Working Group Reports on ITiCSE on Innovation and Technology in Computer Science Education (ITiCSE-WGR '07). ACM, New York, NY, USA, 204--223.
[11]
Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic Mapping Studies in Software Engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE'08). British Computer Society, Swinton, UK, UK, 68--77.
[12]
Alex Radermacher and Gursimran Walia. 2013. Gaps Between Industry Expectations and the Abilities of Graduates. In Proceeding of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE '13) . ACM, New York, NY, USA, 525--530.
[13]
Arto Vihavainen, Jonne Airaksinen, and Christopher Watson. 2014. A Systematic Review of Approaches for Teaching Introductory Programming and Their Influence on Success. In Proceedings of the Tenth Annual Conference on International Computing Education Research (ICER '14) . ACM, New York, NY, USA, 19--26.
[14]
Jacqueline L. Whalley and Anne Philpott. 2011. A Unit Testing Approach to Building Novice Programmers' Skills and Confidence. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (ACE '11). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 113--118. http://dl.acm.org/citation.cfm?id=2459936.2459950
[15]
Claes Wohlin. 2014. Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE'14). ACM, New York, NY, USA, Article 38, bibinfonumpages10 pages.
[16]
He Zhang, Muhammad Ali Babar, and Paolo Tell. 2011. Identifying relevant studies in software engineering. Information and Software Technology, Vol. 53, 6 (2011), 625 -- 637. Special Section: Best papers from the APSEC.

Cited By

View all
  • (2024)CLASSROOM VIGNETTESACM Inroads10.1145/367761115:3(20-23)Online publication date: 21-Aug-2024
  • (2024)Exploring students' opinion on software testing coursesProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661276(570-579)Online publication date: 18-Jun-2024
  • (2024)CLASSROOM VIGNETTES: Experiencing Efficiency of Algorithms and ImplementationsACM Inroads10.1145/364950915:2(10-17)Online publication date: 10-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education
February 2019
1364 pages
ISBN:9781450358903
DOI:10.1145/3287324
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 February 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. introductory programming courses
  2. software testing
  3. systematic mapping

Qualifiers

  • Research-article

Funding Sources

Conference

SIGCSE '19
Sponsor:

Acceptance Rates

SIGCSE '19 Paper Acceptance Rate 169 of 526 submissions, 32%;
Overall Acceptance Rate 1,787 of 5,146 submissions, 35%

Upcoming Conference

SIGCSE TS 2025
The 56th ACM Technical Symposium on Computer Science Education
February 26 - March 1, 2025
Pittsburgh , PA , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)73
  • Downloads (Last 6 weeks)10
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)CLASSROOM VIGNETTESACM Inroads10.1145/367761115:3(20-23)Online publication date: 21-Aug-2024
  • (2024)Exploring students' opinion on software testing coursesProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661276(570-579)Online publication date: 18-Jun-2024
  • (2024)CLASSROOM VIGNETTES: Experiencing Efficiency of Algorithms and ImplementationsACM Inroads10.1145/364950915:2(10-17)Online publication date: 10-May-2024
  • (2024)Evaluating the Effectiveness of a Testing Checklist Intervention in CS2: An Quasi-experimental Replication StudyProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671102(55-64)Online publication date: 12-Aug-2024
  • (2024)CS1 with a Side of AI: Teaching Software Verification for Secure Code in the Era of Generative AIProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630817(345-351)Online publication date: 7-Mar-2024
  • (2024)Narrowing Industry-Academia Gap with a Virtual Laboratory2024 47th MIPRO ICT and Electronics Convention (MIPRO)10.1109/MIPRO60963.2024.10569687(304-310)Online publication date: 20-May-2024
  • (2024)A pragmatic look at education and training of software test engineers: Further cooperation of academia and industry is needed2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW60967.2024.00067(354-360)Online publication date: 27-May-2024
  • (2024)State of the Practice in Software Testing Teaching in Four European Countries2024 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST60714.2024.00015(59-69)Online publication date: 27-May-2024
  • (2024)Towards Understanding Students' Sensemaking of Test Case Design: A One-Page Summary2024 36th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET62301.2024.10663026(1-1)Online publication date: 29-Jul-2024
  • (2024)Assessing the testing skills transfer of model-based testing on testing skill acquisitionSoftware and Systems Modeling10.1007/s10270-023-01141-123:4(953-971)Online publication date: 22-Jan-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media