skip to main content
research-article

Teaching Mathematical Reasoning Principles for Software Correctness and Its Assessment

Published:19 August 2015Publication History
Skip Abstract Section

Abstract

Undergraduate computer science students need to learn analytical reasoning skills to develop high-quality software and to understand why the software they develop works as specified. To accomplish this central educational objective, this article describes a systematic process of introducing reasoning skills into the curriculum and assessing how well students have learned those skills. To facilitate assessment, a comprehensive inventory of principles for reasoning about correctness that captures the finer details of basic skills that students need to learn has been defined and used. The principles can be taught at various levels of depth across the curriculum in a variety of courses.

The use of a particular instructional process is illustrated to inculcate reasoning principles across several iterations of a sophomore-level development foundations course and a junior-level software engineering course. The article summarizes how learning outcomes motivated by the inventory of reasoning principles lead to questions that in turn form the basis for a careful analysis of student understanding and for fine-tuning teaching interventions that together facilitate continuous improvements to instruction.

References

  1. ABET. 2014. Assessment Planning. Retrieved July 14, 2015, from http://abet.org/assessment-planning/.Google ScholarGoogle Scholar
  2. V. L. Almstrum, P. B. Henderson, V. Harvey, C. Heeren, W. Marion, C. Riedesel, L.-K. Soh, and A. E. Tew. 2006. Concept inventories in computer science for the topic discrete mathematics. In Proceedings of the Working Group Reports on ITiCSE on Innovation and Technology in Computer Science Education. ACM, New York, NY, 132--145. DOI:http://dx.doi.org/10.1145/1189215.1189182 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. D. Baldwin, A. Brady, A. Danyluk, J. Adams, and A. Lawrence. 2010a. Case studies of liberal arts computer science programs. ACM Transactions on Computing Education 10, 1, 1--30. DOI:http://dx.doi.org/10.1145/1731041.1731045 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. D. Baldwin, B. Marion, M. Sitaraman, and C. Heeren, 2010b. Some developments in mathematical thinking for computer science education since computing curricula 2001. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 392--393. DOI:http://dx.doi.org/10.1145/1734263.1734397 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. B. S. Bloom, M. Engelhart, E. J. Furst, W. H. Hill, and D. R. Krathwohl. 1956. Taxonomy of Educational Objectives Handbook I: The Cognitive Domain. Longman, New York, NY.Google ScholarGoogle Scholar
  6. P. Bucci, T. J. Long, and B. W. Weide. 2001. Do we really teach abstraction? In Proceedings of the 32nd SIGCSE Technical Symposium on Computer Science Education. ACM, New York, NY, 26--30. DOI:http://dx.doi.org/10.1145/364447.364531 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. L. Burdy, Y. Cheon, D. R. Cok, M. D. Ernst, J. R. Kiniry, G. T. Leavens, K. R. M. Leino, and E. Poll. 2005. An overview of JML tools and applications. International Journal on Software Tools for Technology Transfer 7, 3, 212--232. DOI:http://dx.doi.org/10.1007/s10009-004-0167-4 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. C. T. Cook. 2011. A Web-Integrated Environment for Component-Based Reasoning. Master's Thesis. Clemson University.Google ScholarGoogle Scholar
  9. C. T. Cook, S. Drachova, J. O. Hallstrom, J. E. Hollingsworth, D. P. Jacobs, J. Krone, and M. Sitaraman. 2012a. A systematic approach to teaching abstraction and mathematical modeling. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education. ACM, New York, NY, 357--362. DOI: http://dx.doi.org/10.1145/2325296.2325378 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. C. T. Cook, S. V. Drachova-Strang, Y.-S. Sun, M. Sitaraman, J. C. Carver, and J. E. Hollingsworth. 2013. Specification and reasoning in SE projects using a Web IDE. In Proceedings of the 26th Conference on Software Engineering Education and Training (CSEE&T’’13). IEEE, Los Alamitos, CA, 229--238.Google ScholarGoogle Scholar
  11. C. T. Cook, H. Harton, H. Smith, and M. Sitaraman. 2012b. Specification engineering and modular verification using a Web-integrated verifying compiler. In Proceedings of the 2012 International Conference on Software Engineering. IEEE, Los Alamitos, CA, 1379--1382. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. E. W. Dijkstra, W. H. J. Feijen, and J. Sterringa. 1988. A Method of Programming. Addison-Wesley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. S. Drachova. 2013. Teaching and Assessment of Mathematical Principles for Software Correctness Using a Reasoning Concept Inventory. Ph.D. Dissertation. Clemson University.Google ScholarGoogle Scholar
  14. S. Drachova-Strang, M. Sitaraman, and J. Hollingsworth. 2011. Experimentation with tutors for teaching mathematical reasoning and specification. In Proceedings of the Conference on Frontiers in Education.Google ScholarGoogle Scholar
  15. S. H. Edwards. 2001. A framework for practical, automated black-box testing of component-based software. Software Testing, Verification and Reliability 11, 2, 97--111.Google ScholarGoogle ScholarCross RefCross Ref
  16. S. H. Edwards, W. D. Heym, T. J. Long, M. Sitaraman, and B. W. Weide. 1994. Part II: Specifying components in RESOLVE. ACM SIGSOFT Software Engineering Notes 19, 4, 29--39. DOI: http://dx.doi.org/10.1145/190679.190682 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. M. F. Frias, C. G. L. Pombo, and M. M. Moscato. 2007. Alloy analyzer+PVS in the analysis and verification of alloy specifications. In Proceedings of the 13th International Conference on Tools and Algorithms for the Construction and Analysis of Systems. 587--601. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. L. R. Glasman and D. Albarracín. 2006. Forming attitudes that predict future behavior: A meta-analysis of the attitude-behavior relation. Psychological Bulletin 132, 5, 778--822. DOI: http://dx.doi.org/10.1037/0033-2909.132.5.778Google ScholarGoogle ScholarCross RefCross Ref
  19. K. Goldman, P. Gross, C. Heeren, G. Herman, L. Kaczmarczyk, M. C. Loui, and C. Zilles. 2008. Identifying important and difficult concepts in introductory computing courses using a Delphi process. In Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education. ACM, New York, NY, 256--260. DOI:http://dx.doi.org/10.1145/1352135.1352226 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. D. Gries. 1981. The Science of Programming. Springer, Heidelberg, Germany. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D. Gries, B. Marion, P. Henderson, and D. Schwartz. 2001. How mathematical thinking enchances computer science problem solving. In Proceedings of the 32nd SIGCSE Technical Symposium on Computer Science Education. ACM, New York, NY, 390--391. DOI:http://dx.doi.org/10.1145/364447.364754 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. P. B. Henderson. 2003. Mathematical reasoning in software engineering education. Communications of the ACM 46, 9, 45--50. DOI:http://dx.doi.org/10.1145/903893.903919 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. G. L. Herman, M. C. Loui, and C. Zilles. 2010. Creating the digital logic concept inventory. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 102--106. DOI:http://dx.doi.org/10.1145/1734263.1734298 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. D. Hestenes, M. Wells, and G. Swackhamer. 1992. Force Concept Inventory. Physics Teacher 30, 3, 141--158.Google ScholarGoogle ScholarCross RefCross Ref
  25. J. E. Hollingsworth. 2012. SIGCSE Workshop 2012: Instructional Video Series. Retrieved July 15, 2015, from http://www.cs.clemson.edu/resolve/teaching/ed_ws/sigcse2012/index.html.Google ScholarGoogle Scholar
  26. J. E. Hollingsworth, L. Blankenship, and B. W. Weide. 2000. Experience report: Using RESOLVE/C++ for commercial software. ACM SIGSOFT Software Engineering Notes 25, 6, 11--19. DOI:http://dx.doi.org/10.1145/357474.355048 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. A. Jacobi, J. Martin, J. Mitchell, and T. Newell. 2003. A concept inventory for heat transfer. In Proceedings of the 33rd Annual Frontiers in Education Conference (FIE’03). T3D-12--T3D-16.Google ScholarGoogle Scholar
  28. Joint Task Force on Computing Curricula. 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. J. Krone, D. Baldwin, J. C. Carver, J. E. Hollingsworth, A. Kumar, and M. Sitaraman. 2012. Teaching mathematical reasoning across the curriculum. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 241--242. DOI:http://dx.doi.org/10.1145/2157136.2157208 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. A. Kumar. 2004. Web-based tutors for learning programming in C++/Java. SIGCSE Bulletin 36, 3, 266. DOI:http://dx.doi.org/10.1145/1026487.1008100 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. K. R. M. Leino. 2012. Developing verified programs with Dafny. In Proceedings of the 2012 ACM Conference on High Integrity Language Technology. ACM, New York, NY, 9--10. DOI:http://dx.doi.org/10.1145/2402676.2402682 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. D. P. Leonard, J. O. Hallstrom, and M. Sitaraman. 2009. Injecting rapid feedback and collaborative reasoning in teaching specifications. In Proceedings of the 40th ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 524--528. DOI:http://dx.doi.org/10.1145/1508865.1509046 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. B. Meyer. 1992. Eiffel: The Language. Prentice Hall. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. K. C. Midkiff, T. A. Litzinger, and D. Evans. 2001. Development of engineering thermodynamics concept inventory instruments. In Proceedings of the 31st Annual Frontiers in Education Conference. F2A--F23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. B. M. Notaros. 2002. Concept inventory assessment instruments for electromagnetics education. In Proceedings of the Antennas and Propagation Society International Symposium. IEEE, Los Alamitos, CA, 684--687.Google ScholarGoogle ScholarCross RefCross Ref
  36. A. Remshagen. 2010. Making discrete mathematics relevant. In Proceedings of the 48th Annual Southeast Regional Conference. ACM, New York, NY, 1--6. DOI:http://dx.doi.org/10.1145/1900008.1900060 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Sitaraman, B. Adcock, J. Avigad, D. Bronish, P. Bucci, D. Frazier, H. M. Friedman, H. Harton, W. Heym, J. Kirschenbaum, J. Krone, H. Smith, and B. W. Weide. 2011. Building a push-button RESOLVE verifier: Progress and challenges. Formal Aspects of Computing 23, 5, 607--626. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Sitaraman, J. O. Hallstrom, J. White, S. Drachova-Strang, H. K. Harton, D. Leonard, J. Krone, and R. Pak. 2009. Engaging students in specification and reasoning: “Hands-on” experimentation and evaluation. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education. ACM, New York, NY, 50--54. DOI:http://dx.doi.org/10.1145/1562877.1562899 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. A. Stone, K. Allen, T. R. Rhoads, T. J. Murphy, R. L. Shehab, and C. Saha. 2003. The statistics concept inventory: A pilot study. In Proceedings of the 33rd Annual Conference on Frontiers in Education, IEEE, Los Alamitos, CA, T3D--1-6.Google ScholarGoogle Scholar
  40. A. E. Tew. 2010. Assessing Fundamental Introductory Computing Concept Knowledge in a Language Independent Manner. Ph.D. Dissertation. Georgia Institute of Technology. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. A. E. Tew and M. Guzdial. 2011. The FCS1: A language independent assessment of CS1 knowledge. In Proceedings of the 42nd ACM Technical Symposium on Computer Science Education. ACM, New York, NY, 111--116. DOI:http://dx.doi.org/10.1145/1953163.1953200 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. G. T. Leavens, P. O’Hearn, and S. K. Rajamani (Eds.). 2010. VSTTE’10: Proceedings of the 3rd International Conference on Verified Software: Theories, Tools, Experiments. Springer-Verlag, Heidelberg, Germany. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Teaching Mathematical Reasoning Principles for Software Correctness and Its Assessment

              Recommendations

              Comments

              Login options

              Check if you have access through your login credentials or your institution to get full access on this article.

              Sign in

              Full Access

              • Published in

                cover image ACM Transactions on Computing Education
                ACM Transactions on Computing Education  Volume 15, Issue 3
                September 2015
                69 pages
                EISSN:1946-6226
                DOI:10.1145/2809889
                Issue’s Table of Contents

                Copyright © 2015 ACM

                Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

                Publisher

                Association for Computing Machinery

                New York, NY, United States

                Publication History

                • Published: 19 August 2015
                • Accepted: 1 December 2014
                • Revised: 1 July 2014
                • Received: 1 July 2013
                Published in toce Volume 15, Issue 3

                Permissions

                Request permissions about this article.

                Request Permissions

                Check for updates

                Qualifiers

                • research-article
                • Research
                • Refereed

              PDF Format

              View or Download as a PDF file.

              PDF

              eReader

              View online with eReader.

              eReader