John Aycock
ABSTRACT
We report our experience with a novel course on binary reverse engineering, a university computer science course that was offered at the second-year level to both computer science majors as well as non-majors, with minimal prerequisites. While reverse engineering has known, important uses in computer security, this was pointedly not framed as a security course, because reverse engineering is a skill that has uses outside computer science and can be taught to a more diverse audience. The original course design intended students to perform hands-on exercises during an in-person class; we describe the systems we developed to support that, along with other online systems we used, which allowed a relatively easy pivot to online learning and back as necessitated by the pandemic. Importantly, we detail our application of "ungrading" within the course, an assessment philosophy that has gained some traction primarily in non-STEM disciplines but has seen little to no discussion in the context of computer science education. The combination of pedagogical methods we present has potential uses in other courses beyond reverse engineering.
- N. Altice. 2015. I AM ERROR: The Nintendo Family Computer / Entertainment System Platform. MIT Press.Google Scholar
Cross Ref
- J. Aycock. 2021. The coming tsunami of digital artifacts. Antiquity, Vol. 95, 384 (2021), 1584--1589.Google ScholarCross Ref
- J. Aycock, A. Groeneveldt, H. Kroepfl, and T. Copplestone. 2018. Exercises for Teaching Reverse Engineering. In 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education. 188--193.Google Scholar
- A. Berns. 2020. Scored out of 10: Experiences with Binary Grading Across the Curriculum. In 51st ACM Technical Symposium on Computer Science Education. 1152--1157.Google ScholarDigital Library
- S. D. Blum. 2020a. Not Simple but Essential. See citeNungrading:book, 219--228.Google Scholar
- S. D. Blum (Ed.). 2020b. Ungrading: Why Rating Students Undermines Learning (and What to Do Instead). West Virginia University Press.Google Scholar
- S. Bradley. 2016. Managing Plagiarism in Programming Assignments with Blended Assessment and Randomisation. In 16th Koli Calling International Conference on Computing Education Research. 21--30.Google Scholar
- S. Bridgeman, M. T. Goodrich, S. G. Kobourov, and R. Tamassia. 2000. SAIL: A System for Generating, Archiving, and Retrieving Specialized Assignments using ŁaTeX. In 31st SIGCSE Technical Symposium on Computer Science Education. 300--304.Google Scholar
- S. Brown and V. Chá vez. 2022. Communicating Alternative Grading Schemes: How to Shift Students' Attention to Their Learning from Grades. In 53rd ACM Technical Symposium on Computer Science Education (V.2). 1189.Google Scholar
- J. Burket, P. Chapman, T. Becker, C. Ganas, and D. Brumley. 2015. Automatic Problem Generation for Capture-the-Flag Competitions. In 2015 USENIX Summit on Gaming, Games, and Gamification in Security Education. 8 pages. https://www.usenix.org/system/files/conference/3gse15/3gse15-burket.pdfGoogle Scholar
- S. Cole. 2022. Impact of Capture The Flag (CTF)-style vs. Traditional Exercises in an Introductory Computer Security Class. In 27th ACM Conference on Innovation and Technology in Computer Science Education (Vol. 1). 470--476.Google ScholarDigital Library
- C. Collberg and J. Nagra. 2010. Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection. Addison-Wesley.Google ScholarDigital Library
- P. Elbow. 1997. Grading Student Writing: Making It Simpler, Fairer, Clearer. New Directions for Teaching and Learning, Vol. 1997, 69 (1997), 127--140.Google ScholarCross Ref
- I. Englander and W. Wong. 2021. The Little Man Computer. In The Architecture of Computer Hardware, Systems Software, and Networking: An Information Technology Approach 6th ed.). Wiley, Chapter 6, 152--165.Google Scholar
- W.-C. Feng. 2015. A Scaffolded, Metamorphic CTF for Reverse Engineering. In 2015 USENIX Summit on Gaming, Games, and Gamification in Security Education. 8 pages. https://www.usenix.org/system/files/conference/3gse15/3gse15-feng.pdfGoogle Scholar
- A. Finkel, N. Harris, P. Higginbottom, and M. Tomczyk. 1982. VIC-20 Programmer's Reference Guide. Commodore Business Machines and Howard W. Sams & Co.Google Scholar
- B. Foster and A. Somayaji. 2010. Object-Level Recombination of Commodity Applications. In 12th Annual Conference on Genetic and Evolutionary Computation. 957--963.Google Scholar
- Gather Presence, Inc. [n.,d.]. Gather. https://www.gather.town/ Retrieved 29 December 2022 fromGoogle Scholar
- C. Gorey. 1 Feb 2018. NASA satellite brought back from the dead will need major reverse engineering. Silicon Republic. https://www.siliconrepublic.com/innovation/nasa-image-satellite-reverse-engineeringGoogle Scholar
- Association for Computing Machinery (ACM) Joint Task Force on Computing Curricula and IEEE Computer Society. 2013. Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM. https://doi.org/10.1145/2534860Google ScholarDigital Library
- Kaitai Project. [n.,d.]. Kaitai Struct. https://kaitai.io/ Retrieved 29 December 2022 fromGoogle Scholar
- M. G. Kirschenbaum. 2008. Mechanisms: New Media and the Forensic Imagination. MIT Press.Google ScholarDigital Library
- S. Madnick. 1993. Understanding the Computer (Little Man Computer). Unpublished article, based on 1979 version.Google Scholar
- W. Mahoney and R. A. Gandhi. 2012. Reverse Engineering -- Is It Art? ACM Inroads, Vol. 3, 1 (2012), 56--61. https://doi.org/10.1145/2077808.2077826Google ScholarDigital Library
- A. Mantovani, S. Aonzo, Y. Fratantonio, and D. Balzarotti. 2022. RE-Mind: A First Look Inside the Mind of a Reverse Engineer. In 31st USENIX Security Symposium. 2727--2745.Google Scholar
- N. Montfort and I. Bogost. 2009. Racing the Beam: The Atari video computer system. MIT Press.Google ScholarCross Ref
- Gabriel Moshenska. 2016. Reverse engineering and the archaeology of the modern world. Forum Kritische Archaologie, Vol. 5 (2016), 16--28.Google Scholar
- D. S. Myers. 2022. Designing specifications grading systems. Journal of Computing Sciences in Colleges, Vol. 37, 5 (2022), 91--92.Google Scholar
- National Security Agency. [n.,d.]. Ghidra. https://ghidra-sre.org/ Retrieved 29 December 2022 fromGoogle Scholar
- TJ OConnor, C. Mann, T. Petersen, I. Thomas, and C. Stricklan. 2022. Toward an Automatic Exploit Generation Competition for an Undergraduate Binary Reverse Engineering Course. In 27th ACM Conference on on Innovation and Technology in Computer Science Education (Vol. 1). 442--448.Google Scholar
- TJ OConnor and C. Stricklan. 2021. Teaching a Hands-On Mobile and Wireless Cybersecurity Course. In 27th ACM Conference on on Innovation and Technology in Computer Science Education (Vol. 1). 296--302.Google Scholar
- G. G. Richard III. 2009. A Highly Immersive Approach to Teaching Reverse Engineering. In 2nd Workshop on Cyber Security Experimentation and Test. USENIX, 6 pages. http://usenix.org/event/cset09/tech/full_papers/richard.pdfGoogle Scholar
- C. Riesbeck. 2020. Critique-Driven Learning and Assessment. See [6], Chapter 8, 123--139.Google Scholar
- J. Scott. 14 April 2015. Behold the Emularity. http://ascii.textfiles.com/archives/4604 Retrieved 4 January 2023 fromGoogle Scholar
- J. Stommel. 2020. How to Ungrade. See citeNungrading:book, Chapter 1, 25--41.Google Scholar
- P. Szor. 2005. The Art of Computer Virus Research and Defense. Addison-Wesley.Google Scholar
- K. H. Tan and E. L. Ouh. 2021. Lessons Learnt Conducting Capture the Flag CyberSecurity Competition during COVID-19. In 2021 IEEE Frontiers in Education Conference. 9 pages.Google Scholar
- C. Taylor, P. Arias, J. Klopchic, C. Matarazzo, and E. Dube. 2017. CTF: State-of-the-Art and Building the Next Generation. In 2017 USENIX Workshop on Advances in Security Education. 11 pages.Google Scholar
- C. Taylor and C. Collberg. 2016. A Tool for Teaching Reverse Engineering. In 2016 USENIX Workshop on Advances in Security Education. 8 pages. https://www.usenix.org/system/files/conference/ase16/ase16-paper-taylor.pdfGoogle Scholar
- D. H. Tobey, P. Pusey, and D. L. Burley. 2014. Engaging Learners in Cybersecurity Careers: Lessons from the Launch of the National Cyber League. ACM Inroads, Vol. 5, 1 (2014), 53--56.Google ScholarDigital Library
- J. Togelius, G. N. Yannakakis, K. O. Stanley, and C. Browne. 2011. Search-Based Procedural Content Generation: A Taxonomy and Survey. IEEE Transactions on Computational Intelligence and AI in Games, Vol. 3, 3 (2011), 172--186.Google ScholarCross Ref
- U.S. Bureau of Labor Statistics. [n.,d.]. Occupational Outlook Handbook: Information Security Analysts. https://www.bls.gov/ooh/computer-and-information-technology/information-security-analysts.htm Retrieved 2 January 2023 fromGoogle Scholar
- J. Vykopal, V. Svábenský, and E.-C. Chang. 2020. Benefits and Pitfalls of Using Capture the Flag Games in University Courses. In 51st ACM Technical Symposium on Computer Science Education. 752--758.Google Scholar
- J. Vykopal, V. Svábenský, P. Seda, and P. Celeda. 2022. Preventing Cheating in Hands-on Lab Assignments. In 53rd ACM Technical Symposium on Computer Science Education (V. 1). 78--84.Google Scholar
- H. Wright and J. Aycock. 2020. 10 Binary Games for Computer Science Education. In 51st ACM Technical Symposium on Computer Science Education. 1308. Poster.Google Scholar
Index Terms
- Binary Reverse Engineering for All
Recommendations
Exercises for teaching reverse engineering
ITiCSE 2018: Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science EducationThe ability to reverse engineer binary code is a skill of critical importance within computer security: deciding if an unknown piece of binary code is malicious and, if so, what it does. And yet, there is very little work in computer science education ...
Pairing Ungrading with Project-Based Learning in CS1 for Inherently Flexible Course Design
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1This experience report details the pedagogical approach and curriculum for an introductory programming course for non-majors that combines creative coding, ungrading, and project-based learning, with typical enrollment between 120-140 students. Through a ...
Why teach reverse engineering?
Software reverse engineering is a fascinating discipline of software engineering. But it has failed to attract attention from students. Largely due to the facts that many universities around the world do not offer relevant courses, developing new ...
Comments