Kevin Buffardi
ABSTRACT
Software Engineering courses often incorporate large-scale projects with collaboration between students working in teams. However, it is difficult to objectively assess individual students when their projects are a product of collaborative efforts. This study explores measurements of individuals' contributions to their respective teams.
I analyzed ten Software Engineering team projects (n=42) and evaluations of individual contributions using automated evaluation of the version control system history (Git logs) and user stories completed on their project management (Kanban) boards. Unique insights from meta-data within the Git history and Kanban board user stories reveal complicated relationships between these measurements and traditional assessments, such as peer review and subjective instructor evaluation. From the results, I suggest supplementing and validating traditional assessments with insights from individuals' commit history and user story contributions.
- Howard Abikoff, Mary Courtney, William E. Pelham, and Harold S. Koplewicz. 1993. Teachers' ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology 21, 5 (01 Oct 1993), 519--533. https://doi.org/10.1007/BF00916317Google Scholar
Cross Ref
- ACM. 2013. Computer Science 2013: Curriculum Guidelines for Undergraduate Programs in Computer Science. http://www.acm.org/education/curricula-recommendationsGoogle Scholar
- Francesca Arcelli Fontana and Claudia Raibulet. 2017. Students' Feedback in Using GitHub in a Project Development for a Software Engineering Course. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 380--380. https://doi.org/10.1145/3059009.3072984Google ScholarDigital Library
- Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas. 2001. Agile Manifesto. http://agilemanifesto.org/Google Scholar
- Grant Braught, John Maccormick, James Bowring, Quinn Burke, Barbara Cutler, David Goldschmidt, Mukkai Krishnamoorthy, Wesley Turner, Steven Huss- Lederman, Bonnie Mackellar, and Allen Tucker. 2018. A Multi-Institutional Perspective on H/FOSS Projects in the Computing Curriculum. ACM Trans. Comput. Educ. 18, 2, Article 7 (July 2018), 31 pages. https://doi.org/10.1145/3145476Google ScholarDigital Library
- Bernd Bruegge, Stephan Krusche, and Lukas Alperowitz. 2015. Software Engineering Project Courses with Industrial Clients. Trans. Comput. Educ. 15, 4, Article 17 (Dec. 2015), 31 pages. https://doi.org/10.1145/2732155Google Scholar
- Kevin Buffardi. 2018. Tech Startup Learning Activities: A Formative Evaluation. In IEEE/ACM International Workshop on Software Engineering Education for Millennials (SEEM). 24--31.Google ScholarDigital Library
- Kevin Buffardi. 2019. gitlog2csv. https://github.com/kbuffardi/gitlog2csvGoogle Scholar
- Kevin Buffardi, Colleen Robb, and David Rahn. 2017. Learning Agile with Tech Startup Software Engineering Projects. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 28--33. https://doi.org/10.1145/3059009.3059063Google ScholarDigital Library
- Kevin Buffardi, Colleen Robb, and David Rahn. 2017. Tech Startups: Realistic Software Engineering Projects with Interdisciplinary Collaboration. J. Comput. Sci. Coll. 32, 4 (April 2017), 93--98. http://dl.acm.org/citation.cfm?id=3055338.3055355Google ScholarDigital Library
- Donald T. Campbell. 1979. Assessing the impact of planned social change. Evaluation and Program Planning 2, 1 (1979), 67 -- 90. https://doi.org/10.1016/0149--7189(79)90048-XGoogle ScholarCross Ref
- Nicole Clark. 2005. Evaluating Student Teams Developing Unique Industry Projects. In Proceedings of the 7th Australasian Conference on Computing Education - Volume 42 (ACE '05). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 21--30. http://dl.acm.org/citation.cfm?id=1082424.1082428Google ScholarDigital Library
- N. M. Devadiga. 2017. Software Engineering Education: Converging with the Startup Industry. In 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE T). 192--196. https://doi.org/10.1109/CSEET.2017.38Google Scholar
- Heidi J.C. Ellis and Gregory W. Hislop. 2016. Pathways to Student Learning Within HFOSS. In Proceedings of the 17th Annual Conference on Information Technology Education (SIGITE '16). ACM, New York, NY, USA, 168--168. https://doi.org/10.1145/2978192.2978242Google Scholar
- F. Fagerholm and A. Vihavainen. 2013. Peer assessment in experiential learning Assessing tacit and explicit skills in agile software engineering capstone projects. In 2013 IEEE Frontiers in Education Conference (FIE). 1723--1729. https://doi.org/10.1109/FIE.2013.6685132Google ScholarCross Ref
- Vahid Garousi, Kai Petersen, and Baris Ozkan. 2016. Challenges and best practices in industry-academia collaborations in software engineering: A systematic literature review. Information and Software Technology 79 (2016), 106 -- 127. https://doi.org/10.1016/j.infsof.2016.07.006Google ScholarDigital Library
- GitHub. 2019. The world's leading software development platform - GitHub. https://github.com/ Accessed on August 2019.Google Scholar
- Nicole Herbert. 2007. Quantitative Peer Assessment: Can Students Be Objective?. In Proceedings of the Ninth Australasian Conference on Computing Education - Volume 66 (ACE '07). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 63--71. http://dl.acm.org/citation.cfm?id=1273672.1273680Google Scholar
- Bibb Latane, Kipling Williams, and Stephen Harkins. 1979. Many hands make light the work: The causes and consequences of social loafing. Journal of Personality and Social Psychology 37 (1979), 822--832. Issue 6. https://doi.org/10.1037/0022--3514.37.6.822Google ScholarCross Ref
- Richard A. Layton, Matthew W. Ohland, and Hal Pomeranz. 2007. Software For Student Team Formation And Peer Evaluation: Catme Incorporates Team Maker.Google Scholar
- Robert C. Martin. 2006. Agile Software Development: Principles, Patterns, and Practices. Prentice Hall.Google ScholarDigital Library
- Nancy R. Mead. 2015. Industry/University Collaboration in Software Engineering Education: Refreshing and Retuning Our Strategies. In Proceedings of the 37th International Conference on Software Engineering - Volume 2 (ICSE '15). IEEE Press, Piscataway, NJ, USA, 273--275. http://dl.acm.org/citation.cfm?id=2819009.2819050Google ScholarCross Ref
- Christian Murphy, Kevin Buffardi, Josh Dehlinger, Lynn Lambert, and Nanette Veilleux. 2017. Community Engagement with Free and Open Source Software. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17). ACM, New York, NY, USA, 669--670. https://doi.org/10.1145/3017680.3017682Google ScholarDigital Library
- Natalja Nikitina, Mira Kajko-Mattsson, and Magnus Stråle. 2012. From Scrum to Scrumban: A Case Study of a Process Transition. In Proceedings of the International Conference on Software and System Process (ICSSP '12). IEEE Press, Piscataway, NJ, USA, 140--149. http://dl.acm.org/citation.cfm?id=2664360.2664382Google ScholarCross Ref
- Matthew Ohland, Misty Loughry, Rufus L. Carter, Lisa Bullard, Richard Felder, Cynthia Finelli, Richard Layton, and Douglas Schmucker. 2005. Developing a Peer Evaluation Instrument that is Simple, Reliable, and Valid. Proceedings 2005 American Society of Engineering Education Conference and Exposition Portland Oregon (01 2005).Google Scholar
- MatthewW. Ohland, Misty L. Loughry, David J.Woehr, Lisa G. Bullard, Richard M. Felder, Cynthia J. Finelli, Richard A. Layton, Hal R. Pomeranz, and Douglas G. Schmucker. 2012. The Comprehensive Assessment of Team Member Effectiveness: Development of a Behaviorally Anchored Rating Scale for Self- and Peer Evaluation. Academy of Management Learning & Education 11, 4 (2012), 609--630. https://doi.org/10.5465/amle.2010.0177Google Scholar
- Helen Parker and Mike Holcombe. 1999. Campus-based Industrial Software Projects: Risks and Rewards. SIGCSE Bull. 31, 3 (June 1999), 189--. https://doi.org/10.1145/384267.305935Google Scholar
- Gustavo Pinto, Clarice Ferreira, Cleice Souza, Igor Steinmacher, and Paulo Meirelles. 2019. Training Software Engineers Using Open-source Software: The Students' Perspective. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET '19). IEEE Press, Piscataway, NJ, USA, 147--157. https://doi.org/10.1109/ICSE-SEET.2019.00024Google ScholarDigital Library
- Arnold Rosenbloom, Sadia Sharmin, and Andrew Wang. 2017. GIT: Pedagogy, Use and Administration in Undergraduate CS. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 82--83. https://doi.org/10.1145/3059009.3072980Google ScholarDigital Library
- Frank W. Schneider, Jamie A. Gruman, and Larry M. Coutts. 2012. Applied Social Psychology: Understanding and Addressing Social and Practical Problems. SAGE.Google Scholar
- Therese Mary Smith, Robert McCartney, Swapna S. Gokhale, and Lisa C. Kaczmarczyk. 2014. Selecting Open Source Software Projects to Teach Software Engineering. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14). ACM, New York, NY, USA, 397--402. https://doi.org/10.1145/2538862.2538932Google Scholar
- Jeff Sutherland. 2018. Impediments of Performance Appraisals. https://www.scruminc.com/performance-appraisals/ Accessed November 2019.Google Scholar
- Anya Tafliovich, Andrew Petersen, and Jennifer Campbell. 2015. On the Evaluation of Student Team Software Development Projects. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE '15). ACM, New York, NY, USA, 494--499. https://doi.org/10.1145/2676723.2677223Google ScholarDigital Library
- Linda van den Bergh, Eddie Denessen, Lisette Hornstra, Marinus Voeten, and Rob W. Holland. 2010. The Implicit Prejudiced Attitudes of Teachers: Relations to Teacher Expectations and the Ethnic Achievement Gap. American Educational Research Journal 47, 2 (2010), 497--527. https://doi.org/10.3102/0002831209353594 arXiv:https://doi.org/10.3102/0002831209353594Google ScholarCross Ref
Index Terms
- Assessing Individual Contributions to Software Engineering Projects with Git Logs and User Stories
Recommendations
"Leagile" software development
In recent years there has been a noticeable shift in attention from those who use agile software development toward lean software development, often labelled as a shift "from agile to lean". However, the reality may not be as simple or linear as this ...
Learning Agile with Tech Startup Software Engineering Projects
ITiCSE '17: Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science EducationThe Tech Startup model is an approach to learning software engineering methods by partnering with students studying entrepreneurship to collaborate on real software products. Agile software development methods align with Lean Startup practices so that ...
Software Engineering Practices and Methods in the Game Development Industry
CHI PLAY '19 Extended Abstracts: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended AbstractsApplying current software engineering practices in the game development industry is a rapidly growing but under researched area. Whether game development studios align to traditional software engineering practices such as agile methodologies to develop ...
Comments