Emily M. Hastings
Sneha R. Krishna Kumaran
ABSTRACT
The configuration that an instructor enters into an algorithmic team formation tool determines how students are grouped into teams, impacting their learning experiences. One way to decide the configuration is to solicit input from the students. Prior work has investigated the criteria students prefer for team formation, but has not studied how students prioritize the criteria or to what degree students agree with each other. This paper describes a workflow for gathering student preferences for how to weight the criteria entered into a team formation tool, and presents the results of a study in which the workflow was implemented in four semesters of the same project-based design course. In the most recent semester, the workflow was supplemented with an online peer discussion to learn about students' rationale for their selections. Our results show that students want to be grouped with other students who share the same course commitment and compatible schedules the most. Students prioritize demographic attributes next, and then task skills such as programming needed for the project work. We found these outcomes to be consistent in each instance of the course. Instructors can use our results to guide team formation in their own project-based design courses and replicate our workflow to gather student preferences for team formation in any course.
- 2017. Team-Maker Algorithm Detail. Retrieved 4 April 2019 from https://www. catme.org/faculty/help#TeamMakerScoringGoogle Scholar
- 2018. CATME Smarter Teamwork. Retrieved 31 August 2018 from http://info. catme.orgGoogle Scholar
- Maite Barrios, Georgina Guilera, Laura Nuño, and Juana Gómez-Benito. 2021. Consensus in the delphi method: What makes a decision change? Technological Forecasting and Social Change 163 (feb 2021), 120484. https://doi.org/10.1016/j. techfore.2020.120484Google Scholar
- Julia B Bear and Anita Williams Woolley. 2011. The role of gender in team collaboration and performance. Interdisciplinary science reviews 36, 2 (2011), 146--153.Google Scholar
- Suzanne T Bell. 2007. Deep-level composition variables as predictors of team performance: a meta-analysis. Journal of Applied Psychology 92, 3 (2007), 595--615.Google ScholarCross Ref
- Lt Col James L Brickell, Lt Col David B Porter, Lt Col Michael F Reynolds, and Capt Richard D Cosgrove. 1994. Assigning students to groups for engineering design projects: A comparison of five methods. Journal of Engineering Education 83, 3 (1994), 259--262.Google ScholarCross Ref
- Judy Brown and Gillian Dobbie. 1999. Supporting and evaluating team dynamics in group projects. In The proceedings of the thirtieth SIGCSE technical symposium on Computer science education. 281--285.Google ScholarDigital Library
- Paula E Chan, Kristall J Graham-Day, Virginia A Ressa, Mary T Peters, and Moira Konrad. 2014. Beyond involvement: Promoting student ownership of learning in classrooms. Intervention in School and Clinic 50, 2 (2014), 105--113.Google ScholarCross Ref
- Ben Coleman and Matthew Lang. 2012. Collaboration across the curriculum: a disciplined approach todeveloping team skills. In Proceedings of the 43rd ACM technical symposium on Computer Science Education. 277--282.Google ScholarDigital Library
- David T Conley and Elizabeth M French. 2014. Student ownership of learning as a key component of college readiness. American Behavioral Scientist 58, 8 (2014), 1018--1034.Google ScholarCross Ref
- Mary L Connerley and Fred A Mael. 2001. The importance and invasiveness of student team selection criteria. Journal of management education 25, 5 (2001), 471--494.Google ScholarCross Ref
- Yohanan Eshel and Revital Kohavi. 2003. Perceived classroom control, selfregulated learning strategies, and academic achievement. Educational psychology 23, 3 (2003), 249--260.Google Scholar
- Elena L Glassman, Aaron Lin, Carrie J Cai, and Robert C Miller. 2016. Learnersourcing personalized hints. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. ACM, 1626--1636.Google ScholarDigital Library
- Elena L Glassman and Robert C Miller. 2016. Leveraging Learners for Teaching Programming and Hardware Design at Scale. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion. ACM, 37--40.Google ScholarDigital Library
- Emily M. Hastings, Albatool Alamri, Andrew Kuznetsov, Christine Pisarczyk, Karrie Karahalios, Darko Marinov, and Brian P. Bailey. 2020. LIFT: Integrating Stakeholder Voices into Algorithmic Team Formation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (Chi '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376797Google ScholarDigital Library
- Emily M. Hastings, Farnaz Jahanbakhsh, Karrie Karahalios, Darko Marinov, and Brian P. Bailey. 2018. Structure or Nurture? The Effects of Team-Building Activities and Team Composition on Team Outcomes. In Proceedings of the ACM on Human-Computer Interaction, Vol. 2. ACM.Google ScholarDigital Library
- Tyson R Henry. 2013. Creating effective student groups: an introduction to groupformation. org. In Proceeding of the 44th ACM technical symposium on Computer science education. ACM, 645--650.Google Scholar
- Susan Horwitz, Susan H Rodger, Maureen Biggers, David Binkley, C Kolin Frantz, Dawn Gundermann, Susanne Hambrusch, Steven Huss-Lederman, Ethan Munson, Barbara Ryder, et al. 2009. Using peer-led team learning to increase participation and success of under-represented groups in introductory computer science. ACM SIGCSE Bulletin 41, 1 (2009), 163--167.Google ScholarDigital Library
- Sujin K Horwitz and Irwin B Horwitz. 2007. The effects of team diversity on team outcomes: A meta-analytic review of team demography. Journal of management 33, 6 (2007), 987--1015.Google ScholarCross Ref
- Farnaz Jahanbakhsh, Wai-Tat Fu, Karrie Karahalios, Darko Marinov, and Brian Bailey. 2017. You want me to work with who? Stakeholder perceptions of automated team formation in project-based courses. In Proceedings of the 2017 CHI conference on human factors in computing systems. 3201--3212.Google ScholarDigital Library
- Juho Kim. 2015. Learnersourcing: improving learning with collective learner activity. Ph.D. Dissertation. Massachusetts Institute of Technology.Google Scholar
- Michael S Kirkpatrick. 2017. Student perspectives of team-based learning in a CS course: Summary of qualitative findings. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. 327--332.Google ScholarDigital Library
- Cameron Klein, Deborah DiazGranados, Eduardo Salas, Huy Le, C Shawn Burke, Rebecca Lyons, and Gerald F Goodwin. 2009. Does team building work? Small Group Research 40, 2 (2009), 181--222.Google ScholarCross Ref
- Celine Latulipe, N Bruce Long, and Carlos E Seminario. 2015. Structuring flipped classes with lightweight teams and gamification. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. 392--397.Google ScholarDigital Library
- Richard A Layton, Misty L Loughry, Matthew W Ohland, and George D Ricco. 2010. Design and Validation of a Web-Based System for Assigning Members to Teams Using Instructor-Specified Criteria. Advances in Engineering Education 2, 1 (2010), n1.Google Scholar
- Shang-Wen Daniel Li and Piotr Mitros. 2015. Learnersourced recommendations for remediation. In Advanced Learning Technologies (ICALT), 2015 IEEE 15th International Conference on. IEEE, 411--412.Google Scholar
- Weichen Liu, Sijia Xiao, Jacob T. Browne, Ming Yang, and Steven P. Dow. 2018. ConsensUs. ACM Transactions on Social Computing 1, 1 (feb 2018), 1--26. https: //doi.org/10.1145/3159649Google ScholarDigital Library
- Ioanna Lykourentzou, Angeliki Antoniou, Yannick Naudet, and Steven P Dow. 2016. Personality matters: Balancing for personality types leads to better outcomes for crowd teams. In Proceedings of the 19th ACM Conference on ComputerSupported Cooperative Work & Social Computing. ACM, 260--273.Google ScholarDigital Library
- Ioanna Lykourentzou, Robert E Kraut, and Steven P Dow. 2017. Team Dating Leads to Better Online Ad Hoc Collaborations. In CSCW. 2330--2343.Google Scholar
- Ioanna Lykourentzou, Shannon Wang, Robert E Kraut, and Steven P Dow. 2016. Team dating: A self-organized team formation strategy for collaborative crowdsourcing. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1243--1249.Google ScholarDigital Library
- Jeffrey A Mello. 1993. Improving individual member accountability in small work group settings. Journal of Management Education 17, 2 (1993), 253--259.Google ScholarCross Ref
- Amy Pavel, Dan B Goldman, Björn Hartmann, and Maneesh Agrawala. 2016. VidCrit: video-based asynchronous video review. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 517--528.Google ScholarDigital Library
- Niloufar Salehi and Michael S Bernstein. 2018. Hive: Collective Design Through Network Rotation. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 151.Google ScholarDigital Library
- Thomas J Scott, Lee H Tichenor, Ralph B Bisland Jr, and James H Cross. 1994. Team dynamics in student programming projects. ACM SIGCSE Bulletin 26, 1 (1994), 111--115.Google ScholarDigital Library
- Debra Smarkusky, Richard Dempsey, J Ludka, and Frouke de Quillettes. 2005. Enhancing team knowledge: instruction vs. experience. In Proceedings of the 36th SIGCSE technical symposium on Computer science education. 460--464.Google ScholarDigital Library
- Teri Spinelli. 1983. The Delphi Decision-Making Process. The Journal of Psychology 113, 1 (jan 1983), 73--80. https://doi.org/10.1080/00223980.1983.9923559Google ScholarCross Ref
- James Surowiecki. 2005. The wisdom of crowds. Anchor.Google Scholar
- Anya Tafliovich, Andrew Petersen, and Jennifer Campbell. 2015. On the evaluation of student team software development projects. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. 494--499.Google ScholarDigital Library
- Dai-Yi Wang, Sunny SJ Lin, and Chuen-Tsai Sun. 2007. DIANA: A computersupported heterogeneous grouping system for teachers to conduct successful small learning groups. Computers in Human Behavior 23, 4 (2007), 1997--2010.Google ScholarDigital Library
- Helen Wauck, Yu-Chun Grace Yen, Wai-Tat Fu, Elizabeth Gerber, Steven P Dow, and Brian P Bailey. 2017. From in the Class or in the Wild?: Peers Provide Better Design Feedback Than External Crowds. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5580--5591.Google ScholarDigital Library
- Miaomiao Wen, Keith Maki, Steven Dow, James D. Herbsleb, and Carolyn Rose. 2017. Supporting Virtual Team Formation through Community-Wide Deliberation. Proc. ACM Hum.-Comput. Interact. 1, CSCW, Article 109 (Dec. 2017), 19 pages. https://doi.org/10.1145/3134744Google ScholarDigital Library
- Miaomiao Wen, Keith Maki, Xu Wang, Steven Dow, James D Herbsleb, and Carolyn Penstein Rosé. 2016. Transactivity as a Predictor of Future Collaborative Knowledge Integration in Team-Based Learning in Online Courses.. In EDM. 533--538.Google Scholar
- Joseph Jay Williams, Juho Kim, Anna Rafferty, Samuel Maldonado, Krzysztof Z Gajos, Walter S Lasecki, and Neil Heffernan. 2016. Axis: Generating explanations at scale with learnersourcing and machine learning. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. ACM, 379--388.Google ScholarDigital Library
- Anita Williams Woolley, Christopher F Chabris, Alex Pentland, Nada Hashmi, and Thomas W Malone. 2010. Evidence for a collective intelligence factor in the performance of human groups. science 330, 6004 (2010), 686--688.Google Scholar
Index Terms
- A Learner-Centered Technique for Collectively Configuring Inputs for an Algorithmic Team Formation Tool
Recommendations
LIFT: Integrating Stakeholder Voices into Algorithmic Team Formation
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsTeam formation tools assume instructors should configure the criteria for creating teams, precluding students from participating in a process affecting their learning experience. We propose LIFT, a novel learner-centered workflow where students propose, ...
Composing Team Compositions: An Examination of Instructors' Current Algorithmic Team Formation Practices
CSCWInstructors using algorithmic team formation tools must decide which criteria (e.g., skills, demographics, etc.) to use to group students into teams based on their teamwork goals, and have many possible sources from which to draw these configurations (...
Team formation instruments to enhance learner interactions in open learning environments
Facilitating team formation for project-based learning in MOOCs.Team formation model describing the project definition and team formation process.Validated principles and algorithms for automated formation of teams. Open learning environments, such as ...
Comments