skip to main content
research-article

Intelligent Coaching Systems: Understanding One-to-many Coaching for Ill-defined Problem Solving

Published:16 April 2023Publication History
Skip Abstract Section

Abstract

One-to-many coaching is a common, yet difficult, coaching technique used in environments with many novices learning to solve ill-defined problems. Intelligent systems might be designed to support 1-to-many coaching but designing such systems requires a 1-to-many coaching model that details novices' challenges, coaches' strategies, and coaches' goals. To build such a model, we conducted interaction analysis on 24 1-to-many coaching sessions with novices developing new products in a university incubator and conducted retrospective analyses with 3 coaches and 30 novices. We contribute a model that demonstrates that coaches in a 1-to-many setting not only need to help novices develop metacognitive skills (just as in 1-to-1 coaching), but also need to utilize the presence and expertise of a group of novices to learn from each other, to mitigate their fear of failures, and provide them accountability. Our model informs design implications for future intelligent coaching systems to (1) assist coaches in monitoring and comparing many novices' progress, learning, and expertise; (2) provide novices with checklists, templates, and scaffolds to help them self-evaluate, seek-help, and summarize learning; (3) showcase failures and growth; and (4) publicize planning and progress to provide accountability.

References

  1. Mark S. Ackerman. 2000. The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility. Human--Computer Interaction 15, 2--3 (Sept. 2000), 179--203. https://doi.org/10.1207/S15327051HCI1523_5Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Mark S. Ackerman, Juri Dachtera, Volkmar Pipek, and Volker Wulf. 2013. Sharing Knowledge and Expertise: The CSCW View of Knowledge Management. Computer Supported Cooperative Work (CSCW) 22, 4--6 (Aug. 2013), 531--573. https://doi.org/10.1007/s10606-013--9192--8Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. M. S. Ackerman and T. W. Malone. 1990. Answer Garden: a tool for growing organizational memory. In Proceedings of the ACM SIGOIS and IEEE CS TC-OA conference on Office information systems (COCS '90). Association for Computing Machinery, New York, NY, USA, 31--39. https://doi.org/10.1145/91474.91485Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mark S. Ackerman and David W. McDonald. 1996. Answer Garden 2: merging organizational memory with collaborative help. In Proceedings of the 1996 ACM conference on Computer supported cooperative work (CSCW '96). Association for Computing Machinery, New York, NY, USA, 97--105. https://doi.org/10.1145/240080.240203Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Susan A. Ambrose, Michael W. Bridges, Michele DiPietro, Marsha C. Lovett, and Marie K. Norman. 2010. How learning works: Seven research-based principles for smart teaching. Jossey-Bass, San Francisco, CA, US. Pages: xxii, 301.Google ScholarGoogle Scholar
  6. Marcela Borge, Craig H. Ganoe, Shin-I Shih, and John M. Carroll. 2012. Patterns of team processes and breakdowns in information analysis tasks. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work (CSCW '12). Association for Computing Machinery, New York, NY, USA, 1105--1114. https://doi.org/10.1145/2145204.2145369Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Rodney Brooks, William Scott Green, R. Glenn Hubbard, Dipak C. Jain, Linda Katehi, George McLendon, James Plummer, Myron Roomkin, and Richard Newton. 2007. Entrepreneurship in American Higher Education. SSRN Scholarly Paper ID 1291290. Social Science Research Network, Rochester, NY. https://doi.org/10.2139/ssrn.1291290Google ScholarGoogle ScholarCross RefCross Ref
  8. C.J. Butz, S. Hua, and R.B. Maguire. 2004. A Web-Based Intelligent Tutoring System for Computer Programming. In IEEE/WIC/ACM International Conference on Web Intelligence (WI'04). 159--165. https://doi.org/10.1109/WI.2004.10104Google ScholarGoogle ScholarCross RefCross Ref
  9. Spencer E. Carlson, Daniel G. Rees Lewis, Leesha V. Maliakal, Elizabeth M. Gerber, and Matthew W. Easterday. 2020. The design risks framework: Understanding metacognition for iteration. Design Studies 70 (Sept. 2020), 100961. https://doi.org/10.1016/j.destud.2020.100961Google ScholarGoogle ScholarCross RefCross Ref
  10. Joel Chan, Steven Dang, and Steven P Dow. 2016. Improving crowd innovation with expert facilitation. In Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, Vol. 27. 1223--1235. https://doi.org/10.1145/2818048.2820023Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chun-Wei Chiang, Anna Kasunic, and Saiph Savage. 2018. Crowd Coach: Peer Coaching for Crowd Workers' Skill Growth. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (Nov. 2018), 37:1--37:17. https://doi.org/10.1145/3274306Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Bart Clarysse, Mike Wright, and Jonas Van Hove. 2016. A Look Inside Accelerators in the United Kingdom: Building Technology Businesses. In Technology Entrepreneurship and Business Incubation. https://doi.org/10.1142/9781783269778_0003Google ScholarGoogle ScholarCross RefCross Ref
  13. Allan Collins, John Seely Brown, and Susan E. Newman. 1989. Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In Knowing, learning, and instruction: Essays in honor of Robert Glaser. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ, US, 453--494.Google ScholarGoogle Scholar
  14. A. Corbin, J. & Strauss. 2015. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (4th Edition). arXiv:1011.1669Google ScholarGoogle Scholar
  15. P. G. Dodgson and J. V. Wood. 1998. Self-esteem and the cognitive accessibility of strengths and weaknesses after failure. Journal of Personality and Social Psychology 75, 1 (July 1998), 178--197. https://doi.org/10.1037//0022--3514.75.1.178Google ScholarGoogle ScholarCross RefCross Ref
  16. Matthew W. Easterday, Elizabeth M. Gerber, and Daniel G. Rees Lewis. 2018. Social Innovation Networks: A New Approach to Social Design Education and Impact. Design Issues 34, 2 (April 2018), 64--76. https://doi.org/10.1162/DESI_a_00486 Conference Name: Design Issues.Google ScholarGoogle ScholarCross RefCross Ref
  17. BJ Fogg. 2009. A behavior model for persuasive design. In Proceedings of the 4th International Conference on Persuasive Technology (Persuasive '09). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/1541948.1541999Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Eureka Foong, Steven P. Dow, Brian P. Bailey, and Elizabeth M. Gerber. 2017. Online Feedback Exchange: A Framework for Understanding the Socio-Psychological Factors. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 4454--4467. https://doi.org/10.1145/3025453.3025791Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Eureka Foong, Joy O. Kim, Mira Dontcheva, and Elizabeth M. Gerber. 2021. CrowdFolio: Understanding How Holistic and Decomposed Workflows Influence Feedback on Online Portfolios. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 22:1--22:31. https://doi.org/10.1145/3449096Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Gensler. 2021. The Rise of Academic Incubators. https://www.gensler.com/blog/the-rise-of-academic-incubatorsGoogle ScholarGoogle Scholar
  21. Elizabeth Gerber. 2014. Design for America: Organizing for Civic Innovation. Interactions 21 (March 2014), 42--47. https://doi.org/10.1145/2567840Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Debra M. Gilbuena, Benjamin U. Sherrett, Edith S. Gummer, Audrey B. Champagne, and Milo D. Koretsky. 2015. Feedback on Professional Skills as Enculturation into Communities of Practice. Journal of Engineering Education 104, 1 (2015), 7--34. https://doi.org/10.1002/jee.20061 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/jee.20061.Google ScholarGoogle ScholarCross RefCross Ref
  23. Jonathan Grudin. 1988. Why CSCW applications fail: problems in the design and evaluation of organizational interfaces. In Proceedings of the 1988 ACM conference on Computer-supported cooperative work - CSCW '88. ACM Press, Portland, Oregon, United States, 85--93. https://doi.org/10.1145/62266.62273Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Philip J. Guo. 2015. Codeopticon: Real-Time, One-To-Many Human Tutoring for Computer Programming. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (UIST '15). Association for Computing Machinery, New York, NY, USA, 599--608. https://doi.org/10.1145/2807442.2807469Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Allyson Hadwin, Sanna Järvelä, and Mariel Miller. 2017. Self-Regulation, Co-Regulation, and Shared Regulation in Collaborative Learning Environments. In Handbook of Self-Regulation of Learning and Performance (2 ed.), Dale H. Schunk and Jeffrey A. Greene (Eds.). Routledge, 83--106. https://doi.org/10.4324/9781315697048--6Google ScholarGoogle ScholarCross RefCross Ref
  26. Hazel Hall and Brian Davison. 2007. Social software as support in hybrid learning environments: The value of the blog as a tool for reflective learning and peer support. Library & Information Science Research 29, 2 (June 2007), 163--187. https://doi.org/10.1016/j.lisr.2007.04.007Google ScholarGoogle ScholarCross RefCross Ref
  27. Cindy E. Hmelo-Silver. 2004. Problem-Based Learning: What and How Do Students Learn? Educational Psychology Review 16, 3 (Sept. 2004), 235--266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3Google ScholarGoogle ScholarCross RefCross Ref
  28. Cindy E. Hmelo-Silver and Howard S. Barrows. 2006. Goals and Strategies of a Problem-based Learning Facilitator. Interdisciplinary Journal of Problem-Based Learning 1, 1 (May 2006). https://doi.org/10.7771/1541--5015.1004Google ScholarGoogle ScholarCross RefCross Ref
  29. Cindy E. Hmelo-Silver and Howard S. Barrows. 2008. Facilitating Collaborative Knowledge Building. Cognition and Instruction 26, 1 (Jan. 2008), 48--94. https://doi.org/10.1080/07370000701798495 Publisher: Routledge _eprint: https://doi.org/10.1080/07370000701798495.Google ScholarGoogle ScholarCross RefCross Ref
  30. Tom Hope, Doug Downey, Oren Etzioni, Daniel S. Weld, and Eric Horvitz. 2022. A Computational Inflection for Scientific Discovery. https://doi.org/10.48550/arXiv.2205.02007 arXiv:2205.02007 [cs].Google ScholarGoogle ScholarCross RefCross Ref
  31. Eric Horvitz. 1999. Principles of mixed-initiative user interfaces. Conference on Human Factors in Computing Systems - Proceedings (1999), 159--166. https://doi.org/10.1145/302979.303030 ISBN: 0201485591.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Julie S. Hui, Darren Gergle, and Elizabeth M. Gerber. 2018. IntroAssist: A Tool to Support Writing Introductory Help Requests. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3173596Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Edwin Hutchins. 1995. How a Cockpit Remembers Its Speeds. Cognitive Science 19, 3 (1995), 265--288. https://doi.org/10.1207/s15516709cog1903_1 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog1903_1.Google ScholarGoogle ScholarCross RefCross Ref
  34. David Jonassen, Johannes Strobel, and Chwee Beng Lee. 2006. Everyday Problem Solving in Engineering: Lessons for Engineering Educators. Journal of Engineering Education 95, 2 (2006), 139--151. https://doi.org/10.1002/j.2168--9830.2006.tb00885.x _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/j.2168--9830.2006.tb00885.x.Google ScholarGoogle ScholarCross RefCross Ref
  35. David H. Jonassen and Woei Hung. 2008. All Problems are Not Equal: Implications for Problem-Based Learning. Interdisciplinary Journal of Problem-Based Learning 2, 2 (Oct. 2008). https://doi.org/10.7771/1541--5015.1080Google ScholarGoogle ScholarCross RefCross Ref
  36. Brigitte Jordan and Austin Henderson. 1995. Interaction Analysis: Foundations and Practice. Journal of the Learning Sciences 4, 1 (Jan. 1995), 39--103. https://doi.org/10.1207/s15327809jls0401_2 Publisher: Routledge _eprint: https://doi.org/10.1207/s15327809jls0401_2.Google ScholarGoogle ScholarCross RefCross Ref
  37. Joy Kim, Avi Bagla, and Michael S. Bernstein. 2015. Designing Creativity Support Tools for Failure. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (C&C '15). Association for Computing Machinery, New York, NY, USA, 157--160. https://doi.org/10.1145/2757226.2764542Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Kenneth R. Koedinger, Emma Brunskill, Ryan S.J.d. Baker, Elizabeth A. McLaughlin, and John Stamper. 2013. New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization. AI Magazine 34, 3 (Sept. 2013), 27--41. https://doi.org/10.1609/aimag.v34i3.2484Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Chinmay Kulkarni, Michael S. Bernstein, and Scott Klemmer. 2015. PeerStudio: Rapid peer feedback emphasizes revision and improves performance. In L@S 2015 - 2nd ACM Conference on Learning at Scale. Association for Computing Machinery, Inc, New York, NY, USA, 75--84. https://doi.org/10.1145/2724660.2724670Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Vivian Lai, Chacha Chen, Q. Vera Liao, Alison Smith-Renner, and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. https://doi.org/10.48550/arXiv.2112.11471 arXiv:2112.11471 [cs].Google ScholarGoogle ScholarCross RefCross Ref
  41. Marianna Lamnina, Helena Connolly, and V. Aleven. 2018. The Invention Coach: A computer-based environment that supports the transfer of STEM concepts. https://doi.org/10.1119/perc.2017.pr.052Google ScholarGoogle ScholarCross RefCross Ref
  42. J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 1 (1977), 159--174. https://www.jstor.org/stable/2529310 Publisher: [Wiley, International Biometric Society].Google ScholarGoogle ScholarCross RefCross Ref
  43. Chunggi Lee, Sanghoon Kim, Dongyun Han, Hongjun Yang, Young-Woo Park, Bum Chul Kwon, and Sungahn Ko. 2020. GUIComp: A GUI Design Assistant with Real-Time, Multi-Faceted Feedback. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3313831.3376327Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Yoonjoo Lee, John Joon Young Chung, Tae Soo Kim, Jean Y Song, and Juho Kim. 2022. Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, 1--18. https://doi.org/10.1145/3491102.3502087Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Marcia Linn, Kevin McElhaney, Libby Gerard, and Camillia Matuk. 2018. Inquiry learning and opportunities for technology.Google ScholarGoogle Scholar
  46. Ching Liu, Juho Kim, and Hao-Chuan Wang. 2018. ConceptScape: Collaborative Concept Mapping for Video Learning. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3173574.3173961Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Marsha C. Lovett. 1998. Cognitive Task Analysis in Service of Intelligent Tutoring System Design: A Case Study in Statistics. In Intelligent Tutoring Systems, Barry P. Goettl, Henry M. Halff, Carol L. Redfield, and Valerie J. Shute (Eds.). Springer, Berlin, Heidelberg, 234--243. https://doi.org/10.1007/3--540--68716--5_29Google ScholarGoogle ScholarCross RefCross Ref
  48. Jingyan Lu, Susan Bridges, and Cindy E. Hmelo-Silver. 2014. Problem-based learning. In The Cambridge Handbook of the Learning Sciences, Second Edition. https://doi.org/10.1017/CBO9781139519526.019Google ScholarGoogle ScholarCross RefCross Ref
  49. Kurt Luther and Amy Bruckman. 2008. Leadership in online creative collaboration. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (CSCW '08). Association for Computing Machinery, New York, NY, USA, 343--352. https://doi.org/10.1145/1460563.1460619Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Kurt Luther, Casey Fiesler, and Amy Bruckman. 2013. Redistributing leadership in online creative collaboration. In Proceedings of the 2013 conference on Computer supported cooperative work (CSCW '13). Association for Computing Machinery, New York, NY, USA, 1007--1022. https://doi.org/10.1145/2441776.2441891Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Collin F Lynch, Kevin D Ashley, Vincent Aleven, and Niels Pinkwart. 2006. Defining ?Ill-Defined Domains"; A literature survey. (2006), 10.Google ScholarGoogle Scholar
  52. Thomas W. Malone. 2018. How Human-Computer 'Superminds' Are Redefining the Future of Work. MIT Sloan Management Review 59, 4 (2018), 34--41. https://www.proquest.com/docview/2057250594/abstract/191CBE32E9BE490BPQ/1 Num Pages: 34--41 Place: Cambridge, United States Publisher: Massachusetts Institute of Technology, Cambridge, MA Section: COLLECTIVE INTELLIGENCE.Google ScholarGoogle Scholar
  53. David W. McDonald and Mark S. Ackerman. 2000. Expertise recommender: a flexible recommendation system and architecture. In Proceedings of the 2000 ACM conference on Computer supported cooperative work (CSCW '00). Association for Computing Machinery, New York, NY, USA, 231--240. https://doi.org/10.1145/358916.358994Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Robert C. Miller, Haoqi Zhang, Eric Gilbert, and Elizabeth Gerber. 2014. Pair research: matching people for collaboration, learning, and productivity. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14). Association for Computing Machinery, New York, NY, USA, 1043--1048. https://doi.org/10.1145/2531602.2531703Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Elliot Mitchell, Noemie Elhadad, and Lena Mamykina. 2022. Examining AI Methods for Micro-Coaching Dialogs. In CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, 1--24. https://doi.org/10.1145/3491102.3501886Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Elliot G. Mitchell, Rosa Maimone, Andrea Cassells, Jonathan N. Tobin, Patricia Davidson, Arlene M. Smaldone, and Lena Mamykina. 2021. Automated vs. Human Health Coaching: Exploring Participant and Practitioner Experiences. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 99:1--99:37. https://doi.org/10.1145/3449173Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Molly Jane Nicholas, Sarah Sterman, and Eric Paulos. 2022. Creative and Motivational Strategies Used by Expert Creative Practitioners. In Creativity and Cognition (C&C '22). Association for Computing Machinery, New York, NY, USA, 323--335. https://doi.org/10.1145/3527927.3532870Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Judith Odili Uchidiuno, Jessica Hammer, Ken Koedinger, and Amy Ogan. 2021. Fostering Equitable Help-Seeking for K-3 Students in Low Income and Rural Contexts. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Number 684. Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3411764.3445144Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Charlotte Pauwels, Bart Clarysse, Mike Wright, and Jonas Van Hove. 2016. Understanding a new generation incubation model: The accelerator. Technovation 50--51 (apr 2016), 13--24. https://doi.org/10.1016/j.technovation.2015.09.003Google ScholarGoogle ScholarCross RefCross Ref
  60. Daniel Rees Lewis, Elizabeth Gerber, Spencer Carlson, and Matthew Easterday. 2019. Opportunities for educational innovations in authentic project-based learning: understanding instructor perceived challenges to design for adoption. Educational Technology Research and Development 67 (May 2019). https://doi.org/10.1007/s11423-019-09673--4Google ScholarGoogle ScholarCross RefCross Ref
  61. Daniel Rees Lewis, Elizabeth Gerber, and Matthew Easterday. 2019. Assessing Iterative Planning for Real-world Design Teams. https://doi.org/10.13140/RG.2.2.20452.83840Google ScholarGoogle ScholarCross RefCross Ref
  62. Daniel Rees Lewis, Jamie Gorson, Leesha Maliakal, Spencer Carlson, Elizabeth Gerber, Matthew Easterday, and Christopher Riesbeck. 2018. Planning to Iterate: Supporting Iterative Practices for Real-world Ill-structured Problem-solving. Presented at the 13th International Conference of the Learning Sciences, London, 2018.Google ScholarGoogle Scholar
  63. Horst W. J. Rittel and Melvin M. Webber. 1973. Dilemmas in a General Theory of Planning. Policy Sciences 4, 2 (1973), 155--169. https://www.jstor.org/stable/4531523 Publisher: Springer.Google ScholarGoogle ScholarCross RefCross Ref
  64. Donald A. Schon. 1987. Teaching artistry through reflection in action.Google ScholarGoogle Scholar
  65. Richard A. Schwier. 2010. Focusing Educational Technology Research on Informal Learning Environments. Contemporary Educational Technology 1, 1 (March 2010). https://doi.org/10.30935/cedtech/5965Google ScholarGoogle ScholarCross RefCross Ref
  66. Amy Shannon, Jessica Hammer, H. Thurston, Natalie Diehl, and Steven Dow. 2016. PeerPresents: A Web-Based System for In-Class Peer Feedback during Student Presentations. Conference on Designing Interactive Systems (2016). https://doi.org/10.1145/2901790.2901816Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Herbert A. Simon. 1977. The Structure of Ill-Structured Problems. In Models of Discovery: And Other Topics in the Methods of Science, Herbert A. Simon (Ed.). Springer Netherlands, Dordrecht, 304--325. https://doi.org/10.1007/978--94-010--9521--1_17Google ScholarGoogle ScholarCross RefCross Ref
  68. Ryo Suzuki, Niloufar Salehi, Michelle S. Lam, Juan C. Marroquin, and Michael S. Bernstein. 2016. Atelier: Repurposing Expert Crowdsourcing Tasks as Micro-internships. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). Association for Computing Machinery, New York, NY, USA, 2645--2656. https://doi.org/10.1145/2858036.2858121Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Colby Tofel-Grehl and David F. Feldon. 2013. Cognitive Task Analysis--Based Training: A Meta-Analysis of Studies. Journal of Cognitive Engineering and Decision Making 7, 3 (Sept. 2013), 293--304. https://doi.org/10.1177/1555343412474821 Publisher: SAGE Publications.Google ScholarGoogle ScholarCross RefCross Ref
  70. Cesar Torres, Sarah Sterman, Molly Nicholas, Richard Lin, Eric Pai, and Eric Paulos. 2018. Guardians of Practice: A Contextual Inquiry of Failure-Mitigation Strategies within Creative Practices. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18). Association for Computing Machinery, New York, NY, USA, 1259--1267. https://doi.org/10.1145/3196709.3196795Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Kurt VanLehn, Collin Lynch, Kay Schulze, Joel A. Shapiro, Robert Shelby, Linwood Taylor, Don Treacy, Anders Weinstein, and Mary Wintersgill. 2005. The Andes Physics Tutoring System: Lessons Learned. International Journal of Artificial Intelligence in Education 15, 3 (Jan. 2005), 147--204. https://content.iospress.com/articles/international-journal-of-artificial-intelligence-in-education/jai15--3-02 Publisher: IOS Press.Google ScholarGoogle Scholar
  72. Madhurima Vardhan, Narayan Hegde, Srujana Merugu, Shantanu Prabhat, Deepak Nathani, Martin Seneviratne, Nur Muhammad, Pranay Reddy, Sriram Lakshminarasimhan, Rahul Singh, Karina Lorenzana, Eshan Motwani, Partha Talukdar, and Aravindan Raghuveer. 2022. Walking with PACE - Personalized and Automated Coaching Engine. In Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization (UMAP '22). Association for Computing Machinery, New York, NY, USA, 57--68. https://doi.org/10.1145/3503252.3531301Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Maria Virvou, Christos Troussas, and Sotirios-Christos Sidiropoulos. 2012. Collaborative Support in a Multilingual Tutoring System. In 2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. 502--505. https://doi.org/10.1109/IIH-MSP.2012.127Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Erin Walker, Nikol Rummel, and Kenneth R. Koedinger. 2014. Adaptive Intelligent Support to Improve Peer Tutoring in Algebra. International Journal of Artificial Intelligence in Education 24, 1 (Jan. 2014), 33--61. https://doi.org/10.1007/s40593-013-0001--9Google ScholarGoogle ScholarCross RefCross Ref
  75. Thiemo Wambsganss, Tobias Kueng, Matthias Soellner, and Jan Marco Leimeister. 2021. ArgueTutor: An Adaptive Dialog-Based Learning System for Argumentation Skills. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3411764.3445781Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Rainer Winkler, Matthias Söllner, and Jan Marco Leimeister. 2021. Enhancing problem-solving skills with smart personal assistant technology. Computers & Education 165 (May 2021), 104148. https://doi.org/10.1016/j.compedu.2021.104148Google ScholarGoogle ScholarCross RefCross Ref
  77. Philip H. Winne and Roger Azevedo. 2014. Metacognition. In The Cambridge handbook of the learning sciences, 2nd ed. Cambridge University Press, New York, NY, US, 63--87. https://doi.org/10.1017/CBO9781139519526.006Google ScholarGoogle ScholarCross RefCross Ref
  78. B.L. William Wong. 2003. Critical Decision Method data analysis. The handbook of task analysis for human-computer interaction, 327--346.Google ScholarGoogle Scholar
  79. Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, Baltimore Maryland USA, 1433--1444. https://doi.org/10.1145/2531602.2531604Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Haoqi Zhang, Matthew W. Easterday, Elizabeth M. Gerber, Daniel Rees Lewis, and Leesha Maliakal. 2017. Agile Research Studios: Orchestrating Communities of Practice to Advance Research Training. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). Association for Computing Machinery, New York, NY, USA, 220--232. https://doi.org/10.1145/2998181.2998199Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Haiyi Zhu, Robert Kraut, and Aniket Kittur. 2012. Effectiveness of shared leadership in online communities. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work (CSCW '12). Association for Computing Machinery, New York, NY, USA, 407--416. https://doi.org/10.1145/2145204.2145269Google ScholarGoogle ScholarDigital LibraryDigital Library
  82. Haiyi Zhu, Robert E. Kraut, Yi-Chia Wang, and Aniket Kittur. 2011. Identifying shared leadership in Wikipedia. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). Association for Computing Machinery, New York, NY, USA, 3431--3434. https://doi.org/10.1145/1978942.1979453Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Intelligent Coaching Systems: Understanding One-to-many Coaching for Ill-defined Problem Solving

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image Proceedings of the ACM on Human-Computer Interaction
      Proceedings of the ACM on Human-Computer Interaction  Volume 7, Issue CSCW1
      CSCW
      April 2023
      3836 pages
      EISSN:2573-0142
      DOI:10.1145/3593053
      Issue’s Table of Contents

      Copyright © 2023 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 16 April 2023
      Published in pacmhci Volume 7, Issue CSCW1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader