Skip to main content
SpringerLink
Log in

CPSCoach: The Design and Implementation of Intelligent Collaborative Problem Solving Feedback

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13916))

Included in the following conference series:

Abstract

We present the design of CPSCoach, a fully-automated system that assesses and provides feedback on collaborative problem solving (CPS) competencies during remote collaborations. We leveraged existing data to develop deep NLP models that automatically assess the CPS competencies from speech, achieving moderate to high accuracies (average area under the receiver operating characteristic curve of .78). We engaged 43 participants in an iterative process to design the feedback mechanism, resulting in the first prototype of CPSCoach. We conducted a user study with 20 dyads who engaged with CPSCoach over multiple rounds. Participants thought the system was usable, but they were mixed about the accuracy of the feedback. We discuss design considerations for feedback systems aimed at improving CPS competencies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sun, C., Shute, V.J., Stewart, A.E.B., Yonehiro, J., Duran, N., D’Mello, S.K.: Towards a generalized competency model of collaborative problem solving. Comput Educ. 143, 103672 (2020). https://doi.org/10.1016/j.compedu.2019.103672

  2. Graesser, A.C., Fiore, S.M., Greiff, S., Andrews-Todd, J., Foltz, P.W., Hesse, F.W.: Advancing the science of collaborative problem solving. Psychological Science in the Public Interest. 19, 59–92 (2018). https://doi.org/10.1177/1529100618808244

    Article  Google Scholar 

  3. Schulze, J., Krumm, S.: The “virtual team player”: a review and initial model of knowledge, skills, abilities, and other characteristics for virtual collaboration. Organizational Psychology Review. 7, 66–95 (2017). https://doi.org/10.1177/2041386616675522

    Article  Google Scholar 

  4. Čubraniundefined, D., Storey, M.A.D., Čubranić, D., Storey, M.A.D.: Collaboration support for novice team programming. In: Proceedings of the 2005 International ACM SIGGROUP Conference on Supporting Group Work, pp. 136–139. Association for Computing Machinery, New York, NY, USA (2005). https://doi.org/10.1145/1099203.1099229

  5. Faucett, H.A., Lee, M.L., Carter, S.: I Should Listen More: Real-time Sensing and Feedback of Non-Verbal Communication in Video Telehealth. Proc. ACM Hum.-Comput. Interact. 1, 44:1--44:19 (2017). https://doi.org/10.1145/3134679

  6. Shute, V.J.: Focus on formative feedback. Rev Educ Res. 78, 153–189 (2008). https://doi.org/10.3102/0034654307313795

    Article  Google Scholar 

  7. Shute, V.J., Smith, G., Kuba, R., Dai, C.-P., Rahimi, S., Liu, Z., Almond, R.: The design, development, and testing of learning supports for the physics playground game. Int J Artif Intell Educ. 31, 357–379 (2021). https://doi.org/10.1007/s40593-020-00196-1

    Article  Google Scholar 

  8. Stewart, A.E.B., Keirn, Z., D’Mello, S.K.: Multimodal modeling of collaborative problem-solving facets in triads. User Model User-adapt Interact. 31, 713–751 (2021). https://doi.org/10.1007/s11257-021-09290-y

    Article  Google Scholar 

  9. Wolf, T., et al.: HuggingFace’s Transformers: State-of-the-art Natural Language Processing (2019)

    Google Scholar 

  10. Dixon, W.J., Yuen, K.K.: Trimming and winsorization: A review. Statistische Hefte. 15, 157–170 (1974). https://doi.org/10.1007/BF02922904

    Article  MathSciNet  MATH  Google Scholar 

  11. IBM: https://www.ibm.com/watson/services/speech-to-text/, last accessed 01 May 2018

  12. Brooke, J.: others: SUS-A quick and dirty usability scale. Usability evaluation in industry. 189, 4–7 (1996)

    Google Scholar 

  13. Blandford, A., Furniss, D., Makri, S.: Qualitative HCI Research: Goaing Behind the Scenes. Morgan & Claypool (2016). https://doi.org/10.2200/S00706ED1V01Y201602HCI034

    Article  Google Scholar 

  14. Bostrom, N., Yudkowsky, E.: The ethics of artificial intelligence. The Cambridge handbook of artificial intelligence. 1, 316–334 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Pittsburgh, Pittsburgh, PA, 15260, USA

    Angela E. B. Stewart

  2. University of Colorado Boulder, Boulder, CO, 80309, USA

    Arjun Rao & Sidney K. D’Mello

  3. Arizona State University, Glendale, AZ, 85306, USA

    Amanda Michaels & Nicholas D. Duran

  4. University of Manchester, Manchester, UK

    Chen Sun

  5. Florida State University, Tallahassee, FL, 32306, USA

    Valerie J. Shute

Authors
  1. Angela E. B. Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arjun Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amanda Michaels
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicholas D. Duran
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Valerie J. Shute
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sidney K. D’Mello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Angela E. B. Stewart .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, CA, USA

    Ning Wang

  2. University of British Columbia, Vancouver, BC, Canada

    Genaro Rebolledo-Mendez

  3. North Carolina State University, Raleigh, NC, USA

    Noboru Matsuda

  4. Despacho 3.01, UNED-Grupo de Investigación aDeNu, Madrid, Spain

    Olga C. Santos

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Stewart, A.E.B. et al. (2023). CPSCoach: The Design and Implementation of Intelligent Collaborative Problem Solving Feedback. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_58

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-36272-9_58

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-36271-2

  • Online ISBN: 978-3-031-36272-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation