Skip to main content
SpringerLink
Log in
Book cover

International Conference on Artificial Intelligence in Education

AIED 2021: Artificial Intelligence in Education pp 293–305Cite as

Multidimensional Team Communication Modeling for Adaptive Team Training: A Hybrid Deep Learning and Graphical Modeling Framework

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12748))

Abstract

Team communication modeling offers great potential for adaptive learning environments for team training. However, the complex dynamics of team communication pose significant challenges for team communication modeling. To address these challenges, we present a hybrid framework integrating deep learning and probabilistic graphical models that analyzes team communication utterances with respect to the intent of the utterance and the directional flow of communication within the team. The hybrid framework utilizes conditional random fields (CRFs) that use deep learning-based contextual, distributed language representations extracted from team members’ utterances. An evaluation with communication data collected from six teams during a live training exercise indicate that linear-chain CRFs utilizing ELMo utterance embeddings (1) outperform both multi-task and single-task variants of stacked bidirectional long short-term memory networks using the same distributed representations of the utterances, (2) outperform a hybrid approach that uses non-contextual utterance representations for the dialogue classification tasks, and (3) demonstrate promising domain-transfer capabilities. The findings suggest that the hybrid multidimensional team communication analysis framework can accurately recognize speaker intent and model the directional flow of team communication to guide adaptivity in team training environments.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Salas, E., et al.: Does team training improve team performance? Meta-Anal. Hum. Factors 50(6), 903–933 (2008)

    Article  Google Scholar 

  2. Johnston, J.H., Burke, C.S., Milham, L.A., Ross, W.M., Salas, E.: Challenges and propositions for developing effective team training with adaptive tutors. In: Johnston, J., Sottilare, R., Sinatra, A., Burke, C. (eds.) Building Intelligent Tutoring Systems for Teams, vol. 19, pp. 75–97. Emerald Publishing Limited (2018)

    Google Scholar 

  3. Sottilare, R.A., Burke, C.S., Salas, E., Sinatra, A.M., Johnston, J.H., Gilbert, S.B.: Designing adaptive instruction for teams: a meta-analysis. Int. J. Artif. Intell. Educ. 28(2), 225–264 (2018)

    Article  Google Scholar 

  4. Smith-Jentsch, K.A., Johnston, J.H., Payne, S.C.: Measuring team-related expertise in complex environments. In: Cannon-Bowers, J.A., Salas, E. (eds.). Making Decisions Under stress: Implications for Individual and Team Training, pp. 61–87. American Psychological Association (1998)

    Google Scholar 

  5. Rousseau, V., Aubé, C., Savoie, A.: Teamwork behaviors: a review and an integration of frameworks. Small Group Res. 37(5), 540–570 (2006)

    Article  Google Scholar 

  6. Marks, M.A., Mathieu, J.E., Zaccaro, S.J.: A temporally based framework and taxonomy of team processes. Acad. Manage. Rev. 26(3), 356–376 (2001)

    Article  Google Scholar 

  7. Stout, R.J., Cannon-Bowers, J.A., Salas, E.: The role of shared mental models in developing team situational awareness: Implications for training. In: Salas, E. (ed.) Situational Awareness, pp. 287–318. Routledge (2017)

    Google Scholar 

  8. Gorman, J.C., Foltz, P.W., Kiekel, P.A., Martin, M.J., Cooke, N.J.: Evaluation of latent semantic analysis-based measures of team communications content. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 47, no. 3, pp. 424–428. SAGE Publications (2003)

    Google Scholar 

  9. Deng, L., Liu, Y.: Deep learning in natural language processing. Springer (2018)

    Google Scholar 

  10. Yu, B., Fan, Z.: A comprehensive review of conditional random fields: variants, hybrids and applications. Artif. Intell. Rev. 53(6), 4289–4333 (2019). https://doi.org/10.1007/s10462-019-09793-6

    Article  Google Scholar 

  11. Stolcke, A., et al.: Dialogue act modeling for automatic tagging and recognition of conversational speech. Comput. Linguist. 26(3), 339–373 (2000)

    Article  Google Scholar 

  12. Johnston, J.H., et al.: A team training field research study: extending a theory of team development. Front. Psychol. 10, 1480 (2019)

    Article  Google Scholar 

  13. McNamara, D., Allen, L., Crossley, S., Dascalu, M., Perret, C.A.: Natural language processing and learning analytics. In: Handbook of Learning Analytics, pp. 93–104 (2017)

    Google Scholar 

  14. Litman, D.: Natural language processing for enhancing teaching and learning. In: AAAI Conference on Artificial Intelligence, pp. 4170–4176. AAAI (2016)

    Google Scholar 

  15. Kumar, V.S., Boulanger, D.: Automated essay scoring and the deep learning black box: how are rubric scores determined? Int. J. Artificial Intell. Educ. 1–47 (2020). https://doi.org/10.1007/s40593-020-00211-5

  16. Sung, C., Dhamecha, T.I., Mukhi, N.: Improving short answer grading using transformer-based pre-training. In: International Conference on Artificial Intelligence in Education, pp. 469–481. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23204-7_39

  17. Clarke, S.N., Resnick, L.B., Rosé, C.P.: Discourse analytics for classroom learning. In: Learning Analytics in Education, vol. 139 (2018)

    Google Scholar 

  18. Jensen, E., Dale, M., Donnelly, P.J., Stone, C., Kelly, S., Godley, A., D’Mello, S.K.: Toward automated feedback on teacher discourse to enhance teacher learning. In: 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–13. ACM (2020)

    Google Scholar 

  19. Balyan, R., McCarthy, K.S., McNamara, D.S.: Applying natural language processing and hierarchical machine learning approaches to text difficulty classification. Int. J. Artif. Intell. Educ. 30(3), 337–370 (2020)

    Article  Google Scholar 

  20. Katz, S., Albacete, P., Chounta, I.A., Jordan, P., McLaren, B.M., Zapata-Rivera, D.: Linking dialogue with student modelling to create an adaptive tutoring system for conceptual physics. Int. J. Artif. Intell. Educ. 1–49 (2021). https://doi.org/10.1007/s40593-020-00226-y

  21. Stone, C., Quirk, A., Gardener, M., Hutt, S., Duckworth, A.L., D'Mello, S.K.: Language as thought: using natural language processing to model noncognitive traits that predict college success. In: International Conference on Learning Analytics & Knowledge, pp. 320–329. ACM (2019)

    Google Scholar 

  22. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  23. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)

  24. Hirschberg, J., Manning, C.D.: Advances in natural language processing. Science 349(6245), 261–266 (2015)

    Article  MathSciNet  Google Scholar 

  25. Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing. IEEE Comput. Intell. Mag. 13(3), 55–75 (2018)

    Article  Google Scholar 

  26. Sullivan, F.R., Keith, P.K.: Exploring the potential of natural language processing to support microgenetic analysis of collaborative learning discussions. Br. J. Edu. Technol. 50(6), 3047–3063 (2019)

    Article  Google Scholar 

  27. Park, K., Sohn, H., Mott, B., Min, W., Saleh, A., Glazewski, K., Hmelo-Silver, C., Lester, J.: Detecting disruptive talk in student chat-based discussion within collaborative game-based learning environments. In: International Learning Analytics and Knowledge Conference, pp. 405–415. ACM (2021)

    Google Scholar 

  28. Carpenter, D., Emerson, A., Mott, B.W., Saleh, A., Glazewski, K.D., Hmelo-Silver, C.E., Lester, J.C.: Detecting off-task behavior from student dialogue in game-based collaborative learning. In: International Conference on Artificial Intelligence in Education, pp. 55–66. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52237-7_5

  29. Marlow, S., Lacerenza, C., Paoletti, J., Burke, C., Salas, E.: Does team communication represent a one-size-fits-all approach? a meta-analysis of team communication and performance. Organ. Behav. Hum. Decis. Process. 144, 145–170 (2018)

    Article  Google Scholar 

  30. Foltz, P.W.: Automating the assessment of team collaboration through communication analysis. Design Recommendations for Intell. Tutor. Syst. 6, 179–185 (2018)

    Google Scholar 

  31. Dowell, N.M.M., Nixon, T.M., Graesser, A.C.: Group communication analysis: a computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behav. Res. Methods 51(3), 1007–1041 (2018). https://doi.org/10.3758/s13428-018-1102-z

    Article  Google Scholar 

  32. Ayala, D.F.M., Balasingam, B., McComb, S., Pattipati, K.R.: Markov modeling and analysis of team communication. IEEE Trans. Syst. Man Cybern.: Syst. 50(4), 1230–1241 (2020)

    Article  Google Scholar 

  33. Enayet, A., Sukthankar, G.: Analyzing team performance with embeddings from multiparty dialogues. arXiv preprint arXiv:2101.09421 (2021)

  34. Yu, M., Litman, D., Paletz, S.: Investigating the relationship between multi-party linguistic entrainment, team characteristics, and the perception of team social outcomes. In: International Florida Artificial Intelligence Research Society Conference, pp. 227–232. AAAI (2019)

    Google Scholar 

  35. Saville, J.D., Spain, R., Johnston, J., Lester, J.: Exploration of team communication behaviors from a live training event. In: The 12th International Conference on Applied Human Factors and Ergonomics (2021, to appear)

    Google Scholar 

  36. Lafferty, J., McCallum, A., Pereira, F.C.: Conditional random fields: probabilistic models for segmenting and labeling sequence data (2001)

    Google Scholar 

  37. Kumar, H., Agarwal, A., Dasgupta, R., Joshi, S.: Dialogue act sequence labeling using hierarchical encoder with CRF. In: AAAI Conference on Artificial Intelligence, pp. 3440–3447 (2018)

    Google Scholar 

  38. Tran, T.U., Hoang, H.T.T., Huynh, H.X.: Bidirectional independently long short-term memory and conditional random field integrated model for aspect extraction in sentiment analysis. In: Frontiers in Intelligent Computing: Theory and Applications, pp. 131–140. Springer (2020). https://doi.org/10.1007/978-981-13-9920-6_14

  39. Sutton, C., McCallum, A.: An introduction to conditional random fields for relational learning. In: Introduction to Statistical Relational Learning, vol. 2, pp. 93–128 (2006)

    Google Scholar 

  40. Lacoste-Julien, S., Jaggi, M., Schmidt, M., Pletscher, P.: Block-coordinate Frank-Wolfe optimization for structural SVMs. In: International Conference on Machine Learning, pp. 53–61. PMLR (2013)

    Google Scholar 

  41. Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  42. Müller, A.C., Behnke, S.: PyStruct: learning structured prediction in python. J. Mach. Learn. Res. 15(1), 2055–2060 (2014)

    MathSciNet  MATH  Google Scholar 

  43. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)

    Article  Google Scholar 

  44. Zhang, Y., Yang, Q.: A survey on multi-task learning. arXiv preprint arXiv:1707.08114 (2017)

  45. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

Download references

Acknowledgements

The research was supported by the U.S. Army Research Laboratory under cooperative agreement #W912CG-19–2-0001. Any opinions, findings, and conclusions expressed in this paper are those of the authors and do not necessarily reflect the views of the U.S. Army.

Author information

Authors and Affiliations

  1. North Carolina State University, Raleigh, NC, 27695, USA

    Wookhee Min, Randall Spain, Jason D. Saville, Bradford Mott & James Lester

  2. U.S. Army Combat Capabilities Development Command, Orlando, FL, 32826, USA

    Keith Brawner & Joan Johnston

Authors
  1. Wookhee Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Randall Spain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jason D. Saville
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bradford Mott
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Keith Brawner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Joan Johnston
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. James Lester
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wookhee Min .

Editor information

Editors and Affiliations

  1. Technion – Israel Institute of Technology, Haifa, Israel

    Ido Roll

  2. Arizona State University, Tempe, AZ, USA

    Danielle McNamara

  3. Utrecht University, Utrecht, The Netherlands

    Sergey Sosnovsky

  4. London Knowledge Lab, London, UK

    Rose Luckin

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Min, W. et al. (2021). Multidimensional Team Communication Modeling for Adaptive Team Training: A Hybrid Deep Learning and Graphical Modeling Framework. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2021. Lecture Notes in Computer Science(), vol 12748. Springer, Cham. https://doi.org/10.1007/978-3-030-78292-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78292-4_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78291-7

  • Online ISBN: 978-3-030-78292-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation