Skip to main content

Advertisement

SpringerLink
Log in

Using Dialogue Features to Predict Trouble During Collaborative Learning

  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

An Erratum to this article was published on 01 March 2006

Abstract

A web-based, collaborative distance-learning system that will allow groups of students to interact with each other remotely and with an intelligent electronic agent that will aid them in their learning has the potential for improving on-line learning. The agent would follow the discussion and interact with the participants when it detects learning trouble of some sort, such as confusion about the problem they are working on or a participant who is dominating the discussion or not interacting with the other participants. In order to recognize problems in the dialogue, we investigated conversational elements that can be utilized as predictors for effective and ineffective interaction between human students. These elements can serve as the basis for student and group models. In this paper, we discuss group interaction during collaborative learning, our representation of participant dialogue, and the statistical models we are using to determine the role being played by a participant at any point in the dialogue and the effectiveness of the group. We also describe student and group models that can be built using conversational elements and discuss one set that we built to illustrate their potential value in collaborative learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • J.R. Anderson (1990) The Adaptive Character of Thought Erlbaum Mahwah, NJ

    Google Scholar 

  • J.R. Anderson C.F. Boyle A. Corbett M. Lewis (1990) ArticleTitleCognitive modeling and intelligent tutoring Artificial Intelligence 42 7–49 Occurrence Handle10.1016/0004-3702(90)90093-F

    Article  Google Scholar 

  • Ang J., Dhillon R., Krupski A., Shriberg E., Stolcke A. (2002). Prosody-based automatic detection of annoyance and frustration in human–computer dialog. In: Proceedings of ICSLP-2002, Denver, CO, pp. 2037–2040

  • Batliner A., Noth E., Buckow J., Huber R., Warnke V., Niemann H. (2001). Whence and whither prosody in automatic speech understanding: A case study. In: Proceedings ISCA Tutorial and Research Workshop on Prosody and Speech Recognition and Understanding, Red Bank, NJ

  • R.M. Belbin (2004) Management Teams EditionNumber2 Butterworth-Heinemann Oxford

    Google Scholar 

  • K. Benne P. Sheats (1948) ArticleTitleFunctional roles of group members Journal of Social Issues 4 41–49

    Google Scholar 

  • S. Bloom B. (1984) ArticleTitleThe two sigma problem: The search for methods of group instruction as effective as one-to-one tutoring Educational Researcher. 13 IssueID6 4–16 Occurrence Handle10.2307/1175554

    Article  Google Scholar 

  • Bosma W., André E. (2004). Exploiting emotions to disambiguate dialogue acts. In: Proceedings of the Conference on Intelligent User Interfaces, Portugal, pp. 85–92

  • A. Brown A. Palincsar (1989) Guided, cooperative learning and individual knowledge acquisition Resnick. Lauren B. (Eds) Knowledge, Learning and Instruction. Lawrence Erlbaum Hillsdale, NJ 393–451

    Google Scholar 

  • S. Bull P. Brna H. Pain (1995) ArticleTitleExtending the scope of the student model User Modeling and User-Adapted Interaction 5 45–65 Occurrence Handle10.1007/BF01101801

    Article  Google Scholar 

  • M. Burton P. Brna R. Pilkington Clarissa. (2000) ArticleTitleA Laboratory for the Modelling of Collaboration International Journal of Artificial Intelligence in Education 11 79–105

    Google Scholar 

  • J. Carletta (1996) ArticleTitleAssessing agreement on classification tasks: The Kappa statistic Computational Linguistics. 22 IssueID2 249–254

    Google Scholar 

  • Chan T., Baskin A. (1988). Studying with the prince. The computer as a learning companion. In: Proceedings of the ITS-88 Conference, (Montréal, Canada), pp. 194–200

  • M. Chi (1996) ArticleTitleConstructing self-explanations and scaffolded explanations in tutoring Applied Cognitive Psychology, 10 S33–S49 Occurrence Handle1431973 Occurrence Handle10.1002/(SICI)1099-0720(199611)10:7<33::AID-ACP436>3.0.CO;2-E

    Article  MathSciNet  Google Scholar 

  • A. Collins J.S. Brown S. Newman (1989) Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics L.B. Resnick (Eds) Knowing, learning and Instruction: Essays in Honor of Robert Glaser. Lawrence Erlbaum Hillsdale, NJ 453–494

    Google Scholar 

  • A. Collins (1991) Cognitive Apprenticeship and Instructional Technology L. Idol B.F. Jones (Eds) Educational Values and Cognitive Instruction. Lawrence Erlbaum Hillsdale, NJ 121–138

    Google Scholar 

  • Corbett A.T., Anderson J.R. (1989). ‘Feedback timing and student control in the LISP Intelligent Tutoring System’. In: Proceedings of the Fourth International Conference on AI and Education, pp. 64–72

  • M. Czarkowski J. Kay (2000) Bringing scrutability to adaptive hypertext teaching Gauthier. Frasson. Lehn. Van (Eds) Intelligent Tutoring Systems. Springer Berlin 423–432

    Google Scholar 

  • Dietterich T.G. (2002). Machine learning for sequential data: A review. In: Caelli T. (ed). Lecture Notes in Computer Science. 2396, 15–30

  • R.M. Felder G.N. Felder E.J. Dietz (1998) ArticleTitleA longitudinal study of engineering student performance and retention. V. comparisons with traditionally-taught students Journal of Engineering Education. 87 IssueID4 469–480

    Google Scholar 

  • F. Flores M. Graves B. Hartfield T. Winograd (1988) ArticleTitleComputer systems and the design of organizational interaction ACM Transactions on Office Information Systems. 6 IssueID2 153–172 Occurrence Handle10.1145/45941.45943

    Article  Google Scholar 

  • Forbes-Riley K., Litman D.J. (2004). Predicting emotion in spoken dialogue from multiple knowledge sources. In: Proceedings of HLT-NAACL 2004, Boston, MA, pp. 201–208

  • Goodman B., Iorizzo L. (2000). Learning with reflection: Project PRAXIS. In: Proceedings of the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL

  • Gaimari R., Soller A.L. (1996). Collaborative learning in an intelligent tutoring system. In: Proceedings of the 1996 Conference on Computer-Supported Cooperative Work Workshop on Approaches for Distributed Learning through Computer-Supported Collaborative Learning, Cambridge, MA

  • Goodman B., Soller A., Linton F., Gaimari R. (1996). [Videotaped study: 3 groups of 4–5 students each solving software system design problems using Object Modeling Technique during a one week course at The MITRE Institute]. Unpublished raw data

  • Goodman B., Soller A., Linton F., Gaimari R. (1997). Encouraging student reflection and articulation using a learning companion. In: Proceedings of the AI-ED 97 World Conference on Artificial Intelligence in Education, Kobe, Japan, pp. 151–158

  • B. Goodman A. Soller F. Linton R. Gaimari (1998) ArticleTitleEncouraging student reflection and articulation using a learning companion International Journal of Artificial Intelligence in Education. 9 IssueID3–4 237–255

    Google Scholar 

  • B. Goodman M. Geier L. Haverty F. Linton R. McCready (2001) A framework for asynchronous collaborative learning and problem-solving J. Moore C.L. Redfield W.L. Johnson (Eds) Artificial Intelligence in Education. IOS Press Amsterdam 188–199

    Google Scholar 

  • Goodman B., Hitzeman J., Linton F., Ross H. (2003a). Towards intelligent agents for collaborative learning: recognizing the role of dialogue participants. In: Proceedings of the International Conference on User Modeling, Johnstown, PA, pp. 363–367

  • B. Goodman R. Gaimari J. Zarrella F. Linton et al. (2003) An empirical analysis of learner discourse H.U. Hoppe (Eds) Proceedings of the 10th International Conference on Artificial Intelligence in Education: Artificial Intelligence in Education. IOS Press Amsterdam 416–418

    Google Scholar 

  • Hmelo-Silver C.E. (2002). Collaborative ways of knowing: Issues in facilitation. In: Stahl G. (ed). Proceedings of CSCL 2002, Boulder, CO, pp. 199–208

  • Hillard D., Ostendorf M., Shriberg E. (2003). Detection of agreement vs. disagreement in meetings: Training with unlabeled data. In: Proceedings of HLT-NAACL Conference, Edmonton, Canada, pp. 34–36

  • Hirschberg J., Litman D.J., Swerts M. (2000). Generalizing prosodic prediction of speech recognition errors. In: Proceedings of the 6th International Conference of Spoken Language Processing (ICSLP-2000), Beijing, China, pp. 615–618

  • S. Jarboe (1996) Procedures for enhancing group decision making B. Hirokawa M. Poole (Eds) Communication and Group Decision Making. Sage Publications Thousand Oaks, CA 345–383

    Google Scholar 

  • D. Jonassen S. Land (2000) Theoretical Foundations of Learning Environments Erlbaum Mahwah, NJ

    Google Scholar 

  • D. Johnson R. Johnson E.J. Holubec (1990) Circles of Learning: Cooperation in the Classroom Interaction Book Company Edina, MN

    Google Scholar 

  • Jokinen K., Hurtig T., Hynnä K., Kanto K., Kaipainen M., Kerminen A. (2001). Self-organizing dialogue management. In: Proceedings of the 2nd Workshop on Neural Networks and Natural Language Processing, Natural Language Pacific Rim Symposium (NLPRS), Tokyo, Japan, pp. 78–85

  • S. Katz J. Aronis C. Creitz (1999) Modeling pedagogical interactions with machine learning S.P. Lajoie M. Vivet (Eds) Artificial Intelligence in Education. IOS Press Amsterdam 543–550

    Google Scholar 

  • S. Katz G. O’Donnell H. Kay (2000) ArticleTitleapproach to analyzing the role and structure of reflective dialogue International Journal of Artificial Intelligence in Education 11 320–343

    Google Scholar 

  • Kay J. (1998). Scrutable User Modeling Shell for User-adapted Interaction. Ph.D. Thesis Basser Department of Computer Science. University of Sydney, Australia

  • C. Kneser R. Pilkington T. Treasure-Jones (2001) ArticleTitleThe tutor’s role: An investigation of the power of exchange structure analysis to identify different roles in CMC seminars International Journal of Artificial Intelligence in Education 12 63–84

    Google Scholar 

  • A. Lesgold S. Katz L. Greenberg E. Hughes G. Eggan (1992) Extensions of intelligent tutoring paradigms to support collaborative learning S. Dijkstra H. Krammer J. Merrienboer Particlevan (Eds) Instructional Models in Computer-Based Learning Environments. Springer Berlin 291–311

    Google Scholar 

  • Linton F., Goodman B., Gaimari R., Zarrella J., Ross H. (2003). Student modeling for an intelligent agent in a collaborative learning environment. In: Proceedings of the International Conference on User Modeling, Johnstown, PA, pp. 342–352

  • Litman D.J., Hirschberg J., Swerts M. (2000). Predicting automatic speech recognition performance using prosodic cues. In: Proceedings of the First Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL’00), Seattle, WA, pp. 218–225

  • Lund K., Baker M., Baron M. (1996). Modeling dialogue and beliefs as a basis for generating guidance in a CSCL environment. In: Proceedings of the ITS-96 Conference, Montreal, pp. 206–614

  • M. McManus R. Aiken (1995) ArticleTitleMonitoring computer-based problem-solving Journal of Artificial Intelligence in Education. 6 IssueID4 307–336

    Google Scholar 

  • R. Morales H. Pain T. Conlon (2000) Understandable learner models for a sensorimotor control task Gauthier. Frasson. Lehn. Van (Eds) Intelligent Tutoring Systems. Springer Berlin 222–231

    Google Scholar 

  • M. Mühlenbrock F. Tewissen H.U. Hoppe (1998) ArticleTitleA framework system for intelligent support in open distributed learning environments International Journal of Artificial Intelligence in Education. 9 256–274

    Google Scholar 

  • Rabiner L.R. (1989). A tutorial on hidden Markov models. In: Proceedings of the IEEE, Vol. 77, pp. 257–286

  • Rehg W., McBurney P., Parsons S. (2001). Computer decision support systems for public argumentation: criteria for assessment. In: Hansen H.V., Tindale C.W., Blair J.A., Johnson R.H. (ed). Argumentation and its Applications. Proceedings of the Fourth Biennial Conference of the Ontario Society for the Study of Argumentation, OSSA 2001

  • J. Rumbaugh M. Blaha W. Premerlani F. Eddy W. Lorensen (1991) Object-Oriented modeling and design Prentice Hall Englewood Cliffs, NJ

    Google Scholar 

  • Samuel K., Carberry S., Vijay-Shanker K. (1998). Dialogue act tagging with transformation-based learning. In: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1150–1156

  • J. Searle (1969) DIALOGUE ACTS: An Essay in the Philosophy of Language Cambridge University Press London

    Google Scholar 

  • J. Searle (1996) A taxonomy of illocutionary acts A. Martinich (Eds) The Philosophy of Language EditionNumber3 Oxford University Press New York

    Google Scholar 

  • E. Shriberg R. Bates A. Stolcke P. Taylor D. Jurafsky K. Ries N. Coccaro R. Martin M. Meteer C. Van Ess-Dykema (1998) ArticleTitleCan prosody aid the automatic classification of dialog acts in conversational speech Language and Speech. 41 IssueID3–4 439–487

    Google Scholar 

  • Shriberg E., Stolcke A., Baron D. (2001). Can prosidy aid the automatic processing of multi-party meetings? Evidence from predicting punctuation, disfluencies, and overlapping speech. In: Proceedings of the ISCA Tutorial and Research Workshop on Prosody in Speech Recognition and Understanding, Red Bank, NJ, pp. 139–146

  • V.J. Shute (1990) Rose Garden Promises of Intelligent Tutoring Systems: Blossom or Thorn? Presented at the Space Operations and Research (SOAR) Symposium Albuquerque NM

    Google Scholar 

  • Singley M.K., Singh M., Fairweather P., Farrell R., Swerling S. (2000). Algebra jam: Supporting teamwork and managing roles in a collaborative learning environment. In: Proceedings of Computer Supported Collaborative Work 2000, Philadelphia, PA, pp. 145–154

  • Soller A. (2002). Computational Analysis of Knowledge Sharing in Collaborative Distance Learning. Doctoral Dissertation. Department of Computer Science, University of Pittsburgh

  • Soller A. (2003). Personal communication

  • A. Soller (2004) ArticleTitleComputational modeling and analysis of knowledge sharing in collaborative distance learning User Modeling and User-Adapted Interaction. 14 IssueID4 351–381 Occurrence Handle10.1023/B:USER.0000043436.49168.3b

    Article  Google Scholar 

  • Soller A.L. (1997a). An intelligent CSCL communication interface. In: Proceedings of AI-ED 97 Workshop IV: Collaborative Learning/Working Support System with Networking, Kobe, Japan, pp. 94–95

  • Soller A.L. (1997b). Back to the drawing board: Explaining causal relationships in an argumentation-based ITS. In: Proceedings of the AI-ED 97 World Conference on Artificial Intelligence in Education, Kobe, Japan, pp. 231–238

  • Soller A., Goodman B., Linton F., Gaimari R. (1998). Promoting effective peer interaction in an intelligent collaborative learning environment. In: Proceedings of the Fourth International Conference on Intelligent Tutoring Systems (ITS 98), San Antonio, TX, pp. 186–195

  • A. Soller A. Lesgold (2000) Modeling the process of collaborative learning. International workshop on new technologies in Collaborative Learning Awaji-Yumebutai Japan

    Google Scholar 

  • Soller A., Lesgold A., Linton F., Goodman B. (1999a). What makes peer interaction effective? Modeling effective communication in an intelligent CSCL. In: Proceedings of the 1999 AAAI Fall Symposium: Psychological Models of Communication in Collaborative Systems, Cape Cod, MA, pp. 116–123

  • Soller A., Lesgold A. (1999). Analyzing peer dialogue from an active learning perspective. Proceedings of the AI-ED 99 Workshop: Analysing Educational Dialogue Interaction: Towards Models that Support Learning, LeMans, France, pp. 63–71

  • Soller A., Linton F., Goodman B., Lesgold A. (1999b). Toward intelligent analysis and support of collaborative learning interaction. In: Proceedings of the Ninth International Conference on Artificial Intelligence in Education, LeMans, France, pp. 75–82

  • A. Soller J. Wiebe A. Lesgold (2002) A machine learning approach to assessing knowledge sharing during collaborative learning activities G. Stahl (Eds) Proceedings of CSCL 2002. Boulder CO 128–137

    Google Scholar 

  • A.L. Soller (2001) ArticleTitleSupporting social interaction in an intelligent collaborative learning system International Journal of Artificial Intelligence in Education. 12 IssueID1 40–62

    Google Scholar 

  • Stevens R., Ikeba J., Casillas A., Palacio-Cayetano J., Clyman S. (1999). Artificial neural network-based performance assessments, Computers in Human Behavior, 15, pp. 295–313

  • P. Tesluk J.E. Mathieu S.J. Zaccaro M. Marks (1997) Task and aggregation issues in the analysis and assessment of team performance M.T. Brannick E. Salas C. Prince (Eds) Team Performance Assessment and Measurement. Lawrence Erlbaum Mahwah, NJ 197–224

    Google Scholar 

  • de Vicente A., Bouwer A., Pain H. (1999). Initial impressions on using the DISCOUNT scheme. In: Proceedings of the Workshop on Analysing Educational Dialogue Interaction (AIED 99 Workshop), LeMans, France, pp. 87–94

  • A. Waibel M. Bett M. Finke R. Stiefelhagen (1998) Meeting browser: Tracking and summarizing meetings D.E.M. Penrose (Eds) Proceedings of the Broadcast News Transcription and Understanding Workshop. Morgan Kaufmann Lansdowne, VA 281–286

    Google Scholar 

  • Walker M.A., Wright J., Langkilde I. (2000). Using natural language processing and discourse features to identify understanding errors in a spoken dialogue system. In: Proceedings of the International Conference on Machine Learning, Stanford, CA, pp. 1111–1118

  • Winter M., McCalla G. (2003). An analysis of group performance in terms of the functional knowledge and teamwork skills of group members. In: Proceedings of the UM2003 Workshop on User and Group Models for Web-based Adaptive Collaborative Environments, Johnstown, PA, pp. 35–44

  • A. Wu R. Farrell M. Singley (2002) Scaffolding Group Learning in a Collaborative Networked Environment G. Stahl (Eds) Proceedings of CSCL 2002. Boulder CO 245–254

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The MITRE Corporation, 202 Burlington Road, M/S K302, Bedford, MA, 01730, USA

    Bradley A. Goodman, Frank N. Linton, Robert D. Gaimari, Janet M. Hitzeman, Helen J. Ross & Guido Zarrella

Authors
  1. Bradley A. Goodman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frank N. Linton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert D. Gaimari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Janet M. Hitzeman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Helen J. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guido Zarrella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bradley A. Goodman.

Additional information

An erratum to this article is available at http://dx.doi.org/10.1007/s11257-006-9001-x.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Goodman, B.A., Linton, F.N., Gaimari, R.D. et al. Using Dialogue Features to Predict Trouble During Collaborative Learning. User Model User-Adap Inter 15, 85–134 (2005). https://doi.org/10.1007/s11257-004-5269-x

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-004-5269-x

Keywords

Search

Navigation