Marcelo Worsley
ABSTRACT
Video analysis is a staple of the education research community. For many contemporary education researchers, participation in the video coding process serves as a rite of passage. However, recent developments in multimodal learning analytics may help to accelerate and enhance this process by providing researchers with a more nuanced glimpse into a set of learning experiences. As an example of how to use multimodal learning analytics towards these ends, this paper includes a preliminary analysis from 54 college students, who completed two engineering design tasks in pairs. Gesture, speech and electro-dermal activation data were collected as students completed these tasks. The gesture data was used to learn a set of canonical clusters (N=4). A decision tree was trained based on individual students' cluster frequencies, and pre-post learning gains. The nodes in the decision tree were then used to identify a subset of video segments that were human coded based on prior work in learning analytics and engineering design. The combination of machine learning and human inference helps elucidate the practices that seem to correlate with student learning. In particular, both engagement and disengagement seem to correlate with student learning, albeit in a somewhat nuanced fashion.
- Barron, B. 2003. When smart groups fail. The journal of the learning sciences. January 2014 (2003), 37--41.Google Scholar
- Barron, B., Pea, R. and Engle, R. 2013. Advancing understanding of collaborative learning with data derived from video records. ... handbook of collaborative learning. (2013).Google Scholar
- Bassiou, N., Tsiartas, A., Smith, J., Bratt, H., Richey, C., Shriberg, E., D'Angelo, C. and Alozie, N. 2016. Privacy-preserving speech analytics for automatic assessment of student collaboration. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (2016), 888--892.Google Scholar
- Blikstein, P. and Worsley, M. 2016. Multimodal Learning Analytics: a methodological framework for research in constructivist learning. Journal of Learning Analytics. 3, 2 (2016), 220--238.Google ScholarCross Ref
- Costa, A.L. and Kallick, B. 2000. Getting into the Habit of Reflection. Educational Leadership.Google Scholar
- Goldin-Meadow, S. and Beilock, S.L. 2010. Action's Influence on Thought: The Case of Gesture. Perspectives on Psychological Science. 5, 6 (Dec. 2010), 664--674.Google ScholarCross Ref
- Grafsgaard, J.F. 2014. Multimodal Analysis and Modeling of Nonverbal Behaviors During Tutoring. Proceedings of the 16th International Conference on Multimodal Interaction (New York, NY, USA, 2014), 404--408. Google ScholarDigital Library
- Hatano, G. and Inagaki, K. 1984. Two courses of expertise. Research and Clinical Center for Child Development Annual Report. 6, (1984), 27--36.Google Scholar
- Hmelo-Silver, C.E. and Barrows, H.S. 2008. Facilitating collaborative knowledge building. Cognition and Instruction. 26, 1 (2008), 48--94.Google ScholarCross Ref
- Howison, M., Trninic, D., Reinholz, D. and Abrahamson, D. 2011. The Mathematical Imagery Trainer: From Embodied Interaction to Conceptual Learning. (2011), 1989--1998. Google ScholarDigital Library
- Kress, G. 2001. Multimodal teaching and learning: The rhetorics of the science classroom. (2001).Google Scholar
- Leong, C.W., Chen, L., Feng, G., Lee, C.M. and Mulholland, M. 2015. Utilizing depth sensors for analyzing multimodal presentations: Hardware, software and toolkits. ICMI 2015 - Proceedings of the 2015 ACM International Conference on Multimodal Interaction. 3 (2015), 547--556. Google ScholarDigital Library
- Mercier, E.M., Higgins, S.E. and da Costa, L. 2014. Different leaders: Emergent organizational and intellectual leadership in children's collaborative learning groups. International Journal of Computer-Supported Collaborative Learning. 9, 4 (2014), 397--432.Google ScholarCross Ref
- Núñez, R., Edwards, L. and Matos, J. 1999. Embodied cognition as grounding for situatedness and context in mathematics education. Educational Studies in Mathematics. 39 (1999), 45--65.Google ScholarCross Ref
- Ochoa, X., Chiluiza, K., Méndez, G., Luzardo, G., Guamán, B. and Castells, J. 2013. Expertise estimation based on simple multimodal features. Proceedings of the 15th ACM on International conference on multimodal interaction - ICMI '13. (2013), 583--590. Google ScholarDigital Library
- Ochoa, X. and Worsley, M. 2016. Augmenting Learning Analytics with Multimodal Sensory Data. Journal of Learning Analytics. 3, 2 (2016), 213--219.Google ScholarCross Ref
- Oviatt, S. 2003. Multimodal interfaces. The human-computer interaction handbook: .... (2003). Google ScholarDigital Library
- Oviatt, S. and Cohen, A. 2013. Written and multimodal representations as predictors of expertise and problem-solving success in mathematics. ... 15th ACM on International conference on multimodal .... (2013), 1--8. Google ScholarDigital Library
- Richey, C., D'Angelo, C., Alozie, N., Bratt, H. and Shriberg, E. 2016. The SRI speech-based collaborative learning corpus. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (2016), 1550--1554.Google Scholar
- Roschelle, J. 1992. Learning by Collaborating: Convergent Conceptual Change.Google Scholar
- Scherr, R.E. and Hammer, D. 2009. Student Behavior and Epistemological Framing: Examples from Collaborative Active-Learning Activities in Physics. Cognition and Instruction. 27, 2 (Apr. 2009), 147--174.Google ScholarCross Ref
- Schneider, B. and Blikstein, P. 2015. Unraveling Students' Interaction Around a Tangible Interface using Multimodal Learning Analytics. Journal of Educational Data Mining. (2015).Google Scholar
- Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P. and Pea, R. 2016. Using Mobile Eye-Trackers to Unpack the Perceptual Benefits of a Tangible User Interface for Collaborative Learning. ACM Trans. Comput.-Hum. Interact. 23, 6 (Dec. 2016), 39:1--39:23. Google ScholarDigital Library
- Smith, E.R. and Semin, G.R. 2004. Socially situated cognition: Cognition in its social context. Advances in Experimental Social Psychology. 36, (2004), 53--117.Google Scholar
- Worsley, M. 2017. Engineering Design with Everyday Materials Multi- modal Dataset. (2017).Google Scholar
- Worsley, M. and Blikstein, P. A Multimodal Analysis of Making. International Journal of Artificial Intelligence in Education.Google Scholar
- Worsley, M. and Blikstein, P. 2014. Analyzing Engineering Design through the Lens of Computation. Journal of Learning Analytics. 1, 2 (2014), 151--186.Google ScholarCross Ref
- Worsley, M. and Blikstein, P. 2015. Leveraging Multimodal Learning Analytics to Differentiate Student Learning Strategies. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (New York, NY, USA, 2015), 360--367. Google ScholarDigital Library
- Worsley, M. and Blikstein, P. 2017. Reasoning Strategies in the Context of Engineering Design with Everyday Materials. 2Journal of Pre-College Engineering Education Research (J-PEER). 6, 2 (2017), 58--74.Google Scholar
- Worsley, M. and Blikstein, P. 2013. Towards the Development of Multimodal Action Based Assessment. Proceedings of the Third International Conference on Learning Analytics and Knowledge (New York, NY, USA, 2013), 94--101. Google ScholarDigital Library
- Worsley, M., Scherer, S., Morency, L.P. and Blikstein, P. 2016. Exploring Behavior Representation for Learning Analytics. Proceedings of the 2015 International Conference on Multimodal Interaction (ICMI). (2016), 251--258. Google ScholarDigital Library
- Xavier Ochoa, Marcelo Worsley, Katherine Chiluiza and Saturnino Luz 2014. MLA'14: Third Multimodal Learning Analytics Workshop and Grand Challenges. Proceedings of the 16th International Conference on Multimodal Interaction. (2014), 531--532. Google ScholarDigital Library
Index Terms
- (Dis)engagement matters: identifying efficacious learning practices with multimodal learning analytics
Recommendations
Profiling self-regulation behaviors in STEM learning of engineering design
AbstractEngineering design is a complex process which requires science, technology, engineering, and mathematic (STEM) knowledge. Students' self-regulation plays a critical role in interdisciplinary tasks. However, there is limited research ...
Highlights- A self-regulated learning model was developed in STEM education.
- This examined ...
Multimodal Collaboration in Expository Discourse: Verbal and Nonverbal Moves Alignment
Speech and ComputerAbstractThe paper explores multimodal collaboration in expository discourse considering verbal and nonverbal moves used by the participants in turn-taking. It reports the results of an experiment which tested speech, gesture and gaze alignment as affected ...
Comments