Vanessa Echeverria
Simon Buckingham Shum
ABSTRACT
Collocated, face-to-face teamwork remains a pervasive mode of working, which is hard to replicate online. Team members' embodied, multimodal interaction with each other and artefacts has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. However, the ready availability of sensors makes it increasingly affordable to instrument work spaces to study teamwork and groupwork. The possibility of visualising key aspects of a collaboration has huge potential for both academic and professional learning, but a frontline challenge is the enrichment of quantitative data streams with the qualitative insights needed to make sense of them. In response, we introduce the concept of collaboration translucence, an approach to make visible selected features of group activity. This is grounded both theoretically (in the physical, epistemic, social and affective dimensions of group activity), and contextually (using domain-specific concepts). We illustrate the approach from the automated analysis of healthcare simulations to train nurses, generating four visual proxies that fuse multimodal data into higher order patterns.
- S. S. J. Alhadad (2018). Visualizing Data to Support Judgement, Inference, and Decision Making in Learning Analytics: Insights from Cognitive Psychology and Visualization Science. Journal of Learning Analytics, 5, (2), 60--85.Google Scholar
Cross Ref
- K. Bachour, F. Kaplan and P. Dillenbourg (2010). An Interactive Table for Supporting Participation Balance in Face-to-Face Collaborative Learning. IEEE Transactions on Learning Technologies, 3, (3), 203--213. Google ScholarDigital Library
- J. E. Bardram (2005). Activity-based computing: support for mobility and collaboration in ubiquitous computing. Pers Ubiquit Comput, 9, (5), 312--322. Google ScholarDigital Library
- M. Bilandzic and M. Foth (2012). A review of locative media, mobile and embodied spatial interaction. International Journal of Human-Computer Studies, 70, (1), 66--71. Google ScholarDigital Library
- J. J. Braithwaite, D. G. Watson, R. Jones and M. Rowe (2013). A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology, 49, (1), 1017--1034.Google Scholar
- S. E. Chappel, S. J. J. M. Verswijveren, B. Aisbett, J. Considine and N. D. Ridgers (2017). Nurses' occupational physical activity levels: A systematic review. International Journal of Nursing Studies, 73, (August), 52--62.Google ScholarCross Ref
- M. Cukurova, R. Luckin, E. Millán and M. Mavrikis (2018). The NISPI framework: Analysing collaborative problem-solving from students' physical interactions. Computers & Education, 116, 93--109. Google ScholarDigital Library
- C. W. Darrow (1964). The rationale for treating the change in galvanic skin response as a change in conductance. Psychophysiology, 1, (1), 31--38.Google ScholarCross Ref
- M. E. Dawson, A. M. Schell and D. L. Filion. 2007. The Electrodermal System. In Handbook of Psychophysiology, Gary Berntson, John T. Cacioppo and Louis G. Tassinary (Eds.). Cambridge University Press, Cambridge, 159--181.Google Scholar
- Y. O. de Lima and J. M. de Souza. 2017. The future of work: Insights for CSCW. In Proceedings of the IEEE International Conference on Computer Supported Cooperative Work in Design, CSCWD'17 42--47.Google ScholarCross Ref
- J. Dewey. 1997. How we think. Courier Corporation, Chelmsford, USA.Google Scholar
- D. Di Mitri, J. Schneider, M. Specht and H. Drachsler (2018). From signals to knowledge: A conceptual model for multimodal learning analytics. Journal of Computer Assisted Learning, 34, (4), 338--349.Google ScholarCross Ref
- D. Di Mitri, J. Schneider, R. Klemke, M. Specht and H. Drachsler. 2019. Read Between the Lines: An Annotation Tool for Multimodal Data for Learning. In Proceedings of the International Conference on Learning Analytics and Knowledge, LAK'19, to appear. Google ScholarDigital Library
- Y. Dich, J. Reilly and B. Schneider. 2018. Using Physiological Synchrony as an Indicator of Collaboration Quality, Task Performance and Learning. In Proceedings of the International Conference on Artificial Intelligence in Education, AIED 2018, 98--110.Google ScholarCross Ref
- J. M. DiMicco, K. J. Hollenbach, A. Pandolfo and W. Bender (2007). The impact of increased awareness while face-to-face. Human-Computer Interaction, 22, (1), 47--96. Google ScholarDigital Library
- R. G. dos Santos Almeida, B. M. Jorge, V. D. Souza-Junior, A. Mazzo, J. C. A. Martins, E. C. Negri and I. A. C. Mendes (2018). Trends in research on simulation in the teaching of nursing: an integrative review. Nursing education perspectives, 39, (3), E7-E10.Google Scholar
- S. Dumais, R. Jeffries, D. M. Russell, D. Tang and J. Teevan. 2014. Understanding User Behavior Through Log Data and Analysis. In Ways of Knowing in HCI, S. Judith Olson and A. Wendy Kellogg (Eds.). Springer, New York, 349--372.Google Scholar
- V. Echeverria, R. Martinez-Maldonado, K. Chiluiza and S. B. Shum. 2017. Dbcollab: Automated Feedback for Face-to-Face Group Database Design. In Proceedings of the International Conference on Computers in Education, ICCE'17, 156 - 165.Google Scholar
- V. Echeverria, R. Martinez-Maldonado, R. Granda, K. Chiluiza, C. Conati and S. B. Shum. 2018. Driving data storytelling from learning design. In Proceedings of the International Conference on Learning Analytics and Knowledge, LAK'18, 131140. Google ScholarDigital Library
- T. Erickson, D. Smith, W. Kellogg, M. Laff, J. Richards and E. Bradner. 1999. Socially translucent systems: social proxies, persistent conversation, and the design of "babble". In Proceedings of the SIGCHI Conference on Human Factors in Computer Systems, CHI'99, 72--79. Google ScholarDigital Library
- T. Erickson and W. A. Kellogg (2000). Social translucence: an approach to designing systems that support social processes. ACM Transactions on Computer-Human Interaction, 7, (1), 59--83. Google ScholarDigital Library
- T. Erickson, C. Halverson, W. A. Kellogg, M. Laff and T. Wolf (2002). Social translucence: designing social infrastructures that make collective activity visible. Communications of the ACM, 45, (4), 40--44. Google ScholarDigital Library
- A. C. Evans, J. O. Wobbrock and K. Davis. 2016. Modeling Collaboration Patterns on an Interactive Tabletop in a Classroom Setting. In Proceedings of the International Conference on Computer-Supported Cooperative Work, CSCW'16, 860--871. Google ScholarDigital Library
- P. Fournier-Viger, R. Nkambou and E. M. Nguifo. 2010. Building Intelligent Tutoring Systems for Ill-Defined Domains. In Advances in Intelligent Tutoring Systems, Roger Nkambou, Jacqueline Bourdeau and Riichiro Mizoguchi (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 81--101.Google Scholar
- D. Gasevic, S. Joksimovic, B. R. Eagan and D. W. Shaffer (2018). SENS: Network analytics to combine social and cognitive perspectives of collaborative learning. Computers in Human Behavior, July, (2018).Google Scholar
- A. Gibson and R. Martinez-Maldonado. 2017. That dashboard looks nice, but what does it mean?: towards making meaning explicit in learning analytics design. In Proceedings of the Australian Conference on Computer-Human Interaction, OZCHI'17, 528--532. Google ScholarDigital Library
- E. Gilbert. 2012. Designing social translucence over social networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'12, 2731--2740. Google ScholarDigital Library
- R. M. Gillies, A. F. Ashman and J. Terwel. 2007. The teacher's role in implementing cooperative learning in the classroom. Springer, New York.Google Scholar
- P. A. Gloor, F. Grippa, J. Putzke, C. Lassenius, H. Fuehres, K. Fischbach and D. Schoder (2012). Measuring social capital in creative teams through sociometric sensors. International Journal of Organisational Design and Engineering, 2, (4), 380--401.Google ScholarCross Ref
- P. Goodyear and L. Carvalho. 2014. Framing the analysis of learning network architectures. In The architecture of productive learning networks, Lucila Carvalho and Peter Goodyear (Eds.). Routledge, New York, 48--70.Google Scholar
- A. Green, S. P. Stawicki and M. S. Firstenberg. 2018. Introductory Chapter: Medical Error and Associated Harm-The The Critical Role of Team Communication and Coordination. In Vignettes in Patient Safety-Volume 3, Stanislaw P. Stawicki and Michael S. Firstenberg (Eds.). IntechOpen, London, UK, 1--12.Google Scholar
- E. Haataja, J. Malmberg and S. Järvelä (2018). Monitoring in collaborative learning: Co-occurrence of observed behavior and physiological synchrony explored. Computers in Human Behavior, 87, 337--347.Google ScholarDigital Library
- E. Hutchins (2010). Cognitive ecology. Topics in cognitive science, 2, (4), 705715.Google Scholar
- H. Ishii. 2008. Tangible bits: beyond pixels. In Proceedings of the International Conference on Tangible and Embedded Interaction, TEI'08, xv-xxv. Google ScholarDigital Library
- H.-C. Jetter, H. Reiterer and F. Geyer (2014). Blended Interaction: understanding natural human--computer interaction in post-WIMP interactive spaces. Pers Ubiquit Comput, 18, (5), 1139--1158. Google ScholarDigital Library
- D. M. Johnson, P. Sutton and J. Poon. 2000. Face-to-Face vs. CMC: Student communication in a technologically rich learning environment. In Proceedings of the Annual Conference of the Australian Society for Computers in Learning in Tertiary Education, ASCILITE 2000, 509--520.Google Scholar
- C. Kaendler, M. Wiedmann, N. Rummel and H. Spada (2015). Teacher Competencies for the Implementation of Collaborative Learning in the Classroom: a Framework and Research Review. Educ Psychol Rev, 27, (3), 505536.Google ScholarCross Ref
- T. Kim, A. Chang, L. Holland and A. S. Pentland. 2008. Meeting mediator: enhancing group collaborationusing sociometric feedback. In Proceedings of the International Conference on Computer Supported Cooperative Work, CSCW'08, 457--466. Google ScholarDigital Library
- K. Kitto, S. B. Shum and A. Gibson. 2018. Embracing imperfection in learning analytics. In Proceedings of the International Conference on Learning Analytics and Knowledge, LAK'18, 451--460. Google ScholarDigital Library
- D. S. Kusunoki and A. Sarcevic. 2015. Designing for Temporal Awareness: The Role of Temporality in Time-Critical Medical Teamwork. In Proceedings of the International Conference on Computer Supported Cooperative Work, CSCW'15, 1465--1476. Google ScholarDigital Library
- J. Lazar, J. H. Feng and H. Hochheiser. 2017. Research methods in humancomputer interaction. Morgan Kaufmann. Google ScholarDigital Library
- M. Liggins II, D. Hall and J. Llinas. 2017. Handbook of multisensor data fusion: theory and practice. CRC press, Boca Raton, USA.Google Scholar
- K. Littleton, D. Miell and D. Faulkner. 2004. Learning to collaborate, collaborating to learn: Understanding and promoting educationally productive collaborative work. NSP, Charlotte, USA.Google Scholar
- Z. Liu, Y. Wang, M. Dontcheva, M. Hoffman, S. Walker and A. Wilson (2017). Patterns and sequences: Interactive exploration of clickstreams to understand common visitor paths. IEEE Transactions on Visualization and Computer Graphics, 23, (1), 321--330. Google ScholarDigital Library
- B. Mariani and J. Doolen (2016). Nursing simulation research: What are the perceived gaps? Clinical simulation in nursing, 12, (1), 30--36.Google Scholar
- N. Marquardt, K. Hinckley and S. Greenberg. 2012. Cross-device interaction via micro-mobility and f-formations. In Proceedings of the Symposium on User Interface Software and Technology, UIST'12 13--22. Google ScholarDigital Library
- R. Martinez-Maldonado, A. Clayphan, K. Yacef and J. Kay (2015). MTFeedback: Providing Notifications to Enhance Teacher Awareness of Small Group Work in the Classroom. IEEE Transactions on Learning Technologies, 8, (2), 187--200.Google ScholarDigital Library
- R. Martinez-Maldonado, K. Yacef and J. Kay (2015). TSCL: A conceptual model to inform understanding of collaborative learning processes at interactive tabletops. International Journal of Human-Computer Studies, 83, 62--82. Google ScholarDigital Library
- R. Martinez-Maldonado, P. Goodyear, J. Kay, K. Thompson and L. Carvalho. 2016. An Actionable Approach to Understand Group Experience in Complex, Multi-surface Spaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'16, 2062--2074. Google ScholarDigital Library
- R. Martinez-Maldonado, B. Schneider, S. Charleer, S. B. Shum, J. Klerkx and E. Duval. 2016. Interactive surfaces and learning analytics: data, orchestration aspects, pedagogical uses and challenges. In Proceedings of the International Conference on Learning Analytics & Knowledge, LAK'16, 124--133. Google ScholarDigital Library
- R. Martinez-Maldonado, V. Echeverria, O. C. Santos, A. D. P. Dos Santos and K. Yacef. 2018. Physical learning analytics: A multimodal perspective. In Proceedings of the International Conference on Learning Analytics and Knowledge, LAK'18, 375--379. Google ScholarDigital Library
- F. Mazzocchi (2015). Could Big Data be the end of theory in science?: A few remarks on the epistemology of data-driven science. EMBO reports, 16, (10), 1250--1255.Google ScholarCross Ref
- D. W. McDonald, S. Gokhman and M. Zachry. 2012. Building for social translucence: a domain analysis and prototype system. In Proceedings of the International Conference on Computer Supported Cooperative Work, CSCW'12, 637--646. Google ScholarDigital Library
- M. McGregor and J. C. Tang. 2017. More to Meetings: Challenges in Using Speech-Based Technology to Support Meetings. In Proceedings of the International Conference on Computer Supported Cooperative Work, CSCW'17, 2208--2220. Google ScholarDigital Library
- S. K. Milligan and P. Griffin (2016). Understanding learning and learning design in MOOCs: a measurement-based interpretation. Journal of Learning Analytics, 3, (2), 88--115.Google ScholarCross Ref
- A. Mohan, H. C. Sun, O. Lederman, K. Full and A. S. Pentland. 2018. Measurement and Feedback of Group Activity Using Wearables for Face-to-Face Collaborative Learning. In Proceedings of the IEEE International Conference on Advanced Learning Technologies, ICALT'18, 163--167.Google Scholar
- L. Müller, V. Rivera-Pelayo, C. Kunzmann and A. Schmidt. 2011. From Stress Awareness to Coping Strategies of Medical Staff: Supporting Reflection on Physiological Data. In Proceedings of the International Workshop on Human Behavior Understanding, 93--103. Google ScholarDigital Library
- P. Müller, M. X. Huang and A. Bulling. 2018. Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour. In Proceedings of the International Conference on Intelligent User Interfaces, IUI'18, 153--164. Google ScholarDigital Library
- Y. I. Nakano, S. Nihonyanagi, Y. Takase, Y. Hayashi and S. Okada. 2015. Predicting participation styles using co-occurrence patterns of nonverbal behaviors in collaborative learning. In Proceedings of the International Conference on Multimodal Interaction, ICMI'15, 91--98. Google ScholarDigital Library
- B. A. Nardi and S. Whittaker. 2002. The place of face-to-face communication in distributed work. In Distributed work: New research on working across distance using technology Pamela J. Hinds and Sara Kiesler (Eds.). MIT Press, Cambridge, 83--110.Google Scholar
- C. Neubauer, J. Woolley, P. Khooshabeh and S. Scherer. 2016. Getting to know you: a multimodal investigation of team behavior and resilience to stress. In Proceedings of the International Conference on Multimodal Interaction, ICMI'16, 193--200. Google ScholarDigital Library
- K. Niemantsverdriet, M. Broekhuijsen, H. van Essen and B. Eggen. 2016. Designing for multi-user interaction in the home environment: implementing social translucence. In Proceedings of the International Conference on Designing Interactive Systems, DIS'16, 1303--1314. Google ScholarDigital Library
- F. Nihei, Y. Nakano and Y. Takase. 2017. Predicting meeting extracts in group discussions using multimodal convolutional neural networks. In Proceedings of the International Conference on Multimodal Interaction, ICMI'17, 421--425. Google ScholarDigital Library
- J. Oh, Y. Chee, T. Lim, Y. Cho and I. Y. Kim (2014). Chest compression with kneeling posture in hospital cardiopulmonary resuscitation: a randomised crossover simulation study. Emergency Medicine Australasia, 26, (6), 585--590.Google ScholarCross Ref
- J. S. Olson, S. Teasley, L. Covi and G. Olson. 2002. The (currently) unique advantages of collocated work. In Distributed work: New research on working across distance using technology Pamela J. Hinds and Sara Kiesler (Eds.). MIT Press, Cambridge, 113--136.Google Scholar
- S. Oviatt and A. Cohen. 2013. Written and multimodal representations as predictors of expertise and problem-solving success in mathematics. In Proceedings of the International Conference on Multimodal Interaction, ICMI'13, 599--606. Google ScholarDigital Library
- L. A. Pugh, C. R. Oldroyd, T. S. Ray and M. L. Clark (1966). Muscular effort and electrodermal responses. Journal of Experimental Psychology, 71, (2), 241--248.Google ScholarCross Ref
- W.-M. Roth, J. Tenenberg and D. Socha (2016). Discourse/s in/of CSCW. Computer Supported Cooperative Work, 25, (4--5), 385--407. Google ScholarDigital Library
- N. Ruiz-Robledillo and L. Moya-Albiol (2015). Lower Electrodermal Activity to Acute Stress in Caregivers of People with Autism Spectrum Disorder: An Adaptive Habituation to Stress. Journal of Autism and Developmental Disorders, 45, (2), 576--588.Google ScholarCross Ref
- N. Rummel, H. Spada and S. Hauser (2009). Learning to collaborate while being scripted or by observing a model. International Journal of Computer-Supported Collaborative Learning, 4, (1), 69--92.Google ScholarCross Ref
- K. Schmidt (2002). The problem with awareness: Introductory remarks on awareness in CSCW. Computer Supported Cooperative Work, 11, (3--4), 285--298. Google ScholarDigital Library
- B. Schneider, K. Sharma, S. Cuendet, G. Zufferey, P. Dillenbourg and R. Pea. 2016. Detecting Collaborative Dynamics Using Mobile Eye-Trackers. In Proceedings of the International Conference of the Learning Sciences, ICLS'16, 522--529.Google Scholar
- B. Schneider and R. Pea. 2017. Real-time mutual gaze perception enhances collaborative learning and collaboration quality. In Educational Media and Technology Yearbook, Springer, 99--125.Google Scholar
- D. W. Shaffer. 2017. Quantitative ethnography. Cathcart Press.Google Scholar
- V. J. Shute and M. Ventura. 2013. Stealth assessment: Measuring and supporting learning in video games. MIT Press. Google ScholarDigital Library
- D. Spikol, E. Ruffaldi, L. Landolfi and M. Cukurova. 2017. Estimation of Success in Collaborative Learning Based on Multimodal Learning Analytics Features. In Proceedings of the IEEE International Conference on Advanced Learning Technologies, ICALT'17, 269--273.Google Scholar
- J. Sturm, O. H. V. Herwijnen, A. Eyck and J. Terken. 2007. Influencing social dynamics in meetings through a peripheral display. In Proceedings of the International Conference on Multimodal Interfaces, ICMI'07, 263--270. Google ScholarDigital Library
- S. Tausch, S. Ta and H. Hussmann. 2016. A Comparison of Cooperative and Competitive Visualizations for Co-located Collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'16. Google ScholarDigital Library
- S. Taylor, N. Jaques, W. Chen, S. Fedor, A. Sano and R. Picard. 2015. Automatic identification of artifacts in electrodermal activity data. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'15, 1934--1937.Google Scholar
- S. D. Teasley (2017). Student Facing Dashboards: One Size Fits All? Technology, Knowledge and Learning, 22, (3), 377--384.Google ScholarCross Ref
- S. A. Viswanathan and K. VanLehn (2017). Using the tablet gestures and speech of pairs of students to classify their collaboration. IEEE Transactions on Learning Technologies, 11, (2).Google Scholar
- L. S. Vygotski. 1987. The collected works of LS Vygotsky. Volume 1: Problems of general psychology, including the volume Thinking and Speech. Springer New York.Google Scholar
- G. Wang, X. Zhang, S. Tang, H. Zheng and B. Y. Zhao. 2016. Unsupervised Clickstream Clustering For User Behavior Analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'16, 225--236. Google ScholarDigital Library
- A. F. Wise and D. W. Shaffer (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2, (2), 5--13.Google ScholarCross Ref
- M. Worsley and P. Blikstein (2018). A multimodal analysis of making. International Journal of Artificial Intelligence in Education, 28, (3), 385--419.Google ScholarCross Ref
- Z. Zhang and A. Sarcevic. 2015. Constructing Awareness Through Speech, Gesture, Gaze and Movement During a Time-Critical Medical Task. In Proceedings of the European Conference on Computer Supported Cooperative Work, ECSCW'15, 163--182.Google Scholar
- A. Zolyomi, A. S. Ross, A. Bhattacharya, L. Milne and S. A. Munson. 2018. Values, Identity, and Social Translucence: Neurodiverse Student Teams in Higher Education. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI'18, 1--13. Google ScholarDigital Library
Index Terms
- Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data
Recommendations
Collaboard: a remote collaboration groupware device featuring an embodiment-enriched shared workspace
GROUP '10: Proceedings of the 2010 ACM International Conference on Supporting Group WorkIn this paper we present a mixed presence groupware device called "CollaBoard". The device improves collaboration between co-located and remote partners by providing a high level of workspace awareness. This is achieved by superimposing a life-size ...
Social translucence: an approach to designing systems that support social processes
Special issue on human-computer interaction in the new millennium, Part 1We are interested in desiging systems that support communication and collaboration among large groups of people over computing networks. We begin by asking what properties of the physical world support graceful human-human communication in face-to-face ...
Spatial workspace collaboration: a SharedView video support system for remote collaboration capability
CHI '92: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsCollaboration in three-dimensional space: “spatial workspace collaboration” is introduced and an approach supporting its use via a video mediated communication system is described. Verbal expression analysis is primarily focused on. Based on experiment ...
Comments