research-article

MeetingCoach: An Intelligent Dashboard for Supporting Effective & Inclusive Meetings

Authors:

Samiha Samrose,

Javier Hernandez,

Kevin Moynihan,

Mary CzerwinskiAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 252, Pages 1 - 13

https://doi.org/10.1145/3411764.3445615

Published: 07 May 2021 Publication History

Abstract

Video-conferencing is essential for many companies, but its limitations in conveying social cues can lead to ineffective meetings. We present MeetingCoach, an intelligent post-meeting feedback dashboard that summarizes contextual and behavioral meeting information. Through an exploratory survey (N=120), we identified important signals (e.g., turn taking, sentiment) and used these insights to create a wireframe dashboard. The design was evaluated with in situ participants (N=16) who helped identify the components they would prefer in a post-meeting dashboard. After recording video-conferencing meetings of eight teams over four weeks, we developed an AI system to quantify the meeting features and created personalized dashboards for each participant. Through interviews and surveys (N=23), we found that reviewing the dashboard helped improve attendees’ awareness of meeting dynamics, with implications for improved effectiveness and inclusivity. Based on our findings, we provide suggestions for future feedback system designs of video-conferencing meetings.

References

[1]

Joseph A Allen, Nale Lehmann-Willenbrock, and Steven G Rogelberg. 2015. The Cambridge handbook of meeting science. Cambridge University Press.

[2]

Yadid Ayzenberg, Javier Hernandez, and Rosalind W. Picard. 2012. FEEL: frequent EDA and event logging - a mobile social interaction stress monitoring system. In Extended Abstracts of Human Factors in Computing Systems. 2357–2362.

[3]

Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M Martinez, and Seth D Pollak. 2019. Emotional expressions reconsidered: challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest 20, 1 (2019), 1–68.

[4]

Sigal G Barsade. 2002. The ripple effect: Emotional contagion and its influence on group behavior. Administrative science quarterly 47, 4 (2002), 644–675.

[5]

Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang. 2016. Training deep networks for facial expression recognition with crowd-sourced label distribution. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 279–283.

Digital Library

[6]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.

[7]

Dan Calacci, Oren Lederman, David Shrier, and Alex ‘Sandy’ Pentland. 2016. Breakout: An open measurement and intervention tool for distributed peer learning groups. arXiv preprint arXiv:1607.01443(2016).

[8]

Janis A Cannon-Bowers, Scott I Tannenbaum, Eduardo Salas, and Catherine E Volpe. 1995. Defining competencies and establishing team training requirements. Team effectiveness and decision making in organizations 333 (1995), 380.

[9]

Ross Cutler, Yasaman Hosseinkashi, Jamie Pool, Senja Filipi, Robert Aichner, and Johannes Gehrke. 2020. Meeting Effectiveness and Inclusiveness in Remote Collaboration. arXiv preprint arXiv: To Appear(2020).

[10]

Terry L Dickinson and Robert M McIntyre. 1997. A conceptual framework for teamwork measurement. Team performance assessment and measurement(1997), 19–43.

[11]

Joan Morris DiMicco, Katherine J Hollenbach, Anna Pandolfo, and Walter Bender. 2007. The impact of increased awareness while face-to-face. Human–Computer Interaction 22, 1-2 (2007), 47–96.

Digital Library

[12]

Judith Donath, Alex Dragulescu, Aaron Zinman, Fernanda Viégas, and Rebecca Xiong. 2010. Data portraits. In ACM SIGGRAPH 2010 Art Gallery. 375–383.

Digital Library

[13]

Wen Dong, Bruno Lepri, Taemie Kim, Fabio Pianesi, and Alex Sandy Pentland. 2012. Modeling conversational dynamics and performance in a social dilemma task. In 2012 5th International Symposium on Communications, Control and Signal Processing. IEEE, 1–4.

[14]

Norah E Dunbar and Judee K Burgoon. 2005. Perceptions of power and interactional dominance in interpersonal relationships. Journal of Social and Personal Relationships 22, 2(2005), 207–233.

[15]

Mohamed Ez-Zaouia, Aurélien Tabard, and Elise Lavoué. 2020. Emodash: A dashboard supporting retrospective awareness of emotions in online learning. International Journal of Human-Computer Studies 139 (2020), 102411.

[16]

José-Miguel Fernández-Dolsand Maria-Angeles Ruiz-Belda. 1997. Spontaneous facial behavior during intense emotional episodes: Artistic truth and optical truth. In The psychology of facial expression, James A Russell and José-Miguel Fernández-Dols (Eds.). Cambridge University Press, New York, NY, 255–274.

[17]

C Marlene Fiol and Edward J O’Connor. 2005. Identification in face-to-face, hybrid, and pure virtual teams: Untangling the contradictions. Organization science 16, 1 (2005), 19–32.

[18]

Michelle Gregory, Nancy Chinchor, Paul Whitney, Richard Carter, Elizabeth Hetzler, and Alan Turner. 2006. User-directed sentiment analysis: Visualizing the affective content of documents. In Proceedings of the Workshop on Sentiment and Subjectivity in Text. 23–30.

[19]

Steve Harrison. 2009. Media space 20+ years of mediated life. Springer Science & Business Media.

[20]

J. Hernandez, X. Benavides, P. Maes, D. McDuff, J. Amores, and R.W. Picard. 2014. AutoEmotive: Bringing empathy to the driving experience to manage stress. In Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS. https://doi.org/10.1145/2598784.2602780

Digital Library

[21]

J. Hernandez, D. McDuff, R. Fletcher, and R.W. Picard. 2013. Inside-out: Reflecting on your inner state. In IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2013. https://doi.org/10.1109/PerComW.2013.6529507

[22]

Dirk Heylen. 2005. Challenges ahead: Head movements and other social acts in conversations. Virtual Social Agents(2005), 45–52.

[23]

Ellen A Isaacs and John C Tang. 1994. What video can and cannot do for collaboration: a case study. Multimedia systems 2, 2 (1994), 63–73.

[24]

Alejandro Jaimes, Takeshi Nagamine, Jianyi Liu, Kengo Omura, and Nicu Sebe. 2005. Affective meeting video analysis. In 2005 IEEE International Conference on Multimedia and Expo. IEEE, 1412–1415.

[25]

Jeroen Janssen, Gijsbert Erkens, and Gellof Kanselaar. 2007. Visualization of agreement and discussion processes during computer-supported collaborative learning. Computers in Human Behavior 23, 3 (2007), 1105–1125.

Digital Library

[26]

Florian Jentsch, John Barnett, Clint A Bowers, and Eduardo Salas. 1999. Who is flying this plane anyway? What mishaps tell us about crew member role assignment and air crew situation awareness. Human Factors 41, 1 (1999), 1–14.

[27]

Taemie Kim, Agnes Chang, Lindsey Holland, and Alex Sandy Pentland. 2008. Meeting mediator: enhancing group collaborationusing sociometric feedback. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. 457–466.

[28]

Uroš Krčadinac, Jelena Jovanović, and Vladan Devedžić. 2012. Visualizing the affective structure of students interaction. In International Conference on Hybrid Learning. Springer, 23–34.

Digital Library

[29]

Kostiantyn Kucher, Daniel Cernea, and Andreas Kerren. 2016. Visualizing excitement of individuals and groups. In Proceedings of the 2016 EmoVis Conference on Emotion and Visualization. 15–22.

Digital Library

[30]

Anastasia Kuzminykh and Sean Rintel. 2020. Classification of Functional Attention in Video Meetings. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[31]

G Ross Lawford. 2003. Beyond success: Achieving synergy in teamwork. The Journal for Quality and Participation 26, 3 (2003), 23.

[32]

Harold J Leavitt. 1951. Some effects of certain communication patterns on group performance.The Journal of Abnormal and Social Psychology 46, 1(1951), 38.

[33]

Charlotte P Lee and Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA) A Conceptual Framework of and for CSCW. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 179–194.

[34]

Jina Lee and Stacy C Marsella. 2010. Predicting speaker head nods and the effects of affective information. IEEE Transactions on Multimedia 12, 6 (2010), 552–562.

Digital Library

[35]

Gilly Leshed, Diego Perez, Jeffrey T Hancock, Dan Cosley, Jeremy Birnholtz, Soyoung Lee, Poppy L McLeod, and Geri Gay. 2009. Visualizing real-time language-based feedback on teamwork behavior in computer-mediated groups. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 537–546.

Digital Library

[36]

Madelene Lindström, Anna Ståhl, Kristina Höök, Petra Sundström, Jarmo Laaksolathi, Marco Combetto, Alex Taylor, and Roberto Bresin. 2006. Affective diary: designing for bodily expressiveness and self-reflection. In CHI’06 extended abstracts on Human factors in computing systems. 1037–1042.

[37]

Paul Luff, Christian Heath, Hideaki Kuzuoka, Jon Hindmarsh, Keiichi Yamazaki, and Shinya Oyama. 2003. Fractured ecologies: creating environments for collaboration. Human–Computer Interaction 18, 1-2 (2003), 51–84.

[38]

Sandi Mann and Lynn Holdsworth. 2003. The psychological impact of teleworking: stress, emotions and health. New Technology, Work and Employment 18, 3 (2003), 196–211.

[39]

Rob McCarney, James Warner, Steve Iliffe, Robbert Van Haselen, Mark Griffin, and Peter Fisher. 2007. The Hawthorne Effect: a randomised, controlled trial. BMC medical research methodology 7, 1 (2007), 30.

[40]

Daniel McDuff, Eunice Jun, Kael Rowan, and Mary Czerwinski. 2019. Longitudinal Observational Evidence of the Impact of Emotion Regulation Strategies on Affective Expression. IEEE Transactions on Affective Computing(2019).

[41]

Daniel McDuff, Amy Karlson, Ashish Kapoor, Asta Roseway, and Mary Czerwinski. 2012. AffectAura: an intelligent system for emotional memory. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 849–858.

Digital Library

[42]

Daniel McDuff, Kael Rowan, Piali Choudhury, Jessica Wolk, ThuVan Pham, and Mary Czerwinski. 2019. A Multimodal Emotion Sensing Platform for Building Emotion-Aware Applications. arXiv preprint arXiv:1903.12133(2019).

[43]

Poppy Lauretta McLeod, Jeffrey K Liker, and Sharon A Lobel. 1992. Process feedback in task groups: An application of goal setting. The Journal of applied behavioral science 28, 1 (1992), 15–41.

[44]

Michael Nowak, Juho Kim, Nam Wook Kim, and Clifford Nass. 2012. Social Visualization and Negotiation: Effects of Feedback Configuration and Status. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (Seattle, Washington, USA) (CSCW ’12). ACM, New York, NY, USA, 1081–1090. https://doi.org/10.1145/2145204.2145365

Digital Library

[45]

Isabelle Odermatt, Cornelius J König, Martin Kleinmann, Maria Bachmann, Heiko Röder, and Patricia Schmitz. 2018. Incivility in meetings: Predictors and outcomes. Journal of Business and Psychology 33, 2 (2018), 263–282.

[46]

Steven G Rogelberg. 2018. The surprising science of meetings: How you can lead your team to peak performance. Oxford University Press, USA.

[47]

Marcello Russo. 2012. Diversity in goal orientation, team performance, and internal team environment. Equality, Diversity and Inclusion: An International Journal (2012).

[48]

Samiha Samrose, Ru Zhao, Jeffery White, Vivian Li, Luis Nova, Yichen Lu, Mohammad Rafayet Ali, and Mohammed Ehsan Hoque. 2018. Coco: Collaboration coach for understanding team dynamics during video conferencing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 1–24.

Digital Library

[49]

Sanat Sarda, Martin Constable, Justin Dauwels, Shoko Dauwels, Mohamed Elgendi, Zhou Mengyu, Umer Rasheed, Yasir Tahir, Daniel Thalmann, and Nadia Magnenat-Thalmann. 2014. Real-time feedback system for monitoring and facilitating discussions. In Natural Interaction with Robots, Knowbots and Smartphones. Springer, 375–387.

[50]

Mark Schittekatte and Alain Van Hiel. 1996. Effects of partially shared information and awareness of unshared information on information sampling. Small Group Research 27, 3 (1996), 431–449.

[51]

Abigail J Sellen. 1995. Remote conversations: The effects of mediating talk with technology. Human-computer interaction 10, 4 (1995), 401–444.

Digital Library

[52]

John Short, Ederyn Williams, and Bruce Christie. 1976. The social psychology of telecommunications. John Wiley & Sons.

[53]

Tony L Simons and Randall S Peterson. 2000. Task conflict and relationship conflict in top management teams: The pivotal role of intragroup trust.Journal of applied psychology 85, 1 (2000), 102.

[54]

M.I. Tanveer, E. Lin, and M.E. Hoque. 2015. Rhema: A Real-time in-situ Intelligent Interface to Help People with Public Speaking. In Proceedings of the 20th International Conference on Intelligent User Interfaces. 286–295.

[55]

Ivan J Tashev. 2015. Offline voice activity detector using speech supergaussianity. In 2015 Information Theory and Applications Workshop (ITA). IEEE, 214–219.

[56]

Yla R Tausczik and James W Pennebaker. 2013. Improving teamwork using real-time language feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 459–468.

Digital Library

[57]

Anja Thieme, Ed Cutrell, Cecily Morrison, Alex Taylor, and Abigail Sellen. 2020. Interpretability as a dynamic of human-AI interaction. Interactions 27, 5 (2020), 40–45.

Digital Library

[58]

Wilbert van Vree. 2019. Formalisation and Informalisation of Meeting Manners. In Civilisation and Informalisation. Springer, 291–313.

[59]

Shiliang Zhang, Qingming Huang, Shuqiang Jiang, Wen Gao, and Qi Tian. 2010. Affective visualization and retrieval for music video. IEEE Transactions on Multimedia 12, 6 (2010), 510–522.

Digital Library

[60]

M. Zisook, J. Hernandez, M.S. Goodwin, and W. Picard R.2013. Enabling Visual Exploration of Long-term Physiological Data. In IEEE Conference on Visual Analytics Science and Technology.

[61]

Annuska Zolyomi, Andrew Begel, Jennifer Frances Waldern, John Tang, Michael Barnett, Edward Cutrell, Daniel McDuff, Sean Andrist, and Meredith Ringel Morris. 2019. Managing Stress: The Needs of Autistic Adults in Video Calling. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–29.

Digital Library

Cited By

Arakawa RMaeda KYakura H(2024)ConverSearch: Supporting Experts in Human Behavior Analysis of Conversational Videos with a Multimodal Scene Search ToolACM Transactions on Interactive Intelligent Systems10.1145/370901215:1(1-31)Online publication date: 23-Dec-2024
https://dl.acm.org/doi/10.1145/3709012
Howell NHartsoe WAmin JNamani V(2024)Reflective Design for Informal Participatory Algorithm Auditing: A Case Study with Emotion AIProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685411(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685411
Siniukov MYin YFast EQi YMonga AKim ASoleymani M(2024)SEMPI: A Database for Understanding Social Engagement in Video-Mediated Multiparty InteractionProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685752(546-555)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685752
Show More Cited By

Index Terms

MeetingCoach: An Intelligent Dashboard for Supporting Effective & Inclusive Meetings
1. Human-centered computing
  1. Collaborative and social computing
  2. Human computer interaction (HCI)
2. Social and professional topics
  1. Professional topics
    1. Computing and business

Index terms have been assigned to the content through auto-classification.

Recommendations

An Experiment Assessing Group Support System and Facilitator Effects on Meeting Outcomes

This paper is dedicated to the memory of Bye Wynne, who never held back from life, charging those he touched with his vigor, heart, and mind. Thanks.

This article reports on an experiment investigating the effects of a human facilitator and a ...
The effects visual feedback on social behavior during decision making meetings
HCII'11: Proceedings of the 1st international conference on Human interface and the management of information: interacting with information - Volume Part II

This paper describes the design and evaluation of a visualization that provides feedback for meeting participants on their social behavior (Social Mirror). Our Social Mirror provides feedback on participation level, interactivity level, and level of ...
Meetings and Mood – Related or Not? Insights from Student Software Projects
ESEM '22: Proceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement

[Background:] Teamwork, coordination, and communication are a prerequisite for the timely completion of a software project. Meetings as a facilitator for coordination and communication are an established medium for information exchange. Analyses of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
1,589
Total Downloads

Downloads (Last 12 months)369
Downloads (Last 6 weeks)89

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Arakawa RMaeda KYakura H(2024)ConverSearch: Supporting Experts in Human Behavior Analysis of Conversational Videos with a Multimodal Scene Search ToolACM Transactions on Interactive Intelligent Systems10.1145/370901215:1(1-31)Online publication date: 23-Dec-2024
https://dl.acm.org/doi/10.1145/3709012
Howell NHartsoe WAmin JNamani V(2024)Reflective Design for Informal Participatory Algorithm Auditing: A Case Study with Emotion AIProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685411(1-17)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685411
Siniukov MYin YFast EQi YMonga AKim ASoleymani M(2024)SEMPI: A Database for Understanding Social Engagement in Video-Mediated Multiparty InteractionProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685752(546-555)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685752
Andalibi NStark LMcDuff DPicard RGratch JHowell NFarzan RLópez CCardoso Llach DQuercia DMustafa MNiu SWong-Villacrés M(2024)What should we do with Emotion AI? Towards an Agenda for the Next 30 YearsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3689135(98-101)Online publication date: 11-Nov-2024
https://dl.acm.org/doi/10.1145/3678884.3689135
Arakawa RYakura H(2024)Coaching Copilot: Blended Form of an LLM-Powered Chatbot and a Human Coach to Effectively Support Self-Reflection for Leadership GrowthProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665549(1-14)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1145/3640794.3665549
Hosseinkashi YTankelevitch LPool JCutler RMadan C(2024)Meeting Effectiveness and Inclusiveness: Large-scale Measurement, Identification of Key Features, and Prediction in Real-world Remote MeetingsProceedings of the ACM on Human-Computer Interaction10.1145/36373708:CSCW1(1-39)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637370
Wang RQiu LCranshaw JZhang A(2024)Meeting Bridges: Designing Information Artifacts that Bridge from Synchronous Meetings to Asynchronous CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/36373128:CSCW1(1-29)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637312
Scott ATankelevitch LRintel S(2024)Mental Models of Meeting Goals: Supporting Intentionality in Meeting TechnologiesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642670(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642670
Bedmutha MTsedenbal ATobar KBorsotto SSladek KSingh DCasanova-Perez RBascom EWood BSabin JPratt WHartzler AWeibel N(2024)ConverSense: An Automated Approach to Assess Patient-Provider Interactions using Social SignalsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641998(1-22)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641998
Xia WSun GLi TChang BTang JZhang GLiang R(2024)Video Visualization and Visual Analytics: A Task-Based and Application- Driven InvestigationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.342340234:11(11316-11339)Online publication date: Nov-2024
https://doi.org/10.1109/TCSVT.2024.3423402
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten