skip to main content
10.1145/3313831.3376546acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Classification of Functional Attention in Video Meetings

Published: 23 April 2020 Publication History

Abstract

Participants in video meetings have long struggled with asymmetrical attention levels, especially when participants are distributed unevenly. While technological advances offer exciting opportunities to augment remote users' attention, the phenomenological complexity of attention means that to design attention-fostering features we must first understand what aspects of it are functionally meaningful to support. In this paper, we present a functional classification of observable attention for video meetings. The classification was informed by two studies on sense-making and selectiveness of attention in work meetings. It includes categories of attention accessible for technological support, their functions in a meeting process, and meeting-related activities that correspond to these functions. This classification serves as a multi-level representation of attention and informs the design of features aiming to support remote participants' attention in video meetings.

Supplemental Material

MP4 File

References

[1]
Manabu Arai, Ellen Gurman Bard, and Robin Hill. 2009. Referring and gaze alignment: Accessibility is alive and well in situated dialogue. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 31.
[2]
Michael Argyle and Mark Cook. 1976. Gaze and mutual gaze. (1976).
[3]
Frank Biocca and Chad Harms. 2002. Defining and measuring social presence: Contribution to the networked minds theory and measure. Proceedings of PRESENCE 2002 (2002), 7--36.
[4]
Frank Biocca, Chad Harms, and Jenn Gregg. 2001. The networked minds measure of social presence: Pilot test of the factor structure and concurrent validity. In 4th annual international workshop on presence, Philadelphia, PA. 1--9.
[5]
Elina Birmingham, Walter F Bischof, and Alan Kingstone. 2008a. Gaze selection in complex social scenes. Visual Cognition 16, 2--3 (2008), 341--355.
[6]
Elina Birmingham, Walter F Bischof, and Alan Kingstone. 2008b. Social attention and real-world scenes: The roles of action, competition and social content. The Quarterly Journal of Experimental Psychology 61, 7 (2008), 986--998.
[7]
Jeremy Birnholtz, Lindsay Reynolds, Eli Luxenberg, Carl Gutwin, and Maryam Mustafa. 2010. Awareness beyond the desktop: exploring attention and distraction with a projected peripheral-vision display. In Proceedings of Graphics Interface 2010. Canadian Information Processing Society, 55--62.
[8]
Ingar Brinck. 2001. Attention and the evolution of intentional communication. Pragmatics & Cognition 9, 2 (2001), 259--277.
[9]
Allison Bruce, Illah Nourbakhsh, and Reid Simmons. 2002. The role of expressiveness and attention in human-robot interaction. In Robotics and Automation, 2002. Proceedings. ICRA'02. IEEE International Conference on, Vol. 4. IEEE, 4138--4142.
[10]
Herbert H Clark and Deanna Wilkes-Gibbs. 1986. Referring as a collaborative process. Cognition 22, 1 (1986), 1--39.
[11]
Juliet Corbin and Anselm Strauss. 1990. Grounded theory research: Procedures, canons and evaluative criteria. Zeitschrift für Soziologie 19, 6 (1990), 418--427.
[12]
Guoqiang Cui, Barbara Lockee, and Cuiqing Meng. 2013. Building modern online social presence: A review of social presence theory and its instructional design implications for future trends. Education and information technologies 18, 4 (2013), 661--685.
[13]
Richard L Daft and Robert H Lengel. 1986. Organizational information requirements, media richness and structural design. Management science 32, 5 (1986), 554--571.
[14]
Madeleine Dean. 2019. Skype now supports Speaker View in group calls. (3 2019). Retrieved September 07, 2019 from https://windowsreport.com/skype-speaker-view-group-calls/.
[15]
Gwyneth Doherty-Sneddon and Fiona G Phelps. 2005. Gaze aversion: A response to cognitive or social difficulty? Memory & cognition 33, 4 (2005), 727--733.
[16]
John V Draper, David B Kaber, and John M Usher. 1998. Telepresence. Human factors 40, 3 (1998), 354--375.
[17]
Mica R Endsley. 1995. Measurement of situation awareness in dynamic systems. Human factors 37, 1 (1995), 65--84.
[18]
Florian Eyben, Felix Weninger, Lucas Paletta, and Björn W Schuller. 2013. The acoustics of eye contact: detecting visual attention from conversational audio cues. In Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction. ACM, 7--12.
[19]
Kathleen E Finn, Abigail J Sellen, and Sylvia B Wilbur. 1997. Video-mediated communication. L. Erlbaum Associates Inc.
[20]
Megan Freeth, Tom Foulsham, and Alan Kingstone. 2013. What affects social attention? Social presence, eye contact and autistic traits. PloS one 8, 1 (2013), e53286.
[21]
Alexandra Frischen, Andrew P Bayliss, and Steven P Tipper. 2007. Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychological bulletin 133, 4 (2007), 694.
[22]
Samratul Fuady, Masato Orishige, Haoyan Li, Hironori Mitake, and Shoichi Hasegawa. 2016. Natural Interaction in Asymmetric Teleconference using Stuffed-toy Avatar Robot. In ICAT-EGVE. 93--98.
[23]
William W Gaver, Abigail Sellen, Christian Heath, and Paul Luff. 1993. One is not enough: Multiple views in a media space. In Proceedings of the INTERACT'93 and CHI'93 Conference on Human Factors in Computing Systems. ACM, 335--341.
[24]
Darren Gergle, Robert E Kraut, and Susan R Fussell. 2013. Using visual information for grounding and awareness in collaborative tasks. Human--Computer Interaction 28, 1 (2013), 1--39.
[25]
Arthur M Glenberg, Jennifer L Schroeder, and David A Robertson. 1998. Averting the gaze disengages the environment and facilitates remembering. Memory & cognition 26, 4 (1998), 651--658.
[26]
Charles Goodwin. 2000. Action and embodiment within situated human interaction. Journal of pragmatics 32, 10 (2000), 1489--1522.
[27]
Charles Goodwin. 2003. Pointing as situated practice. In Pointing. Psychology Press, 225--250.
[28]
David M Grayson and Andrew F Monk. 2003. Are you looking at me? Eye contact and desktop video conferencing. ACM Transactions on Computer-Human Interaction (TOCHI) 10, 3 (2003), 221--243.
[29]
Carl Gutwin and Saul Greenberg. 2002. A descriptive framework of workspace awareness for real-time groupware. Computer Supported Cooperative Work (CSCW) 11, 3--4 (2002), 411--446.
[30]
Joy E Hanna and Susan E Brennan. 2007. Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language 57, 4 (2007), 596--615.
[31]
Steve Harrison. 2009. A brief history of media space research and mediated life. In Media Space 20+ Years of Mediated Life. Springer, 9--16.
[32]
Christian Heath and Paul Luff. 1991. Disembodied conduct: communication through video in a multi-media office environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 99--103.
[33]
Christian Heath, Marcus Sanchez Svensson, Jon Hindmarsh, Paul Luff, and Dirk Vom Lehn. 2002. Configuring awareness. Computer Supported Cooperative Work (CSCW) 11, 3--4 (2002), 317--347.
[34]
Jim Hollan and Scott Stornetta. 1992. Beyond being there. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 119--125.
[35]
Eric Horvitz, Edward Cutrell, and Mary Czerwinski. 2001. Notification, disruption, and memory: Effects of messaging interruptions on memory and performance. In Proceedings of INTERACT. 263.
[36]
Eric Horvitz, Carl Kadie, Tim Paek, and David Hovel. 2003. Models of attention in computing and communication: from principles to applications. Commun. ACM 46, 3 (2003), 52--59.
[37]
Eric Horvitz and Tim Paek. 2001. Harnessing models of users' goals to mediate clarification dialog in spoken language systems. In International Conference on User Modeling. Springer, 3--13.
[38]
Bernardo A Huberman and Fang Wu. 2007. The economics of attention: maximizing user value in information-rich environments. In Proceedings of the 1st international workshop on Data mining and audience intelligence for advertising. ACM, 16--20.
[39]
Rodney H Jones. 2005. 14 Sites of engagement as sites of attention: Time, space and culture in electronic discourse. (2005).
[40]
Victor Kaptelinin and Bonnie A Nardi. 2006. Acting with technology: Activity theory and interaction design. MIT press.
[41]
Annica Kristoffersson, Silvia Coradeschi, and Amy Loutfi. 2013. A review of mobile robotic telepresence. Advances in Human-Computer Interaction 2013 (2013), 3.
[42]
Aleksei N Leont'ev. 1974. The problem of activity in psychology. Soviet psychology 13, 2 (1974), 4--33.
[43]
Paul Luff, Christian Heath, Hideaki Kuzuoka, Jon Hindmarsh, Keiichi Yamazaki, and Shinya Oyama. 2003. Fractured ecologies: creating environments for collaboration. Human-Computer Interaction 18, 1 (2003), 51--84.
[44]
Albert Mehrabian and others. 1971. Silent messages. Vol. 8. Wadsworth Belmont, CA.
[45]
AJung Moon, Daniel M Troniak, Brian Gleeson, Matthew KXJ Pan, Minhua Zheng, Benjamin A Blumer, Karon MacLean, and Elizabeth A Croft. 2014. Meet me where i'm gazing: how shared attention gaze affects human-robot handover timing. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. ACM, 334--341.
[46]
Peter Mundy and Lisa Newell. 2007. Attention, joint attention, and social cognition. Current directions in psychological science 16, 5 (2007), 269--274.
[47]
Brid O'Conaill, Steve Whittaker, and Sylvia Wilbur. 1993. Conversations over video conferences: An evaluation of the spoken aspects of video-mediated communication. Human-computer interaction 8, 4 (1993), 389--428.
[48]
Mai Otsuki, Taiki Kawano, Keita Maruyama, Hideaki Kuzuoka, and Yusuke Suzuki. 2017. ThirdEye: Simple Add-on Display to Represent Remote Participant's Gaze Direction in Video Communication. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5307--5312.
[49]
Mai Otsuki, Keita Maruyama, Hideaki Kuzuoka, and Yusuke Suzuki. 2018. Effects of Enhanced Gaze Presentation on Gaze Leading in Remote Collaborative Physical Tasks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 368.
[50]
Ye Pan and Anthony Steed. 2012. Preserving gaze direction in teleconferencing using a camera array and a spherical display. In 2012 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON). IEEE, 1--4.
[51]
Rosalind W. Picard. 1997. Affective computing. The MIT Press, Cambridge (MA) 167 (1997), 170.
[52]
Aina Puce and Bennett I. Bertenthal. 2015. The Many Faces of Social Attention: Behavioral and Neural Measures. Springer.
[53]
Irene Rae, Gina Venolia, John C. Tang, and David Molnar. 2015. A framework for understanding and designing telepresence. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. ACM, 1552--1566.
[54]
Daniel C Richardson, Rick Dale, and John M Tomlinson. 2009. Conversation, gaze coordination, and beliefs about visual context. Cognitive Science 33, 8 (2009), 1468--1482.
[55]
Sean Rintel. 2018. Mobile Sharing and Companion Experiences for Microsoft Teams Meetings. (3 2018). Retrieved September 07, 2019 from https://www.microsoft.com/en-us/garage/wall-of-fame/companionexperiences/.
[56]
Evan F Risko, Daniel C Richardson, and Alan Kingstone. 2016. Breaking the fourth wall of cognitive science: Real-world social attention and the dual function of gaze. Current Directions in Psychological Science 25, 1 (2016), 70--74.
[57]
Ben Robins, Paul Dickerson, Penny Stribling, and Kerstin Dautenhahn. 2004. Robot-mediated joint attention in children with autism: A case study in robot-human interaction. Interaction studies 5, 2 (2004), 161--198.
[58]
Hanan Salam and Mohamed Chetouani. 2015. A multi-level context-based modeling of engagement in human-robot interaction. In Automatic Face and Gesture Recognition (FG), 2015 11th IEEE International Conference and Workshops on, Vol. 3. IEEE, 1--6.
[59]
Brenda Salley and John Colombo. 2016. Conceptualizing social attention in developmental research. Social Development 25, 4 (2016), 687--703.
[60]
Samiha Samrose, Ru Zhao, Jeffery White, Vivian Li, Luis Nova, Yichen Lu, Mohammad Rafayet Ali, and Mohammed Ehsan Hoque. 2018. CoCo: Collaboration Coach for Understanding Team Dynamics during Video Conferencing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 160.
[61]
Ted Selker, Andrea Lockerd, and Jorge Martinez. 2001. Eye-R, a glasses-mounted eye motion detection interface. In CHI'01 extended abstracts on Human factors in computing systems. ACM, 179--180.
[62]
Abigail J Sellen. 1995. Remote conversations: The effects of mediating talk with technology. Human-computer interaction 10, 4 (1995), 401--444.
[63]
Samira Sheikhi, Dinesh Babu Jayagopi, Vasil Khalidov, and Jean-Marc Odobez. 2013. Context aware addressee estimation for human robot interaction. In Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction. ACM, 1--6.
[64]
John Short, Ederyn Williams, and Bruce Christie. 1976. The social psychology of telecommunications. (1976).
[65]
Garriy Shteynberg. 2014. A social host in the machine? The case of group attention. Journal of Applied Research in Memory and Cognition 3, 4 (2014), 307--311.
[66]
Garriy Shteynberg. 2015. Shared attention. Perspectives on Psychological Science 10, 5 (2015), 579--590.
[67]
Maria Staudte and Matthew W Crocker. 2009. Visual attention in spoken human-robot interaction. In Human-Robot Interaction (HRI), 2009 4th ACM/IEEE International Conference on. IEEE, 77--84.
[68]
Rainer Stiefelhagen, Jie Yang, and Alex Waibel. 2001. Estimating focus of attention based on gaze and sound. In Proceedings of the 2001 workshop on Perceptive user interfaces. ACM, 1--9.
[69]
Roel Vertegaal. 1999. The GAZE groupware system: mediating joint attention in multiparty communication and collaboration. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 294--301.
[70]
Roel Vertegaal, Jeffrey S Shell, Daniel Chen, and Aadil Mamuji. 2006. Designing for augmented attention: Towards a framework for attentive user interfaces. Computers in Human Behavior 22, 4 (2006), 771--789.
[71]
Robert S Weiss. 1995. Learning from strangers: The art and method of qualitative interview studies. Simon and Schuster.
[72]
Christopher D Wickens, Justin G Hollands, Simon Banbury, and Raja Parasuraman. 2015. Engineering psychology and human performance. Psychology Press.
[73]
Bin Xu, Jason Ellis, and Thomas Erickson. 2017. Attention from Afar: Simulating the Gazes of Remote Participants in Hybrid Meetings. In Proceedings of the 2017 Conference on Designing Interactive Systems. ACM, 101--113.
[74]
Lisa C Yamagata-Lynch. 2010. Understanding cultural historical activity theory. In Activity systems analysis methods. Springer, 13--26.
[75]
Jie Yang, Leejay Wu, and Alex Waibel. 1996. Focus of attention in video conferencing. Technical Report. CARNEGIE-MELLON UNIV PITTSBURGH PA SCHOOL OF COMPUTER SCIENCE.
[76]
Weilie Yi and Dana Ballard. 2009. Recognizing behavior in hand-eye coordination patterns. International Journal of Humanoid Robotics 6, 03 (2009), 337--359.
[77]
Cha Zhang, Qin Cai, Philip A Chou, Zhengyou Zhang, and Ricardo Martin-Brualla. 2013. Viewport: A distributed, immersive teleconferencing system with infrared dot pattern. IEEE MultiMedia 20, 1 (2013), 17--27.

Cited By

View all
  • (2024)An experimental comparison of participants’ experience in face-to-face, video, and virtual reality meetingsFrontiers in Virtual Reality10.3389/frvir.2024.14631895Online publication date: 17-Dec-2024
  • (2024)Immersive gathering: insights into virtual workplace meetingsFrontiers in Virtual Reality10.3389/frvir.2024.13916625Online publication date: 2-Sep-2024
  • (2024)Use of Technologies in Intercultural Communication: Virtual Tools for Business Negotiations with Foreign PartnersInformation and Innovations10.31432/1994-2443-2023-18-4-58-7018:4(58-70)Online publication date: 30-Jan-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
April 2020
10688 pages
ISBN:9781450367080
DOI:10.1145/3313831
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. attention
  2. engagement
  3. features
  4. meetings
  5. video-mediated communication

Qualifiers

  • Research-article

Conference

CHI '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)118
  • Downloads (Last 6 weeks)7
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)An experimental comparison of participants’ experience in face-to-face, video, and virtual reality meetingsFrontiers in Virtual Reality10.3389/frvir.2024.14631895Online publication date: 17-Dec-2024
  • (2024)Immersive gathering: insights into virtual workplace meetingsFrontiers in Virtual Reality10.3389/frvir.2024.13916625Online publication date: 2-Sep-2024
  • (2024)Use of Technologies in Intercultural Communication: Virtual Tools for Business Negotiations with Foreign PartnersInformation and Innovations10.31432/1994-2443-2023-18-4-58-7018:4(58-70)Online publication date: 30-Jan-2024
  • (2024)The Rise of AI Avatars: Legal Personhood, Rights and Liabilities in an Evolving MetaverseJournal of Digital Technologies and Law10.21202/jdtl.2024.422:4(857-885)Online publication date: 28-Dec-2024
  • (2024)Hybridge: Bridging Spatiality for Inclusive and Equitable Hybrid MeetingsProceedings of the ACM on Human-Computer Interaction10.1145/36870408:CSCW2(1-39)Online publication date: 8-Nov-2024
  • (2024)"Mirror, mirror in the call": Exploring the Ambivalent Nature of the Self-view in Video Meeting Systems with Self-Reported & Eye-Tracking DataProceedings of the ACM on Human-Computer Interaction10.1145/36869318:CSCW2(1-37)Online publication date: 8-Nov-2024
  • (2024)Comparing the Agency of Hybrid Meeting Remote Users in 2D and 3D Interfaces of the Hybridge SystemExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651103(1-12)Online publication date: 11-May-2024
  • (2024)Mental Models of Meeting Goals: Supporting Intentionality in Meeting TechnologiesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642670(1-17)Online publication date: 11-May-2024
  • (2024)Whispering Through Walls: Towards Inclusive Backchannel Communication in Hybrid MeetingsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642419(1-16)Online publication date: 11-May-2024
  • (2024)Circle Back Next Week: The Effect of Meeting-Free Weeks on Distributed Workers’ Unstructured Time and Attention NegotiationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642175(1-17)Online publication date: 11-May-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media