Jung In Koh
ABSTRACT
The prevalence of virtual meeting software and webcams have normalized holding group meetings both remotely and distributed. However, shifting group meetings virtually with typical computing setups, while convenient, also reduces fluidity of communications. That is, identifying body language of any attendee is complicated from small or lacking on-screen camera views, while distinguishing between multiple attendees’ voices is challenging with lack of spatial audio from commodity speakers and headphones. For my dissertation research, I propose designing and developing intelligent virtual meeting interfaces to narrow communication fluidity gaps between virtual meeting platforms and in-person meeting arrangements. In order to better inform hosts on attendees’ overall body language and attendees on each others’ shared responses, I explore methodologies to automatically recognize users’ physical gestures and intuitively relate them to camera video filters. Therefore, my dissertation goal is to optimally leverage nonverbal communication for improving intelligent user interfaces in virtual meetings.
- Cisco. 2022. Webex App | Use reactions in your meetings. https://help.webex.com/en-us/article/bxg7wz/Webex-App-%7C-Use-reactions-in-your-meetings (accessed 2023/01/08).Google Scholar
- P. D. Hills, M. V. Q. Clavin, M. R. A. Tufft, M. S. Gobel, and D. C. Richardson. 2022. Video Meeting Signals: Experimental Evidence for a Technique to Improve the Experience of Video Conferencing. PloS one 17, 8 (aug 2022), e0270399.Google ScholarCross Ref
- J. Jesna, Athi S. Narayanan, and Kamal Bijlani. 2018. Automatic Hand Raise Detection by Analyzing the Edge Structures. In Emerging Research in Computing, Information, Communication and Applications, N. R. Shetty, L. M. Patnaik, N. H. Prasad, and N. Nalini (Eds.). Springer Singapore, Singapore, Singapore, 171–180.Google Scholar
- Jung In Koh, Josh Cherian, Paul Taele, and Tracy Hammond. 2019. Developing a Hand Gesture Recognition System for Mapping Symbolic Hand Gestures to Analogous Emojis in Computer-Mediated Communication. ACM Trans. Interact. Intell. Syst. 9, 1, Article 6 (March 2019), 35 pages. https://doi.org/10.1145/3297277Google ScholarDigital Library
- Jung In Koh, Samantha Ray, Josh Cherian, Paul Taele, and Tracy Hammond. 2022. Show of Hands: Leveraging Hand Gestural Cues in Virtual Meetings for Intelligent Impromptu Polling Interactions. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22). Association for Computing Machinery, New York, NY, USA, 292–309. https://doi.org/10.1145/3490099.3511153Google ScholarDigital Library
- Jiaojiao Lin, Fei Jiang, and Ruimin Shen. 2018. Hand-raising gesture detection in real classroom. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, IEEE, New York, NY, USA, 6453–6457.Google ScholarDigital Library
- MeetinVR. 2023. MeetinVR. https://www.meetinvr.com/ (accessed 2023/01/08).Google Scholar
- Meta. 2023. Bring your teams together in Meta Horizon Workrooms. https://www.meta.com/work/workrooms/ (accessed 2023/01/08).Google Scholar
- Meredith Ringel Morris. 2012. Web on the Wall: Insights from a Multimodal Interaction Elicitation Study. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces(Cambridge, Massachusetts, USA) (ITS ’12). Association for Computing Machinery, New York, NY, USA, 95–104. https://doi.org/10.1145/2396636.2396651Google ScholarDigital Library
- Loïc Rosset, Hamed Alavi, Sailin Zhong, and Denis Lalanne. 2021. Already It Was Hard to Tell Who’s Speaking Over There, and Now Face Masks! Can Binaural Audio Help Remote Participation in Hybrid Meetings?Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3411763.3451802Google ScholarDigital Library
- Advait Sarkar, Sean Rintel, Damian Borowiec, Rachel Bergmann, Sharon Gillett, Danielle Bragg, Nancy Baym, and Abigail Sellen. 2021. The Promise and Peril of Parallel Chat in Video Meetings for Work. Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3411763.3451793Google ScholarDigital Library
- Zoom. 2022. Using Gesture Recognition. https://support.zoom.us/hc/en-us/articles/4407537406093-Using-gesture-recognition (accessed 2023/01/08).Google Scholar
Index Terms
- Leveraging Nonverbal Communication for Intelligent Virtual Meeting Interfaces
Recommendations
Show of Hands: Leveraging Hand Gestural Cues in Virtual Meetings for Intelligent Impromptu Polling Interactions
IUI '22: Proceedings of the 27th International Conference on Intelligent User InterfacesIncreased virtual meeting software usage has allowed people to meet remotely in a more seamless fashion. However, compared to in-person meetings, valuable interaction cues such as impromptu group polling are less optimally executed due to increased ...
An Intelligent Virtual Meeting App for Seamlessly Polling Virtual Participants “On-the-Fly” with Nonverbal Communication Cues
IUI '23 Companion: Companion Proceedings of the 28th International Conference on Intelligent User InterfacesThe prevalence of virtual meeting software and webcams have normalized holding group meetings both remotely and distributed. However, virtual meetings reduce viewing to computing displays and listening to unidirectional speakers, which disrupt real-life ...
Exploring Embodied Gestures and Video Filters for More Expressive Virtual Group Meeting Interactions
TEI '22: Proceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied InteractionThis paper explores my doctoral research work on supporting how people express their embodied virtual meeting interactions through the synthesis of physical gesticulations and video filters. The goal is to suggest novel interaction methods that first ...
Comments