research-article

TeleHuman: effects of 3d perspective on gaze and pose estimation with a life-size cylindrical telepresence pod

Authors:

Audrey Girouard,

Jeremy Cooperstock,

Roel VertegaalAuthors Info & Claims

CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 2531 - 2540

https://doi.org/10.1145/2207676.2208640

Published: 05 May 2012 Publication History

Abstract

In this paper, we present TeleHuman, a cylindrical 3D display portal for life-size human telepresence. The TeleHuman 3D videoconferencing system supports 360 degree motion parallax as the viewer moves around the cylinder and optionally, stereoscopic 3D display of the remote person. We evaluated the effect of perspective cues on the conveyance of nonverbal cues in two experiments using a one-way telecommunication version of the system. The first experiment focused on how well the system preserves gaze and hand pointing cues. The second experiment evaluated how well the system conveys 3D body postural information. We compared 3 perspective conditions: a conventional 2D view, a 2D view with 360 degree motion parallax, and a stereoscopic view with 360 degree motion parallax. Results suggest the combined presence of motion parallax and stereoscopic cues significantly improved the accuracy with which participants were able to assess gaze and hand pointing cues, and to instruct others on 3D body poses. The inclusion of motion parallax and stereoscopic cues also led to significant increases in the sense of social presence and telepresence reported by participants.

Supplementary Material

MP4 File (paperfile1830-3.mp4)

Supplemental video for “TeleHuman: effects of 3d perspective on gaze and pose estimation with a life-size cylindrical telepresence pod”

Download
23.94 MB

References

[1]

Alexander, O., Lambeth, W., and Debevec, P. Creating a Photoreal Digital Actor: The Digital Emily Project Previous Efforts at Photoreal Digital Humans. Proc. SIGGRAPH, (2009), 1--15.

[2]

Argyle, M. and Ingham, R. Gaze, Mutual Gaze and proximity. Semiotica 6, 1 (1972), 32--49.

[3]

Buxton, B. Telepresence: Integrating Shared Task and Person Spaces. Proc. GI, (1992), 123--129.

Digital Library

[4]

Böcker, M. and Mühlbach, L. Communicative Presence in Videocommunications. Proc. Human Factors and Ergonomics Society, (1993), 249--253.

[5]

Böcker, M., Blohm, W., and Mühlbach, L. Anthropometric Data on Horizontal Head Movements in Videocommunications. Proc. CHI, (1996), 95--96.

Digital Library

[6]

Böcker, M., Rundel, D., and Mühlbach, L. On the Reproduction of Motion Parallax in Videocommunications. Proc. HFES, (1995), 198--20.

[7]

Cooperstock, J. R. Multimodal Telepresence Systems: Supporting Demanding Collaborative Human Activities. IEEE Signal Processing Magazine, Special Issue on Immersive Communications 28, 1 (2011), 77--86.

[8]

Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., and Sasse, M. A. The Impact of Avatar Realism and Eye Gaze Control on Perceived Quality of Communication in a Shared Immersive Virtual Environment. Proc. CHI, 5 (2003), 529.

Digital Library

[9]

Gross, M., Würmlin, S., Naef, M., et al. Blue-c: A Spatially Immersive Display and 3D Video Portal for Telepresence. ACM Transactions on Graphics 22, 3 (2003), 819--827.

Digital Library

[10]

Harrison, C. and Hudson, S. Pseudo-3D Video Conferencing with a Generic Webcam. Proc. IEEE International Symposium on Multimedia, (2008), 236--241.

Digital Library

[11]

Jones, A., Lang, M., Fyffe, G., et al. Achieving Eye contact in a One-to-many 3D Video Teleconferencing System. ACM Transactions on Graphics 28, 3 (2009), 64:3--64:8.

Digital Library

[12]

Jouppi, N. P., Iyer, S., Thomas, S., and Slayden, A. BiReality: Mutually-immersive Telepresence. Proc. MM, (2004), 860--867.

Digital Library

[13]

Kendon, A. Some Functions of Gaze Direction in Social Interaction. Acta Psychologica 26, 44 (1967), 22--63.

[14]

Lightspeed Design. DepthQ Stereoscopic Projector. http://www.depthq.com.

[15]

Lincoln, P., Nashel, A., Ilie, A., Towles, H., Welch, G., and Fuchs, H. Multi-view Lenticular Display for Group Teleconferencing. Proc IMMERSCOM, (2009).

Digital Library

[16]

Litefast. http://www.litefast-display.com/

[17]

Matusik, W. and Pfister, H. 3D TV: A Scalable system for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes. ACM Transactions on Graphics, (2004), 814--824.

Digital Library

[18]

Microsoft. Kinect. http://www.xbox.com/kinect

[19]

Mori, M. The Uncanny Valley. Energy 7, 4 (1970), 33--35.

[20]

Morikawa, O. and Maesako, T. HyperMirror: Toward Pleasant-to-use Video Mediated Communication System. Proc. CSCW, (1998), 149--158.

Digital Library

[21]

NVidia. nVidia 3D Vision Kit. http://www.nvidia.com/object/3d-vision-main.html

[22]

Naef, M., Lamboray, E., Staadt, O., and Gross, M. The Blue-c Distributed Scene Graph. Proc. IPT/EGVE Workshop, (2003), 125--133.

Digital Library

[23]

Negroponte, N. Being Digital. Vintage Books, New York, NY, USA, 1995.

Digital Library

[24]

Nguyen, D. and Canny, J. MultiView: Spatially Faithful Group Video Conferencing. Proc. CHI, (2005), 799--808.

Digital Library

[25]

Nowak, K. L. and Biocca, F. The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments. Presence Teleoperators and Virtual Environments 12, 5 (2003), 481--494.

Digital Library

[26]

Okada, K.-I., Maeda, F., Ichikawaa, Y., and Matsushita, Y. Multiparty Videoconferencing at Virtual Social Distance: MAJIC Design. Proc. CSCW, (1994), 385--393.

Digital Library

[27]

Olmos, A., Lachapelle, K., and Cooperstock, J. R. Multiple Angle Viewer for Remote Medical Training. Proc. International Workshop on Multimedia Technologies for Distance Leaning, (2010), 19--24.

Digital Library

[28]

OpenCV. http://opencv.willowgarage.com/

[29]

OpenNI. http://openni.org/

[30]

Rosenthal, A. Two-way Television Communication Unit. (1947).

[31]

Sellen, A., Buxton, B., and Arnott, J. Using Spatial Cues to Improve Videoconferencing. Proc. CHI, (1992), 651--652.

Digital Library

[32]

Tang, J. C. and Minneman, S. VideoWhiteboard: Video Shadows to Support Remote Collaboration. Proc. CHI, (1991), 315--322.

Digital Library

[33]

Vertegaal, R. and Ding, Y. Explaining Effects of Eye Gaze on Mediated Group Conversations: Amount or Synchronization/? Proc. CSCW, (2002), 278--285.

Digital Library

[34]

Vertegaal, R., Slagter, R., Van der Veer, G., and Nijholt, A. Eye Gaze Patterns in Conversations: There is More to Conversational Agents than Meets the Eyes. Proc. CHI, (2001), 301--308.

Digital Library

[35]

Vertegaal, R., Weevers, I., Sohn, C., and Cheung, C. GAZE-2: Conveying Eye Contact in Group Video Conferencing Using Eye-controlled Camera Direction. Proc. CHI, (2003), 521--528.

Digital Library

[36]

Vertegaal, R. The GAZE Groupware System: Mediating Joint Attention in Multiparty Communication and Collaboration. Proc. CHI, (1999), 294--301.

Digital Library

[37]

Ware, C., Arthur, K., and Booth, K. S. Fish Tank Virtual Reality. Proc. CHI, (1993), 37--42.

Digital Library

[38]

Wright, M. and Freed, A. Open Sound Control: A New Protocol for Communicating with Sound Synthesizers. Proc. International Computer Music Conference, (1997), 101--104.

[39]

Zhang, C., Yin, Z., and Florencio, D. Improving Depth Perception with Motion Parallax and its Application in Teleconferencing. Proc. IEEE International Workshop on Multimedia Signal Processing, (2009), 1--6.

Cited By

Dybboe MEllemose JVastrup ABoudouraki ARintel SPetersen MGrønbæk JKlokmose C(2024)TableBot: Getting a Handle on Hybrid Collaboration by Negotiating Control of a Tabletop Telepresence RobotProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685350(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685350
Schuessler MHormann LDachselt RBlake ARother C(2024)Gazing Heads: Investigating Gaze Perception in Video-Mediated CommunicationACM Transactions on Computer-Human Interaction10.1145/366034331:3(1-31)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1145/3660343
Park SSong JKarger DMalone T(2024)Who2chat: A Social Networking System for Academic Researchers in Virtual Social Hours Enabling Coordinating, Overcoming Barriers and Social SignalingProceedings of the ACM on Human-Computer Interaction10.1145/36374358:CSCW1(1-34)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637435
Show More Cited By

Index Terms

TeleHuman: effects of 3d perspective on gaze and pose estimation with a life-size cylindrical telepresence pod
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

TeleHuman2: A Cylindrical Light Field Teleconferencing System for Life-size 3D Human Telepresence
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

For telepresence to support the richness of multiparty conversations, it is important to convey motion parallax and stereoscopy without head-worn apparatus. TeleHuman2 is a "hologrammatic" telepresence system that conveys full-body 3D video of ...
LightBee: A Self-Levitating Light Field Display for Hologrammatic Telepresence
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

LightBee is a novel "hologrammatic" telepresence system featuring a self-levitating light field display. It consists of a drone that flies a projection of a remote user's head through 3D space. The movements of the drone are controlled by the remote ...
Movable cameras enhance social telepresence in media spaces
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Media space is a promising but still immature technology to connect distributed sites. We developed a simple additional function that moved a remote camera forward when a local user approached a display so that the approach was amplified by a remote ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

May 2012

3276 pages

ISBN:9781450310154

DOI:10.1145/2207676

General Chair:
Joseph A. Konstan
University of Minnesota
,
Program Chairs:
Ed H. Chi
Google
,
Kristina Höök
Mobile Life at KTH

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 May 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '12

Sponsor:

SIGCHI

CHI '12: CHI Conference on Human Factors in Computing Systems

May 5 - 10, 2012

Texas, Austin, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

77
Total Citations
View Citations
1,644
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)5

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dybboe MEllemose JVastrup ABoudouraki ARintel SPetersen MGrønbæk JKlokmose C(2024)TableBot: Getting a Handle on Hybrid Collaboration by Negotiating Control of a Tabletop Telepresence RobotProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685350(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685350
Schuessler MHormann LDachselt RBlake ARother C(2024)Gazing Heads: Investigating Gaze Perception in Video-Mediated CommunicationACM Transactions on Computer-Human Interaction10.1145/366034331:3(1-31)Online publication date: 11-Jun-2024
https://dl.acm.org/doi/10.1145/3660343
Park SSong JKarger DMalone T(2024)Who2chat: A Social Networking System for Academic Researchers in Virtual Social Hours Enabling Coordinating, Overcoming Barriers and Social SignalingProceedings of the ACM on Human-Computer Interaction10.1145/36374358:CSCW1(1-34)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637435
Kumar CSaini BStaab S(2024)Enhancing Online Meeting Experience through Shared Gaze-AttentionExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651068(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651068
Sayffaerth CRasch JMüller F(2024)“Tele” Me More: Using Telepresence Charades to Connect Strangers and Exhibits in Different MuseumsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650834(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650834
Wang XZhang WFu H(2024)A3RT: Attention-Aware AR Teleconferencing with Life-Size 2.5D Video Avatars2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00044(211-221)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00044
Kang SKim GLee KKim S(2023)How Do Background and Remote User Representations Affect Social Telepresence in Remote Collaboration?: A Study with Portal Display, a Head Pose-Responsive Video Teleconferencing SystemElectronics10.3390/electronics1220433912:20(4339)Online publication date: 19-Oct-2023
https://doi.org/10.3390/electronics12204339
Yu KRoth DStrak RPankratz FReichling JKraetsch CWeidert SLazarovici MNavab NEck U(2023)Mixed Reality 3D Teleconsultation for Emergency Decompressive Craniotomy: An Evaluation with Medical Residents2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00081(662-671)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00081
Campo AMichałko AVan Kerrebroeck BStajic BPokric MLeman M(2023)The assessment of presence and performance in an AR environment for motor imitation learningComputers in Human Behavior10.1016/j.chb.2023.107810146:COnline publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1016/j.chb.2023.107810
Surale HTham YSmith BVaish R(2022)ARcall: Real-Time AR Communication using Smartphones and SmartglassesProceedings of the Augmented Humans International Conference 202210.1145/3519391.3519398(46-57)Online publication date: 13-Mar-2022
https://dl.acm.org/doi/10.1145/3519391.3519398
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten