research-article

LightBee: A Self-Levitating Light Field Display for Hologrammatic Telepresence

Authors:

Timothy Merritt,

Roel VertegaalAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 12, Pages 1 - 10

https://doi.org/10.1145/3290605.3300242

Published: 02 May 2019 Publication History

Abstract

LightBee is a novel "hologrammatic" telepresence system featuring a self-levitating light field display. It consists of a drone that flies a projection of a remote user's head through 3D space. The movements of the drone are controlled by the remote user's head movements, offering unique support for non-verbal cues, especially physical proxemics. The light field display is created by a retro-reflective sheet that is mounted on the cylindrical quadcopter. 45 smart projectors, one per 1.3 degrees, are mounted in a ring, each projecting a video stream rendered from a unique perspective onto the retroreflector. This creates a light field that naturally provides motion parallax and stereoscopy without requiring any headset nor stereo glasses. LightBee allows multiple local users to experience their own unique and correct perspective of the remote user's head. The system is currently one-directional: 2 small cameras mounted on the drone allow the remote user to observe the local scene.

Supplementary Material

ZIP File (pn3957.zip)

Change doc

Download
49.49 MB

MOV File (paper012.mov)

Supplemental video

Download
57.57 MB

MP4 File (paper012.mp4)

Download
380.94 MB

References

[1]

Sigurdur O. Adalgeirsson and Cynthia Breazeal. 2010. MeBot: a robotic platform for socially embodied presence. In Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction (HRI '10). IEEE Press, Piscataway, NJ, USA, 15--22.

Digital Library

[2]

Michael Argyle and Janet Dean. 1965. Eye-Contact, Distance and Affiliation. Sociometry 28, 3: 289.

[3]

Marius Braun, Ulrich Leiner, and Detlef Ruschin. 2011. Evaluating motion and binocular parallax as depth cues for autostereoscopic displays. of SPIE, the International Society for Optical Engineering. Society of Photo-Optical Instrumentation Engineers.

[4]

William A. S. Buxton, Abigail J. Sellen, and Michael C. Sheasby. 1997. Interfaces for multiparty videoconferences. Video-mediated.

[5]

Mina Choi, Rachel Kornfield, Leila Takayama, and Bilge Mutlu. 2017. Movement Matters: Effects of Motion and Mimicry on Perception of Similarity and Closeness in Robot-Mediated Communication. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 325--335.

Digital Library

[6]

Docomo Drone Project. Retrieved September 14, 2017 from https://phys.org/news/2017-04-world-sphericaldrone.html

[7]

Neil A. Dodgson. 2004. Variation and extrema of human interpupillary distance. Electronic imaging 2004.

[8]

Double 2. Retrieved September 14, 2017 from https://www.doublerobotics.com/double2.html

[9]

Antonio Gomes, Calvin Rubens, Sean Braley, and Roel Vertegaal. 2016. BitDrones: Towards Using 3D Nanocopter Displays As Interactive Self-Levitating Programmable Matter. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems: 770--780.

Digital Library

[10]

Rita Gorawara-Bhat, Mary Ann Cook, and Greg A. Sachs. 2007. Nonverbal communication in doctorelderly patient transactions (NDEPT): Development of a tool. Patient Education and Counseling 66, 2: 223-- 234.

[11]

Dan Gotsch, Xujing Zhang, Tim Merritt, and Roel Vertegaal. 2018. TeleHuman2: A Cylindrical Light Field Teleconferencing System for Life-size 3D Human Telepresence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York.

Digital Library

[12]

Maureen Graham and Brian Rogers. 1982. Simultaneous and Successive Contrast Effects in the Perception of Depth from Motion-Parallax and Stereoscopic Information. Perception 11, 3: 247--262.

[13]

Edward Hall. 1969. The Hidden Dimension?: man's use of space in public and in private.

[14]

Keita Higuchi and Jun Rekimoto. 2013. Flying head: a head motion synchronization mechanism for unmanned aerial vehicle control. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, New York, NY, USA, 2029--2038.

Digital Library

[15]

iRobot. Retrieved September 14, 2017 from http://www.irobot.com/filelibrary/pdfs/hrd/ava/AvaBrochure-May2--2011.pdf

[16]

Frederic E. Ives. 1903. Parallax stereogram and process of making same.

[17]

Andrew Jones, Koki Nagano, Jing Liu, Jay Busch, Xueming Yu, Mark Bolas, Paul Debevec. 2014. Interpolating vertical parallax for an autostereoscopic three-dimensional projector array. Journal of Electronic Imaging.

[18]

Andrew Jones, Magnus Lang, Graham Fyffe, Xueming Yu, Jay Busch, Ian McDowall, Mark Bolas, and Paul Debevec. 2009. Achieving eye contact in a one-tomany 3D video teleconferencing system. ACM Transactions on Graphics 28, 212: 1.

Digital Library

[19]

Handry Khoswanto, Petrus Santoso, and Resmana Lim. 2016. Odometry Algorithm with Obstacle Avoidance on Mobile Robot Navigation. In Proceedings of Second International Conference on Electrical Systems, Technology and Information 2015 (ICESTI 2015), pp. 155--161. Springer, Singapore.

[20]

Kibum Kim, John Bolton, Audrey Girouard, Jeremy Cooperstock, and Roel Vertegaal. 2012. TeleHuman. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12, 2531.

[21]

Min Kyung Lee and Leila Takayama. 2011. "Now, i have a body": uses and social norms for mobile remote presence in the workplace. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 33--42.

Digital Library

[22]

Peter Lincoln, Greg Welch, Andrew Nashel, Adrian Ilie, Andrei State, and Henry Fuchs. 2009. Animatronic Shader Lamps Avatars. In Proceedings of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '09). IEEE Computer Society, Washington, DC, USA, 27--33.

Digital Library

[23]

Gabriel Lippmann. 1908. Epreuves reversibles. Photographies integrals. Comptes-Rendus Academie des Sciences 146: 446--451

[24]

Akira Matsuda, Takashi Miyaki, and Jun Rekimoto. 2017. ScalableBody: a telepresence robot that supports face position matching using a vertical actuator. In Proceedings of the 8th Augmented Human International Conference (AH '17). ACM, New York, NY, USA, Article 13, 9 pages.

Digital Library

[25]

Rachel McDonnell, Sophie Jörg, Joanna McHugh, Fiona N. Newell, and Carol O'Sullivan. 2009. Investigating the role of body shape on the perception of emotion. ACM Transactions on Applied Perception 6, 3: 1--11.

Digital Library

[26]

Masahiro Mori. 1970. The uncanny valley. IEEE Robotics & Automation Magazine. 19 (2): 98--100.

[27]

Koki Nagano, Andrew Jones, Jing Liu, et al. 2013. An autostereoscopic projector array optimized for 3D facial display. In ACM SIGGRAPH 2013 Emerging Technologies on - SIGGRAPH '13, 1--1.

Digital Library

[28]

Carman Neustaedter, Gina Venolia, Jason Procyk, and Daniel Hawkins. 2016. To Beam or Not to Beam: A Study of Remote Telepresence Attendance at an Academic Conference. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW '16). ACM, New York, NY, USA, 418--431.

Digital Library

[29]

David Nguyen and John Canny. 2005. MultiView: spatially faithful group video conferencing. Proceedings of the SIGCHI conference on Human Factors in Computing Systems: 799--808.

Digital Library

[30]

Ken-Ichi Okada, Fumihiko Maeda, Yusuke Ichikawaa, and Yutaka Matsushita. 1994. Multiparty videoconferencing at virtual social distance: MAJIC design. CSCW '94: Proceedings of the 1994 ACM conference on Computer supported cooperative work: 385--393.

Digital Library

[31]

Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, and Shahram Izadi. 2016. Holoportation: Virtual 3D Teleportation in Real-time. Proceedings of the 29th Annual Symposium on User Interface Software and Technology: 741--754.

Digital Library

[32]

Ye Pan and Anthony Steed. 2014. A gaze-preserving situated multiview telepresence system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2173--2176.

Digital Library

[33]

Eric Paulos and John Canny. 1998. PRoP: Personal Roving Presence. In Proceedings of the Conference on Human Factors in Computing Systems. Los Angeles, CA, 296--303

Digital Library

[34]

Ekaterina Peshkova, Martin Hitz, and Bonifaz Kaufmann. 2017. Natural Interaction Techniques for an Unmanned Aerial Vehicle System. IEEE Pervasive Computing.

Digital Library

[35]

Fábio Policarpo, Manuel M. Oliveira, and João L. D. Comba. 2005. Real-time Relief Mapping on Arbitrary Polygonal Surfaces. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games, 155--162.

Digital Library

[36]

Irene Rae, Bilge Mutlu, and Leila Takayama. 2014. Bodies in motion: mobility, presence, and task awareness in telepresence. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 21532162.

Digital Library

[37]

Irene Rae, Leila Takayama, and Bilge Mutlu. The influence of height in robot-mediated communication. In Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction, pp. 1--8. IEEE Press, 2013.

Digital Library

[38]

Abigail Sellen, Bill Buxton, and John Arnott. 1992. Using spatial cues to improve videoconferencing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92), Penny Bauersfeld, John Bennett, and Gene Lynch (Eds.). ACM, New York, NY, USA, 651--652.

Digital Library

[39]

John Short, Ederyn Williams, and Bruce Christie. 1976. The social psychology of telecommunications.

[40]

Mohammed Tahri Sqalli, Kyoichi Tatsuno, Koichi Kurabe, Hiroto Ando, Hideharu Obitsu, Ryoma Itakura, Takatoshi Aoto, and Katsumi Yoshino. 2016. Improvement of a tele-presence robot autonomous navigation Using SLAM algorithm. In MicroNanoMechatronics and Human Science (MHS), 2016 International Symposium on, pp. 1--7. IEEE.

[41]

Susumu Tachi, Kouichi Watanabe, Keisuke Takeshita, Kouta Minamizawa, Takumi Yoshida, and Katsunari Sato. 2011. Mutual telexistence surrogate system: Telesar4-telexistence in real environments using autostereoscopic immersive display. In Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, pp. 157--162. IEEE.

[42]

Leila Takayama,Eitan Marder-Eppstein, Helen Harris, and Jenay M. Beer. 2011. Assisted driving of a mobile remote presence system: System desiagn and controlled user evaluation. In Robotics and Automation (ICRA), 2011 IEEE International Conference on, pp. 1883--1889. IEEE.

[43]

John C. Tang and Scott Minneman. 1991. VideoWhiteboard: Video Shadows to Support Remote Collaboration. Proceedings of the SIGCHI conference on Human factors in computing systems Reaching through technology - CHI '91: 315--322.

Digital Library

[44]

H Tobita, S Maruyama, and T Kuzi. 2011. Floating avatar: telepresence system using blimps for communication and entertainment. CHI'11 Extended Abstracts on Human.

Digital Library

[45]

Roel Vertegaal and Yaping Ding. 2002. Explaining effects of eye gaze on mediated group conversations: In Proceedings of the 2002 ACM conference on Computer supported cooperative work - CSCW '02, 41.

Digital Library

[46]

Roel Vertegaal, Ivo Weevers, Changuk Sohn, and Chris Cheung. 2003. GAZE-2: conveying eye contact in group video conferencing using eye-controlled camera direction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 521--528.

Digital Library

Cited By

Guindy MKara P(2024)Lessons Learned from Implementing Light Field Camera Animation: Implications, Limitations, Potentials, and Future Research EffortsMultimodal Technologies and Interaction10.3390/mti80800688:8(68)Online publication date: 1-Aug-2024
https://doi.org/10.3390/mti8080068
Javidi KMartini MKara P(2024)Investigation of the separation zone for split-domain light field visualization2024 28th International Conference on Information Technology (IT)10.1109/IT61232.2024.10475755(1-4)Online publication date: 21-Feb-2024
https://doi.org/10.1109/IT61232.2024.10475755
Kara PWippelhauser ABalogh TBokor L(2023)How I Met Your V2X Sensor Data: Analysis of Projection-Based Light Field Visualization for Vehicle-to-Everything Communication Protocols and Use CasesSensors10.3390/s2303128423:3(1284)Online publication date: 22-Jan-2023
https://doi.org/10.3390/s23031284
Show More Cited By

Index Terms

LightBee: A Self-Levitating Light Field Display for Hologrammatic Telepresence
1. Hardware
  1. Emerging technologies
    1. Emerging interfaces

Recommendations

TeleHuman2: A Cylindrical Light Field Teleconferencing System for Life-size 3D Human Telepresence
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

For telepresence to support the richness of multiparty conversations, it is important to convey motion parallax and stereoscopy without head-worn apparatus. TeleHuman2 is a "hologrammatic" telepresence system that conveys full-body 3D video of ...
TeleHuman: effects of 3d perspective on gaze and pose estimation with a life-size cylindrical telepresence pod
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

In this paper, we present TeleHuman, a cylindrical 3D display portal for life-size human telepresence. The TeleHuman 3D videoconferencing system supports 360 degree motion parallax as the viewer moves around the cylinder and optionally, stereoscopic 3D ...
Camera array calibration for light field acquisition

Light field cameras are becoming popular in computer vision and graphics, with many research and commercial applications already having been proposed. Various types of cameras have been developed with the camera array being one of the ways of acquiring ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,827
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Guindy MKara P(2024)Lessons Learned from Implementing Light Field Camera Animation: Implications, Limitations, Potentials, and Future Research EffortsMultimodal Technologies and Interaction10.3390/mti80800688:8(68)Online publication date: 1-Aug-2024
https://doi.org/10.3390/mti8080068
Javidi KMartini MKara P(2024)Investigation of the separation zone for split-domain light field visualization2024 28th International Conference on Information Technology (IT)10.1109/IT61232.2024.10475755(1-4)Online publication date: 21-Feb-2024
https://doi.org/10.1109/IT61232.2024.10475755
Kara PWippelhauser ABalogh TBokor L(2023)How I Met Your V2X Sensor Data: Analysis of Projection-Based Light Field Visualization for Vehicle-to-Everything Communication Protocols and Use CasesSensors10.3390/s2303128423:3(1284)Online publication date: 22-Jan-2023
https://doi.org/10.3390/s23031284
Kara PSimon A(2023)The Good News, the Bad News, and the Ugly Truth: A Review on the 3D Interaction of Light Field DisplaysMultimodal Technologies and Interaction10.3390/mti70500457:5(45)Online publication date: 27-Apr-2023
https://doi.org/10.3390/mti7050045
Praveena PWang YSenft EGleicher MMutlu B(2023)Periscope: A Robotic Camera System to Support Remote Physical CollaborationProceedings of the ACM on Human-Computer Interaction10.1145/36101997:CSCW2(1-39)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610199
Inoue MTakashima KFujita KKitamura Y(2023)BirdViewAR: Surroundings-aware Remote Drone Piloting Using an Augmented Third-person PerspectiveProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580681(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580681
Kara PTamboli RAdhikarla VBalogh TGuindy MSimon A(2023)Connected without disconnection: Overview of light field metaverse applications and their quality of experienceDisplays10.1016/j.displa.2023.10243078(102430)Online publication date: Jul-2023
https://doi.org/10.1016/j.displa.2023.102430
Trepnau DRichter K(2023)Avatar: A Telepresence System for the Participation in Remote EventsReliability and Statistics in Transportation and Communication10.1007/978-3-031-26655-3_23(252-258)Online publication date: 21-Feb-2023
https://doi.org/10.1007/978-3-031-26655-3_23
Guindy MBarsi AKara PAdhikarla VBalogh TSimon A(2022)Camera Animation for Immersive Light Field ImagingElectronics10.3390/electronics1117268911:17(2689)Online publication date: 27-Aug-2022
https://doi.org/10.3390/electronics11172689
Kara PTamboli RShafiee EMartini MSimon AGuindy M(2022)Beyond Perceptual Thresholds and Personal Preference: Towards Novel Research Questions and Methodologies of Quality of Experience Studies on Light Field VisualizationElectronics10.3390/electronics1106095311:6(953)Online publication date: 19-Mar-2022
https://doi.org/10.3390/electronics11060953
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten