research-article

Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration

Authors:

Daniel Leithinger,

Hiroshi IshiiAuthors Info & Claims

UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

Pages 461 - 470

https://doi.org/10.1145/2642918.2647377

Published: 05 October 2014 Publication History

Abstract

We propose a new approach to Physical Telepresence, based on shared workspaces with the ability to capture and remotely render the shapes of people and objects. In this paper, we describe the concept of shape transmission, and propose interaction techniques to manipulate remote physical objects and physical renderings of shared digital content. We investigate how the representation of user's body parts can be altered to amplify their capabilities for teleoperation. We also describe the details of building and testing prototype Physical Telepresence workspaces based on shape displays. A preliminary evaluation shows how users are able to manipulate remote objects, and we report on our observations of several different manipulation techniques that highlight the expressive nature of our system.

Supplementary Material

suppl.mov (uistf2493-file3.mp4)

Supplemental video

Download
73.72 MB

References

[1]

Benko, H., Jota, R., and Wilson, A. MirageTable: freehand interaction on a projected augmented reality tabletop. In ACM CHI '12 (2012), 199--208.

Digital Library

[2]

Brave, S., Ishii, H., and Dahley, A. Tangible interfaces for remote collaboration and communication. In ACM CSCW '98 (Nov. 1998), 169--178.

Digital Library

[3]

Cutkosky, M., and Howe, R. Human grasp choice and robotic grasp analysis. In Dextrous Robot Hands, S. Venkataraman and T. Iberall, Eds. Springer New York, 1990, 5--31.

Digital Library

[4]

Everitt, K. M., Klemmer, S. R., Lee, R., and Landay, J. A. Two worlds apart: bridging the gap between physical and virtual media for distributed design collaboration. In ACM CHI '03 (Apr. 2003), 553--560.

Digital Library

[5]

Festo. WaveHandling: Transporting and sorting in one. In http://www.festo.com/cms/en corp/13136.htm (2013).

[6]

Follmer, S., Leithinger, D., Olwal, A., Cheng, N., and Ishii, H. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In ACM UIST '12 (Oct. 2012), 519--528.

Digital Library

[7]

Follmer, S., Leithinger, D., Olwal, A., Hogge, A., and Ishii, H. inFORM: dynamic physical affordances and constraints through shape and object actuation. In ACM UIST '13 (Oct. 2013), 417--426.

Digital Library

[8]

Genecov, A. M., Stanley, A. A., and Okamura, A. M. Perception of a Haptic Jamming display: Just noticeable differences in stiffness and geometry. In 2014 IEEE Haptics Symposium (HAPTICS), IEEE (Feb. 2014), 333--338.

[9]

Gross, M., Lang, S., Strehlke, K., Moere, A. V., Staadt, O., Wüurmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., and Van Gool, L. blue-c: a spatially immersive display and 3D video portal for telepresence. ACM Transactions on Graphics 22, 3 (July 2003), 819--827.

Digital Library

[10]

Hilliges, O., Kim, D., Izadi, S., Weiss, M., and Wilson, A. HoloDesk: direct 3d interactions with a situated see-through display. In ACM CHI '12 (May 2012), 2421--2430.

Digital Library

[11]

Ishii, H., and Kobayashi, M. ClearBoard: a seamless medium for shared drawing and conversation with eye contact. In ACM CHI '92 (June 1992), 525--532.

Digital Library

[12]

Iwata, H., Yano, H., Nakaizumi, F., and Kawamura, R. Project FEELEX: adding haptic surface to graphics. In ACM SIGGRAPH '01 (Aug. 2001), 469--476.

Digital Library

[13]

Lee, M. K., and Takayama, L. "Now, i have a body": uses and social norms for mobile remote presence in the workplace. In ACM CHI '11 (May 2011), 33--42.

Digital Library

[14]

Leithinger, D., Follmer, S., Olwal, A., Luescher, S., Hogge, A., Lee, J., and Ishii, H. Sublimate: state-changing virtual and physical rendering to augment interaction with shape displays. In ACM CHI '13 (Apr. 2013), 1441--1450.

Digital Library

[15]

Leithinger, D., Lakatos, D., DeVincenzi, A., Blackshaw, M., and Ishii, H. Direct and gestural interaction with relief: a 2.5D shape display. In ACM UIST '11 (Oct. 2011), 541--548.

Digital Library

[16]

Lum, M. J., Rosen, J., Lendvay, T. S., Sinanan, M. N., and Hannaford, B. Effect of time delay on telesurgical performance. In IEEE ICRA'09 (2009), 4246--4252.

Digital Library

[17]

Mann, S. Telepointer: Hands-Free Completely Self Contained Wearable Visual Augmented Reality without Headwear and without any Infrastructural Reliance. In 2012 16th International Symposium on Wearable Computers (Oct. 2000), 177--178.

Digital Library

[18]

Mueller, F. F., Cole, L., O'Brien, S., and Walmink, W. Airhockey over a distance. In CHI '06 extended abstracts (Apr. 2006), 1133--1138.

Digital Library

[19]

Pangaro, G., Maynes-Aminzade, D., and Ishii, H. The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In ACM UIST '02 (Oct. 2002), 181--190.

Digital Library

[20]

Paul, R. P. Robot manipulators: mathematics, programming, and control: the computer control of robot manipulators. Richard Paul, 1981.

Digital Library

[21]

Paulos, E., and Canny, J. PRoP: personal roving presence. In ACM CHI '98 (Jan. 1998), 296--303.

Digital Library

[22]

Piper, B., and Ishii, H. PegBlocks: a learning aid for the elementary classroom. In ACM CHI '02 extended abstracts (Apr. 2002), 686--687.

Digital Library

[23]

Poupyrev, I., Nashida, T., and Okabe, M. Actuation and Tangible User Interfaces: the Vaucanson Duck, Robots, and Shape Displays. In ACM TEI '07 (Feb. 2007), 205--212.

Digital Library

[24]

Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. The office of the future: a unified approach to image-based modeling and spatially immersive displays. In ACM SIGGRAPH '98 (July 1998), 179--188.

Digital Library

[25]

Rasmussen, M. K., Pedersen, E. W., Petersen, M. G., and Hornbæ k, K. Shape-changing interfaces: a review of the design space and open research questions. In ACM CHI '12 (May 2012), 735--744.

Digital Library

[26]

Richter, J., Thomas, B. H., Sugimoto, M., and Inami, M. Remote active tangible interactions. In ACM TEI '07 (Feb. 2007), 39--42.

Digital Library

[27]

Riedenklau, E., Hermann, T., and Ritter, H. An integrated multi-modal actuated tangible user interface for distributed collaborative planning. In ACM TEI '12 (Feb. 2012), 169--174.

Digital Library

[28]

Sodhi, R. S., Jones, B. R., Forsyth, D., Bailey, B. P., and Maciocci, G. BeThere: 3D mobile collaboration with spatial input. In ACM CHI '13 (Apr. 2013), 179--188.

Digital Library

[29]

Staub, C., Ono, K., Mayer, H., Knoll, A., Ulbrich, H., and Bauernschmitt, R. Remote minimally invasive surgery --haptic feedback and selective automation in medical robotics. Applied Bionics and Biomechanics 8, 2 (Apr. 2011), 221--236.

Digital Library

[30]

Tang, J. C., and Minneman, S. VideoWhiteboard: video shadows to support remote collaboration. In ACM CHI '91 (Mar. 1991), 315--322.

Digital Library

[31]

Tang, J. C., and Minneman, S. L. Videodraw: a video interface for collaborative drawing. ACM Transactions on Information Systems 9, 2 (Apr. 1991), 170--184.

Digital Library

[32]

Tsui, K. M., Desai, M., Yanco, H. A., and Uhlik, C. Exploring use cases for telepresence robots. In ACM/IEEE HRI '11 (Mar. 2011), 11--18.

Digital Library

[33]

Wilson, A. D., and Robbins, D. C. Playtogether: Playing games across multiple interactive tabletops. In IUI Workshop on Tangible Play: Research and Design for Tangible and Tabletop Games (2007).

[34]

Yarosh, S., Tang, A., Mokashi, S., and Abowd, G. D. "almost touching": parent-child remote communication using the sharetable system. In ACM CSCW '13 (Feb. 2013), 181--192.

Digital Library

Cited By

Wang YWang KWang ZPerlin K(2025)Robotecture: A Modular Shape-changing Interface Using Actuated Support BeamsProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704925(1-11)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3704925
Wu QLi JSeifi HHornbæk K(2025)Vipins: Combining a pin array and vibrotactile actuators to render complex shapes and texturesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2025.103464197(103464)Online publication date: Mar-2025
https://doi.org/10.1016/j.ijhcs.2025.103464
Wittig NDrey TWettig TAuda JKoelle MGoedicke DSchneegass S(2024)LeARn at Home: Comparing Augmented Reality and Video Conferencing Remote TutoringProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701577(255-263)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701577
Show More Cited By

Index Terms

Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

inFORM: dynamic physical affordances and constraints through shape and object actuation
UIST '13: Proceedings of the 26th annual ACM symposium on User interface software and technology

Past research on shape displays has primarily focused on rendering content and user interface elements through shape output, with less emphasis on dynamically changing UIs. We propose utilizing shape displays in three different ways to mediate ...
Kinetic Blocks: Actuated Constructive Assembly for Interaction and Display
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology

Pin-based shape displays not only give physical form to digital information, they have the inherent ability to accurately move and manipulate objects placed on top of them. In this paper we focus on such object manipulation: we present ideas and ...
shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

We explore interactions enabled by 2D spatial manipulation and self-actuation of a tabletop shape display. To explore these interactions, we developed shapeShift, a compact, high-resolution (7 mm pitch), mobile tabletop shape display. shapeShift can be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology

October 2014

722 pages

ISBN:9781450330695

DOI:10.1145/2642918

General Chair:
Hrvoje Benko
Microsoft Research, USA
,
Program Chairs:
Mira Dontcheva
Adobe, USA
,
Daniel Wigdor
University of Toronto, Canada

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 October 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

UIST '14

Sponsor:

UIST '14: The 27th Annual ACM Symposium on User Interface Software and Technology

October 5 - 8, 2014

Hawaii, Honolulu, USA

Acceptance Rates

UIST '14 Paper Acceptance Rate 74 of 333 submissions, 22%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

121
Total Citations
View Citations
2,060
Total Downloads

Downloads (Last 12 months)147
Downloads (Last 6 weeks)38

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YWang KWang ZPerlin K(2025)Robotecture: A Modular Shape-changing Interface Using Actuated Support BeamsProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704925(1-11)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3704925
Wu QLi JSeifi HHornbæk K(2025)Vipins: Combining a pin array and vibrotactile actuators to render complex shapes and texturesInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2025.103464197(103464)Online publication date: Mar-2025
https://doi.org/10.1016/j.ijhcs.2025.103464
Wittig NDrey TWettig TAuda JKoelle MGoedicke DSchneegass S(2024)LeARn at Home: Comparing Augmented Reality and Video Conferencing Remote TutoringProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701577(255-263)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701577
Burnah TImtiaz MGuesgen HRudolph GBlagojevic R(2024)Passive Stylus Tracking: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36981448:ISS(427-461)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698144
Huang XYin MXia ZXiao R(2024)VirtualNexus: Enhancing 360-Degree Video AR/VR Collaboration with Environment Cutouts and Virtual ReplicasProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676377(1-12)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676377
Qian WGao CSathya ASuzuki RNakagaki K(2024)SHAPE-IT: Exploring Text-to-Shape-Display for Generative Shape-Changing Behaviors with LLMsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676348(1-29)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676348
Wang KWang ZNakagaki KPerlin K(2024)“Push-That-There”: Tabletop Multi-robot Object Manipulation via Multimodal 'Object-level Instruction'Proceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661542(2497-2513)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661542
Patibanda ROverdevest NNisal SSaini AElvitigala DKnibbe JVan Den Hoven EMueller F(2024)Shared Bodily Fusion: Leveraging Inter-Body Electrical Muscle Stimulation for Social PlayProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660723(2088-2106)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660723
Hunt CSun KTseng KBalasubramaniyam PDruin AHuynh ALeithinger DYip J(2024)Making a Metaphor Sandwich: Analyzing Children's use of Metaphor During Tabletop Telepresence Robot Supported Participatory DesignProceedings of the 23rd Annual ACM Interaction Design and Children Conference10.1145/3628516.3656272(173-188)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3628516.3656272
Ichihashi SKuroki SNishimura MKasaura KHiraki TTanaka KYoshida S(2024)Swarm Body: Embodied Swarm RobotsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642870(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642870
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten