research-article

GridDrones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface Deformations

Authors:

Timothy Merritt,

Roel VertegaalAuthors Info & Claims

UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

Pages 87 - 98

https://doi.org/10.1145/3242587.3242658

Published: 11 October 2018 Publication History

Abstract

We present GridDrones, a self-levitating programmable matter platform that can be used for representing 2.5D voxel grid relief maps capable of rendering unsupported structures and 3D transformations. GridDrones consists of cube-shaped nanocopters that can be placed in a volumetric 1xnxn mid-air grid, which is demonstrated here with 15 voxels. The number of voxels and scale is only limited by the size of the room and budget. Grid deformations can be applied interactively to this voxel lattice by manually selecting a set of voxels, then assigning a continuous topological relationship between voxel sets that determines how voxels move in relation to each other and manually drawing out selected voxels from the lattice structure. Using this simple technique, it is possible to create unsupported structures that can be translated and oriented freely in 3D. Shape transformations can also be recorded to allow for simple physical shape morphing animations. This work extends previous work on selection and editing techniques for 3D user interfaces.

Supplementary Material

suppl.mov (ufp1482.m4v)

Supplemental video

Download
80.91 MB

suppl.mov (ufp1482p.mp4)

Supplemental video

Download
7.52 MB

MP4 File (p87-braley.mp4)

Download
297.18 MB

References

[1]

Harshit Agrawal, Udayan Umapathi, Robert Kovacs, Johannes Frohnhofen, Hsiang-Ting Chen, Stefanie Mueller, and Patrick Baudisch. 2015. Protopiper: Physically Sketching Room-Sized Objects at Actual Scale. In Proc. UIST '15, 427--436.

Digital Library

[2]

Javier Alonso-Mora, Andreas Breitenmoser, Martin Rufli, Roland Siegwart, and Paul Beardsley. 2011. Multi-Robot System for Artistic Pattern Formation. ICRA'11, 4512--4517.

[3]

Javier Alonso-Mora, Stefan Lohaus, Phillip Leemann, Roland Siegwart and Paul Beardsley. 2015. Gesture Based Human-Multi-Robot Swarm Interaction and its Application to an Interactive Display. ICRA'15, 5948--5953.

[4]

Raffaello D'Andrea. 2013. Humans and the coming machine revolution. In Proc. UIST'13, 1--2.

Digital Library

[5]

Ars Electronica Futurelab. 2016. Drone 100. http://tinyurl.com/drone100. Retrieved April 2016.

[6]

Matthew Blackshaw, Anthony DeVincenzi, David Lakatos, Daniel Leithinger, and Hiroshi Ishii. 2011. Recompose: Direct and Gestural Interaction with an Actuated Surface. In Proc. CHI EA '11, 1237--1242.

Digital Library

[7]

Sean Braley, Calvin Rubens, Timothy R. Merritt, and Roel Vertegaal. 2018. GridDrones: A Self-Levitating Physical Voxel Lattice for 3D Surface Deformations. In Ext. Abstracts CHI'18. Paper D200, 4 pages.

Digital Library

[8]

Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces. Personal Ubiquitous Computing. 8, 5, 347--358.

Digital Library

[9]

Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: Dynamic Physical Affordances and Constraints Through Shape and Object Actuation. In Proc. UIST'13, 417--426.

Digital Library

[10]

Sean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, and Hiroshi Ishii. 2012. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In Proc. UIST'12, 519--528.

Digital Library

[11]

Antonio Gomes, Calvin Rubens, Sean Braley, and Roel Vertegaal. 2016. BitDrones: Towards Using 3D Nanocopter Displays as Interactive Self-Levitating Programmable Matter. In Proc. CHI '16, 770--780.

Digital Library

[12]

Seth C. Goldstein, and Todd C. Mowry. 2004. Claytronics: A Scalable Basis for Future Robots. In RoboSphere'04.

[13]

Seth C. Goldstein, Jason D. Campbell, and Todd C. Mowry. Programmable Matter. 2005. IEEE Computer, 38, 6, 99--101.

Digital Library

[14]

Mathieu Le Goc, Lawrence H. Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Dragicevic, and Sean Follmer. 2016. Zooids: Building Blocks for Swarm User Interfaces. In Proc. UIST'16, 97--109.

Digital Library

[15]

Tovi Grossman and Ravin Balakrishnan. 2006. The design and evaluation of selection techniques for 3D volumetric displays. In Proc UIST'06. ACM, New York, NY, USA, 3--12.

Digital Library

[16]

Hooman Hedayati, Michael Walker, and Daniel Szafir. 2018. Improving Collocated Robot Teleoperation with Augmented Reality. In Proc HRI'18, 78--86.

Digital Library

[17]

Jacques Heyman. 1966. The stone skeleton. International Journal of Solids and Structures 2, 2: 249--279.

[18]

Hiroshi Ishii, and Brygg Ullmer. 1997. Tangible Bits: Beyond Pixels. In Proc. CHI'97, 234--241.

Digital Library

[19]

Hiroshi Ishii, Dávid Lakatos, Leonardo Bonanni, and Jean-Baptiste Labrune. 2012. Radical atoms: Beyond tangible bits, toward transformable materials. Interactions 19, 1, 38--51.

Digital Library

[20]

Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2013. Evaluating the efficiency of physical visualizations. In Proc. CHI'13, 2593--2602.

Digital Library

[21]

Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper Hornbæk. 2015. Opportunities and Challenges for Data Physicalization. In Proc. CHI '15, 3227--3236.

Digital Library

[22]

Shunichi Kasahara, Ryuma Niiyama, Valentin Heun, and Hiroshi Ishii. 2013. exTouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In Proc TEI'13, 223--228.

Digital Library

[23]

Brian Kirby, Jason Campbell, Burak Aksak, Padmanabhan Pillai, James Hoburg, Todd C. Mowry and Seth C. Goldstein. 2005. Catoms: Moving robots without moving parts. AAAI Robotics Exhibition'05, 1730--1731.

Digital Library

[24]

Haruhisa Kurokawa, Kohji Tomita, Akiya Kamimura, Shigeru Kokaji, Takashi Hasuo, and Satoshi Murata. 2008. Distributed Self-Reconfiguration of M-TRAN III Modular Robotic System. Int. J. Rob. Res. 27, 373--386.

Digital Library

[25]

Haruhisa Kurokawa, Satoshi Murata, Eiji Yoshida, Kohji Tomita, and Shigeru Kokaji. 1998. A 3-D Self-Reconfigurable Structure and Experiments," in Intelligent Robots and Systems, 860--865.

[26]

Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. ZeroN: Mid-air tangible interaction enabled by computer controlled magnetic levitation. In Proc. UIST'11, 327--336.

Digital Library

[27]

Daniel Leithinger and Hiroshi Ishii. 2010. Relief: A Scalable Actuated Shape Display. In Proc TEI'10, 221--222.

Digital Library

[28]

Henry Packard Moreton. 1992. Minimum Curvature Variation Curves, Networks, and Surfaces for Fair Free-Form Shape Design. Ph.D. Dissertation. University of California at Berkeley, Berkeley, CA, USA.

[29]

Javier Alonso-Mora, Andreas Breitenmoser, Martin Rufli, Roland Siegwart, and Paul Beardsley. 2012. Image and animation display with multiple mobile robots. Int. J. Rob. Res. 31, 6, 753--773.

Digital Library

[30]

Hiroki Nozaki. 2014. Flying display: A movable display pairing projector and screen in the air. In Proc. CHI EA'14, 909--914.

Digital Library

[31]

Yoichi Ochiai, Takayuki Hoshi and Jun Rekimoto. 2014. Pixie dust: Graphics generated by levitated and animated objects in computational acoustic-potential field. ACM Trans. Graph. 33, 4, 85.

Digital Library

[32]

Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto, and Yasufumi Yamaji. 2004. Lumen: Interactive visual and shape display for calm computing. In Proc SIGGRAPH '04, 17.

Digital Library

[33]

Gang Ren and Eamonn O'Neill. 2013. 3D selection with freehand gesture. Computers & Graphics 37, 3: 101--120.

Digital Library

[34]

John W. Romanishin, Kyle Gilpin and Daniela Rus. 2013. M-Blocks: Momentum-driven, Magnetic Modular Robots. In Intelligent Robots and Systems'13, 4288--4295

[35]

Michael Rubenstein, Alejandro Cornejo and Radhika Nagpal. 2014. Programmable self-assembly in a thousand-robot swarm. Science 345(6198), 795--799.

[36]

Michael Rubenstein, Christian Ahler and Radhika Nagpal. 2012. Kilobot: A low cost scalable robot system for collective behaviors. ICRA'12, 3293--98.

[37]

Juergen Scheible, Achim Hoth, Julian Saal and Haifeng Su. 2013. Displaydrone: A flying robot based interactive display. In Proc. PerDis'13, 49--54.

Digital Library

[38]

Philipp Schoessler, Daniel Windham, Daniel Leithinger, Sean Follmer, and Hiroshi Ishii. 2015. Kinetic Blocks: Actuated Constructive Assembly for Interaction and Display. In Proc. UIST '15, 341--349.

Digital Library

[39]

Stefan Schneegass, Florian Alt, Jürgen Scheible, Albrecht Schmidt, and Haifeng Su. 2014. Midair displays: Exploring the concept of free-floating public displays. In Proc. CHI EA'14, 2035--2040.

Digital Library

[40]

Alexa F. Siu, Eric J. Gonzalez, Shenli Yuan, Jason B. Ginsberg, and Sean Follmer. 2018. shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction. In Proc CHI'18. Paper 291, 13 pages.

Digital Library

[41]

Ivan Sutherland. 1965. The Ultimate Display. In Proc. IFIP 65, 2, 506--508.

[42]

Tomasso Toffoli and Norman Margolus. 1991. Programmable matter: Concepts and realization. Physica D: Nonlinear Phenomena 47, 1, 263--272.

Digital Library

[43]

Roel Vertegaal and Ivan Poupyrev. 2008. Organic User Interfaces. Communications of the ACM 51, 6, 26--30

Digital Library

[44]

Justin Werfel, Kirstin Petersen, and Radhika Nagpal. 2014. Designing collective behavior in a termite-inspired robot construction team. Science 343, 6172, 754--758.

[45]

Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva, and Hiroshi Ishii. 2013. PneUI: pneumatically actuated soft composite materials for shape changing interfaces. In Proc. UIST '13, 13--22.

Digital Library

[46]

Toshiya Yui and Tomoko Hashida. 2016. Floatio: Floating Tangible User Interface Based on Animacy Perception. In Proc. UIST '16 Adjunct, 43--45.

Digital Library

[47]

Robert Kovacs, Anna Seufert, Ludwig Wall, Hsiang-Ting Chen, Florian Meinel, Willi Müller, Sijing You, Maximilian Brehm, Jonathan Striebel, Yannis Kommana, Alexander Popiak, Thomas Bläsius, and Patrick Baudisch. 2017. TrussFab: Fabricating Sturdy Large-Scale Structures on Desktop 3D Printers. In Proc. CHI '17, 2606--2616.

Digital Library

Cited By

Cauchard JGover WChen WCartwright SSharlin E(2024)Drones in Wonderland – Disentangling Collocated Interaction Using Radical FormIEEE Robotics and Automation Letters10.1109/LRA.2021.31036539:2(1636-1643)Online publication date: Feb-2024
https://doi.org/10.1109/LRA.2021.3103653
Lykov AKaraf SMartynov MSerpiva VFedoseev AKonenkov MTsetserukou D(2024)FlockGPT: Guiding UAV Flocking with Linguistic Orchestration2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00141(485-488)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00141
Chen YAlimohammadzadeh HGhandeharizadeh SCulbertson H(2024)Force-Feedback Through Touch-based Interactions With A Nanocopter2024 IEEE Haptics Symposium (HAPTICS)10.1109/HAPTICS59260.2024.10520851(271-277)Online publication date: 7-Apr-2024
https://doi.org/10.1109/HAPTICS59260.2024.10520851
Show More Cited By

Index Terms

GridDrones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface Deformations
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques

Recommendations

BitDrones: Towards Using 3D Nanocopter Displays as Interactive Self-Levitating Programmable Matter
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We present BitDrones, a toolbox for building interactive real reality 3D displays that use nano-quadcopters as self-levitating tangible building blocks. Our prototype is a first step towards interactive self-levitating programmable matter, in which the ...
GridDrones: A Self-Levitating Physical Voxel Lattice for 3D Surface Deformations
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

We present GridDrones, a self-levitating programmable matter platform that can be used for representing 2.5D 15 voxel grid relief maps with capabilities of rendering overhangs and 3D spatial transformations. GridDrones consists of 15 cube-shaped ...
BitDrones: Towards Levitating Programmable Matter Using Interactive 3D Quadcopter Displays
UIST '15 Adjunct: Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology

In this paper, we present BitDrones, a platform for the construction of interactive 3D displays that utilize nano quadcopters as self-levitating tangible building blocks. Our prototype is a first step towards supporting interactive mid-air, tangible ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology

October 2018

1016 pages

ISBN:9781450359481

DOI:10.1145/3242587

General Chairs:
Patrick Baudisch
Hasso-Plattner Institute, Germany
,
Albrecht Schmidt
LMU, Germany
,
Program Chair:
Andy Wilson
Microsoft Research, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '18

Sponsor:

UIST '18: The 31st Annual ACM Symposium on User Interface Software and Technology

October 14, 2018

Berlin, Germany

Acceptance Rates

UIST '18 Paper Acceptance Rate 80 of 375 submissions, 21%;

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
654
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cauchard JGover WChen WCartwright SSharlin E(2024)Drones in Wonderland – Disentangling Collocated Interaction Using Radical FormIEEE Robotics and Automation Letters10.1109/LRA.2021.31036539:2(1636-1643)Online publication date: Feb-2024
Lykov AKaraf SMartynov MSerpiva VFedoseev AKonenkov MTsetserukou D(2024)FlockGPT: Guiding UAV Flocking with Linguistic Orchestration2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00141(485-488)Online publication date: 21-Oct-2024
Chen YAlimohammadzadeh HGhandeharizadeh SCulbertson H(2024)Force-Feedback Through Touch-based Interactions With A Nanocopter2024 IEEE Haptics Symposium (HAPTICS)10.1109/HAPTICS59260.2024.10520851(271-277)Online publication date: 7-Apr-2024
Lingam SFranssen MPetermeijer SMartens M(2024)Challenges and Future Directions for Human-Drone Interaction Research: An Expert PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2400756(1-17)Online publication date: 12-Sep-2024
Gao LIrani PSubramanian SPrabhakar GMartinez Plasencia DHirayama R(2023)DataLev: Mid-air Data Physicalisation Using Acoustic LevitationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581016(1-14)Online publication date: 19-Apr-2023
Bae SZheng CWest MDo EHuron SSzafir D(2022)Making Data Tangible: A Cross-disciplinary Design Space for Data PhysicalizationProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501939(1-18)Online publication date: 29-Apr-2022
Herdel VYamin LCauchard J(2022)Above and Beyond: A Scoping Review of Domains and Applications for Human-Drone InteractionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501881(1-22)Online publication date: 29-Apr-2022
Fedoseev ABaza AGupta ADorzhieva EGujarathi RTsetserukou D(2022)DandelionTouch: High Fidelity Haptic Rendering of Soft Objects in VR by a Swarm of Drones2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53654.2022.9945390(1078-1083)Online publication date: 9-Oct-2022
Ullmer BShaer OMazalek AHummels C(2022)Weaving Fire into FormundefinedOnline publication date: 20-Jul-2022
Tsykunov EFedoseev ADorzhieva EAgishev RIbrahimov RVasquez DLabazanova LTsetserukou D(2021)DroneStick: Flying Joystick as a Novel Type of InterfaceSIGGRAPH Asia 2021 Emerging Technologies10.1145/3476122.3484845(1-2)Online publication date: 14-Dec-2021
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten