skip to main content
10.1145/3242587.3242658acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

GridDrones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface Deformations

Published: 11 October 2018 Publication History

Abstract

We present GridDrones, a self-levitating programmable matter platform that can be used for representing 2.5D voxel grid relief maps capable of rendering unsupported structures and 3D transformations. GridDrones consists of cube-shaped nanocopters that can be placed in a volumetric 1xnxn mid-air grid, which is demonstrated here with 15 voxels. The number of voxels and scale is only limited by the size of the room and budget. Grid deformations can be applied interactively to this voxel lattice by manually selecting a set of voxels, then assigning a continuous topological relationship between voxel sets that determines how voxels move in relation to each other and manually drawing out selected voxels from the lattice structure. Using this simple technique, it is possible to create unsupported structures that can be translated and oriented freely in 3D. Shape transformations can also be recorded to allow for simple physical shape morphing animations. This work extends previous work on selection and editing techniques for 3D user interfaces.

Supplementary Material

suppl.mov (ufp1482.m4v)
Supplemental video
suppl.mov (ufp1482p.mp4)
Supplemental video
MP4 File (p87-braley.mp4)

References

[1]
Harshit Agrawal, Udayan Umapathi, Robert Kovacs, Johannes Frohnhofen, Hsiang-Ting Chen, Stefanie Mueller, and Patrick Baudisch. 2015. Protopiper: Physically Sketching Room-Sized Objects at Actual Scale. In Proc. UIST '15, 427--436.
[2]
Javier Alonso-Mora, Andreas Breitenmoser, Martin Rufli, Roland Siegwart, and Paul Beardsley. 2011. Multi-Robot System for Artistic Pattern Formation. ICRA'11, 4512--4517.
[3]
Javier Alonso-Mora, Stefan Lohaus, Phillip Leemann, Roland Siegwart and Paul Beardsley. 2015. Gesture Based Human-Multi-Robot Swarm Interaction and its Application to an Interactive Display. ICRA'15, 5948--5953.
[4]
Raffaello D'Andrea. 2013. Humans and the coming machine revolution. In Proc. UIST'13, 1--2.
[5]
Ars Electronica Futurelab. 2016. Drone 100. http://tinyurl.com/drone100. Retrieved April 2016.
[6]
Matthew Blackshaw, Anthony DeVincenzi, David Lakatos, Daniel Leithinger, and Hiroshi Ishii. 2011. Recompose: Direct and Gestural Interaction with an Actuated Surface. In Proc. CHI EA '11, 1237--1242.
[7]
Sean Braley, Calvin Rubens, Timothy R. Merritt, and Roel Vertegaal. 2018. GridDrones: A Self-Levitating Physical Voxel Lattice for 3D Surface Deformations. In Ext. Abstracts CHI'18. Paper D200, 4 pages.
[8]
Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces. Personal Ubiquitous Computing. 8, 5, 347--358.
[9]
Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: Dynamic Physical Affordances and Constraints Through Shape and Object Actuation. In Proc. UIST'13, 417--426.
[10]
Sean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, and Hiroshi Ishii. 2012. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In Proc. UIST'12, 519--528.
[11]
Antonio Gomes, Calvin Rubens, Sean Braley, and Roel Vertegaal. 2016. BitDrones: Towards Using 3D Nanocopter Displays as Interactive Self-Levitating Programmable Matter. In Proc. CHI '16, 770--780.
[12]
Seth C. Goldstein, and Todd C. Mowry. 2004. Claytronics: A Scalable Basis for Future Robots. In RoboSphere'04.
[13]
Seth C. Goldstein, Jason D. Campbell, and Todd C. Mowry. Programmable Matter. 2005. IEEE Computer, 38, 6, 99--101.
[14]
Mathieu Le Goc, Lawrence H. Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Dragicevic, and Sean Follmer. 2016. Zooids: Building Blocks for Swarm User Interfaces. In Proc. UIST'16, 97--109.
[15]
Tovi Grossman and Ravin Balakrishnan. 2006. The design and evaluation of selection techniques for 3D volumetric displays. In Proc UIST'06. ACM, New York, NY, USA, 3--12.
[16]
Hooman Hedayati, Michael Walker, and Daniel Szafir. 2018. Improving Collocated Robot Teleoperation with Augmented Reality. In Proc HRI'18, 78--86.
[17]
Jacques Heyman. 1966. The stone skeleton. International Journal of Solids and Structures 2, 2: 249--279.
[18]
Hiroshi Ishii, and Brygg Ullmer. 1997. Tangible Bits: Beyond Pixels. In Proc. CHI'97, 234--241.
[19]
Hiroshi Ishii, Dávid Lakatos, Leonardo Bonanni, and Jean-Baptiste Labrune. 2012. Radical atoms: Beyond tangible bits, toward transformable materials. Interactions 19, 1, 38--51.
[20]
Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2013. Evaluating the efficiency of physical visualizations. In Proc. CHI'13, 2593--2602.
[21]
Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper Hornbæk. 2015. Opportunities and Challenges for Data Physicalization. In Proc. CHI '15, 3227--3236.
[22]
Shunichi Kasahara, Ryuma Niiyama, Valentin Heun, and Hiroshi Ishii. 2013. exTouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In Proc TEI'13, 223--228.
[23]
Brian Kirby, Jason Campbell, Burak Aksak, Padmanabhan Pillai, James Hoburg, Todd C. Mowry and Seth C. Goldstein. 2005. Catoms: Moving robots without moving parts. AAAI Robotics Exhibition'05, 1730--1731.
[24]
Haruhisa Kurokawa, Kohji Tomita, Akiya Kamimura, Shigeru Kokaji, Takashi Hasuo, and Satoshi Murata. 2008. Distributed Self-Reconfiguration of M-TRAN III Modular Robotic System. Int. J. Rob. Res. 27, 373--386.
[25]
Haruhisa Kurokawa, Satoshi Murata, Eiji Yoshida, Kohji Tomita, and Shigeru Kokaji. 1998. A 3-D Self-Reconfigurable Structure and Experiments," in Intelligent Robots and Systems, 860--865.
[26]
Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. ZeroN: Mid-air tangible interaction enabled by computer controlled magnetic levitation. In Proc. UIST'11, 327--336.
[27]
Daniel Leithinger and Hiroshi Ishii. 2010. Relief: A Scalable Actuated Shape Display. In Proc TEI'10, 221--222.
[28]
Henry Packard Moreton. 1992. Minimum Curvature Variation Curves, Networks, and Surfaces for Fair Free-Form Shape Design. Ph.D. Dissertation. University of California at Berkeley, Berkeley, CA, USA.
[29]
Javier Alonso-Mora, Andreas Breitenmoser, Martin Rufli, Roland Siegwart, and Paul Beardsley. 2012. Image and animation display with multiple mobile robots. Int. J. Rob. Res. 31, 6, 753--773.
[30]
Hiroki Nozaki. 2014. Flying display: A movable display pairing projector and screen in the air. In Proc. CHI EA'14, 909--914.
[31]
Yoichi Ochiai, Takayuki Hoshi and Jun Rekimoto. 2014. Pixie dust: Graphics generated by levitated and animated objects in computational acoustic-potential field. ACM Trans. Graph. 33, 4, 85.
[32]
Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto, and Yasufumi Yamaji. 2004. Lumen: Interactive visual and shape display for calm computing. In Proc SIGGRAPH '04, 17.
[33]
Gang Ren and Eamonn O'Neill. 2013. 3D selection with freehand gesture. Computers & Graphics 37, 3: 101--120.
[34]
John W. Romanishin, Kyle Gilpin and Daniela Rus. 2013. M-Blocks: Momentum-driven, Magnetic Modular Robots. In Intelligent Robots and Systems'13, 4288--4295
[35]
Michael Rubenstein, Alejandro Cornejo and Radhika Nagpal. 2014. Programmable self-assembly in a thousand-robot swarm. Science 345(6198), 795--799.
[36]
Michael Rubenstein, Christian Ahler and Radhika Nagpal. 2012. Kilobot: A low cost scalable robot system for collective behaviors. ICRA'12, 3293--98.
[37]
Juergen Scheible, Achim Hoth, Julian Saal and Haifeng Su. 2013. Displaydrone: A flying robot based interactive display. In Proc. PerDis'13, 49--54.
[38]
Philipp Schoessler, Daniel Windham, Daniel Leithinger, Sean Follmer, and Hiroshi Ishii. 2015. Kinetic Blocks: Actuated Constructive Assembly for Interaction and Display. In Proc. UIST '15, 341--349.
[39]
Stefan Schneegass, Florian Alt, Jürgen Scheible, Albrecht Schmidt, and Haifeng Su. 2014. Midair displays: Exploring the concept of free-floating public displays. In Proc. CHI EA'14, 2035--2040.
[40]
Alexa F. Siu, Eric J. Gonzalez, Shenli Yuan, Jason B. Ginsberg, and Sean Follmer. 2018. shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction. In Proc CHI'18. Paper 291, 13 pages.
[41]
Ivan Sutherland. 1965. The Ultimate Display. In Proc. IFIP 65, 2, 506--508.
[42]
Tomasso Toffoli and Norman Margolus. 1991. Programmable matter: Concepts and realization. Physica D: Nonlinear Phenomena 47, 1, 263--272.
[43]
Roel Vertegaal and Ivan Poupyrev. 2008. Organic User Interfaces. Communications of the ACM 51, 6, 26--30
[44]
Justin Werfel, Kirstin Petersen, and Radhika Nagpal. 2014. Designing collective behavior in a termite-inspired robot construction team. Science 343, 6172, 754--758.
[45]
Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva, and Hiroshi Ishii. 2013. PneUI: pneumatically actuated soft composite materials for shape changing interfaces. In Proc. UIST '13, 13--22.
[46]
Toshiya Yui and Tomoko Hashida. 2016. Floatio: Floating Tangible User Interface Based on Animacy Perception. In Proc. UIST '16 Adjunct, 43--45.
[47]
Robert Kovacs, Anna Seufert, Ludwig Wall, Hsiang-Ting Chen, Florian Meinel, Willi Müller, Sijing You, Maximilian Brehm, Jonathan Striebel, Yannis Kommana, Alexander Popiak, Thomas Bläsius, and Patrick Baudisch. 2017. TrussFab: Fabricating Sturdy Large-Scale Structures on Desktop 3D Printers. In Proc. CHI '17, 2606--2616.

Cited By

View all
  • (2024)Drones in Wonderland – Disentangling Collocated Interaction Using Radical FormIEEE Robotics and Automation Letters10.1109/LRA.2021.31036539:2(1636-1643)Online publication date: Feb-2024
  • (2024)FlockGPT: Guiding UAV Flocking with Linguistic Orchestration2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00141(485-488)Online publication date: 21-Oct-2024
  • (2024)Force-Feedback Through Touch-based Interactions With A Nanocopter2024 IEEE Haptics Symposium (HAPTICS)10.1109/HAPTICS59260.2024.10520851(271-277)Online publication date: 7-Apr-2024
  • Show More Cited By

Index Terms

  1. GridDrones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface Deformations

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
    October 2018
    1016 pages
    ISBN:9781450359481
    DOI:10.1145/3242587
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 October 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. claytronics
    2. organic user interfaces
    3. programmable matter
    4. radical atoms
    5. swarm user interfaces

    Qualifiers

    • Research-article

    Conference

    UIST '18

    Acceptance Rates

    UIST '18 Paper Acceptance Rate 80 of 375 submissions, 21%;
    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)49
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 25 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Drones in Wonderland – Disentangling Collocated Interaction Using Radical FormIEEE Robotics and Automation Letters10.1109/LRA.2021.31036539:2(1636-1643)Online publication date: Feb-2024
    • (2024)FlockGPT: Guiding UAV Flocking with Linguistic Orchestration2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00141(485-488)Online publication date: 21-Oct-2024
    • (2024)Force-Feedback Through Touch-based Interactions With A Nanocopter2024 IEEE Haptics Symposium (HAPTICS)10.1109/HAPTICS59260.2024.10520851(271-277)Online publication date: 7-Apr-2024
    • (2024)Challenges and Future Directions for Human-Drone Interaction Research: An Expert PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2400756(1-17)Online publication date: 12-Sep-2024
    • (2023)DataLev: Mid-air Data Physicalisation Using Acoustic LevitationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581016(1-14)Online publication date: 19-Apr-2023
    • (2022)Making Data Tangible: A Cross-disciplinary Design Space for Data PhysicalizationProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501939(1-18)Online publication date: 29-Apr-2022
    • (2022)Above and Beyond: A Scoping Review of Domains and Applications for Human-Drone InteractionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501881(1-22)Online publication date: 29-Apr-2022
    • (2022)DandelionTouch: High Fidelity Haptic Rendering of Soft Objects in VR by a Swarm of Drones2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53654.2022.9945390(1078-1083)Online publication date: 9-Oct-2022
    • (2022)Weaving Fire into FormundefinedOnline publication date: 20-Jul-2022
    • (2021)DroneStick: Flying Joystick as a Novel Type of InterfaceSIGGRAPH Asia 2021 Emerging Technologies10.1145/3476122.3484845(1-2)Online publication date: 14-Dec-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media