research-article

Flying LEGO Bricks: Observations of Children Constructing and Playing with Programmable Matter

Authors:

Julie Torpegaard,

Roel Vertegaal,

Timothy MerrittAuthors Info & Claims

TEI '20: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction

Pages 193 - 205

https://doi.org/10.1145/3374920.3374948

Published: 09 February 2020 Publication History

Abstract

In this paper, we present a case study that explores how children could learn to interact with programmable matter. Flying drone swarms enable physical visualizations of complex data and simulation of physical objects and processes, e.g., planetary movements. The swarms can be digitally controlled as an ensemble as a form of (sparse) "programmable matter." We worked with the toy company LEGO®, to design and evaluate a "build and fly" experience with 240 children in a public exhibition. The children decorated a bendable handheld controller with LEGO® bricks and then used this controller to animate the flight of a 10-drone swarm. Results indicate that children enjoyed the constructive play and performance aspects of the system. Four main patterns of player behavior emerged, which we discuss in relation to possible improvements to the system. We provide implications for design of programmable matter systems for supporting child play experiences.

Supplementary Material

MP4 File (teifp1140.mp4)

Supplemental video

Download
73.68 MB

References

[1]

Hal Abelson, Nat Goodman, and Lee Rudolph. 1974. LOGO Manual. Retrieved September 16, 2018 from http://dspace.mit.edu/handle/1721.1/6226

[2]

Dor Abrahamson and Arthur Bakker. 2016. Making sense of movement in embodied design for mathematics learning. Cognitive Research: Principles and Implications 1, 1: 33. https://doi.org/10.1186/s41235-016-0034--3

[3]

Edith Ackermann. 1996. Perspective-taking and object construction: Two keys to learning. In Constructionism in Practice: Designing, Thinking, and Learning in A Digital World (1 edition), Yasmin B. Kafai and Mitchel Resnick (eds.). Routledge, Mahwah, N.J, 39--50.

[4]

Edith Ackermann. 2001. Piaget's Constructivism, Papert's Constructionism: What's the difference? Future of learning group publication 5, 3: 438.

[5]

J. Alonso-Mora, A. Breitenmoser, M. Rufli, R. Siegwart, and P. Beardsley. 2011. Multi-robot system for artistic pattern formation. In 2011 IEEE International Conference on Robotics and Automation, 4512--4517. https://doi.org/10.1109/ICRA.2011.5980269

[6]

J. Alonso-Mora, S. Haegeli Lohaus, P. Leemann, R. Siegwart, and P. Beardsley. 2015. Gesture based human - Multi-robot swarm interaction and its application to an interactive display. In 2015 IEEE International Conference on Robotics and Automation (ICRA), 5948--5953. https://doi.org/10.1109/ICRA.2015.7140033

[7]

Alissa N. Antle. 2007. The CTI Framework: Informing the Design of Tangible Systems for Children. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07), 195--202. https://doi.org/10.1145/1226969.1227010

Digital Library

[8]

F. Augugliaro, S. Lupashin, M. Hamer, C. Male, M. Hehn, M. W. Mueller, J. S. Willmann, F. Gramazio, M. Kohler, and R. D'Andrea. 2014. The Flight Assembled Architecture installation: Cooperative construction with flying machines. IEEE Control Systems Magazine 34, 4: 46--64. https://doi.org/10.1109/MCS.2014.2320359

[9]

Jon Back, Elena Márquez Segura, and Annika Waern. 2017. Designing for Transformative Play. ACM Trans. Comput.-Hum. Interact. 24, 3: 18:1--18:28. https://doi.org/10.1145/3057921

Digital Library

[10]

Saskia Bakker, Elise van den Hoven, and Alissa N. Antle. 2011. MoSo Tangibles: Evaluating Embodied Learning. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11), 85--92. https://doi.org/10.1145/1935701.1935720

[11]

Sean Braley, Calvin Rubens, Timothy R. Merritt, and Roel Vertegaal. 2018. GridDrones: A Self-Levitating Physical Voxel Lattice for 3D Surface Deformations. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18), D200:1--D200:4. https://doi.org/10.1145/3170427.3186477

Digital Library

[12]

Sean Braley, Calvin Rubens, Timothy R. Merritt, and Roel Vertegaal. 2018. GridDrones: A Self-Levitating Physical Voxel Lattice for Interactive 3D Surface Deformations. In UIST '18 (UIST '18), D200:1--D200:4.

Digital Library

[13]

Andy Clark. 1996. Being There: Putting Brain, Body, and World Together Again. MIT Press, Cambridge, MA, USA.

[14]

Paul Dourish. 2004. Where the Action Is. MIT Press, Cambridge, Mass.

[15]

Hubert L. Dreyfus. 1991. Being-in-the-world: A Commentary on Heidegger's Being and Time, División I. MIT Press.

[16]

Kenneth P. Fishkin. 2004. A Taxonomy for and Analysis of Tangible Interfaces. Personal Ubiquitous Comput. 8, 5: 347--358. https://doi.org/10.1007/s00779-004-0297--4

Digital Library

[17]

Kenneth P. Fishkin, Thomas P. Moran, and Beverly L. Harrison. 1999. Embodied User Interfaces: Towards Invisible User Interfaces. In Engineering for Human-Computer Interaction: IFIP TC2/TC13 WG2.7/WG13.4 Seventh Working Conference on Engineering for Human-Computer Interaction September 14--18, 1998, Heraklion, Crete, Greece, Stéphane Chatty and Prasun Dewan (eds.). Springer US, Boston, MA, 1--18. https://doi.org/10.1007/978-0--387--35349--4_1

Digital Library

[18]

George W. Fitzmaurice, Hiroshi Ishii, and William A. S. Buxton. 1995. Bricks: Laying the Foundations for Graspable User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), 442--449. https://doi.org/10.1145/223904.223964

Digital Library

[19]

B. Gabrich, D. Saldaña, V. Kumar, and M. Yim. 2018. A Flying Gripper Based on Cuboid Modular Robots. In 2018 IEEE International Conference on Robotics and Automation (ICRA), 7024--7030. https://doi.org/10.1109/ICRA.2018.8460682

[20]

S. C. Goldstein, J. D. Campbell, and T. C. Mowry. 2005. Programmable matter. Computer 38, 6: 99--101. https://doi.org/10.1109/MC.2005.198

Digital Library

[21]

Antonio Gomes, Calvin Rubens, Sean Braley, and Roel Vertegaal. 2016. BitDrones: Towards Using 3D Nanocopter Displays As Interactive Self-Levitating Programmable Matter. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 770--780. https://doi.org/10.1145/2858036.2858519

Digital Library

[22]

Jens Emil Grønbæk, Kasper Buhl Jakobsen, Marianne Graves Petersen, Majken Kirkegård Rasmussen, Jakob Winge, and Jeppe Stougaard. 2016. Designing for Children's Collective Music Making: How Spatial Orientation and Configuration Matter. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI '16), 60:1--60:10. https://doi.org/10.1145/2971485.2971552

Digital Library

[23]

David Holman and Roel Vertegaal. 2008. Organic user interfaces: Designing computers in any way, shape, or form. Communications of the ACM 51, 6: 48--55. https://doi.org/10.1145/1349026.1349037

Digital Library

[24]

Immanuel Kant, Translated by Ian Johnston, and Illustrated by Ian Crowe. 2009. Universal Natural History and Theory of the Heavens. Richer Resources Publications, Arlington, VA.

[25]

Alan C. Kay. 1996. History of Programming languages-II. In Thomas J. Bergin Jr. and Richard G. Gibson Jr. (eds.). ACM, New York, NY, USA, 511--598. https://doi.org/10.1145/234286.1057828

[26]

Eng Tat Khoo, Tim Merritt, and Adrian David Cheok. 2010. Designing a Mixed Reality Intergenerational Entertainment System. In The Engineering of Mixed Reality Systems, Emmanuel Dubois, Philip Gray and Laurence Nigay (eds.). Springer London, London, 121--141. https://doi.org/10.1007/978--1--84882--733--2_7

[27]

Sofie Kinch and Marie Koldkjær Højlund. 2013. Kidkit Guides Children into Alarming Atmospheres: Designing for Embodied Habituation in Hospital Wards. In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces (DPPI '13), 1--10. https://doi.org/10.1145/2513506.2513507

Digital Library

[28]

Timothy Koschmann, Kari Kuutti, and Larry Hickman. 1998. The Concept of Breakdown in Heidegger, Leont'ev, and Dewey and Its Implications for Education. Mind, Culture, and Activity 5, 1: 25--41. https://doi.org/10.1207/s15327884mca0501_3

[29]

Mathieu Le Goc, Lawrence H. Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Dragicevic, and Sean Follmer. 2016. Zooids: Building Blocks for Swarm User Interfaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16), 97--109. https://doi.org/10.1145/2984511.2984547

Digital Library

[30]

Jonas Löwgren. 2013. Annotated Portfolios and Other Forms of Intermediate-level Knowledge. Interactions 20, 1: 30--34. https://doi.org/10.1145/2405716.2405725

Digital Library

[31]

Paul Marshall. 2007. Do Tangible Interfaces Enhance Learning? In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07), 163--170. https://doi.org/10.1145/1226969.1227004

Digital Library

[32]

Catherine A. McMahon and Annie Bernier. 2017. Twenty years of research on parental mind-mindedness: Empirical findings, theoretical and methodological challenges, and new directions. Developmental Review 46: 54--80. https://doi.org/10.1016/j.dr.2017.07.001

[33]

Edward F. Melcer and Katherine Isbister. 2016. Bridging the Physical Divide: A Design Framework for Embodied Learning Games and Simulations. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 2225--2233. https://doi.org/10.1145/2851581.2892455

[34]

Maurice Merleau-Ponty. 2013. Phenomenology of perception (translated). Routledge.

[35]

G. Michelitsch, J. Williams, M. Osen, B. Jimenez, and S. Rapp. 2004. Haptic chameleon: a new concept of shape-changing user interface controls with force feedback. 1305. https://doi.org/10.1145/985921.986050

[36]

Jaime Montemayor, Allison Druin, Gene Chipman, Allison Farber, and Mona Leigh Guha. 2004. Tools for Children to Create Physical Interactive Storyrooms. Comput. Entertain. 2, 1: 12--12. https://doi.org/10.1145/973801.973821

Digital Library

[37]

Patricia Moya. 2014. Habit and embodiment in Merleau-Ponty. Frontiers in Human Neuroscience 8. https://doi.org/10.3389/fnhum.2014.00542

[38]

Florian Mueller and Katherine Isbister. 2014. Movement-based Game Guidelines. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14), 2191--2200. https://doi.org/10.1145/2556288.2557163

Digital Library

[39]

Hiroki Nozaki. 2014. Flying Display: A Movable Display Pairing Projector and Screen in the Air. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14), 909--914. https://doi.org/10.1145/2559206.2579410

Digital Library

[40]

Seymour Papert. 1980. Mindstorms: Children, Computers, and Powerful Ideas. Basic Books, Inc., New York, NY, USA.

Digital Library

[41]

Jean Piaget. 1954. The construction of reality in the child. Basic Books, New York, NY, US. https://doi.org/10.1037/11168-000

[42]

Glenda Revelle, Oren Zuckerman, Allison Druin, and Mark Bolas. 2005. Tangible User Interfaces for Children. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05), 2051--2052. https://doi.org/10.1145/1056808.1057095

[43]

M. Rubenstein, C. Ahler, and R. Nagpal. 2012. Kilobot: A low cost scalable robot system for collective behaviors. In 2012 IEEE International Conference on Robotics and Automation, 3293--3298. https://doi.org/10.1109/ICRA.2012.6224638

[44]

M. Rubenstein, A. Cornejo, and R. Nagpal. 2014. Programmable self-assembly in a thousand-robot swarm. Science 345, 6198: 795--799. https://doi.org/10.1126/science.1254295

[45]

Kimiko Ryokai, Stefan Marti, and Hiroshi Ishii. 2004. I/O Brush: Drawing with Everyday Objects As Ink. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04), 303--310. https://doi.org/10.1145/985692.985731

Digital Library

[46]

Ofir Sadka, Hadas Erel, Andrey Grishko, and Oren Zuckerman. 2018. Tangible Interaction in Parent-child Collaboration: Encouraging Awareness and Reflection. In Proceedings of the 17th ACM Conference on Interaction Design and Children (IDC '18), 157--169. https://doi.org/10.1145/3202185.3202746

Digital Library

[47]

Corina Sas, Steve Whittaker, Steven Dow, Jodi Forlizzi, John Zimmerman, Corina Sas, Steve Whittaker, Steven Dow, Jodi Forlizzi, and John Zimmerman. 2014. Generating implications for design through design research. 1971--1980. https://doi.org/10.1145/2556288.2557357

[48]

Jürgen Scheible, Achim Hoth, Julian Saal, and Haifeng Su. 2013. Displaydrone: A Flying Robot Based Interactive Display. In Proceedings of the 2Nd ACM International Symposium on Pervasive Displays (PerDis '13), 49--54. https://doi.org/10.1145/2491568.2491580

Digital Library

[49]

Stefan Schneegass, Florian Alt, Jürgen Scheible, Albrecht Schmidt, and Haifeng Su. 2014. Midair Displays: Exploring the Concept of Free-floating Public Displays. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14), 2035--2040. https://doi.org/10.1145/2559206.2581190

[50]

Lawrence Shapiro. 2010. Embodied Cognition. Taylor & Francis.

[51]

Alexander Skulmowski and Günter Daniel Rey. 2018. Embodied learning: introducing a taxonomy based on bodily engagement and task integration. Cognitive Research: Principles and Implications 3, 1: 6. https://doi.org/10.1186/s41235-018-0092--9

[52]

Skylar Tibbits. 2014. 4D Printing: Multi-Material Shape Change. Architectural Design 84, 1: 116--121. https://doi.org/10.1002/ad.1710

[53]

Tommaso Toffoli and Norman Margolus. 1991. Programmable matter: Concepts and realization. Physica D: Nonlinear Phenomena 47, 1: 263--272. https://doi.org/10.1016/0167--2789(91)90296-L

Digital Library

[54]

Deena Skolnick Weisberg, Kathy Hirsh-Pasek, Roberta Michnick Golinkoff, Audrey K. Kittredge, and David Klahr. 2016. Guided Play: Principles and Practices. Current Directions in Psychological Science 25, 3: 177--182. https://doi.org/10.1177/0963721416645512

[55]

Justin Werfel, Kirstin Petersen, and Radhika Nagpal. 2014. Designing Collective Behavior in a Termite-Inspired Robot Construction Team. Science 343, 6172: 754--758. https://doi.org/10.1126/science.1245842

[56]

Peta Wyeth. 2006. Ethnography in the Kindergarten: Examining Children's Play Experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 1225--1228. https://doi.org/10.1145/1124772.1124956

Digital Library

[57]

Drone 100 - Futurelab. Retrieved September 21, 2018 from https://ars.electronica.art/futurelab/en/project/drone-100/

Cited By

Dominiak JWalczak AStefanidi EAdamkiewicz KGrudzień KNiess JWoźniak P(2024)ProtoBricks: A Research Toolkit for Tangible Prototyping & Data PhysicalizationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661573(476-495)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661573
Hödl O(2024)An Electronic Engineering Approach for Turning a Lego Brick Piano into a Musical InstrumentIEEE Access10.1109/ACCESS.2024.338636112(51319-51329)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3386361
Stemasov EHohn JCordts MSchikorr ARukzio EGugenheimer J(2023)BrickStARt: Enabling In-situ Design and Tangible Exploration for Personal Fabrication using Mixed RealityProceedings of the ACM on Human-Computer Interaction10.1145/36264657:ISS(64-92)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626465
Show More Cited By

Index Terms

Flying LEGO Bricks: Observations of Children Constructing and Playing with Programmable Matter
1. Hardware
  1. Emerging technologies
    1. Emerging interfaces
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
  2. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing

Recommendations

BitDrones: Towards Using 3D Nanocopter Displays as Interactive Self-Levitating Programmable Matter
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

We present BitDrones, a toolbox for building interactive real reality 3D displays that use nano-quadcopters as self-levitating tangible building blocks. Our prototype is a first step towards interactive self-levitating programmable matter, in which the ...
Autonomous Flying Robots: Unmanned Aerial Vehicles and Micro Aerial Vehicles
Exploiting Augmented Reality in LEGO Therapy for Children with Autism Spectrum Disorder
Virtual Reality and Mixed Reality
Abstract
Numerous computer-based therapies have been designed for cognitive-behavioral interventions to support children with Autism Spectrum Disorder (ASD) in recent years. Among these technologies, Augmented Reality (AR) offers unique educational ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

TEI '20: Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction

February 2020

978 pages

ISBN:9781450361071

DOI:10.1145/3374920

General Chairs:
Elise van den Hoven
University of Technology Sydney (Australia) & Eindhoven University of Technology (the Netherlands)
,
Lian Loke
University of Sydney (Australia)
,
Program Chairs:
Orit Shaer
Wellesley College (USA)
,
Jelle van Dijk
University of Twente (Netherlands)
,
Andrew Kun
University of New Hampshire (USA)

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 February 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

TEI '20

Sponsor:

SIGCHI

TEI '20: Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction

February 9 - 12, 2020

Sydney NSW, Australia

Acceptance Rates

TEI '20 Paper Acceptance Rate 37 of 132 submissions, 28%;

Overall Acceptance Rate 393 of 1,367 submissions, 29%

Upcoming Conference

TEI '25

Sponsor:
sigchi

Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction

March 4 - 7, 2025

Bordeaux / Talence , France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
602
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)8

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dominiak JWalczak AStefanidi EAdamkiewicz KGrudzień KNiess JWoźniak P(2024)ProtoBricks: A Research Toolkit for Tangible Prototyping & Data PhysicalizationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661573(476-495)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661573
Hödl O(2024)An Electronic Engineering Approach for Turning a Lego Brick Piano into a Musical InstrumentIEEE Access10.1109/ACCESS.2024.338636112(51319-51329)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3386361
Stemasov EHohn JCordts MSchikorr ARukzio EGugenheimer J(2023)BrickStARt: Enabling In-situ Design and Tangible Exploration for Personal Fabrication using Mixed RealityProceedings of the ACM on Human-Computer Interaction10.1145/36264657:ISS(64-92)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3626465
Gamboa MLjungblad SJohal WMubin OObaid M(2023)Championing Design Knowledge in Human-Drone Interaction ResearchProceedings of the 11th International Conference on Human-Agent Interaction10.1145/3623809.3623927(365-368)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3623809.3623927
Camara Olim SNisi VRubegni E(2023)“Periodic Fable Discovery” Using Tangible Interactions and Augmented Reality to Promote STEM SubjectsProceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3569009.3572804(1-15)Online publication date: 26-Feb-2023
https://dl.acm.org/doi/10.1145/3569009.3572804
Gamboa M(2022)Living with Drones, Robots, and Young Children: Informing Research through Design with AutoethnographyNordic Human-Computer Interaction Conference10.1145/3546155.3546658(1-14)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546658
Herdel VYamin LCauchard J(2022)Above and Beyond: A Scoping Review of Domains and Applications for Human-Drone InteractionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501881(1-22)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501881
Li YLiang MPreissing JBachl NDutoit MWeber TMayer SHussmann H(2022)A Meta-Analysis of Tangible Learning Studies from the TEI ConferenceProceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3490149.3501313(1-17)Online publication date: 13-Feb-2022
https://dl.acm.org/doi/10.1145/3490149.3501313
Bispo LNolasco APereira AUcella-Filho JDelatorre FCupertino GGomes Rde Souza EBrito JDias Júnior A(2022)Design and Production Process of Toy Prototypes Using Urban Forestry WasteToys and Sustainability10.1007/978-981-16-9673-2_1(1-18)Online publication date: 18-Feb-2022
https://doi.org/10.1007/978-981-16-9673-2_1
Pruszko LLaurillau YPiranda BBourgeois JCoutrix C(2021)Impact of the Size of Modules on Target Acquisition and Pursuit for Future Modular Shape-changing Physical User InterfacesProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479936(297-307)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479936
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten