skip to main content
10.1145/3173574.3174089acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Project Zanzibar: A Portable and Flexible Tangible Interaction Platform

Published: 21 April 2018 Publication History

Abstract

We present Project Zanzibar: a flexible mat that can locate, uniquely identify and communicate with tangible objects placed on its surface, as well as sense a user's touch and hover hand gestures. We describe the underlying technical contributions: efficient and localised Near Field Communication (NFC) over a large surface area; object tracking combining NFC signal strength and capacitive footprint detection, and manufacturing techniques for a rollable device form-factor that enables portability, while providing a sizable interaction area when unrolled. In addition, we detail design patterns for tangibles of varying complexity and interactive capabilities, including the ability to sense orientation on the mat, harvest power, provide additional input and output, stack, or extend sensing outside the bounds of the mat. Capabilities and interaction modalities are illustrated with self-generated applications. Finally, we report on the experience of professional game developers building novel physical/digital experiences using the platform.

References

[1]
Alissa N. Antle. 2007. The CTI Framework: Informing the Design of Tangible Systems for Children. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07). ACM, New York, NY, USA, 195--202.
[2]
Ayah Bdeir. 2009. Electronics As Material: LittleBits. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI '09). ACM, New York, NY, USA, 397--400.
[3]
Andrea Bianchi and Ian Oakley. 2013. Designing Tangible Magnetic Appcessories. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (TEI '13). ACM, New York, NY, USA, 255--258.
[4]
Scott Brave, Hiroshi Ishii, and Andrew Dahley. 1998. Tangible Interfaces for Remote Collaboration and Communication. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work (CSCW '98). ACM, New York, NY, USA, 169--178.
[5]
Michael Buettner, Richa Prasad, Alanson Sample, Daniel Yeager, Ben Greenstein, Joshua R. Smith, and David Wetherall. 2008. RFID Sensor Networks with the Intel WISP. In Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems (SenSys '08). ACM, New York, NY, USA, 393--394.
[6]
Paul H. Dietz and Benjamin D. Eidelson. 2009. SurfaceWare: Dynamic Tagging for Microsoft Surface. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI '09). ACM, New York, NY, USA, 249--254.
[7]
Paul Dourish. 2004. Where the action is: the foundations of embodied interaction. MIT press.
[8]
George W. Fitzmaurice, Hiroshi Ishii, and William A. S. Buxton. 1995. Bricks: Laying the Foundations for Graspable User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 442--449.
[9]
Audrey Girouard, Erin Treacy Solovey, Leanne M. Hirshfield, Stacey Ecott, Orit Shaer, and Robert J. K. Jacob. 2007. Smart Blocks: A Tangible Mathematical Manipulative. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07). ACM, New York, NY, USA, 183--186.
[10]
Tobias Grosse-Puppendahl, Christian Holz, Gabe Cohn, Raphael Wimmer, Oskar Bechtold, Steve Hodges, Matthew S. Reynolds, and Joshua R. Smith. 2017. Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3293--3315.
[11]
Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O'Hara, Gavin Smyth, and William Buxton. 2016. Pre-Touch Sensing for Mobile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2869--2881.
[12]
Christian Holz and Patrick Baudisch. 2013. Fiberio: A Touchscreen That Senses Fingerprints. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, New York, NY, USA, 41--50.
[13]
Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3011--3014.
[14]
Christian Holz and Marius Knaust. 2015. Biometric Touch Sensing: Seamlessly Augmenting Each Touch with Continuous Authentication. In Proceedings of the 28th Annual ACM Symposium on User Interface Software&Technology (UIST '15). ACM, New York, NY, USA, 303--312.
[15]
Michael S. Horn, Erin Treacy Solovey, and Robert J. K. Jacob. 2008. Tangible Programming and Informal Science Learning: Making TUIs Work for Museums. In Proceedings of the 7th International Conference on Interaction Design and Children (IDC '08). ACM, New York, NY, USA, 194--201.
[16]
Hiroshi Ishii. 2008. The Tangible User Interface and Its Evolution. Commun. ACM 51, 6 (June 2008), 32--36.
[17]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, New York, NY, USA, 234--241.
[18]
Sergi Jordà, Günter Geiger, Marcos Alonso, and Martin Kaltenbrunner. 2007. The reacTable: Exploring the Synergy Between Live Music Performance and Tabletop Tangible Interfaces. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07). ACM, New York, NY, USA, 139--146.
[19]
Lego. 2017. Lego Dimensions. Website. (18 Sep 2017). Retrieved September 17, 2017 from https://www.lego.com/en-us/dimensions/products.
[20]
Hanchuan Li, Can Ye, and Alanson P. Sample. 2015. IDSense: A Human Object Interaction Detection System Based on Passive UHF RFID. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2555--2564.
[21]
Mattel. 2017. Mattel Brings Powerhouse Brands to the iPad With Apptivity Play. Website. (11 Sep 2017). Retrieved September 11, 2017 from http://news.mattel.com/news/mattel-brings-powerhouse -brands-to-the-ipad-with-apptivityTM-play.
[22]
David Merrill, Jeevan Kalanithi, and Pattie Maes. 2007. Siftables: towards sensor network user interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction. ACM, 75--78.
[23]
Hendrik Müller, Jennifer Gove, and John Webb. 2012. Understanding Tablet Use: A Multi-method Exploration. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, New York, NY, USA, 1--10.
[24]
Nintendo. 2017. Nintendo Amiibo. Website. (18 Sep 2017). Retrieved September 17, 2017 from http://www.nintendo.com/amiibo.
[25]
Uwe Oestermeier, Philipp Mock, Jörg Edelmann, and Peter Gerjets. 2015. LEGO Music: Learning Composition with Bricks. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15). ACM, New York, NY, USA, 283--286.
[26]
Alex Olwal and Andrew D. Wilson. 2008. SurfaceFusion: Unobtrusive Tracking of Everyday Objects in Tangible User Interfaces. In Proceedings of Graphics Interface 2008 (GI '08). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 235--242. http://dl.acm.org/citation.cfm?id=1375714.1375754
[27]
Osmo. 2017. Osmo. Website. (11 Sep 2017). Retrieved September 11, 2017 from https://www.playosmo.com.
[28]
Seymour Papert. 1980. Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc.
[29]
James Patten, Hiroshi Ishii, Jim Hines, and Gian Pangaro. 2001. Sensetable: A Wireless Object Tracking Platform for Tangible User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 253--260.
[30]
Clément Pillias, Raphaël Robert-Bouchard, and Guillaume Levieux. 2014. Designing Tangible Video Games: Lessons Learned from the Sifteo Cubes. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3163--3166.
[31]
Ben Piper, Carlo Ratti, and Hiroshi Ishii. 2002. Illuminating Clay: A 3-D Tangible Interface for Landscape Analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, New York, NY, USA, 355--362.
[32]
S Price, Y Rogers, M Scaife, D Stanton, and H Neale. 2003. Using ?tangibles' to promote novel forms of playful learning. Interacting with Computers 15, 2 (2003), 169--185.
[33]
Jun Rekimoto. 2002. SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, New York, NY, USA, 113--120.
[34]
Jun Rekimoto and Masanori Saitoh. 1999. Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 378--385.
[35]
Jun Rekimoto, Brygg Ullmer, and Haruo Oba. 2001. DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 269--276.
[36]
Eric Schweikardt and Mark D. Gross. 2006. roBlocks: A Robotic Construction Kit for Mathematics and Science Education. In Proceedings of the 8th International Conference on Multimodal Interfaces (ICMI '06). ACM, New York, NY, USA, 72--75.
[37]
Arnan Sipitakiat and Nusarin Nusen. 2012. Robo-Blocks: Designing Debugging Abilities in a Tangible Programming System for Early Primary School Children. In Proceedings of the 11th International Conference on Interaction Design and Children (IDC '12). ACM, New York, NY, USA, 98--105.
[38]
Skylanders. 2017. Skylanders Video Game. Website. (18 Sep 2017). Retrieved September 17, 2017 from https://www.skylanders.com/.
[39]
Andrew Spielberg, Alanson Sample, Scott E. Hudson, Jennifer Mankoff, and James McCann. 2016. RapID: A Framework for Fabricating Low-Latency Interactive Objects with RFID Tags. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5897--5908.
[40]
Hideyuki Suzuki and Hiroshi Kato. 1995. Interaction-level Support for Collaborative Learning: AlgoBlock&Mdash;an Open Programming Language. In The First International Conference on Computer Support for Collaborative Learning (CSCL '95). L. Erlbaum Associates Inc., Hillsdale, NJ, USA, 349--355.
[41]
Paula Te. 2015. TADCAD: A Tangible and Gestural 3D Modeling&Printing Platform for Building Creativity. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15). ACM, New York, NY, USA, 406--409.
[42]
John Underkoffler and Hiroshi Ishii. 1999. Urp: A Luminous-tangible Workbench for Urban Planning and Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 386--393.
[43]
Simon Voelker, Christian Cherek, Jan Thar, Thorsten Karrer, Christian Thoresen, Kjell Ivar Øvergård, and Jan Borchers. 2015. PERCs: Persistently Trackable Tangibles on Capacitive Multi-Touch Displays. In Proceedings of the 28th Annual ACM Symposium on User Interface Software&Technology (UIST '15). ACM, New York, NY, USA, 351--356.
[44]
Danli Wang, Cheng Zhang, and Hongan Wang. 2011. T-Maze: A Tangible Programming Tool for Children. In Proceedings of the 10th International Conference on Interaction Design and Children (IDC '11). ACM, New York, NY, USA, 127--135.
[45]
Roy Want, Kenneth P. Fishkin, Anuj Gujar, and Beverly L. Harrison. 1999. Bridging Physical and Virtual Worlds with Electronic Tags. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 370--377.
[46]
Malte Weiss, Julie Wagner, Yvonne Jansen, Roger Jennings, Ramsin Khoshabeh, James D. Hollan, and Jan Borchers. 2009. SLAP Widgets: Bridging the Gap Between Virtual and Physical Controls on Tabletops. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 481--490.
[47]
Pierre Wellner. 1991. The DigitalDesk Calculator: Tangible Manipulation on a Desk Top Display. In Proceedings of the 4th Annual ACM Symposium on User Interface Software and Technology (UIST '91). ACM, New York, NY, USA, 27--33.
[48]
Andrew D. Wilson. 2005. PlayAnywhere: A Compact Interactive Tabletop Projection-vision System. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST '05). ACM, New York, NY, USA, 83--92.
[49]
Neng-Hao Yu, Li-Wei Chan, Seng Yong Lau, Sung-Sheng Tsai, I-Chun Hsiao, Dian-Je Tsai, Fang-I Hsiao, Lung-Pan Cheng, Mike Chen, Polly Huang, and Yi-Ping Hung. 2011. TUIC: Enabling Tangible Interaction on Capacitive Multi-touch Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2995--3004.
[50]
Zowie. 2017. Zowie Playsets. Website. (19 Sep 2017). Retrieved September 19, 2017 from http://www.piernot.com/proj/zowie/.
[51]
Oren Zuckerman, Saeed Arida, and Mitchel Resnick. 2005. Extending Tangible Interfaces for Education: Digital Montessori-inspired Manipulatives. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05). ACM, New York, NY, USA, 859--868.

Cited By

View all
  • (2024)A Narrative Review of the Sociotechnical Landscape and Potential of Computer-Assisted Dynamic Assessment for Children with Communication Support NeedsMultimodal Technologies and Interaction10.3390/mti80500388:5(38)Online publication date: 7-May-2024
  • (2024)Democratizing Intelligent Soft WearablesAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686707(1-3)Online publication date: 13-Oct-2024
  • (2024)Game Jam with CARDinality: A Case Study of Exploring Play-based Interactive ApplicationsAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686347(1-3)Online publication date: 13-Oct-2024
  • Show More Cited By

Index Terms

  1. Project Zanzibar: A Portable and Flexible Tangible Interaction Platform

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
    April 2018
    8489 pages
    ISBN:9781450356206
    DOI:10.1145/3173574
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 April 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    • Best Paper

    Author Tags

    1. capacitive sensing
    2. flexible devices
    3. games
    4. near field communication
    5. play
    6. tangible user interfaces

    Qualifiers

    • Research-article

    Conference

    CHI '18
    Sponsor:

    Acceptance Rates

    CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)148
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)A Narrative Review of the Sociotechnical Landscape and Potential of Computer-Assisted Dynamic Assessment for Children with Communication Support NeedsMultimodal Technologies and Interaction10.3390/mti80500388:5(38)Online publication date: 7-May-2024
    • (2024)Democratizing Intelligent Soft WearablesAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686707(1-3)Online publication date: 13-Oct-2024
    • (2024)Game Jam with CARDinality: A Case Study of Exploring Play-based Interactive ApplicationsAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686347(1-3)Online publication date: 13-Oct-2024
    • (2024)CARDinality: Interactive Card-shaped Robots with Locomotion and Haptics using VibrationProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676421(1-14)Online publication date: 13-Oct-2024
    • (2024)ProtoBricks: A Research Toolkit for Tangible Prototyping & Data PhysicalizationProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661573(476-495)Online publication date: 1-Jul-2024
    • (2024)ViObjectProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435478:1(1-26)Online publication date: 6-Mar-2024
    • (2024)WonderMap: Capturing and Connecting the Edges of CulturesProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3635865(1-4)Online publication date: 11-Feb-2024
    • (2024)Act2Auth – A Novel Authentication Concept based on Embedded Tangible Interaction at DesksProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633360(1-15)Online publication date: 11-Feb-2024
    • (2024)Screenless Interactive Tabletop Gaming with Capacitive Surface SensingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642654(1-10)Online publication date: 11-May-2024
    • (2024)Wearable smart textiles—recent development and applicationsSmart Textiles from Natural Resources10.1016/B978-0-443-15471-3.00003-0(395-442)Online publication date: 2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media