skip to main content
10.1145/3411763.3451586acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
poster

Towards On-the-wall Tangible Interaction: Using Walls as Interactive, Dynamic, and Responsive User Interface

Published: 08 May 2021 Publication History

Abstract

This paper presents our vision of on-the-wall tangible interaction. We envision a future where tangible interaction can be extended from conventional horizontal surfaces to vertical surfaces; indoor vertical areas such as walls, windows, and ceilings can be used for dynamic and direct physical manipulation. We first discuss the unique properties that vertical surfaces may offer for tangible interaction and the interaction scenarios they imbue. We then propose two potential paths for realizing on-the-wall interaction and the technical challenges we face. We follow with one prototype called Climbot. We showcase how Climbot can be used as an on-the-wall tangible user interface for dynamic lighting and as a wall switch controller. We conclude with a discussion of future work.

References

[1]
Thomas Augsten, Konstantin Kaefer, René Meusel, Caroline Fetzer, Dorian Kanitz, Thomas Stoff, Torsten Becker, Christian Holz, and Patrick Baudisch. 2010. Multitoe: high-precision interaction with back-projected floors based on high-resolution multi-touch input. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (UIST '10). Association for Computing Machinery, New York, NY, USA, 209–218. /10.1145/1866029.1866064
[2]
Wanghui Bu, Weiqi Zhou, Laixin Fang, Jing Chen, Xianghua An, Jingkai Huang. (2019, August). A Novel Cable-Driven Parallel Robot for Inner Wall Cleaning of the Large Storage Tank. In International Conference on Mechanical Design (pp. 28-40). Springer, Singapore.
[3]
Artem Dementyev, Hsin-Liu (Cindy) Kao, Inrak Choi, Deborah Ajilo, Maggie Xu, Joseph A. Paradiso, Chris Schmandt, and Sean Follmer. 2016. Rovables: Miniature On-Body Robots as Mobile Wearables. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST'16). Association for Computing Machinery, New York, NY, USA, 111–120.
[4]
Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. InFORM: dynamic physical affordances and constraints through shape and object actuation. In Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST '13). Association for Computing Machinery, New York, NY, USA, 417–426.
[5]
Mark D. Gross and Keith Evan Green. 2012. Architectural robotics, inevitably. interactions 19, 1 (January + February 2012), 28–33.
[6]
François Guimbretière, Maureen Stone, and Terry Winograd. 2001. Fluid interaction with high-resolution wall-size displays. In Proceedings of the 14th annual ACM symposium on User interface software and technology UIST '01). Association for Computing Machinery, New York, NY, USA, 21–30.
[7]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (CHI '97). Association for Computing Machinery, New York, NY, USA, 234–241.
[8]
Hiroshi Ishii, Daniel Leithinger, Sean Follmer, Amit Zoran, Philipp Schoessler, and Jared Counts. 2015. TRANSFORM: Embodiment of "Radical Atoms" at Milano Design Week. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA'15). Association for Computing Machinery, New York, NY, USA, 687–694.
[9]
Hiroshi Ishii, Dávid Lakatos, Leonardo Bonanni, and Jean-Baptiste Labrune. 2012. Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19, 1 (January + February 2012), 38–51.
[10]
Sergi Jordà. 2010. The reactable: tangible and tabletop music performance. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA'10). Association for Computing Machinery, New York, NY, USA, 2989–2994.
[11]
Gierad Laput, Yang Zhang, and Chris Harrison. 2017. Synthetic Sensors: Towards General-Purpose Sensing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 3986–3999.
[12]
Mathieu Le Goc, Lawrence H. Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Dragicevic, and Sean Follmer. 2016. Zooids: Building Blocks for Swarm User Interfaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). Association for Computing Machinery, New York, NY, USA, 97–109.
[13]
Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang, De-Nian Yang, and Bing-Yu Chen. 2014. GaussBricks: magnetic building blocks for constructive tangible interactions on portable displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). Association for Computing Machinery, New York, NY, USA, 3153–3162.
[14]
Hila Mor, Tianyu Yu, Ken Nakagaki, Benjamin Harvey Miller, Yichen Jia, and Hiroshi Ishii. 2020. Venous Materials: Towards Interactive Fluidic Mechanisms. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI '20). Association for Computing Machinery, New York, NY, USA, 1–14.
[15]
Md. Farhan Tasnim Oshim, Julian Killingback, Dave Follette, Huaishu Peng, and Tauhidur Rahman. 2020. MechanoBeat: Monitoring Interactions with Everyday Objects using 3D Printed Harmonic Oscillators and Ultra-Wideband Radar. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST '20). Association for Computing Machinery, New York, NY, USA, 430–444.
[16]
Gian Pangaro, Dan Maynes-Aminzade, and Hiroshi Ishii. 2002. The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In Proceedings of the 15th annual ACM symposium on User interface software and technology (UIST '02). Association for Computing Machinery, New York, NY, USA, 181–190.
[17]
Joseph A. Paradiso, Che King Leo, Nisha Checka, and Kaijen Hsiao. 2002. Passive acoustic knock tracking for interactive windows. In CHI '02 Extended Abstracts on Human Factors in Computing Systems(CHI EA '02). Association for Computing Machinery, New York, NY, USA, 732–733.
[18]
James Patten and Hiroshi Ishii. 2007. Mechanical constraints as computational constraints in tabletop tangible interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). Association for Computing Machinery, New York, NY, USA, 809–818.
[19]
Kimiko Ryokai, Stefan Marti, and Hiroshi Ishii. 2004. I/O brush: drawing with everyday objects as ink. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). Association for Computing Machinery, New York, NY, USA, 303–310.
[20]
Aravind Sekhar R, Amritha Mary, Sanju N. Raju, Akhil G. Ravi, Vijitha Sharma, Gauri Bala. A Novel Design Technique to Develop a Low Cost and Highly Stable Wall Climbing Robot. 2013 4th International Conference on Intelligent Systems, Modelling and Simulation, Bangkok, 2013, pp. 360-363. 10.1109/ISMS.2013.114.
[21]
Dominik Schmidt, Raf Ramakers, Esben W. Pedersen, Johannes Jasper, Sven Köhler, Aileen Pohl, Hannes Rantzsch, Andreas Rau, Patrick Schmidt, Christoph Sterz, Yanina Yurchenko, and Patrick Baudisch. 2014. Kickables: tangibles for feet. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI '14). Association for Computing Machinery, New York, NY, USA, 3143–3152.
[22]
Teddy Seyed, Xing-Dong Yang, and Daniel Vogel. 2016. Doppio: A Reconfigurable Dual-Face Smartwatch for Tangible Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI '16). Association for Computing Machinery, New York, NY, USA, 4675–4686.
[23]
Kaige Shi and Xin Li. (2020). Vacuum suction unit based on the zero pressure difference method. Physics of Fluids, 32(1), 017104.
[24]
Ryo Suzuki, Ryosuke Nakayama, Dan Liu, Yasuaki Kakehi, Mark D. Gross, and Daniel Leithinger. 2020. LiftTiles: Constructive Building Blocks for Prototyping Room-scale Shape-changing Interfaces. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '20). Association for Computing Machinery, New York, NY, USA, 143–151.
[25]
Ryo Suzuki, Ryosuke Nakayama, Dan Liu, Yasuaki Kakehi, Mark D. Gross, and Daniel Leithinger. 2019. LiftTiles: Modular and Reconfigurable Room-scale Shape Displays through Retractable Inflatable Actuators. In The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology(UIST '19). Association for Computing Machinery, New York, NY, USA, 30–32.
[26]
Saiganesh Swaminathan, Jonathon Fagert, Michael Rivera, Andrew Cao, Gierad Laput, Hae Young Noh, and Scott E. Hudson. 2020. OptiStructures: Fabrication of Room-Scale Interactive Structures with Embedded Fiber Bragg Grating Optical Sensors and Displays. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.4, 2, Article 50 (June 2020), 21 pages.
[27]
Taghavi, M., Iturralde, K., & Bock, T. (2018). Cable-driven parallel robot for curtain wall modules automatic installation. In Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC) (pp. 396-403).
[28]
John Underkoffler and Hiroshi Ishii. 1999. Urp: a luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99). Association for Computing Machinery, New York, NY, USA, 386–393.
[29]
Ozgur Unver and Mettin Sitti. (2010). Tankbot: A palm-size, tank-like climbing robot using soft elastomer adhesive treads. The International Journal of Robotics Research, 29(14), 1761-1777.
[30]
Zhiwei Yu, Ye Shi, Jiaxing Xie, Simon X. Yang, and Zhendong Dai. Design and analysis of a bionic adhesive foot for gecko robot climbing the ceiling. International Journal of Robotics and Automation 33.4 (2018). 10.2316/Journal.206.2018.4.206-5412
[31]
Yang Zhang, Chouchang (Jack) Yang, Scott E. Hudson, Chris Harrison, and Alanson Sample. 2018. Wall++: Room-Scale Interactive and Context-Aware Sensing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems(CHI '18). Association for Computing Machinery, New York, NY, USA, Paper 273, 1–15.
[32]
Yang Zhang, Gierad Laput, and Chris Harrison. 2018. Vibrosight: Long-Range Vibrometry for Smart Environment Sensing. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology(UIST '18). Association for Computing Machinery, New York, NY, USA, 225–236.

Cited By

View all
  • (2023)Demonstrating Swarm Robots Capable of Cooperative Transitioning between Table and WallAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586182.3615763(1-4)Online publication date: 29-Oct-2023
  • (2023)AeroRigUI: Actuated TUIs for Spatial Interaction using Rigging Swarm Robots on Ceilings in Everyday SpaceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581437(1-18)Online publication date: 19-Apr-2023
  • (2023)ThrowIO: Actuated TUIs that Facilitate “Throwing and Catching” Spatial Interaction with Overhanging Mobile Wheeled RobotsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581267(1-17)Online publication date: 19-Apr-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
May 2021
2965 pages
ISBN:9781450380959
DOI:10.1145/3411763
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Robotics, Climbing Robot
  2. Tangible User Interface, Tangible Interaction, Wall Interaction, Wall

Qualifiers

  • Poster
  • Research
  • Refereed limited

Conference

CHI '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)93
  • Downloads (Last 6 weeks)10
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Demonstrating Swarm Robots Capable of Cooperative Transitioning between Table and WallAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586182.3615763(1-4)Online publication date: 29-Oct-2023
  • (2023)AeroRigUI: Actuated TUIs for Spatial Interaction using Rigging Swarm Robots on Ceilings in Everyday SpaceProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581437(1-18)Online publication date: 19-Apr-2023
  • (2023)ThrowIO: Actuated TUIs that Facilitate “Throwing and Catching” Spatial Interaction with Overhanging Mobile Wheeled RobotsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581267(1-17)Online publication date: 19-Apr-2023
  • (2022)Living in a Historic Neighborhood in the Technology-Era: Understanding Residents’ Expectations from Domestic SpacesNordic Human-Computer Interaction Conference10.1145/3546155.3546671(1-11)Online publication date: 8-Oct-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media