skip to main content
10.1145/3313831.3376838acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

PneuModule: Using Inflatable Pin Arrays for Reconfigurable Physical Controls on Pressure-Sensitive Touch Surfaces

Published: 23 April 2020 Publication History

Abstract

We present PneuModule, a tangible interface platform that enables users to reconfigure physical controls on pressure-sensitive touch surfaces using pneumatically-actuated inflatable pin arrays. PneuModule consists of a main module and extension modules. The main module is tracked on the touch surface and forwards continuous inputs from attached multiple extension modules to the touch surface. Extension modules have distinct mechanisms for user input, which pneumatically actuates the inflatable pins at the bottom of the main module through internal air pipes. The main module accepts multi-dimensional inputs since each pin is individually inflated by the corresponding air chamber. Also, since the extension modules are swappable and identifiable owing to the marker design, users can quickly customize the interface layout. We contribute to design details of inflatable pins and diverse pneumatic input control design examples for PneuModule. We also showcase the feasibility of PneuModule through a series of evaluations and interactive prototypes.

Supplemental Material

MP4 File
MP4 File
Preview video
MP4 File
Supplemental video

References

[1]
Anna Xambó Sedó. 2014. Tabletop Tangible Interfaces for Music Performance: Design and Evaluation. Ph.D. Dissertation. The Open University. http://annaxambo.me/pub/Xambo_2015_thesis.screen.pdf
[2]
Tom Bartindale and Chris Harrison. 2009. Stacks on the surface: resolving physical order using fiducial markers with structured transparency. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '09. ACM Press, New York, New York, USA, 57.
[3]
Patrick Baudisch, Torsten Becker, and Frederik Rudeck. 2010. Lumino: Tangible Blocks for Tabletop Computers Based on Glass Fiber Bundles. In Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. ACM Press, New York, New York, USA, 1165--1174.
[4]
Alberto Boem and Giovanni Maria Troiano. 2019. Non-Rigid HCI: A Review of Deformable Interfaces and Input. In Proceedings of the 2019 on Designing Interactive Systems Conference - DIS '19. ACM Press, New York, New York, USA, 885--906.
[5]
Alan Bränzel, Christian Holz, Daniel Hoffmann, Dominik Schmidt, Marius Knaust, Patrick Lühne, René Meusel, Stephan Richter, and Patrick Baudisch. 2013. GravitySpace: Tracking Users and Their Poses in a Smart Room Using a Pressure-Sensing Floor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI '13. ACM Press, New York, New York, USA, 725--734.
[6]
Liwei Chan, Stefanie Müller, Anne Roudaut, and Patrick Baudisch. 2012. CapStones and ZebraWidgets: Sensing Stacks of Building Blocks, Dials and Sliders on Capacitive Touch Screens. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12. ACM Press, New York, New York, USA, 2189.
[7]
Alix Goguey, Timothy Neate, Jennifer Pearson, Simon Robinson, Matt Jones, Cameron Steer, Andrés Lucero, Laurence Nigay, Deepak Ranjan Sahoo, Céline Coutrix, Anne Roudaut, Sriram Subramanian, and Yutaka Tokuda. 2019. PickCells: A Physically Reconfigurable Cell-composed Touchscreen. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19. ACM Press, New York, New York, USA, 1--14.
[8]
Saul Greenberg and Chester Fitchett. 2001. Phidgets: Easy Development of Physical Interfaces through Physical Widgets. In Proceedings of the 14th annual ACM symposium on User interface software and technology - UIST '01. ACM Press, New York, New York, USA, 209.
[9]
Sebastian Günther, Martin Schmitz, Florian Müller, Jan Riemann, and Max Mühlhäuser. 2017. BYO*: Utilizing 3D Printed Tangible Tools for Interaction on Interactive Surfaces. In Proceedings of the 2017 ACM Workshop on Interacting with Smart Objects - SmartObject '17. ACM Press, New York, New York, USA, 21--26.
[10]
Changyo Han, Katsufumi Matsui, and Takeshi Naemura. 2020. ForceStamps: Fiducial Markers for Pressure-sensitive Touch Surfaces to Support Rapid Porototyping of Physical Control Interfaces. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '20.
[11]
Changyo Han and Takeshi Naemura. 2019. BumpMarker: a 3D-printed tangible marker for simultaneous tagging, tracking, and weight measurement. ITE Transactions on Media Technology and Applications 7, 1 (2019), 11--19.
[12]
Fabian Hennecke, Franz Berwein, and Andreas Butz. 2011. Optical Pressure Sensing for Tangible User Interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces - ITS '11. ACM Press, New York, New York, USA, 45--48.
[13]
Sanghwa Hong, Eunseok Jeong, Seongkook Heo, and Byungjoo Lee. 2018. FDSense: Estimating Young's Modulus and Stiffness of End Effectors to Facilitate Kinetic Interaction on Touch Surfaces. In The 31st Annual ACM Symposium on User Interface Software and Technology - UIST '18. ACM Press, New York, New York, USA, 809--823.
[14]
Meng-Ju Hsieh, Rong-Hao Liang, Da-Yuan Huang, Jheng-You Ke, and Bing-Yu Chen. 2018. RFIBricks: Interactive Building Blocks Based on RFID. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. ACM Press, New York, New York, USA, 1--10.
[15]
Sungjae Hwang, Myungwook Ahn, and Kwang-yun Wohn. 2013. MagGetz: Customizable Passive Tangible Controllers On and Around Conventional Mobile Devices. In Proceedings of the 26th annual ACM symposium on User interface software and technology UIST '13. ACM Press, New York, New York, USA, 411--416.
[16]
Kaori Ikematsu and Itiro Siio. 2018. Ohmic-Touch: Extending Touch Interaction by Indirect Touch through Resistive Objects. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems CHI '18. ACM Press, New York, New York, USA, 1--8.
[17]
Michael D. Jones, Casey Walker, Zann Anderson, Candice Lusk, and Andrew Bryce. 2017. Exploring a Modular Approach to Redesigning Interfaces for Physical Interactive Devices. In Proceedings of the Tenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '17. ACM Press, New York, New York, USA, 465--471.
[18]
Yasuaki Kakehi, Kensei Jo, Katsunori Sato, Kouta Minamizawa, Hideaki Nii, Naoki Kawakami, Takeshi Naemura, and Susumu Tachi. 2008. ForceTile: Tabletop Tangible Interface with Vision-based Force Distribution Sensing. In ACM SIGGRAPH 2008 new tech demos on SIGGRAPH '08. ACM Press, New York, New York, USA, 1--1.
[19]
Hyunyoung Kim, Céline Coutrix, and Anne Roudaut. 2018. KnobSlider: Design of a Shape-Changing UI for Parameter Control. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems CHI '18. ACM Press, New York, New York, USA, 1--13.
[20]
Gierad Laput, Eric Brockmeyer, Scott E. Hudson, and Chris Harrison. 2015. Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15. ACM Press, New York, New York, USA, 2161--2170.
[21]
Johnny C. Lee, Daniel Avrahami, Scott E. Hudson, Jodi Forlizzi, Paul H. Dietz, and Darren Leigh. 2004. The Calder Toolkit: Wired and Wireless Components for Rapidly Prototyping Interactive Devices. In Proceedings of the 2004 conference on Designing interactive systems processes, practices, methods, and techniques - DIS '04. ACM Press, New York, New York, USA, 167.
[22]
Jakob Leitner and Michael Haller. 2011. Geckos: Combining Magnets and Pressure Images to Enable New Tangible-object Design and Interaction. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11. ACM Press, New York, New York, USA, 2985--2994.
[23]
Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang, De-Nian Yang, Bing-Yu Chen, Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang, De-Nian Yang, and Bing-Yu Chen. 2014a. GaussBricks: Magnetic Building Blocks for Constructive Tangible Interactions on Portable Displays. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14. ACM Press, New York, New York, USA, 3153--3162.
[24]
Rong-Hao Liang, Han-Chih Kuo, Liwei Chan, De-Nian Yang, and Bing-Yu Chen. 2014b. GaussStones: Shielded Magnetic Tangibles for Multi-Token Interactions on Portable Displays Rong-Hao. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST '14. ACM Press, New York, New York, USA, 365--372.
[25]
Satoshi Nakamaru, Ryosuke Nakayama, Ryuma Niiyama, and Yasuaki Kakehi. 2017. FoamSense: Design of Three Dimensional Soft Sensors with Porous Materials. In UIST 2017 - Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. ACM Press, New York, New York, USA, 437--447.
[26]
Ryosuke Nakayama, Ryo Suzuki, Satoshi Nakamaru, Ryuma Niiyama, Yoshihiro Kawahara, and Yasuaki Kakehi. 2019. MorphIO: Entirely Soft Sensing and Actuation Modules for Programming Shape Changes through Tangible Interaction. In Proceedings of the 2019 on Designing Interactive Systems Conference - DIS '19. ACM Press, New York, New York, USA, 975--986.
[27]
Simon Olberding, Sergio Soto Ortega, Klaus Hildebrandt, and Jürgen Steimle. 2015. Foldio: Digital Fabrication of Interactive and Shape- Changing Objects With Foldable Printed Electronics. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST '15. ACM Press, New York, New York, USA, 223--232.
[28]
James Patten, Hiroshi Ishii, Jim Hines, and Gian Pangaro. 2001. Sensetable: a wireless object tracking platform for tangible user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '01. ACM Press, New York, New York, USA, 253--260.
[29]
Parinya Punpongsanon, Daisuke Iwai, and Kosuke Sato. 2014. Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture. Virtual Reality 19, 1 (2014), 45--56.
[30]
Simon Robinson, Céline Coutrix, Jennifer Pearson, Juan Rosso, Matheus Fernandes Torquato, Laurence Nigay, and Matt Jones. 2016. Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16. ACM Press, New York, New York, USA, 3793--3805.
[31]
Valkyrie Savage, Ryan Schmidt, Tovi Grossman, George Fitzmaurice, and Björn Hartmann. 2014. A Series of Tubes: Adding Interactivity to 3D Prints Using Internal Pipes. In Proceedings of the 27th annual ACM symposium on User interface software and technology UIST '14. ACM Press, New York, New York, USA, 3--12.
[32]
Dominik Schmidt, Christoph Sterz, Yanina Yurchenko, Patrick Baudisch, Raf Ramakers, Esben W. Pedersen, Johannes Jasper, Sven Köhler, Aileen Pohl, Hannes Rantzsch, Andreas Rau, and Patrick Schmidt. 2014. Kickables: Tangibles for Feet. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14. ACM Press, New York, New York, USA, 3143--3152.
[33]
Martin Schmitz, Jürgen Steimle, Jochen Huber, Niloofar Dezfuli, and Max Mühlhäuser. 2017. Flexibles: Deformation-Aware 3D-Printed Tangibles for Capacitive Touchscreens. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems CHI '17. ACM Press, New York, New York, USA, 1001--1014.
[34]
Timothy M. Simon, Bruce H. Thomas, Ross T. Smith, and Mark Smith. 2013. Adding Input Controls and Sensors to RFID Tags to support Dynamic Tangible User Interfaces. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction - TEI '14. ACM Press, New York, New York, USA, 165--172.
[35]
Ronit Slyper and Jessica Hodgins. 2012. Prototyping Robot Appearance, Movement, and Interactions Using Flexible 3D Printing and Air Pressure Sensors. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 6--11.
[36]
Paul Strohmeier, Victor Håkansson, Cedric Honnet, Daniel Ashbrook, and Kasper Hornbæk. 2019. Optimizing Pressure Matrices: Interdigitation and Interpolation Methods for Continuous Position Input. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '19. ACM Press, New York, New York, USA, 117--126.
[37]
Yuta Sugiura, Calista Lee, Masayasu Ogata, Anusha Withana, Yasutoshi Makino, Daisuke Sakamoto, Masahiko Inami, and Takeo Igarashi. 2012. PINOKY: A Ring That Animates Your Plush Toys. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12. ACM Press, New York, New York, USA, 725.
[38]
Koray Tahiroglu, Thomas Svedström, Valtteri Wikström, Simon Overstall, Johan Kildal, and Teemu Ahmaniemi. 2014. SoundFLEX: Designing Audio to Guide Interactions with Shape-Retaining Deformable Interfaces. In Proceedings of the 16th International Conference on Multimodal Interaction - ICMI '14. ACM Press, New York, New York, USA, 267--274.
[39]
Giovanni Maria Troiano, Esben Warming Pedersen, and Kasper Hornbæk. 2015. Deformable Interfaces for Performing Music. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15. ACM Press, New York, New York, USA, 377--386.
[40]
Bert Vandenberghe, Kathrin Gerling, Luc Geurts, and Vero Vanden Abeele. 2019. Skweezee for Processing: A Software Library to Make Squeeze Interactions. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '19. ACM Press, New York, New York, USA, 375--381.
[41]
Marynel Vázquez, Eric Brockmeyer, Ruta Desai, Chris Harrison, and Scott E. Hudson. 2015. 3D Printing Pneumatic Device Controls with Variable Activation Force Capabilities. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15. ACM Press, New York, New York, USA, 1295--1304.
[42]
Nicolas Villar and Hans Gellersen. 2007. A Malleable Control Structure for Softwired User Interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction - TEI '07. ACM Press, New York, New York, USA, 49--56.
[43]
Nicolas Villar, Haiyan Zhang, Daniel Cletheroe, Greg Saul, Christian Holz, Tim Regan, Oscar Salandin, Misha Sra, Hui-Shyong Yeo, and William Field. 2018. Project Zanzibar: A Portable and Flexible Tangible Interaction Platform. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. ACM Press, New York, New York, USA, 1--13.
[44]
Simon Voelker, Christian Cherek, Jan Thar, Thorsten Karrer, Christian Thoresen, Kjell Ivar Øvergård, and Jan Borchers. 2015. PERCs: Persistently Trackable Tangibles on Capacitive Multi-Touch Displays. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology - UIST '15. ACM Press, New York, New York, USA, 351--356.
[45]
Malte Weiss, Julie Wagner, Yvonne Jansen, Roger Jennings, Ramsin Khoshabeh, James D. Hollan, and Jan Borchers. 2009. SLAP widgets: bridging the gap between virtual and physical controls on tabletops. In Proceedings of the 27th international conference on Human factors in computing systems - CHI 09. ACM Press, New York, New York, USA, 481.
[46]
Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva, and Hiroshi Ishii. 2013. PneUI: pneumatically actuated soft composite materials for shape changing interfaces. In ACM Transactions on Graphics. ACM Press, New York, New York, USA, 13--22.
[47]
Sang Ho Yoon, Luis Paredes, Ke Huo, and Karthik Ramani. 2018. MultiSoft: Soft Sensor Enabling Real-Time Multimodal Sensing with Contact Localization and Deformation Classification. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3 (2018), 1--22.
[48]
Neng-Hao Yu, Polly Huang, Yi-Ping Hung, Li-Wei Chan, Seng Yong Lau, Sung-Sheng Tsai, I-Chun Hsiao, Dian-Je Tsai, Fang-I Hsiao, Lung-Pan Cheng, and Mike Chen. 2011. TUIC: Enabling Tangible Interaction on Capacitive Multi-touch Display. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11. ACM Press, New York, New York, USA, 2995--3004.
[49]
Yuan-Fang Zhang, Colin Ju-Xiang Ng, Zhe Chen, Wang Zhang, Sahil Panjwani, Kavin Kowsari, Hui Ying Yang, and Qi Ge. 2019. Miniature Pneumatic Actuators for Soft Robots by High-Resolution Multimaterial 3D Printing. Advanced Materials Technologies 1900427 (8 2019), 1--9.

Cited By

View all
  • (2025)Impedius: A Signal-Space Multiplexing Technique Using Individual Elements of Impedance for Chained Passive SensorsProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704949(1-11)Online publication date: 4-Mar-2025
  • (2025)Elastic robotic structures: a multidisciplinary framework for the design and control of shape-morphing elastic system for architectural and design applicationsConstruction Robotics10.1007/s41693-024-00128-89:1Online publication date: 13-Jan-2025
  • (2024)Passive Stylus Tracking: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36981448:ISS(427-461)Online publication date: 24-Oct-2024
  • Show More Cited By

Index Terms

  1. PneuModule: Using Inflatable Pin Arrays for Reconfigurable Physical Controls on Pressure-Sensitive Touch Surfaces

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
    April 2020
    10688 pages
    ISBN:9781450367080
    DOI:10.1145/3313831
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 April 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    • Honorable Mention

    Author Tags

    1. pneumatic actuation
    2. pressure-sensitive touch surfaces
    3. reconfigurable physical controls
    4. tangible user interfaces

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)80
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 22 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Impedius: A Signal-Space Multiplexing Technique Using Individual Elements of Impedance for Chained Passive SensorsProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704949(1-11)Online publication date: 4-Mar-2025
    • (2025)Elastic robotic structures: a multidisciplinary framework for the design and control of shape-morphing elastic system for architectural and design applicationsConstruction Robotics10.1007/s41693-024-00128-89:1Online publication date: 13-Jan-2025
    • (2024)Passive Stylus Tracking: A Systematic Literature ReviewProceedings of the ACM on Human-Computer Interaction10.1145/36981448:ISS(427-461)Online publication date: 24-Oct-2024
    • (2022)Conductive Fiducial Tangibles for EveryoneProceedings of the ACM on Human-Computer Interaction10.1145/35467186:MHCI(1-22)Online publication date: 20-Sep-2022
    • (2022)InfraredTags: Embedding Invisible AR Markers and Barcodes Using Low-Cost, Infrared-Based 3D Printing and Imaging ToolsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501951(1-12)Online publication date: 29-Apr-2022
    • (2021)3D Printing Firm Inflatables with Internal TethersExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3451613(1-7)Online publication date: 8-May-2021

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media