research-article

Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices

Authors:

Daniel Leithinger,

Hiroshi IshiiAuthors Info & Claims

UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technology

Pages 519 - 528

https://doi.org/10.1145/2380116.2380181

Published: 07 October 2012 Publication History

Abstract

Malleable and organic user interfaces have the potential to enable radically new forms of interactions and expressiveness through flexible, free-form and computationally controlled shapes and displays. This work, specifically focuses on particle jamming as a simple, effective method for flexible, shape-changing user interfaces where programmatic control of material stiffness enables haptic feedback, deformation, tunable affordances and control gain. We introduce a compact, low-power pneumatic jamming system suitable for mobile devices, and a new hydraulic-based technique with fast, silent actuation and optical shape sensing. We enable jamming structures to sense input and function as interaction devices through two contributed methods for high-resolution shape sensing using: 1) index-matched particles and fluids, and 2) capacitive and electric field sensing. We explore the design space of malleable and organic user interfaces enabled by jamming through four motivational prototypes that highlight jamming's potential in HCI, including applications for tabletops, tablets and for portable shape-changing mobile devices.

Supplementary Material

JPG File (paper_0282-file3.jpg)

Download
18.69 KB

suppl.mov (paper_0282-file3.m4v)

Supplemental video

Download
76.50 MB

References

[1]

Amend, J., Brown, E., Rodenberg, N., Jaeger, H., and Lipson, H. A Positive Pressure Universal Gripper Based on the Jamming of Granular Material. IEEE Transactions on Robotics (2012), 341--350.

[2]

Benko, H. Beyond flat surface computing: challenges of depth-aware and curved interfaces. In ACM MM '09, ACM Press (New York, New York, USA, Oct. 2009), 935--944.

Digital Library

[3]

Brown, E., Rodenberg, N., Amend, J., Mozeika, A., Steltz, E., Zakin, M. R., Lipson, H., and Jaeger, H. M. Universal robotic gripper based on the jamming of granular material. Proceedings of the National Academy of Sciences 107, 44 (Oct. 2010), 18809--18814.

[4]

Cassinelli, A., and Ishikawa, M. Khronos projector. In ACM SIG-GRAPH 2005 Emerging Technologies, SIGGRAPH '05, ACM Press (New York, New York, USA, 2005).

Digital Library

[5]

Cates, M., Wittmer, J., Bouchaud, J. P., and Claudin, P. Jamming, Force Chains, and Fragile Matter. Physical Review Letters 81, 9 (Aug. 1998), 1841--1844.

[6]

Cheng, N., Lobovsky, M., Keating, S., Setapen, A., Gero, K. I., Hosoi, A., and Iagnemma, K. Design and Analysis of a Robust, Low-Cost, Highly Articulated Manipulator Enabled by Jamming of Granular Media. IEEE ICRA (2012), 4328--4333.

[7]

Coelho, M., and Zigelbaum, J. Shape-changing interfaces. Personal and Ubiquitous Computing 15, 2 (July 2010), 161--173.

Digital Library

[8]

Follmer, S., Johnson, M., Adelson, E., and Ishii, H. deForm: an interactive malleable surface for capturing 2.5D arbitrary objects, tools and touch. In UIST '11, ACM Press (Oct. 2011), 527--536.

Digital Library

[9]

Gupta, S., Campbell, T., Hightower, J. R., and Patel, S. N. SqueezeBlock: using virtual springs in mobile devices for eyes-free interaction. In UIST '10, ACM Press (Oct. 2010), 101--104.

Digital Library

[10]

Harrison, C., and Hudson, S. E. Providing dynamically changeable physical buttons on a visual display. In CHI '09, ACM Press (Apr. 2009), 299--308.

Digital Library

[11]

Hemmert, F., Hamann, S., Löwe, M., Wohlauf, A., and Joost, G. Shape-changing mobiles: tapering in one-dimensional deformational displays in mobile phones. In TEI '10, ACM Press (Jan. 2010), 249-- 252.

Digital Library

[12]

Herkenrath, G., Karrer, T., and Borchers, J. Twend: twisting and bending as new interaction gesture in mobile devices. In CHI EA '08, ACM Press (New York, New York, USA, Apr. 2008), 3819--3824.

Digital Library

[13]

Hilliges, O., Kim, D., and Izadi, S. Creating malleable interactive surfaces using liquid displacement sensing. In Proc. Tabletop 2008, IEEE Press (Oct. 2008), 157--160.

[14]

Hook, J., Taylor, S., Butler, A., Villar, N., and Izadi, S. A reconfigurable ferromagnetic input device. In Proc. UIST 2009, ACM Press (New York, New York, USA, 2009), 51--54.

Digital Library

[15]

Huber, J. E., Fleck, N. a., and Ashby, M. F. The selection of mechanical actuators based on performance indices. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 453, 1965 (Oct. 1997), 2185--2205.

[16]

Huijben, F., Herwijnen, F. V. A. N., and Nijsse, R. Vacuumatics 3D-Formwork Systems : Customized free-form solidification. Nano, figure 1 (2009), 1--4.

[17]

Ishii, H., Ratti, C., Piper, B., Wang, Y., Biderman, A., and Ben-Joseph, E. Bringing Clay and Sand into Digital Design Continuous Tangible user Interfaces. BT Technology Journal 22, 4 (Oct. 2004), 287--299.

Digital Library

[18]

Iwata, H., Yano, H., Nakaizumi, F., and Kawamura, R. Project FEELEX: adding haptic surface to graphics. In SIGGRAPH '01, ACM Press (Aug. 2001), 469--476.

Digital Library

[19]

Jansen, Y., Karrer, T., and Borchers, J. MudPad: tactile feedback and haptic texture overlay for touch surfaces. In Proc. ITS 2010, ACM Press (2010), 11--14.

Digital Library

[20]

Kim, S., Kim, H., Lee, B., Nam, T.-J., and Lee, W. Inflatable mouse: volume-adjustable mouse with air-pressure-sensitive input and haptic feedback. In CHI '08, ACM Press (Apr. 2008), 211--224.

Digital Library

[21]

Lahey, B., Girouard, A., Burleson, W., and Vertegaal, R. PaperPhone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays. In CHI '11, ACM Press (May 2011), 1303--1312.

Digital Library

[22]

Liu, A. Jamming is not just cool any more. Nature 396, November (1998), 21--22.

[23]

Loeve, A. J., Ven, O. S., Vogel, J. G., Breedveld, P., and Dankelman, J. Vacuum packed particles as flexible endoscope guides with controllable rigidity. Granular Matter 12, 6 (June 2010), 543--554.

[24]

Marquardt, N., Nacenta, M. A., Young, J. E., Carpendale, S., Greenberg, S., and Sharlin, E. The Haptic Tabletop Puck: tactile feedback for interactive tabletops. In ITS '09, ACM Press (Nov. 2009), 85--92.

Digital Library

[25]

Matoba, Y., Sato, T., Takahashi, N., and Koike, H. Claytricsurface: an interactive surface with dynamic softness control capability. In SIGGRAPH '12 Emerging Technologies, ACM (2012), 6:1--6:1.

Digital Library

[26]

Mazzone, A., Spagno, C., and Kunz, A. The HoverMesh: a deformable structure based on vacuum cells: new advances in the research of tangible user interfaces. In ACE '04, ACM Press (Sept. 2004), 187--193.

Digital Library

[27]

Mitsuda, T., Kuge, S., Wakabayashi, M., and Kawamura, S. Wearable haptic display by the use of a Particle Mechanical Constraint. Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002 (2002), 153--158.

Digital Library

[28]

Overholt, D. The MATRIX: a novel controller for musical expression. In Proc. NIME 2001, NIME '01, AMC (Singapore, 2001), 1--4.

Digital Library

[29]

Park, Y.-L., Majidi, C., Kramer, R., Beérard, P., and Wood, R. J. Hyperelastic pressure sensing with a liquid-embedded elastomer. Journal of Micromechanics and Microengineering 20, 12 (Dec. 2010), 125029.

[30]

Parkes, A., Poupyrev, I., and Ishii, H. Designing kinetic interactions for organic user interfaces. Communications of the ACM 51, 6 (June 2008), 58--65.

Digital Library

[31]

Rekimoto, J. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proc. CHI 2002, CHI '02, ACM Press (New York, New York, USA, 2002), 113--120.

Digital Library

[32]

Schwesig, C., Poupyrev, I., and Mori, E. Gummi: a bendable computer. In Proceedings of the 2004 conference on Human factors in computing systems - CHI '04, ACM Press (New York, New York, USA, Apr. 2004), 263--270.

Digital Library

[33]

Sergio, M., Manaresi, N., and Tartagni, M. A Textile Based Capacitive Pressure Sensor. Sensors, 2002. (2002), 1625--1630.

[34]

Smith, R. T., Thomas, B. H., and Piekarski, W. Digital foam interaction techniques for 3D modeling. In Virtual Reality Software and Technology (2008), 61--68.

Digital Library

[35]

Steltz, E., Mozeika, A., and Rembisz, J. Jamming as an enabling technology for soft robotics. Electroactive Polymer Actuators and Devices (EAPAD), 764225.

[36]

Steltz, E., Mozeika, A., Rodenberg, N., Brown, E., and Jaeger, H. JSEL: Jamming Skin Enabled Locomotion. 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (Oct. 2009), 5672--5677.

Digital Library

[37]

Taylor, B. T., and Bove, V. M. Graspables: grasp-recognition as a user interface. In Proceedings of the 27th international conference on Human factors in computing systems - CHI '09, ACM Press (New York, New York, USA, Apr. 2009), 917--926.

Digital Library

[38]

Torquato, S., Truskett, T., and Debenedetti, P. Is random close packing of spheres well defined? Physical review letters 84, 10 (Mar. 2000), 2064--7.

[39]

Vertegaal, R., and Poupyrev, I. Organic User Interfaces. Communications of the ACM 51, 6 (June 2008), 26--30.

Digital Library

[40]

Vlack, K., Mizota, T., Kawakami, N., Kamiyama, K., Kajimoto, H., and Tachi, S. Gelforce: a vision-based traction field computer interface. In Ext. Abstracts CHI 2005, ACM Press (2005), 1154--1155.

Digital Library

[41]

White, Tom. Introducing Liquid Haptics in High Bandwidth Human Computer Interfaces. PhD thesis, MIT, 1998.

Cited By

Kwok TZhang BChow W(2025)A Wearable Stiffness-Rendering Haptic Device with a Honeycomb Jamming Mechanism for Bilateral TeleoperationMachines10.3390/machines1301002713:1(27)Online publication date: 6-Jan-2025
https://doi.org/10.3390/machines13010027
Miyajima MUmedachi TIwamoto N(2025)Variable-Corner Robotic Surfaces with Minimal Area and Their KinematicsJournal of Robotics and Mechatronics10.20965/jrm.2025.p014337:1(143-152)Online publication date: 20-Feb-2025
https://doi.org/10.20965/jrm.2025.p0143
Zhang LHuai YHu ZWu JCassinelli AZhu K(2025)ThreadTessel: A Modularized Tangible Toolkit Leveraging Origami Tessellation for Designing Thread-actuated Shape-changing StructuresProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704929(1-15)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3704929
Show More Cited By

Index Terms

Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Shape-changing interfaces: a review of the design space and open research questions
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Shape change is increasingly used in physical user interfaces, both as input and output. Yet, the progress made and the key research questions for shape-changing interfaces are rarely analyzed systematically. We review a sample of existing work on shape-...
SnowGlobe: a spherical fish-tank VR display
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing Systems

In this paper, we present a spherical display with Fish-Tank VR as a means for interacting with three- dimensional objects. We implemented the spherical display by reflecting a projected image off a hemispherical mirror, allowing for a seamless ...
FlexView: an evaluation of depth navigation on deformable mobile devices
TEI '13: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction

We present FlexView, a set of interaction techniques for Z-axis navigation on touch-enabled flexible mobile devices. FlexView augments touch input with bend to navigate through depth-arranged content. To investigate Z-axis navigation with FlexView, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technology

October 2012

608 pages

ISBN:9781450315807

DOI:10.1145/2380116

General Chair:
Rob Miller
MIT CSAIL, USA
,
Program Chairs:
Hrvoje Benko
Microsoft Research, USA
,
Celine Latulipe
University of North Carolina at Charlotte, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '12

Sponsor:

UIST '12: The 25th Annual ACM Symposium on User Interface Software and Technology

October 7 - 10, 2012

Massachusetts, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25

Sponsor:
sigchi
sigchi

The 38th Annual ACM Symposium on User Interface Software and Technology

September 28 - October 1, 2025

Busan , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

213
Total Citations
View Citations
3,675
Total Downloads

Downloads (Last 12 months)301
Downloads (Last 6 weeks)21

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kwok TZhang BChow W(2025)A Wearable Stiffness-Rendering Haptic Device with a Honeycomb Jamming Mechanism for Bilateral TeleoperationMachines10.3390/machines1301002713:1(27)Online publication date: 6-Jan-2025
https://doi.org/10.3390/machines13010027
Miyajima MUmedachi TIwamoto N(2025)Variable-Corner Robotic Surfaces with Minimal Area and Their KinematicsJournal of Robotics and Mechatronics10.20965/jrm.2025.p014337:1(143-152)Online publication date: 20-Feb-2025
https://doi.org/10.20965/jrm.2025.p0143
Zhang LHuai YHu ZWu JCassinelli AZhu K(2025)ThreadTessel: A Modularized Tangible Toolkit Leveraging Origami Tessellation for Designing Thread-actuated Shape-changing StructuresProceedings of the Nineteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3689050.3704929(1-15)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.1145/3689050.3704929
Brown JFarkhatdinov IJenkin M(2025)ROV Teleoperation in the Presence of Cross‐Currents Using Soft HapticsJournal of Field Robotics10.1002/rob.22533Online publication date: 23-Feb-2025
https://doi.org/10.1002/rob.22533
Kedwan F(2024)Intermolecular Forces in a Physical Molecular Visualization System: A Human-Computer Interaction ApplicationEuropean Journal of Applied Science, Engineering and Technology10.59324/ejaset.2024.2(2).142:2(212-223)Online publication date: 1-Mar-2024
https://doi.org/10.59324/ejaset.2024.2(2).14
de Rodrigo IBelart JLopez-Lopez A(2024)Universal Jamming Gripper: Experimental Analysis on Envelope and Granular MaterialsMachines10.3390/machines1201005212:1(52)Online publication date: 11-Jan-2024
https://doi.org/10.3390/machines12010052
Kedwan F(2024)Intermolecular Forces in a Physical Molecular Visualization System: A Human-Computer Interaction ApplicationSSRN Electronic Journal10.2139/ssrn.4840189Online publication date: 2024
https://doi.org/10.2139/ssrn.4840189
Kang SKim GHwang SPark JElsharkawy AKim S(2024)Flip-Pelt: Motor-Driven Peltier Elements for Rapid Thermal Stimulation and Congruent Pressure Feedback in Virtual RealityProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676363(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676363
Guo YWang RFeng YLu YWang XTanaka FMi H(2024)ROMEO: Towards the Design of Robot with Haptic Mediation for Remote ConflictExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650781(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650781
Nash JSteer CDinca TSharma AFavaratto Santos AWildgoose BAger AClarke CAlexander J(2024)DeformIO: Dynamic Stiffness Control on a Deformable Force-sensing DisplayExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650772(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650772
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten