Sean Follmer
Daniel Leithinger
Alex Olwal
Hiroshi Ishii
ABSTRACT
Past research on shape displays has primarily focused on rendering content and user interface elements through shape output, with less emphasis on dynamically changing UIs. We propose utilizing shape displays in three different ways to mediate interaction: to facilitate by providing dynamic physical affordances through shape change, to restrict by guiding users with dynamic physical constraints, and to manipulate by actuating physical objects. We outline potential interaction techniques and introduce Dynamic Physical Affordances and Constraints with our inFORM system, built on top of a state-of-the-art shape display, which provides for variable stiffness rendering and real-time user input through direct touch and tangible interaction. A set of motivating examples demonstrates how dynamic affordances, constraints and object actuation can create novel interaction possibilities.
Supplemental Material
- Audet, S., and Okutomi, M. A user-friendly method to geometrically calibrate projector-camera systems. In IEEE CVPR 2009 (2009), 47--54.Google Scholar
Cross Ref
- Bau, O., Petrevski, U., and Mackay, W. Bubblewrap: a textile-based electromagnetic haptic display. In CHI EA '09, ACM (2009), 3607--3612. Google ScholarDigital Library
- Bier, E. A., Stone, M. C., Pier, K., Buxton, W., and DeRose, T. D. Toolglass and magic lenses: the see-through interface. In SIGGRAPH '93, ACM (1993), 73--80. Google ScholarDigital Library
- Ciesla, C. M., and Yairi, M. B. User interface system and method, 07 2009.Google Scholar
- Coelho, M., and Zigelbaum, J. Shape-changing interfaces. Personal Ubiquitous Comput. 15, 2 (Feb. 2011), 161--173. Google ScholarDigital Library
- Crampton Smith, G. The hand that rocks the cradle. ID magazine (1995), 60--65.Google Scholar
- Fitzmaurice, G. W., Ishii, H., and Buxton, W. A. S. Bricks: laying the foundations for graspable user interfaces. In CHI '95, ACM Press/Addison-Wesley Publishing Co. (1995), 442--449. Google ScholarDigital Library
- Follmer, S., Leithinger, D., Olwal, A., Cheng, N., and Ishii, H. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In UIST '12, ACM (2012), 519--528. Google ScholarDigital Library
- Gaver, W. W. Technology affordances. In CHI '91, ACM (1991), 79--84. Google ScholarDigital Library
- Gibson, J. J. The Senses Considered as Perceptual Systems. Houghton Mifflin Co., 1966.Google Scholar
- Gibson, J. J. The ecological approach to visual perception. Psychology Press, 1986.Google Scholar
- Harrison, C., and Hudson, S. E. Providing dynamically changeable physical buttons on a visual display. In CHI '09, ACM (2009), 299--308. Google ScholarDigital Library
- Hartson, R. Cognitive, physical, sensory, and functional affordances in interaction design. Behaviour & Information Technology 22, 5 (2003), 315--338.Google ScholarCross Ref
- Hemmert, F., Hamann, S., Löwe, M., Wohlauf, A., Zeipelt, J., and Joost, G. Take me by the hand: Haptic compasses in mobile devices through shape change and weight shift. In NordiCHI '10 (2010). Google ScholarDigital Library
- Ishii, H. Tangible bits: beyond pixels. In TEI '08, ACM (2008), xv--xxv. Google ScholarDigital Library
- Ishii, H., Lakatos, D., Bonanni, L., and Labrune, J.-B. Radical atoms: beyond tangible bits, toward transformable materials. interactions 19, 1 (Jan. 2012), 38--51. Google ScholarDigital Library
- Ishii, H., and Ullmer, B. Tangible bits: towards seamless interfaces between people, bits and atoms. In CHI '97, ACM (1997), 234--241. Google ScholarDigital Library
- Iwata, H., Yano, H., Nakaizumi, F., and Kawamura, R. Project feelex: adding haptic surface to graphics. In SIGGRAPH '01, ACM (2001), 469--476. Google ScholarDigital Library
- Jansen, Y., Karrer, T., and Borchers, J. Mudpad: tactile feedback and haptic texture overlay for touch surfaces. In ITS '10, ACM (2010), 11--14. Google ScholarDigital Library
- Kaptelinin, V., and Nardi, B. Affordances in hci: toward a mediated action perspective. In CHI '12, ACM (2012), 967--976. Google ScholarDigital Library
- Klemmer, S. R., Hartmann, B., and Takayama, L. How bodies matter: five themes for interaction design. In DIS '06, ACM (2006), 140--149. Google ScholarDigital Library
- KOJIMA, M., SUGIMOTO, M., NAKAMURA, A., TOMITA, M., INAMI, M., and NII, H. Augmented coliseum: An augmented game environment with small vehicles. In TABLETOP '06, IEEE Computer Society (2006), 3--8. Google ScholarDigital Library
- Lee, N., Kim, J., Lee, J., Shin, M., and Lee, W. Molebot: mole in a table. In SIGGRAPH '11, ACM (2011), 9:1--9:1. Google ScholarDigital Library
- Leithinger, D., Follmer, S., Olwal, A., Luescher, S., Hogge, A., Lee, J., and Ishii, H. Sublimate: State-changing virtual and physical rendering to augment interaction with shape displays. In CHI '13, ACM (2013), 1441--1450. Google ScholarDigital Library
- Leithinger, D., Lakatos, D., DeVincenzi, A., Blackshaw, M., and Ishii, H. Direct and gestural interaction with relief: a 2.5d shape display. In UIST '11, ACM (2011), 541--548. Google ScholarDigital Library
- Macknik, S. L., King, M., Randi, J., Robbins, A., et al. Attention and awareness in stage magic: turning tricks into research. Nature Reviews Neuroscience 9, 11 (2008), 871--879.Google ScholarCross Ref
- MacLean, K. E., Snibbe, S. S., and Levin, G. Tagged handles: merging discrete and continuous manual control. In CHI '00, ACM (2000), 225--232. Google ScholarDigital Library
- McGuffin, M., and Balakrishnan, R. Acquisition of expanding targets. In CHI '02, ACM (2002), 57--64. Google ScholarDigital Library
- Michelitsch, G., Williams, J., Osen, M., Jimenez, B., and Rapp, S. Haptic chameleon: a new concept of shape-changing user interface controls with force feedback. In CHI EA '04, ACM (2004), 1305--1308. Google ScholarDigital Library
- Norman, D. A. The Psychology of Everyday Things. Basic Books, Inc., 1988.Google Scholar
- Pangaro, G., Maynes-Aminzade, D., and Ishii, H. The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In UIST '02, ACM (2002), 181--190. Google ScholarDigital Library
- Patten, J., and Ishii, H. Mechanical constraints as computational constraints in tabletop tangible interfaces. In CHI '07, ACM (2007), 809--818. Google ScholarDigital Library
- Poupyrev, I., Nashida, T., and Okabe, M. Actuation and tangible user interfaces: the vaucanson duck, robots, and shape displays. In TEI '07, ACM (2007), 205--212. Google ScholarDigital Library
- Rasmussen, M. K., Pedersen, E. W., Petersen, M. G., and Hornbaek, K. Shape-changing interfaces: a review of the design space and open research questions. In CHI '12, ACM (2012), 735--744. Google ScholarDigital Library
- Rekimoto, J., Ullmer, B., and Oba, H. Datatiles: a modular platform for mixed physical and graphical interactions. In CHI '01, ACM (2001), 269--276. Google ScholarDigital Library
- Reznik, D., and Canny, J. A flat rigid plate is a universal planar manipulator. In IEEE ICRA 1998, vol. 2, IEEE (1998), 1471--1477.Google ScholarCross Ref
- Salisbury, K., Conti, F., and Barbagli, F. Haptic rendering: introductory concepts. Computer Graphics and Applications, IEEE 24, 2 (2004), 24--32. Google ScholarDigital Library
- Ullmer, B., Ishii, H., and Jacob, R. J. K. Token+constraint systems for tangible interaction with digital information. ACM Trans. Comput.-Hum. Interact. 12, 1 (Mar. 2005), 81--118. Google ScholarDigital Library
- Vaughan, L. C. Understanding movement. In CHI '97, ACM (1997), 548--549. Google ScholarDigital Library
- Vermeulen, J., Luyten, K., van den Hoven, E., and Coninx, K. Crossing the bridge over norman's gulf of execution: revealing feedforward's true identity. In CHI '13, ACM (2013), 1931--1940. Google ScholarDigital Library
- Weiss, M., Remy, C., and Borchers, J. Rendering physical effects in tabletop controls. In CHI '11, ACM (2011), 3009--3012. Google ScholarDigital Library
- Weiss, M., Schwarz, F., Jakubowski, S., and Borchers, J. Madgets: actuating widgets on interactive tabletops. In UIST '10, ACM (2010), 293--302. Google ScholarDigital Library
- Weiss, M., Wacharamanotham, C., Voelker, S., and Borchers, J. Fingerflux: near-surface haptic feedback on tabletops. In UIST '11, ACM (2011), 615--620. Google ScholarDigital Library
- Weiss, M., Wagner, J., Jansen, Y., Jennings, R., Khoshabeh, R., Hollan, J. D., and Borchers, J. Slap widgets: bridging the gap between virtual and physical controls on tabletops. In CHI '09, ACM (2009), 481--490. Google ScholarDigital Library
Index Terms
- inFORM: dynamic physical affordances and constraints through shape and object actuation
Recommendations
shapeShift: 2D Spatial Manipulation and Self-Actuation of Tabletop Shape Displays for Tangible and Haptic Interaction
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsWe explore interactions enabled by 2D spatial manipulation and self-actuation of a tabletop shape display. To explore these interactions, we developed shapeShift, a compact, high-resolution (7 mm pitch), mobile tabletop shape display. shapeShift can be ...
Shape-changing interfaces: a review of the design space and open research questions
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsShape change is increasingly used in physical user interfaces, both as input and output. Yet, the progress made and the key research questions for shape-changing interfaces are rarely analyzed systematically. We review a sample of existing work on shape-...
Kinetic Blocks: Actuated Constructive Assembly for Interaction and Display
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & TechnologyPin-based shape displays not only give physical form to digital information, they have the inherent ability to accurately move and manipulate objects placed on top of them. In this paper we focus on such object manipulation: we present ideas and ...
Comments