Hrvoje Benko
ABSTRACT
In the past decade, multi-touch-sensitive interactive surfaces have transitioned from pure research prototypes in the lab, to commercial products with wide-spread adoption. One of the longer term visions of this research follows the idea of ubiquitous computing, where everyday surfaces in our environment are made interactive. However, most of current interfaces remain firmly tied to the traditional flat rectangular displays of the today's computers and while they benefit from the directness and the ease of use, they are often not much more than touch-enabled standard desktop interfaces.
In this paper, we argue for explorations that transcend the traditional notion of the flat display, and envision interfaces that are curved, three-dimensional, or that cross the boundary between the digital and physical world. In particular, we present two research directions that explore this idea: (a) exploring the three-dimensional interaction space above the display and (b) enabling gestural and touch interactions on curved devices for novel interaction possibilities. To illustrate both of these, we draw examples from our own work and the work of others, and guide the reader through several case studies that highlight the challenges and benefits of such novel interfaces. The implications on media requirements and collaboration aspects are discussed in detail, and, whenever possible, we highlight promising directions of future research. We believe that the compelling application design for future non-flat user interfaces will greatly depend on exploiting the unique characteristics of the given form factor.
- Benko, H., Ishak, E.W., Feiner, S. (2005). Cross-Dimensional Gestural Interaction Techniques for Hybrid Immersive Environments. In Proceedings of IEEE Conference on Virtual Reality (VR). p. 209--116. Google Scholar
Digital Library
- Benko, H., Wilson, A., and Balakrishnan, R. (2008). Sphere: Multi-Touch Interactions on a Spherical Display. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). p. 77--86. Google ScholarDigital Library
- Benko, H., Wilson, A. (2009). DepthTouch: Using Depth-Sensing Camera to Enable Freehand Interactions On and Above the Interactive Surface. Microsoft Research Technical Report MSR-TR-2009-23. March, 2009.Google Scholar
- Bowman, D.A., Kruijff, E., LaViola, J.J., and Poupyrev, I. (2004). 3D User Interfaces: Theory and Practice. Addison-Wesley, Boston. Google ScholarDigital Library
- Butz, A., Höllerer, T., Feiner, S., MacIntyre, B., and Beshers, C. (1999). Enveloping users and Computers in a Collaborative 3D Augmented Reality. In Proceedings International Workshop on Augmented Reality (IWAR). p. 35--44. Google ScholarDigital Library
- Cassinelli, A. and Ishikawa, M. (2005). Khronos projector. In ACM SIGGRAPH 2005 Emerging Technologies. p. 10. Google ScholarDigital Library
- Companje, R., van Dijk, N., Hogenbirk, H. and Mast, D. (2007). Globe4D, Time-Traveling with an Interactive Four-Dimensional Globe. In Proceedings of ACM Conference on Multimedia (MULTIMEDIA). p. 959--960. Google ScholarDigital Library
- Chen, Y., Au, J., Kazlas, P., Ritenour, A., Gates, H. and McCreary, M. (2003). Flexible Active-Matrix Electronic Ink Display. Nature. 423. p. 136.Google Scholar
- Cruz-Neira,C., Sandin, D.J., DeFanti, T.A., Kenyon, R.V., and Hart, J.C. (1992). The CAVE: Audio Visual Experience Automatic Virtual Environment. Communications of the ACM. 35, 6. (June 1992). p. 65--72. Google ScholarDigital Library
- Cutler, L.D., Fröhlich, B., and Hanrahan, P. (1997). Two-handed Direct Manipulation on the Responsive Workbench. In Proceedings of the ACM Symposium on Interactive 3D Graphics (I3D). p. 107--114. Google ScholarDigital Library
- Dietz, P. and Leigh, D. (2001). DiamondTouch: A Multi-User Touch Technology. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). p. 219--226. Google ScholarDigital Library
- Fitzmaurice, G., Khan, A., Buxton, W., Kurtenbach, G., and Balakrishnan, R. (2003). Sentient data access via a diverse society of devices. ACM Queue. p. 53--62. Google ScholarDigital Library
- Grossman, T. and Wigdor, D. (2007). Going Deeper: a Taxonomy of 3D on the Tabletop. In Proceedings of the IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP). p.137--144,Google ScholarCross Ref
- Grossman, T., Wigdor, D. and Balakrishnan, R. (2004). Multi-Finger Gestural Interaction with 3D Volumetric Displays. In Proceedings of ACM Symposium on User Interface Software and Technology (UIST). p. 61--70. Google ScholarDigital Library
- Han, J. (2005). Low-cost multi-touch sensing through frustrated total internal reflection. In Proceedings of ACM Symposium on User Interface Software and Technology (UIST). p. 115--118. Google ScholarDigital Library
- Holman, D. and Vertegaal, R. (2008). Organic user interfaces: designing computers in any way, shape, or form. Communications of the ACM 51, 6 (Jun. 2008), p. 48--55. Google ScholarDigital Library
- Hua, H., Brown, L. D., Gao, C., and Ahuja, N. (2003). A New Collaborative Infrastructure: SCAPE. In Proceedings of IEEE Virtual Reality (VR). p. 171--179. Google ScholarDigital Library
- Iddan, G. J. and Yahav, G. (2001). 3D Imaging in the Studio. SPIE, vol. 4298. p. 48.Google Scholar
- Izadi, S., Hodges, S., Taylor, S., Rosenfeld, D., Villar, N., Butler, A., and Westhues, J. (2008). Going beyond the display: a surface technology with an electronically switchable diffuser. In Proceedings of ACM Symposium on User interface Software and Technology (UIST). p. 269--278. Google ScholarDigital Library
- Kettner, S., Madden, C. and Ziegler, R. (2004). Direct Rotational Interaction with a Spherical Projection. In Proceedings of Creativity&Cognition Symposium on Interaction: Systems, Practice and Theory.Google Scholar
- Krueger, M. W., Gionfriddo, T., and Hinrichsen, K. (1985). VIDEOPLACE - an artificial reality. SIGCHI Bulletin 16(4) (Apr. 1985). p. 35--40. Google ScholarDigital Library
- Kruger, R., Carpendale, S., Scott, S. and Greenberg, S. (2003). How People Use Orientation on Tables: Comprehension, Coordination and Communication. In Proceedings of ACM Conference on Supporting Group Work (SIGGROUP). p. 369--378. Google ScholarDigital Library
- Matsushita, N. and Rekimoto, J. (1997). HoloWall: Designing a Finger, Hand, Body, and Object Sensitive Wall. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). p. 209--210. Google ScholarDigital Library
- Pinhanez, C. S. (2001) The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. In Proceedings of the International Conference on Ubiquitous Computing (UBICOMP). p. 315--331. Google ScholarDigital Library
- Piper, B., Ratti, C., and Ishii, H. (2002) Illuminating Clay: A 3-D Tangible Interface for Landscape Analysis. In Proceedings of Conference on Human Factors in Computing Systems (CHI). p. 355--362. Google ScholarDigital Library
- Poupyrev, I., Newton-Dunn, H., Bau, O. (2006). D20: Interaction with Multifaceted Display Devices. In Proceedings of Conference on Human Factors in Computing Systems (CHI). Extended Abstracts. p. 1241--1246. Google ScholarDigital Library
- Rekimoto, J. and Saitoh, M. (1999). Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments. In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI). p. 378--385. Google ScholarDigital Library
- Rekimoto, J. (2002). SmartSkin: An Infrastructure for Free-hand Manipulation on Interactive Surfaces. In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI). p. 113--120. Google ScholarDigital Library
- Shen, C., Vernier, F., Forlines, C. and Ringel, M. (2004). DiamondSpin: An Extensible Toolkit for Around-the-Table Interaction. In Proceedings of ACM Conference on Human Factors in Computing Systems (CHI). p. 167--174. Google ScholarDigital Library
- Starner, T., Leibe, B., Minnen, D., Westeyn, T., Hurst, A., and Weeks, J. (2003). The Perceptive Workbench: Computer-Vision-Based Gesture Tracking, Object Tracking, and 3D Reconstruction for Augmented Desks," Machine Vision and Applications, vol. 14, p. 51--71.Google ScholarDigital Library
- Taylor, B. T. and Bove, V. M. (2009). Graspables: grasp-recognition as a user interface. In Proceedings of Conference on Human Factors in Computing Systems (CHI). p. 917--926. Google ScholarDigital Library
- Weiser, M. (1991). The computer for the 21st century. Scientific American. 265, 3. December 11, 1991. p. 94--104.Google ScholarCross Ref
- Wellner, P. (1993). Interacting with paper on the DigitalDesk. Communications of the ACM. 36, 7 (Jul. 1993). p. 87--96. Google ScholarDigital Library
- Wilson, A. (2004). TouchLight: An Imaging Touch Screen and Display for Gesture-Based Interaction. In Proceedings of the Conference on Multimodal Interfaces (ICMI). p. 69--76. Google ScholarDigital Library
- Wilson, A. (2005). PlayAnywhere: A Compact Tabletop Computer Vision System. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). p. 83--92. Google ScholarDigital Library
- Wilson, A. (2006) Robust Computer Vision-Based Detection of Pinching for One and Two-Handed Gesture Input. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). p. 255--258. Google ScholarDigital Library
- Wilson, A. (2007) Depth-Sensing Video Cameras for 3D Tangible Tabletop Interaction. In Proceedings of the IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP). p. 201--204.Google ScholarCross Ref
- Wilson, A. D., Izadi, S., Hilliges, O., Garcia-Mendoza, A., and Kirk, D. 2008. Bringing physics to the surface. In Proceedings of ACM Symposium on User interface Software and Technology (UIST). p. 67--76. Google ScholarDigital Library
- Wu, M. and Balakrishnan, R. (2003). Multi-Finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). p. 193--202. Google ScholarDigital Library
Index Terms
- Beyond flat surface computing: challenges of depth-aware and curved interfaces
Recommendations
Interactions in the air: adding further depth to interactive tabletops
UIST '09: Proceedings of the 22nd annual ACM symposium on User interface software and technologyAlthough interactive surfaces have many unique and compelling qualities, the interactions they support are by their very nature bound to the display surface. In this paper we present a technique for users to seamlessly switch between interacting on the ...
Poker surface: combining a multi-touch table and mobile phones in interactive card games
MobileHCI '09: Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and ServicesIn this paper, we introduce the design and implementation of a digital card game that combines mobile phone gestures and multi-touch tabletop interactions. Conceptually based on interactions in traditional card games, our Poker game application explores ...
Multitouch and surface computing
CHI EA '09: CHI '09 Extended Abstracts on Human Factors in Computing SystemsNatural user interfaces (NUI) such as multitouch and surface computing are positioned as the next major evolution in computing and user interfaces. Just graphical user interfaces (GUIs) brought unprecedented interaction capabilities to their command-...
Comments