Abstract
This paper presents two novel handheld projector systems for indoor pervasive computing spaces. These projection-based devices are “aware” of their environment in ways not demonstrated previously. They offer both spatial awareness, where the system infers location and orientation of the device in 3D space, and geometry awareness, where the system constructs the 3D structure of the world around it, which can encompass the user as well as other physical objects, such as furniture and walls. Previous work in this area has predominantly focused on infrastructure-based spatial-aware handheld projection and interaction. Our prototypes offer greater levels of environment awareness, but achieve this using two opposing approaches; the first infrastructure-based and the other infrastructure-less sensing. We highlight a series of interactions including direct touch, as well as in-air gestures, which leverage the shadow of the user for interaction. We describe the technical challenges in realizing these novel systems; and compare them directly by quantifying their location tracking and input sensing capabilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Beardsley, P., Baar, J.V., Raskar, R., Forlines, C.: Interaction using a handheld projector. IEEE Computer Graphics and Applications 25(1), 39–43 (2005)
Brumitt, B., Meyers, B., Krumm, J., Kern, A., Shafer, S.: EasyLiving: Technologies for Intelligent Environments. In: Thomas, P., Gellersen, H.-W. (eds.) HUC 2000. LNCS, vol. 1927, pp. 12–27. Springer, Heidelberg (2000)
Cao, X., Balakrishnan, R.: Interacting with dynamically defined information spaces using a handheld projector and pen. In: Proc. of ACM UIST 2006, pp. 225–234 (2006)
Cao, X., Forlines, C., Balakrishnan, R.: Multi-user interaction using handheld projectors. In: Proc. ACM UIST 2007, pp. 43–52 (2007)
Cowan, L., Li, K.: ShadowPuppets: collocated interaction with mobile projector phones using hand shadows. In: Proc: ACM CHI 2011, pp. 2707–2716 (2011)
Du, H., et al.: Interactive 3D modeling of indoor environments with a consumer depth camera. In: Proc. Ubiquitous Computing (UbiComp 2011), pp. 75–84. ACM (2011)
Ehnes, J., Hirota, K., Hirose, M.: Projected augmentation - AR using rotatable video projectors. In: Proc: 3rd IEEE and ACM ISMAR, pp. 26–35 (2004)
Harrison, C., Benko, H., Wilson, A.D.: OmniTouch: Wearable Multitouch Interaction Everywhere. In: Proc. ACM UIST 2011 (2011)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003)
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., et al.: KinectFusion: RealTime Interactions with Dynamic 3D Surface Reconstructions. In: Proc. ACM UIST 2011 (2011)
Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proc. Geometry Processing (SGP 2006), Switzerland, pp. 61–70 (2006)
Kidd, C.D., et al.: The Aware Home: A Living Laboratory for Ubiquitous Computing Research. In: Yuan, F., Hartkopf, V. (eds.) CoBuild 1999. LNCS, vol. 1670, pp. 191–198. Springer, Heidelberg (1999)
Krueger, M.: Artificial Reality 2. Addison-Wesley Professional (1991)
Mistry, P., Maes, P., Chang, L.: WUW - Wear Ur World - A Wearable Gestural Interface. In: ACM CHI 2009 Extended Abstracts (2009)
Molyneaux, D., Gellersen, H., Kortuem, G., Schiele, B.: Cooperative Augmentation of Smart Objects with Projector-Camera Systems. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 501–518. Springer, Heidelberg (2007)
Naemura, T., Nitta, T., Mimura, A., Harashima, H.: Virtual Shadows- Enhanced Interaction in Mixed Reality Environment. In: IEEE VR 2002, p. 293 (2002)
Raskar, R., et al.: The office of the future: a unified approach to image-based modeling and spatially immersive displays. In: Proc. SIGGRAPH 1998, pp. 179–188 (1998)
Raskar, R., VanBaar, J., Beardsley, P., et al.: iLamps: geometrically aware and self-configuring projectors. ACM Trans. Graph. 22(3), 809–818 (2003)
Raskar, R., et al.: RFIG Lamps: interacting with a self-describing world via photosensing wireless tags and projectors. ACM ToG 23(3), 406–415
Pinhanez, C.: The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. In: Abowd, G.D., Brumitt, B., Shafer, S. (eds.) UbiComp 2001. LNCS, vol. 2201, pp. 315–331. Springer, Heidelberg (2001)
Sato, Y., et al.: Fast tracking of hands and fingertips in infrared images for augmented desk interface. In: Proc: AFGR. IEEE (2000)
Shotton, J., et al.: Real-time human pose recognition in parts from single depth images. In: Proc. CVPR 2011, pp. 1297–1304 (2011)
Sugimoto, M., Miyahara, K., Inoue, H., Tsunesada, Y.: Hotaru: Intuitive Manipulation Techniques for Projected Displays of Mobile Devices. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 57–68. Springer, Heidelberg (2005)
Teller, S., Chen, J., Balakrishnan, H.: Pervasive pose-aware applications and Infrastructure. IEEE CG&A 23(4), 14–18 (2003)
Weiser, M.: The Computer for the Twenty-First Century. Scientific American 265(3), 94–100 (1991)
Willis, K.D.D., Poupyrev, I., Shiratori, T.: Motionbeam: Character interaction with handheld projectors. In: Proc: ACM CHI 2011, pp. 1031–1040 (2011)
Willis, K.D.D., Poupyrev, I., et al.: SideBySide: ad-hoc multi-user interaction with handheld projectors. In: Proc. ACM UIST 2011, pp. 431–440 (2011)
Wilson, A., Izadi, D., Hilliges, O., et al.: Bringing physics to the surface. In: Proc: ACM UIST 2008, pp. 67–76 (2008)
Wilson, A., Benko, H.: Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In: Proc: ACM UIST 2010, pp. 273–282 (2010)
Xu, H., Iwai, D., Hiura, S., Sato, K.: User Interface by Virtual Shadow Projection. In: Proc: SICE-ICASE, pp. 4817–4818 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Molyneaux, D. et al. (2012). Interactive Environment-Aware Handheld Projectors for Pervasive Computing Spaces. In: Kay, J., Lukowicz, P., Tokuda, H., Olivier, P., Krüger, A. (eds) Pervasive Computing. Pervasive 2012. Lecture Notes in Computer Science, vol 7319. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31205-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-642-31205-2_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31204-5
Online ISBN: 978-3-642-31205-2
eBook Packages: Computer ScienceComputer Science (R0)