Peter Gyory
Ellen Yi-Luen Do
ABSTRACT
We present Beholder, a computer vision (CV) toolkit for building tangible controllers for interactive computer systems. Beholder facilitates designers to build physical inputs that are instrumented with CV markers. By observing the properties of these markers, a CV system can detect physical interactions that occur. Beholder provides a software editor that enables designers to map CV marker behavior to keyboard events; thus connecting the CV-driven tangible controllers to any software that responds to keyboard input. We propose three design scenarios for Beholder—controllers to support everyday work, alternative controllers for games, and transforming physical therapy equipment into controllers to monitor patient progress.
Supplemental Material
- S. Garrido-Jurado, R. Muñoz-Salinas, F.J. Madrid-Cuevas, and M.J. Marín-Jiménez. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47, 6 (2014), 2280–2292. https://doi.org/10.1016/j.patcog.2014.01.005Google Scholar
Digital Library
- Peter Gyory, Perry Owens, Matthew Bethancourt, Amy Banic, Clement Zheng, and Ellen Yi-Luen Do. 2022. Build Your Own Arcade Machine with Tinycade. In Creativity and Cognition. ACM, Venice Italy, 312–322. https://doi.org/10.1145/3527927.3533023Google ScholarDigital Library
- Sergi Jordà, Günter Geiger, Marcos Alonso, and Martin Kaltenbrunner. 2007. The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction (2007-02-15) (TEI ’07). Association for Computing Machinery, New York, NY, USA, 139–146. https://doi.org/10.1145/1226969.1226998Google ScholarDigital Library
- Martin Kaltenbrunner and Ross Bencina. 2007. reacTIVision: a computer-vision framework for table-based tangible interaction. In Proceedings of the 1st international conference on Tangible and embedded interaction (2007-02-15) (TEI ’07). Association for Computing Machinery, New York, NY, USA, 69–74. https://doi.org/10.1145/1226969.1226983Google ScholarDigital Library
- Mitchel Resnick and Eric Rosenbaum. 2013. Designing for Tinkerability. In Design, Make, Play. Routledge, Abingdon, Oxfordshire, UK.Google Scholar
- Valkyrie Savage, Colin Chang, and Björn Hartmann. 2013. Sauron: embedded single-camera sensing of printed physical user interfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology (2013-10-08) (UIST ’13). Association for Computing Machinery, New York, NY, USA, 447–456. https://doi.org/10.1145/2501988.2501992Google ScholarDigital Library
- Clement Zheng, Peter Gyory, and Ellen Yi-Luen Do. 2020. Tangible Interfaces with Printed Paper Markers. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (2020-07-03) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 909–923. https://doi.org/10.1145/3357236.3395578Google ScholarDigital Library
Index Terms
- Directing Tangible Controllers with Computer Vision and Beholder
Recommendations
Tangible comics: a performance space with full-body interaction
ACE '07: Proceedings of the international conference on Advances in computer entertainment technologyTangible Comics is a computer vision based full-body interactive storytelling environment that also functions as a comics generator. Prevailing applications of full-body computer vision have not utilized the full storytelling or performance potential of ...
Robust computer vision-based detection of pinching for one and two-handed gesture input
UIST '06: Proceedings of the 19th annual ACM symposium on User interface software and technologyWe present a computer vision technique to detect when the user brings their thumb and forefinger together (a pinch gesture) for close-range and relatively controlled viewing circumstances. The technique avoids complex and fragile hand tracking ...
Computer Vision for Interactive Computer Graphics
Computers looking through a camera at people is a potentially powerful technique to facilitate human-computer interaction. The computer can interpret the user's movements, gestures, and glances. Fundamental visual algorithms include tracking, shape ...
Comments