Vitus Maierhöfer
Andreas Schmid
Raphael Wimmer
ABSTRACT
Interactive tabletops do not only offer a large surface for collaborative interaction. They also offer quick access to digital tools directly at the table - where a large number of everyday activities take place. Tabletops with an embedded display are generally less flexible and more fragile than ordinary massive tabletops. Physical objects on the tabletop occlude the digital content. In contrast, top-down-projected interfaces using an overhead projector-camera system allow for augmenting arbitrary tables and objects lying on them. However, detecting pointing input only via a camera image captured from above requires robustly recognizing whether a finger or pen touches the tabletop or whether it hovers slightly above it. In this demonstration, we showcase a solution for reliably tracking a pen on arbitrary tabletop surfaces. The pen emits infrared light via a tip made of optical fiber. A camera captures position and shape of the light point on the surface. Our open-source tracking algorithm combines heuristics and a neural network to distinguish between drawing and hovering. This system can be reliably used for drawing and writing on tabletops. However, occlusion by users’ hands can deteriorate tracking of the pen.
Supplemental Material
- G. Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools(2000).Google Scholar
- Rui Chen, Po-Jui (Ray) Chen, Rui Feng, Yilin (Elaine) Liu, Andy Wu, and Ali Mazalek. 2014. SciSketch: a tabletop collaborative sketching system. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction - TEI ’14. ACM Press, Munich, Germany, 247–250. https://doi.org/10.1145/2540930.2540973Google ScholarDigital Library
- Florian Echtler. 2018. SurfaceStreams: A Content-Agnostic Streaming Toolkit for Interactive Surfaces. In The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings - UIST ’18 Adjunct. ACM Press, Berlin, Germany, 10–12. https://doi.org/10.1145/3266037.3266085Google ScholarDigital Library
- Florian Echtler and Raphael Wimmer. 2014. The Interactive Dining Table, or Pass the Weather Widget, Please. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces - ITS ’14. ACM Press, Dresden, Germany, 419–422. https://doi.org/10.1145/2669485.2669525Google ScholarDigital Library
- Markus Funk, Sven Mayer, and Albrecht Schmidt. 2015. Using In-Situ Projection to Support Cognitively Impaired Workers at the Workplace. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS ’15. ACM Press, Lisbon, Portugal, 185–192. https://doi.org/10.1145/2700648.2809853Google ScholarDigital Library
- John Hardy. 2012. Experiences: a year in the life of an interactive desk. In Proceedings of the Designing Interactive Systems Conference(DIS ’12). Association for Computing Machinery, Newcastle Upon Tyne, United Kingdom, 679–688. https://doi.org/10.1145/2317956.2318058Google ScholarDigital Library
- Sergi Jordà, Günter Geiger, Marcos Alonso, and Martin Kaltenbrunner. 2007. The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st international conference on Tangible and embedded interaction(TEI ’07). Association for Computing Machinery, New York, NY, USA, 139–146. https://doi.org/10.1145/1226969.1226998Google ScholarDigital Library
- Sergi Jordà, Martin Kaltenbrunner, Günter Geiger, and Marcos Alonso. 2006. The reacTable: a tangible tabletop musical instrument and collaborative workbench. In ACM SIGGRAPH 2006 Sketches on - SIGGRAPH ’06. ACM Press, Boston, Massachusetts, 91. https://doi.org/10.1145/1179849.1179963Google ScholarDigital Library
- Thomas Kosch, Paweł W. Woźniak, Erin Brady, and Albrecht Schmidt. 2018. Smart Kitchens for People with Cognitive Impairments: A Qualitative Study of Design Requirements. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, Montreal QC, Canada, 1–12. https://doi.org/10.1145/3173574.3173845Google ScholarDigital Library
- Johnny Chung Lee. 2008. Hacking the Nintendo Wii Remote. IEEE Pervasive Computing 7, 3 (July 2008), 39–45. https://doi.org/10.1109/MPRV.2008.53Google ScholarDigital Library
- Kuk-Seon Lee, Sang-Heon Oh, Kuk-Hui Jeon, Seong-Soo Kang, Dong-Hee Ryu, and Byung-Gyu Kim. 2012. A Study on Smart Touch Projector System Technology Using Infrared (IR) Imaging Sensor. Journal of Korea Multimedia Society 15, 7 (July 2012), 870–878. https://doi.org/10.9717/KMMS.2012.15.7.870Google ScholarCross Ref
- Vitus Maierhöfer, Andreas Schmid, and Raphael Wimmer. 2022. Using an Infrared Pen as an Input Device for Projected Augmented Reality Tabletops. In Mensch und Computer 2022 – Workshopband, Marky, K., Grünefeld, U., and Kosch, T. (Eds.). Gesellschaft für Informatik e.V., Darmstadt. https://doi.org/10.18420/muc2022-mci-ws04-274Google Scholar
- G. Margetis, A. Ntelidakis, X. Zabulis, S. Ntoa, P. Koutlemanis, and C. Stephanidis. 2013. Augmenting physical books towards education enhancement. In 2013 1st IEEE Workshop on User-Centered Computer Vision (UCCV). 43–49. https://doi.org/10.1109/UCCV.2013.6530807Google ScholarCross Ref
- Raphael Wimmer. 2017. Some Thoughts on Ergonomic, Practical, and Economic Properties of Interactive Tabletops. Brighton, UK. https://thedisappearingworkshop.files.wordpress.com/2017/06/disappearingtabletop2017_paper_5.pdfGoogle Scholar
- Raphael Wimmer and Florian Echtler. 2019. VIGITIA: Unterstützung von alltäglichen Tätigkeiten an Tischen durch Projected AR. In Mensch und Computer 2019 - Workshopband. Gesellschaft für Informatik e.V., 6. https://doi.org/10.18420/muc2019-ws-620Google Scholar
- Qin Wu, Jiayuan Wang, Sirui Wang, Tong Su, and Chenmei Yu. 2019. MagicPAPER: tabletop interactive projection device based on tangible interaction. In ACM SIGGRAPH 2019 Posters on - SIGGRAPH ’19. ACM Press, Los Angeles, California, 1–2. https://doi.org/10.1145/3306214.3338575Google ScholarDigital Library
Recommendations
TipTrack: Precise, Low-Latency, Robust Optical Pen Tracking on Arbitrary Surfaces Using an IR-Emitting Pen Tip
TEI '24: Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied InteractionTables are focus points for social interactions and support everyday activities, such as learning, crafting, or dining. These physical interactions on and around the table may be augmented with digital information and tools projected onto the tabletop. ...
Joint 5D Pen Input for Light Field Displays
UIST '15: Proceedings of the 28th Annual ACM Symposium on User Interface Software & TechnologyLight field displays allow viewers to see view-dependent 3D content as if looking through a window; however, existing work on light field display interaction is limited. Yet, they have the potential to parallel 2D pen and touch screen systems, which ...
A Camera-Based Input Device for Large Interactive Displays
Humanýcomputer interaction using large-format displays is an active area of research that focuses on how humans can better work with computers or other machines. For this to happen, there must be an enabling technology that creates the interface between ...
Comments