Otto Korkalo
ABSTRACT
We applied a computer vision based method and developed multi-touch technology to be adopted in various display types. In the selected design, multiple cameras are placed on the side of the display with their optical axis parallel to the screen. The display edges opposite the cameras are illuminated and fingers are detected since they block the light in the camera images. The approach is scalable and can be used in a wide variety of displays. Due to self-occlusion of the touchpoints, it is challenging to relate camera measurements to tracked points. In this paper, we present our approach for tracking and managing multiple touchpoints in such camera set-ups. We describe the mathematical background for modeling and calibrating the cameras, the design of the extended Kalman filter for point tracking, and the logic for adding, updating and removing the touchpoints. We analyze the potential accuracy and robustness of the system using several simulations and present two different real-life implementations of the approach.
Supplemental Material
- Shai Avidan and Ariel Shamir. Seam carving for content-aware image resizing. ACM Transactions on Graphics, 26(3):10, 2007. Google Scholar
Digital Library
- Yaakov Barshalom, Rong X. Li, and Thiagalingam Kirubarajan. Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software (A Wiley-Interscience publication). WileyBlackwell, July 2001. Google ScholarDigital Library
- François Bérard and Yann Laurillau. Single user multi-touch on the DiamondTouch: from 2 x 1D to 2D. In ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, pages 1--8, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- Rong Chang, Feng Wang, and Pengfei You. A survey on the development of multi-touch technology. In AP-WCS 2010: Proceedings of the Asia-Pacific Conference on Wearable Computing Systems, volume 0, pages 363--366, Los Alamitos, CA, USA, 2010. IEEE Computer Society. Google ScholarDigital Library
- Florian Echtler, Andreas Dippon, Marcus Tönnis, and Gudrun Klinker. Inverted FTIR: easy multitouch sensing for flatscreens. In ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, pages 29--32, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- Thomas E. Fortmann, Yaakov Bar-Shalom, and Molly Scheffe. Multi-target tracking using joint probabilistic data association. In Proceedings of the 19th IEEE Conference on Decision and Control, volume 19, pages 807--812, 1980.Google ScholarCross Ref
- Jefferson Y. Han. Low-cost multi-touch sensing through frustrated total internal reflection. In UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology, pages 115--118, New York, NY, USA, 2005. ACM. Google ScholarDigital Library
- Juho Kannala and Sami S. Brandt. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8):1335--1340, 2006. Google ScholarDigital Library
- Ian Maxwell. An overview of optical-touch technologies. Information Display, 23(12), 2007.Google Scholar
- Gerald Morrison and David Holmgren. Toward a touch-sensitive display wall. SID Symposium Digest of Technical Papers, 34(1):1458--1461, 2003.Google ScholarCross Ref
- NASA. The sensor frame graphic manipulator, final report. Technical Report NASA-CR-194243, NASA, 1992.Google Scholar
- D. Reid. An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control, 24(6):843--854, 1979.Google ScholarCross Ref
- Greg Welch and Gary Bishop. SCAAT: Incremental tracking with incomplete information. Computer Graphics, 31(Annual Conference Series):333--344, 1997. Google ScholarDigital Library
- Greg Welch and Gary Bishop. An introduction to the Kalman filter. Technical report, Chapel Hill, NC, USA, 2006.Google Scholar
- Andrew D. Wilson. Touchlight: an imaging touch screen and display for gesture-based interaction. In ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces, pages 69--76, New York, NY, USA, 2004. ACM. Google ScholarDigital Library
Index Terms
- Construction and evaluation of multi-touch screens using multiple cameras located on the side of the display
Recommendations
Robust Relocalization and Its Evaluation for Online Environment Map Construction
The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation ...
Object tracking in the presence of occlusions using multiple cameras: A sensor network approach
This article describes a sensor network approach to tracking a single object in the presence of static and moving occluders using a network of cameras. To conserve communication bandwidth and energy, we combine a task-driven approach with camera subset ...
Multi-touch wall display system using multiple laser range scanners
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and SurfacesIn this work we present a multi-touch wall display system that equips easily and considers occlusions problem using multiple range scanners. Our system is implemented on an existing large display embedded in a wall with two laser range scanners. Each ...
Comments