Yosra Rekik
Radu-Daniel Vatavu
Laurent Grisoni
ABSTRACT
We present a simple, two-step technique for recognizing multi-touch gesture input that is invariant to how users articulate gestures, i.e., by using one or two hands, one or multiple fingers, one or multiple strokes, synchronous or asynchronous stroke input. We introduce, for the first time in the gesture literature, a preprocessing step that is specific to multi-touch gestures (Match-Up) that clusters together similar strokes produced by different fingers, before running a gesture recognizer (Conquer). We report gains in recognition accuracy up to 10% leveraged by our new preprocessing step, which manages to construct a more adequate representation of multi-touch gestures in terms of key strokes. It is our hope that the Match-Up technique will add to the practitioners' toolkit of gesture preprocessing techniques, as a first step toward filling today's lack of algorithmic knowledge to process multi-touch input and leading toward the design of more efficient and accurate recognizers for touch surfaces.
- L. Anthony, Q. Brown, B. Tate, J. Nias, R. Brewer, and G. Irwin. Designing smarter touch-based interfaces for educational contexts. Journal of Personal and Ubiquitous Computing, November 2013.Google Scholar
- L. Anthony, R.-D. Vatavu, and J. O. Wobbrock. Understanding the consistency of users' pen and finger stroke gesture articulation. proceedings of graphics interface. In Proc. of GI '13, pages 87--94. Google ScholarDigital Library
- L. Anthony and J. O. Wobbrock. A lightweight multistroke recognizer for user interface prototypes. In Proc. of GI '10, pages 245--252. Canadian Information Processing Society. Google ScholarDigital Library
- L. Anthony and J. O. Wobbrock. $N-Protractor: a fast and accurate multistroke recognizer. In Proc. of GI '12, pages 117--120. Google ScholarDigital Library
- G. Bailly, J. Müller, and E. Lecolinet. Design and evaluation of finger-count interaction: Combining multitouch gestures and menus. Int. J. Hum.-Comput. Stud., 70(10):673--689. Google ScholarDigital Library
- O. Bau and W. E. Mackay. Octopocus: a dynamic guide for learning gesture-based command sets. In Proc. of UIST '08, pages 37--46. Google ScholarDigital Library
- T. Cormen, C. Leiserson, R. Rivest, and C. Stein. Introduction to Algorithms, 2nd Ed. MIT Press. Google ScholarDigital Library
- N. Donmez and K. Singh. Concepture: a regular language based framework for recognizing gestures with varying and repetitive patterns. In Proc. of SBIM '12, pages 29--37. Google ScholarDigital Library
- D. Freeman, H. Benko, M. R. Morris, and D. Wigdor. Shadowguides: visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proc. of ITS '09, pages 165--172. ACM Press. Google ScholarDigital Library
- U. Hinrichs and S. Carpendale. Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits. In Proc. of CHI '11, pages 3023--3032. Google ScholarDigital Library
- Y. Jiang, F. Tian, X. Zhang, W. Liu, G. Dai, and H. Wang. Unistroke gestures on multi-touch interaction: supporting flexible touches with key stroke extraction. In Proc. of IUI '12, pages 85--88. ACM Press. Google ScholarDigital Library
- K. Kin, B. Hartmann, and M. Agrawala. Two-handed marking menus for multitouch devices. ACM Trans. Comput.-Hum. Interact., 18(3):16:1--16:23. Google ScholarDigital Library
- K. Kin, B. Hartmann, T. DeRose, and M. Agrawala. Proton: multitouch gestures as regular expressions. In Proc. of CHI '12, pages 2885--2894. ACM Press. Google ScholarDigital Library
- C. Kray, D. Nesbitt, J. Dawson, and M. Rohs. User-defined gestures for connecting mobile phones, public displays, and tabletops. In Proc. of MobileHCI '10, pages 239--248. Google ScholarDigital Library
- Y. Li. Protractor: A fast and accurate gesture recognizer. In Proc. of CHI '10, pages 2169--2172, New York, NY, USA, 2010. ACM Press. Google ScholarDigital Library
- H. Lü and Y. Li. Gesture coder: a tool for programming multi-touch gestures by demonstration. In Proc. of CHI '12, pages 2875--2884. Google ScholarDigital Library
- M. R. Morris, A. Huang, A. Paepcke, and T. Winograd. Cooperative gestures: Multi-user gestural interactions for co-located groupware. In Proc. of CHI '06, pages 1201--1210. ACM Press. Google ScholarDigital Library
- U. Oh and L. Findlater. The challenges and potential of end-user gesture customization. In Proc. of CHI '13, pages 1129--1138. Google ScholarDigital Library
- Y. Rekik, L. Grisoni, and N. Roussel. Towards many gestures to one command: A user study for tabletops. In Proc. of INTERACT '13. Springer-Verlag.Google Scholar
- M. Ringel, K. Ryall, C. Shen, C. Forlines, and F. Vernier. Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables. In CHI EA '04, pages 1441--1444. Google ScholarDigital Library
- D. Rubine. Specifying gestures by example. In Proc. of SIGGRAPH '91, pages 329--337. ACM Press. Google ScholarDigital Library
- J. Ruiz, Y. Li, and E. Lank. User-defined motion gestures for mobile interaction. In Proc. of CHI '11, pages 197--206. ACM Press. Google ScholarDigital Library
- T. M. Sezgin and R. Davis. HMM-based efficient sketch recognition. In Proc. of IUI '05, pages 281--283. ACM Press. Google ScholarDigital Library
- C. Sima and E. Dougherty. The peaking phenomenon in the presence of feature-selection. Pattern Recognition Letters, 29:1667--1674. Google ScholarDigital Library
- R.-D. Vatavu. The effect of sampling rate on the performance of template-based gesture recognizers. In Proc. of ICMI '11, pages 271--278. ACM Press. Google ScholarDigital Library
- R.-D. Vatavu. The impact of motion dimensionality and bit cardinality on the design of 3D gesture recognizers. Int. Journ. of Human-Computer Studies, 71(4):387--409. Google ScholarDigital Library
- R.-D. Vatavu, L. Anthony, and J. O. Wobbrock. Gestures as point clouds: a $P recognizer for user interface prototypes. In Proc. of ICMI '12, pages 273--280. ACM Press. Google ScholarDigital Library
- R.-D. Vatavu, D. Vogel, G. Casiez, and L. Grisoni. Estimating the perceived difficulty of pen gestures. In Proc. of INTERACT'11, pages 89--106. Springer-Verlag. Google ScholarDigital Library
- A. Webb. Statistical pattern recognition. John Wiley & Sons.Google Scholar
- J. O. Wobbrock, M. R. Morris, and A. D. Wilson. User-defined gestures for surface computing. In Proc. of CHI '09, pages 1083--1092. ACM Press. Google ScholarDigital Library
- J. O. Wobbrock, A. D. Wilson, and Y. Li. Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In Proc. of UIST '07, pages 159--168. ACM Press. Google ScholarDigital Library
- M. Wu, C. Shen, K. Ryall, C. Forlines, and R. Balakrishnan. Gesture registration, relaxation, and reuse for multi-point direct-touch surfaces. In Proc. of TABLETOP '06, pages 185--192. Google ScholarDigital Library
Index Terms
- Match-up & conquer: a two-step technique for recognizing unconstrained bimanual and multi-finger touch input
Recommendations
Smartphone's Based Gesture Recognition in Air
ICARCSET '15: Proceedings of the 2015 International Conference on Advanced Research in Computer Science Engineering & Technology (ICARCSET 2015)In the recent years, sensors within the mobile devices are the center of attraction for people. This paper proposes a system for gesture recognition. System uses in-built accelerometer in the mobile phone to recognize the gesture. Simply by holding a ...
Gestures à Go Go: Authoring Synthetic Human-Like Stroke Gestures Using the Kinematic Theory of Rapid Movements
Special Issue on Causal Discovery and InferenceTraining a high-quality gesture recognizer requires providing a large number of examples to enable good performance on unseen, future data. However, recruiting participants, data collection, and labeling, etc., necessary for achieving this goal are ...
G3: bootstrapping stroke gestures design with synthetic samples and built-in recognizers
MobileHCI '16: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services AdjunctStroke gestures are becoming increasingly important with the ongoing success of touchscreen-capable devices. However, training a high-quality gesture recognizer requires providing a large number of examples to enable good performance on unseen, future ...
Comments