Hostname: page-component-6bf8c574d5-pdxrj Total loading time: 0 Render date: 2025-03-07T00:50:04.409Z Has data issue: false hasContentIssue false
  • English
  • Français

RATA.Gesture: A gesture recognizer developed using data mining

Published online by Cambridge University Press:  14 August 2012

Samuel Hsiao-Heng Chang ,
Rachel Blagojevic  and
Beryl Plimmer
Samuel Hsiao-Heng Chang
Affiliation:
Department of Computer Science, University of Auckland, Auckland, New Zealand
Rachel Blagojevic
Affiliation:
Department of Computer Science, University of Auckland, Auckland, New Zealand
Beryl Plimmer*
Affiliation:
Department of Computer Science, University of Auckland, Auckland, New Zealand
*
Reprint requests to: Beryl Plimmer, Department of Computer Science, University of Auckland, Private Bag 92019, Auckland 1142, New Zealand. E-mail: beryl@cs.auckland.ac.nz
Get access Rights & Permissions [Opens in a new window]

Abstract

Although many approaches to digital ink recognition have been proposed, most lack the flexibility and adaptability to provide acceptable recognition rates across a variety of problem spaces. This project uses a systematic approach of data mining analysis to build a gesture recognizer for sketched diagrams. A wide range of algorithms was tested, and those with the best performance were chosen for further tuning and analysis. Our resulting recognizer, RATA.Gesture, is an ensemble of four algorithms. We evaluated it against four popular gesture recognizers with three data sets; one of our own and two from other projects. Except for recognizer–data set pairs (e.g., PaleoSketch recognizer and PaleoSketch data set) the results show that it outperforms the other recognizers. This demonstrates the potential of this approach to produce flexible and accurate recognizers.

Keywords

Pen-Based InterfacesRecognition AlgorithmsSketch RecognitionSketch Tools
Type
Special Issue Articles
Information
AI EDAM , Volume 26 , Issue 3: Sketching and Pen-Based Design Interaction , August 2012 , pp. 351 - 366
Copyright
Copyright © Cambridge University Press 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Alimoglu, F., & Alpaydin, E. (2001). Combining multiple representations and classifiers for pen-based handwritten digit recognition. Turkish Journal of Electrical Engineering and Computer Sciences 9(1), 1–12.Google Scholar
Alvarado, C., & Davis, R. (2001). Resolving ambiguities to create a natural computer-based sketching environment. Proc. IJCAI-01, pp. 1365–1374.Google Scholar
Apte, A., Vo, V., & Kimura, T.D. (1993). Recognizing multistroke geometric shapes: an experimental evaluation. Proc. 6th Annual ACM Symp. User Interface Software and Technology, pp. 121–128.CrossRefGoogle Scholar
Basili, R., Serafini, A., & Stellato, A. (2004). Classification of musical genre: a machine learning approach. 5th Int. Conf. Music Information Retrieval (ISMIR'04), Barcelona.Google Scholar
Ben-Gal, I. (2007). Bayesian networks. In Encyclopedia of Statistics in Quality and Reliability (Ruggeri, F., Faltin, F., & Kenett, R., Eds.). Hoboken, NJ: Wiley.Google Scholar
Blagojevic, R. (2011). Using data mining for digital ink recognition. PhD Thesis, University of Auckland.Google Scholar
Blagojevic, R., Chang, S.H.-H., & Plimmer, B. (2010). The power of automatic feature selection: Rubine on steroids. Proc. Eurographics 2010, Sketch Based Interfaces and Modeling, pp. 79–86, Annecy, France.Google Scholar
Blagojevic, R., Plimmer, B., Grundy, J., & Wang, Y. (2008). A data collection tool for sketched diagrams. Proc. Eurographics 2010, Sketch Based Interfaces and Modeling, pp. 73–80, Annecy, France.Google Scholar
Breiman, L. (1996). Bagging predictors. Machine Learning 24(2), 123–140.CrossRefGoogle Scholar
Breiman, L. (2001). Random forests. Machine Learning 45(1), 5–32.CrossRefGoogle Scholar
Calhoun, C., Stahovich, T.F., Kurtoglu, T., & Kara, L.B. (2002). Recognizing multi-stroke symbols. AAAI Spring Symp., Sketch Understanding, pp. 15–23.Google Scholar
Chang, S.H.-H., Plimmer, B., & Blagojevic, R. (2010). Rata.SSR: data mining for pertinent stroke recognizers. Proc. Eurographics 2010, Sketch Based Interfaces and Modeling, pp. 95–102, Annecy, France.Google Scholar
Connell, S.D., Sinha, R.M.K., & Jain, A.K. (2000). Recognition of unconstrained on-line Devanagari characters. Proc. 15th ICPR, pp. 368–371.Google Scholar
Dong, L., Frank, E., & Kramer, S. (2005). Ensembles of balanced nested dichotomies for multi-class problems. Knowledge Discovery in Databases: PKDD 2005, pp. 84–95.CrossRefGoogle Scholar
Field, M., Gordon, S., Peterson, E., Robinson, R., Stahovich, T., et al. (2009). The effect of task on classification accuracy: using gesture recognition techniques in free-sketch recognition. CAD/GRAPHICS 2009, pp. 499–512.Google Scholar
Fonseca, M.J., Pimentel, C.E., & Jorge, J.A. (2002). CALI: an online scribble recogniser for calligraphic interfaces. AAAI Spring Symp. Sketch Understanding, pp. 51–58. New York: IEEE.Google Scholar
Frank, E., & Kramer, S. (2004). Ensembles of nested dichotomies for multi-class problems. Proc. 21st Int. Conf. Machine Learning, Banff, AB, Canada.CrossRefGoogle Scholar
Freeman, I., & Plimmer, B. (2007). Connector semantics for sketched diagram recognition. AUIC, pp. 71–78, Ballarat, Australia.Google Scholar
Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Annals of Statistics 28(2), 337–407.CrossRefGoogle Scholar
Fu, L., & Kara, L.B. (2011). From engineering diagrams to engineering models: visual recognition and applications. Computer-Aided Design 43(3), 278–292.CrossRefGoogle Scholar
Gross, M. (1994). Recognizing and interpreting diagrams in design. AVI 94, pp. 88–94, Bari, Italy.CrossRefGoogle Scholar
Hammond, T., Eoff, B., Paulson, B., Wolin, A., Dahmen, K., et al. (2008). Free-sketch recognition: putting the CHI in sketching. 26th Annual SIGCHI Conf. Human Factors in Computing Systems (CHI 2008) Works in Progress, pp. 3027–3032, Florence, Italy.CrossRefGoogle Scholar
Hastie, T., & Tibshirani, R. (1998). Classification by pairwise coupling. Advances in Neural Information Processing Systems, pp. 507–513, Denver, CO.Google Scholar
Holmes, G., Pfahringer, B., Kirkby, R., Frank, E., & Hall, M. (2002). Multiclass alternating decision trees. Machine Learning: ECML 2002, pp. 105–122.Google Scholar
Johnson, G., Gross, M.D., Hong, J., & Do, E.Y.-L. (2009). Computational support for sketching in design: a review. Foundations and Trends in Human–Computer Interaction 2(1), 1–93.CrossRefGoogle Scholar
Kara, L.B., & Stahovich, T.F. (2004). Hierarchical parsing and recognition of handsketched diagrams. UIST '04, pp. 13–22, Santa Fe, NM.CrossRefGoogle Scholar
Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., & Murthy, K.R.K. (2001). Improvements to Platt's SMO algorithm for SVM classifier design. Neural Computation 13(3), 637–649.CrossRefGoogle Scholar
Kohavi, R., & John, G.H. (1997). Wrappers for feature subset selection. Artificial Intelligence 97(1–2), 273–324.CrossRefGoogle Scholar
Landwehr, N., Hall, M., & Frank, E. (2005). Logistic model trees. Machine Learning 59(1–2), 161–205.CrossRefGoogle Scholar
LaViola, J.J. Jr., & Zeleznik, R.C. (2004). MathPad2: a system for the creation and exploration of mathematical sketches. ACM Transactions in Graphics 23(3), 432–440.CrossRefGoogle Scholar
Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., & Euler, T. (2006). YALE: rapid prototyping for complex data mining tasks. 12th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (KDD '06), pp. 935–940, New York.CrossRefGoogle Scholar
Minsky, M., & Papert, S. (1969). Perceptrons. Cambridge, MA: MIT Press.Google Scholar
Ouyang, T.Y., & Davis, R. (2009). A visual approach to sketched symbol recognition. Proc. 21st Int. Joint Conf. Artificial Intelligence, pp. 1463–1468.Google Scholar
Patel, R., Plimmer, B., Grundy, J., & Ihaka, R. (2007). Ink features for diagram recognition. Eurographics 2007, 4th Eurographics Workshop on Sketch-Based Interfaces and Modeling, pp. 131–138, Riverside, CA.CrossRefGoogle Scholar
Paulson, B., & Hammond, T. (2008). PaleoSketch: accurate primitive sketch recognition and beautification. Intelligent User Interfaces (IUI ‘08), pp. 1–10, New York.Google Scholar
Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods—Support Vector Learning, pp. 185–208. Cambridge, MA: MIT Press.Google Scholar
Plimmer, B., & Freeman, I. (2007). A toolkit approach to sketched diagram recognition. HCI, eWiC, pp. 205–213, Lancaster, UK.Google Scholar
Rubine, D.H. (1991). Specifying gestures by example. Proc. Siggraph '91, pp. 329–337.CrossRefGoogle Scholar
Rumelhart, D.E., Hinton, G.E., & Williams, R.J. (1986). Learning Internal Representations by Error Propagation. Cambridge, MA: MIT Press.Google Scholar
Schmieder, P. (2009). Comparing basic shape classifiers: a platform for evaluating sketch recognition algorithms. MS Thesis, University of Auckland.Google Scholar
Schmieder, P., Plimmer, B., & Blagojevic, R. (2009). Automatic evaluation of sketch recognizers. SBIM '09, Sketch Based Interfaces and Modelling, pp. 85–92, New Orleans.Google Scholar
Sezgin, T.M., & Davis, R. (2007). Sketch interpretation using multiscale models of temporal patterns. IEEE Computer Graphics and Applications 27(1), 28–37.CrossRefGoogle ScholarPubMed
Sezgin, T.M., Stahovich, T., & Davis, R. (2001). Sketch based interfaces: early processing for sketch understanding. Proc. 2001 Workshop on Perceptive User Interfaces, pp. 1–8, Orlando, FL.CrossRefGoogle Scholar
Sumner, M., Frank, E., & Hall, M. (2005). Speeding up logistic model tree induction. 9th European Conf. Principles and Practice of Knowledge Discovery in Databases, pp. 675–683, Porto, Portugal.CrossRefGoogle Scholar
Tay, K.S. (2008). Improving digital ink interpretation through expected type prediction and dynamic dispatch. Pattern Recognition (ICPR), pp. 1–4.Google Scholar
Vogt, T., & Andre, E. (2005). Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition. Multimedia and Expo (ICME), pp. 474–477.Google Scholar
Willems, D., Niels, R., Gerven, M.v., & Vuurpijl, L. (2009). Iconic and multi-stroke gesture recognition. Pattern Recognition 42(12), 3303–3312.CrossRefGoogle Scholar
Witten, I.H., & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques. San Francisco, CA: Morgan Kaufmann.Google Scholar
Wobbrock, J.O., Wilson, A.D., & Li, Y. (2007). Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. User Interface Software and Technology, pp. 159–168. Newport, RI: ACM.Google Scholar
Wobbrock, J.O., Wilson, A.D., & Li, Y. (2009). $1 Unistroke Recognizer. Accessed at http://depts.washington.edu/aimgroup/proj/dollar/Google Scholar
Yu, B., & Cai, S. (2003). A domain-independent system for sketch recognition. Proc. 1st Int. Conf. Computer Graphics and Interactive Techniques in Australasia and South East Asia, pp. 141–146, Melbourne, Australia.CrossRefGoogle Scholar