Skip to main content
SpringerLink
Log in

Sky Writer: Towards an Intelligent Smart-phone Gesture Tracing and Recognition Framework

  • Conference paper
  • First Online:
Discovery Science (DS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11198))

Included in the following conference series:

  • 819 Accesses

Abstract

We present Sky Writer, an intelligent smartphone gesture tracking and recognition framework for free-form gestures. The design leverages anthropomorphic kinematics and device orientation to estimate the trajectory of complex gestures instead of employing traditional acceleration based techniques. Orientation data are transformed, using the kinematic model, to a 3D positional data stream, which is flattened, scaled down, and curve fitted to produce a gesture trace and a set of accompanying features for a support vector machine (SVM) classifier. SVM is the main classifier we adopted but for the sake of comparison, we couple our results with the hidden Markov models (HMM). In this experiment, a dataset of size 1200 is collected from 15 participants that performed 5 instances for each of 16 distinct custom developed gestures after being instructed on how to handle the device. User-dependent, user-independent, and hybrid/mixed learning scenarios are used to evaluate the proposed design. This custom developed gesture set achieved using SVM 96.55%, 96.1%, and 97.75% average recognition rates across all users for the respective learning scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Android developers, “Sensor Overview” http://developer.android.com/guide/topics/sensors/sensors_overview.html (2016). Accessed 5 Jan 2016

  • Amma, C., Georgi, M., Schultz, T.: Airwriting: hands-free mobile text input by spotting and continuous recognition of 3d-space handwriting with inertial sensors. In: 2012 16th International Symposium on Wearable Computers (ISWC), pp. 52–59 (2012). Accessed 18–22 June 2012

    Google Scholar 

  • Awad, M., Khanna, R.: Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers. A Press (2015)

    Google Scholar 

  • Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the 5th Annual Workshop on Computational Learning Theory, pp. 144–152 (1992)

    Google Scholar 

  • Chan, K., Koh, C., Lee, C.S.G.: A 3-D-point-cloud system for human-pose estimation. IEEE Trans. Syst. Man Cybern.: Syst. 44(11), 1486–1497 (2014)

    Article  Google Scholar 

  • Cho, S., et al.: Magic wand: a hand-drawn gesture input device in 3-D space with inertial sensors. In: IWFHR-9 2004. Ninth International Workshop on Frontiers in Handwriting Recognition, 2004, pp. 106–111 (2004) 26–29 Oct 2004

    Google Scholar 

  • Choi, E.S., et al.: Beatbox music phone: gesture-based interactive mobile phone using a tri-axis accelerometer. In: IEEE International Conference on Industrial Technology, 2005. ICIT 2005, pp. 97–102 (2005). Accessed 14–17 Dec 2005

    Google Scholar 

  • Choi, S., Lee, A.S., Lee, S.Y.: On-line handwritten character recognition with 3D accelerometer. In: International Conference on Information Acquisition, pp. 845–850 (2006). Accessed 20–23 Aug 2006

    Google Scholar 

  • Craig, J.J.: Introduction to Robotics: Mechanics and Control, 3rd edn. Prentice Hall, Upper Saddle River (2004)

    Google Scholar 

  • Elmezain, M., et al.: A Hidden Markov Model-based continuous gesture recognition system for hand motion trajectory. In: ICPR 2008. 19th International Conference on Pattern Recognition, 2008, pp. 1–4 (2008). Accessed 8–11 Dec 2008

    Google Scholar 

  • Fuccella, V., Costagliola, G.: Unistroke Gesture recognition through polyline approximation and alignment. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3351–3354. ACM (2015)

    Google Scholar 

  • Fujioka, H., et al.: Constructing and reconstructing characters, words, and sentences by synthesizing writing motions. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 36(4), 661–670 (2006)

    Article  Google Scholar 

  • GestureTek mobile. http://www.gesturetekmobile.com

  • He, Z.: A new feature fusion method for gesture recognition based on 3D accelerometer. In: 2010 Chinese Conference on Pattern Recognition (CCPR), pp. 1–5 (2010). Accessed 21–23 Oct 2010

    Google Scholar 

  • He, Z.: Accelerometer based gesture recognition using fusion features and SVM. J. Softw. 6(6), 1042–1049 (2011)

    Google Scholar 

  • Kallio, S., Kela, J., Mantyjarvi, J.: Online gesture recognition system for mobile interaction. In: IEEE International Conference on Systems, Man and Cybernetics, 2003, vol. 3, pp. 2070–2076 (2003). Accessed 5–8 Oct 2003

    Google Scholar 

  • Kratz, S., Rohs, M., Essl, G.: Combining acceleration and gyroscope data for motion gesture recognition using classifiers with dimensionality constraints. In: Proceedings of the 2013 International Conference on Intelligent user Interfaces, pp. 173–178. ACM (2013)

    Google Scholar 

  • Liu, J., et al.: uWave: accelerometer-based personalized gesture recognition and its applications. In: PerCom 2009. IEEE International Conference on Pervasive Computing and Communications, 2009, pp. 1–9 (2009). Accessed 9–13 March 2009

    Google Scholar 

  • Mitri, N., Wilkerson, C., Awad, M.: Recognition of free-form gestures from orientation tracking of a handheld or wearable device (2016). http://www.freepatentsonline.com/y2016/0092504.html

  • Prigge, E.: A positioning system with no line-of-sight restrictions for cluttered environments. PhD dissertation, Stanford University, p. 1 (2004)

    Google Scholar 

  • Kauppila, M., et al.: Accelerometer based gestural control of browser applications. In: Proceedings of UCS, Tokyo (2007)

    Google Scholar 

  • Khan, M.: Cubic Bezier least square fitting. Matlab Central (2007). http://www.mathworks.com/matlabcentral/fileexchange/15542-cubic-bezier-least-square-fitting

  • Pylvänäinen, T.: Accelerometer Based gesture recognition using continuous HMMs. In: Marques, Jorge S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA 2005. LNCS, vol. 3522, pp. 639–646. Springer, Heidelberg (2005). https://doi.org/10.1007/11492429_77

    Chapter  Google Scholar 

  • Raffa, G., et al.: Don’t slow me down: bringing energy efficiency to continuous gesture recognition. In: 2010 International Symposium on Wearable Computers (ISWC), pp. 1–8. Accessed 10–13 Oc. 2010

    Google Scholar 

  • Rico, J., Brewster, S.: Gesture and voice prototyping for early evaluations of social acceptability in multimodal interfaces. In: Proceeding ICMI-MLMI ‘10 International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, Article No. 16

    Google Scholar 

  • Ruiz, J., Li, Y., Lank, E.: User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 197–206. ACM (2011)

    Google Scholar 

  • SmartWatchr: 2015 Smartwatch Specs Comparison Chart. Smartwatch News. (2015). http://www.smartwatch.me/t/2015-smartwatch-specs-comparison-chart/979

  • Saab, A., Mitri, N., Awad, M.: Ham or Spam? A comparative study for some content-based classification algorithms for email filtering. In: 17th IEEE Mediterranean Electrotechnical Conference - Information & Communication Systems, Beirut, Lebanon (2014). Accessed 13–16 April 2014

    Google Scholar 

  • Timo, P.: Accelerometer based gesture recognition using continuous HMMs. In: Pattern Recognition and Image Analysis Lecture Notes in Computer Science, vol. 3522, pp. 639–646 (2005)

    Google Scholar 

  • Vapnik, V.: The Nature of Statistic Learning Theory. Springer, New York (1995)

    Book  Google Scholar 

  • Voica, A.: PowerVR GX5300: the world’s smallest GPU for next-generation wearables and IoT. Imagination Blog. 2014. http://blog.imgtec.com/powervr/powervr-gx5300-the-worlds-smallest-gpu-for-next-generation-wearables-and-iot. Accessed 5 Jan 2016

  • Wang, X., et al.: Gesture recognition using mobile phone’s inertial sensors. In: Distributed Computing and Artificial Intelligence, pp. 173–184. Springer, Berlin (2012)

    Google Scholar 

  • Wobbrock, J.O., Wilson, A.D., Li, Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, pp. 159–168. ACM (2007)

    Google Scholar 

  • Wu, J., et al.: Gesture recognition with a 3-D accelerometer. In: UIC ‘09 Proceedings of the 6th International Conference on Ubiquitous Intelligence and Computing, pp. 25–38 Brisbane (2009). Accessed July 7–9 2009

    Google Scholar 

  • Zhang, X., et al.: A framework for hand gesture recognition based on accelerometer and EMG sensors. In: IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 41(6), 1064–1076 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon

    Nicholas Mitri & Mariette Awad

Authors
  1. Nicholas Mitri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariette Awad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mariette Awad .

Editor information

Editors and Affiliations

  1. Goldsmiths University of London, London, UK

    Larisa Soldatova

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Joaquin Vanschoren

  3. University of Cyprus, Nicosia, Cyprus

    George Papadopoulos

  4. Università degli Studi di Bari Aldo Moro, Bari, Italy

    Michelangelo Ceci

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mitri, N., Awad, M. (2018). Sky Writer: Towards an Intelligent Smart-phone Gesture Tracing and Recognition Framework. In: Soldatova, L., Vanschoren, J., Papadopoulos, G., Ceci, M. (eds) Discovery Science. DS 2018. Lecture Notes in Computer Science(), vol 11198. Springer, Cham. https://doi.org/10.1007/978-3-030-01771-2_29

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01771-2_29

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01770-5

  • Online ISBN: 978-3-030-01771-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation