Skip to main content
SpringerLink
Log in

Extracting Commands from Gestures: Gesture Spotting and Recognition for Real-Time Music Performance

  • Conference paper
  • First Online:
Sound, Music, and Motion (CMMR 2013)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 8905))

Included in the following conference series:

Abstract

Our work allows an interactive music system to spot and recognize “command” gestures from musicians in real time. The system gives the musician gestural control over sound and the flexibility to make distinct changes during the performance by interpreting gestures as discrete commands. We combine a gesture threshold model with a Dynamic Time Warping (DTW) algorithm for gesture spotting and classification. The following problems are addressed: i) how to recognize discrete commands embedded within continuous gestures, and ii) an automatic threshold and feature selection method based on F-measure to find good system parameters according to training data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akl, A., Valaee, S.: Accelerometer-based gesture recognition via dynamic-time warping, affinity propagation, and compressive sensing. In: ICASSP, pp. 2270–2273, (2010)

    Google Scholar 

  2. Alon, J., Athitsos, V., Yuan, Q., Sclaroff, S.: A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1685–1699 (2008)

    Article  Google Scholar 

  3. Bautista, M.A., Hernández, A., Ponce, V., Perez-Sala, X., Baró, X., Pujol, O., Angulo, C., Escalera, S.: Probability-based dynamic time warping for gesture recognition on RGB-D data. In: Jiang, X., Bellon, O.R.P., Goldgof, D., Oishi, T. (eds.) WDIA 2012. LNCS, vol. 7854, pp. 126–135. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  4. Bevilacqua, F., Schnell, N.: Wireless sensor interface and gesture-follower for music pedagogy. In: Proceedings of the 7th International Conference on New Interfaces for Musical Expression, pp. 124–129, (2007)

    Google Scholar 

  5. Corradini, A.: Dynamic time warping for off-line recognition of a small gesture vocabulary. In: Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems (RATFG-RTS’01). IEEE Computer Society. Washington, DC, USA (2001)

    Google Scholar 

  6. Elmezain, M., Al-Hamadi, A., Michaelis, B.: Robust methods for hand gesture spotting and recognition using hidden markov models and conditional random fields. In: IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) (2010)

    Google Scholar 

  7. Fels, S., Hinton, G.: Glove-talk: a neural network interface between a data-glove and a speech synthesizer. IEEE Trans. Neural Netw. 4(1), 2–8 (1993)

    Article  Google Scholar 

  8. Fiebrink, R.: Real-time human interaction with supervised learning algorithms for music composition and performance. Ph.D. dissertation, Faculty of Princeton University (2011)

    Google Scholar 

  9. Francoise, J.: Real time segmentation and recognition of gestures using hierarchical markov models. Master’s thesis, Université Pierre et Marie Curie, Ircam (2011)

    Google Scholar 

  10. Gillian, N., Knapp, R.: A machine learning tool-box for musician computer interaction. In: Proceedings of the 2011 Conference on New Interfaces for Musical Expression (2011)

    Google Scholar 

  11. Gillian N., Knapp R.B., O’Modhrain, S.: Recognition of multivariate temporal musical gestures using N-dimensional dynamic time warping. In: Proceedings of the 11th International Conference on New Interfaces for Musical Expression, pp. 337–342 (2011)

    Google Scholar 

  12. Kjeldsen, R., Kender, J.: Visual hand gesture recognition for window system control. In: Proceedings of Int’l Workshop Automatic Face- and Gesture-Recognition, pp. 184–188, Zurich, Switzerland (1995)

    Google Scholar 

  13. Krishnan, N.C., Lade, P., Panchanathan, S.: Activity gesture spotting using a threshold model based on adaptive boosting. In: IEEE International Conference on Multimedia and Expo (ICME), vol. 1, pp. 155–160 (2010)

    Google Scholar 

  14. Lee, H., Kim, J.: An HMM-based threshold model approach for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intel. 21(10), 961–973 (1999)

    Article  Google Scholar 

  15. Liu, J., Wang, Z., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uwave: Accelerometer-based personalized gesture recognition and its applications. In: IEEE International Conference on Pervasive Computing and Communications, pp. 1–9 (2009)

    Google Scholar 

  16. Murph, K.P., Paskin, M.A.: Linear Time Inference in Hierarchical HMMs. In: Dietterich, T., Becker, S., Gharahmani, Z. (eds.) Advances in Neural Information Processing Systems, vol. 2. MIT Press, Cambridge (2001)

    Google Scholar 

  17. Takahashi, K., Seki, S., Oka, R.: spotting recognition of human gestures from motion images (in Japanese). Technical report IE92 134. Institute of Electronics, Information and Communication Engineers, Japan, pp. 9–16 (1992)

    Google Scholar 

  18. Xu, D.: A neural approach for hand gesture recognition in virtual reality driving training system of SPG. In: International Conference on Pattern Recognition, vol. 3, pp. 519–522 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, USA

    Jiuqiang Tang & Roger B. Dannenberg

Authors
  1. Jiuqiang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roger B. Dannenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roger B. Dannenberg .

Editor information

Editors and Affiliations

  1. CNRS - LMA, Marseille, France

    Mitsuko Aramaki

  2. Toulon-Var University and CNRS - LMA, Marseille, France

    Olivier Derrien

  3. CNRS - LMA, Marseille, France

    Richard Kronland-Martinet

  4. CNRS - LMA, Marseille, France

    Sølvi Ystad

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Tang, J., Dannenberg, R.B. (2014). Extracting Commands from Gestures: Gesture Spotting and Recognition for Real-Time Music Performance. In: Aramaki, M., Derrien, O., Kronland-Martinet, R., Ystad, S. (eds) Sound, Music, and Motion. CMMR 2013. Lecture Notes in Computer Science(), vol 8905. Springer, Cham. https://doi.org/10.1007/978-3-319-12976-1_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-12976-1_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-12975-4

  • Online ISBN: 978-3-319-12976-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation