Jules Françoise
Agnès Roby-Brami
ABSTRACT
Movement sequences are essential to dance and expressive movement practice; yet, they remain underexplored in movement and computing research, where the focus on short gestures prevails. We propose a method for movement sequence analysis based on motion trajectory synthesis with Hidden Markov Models. The method uses Hidden Markov Regression for jointly synthesizing motion feature trajectories and their associated variances, that serves as basis for investigating performers' consistency across executions of a movement sequence. We illustrate the method with a use-case in Tai Chi performance, and we further extend the approach to cross-modal analysis of vocalized movements.
- Bevilacqua, F., Baschet, F., and Lemouton, S. The augmented string quartet: Experiments and gesture following. Journal of New Music Research 41, 1 (2012), 103--119.Google Scholar
Cross Ref
- Bevilacqua, F., Zamborlin, B., Sypniewski, A., Schnell, N., Guédy, F., and Rasamimanana, N. Continuous realtime gesture following and recognition. Gesture in Embodied Communication and Human-Computer Interaction (2010), 73--84. Google ScholarCross Ref
- Brand, M., and Hertzmann, A. Style machines. Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 (2000), 183--192. Google ScholarDigital Library
- Calinon, S., D'halluin, F., Sauser, E., Caldwell, D., and Billard, A. Learning and reproduction of gestures by imitation: An approach based on Hidden Markov Model and Gaussian Mixture Regression. Robotics & Automation Magazine, IEEE 17, 2 (2010), 44--54.Google ScholarCross Ref
- Camurri, A., Hashimoto, S., and Ricchetti, M. Eyesweb: Toward gesture and affect recognition in interactive dance and music systems. Computer Music Journal 24, 1 (2000), 57--69. Google ScholarDigital Library
- Camurri, A., Mazzarino, B., and Ricchetti, M. Multimodal analysis of expressive gesture in music and dance performances. Gesture-Based Communication in Human-Computer Interaction (2004), 20--39.Google ScholarCross Ref
- Caramiaux, B., Wanderley, M. M., and Bevilacqua, F. Segmenting and Parsing Instrumentalist's Gestures. Journal of New Music Research 41, 1 (2012), 1--27.Google ScholarCross Ref
- Eickeler, S., Kosmala, A., and Rigoll, G. Hidden markov model based continuous online gesture recognition. Pattern Recognition, 1998. Proceedings. Fourteenth International Conference on 2 (1998), 1206--1208. Google ScholarDigital Library
- Fdili Alaoui, S., Caramiaux, B., Serrano, M., and Bevilacqua, F. Movement Qualities as Interaction Modality. In ACM Designing Interactive Systems (DIS 2012) (Newcastle, UK, 2012). Google ScholarDigital Library
- Ferguson, S., Schubert, E., and Stevens, C. Dynamic dance warping: Using dynamic time warping to compare dance movement performed under different conditions. In Proceedings of the International Workshop on Movement and Computing, MOCO'14, ACM (Paris, France, 2014), 94--99. Google ScholarDigital Library
- Fine, S., Singer, Y., and Tishby, N. The hierarchical hidden Markov model: Analysis and applications. Machine learning 32, 1 (1998), 41--62. Google ScholarDigital Library
- Françoise, J. Realtime Segmentation and Recognition of Gestures using Hierarchical Markov Models. Master's thesis, Université Pierre et Marie Curie, Ircam, 2011.Google Scholar
- Françoise, J., Fdili Alaoui, S., Schiphorst, T., and Bevilacqua, F. Vocalizing Dance Movement for Interactive Sonification of Laban Effort Factors. In Proceedings of the 2014 Conference on Designing Interactive Systems, DIS '14, ACM (Vancouver, Canada, 2014), 1079--1082. Google ScholarDigital Library
- Françoise, J., Schnell, N., and Bevilacqua, F. A Multimodal Probabilistic Model for Gesture-based Control of Sound Synthesis. In Proceedings of the 21st ACM international conference on Multimedia (MM'13) (Barcelona, Spain, 2013), 705--708. Google ScholarDigital Library
- Gillian, N., Knapp, B., and O'Modhrain, S. Recognition Of Multivariate Temporal Musical Gestures Using N-Dimensional Dynamic Time Warping. In Proceedings of the 2011 International Conference on New Interfaces for Musical Expression (NIME'11), Oslo, Norway (Oslo, Norway, 2011), 337--342.Google Scholar
- Karg, M., Seiber, W., Hoey, J., and Kulic, D. Human Movement Analysis: Extension of the F-Statistic to Time Series Using HMM. In IEEE International Conference on Systems, Man, and Cybernetics, IEEE (Oct. 2013), 3870--3875. Google ScholarDigital Library
- Karg, M., Venture, G., Hoey, J., and Kulić, D. Human movement analysis as a measure for fatigue: a hidden Markov-based approach. Neural Systems and Rehabilitation Engineering, IEEE Transactions on 22, 3 (May 2014), 470--81.Google Scholar
- McNeill, D. Hand and mind: What gestures reveal about thought. University of Chicago Press, 1996.Google Scholar
- Pohl, H., and Hadjakos, A. Dance Pattern Recognition using Dynamic Time Warping. In Proceedings of the Sound and Music Computing Conference (2010).Google Scholar
- Rabiner, L. R. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77, 2 (1989), 257--286.Google ScholarCross Ref
- Rasamimanana, N., Kaiser, F., and Bevilacqua, F. Perspectives on gesture-sound relationships informed from acoustic instrument studies. Organised Sound 14, 02 (2009), 208--216. Google ScholarDigital Library
- Tilmanne, J. Data-driven Stylistic Humanlike Walk Synthesis. Phd dissertation, University of Mons, 2013.Google Scholar
- Tilmanne, J., Moinet, A., and Dutoit, T. Stylistic gait synthesis based on hidden Markov models. EURASIP Journal on Advances in Signal Processing 2012, 1 (2012), 72.Google ScholarCross Ref
- Yang, J., and Bolt, J. Tai Chi Sword Classical Yang Style 2nd ed. YMAA Publication Center, Inc., 2015.Google Scholar
Index Terms
- Movement sequence analysis using hidden Markov models: a case study in Tai Chi performance
Recommendations
Coding with partially hidden Markov models
DCC '95: Proceedings of the Conference on Data CompressionPartially hidden Markov models (PHMM) are introduced. They are a variation of the hidden Markov models (HMM) combining the power of explicit conditioning on past observations and the power of using hidden states. (P)HMM may be combined with arithmetic ...
Structural hidden Markov models: An application to handwritten numeral recognition
We introduce in this paper a generalization of the widely used hidden Markov models (HMM's), which we name "structural hidden Markov models" (SHMM). Our approach is motivated by the need of modeling complex structures which are encountered in many ...
The evaluation problem in discrete semi-hidden Markov models
This paper is devoted to discrete semi-hidden Markov models (SHMM), which are related to the well-known hidden Markov models (HMM). In particular, the HMM associated to an SHMM is defined, and the forward algorithm for solving the evaluation problem in ...
Comments