Perrine Chassat
ABSTRACT
The analysis of human movement poses a well-known challenge that has already been addressed in various ways and needs to be adapted to the type of movement being considered. The focus here is on the analysis of hand movement in sign language. This study aims to characterize and model the different variations present in the data to develop a realistic generative model of hand movement in sign language. We identify two types of variations that play a key role in characterizing human movement: temporal variations and shape variations, i.e., variations in the speed of movement and the geometry of movement. However, separating these variations or understanding their relationship is a non-trivial task. A well-known model for the relationship between time, speed, and geometry is the 2/3 power-law demonstrated for several human movements, mainly constrained and planar. We find that the generalization of this law to a three-dimensional motion is not sufficient to explain variations in hand movement in sign language. We develop a new statistical modeling framework that is flexible and can respect the geometry of the movement signals. The two different variations are identified using the Frenet-Serret representation and modeled by mean geometry, mean speed, and their nonlinear transformations. The nonlinear variations in time and geometry are analyzed by functional principal component analysis. Then the generative model for the hand movement in sign language is built by imposing a joint probability model on the principal coefficients of these components.
- N. Brunel and J. Park. 2019. The Frenet-Serret Framework for Aligning Geometric Curves. In Geometric Science of Information, Frank Nielsen and Frédéric Barbaresco (Eds.). Springer, 608–617.Google Scholar
- T. Flash and A. Berthoz. 2021. Space-Time Geometries for Motion and Perception in the Brain and the Arts. Springer.Google Scholar
- P.T. Fletcher, Conglin Lu, S.M. Pizer, and Sarang Joshi. 2004. Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging 23, 8 (2004), 995–1005. https://doi.org/10.1109/TMI.2004.831793Google ScholarCross Ref
- S. Gibet, F. Lefebvre-Albaret, L. Hamon, R. Brun, and A. Turki. 2016. Interactive editing in French Sign Language dedicated to virtual signers: requirements and challenges.Universal Access in the Information Society, Springer Verlag (2016).Google Scholar
- Jolliffe IT.2002. Principal component analysis. 2nd edn. New York, NY: Springer-Verlag.Google Scholar
- F. Lacquaniti, C. Terzuolo, and P. Viviai. 1983. The law relating the kinematic and figural aspects of drawing movements.Acta Psychologica (Amst.) 54 (1983), 115–130.Google ScholarCross Ref
- U. Maoz, A. Berthoz, and T. Flash. 2009. Complex unconstrained three-dimensional hand movement and constant equi-affine speed.Journal of Neurophysiology 101 (2009), 1002–1015.Google ScholarCross Ref
- J. Park, N. Brunel, and P. Chassat. 2022. Curvature and Torsion estimation of 3D functional data: A geometric approach to build the mean shape under the Frenet Serret framework. https://doi.org/10.48550/ARXIV.2203.02398Google Scholar
- F. Pollick, U. Maoz, A. Handzel, P. Giblin, G. Sapiro, and T. Flash. 2009. Three-dimensional arm movements at constant equi-affine speed.Cortex 45(2009), 325–339.Google ScholarCross Ref
- J.O. Ramsay and B.W. Silverman. 2005. Functional data analysis(second ed.). Springer. xx+426 pages.Google Scholar
- A. Srivastava, E. Klassen, S.H. Joshi, and I.H Jermyn. 2011. Shape analysis of elastic curves in Euclidean spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(7) (2011), 1415–1428.Google ScholarDigital Library
- J.D. Tucker, W. Wu, and A. Srivastava. 2013. Generative models for functional data using phase and amplitude separation.Computational Statistics and Data Analysis 61 (2013), 50–66.Google ScholarDigital Library
Index Terms
- Analysis of variability in sign language hand trajectories: development of generative model
Recommendations
Analysis of hand movement variation related to speed in Japanese sign language
IUCS '09: Proceedings of the 3rd International Universal Communication SymposiumTo achieve the greater accessibility for deaf people, sign language recognition systems and sign language animation systems must be developed. In Japanese sign language (JSL), previous studies have suggested that emphasis and emotion cause changes in ...
Improved Face and Hand Tracking for Sign Language Recognition
ITCC '05: Proceedings of the International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II - Volume 02In this paper, we develop face and hand tracking for sign language recognition system. The system is divided into two stages; the initial and tracking stages. In initial stage, we use the skin feature to localize face and hands of signer. The ellipse ...
A sign language recognition system using hidden markov model and context sensitive search
VRST '96: Proceedings of the ACM Symposium on Virtual Reality Software and TechnologyHand gesture is one of the most natural and expressive ways for the hearing impaired. However, because of the complexity of dynamic gestures, most researches are focused either on static gestures, postures, or a small set of dynamic gestures. As real-...
Comments