Abstract
Recognizing rotational and angular hand movements using a non-invasive way is a challenging task. In this paper, we present a web-based multimedia hand-Therapy framework that can dynamically provide therapy services to a patient having Hemiplegia disability. Using off the shelf 3D motion control sensor called LEAP, the framework can detect, recognize and track different high-level therapeutic movements, both rotational as well as angular, originated from different joints of a Hemiplegic patient’s hand and deduce live kinematic data from these movements. The obtained kinematic data consists of a wide span of hand joint and therapy parameters that is assumed to help medical professionals in their clinical decision making. The multimedia environment uses forward kinematics to receive the live sensory data from the 3D motion tracking sensors and uses inverse kinematics to analyze the sensory data streams in real-time. A subject performs a rehabilitation therapy prescribed by the physician and using both forward and inverse kinematics, the system validates the angular and rotational motion of the joints with respect to the correct therapeutic posture and provides live feedback to the subject. The proposed method is non-invasive as the patient does not need to wear any external devices in the hand and hence can perform therapy exercises even at home. The system uses serious game interfaces to provide immersive and engaging perspective to the therapy domain. To the best of our knowledge, this study is the first of its kind to propose a non-invasive way of tracking live forward and inverse kinematic data in the context of hand therapy. We share the implementation details and our initial test results. Finally, our experiment shows that LEAP has the potential to be used for obtaining kinematic data from primitive and complex hand therapies.
Similar content being viewed by others
Notes
Al Noor Specialist Hospital, Makkah, Saudi Arabia
Disabled Children Hospital (DCA), Makkah, Saudi Arabia
Basma Disability Center, Iskan, Makkah, Saudi Arabia
Basma Center for Disabled Kids,
Disabled Children Association Hospital in Makkah (http://www.dca.org.sa)
Al Noor Hospital, Makkah (http://alnoorhospital.com/HandTherapy.aspx).
References
Alamri A, Kim H-N, Cha J, El Saddik A (2012) Serious games for rehabilitation of stroke patients with vibrotactile feedback. Int J Comp Sci Sport 9(2)
Albrecht I, Haber J, Seidel H-P (2003) Construction and animation of anatomically based human hand models. Proc SCA 98–109
Anderson SA, Blandine C-G (1993) Anatomy of movement. Eastland Press, Chicago. ISBN 0-939616-17-3
Aziz O, Atallah L, Lo B, Elhelw M, Wang L, Yang GZ, Darzi A (2007) A pervasive body sensor network for measuring postoperative recovery at home. J Surg Innov 14(2):83–90
Baran M, Lehrer N, Siwiak D, Chen Y, Duff M, Ingalls T, Rikakis T (2011) Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors. Eng Med Biol Soc EMBC7602–7605
Chang Y, Chen S, Huang J (2011) A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res Dev Disabil 32(6):2566–2570
Chuanchu W, Kok SP, Kai KA, Cuntai G, Haihong Z, Rongsheng L, Sui GC, Geok Chua K, Beng TA, Kuah CWK (2009) A feasibility study of non-invasive motor-imagery BCI-based robotic rehabilitation for Stroke patients. 4th International IEEE/EMBS Conference on Neural Engineering. pp 271–274
Dolphin JA (1965) Extensor tenotomy for chronic boutonnière deformity of the finger report of two cases. J Bone Joint Surg 47(1):161–164
Erdemir A, McLean S, Herzog W, van den Bogert AJ (2007) Model-based estimation of muscle forces exerted during movements. J Clin Biomech, Elsevier 22(2):131–154
Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: a review. Comp Vision Image Underst Sci Dir 108(1–2):52–57
Gabel M, Renshaw E, Schuster A, Gilad-Bachrach R (2012) Full body gait analysis with kinect. IEEE Eng Med Biol Soc (EMBC) 1964–1967
Gips J, Betke M, DiMattia PA (2007) Early experiences using visual tracking for computer access by people with profound physical disabilities. Disabil Rehabil Assist Technol 2(4):189–199
Goto A, Leng S, Sugamoto K, Cooney WP III, Kakar S, Zhao K (2014) In vivo pilot study evaluating the thumb carpometacarpal joint during circumduction. Clin Orthop Relat Res 472(4):1106–1113
Gustus A, Stillfried G, Visser J, Jörntell H, van der Smagt P (2012) Human hand modelling: kinematics, dynamics, applications. J Biol Cybern, Springer 106:741–755
Hanna SE, Bartlett DJ, Rivard LM, Russell DJ (2008) Reference curves for the gross motor function measure: percentiles for clinical description and tracking over time among children with cerebral palsy. Phys Ther 88(5):596–607
Harley L, Robertson S, Gandy M, Harbert S, Britton D (2011) The design of an interactive stroke rehabilitation gaming system. Proc. of the 14th Intl. Conf. Human-computer interaction: users and applications, Orlando FL, pp 167–173
Herman IP (2007) Physics of human body. Springer, ISBN-10 3-540-29603-4
Holden MK (2005) Virtual environments for motor rehabilitation: review. Cyber Psychol Behav 8(3):187–211
Hondori HM, Khademi M, Dodakian L, Cramer SC, Lopes CV (2013) A spatial augmented reality rehab system for post-stroke hand rehabilitation In proceeding of 2013 Conference on Medicine Meets Virtual Reality, NextMed/MMVR20
Hovet L, Ryall K, Mcdonnell R, O’Sullivan C (2012) Sleight of hand: perception of finger motion from reduced marker sets. Proc. of the ACM Symposium on Interactive 3D Graphics and Games, pp 79–86
Hume MC, Gellman H, McKellop H, Brumfield RH (1990) Functional range of motion of the joints of the hand. J Hand Surg 15(2):240–243
Industrial-robotics-theory-modelling-control. In: Cubero S (ed) ARS/plV, Germany, ISBN 3-86611-285-8, p 964
Jenkins M, Bamberger HB, Black L, Nowinski R (1998) Thumb joint motion: what is normal? J Hand Surg 23(6):796–797
Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12:1437–1454
Kim D, Hilliges O, Iazddi S, Butler A, Chen J, Oikonomidis I, Olivier P (2012) Digits: free 3D interactions anywhere using a wrist-worn gloveless sensor dithered color quantization. Proc ACM UIST 167–176
LaBelle K (2011) Evaluation of kinect joint tracking for clinical and in-home stroke rehabilitation tools. Undergraduate thesis, Notre Dame, Indiana
Lange B, Koenig S, McConnell E, Chang C, Juang R, Suma E, Bolas M, Rizzo A (2012) Interactive game-based rehabilitation using the microsoft kinect. IEEE Virt Real 171–172
Ma M, Bechkoum K Serious games for movement therapy after stroke systems. In Proceedings of IEEE International Conference on Man and Cybernetics. Singapore, 1872–1877
Mallon WJ, Brown HR, Nunley JA (1991) Digital ranges of motion: normal values in young adults. J Hand Surg 16A:882
Mehm F, Hardy S, Göbel S, Steinmetz R (2011) Collaborative authoring of serious games for health. Proc. of the 19th ACM international conference on Multimedia. ACM, New York, pp 807–808
Minear WL (1956) A classification of cerebral palsy, pediatrics. Am Acad Pediatr 18(5):841–852
Rahman MA, Basalamah S, Toonsi AH, El Saddik A (2013) A Framework toward detecting and visualizing kinematic data for children with hemiplegia. IEEE Healthcom, Lisbon, pp 9–12
Rahman MA, El Saddik A (2013) Modeling e-health framework towards personalized and context-aware multimedia services. In: Furht B, Agarwal A (eds) Handbook of innovative medical technologies, Springer, ISBN 978-1-4614-8494-3
Rahman MA, Qamar AM, Ahmed MA, Rahman MA, Basalamah S (2013) Multimedia interactive therapy environment for children having physical disabilities. International Conference on Multimedia Retrieval, Dallas, pp 16–19
Saini S, Sulaiman S, Zakaria M, Shukri S (2012) A Low-cost game framework for a home-based stroke rehabilitation system. Proc. of International Conference on Computer and Information Science (ICCIS), pp 55–60
Skarilova B, Plevkova A (1996) Ranges of joint motion of the adult hand. Acta Chir Plast 38(67)
Smahel Z, Klimova A (2004) The influence of age and exercise on the mobility of hand joints 1: metacarpophalangeal joint of the threephalangeal fingers. Acta Chir Plast 46(81)
Smahel Z, Klimova A (2004) The influence of age and exercise on the mobility of hand joints 2: interphalangeal joints of the three-phalangeal fingers. Acta Chir Plast 46(122)
Stanley FJ, Blair E, Alberman E (2000) Cerebral palsies: epidemiology and causal pathways. MacKeith Press, London
Stone EE, Skubic M (2011) Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing. Proc. of IEEE Engineering in Medicine and Biology Society (EMBC). Boston, USA, pp 6491–6494
Sueda S, Kaufman A, PAI DK (2008) Musculotendon simulation for hand animation. ACM Trans Graph 27(3)
Tsang W, Singh K, Eugene F (2005) Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. Proc SCA 319–328
Weichert F, Bachmann D, Rudak B, Fisseler D (2013) Analysis of the accuracy and robustness of the leap motion controller. Sensors 13:6380–6393
Winter DA (2009) Biomechanics and motor control of human movement. John Wiley & Sons, ISBN: 978-0-470-39818-0
Zhao W, Chai J, Xu Y (2012) Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. ACM SIGGRAPH 33–42
Acknowledgments
The author would like to thank the therapists and medical doctors who have given valuable feedback and domain specific therapy knowledge. The author appreciates the suggestion provided by Dr. Asad Toonsi of Maternity and Children Hospital and Dr. Fawaz of Disabled Children Hospital in Makkah, Saudi Arabia. The author would also like to thank Jaanga for making his LEAP code available for reusing. The author would also like to thank Ahmad Qamar, Faizan Ur Rehman, and Delwar Hossain of Advanced Media Laboratory of Umm Al-Qura University for helping in demo and usability testing.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rahman, M.A. Web-based multimedia hand-therapy framework for measuring forward and inverse kinematic data. Multimed Tools Appl 76, 8227–8255 (2017). https://doi.org/10.1007/s11042-016-3447-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-3447-6