Skip to main content
Log in

Web-based multimedia hand-therapy framework for measuring forward and inverse kinematic data

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recognizing rotational and angular hand movements using a non-invasive way is a challenging task. In this paper, we present a web-based multimedia hand-Therapy framework that can dynamically provide therapy services to a patient having Hemiplegia disability. Using off the shelf 3D motion control sensor called LEAP, the framework can detect, recognize and track different high-level therapeutic movements, both rotational as well as angular, originated from different joints of a Hemiplegic patient’s hand and deduce live kinematic data from these movements. The obtained kinematic data consists of a wide span of hand joint and therapy parameters that is assumed to help medical professionals in their clinical decision making. The multimedia environment uses forward kinematics to receive the live sensory data from the 3D motion tracking sensors and uses inverse kinematics to analyze the sensory data streams in real-time. A subject performs a rehabilitation therapy prescribed by the physician and using both forward and inverse kinematics, the system validates the angular and rotational motion of the joints with respect to the correct therapeutic posture and provides live feedback to the subject. The proposed method is non-invasive as the patient does not need to wear any external devices in the hand and hence can perform therapy exercises even at home. The system uses serious game interfaces to provide immersive and engaging perspective to the therapy domain. To the best of our knowledge, this study is the first of its kind to propose a non-invasive way of tracking live forward and inverse kinematic data in the context of hand therapy. We share the implementation details and our initial test results. Finally, our experiment shows that LEAP has the potential to be used for obtaining kinematic data from primitive and complex hand therapies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig 15
Fig. 16

Similar content being viewed by others

Notes

  1. http://www.json.org/

  2. http://en.wikipedia.org/wiki/Biovision_Hierarchy

  3. http://threejs.org/

  4. http://www.jqplot.com/

  5. Al Noor Specialist Hospital, Makkah, Saudi Arabia

    Disabled Children Hospital (DCA), Makkah, Saudi Arabia

    Basma Disability Center, Iskan, Makkah, Saudi Arabia

  6. Basma Center for Disabled Kids,

    Disabled Children Association Hospital in Makkah (http://www.dca.org.sa)

    Al Noor Hospital, Makkah (http://alnoorhospital.com/HandTherapy.aspx).

References

  1. Alamri A, Kim H-N, Cha J, El Saddik A (2012) Serious games for rehabilitation of stroke patients with vibrotactile feedback. Int J Comp Sci Sport 9(2)

  2. Albrecht I, Haber J, Seidel H-P (2003) Construction and animation of anatomically based human hand models. Proc SCA 98–109

  3. Anderson SA, Blandine C-G (1993) Anatomy of movement. Eastland Press, Chicago. ISBN 0-939616-17-3

    Google Scholar 

  4. Aziz O, Atallah L, Lo B, Elhelw M, Wang L, Yang GZ, Darzi A (2007) A pervasive body sensor network for measuring postoperative recovery at home. J Surg Innov 14(2):83–90

    Article  Google Scholar 

  5. Baran M, Lehrer N, Siwiak D, Chen Y, Duff M, Ingalls T, Rikakis T (2011) Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors. Eng Med Biol Soc EMBC7602–7605

  6. Chang Y, Chen S, Huang J (2011) A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. Res Dev Disabil 32(6):2566–2570

    Article  Google Scholar 

  7. Chuanchu W, Kok SP, Kai KA, Cuntai G, Haihong Z, Rongsheng L, Sui GC, Geok Chua K, Beng TA, Kuah CWK (2009) A feasibility study of non-invasive motor-imagery BCI-based robotic rehabilitation for Stroke patients. 4th International IEEE/EMBS Conference on Neural Engineering. pp 271–274

  8. Dolphin JA (1965) Extensor tenotomy for chronic boutonnière deformity of the finger report of two cases. J Bone Joint Surg 47(1):161–164

    Article  Google Scholar 

  9. Erdemir A, McLean S, Herzog W, van den Bogert AJ (2007) Model-based estimation of muscle forces exerted during movements. J Clin Biomech, Elsevier 22(2):131–154

    Article  Google Scholar 

  10. Erol A, Bebis G, Nicolescu M, Boyle RD, Twombly X (2007) Vision-based hand pose estimation: a review. Comp Vision Image Underst Sci Dir 108(1–2):52–57

    Article  Google Scholar 

  11. Gabel M, Renshaw E, Schuster A, Gilad-Bachrach R (2012) Full body gait analysis with kinect. IEEE Eng Med Biol Soc (EMBC) 1964–1967

  12. Gips J, Betke M, DiMattia PA (2007) Early experiences using visual tracking for computer access by people with profound physical disabilities. Disabil Rehabil Assist Technol 2(4):189–199

    Article  Google Scholar 

  13. Goto A, Leng S, Sugamoto K, Cooney WP III, Kakar S, Zhao K (2014) In vivo pilot study evaluating the thumb carpometacarpal joint during circumduction. Clin Orthop Relat Res 472(4):1106–1113

  14. Gustus A, Stillfried G, Visser J, Jörntell H, van der Smagt P (2012) Human hand modelling: kinematics, dynamics, applications. J Biol Cybern, Springer 106:741–755

    Article  MathSciNet  MATH  Google Scholar 

  15. Hanna SE, Bartlett DJ, Rivard LM, Russell DJ (2008) Reference curves for the gross motor function measure: percentiles for clinical description and tracking over time among children with cerebral palsy. Phys Ther 88(5):596–607

    Article  Google Scholar 

  16. Harley L, Robertson S, Gandy M, Harbert S, Britton D (2011) The design of an interactive stroke rehabilitation gaming system. Proc. of the 14th Intl. Conf. Human-computer interaction: users and applications, Orlando FL, pp 167–173

  17. Herman IP (2007) Physics of human body. Springer, ISBN-10 3-540-29603-4

  18. Holden MK (2005) Virtual environments for motor rehabilitation: review. Cyber Psychol Behav 8(3):187–211

    Article  Google Scholar 

  19. Hondori HM, Khademi M, Dodakian L, Cramer SC, Lopes CV (2013) A spatial augmented reality rehab system for post-stroke hand rehabilitation In proceeding of 2013 Conference on Medicine Meets Virtual Reality, NextMed/MMVR20

  20. Hovet L, Ryall K, Mcdonnell R, O’Sullivan C (2012) Sleight of hand: perception of finger motion from reduced marker sets. Proc. of the ACM Symposium on Interactive 3D Graphics and Games, pp 79–86

  21. Hume MC, Gellman H, McKellop H, Brumfield RH (1990) Functional range of motion of the joints of the hand. J Hand Surg 15(2):240–243

    Article  Google Scholar 

  22. Industrial-robotics-theory-modelling-control. In: Cubero S (ed) ARS/plV, Germany, ISBN 3-86611-285-8, p 964

  23. Jenkins M, Bamberger HB, Black L, Nowinski R (1998) Thumb joint motion: what is normal? J Hand Surg 23(6):796–797

    Article  Google Scholar 

  24. Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12:1437–1454

    Article  Google Scholar 

  25. Kim D, Hilliges O, Iazddi S, Butler A, Chen J, Oikonomidis I, Olivier P (2012) Digits: free 3D interactions anywhere using a wrist-worn gloveless sensor dithered color quantization. Proc ACM UIST 167–176

  26. LaBelle K (2011) Evaluation of kinect joint tracking for clinical and in-home stroke rehabilitation tools. Undergraduate thesis, Notre Dame, Indiana

  27. Lange B, Koenig S, McConnell E, Chang C, Juang R, Suma E, Bolas M, Rizzo A (2012) Interactive game-based rehabilitation using the microsoft kinect. IEEE Virt Real 171–172

  28. Ma M, Bechkoum K Serious games for movement therapy after stroke systems. In Proceedings of IEEE International Conference on Man and Cybernetics. Singapore, 1872–1877

  29. Mallon WJ, Brown HR, Nunley JA (1991) Digital ranges of motion: normal values in young adults. J Hand Surg 16A:882

    Article  Google Scholar 

  30. Mehm F, Hardy S, Göbel S, Steinmetz R (2011) Collaborative authoring of serious games for health. Proc. of the 19th ACM international conference on Multimedia. ACM, New York, pp 807–808

    Google Scholar 

  31. Minear WL (1956) A classification of cerebral palsy, pediatrics. Am Acad Pediatr 18(5):841–852

    Google Scholar 

  32. Rahman MA, Basalamah S, Toonsi AH, El Saddik A (2013) A Framework toward detecting and visualizing kinematic data for children with hemiplegia. IEEE Healthcom, Lisbon, pp 9–12

    Google Scholar 

  33. Rahman MA, El Saddik A (2013) Modeling e-health framework towards personalized and context-aware multimedia services. In: Furht B, Agarwal A (eds) Handbook of innovative medical technologies, Springer, ISBN 978-1-4614-8494-3

  34. Rahman MA, Qamar AM, Ahmed MA, Rahman MA, Basalamah S (2013) Multimedia interactive therapy environment for children having physical disabilities. International Conference on Multimedia Retrieval, Dallas, pp 16–19

    Google Scholar 

  35. Saini S, Sulaiman S, Zakaria M, Shukri S (2012) A Low-cost game framework for a home-based stroke rehabilitation system. Proc. of International Conference on Computer and Information Science (ICCIS), pp 55–60

  36. Skarilova B, Plevkova A (1996) Ranges of joint motion of the adult hand. Acta Chir Plast 38(67)

  37. Smahel Z, Klimova A (2004) The influence of age and exercise on the mobility of hand joints 1: metacarpophalangeal joint of the threephalangeal fingers. Acta Chir Plast 46(81)

  38. Smahel Z, Klimova A (2004) The influence of age and exercise on the mobility of hand joints 2: interphalangeal joints of the three-phalangeal fingers. Acta Chir Plast 46(122)

  39. Stanley FJ, Blair E, Alberman E (2000) Cerebral palsies: epidemiology and causal pathways. MacKeith Press, London

    Google Scholar 

  40. Stone EE, Skubic M (2011) Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing. Proc. of IEEE Engineering in Medicine and Biology Society (EMBC). Boston, USA, pp 6491–6494

  41. Sueda S, Kaufman A, PAI DK (2008) Musculotendon simulation for hand animation. ACM Trans Graph 27(3)

  42. Tsang W, Singh K, Eugene F (2005) Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. Proc SCA 319–328

  43. Weichert F, Bachmann D, Rudak B, Fisseler D (2013) Analysis of the accuracy and robustness of the leap motion controller. Sensors 13:6380–6393

    Article  Google Scholar 

  44. Winter DA (2009) Biomechanics and motor control of human movement. John Wiley & Sons, ISBN: 978-0-470-39818-0

  45. Zhao W, Chai J, Xu Y (2012) Combining marker-based mocap and RGB-D camera for acquiring high-fidelity hand motion data. ACM SIGGRAPH 33–42

Download references

Acknowledgments

The author would like to thank the therapists and medical doctors who have given valuable feedback and domain specific therapy knowledge. The author appreciates the suggestion provided by Dr. Asad Toonsi of Maternity and Children Hospital and Dr. Fawaz of Disabled Children Hospital in Makkah, Saudi Arabia. The author would also like to thank Jaanga for making his LEAP code available for reusing. The author would also like to thank Ahmad Qamar, Faizan Ur Rehman, and Delwar Hossain of Advanced Media Laboratory of Umm Al-Qura University for helping in demo and usability testing.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Md. Abdur Rahman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahman, M.A. Web-based multimedia hand-therapy framework for measuring forward and inverse kinematic data. Multimed Tools Appl 76, 8227–8255 (2017). https://doi.org/10.1007/s11042-016-3447-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3447-6

Keywords

Navigation