Skip to main content

Advertisement

Springer Nature Link
Log in

Two-channel lstm for severity rating of parkinson’s disease using 3d trajectory of hand motion

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hand movement is one of the important bases for the severity rating of Parkinson’s disease. While observing hand motion of patients, medical specialists evaluate the degree of motor deterioration according to established rating scales. This diagnostic procedure is inefficient and can be easily affected by different doctors’ subjectivity, even though several studies showed rating scales are reliable. In this paper, we propose an automatic method based on hand exercise data including finger-tapping and fist movements, which is recorded by ordinary camera. We estimate 3D hand pose from regular RGB images and proposed a two-channel long short-term memory model to learn the patterns of 3D position changing trajectory of hand joints. Experiments on our dataset, the proposed method outperforms literature including popular machine learning methods with 95.7% of the precision, 95.8% of the sensitivity and 92.8% of the specificity respectively on average. We believe the quantitative evaluation of hand movement will benefit the clinical PD diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Alty JE, Cosgrove J, Lones MA, Smith SL, Possin K, Schuff N, Jamieson S (2016) Clinically ‘slight’ bradykinesia in parkinson’s disease is accurately detected using evolutionary computation analysis of finger tapping. In: International Parkinsons and Movement Disorders Society Congress

  2. Ariyanto M, Caesarendra W, Mustaqim KA, Irfan M, Pakpahan JA, Setiawan JD, Winoto AR (2015) Finger movement pattern recognition method using artificial neural network based on electromyography (emg) sensor. In: Automation, Cognitive Science, Optics, Micro Electro-Mechanical System, and Information Technology (ICACOMIT), 2015 International Conference on. IEEE, pp 12–17

  3. Arora S, Venkataraman V, Zhan A, Donohue S, Biglan KM, Dorsey ER, Little MA (2015) Detecting and monitoring the symptoms of parkinson’s disease using smartphones: a pilot study. Parkinsonism Relat Dis 21(6):650–653

    Article  Google Scholar 

  4. Camgöz NC, Kindiroglu AA, Akarun L (2014) Gesture recognition using template based random forest classifiers.. In: ECCV Workshops (1), pp 579–594

  5. Chen HL, Wang G, Ma C, Cai ZN, Liu WB, Wang SJ (2016) An efficient hybrid kernel extreme learning machine approach for early diagnosis of parkinson’s disease. Neurocomputing 184(C):131–144

    Article  Google Scholar 

  6. Diaz M, Ferrer MA, Impedovo D, Pirlo G, Vessio G (2019) Dynamically enhanced static handwriting representation for parkinson’s disease detection. Pattern Recognition Letters

  7. Escalante HJ, Morales EF, Sucar LE (2016) A naive bayes baseline for early gesture recognition. Pattern Recogn Lett 73:91–99

    Article  Google Scholar 

  8. Gage H, Hendricks A, Zhang S, Kazis L (2003) The relative health related quality of life of veterans with parkinson’s disease. J Neurol Neurosurg Psychiatry 74(2):163

    Article  Google Scholar 

  9. Giancardo L, Sánchez-Ferro A, Arroyo-Gallego T, Butterworth I, Mendoza CS, Montero P, Matarazzo M, Obeso JA, Gray ML, Estépar RSJ (2016) Computer keyboard interaction as an indicator of early parkinson’s disease. Sci Rep 6(10):1–10

    Google Scholar 

  10. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  11. Ireland D, Wang Z, Lamont R, Liddle J (2016) Classification of movement of people with parkinsons disease using wearable inertial movement units and machine learning. Stud Health Technol Inform 227:61

    Google Scholar 

  12. Keisuke S, Toshio T, Akihiko K, Masaru Y, Saburo S (2009) Measurement and evaluation of finger tapping movements using log-linearized gaussian mixture networks. Sensors 9(3):2187–201

    Article  Google Scholar 

  13. Khan T, Nyholm D, Westin J, Dougherty M (2014) A computer vision framework for finger-tapping evaluation in parkinson’s disease. Artif Intell Med Artif Intell Med 60(1):27–40

    Article  Google Scholar 

  14. Kim H, Lee S, Lee D, Choi S, Ju J, Myung H (2015) Real-time human pose estimation and gesture recognition from depth images using superpixels and svm classifier. Sensors 15(6):12410–12427

    Article  Google Scholar 

  15. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. Computer Science

  16. Krupicka R, Szabo Z, Jirina M (2011) Motion camera system for measuring finger tapping in parkinson’s disease. Springer, Berlin Heidelberg

    Book  Google Scholar 

  17. Kupryjanow A, Kunka B, Kostek B (2010) Updrs tests for diagnosis of parkinson’s disease employing virtual-touchpad. In: Database and Expert Systems Applications, pp 132–136

  18. Li F, Ge R, Zhou H, Wang Y, Liu Z, Yu X (2020) Tesia: A trusted efficient service evaluation model in internet of things based on improved aggregation signature. Concurrency and Computation: Practice and Experience

  19. Li Y, Yang L, Wang P, Zhang C, Xiao J, Zhang Y, Qiu M (2017) Classification of parkinson’s disease by decision tree based instance selection and ensemble learning algorithms. Journal of Medical Imaging & Health Informatics 7(2)

  20. Liu X, Xia Y, Yu H, Dong J, Jian M, D. Pham T (2021) Region based parallel hierarchy convolutional neural network for automatic facial nerve paralysis evaluation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, pp 2325–2332

  21. Liu X, Xia Y, Yu H, Dong J, Jian M, Pham T (2020) Region based parallel hierarchy convolutional neural network for automatic facial nerve paralysis evaluation. IEEE Trans Neural Syst Rehabilitation Eng 10:2325–2332

    Article  Google Scholar 

  22. Molchanov P, Gupta S, Kim K, Kautz J (2015) Hand gesture recognition with 3d convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 1–7

  23. Molchanov P, Yang X, Gupta S, Kim K, Tyree S, Kautz J (2016) Online detection and classification of dynamic hand gestures with recurrent 3d convolutional neural network. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 4207–4215

  24. Om MS, Duarte M, Diana R, Divyanshu D, Bahman J (2018) Botulinum toxin in essential hand tremor - a randomized double-blind placebo-controlled study with customized injection approach. Parkinsonism Relat Dis 56:65–69

    Article  Google Scholar 

  25. Papadopoulos A, Kyritsis K, Klingelhoefer L, Bostanjopoulou S, Delopoulos A (2019) Detecting parkinsonian tremor from imu data collected in-the-wild using deep multiple-instance learning. IEEE J Biomed Health Inform 24(9):2559–2569

    Article  Google Scholar 

  26. Parziale A, Senatore R, Cioppa A D, Marcelli A (2021) Cartesian genetic programming for diagnosis of parkinson disease through handwriting analysis: Performance vs. interpretability issues. Artif Intell Med 111:1–13

    Article  Google Scholar 

  27. Printy BP, Renken LM, Herrmann JP, Lee I, Johnson B, Knight E, Varga G, Whitmer D (2014) Smartphone application for classification of motor impairment severity in parkinson’s disease. Conf Proc IEEE Eng Med Biol Soc 2014:2686–2689

    Google Scholar 

  28. Sano Y, Kandori A, Shima K, Yamaguchi Y, Tsuji T, Noda M, Higashikawa F, Yokoe M, Sakoda S (2016) Quantifying parkinson’s disease finger-tapping severity by extracting and synthesizing finger motion properties. Med Biol Eng Comput 54(6):953–965

    Article  Google Scholar 

  29. Stamatakis J, Ambroise J, Crémers J, Sharei H, Delvaux V, Macq B, Garraux G (2013) Finger tapping clinimetric score prediction in parkinson’s disease using low-cost accelerometers. Computational Intelligence and Neuroscience,2013,(2013-4-16) 2013(2):1

    Google Scholar 

  30. Tsironi E, Barros P, Weber C, Wermter S (2017) An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing

  31. Wang Y, Dong X, Li G, Dong J, Yu H (2021) Cascade regression-based face frontalization for dynamic facial expression analysis. Cognitive Computation 99(3)

  32. Wei SE (2016) Convolutional pose machines: A deep architecture for estimating articulated poses. PhD thesis

  33. Yokoe M, Okuno R, Hamasaki T, Kurachi Y, Akazawa K, Sakoda S (2009) Opening velocity, a novel parameter, for finger tapping test in patients with parkinson’s disease. Parkinsonism Relat Dis 15(6):440–444

    Article  Google Scholar 

  34. Yu X, Li F, Li T, Wu N, Zhou H (2020) Trust-based secure directed diffusion routing protocol in wsn. J Ambient Intell Humaniz Comput J Amb Intel Hum Comp 99(5):1–13

    Google Scholar 

  35. Zeng W, Liu F, Wang Q, Wang Y, Ma L, Zhang Y (2016) Parkinson’s disease classification using gait analysis via deterministic learning. Neurosci Lett 633:268–278

    Article  Google Scholar 

  36. Zhou Y, Jenkins M E, Naish MD, Trejos A L (2018) Characterization of parkinsonian hand tremor and validation of a high-order tremor estimator. IEEE Trans Neural Syst Rehabilitation Eng 26(9):1823–1834

    Article  Google Scholar 

  37. Zimmermann C, Brox T (2017) Learning to estimate 3d hand pose from single rgb images. ICCV 2017

Download references

Acknowledgements

This research was supported in part by National Key Research and Development Plan Key Special Projects under Grant No. 2018YFB2100303, Shandong Province colleges and universities youth innovation technology plan innovation team project under Grant No. 2020KJN011, Shandong Provincial Natural Science Foundation under Grant No. ZR2020MF060, Program for Innovative Postdoctoral Talents in Shandong Province under Grant No. 40618030001, National Natural Science Foundation of China under Grant No. 61802216, and Postdoctoral Science Foundation of China under Grant No.2018M642613. National Natural Science Foundation of China under Grant No.62106117, and Shandong Provincial Natural Science Foundation under Grant No.ZR2021QF084.

Author information

Authors and Affiliations

  1. School of Business, Qingdao University, Qingdao, China

    Aite Zhao

  2. College of Computer Science and Technology, Qingdao University, Qingdao, China

    Jianbo Li

Authors
  1. Aite Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianbo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianbo Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, A., Li, J. Two-channel lstm for severity rating of parkinson’s disease using 3d trajectory of hand motion. Multimed Tools Appl 81, 33851–33866 (2022). https://doi.org/10.1007/s11042-022-12659-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12659-9

Keywords

Search

Navigation