Skip to main content
SpringerLink
Log in

Exercise classification and event segmentation in Hammersmith Infant Neurological Examination videos

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Image and video processing techniques are being frequently used in medical science applications. Computer vision-based systems have successfully replaced various manual medical processes such as analyzing physical and biomechanical parameters, physical examination of patients. These systems are gaining popularity because of their robustness and the objectivity they bring to various medical procedures. Hammersmith Infant Neurological Examinations (HINE) is a set of physical tests that are carried out on infants in the age group of 3–24 months with neurological disorders. However, these tests are graded through visual observations, which can be highly subjective. Therefore, computer vision-aided approach can be used to assist the experts in the grading process. In this paper, we present a method of automatic exercise classification through visual analysis of the HINE videos recorded at hospitals. We have used scale-invariant-feature-transform features to generate a bag-of-words from the image frames of the video sequences. Frequency of these visual words is then used to classify the video sequences using HMM. We also present a method of event segmentation in long videos containing more than two exercises. Event segmentation coupled with a classifier can help in automatic indexing of long and continuous video sequences of the HINE set. Our proposed framework is a step forward in the process of automation of HINE tests through computer vision-based methods. We conducted tests on a dataset comprising of 70 HINE video sequences. It has been found that the proposed method can successfully classify exercises with accuracy as high as 84%. The proposed work has direct applications in automatic or semiautomatic analysis of “vertical suspension” and “ventral suspension” tests of HINE. Though some of the critical tests such as “pulled-to-sit,” “lateral tilting,” or “adductor’s angle measurement” have already been addressed using image- and video-guided techniques, scopes are there for further improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Zhang, R., Vogler, C., Metaxas, D.: Human gait recognition at sagittal plane. Image Vis. Comput. 25(3), 321–330 (2007)

    Article  Google Scholar 

  2. Liu, Q., Sclabassi, R., Sun, M.: Change detection in epilepsy monitoring video based on Markov random field theory. In: Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems, pp. 63–66 (2004)

  3. Viola, P., Jones., M.J., Snow, D.: Detecting pedestrians using patterns of motion and appearance. In: Ninth IEEE International Conference on Computer Vision, pp. 734–741 (2003)

  4. Singh, S., Hsiao, H.: Infant telemonitoring system. In: Proceedings of the 25th Annual International Conference of the IEEE on Engineering in Medicine and Biology Society, vol. 2, pp. 1354–1357 (2003)

  5. Nishida, Y., Motomura, Y., Kitamura, K., Mizoguchi, H.: Infant behavior simulation based on an environmental model and a developmental behavior model. In: Proceedings of IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1555–1560 (2004)

  6. Dubowitz, L., Dubowitz, V., Mercuri, E.: The Neurological Assessment of the Preterm and Full Term Infant, Clinics in Developmental Medicine. Heinemann, London (2000)

    Google Scholar 

  7. Romeo, D., Guzzetta, A., Scoto, M., Cioni, M., Patusi, P., Mazzone, D., Romeo, M.: Early neurologic assessment in preterm infants: integration of traditional neurologic examination and observation of general movements. Eur. J. Paediatr. Neurol. 12(3), 183–189 (2008)

    Article  Google Scholar 

  8. Dogra, D., Majumdar, A., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Analysis of adductors angle measurement in hammersmith infant neurological examinations using mean shift segmentation and feature point based object tracking. Comput. Biol. Med. 42(9), 925–934 (2012)

    Article  Google Scholar 

  9. Dogra, D.P., Badri, V., Majumdar, A.K., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Video analysis of Hammersmith lateral tilting examination using Kalman filter guided multi-path tracking. Med. Biol. Eng. Comput. 52(9), 759–772 (2014)

    Article  Google Scholar 

  10. Dogra, D.P., Majumdar, A.K., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Automatic adductors angle measurement for neurological assessment of post-neonatal infants during follow up. Pattern Recognit. Mach. Intell. Lect. Notes Comput. Sci. 6744, 160–166 (2011)

    Article  Google Scholar 

  11. Dogra, D.P., Majumdar, A.K., Sural, S., Mukherjee, J., Mukherjee, S., Singh, A.: Toward automating Hammersmith pulled-to-sit examination of infants using feature point based video object tracking. IEEE Trans. Neural Syst. Rehabil. Eng. 20(1), 38–47 (2012)

    Article  Google Scholar 

  12. Romeo, D., Cioni, M., Scoto, M., Mazzone, L., Palermo, F., Romeo, M.: Neuromotor development in infants with cerebral palsy investigated by the Hammersmith Infant Neurological Examination during the first year of age. Eur. J. Paediatr. Neurol. 12(1), 24–31 (2008)

    Article  Google Scholar 

  13. Dey, P., Dogra, D.P., Roy, P.P., Bhaskar, H.: Autonomous vision-guided approach for the analysis and grading of vertical suspension tests during HINE. In: Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 863–866 (2016)

  14. Roy, S., Dogra, D.P., Bhattacharya, S., Saha, B., Biswas, A., Majumdar, A.K., Mukhopadhyay, J., Majumdar, B., Singh, A., Paria, A., Mukherjee, S.: A web enabled health information system for Neonatal Intensive Care Unit (NICU). In: Proceedings of the 7th IEEE World Congress on Services (SERVICES), pp. 451–458 (2011)

  15. Dogra, D.P., Nandam, K., Majumdar, A.K., Sural, S., Mukhopadhyay, J., Majumdar, B., Mukherjee, S., Singh, A.: A tool for automatic Hammersmith infant neurological examination. E-Health Med. Commun. 2(2), 1–13 (2011)

    Article  Google Scholar 

  16. Ansari, A.F., Roy, P.P., Dogra, D.P.: Posture recognition in HINE exercises. In: Proceedings of International Conference on Computer Vision and Image Processing (CVIP) (2016)

  17. Poleg, Y., Arora, C., Peleg, S.: Temporal segmentation of egocentric videos. In: CVPR, pp. 2537–2544 (2014)

  18. Ballan, L., Bertini, M., Bimbo, A.D., Serra, G.: Action categorization in soccer videos using string kernels. In: Seventh International Workshop on Content-Based Multimedia Indexing, pp. 13–18 (2009)

  19. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  20. Luhn, H.P.: A statistical approach to mechanized encoding and searching of literary information. IBM J. Res. Dev. 1(4), 315 (1957)

    Article  MathSciNet  Google Scholar 

  21. Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. J. Doc. 28, 11–21 (1972)

    Article  Google Scholar 

  22. Chai, J., Ngan, K.: Face segmentation using skin-color map in videophone applications. IEEE Trans. Circuits Syst. Video Technol. 9(4), 551–564 (1999)

    Article  Google Scholar 

  23. Hu, J., Lim, S.G., Brown, M.K.: Writer independent on-line handwriting recognition using an HMM approach. Pattern Recogn. 33(1), 133–147 (2000)

    Article  Google Scholar 

  24. Huang, X.D., Arikiand, Y., Jack, M.A.: Hidden Markov Models for Speech Recognition. Edinburgh University Press, Edinburgh (1990)

    Google Scholar 

  25. Yang, J., Xu, Y.: Hidden Markov model for gesture recognition. Technical Report, Robotics Institute, Carnegie Mellon University, 10 (1994)

  26. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vision. 12(1), 43–77 (1994)

    Article  Google Scholar 

  27. Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., Baskurt, A.: Action classification in soccer videos with long short-term memory recurrent neural networks. In: 20th International Conference on Artificial Neural Networks, pp. 154–159 (2010)

  28. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks. 18(5), 602–610 (2005)

    Article  Google Scholar 

  29. Guo, Z., Hall, R.W.: Parallel thinning with two-subiteration algorithms. Commun. ACM 32(3), 359–373 (1989)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Civil Engineering, IIT Roorkee, Roorkee, 247667, India

    Abdul Fatir Ansari

  2. Department of Computer Science and Engineering, IIT Roorkee, Roorkee, 247667, India

    Partha Pratim Roy

  3. School of Electrical Sciences, IIT Bhubaneswar, Bhubaneswar, 751013, India

    Debi Prosad Dogra

Authors
  1. Abdul Fatir Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Partha Pratim Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debi Prosad Dogra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Debi Prosad Dogra.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ansari, A.F., Roy, P.P. & Dogra, D.P. Exercise classification and event segmentation in Hammersmith Infant Neurological Examination videos. Machine Vision and Applications 29, 233–245 (2018). https://doi.org/10.1007/s00138-017-0896-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-017-0896-5

Keywords

Search

Navigation