Skip to main content
SpringerLink
Log in

Space-time flexible kernel for recognizing activities from wearable cameras

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Recognizing activities of daily living is useful for ambient assisted living. In this regard, the use of wearable cameras is a promising technology. In this paper, we propose a novel approach for recognizing activities of daily living using egocentric viewpoint video clips. First, in every frame, the appearing objects are detected and labelled depending if they are being used or not by the subject. Later, the video clip is divided into spatiotemporal bins created with an object-centric cut. Finally, a support vector machine classifier is computed using a spatiotemporal flexible kernel between video clips. The validity of the proposed method has been proved by conducting experiments in the ADL dataset. Results confirm the suitability of using the space-time location of objects as information for the classification of activities using an egocentric viewpoint.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Economic and Financial Affairs (2017) The 2018 ageing report: underlying assumptions and projection methodologies. Technical report, European Commission

  2. Colby S, Ortman JM (2015) Projections of the size and composition of the U.S. population 2014 to 2060. Department of Commerce, Economics and Statistics Administration, technical report, U.S., p 2015

  3. Cardinaux F, Bhowmik D, Abhayaratne C, Hawley MS (2011) Video based technology for ambient assisted living: a review of the literature. J Ambient Intell Smart Environ 3:253–269

    Article  Google Scholar 

  4. Nguyen T-H-C, Nebel J-C, Florez-Revuelta F (2016) Recognition of activities of daily living with egocentric vision: a review. Sensors 16(1):72

    Article  Google Scholar 

  5. Hodges S, Berry E, Wood K (2011) Sensecam: a wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19(7):685–696

    Article  Google Scholar 

  6. Rodriguez M, Orrite C, Medrano C, Makris D (2016) A time flexible kernel framework for video-based activity recognition. Image Vis Comput 48:26–36

    Article  Google Scholar 

  7. Queiros A, Dias A, Silva AG, Rocha NP (2017) Ambient assisted living and health-related outcomes: a systematic literature review. Informatics 4(3):19

    Article  Google Scholar 

  8. Blackman S, Matlo C, Bobrovitskiy C, Waldoch A, Fang ML, Jackson P, Mihailidis A, Nygård L, Astell A, Sixsmith A (2016) Ambient assisted living technologies for aging well: a scoping review. J Intell Syst 25(1):55–69

    Article  Google Scholar 

  9. Reissner L, Gabriella F, List R, Giovanoli P, Calcagni M (2019) Assessment of hand function during activities of daily living using motion tracking cameras: a systematic review. Proc Inst Mech Eng H J Eng Med 233:095441191985130

    Article  Google Scholar 

  10. Martin SK, Jude HD (2017) Hand posture and gesture recognition techniques for virtual reality applications: a survey. Virtual Real 21(2):91–107

    Article  Google Scholar 

  11. Rodríguez M, Orrite C, Medrano C, Makris D (2017) Fast simplex-hmm for one-shot learning activity recognition. In: 2017 IEEE conference on computer vision and pattern recognition workshops, CVPR workshops 2017, Honolulu, HI, USA, July 21–26, 2017. IEEE Computer Society, pp 1259–1266

  12. Martin SK, Jude HD (2019) A probabilistic model for state sequence analysis in hidden Markov model for hand gesture recognition. Comput Intell 35(1):59–81

    Article  MathSciNet  Google Scholar 

  13. Pirsiavash H, Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE

  14. McCandless T, Grauman K (2013) Object-centric spatio-temporal pyramids for egocentric activity recognition. In: British machine vision conference, BMVC 2013, Bristol, UK, September 9–13, 2013

  15. Matsuo K, Yamada K, Ueno S, Naito S (2014) An attention-based activity recognition for egocentric video. In: IEEE conference on computer vision and pattern recognition, CVPR workshops 2014, Columbus, OH, USA, June 23–28, 2014, pp 565–570

  16. Wang H, Schmid C (2013) Action recognition with improved trajectories. In: 2013 IEEE international conference on computer vision, IEEE, Sydney, Australia, Dec 1–8 2013. https://doi.org/10.1109/ICCV.2013.441

  17. Fathi A, Farhadi A, Rehg JM (2011) Understanding egocentric activities. In: 2011 IEEE international conference on computer vision (ICCV). IEEE, pp 407–414

  18. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. IEEE Conf Comput Vis Pattern Recognit (CVPR) 2:2169–2178

    Google Scholar 

  19. Zhou Y, Ni B, Hong R, Yang X, Tian Q (2016) Cascaded interactional targeting network for egocentric video analysis. In: 2016 IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016. IEEE Computer Society, pp 1904–1913

  20. Wang M, Luo C, Ni B, Yuan J, Wang J, Yan S (2018) First-person daily activity recognition with manipulated object proposals and non-linear feature fusion. IEEE Trans Circuits Syst Video Technol 28(10):2946–2955

    Article  Google Scholar 

  21. Zou Z, Shi Z, Guo Y, Ye J (2019) Object detection in 20 years: a survey. CoRR. arXiv:1905.05055

  22. Liu L, Ouyang W, Wang X, Fieguth PW, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128(2):261–318

    Article  Google Scholar 

  23. Jiao L, Zhang F, Liu F, Yang S, Li L, Feng Z, Rong Q (2019) A survey of deep learning-based object detection. IEEE Access 7:128837–128868

    Article  Google Scholar 

  24. Rodriguez M, Orrite C, Medrano C (2017) Space-time flexible kernel for recognizing activities from wearable cameras. In: Alexandre LA, Sánchez JS, Rodrigues JMF (eds) Pattern recognition and image analysis. Springer, Cham, pp 511–518

    Chapter  Google Scholar 

  25. Jebara T, Kondor R, Howard A (2004) Probability product kernels. J Mach Learn Res 5:819–844

    MathSciNet  MATH  Google Scholar 

  26. Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, New York

    MATH  Google Scholar 

  27. Choi J, Wang Z, Lee S-C, Jeon WJ (2013) A spatio-temporal pyramid matching for video retrieval. Comput Vis Image Underst 117(6):660–669

    Article  Google Scholar 

  28. Zhang S, Nugent C, Lundström J, Sheng M (2018) Ambient assisted living for improvement of health and quality of life. Informatics 5(1):4

    Article  Google Scholar 

  29. Damen D, Doughty H, Farinella GM, Fidler S, Furnari A, Kazakos E, Moltisanti D, Munro J, Perrett T, Price W, Wray M (2020) The EPIC-KITCHENS dataset: collection, challenges and baselines. CoRR. arXiv:2005.00343

Download references

Acknowledgements

This work was supported by the Spanish Project TIN2017-88841-R (M. de Economia, Industria y Competitividad/FEDER, UE).

Author information

Authors and Affiliations

  1. ITCL, Burgos, Spain

    Mario Rodriguez

  2. CVLab, I3A, Zaragoza University, Zaragoza, Spain

    Carlos Orrite

  3. EduQTech, IIS, Zaragoza University, Teruel, Spain

    Carlos Medrano

Authors
  1. Mario Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Orrite
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carlos Medrano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Orrite.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodriguez, M., Orrite, C. & Medrano, C. Space-time flexible kernel for recognizing activities from wearable cameras. Pattern Anal Applic 24, 843–852 (2021). https://doi.org/10.1007/s10044-020-00942-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-020-00942-0

Keywords

Search

Navigation