Skip to main content
SpringerLink
Log in

Ontology-driven semantic unified modelling for concurrent activity recognition (OSCAR)

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Activity recognition has a vital role in smart home operations. One of the major challenges in object-sensor-based activity recognition is to learn the complete activity model derived from a generic activity model for sequential and parallel activities. Such challenge exists due to erratic degrees of dissimilar activities in which inhabitants perform activities in sequential and interleaved fashion while interacting with different objects. The proposed work focuses on recognizing a complete set of actions (of activity) by exploiting different knowledge engineering techniques, ontology-based temporal formalisms and data driven techniques. Semantic Segmentation has been employed to establish the generic activity model. The spurious semantic segmentation produced by sensor noise or erratic behaviour is removed by Allen’s temporal formalism. Moreover, Tversky’s feature-based similarity has been used to remove the highly similar spurious activities produced as a result of mistaken interactions with wrong home objects. The duration to perform activities varies among inhabitants; such duration intervals are identified dynamically using the proposed model in order to have a complete activity model. A comprehensive set of experiments has been carried out for evaluating the proposed model where the results based upon different metrics assert its effectiveness especially when compared with other contemporary techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Allen JF (1983) Maintaining knowledge about temporal intervals. Commun ACM 26.11:832–843

    Article  Google Scholar 

  2. Allen JF, Ferguson G (1994) Actions and events in interval temporal logic. J Log Comput 4.5:531–579

    Article  MathSciNet  Google Scholar 

  3. Azkune G et al (2015) Extending knowledge-driven activity models through data-driven learning techniques. Expert Syst Appl 42.6:3115–3128

    Article  Google Scholar 

  4. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In International Conference on Pervasive Computing (pp. 1–17). Springer Berlin Heidelberg

  5. Caragliu A, Del Bo C, Nijkamp P (2011) Smart cities in Europe. J Urban Technol 18(2):65–82

    Article  Google Scholar 

  6. Chen L, Nugent C (2009) Ontology-based activity recognition in intelligent pervasive environments. Int J Web Info Syst 5(4):410–430

    Article  Google Scholar 

  7. Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24(6):961–974

    Article  Google Scholar 

  8. Chen L, Hoey J, Nugent CD, Cook DJ, Yu Z (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C Appl Rev 42(6):790–808

    Article  Google Scholar 

  9. Chen L, Nugent C, Okeyo G (2014) An ontology-based hybrid approach to activity modeling for smart homes. IEEE Trans Human-Mach Syst 44(1):92–105

    Article  Google Scholar 

  10. Cook D, Schmitter-Edgecombe M, Crandall A, Sanders C, Thomas B (2009) Collecting and disseminating smart home sensor data in the CASAS project. Proc CHI Workshop Dev Shared Home Behav Datasets Adv HCI Ubiquitous Comput Res: 1–7)

  11. Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low-and high-dimensional approaches. IEEE Trans Syst, Man, Cybernet: Syst 43(4):996–1002

    Article  Google Scholar 

  12. Díaz-Rodríguez N, Cadahía OL, Cuéllar MP, Lilius J, Calvo-Flores MD (2014) Handling real-world context awareness, uncertainty and vagueness in real-time human activity tracking and recognition with a fuzzy ontology-based hybrid method. Sensors 14(10):18131–18171

    Article  Google Scholar 

  13. Gayathri KS, Elias S, Shivashankar S (2015) Composite activity recognition in smart homes using Markov logic network, IEEE Computer Society

  14. Gayathri KS, Elias S, Shivashankar S (2015) Composite activity recognition in smart homes using Markov logic network. Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom), 2015 IEEE 12th Intl Conf on, pp. 46-53. IEEE

  15. Georis, B., Maziere, M., Bremond, F., and Thonnat, M, (2004) A video interpretation platform applied to bank agency monitoring. Proc 2nd workshop Intell Distrib Surv Syst: 46–50

  16. Gu T, Wu Z, Tao X, Pung HK, Lu J 2009, March. epsicar: an emerging patterns-based approach to sequential, interleaved and concurrent activity recognition. Pervasive Comput Commun, 2009. PerCom 2009. IEEE Int Conf (pp. 1–9). IEEE

  17. Gu T, Wang L, Wu Z, Tao X, Lu J (2011) A pattern mining approach to sensor-based human activity recognition. IEEE Trans Knowl Data Eng 23(9):1359–1372

    Article  Google Scholar 

  18. Gubbi J, Buyya R, Marusic S, Palaniswami M (2013) Internet of things (IoT): a vision, architectural elements, and future directions. Future Gen Comput Syst 29(7):1645–1660

    Article  Google Scholar 

  19. Hakeem A, Shah M (2004) Ontology and taxonomy collaborated framework for meeting classification. Proc Int Conf Pattern Recogn: 219–222

  20. Helaoui R, Niepert M, Stuckenschmidt H (2011) Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships. Perva Mobile Comput 7.6:660–670

    Article  Google Scholar 

  21. Helaoui R, Niepert M, Stuckenschmidt H (2011) Recognizing interleaved and concurrent activities: A statistical-relational approach. Perva Comput Commun (PerCom), 2011 IEEE Int Conf (pp. 1–9). IEEE

  22. Hemalatha M, Uma V, Aghila G (2012) Time ontology with reference event based temporal relations (retr). Int J Web Semantic Technol 3.1

  23. http://boxlab.wikispaces.com/List+of+Home+Datasets (accessed April 2018)

  24. Kakde A, Gulhane V (2015) Real time composite user activity modelling using hybrid approach for recognition. In Electrical, Computer and Communication Technologies (ICECCT), 2015 IEEE International Conference on (pp. 1–6). IEEE

  25. Khan S, Safyan M (2014) Semantic matching in hierarchical ontologies. J King Saud Univ-Comput Inf Sci 26(3):247–257

    Google Scholar 

  26. Likavec S, Cena F (2015) Property-based semantic similarity: what counts?. AIC

  27. Liu Y, Zhang X, Cui J, Wu C, Aghajan H, Zha H (2010) Visual analysis of child-adult interactive behaviors in video sequences. 16th Int Conf Virtual Syst Multimed (VSMM) 2010:26–33 IEEE

    Article  Google Scholar 

  28. Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. Pattern Recogn (ICPR), 2012 21st Int Conf (pp. 898–901). IEEE

  29. Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. IJCAI: 1617–1623

  30. Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model. AAAI 30:1266–1272

    Google Scholar 

  31. Liu Y, Zheng Y, Liang Y, Liu S, Rosenblum DS (2016) Urban water quality prediction based on multi-task multi-view learning

  32. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. AAAI: 201–207

  33. Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools Appl 76(8):10701–10719

    Article  Google Scholar 

  34. Mckeever S, Ye J, Coyle L, Bleakley C, Dobson S (2010) Activity recognition using temporal evidence theory. J Ambient Intell Smart Environ 2.3:253–269

    Google Scholar 

  35. Meditskos G, Kontopoulos E, Kompatsiaris I (2015) ReDef: context-aware recognition of interleaved activities using OWL 2 and defeasible reasoning. In SSN-TC/OrdRing@ ISWC: 31–42

  36. Milea V, Frasincar F, Kaymak U (2008) Knowledge engineering in a temporal semantic web context. Web Eng, 2008. ICWE'08. Eighth Int Conf. IEEE

  37. Nevatia R, Hobbs J, Bolles B (2004) An ontology for video event representation. Proc IEEE Workshop Event Detect Recognit: 119

  38. Okeyo G, Chen L, Wang H, Sterritt R (2012) A knowledge-driven approach to composite activity recognition in smart environments ubiquitous computing and ambient intelligence vol 7656 of the series lecture notes in computer science. Springer, Berlin Heidelberg, pp 322–329

    Google Scholar 

  39. Online referencing http://ailab.wsu.edu/casas/datasets/ (accessed April 2018)

  40. Pantelopoulos A, Bourbakis NG (2010) A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Trans Syst Man Cybern Part C Appl Rev 40(1):1–12

    Article  Google Scholar 

  41. Poppe R 2010. A survey on vision-based human action recognition. Image and vision Pantelopoulos, A. and Bourbakis, N.G., 2010

  42. Riboni D, Bettini C (2011) COSAR: hybrid reasoning for context-aware activity recognition. Pers Ubiquit Comput 15(3):271–289

    Article  Google Scholar 

  43. Riboni D, Pareschi L, Radaelli L, Bettini C (2011) Is ontology-based activity recognition really effective?" Pervasive Computing and Communications Workshops (PERCOM Workshops), IEEE International Conference on. IEEE

  44. Riboni D, Sztyler T, Civitarese G, Stuckenschmidt H (2016) Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 1–12). ACM

  45. Singla G, Cook DJ, Schmitter-Edgecombe M (2009) Tracking activities in complex settings using smart environment technologies. Int J Biosci Psychiatr Technol (IJBSPT) 1(1):25

    Google Scholar 

  46. Tapia EM, Intille SS, Larson K (2004) Activity recognition in the home using simple and ubiquitous sensors. In International conference on pervasive computing (pp. 158–175). Springer, Berlin, Heidelberg

  47. Van Kasteren T, Alemdar H, Ersoy C (2011) Effective performance metrics for evaluating activity recognition methods. In ARCS 2011 VDE

  48. Welty C, Fikes R, Makarios S (2006) A reusable ontology for fluent in OWL. FOIS 150:226–236

    Google Scholar 

  49. Ye J, Stevenson G, Dobson S (2015) KCAR: a knowledge-driven approach for concurrent activity recognition. Perva Mobile Comput 19:47–70

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Iqra University Islamabad, Islamabad, Pakistan

    Muhammad Safyan

  2. University of Gujarat, Gujarat, Pakistan

    Muhammad Safyan, Zia Ul Qayyum & Sohail Sarwar

  3. Ontology Engineering Group, Universidad Politécnica de Madrid, Madrid, Spain

    Raúl García-Castro

  4. GC University, Lahore, Pakistan

    Mehtab Ahmed

Authors
  1. Muhammad Safyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zia Ul Qayyum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sohail Sarwar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raúl García-Castro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mehtab Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Safyan.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(RAR 15964 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Safyan, M., Qayyum, Z.U., Sarwar, S. et al. Ontology-driven semantic unified modelling for concurrent activity recognition (OSCAR). Multimed Tools Appl 78, 2073–2104 (2019). https://doi.org/10.1007/s11042-018-6318-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6318-5

Keywords

Search

Navigation