Skip to main content
Springer Nature Link
Log in

Extended SESIM: A Tool to Support the Generation of Synthetic Datasets for Human Activity Recognition

  • Conference paper
  • First Online:
The Role of Digital Technologies in Shaping the Post-Pandemic World (I3E 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13454))

Included in the following conference series:

Abstract

The accessibility of datasets that capture the performance of Activities of Daily Living is limited by the difficulties in setting up test beds. The Covid-19 pandemic recently compounded such challenges. Smart Environments employed as test-beds consist of sensors and applications formulated to develop a comfortable and safe environment for their inhabitants. Despite the increase in quantities of Smart Environments, accessibility of these spaces for researchers has become even more challenging amidst a pandemic. Computing power has enabled researchers to generate virtual Smart Environments with fewer overheads and less complexity. This article proposes an Extended Smart Environment Simulator (ESESIM), with multiple inhabitants possibly utilised for dataset generation. The proposed simulation tool has a virtual space with multiple script-regulated inhabitants. While the various inhabitants probe the Smart Environment, sensor readings are recorded and stored in a dataset. The virtual space developed in this study generated synthetic datasets that can be employed for Human Activity recognition in machine learning. This study also evaluated two deep machine learning models and performance mechanisms in recognising four activities of daily living, namely personal hygiene, dressing, cooking and sleeping, on the SESIM dataset. Findings from this study indicate that simulation can be used as a tool for generating human activity datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. World Health Organization: Coronavirus (COVID-19) events as they happen (2020). https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen. Accessed 03 Apr 2022

  2. Khan, H., Kushwah, K.K., Singh, S., Urkude, H., Maurya, M.R., Sadasivuni, K.K.: Smart technologies driven approaches to tackle COVID-19 pandemic: a review. 3 Biotech 11(2), 1–22 (2021). https://doi.org/10.1007/s13205-020-02581-y

    Article  Google Scholar 

  3. Ribeiro-Navarrete, S., Saura, J.R., Palacios-Marqués, D.: Towards a new era of mass data collection: assessing pandemic surveillance technologies to preserve user privacy. Technol. Forecast. Soc. Change 167 (2021). https://doi.org/10.1016/J.TECHFORE.2021.120681

  4. Uelschen, M., Schaarschmidt, M.: Software design of energy-aware peripheral control for sustainable internet-of-things devices. In: Proceedings of the 55th Hawaii International Conference on System Sciences, vol. 7, pp. 7762–7771 (2022). https://doi.org/10.24251/hicss.2022.933

  5. Shalaby, E., ElShennawy, N., Sarhan, A.: Utilizing deep learning models in CSI-based human activity recognition. Neural Comput. Appl. 34(8), 5993–6010 (2021). https://doi.org/10.1007/s00521-021-06787-w

    Article  Google Scholar 

  6. Cedillo, P., Sanchez, C., Campos, K., Bermeo, A.: A systematic literature review on devices and systems for ambient assisted living: solutions and trends from different user perspectives (2018). https://doi.org/10.1109/ICEDEG.2018.8372367

  7. Zhang, S., et al.: Deep learning in human activity recognition with wearable sensors: a review on advances. Sensors 22(4), 1476 (2022). https://doi.org/10.3390/s22041476

    Article  Google Scholar 

  8. Dang, L.M., Min, K., Wang, H., Piran, M.J., Lee, C.H., Moon, H.: Sensor-based and vision-based human activity recognition: a comprehensive survey. Pattern Recogn. 108, 107561 (2020). https://doi.org/10.1016/j.patcog.2020.107561

  9. Chiridza, T.: A smart home environment to support saftey and risk monitoring for the elderly living independently. Nelson Mandela University (2017)

    Google Scholar 

  10. Kim, Y., An, J., Lee, M., Lee, Y.: An activity-embedding approach for next-activity prediction in a multi-user smart space (2017). https://doi.org/10.1109/SMARTCOMP.2017.7946985

  11. Jalal, A., Mahmood, M., Hasan, A.S.: Multi-features descriptors for human activity tracking and recognition in indoor-outdoor environments. In: Proceedings of 2019 16th International Bhurban Conference on Applied Sciences and Technology (IBCAST) 2019, pp. 371–376 (2019). https://doi.org/10.1109/IBCAST.2019.8667145

  12. Nafea, O., Abdul, W., Muhammad, G., Alsulaiman, M.: Sensor-based human activity recognition with spatio-temporal deep learning. Sensors 21(6), 1–20 (2021). https://doi.org/10.3390/s21062141

    Article  Google Scholar 

  13. Awad, M.M.: Forest mapping: a comparison between hyperspectral and multispectral images and technologies. J. For. Res. 29(5), 1395–1405 (2017). https://doi.org/10.1007/s11676-017-0528-y

    Article  Google Scholar 

  14. Gupta, S.: Deep learning based human activity recognition (HAR) using wearable sensor data. Int. J. Inf. Manag. Data Insights 1(2), 100046 (2021). https://doi.org/10.1016/j.jjimei.2021.100046

    Article  Google Scholar 

  15. Cao, C., et al.: Deep learning and its applications in biomedicine. Genomics Proteomics Bioinf. 16(1), 17–32 (2018). https://doi.org/10.1016/j.gpb.2017.07.003

  16. Lee, Y., Choi, T.J., Ahn, C.W.: Multi-objective evolutionary approach to select security solutions. CAAI Trans. Intell. Technol. 2(2), 64–67 (2017). https://doi.org/10.1049/trit.2017.0002

    Article  Google Scholar 

  17. Irvine, N., Nugent, C., Zhang, S., Wang, H., Ng, W.W.Y.: Neural network ensembles for sensor-based human activity recognition within smart environments. Sensors (Switzerland) 20(1) (2020). https://doi.org/10.3390/s20010216

  18. Ho, B., Vogts, D., Wesson, J.: A smart home simulation tool to support the recognition of activities of daily living. In: ACM International Conference Proceeding Series (2019). https://doi.org/10.1145/3351108.3351132

  19. Nugent, C., et al.: Improving the quality of user generated data sets for activity recognition. In: García, C.R., Caballero-Gil, P., Burmester, M., Quesada-Arencibia, A. (eds.) UCAmI/IWAAL/AmIHEALTH 2016. LNCS, vol. 10070, pp. 104–110. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48799-1_13

    Chapter  Google Scholar 

  20. Friday Nweke, H., Wah Teh, Y., Al-Garadi, M.A., Alo, R.: Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst. Appl. 105, 233–261 (2018). https://doi.org/10.1016/j.eswa.2018.03.056

  21. Fysarakis, K., Soultatos, O., Manifavas, C., Papaefstathiou, I., Askoxylakis, I.: XSACd—cross-domain resource sharing & access control for smart environments. Futur. Gener. Comput. Syst. 80, 572–582 (2018). https://doi.org/10.1016/j.future.2016.05.023

    Article  Google Scholar 

  22. Dorri, A., Kanhere, S.S., Jurdak, R., Gauravaram, P.: Blockchain for IoT security and privacy: the case study of a smart home. In: 2017 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), pp. 618–623 (2017) https://doi.org/10.1109/PERCOMW.2017.7917634

  23. Nazari Shirehjini, A.A., Semsar, A.: Human interaction with IoT-based smart environments. Multimed. Tools Appl. 76(11), 13343–13365 (2016). https://doi.org/10.1007/s11042-016-3697-3

    Article  Google Scholar 

  24. Alshammari, N., Alshammari, T., Sedky, M., Champion, J., Bauer, C.: OpenSHS: open smart home simulator. Sensors 17(5), 1003 (2017). https://doi.org/10.3390/s17051003

    Article  Google Scholar 

  25. Lee, J.W., Helal, A., Sung, Y., Cho, K.: Context-driven control algorithms for scalable simulation of human activities in smart homes. In: Proceedings - IEEE 10th International Conference on Ubiquitous Intelligence and Computing, UIC 2013 and IEEE 10th International Conference on Autonomic and Trusted Computing, ATC 2013, pp. 285–292 (2013). https://doi.org/10.1109/UIC-ATC.2013.68

  26. Wixom, B.H., Watson, H.J., Reynolds, A.M., Hoffer, J.A.: Continental airlines continues to soar with business intelligence (2015)

    Google Scholar 

  27. Ho, B., Vogts, D., Wesson, J.: SESim: a smart environment simulation tool to support human activity recognition (2018)

    Google Scholar 

  28. Lee, J.W., Cho, S., Liu, S., Cho, K., Helal, S.: Persim 3D: context-driven simulation and modelling of human activities in smart spaces. IEEE Trans. Autom. Sci. Eng. 12, 1243–1256 (2015). https://doi.org/10.1109/TASE.2015.2467353

    Article  Google Scholar 

  29. Forbes, G.: Employing multi-modal sensors for personalised smart home health monitoring (2019). www.rgu.ac.uk/dmstaff/forbes-glenn. Accessed 28 June 2020

  30. Kormányos, B., Pataki, B.: Multi-level simulation of daily activities: why and how? In: Proceedings of the 2013 IEEE International Conference Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), pp. 1–6 (2013). https://doi.org/10.1109/CIVEMSA.2013.6617386

  31. Oort, Q., Taphoorn, M.J.B., Sikkes, S.A.M., Uitdehaag, B.M.J., Reijneveld, J.C., Dirven, L.: Evaluation of the content coverage of questionnaires containing basic and instrumental activities of daily living (ADL) used in adult patients with brain tumors. J. Neurooncol. 143(1), 1–13 (2019). https://doi.org/10.1007/s11060-019-03136-9

    Article  Google Scholar 

  32. Spector, W.D., Katz, S., Murphy, J.B., Fulton, J.P.: The hierarchical relationship between activities of daily living and instrumental activities of daily living. J. Chronic Dis. (1987). https://doi.org/10.1016/0021-9681(87)90004-X

  33. Qian, H., Pan, S.J., Da, B., Miao, C.: A novel distribution-embedded neural network for sensor-based activity recognition. In: IJCAI International Joint Conference on Artificial Intelligence, vol. 2019, pp. 5614–5620 (2019). https://doi.org/10.24963/ijcai.2019/779

  34. Reyes-Ortiz, J.L., Oneto, L., Samà, A., Parra, X., Anguita, D.: Transition-aware human activity recognition using smartphones. Neurocomputing (2016). https://doi.org/10.1016/j.neucom.2015.07.085

  35. Chiristian Debes, M.N., Sukhanov, S., Matheas, A., et al.: Monitoring activities of daily living in smart homes: understanding human behaviour. IEEE Signal Process. Mag. 33(2), 81–94 (2016). https://doi.org/10.1109/MSP.2015.2503881

  36. Roy, N., Misra, A., Cook, D.: Ambient and smartphone sensor assisted ADL recognition in multi-inhabitant smart environments. J. Ambient. Intell. Humaniz. Comput. 7(1), 1–19 (2015). https://doi.org/10.1007/s12652-015-0294-7

    Article  Google Scholar 

  37. Ferrari, A., Micucci, D., Mobilio, M., Napoletano, P.: Deep learning and model spersonalisation in sensor-based human activity recognition. J. Reliab. Intell. Environ. (2022). https://doi.org/10.1007/s40860-021-00167-w

  38. Bradfield, K., Allen, C.: Advances in Informatics and Computing in Civil and Construction Engineering. Springer, Cham (2019)

    Google Scholar 

  39. Leodolter, M., Widhalm, P., Plant, C., Brandle, N.: Semi-supervised segmentation of accelerometer time series for transport mode classification (2017). https://doi.org/10.1109/MTITS.2017.8005596

  40. ECMA-404: The JSON data interchange format. ECMA Int. (2013). https://doi.org/10.17487/rfc7158

  41. Chauhan, N.K., Singh, K.: A review on conventional machine learning vs deep learning. In: 2018 International Conference on Computing, Power and Communication Technologies (GUCON), pp. 347–352 (2018). https://doi.org/10.1109/GUCON.2018.8675097

  42. Wang, H., et al.: Wearable sensor-based human activity recognition using hybrid deep learning techniques. Secur. Commun. Netw. 2020, 1–12 (2020). https://doi.org/10.1155/2020/2132138

    Article  Google Scholar 

  43. Murad, A., Pyun, J.Y.: Deep recurrent neural networks for human activity recognition. Sensors (Switzerland) 17(11), 2556 (2017). https://doi.org/10.3390/s17112556

  44. Inoue, M., Inoue, S., Nishida, T.: Deep recurrent neural network for mobile human activity recognition with high throughput. Artif. Life Robot. 23(2), 173–185 (2017). https://doi.org/10.1007/s10015-017-0422-x

    Article  Google Scholar 

  45. Hamad, R.A., Kimura, M., Yang, L., Woo, W.L., Wei, B.: Dilated causal convolution with multi-head self attention for sensor human activity recognition. Neural Comput. Appl. 5 (2021). https://doi.org/10.1007/s00521-021-06007-5

  46. Zebin, T., Sperrin, M., Peek, N., Casson, A.J.: Human activity recognition from inertial sensor time-series using batch snormalised deep LSTM recurrent networks. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, vol. 2018-July, pp. 1–4 (2018). https://doi.org/10.1109/EMBC.2018.8513115

  47. Kim, Y., Toomajian, B.: Hand gesture recognition using micro-doppler signatures with convolutional neural network. IEEE Access (2016). https://doi.org/10.1109/ACCESS.2016.2617282

  48. Badura, M., Batog, P., Drzeniecka-Osiadacz, A., Modzel, P.: Evaluation of low-cost sensors for ambient PM2.5 monitoring. J. Sensors (2018). https://doi.org/10.1155/2018/5096540

Download references

Author information

Authors and Affiliations

  1. Nelson Mandela University, Gqeberha, 6001, South Africa

    Timothy Musharu & Dieter Vogts

Authors
  1. Timothy Musharu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dieter Vogts
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy Musharu .

Editor information

Editors and Affiliations

  1. Newcastle University Business School, Newcastle upon Tyne, UK

    Savvas Papagiannidis

  2. Newcastle University Business School, Newcastle upon Tyne, UK

    Eleftherios Alamanos

  3. Newcastle University Business School, Newcastle upon Tyne, UK

    Suraksha Gupta

  4. Swansea University, Swansea, UK

    Yogesh K. Dwivedi

  5. University of Turku, Turku, Finland

    Matti Mäntymäki

  6. University of Agder, Kristiansand, Norway

    Ilias O. Pappas

Rights and permissions

Reprints and permissions

Copyright information

© 2022 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Musharu, T., Vogts, D. (2022). Extended SESIM: A Tool to Support the Generation of Synthetic Datasets for Human Activity Recognition. In: Papagiannidis, S., Alamanos, E., Gupta, S., Dwivedi, Y.K., Mäntymäki, M., Pappas, I.O. (eds) The Role of Digital Technologies in Shaping the Post-Pandemic World. I3E 2022. Lecture Notes in Computer Science, vol 13454. Springer, Cham. https://doi.org/10.1007/978-3-031-15342-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-15342-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-15341-9

  • Online ISBN: 978-3-031-15342-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships