Skip to main content
SpringerLink
Log in

Human behavior sensing: challenges and approaches

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

In recent years, Activities of Daily Living Scale (ADLs) is widely used to evaluate living abilities of the patients and the elderly. So, the study of behavior sensing has attracted more and more attention of researchers. Behavior sensing technology is of strong theoretical and practical value in the fields of smart home and virtual reality. Most of the currently proposed approaches for tracking indicators of ADLs are human-centric, which classify activities using physical information of the observed persons. Considering the privacy concerns of the human-centric approaches (e.g. images of home environment, private behavior), researchers have also proposed some thing-centric approaches, which use environmental information on things (e.g. the vibration of things) to infer human activity. In this paper, by considering the unified steps in both the human-centric approaches and the thing-centric approaches, we make a comprehensive survey on the challenges and proposed methods to do behavior sensing, which are signal collection, preprocessing, feature extraction, and activity recognition. Moreover, based on the latest research progress, we post a perspective from our standpoint, discussing future outlook and challenges of human behavior sensing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Aceto G, Ciuonzo D, Montieri A, Pescapè A (2018) Multi-classification approaches for classifying mobile APP traffic. J Netw Comput Appl 103(C):131–145

    Article  Google Scholar 

  • Aceto G, Ciuonzo D, Montieri A, Pescapè A (2019a) Know your big data trade-offs when classifying encrypted mobile traffic with deep learning. In: IEEE/IFIP network traffic measurement and analysis conference (TMA) 2019, pp 121–128

  • Aceto G, Ciuonzo D, Montieri A, Pescapè A (2019b) Mobile encrypted traffic classification using deep learning: experimental evaluation, lessons learned, and challenges. IEEE Trans Netw Serv Manage 16(2):445–458

    Article  Google Scholar 

  • Adib F, Kabelac Z, Katabi D et al (2014) 3D tracking via body radio reflections. In: USENIX NSDI 2014, pp 317–329

  • Alemdar H, Ersoy C (2017) Multi-resident activity tracking and recognition in smart environments. J Ambient Intell Humaniz Comput 8:513–529

    Article  Google Scholar 

  • Ali K, Liu AX, Wang W et al (2015) Keystroke recognition using WiFi signals. In: ACM MobiCom, pp 90–102

  • Aly H, Youssef M (2013) New Insights into WiFi-based Device-Free Localization. In: ACM UbiComp, pp 541–548

  • Cai JF, Cands EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  Google Scholar 

  • Caine K, Fisk A, Rogers W (2016) Benefits and privacy concerns of a home equipped with a visual sensing system: a perspective from older adults. In: The human factors and ergonomics society annual meeting 2016

  • Casale P, Pujol O, Radeva P (2011) Human activity recognition from accelerometer data using a wearable device. In: Vitrià J, Sanches JM, Hernández M (eds) Pattern recognition and image analysis. IbPRIA 2011. Lecture notes in computer science, vol 6669. Springer, Berlin, Heidelberg, pp 289–296

  • Chen H, Liu X, Zhao Z, et al (2019) TaRad: a thing-centric sensing system for detecting activities of daily living. In: the 12th international conference on internet and distributed computing systems (IDCS), Napoli, Italy 2019

  • Chikhaoui B, Ye B, Mihailidis A (2017) Feature-level combination of skeleton joints and body parts for accurate aggressive and agitated behavior recognition. J Ambient Intell Humaniz Comput 8:957–976

    Article  Google Scholar 

  • Chikhaoui B, Ye B, Mihailidis A (2018) Aggressive and agitated behavior recognition from accelerometer data using non-negative matrix factorization. J Ambient Intell Humaniz Comput 9(5):1375–1389

    Article  Google Scholar 

  • Daubechies I, Heil C (1992) Ten lectures on wavelets. Comput Phys 61

  • Debes C, Merentitis A, Sukhanov S et al (2016) Monitoring activities of daily living in smart homes: understanding human behavior. IEEE Signal Process Mag 33(2):81–94

    Article  Google Scholar 

  • Dickerson R, Gorlin E, Stankovic J (2011) Empath: a continuous remote emotional health monitoring system for depressive illness. In: ACM conference on wireless health, pp 5–14

  • Erickson VL, Carreira-Perpinan MA, Cerpa AE (2011) OBSERVE: occupancy-based system for efficient reduction of HVAC energy. In: ACM IPSN 2011, pp 258–269

  • Fornasier M, Rauhut H, Ward R (2011) Low-rank matrix recovery via iteratively reweighted least squares minimization. SIAM J Optim 21(4):1614–1640

    Article  MathSciNet  Google Scholar 

  • Franco GC, Gallay F, Berenguer M, Mourrain C, Couturier P (2008) Non-invasive monitoring of the activities of daily living of elderly people at home—a pilot study of the usage of domestic appliances. J Telemed Telec 14(5):231–235

    Article  Google Scholar 

  • Fujinami T, Miura M, Takatsuka R, et al (2011) Toward useful services for elderly and people with disabilities, Springer, chap a study of long term tendencies in residents activities of daily living at a group home for people with dementia using RFID slippers, pp 303–307

  • Galluzzi V, Herman T, Polgreen P (2015) Hand hygiene duration and technique recognition using wrist-worn sensors. In: ACM IPSN 2015, pp 106–117

  • Hao T, Xing G, Zhou G (2013) iSleep: unobtrusive sleep quality monitoring using smartphones. In: ACM SenSys 2013, p 4

  • Herath S, Harandi M, Porikli F (2017) Going deeper into action recognition: a survey. Image Vis Comput 4–21

  • Keally M, Zhou G, Xing G et al (2011) Pbn: towards practical activity recognition using smartphone-based body sensor networks. In: ACM SenSys, pp 246–259

  • Kellogg B, Talla V, Gollakota S (2014) Bringing gesture recognition to all devices. In: USENIX NSDI 2014, pp 303–316

  • Kerola T, Inoue N, Shinoda K (2014) Spectral graph skeletons for 3D action recognition. In: IEEE ACCV 2014, pp 417–432

  • Kosba AE, Saeed A, Youssef M (2012) Robust WLAN device-free passive motion detection. In: IEEE WCNC 2012, pp 3284–3289

  • Lao W, Han J, de With P (2009) Automative video-based human motion analysis for consumer surveillance system. IEEE Trans Consumer Electron 55(2):591–598

    Article  Google Scholar 

  • Lawton M, Brody E (1970) Assessment of older people: self-maintaining and instrumental activities of daily living. Nurs Res 19(3):278

    Article  Google Scholar 

  • Lee S, Kim Y, Ahn D et al (2015) Non-obstructive room-level locating system in home environments using activity fingerprints from smartwatch. In: ACM UbiComp, pp 939–950

  • Li H, Yang W, Wang J et al (2016a) Wi-Finger: talk to your smart devices with finger-grained gesture. In: ACM UbiComp 2016, pp 250–261

  • Li X, Zhang Y, Li M, Marsic I, Yang JW et al (2016b) Deep neural network for RFID-based activity recognition. In: ACM MobiCom 2016, pp 24–26

  • Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:10095055

  • Ling K, Dai H, Liu Y et al (2018) UltraGesture: fine-grained gesture sensing and recognition. In: IEEE SECON 2018, pp 1–9

  • Liu Q, Liu Z (2012) A comparison of improving multi-class imbalance for internet traffic classification. Inf Syst Front 16(3):509–521

    Article  Google Scholar 

  • Liu X, Chen H, Zhang X, et al (2019) Human Action Counting and Recognition with Wi-Fi Signals. In: The 4th International Conference on Computing, Communications and Security (ICCCS), Rome, Italy 2019

  • Liu J, Wang Y, Chen Y et al (2015) Tracking vital signs during sleep leveraging off-the-shelf WiFi. In: ACM MobiHoc 2015, pp 267–276

  • Lyonnet B, Ioana C, Amin MG (2010) Human gait classification using microdoppler time-frequency signal representations. In: IEEE Radar Conference 2010, pp 915–919

  • Mitchell D, Morrow P, Nugent C (2014) A sensor and video based ontology for activity recognition in smart environments. In: IEEE EMBS 2014

  • Morales F, Toledo P, Sanchis A (2013) Activity recognition using hybrid generative/discriminative models on home environments using binary sensors. Sensors 13(5):5460–5477

    Article  Google Scholar 

  • Nguyen V, Ibrahim M, Rupavatharam S, et al (2018) EyeLight: light-based occupancy estimation and activity recognition from shadows on the floor. In: IEEE INFOCOM 2018

  • O’Shea TJ, West N, Vondal M et al (2017) Semi-supervised radio signal identification. In: IEEE ICACT 2017, pp 33–38

  • Pu Q, Gupta S, Gollakota S et al (2013) Whole-home gesture recognition using wireless signals. In: ACM SIGCOMM 2013, pp 27–38

  • Sabek I, Youssef M (2012) Multi-entity device-free WLAN localization. In: IEEE GLOBECOM 2012, pp 2018–2023

  • Sigg S, Shi S, Ji Y (2013) RF-based Device-free recognition of simultaneously conducted activities. In: ACM UbiComp 2013, pp 531–540

  • Spagnolo P, Mazzeo PL, Distante C (eds) (2014) Human behavior understanding in networked sensing. Springer, Berlin

    Google Scholar 

  • Srivastava M, Abdelzaher T, Szymanski B (2012) Human-centric sensing. Philos Trans 176–197

  • Tan S, Yang J (2016) WiFinger: leveraging commodity WiFi for fine-grained finger gesture recognition. In: ACM MobiHoc 2016, pp 201–210

  • Wang Y, Liu J, Chen Y et al (2014) E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures. In: ACM MobiCom 2014, pp 617–628

  • Wang W, Liu AX, Shahzad M et al (2015) Understanding and modelling of WiFi signal based human activity recognition. In: ACM MobiCom 2015, pp 65–76

  • Wang Y, Wu K, Ni LM (2016) Wifall: device-free fall detection by wireless networks. In: IEEE INFOCOM 2016, pp 581–594

  • Wilson J, Patwari N (2011) See-through walls: motion tracking using variance based radio tomography networks. IEEE Trans Mob Comput 10(5):612–621

    Article  Google Scholar 

  • World Health Organization (2011) Global Health and Ageing. Tech. rep., US National Institute of Aging

  • Wu CS, Yang Z, Zhou Z et al (2015) Non-invasive detection of moving and stationary human with WiFi. IEEE JSAC 33(11):2329–2342

    Google Scholar 

  • Wu X, Chu Z, Yang P et al (2018) TW-See: human activity recognition through the wall with commodity WiFi devices. IEEE Trans Veh Technol 68(1):306–319

    Article  Google Scholar 

  • Xi W, Zhao J, Li XY et al (2014) Electronic Frog Eye: Counting Crowd using WiFi. In: IEEE INFOCOM 2014, pp 361–369

  • Xin T et al (2018) FreeSense: human-behavior understanding using Wi-Fi signals. J Ambient Intell Humaniz Comput 9(5):1611–1622

    Article  Google Scholar 

  • Xu Y, Yang W, Wang J, et al (2018) WiStep: Device-free step counting with WiFi signals. ACM IMWUT 2018, p 172

  • Yang J, Lee J, Choi J (2011) Activity recognition based on RFID object usage for smart mobile devices. J Comput Sci Technol 26(2):239–246

    Article  Google Scholar 

  • Yang X, Tian Y (2012) Eigen joints-based action recognition using Naive–Bayes-nearest-neighbor. In: IEEE CVPRW, pp 14–19

  • Yang Y, Hao J, Luo J, Pan SJ (2017) Ceilingsee: device-free occupancy inference through lighting infrastructure based LED sensing. In: IEEE Percom 2017

  • Yatani K, Truong KN (2012) Bodyscope: a Wearable Acoustic Sensor for Activity Recognition. In: ACM UbiComp 2012, pp 341–350

  • Yun J, Lee SS (2014) Human movement detection and identification using pyroelectric infra-red sensors. Sensors 14(5):8057–8081

    Article  Google Scholar 

  • Zeng Y, Pathak PH, Mohapatra P (2016) WiWho: Wifi-based Person Identification in Smart Spaces. In: ACM IPSN 2016, p 4

  • Zhou Z, Yang Z, Wu C et al (2013) Towards omni-directional passive human detection. In: IEEE INFOCOM 2013, pp 3057–3065

Download references

Acknowledgements

Partial work of this paper is supported by the Zhejiang Provincial Natural Science Foundation of China (LY18F020011), Ningbo Natural Science Foundation (2018A610154) and the K. C. Wong Magna Fund in Ningbo University.

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China

    Xiwen Liu & Haiming Chen

  2. Department of Electrical Engineering and Information Technology, University of Napoli Federico II, Naples, Italy

    Antonio Montieri & Antonio Pescapè

Authors
  1. Xiwen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Montieri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonio Pescapè
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiming Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Chen, H., Montieri, A. et al. Human behavior sensing: challenges and approaches. J Ambient Intell Human Comput 11, 6043–6058 (2020). https://doi.org/10.1007/s12652-020-01861-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-01861-y

Keywords

Search

Navigation