research-article

A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition

Authors:

Youngjae Chang,

Anton Isopoussu,

Fahim KawsarAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 4, Issue 1

Article No.: 39, Pages 1 - 30

https://doi.org/10.1145/3380985

Published: 18 March 2020 Publication History

Abstract

Wearable sensors are increasingly becoming the primary interface for monitoring human activities. However, in order to scale human activity recognition (HAR) using wearable sensors to million of users and devices, it is imperative that HAR computational models are robust against real-world heterogeneity in inertial sensor data. In this paper, we study the problem of wearing diversity which pertains to the placement of the wearable sensor on the human body, and demonstrate that even state-of-the-art deep learning models are not robust against these factors. The core contribution of the paper lies in presenting a first-of-its-kind in-depth study of unsupervised domain adaptation (UDA) algorithms in the context of wearing diversity -- we develop and evaluate three adaptation techniques on four HAR datasets to evaluate their relative performance towards addressing the issue of wearing diversity. More importantly, we also do a careful analysis to learn the downsides of each UDA algorithm and uncover several implicit data-related assumptions without which these algorithms suffer a major degradation in accuracy. Taken together, our experimental findings caution against using UDA as a silver bullet for adapting HAR models to new domains, and serve as practical guidelines for HAR practitioners as well as pave the way for future research on domain adaptation in HAR.

References

[1]

2017. New technology tracks food intake by monitoring wrist movements. http://gadgetsandwearables.com/2017/03/29/food-tracking/. Accessed: February 10, 2020.

[2]

P Aggarwal, Z Syed, X Niu, and N El-Sheimy. 2008. A standard testing and calibration procedure for low cost MEMS inertial sensors and units. The Journal of Navigation 61, 2 (2008), 323--336.

[3]

Ali Akbari and Roozbeh Jafari. 2019. Transferring activity recognition models for new wearable sensors with deep generative domain adaptation. In IPSN.

[4]

Allan, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind K. Dey, Tobias Sonne, and Mads Møller Jensen. 2015. Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, SenSys 2015, Seoul, South Korea, November 1-4, 2015. 127--140. https://doi.org/10.1145/2809695.2809718

[5]

Bandar Almaslukh, Abdel Artoli, and Jalal Al-Muhtadi. 2018. A Robust Deep Learning Approach for Position-Independent Smartphone-Based Human Activity Recognition. Sensors 18, 11 (2018), 3726.

[6]

Stephen A. Antos, Mark V. Albert, and Konrad P. Kording. 2014. Hand, belt, pocket or bag: Practical activity tracking with mobile phones. Journal of Neuroscience Methods 231 (2014), 22--30. https://doi.org/10.1016/j.jneumeth.2013.09.015 Motion Capture in Animal Models and Humans.

[7]

Phil Blunsom, Changhao Chen, Xiaoxuan Lu, Andrew Markham, Yishu Miao, Agathoniki Trigoni, and Linhai Xie. 2019. MotionTransformer: Transferring Neural Inertial Tracking Between Domains.

[8]

Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 3 (2014), 33.

Digital Library

[9]

Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang. 2018. Partial adversarial domain adaptation. In Proceedings of the European Conference on Computer Vision (ECCV). 135--150.

[10]

Liwei Chan, Chien-Ting Weng, Rong-Hao Liang, and Bing-Yu Chen. 2014. AnyButton: Unpowered, Modeless and Highly Available Mobile Input Using Unmodified Clothing Buttons. In Proceedings of the 5th Augmented Human International Conference (Kobe, Japan) (AH '14). ACM, New York, NY, USA, Article 24, 2 pages. https://doi.org/10.1145/2582051.2582075

Digital Library

[11]

Blunck et al. 2013. On heterogeneity in mobile sensing applications aiming at representative data collection. In Proceedings of the 2013 ACM Ubicomp. ACM, 1087--1098.

Digital Library

[12]

Davide Figo, Pedro C. Diniz, Diogo R. Ferreira, and João M. P. Cardoso. 2010. Preprocessing techniques for context recognition from accelerometer data. Personal and Ubiquitous Computing 14, 7 (01 Oct 2010), 645--662. https://doi.org/10.1007/s00779-010-0293-9

[13]

Jérémy Frey, May Grabli, Ronit Slyper, and Jessica R. Cauchard. 2018. Breeze: Sharing Biofeedback Through Wearable Technologies. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, Article 645, 12 pages. https://doi.org/10.1145/3173574.3174219

[14]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17, 1 (2016), 2096--2030.

Digital Library

[15]

Andreas Grammenos, Cecilia Mascolo, and Jon A Crowcroft. 2018. You Are Sensing, but Are You Biased?: A User Unaided Sensor Calibration Approach for Mobile Sensing. IMWUT 2 (2018), 11:1--11:26.

Digital Library

[16]

Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. Journal of Machine Learning Research 13, Mar (2012), 723--773.

Digital Library

[17]

Nils Y. Hammerla, Shane Halloran, and Thomas Plötz. 2016. Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables. In IJCAI.

[18]

W. Hang, W. Feng, R. Du, S. Liang, Y. Chen, Q. Wang, and X. Liu. 2019. Cross-Subject EEG Signal Recognition Using Deep Domain Adaptation Network. IEEE Access 7 (2019), 128273--128282. https://doi.org/10.1109/ACCESS.2019.2939288

[19]

F. Kawsar, C. Min, A. Mathur, and A. Montanari. 2018. Earables for Personal-Scale Behavior Analytics. IEEE Pervasive Computing 17, 3 (Jul 2018), 83--89. https://doi.org/10.1109/MPRV.2018.03367740

Digital Library

[20]

Adil Khan, Muhammad Siddiqi, and Seok-Won Lee. 2013. Exploratory Data Analysis of Acceleration Signals to Select Light-Weight and Accurate Features for Real-Time Activity Recognition on Smartphones. Sensors 13, 10 (Sep 2013), 13099--13122. https://doi.org/10.3390/s131013099

[21]

Md Abdullah Hafiz KHAN, Nirmalya Roy, and Archan Misra. 2018. Scaling human activity recognition via deep learning-based domain adaptation. (2018).

[22]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[23]

Diederik P. Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. CoRR abs/1312.6114 (2013).

[24]

Nicholas D Lane, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, and Andrew T Campbell. 2010. A survey of mobile phone sensing. IEEE Communications magazine 48, 9 (2010), 140--150.

Digital Library

[25]

Nicholas D. Lane, Ye Xu, Hong Lu, Shaohan Hu, Tanzeem Choudhury, Andrew T. Campbell, and Feng Zhao. 2011. Enabling large-scale human activity inference on smartphones using community similarity networks (csn). In UbiComp.

[26]

Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, and Timothy M. Hospedales. 2019. Episodic Training for Domain Generalization. arXiv:cs.CV/1902.00113

[27]

Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. 2013. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, Vol. 30. 3.

[28]

Akhil Mathur, Anton Isopoussu, Fahim Kawsar, Nadia Berthouze, and Nicholas Lane. 2019. FlexAdapt: Flexible Cycle-Consistent Domain Adaptation. In Proceedings of the IEEE International Conference on Machine Learning and Applications.

[29]

Chulhong Min, Akhil Mathur, Alessandro Montanari, and Fahim Kawsar. 2019. An Early Characterisation of Wearing Variability on Motion Signals for Wearables. In Proceedings of the 23rd International Symposium on Wearable Computers (London, United Kingdom) (ISWC '19). Association for Computing Machinery, New York, NY, USA, 166--168. https://doi.org/10.1145/3341163.3347716

Digital Library

[30]

Prashanth Mohan, Venkata N Padmanabhan, and Ramachandran Ramjee. 2008. Nericell: rich monitoring of road and traffic conditions using mobile smartphones. In Proceedings of the 6th ACM conference on Embedded network sensor systems. ACM, 323--336.

Digital Library

[31]

Le T. Nguyen, Ming Zeng, Patrick Tague, and Joy Zhang. 2015. I Did Not Smoke 100 Cigarettes Today!: Avoiding False Positives in Real-world Activity Recognition. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Osaka, Japan) (UbiComp '15). ACM, New York, NY, USA, 1053--1063. https://doi.org/10.1145/2750858.2804256

Digital Library

[32]

Nhan Duc Nguyen, Duong Trong Bui, Phuc Huu Truong, and Gu-Min Jeong. 2018. Position-Based Feature Selection for Body Sensors regarding Daily Living Activity Recognition. J. Sensors 2018 (2018), 9762098:1--9762098:13.

[33]

Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. 2018. Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. In Advances in Neural Information Processing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.). Curran Associates, Inc., 3235--3246. http://papers.nips.cc/paper/7585-realistic-evaluation-of-deep-semi-supervised-learning-algorithms.pdf

[34]

Vern I Paulsen and Mrinal Raghupathi. 2016. An introduction to the theory of reproducing kernel Hilbert spaces. Vol. 152. Cambridge University Press.

[35]

Liangying Peng, Ling Chen, Zhenan Ye, and Yi Zhang. 2018. AROMA: A Deep Multi-Task Learning Based Simple and Complex Human Activity Recognition Method Using Wearable Sensors. IMWUT 2 (2018), 74:1--74:16.

[36]

Attila Reiss and Didier Stricker. 2012. Introducing a New Benchmarked Dataset for Activity Monitoring. In Proceedings of the 2012 16th Annual International Symposium on Wearable Computers (ISWC) (ISWC '12). IEEE Computer Society, Washington, DC, USA, 108--109. https://doi.org/10.1109/ISWC.2012.13

Digital Library

[37]

D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. FÃűrster, G. TrÃűster, P. Lukowicz, D. Bannach, G. Pirkl, A. Ferscha, J. Doppler, C. Holzmann, M. Kurz, G. Holl, R. Chavarriaga, H. Sagha, H. Bayati, M. Creatura, and J. d. R. MillÃăn. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh International Conference on Networked Sensing Systems (INSS). 233--240. https://doi.org/10.1109/INSS.2010.5573462

[38]

P. Siirtola and J. RÃűning. 2013. Ready-to-use activity recognition for smartphones. In 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM). 59--64. https://doi.org/10.1109/CIDM.2013.6597218

[39]

Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. 2014. Striving for Simplicity: The All Convolutional Net. arXiv:cs.LG/1412.6806

[40]

Timo Sztyler and Heiner Stuckenschmidt. 2016. On-body Localization of Wearable Devices: An Investigation of Position-Aware Activity Recognition. In 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE Computer Society, 1--9. https://doi.org/10.1109/PERCOM.2016.7456521 http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7456521.

[41]

Timo Sztyler, Heiner Stuckenschmidt, and Wolfgang Petrich. 2017. Position-aware activity recognition with wearable devices. Pervasive and mobile computing 38 (2017), 281--295.

[42]

Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).

[43]

Dmitry Ulyanov, Andrea Vedaldi, and Victor S Lempitsky. 2017. Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis. In CVPR, Vol. 1. 3.

[44]

Terry T. Um, Franz M. J. Pfister, Daniel Pichler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fietzek, and Dana Kulić. 2017. Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring Using Convolutional Neural Networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (Glasgow, UK) (ICMI 2017). ACM, New York, NY, USA, 216--220. https://doi.org/10.1145/3136755.3136817

Digital Library

[45]

Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John Duchi, Vittorio Murino, and Silvio Savarese. 2018. Generalizing to Unseen Domains via Adversarial Data Augmentation. In Proceedings of the 32Nd International Conference on Neural Information Processing Systems (Montréal, Canada) (NIPS'18). Curran Associates Inc., USA, 5339--5349. http://dl.acm.org/citation.cfm?id=3327345.3327439

Digital Library

[46]

Dian xi Shi, Ran Wang, Yuan Wu, Xiaoyun Mo, and Jing Wei. 2017. A novel orientation- and location-independent activity recognition method. Personal and Ubiquitous Computing 21 (2017), 427--441.

Digital Library

[47]

Chao Xu, Parth H. Pathak, and Prasant Mohapatra. 2015. Finger-writing with Smartwatch: A Case for Finger and Hand Gesture Recognition Using Smartwatch. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications (Santa Fe, New Mexico, USA) (HotMobile '15). ACM, New York, NY, USA, 9--14. https://doi.org/10.1145/2699343.2699350

Digital Library

Cited By

Xue MZhu YXie WWang ZChen YJiang KZhang Q(2025)MobHAR: Source-free Knowledge Transfer for Human Activity Recognition on Mobile DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37126209:1(1-24)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712620
Busso MBontempelli AMalcotti LMeegahapola LKun PDiwakar SNutakki CBritez MXu HSong DCorrea SMendoza-Lara AGaskell GStares SBidoglia MGanbold AChagnaa ACernuzzi LHume AChenu-Abente RAsiku RKayongo IGatica-Perez Dde Götzen ABison IGiunchiglia F(2025)DiversityOne: A Multi-Country Smartphone Sensor Dataset for Everyday Life Behavior ModelingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122899:1(1-49)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712289
Thukral MDhekane SHiremath SHaresamudram HPloetz T(2025)Layout-Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST)Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122789:1(1-38)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712278
Show More Cited By

Index Terms

A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
2. Human-centered computing
  1. Ubiquitous and mobile computing

Recommendations

Unsupervised Domain Adaptation for Human Activity Recognition
Intelligent Data Engineering and Automated Learning – IDEAL 2018
Abstract
Human Activity Recognition has been primarily investigated as a machine learning classification task forcing it to handle with two main limitations. First, it must assume that the testing data has an equal distribution with the training sample. ...
A survey on unsupervised learning for wearable sensor-based activity recognition
Abstract
Human Activity Recognition (HAR) is an essential task in various applications such as pervasive healthcare, smart environment, and security and surveillance. The need to develop accurate HAR systems has motivated researchers to propose various ...
Highlights
- Adoption of unsupervised learning in wearable sensor-datasets.
- Survey to investigate the evolution of human activity recognition.
- Existing wearable sensor-based datasets and state-of-the-art clustering approach.
- Existing data ...
Multiple-instance domain adaptation for cost-effective sensor-based human activity recognition
Abstract
Machine learning-based human activity recognition (HAR) is important as the means of human–computer interaction to empower the existing systems in many areas, such as healthcare, entertainment, logistics, and manufacturing. To build ...
Highlights
- Human activity recognition (HAR) and its application are beneficial in real-life.

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 4, Issue 1

March 2020

1006 pages

EISSN:2474-9567

DOI:10.1145/3388993

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 March 2020

Published in IMWUT Volume 4, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

104
Total Citations
View Citations
2,269
Total Downloads

Downloads (Last 12 months)267
Downloads (Last 6 weeks)37

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xue MZhu YXie WWang ZChen YJiang KZhang Q(2025)MobHAR: Source-free Knowledge Transfer for Human Activity Recognition on Mobile DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37126209:1(1-24)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712620
Busso MBontempelli AMalcotti LMeegahapola LKun PDiwakar SNutakki CBritez MXu HSong DCorrea SMendoza-Lara AGaskell GStares SBidoglia MGanbold AChagnaa ACernuzzi LHume AChenu-Abente RAsiku RKayongo IGatica-Perez Dde Götzen ABison IGiunchiglia F(2025)DiversityOne: A Multi-Country Smartphone Sensor Dataset for Everyday Life Behavior ModelingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122899:1(1-49)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712289
Thukral MDhekane SHiremath SHaresamudram HPloetz T(2025)Layout-Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST)Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122789:1(1-38)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712278
Thukral MHaresamudram HPlötz T(2025)Cross-Domain HAR: Few-Shot Transfer Learning for Human Activity RecognitionACM Transactions on Intelligent Systems and Technology10.1145/370492116:1(1-35)Online publication date: 20-Jan-2025
https://dl.acm.org/doi/10.1145/3704921
Li JKang PBradley Shull P(2025)Multitarget Unsupervised Transfer Learning Improves Wearable-Sensor Cross-Body-Position Activity Recognition via Inter-Target Shared RepresentationIEEE Sensors Journal10.1109/JSEN.2025.352844125:5(9101-9112)Online publication date: 1-Mar-2025
https://doi.org/10.1109/JSEN.2025.3528441
Mondal SZingre SChakraborty S(2025)SlidEar: Exploring eSense in-Ear Wearable for Voice-Assisted Smart Slide Supervision2025 17th International Conference on COMmunication Systems and NETworks (COMSNETS)10.1109/COMSNETS63942.2025.10885592(811-813)Online publication date: 6-Jan-2025
https://doi.org/10.1109/COMSNETS63942.2025.10885592
Jeong YKim S(2024)COLDOG: Human Activity Recognition through Collaborative Learning for Domain GeneralizationJournal of the Korean Institute of Industrial Engineers10.7232/JKIIE.2024.50.2.07550:2(75-82)Online publication date: 15-Apr-2024
https://doi.org/10.7232/JKIIE.2024.50.2.075
Ye XSakurai KNair NWang K(2024)Machine Learning Techniques for Sensor-Based Human Activity Recognition with Data Heterogeneity—A ReviewSensors10.3390/s2424797524:24(7975)Online publication date: 13-Dec-2024
https://doi.org/10.3390/s24247975
Wang ZLiu HZhao BShen QLi MQue NYan MXin JLarson K(2024)Multi-scale context-aware networks based on fragment association for human activity recognitionProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/351(3169-3177)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/351
Chen WCheng JWang LZhao WMatusik W(2024)Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable SensorsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997478:4(1-26)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699747
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents