skip to main content
10.1145/3412382.3458278acmconferencesArticle/Chapter ViewAbstractPublication PagescpsweekConference Proceedingsconference-collections
research-article

Optimal sensor channel selection for resource-efficient deep activity recognition

Published: 18 May 2021 Publication History

Abstract

Deep learning has permitted unprecedented performance in sensor-based human activity recognition (HAR). However, deep learning models often present high computational overheads, which poses challenges to their implementation on resource-constraint devices such as microcontrollers. Usually, the computational overhead increases with the input size. One way to reduce the input size is by constraining the number of sensor channels. We refer to sensor channel as a specific data modality (e.g. accelerometer) placed on a specific body location (e.g. chest). Identifying and removing irrelevant and redundant sensor channels is feasible via exhaustive search only in cases where few candidates exist. In this paper, we propose a smarter and more efficient way to optimize the sensor channel selection during the training of deep neural networks for HAR. Firstly, we propose a light-weight deep neural network architecture that learns to minimize the use of redundant and irrelevant information in the classification task, while achieving high performance. Secondly, we propose a sensor channel selection algorithm that utilizes the knowledge learned by the neural network to rank the sensor channels by their contribution to the classification task. The neural network is then trimmed by removing the sensor channels with the least contribution from the input and pruning the corresponding weights involved in processing them. The pipeline that consists of the above two steps iterates until the optimal set of sensor channels has been found to balance the trade-off between resource consumption and classification performance. Compared with other selection methods in the literature, experiments on 5 public datasets showed that our proposal achieved significantly higher F1-scores at the same time as utilizing from 76% to 93% less memory, with up to 75% faster inference time and as far as 76% lower energy consumption.

References

[1]
O. Aziz, S. N. Robinovitch, and E. J. Park. 2016. Identifying the number and location of body worn sensors to accurately classify walking, transferring and sedentary activities. In 2016 EMBC. 5003--5006.
[2]
M. Bachlin, M. Plotnik, D. Roggen, I. Maidan, J. M. Hausdorff, N. Giladi, and G. Troster. 2010. Wearable Assistant for Parkinson's Disease Patients With the Freezing of Gait Symptom. IEEE Transactions on Information Technology in Biomedicine 14, 2 (March 2010), 436--446.
[3]
O. Banos, J. A. Moral-Munoz, I. Diaz-Reyes, M. Arroyo-Morales, M. Damas, E. Herrera-Viedma, C. S. Hong, S. Lee, H. Pomares, I. Rojas, and C. Villalonga. 2015. mDurance: A Novel Mobile Health System to Support Trunk Endurance Assessment. Sensors 15, 6 (2015), 13159--13183.
[4]
K. A. Bhaskaran, A. G. Nair, K. D. Ram, K. Ananthanarayanan, and H. R. Nandi Vardhan. 2016. Smart gloves for hand gesture recognition: Sign language to speech conversion system. In 2016 RAHA. 1--6.
[5]
J. Cao, W. Li, C. Ma, and Z. Tao. 2018. Optimizing Multi-Sensor Deployment via Ensemble Pruning for Wearable Activity Recognition. Inf. Fusion 41, C (May 2018), 68--79.
[6]
R. Chavarriaga, H. Sagha, A. Calatroni, S. T. Digumarti, G. Tröster, J. del R. Millán, and D. Roggen. 2013. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 15 (2013), 2033 - 2042.
[7]
Y. Du, Y. Lim, and Y. Tan. 2019. A Novel Human Activity Recognition and Prediction in Smart Home Based on Interaction. Sensors 19, 20 (2019), 1--16.
[8]
O. Ertugrul and Y. Kaya. 2016. Determining the optimal number of body-worn sensors for human activity recognition. Soft Computing 21 (02 2016), 5053--5060.
[9]
M. Espinilla, J. Medina, A. Calzada, J. Liu, L. Martínez, and C. Nugent. 2017. Optimizing the configuration of an heterogeneous architecture of sensors for activity recognition, using the extended belief rule-based inference methodology. Microprocessors and Microsystems 52 (2017), 381--390.
[10]
Y. Guan and T. Plötz. 2017. Ensembles of Deep LSTM Learners for Activity Recognition Using Wearables. IMWUT 1, 2, Article 11 (June 2017), 28 pages.
[11]
N. Y. Hammerla, S. Halloran, and T. Plötz. 2016. Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables. In IJCAI'16. AAAI Press, 1533--1540.
[12]
G. Hinton, O. Vinyals, and J. Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv e-prints arXiv:abs/1503.02531 (March 2015), 1--9.
[13]
S. Hochreiter and J. Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. 9, 8 (Nov. 1997), 1735--1780.
[14]
J. Hsiao, Y. Deng, T. Pao, H. Chou, and J. Chang. 2017. Design ofa Wireless 3D Hand Motion Tracking and Gesture Recognition Glove for Virtual Reality Applications, Vol. ASME 2017 Conference on Information Storage and Processing Systems. 1--3.
[15]
M. Keally, G. Zhou, G. Xing, J. Wu, and A. Pyles. 2011. PBN: Towards Practical Activity Recognition Using Smartphone-Based Body Sensor Networks. In Sensys '11 (Seattle, Washington). Association for Computing Machinery, New York, NY, USA, 246--259.
[16]
J. Lei Ba, J. R. Kiros, and G. E. Hinton. 2016. Layer Normalization. arXiv e-prints arXiv:abs/1607.06450 (July 2016), 1--14.
[17]
C. Min, A. Montanari, A. Mathur, and F. Kawsar. 2019. A Closer Look at Quality-Aware Runtime Assessment of Sensing Models in Multi-Device Environments. In Sensys '19 (New York, New York). Association for Computing Machinery, New York, NY, USA, 271--284.
[18]
E. B. Neves, J. Vilaça-Alves, N. Antunes, I. M. V. Felisberto, C. Rosa, and V. M. Reis. 2015. Different responses of the skin temperature to physical exercise: Systematic review. In 2015 EMBC. 1307--1310.
[19]
F. J. Ordóñez and D. Roggen. 2016. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors 16, 1 (2016), 1--25.
[20]
L. Peng, L. Chen, Z. Ye, and Y. Zhang. 2018. AROMA: A Deep Multi-Task Learning Based Simple and Complex Human Activity Recognition Method Using Wearable Sensors. ACM IMWUT 2, 2, Article 74 (July 2018), 16 pages.
[21]
M. Phuong and C. Lampert. 2019. Towards Understanding Knowledge Distillation. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), K. Chaudhuri and R. Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 5142--5151.
[22]
A. Reiss and D. Stricker. 2012. Introducing a New Benchmarked Dataset for Activity Monitoring. In 2012 16th International Symposium on Wearable Computers. 108--109.
[23]
L. Schrader, A. Vargas Toro, S. Konietzny, S. Rüping, B. Schäpers, M. Steinböck, C. Krewer, F. Müller, J. Güttler, and T. Bock. 2020. Advanced Sensing and Human Activity Recognition in Early Intervention and Rehabilitation of Elderly People. Journal of Population Ageing 13, 2 (01 Jun 2020), 139--165.
[24]
C. F. Souza Leite and Y. Xiao. 2020. Improving Cross-Subject Activity Recognition via Adversarial Learning. IEEE Access 8 (2020), 90542--90554.
[25]
C. F. Souza Leite and Y. Xiao. 2020. Improving Resource Efficiency of Deep Activity Recognition via Redundancy Reduction. In HotMobile '20. Association for Computing Machinery, 33--38.
[26]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15, 56 (2014), 1929--1958.
[27]
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. 2014. Going Deeper with Convolutions. CoRR arXiV:abs/1409.4842 (2014), 1--12.
[28]
C. Xu, D. Chai, J. He, X. Zhang, and S. Duan. 2019. InnoHAR: A Deep Neural Network for Complex Human Activity Recognition. IEEE Access 7 (2019), 9893--9902.
[29]
X. Yang, Yiqiang Chen, H. Yu, Y. Zhang, W. Lu, and R. Sun. [n.d.]. Instance-wise Dynamic Sensor Selection for Human Activity Recognition. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020. AAAI Press, 1104--1111.
[30]
P. Zappi, C. Lombriser, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Tröster. 2008. Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection. In Wireless Sensor Networks, Verdone R. (Ed.). Springer, Berlin, Heidelberg, 17--33.
[31]
P. Zappi, D. Roggen, E. Farella, G. Tröster, and L. Benini. 2012. Network-Level Power-Performance Trade-Off in Wearable Activity Recognition: A Dynamic Sensor Selection Approach. ACM Trans. Embed. Comput. Syst. 11, 3, Article 68 (Sept. 2012), 30 pages.
[32]
P. Zappi, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Troster. 2007. Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness. In 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information. 281--286.
[33]
M. Zeng, H. Gao, T. Yu, O. J. Mengshoel, H; Langseth, I. Lane, and X. Liu. 2018. Understanding and Improving Recurrent Networks for Human Activity Recognition by Continuous Attention. In ISWC '18 (Singapore, Singapore). ACM, 56--63.

Cited By

View all
  • (2024)AAUConvNeXt: Enhancing Crop Lodging Segmentation with Optimized Deep Learning ArchitecturesPlant Phenomics10.34133/plantphenomics.01826(0182)Online publication date: 2024
  • (2024)Artificial Intelligence of Things: A SurveyACM Transactions on Sensor Networks10.1145/369063921:1(1-75)Online publication date: 30-Aug-2024
  • (2024)Minimum-Cost Channel Selection in Wearables2024 IEEE 20th International Conference on Body Sensor Networks (BSN)10.1109/BSN63547.2024.10780744(1-4)Online publication date: 15-Oct-2024
  • Show More Cited By

Index Terms

  1. Optimal sensor channel selection for resource-efficient deep activity recognition

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      IPSN '21: Proceedings of the 20th International Conference on Information Processing in Sensor Networks (co-located with CPS-IoT Week 2021)
      May 2021
      423 pages
      ISBN:9781450380980
      DOI:10.1145/3412382
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 18 May 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Sensor channel selection
      2. deep learning
      3. human activity recognition

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      Conference

      IPSN '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 143 of 593 submissions, 24%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)32
      • Downloads (Last 6 weeks)5
      Reflects downloads up to 27 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)AAUConvNeXt: Enhancing Crop Lodging Segmentation with Optimized Deep Learning ArchitecturesPlant Phenomics10.34133/plantphenomics.01826(0182)Online publication date: 2024
      • (2024)Artificial Intelligence of Things: A SurveyACM Transactions on Sensor Networks10.1145/369063921:1(1-75)Online publication date: 30-Aug-2024
      • (2024)Minimum-Cost Channel Selection in Wearables2024 IEEE 20th International Conference on Body Sensor Networks (BSN)10.1109/BSN63547.2024.10780744(1-4)Online publication date: 15-Oct-2024
      • (2024)Simulation-driven design of smart gloves for gesture recognitionScientific Reports10.1038/s41598-024-65069-214:1Online publication date: 27-Jun-2024
      • (2022)Resource-Efficient Continual Learning for Sensor-Based Human Activity RecognitionACM Transactions on Embedded Computing Systems10.1145/353091021:6(1-25)Online publication date: 18-Oct-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media