research-article

A Comparison of Machine Learning Models with Data Augmentation Techniques for Skeleton-based Human Action Recognition

Authors:

Kyoung Shin ParkAuthors Info & Claims

BCB '23: Proceedings of the 14th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics

Article No.: 19, Pages 1 - 6

https://doi.org/10.1145/3584371.3612999

Published: 04 October 2023 Publication History

Abstract

3D skeleton motion recognition plays a crucial role in the field of human action recognition (HAR) due to its efficiency and reliability. This paper introduces data augmentation techniques applied to skeleton data with the aim of improving the accuracy of machine learning models and examining the effectiveness of different augmentation methods for different activities. Spatial transformations are applied to generate augmented samples from the original 3D skeleton sequences, while temporal augmentation techniques are employed to capture the temporal differences in motion. We evaluated the effects of spatial and temporal data augmentation at different levels on the MSR Daily Activity 3D dataset using SVM, CNN, LSTM, and CNNLSTM models. The results show that temporal augmentation significantly improves model performance, while spatial augmentation has only a limited effect on model performance. Simultaneously applying both spatial and temporal augmentation further improved model performance, highlighting the importance of temporal information in action sequence data.

References

[1]

Hong-Bo Zhang, Yi-Xiang Zhang, Bineng Zhong, Qing Lei, Lijie Yang, Ji-Xiang Du and Duan-Sheng Chen. 2019. A Comprehensive Survey of Vision-Based Human Action Recognition Methods. Sensors. 19, 5

[2]

Zehua Sun, Qiuhong Ke, Hossein Rahmani, Mohammed Bennamoun, Gang Wang, and Jun Liu. 2023. Human Action Recognition From Various Data Modalities: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 3, 3200--3225

[3]

Nils Y. Hammerla, Shane Halloran and Thomas Plötz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. July 2016. 1533--1540.

Digital Library

[4]

Federico Cruciani, Anastasios Vafeiadis, Chris Nugent, Ian Cleland, Paul McCullagh, Konstantinos Votis, Dimitrios Giakoumis, Dimitrios Tzovaras, Liming Chen and Raouf Hamzaoui. 2020. Feature learning for Human Activity Recognition using Convolutional Neural Networks. CCF Trans. Pervasive Comp. Interact. 2, 18--32 (2020).

[5]

Yeon-Wook Kim, Woo-Hyeong Cho, Kyu-Sung Kim and Sangmin Lee. 2022. Inertial-Measurement-Unit-Based Novel Human Activity Recognition Algorithm Using Conformer. Sensors. 22, 10

[6]

Pichao Wang, Wanqing Li, Zhimin Gao, Jing Zhang, Chang Tang and Philip O. Ogunbona. 2016. Action Recognition From Depth Maps Using Deep Convolutional Neural Networks. IEEE Trans. Human-Machine Systems. 46, 4

[7]

Xuesong Gao, Keqiu Li, Yu Zhang, Qiguang Miao, Lijie Sheng, Jun Xie, Jinfu Xu. 2019. 3D Skeleton-Based Video Action Recognition by Graph Convolution Network. 2019 IEEE International Conference on Smart Internet of Things (SmartIoT), Tianjin, China, 2019, pp. 500--501

[8]

Jiahui An, Xinrong Cheng, Qing Wang, Hong Chen, Jiayue Li and Shiji Li. 2020. Human action recognition based on Kinect. J. Phys.: Conf. Ser. 1693 012190.

[9]

Santosh Kumar Yadav, Kamlesh Tiwari, Hari Mohan Pandey and Shaik Ali Akbar. Skeleton-based human activity recognition using ConvLSTM and guided feature learning. Soft Comput 26, 877--890.

Digital Library

[10]

Zhengrui Shi, Lujie Cao, Yu Han, Haixing Liu, Fengshou Jiang and Yu Ren. 2020. Research on Recognition of Motion Behaviors of Copepods. IEEE Access, 8, 141224-141233.

[11]

Juanhui Tu, Hong Liu, Fanyang Meng, Mengyuan Liu and Runwei Ding. 2018. Spatial-temporal data augmentation based on LSTM autoencoder network for skeleton-based human action recognition. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece. September 2018. 3478--3482.

[12]

Fanyang Meng, Hong Liu, Member, IEEE, Yongsheng Liang, Juanhui Tu, and Mengyuan Liu. 2019. Sample fusion network: An end-to-end data augmentation network for skeleton-based human action recognition. IEEE Transactions on Image Processing, 28, 11, 5281--5295

Digital Library

[13]

Thien Huynh-The, Cam-Hao Hua and Dong-Seong Kim. 2020. Encoding Pose Features to Images With Data Augmentation for 3-D Action Recognition. IEEE Transactions on Industrial Informatics, 16, 5, 3100--3111

[14]

Junxiao Shen, John Dudley and Per Ola Kristensson. 2021. The Imaginative Generative Adversarial Network: Automatic Data Augmentation for Dynamic Skeleton-Based Hand Gesture and Human Action Recognition. In Proceedings of the 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), Jodhpur, India, 2021, 1--8

Digital Library

[15]

Junjie Chen, Wei Yang, Chenqi Liu and Leiyue Yao. 2021. A Data Augmentation Method for Skeleton-Based Action Recognition with Relative Features. Appl. Sci. 11, 23, 11481

[16]

Parsa Riazi Bakhshayesh, Mehdi Ejtehadi, Alireza Taheri, and Saeed Behzadipour. 2022. The Effects of Data Augmentation Methods on the Performance of Human Activity Recognition. In Proceedings of 8th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS). Behshahr, Iran, Islamic Republic of, 1--6

[17]

Terry T. Um, Franz M. J. Pfister, Daniel Pichler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fietzek, and Dana Kulić. 2017. Data augmentation of wearable sensor data for parkinson's disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, November 2017, 216--220, Brian Kenji Iwana and Seiichi Uchida. 2021. An empirical survey of data augmentation for time series classification with neural networks. PLoS ONE 16, 7, e0254841. https://doi.org/10.1371/journal.pone.0254841

Digital Library

[18]

Brian Kenji Iwana and Seiichi Uchida. 2021. An empirical survey of data augmentation for time series classification with neural networks. PLoS ONE 16, 7, e0254841.

[19]

Connor Shorten and Taghi M. Khoshgoftaar. 2019. A Survey on Image Data Augmentation for Deep Learning. J Big Data. 6, 60

[20]

Jiang Wang, Zicheng Liu, Ying Wu and Junsong Yuan. 2012. Mining Actionlet Ensemble for Action Recognition with Depth Cameras. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 1290--1297

[21]

Sujie Xu, Shang Gao, Mengkun Zhang and Lejun Zhu. 2022. Research on Class Individual Action Recognition Based on 3D Skeleton Data. In Proceedings of Computer & Engineering, VoL. 50 No. 8, http://journal16.magtechjournal.com/Jweb_jsjyszgc/CN/abstract/abstract3239.shtml

[22]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation. 9, 8, 1735--1780.

Digital Library

[23]

Daeun Kim, Woohyun Kim and Kyoung Shin Park. 2022. Effects of Exercise Type and Gameplay Mode on Physical Activity in Exergame. Electronics. 11, 9, 3086

Cited By

Xin CKim SCho YPark K(2024)Enhancing Human Action Recognition with 3D Skeleton Data: A Comprehensive Study of Deep Learning and Data AugmentationElectronics10.3390/electronics1304074713:4(747)Online publication date: 13-Feb-2024
https://doi.org/10.3390/electronics13040747
Daud NKurnianingtyas D(2024)Food Image Classification for Maternal Nutritional Fulfillment Using MobileNet2024 IEEE International Conference on Communication, Networks and Satellite (COMNETSAT)10.1109/COMNETSAT63286.2024.10862862(529-535)Online publication date: 28-Nov-2024
https://doi.org/10.1109/COMNETSAT63286.2024.10862862
Zakka VDai ZManso L(2024)Action Recognition for Privacy-Preserving Ambient Assisted LivingArtificial Intelligence in Healthcare10.1007/978-3-031-67285-9_15(203-217)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-67285-9_15

Index Terms

A Comparison of Machine Learning Models with Data Augmentation Techniques for Skeleton-based Human Action Recognition
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Comparison of Affine and DCGAN-based Data Augmentation Techniques for Chest X-Ray Classification
Abstract
Data augmentation, also called implicit regularization, is one of the popular strategies to improve the generalization capability of deep neural networks. It is crucial in situations where there is a scarcity of high-quality ground-truth data. ...
Extending Temporal Data Augmentation for Video Action Recognition
Image and Vision Computing
Abstract
Pixel space augmentation has grown in popularity in many Deep Learning areas, due to its effectiveness, simplicity, and low computational cost. Data augmentation for videos, however, still remains an under-explored research topic, as most works ...
Data Augmentation of Minority Class with Transfer Learning for Classification of Imbalanced Breast Cancer Dataset Using Inception-V3
Pattern Recognition and Image Analysis
Abstract
In this paper, deep learning based experiments are conducted to investigate the effect of data augmentation on the minority class for the imbalanced breast cancer histopathology dataset (BREAKHIS). Two different pre-trained networks are fine-tuned ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

BCB '23: Proceedings of the 14th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics

September 2023

626 pages

ISBN:9798400701269

DOI:10.1145/3584371

General Chairs:
May D. Wang
SGeorgia Institute of Technology
,
Byung-Jun Yoon

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGBio: ACM Special Interest Group on Bioinformatics

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

BCB '23

Sponsor:

SIGBio

BCB '23: 14th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics

September 3 - 6, 2023

TX, Houston, USA

Acceptance Rates

Overall Acceptance Rate 254 of 885 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
119
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)6

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xin CKim SCho YPark K(2024)Enhancing Human Action Recognition with 3D Skeleton Data: A Comprehensive Study of Deep Learning and Data AugmentationElectronics10.3390/electronics1304074713:4(747)Online publication date: 13-Feb-2024
https://doi.org/10.3390/electronics13040747
Daud NKurnianingtyas D(2024)Food Image Classification for Maternal Nutritional Fulfillment Using MobileNet2024 IEEE International Conference on Communication, Networks and Satellite (COMNETSAT)10.1109/COMNETSAT63286.2024.10862862(529-535)Online publication date: 28-Nov-2024
https://doi.org/10.1109/COMNETSAT63286.2024.10862862
Zakka VDai ZManso L(2024)Action Recognition for Privacy-Preserving Ambient Assisted LivingArtificial Intelligence in Healthcare10.1007/978-3-031-67285-9_15(203-217)Online publication date: 4-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-67285-9_15

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten