Abstract
In video sequences, human action recognition is a challenging problem due to motion variation, in frame person difference, and setting of video recording in the field of computer vision. Since last few years, applications of human activity recognition have increased significantly. In the literature, many techniques are implemented for human action recognition, but still they face problem in contrast of foreground region, segmentation, feature extraction, and feature selection. This article contributes a novel human action recognition method by embedding the proposed frames fusion working on the principle of pixels similarity. An improved hybrid feature extraction increases the recognition rate and allows efficient classification in the complex environment. The design consists of four phases, (a) enhancement of video frames (b) threshold-based background subtraction and construction of saliency map (c) feature extraction and selection (d) neural network (NN) for human action classification. Results have been tested using five benchmark datasets including Weizmann, KTH, UIUC, Muhavi, and WVU and obtaining recognition rate 97.2, 99.8, 99.4, 99.9, and 99.9%, respectively. Contingency table and graphical curves support our claims. Comparison with existent techniques identifies the recognition rate and trueness of our proposed method.











Similar content being viewed by others
References
Ye Q, Han Z, Jiao J, Liu J (2013) Human detection in images via piecewise linear support vector machines. IEEE Trans Image Process 22(2):778–789
Liang C-W, Juang C-F (2015) Moving object classification using local shape and HOG features in wavelet-transformed space with hierarchical SVM classifiers. Appl Soft Comput 28:483–497
Xu Y, Dong X, Lin S, Han TX, Cao X, Li X (2012) Detection of sudden pedestrian crossings for driving assistance systems. IEEE Trans Syst Man Cybern B Cybern 42(3):729–739
Juang C-F, Chang C-M (2007) Human body posture classification by a neural fuzzy network and home care system application. IEEE Trans Syst Man Cybern Part A Syst Hum 37(6):984–994
Ye Q, Liang J, Jiao J (2012) Pedestrian detection in video images via error correcting output code classification of manifold subclasses. IEEE Trans Intell Transp Syst 13(1):193–202
Yeguas-Bolivar E, Muoz-Salinas R, Medina-Carnicer R, Carmona-Poyato A (2014) Comparing evolutionary algorithms and particle filters for Markerless Human Motion Capture. Appl Soft Comput 17:153–166
Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv (CSUR) 43(3):16
Chathuramali KGM, Rodrigo R (2013) Faster human activity recognition with SVM. In: IEEE international conference on advances in ICT for emerging regions, pp 197–203
Ali S, Shah M (2010) Human action recognition in videos using kinematic features and multiple instance learning. IEEE Trans Pattern Anal Mach Intell 32(2):288–303
Dayananda Kumar NC, Suresh KV (2014) HOG-PCA descriptor with optical flow based human detection and tracking. In: International conference on communications and signal processing (ICCSP). IEEE, pp 900–904
Porikli F, Tuzel O, Meer P (2016) Designing a boosted classifier on riemannian manifolds. In: Riemannian computing in computer vision. Springer, pp 281–301
Mu Y, Yan S, Liu Y, Huang T, Zhou B (2008) Discriminative local binary patterns for human detection in personal album. In: Proceedings of IEEE international conference computer vision pattern recognition, pp 1–8
Wang X, Han TX, Yan S (2009) An HOG-LBP human detector with partial occlusion handling. In: Proceedings of IEEE international conference on computer vision, pp 32–39
Nguyen DT, Ogunbona PO, Li W (2013) A novel shape-based non-redundant local binary pattern descriptor for object detection. Pattern Recognit 46(5):1485–1500
Xia L, Chen C-C, Aggarwal JK (2011) Human detection using depth information by kinect. In: 2011 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 15–22
Kramer O (2013) K-nearest neighbors. In: Dimensionality reduction with unsupervised nearest neighbors. Springer, Berlin, pp 13–23
Shen H-B, Chou K-C (2006) Ensemble classifier for protein fold pattern recognition. Bioinformatics 22(14):1717–1722
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of IEEE international conference on computer vision pattern recognition, pp 886–893
Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision, Springer
Tran D, Sorokin A, Forsyth D (2008) Human activity recognition with metric learning, Computer Vision – ECCV 2008, pp 548-561
Gilbert A, Illingworth J, Bowden R (2009) Fast realistic multi-action recognition using mined dense spatio-temporal features. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 925–931
Bregonzio M, Gong S, Xiang T (2009) Recognising action as clouds of space-time interest points. In: IEEE conference on computer vision and pattern recognition, 2009, CVPR. IEEE, pp 1948–1955
Yang M, Lv F, Xu W, Yu K, Gong Y (2009) Human action detection by boosting efficient motion features. In: 2009 IEEE 12th international conference on computer vision workshops (ICCV Workshops). IEEE, pp 522–529
Liu L, Shao L, Rockett P (2013) Human action recognition based on boosted feature selection and naive Bayes nearest-neighbor classification. Signal Process 93(6):1521–1530
Qian H, Mao Y, Xiang W, Wang Z (2010) Recognition of human activities using SVM multi-class classifier. Pattern Recognit Lett 31(2):100–111
Mahbub U, Imtiaz H, Ahad MAR (2012) Motion clustering-based action recognition technique using optical flow. In: 2012 International conference on informatics, electronics & vision (ICIEV). IEEE, pp 919–924
Varol G, Laptev I, Schmid C (2017) Long-term temporal convolutions for action recognition. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2017.2712608
Zhang H, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520–531
Li C, Zhong Q, Xie D, Pu S (2017) Skeleton-based action recognition with convolutional neural networks. arXiv preprint arXiv:1704.07595
Kushwaha AKS, Singh S, Srivastava R (2017) Multi-view human activity recognition based on silhouette and uniform rotation invariant local binary patterns. Multimed Syst 23(4):451–467
Gonzalez RC, Woods RE (2002) Digital image processing. Prentice hall, New Jersey
Hunter RS (1948) Photoelectric color-difference meter. JOSA 38(7):661 (Proceedings of the Winter Meeting of the Optical Society of America)
Barron JL, Fleet DJ, Beauchemin SS (1994) Performance of optical flow techniques. Int J Comput Vision 12(1):43–77
Kim J, Han D, Tai Y-W, Kim J (2014) Salient region detection via high-dimensional color transform. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 883–890
Kempe J, Regev O, Toner B (2010) Unique games with entangled provers are easy. SIAM J Comput 39(7):3207–3229
Ito T, Kobayashi H, Matsumoto K (2009) Oracularization and two-prover one-round interactive proofs against nonlocal strategies. In: 24th annual IEEE conference on computational complexity, 2009. CCC’09. IEEE, pp 217–228
Duan Q, Akram T, Duan P, Wang X (2016) Visual saliency detection using information contents weighting. Optik-Int J Light Electron Opt 127(19):7418–7430
Costa AF, Humpire-Mamani G, Traina AJM (2012) An efficient algorithm for fractal analysis of textures. In: 2012 25th SIBGRAPI conference on graphics, patterns and images (SIBGRAPI). IEEE, pp 39–46
Xiong W, Zhang L, Bo D, Tao D (2017) Combining local and global: rich and robust feature pooling for visual recognition. Pattern Recognit 62:225–235
Zhang L, Zhang Q, Zhang L, Tao D, Huang X, Bo D (2015) Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding. Pattern Recognit 48(10):3102–3112
Jolliffe I (2002) Principal component analysis. Wiley, Hoboken
Kamran M, Haider SA, Akram T, Naqvi SR, He SK (2016) Prediction of IV curves for a superconducting thin film using artificial neural networks. Superlattices Microstruct 95:88–94
Baumann F, Ehlers A, Rosenhahn B, Liao J (2016) Recognizing human actions using novel space-time volume binary patterns. Neurocomputing 173:54–63
Wang H, Kläser A, Schmid C, Liu C-L (2013) Dense trajectories and motion boundary descriptors for action recognition. Int J Comput Vision 103(1):60–79
Singh S, Velastin SA, Ragheb H (2010) Muhavi: a multicamera human action video dataset for the evaluation of action recognition methods. In: 2010 Seventh IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 48–55
Ramagiri S, Kavi R, Kulathumani V (2011) Real-time multi-view human action recognition using a wireless camera network. In: 2011 Fifth ACM/IEEE international conference on distributed smart cameras (ICDSC). IEEE, pp 1–6
Maity S, Bhattacharjee D, Chakrabarti A (2017) A novel approach for human action recognition from silhouette images. IETE J Res 63(2):160–171
Xiao Q, Song R (2017) Action recognition based on hierarchical dynamic Bayesian network. Multimedia Tools and Applications. NY, USA, New York, pp 1–14
Wang H, Yuan C, Weiming H, Sun C (2012) Supervised class-specific dictionary learning for sparse modeling in action recognition. Pattern Recognit 45(11):3902–3911
Chaaraoui AA, Climent-Prez P, Flrez-Revuelta F (2013) Silhouette-based human action recognition using sequences of key poses. Pattern Recognit Lett 34(15):1799–1807
Nasiri JA, Charkari NM, Mozafari K (2014) Energy-based model of least squares twin Support Vector Machines for human action recognition. Signal Process 104:248–257
Ahmad T, Rafique J, Muazzam H, Rizvi T (2015) Using discrete cosine transform based features for human action recognition. J Image Gr 3(2)
Kumar SS, John M (2016) Human activity recognition using optical flow based feature set. In: 2016 IEEE international carnahan conference on security technology (ICCST). IEEE, pp 1–5
Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–8
Gilbert A, Illingworth J, Bowden R (2011) Action recognition using mined hierarchical compound features. IEEE Trans Pattern Anal Mach Intell 33(5):883–897
Abdul-Azim HA, Hemayed EE (2015) Human action recognition using trajectory-based representation. Egypt Inform J 16(2):187–198
Shao L, Liu L, Mengyang Y (2016) Kernelized multiview projection for robust action recognition. Int J Comput Vision 118(2):115–129
Tong M, Wang H, Tian W, Yang S (2017) Action recognition new framework with robust 3D-TCCHOGAC and 3D-HOOFGAC. Multimed Tools Appl 76(2):3011–3030
Liu J, Kuipers B, Savarese S (2011) Recognizing human actions by attributes. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3337–3344
Parikh D, Grauman K (2011) Relative attributes. In: 2011 International conference on computer vision. IEEE, pp 503–510
Zhang Z, Wang C, Xiao B, Zhou W, Liu S (2015) Robust relative attributes for human action recognition. Pattern Anal Appl 18(1):157–171
Wu X, Jia Y (2012) View-invariant action recognition using latent kernelized structural SVM. In: European conference on computer vision. Springer, Berlin, pp 411–424
Cheema S, Eweiwi A, Thurau C, Bauckhage C (2011) Action recognition by learning discriminative key poses. In: 2011 IEEE international conference on computer vision workshops (ICCV Workshops). IEEE, pp 1302–1309
Zheng J, Jiang Z, Chellappa R (2016) Cross-view action recognition via transferable dictionary learning. IEEE Trans Image Process 25(6):2542–2556
Murtaza F, Yousaf MH, Velastin SA (2016) Multi-view human action recognition using 2D motion templates based on MHIs and their HOG description. IET Comput Vision 10(7):758–767
Haq A, Gondal I, Murshed M (2013) On temporal order invariance for view-invariant action recognition. IEEE Trans Circuits Syst Video Technol 23(2):203–211
Kavi R, Kulathumani V, Rohit F, Kecojevic V (2016) Multiview fusion for activity recognition using deep neural networks. J Electron Imaging 25(4):043010–043010
Iosifidis A, Tefas A, Pitas I (2013) Multi-view action recognition based on action volumes, fuzzy distances and cluster discriminant analysis. Signal Process 93(6):1445–1457
Hashemi SM, Rahmati M (2016) View-independent action recognition: a hybrid approach. Multimed Tools Appl 75(12):6755–6775
Liu R, Gillies DF (2016) Overfitting in linear feature extraction for classification of high-dimensional image data. Pattern Recognit 53:73–86
Wu D, Sharma N, Blumenstein M (2017) Recent advances in video-based human action recognition using deep learning: a review. In: 2017 International joint conference on neural networks (IJCNN). IEEE, pp 2865–2872
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Khan, M.A., Akram, T., Sharif, M. et al. An implementation of optimized framework for action classification using multilayers neural network on selected fused features. Pattern Anal Applic 22, 1377–1397 (2019). https://doi.org/10.1007/s10044-018-0688-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-018-0688-1