Skip to main content
SpringerLink
Log in

Enhanced hand-gesture recognition by improved beetle swarm optimized probabilistic neural network for human–computer interaction

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The dynamic hand gesture recognition is a new research area due to the emerging nature of pervasive somatosensory devices, which has been gained more attention and been broadly utilized in human–computer interaction (HCI). This paper tactics to develop an intelligent hand gesture recognition model through considering the use of hand gestures for HCI. The user-friendly, less intrusive, intuitive, and more natural HCI for controlling an application by executing hand gestures are presented. The overall models cover diverse steps like data collection, pre-processing, segmentation, feature extraction, and recognition. The hand gestures videos from different benchmark sources are collected, and further, the pre-processing is performed by median filtering. Further, the goal of the segmentation is to evaluate the temporal hand trajectories from the detected hand poses, which is carried out through the adaptive region-based active contour (ARAC) method based on the meta-heuristic basis by the opposition strategic velocity updated beetle swarm optimization (OSV-BSO). Feature extraction is the next step, which is done by combining a set of features like oriented FAST and rotated BRIEF (ORB), histogram of oriented gradients (HOG), and regionprops, which are further given to the principal component analysis (PCA) for dimensionality reduction. With these relevant features, the recognition phase is accomplished by the optimized probabilistic neural network (PNN). To improvise the existing performance of PNN as a deep learning model, the weights in training are tuned based on the meta-heuristic basis by the OSV-BSO. The objective model of the optimized PNN is to minimize the error between the desired and actual outcomes. Finally, the designed “gesture recognition approach” makes the HCI process more user-specific and intuitive, which is proved by comparing it over the existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability statement

The data underlying this article are available in “https://www.kaggle.com/ayuraj/american-sign-language-dataset:Access Date: 2021-01-11”. “https://www.kaggle.com/muhammadkhalid/sign-language-for-alphabets: Access Date: 2021-01-11” and “https://www.kaggle.com/muhammadkhalid/sign-language-for-numbers/vaersion/1: Access Date: 2021-01-11”.

References

  • Aglave P, Kolkure VS (2015) Implementation of high-performance feature extraction method using oriented fast and rotated brief algorithm. IJRET Int J Res Eng Technol 04(02):394–397

    Google Scholar 

  • Ameur S, Khalifa AB, Bouhlel MS (2020) A novel hybrid bidirectional unidirectional LSTM network for dynamic hand gesture recognition with leap motion. Entertain Comput 35:100373

    Google Scholar 

  • Bao P, Maqueda AI, del-Blanco CR, García N (2017) Tiny hand gesture recognition without localization via a deep convolutional network. IEEE Trans Consum Electron 63(3):251–257

    Google Scholar 

  • Bonyadi MR, Michalewicz Z (2016) Analysis of stability, local convergence, and transformation sensitivity of a variant of the particle swarm optimization algorithm. IEEE Trans Evol Comput 20(3):370–385

    Google Scholar 

  • Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound rank-k projections for bilinear analysis. IEEE Trans Neural Netw Learn Syst 27(7):1502–1513

    MathSciNet  Google Scholar 

  • Chen T, Zhu Y, Teng J (2018) Beetle swarm optimisation for solving investment portfolio problems. J Eng 2018(16):1600–1605

    Google Scholar 

  • Chen F et al (2019) WristCam: a wearable sensor for hand trajectory gesture recognition and intelligent human–robot interaction. IEEE Sens J 19(19):8441–8451

    Google Scholar 

  • Deng M (2020) Robust human gesture recognition by leveraging multi-scale feature fusion. Signal Process Image Commun 83:115768

    Google Scholar 

  • Duan H, Huang M, Yang Y, Hao J, Chen L (2020) Ambient light-based hand gesture recognition enabled by recurrent neural network. IEEE Access 8:7303–7312

    Google Scholar 

  • Hu Y, Wong Y, Dai Q, Kankanhalli M, Geng W, Li X (2019) sEMG-based gesture recognition with embedded virtual hand poses and adversarial learning. IEEE Access 7:104108–104120

    Google Scholar 

  • Huan G, Li Y, Song Z (2016) A novel robust principal component analysis method for image and video processing. Appl Math 61:197–214

    MathSciNet  MATH  Google Scholar 

  • Kang X, Xiang X, Li S, Benediktsson JA (2017) PCA-based edge-preserving features for hyperspectral image classification. IEEE Trans Geosci Remote Sens 55(12):7140–7151

    Google Scholar 

  • Kılıboz NÇ, Güdükbay U (2015) A hand gesture recognition technique for human–computer interaction. J Vis Commun Image Represent 28:97–104

    Google Scholar 

  • Kobayashi T, Hidaka A, Kurita T (2008) Selection of histograms of oriented gradients features for pedestrian detection. In: International conference on neural information processing, p 498

  • Köpüklü O, Gunduz A, Kose N, Rigoll G (2020) Online dynamic hand gesture recognition including efficiency analysis. IEEE Trans Biom Behav Identity Sci 2(2):85–97

    Google Scholar 

  • Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N (2018) Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans Neural Netw Learn Syst 29(12):6323–6332

    Article  MathSciNet  Google Scholar 

  • Li Z, Yao L, Chang X, Zhan K, Sun J, Zhang H (2019) Zero-shot event detection via event-adaptive concept relevance mining. Pattern Recognit 88:595–603

    Google Scholar 

  • Li H, Wu L, Wang H, Han C, Quan W, Zhao J (2020) Hand gesture recognition enhancement based on spatial fuzzy matching in leap motion. IEEE Trans Ind Inf 16(3):1885–1894

    Google Scholar 

  • Liu Y, Zhang Y, Zeng M (2018) Novel algorithm for hand gesture recognition utilizing a wrist-worn inertial sensor. IEEE Sens J 18(24):10085–10095

    Google Scholar 

  • Liu L, Huai Y (2019) Dynamic hand gesture recognition using LMC for flower and plant interaction. Int J Pattern Recognit Artif Intell 33(01):1950003

    Google Scholar 

  • Lu Z, Chen X, Li Q, Zhang X, Zhou P (2014) A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE Trans Hum Mach Syst 44(2):293–299

    Google Scholar 

  • Luo M, Chang X, Nie L, Yang Y, Hauptmann AG, Zheng Q (2018a) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans Cybern 48(2):648–660

    Google Scholar 

  • Luo M, Nie F, Chang X, Yang Y, Hauptmann AG, Zheng Q (2018b) Adaptive unsupervised feature selection with structure regularization. IEEE Trans Neural Netw Learn Syst 29(4):944–956

    Google Scholar 

  • Maqueda AI, del-Blanco CR, Jaureguizar F, García N (2016) Temporal pyramid matching of local binary subpatterns for hand-gesture recognition. IEEE Signal Process Lett 23(8):1037–1041

    Google Scholar 

  • Maqueda AI, del-Blanco CR, Jaureguizar F, Garcıa N (2015) Human–computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns. Comput Vis Image Underst 141:126–137

    Google Scholar 

  • Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–56

    Google Scholar 

  • Murthy MYB, Koteswararao A, Babu MS (2021) Adaptive fuzzy deformable fusion and optimized CNN with ensemble classification for automated brain tumor diagnosis. Biomed Eng Lett 12:37–58

    Google Scholar 

  • Ohn-Bar E, Trivedi MM (2014) Hand gesture recognition in real-time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans Intell Transp Syst 15(6):2368–2377

    Google Scholar 

  • Plouffe G, Cretu A (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Meas 65(2):305–316

    Google Scholar 

  • Pratondo A, Chui C-K, Ong S-H (2017) Integrating machine learning with region-based active contour models in medical image segmentation. J Vis Commun Image Represent 43:1–9

    Google Scholar 

  • Qi J, Jiang G, Li G, Sun Y, Tao B (2019) Intelligent human–computer interaction based on surface EMG gesture recognition. IEEE Access 7:61378–61387

    Google Scholar 

  • Rautaray SS, Agrawal A (2015) Vision-based hand gesture recognition for human-computer interaction: a survey. Artif Intell Rev 43:1–54

    Google Scholar 

  • Ren P, Xiao Y, Chang X, Huang P-Y, Li Z, Chen X, Wang X (2021) A comprehensive survey of neural architecture search: challenges and solutions 37(4): 111: 33

  • Skaria S, Al-Hourani A, Lech M, Evans RJ (2019) Hand-gesture recognition using two-antenna doppler radar with deep convolutional neural networks. IEEE Sens J 19(8):3041–3048

    Google Scholar 

  • Song T, Zhao H, Liu Z, Liu H, Hu Y, Sun D (2020) Intelligent human hand gesture recognition by local-global fusing quality-aware features. Future Gener Comput Syst 115:298–303

    Google Scholar 

  • Sreedharan NPN, Ganesan B, Raveendran R, Sarala P, Dennis B, Boothalingam RR (2018) Grey Wolf optimization-based feature selection and classification for facial emotion recognition. IET Biom 7(5):490–499

    Google Scholar 

  • Su M-C (2000) A fuzzy rule-based approach to Spatio-temporal hand gesture recognition. IEEE Trans Syst Man Cybern 30(2):276–281

    Google Scholar 

  • Swamy SM, Rajakumar BR and Valarmathi IR (2013) Design of hybrid wind and photovoltaic power system using opposition-based genetic algorithm with cauchy mutation. In: IET Chennai fourth international conference on sustainable energy and intelligent systems (SEISCON 2013)

  • Tsai T, Huang C, Zhang K (2020) Design of hand gesture recognition system for human-computer interaction. Multimed Tools Appl 79:5989–6007

    Google Scholar 

  • Tsang S, Kao B, Yip KY, Ho W, Lee SD (2011) Decision trees for uncertain data. IEEE Trans Knowl Data Eng 23(1):64–78

    Google Scholar 

  • Tu P, Huang C (2019) Mechanical arm teleoperation control system by dynamic hand gesture recognition based on kinect device. J Eng 2019(23):9110–9113

    Google Scholar 

  • Tuccitto N, Bombace A, Torrisi A, Licciardello A, Sciuto GL, Capizzi G, Wozniak M (2019) Probabilistic neural network-based classifier of ToF-SIMS single-pixel spectra. Chemom Intell Lab Syst 191:138–142

    Google Scholar 

  • Wu J, Yang H (2015) Linear regression-based efficient SVM learning for large-scale classification. IEEE Trans Neural Netw Learn Syst 26(10):2357–2369

    MathSciNet  Google Scholar 

  • Wu D et al (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1583–1597

    Google Scholar 

  • Yadav K, Bhattacharya J (2016) Real-time hand gesture detection and recognition for human–computer interaction. Intell Syst Technol Appl 384

  • Yang M, Kpalma K, Ronsin J (2008) A survey of shape feature extraction techniques. Pattern Recognition. In-Tech, p 43–90

  • Yu E, Sun J, Li J, Chang X, Han X-H, Hauptmann AG (2019) Adaptive semi-supervised feature selection for cross-modal retrieval. IEEE Trans Multimed 21(5):1276–1288

    Google Scholar 

  • Zhang L, Luo M, Liu J, Chang X, Yang Y, Hauptmann AG (2020) Deep top-k ranking for image-sentence matching. IEEE Trans Multimed 22(3):775–785

    Google Scholar 

  • Zhihui L, Feiping N, Xiaojun C, Liqiang N, Huaxiang Z, Yi Y (2018) Rank-constrained spectral clustering with flexible embedding. IEEE Trans Neural Netw Learn Syst 29(12):6073–6082

    MathSciNet  Google Scholar 

  • Zhou R, Chang X, Shi L, Shen Y-D, Yang Y, Nie F (2020) Person reidentification via multi-feature fusion with adaptive graph learning. IEEE Trans Neural Netw Learn Syst 31(5):1592–1601

    Google Scholar 

  • Zhu Y, Huang C (2012) An improved median filtering algorithm for image noise reduction. Phys Proc 25:609–616

    Google Scholar 

  • Zhu Y, Xu G (2002) A real-time approach to the spotting, representation, and recognition of hand gestures for human-computer interaction. Comput Vis Image Underst 85:189–208

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ABES Engineering College, Ghaziabad, Uttar Pradesh, 201009, India

    Anil Kumar Dubey

Authors
  1. Anil Kumar Dubey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anil Kumar Dubey.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dubey, A.K. Enhanced hand-gesture recognition by improved beetle swarm optimized probabilistic neural network for human–computer interaction. J Ambient Intell Human Comput 14, 12035–12048 (2023). https://doi.org/10.1007/s12652-022-03753-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-03753-9

Keywords

Search

Navigation