Skip to main content
SpringerLink
Log in

Trbaggboost: an ensemble-based transfer learning method applied to Indian Sign Language recognition

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

An efficient sign language recognition (SLR) system would help speech and hearing-impaired people to communicate with normal people. This work aims to develop a SLR system for Indian sign language using data acquired from multichannel surface electromyogram, tri-axis accelerometers and tri-axis gyroscopes placed on both the forearms of signers. A novel ensemble-based transfer learning algorithm called Trbaggboost is proposed, which uses small amount of labeled data from a new subject along with labelled data from other subjects to train an ensemble of learners for predicting unlabeled data from the new subject. Conventional machine learning algorithms such as decision tree, support vector machine and random forest (RF) are used as base learners. The results for classification of signs using Trbaggboost are compared with commonly used transfer learning algorithms such as TrAdaboost, TrResampling, TrBagg, and simple bagging approach such as RF. Average accuracy for classification of signs performed by a new subject is achieved as 69.56% when RF is used without transfer learning. When just two observations of labeled data from a new subject are integrated with training data of an existing SLR system, average classification accuracy for TrAdaboost, TrResampling, TrBagg and RF are 71.07%, 72.92%, 76.10% and 76.79%, respectively. However, for the same number of labelled data from the new subject, Trbaggboost yields an average classification accuracy of 80.44%, indicating the effectiveness of the algorithm. Moreover, the classification accuracy for Trbaggboost improves up to 97.04% as the number of labelled data from the new user increase.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Ahmed MA, Zaidan BB, Zaidan AA, Salih MM, Lakulu MM (2018) A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors 18(7):2208. https://doi.org/10.3390/s18072208

    Article  Google Scholar 

  • Ali SM, Augusto JC, Windridge D (2019) A survey of user-centred approaches for smart home transfer learning and new user home automation adaptation. Appl Artif Intell 33(8):747–774. https://doi.org/10.1080/08839514.2019.1603784

    Article  Google Scholar 

  • Bickel S, Brückner M, Scheffer T (2009) Discriminative learning under covariate shift. J Mach Learn Res 10:2137–2155

    MathSciNet  MATH  Google Scholar 

  • Chiang YT, Lu CH, Hsu JY (2017) A feature-based knowledge transfer framework for cross-environment activity recognition toward smart home applications. IEEE Trans Hum Mach Syst 47(3):310–322. https://doi.org/10.1109/THMS.2016.2641679

    Article  Google Scholar 

  • Chong TW, Lee BG (2018) American sign language recognition using leap motion controller with machine learning approach. Sensors 18(10):3554. https://doi.org/10.3390/s18103554

    Article  Google Scholar 

  • Dai W, Yang Q, Xue GR, Yu Y (2007) Boosting for transfer learning. In Proceedings of the 24th international conference on Machine learning ACM, pp. 193–200. doi: 10.1145/1273496.1273521

  • Farhadi A, Forsyth D, White R (2007) Transfer learning in sign language. IEEE conference on computer vision and pattern recognition, pp. 1–8. doi: 10.1109/CVPR.2007.383346

  • Gattupalli S, Ghaderi A, Athitsos V (2016) Evaluation of deep learning based pose estimation for sign language recognition. In Proceedings of the 9th ACM international conference on pervasive technologies related to assistive environments, 12, pp. 1–7. doi: 10.1145/2910674.2910716

  • Hu C, Chen Y, Peng X, Yu H, Gao C, Hu L (2018) A Novel feature incremental learning method for sensor-based activity recognition. IEEE Trans Knowl Data Eng 31(6):1038–1050. https://doi.org/10.1109/TKDE.2018.2855159

    Article  Google Scholar 

  • Indian Sign Language Dictionary (2015) Ramakrishna mission Vivekananda University, Coimbatore. http://indiansignlanguage.org/dictionary/. Accessed 25 May 2020

  • Kamishima T, Hamasaki M, Akaho S (2009) TrBagg: a simple transfer learning method and its application to personalization in collaborative tagging. Ninth IEEE international conference on data mining, 6, pp. 219-228. doi: 10.1109/ICDM.2009.9

  • Khan MA, Roy N (2017) Transact: transfer learning enabled activity recognition. IEEE international conference on pervasive computing and communications workshops, pp. 545–550. doi: 10.1109/PERCOMW.2017.7917621

  • Kyranou I, Vijayakumar S, Erden MS (2018) Causes of performance degradation in electromyographic pattern recognition in upper limb prostheses. Front Neurorobotics 12:58. https://doi.org/10.3389/fnbot.2018.00058

    Article  Google Scholar 

  • Liu X, Wang G, Cai Z, Zhang H (2016) Bagging based ensemble transfer learning. J Ambient Intell Hum Comput 7(1):29–36. https://doi.org/10.1007/s12652-015-0296-5

    Article  Google Scholar 

  • Liu X, Liu Z, Wang G, Cai Z, Zhang H (2017) Ensemble transfer learning algorithm. IEEE Access 6:2389–2396. https://doi.org/10.1109/THMS.2016.2641679

    Article  Google Scholar 

  • Ma Y, Zhou G, Wang S, Zhao H, Jung W (2018) SignFi: sign language recognition using WiFi. Proc ACM Interac Mob Wearable Ubiquitous Technol 2(1):23. https://doi.org/10.1145/3191755

    Article  Google Scholar 

  • Mocialov B, Hastie H, Turner G (2018) Transfer learning for British Sign Language Modelling. In Proceedings of the fifth workshop on NLP for similar languages, varieties and dialects, pp. 101–110

  • Phinyomark A, Phukpattaranont P, Limsakul C (2012) Feature reduction and selection for EMG signal classification. Expert Syst Appl 39(8):7420–7431. https://doi.org/10.1016/j.eswa.2012.01.102

    Article  Google Scholar 

  • Raheja JL, Mishra A, Chaudhary A (2016) Indian sign language recognition using SVM. Pattern Recognit Image Anal 26(2):434–441. https://doi.org/10.1134/S1054661816020164

    Article  Google Scholar 

  • Seiffert C, Khoshgoftaar TM, Van Hulse J, Napolitano A (2008) Resampling or reweighting: a comparison of boosting implementations. 20th IEEE international conference on tools with artificial intelligence, 1, pp. 445–451. doi: 10.1109/ICTAI.2008.59

  • Sharma S, Gupta R, Kumar A (2019) On the use of multi-modal sensing in sign language classification. 6th IEEE international conference on signal processing and integrated networks (SPIN), pp. 495–500. doi: 10.1109/SPIN.2019.8711702

  • Song P, Zheng W (2018) Feature selection based transfer subspace learning for speech emotion recognition. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2018.2800046

    Article  Google Scholar 

  • Suharjito, Anderson R, Wiryana F, Ariesta MC, Kusuma GP (2017) Sign language recognition application systems for deaf-mute people: a review based on input-process-output. Proced Comput Sci 116:441–448. https://doi.org/10.1016/j.procs.2017.10.028

    Article  Google Scholar 

  • Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3(1):9. https://doi.org/10.1186/s40537-016-0043-6

    Article  Google Scholar 

  • Wu J, Sun L, Jafari R (2016) A wearable system for recognizing American sign language in real-time using IMU and surface EMG sensors. IEEE J Biomed Health Inform 20(5):1281–1290. https://doi.org/10.1109/JBHI.2016.2598302

    Article  Google Scholar 

  • Xi X, Tang M, Miran SM, Luo Z (2017) Evaluation of feature extraction and recognition for activity monitoring and fall detection based on wearable sEMG sensors. Sensors 17(6):1229. https://doi.org/10.3390/s17061229

    Article  Google Scholar 

  • Yang X, Chen X, Cao X, Wei S, Zhang X (2016) Chinese sign language recognition based on an optimized tree-structure framework. IEEE J Biomed Health Inform 21(4):994–1004. https://doi.org/10.1109/JBHI.2016.2560907

    Article  Google Scholar 

  • Zhou ZH (2015) Ensemble learning. Encycl Biom. https://doi.org/10.1007/978-1-4899-7488-4

    Article  Google Scholar 

  • Zhu Z, Wang X, Kapoor A, Zhang Z, Pan T, Yu Z (2018) EIS: a wearable device for epidermal American Sign Language recognition. Proc ACM Interact Mob Wearable Ubiquitous Technol 2(4):202. https://doi.org/10.1145/3287080

    Article  Google Scholar 

Download references

Acknowledgements

The author thankfully acknowledges the support and funds provided by Science and Engineering Research Board (SERB), a statutory body from the Department of Science and Technology (DST), (ECR/2016/000637) Government of India. The author also thanks for the support and patience of all the volunteers in recording the data.

Author information

Authors and Affiliations

  1. Electronics and Communication Engineering Department, Amity University Uttar Pradesh, Noida, India

    S. Sharma & R. Gupta

  2. Centre for Applied Research in Electronics, Indian Institute of Technology Delhi, New Delhi, India

    A. Kumar

Authors
  1. S. Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Sharma.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sharma, S., Gupta, R. & Kumar, A. Trbaggboost: an ensemble-based transfer learning method applied to Indian Sign Language recognition. J Ambient Intell Human Comput 13, 3527–3537 (2022). https://doi.org/10.1007/s12652-020-01979-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-01979-z

Keywords

Search

Navigation