Skip to main content
SpringerLink
Log in

Machine-learning-based hand motion recognition system by measuring forearm deformation with a distance sensor array

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

Studies on hand motion recognition based on biosignals have become popular as such recognition can be applied to various input interfaces and motion measurements for human–robot/computer interaction. In recent years, many machine-learning-based technologies have been developed to analyze such biosignals more accurately. Among various possible biosignals, we focus on forearm deformation which is an alternative source of information for hand motion recognition. The activities of surface and deep layer muscles, tendons, and bones can be extracted from forearm deformation in a non-invasive manner. In this study, a hand motion recognition system is proposed based on forearm deformation. By using machine-learning-based technology, the proposed method can be applied to various users and various measurement conditions. First, a distance sensor array is developed to measure forearm deformation. Then, we test and verify the suitableness of three types of machine-learning-based classifiers (k-NN, SVM, and DNN) using the measured forearm deformation. In experiments, we verified the accuracy of the proposed system with various users. We also test the system for different elbow postures, and when measuring the data over the clothing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Adewuyi, A.A., Hargrove, L.J., Kuiken, T.A.: An analysis of intrinsic and extrinsic hand muscle emg for improved pattern recognition control. IEEE Trans. Neural Syst. Rehabil. Eng. 24(4), 485–494 (2016)

    Article  Google Scholar 

  • Bengio, Y., et al.: Learning deep architectures for AI. Foundations and trends. Mach. Learn. 2(1), 1–127 (2009)

    Article  MathSciNet  Google Scholar 

  • Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: the ACM Annual Workshop on Computational learning theory (COLT), pp. 144–152. ACM (1992)

  • Cai, Y., Ge, L., Cai, J., Yuan, J.: Weakly-supervised 3d hand pose estimation from monocular rgb images. In: the European Conference on Computer Vision (ECCV), pp. 666–682 (2018)

    Chapter  Google Scholar 

  • Cipriani, C., Zaccone, F., Micera, S., Carrozza, M.C.: On the shared control of an emg-controlled prosthetic hand: analysis of user-prosthesis interaction. IEEE Trans. Robot. 24(1), 170–184 (2008)

    Article  Google Scholar 

  • Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967)

    Article  Google Scholar 

  • Fang, B., Sun, F., Liu, H., Liu, C.: 3D human gesture capturing and recognition by the immu-based data glove. Neurocomputing 277, 198–207 (2018)

    Article  Google Scholar 

  • Fantozzi, S., Giovanardi, A., Magalhães, F.A., Di Michele, R., Cortesi, M., Gatta, G.: Assessment of three-dimensional joint kinematics of the upper limb during simulated swimming using wearable inertial-magnetic measurement units. J. Sports Sci. 34(11), 1073–1080 (2016)

    Article  Google Scholar 

  • Fukui, R., Watanabe, M., Shimosaka, M., Sato, T.: Hand-shape classification with a wrist contour sensor: analyses of feature types, resemblance between subjects, and data variation with pronation angle. Int. J. Robot. Res. 33(4), 658–671 (2014)

    Article  Google Scholar 

  • Guo, W., Sheng, X., Liu, J., Hua, L., Zhang, D., Zhu, X.: Towards zero training for myoelectric control based on a wearable wireless semg armband. In: the IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 196–201. IEEE (2015)

  • Ho, N.S.K., Tong, K.Y., Hu, X.L., Fung, K.L., Wei, X.J., Rong, W., Susanto, E.A.: An emg-driven exoskeleton hand robotic training device on chronic stroke subjects: Task training system for stroke rehabilitation. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–5 (2011)

  • Jung, P.G., Lim, G., Kim, S., Kong, K.: A wearable gesture recognition device for detecting muscular activities based on air-pressure sensors. IEEE Trans. Ind. Inf. 11(2), 485–494 (2015)

    Google Scholar 

  • Kato, A., Matsumoto, Y., Kobayashi, Y., Sugano, S., Fujie, M.G.: Estimating a joint angle by means of muscle bulge movement along longitudinal direction of the forearm. In: the IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 614–619. IEEE (2015)

  • Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  • Kortier, H.G., Antonsson, J., Schepers, H.M., Gustafsson, F., Veltink, P.H.: Hand pose estimation by fusion of inertial and magnetic sensing aided by a permanent magnet. IEEE Trans. Neural Syst. Rehabil. Eng. 23(5), 796–806 (2015)

    Article  Google Scholar 

  • Kortier, H.G., Sluiter, V.I., Roetenberg, D., Veltink, P.H.: Assessment of hand kinematics using inertial and magnetic sensors. J. NeuroEng. Rehabil. 11(1), 70–83 (2014)

    Article  Google Scholar 

  • Leonardis, D., Barsotti, M., Loconsole, C., Solazzi, M., Troncossi, M., Mazzotti, C., Castelli, V.P., Procopio, C., Lamola, G., Chisari, C., Bergamasco, M., Frisoli, A.: An EMG-controlled robotic hand exoskeleton for bilateral rehabilitation. IEEE Trans. Haptics 8(2), 140–151 (2015)

    Article  Google Scholar 

  • Li, N., Wei, S., Wei, M., Liu, B., Huo, H., Jiang, L.: Hand motion recognition based on pressure distribution maps and ls-svm. In: 2014 International Conference on Mechatronics and Control (ICMC), pp. 1027–1031. IEEE (2014)

  • Ploderer, B., Fong, J., Withana, A., Klaic, M., Nair, S., Crocher, V., Vetere, F., Nanayakkara, S.: Armsleeve: a patient monitoring system to support occupational therapists in stroke rehabilitation. In: the 2016 ACM Conference on Designing Interactive Systems, pp. 700–711. ACM (2016)

  • Radmand, A., Scheme, E., Englehart, K.: High-density force myography: a possible alternative for upper-limb prosthetic control. J. Rehabil. Res. Dev. 53(4), 443–457 (2016)

    Article  Google Scholar 

  • Rakita, D., Mutlu, B., Gleicher, M.: A motion retargeting method for effective mimicry-based teleoperation of robot arms. In: the ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 361–370. ACM (2017)

  • Shima, K., Tsuji, T.: Classification of combined motions in human joints through learning of individual motions based on muscle synergy theory. In: the IEEE/SICE International Symposium on System Integration (SII), pp. 323–328. IEEE (2010)

  • Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1145–1153 (2017)

  • Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tröster, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7(2), (2008)

    Article  Google Scholar 

  • Xiao, Z.G., Menon, C.: Performance of forearm FMG and SEMG for estimating elbow, forearm and wrist positions. J. Bionic Eng. 14(2), 284–295 (2017)

    Article  Google Scholar 

  • Yoshikawa, M., Mikawa, M., Tanaka, K.: A myoelectric interface for robotic hand control using support vector machine. In: the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2723–2728. IEEE (2007)

  • Yoshikawa, M., Taguchi, Y., Sakamoto, S., Yamanaka, S., Matsumoto, Y., Ogasawara, T., Kawashima, N.: Trans-radial prosthesis with three opposed fingers. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1493–1498. IEEE (2013)

Download references

Author information

Authors and Affiliations

  1. Graduate School of Information Science, Nara Institute of Science and Technology, 8916-5 Takayama, Ikoma, Nara, 630-0192, Japan

    Sung-Gwi Cho, Ming Ding, Jun Takamatsu & Tsukasa Ogasawara

  2. Faculty of Robotics and Design Engineering, Osaka Institute of Technology, 1-45 Chayamachi, Kita, Osaka, 530-8568, Japan

    Masahiro Yoshikawa

Authors
  1. Sung-Gwi Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masahiro Yoshikawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Takamatsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tsukasa Ogasawara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sung-Gwi Cho.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This research was supported by JSPS KAKENHI (Grant number 16K01549) and Tateisi Science and Technology Foundation (Grant number 2187007).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cho, SG., Yoshikawa, M., Ding, M. et al. Machine-learning-based hand motion recognition system by measuring forearm deformation with a distance sensor array. Int J Intell Robot Appl 3, 418–429 (2019). https://doi.org/10.1007/s41315-019-00115-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-019-00115-1

Keywords

Search

Navigation