Skip to main content
SpringerLink
Log in

Human-Computer Interactive Digital-Twin System Driven by Magnetic-Inertia Fusion Data

  • Conference paper
  • First Online:
Intelligent Robotics and Applications (ICIRA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14274))

Included in the following conference series:

  • 401 Accesses

Abstract

As the basic interface for human-computer interaction, human hand allows us to operate digital devices in a natural and intuitive way. With the advent of touch screens, gestures and other forms of gesture recognition technology, the role of the hand in human-computer interaction has become even more important. From mobile devices to game consoles, virtual reality systems to industrial systems, it enables us to quickly manipulate objects and perform complex tasks that are difficult to achieve with traditional input devices. In this paper, we develop a digital twin system based on magnetic inertial data driven data glove, and build a personalized 3D virtual hand model in Solidworks. Import Simulink through the United Robotics Description Format (URDF) plug-in. Then Simscpae Multibody is used to drive the virtual hand model in real time. The simulation results show that the virtual finger model obtained by the inverse solution of magnetic positioning can follow the human hand to complete the corresponding action under the interface of Simulink, and the real-time performance is good.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mohammed, Y.B., Karagozlu, D.: A review of human-computer interaction design approaches towards information systems development. BRAIN-Broad Res. Artif. Intell. Neurosci. 12(1), 229–250 (2021)

    Google Scholar 

  2. Vera Anaya, D.F., Yuce, M.R.: A hands-free human-computer-interface platform for paralyzed patients using a TENG-based eyelash motion sensor. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 4567–4570. Montreal, QC, Canada (2020)

    Google Scholar 

  3. Cai, J., Cai, J.: Human-computer interaction-based robotic design and development and control system testing —Prototyping and implementation testing study of a robotic hand. In: 2022 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC), pp. 471–475. Dalian, China (2022)

    Google Scholar 

  4. Liu, X.: Earth Environ. Sci. 252, 042090 (2019)

    Google Scholar 

  5. Królak, A., Strumiłło, P.: Eye-blink detection system for human–computer interaction. Univ. Access Inf. Soc. 11, 409–419 (2012)

    Article  Google Scholar 

  6. Hao, Z., et al.: Biofabrication of a Low Modulus Bioelectroprobe for Neurons to Grow Into. Materials (2012)

    Google Scholar 

  7. Jiang, S., Kang, P., Song, X., Lo, B.P.L., Shull, P.B.: Emerging wearable interfaces and algorithms for hand gesture recognition. In: IEEE Reviews in Biomedical Engineering, vol. 15, pp. 85–102 (2022)

    Google Scholar 

  8. Pan, M., Tang, Y., Li, H.: State-of-the-art in data gloves: a review of hardware, algorithms, and applications. In: IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1–15 (2023). Art no. 4002515

    Google Scholar 

  9. Almeida, L., et al.: Towards natural interaction in immersive reality with a cyber-glove. In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 2653–2658. Bari, Italy (2019)

    Google Scholar 

  10. Chou, W., Wang, T., Hu, L.: Design of data glove and arm type haptic interface. In: 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings, pp. 422–427. Los Angeles, CA, USA (2003)

    Google Scholar 

  11. Michał, S., Leszek, B., Jarosław, P., Wojciech, K.: Modeling and simulation of movement of dispersed group of mobile robots using Simscape multibody software. In: AIP Conference Proceedings (2019)

    Google Scholar 

  12. Jiang, Y., Peng, P., Wang, L., Wang, J., Liu, Y., Wu, J.: Modeling and Simulation of Unmanned Driving System for Load Haul Dump Vehicles in Underground Mines. Sustainability (2022)

    Google Scholar 

  13. Kaur, G., Kumar, S.: Comparison: Matrix Converter, Cycloconverter, and DC Link Converter. In: Singh, S., Wen, F., Jain, M. (eds.) Advances in Energy and Power Systems. Lecture Notes in Electrical Engineering, vol. 508. Springer, Singapore (2018)

    Google Scholar 

  14. Shen, H., Ge, R., Gu, X., Cai, X., Gan, Y., Yang, G.: Method of dexterous finger unit full-pose modeling based on magnetic-inertial sensor fusion. Jixie Gongcheng Xuebao 58(18), 133–140 (2022)

    Google Scholar 

  15. Zhu, X., Tian, W., Li, G., Yu, J.: Research on localization vehicle based on multiple sensors fusion system. In: 2017 International Conference on Computer Network, Electronic and Automation (ICCNEA), pp. 491–494. Xi'an, China (2017)

    Google Scholar 

Download references

Acknowledgement

This work is supported by National Natural Science Foundation of China (No. 52175055).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Minghao Zhou, Ruikang Ge, Xiaotong Cai & Huimin Shen

Authors
  1. Minghao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruikang Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaotong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huimin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huimin Shen .

Editor information

Editors and Affiliations

  1. Zhejiang University, Hangzhou, China

    Huayong Yang

  2. Harbin Institute of Technology, Shenzhen, China

    Honghai Liu

  3. Zhejiang University, Hangzhou, China

    Jun Zou

  4. Huazhong University of Science and Technology, Wuhan, China

    Zhouping Yin

  5. Shenyang Institute of Automation, Shenyang, Liaoning, China

    Lianqing Liu

  6. Zhejiang University, Hangzhou, China

    Geng Yang

  7. Zhejiang University, Hangzhou, China

    Xiaoping Ouyang

  8. Harbin Institute of Technology, Shenzhen, China

    Zhiyong Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, M., Ge, R., Cai, X., Shen, H. (2023). Human-Computer Interactive Digital-Twin System Driven by Magnetic-Inertia Fusion Data. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14274. Springer, Singapore. https://doi.org/10.1007/978-981-99-6501-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-6501-4_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-6500-7

  • Online ISBN: 978-981-99-6501-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation