Processing math: 100%
A Novel PPG-FMG-ACC Wristband for Hand Gesture Recognition | IEEE Journals & Magazine | IEEE Xplore

A Novel PPG-FMG-ACC Wristband for Hand Gesture Recognition


Abstract:

Wrist-based hand gesture recognition has the potential to unlock naturalistic human-computer interaction for a vast array of virtual and augmented reality applications. P...Show More

Abstract:

Wrist-based hand gesture recognition has the potential to unlock naturalistic human-computer interaction for a vast array of virtual and augmented reality applications. Photoplethysmography (PPG), force myography (FMG), and accelerometry (ACC) have generally been proposed as isolated single sensing modalities for gesture recognition, but any of these alone is inherently limited in the amount of biological information it can collect during finger and hand movements. We thus propose a novel, wrist-based, PPG-FMG-ACC combined sensing approach based on a multi-head attention mechanism fusion convolutional neural network (CNN-AF) for gesture recognition. Nine subjects performed twelve hand gestures involving various wrist and finger postures. Experimental results showed that multi-modal fusion improved classification performance significantly (p < 0.01) compared to any single sensing modality, and the F1-score of the combined PPG-FMG-ACC approach was 40.1% higher than PPG alone, 27.4% higher than ACC alone, and 11.9% higher than FMG alone. To the best of our knowledge, this paper is the first to combine wrist-based PPG, FMG, and ACC signals for hand gesture recognition. These results could serve to inform wrist-based gesture recognition design (e.g., via a smartwatch) and thus expand the capabilities of intuitive and ubiquitous human-machine interaction.
Published in: IEEE Journal of Biomedical and Health Informatics ( Volume: 26, Issue: 10, October 2022)
Page(s): 5097 - 5108
Date of Publication: 26 July 2022

ISSN Information:

PubMed ID: 35881605

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.