Loading [MathJax]/extensions/MathMenu.js
Pose-Promote: Progressive Visual Perception for Activities of Daily Living | IEEE Journals & Magazine | IEEE Xplore

Pose-Promote: Progressive Visual Perception for Activities of Daily Living


Abstract:

Poses are effective in interpreting fine-grained human activities, especially when encountering complex visual information. Unimodal methods for action recognition unsati...Show More

Abstract:

Poses are effective in interpreting fine-grained human activities, especially when encountering complex visual information. Unimodal methods for action recognition unsatisfactorily to daily activities due to the lack of a more comprehensive perspective. Multimodal methods to combine pose and visual are still not exhaustive enough in mining complementary information. Therefore, we propose a Pose-promote (Ppromo) framework that utilizes a priori knowledge of pose joints to perceive visual information progressively. We first introduce a temporal promote module to activate each video segment using temporally synchronized joint weights. Then a spatial promote module is proposed to capture the key regions in visuals using the learned pose attentions. To further refine the bimodal associations, the global inter-promote module is proposed to align global pose-visual semantics at the feature granularity. Finally, a learnable late fusion strategy between visual and pose is applied for accurate inference. Ppromo achieves state-of-the-art performance on three publicly available datasets.
Published in: IEEE Signal Processing Letters ( Volume: 31)
Page(s): 2950 - 2954
Date of Publication: 14 October 2024

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.