Loading [a11y]/accessibility-menu.js
Efficient Vision Transformer for Human-Centric AIoT Applications Through Token Tracking Assignment | IEEE Journals & Magazine | IEEE Xplore

Efficient Vision Transformer for Human-Centric AIoT Applications Through Token Tracking Assignment


Abstract:

The integration of Artificial Intelligence Internet of Things (AIoT) with consumer electronics has resulted in enhanced connectivity and intelligence within the consumer ...Show More

Abstract:

The integration of Artificial Intelligence Internet of Things (AIoT) with consumer electronics has resulted in enhanced connectivity and intelligence within the consumer electronics sector, thereby facilitating a more convenient living experience for individuals. Particularly, the utilization of Vision Transformer (ViT) in AIoT has further improved the decision-making capabilities of consumer electronics devices. To cater to the increasing demand for efficient and intelligent consumer electronics, the prevalent approach involves transferring ViT models, which are trained on AIoT cloud servers, to edge devices for deployment. However, the incongruity between resource-constrained edge devices and computationally intensive ViT models necessitates the reduction of redundant computations in ViT to optimize the utilization of edge devices. This paper introduces a novel approach named TMTP, which combines token merging and pruning techniques. TMTP encompasses three key blocks: the Token Merging Block (TMB), the Token Tracking Assignment Block (TAB), and the Token Pruning Block (TPB). TMTP effectively diminishes redundant computations in the ViT model while preserving its accuracy. The experimental results demonstrate that by employing TMTP to hierarchically prune 80% of the input tokens in the SWAG model, the number of GFLOPs is significantly reduced by 35%. Interestingly, the accuracy of ImageNet-1k experiences a mere reduction of 0.05%-0.18%.
Published in: IEEE Transactions on Consumer Electronics ( Volume: 70, Issue: 1, February 2024)
Page(s): 1029 - 1039
Date of Publication: 13 October 2023

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.