SpARC: Token Similarity-Aware Sparse Attention Transformer Accelerator via Row-wise Clustering
Abstract
References
Index Terms
- SpARC: Token Similarity-Aware Sparse Attention Transformer Accelerator via Row-wise Clustering
Recommendations
Layer-Wise Sparse Training of Transformer via Convolutional Flood Filling
Advances in Knowledge Discovery and Data MiningAbstractSparsifying the Transformer has garnered considerable interest, as training the Transformer is very computationally demanding. Prior efforts to sparsify the Transformer have either used a fixed pattern or data-driven approach to reduce the number ...
Sparse self-attention transformer for image inpainting
AbstractLearning-based image inpainting methods have made remarkable progress in recent years. Nevertheless, these methods still suffer from issues such as blurring, artifacts, and inconsistent contents. The use of vanilla convolution kernels, which have ...
Highlights- To accommodate the long-range modeling capacity of transformer blocks while reducing the computational burden, we introduce a novel U-Net style transformer-based network, called sparse self-attention transformer (Spa-former), to approach ...
Sparse subspace clustering via nonconvex approximation
Among existing clustering methods, sparse subspace clustering (SSC) obtains superior clustering performance in grouping data points from a union of subspaces by solving a relaxed $$\ell _{0}$$ℓ0-minimization problem by $$\ell _{1}$$ℓ1-norm. The use of $$...
Comments
Information & Contributors
Information
Published In
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Check for updates
Qualifiers
- Research-article
Funding Sources
- National Research Foundation of Korea
- Samsung Research Funding and Incubation Center of Samsung Electronics
- Institute of Information & communications Technology Planning & Evaluation
Conference
Acceptance Rates
Upcoming Conference
- Sponsor:
- sigda
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 267Total Downloads
- Downloads (Last 12 months)267
- Downloads (Last 6 weeks)94
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in