FBCA: FPGA-Based Balanced Convolutional Attention Module

Abstract

Large-scale computation and data processing are common tasks in machine learning. While traditional central processors are capable of performing these tasks, their computational speed is often inadequate when dealing with large-scale data sets and deep neural networks. As a result, many accelerators have emerged, such as graphics processors, field-programmable gate arrays, etc. FPGA have become a widely used type of accelerator compared to other accelerators due to their high flexibility, high performance, low power consumption, and low latency. However, most of the existing FPGA accelerators only accelerate single modules of CNN, RNN, and attention modules, and few cases of joint acceleration for different types of network combinations are mentioned. Therefore, this work is based on the hardware design of a model with a combination of convolutional and attention modules, and the way they combine to process the data is a perfect fit for the core of hardware acceleration. On the hardware device, the data in this model can flow into the computation at the same time to obtain parallel processing speed. We use a cut that is more suitable for hardware parallelism to process the data coming into both modules, thus making the best use of resources and keeping the time of both modules close to each other. In the same way, for the most computationally heavy loop structure, we have adapted the array structure for faster computation. We also parallelize the design of the serial linear layer in the attention module after the efforts in this paper, the model is further streamlined and accelerated, and finally, our model achieves a speedup of 12.5 times with only a 0.25 decrease in BLEU.