Hybrid precoding design is a high-complexity problem due to the coupling of analog and digital precoders as well as the constant modulus constraint for the analog precoder. Fortunately, the deep learning based hybrid precoding methods can significantly reduce the complexity, but the performance remains limited. In this paper, inspired by the attention mechanism recently developed for machine learning, we propose an attention-based hybrid precoding scheme for millimeter-wave (mmWave) MIMO systems with improved performance and low complexity. The key idea is to design each user’s beam pattern according to its attention weights to other users’. Specifically, the proposed attention-based hybrid precoding scheme consists of two parts, i.e., the attention layer and the convolutional neural network (CNN) layer. The attention layer is used to identify the features of inter-user interferences. Then, these features are processed by the CNN layer for the analog precoder design to maximize the achievable sum-rate. Simulation results demonstrate that the attention layer could mitigate the inter-user interferences, and the proposed attention-based hybrid precoding with low complexity can achieve higher achievable sum-rate than the existing deep learning based method.