The auto-correlation enhanced transformer for information diffusion prediction in social networks

Abstract

Information diffusion prediction aims to analyze the patterns of information propagation in social media to understand and forecast the process of information dissemination. Previous Transformer-based methods are usually limited to dynamic short cascades and do not combine temporal and structural information effectively. Additionally, the self-attention mechanism in Transformers cannot capture the periodic dependencies of sub-sequences, thereby reducing prediction precision. To address the above issues, we propose a dynamic heterogeneous social network information diffusion prediction model called Auto-Correlation Enhanced Transformer (ACET). First, we construct a heterogeneous graph composed of a social network graph and a dynamic diffusion graph to learn users’ structural characteristics. We also construct a user co-occurrence graph based on the information diffusion sequence, leading to more accurate user embeddings. Next, to discover the dependencies at the sub-sequences level and reduce computational complexity, we replace self-attention in Transformer with the auto-correlation mechanism resulting in improved prediction accuracy of the model. Finally, for the effective fusion of user similarity, temporal features, and structural features, a novel residual fusion method is proposed to replace the original one in Transformer. Experimental results show that the performance of ACET on three public datasets is all improved. Specially, the Map@k and Hits@k on Douban dataset are improved by an average of 22.95% and 15.88%, respectively.