The inherent dependencies among video content, personal characteristics, and perceptual emotion are crucial for personalized video emotion tagging, but have not been thoroughly exploited. To address this, we propose a novel topic model to capture such inherent dependencies. We assume that there are several potential human factors, or “topics,” that affect the personal characteristics and the personalized emotion responses to videos. During training, the proposed topic model exploits the latent space to model the relationships among personal characteristics, video content and video tagging behaviors. After learning, the proposed model can generate meaningful latent topics, which help personalized video emotion tagging. Efficient learning and inference algorithms of the model are proposed. Experimental results on the CP-QAE-I database demonstrate the effectiveness of the proposed approach in modeling complex relationships among video content, personal characteristics, and perceptual emotion, as well as its good performance in personalized video emotion.