The paper proposes a multi-factor interlinked POI recommendation model called MFIP. Extracting user similarity for user-sensitive implicit modeling to enrich user representation. Using contextual information such as sequential, geographical and social to construct a POI recommendation model with collaborative influence of multi-factor, alleviating data sparsity. A novel multi-factor interlinked strategy(FIS) is proposed that can dynamically adjust the user preferences of different factors to obtain the comprehensive impact of user personalization. In addition, we propose a active area selection algorithm based on segmentation to model the geographical information more effectively. Finally, we conduct a comprehensive performance evaluation for MFIP on two large-scale real world check-in datasets collected from Gowalla and Yelp. Experimental results show that MFIP achieves significantly superior recommendation performance compared to other state-of-the-art POI recommendation models.