LSVP: A visual based deep neural direction learning model for point-of-interest recommendation on sparse check-in data

Abstract

Recently accumulated massive amounts of geo-tagged photos provide an excellent opportunity to understand human behaviors and can be used for personalized POI recommendation. However, no existing work has considered both the visual contents in these photos and the sequential patterns of users’ check-ins for POI recommendation. To this end, in this paper, we propose an attentional network named LSVP for POI recommendation, which adaptively considers the joint effects of users’ long-term, short-term and visual preferences. Specifically, we first extract visual preferences from photos, then extract long-term and short-term preferences from check-in sequences. At last, an adaptive attention mechanism is used to balance all the extracted users’ preferences. Experimental results on two real-world datasets collected show that LSPV provides significantly superior performances compared to other state-of-the-art POI recommendation models in terms of accuracy.

Introduction

With the popularity of location-based social networks (LBSNs) and mobile devices, users tend to share their locations and experiences, leading to a great amount of geo-tagged data to be accumulated. The availability of user check-in data brings in excellent opportunities to understand human behaviors and make recommendations to users.

Different from traditional recommendation, the check-in data for POI recommendation is extremely sparse. Because users must physically visit the POI to rate it, the cost is more expensive than a click or rating on a movie online. Besides, the cold start problem (no historical check-in records for new users or new POIs) is even more severe in personalized POI recommendation. Therefore, additional information needs to be incorporated to address these issues. Various information are integrated to traditional recommendation models, i.e., geographical influence [33], social correlations [13], [21] and textual contents [27], [25]. These information have been proven to be effective for improving POI recommendations.

We usually use users’ comments on POIs or the labels specified by the platform to obtain additional textual information [26]. However, in some social networks that dominate pictures such as Instagram and Flickr, users are willing to share photos on it than describe the attributes of POIs through text. To make a better recommendation for such users, we find that visual features in geo-tagged photos taken by users can provide meaningful context information for predicting users’ visit preferences [41]. For example, Fig. 1 shows some POI photos from two users. We can observe that user A prefers to visit the natural landscape while user B tends to visit the cultural place. From these photos, the POI information can be inferred, also users’ behaviors and preferences can be revealed. Previous work [18], [42] also suggests that photos and POIs have strong connections. Thus, incorporating photos could improve the performance of POI recommendations.

Unfortunately, most models that incorporate visual contents only capture users’ long-term preferences, which are considered statistical data or change slowly over time. Wang et al. [29] incorporate visual contents into a probabilistic model to learn the potential functions of users and POIs, Zhang et al. [39] integrates visual contents and geographic influence into a matrix decomposition model. However, the intention of user behavior is inherently variable. It will be affected by various factors during a specific period of time, such as the evolution of interests, instant demand, and global mainstream fashion [34]. Therefore, it is necessary to consider long-term, short-term and visual preferences of users.

In this paper, we integrate the visual contents of geo-tagged photos and the sequential patterns of check-in sequences into an attention-based neural model to capture users’ Long-term, Short-term, and Visual Preferences (LSVP). Specifically, we extract visual features from photos taken by users, and use them to understand the style of POIs and users’ visual preferences. Then, we adopt an LSTM-based network with temporal gating, which learns users’ long-term and short-term preferences from users’ historical check-ins. Finally, an adaptive attention mechanism is used to balance all the extracted users’ preferences. Experimental results show that our proposed model is significantly better than some of the latest models. The main contributions of this article are summarized as follows:

• We propose a deep neural network model LSVP to make POI recommendation for users who are active in social networks through pictures. We integrate visual contents and sequential patterns to achieve more accurate recommendation, so that the data sparsity problem can be well resolved.

• We design a two-layer attentional network for POI recommendation to effectively integrate the joint effects of users’ long-term, short-term and visual preferences through the specific contextual information, such that their weights are reasonably balanced for the personalized recommendation.

• We conduct extensive experiments on two real datasets. Experimental results demonstrate that our proposed model outperforms several state-of-the-art models significantly.

The remainder of the paper is organized as follows. We first review related work from POI recommendation and visual contents for recommendation in Section 2. Afterwards, we define some important concepts used in this paper and formulate the problem in Section 3, before delving into details of the proposed method in Section 4. We perform extensive empirical research in Section 5 and conclude the paper in Section 6.