Itinerary recommender system with semantic trajectory pattern mining from geo-tagged photos

doi:10.1016/j.eswa.2017.10.049

Expert Systems with Applications

Volume 94, 15 March 2018, Pages 32-40

https://doi.org/10.1016/j.eswa.2017.10.049 Get rights and content

Highlights

•
Propose a semantic-level itinerary recommender system from geo-tagged photos.
•
Incorporation of semantic trajectory patterns into itinerary recommendations.
•
Overcome the inefficiency of traditional itinerary recommender systems.
•
Experimental results to demonstrate the effectiveness of proposed framework.

Abstract

A large number of geo-tagged photos become available online due to the advances in geo-tagging services and Web technologies. These geo-tagged photos are indicative of photo-takers’ trails and movements, and have been used for mining people movements and trajectory patterns. These geo-tagged photos are inherently spatio-temporal, sequential and implicitly containing aspatial semantics. and recommender systems are collaborative filtering based. There have been some studies to build itinerary recommender systems from these geo-tagged photos, but they fail to consider these dimensions and share some common drawbacks, especially lacking aspatial semantics or temporal information. This paper proposes an itinerary recommender system with semantic trajectory pattern mining from geo-tagged photos by discovering sequential points-of-interest with temporal information from other users’ visiting sequences and preferences. Our system considers spatio-temporal, sequential, and aspatial semantics dimensions, and also takes into account user-specified preferences and constraints to customise their requests. It generates a set of customised and targeted semantic-level itineraries meeting the user specified constraints. The proposed method generates these semantic itineraries from historic people’s movements by mining frequent travel patterns from geo-tagged photos. Experimental results demonstrate the informativeness, efficiency and effectiveness of our proposed method over traditional approaches.

Introduction

Travel itinerary recommender systems attempt to assist users on travel planning (Yoon, Zheng, Xie, & Woo, 2012). They provide useful suggestions on a tour about popular places to visit and ideas on a travel route of places and corresponding stay times for users who travel an unfamiliar destination. Intelligent recommender systems play an important role in decision-making and intelligent systems, and require a mixture of knowledge in expert systems, intelligent decision-making systems, and data mining. An itinerary is a detailed trip plan with a travel route associated with stay time information, where the travel route is a sequence of places. With the advance of social media platforms, a large number of online users generates and shares photos they have taken during their trips with their lovers, friends, and families. These photos are about their travel, activities and life, and are indicative of their movements and activities during their travel. Through the geo-tagging service, a large number of photos is becoming tagged with geographic locations. A photo together with geographic information and time stamp indicates a user’s footprint, the place the user visits and the time the user spends there. A series of geo-tagged photos reflects the user’s movement and trajectory. Consequently, the enormous amount of online photo data has become a potential data repository for discovering useful travel information and building travel recommender systems (Beel, Gipp, Langer, Breitinger, 2016, Bobadilla, Ortega, Hernando, Gutiérrez, 2013), like location recommendation (Popescu, Grefenstette, 2011, Waga, Tabarcea, Fränti, 2012, Yamasaki, Gallagher, Chen, 2013) and travel route recommendation (Okuyama & Yanai, 2013).

Existing itinerary recommender systems generate specific itineraries with geographic location information from available geo-tagged photos. Typically, they generate popular Points-of-Interest (PoIs) where a number of photos taken, and map-match PoIs with specific geographic place types to construct a suggestive itinerary. However, traditional approaches share a common major drawback. They are mainly based on geographic spatial information only when they recommend an itinerary. That is, they do not take any aspatial semantic information into account in the recommender system. In many real world scenarios, a user wants to visit a certain place type for instance “zoo” in a given trip. This specific semantically enhanced request is not considered at all in the traditional approaches. Instead, they accommodate this aspatial semantic information as a post-processing stage for their spatial information only recommender systems. Therefore, the traditional recommender systems are not able to accommodate users’ semantically enhanced requests to generate meaningful and semantically enhanced itineraries.

The semantic place type request is an important feature in the user’s travel planning. For users who are unfamiliar with specific geographic locations and PoIs in a certain destination, they prefer to list some place types (categories) they would like to visit (Gionis, Lappas, Pelechrinis, & Terzi, 2014). For instance, a user may want visit “Great Barrier Reef”, “Rain Forest” and “Cultural aboriginal park” in a trip to Cairns, a popular gateway to rain forest and Great Barrier Reef in Australia. In addition, the user may want to visit “Great Barrier Reef” in a clear day to enjoy swimming with fish and exploring the beauty of reefs, whilst “Rain Forest” in a rainy day. The recommended itineraries are expected to contain a set of these requested place types. However, existing itinerary recommender systems fail to consider this constraint considering semantic information in the recommender system. This study presents an itinerary recommender system that considers users’ predefined semantic spatial and aspatial constraints on place types, weather conditions and travel duration time. A semantic-level itinerary is a detailed journey planning with semantic spatial and aspatial information incorporated. It is more detailed and specific than the general spatial location alone itinerary. It shows a sequence of movements among different place types with certain weather conditions and certain stay times. This semantic-level itinerary provides users with flexible choices (rain forest in Kuranda or rain forest in Port Douglas) of specific geographic level route that satisfy their actual requests (Chen, Ku, Sun, & Zimmermann, 2011).

Note that, trajectories generated from geo-tagged photos are inherently spatio-temporal and sequential, and implicitly containing aspatial semantics. Therefore, itineary recommender systems from these spatio-temporal trajectories should incorporate: locational spatial dimension, temporal dimension, sequence and aspatial semantics in addition to two basic features Collaborative Filtering (CF) to benefit from other users and user-specified constraints to refine search result. These six are important features in this type of recommender system, and to the best of our knowledge, there is no known itinerary recommender system that produces a semantic-level itinerary with a set of spatial and aspatial user-specified constraints meeting this set of requirements.

This study develops a semantic-level itinerary recommendation system from geo-tagged photos. This system considers users’ semantic spatial and aspatial requests, and travel duration constraints, and generates semantic-level itineraries that meet the user constraints. The proposed semantic-level itinerary recommendation system aims to provide users with higher level advice on place types, weather conditions and stay times. We generate itineraries based on mining semantic trajectory patterns from geo-tagged photos. We construct people trajectories from geo-tagged photos (raw trajectories), enhance the raw trajectories with required semantics to build semantically enhanced trajectories (semantic trajectories), and mine semantic trajectory patterns that will be basically used to build semantic-level itineraries. We test our algorithm with real datasets from Flickr¹ against traditional spatial only recommender systems. The experimental results support the effectiveness and efficiency of our recommendation system.

The rest of paper is organised as follows. Section 2 reviews current studies in itinerary recommender systems, and Section 3 formulates problems and provides problem statements derived from the literature review. Section 4 introduces a framework of our proposed itinerary recommender system based on trajectory pattern mining from geo-tagged photos whilst Section 4 outlines our experimental design and datasets used. Section 6 provides experimental results to demonstrate the effectiveness and efficiency of our framework over traditional approaches, and analyses the results. Section 7 draws conclusion and provides future work.

Section snippets

Literature review

On-line user generated massive databases have become a potential and useful resource for tourism related research community to build collective intelligence and to generate collectively filtered and recommended travel itineraries (De Choudhury et al., 2010). Advances of Web technologies promote a speedy increase of online user generated photo data, and people are uploading and storing their photos on-line to share their experience and moments with their friends, colleagues and families. Photos

Problem statements and definitions

This study is to build a semantic itinerary recommender system using massive on-line geo-tagged photos. The problem could be described as follows: given a query including a set of required place types and travel duration, $q u e r y = {< T y p e s >, D u r a t i o n},$ our system replies a list of semantic-level candidate itineraries. An itinerary is a sequence of place types with interval time information, that contains some of required place types and satisfies the travel duration. A stop of place type is

Semantic itinerary recommender system and methods

Fig. 1 shows our architecture of semantic itinerary recommender system. The framework includes two main components which are “offline” semantic trajectory pattern mining from geo-tagged photos and “online” itinerary recommendation. In the offline component, we construct people trajectories from geo-tagged photos, generate semantic trajectories associated with basic place type semantics and additional other contextual environment semantics, and extract previous users’ semantic trajectory

Experimental design

We conduct experiments to evaluate the efficiency and effectiveness of the proposed recommender system. The first evaluation mainly focuses on the effectiveness of our system. Specifically, we validate the performance of recommended semantic itineraries that the number of user’s requests the recommended itineraries contains. The second experiment is about evaluations for the informativeness of recommendations. We present what additional useful information our recommendations can provide with a

Effectiveness of recommendation results

We use each testing route as a user query. A query includes a set of place types and a travel duration constraint. In evaluation experiments for the effectiveness of itinerary recommendation results, we choose top five candidate itineraries as final recommendations because the top five candidates provide a temperate diversity and number of recommendations for all three methods. When using more than five candidates, the redundancy of itineraries increases.

The first experimental result is about

Conclusion

In this study, we present an itinerary recommender system using on-line geo-tagged photos. Our system allows user to customise a set of place types and travel duration in the query. The system generates itinerary recommendations based on the previous people semantic trajectory patterns extracted from their historic photo data. Experimental results show that our system is able to produce itinerary recommendations which satisfy user’s predefined requirements. Our system recommends semantic-level

References (43)

J. Bobadilla et al.
Recommender systems survey
Knowledge-Based Systems
(2013)
G. Cai et al.
Sequential pattern mining of geo-tagged photos with an arbitrary regions-of-interest detection method
Expert Systems with Applications
(2014)
R. Feick et al.
A multi-scale approach to exploring urban places in geotagged photographs
Computers, Environment and Urban Systems
(2015)
K.H. Lim
Recommending tours and places-of-interest based on user interests from geo-tagged photos
Proceedings of the 2015 ACM SIGMOD on PhD symposium
(2015)
Y. Sun et al.
Road-based travel recommendation using geo-tagged images
Computers, Environment and Urban Systems
(2015)
P. Symeonidis et al.
Recommender systems for location-based social networks
(2014)
Z. Xu et al.
Topic based context-aware travel recommendation method exploiting geotagged photos
Neurocomputing
(2015)
T. Yamasaki et al.
Personalized intra-and inter-city travel recommendation using large-scale geotags
Proceedings of the 2nd ACM international workshop on geotagging and its applications in multimedia
(2013)
J. Beel et al.
Research-paper recommender systems: A literature survey
International Journal on Digital Libraries
(2016)
A. Bujari et al.
Using gamification to discover cultural heritage locations from geo-tagged photos
Personal and Ubiquitous Computing
(2017)

G. Cai et al.

A framework for mining semantic-level tourist movement behaviours from geo-tagged photos

Australasian joint conference on artificial intelligence

(2016)

G. Cai et al.

Mining semantic sequential patterns from geo-tagged photos

Proceedings of the 2016 49th Hawaii international conference on system sciences (HICSS)

(2016)

L. Cao et al.

A worldwide tourism recommendation system based on geotaggedweb photos

ICASSP

(2010)

H. Chen et al.

The partial sequenced route query with traveling rules in road networks

GeoInformatica

(2011)

Y.-Y. Chen et al.

Travel recommendation by mining people attributes and travel group types from community-contributed photos

IEEE Transactions on Multimedia

(2013)

M. De Choudhury et al.

Automatic construction of travel itineraries using social breadcrumbs

Proceedings of the 21st ACM conference on hypertext and hypermedia

(2010)

M. Ester et al.

A density-based algorithm for discovering clusters in large spatial databases with noise

KDD

(1996)

O. Gaggi

Discovering local attractions from geo-tagged photos

Proceedings of the 28th annual ACM symposium on applied computing

(2013)

F. Giannotti et al.

Efficient mining of temporally annotated sequences

SDM

(2006)

F. Giannotti et al.

Trajectory pattern mining

Proceedings of the 13th ACM SIGKDD international conference on knowledge discovery and data mining

(2007)

A. Gionis et al.

Customized tour recommendations in urban areas

Proceedings of the 7th ACM international conference on web search and data mining

(2014)

Cited by (96)

Selection and visiting sequence of daily attractions: Multi-day travel itinerary recommendation based on multi-source online data
2024, Expert Systems with Applications
This article investigates the problem of recommending multi-day travel itineraries based on multi-source online data to meet the requirements of tourists who wish to plan selection and visiting sequence of daily attractions. Current research mainly uses optimization model-based methods or data-driven methods for multi-day travel itinerary recommendation. However, the former is difficult to utilize others' travel experiences and may lead to over-recommending similar category attractions, while the latter is unable to help tourists determine their specific daily itineraries. Therefore, in this study, a multi-day travel itinerary recommendation framework is proposed based on travelogue data, review data, attraction basic information from travel websites, and traffic data from online maps. The framework aims to achieve multi-day travel itinerary recommendation based on the personalized attraction preferences of tourists, while fully leveraging the travel experiences of others. Four corresponding methods are proposed for the four stages of this framework: 1) a method for mining the preferences of the target tourist for tourist attractions, 2) a method for constructing a candidate multi-day itinerary set, 3) a method for constructing a candidate single-day itinerary set, and 4) a method for recommending multi-day itineraries based on the combination of single-day itineraries. In addition, we conduct experimental studies by using real data to verify the feasibility and effectiveness of the framework. The results show that compared with existing related research, the proposed framework and methods have a significant advantage in hit rate and are helpful for tourists to determine specific daily itineraries during their travel time.
A visiting sequence recommendation framework: Enhanced by dynamic landmark and stay time
2023, Expert Systems with Applications
The content shared via social media significantly influences tourists’ travel planning process and even destination decisions. By analyzing the shared text, photos, and videos on social media, we can gain insight into what tourists like and how they move. Many researchers have studied tour recommendation problems and proposed various solutions based on location-based social media to address this issue. Despite achieving some progress, research on personalized tours still reveals some difficulties. In this paper, we propose a framework for recommending a visiting sequence containing orderly landmarks and stay time (LST) to meet the user’s specified constraints by using user-generated content in a photo-sharing social media. First, the landmarks, also known as POIs, are extracted from the social media photo dataset by clustering photos close to each other, where the closeness can be derived by the geotags in photos. Second, implicit preferences (topics) among the photos are derived by a topic modeling approach called Latent Dirichlet Allocation (LDA). These topics are then used to generate landmark and user feature vectors. Third, users are assigned to the class whose user vectors are similar, and the visiting sequences in the class are used as the base for constructing the prediction model. Fourth, we take advantage of the long-short term memory (LSTM) model, an enhanced version of RNN, and develop an LST-LSTM prediction model to predict the next landmark and its stay time. Finally, the LST-LSTM model keeps generating the most likely landmark and stay time by the proposed time-based tour recommender until the user’s specified constraints are researched. The experiments show that the proposed LST-LSTM predictor performs better than the baseline L-LSTM predictor, which does not consider stay time in all metrics. Moreover, our experiment shows that the proposed recommender can help users visit more landmarks and spend less time when compared with the baseline recommender.
Examining the importance of spatial aspects of travel routes: A multi-method approach
2023, Information Processing and Management
A well-designed travel route can increase tourists’ satisfaction and revisit intentions by helping them explore popular or/and hidden places conveniently and thus better experiencing the destination. The popularity of travel route development has resulted in tourists using cities’ infrastructure to enable travel from areas with well-structured accessibility to more inaccessible areas. Despite the importance of the accessibility of travel routes to enhance travel experiences, the tourism literature lacks an understanding of how travel routes should consider not only the physical distance between tourist destinations but also accessibility using mobility within smart tourism cities. Therefore, this research aims to examine the importance of considering accessibility in assessing a travel route. Drawing on the distance decay theory and the concept of the effective tourism exclusion zone (ETEZ), we argue that a travel route can be adopted when every part of the route is accessible in terms of time cost. To achieve the objective, this research adopted a multi-method approach. We explored how much developers consider accessibility when generating a travel route via a case study in study 1. An experiment, study 2, strengthened the results of study 1, presenting cause-and-effect relationships. This study provides theoretical and practical implications for developers of travel routes.
DSER: Deep-Sequential Embedding for single domain Recommendation
2022, Expert Systems with Applications
Recently, Deep Neural Networks (DNNs) have proved their capability to model nonlinear relationships between users and items in recommender systems. Therefore, many researchers have exploited the DNN to model the non-linearity of individual interactions between users and items. However, it has been barely studied to model the non-linearity of a user’s sequential item interactions that may be the fundamental clue to represent a user’s interest naturally. This paper proposes a novel Deep Sequential Embedding for a single domain Recommendation based on implicit feedback, named DSER. It exploits Doc2vec techniques to convert sequential user–item interactions, rather than textual information, to the feature vectors of users and items. In our framework, a user’s interaction is like a document containing items as words in the word embedding techniques. The obtained user and item vectors are then fed to Multi-Layer Perceptron (MLP) to model a high level of non-linearity for user–item interactions. Finally, the non-linearity of MLP is fused with the linear user–item relationships trained by matrix factorization to predict the possibility that a target user likes items. Extensive experiments on four real-world datasets demonstrate significant improvements of our DSER over state-of-the-art approaches. Moreover, empirical evidence shows that using word embedding offers considerably better predictive performance on sparse data of the smart tourism field, whose new user ratio is relatively high.
A novel Sequence-Aware personalized recommendation system based on multidimensional information
2022, Expert Systems with Applications
Due to the rapid growth of the information overload issue, recommender systems have become necessary and are implemented in numerous facets of human life, including the tourism industry. Today, technological advancements have significantly altered our travels, and these advancements promise even more interactive and exciting experiences in the future. Nowadays, planning and arranging a customized trip well in advance is ideal, as the process can be challenging and time-consuming. This paper proposes a hybrid approach that leverages multidimensional data to enhance personalized trip recommendations while addressing several shortcomings of existing recommender systems, such as their inability to account for dynamic user preferences and diverse contexts. To this end, the proposed method, considering the data sparsity issue, employs a clustering algorithm to reduce the time complexity of discovering Points of Interest (POIs). This approach utilizes user demographic information to address the cold start issue while improving the collaborative filtering (CF) paradigm through an asymmetric schema. Furthermore, this study exploits the context vector model and the Term-Frequency-Inverse-Document-Frequency (TF-IDF) algorithm to present a novel method for quantifying context similarity. Moreover, the method retrieves and ranks a list of candidate routes optimized by applying personalized POIs to sequential travel patterns. Finally, the experimental results demonstrate the superiority of the approach in terms of Precision, Recall, RMSE, MAP, F-Score, and nDCG compared to previous works based on Flickr and STS datasets.
CAFOB: Context-aware fuzzy-ontology-based tourism recommendation system
2022, Expert Systems with Applications
The ever-increasing volume of information available on tourist attractions in cyberspace has made the tourist decision-making process a crucial task. Therefore, tourism recommendation systems can significantly benefit tourists in terms of comfort and satisfaction. In this paper, a context-aware fuzzy-ontology-based tourism recommendation system is proposed. In the proposed system, we have two new propositions that can be individually used in other tourism recommendation systems: a fuzzy-weighted ontology and a new sentiment/emotion score scheme. In CAFOB, the ontology-based scores of a user’s reviews are then multiplicatively modulated by sentiment/emotion scores to generate the total scores of the reviews’ ontology words, representing the user preferences. Then, the nearby-open- $3^{+}$ -bubbled touristic attractions’ characteristics are extracted based on their past visitors’ reviews. Finally, a recommendation list is produced using a maximum hybrid semantic similarity between the user preferences and attractions’ characteristics. The employment of contextual information, including weather, location, and time makes CAFOB context-aware and improves the accuracy and quality of the recommendations. The F-measure, NDCG, and MRR results show the outperformance of CAFOB against the state-of-the-art tourism recommendation systems for all relevant Top N recommendation options and different geographical spans of interest.

View all citing articles on Scopus

View full text