Time-aware adaptive tweets ranking through deep learning

Highlights

• Time-aware adaptive and personalized learning to rank algorithm for tweets.

• Comparative multilayer perceptron feed-forward neural network to train comparison among tweets.

• Tweet content wikification to semantically categorize the posts by linking tweet text to Wikipedia articles.

• Ranking model evaluation by measuring how many re-tweeted tweets are included in the top-ranked tweet list.

Abstract

Generally, tweets about brands, news and so forth, are mostly delivered to the Twitter user in a reverse chronological order choosing among those twitted by the so-called followed users. Recently, Twitter is facing with information overload by introducing new filtering features, such as “while you are away”, in order to show only a few tweets summarizing the posted ones, and ranking the tweets considering the quality, in addition to timeliness. Trivially enough we state that the strategy to rank the tweets to maximize the user engagement and, why not, augmenting the tweet and re-tweet rates, is not unique. There are several dimensions affecting the ranking, such as time, location, semantic, publisher authority, quality, and so on. We point out that the tweet ranking model should vary according to the user’s context, interests and how those change along the timeline, cyclically, weekly or at specific date-time when the user logs in.

In this work, we introduce a deep learning method attempting to re-adapt the ranking of the tweets by preferring those that are more likely interesting for the user. User’s interests are extracted by mainly considering previous user re-tweets, replies and also the time when they occurred.

We evaluate a ranking model by measuring how many tweets that will be re-tweeted in the near future were included in the top-ranked tweet list. The results of the proposed ranking model revealed good performances overcoming the methods that consider only the reverse-chronological order or user’s interest score. In addition, we pointed out that in our dataset the most impacting features on the performance of proposed ranking model are: publisher authority, tweet content measures, and time-awareness.

Introduction

Context. Nowadays, we are assisting to a social data explosion. Facebook and Twitter are very popular communication platforms so that they are playing an important role in cultural, social, and political events. Social networking is a core part of the online experience [1]. Nevertheless, tons of tweets are daily posted, thousands of them happen every second and people are overwhelmed by the incoming information. Posts are authored by anyone from wherever around the world, and so, Twitter and Facebook have become attractive for spammers [2] compromising also the worth of the information source. The provisioning of the valuable tweet at the right time requires facing with information overload problem introducing filtering and ranking methods considering the user’s interests, the activity that he/she is performing, the quality and relevance of the content, and so on.

In general, tweets are mostly delivered to the user in a reverse chronological order by considering ones that are published by the followed users. Recently, Twitter is facing with information overload proposing a new version of its timeline that ranks tweets by considering also the quality¹ and the relevance² in addition to the timeliness as stated in the official blog. New features are available on Twitter to show you relevant tweets list “in case you missed it”, to give you a subset of tweets based on their popularity, and how you interact with the tweet publisher. From the research point of view, some works are dealing with information overload on Twitter by defining tweets recommendation algorithms [3], [4], personalized ranking [5], filtering, and summarization [6], [7], [8], [9], [10], [11], customized according to several criteria.

In this paper, we emphasize that there is no unique and optimal criteria to rank the tweets maximizing users engagement and, why not, augmenting tweet and re-tweet rates creating more live commentary and conversations. There are several dimensions affecting the ranking, such as time, location, semantic, interestingness, publisher authority, and so on. Their impact on the ranking algorithm changes according to the user’s context, the day of the week, the period of the year, and so forth. Indeed, the preferences change not only for different users but also for the same user according to the context in which user is when he/she comes to social media (i.e., Twitter). In fact, the same user may prefer to be updated by reading breaking news coming from social media when he/she is having a break, or when he/she is watching TV. Some users may prefer tweets related to the sporting event, but only in the hours following football matches. Unlikely, they may prefer to know that something important is happening in the nearby whenever it happens, even if they are searching something else.

Problem definition. Formally, given a time-stamped finite tweet stream $TW=〈{\mathit{tw}}_1,{\mathit{tw}}_2,\dots ,{\mathit{tw}}_n〉$, with some related information about publisher authority and user $u$, the task goal is to identify a function to rank the tweets in $TW$ from those that are more relevant for $u$ considering his/her own history (tweets, re-tweets, follows, etc.). The resulting ranking model should be adaptive, personalized and time-aware considering that the user’s interests may change along the timeline and depend on the current context when the user logs in Twitter.

Proposed solution. To achieve the aforementioned goal we define a learning to rank algorithm to sort a set of tweets (sketched in Fig. 1). Actually, learning to rank is a research area intensively investigated and many algorithms have been proposed, and consequently used in several fields including information retrieval tasks, focused search engines, and more recently, they are being adopted also for tweets ranking or recommendation [12]. In literature, we can distinguish the following main supervised approaches [13]: pointwise, pairwise or listwise. The main limitation of these algorithms is that supervised learning powers on the availability of user’s feedbacks about the ranking of items, which are not easy to collect. In this sense, the most promising and natural approach is the pairwise that requires users’ feedback only to determine what are the users’ preferences with respect to pairs of items (i.e., tweets) instead of complete rank lists of them. We adopted a pairwise approach in which user’s preferences are implicitly expressed by re-tweets and replies that we interpret as pairwise comparisons with respect to other tweets, for example, those shown in reverse chronological order, that have not been mentioned by the user. Among others, the pairwise algorithms, such as RankNet [14] and its deep version [15], has revealed good performances in ranking web pages to improve web search experience. In this work, we adopt an algorithm inspired to SortNet [16], a ranking algorithm based on deep neural network to rank tweets including several features to represent user, content, publisher, and so on. The aim is to learn a function to evaluate the choices between two tweets, i.e., ${\mathit{tw}}_i$ and ${\mathit{tw}}_j$. Given a pair of tweets ${\mathit{tw}}_i,{\mathit{tw}}_j\in TW$, the aim is to learn a preference function $P:TW\times TW$  $\to \{>,<\}$ which evaluates the user’s interests with respect to the pair of tweets, i.e. ${\mathit{tw}}_i>{\mathit{tw}}_j$, if ${\mathit{tw}}_i$ should be preferred to ${\mathit{tw}}_j$, and ${\mathit{tw}}_i<{\mathit{tw}}_j$, vice versa.

Contributions. Unlike other application domains, for instance, web search where learning to rank algorithms have already been widely applied, the strong dynamic nature of the microblogging stresses the importance of the model re-adaptation. This work introduces a deep learning method for tweet ranking capable to re-adapt itself along the timeline and considering different tweet and user’s interests. Time-awareness is implemented by using datetime of the tweets during the ranking model training. More precisely, the main contributions of the proposed research are:

• Definition of a learning to rank algorithm for tweets; in particular, we use a pairwise algorithm assuming that each re-tweet and/or reply represents a user’s feedback expressing preference for that topic, the publisher’s authority, and so forth;

• Integration of datetime of the tweet, re-tweet, or reply during the training phase in order to provide different ranking results considering the moment when user logs into the Twitter; in fact, the occurrence of user’s interest may recur cyclically in a given time slot (e.g., weekend, evening, etc.);

• Implementation of a continuous learning giving new sample items as input tuples for training the ranking model at each time the user expresses his/her preference replying or re-tweeting something;

• Adoption of tweet content wikification to semantically categorize the posts by linking tweet text to Wikipedia articles; this practice enables us to use corresponding Wikipedia entities to characterize the user’s topics of interest.

Experimental results. Starting from the collected tweet stream, we adopt our framework to perform a personalized tweet rank simulating different accessing time slots, and we evaluate its precision by applying Mean Average Precision (MAP) and Normalized Discount Cumulative Gain (NDCG) metrics. Performances have also been evaluated by omitting some significant features (i.e., tweet publisher’s authority, social relation between tweet author and user, and time-awareness) in order to estimate their impact on the method performance. We evaluate the tweets ranking improvement counting how many top-ranked tweets will be re-tweeted/replied in the near future with respect to the ignored ones. The experimental results reveal promising performance and confirm the unsuitability of a simply reverse chronological order. In addition, we point out that time features play an important role because ranking preferences improves by including time features in the learning phase and considering the time slot when the users log in Twitter.

Outlines. The paper is structured as follows: Section 2 describes some related works; Section 3 discusses the deep neural network architecture used for the ranking model; Section 4 details the features selected to train the model and illustrates how the tweets components are modeled; then, in Section 5 the evaluation results are discussed; finally, the conclusion and future works close the paper.