SocialLink: exploiting graph embeddings to link DBpedia entities to Twitter profiles

Abstract

SocialLink is a project designed to match social media profiles on Twitter to corresponding entities in DBpedia. Built to bridge the vibrant Twitter social media world and the Linked Open Data cloud, SocialLink enables knowledge transfer between the two, both assisting Semantic Web practitioners in better harvesting the vast amounts of information available on Twitter and allowing leveraging of DBpedia data for social media analysis tasks. In this paper, we further extend the original SocialLink approach by exploiting graph-based features based on both DBpedia and Twitter, represented as graph embeddings learned from vast amounts of unlabeled data. The introduction of such new features required to redesign our deep neural network-based candidate selection algorithm and, as a result, we experimentally demonstrate a significant improvement of the performances of SocialLink.

Appendix A: On the choice of word embeddings

Pre-trained word representations, or embeddings, have become a staple technique for modeling textual data in a convenient low-dimensional form. Such word representations are typically allocated in the resulting vector space according to the distributional semantics hypothesis, i.e., the words that appear in similar contexts tend to have similar meanings and will be placed close to each other. Pre-trained word representations allow the usage of large unsupervised corpora to model the target languages efficiently. Over the last 5 years, since the introduction of the word2vec algorithm, many new approaches were proposed to improve different aspects of such representations yielding better performance than the original word2vec in many tasks. Before word2vec, methods, such as LSA, HAL and autoencoders were also widely used. However, to the best of our knowledge at the time of writing, there is still no consensus in the community about whether any of the proposed approaches is clearly superior to others and should be used by default to represent text. This conclusion is consistent with the famous “no free lunch” concept, meaning that for each task the choice of particular word embeddings should be justified and supported with experiments.

In this appendix, we measure the impact of different pre-trained word representations on our task. In our original paper, as mentioned in Sect. 3.1, we chose the latent semantic analysis (LSA) approach to represent text. The choice was mainly driven by our confidence in our LSA-based model that was used in DBpedia-related tasks before. In this paper, in order to have a precise evaluation of the new feature sets, we have opted to use the same model. However, we believe that an additional set of experiments to measure the impact of different embeddings could serve as an extra data point for the community, not to mention potentially improve the performance of our approach.

1.1 A.1 Experimental setting

We compare the previously chosen LSA model to two recent embedding types: GloVe [29] and fastText [3]. To this end, we modify the computation of the “Description” scalars in our original base feature set (see Table 3). Each scalar is computed as follows. Each user text and entity text is converted into the sparse vector $\mathbf {x}_{\text {sparse}} \in {\mathbb {R}}^v$, where v is the size of the vocabulary for the given language model. Each vector contains tf-idf scores for each token t present in a text:

$$\begin{aligned} x_t&= \text {tf}(t) \cdot \text {idf}(t,D) \\&= \log (1+\text {freq}_{t}) \cdot \log \left( 1+ \frac{ |D| }{ 1 + |\{d \in D : t \in d\}| }\right) \end{aligned}$$

where D is a corpus on which a chosen language model was trained. As can be seen, the computation of such vector requires IDF statistics from the corpus. Here we use precomputed vectors provided by the authors of the respective approaches which do not include such data along with the embeddings. Therefore, we use the same IDF scores acquired from Wikipedia for all models. Each approach is represented by the embedding matrix $M \in {\mathbb {R}}^{v \times d}$, where d is the embedding size. Then the dense text vector is acquired: $\mathbf {x}_{\text {dense}} = \mathbf {x}_{\text {sparse}}^T \cdot M$. Finally, the Description scalars are computed as a cosine similarity between the user text $\mathbf {u}_{\text {dense}}$ and the entity text $\mathbf {e}_{\text {dense}}$. In short, in order to test other representation approaches we substitute the embedding matrix $M_{\text {lsa}}$ we employed originally with $M_{\texttt {fastText}}$ and $M_{\text {GloVe}}$.

We test four different models and measure their impact on the performance of the base_kb_sg_tl approach described in the paper:

• 1. LSA ($v = 972,001; d = 100$). The same LSA-based approach we used throughout the paper. The model is derived from Wikipedia and is described in [1].

• 2. GloVe ($v = 1,917,494; d = 300$). The model is trained on 42B token Common Crawl corpus and provided by the authors on their website.¹⁷ This approach is also described in Section 4.1 and used to populate the RDF embeddings we used.

• 3. fastText ($v = 2,519,370; d = 300$). The word2vec-based model exploiting subword information. The model was provided¹⁸ by the authors and is trained on Wikipedia.

• 4. ALL. Description scalars produced by each model used together. Effectively an ensemble of embeddings.

Additionally, we slightly modify our data acquisition phase. During this phase, we would typically gather the textual content for each candidate from the stream of tweets. As a consequence, we would have a lot of textual content for more popular users and much less (as little as a description field in the profile) for others. This makes it so the “Description” features get a strong implicit notion of the user popularity instead of capturing only the textual similarity. To alleviate this bias, we freshly captured the most recent 200 tweets for each candidate for each entity in our gold standard using Twitter ReST API.

Fig. 8

P/R curves of four models using the base_kb_sg_tl model

All approaches are evaluated using stratified fivefold cross-validation, additionally computing 95% confidence intervals and performing statistical significance test using the approximate randomization test (see Sect. 6.1).

1.2 A.2 Experimental results

Evaluation results for the four models are provided in Table 7, while Fig. 8 provides the precision/recall curves. As can be seen, the difference between the four models is minimal. However, the ALL model does provide statistically significant improvement over the LSA model we employed originally, and similarly over fastText and GloVe.

The absence of significant differences between separate models shows that our pipeline is not particularly sensitive toward the choice of the particular word embeddings. In future, the model can be modified to include textual representations in a similar way we have incorporated the graphical ones, to give more opportunity for the neural network to utilize textual data efficiently.