Linking place records using multi-view encoders

Abstract

Extracting information about Web entities has become commonplace in the academy and industry alike. In particular, data about places distinguish themselves as rich sources of geolocalized information and spatial context, serving as a foundation for a series of applications. This data, however, is inherently noisy and has several issues, such as data replication. In this work, we aim to detect replicated places using a deep-learning model, named PlacERN, that relies on multi-view encoders. These encoders learn different representations from distinct information levels of a place, using intermediate mappings and non-linearities. They are then compared in order to predict whether a place pair is a duplicate or not. We then indicate how this model can be used to solve the place linkage problem in an end-to-end fashion by fitting it into a pipeline. PlacERN is evaluated on top of two distinct datasets, containing missing values and high class imbalance. The results show that: (1) PlacERN is effective in performing place deduplication, even on such challenging datasets; and (2) it outperforms previous place deduplication approaches, and competitive algorithms, namely Random Forest and LightGBM using pairwise features, on both datasets in regards to different metrics (F score, Gini Coefficient and Area Under Precision-recall Curve).

Appendix: Hyperparameters

To improve reproducibility of our results, we describe in this appendix the optimization process utilized for the previous approaches, for our supervised models, and for PlacERN. We also provide the best hyperparameter values achieved during optimization. The value ranges used as input to the optimization algorithms may be found in [10].

1.1 Previous approaches

We utilize Bayesian optimizations from Optuna [1] to tune the parameters of PE [56], and a grid search for the model of Dalvi et al. [13], both using validation data. The EM algorithm from [13] is executed for 10 iterations in $Pairs_{BR}$ and 5 iterations in $Pairs_{US}$, while PE [56] runs for 100 Optuna trials in $Pairs_{BR}$ and 50 trials in $Pairs_{US}$, using a median pruner after 5 warm-up trials and 3 warm-up steps. Geohashes are utilized as a means for the creation of tiles in both methods.

The optimal values obtained for the hyperparameters of [13] in the $Pairs_{BR}$ dataset are: $\lambda = 0.0$, $\text {Geohash}\, \text {precision} = 6$, $\alpha = 0.3$, and $t_c = 0.5$. For the $Pairs_{US}$: $\lambda = 0.0$, Geohash precision $= 6$, $\alpha = 0.9$, and $t_c = 0.5$.

Meanwhile, the embedding smoothing process from PE runs for 10 full epochs and uses 100,000 smoothing random walks with a fixed length of 10, a minimum frequency of 1, and a half window size of 5. The training batch size is fixed as 512 for $Pairs_{BR}$ and 1024 for $Pairs_{US}$, and the MLP uses 3 feedforward layers. The best values for the tuned hyperparameters of PE in the $Pairs_{BR}$ dataset are: $\alpha = 0.4$, smoothing negative sampling ratio $= 20$, $\text {neurons} = 512, 128, 128$, $t_c = 0.75$. For the $Pairs_{US}$ set: $\alpha = 0.5$, smoothing negative sampling ratio $= 5$, $\text {neurons} = 512, 128, 128$, $t_c = 0.75$.

1.2 Supervised baseline models

Both models (PRF and PLGBM) are tuned by Bayesian optimization, using Optuna for 100 trials in the scope of the $Pairs_{BR}$ and $Pairs_{US}$ validation data sets. The description of the hyperparameters for each model follows the parameter names from their respective libraries, with any value not shown assuming the default value.

PRF utilized the class_weight parameter set to balanced and the oob_score parameter set to True in both sets. The optimal tuned hyperparameter values for PRF in the scope of $Pairs_{BR}$ are: n_estimators $= 110$, max_features $= log2$, max_leaf_nodes $= 150$, min_samples_split $= 3$, $t_c = 0.9$. In the $Pairs_{US}$ set: n_estimators $= 150$, max_features $= sqrt$, max_leaf_nodes $= 150$, min_sample_splits $= 5$, $t_c = 0.9$.

The PLGBM model utilizes 10 early stopping rounds for 100 iterations in each trial, using a fixed class_weight value of balanced. The optimal values for its hyperparameters in $Pairs_{BR}$ are: lambda_l1 $= 1.247 \cdot 10^{-8}$, lambda_l2 $= 0.659$, num_leaves $= 95$, feature_fraction $= 0.5$, bagging_fraction $= 1.0$, bagging_freq $= 0$, min_child_samples $= 5$, $t_c = 0.5$. In the $Places_{US}$ set, the values are: lambda_l1 $= 1.132 \cdot 10^{-8}$, lambda_l2 $= 0.247$, num_leaves $= 256$, feature_fraction $= 0.62$, bagging_fraction $= 1.0$, bagging_freq $= 0$, min_child_samples $= 20$, $t_c = 0.5$

1.3 PlacERN

PlacERN is implemented with $L = 3$ feed-forward network layers, the first of them having $H_0 = 256$ neurons. Regarding the sequence lengths noted in Sect. 3, we use the 90th percentile of lengths for each field to extract $S^w = 5$, and $S^{ct} = 3$ for both sets, $S^{ch}_n = 42, S^{ch}_a = 32$ for $Pairs_{BR}$, and $S^{ch}_n = 26, S^{ch}_a = 23$ for $Pairs_{US}$. We use $B = 100$ distance buckets in the geographical encoder, and $m = 100$ dimensions for the embedding layers.

To use the pre-trained FastText embeddings in our model, their dimensionality is reduced to 100 beforehand by means of a Principal Component Analysis [39] dimensionality reduction script.⁸ In order to improve reproducibility, we also fix the random seeds as 6810818.

The model is tuned by Bayesian optimization from Optuna for 50 trials on top of the $Pairs_{BR}$ validation data set and 20 trials in the $Pairs_{US}$ one, with a median pruner after 3 warm-up trials and 1 warm-up step. An additional early stopping callback with a patience of 2 epochs and a minimum change of $10^{-3}$ is also added as insurance against degenerate cases not detected by the median pruner. A batch size of 64 for $Pairs_{BR}$ and 1024 for $Pairs_{US}$ is utilized during training. The best tuned hyperparameter values for PlacERN and its ablated versions in both datasets are described in Table 5.

Table 5 Hyperparameters of PlacERN and its ablations, with the best values obtained in each data set