Knowledge extraction from crowdsourced data for the enrichment of road networks

Abstract

In current navigation systems quantitative metrics such as distance, time and energy are used to determine optimal paths. Yet, a “best path”, as judged by users, might take qualitative features into account, for instance the scenery or the touristic attractiveness of a path. Machines are unable to quantify such “soft” properties. Crowdsourced data provides us with a means to record user choices and opinions. In this work, we survey heterogeneous sources of spatial, spatio-temporal and textual crowdsourced data as a proxy for qualitative information of users in movement. We (i) explore the process of extracting qualitative information from uncertain crowdsourced data sets employing different techniques, (ii) investigate the enrichment of road networks with the extracted information by adjusting its properties and by building a meta-network, and (iii) show how to use the enriched networks for routing purposes. An extensive experimental evaluation of our proposed methods on real-world data sets shows that qualitative properties as captured by crowdsourced data can indeed be used to improve the quality of routing suggestions while not sacrificing their quantitative aspects.

Appendix: Mining Spatial Closeness from Textual Data

As presented in [3], we rely on a probabilistic model to counter the ambiguity inherent in textual data. This is done in three steps. First, we create feature vectors for each occurrence of a particular spatial closeness relation. Second, employing the feature vectors we train a Gaussian Mixture Model (GMM) for each kind of closeness relation. Third, we use the derived GMMs to infer a closeness score which can be used to enrich the underlying road network.

For each occurrence of a closeness relation (e.g., “next to”) mined from the text (e.g., the triplet (P _i ,“next to”,P _j )), a spatial feature vector v _{i j} = (r, ϕ) is created. v _{i j} describes the Euclidean distance between P _i and P _j and the orientation as the counterclockwise rotation of the x-axis, centered at P _j , to P _i . Thus, for any triplet extracted from the corpus (sufficiently often) we obtain a two-dimensional feature vector. We group these vectors by the relation they represent, obtaining a set of vectors $V_{REL^{k}}$ for each of the closeness relations R E L ^k. For each relation, we propose to train a probabilistic model. Following the argumentation in [43] where similar relations were examined, we propose employing GMMs, a well-proven and extensively studied method for many supervised and unsupervised learning problems [56].

A GMM is a weighted sum of a fixed number M of Gaussian component densities

$$\mathbb{P}_{\theta}(v) = \sum\limits_{i = 1}^{M} w_{i} g(v;\mu_{i},{\Sigma}_{i}) $$

where v is an l-dimensional vector, w _i are the mixture weights (summing to 1) and

$$g(v;\mu_{i}, {\Sigma}_{i}) = \frac{1}{\left( (2\pi)^{l} \det({\Sigma}_{i})\right)^{1/2}} \exp \left( -\frac{1}{2}(v-\mu_{i})^{T}{\Sigma}_{i}(v-\mu_{i})\right) $$

is an l-variate Gaussian density function with mean vector $\mu _{i} \in \mathbb {R}^{l}$ and covariance matrix ${\Sigma }_{i} \in \mathbb {R}^{l\times l}$. The model is fully characterized by the weights, mean vectors and covariance matrices, collectively represented in 𝜃 = {w _i , μ _i ,Σ _i },i = 1,…,M. In our case, l = 2, the dimensionality of the spatial feature vectors v. For the parameter estimation of each Gaussian component, Expectation Maximization (EM) [57] is the state-of-the-art technique. It updates the parameters of the components iteratively w.r.t. a given (feature) vector set until a convergence threshold is reached.

Employing EM, we obtain a set of probabilistic models $\mathbb {P}(\cdot \mid \theta ^{k})$, one for each closeness relation R E L ^k ∈{R E L ¹,…,R E L ^m} in the set of all closeness relations. For a pair of POIs P _i and P _j with spatial feature vector v _{i j} , $\mathbb {P}_{\theta ^{k}}(v_{ij})$ is the probability that P _i and P _j stand in spatial relation R E L ^k to each other. Based on this information, we now derive a closeness score for pairs of POIs by Bayesian inference.

For two distinct POIs, let R E L _{i j} denote the set of all closeness relations existing between P _i and P _j . Note that R E L ^k denotes an abstract relation, while R E L _{i j} denotes the set of occurrences of relations between a pair of POIs. Furthermore, let v _{i j} denote the spatial feature vector of P _i and P _j . In order to determine a numeric score describing the closeness of the POIs, we estimate the posterior probability of all closeness relations R E L ^k and accumulate them. The posterior probability of relation R E L ^k is given by

$$\mathbb{P}(REL^{k} \mid v_{ij}) = \frac{\mathbb{P}(v_{ij} \mid \theta^{k})\ \mathbb{P}(\theta^{k})}{{\sum}_{l=1}^{m} \mathbb{P}(v_{ij} \mid \theta^{l})\ \mathbb{P}(\theta^{l})} $$

where $\mathbb {P}(\theta ^{k}) = \mathbb {P}(REL^{k})$ denotes the prior probability of relation R E L ^k given by the trained model represented by 𝜃 ^k.

In a traditional classification problem the task would be to choose the closeness relation with the highest posterior and to assign the pair of POIs to this class. We, in contrast, consider each posterior probability $\mathbb {P}(REL^{k} \mid v_{ij})$ as a measure of confidence of the existence of R E L ^k between P _i and P _j . Stressing that all considered relations represent spatial closeness, we combine all posteriors into one measure which we call closeness score c s _{i j} of the pair of POIs P _i and P _j :

$$cs_{ij} = \frac{1}{m} \sum\limits_{k=1}^{m} \frac{\mathbb{P}(REL^{k} \mid v_{ij})}{\max\{\mathbb{P}(REL^{k} \mid v_{ij}) \mid \forall \ i\neq j\}} $$

This closeness score can be used to enrich the underlying road network.