A distributed group recommendation system based on extreme gradient boosting and big data technologies

Abstract

Personalized recommendation systems have emerged as useful tools for recommending the appropriate items to individual users. However, in such situations, some items tend to be consumed by groups of users, such as tourist attractions or television programs. With this purpose in mind, Group Recommender Systems (GRSs) are tailored to help groups of users to find suitable items according to their preferences and needs. In general, these systems often confront the sparsity problem, which negatively affects their efficiency. With the increase in the number of users, items, groups, and ratings in the system. Data becomes too big to be processed efficiently by traditional systems. Thus, there is an increasing need for distributed recommendation approaches able to manage the issues related to Big Data and sparsity problem. In this paper, we propose a distributed group recommendation system, which is designed based on Apache Spark to handle large-scale data. It integrates a novel proposed recommendation method, a dimension reduction technique, with supervised and unsupervised learning for dealing efficiently with the curse of dimensionality problem, detecting the groups of users, and improving the prediction quality. Experimental results on three real-world data sets show that our proposal is significantly better than other competitors.

Appendix: Extreme gradient boosting

XGBoost (Extreme gradient boosting) represents a powerful machine learning technique. It is a tree ensemble, which consists of a set of classification and regression trees (CARTs).

Mathematically, the XGBoost model is described as follows:

$$ \hat y_{i}=\sum\limits_{k=1}^{K} f_{k}(x_{i}), f_{k} \in \digamma $$

(35)

where K is the number of trees, f is a function in the functional space г, and г is the set of all possible CARTs.

The objective function to optimize is written as follows:

$$ obj=\sum\limits_{i=1}^{n} l(y_{i},\hat y_{i})+ \sum\limits_{{k=1}}^{K} {\Omega}(f_{k}) $$

(36)

where l is the loss function that measures the difference between the predicted value $\hat y_i$ and the target value y_i. While Ω represents the regularization term.

With regard to the training task, XGBoost is trained in an additive manner. The prediction value $\hat y^{(t)}_i$ of the i_th instance at the t_th iteration is given by:

$$ \hat y^{(t)}_{i}=\hat y_{i}^{(t-1)}+f_{t}(x_{i}) $$

(37)

Therefore, the objective function at the t_th iteration is defined as follows:

$$ obj^{(t)}=\sum\limits_{i=1}^{n} l(y_{i},\hat y^{(t-1)}_{i}+f_{t}(x_{i}))+ {\Omega}(f_{t}) $$

(38)

In order to approximate the objective function, Taylor expansion is employed. The formulation can be written as follows:

$$ obj^{(t)}=\sum\limits_{i=1}^{n} [ g_{i} f_{t}(x_{i}) + \frac{1}{2} h_{i} {f^{2}_{t}}(x_{i})]+ {\Omega}(f_{t}) $$

(39)

where g_i and h_i are defined as:

$$ \begin{array}{@{}rcl@{}} g_{i}&=&\partial_{\hat y^{(t-1)}_{i}} l(y_{i},\hat y^{(t-1)}_{i})\\ h_{i}&=&\partial^{2}_{\hat y^{(t-1)}_{i}} l(y_{i},\hat y^{(t-1)}_{i}) \end{array} $$

(40)

Let I_j be a set of instances assigned to the leaf j, after defining the regularization term Ω, (39) can be rewritten as follows:

$$ \begin{array}{@{}rcl@{}} obj^{(t)}&=&\sum\limits_{{i=1}}^{n} [ g_{i} f_{t}(x_{i}) + \frac{1}{2} h_{i} {f^{2}_{t}}(x_{i})]+ \gamma T + \frac{1}{2} \lambda \sum\limits_{{j=1}}^{T} {w^{2}_{j}}\\ &=&\sum\limits_{{j=1}}^{T}[(\sum\limits_{{i \in I_{j}}} g_{i}) w_{j} + \frac{1}{2}(\sum\limits_{{i \in I_{j}}} h_{i}+\lambda) {w_{j}^{2}}] + \gamma T \end{array} $$

(41)

where T is the number of leaves in the tree, λ and γ are the regularization parameters, and w_j is the weight of leaf j.

Given a fixed tree structure, the optimal weight wj∗ and the objective are computed as follows:

$$ \begin{array}{@{}rcl@{}} w_{j}^{*}=-\frac{{\sum}_{{i \in I_{j}}} g_{i}}{{\sum}_{{i \in I_{j}}} h_{i}+\lambda}&&\\ obj^{*}=-\frac{1}{2} \sum\limits_{{j=1}}^{T} \frac{{\sum}_{{i \in I_{j}}} g_{i}}{{\sum}_{{i \in I_{j}}} h_{i}+\lambda}+\gamma T&& \end{array} $$

(42)

The smaller objective value indicates the better structure.

Regarding the construction of trees, XGBoost adopts a greedy algorithm, which starts from tree with depth 0, and iteratively splits the leaves of the tree.

The gain after adding a split is measured as follows:

$$ \begin{array}{@{}rcl@{}} Gain&=&\frac{1}{2}[\frac{{\sum}_{{i \in I_{R}}} {g_{i}^{2}}}{{\sum}_{{i \in I_{R}}} h_{i}+\lambda}+\frac{{\sum}_{{i \in I_{L}}} {g_{i}^{2}}}{{\sum}_{{i \in I_{L}}} h_{i}+\lambda}\\ &&-\frac{{\sum}_{{i \in I}} {g_{i}^{2}}}{{\sum}_{{i \in I}} h_{i}+\lambda}]-\gamma \end{array} $$

(43)

where I_R and I_L denote the instance sets of right and left nodes after the split, respectively [35].