End-to-End Optimization of Quantization-Based Structure Learning and Interventional Next-Item Recommendation

Abstract

With the development of deep learning, more and more related techniques are used in recommender system, making it more effective and reliable. However, due to the various distribution of real-time data, those deep-learning-based methods can merely learn the correlation between data rather than the actual causal effect, decreasing the performance of recommenders when a distribution shift occurs. Therefore, causal structure learning, which has been proposed to search for causal relationships between variables, is applied in recommender systems. However, existing methods assume that the recommender system is a non-interventional environment, making the causal graph learned not entirely correct. In this paper, we propose to decouple the recommender module and the causal module to consider the intervention of recommender system when building a causal graph. We utilize vector quantization to learn a cluster-level graph rather than an item-level graph to guarantee an acceptable training time. With an adjustable number of clusters, our model can adapt datasets of any size and be trained within a certain period. We conduct extensive experiments on both real-world and synthetic OOD datasets to demonstrate that our model is more effective than other state-of-the-art sequential recommenders.

A Appendix

1.1 A.1 Experimental Setup

Datasets. We conduct experiments based on the following benchmark datasets, of which the statistics are shown in Table 4:

• Movielens-1M⁴ contains anonymous ratings of movies made by MovieLens users who joined MovieLens in 2000. Besides, it also includes the metadata of each movie, such as the categories it belongs to.

• Amazon-Music⁵ is an e-commerce dataset collected from Amazon.com, which also includes the category information of a product.

• Netflix⁶ is another movie rating file constructed to support participants in the Netflix Prize. In this experiment, We use the preprocessed dataset provided in NATR [5].

In order to keep dataset quality, We discard items with fewer than 20 related appearances, while for user filtering, we adopt 5-core settings for Amazon and Netflix and 20-core settings for Movielens due to their different sparsity.

For the three datasets above, we sort the user-item interactions in chronological order and divide the training and testing sets according to the ratio of 8:2. In our experiments, the validation set is consistent with the test set. For each interaction, We regard those feedback with positive ratings as positive.

OOD Datasets. Following the data splitting paradigm of DESMIL [15], we build the OOD datasets by adjusting the distribution of user groups in the training set and test set. We randomly select a user and calculate the Jaccard similarity⁷ between other users and the selected user. Then we choose 80% users with the smallest similarity as the training set and use the whole dataset for testing, where the training set and testing set contain different user groups.

Table 4. Statistics of the datasets.

Baselines. For the implementation of our method, we choose YoutubeDNN [3] as the base model, which is a classic deep learning recommendation system. Since our model is a sequential recommendation model, we ignore the graph recommendation model and focus on the sequential recommendation model. As for those collaborative filtering (CF) recommendation models, we adjust them to apply the historical sequence to the model. The compared models are shown as the following:

• YoutubeDNN [3]. As one of the industry’s most commonly used recommender systems, it represents each user with his historical interaction data and combines them to get the final predictive score.

• GRU4Rec [10]. GRU4Rec is a representative recommendation model in session based recommendation which adopts recurrent neural networks to train the relationship between target items and item sequences.

• NeuMF [8]. NeuMF combines traditional matrix decomposition and multi-layer perceptron to extract both low- and high-dimensional features simultaneously.

• SASRec [13]. SASRec applies self-attention mechanisms in sequential recommendation to help learn more valuable information.

• Caser [17]. Caser is a state-of-the-art model that utilizes vertical and horizontal convolutional networks to extract short-term preferences from user interaction sequences.

• AttRec [29]. Similar to SASRec, AttRec makes use of self-attention mechanisms to capture long-term preference. Moreover, it uses metric learning to model the consequences of short-term interests versus long-term preferences.

• PAUP [33]. PAUP proposes a down-sampling convolution module and an unsymmetrical positional encoding strategy to effectively and efficiently capture both short- and long-term patterns.

• FMLP-Rec [32]. FMLP-Rec makes use of filtering algorithms in signal processing instead of the traditional transformer to relieve the noise in the original dataset.

Evaluation Metrics. The three evaluation metrics we used are the same as the framework in ComiRec [1]. In recommendation system, accuracy, that is, whether what the user wants to see is correctly recommended to the user, is crucial. Thus we use Recall and Hit Rate to evaluate the accuracy of a model. Another metric used in our experiment is NDCG(Normalized Discounted Cumulative Gain), which pays more attention to the ratings in the recommendation list of items desired by users and is more convincing than recall. The higher the scores of the three indicators, the better the effect of the recommendation model.

Implementation Details. For all the models mentioned above, we use Adam [14] as the optimizer and 0.001 as the learning rate. All of them are implemented with an item embedding with size $d=64$. The number of horizontal and vertical convolution filters are 4 and 16 in Caser, respectively. For each sequence in Caser, AttRec, and SASRec, we randomly sample 10 items not interacted with by the user as negative samples. While for GRU4Rec, we follow the method used in the original paper, namely, using other items in the same batch except for the target item as negative samples. For the NeuMF baseline, we create an embedding for each user with the history of interactions over time to adapt sequential recommendations.