Abstract
Multi-task learning (MTL), which jointly tackles multiple tasks through information sharing, has been widely applied to many recommendation applications. Recently, current efforts targeted for recommendation focus on learning task relationships based on the Multi-gate Mixture-of-Experts (MMoE) architecture with shared input features (i.e., subtle feature engineering for user-item interaction). Recent evidences suggest the Graph Neural Network (GNN) as a powerful component in characterizing deep interaction context for recommendation, greatly contributing to easing the data sparseness issue in online advertising services. Hence, we make the first attempt to explore the GNN towards multi-task recommendation, by designing Mixture of Graph enhanced Expert Networks (MoGENet). Specifically, we propose a novel multi-channel graph neural network to jointly model high-order information with the user-item bipartite graph as well as derived collaborative similarity graphs for users and items. On the top of the learned deep interaction context, a group of graph enhanced expert networks are introduced for contributing to the multi-task recommendation in a cooperative manner. Experimental results on three real-world datasets show that MoGENet consistently and significantly outperforms state-of-the-art baselines across all target tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
References
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
Fan, W., et al.: Graph neural networks for social recommendation. In: WWW, pp. 417–426 (2019)
Gao, C., et al.: Neural multi-task recommendation from multi-behavior data. In: ICDE, pp. 1554–1557 (2019)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: NIPS, pp. 1024–1034 (2017)
Hsu, W.T., Lin, C.K., Lee, M.Y., Min, K., Tang, J., Sun, M.: A unified model for extractive and abstractive summarization using inconsistency loss. In: ACL, pp. 132–141 (2018)
Kendall, A., Gal, Y.: What uncertainties do we need in bayesian deep learning for computer vision. In: NIPS, pp. 5574–5584 (2017)
Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: CVPR, pp. 7482–7491 (2018)
Kim, S., Ahn, N., Sohn, K.A.: Restoring spatially-heterogeneous distortions using mixture of experts network. In: ACCV (2020)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Li, D., Li, X., Wang, J., Li, P.: Video recommendation with multi-gate mixture of experts soft actor critic. In: SIGIR, pp. 1553–1556 (2020)
Liu, Z., et al.: GeniePath: graph neural networks with adaptive receptive paths. In: AAAI, pp. 4424–4431 (2019)
Ma, J., Zhao, Z., Chen, J., Li, A., Hong, L., Chi, E.H.: SNR: sub-network routing for flexible parameter sharing in multi-task learning. In: AAAI, pp. 216–223 (2019)
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., Chi, E.H.: Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In: SIGKDD, pp. 1930–1939 (2018)
Ma, X., et al.: Entire space multi-task model: an effective approach for estimating post-click conversion rate. In: SIGIR, pp. 1137–1140 (2018)
Misra, I., Shrivastava, A., Gupta, A., Hebert, M.: Cross-stitch networks for multi-task learning. In: CVPR, pp. 3994–4003 (2016)
Niu, X., Li, B., Li, C., Tan, J., Xiao, R., Deng, H.: Heterogeneous graph augmented multi-scenario sharing recommendation with tree-guided expert networks. In: WSDM, pp. 1038–1046 (2021)
Qin, Z., Cheng, Y., Zhao, Z., Chen, Z., Metzler, D., Qin, J.: Multitask mixture of sequential experts for user activity streams. In: SIGKDD, pp. 3083–3091 (2020)
Shen, Q., Tao, W., Zhang, J., Wen, H., Chen, Z., Lu, Q.: SAR-Net: a scenario-aware ranking network for personalized fair recommendation in hundreds of travel scenarios. In: CIKM, pp. 4094–4103 (2021)
Tang, H., Liu, J., Zhao, M., Gong, X.: Progressive layered extraction (PLE): a novel multi-task learning (MTL) model for personalized recommendations. In: Recommender Systems, pp. 269–278 (2020)
Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: ICLR (2018)
Wang, M., Lin, Y., Lin, G., Yang, K., Wu, X.M.: M2GRL: a multi-task multi-view graph representation learning framework for web-scale recommender systems. In: SIGKDD, pp. 2349–2358 (2020)
Wang, X., He, X., Cao, Y., Liu, M., Chua, T.S.: KGAT: knowledge graph attention network for recommendation. In: SIGKDD, pp. 950–958 (2019)
Wang, X., He, X., Wang, M., Feng, F., Chua, T.S.: Neural graph collaborative filtering. In: SIGIR, pp. 165–174 (2019)
Wang, X., et al.: Deep mixture of experts via shallow embedding. In: UAI, pp. 552–562 (2020)
Wen, H., et al.: Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In: SIGIR, pp. 2377–2386 (2020)
Wu, Q., et al.: Dual graph attention networks for deep latent representation of multifaceted social effects in recommender systems. In: WWW, pp. 2091–2102 (2019)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32(1), 4–24 (2020)
Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K.i., Jegelka, S.: Representation learning on graphs with jumping knowledge networks. In: ICML, pp. 5449–5458 (2018)
Yan, L., Li, W.J., Xue, G.R., Han, D.: Coupled group lasso for web-scale CTR prediction in display advertising. In: ICML, pp. 802–810 (2014)
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W.L., Leskovec, J.: Graph convolutional neural networks for web-scale recommender systems. In: SIGKDD, pp. 974–983 (2018)
Zhou, J., et al.: KunPeng: parameter server based distributed learning systems and its applications in Alibaba and ant financial. In: SIGKDD, pp. 1693–1702 (2017)
Zhu, H., et al.: Optimized cost per click in taobao display advertising. In: SIGKDD, pp. 2191–2200 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hu, B. et al. (2022). Mixture of Graph Enhanced Expert Networks for Multi-task Recommendation. In: Khanna, S., Cao, J., Bai, Q., Xu, G. (eds) PRICAI 2022: Trends in Artificial Intelligence. PRICAI 2022. Lecture Notes in Computer Science, vol 13631. Springer, Cham. https://doi.org/10.1007/978-3-031-20868-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-20868-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20867-6
Online ISBN: 978-3-031-20868-3
eBook Packages: Computer ScienceComputer Science (R0)