skip to main content
10.1145/3511808.3557140acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Multi-Faceted Hierarchical Multi-Task Learning for Recommender Systems

Published: 17 October 2022 Publication History

Abstract

There have been many studies on improving the efficiency of shared learning in Multi-Task Learning (MTL). Previous works focused on the "micro" sharing perspective for a small number of tasks, while in Recommender Systems (RS) and many other AI applications, we often need to model a large number of tasks. For example, when using MTL to model various user behaviors in RS, if we differentiate new users and new items from old ones, the number of tasks will increase exponentially with multidimensional relations. This work proposes a Multi-Faceted Hierarchical MTL model (MFH) that exploits the multidimensional task relations in large scale MTLs with a nested hierarchical tree structure. MFH maximizes the shared learning through multi-facets of sharing and improves the performance with heterogeneous task tower design. For the first time, MFH addresses the "macro" perspective of shared learning and defines a "switcher" structure to conceptualize the structures of macro shared learning. We evaluate MFH and SOTA models in a large industry video platform of 10 billion samples and hundreds of millions of monthly active users. Results show that MFH outperforms SOTA MTL models significantly in both offline and online evaluations across all user groups, especially remarkable for new users with an online increase of 9.1% in app time per user and 1.85% in next-day retention rate. MFH currently has been deployed in WeSee, Tencent News, QQ Little World and Tencent Video, several products of Tencent. MFH is especially beneficial to the cold-start problems in RS where new users and new items often suffer from a "local overfitting" phenomenon that we first formalize in this paper.

Supplementary Material

MP4 File (CIKM22-jim-app230.mp4)
CIKM22 paper - MFH:Multi-Faceted Hierarchical MTL presentation recorded video

References

[1]
Charu C Aggarwal et al. 2016. Recommender Systems. Vol. 1. Springer.
[2]
Trapit Bansal, David Belanger, and Andrew McCallum. 2016. Ask the GRU: Multi-task Learning for Deep Text Recommendations. In Proceedings of the 10th RecSys. 107--114.
[3]
Rich Caruana. 1997. Multitask Learning. Machine learning, Vol. 28, 1 (1997), 41--75.
[4]
Xiaokai Chen, Xiaoguang Gu, and Libo Fu. 2021. Boosting Share Routing for Multi-task Learning. In Companion Proceedings of the Web Conference 2021.
[5]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & Deep Learning for Recommender Systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 7--10.
[6]
Ronan Collobert and Jason Weston. 2008. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. In Proceedings of the 25th International Conference on Machine Learning. 160--167.
[7]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for Youtube Recommendations. In Proceedings of the 10th ACM RecSys. 191--198.
[8]
James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, et al. 2010. The YouTube Video Recommendation System. In Proceedings of the 4th ACM RecSys. 293--296.
[9]
Ke Ding, Xin Dong, Yong He, Lei Cheng, Chilin Fu, Zhaoxin Huan, Hai Li, Tan Yan, Liang Zhang, Xiaolu Zhang, et al. 2021. MSSM: A Multiple-level Sparse Sharing Model for Efficient Multi-Task Learning. In Proceedings of the 44th ACM SIGIR. 2237--2241.
[10]
Jianping Fan, Tianyi Zhao, Zhenzhong Kuang, Yu Zheng, Ji Zhang, Jun Yu, and Jinye Peng. 2017. HD-MTL: Hierarchical Deep Multi-task Learning for Large-scale Visual Recognition. IEEE Transactions on Image Processing, Vol. 26, 4 (2017), 1923--1938.
[11]
Alexander Felfernig, Ludovico Boratto, Martin Stettinger, and Marko Tkalvc ivc. 2018. Group Recommender Systems: An Introduction. Springer.
[12]
Yulong Gu, Zhuoye Ding, Shuaiqiang Wang, Lixin Zou, Yiding Liu, and Dawei Yin. 2020. Deep Multifaceted Transformers for Multi-objective Ranking in Large-scale E-commerce Recommender Systems. In Proceedings of the 29th ACM CIKM. 2493--2500.
[13]
F Maxwell Harper and Joseph A Konstan. 2015. The Movielens Datasets: History and Context. Acm Transactions on Interactive Intelligent Systems, Vol. 5, 4 (2015), 1--19.
[14]
Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991. Adaptive Mixtures of Local Experts. Neural Computation, Vol. 3, 1 (1991), 79--87.
[15]
Alex Kendall, Yarin Gal, and Roberto Cipolla. 2018. Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7482--7491.
[16]
Xiao Lin, Hongjie Chen, Changhua Pei, Fei Sun, Xuanji Xiao, Hanxiao Sun, Yongfeng Zhang, Wenwu Ou, and Peng Jiang. 2019. A Pareto-efficient Algorithm for Multiple Objective Optimization in E-commerce Recommendation. In Proceedings of the 13th ACM RecSys. 20--28.
[17]
Yichao Lu, Ruihai Dong, and Barry Smyth. 2018. Why I like it: Multi-task Learning for Recommendation and Explanation. In Proceedings of the 12th ACM RecSys. 4--12.
[18]
Jiaqi Ma, Zhe Zhao, Jilin Chen, Ang Li, Lichan Hong, and Ed H Chi. 2019. SNR: Sub-network Routing for Flexible Parameter Sharing in Multi-task Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 216--223.
[19]
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018b. Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-experts. In Proceedings of the 24th ACM SIGKDD. 1930--1939.
[20]
Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018a. Entire Dpace Multi-task Model: An Effective Approach for Estimating Post-click Conversion Rate. In Proceedings of the 41st ACM SIGIR. 1137--1140.
[21]
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert. 2016. Cross-stitch Networks for Multi-task Learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3994--4003.
[22]
Duy-Kien Nguyen and Takayuki Okatani. 2019. Multi-task Learning of Hierarchical Vision-language Representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 10492--10501.
[23]
Evaggelia Pitoura, Kostas Stefanidis, and Georgia Koutrika. 2021. Fairness in Rankings and Recommendations: An Overview. The VLDB Journal (2021), 1--28.
[24]
Clemens Rosenbaum, Tim Klinger, and Matthew Riemer. 2017. Routing Networks: Adaptive Selection of Non-linear Functions for Multi-task Learning. arXiv preprint arXiv:1711.01239 (2017).
[25]
Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, and Anders Søgaard. 2017. Sluice Networks: Learning What to Share Between Loosely Related Tasks. arXiv preprint arXiv:1705.08142, Vol. 2 (2017).
[26]
Victor Sanh, Thomas Wolf, and Sebastian Ruder. 2019. A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6949--6956.
[27]
Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations. In Proceedings of the 14th ACM RecSys. 269--278.
[28]
Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative Deep Learning for Recommender Systems. In Proceedings of the 21th ACM SIGKDD. 1235--1244.
[29]
Hong Wen, Jing Zhang, Fuyu Lv, Wentian Bao, Tianyi Wang, and Zulong Chen. 2021. Hierarchically Modeling Micro and Macro Behaviors via Multi-Task Learning for Conversion Rate Prediction. In Proceedings of the 44th ACM SIGIR. 2187--2191.
[30]
Ruobing Xie, Rui Wang, Shaoliang Zhang, Zhihong Yang, Feng Xia, and Leyu Lin. 2021. Real-time Relevant Recommendation Suggestion. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 112--120.
[31]
Yu Zhang and Qiang Yang. 2021. A Survey on Multi-Task Learning. IEEE Transactions on Knowledge and Data Engineering (2021), 1--1.
[32]
Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending What Video to Watch Next: a Multitask Ranking System. In Proceedings of the 13th ACM RecSys. 43--51.
[33]
Barret Zoph and Quoc V Le. 2016. Neural Architecture Search with Reinforcement Learning. arXiv preprint arXiv:1611.01578 (2016).

Cited By

View all
  • (2024)Multi-Task Recommendation with Task Information DecouplingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679621(2786-2795)Online publication date: 21-Oct-2024
  • (2024)Modeling multi-behavior sequence via HyperGRU contrastive network for micro-video recommendationKnowledge-Based Systems10.1016/j.knosys.2024.111841295(111841)Online publication date: Jul-2024
  • (2023)MT-BICN: Multi-task Balanced Information Cascade Network for RecommendationKnowledge Science, Engineering and Management10.1007/978-3-031-40289-0_34(423-435)Online publication date: 16-Aug-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management
October 2022
5274 pages
ISBN:9781450392365
DOI:10.1145/3511808
  • General Chairs:
  • Mohammad Al Hasan,
  • Li Xiong
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. multi-task learning
  2. recommendation systems

Qualifiers

  • Research-article

Conference

CIKM '22
Sponsor:

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)57
  • Downloads (Last 6 weeks)7
Reflects downloads up to 01 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Multi-Task Recommendation with Task Information DecouplingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679621(2786-2795)Online publication date: 21-Oct-2024
  • (2024)Modeling multi-behavior sequence via HyperGRU contrastive network for micro-video recommendationKnowledge-Based Systems10.1016/j.knosys.2024.111841295(111841)Online publication date: Jul-2024
  • (2023)MT-BICN: Multi-task Balanced Information Cascade Network for RecommendationKnowledge Science, Engineering and Management10.1007/978-3-031-40289-0_34(423-435)Online publication date: 16-Aug-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media