skip to main content
10.1145/3331184.3331196acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Bayesian Personalized Feature Interaction Selection for Factorization Machines

Published: 18 July 2019 Publication History

Abstract

Factorization Machines (FMs) are widely used for feature-based collaborative filtering tasks, as they are very effective at modeling feature interactions. Existing FM-based methods usually take all feature interactions into account, which is unreasonable because not all feature interactions are helpful: incorporating useless feature interactions will introduce noise and degrade the recommendation performance. Recently, methods that perform Feature Interaction Selection (FIS) have attracted attention because of their effectiveness at filtering out useless feature interactions. However, they assume that all users share the same feature interactions, which is not necessarily true, especially for collaborative filtering tasks. In this work, we address this issue and study Personalized Feature Interaction Selection (P-FIS) by proposing a Bayesian Personalized Feature Interaction Selection (BP-FIS) mechanism under the Bayesian Variable Selection (BVS) theory. Specifically, we first introduce interaction selection variables with hereditary spike and slab priors for P-FIS. Then, we form a Bayesian generative model and derive the Evidence Lower Bound (ELBO), which can be optimized by an efficient Stochastic Gradient Variational Bayes (SGVB) method to learn the parameters. Finally, because BP-FIS can be seamlessly integrated with different variants of FMs, we implement two FM variants under the proposed BP-FIS. We carry out experiments on three benchmark datasets. The empirical results demonstrate the effectiveness of BP-FIS for selecting personalized interactions and improving the recommendation performance.

Supplementary Material

MP4 File (cite2-11h00-d3.mp4)

References

[1]
Michal Aharon, Natalie Aizenberg, Edward Bortnikov, Ronny Lempel, Roi Adadi, and etc. 2013. OFF-set: One-pass Factorization of Feature Sets for Online Recommendation in Persistent Cold Start Settings. In RecSys. ACM, 375--378.
[2]
Taha Alshaybawee, Rahim Alhamzawi, Habshah Midi, and Intisar Ibrahim Allyas. 2018. Bayesian Variable Selection and Coefficient Estimation in Heteroscedastic Linear Regression Model. Journal of Applied Statistics, Vol. 45, 14 (2018), 2643--2657.
[3]
Cédric Archambeau and Francis R. Bach. 2009. Sparse Probabilistic Projections. In NIPS. 73--80.
[4]
Mathieu Blondel, Akinori Fujino, Naonori Ueda, and Masakazu Ishihata. 2016. Higher-Order Factorization Machines. In NIPS. 3351--3359.
[5]
Mathieu Blondel, Masakazu Ishihata, Akinori Fujino, and Naonori Ueda. 2016. Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms. In ICML. ACM, 850--858.
[6]
Iván Cantador, Peter Brusilovsky, and Tsvi Kuflik. 2011. 2nd Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec 2011). In RecSys. ACM.
[7]
Kailong Chen, Tianqi Chen, Guoqing Zheng, Ou Jin, Enpeng Yao, and Yong Yu. 2012. Collaborative Personalized Tweet Recommendation. In SIGIR. ACM, 661--670.
[8]
Yifan Chen, Yang Wang, Xiang Zhao, Hongzhi Yin, Ilya Markov, and Maarten de Rijke. 2019. Local Variational Feature-based Similarity Models for Recommending Top-N New Items. ACM Transactions on Information Systems (2019). To appear.
[9]
Yifan Chen, Xiang Zhao, and Maarten de Rijke. 2017. Top-N Recommendation with High-dimensional Side Information via Locality Preserving Projection. In SIGIR. ACM, 985--988.
[10]
Yifan Chen, Xiang Zhao, Xuemin Lin, Yang Wang, and Deke Guo. 2019. Efficient Mining of Frequent Patterns on Uncertain Graphs. IEEE Trans. Knowl. Data Eng., Vol. 31, 2 (2019), 287--300.
[11]
Chen Cheng, Fen Xia, Tong Zhang, Irwin King, and Michael R. Lyu. 2014. Gradient Boosting Factorization Machines. In RecSys. ACM, 265--272.
[12]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah. 2016. Wide & Deep Learning for Recommender Systems. In DLRS. 7--10.
[13]
Nam Hee Choi, William Li, and Ji Zhu. 2010. Variable Selection with the Strong Heredity Constraint and its Oracle Property. J. Amer. Statist. Assoc., Vol. 105, 489 (2010), 354--364.
[14]
Petros Dellaportas, Jonathan J. Forster, and Ioannis Ntzoufras. 2002. On Bayesian Model and Variable Selection using MCMC. Statistics and Computing, Vol. 12, 1 (2002), 27--36.
[15]
Cédric Févotte and Simon J. Godsill. 2006. Sparse Linear Regression in Unions of Bases via Bayesian Variable Selection. IEEE Signal Processing Letters, Vol. 13, 7 (2006), 441--444.
[16]
F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Inter. Intel. Syst., Vol. 5, 4 (2015), 19:1--19:19.
[17]
Xiangnan He and Tat-Seng Chua. 2017. Neural Factorization Machines for Sparse Predictive Analytics. In SIGIR. ACM, 355--364.
[18]
Yuchin Juan, Damien Lefortier, and Olivier Chapelle. 2017. Field-aware Factorization Machines in a Real-world Online Advertising System. In WWW. 680--688.
[19]
Yu-Chin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware Factorization Machines for CTR Prediction. In RecSys. ACM, 43--50.
[20]
Santosh Kabbur, Xia Ning, and George Karypis. 2013. FISM: Factored Item Similarity Models for Top-N Recommender Systems. In SIGKDD. ACM, 659--667.
[21]
George Karypis. 2001. Evaluation of Item-Based Top-N Recommendation Algorithms. In CIKM. ACM, 247--254.
[22]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
[23]
Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In ICLR.
[24]
Noam Koenigstein and Ulrich Paquet. 2013. Xbox Movies Recommendations: Variational Bayes Matrix Factorization with Embedded Feature Selection. In RecSys. ACM, 129--136.
[25]
Yehuda Koren. 2008. Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model. In SIGKDD. ACM, 426--434.
[26]
Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural Attentive Session-based Recommendation. In CIKM. ACM, 1419--1428.
[27]
Jundong Li, Liang Wu, Harsh Dani, and Huan Liu. 2018. Unsupervised Personalized Feature Selection. In AAAI. 3514--3521.
[28]
Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In WWW. 689--698.
[29]
Xiao Lin, Wenpeng Zhang, Min Zhang, Wenwu Zhu, Jian Pei, Peilin Zhao, and Junzhou Huang. 2018. Online Compact Convexified Factorization Machine. In WWW. 1633--1642.
[30]
Chun-Ta Lu, Lifang He, Weixiang Shao, Bokai Cao, and Philip S. Yu. 2017. Multilinear Factorization Machines for Multi-Task Multi-View Learning. In WSDM. ACM, 701--709.
[31]
Xueyu Mao, Saayan Mitra, and Viswanathan Swaminathan. 2017. Feature Selection for FM-Based Context-Aware Recommendation Systems. In ISM. IEEE, 252--255.
[32]
Peter McCullagh and John A. Nelder. 1989. Generalized Linear Models. Vol. 37. CRC press.
[33]
Lei Mei, Pengjie Ren, Zhumin Chen, Liqiang Nie, Jun Ma, and Jian-Yun Nie. 2018. An Attentive Interaction Network for Context-aware Recommendations. In CIKM. ACM, 157--166.
[34]
Toby J. Mitchell and John J. Beauchamp. 1988. Bayesian Variable Selection in Linear Regression. J. Amer. Statist. Assoc., Vol. 83, 404 (1988), 1023--1032.
[35]
Trung V. Nguyen, Alexandros Karatzoglou, and Linas Baltrunas. 2014. Gaussian Process Factorization Machines for Context-aware Recommendations. In SIGIR. ACM, 63--72.
[36]
Xia Ning and George Karypis. 2011. SLIM: Sparse Linear Methods for Top-N Recommender Systems. In ICDM. IEEE, 497--506.
[37]
Zhen Pan, Enhong Chen, Qi Liu, Tong Xu, Haiping Ma, and Hongjie Lin. 2016. Sparse Factorization Machines for Click-through Rate Prediction. In ICDM. IEEE, 400--409.
[38]
Rajiv Pasricha and Julian McAuley. 2018. Translation-based Factorization Machines for Sequential Recommendation. In RecSys. ACM, 63--71.
[39]
Steffen Rendle. 2010. Factorization Machines. In ICDM. IEEE, 995--1000.
[40]
Steffen Rendle. 2012. Factorization Machines with libFM. ACM Trans. Intel. Syst. & Tech., Vol. 3, 3 (2012), 57:1--57:22.
[41]
Steffen Rendle. 2012. Learning Recommender Systems with Adaptive Regularization. In WSDM. ACM, 133--142.
[42]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452--461.
[43]
Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2011. Fast Context-aware Recommendations with Factorization Machines. In SIGIR. ACM, 635--644.
[44]
Steffen Rendle and Lars Schmidt-Thieme. 2010. Pairwise Interaction Tensor Factorization for Personalized Tag Recommendation. In WSDM. ACM, 81--90.
[45]
Royi Ronen, Noam Koenigstein, Elad Ziklik, and Nir Nice. 2013. Selecting Content-based Features for Collaborative Filtering Recommenders. In RecSys. ACM, 407--410.
[46]
Ying Shan, T. Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and J. C. Mao. 2016. Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features. In SIGKDD. ACM, 255--262.
[47]
Nathan Srebro, Jason D. M. Rennie, and Tommi S. Jaakkola. 2004. Maximum-Margin Matrix Factorization. In NIPS. 1329--1336.
[48]
Ingo Steinwart and Andreas Christmann. 2008. Support Vector Machines .Springer Science & Business Media.
[49]
Robert Tibshirani. 1996. Regression Shrinkage and Selection via the Lasso. J. Royal Statist. Soci. (1996), 267--288.
[50]
Michalis K. Titsias and Miguel Lázaro-Gredilla. 2011. Spike and Slab Variational Inference for Multi-Task and Multiple Kernel Learning. In NIPS. 2339--2347.
[51]
Seiya Tokui and Issei Sato. 2016. Reparameterization Trick for Discrete Variables. CoRR, Vol. abs/1611.01239 (2016). arxiv: 1611.01239
[52]
Meng Wang, Hao Li, Dacheng Tao, Ke Lu, and Xindong Wu. 2012. Multimodal Graph-Based Reranking for Web Image Search. IEEE Trans. Image Processing, Vol. 21, 11 (2012), 4649--4661.
[53]
Meng Wang, Xueliang Liu, and Xindong Wu. 2015. Visual Classification by ℓ1-Hypergraph Modeling. IEEE Trans. Knowl. Data Eng., Vol. 27, 9 (2015), 2564--2574.
[54]
Yang Wang, Xuemin Lin, Lin Wu, Wenjie Zhang, Qing Zhang, and Xiaodi Huang. 2015. Robust Subspace Clustering for Multi-View Data by Exploiting Correlation Consensus. IEEE Trans. Image Processing, Vol. 24, 11 (2015), 3939--3949.
[55]
Mike West. 2003. Bayesian Factor Regression Models in the "Large p, Small n" Paradigm. In Bayesian Statistics. Oxford University Press, 723--732.
[56]
Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative Denoising Auto-Encoders for Top-N Recommender Systems. In WSDM. ACM, 153--162.
[57]
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks. In IJCAI. 3119--3125.
[58]
Jianpeng Xu, Kaixiang Lin, Pang-Ning Tan, and Jiayu Zhou. 2016. Synergies that Matter: Efficient Interaction Selection via Sparse Factorization Machine. In SDM. SIAM, 108--116.
[59]
Makoto Yamada, Wenzhao Lian, Amit Goyal, Jianhui Chen, Kishan Wimalawarne, and etc. 2017. Convex Factorization Machine for Toxicogenomics Prediction. In SIGKDD. ACM, 1215--1224.
[60]
Fajie Yuan, Guibing Guo, Joemon M. Jose, Long Chen, Haitao Yu, and Weinan Zhang. 2017. BoostFM: Boosted Factorization Machines for Top-N Feature-based Recommendation. In IUI. 45--54.
[61]
Ming Yuan and Yi Lin. 2006. Model Selection and Estimation in Regression with Grouped Variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 68, 1 (2006), 49--67.
[62]
Huan Zhao, Quanming Yao, Jianda Li, Yangqiu Song, and Dik Lun Lee. 2017. Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks. In SIGKDD. ACM, 635--644.
[63]
Yong Zheng, Robin D. Burke, and Bamshad Mobasher. 2013. Recommendation with Differential Context Weighting. In UMAP. 152--164.

Cited By

View all
  • (2024)When large language models meet personalization: perspectives of challenges and opportunitiesWorld Wide Web10.1007/s11280-024-01276-127:4Online publication date: 28-Jun-2024
  • (2023)AutoML for Deep Recommender Systems: A SurveyACM Transactions on Information Systems10.1145/357935541:4(1-38)Online publication date: 22-Mar-2023
  • (2023)Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge DistillationProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3570384(715-723)Online publication date: 27-Feb-2023
  • Show More Cited By

Index Terms

  1. Bayesian Personalized Feature Interaction Selection for Factorization Machines

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2019
    1512 pages
    ISBN:9781450361729
    DOI:10.1145/3331184
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 July 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. bayesian variable selection
    2. factorization machines
    3. personalized feature interaction selection
    4. recommender systems

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • Ahold Delhaize
    • Innovation Center for Artificial Intelligence (ICAI)
    • Association of Universities in the Netherlands (VSNU)
    • China Scholarship Council

    Conference

    SIGIR '19
    Sponsor:

    Acceptance Rates

    SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;
    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)34
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 22 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)When large language models meet personalization: perspectives of challenges and opportunitiesWorld Wide Web10.1007/s11280-024-01276-127:4Online publication date: 28-Jun-2024
    • (2023)AutoML for Deep Recommender Systems: A SurveyACM Transactions on Information Systems10.1145/357935541:4(1-38)Online publication date: 22-Mar-2023
    • (2023)Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge DistillationProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3570384(715-723)Online publication date: 27-Feb-2023
    • (2023)Reinforcement Routing on Proximity Graph for Efficient RecommendationACM Transactions on Information Systems10.1145/351276741:1(1-27)Online publication date: 10-Jan-2023
    • (2023)A Novel Hybrid Feature Importance and Feature Interaction Detection Framework for Predictive Optimization in Industry 4.0 ApplicationsIECON 2023- 49th Annual Conference of the IEEE Industrial Electronics Society10.1109/IECON51785.2023.10312491(1-6)Online publication date: 16-Oct-2023
    • (2022)DA-GAN: Dual Attention Generative Adversarial Network for Cross-Modal RetrievalFuture Internet10.3390/fi1402004314:2(43)Online publication date: 27-Jan-2022
    • (2022)Causal factorization machine for robust recommendationProceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries10.1145/3529372.3530921(1-9)Online publication date: 20-Jun-2022
    • (2022)DaisyRec 2.0: Benchmarking Recommendation for Rigorous EvaluationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.3231891(1-20)Online publication date: 2022
    • (2022)Enhancing Factorization Machines With Generalized Metric LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.303461334:8(3740-3753)Online publication date: 1-Aug-2022
    • (2022)Bayesian feature interaction selection for factorization machinesArtificial Intelligence10.1016/j.artint.2021.103589302:COnline publication date: 1-Jan-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media