skip to main content
10.1145/3394486.3403314acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction

Published: 20 August 2020 Publication History

Abstract

Learning feature interactions is crucial for click-through rate (CTR) prediction in recommender systems. In most existing deep learning models, feature interactions are either manually designed or simply enumerated. However, enumerating all feature interactions brings large memory and computation cost. Even worse, useless interactions may introduce noise and complicate the training process. In this work, we propose a two-stage algorithm called Automatic Feature Interaction Selection (AutoFIS). AutoFIS can automatically identify important feature interactions for factorization models with computational cost just equivalent to training the target model to convergence. In the search stage, instead of searching over a discrete set of candidate feature interactions, we relax the choices to be continuous by introducing the architecture parameters. By implementing a regularized optimizer over the architecture parameters, the model can automatically identify and remove the redundant feature interactions during the training process of the model. In the re-train stage, we keep the architecture parameters serving as an attention unit to further boost the performance. Offline experiments on three large-scale datasets (two public benchmarks, one private) demonstrate that AutoFIS can significantly improve various FM based models. AutoFIS has been deployed onto the training platform of Huawei App Store recommendation service, where a 10-day online A/B test demonstrated that AutoFIS improved the DeepFM model by 20.3% and 20.1% in terms of CTR and CVR respectively.

Supplementary Material

MP4 File (3394486.3403314.mp4)
A recorded presentation video which introduces our KDD 2020 paper: AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction.

References

[1]
Gabriel Bender. 2019. Understanding and simplifying one-shot architecture search. In CVPR.
[2]
Alex Beutel, Paul Covington, Sagar Jain, Can Xu, Jia Li, Vince Gatto, and Ed H. Chi. 2018. Latent Cross: Making Use of Context in Recurrent Recommender Systems. In WSDM. 46--54.
[3]
Shih-Kang Chao and Guang Cheng. 2019. A generalization of regularized dual averaging and its dynamics. In CoRR. abs/1909.10072 (2019).
[4]
Tianqi Chen and Carlos Guestrin. 2016. XGBoost:A Scalable Tree Boosting System. In SIGKDD. 785--794.
[5]
Hengtze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Deepak Chandra, Hrishi Aradhye, Glen Anderson, Greg S Corrado, Wei Chai, Mustafa Ispir, et almbox. 2016. Wide & Deep Learning for Recommender Systems. In DLRS@RecSys.
[6]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations. In RecSys. 191--198.
[7]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In IJCAI. 1725--1731.
[8]
Chaoyang He, Haishan Ye, Li Shen, and Tong Zhang. 2020. MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation. In CVPR.
[9]
Xiangnan He and Tat-Seng Chua. 2017. Neural Factorization Machines for Sparse Predictive Analytics. In SIGIR. 355--364.
[10]
Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, and Stuart Bowers. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. In ADKDD@KDD. 5:1--5:9.
[11]
Kurt Hornik, Maxwell B. Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. In Neural Networks.
[12]
Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In ICML. 448--456.
[13]
Yu-Chin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware Factorization Machines for CTR Prediction. In RecSys.
[14]
YuChin Juan, Yong Zhuang, and Wei-Sheng Chin. 2014. 3 Idiots' Approach for Display Advertising Challenge. https://www.csie.ntu.edu.tw/ r01922136/kaggle-2014-criteo.pdf.
[15]
Yehuda Koren, Robert M. Bell, and Chris Volinsky. 2009. Matrix Factorization Techniques for Recommender Systems. IEEE Computer, Vol. 42, 8 (2009), 30--37.
[16]
Guilin Li, Xing Zhang, Zitong Wang, Zhenguo Li, and Tong Zhang. 2019. StacNAS: Towards stable and consistent optimization for differentiable Neural Architecture Search. arXiv preprint arXiv:1909.11926 (2019).
[17]
Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems. In KDD.
[18]
Bin Liu, Ruiming Tang, Yingzhi Chen, Jinkai Yu, Huifeng Guo, and Yuzhou Zhang. 2019 b. Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction. In WWW. ACM, 1119--1129.
[19]
Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019 a. DARTS: Differentiable Architecture Search. In ICLR.
[20]
Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen, Wenyuan Dai, and Qiang Yang. 2019. AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications. In KDD. 1936--1945.
[21]
H. Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, and Jeremy Kubica. 2013. Ad click prediction: a view from the trenches. In KDD.
[22]
Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018. Field-weighted factorization machines for click-through rate prediction in display advertising. In WWW. 1349--1357.
[23]
Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-Based Neural Networks for User Response Prediction. (2016), 1149--1154.
[24]
Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2019. Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data. ACM Trans. Inf. Syst., Vol. 37, 1 (2019), 5:1--5:35.
[25]
James Kwok Yong Li Cho-Jui Hsieh Quanming Yao, Xiangning Chen. 2020. Efficient Neural Interaction Function Search for Collaborative Filtering. In WWW.
[26]
Steffen Rendle. 2010. Factorization Machines. In ICDM. 995--1000.
[27]
Shai Shalev-Shwartz, Ohad Shamir, and Shaked Shammah. 2017. Failures of Gradient-Based Deep Learning. In ICML. 3067--3075.
[28]
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In ADKDD@KDD. 12.
[29]
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks. In IJCAI. 3119--3125.
[30]
Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. In European conference on information retrieval. Springer, 45--57.
[31]
Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click-Through Rate Prediction. In KDD. 1059--1068.

Cited By

View all
  • (2025)Deep Double Towers Click Through Rate Prediction Model with Multi-Head Bilinear FusionSymmetry10.3390/sym1702015917:2(159)Online publication date: 22-Jan-2025
  • (2025)FinalGNN: A dual feature graph enhanced model for CTR predictionNeurocomputing10.1016/j.neucom.2024.129181619(129181)Online publication date: Feb-2025
  • (2025)GraphFM: Graph Factorization Machines for Feature Interaction ModellingMachine Intelligence Research10.1007/s11633-024-1505-5Online publication date: 7-Jan-2025
  • Show More Cited By

Index Terms

  1. AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
    August 2020
    3664 pages
    ISBN:9781450379984
    DOI:10.1145/3394486
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 August 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. factorization machine
    2. feature selection
    3. neural architecture search
    4. recommendation

    Qualifiers

    • Research-article

    Conference

    KDD '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)145
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Deep Double Towers Click Through Rate Prediction Model with Multi-Head Bilinear FusionSymmetry10.3390/sym1702015917:2(159)Online publication date: 22-Jan-2025
    • (2025)FinalGNN: A dual feature graph enhanced model for CTR predictionNeurocomputing10.1016/j.neucom.2024.129181619(129181)Online publication date: Feb-2025
    • (2025)GraphFM: Graph Factorization Machines for Feature Interaction ModellingMachine Intelligence Research10.1007/s11633-024-1505-5Online publication date: 7-Jan-2025
    • (2024)Estrategias tecnológicas enfocadas en mejorar el posicionamiento digital de las empresas de la ciudad de PortoviejoCódigo Científico Revista de Investigación10.55813/gaea/ccri/v5/n1/3795:1(192-219)Online publication date: 30-Jun-2024
    • (2024)Feature-Interaction-Enhanced Sequential Transformer for Click-Through Rate PredictionApplied Sciences10.3390/app1407276014:7(2760)Online publication date: 26-Mar-2024
    • (2024)Cross Feature Engineering for Anti-Fraud Task in InsuranceArtificial Intelligence and Robotics Research10.12677/AIRR.2024.13204813:02(467-477)Online publication date: 2024
    • (2024)CETN: Contrast-enhanced Through Network for Click-Through Rate PredictionACM Transactions on Information Systems10.1145/368857143:1(1-34)Online publication date: 12-Aug-2024
    • (2024)AdaGIN: Adaptive Graph Interaction Network for Click-Through Rate PredictionACM Transactions on Information Systems10.1145/368178543:1(1-31)Online publication date: 4-Nov-2024
    • (2024)SimCEN: Simple Contrast-enhanced Network for CTR PredictionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681203(2311-2320)Online publication date: 28-Oct-2024
    • (2024)Multi-Scenario and Multi-Task Aware Feature Interaction for Recommendation SystemACM Transactions on Knowledge Discovery from Data10.1145/365131218:6(1-20)Online publication date: 12-Apr-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media