Wei Guo
ABSTRACT
Product based models, which represent multi-field categorical data as embedding vectors of features, then model feature interactions in terms of vector product of shared embedding, have been extensively studied and have become one of the most popular techniques for CTR prediction. However, if the shared embedding is applied: (1) the angles of feature interactions of different orders may conflict with each other, (2) the gradients of feature interactions of high-orders may vanish, which result in learned feature interactions less effective. To solve these problems, we propose a novel technique named Order-aware Embedding (i.e., multi-embeddings are learned for each feature, and different embeddings are applied for feature interactions of different orders), which can be applied to various models and generates feature interactions more effectively. We further propose a novel order-aware embedding neural network (OENN) based on this embedding technique for CTR prediction. Extensive experiments on three publicly available datasets demonstrate the effectiveness of Order-aware Embedding and show that our OENN outperforms the state-of-the-art models.
- Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proc. of the 1st Workshop on DLRS. ACM, 7--10. Google Scholar
Digital Library
- Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proc. of the 26th IJCAI. 1725--1731. Google ScholarDigital Library
- Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 355--364. Google ScholarDigital Library
- Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware factorization machines for CTR prediction. In Proc. of the 10th RecSys. ACM, 43--50. Google ScholarDigital Library
- Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. In Proc. of the 18th SIGKDD. ACM, 768--776. Google ScholarDigital Library
- Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems. In Proc. of the 24nd SIGKDD. 1754--1763. Google ScholarDigital Library
- Sebastian Prillo. 2017. An Elementary View on Factorization Machines. In Proc. of the 10th RecSys. ACM, 179--183. Google ScholarDigital Library
- Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2017. Product-Based Neural Networks for User Response Prediction. In IEEE International Conference on Data Mining. 1149--1154.Google Scholar
- Yanru Qu, Bohui Fang, Weinan Zhang, Ruiming Tang, Minzhe Niu, Huifeng Guo, Yong Yu, and Xiuqiang He. 2018. Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data. ACM Transactions on Information Systems (TOIS), Vol. 37, 1 (2018), 5. Google ScholarDigital Library
- Steffen Rendle. 2010. Factorization machines. In ICDM. IEEE, 995--1000. Google ScholarDigital Library
- Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In Proc. of the ADKDD'17. ACM, 12. Google ScholarDigital Library
- Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. In European conference on information retrieval. Springer, 45--57.Google ScholarCross Ref
Index Terms
- Order-aware Embedding Neural Network for CTR Prediction
Recommendations
Causality-based CTR prediction using graph neural networks
Highlights- Propose a causality-based CTR prediction model in the GNNs framework (Causal-GNN) integrating representations of feature graph, user graph and ad graph.
AbstractAs a prevalent problem in online advertising, CTR prediction has attracted plentiful attention from both academia and industry. Recent studies have been reported to establish CTR prediction models in the graph neural networks (GNNs) ...
Field-aware probabilistic embedding neural network for CTR prediction
RecSys '18: Proceedings of the 12th ACM Conference on Recommender SystemsFor Click-Through Rate (CTR) prediction, Field-aware Factorization Machines (FFM) have exhibited great effectiveness by considering field information. However, it is also observed that FFM suffers from the overfitting problem in many practical ...
Dual Graph enhanced Embedding Neural Network for CTR Prediction
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data MiningCTR prediction, which aims to estimate the probability that a user will click an item, plays a crucial role in online advertising and recommender system. Feature interaction modeling based and user interest mining based methods are the two kinds of most ...
Comments