Yangfan Li
Skip Abstract Section
Abstract
Both accurate and fast mobile recommendation systems based on click behaviors analysis are crucial in e-business. Deep learning has achieved state-of-the-art accuracy and the traditional wisdom often hosts these computation-intensive models in powerful cloud centers. However, the cloud-only approaches put significant computational pressure on cloud servers and increase the latency in heavy-load scenarios. Moreover, existing work often adopts RNN structures to model behaviors that suffer from low processing speed for under-utilization of parallel devices such as GPUs. In this work, we propose an efficient internet behavior-based recommendation framework with edge-cloud collaboration on deep CNNs (CoRec) to improve both the accuracy and speed for mobile recommendation. A novel convolutional interest network (CIN) improves the accuracy by modeling the long- and short-term interests and accelerates the prediction through parallel-friendly convolutions. To further improve the serving throughput and latency, a novel device-cloud collaboration strategy reduces workloads by pre-computing and caching long-term interests in the cloud offline and real-time computation of short-term interests in devices. Extensive experiments on real-world datasets show that CoRec significantly outperforms the state-of-the-art methods in accuracy and has achieved at least an order of magnitude improvement in latency and throughput compared to cloud-only RNN-based approaches for long behaviors.
- [1] . 2016. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI16). 265–283.Google Scholar
- [2] . 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- [3] . 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018).Google Scholar
- [4] . 2008. Contextual advertising by combining relevance with click feedback. In Proceedings of the 17th International Conference on World Wide Web. 417–426.Google ScholarDigital Library
- [5] . 2021. Sequential recommendation with graph neural networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 378–387.Google ScholarDigital Library
- [6] . 2014. Simple and scalable response prediction for display advertising. ACM Trans. Intell. Syst. Technol. 5, 4 (2014), 1–34.Google ScholarDigital Library
- [7] . 2018. FlinkCL: An openCL-based in-memory computing architecture on heterogeneous CPU-GPU clusters for big data. IEEE Trans. Comput. 67, 12 (2018), 1765–1779.Google ScholarDigital Library
- [8] . 2018. GFlink: An in-memory computing architecture on heterogeneous CPU-GPU clusters for big data. IEEE Trans. Parallel Distrib. Syst. 29, 6 (2018), 1275–1288.Google ScholarCross Ref
- [9] . 2020. Citywide traffic flow prediction based on multiple gated spatio-temporal convolutional neural networks. ACM Trans. Knowl. Discov. Data 14, 4 (2020), 1–23.Google ScholarDigital Library
- [10] . 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. ACM, 7–10.Google ScholarDigital Library
- [11] . 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).Google Scholar
- [12] . 2017. Language modeling with gated convolutional networks. In Proceedings of the 34th International Conference on Machine Learning. JMLR. org, 933–941.Google ScholarDigital Library
- [13] . 2010. Learning the click-through rate for rare/new ads from similar ads. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 897–898.Google ScholarDigital Library
- [14] . 2019. Serving deep neural networks at the cloud edge for vision applications on mobile platforms. In Proceedings of the 10th ACM Multimedia Systems Conference. 36–47.Google ScholarDigital Library
- [15] . 2019. Deep session interest network for click-through rate prediction. arXiv preprint arXiv:1905.06482 (2019).Google Scholar
- [16] . 2016. A convolutional encoder model for neural machine translation. arXiv preprint arXiv:1611.02344 (2016).Google Scholar
- [17] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google ScholarCross Ref
- [18] . 2014. Practical lessons from predicting clicks on ads at Facebook. In Proceedings of the 8th International Workshop on Data Mining for Online Advertising. 1–9.Google ScholarDigital Library
- [19] . 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4700–4708.Google ScholarCross Ref
- [20] . 2020. A lightweight collaborative deep neural network for the mobile web in edge cloud. IEEE Trans. Mob. Comput. 21, 7 (2020).Google Scholar
- [21] . 2020. DeepAdapter: A collaborative deep learning framework for the mobile web using context-aware network pruning. In Proceedings of the IEEE Conference on Computer Communications. IEEE, 834–843.Google ScholarDigital Library
- [22] . 2016. Neural machine translation in linear time. arXiv preprint arXiv:1610.10099 (2016).Google Scholar
- [23] . 2017. Neurosurgeon: Collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Comput. Archit. News 45, 1 (2017), 615–629.Google ScholarDigital Library
- [24] . 2017. Modeling long- and short-term temporal patterns with deep neural networks. In the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 95–104. .Google ScholarDigital Library
- [25] . 2018. xDeepFM: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1754–1763.Google ScholarDigital Library
- [26] . 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2117–2125.Google ScholarCross Ref
- [27] . 2019. Feature generation by convolutional neural network for click-through rate prediction. In Proceedings of the World Wide Web Conference. ACM, 1119–1129.Google ScholarDigital Library
- [28] . 2021. A dual input-aware factorization machine for CTR prediction. In Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence. 3139–3145.Google Scholar
- [29] . 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, Nov. (2008), 2579–2605.Google Scholar
- [30] . 2017. MoDNN: Local distributed mobile computing system for deep neural network. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 1396–1401.Google ScholarCross Ref
- [31] . 2019. MobiEye: An efficient cloud-based video detection system for real-time mobile applications. In Proceedings of the 56th Annual Design Automation Conference. 1–6.Google ScholarDigital Library
- [32] . 2013. Ad click prediction: A view from the trenches. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1222–1230.Google ScholarDigital Library
- [33] . 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 3111–3119.Google Scholar
- [34] . 2014. Predicting response in mobile advertising with hierarchical importance-aware factorization machine. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining. 123–132.Google ScholarDigital Library
- [35] . 2016. Product-based neural networks for user response prediction. In Proceedings of the IEEE 16th International Conference on Data Mining (ICDM). IEEE, 1149–1154.Google ScholarCross Ref
- [36] . 2020. Recommender systems for smart cities. Inf. Syst. 92 (2020), 101545.Google ScholarCross Ref
- [37] . 2018. DeepDecision: A mobile deep learning framework for edge video analytics. In Proceedings of the IEEE Conference on Computer Communications. IEEE, 1421–1429.Google ScholarDigital Library
- [38] . 2010. Factorization machines. In Proceedings of the IEEE International Conference on Data Mining. IEEE, 995–1000.Google ScholarDigital Library
- [39] . 2012. Factorization machines with libFM. ACM Trans. Intell. Syst. Technol. 3, 3 (2012), 1–22.Google ScholarDigital Library
- [40] . 2007. Predicting clicks: Estimating the click-through rate for new ads. In Proceedings of the 16th International Conference on World Wide Web. ACM, 521–530.Google ScholarDigital Library
- [41] . 2018. MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4510–4520.Google ScholarCross Ref
- [42] . 2017. Distributed deep neural networks over the cloud, the edge and end devices. In Proceedings of the IEEE 37th International Conference on Distributed Computing Systems (ICDCS). IEEE, 328–339.Google ScholarCross Ref
- [43] . 2014. Coupled group lasso for web-scale CTR prediction in display advertising. In Proceedings of the International Conference on Machine Learning. 802–810.Google Scholar
- [44] . 2019. Hierarchical temporal convolutional networks for dynamic recommender systems. In Proceedings of the World Wide Web Conference. ACM, 2236–2246.Google ScholarDigital Library
- [45] . 2016. A dynamic recurrent model for next basket recommendation. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 729–732.Google ScholarDigital Library
- [46] . 2018. Deep layer aggregation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2403–2412.Google ScholarCross Ref
- [47] . 2018. DeepCPU: Serving RNN-based deep learning models 10x faster. In Proceedings of the USENIX Annual Technical Conference (USENIXATC’18). 951–965.Google Scholar
- [48] . 2021. Deep-learning-based probabilistic estimation of solar PV soiling loss. IEEE Trans. Sustain. Energy 12, 4 (2021), 2436–2444.Google ScholarCross Ref
- [49] . 2018. An improved quantile regression neural network for probabilistic load forecasting. IEEE Trans. Smart Grid 10, 4 (2018).Google Scholar
- [50] . 2018. Parallel and reliable probabilistic load forecasting via quantile regression forest and quantile determination. Energy 160 (2018), 810–819.Google ScholarCross Ref
- [51] . 2019. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. 5941–5948.Google ScholarDigital Library
- [52] . 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1059–1068.Google ScholarDigital Library
Index Terms
- CoRec: An Efficient Internet Behavior-based Recommendation Framework with Edge-cloud Collaboration on Deep Convolution Neural Networks
Recommendations
Efficient densely connected convolutional neural networks
Highlights- Proposed two efficient densely connected ConvNets, DesneDsc and Dense2Net.
- ...
Graphical abstractDisplay Omitted
AbstractRecent works have shown that convolutional neural networks (CNNs) are parameter redundant, which limits the application of CNNs in Mobile devices with limited memory and computational resources. In this paper, two novel and efficient ...
Joint Deep Modeling of Users and Items Using Reviews for Recommendation
WSDM '17: Proceedings of the Tenth ACM International Conference on Web Search and Data MiningA large amount of information exists in reviews written by users. This source of information has been ignored by most of the current recommender systems while it can potentially alleviate the sparsity problem and improve the quality of recommendations. ...
3D Convolutional Networks for Session-based Recommendation with Content Features
RecSys '17: Proceedings of the Eleventh ACM Conference on Recommender SystemsIn many real-life recommendation settings, user profiles and past activities are not available. The recommender system should make predictions based on session data, e.g. session clicks and descriptions of clicked items. Conventional recommendation ...
Comments