Abstract
In this paper we investigate the performance of machine learning based recommender system with real-time log streaming on a large real-estate site, in the views of system robustness, business productivity and algorithm performance. Our proposed recommender system, providing personalized contents as opposed to item/query based recommendation, consists of a real-time log processor, auto-scaling recommender API and machine learning modules. System is carefully designed to let data scientists focus on improving core algorithms and features (instead of taking care of distributing systems) and achieves weekly release cycle in production environment. On Suumo, the largest real-estate portal site in Japan, the system returns more than 99.9% of the API calls successfully in real-time and shows finally a 250% improvement of conversion rate compared to the existing recommendation. With its flexible nature, we would also expect the system to be applied in various kinds of real-time recommendation in the near future.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 133–142. ACM, New York (2002)
Hopfgartner, F., Kille, B., Heintz, T., Turrin, R.: Real-time recommendation of streamed data. In: Proceedings of the 9th ACM Conference on Recommender Systems, pp. 361–362. ACM, New York (2015)
Freno, A., Saveski, M., Jenatton, R., Archambeau, C.: One-pass ranking models for low-latency product recommendations. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1789–1798. ACM, New York (2015)
Wang, F., Yuan, C., Xu, X., van Beek, P.: Supervised and semi-supervised online boosting tree for industrial machine vision application. In: Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data, pp. 43–51. ACM, New York (2011)
Bottou, L., Le Cun, Y.: Large scale online learning. Adv. Neural Inf. Process. Syst. 16, 217 (2004)
O’Sullivan, S.: Webinar: working together at the intersection of data science and data engineering (2015). https://datascience.berkeley.edu/blog/webinar-data-science-engineering/. Accessed 27 May 2016
Schleier-Smith, J.: An architecture for agile machine learning in real-time applications. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2059–2068. ACM, New York (2015)
Huang, Y., Cui, B., Zhang, W., Jiang, J., Xu, Y.: Tencentrec: real-time stream recommendation in practice. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 227–238. ACM, New York (2015)
Yuan, X., Lee, J.-H., Kim, S.-J., Kim, Y.-H.: Toward a user-oriented recommendation system for real estate websites. Inf. Syst. 38(2), 231–243 (2013)
Ho, H.-P., Chang, C.-T., Cheng-Yuan, K.: House selection via the internet by considering homebuyers risk attitudes with s-shaped utility functions. Eur. J. Oper. Res. 241(1), 188–201 (2015)
Wang, Y., Liao, X., Wu, H., Wu, J.: Incremental collaborative filtering considering temporal effects. arXiv preprint arXiv:1203.5415 (2012)
Iwanaga, J., Nabetani, K., Kajiwara, Y., Igarashi, K.: About the recommendation method based on the frequency and recency. J. Oper. Res. Soc. Jpn. 2013, 194–195 (2013) (in Japanese)
Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 1189–1232 (2001)
Rendle, S.: Factorization machines. In: 2010 IEEE 10th International Conference on Data Mining (ICDM), pp. 995–1000. IEEE (2010)
Distributed (Deep) Machine Learning Community. Xgboost (2016). https://github.com/dmlc/xgboost
Chen, T., He, T.: XGboost: extreme gradient boosting. R package version 0.4-2 (2015)
Bergstra, J., Yamins, D., Cox, D.D.: Hyperopt: a python library for optimizing the hyperparameters of machine learning algorithms. In: Proceedings of the 12th Python in Science Conference, pp. 13–20 (2013)
Acknowledgments
We are grateful to Yoichi Maejima for useful discussions about the model specification. Special thanks to Iwao Watanabe and Nobuaki Oshiro for building the fast API, Kentaro Hashimoto and all the guys in the infrastructure team for indulging us with general help on the log processing platform.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Li, S., Nomura, S., Kikuta, Y., Arino, K. (2017). Web-Scale Personalized Real-Time Recommender System on Suumo. In: Kim, J., Shim, K., Cao, L., Lee, JG., Lin, X., Moon, YS. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2017. Lecture Notes in Computer Science(), vol 10235. Springer, Cham. https://doi.org/10.1007/978-3-319-57529-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-319-57529-2_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-57528-5
Online ISBN: 978-3-319-57529-2
eBook Packages: Computer ScienceComputer Science (R0)