Abstract
Recommending news articles is a challenging task due to the continuous changes in the set of available news articles and the context-dependent preferences of users. In addition, news recommenders must fulfill high requirements with respect to response time and scalability. Traditional recommender approaches are optimized for the analysis of static data sets. In news recommendation scenarios, characterized by continuous changes, high volume of messages, and tight time constraints, alternative approaches are needed. In this work we present a highly scalable recommender system optimized for the processing of streams. We evaluate the system in the CLEF NewsREEL challenge. Our system is built on Apache Spark enabling the distributed processing of recommendation requests ensuring the scalability of our approach. The evaluation of the implemented system shows that our approach is suitable for the news recommendation scenario and provides high-quality results while satisfying the tight time constraints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Diaz-Aviles, E., Drumond, L., Schmidt-Thieme, L., Nejdl, W.: Real-time top-n recommendation in social streams. In: Proceedings of the 6th ACM Conference on Recommender Systems, pp. 59–66. ACM, New York (2012). ISBN: 978-1-4503-1270-7
Ekstrand, M.D., Ludwig, M., Konstan, J.A., Riedl, J.T.: Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit. In: Proceedings of the 5th ACM Conference on Recommender Systems, pp. 133–140. ACM (2011)
Hopfgartner, F., Kille, B., Lommatzsch, A., Plumbaum, T., Brodt, T., Heintz, T.: Benchmarking news recommendations in a living lab. In: Kanoulas, E., Lupu, M., Clough, P., Sanderson, M., Hall, M., Hanbury, A., Toms, E. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 250–267. Springer, Cham (2014). doi:10.1007/978-3-319-11382-1_21
Hopfgartner, F., Brodt, T., Seiler, J., Kille, B., Lommatzsch, A., Larson, M., Turrin, R., Serény, A., Recommendations, B.N.: The CLEF NewsREEL use case. SIGIR Forum 49(2), 129–136 (2016). doi:10.1145/2888422.2888443. ISSN: 0163–5840
Linden, A., Krensky, P., Hare, J., Idoine, C.J., Sicular, S., Vashisth, S.: Magic quadrant for data science platforms. In: Gartner & Forrester & Aragon, collection hiver 2017, pp. 28–29 (2017). https://myleadcorner.files.wordpress.com/2017/04/magic-quadrant-for-data-science-platforms-feb-2017-1.pdf
Lommatzsch, A., Albayrak, S.: Real-time recommendations for user-item streams. In: Proceedings of the 30th ACM Symposium on Applied Computing (SAC 2015), pp. 1039–1046. ACM, New York (2015). ISBN: 978-1-4503-3196-8
Seminario, C.E., Wilson, D.C.: Case Study evaluation of mahout as a recommender platform. In: RUE @ RecSys, pp. 45–50 (2012)
Walunj, S.G., Sadafale,K.: An online recommendation system for e-commerce based on apache mahout framework. In: Proceedings of the 2013 Conference on Computers and People Research, pp. 153–158. ACM (2013)
Zangerle, E., Gassler, W., Specht, G.: Exploiting Twitter’s collective knowledge for music recommendations. In: Rowe, M., Stankovic, M., Dadzie, A.-S. (eds.) Making Sense of Microposts, pp. 14–17, April 2012. http://ceur-ws.org/Vol-838
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Domann, J., Lommatzsch, A. (2017). A Highly Available Real-Time News Recommender Based on Apache Spark. In: Jones, G., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2017. Lecture Notes in Computer Science(), vol 10456. Springer, Cham. https://doi.org/10.1007/978-3-319-65813-1_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-65813-1_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65812-4
Online ISBN: 978-3-319-65813-1
eBook Packages: Computer ScienceComputer Science (R0)