Skip to main content
SpringerLink
Log in

Online Gradient Boosting for Incremental Recommender Systems

  • Conference paper
  • First Online:
Discovery Science (DS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11198))

Included in the following conference series:

Abstract

Ensemble models have been proven successful for batch recommendation algorithms, however they have not been well studied in streaming applications. Such applications typically use incremental learning, to which standard ensemble techniques are not trivially applicable. In this paper, we study the application of three variants of online gradient boosting to top-N recommendation tasks with implicit data, in a streaming data environment. Weak models are built using a simple incremental matrix factorization algorithm for implicit feedback. Our results show a significant improvement of up to 40% over the baseline standalone model. We also show that the overhead of running multiple weak models is easily manageable in stream-based applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.grouplens.org/data [Jan 2013].

  2. 2.

    https://webscope.sandbox.yahoo.com/catalog.php?datatype=r [Jan 2013].

  3. 3.

    http://last.fm/.

  4. 4.

    http://ocelma.net/MusicRecommendationDataset [Jan 2013].

  5. 5.

    https://rdm.inesctec.pt/dataset/cs-2017-003, file: playlisted_tracks.tsv.

  6. 6.

    http://www.palcoprincipal.com/.

References

  1. Beygelzimer, A., Hazan, E., Kale, S., Luo, H.: Online gradient boosting. In: Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7–12, 2015, Montreal, Quebec, Canada, pp. 2458–2466 (2015)

    Google Scholar 

  2. Bifet, A., Holmes, G., Pfahringer, B.: Leveraging bagging for evolving data streams. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6321, pp. 135–150. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15880-3_15

    Chapter  Google Scholar 

  3. Bifet, A., Holmes, G., Pfahringer, B., Kirkby, R., Gavaldà, R.: New ensemble methods for evolving data streams. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, June 28 - July 1, 2009, pp. 139–148. ACM (2009)

    Google Scholar 

  4. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MathSciNet  MATH  Google Scholar 

  5. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  6. Chen, S., Lin, H., Lu, C.: An online boosting algorithm with theoretical justifications. In: Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, June 26 - July 1, 2012. icml.cc / Omnipress (2012)

    Google Scholar 

  7. Chowdhury, N., Cai, X., Luo, C.: BoostMF: boosted matrix factorisation for collaborative ranking. In: Appice, A., Rodrigues, P.P., Santos Costa, V., Gama, J., Jorge, A., Soares, C. (eds.) ECML PKDD 2015. LNCS (LNAI), vol. 9285, pp. 3–18. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23525-7_1

    Chapter  Google Scholar 

  8. Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of the 13th Intl. Conference on Machine Learning ICML ’96, pp. 148–156. Morgan Kaufmann (1996)

    Google Scholar 

  9. Friedman, J.H.: Stochastic gradient boosting. Comput. Stat. Data Anal. 38(4), 367–378 (2002)

    Article  MathSciNet  Google Scholar 

  10. Gama, J., Medas, P., Rocha, R.: Forest trees for on-line data. In: Proceedings of the 2004 ACM Symposium on Applied Computing (SAC), Nicosia, Cyprus, March 14–17, 2004, pp. 632–636. ACM (2004)

    Google Scholar 

  11. Gama, J., Sebastião, R., Rodrigues, P.P.: On evaluating stream learning algorithms. Mach. Learn. 90(3), 317–346 (2013)

    Article  MathSciNet  Google Scholar 

  12. Gomes, H.M., Barddal, J.P., Enembreck, F., Bifet, A.: A survey on ensemble learning for data stream classification. ACM Comput. Surv. 50(2), 23:1–23:36 (2017)

    Article  Google Scholar 

  13. Hu, H., Sun, W., Venkatraman, A., Hebert, M., Bagnell, J.A.: Gradient boosting on stochastic data streams. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20–22 April 2017, Fort Lauderdale, FL, USA. Proceedings of Machine Learning Research, vol. 54, pp. 595–603. PMLR (2017)

    Google Scholar 

  14. Jahrer, M., Töscher, A., Legenstein, R.A.: Combining predictions for accurate recommender systems. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2010, pp. 693–702. ACM (2010)

    Google Scholar 

  15. Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Wozniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)

    Article  Google Scholar 

  16. Lee, H.K.H., Clyde, M.A.: Lossless online bayesian bagging. J. Mach. Learn. Res. 5, 143–151 (2004)

    MathSciNet  Google Scholar 

  17. Oza, N.C., Russell, S.J.: Experimental comparisons of online and batch versions of bagging and boosting. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2001, pp. 359–364. ACM (2001)

    Google Scholar 

  18. Schclar, A., Tsikinovsky, A., Rokach, L., Meisels, A., Antwarg, L.: Ensemble methods for improving the performance of neighborhood-based collaborative filtering. In: Proceedings of the 2009 ACM Conference on Recommender Systems, RecSys 2009, pp. 261–264. ACM (2009)

    Google Scholar 

  19. Segrera, S., Moreno, M.N.: An experimental comparative study of web mining methods for recommender systems. In: Proceedings of the 6th WSEAS Intl. Conf. on Distance Learning and Web Engineering, pp. 56–61. WSEAS (2006)

    Google Scholar 

  20. Sill, J., Takács, G., Mackey, L.W., Lin, D.: Feature-weighted linear stacking. CoRR (2009). arXiv:0911.0460

  21. Vinagre, J., Jorge, A.M., Gama, J.: Fast incremental matrix factorization for recommendation with positive-only feedback. In: Dimitrova, V., Kuflik, T., Chin, D., Ricci, F., Dolog, P., Houben, G.-J. (eds.) UMAP 2014. LNCS, vol. 8538, pp. 459–470. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08786-3_41

    Chapter  Google Scholar 

  22. Vinagre, J., Jorge, A.M., Gama, J.: Online bagging for recommender systems. Expert Syst. 35(4) (2018). https://doi.org/10.1111/exsy.12303

    Article  Google Scholar 

  23. Wickramaratna, J., Holden, S.B., Buxton, B.F.: Performance degradation in boosting. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 11–21. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-48219-9_2

    Chapter  Google Scholar 

  24. Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992)

    Article  Google Scholar 

Download references

Acknowledgments

This work is financed by the European Regional Development Fund (ERDF), through the Incentive System to Research and Technological development, within the Portugal2020 Competitiveness and Internationalization Operational Program – COMPETE 2020 – within project PushNews (POCI-01- 0247-FEDER-0024257). The work is also financed by the ERDF through COMPETE 2020 within project POCI-01-0145-FEDER-006961, and by national funds through the Portuguese Foundation for Science and Technology (FCT) as part of project UID/EEA/50014/2013.

Author information

Authors and Affiliations

  1. FCUP - University of Porto, Porto, Portugal

    João Vinagre & Alípio Mário Jorge

  2. FEP - University of Porto, Porto, Portugal

    João Gama

  3. LIAAD - INESC TEC, Porto, Portugal

    João Vinagre, Alípio Mário Jorge & João Gama

Authors
  1. João Vinagre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alípio Mário Jorge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. João Gama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Vinagre .

Editor information

Editors and Affiliations

  1. Goldsmiths University of London, London, UK

    Larisa Soldatova

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Joaquin Vanschoren

  3. University of Cyprus, Nicosia, Cyprus

    George Papadopoulos

  4. Università degli Studi di Bari Aldo Moro, Bari, Italy

    Michelangelo Ceci

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vinagre, J., Mário Jorge, A., Gama, J. (2018). Online Gradient Boosting for Incremental Recommender Systems. In: Soldatova, L., Vanschoren, J., Papadopoulos, G., Ceci, M. (eds) Discovery Science. DS 2018. Lecture Notes in Computer Science(), vol 11198. Springer, Cham. https://doi.org/10.1007/978-3-030-01771-2_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01771-2_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01770-5

  • Online ISBN: 978-3-030-01771-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation