Skip to main content
SpringerLink
Log in

ADCB: Adaptive Dynamic Clustering of Bandits for Online Recommendation System

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

To deal with the insufficient feedbacks and dynamics of individual arrival and item popularity in online recommender, collaborative multi-armed bandit (MAB) schemes intentionally utilize the explicitly known or implicitly inferred social relationships among individuals to collaboratively recommend. Especially, without assuming the social relationships among individuals given, the dynamic cluster of bandits simultaneously infers the relationships, and recommends items through using the inferred relationships in multi-round interactive steps. However, the existed clustering bandit algorithms have two weakpoints: first they either fix the number of clusters in advance, or assign two individuals into the same cluster if there exists a path between two users in graph structure, which may lead to the wrongly cluster users. Second, they usually exploit only the cluster’s accumulated parameters of cluster as the inferred preference of individual in the cluster, which can’t fully accurately learn individual’s latent preference. To address issues above, we propose new clustering MAB based online recommendation methods, ADCB and ADCB+, based on adaptively splitting and merging clusters, which incrementally enforce both user-level re-assignment and cluster-level re-adjustment in recommendation rounds to efficiently and effectively learn the individuals’ preferences and their clustering structure. Especially, the proposed ADCB+ method further exploits both the accumulated cluster preference parameters and each individual’s personalized feature through the adaptively weighting of the two influences according to the number of user interactions. The experiments on three real online rating datasets (i.e., MovieLens-2k, Delicious-2k, LastFM-2k) consistently show that, in terms of the cumulative reward over recommendation rounds, and the average Click-Through-Rate, our proposed ADCB and ADCB+ schemes outperform than some existing dynamic clustering based online recommendation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749

    Article  Google Scholar 

  2. Zhang J, Wang Y, Yuan Z, Jin Q (2020) Personalized real-time movie recommendation system: practical prototype and evaluation. Tsinghua Sci Technol 25(2):180–191

    Article  Google Scholar 

  3. Cai W, Wang Y, Ma J, Jin Q (2022) CAN: Effective cross features by global attention mechanism and neural network for Ad click prediction. Tsinghua Sci Technol 27(1):186–195

    Article  Google Scholar 

  4. Belem FM, Heringer AG, Almeida JM, Goncalves MA (2019) Exploiting syntactic and neighbourhood attributes to address cold start in tag recommendation. Inform Process Manag 56(3):771–790

    Article  Google Scholar 

  5. Tang L, Jiang Y, Li L, Zeng C, Li T (2015) Personalized recommendation via parameter-Free contextual bandits. In: Proceedings of the 38th International ACM SIGIR conference on research and development in information retrieval, pp.323–332

  6. Wang Q et al (2019) Online interactive collaborative filtering using multi-armed bandit with dependent arms. IEEE Trans Knowl Data Eng 31(8):1569–1580

    Article  Google Scholar 

  7. Wu H, Guo X, Liu X (2018) Adaptive exploration-exploitation tradeoff for opportunistic bandits. In: Proceedings of the 35th international conference on machine learning, pp.5306–5314

  8. Glowacka D (2019) Bandit algorithms in recommender systems. In: Proceedings of the 13th ACM conference on recommender systems, pp. 574–575

  9. Li L, Chu W, Langford J, Schapire RE (2010) A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th international conference on World wide web, pp.661–670

  10. Li S, Wang B, Zhang S, Chen W (2016) Contextual combinatorial cascading bandits. In: Proceedings of the33rd international conference on machine learning, pp 1245–1253

  11. Cesa-Bianchi N, Gentile C, Zappella G (2013) A gang of bandits. In: Proceedings of the 27th annual conference on neural information processing systems, pp.737–745

  12. Wang M, Wu Z, Sun X et al (2019) Trust-aware collaborative filtering with a denoising autoencoder. Neural Process Lett 49:835–849

    Article  Google Scholar 

  13. Gentile C, Li S, Zappella G (2014) Online clustering of bandits. In: Proceedings of the 31th international conference on machine learning, pp.757–765

  14. Wu Q, Wang H, Gu Q, and Wang H (2016) Contextual bandits in a collaborative environment. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval, pp. 529–538

  15. Turgay E, Oner D, Tekin C (2018) Multi-objective contextual bandit problem with similarity information. In: Proceedings of the international conference on artificial intelligence and statistics, pp 1673–1681

  16. Christakopoulou K, Banerjee A (2018) Learning to interact with users: a collaborative-bandit approach. In: Proceedings of the 2018 SIAM International Conference on Data Mining, pp.612–620

  17. Heckel R, Vlachos M, Parnell T, Duenner C (2017) Scalable and interpretable product recommendations via overlapping co-clustering. In: Proceedings of the IEEE 33rd international conference on data engineering (ICDE), pp. 1033–1044

  18. Nguyen TT, Lauw HW (2014) Dynamic clustering of contextual multi-armed bandits. In: Proceedings of the 23rd ACM international conference on conference on information and knowledge management, pp 1959–1962

  19. Yang L, Liu B, Lin L, Xia F, Chen K, and Yang Q (2020) Exploring clustering of bandits for online recommendation system. In: Proceedings of the Fourteenth ACM conference on recommender systems, pp. 120–129

  20. Zhao X, Zhang W, Wang J (2013) Interactive collaborative filtering. In: Proceedings of the 22nd ACM international conference on Information and Knowledge Management, pp.1411–1420

  21. Gentile C, Li S, Kar P (2017) On context-dependent clustering of bandits. In: Proceedings of the international conference on machine learning, pp.1253–1262

  22. Li S, Karatzoglou A, Gentile C (2016) Collaborative filtering bandits. In: Proceedings of the 39th International ACM SIGIR conference on research and development in information retrieval, pp.539–548

  23. Li S, Chen W, Li S, Leung KS (2019) Improved algorithm on online clustering of bandits. In: Proceedings of the 28th international joint conference on artificial intelligence, pp.2923–2929

  24. Wang Z, Wang S, Du H, Guo H (2020) Fuzzy C-means clustering algorithm for automatically determining the number of clusters. In: Proceedings of the 16th international conference on computational intelligence and security

  25. Yu H, Liu Z, Wang G (2014) An automatic method to determine the number of clusters using decision-theoretic rough set. Int J Approx Reason 55(1):101–115

    Article  MathSciNet  MATH  Google Scholar 

  26. Li S, Zhang S (2018) Online clustering of contextual cascading bandits. In: Proceedings of the AAAI Conference on Artificial Intelligence, 32(1)

  27. Jolliffe I (2005) Principal component analysis. Wiley, Hoboken

    MATH  Google Scholar 

  28. Davies DL, Bouldin DW (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intel PAMI-1 2:224–227

    Article  Google Scholar 

Download references

Acknowledgements

The work was sponsored by QingLan Project of JiangSu Province, and JiangSu Provincial Key Research and Development Program (No. BE2020084-1).

Author information

Authors and Affiliations

  1. Nanjing University of Posts and Telecommunications, Nanjing, 210000, China

    Yufeng Wang & Weidong Zhang

  2. Hosei University, Tokyo, 184-8584, Japan

    Jianhua Ma

  3. Waseda University, Saitama, 359-1192, Japan

    Qun Jin

Authors
  1. Yufeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weidong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianhua Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qun Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yufeng Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Zhang, W., Ma, J. et al. ADCB: Adaptive Dynamic Clustering of Bandits for Online Recommendation System. Neural Process Lett 55, 1155–1172 (2023). https://doi.org/10.1007/s11063-022-10931-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10931-5

Keywords

Search

Navigation