Skip to main content
SpringerLink
Log in

Magic barrier estimation models for recommended systems under normal distribution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Real data are usually imperfect. The inherent nature of data determines the magical barrier of a machine learning task. In this paper, we propose three normal distribution models to estimate the magic barrier of recommender systems in terms of mean absolute error (MAE). The first model assumes that the users’ ratings are all subject to the same normal distribution. The second assumes that there are different appropriate standard deviation settings for different rating levels. The third divides users into different groups and assumes that the settings for the standard deviations are related to the rating levels and the groups of users. In this way, the latter models are more realistic than the former ones. Experimental results on three well-known datasets show that three models are consistent since the estimated values are close to each other. Popular recommendation algorithms also approach the magic barriers closely.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Pearson RK (2005) Mining imperfect data: Dealing with contamination and incomplete records society for industrial and applied mathematics

  2. Simmhan YL, Plale B, Gannon D (2005) A survey of data provenance in e-science. Acm Sigmod Record 34(3):31–36

    Article  Google Scholar 

  3. Denegar CR, Ball DW (1993) Assessing reliability and precision of measurement: an introduction to intraclass correlation and standard error of measurement. J Sport Rehabil 2(1):35– 42

    Article  Google Scholar 

  4. Taghva K, Borsack J, Condit A, Erva S (1994) The effects of noisy data on text retrieval. Journal of the Association for Information Science & Technology 45(1):50–58

    Google Scholar 

  5. Ferris JA (2009) Missing data: a gentle introduction. Drug & Alcohol Review 28(1):90–91

    Article  Google Scholar 

  6. Jasberg K, Sizov S (2017) The magic barrier revisited: Accessing natural limitations of recommender assessment

  7. Amatriain X, Pujol JM, Oliver N (2009) I like it... i like it not: Evaluating user ratings noise in recommender systems. In: International conference on user modeling, adaptation, and personalization. Springer, pp 247-258

  8. O’Mahony MP, Hurley NJ, Silvestre G (2006) Detecting noise in recommender system databases. In: Proceedings of the 11th international conference on Intelligent user interfaces. ACM, pp 109–115

  9. Said A, Jain BJ, Narr S, Plumbaum T (2012) Users and noise: The magic barrier of recommender systems. In: International conference on user modeling, adaptation, and personalization. Springer pp 237-248

  10. Said A, Jain BJ, Narr S, Plumbaum T, Albayrak S, Scheel C (2012) Estimating the magic barrier of recommender systems: a user study. In: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 1061–1062

  11. Bellogín A, Said A, de Vries AP (2014) The magic barrier of recommender systems–no magic, just ratings. In: International Conference on User Modeling, Adaptation, and Personalization. Springer pp 25–36

  12. And KYL, Zeger SL (1993) Regression analysis for correlated data. Annu Rev Public Health 14(1):43–68

    Article  Google Scholar 

  13. Rosipal R, Girolami M, Trejo LJ, Cichocki A (2001) Kernel pca for feature extraction and de-noising in nonlinear regression. Neural Comput & Applic 10(3):231–243

    Article  Google Scholar 

  14. Zhang HR, Min F, Shi B (2017) Regression-based three-way recommendation. Inf Sci 378:444–461

    Article  Google Scholar 

  15. Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. MIT Press, Cambridge

    Google Scholar 

  16. Joachims T (1999) Transductive inference for text classification using support vector machines. In: The 16th International Conference on Machine Learning, pp 200–209

  17. Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. J R Stat Soc 28 (1):100–108

    MATH  Google Scholar 

  18. Rodriguez A, Laio A (2014) Clustering by fast search and find of density peaks. Science 344(6191):1492–1496

    Article  Google Scholar 

  19. Bobadilla J, Ortega F, Hernando A, Gutiérrez A (2013) Recommender systems survey. Knowl-Based Syst 46:109–132

    Article  Google Scholar 

  20. Zhang J, Lin YJ, Lin ML, Liu JH (2016) An effective collaborative filtering algorithm based on user preference clustering. Appl Intell 45(2):230–240

    Article  Google Scholar 

  21. Murua A (2002) Upper bounds for error rates of linear combinations of classifiers. IEEE Transactions on Pattern Analysis & Machine Intelligence 24(5):591–602

    Article  Google Scholar 

  22. Herlocker JL, Konstan JA, Terveen LG, Riedl JT (2004) Evaluating collaborative filtering recommender systems. ACM Trans Inf Syst 22(1):5–53

    Article  Google Scholar 

  23. Hill W, Stead L, Rosenstein M, Furnas G (1995) Recommending and evaluating choices in a virtual community of use. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press/Addison-Wesley Publishing Co, pp 194–201

  24. Pewsey A (2000) The wrapped skew-normal distribution on the circle. Communications in Statistics-Theory and Methods 29(11):2459–2472

    Article  MathSciNet  Google Scholar 

  25. Huang JJ, Zhu KL, Zhong N (2016) A probabilistic inference model for recommender systems. Appl Intell 45(3):686–694

    Article  Google Scholar 

  26. Stein CM (1981) Estimation of the mean of a multivariate normal distribution. Ann Stat 9(6):1135–1151

    Article  MathSciNet  Google Scholar 

  27. Kannan R, Ishteva M, Park H (2014) Bounded matrix factorization for recommender system. Knowl Inf Syst 39(3):491–511

    Article  Google Scholar 

  28. Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web. ACM, pp 285–295

  29. Zheng M, Min F, Zhang HR, Chen WB (2016) Fast recommendations with the m-distance. IEEE Access 4:1464–1468

    Article  Google Scholar 

  30. Desrosiers C, Karypis G (2011) A comprehensive survey of neighborhood-based recommendation methods. In: Recommender systems handbook. Springer, pp 107–144

  31. Zhang HR, Min F (2016) Three-way recommender systems based on random forests. Knowl-Based Syst 91:275–286

    Article  Google Scholar 

  32. Herlocker JL, Konstan JA, Riedl JT (2002) An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inf Retr 5(4):287–310

    Article  Google Scholar 

  33. Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the 4th conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 43–52

  34. Schafer J, Frankowski D, Herlocker J, Sen S (2007) Collaborative filtering recommender systems. The Adaptive Web 4321:291–324

    Article  Google Scholar 

  35. Xie HR, Wang DD, Rao YH, Wong TL, Raymond LY, Chen L, Wang FL (2018) Incorporating user experience into critiquing-based recommender systems: a collaborative approach based on compound critiquing. Int J Mach Learn Cybern 9(5):837–852

    Article  Google Scholar 

  36. Jiang XM, Song WG, Feng WG (2006) Optimizing collaborative filtering by interpolating the individual and group behaviors. Frontiers of WWW Research and Development-APWeb 2006, pp 568–578

  37. Ghauth KIB, Abdullah NA (2009) Building an e-learning recommender system using vector space model and good learners average rating. In: The 9th IEEE International Conference on Advanced Learning Technologies. IEEE, pp 194–196

  38. Cremonesi P, Koren Y, Turrin R (2010) Performance of recommender algorithms on top-n recommendation tasks. In: Proceedings of the 4th ACM conference on Recommender systems. ACM, pp 39–46

  39. Deshpande M, Karypis G (2004) Item-based top-n recommendation algorithms. ACM Trans Inf Syst 22 (1):143–177

    Article  Google Scholar 

  40. Karypis G (2001) Evaluation of item-based top-n recommendation algorithms. In: Proceedings of the 10th international conference on Information and knowledge management. ACM, pp 247–254

  41. Li JJ, Sun LM, Wang J (2011) A slope one collaborative filtering recommendation algorithm using uncertain neighbors optimizing. In: International Conference on Web-Age Information Management. Springer, pp 160–166

  42. Lemire D, Maclachlan A (2005) Slope one predictors for online rating-based collaborative filtering. In: Proceedings of the 2005 SIAM International Conference on Data Mining. SIAM, pp 471-475

    Chapter  Google Scholar 

  43. Gheisari S, Meybodi M, Dehghan M, Ebadzadeh M (2017) Bayesian network structure training based on a game of learning automata. Int J Mach Learn Cybern 8(4):1093–1105

    Article  Google Scholar 

  44. Krzysztofowicz R (1999) Bayesian theory of probabilistic forecasting via deterministic hydrologic model. Water Resour Res 35(9):2739–2750

    Article  Google Scholar 

  45. Montgomery DC, Runger GC (2010) Applied statistics and probability for engineers. Wiley, New York

    MATH  Google Scholar 

  46. Bathe KJ, Wilson EL (1976) Numerical methods in finite element analysis 31(139):6–12

  47. Qian YH, Zhang H, Sang YL, Liang JY (2014) Multigranulation decision-theoretic rough sets. Int J Approx Reason 55(1):225–237

    Article  MathSciNet  Google Scholar 

  48. Chen YN, Liu RF, Xu WR (2016)

  49. Chen JM, Tang Y, Li JG, Xiao J, Jiang WL (2014) Regularized matrix factorization with cognition degree for collaborative filtering. In: International Conference on Human Centered Computing. Springer, pp 300–310

  50. Zheng J, Liu J, Shi C, Zhuang FZ, Li JZ, Wu B (2016) Dual similarity regularization for recommendation. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, pp 542–554

Download references

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China (Grants 61379089, 41604114), the Innovation and Entrepreneurship Foundation of Southwest Petroleum University (Grant SWPUSC16-003), and the Natural Science Foundation of the Department of Education of Sichuan Province (Grant 16ZA0060).

Author information

Authors and Affiliations

  1. School of Computer Science, Southwest Petroleum University, Chengdu, 610500, China

    Heng-Ru Zhang, Fan Min, Yan-Xue Wu, Zhuo-Lin Fu & Lei Gao

Authors
  1. Heng-Ru Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan-Xue Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhuo-Lin Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Min.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, HR., Min, F., Wu, YX. et al. Magic barrier estimation models for recommended systems under normal distribution. Appl Intell 48, 4678–4693 (2018). https://doi.org/10.1007/s10489-018-1237-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1237-8

Keywords

Search

Navigation