Elsevier

Applied Soft Computing

Volume 114, January 2022, 108162
Applied Soft Computing

A Mixture-of-Gaussians model for estimating the magic barrier of the recommender system

https://doi.org/10.1016/j.asoc.2021.108162Get rights and content

Highlights

  • We propose a MoG model to estimate the magic barrier of recommender systems.

  • The noise parameters are estimated instead of being specified by an expert.

  • The barriers were validated by comparison with the results of SOA approaches.

Abstract

The rating data collected by the recommender system usually contains noise due to external factors such as human uncertainty and inconsistency. Such noise, usually modeled by a normal distribution, leads to a magic barrier (MGBR) to the recommender system. However, existing MGBR estimation approaches require a user-specified standard deviation of noise, or make strong assumptions about true ratings, or need additional information from experts or users. In this paper, we propose a Mixture of Gaussians (MoG) model without user intervention to handle this issue. First, the user uncertainties are modeled using MoG, which is a universal approximator for any continuous distribution. Second, we employ the expectation–maximization algorithm to determine the parameters of user uncertainty. Finally, the MGBR is computed by Bayesian formula with the parameters. Experimental results on four well-known datasets show that the MGBRs estimated by the new model are close to the results of the state-of-the-art algorithms.

Section snippets

Code metadata

Permanent link to reproducible Capsule: https://doi.org/10.24433/CO.8965551.v1.

Related work

This section first reviews the definition of the rating system. Second, we present the problem statement for collaborative filtering (CF). Finally, we analyze existing approaches for estimating MGBR.

Table 1 lists the notations used in the paper.

The estimation model with MoG noise

In this section, we first describe the estimating framework. Second, we present a MoG-based noise assumption. Third, we estimate the noise parameters related to the computation of the MGBR. Finally, we estimate the MGBR based on conditional-probability computation.

Experiments

In this section, we present the experimental scheme to answer the questions of using the MoG model to calculate the magic barriers:

  • (1)

    Given a dataset, how many Gaussian distributions are needed to approximate the user uncertainty? and

  • (2)

    Are the magic barriers estimated by the new model close to the results of the state-of-the-art algorithms?

The answers to these problems help evaluate the rationality of the proposed model.

We repeat the calculation 10 times using different initial settings of the

Conclusions and further works

We have proposed a MoG model to estimate the MGBR of recommender system in terms of MAE and RMSE. The parameter σ can be calculated directly. The model is validated by state-of-the-art recommender algorithms on the real-world datasets. Our estimated magic barriers can be used to determine how much room for improvement in existing algorithms and to evaluate the quality of the data collected by recommender system.

The estimation model takes discrete rating as input when calculating conditional

CRediT authorship contribution statement

Heng-Ru Zhang: Conceptualization, Methodology, Writing, Software. Jie Qian: Data preprocessing, Writing. Hui-Lin Qu: Validation, Editing. Fan Min: Conceptualization, Writing – reviewing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China (Grant 61902328), Natural Science Foundation of Sichuan Province, China (Grant 2019YJ0314), and Scientific Innovation Group for Youths of Sichuan Province, China (Grant 2019JDTD0017).

References (47)

  • ReasonJ.

    Human error: models and management

    Bmj

    (2000)
  • HsuM. et al.

    Neural systems responding to degrees of uncertainty in human decision-making

    Science

    (2005)
  • SaidA. et al.

    Coherence and inconsistencies in rating behavior: estimating the magic barrier of recommender systems

    User Model User-Adapt Int

    (2018)
  • GunesI. et al.

    Shilling attacks against recommender systems: a comprehensive survey

    Artif. Intell. Rev.

    (2014)
  • SaidA. et al.

    Estimating the magic barrier of recommender systems: a user study

  • HammerB. et al.

    How to process uncertainty in machine learning?

  • NorthcuttC. et al.

    Confident learning: Estimating uncertainty in dataset labels

    J. Artificial Intelligence Res.

    (2021)
  • HüllermeierE. et al.

    Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

    Mach. Learn.

    (2021)
  • TeyeM. et al.

    Bayesian uncertainty estimation for batch normalized deep networks

  • ZhangH.-R. et al.

    Magic barrier estimation models for recommended systems under normal distribution

    Appl. Intell.

    (2018)
  • HerlockerJ.L. et al.

    Evaluating collaborative filtering recommender systems

    ACM Trans. Inf. Syst.

    (2004)
  • NguyenT.T. et al.

    Rating support interfaces to improve user experience and recommender accuracy

  • BellogínA. et al.

    The magic barrier of recommender systems–no magic, just ratings

  • Cited by (4)

    • Lower bound estimation of recommendation error through user uncertainty modeling

      2023, Pattern Recognition
      Citation Excerpt :

      However, the inaccuracy of these artificially specified parameters generates additional uncertainty. Theoretical methods constitute SG [8] or MoG [9] models for obtaining the rating data and followed by a probabilistic model to estimate the MGBR. However, real-world uncertainty comprises a mixture of super- and sub-Gaussian components.

    The code (and data) in this article has been certified as Reproducible by Code Ocean: (https://codeocean.com/). More information on the Reproducibility Badge Initiative is available at https://www.elsevier.com/physical-sciences-and-engineering/computer-science/journals.

    View full text