Abstract
The collection and exploitation of ratings from users are modern pillars of collaborative filtering. Likert scale is a psychometric quantifier of ratings popular among the electronic commerce sites. In this paper, we consider the tasks of collecting Likert scale ratings of items and of finding the n-k best-rated items, i.e., the n items that are most likely to be the top-k in a ranking constructed from these ratings. We devise an algorithm, Pundit, that computes the n-k best-rated items. Pundit uses the probability-generating function constructed from the Likert scale responses to avoid the combinatorial exploration of the possible outcomes and to compute the result efficiently. Selection of the best-rated items meets, in practice, the major obstacle of the scarcity of ratings. We propose an approach that learns from the available data how many ratings are enough to meet a prescribed error. We empirically validate with real datasets the effectiveness of our method to recommend the collection of additional ratings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
http://www.comp.nus.edu.sg/~liuqing/tech-reports/TRB6-17-likert.pdf
Chen, X., Bennett, P.N., Collins-Thompson, K., Horvitz, E.: Pairwise ranking aggregation in a crowdsourced setting. In: WSDM, pp. 193–202 (2013)
Guo, S., Parameswaran, A., Garcia-Molina, H.: So who won?: dynamic max discovery with the crowd. In: SIGMOD, pp. 385–396 (2012)
Hua, M., Pei, J., Zhang, W., Lin, X.: Ranking queries on uncertain data: a probabilistic threshold approach. In: SIGMOD, pp. 673–686 (2008)
Jamieson, S., et al.: Likert scales: how to (ab)use them. Med. Educ. 38(12), 1217–1218 (2004)
Jin, R., Si, L., Zhai, C., Callan, J.: Collaborative filtering with decoupled models for preferences and ratings. In: CIKM, pp. 309–316. ACM (2003)
Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)
Lee, J., Lee, D., Lee, Y.C., Hwang, W.S., Kim, S.W.: Improving the accuracy of top-n recommendation using a preference model. Inf. Sci. 348, 290–304 (2016)
Li, J., Deshpande, A.: Ranking continuous probabilistic datasets. VLDB 3(1–2), 638–649 (2010)
Li, J., Saha, B., Deshpande, A.: A unified approach to ranking in probabilistic databases. VLDB 2(1), 502–513 (2009)
Liu, Q., Basu, D., Abdessalem, T., Bressan, S.: Top-\(k\) queries over uncertain scores. In: Debruyne, C., et al. (eds.) OTM 2016. LNCS, vol. 10033, pp. 245–262. Springer, Cham (2016). doi:10.1007/978-3-319-48472-3_14
McAuley, J., Pandey, R., Leskovec, J.: Inferring networks of substitutable and complementary products. In: SIGKDD, pp. 785–794 (2015)
McAuley, J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: SIGIR, pp. 43–52 (2015)
Niu, S., Lan, Y., Guo, J., Cheng, X., Yu, L., Long, G.: Listwise approach for rank aggregation in crowdsourcing. In: WSDM, pp. 253–262 (2015)
Robert, C.P., Casella, G.: Monte Carlo Statistical Methods. Springer, New York (2004). doi:10.1007/978-1-4757-4145-2
Soliman, M.A., Ilyas, I.F., Ben-David, S.: Supporting ranking queries on uncertain and incomplete data. VLDB 19(4), 477–501 (2010)
Ye, P., Doermann, D.: Combining preference and absolute judgements in a crowd-sourced setting. In: ICML, pp. 1–7 (2013)
Zhang, X., Li, G., Feng, J.: Crowdsourced top-\(k\) algorithms: an experimental evaluation. VLDB 9(8), 612–623 (2016)
Acknowledgement
This work is supported by the National University of Singapore under a grant from Singapore Ministry of Education for research project number T1 251RES1607 and is partially funded by the Big Data and Market Insights Chair of Télécom ParisTech.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Liu, Q., Basu, D., Goel, S., Abdessalem, T., Bressan, S. (2017). How to Find the Best Rated Items on a Likert Scale and How Many Ratings Are Enough. In: Benslimane, D., Damiani, E., Grosky, W., Hameurlain, A., Sheth, A., Wagner, R. (eds) Database and Expert Systems Applications. DEXA 2017. Lecture Notes in Computer Science(), vol 10439. Springer, Cham. https://doi.org/10.1007/978-3-319-64471-4_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-64471-4_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64470-7
Online ISBN: 978-3-319-64471-4
eBook Packages: Computer ScienceComputer Science (R0)