Abstract
In this work, we focus on online review systems, in which users provide opinions about a set of entities (movies, restaurants, etc.) based on their experiences and in turn can check what others prefer. These systems have been proved to be sensitive to fraud and have shown some shortcomings as a result of capturing opinions through numerical ratings. Thus, supported by recent work on the field, we tackle the problem of fraud in such systems by designing a mechanism based on pairwise comparisons, coupled with an incentive policy attempting to foster the collection of majority opinions over individual experiences. As a result, we propose a new mechanism called iPWRM (incentive-based PWRM), where users are persuaded to reply honestly to pairwise queries based on opinion polls. The idea is: (1) to give a positive reward when all users agree in their reviews; (2) to give a positive reward when a user agrees the majority’s choice; and finally, (3) to give a low incentive—possibly null—when user’s review does not match the majority. Therefore, it is able (1) to overcome the bias introduced into reputation rankings by fraud reviews in ORSs, as well as (2) to mitigate potential biased problems derived from the use of numerical ratings. We exhaustively test the performance of the mechanism by using two different well-known existing datasets Flixster and HetRec2011—real world datasets on movie reviews, aiming to test the performance of the mechanism as well as the effectiveness and efficiency of iPWRM when fraud comes into play.
Similar content being viewed by others
Notes
For every match, a different set of users may be chosen. Moreover, note the size of this subset is very small compared to the size of the potential users that might be queried.
It is important to remark that reward does not have to be necessarily money based, but it might be points, virtual money, or any other resource considered as valuable for users.
It is assumed users always reply a match query.
References
Akoglu L, Chandy R, Faloutsos C (2013) Opinion fraud detection in online reviews by network effects. In: ICWSM
Balakrishnan S, Chopra S (2010) Two of a kind or the ratings game? Adaptive pairwise preferences and latent factor models. In: ICDM, pp 725–730
Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci 99(suppl 3):7280–7287
Centeno R, Hermoso R, Fasli M (2014) On the inaccuracy of numerical ratings: dealing with biased opinions in social networks. Inf Syst Front 17:1–17
Chen K.-Y, Fine LR, Huberman BA (2001) Forecasting uncertain events with small groups. In: ACM EC, pp 58–64
Dalvi N, Kumar R, Pang B (2013) Para ’normal’ activity: on the distribution of average ratings. In: ICWSM, pp 110–119
Dellarocas C (2000) Mechanisms for coping with unfair ratings and discriminatory behavior in online reputation reporting systems. In: ICIS, pp 520–525
De Meo P, Musial-Gabrys K, Rosaci D, Sarnè GML, Aroyo L (2017) Using centrality measures to predict helpfulness-based reputation in trust networks. ACM Trans Internet Technol 17:8:1–8:20. doi:10.1145/2981545
Duan W, Gu B, Whinston AB (2008) Do online reviews matter? An empirical investigation of panel data. Decis Support Syst 45(4):1007–1016
Fang H, Zhang J, Bao Y, Zhu Q (2013) Towards effective online review systems in the chinese context: a cross-cultural empirical study. Electron Commer Res Appl 12(3):208–220
Farahbakhsh R, Ángel C, Crespi N (2015) Characterization of cross-posting activity for professional users across major osns. In: 2015 IEEE/ACM international conference on advances in social networks analysis and mining, pp 645–650
Forsythe R, Rietz TA, Ross TW (1999) Wishes, expectations and actions: a survey on price formation in election stock markets. J Econ Behav Organ 39(1):83–110
Golbeck J, Hansen D (2014) A method for computing political preference among twitter followers. Soc Netw 36:177–184 (Special Issue on Political Networks)
Hermoso R, Centeno R, Fasli M (2013) From blurry numbers to clear preferences: a mechanism to extract reputation in social networks. Expert Syst Appl 41(5):2269–2285
Hu N, Bose I, Koh NS, Liu L (2012) Manipulation of online reviews: an analysis of ratings, readability, and sentiments. Decis Support Syst 52(3):674–684
Hu N, Liu L, Sambamurthy V (2011) Fraud detection in online consumer reviews. Decis Support Syst 50(3):614–626
Huynh TD, Jennings NR, Shadbolt NR (2004) Fire: an integrated trust and reputation model for open multi-agent systems. In: Proceedings of the 16th European conference on artificial intelligence, ECAI’04. IOS Press, Amsterdam, pp 23–27
Huynh TD, Jennings NR, Shadbolt NR (2006) An integrated trust and reputation model for open multi-agent systems. Auton Agents Multi Agent Syst 13(2):119–154
Ismail R, Josang A (2002) The beta reputation system. In: BLED 2002 proceedings, p 41
Jaramillo JJ, Srikant R (2010) A game theory based reputation mechanism to incentivize cooperation in wireless ad hoc networks. Ad Hoc Netw 8(4):416–429
Jindal N, Liu B (2008) Opinion spam and analysis. In: Proceedings of the 2008 international conference on web search and data mining. ACM, pp 219–230
Jøsang A, Ismail R (2002) The beta reputation system. In: Bled eConference
Jøsang A, Ismail R, Boyd C (2007) A survey of trust and reputation systems for online service provision. Decis Support Syst 43(2):618–644
Jurca R, Faltings B (2008) Incentives for expressing opinions in online polls. In: ACM EC, pp 119–128
Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1/2):81–93
Khosravifar B, Bentahar J, Gomrokchi M, Alam R (2012) CRM: an efficient trust and reputation model for agent computing. Knowl Based Syst 30:1–16
Koh NS, Hu N, Clemons EK (2010) Do online reviews reflect a products true perceived quality? An investigation of online movie reviews across cultures. Electron Commer Res Appl 9(5):374–385
Kost A (2012) Woman paid to post five-star google feedback. http://www.thedenverchannel.com/news/woman-paid-to-post-five-star-google-feedback
Koutsoupias E, Papadimitriou C (1999) Worst-case equilibria. In: Proceedings of the 16th annual conference on theoretical aspects of computer science, STACS’99. Springer, Berlin, pp 404–413
Lerman K, Hogg T (2014) Leveraging position bias to improve peer recommendation. PLOS ONE 9(6):1–8
Lewis-Beck MS, Skalaban A (1989) Citizen forecasting: can voters see into the future? Br J Polit Sci 19:146–153
Lim E-P, Nguyen V-A, Jindal N, Liu B, Lauw HW (2010) Detecting product review spammers using rating behaviors. In: Proceedings of the 19th ACM international conference on Information and knowledge management. ACM, pp 939–948
Moon S, Bergey PK, Iacobucci D (2010) Dynamic effects among movie ratings, movie revenues, and viewer satisfaction. J Mark 74(1):108–121
Negahban S, Oh S, Shah D (2012) Iterative ranking from pair-wise comparisons. In: NIPS, pp 2483–2491
Pinyol I, Sabater-Mir J (2013) Computational trust and reputation models for open multi-agent systems: a review. Artif Intell Rev 40(1):1–25
Prelec D (2004) A bayesian truth serum for subjective data. Science 306(5695):462–466
Ramchurn SD, Huynh D, Jennings NR (2004) Trust in multi-agent systems. Knowl Eng Rev 19(1):1–25
Resnick P, Kuwabara K, Zeckhauser R, Friedman E (2000) Reputation systems. Commun ACM 43(12):45–48
Sabater J, Sierra C (2001) Regret: reputation in gregarious societies. In: Proceedings of the fifth international conference on autonomous agents, AGENTS ’01. ACM, New York, pp 194–195. doi:10.1145/375735.376110
Sparling EI, Sen S (2011) Rating: how difficult is it?. In: RecSys, pp 149–156
Streitfeld D (2011) In a race to out-rave, 5-star web reviews go for $5’. http://www.nytimes.com/2011/08/20/technology/finding-fake-reviews-online.html
Teacy WL, Luck M, Rogers A, Jennings NR (2012) An efficient and versatile approach to trust and reputation using hierarchical bayesian modelling. Artif Intell 193:149–185
Teacy WTL, Patel J, Jennings NR, Luck M (2006) Travos: Trust and reputation in the context of inaccurate information sources. Auton Agents Multi Agent Syst 12(2):183–198
Vu T, Altman A, Shoham Y (2009) On the complexity of schedule control problems for knockout tournaments. In: Proceedings of AAMAS’09, vol 1 of AAMAS ’09. IFAAMAS, Richland, pp 225–232
Vu T, Shoham Y (2011) Fair seeding in knockout tournaments. ACM Trans Intell Syst Technol 3(1):9:1–9:17
Wang Y, Vassileva J (2007) Toward trust and reputation based web service selection: a survey. Int Trans Syst Sci Appl 3(2):118–132
Witkowski J, Bachrach Y, Key P, Parkes DC (2013) Dwelling on the negative: incentivizing effort in peer prediction. In: HCOMP
Wooldridge M (2009) An introduction to multiagent systems, 2nd edn. Wiley Publishing, London
Xie S, Wang G, Lin S, Yu PS (2012) Review spam detection via temporal pattern discovery. In: KDD, pp 823–831
Yan Q, Wang Q, Liu X (2014) Research on the interactive effects of online scores. Electron Commer Res Appl 13(6):402–408
Yu B, Singh MP (2002) An evidential model of distributed reputation management. In AAMAS, pp 294–301
Acknowledgements
We thank anonymous reviewers for their very useful comments and suggestions. The work was supported by eMadrid project S2013-ICE-2715, Spanish Ministry of Economy and Competitiveness (TIN2012-36586-C03-02-iHAS) and by the Autonomous Region of Madrid (P2013/ICE-3019-MOSI-AGIL-CM, co-funded by EU-FSE and FEDER”).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Centeno, R., Hermoso, R. Estimating global opinions by keeping users from fraud in online review systems. Knowl Inf Syst 55, 467–491 (2018). https://doi.org/10.1007/s10115-017-1089-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-017-1089-2