skip to main content
10.1145/2507157.2507188acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
research-article

Rating support interfaces to improve user experience and recommender accuracy

Published: 12 October 2013 Publication History

Abstract

One of the challenges for recommender systems is that users struggle to accurately map their internal preferences to external measures of quality such as ratings. We study two methods for supporting the mapping process: (i) reminding the user of characteristics of items by providing personalized tags and (ii) relating rating decisions to prior rating decisions using exemplars. In our study, we introduce interfaces that provide these methods of support. We also present a set of methodologies to evaluate the efficacy of the new interfaces via a user experiment. Our results suggest that presenting exemplars during the rating process helps users rate more consistently, and increases the quality of the data.

References

[1]
G. Adomavicius, J. Bockstedt, S. Curley, and J. Zhang. Recommender systems, consumer preferences, and anchoring effects. InDecisions@RecSys Workshop, pages 35--42, Chicago, 2011.
[2]
X. Amatriain, J. M. Pujol, N. Tintarev, and N. Oliver. Rate it again: increasing recommendation accuracy by user re-rating. In Proc. of RecSys '09, pages 173--180, New York, NY, USA, 2009. ACM.
[3]
J. R. Bettman, M. F. Luce, and J. W. Payne. Constructive consumer choice processes. Journal of Consumer Research, 25(3):187 -- 217, Dec. 1998. ArticleType: research-article /Full publication date: December 1998 / Copyright 1998 Journal of Consumer Research Inc.
[4]
D. Bollen, M. Graus, and M. C. Willemsen. Remembering the stars': effect of time on preference retrieval from memory. In Proc. of RecSys '12, pages 217 -- 220, New York, NY, USA, 2012. ACM.
[5]
D. Cosley, S. K. Lam, I. Albert, J. A. Konstan, and J. Riedl. Is seeing believing?: how recommender system interfaces affect users' opinions. In In Proc. of CHI '03, pages 585--592, New York, NY, USA, 2003. ACM.
[6]
M. D. Ekstrand, M. Ludwig, J. A. Konstan, and J. T. Riedl. Rethinking the recommender research ecosystem: reproducibility, openness, and LensKit. In In Proc. of RecSys '11, pages 133 -- 140, New York, NY, USA, 2011. ACM.
[7]
B. Fischho. Value elicitation: Is there anything in there? American Psychologist, 46(8):835--847, 1991.
[8]
F. M. Harper, X. Li, Y. Chen, and J. A. Konstan. An economic model of user rating in an online recommender 7 http://www.net ixprize.com system. In User Modeling 2005, pages 307--316. Springer, 2005.
[9]
J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1):5--53, 2010.
[10]
W. Hill, L. Stead, M. Rosenstein, and G. Furnas. Recommending and evaluating choices in a virtual community of use. In In Proc. of CHI '95, pages 194--201, New York, NY, USA, 1995.
[11]
C. K. Hsee. The evaluability hypothesis: An explanation for preference reversals between joint and separate evaluations of alternatives. Organizational Behavior and Human Decision Processes, 67(3):247--257, Sept. 1996.
[12]
L.-t. Hu and P. Bentler. Cuto criteria for t indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1):1--55, 1999.
[13]
D. Kluver, T. T. Nguyen, M. Ekstrand, S. Sen, and J. Riedl. How many bits per rating? In In Proc. of RecSys '12, pages 99 -- 106, New York, NY, USA, 2012. ACM.
[14]
B. P. Knijnenburg, M. C. Willemsen, Z. Gantner, H. Soncu, and C. Newell. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, 22(4-5):441--504, Mar. 2012.
[15]
S. Lichtenstein and P. Slovic. The Construction of Preference. Cambridge University Press, Sept. 2006.
[16]
S. M. McNee, S. K. Lam, J. A. Konstan, and J. Riedl. Interfaces for eliciting new user preferences in recommender systems. In P. Brusilovsky, A. Corbett, and F. d. Rosis, editors, User Modeling 2003, number 2702 in Lecture Notes in Computer Science, pages 178--187. Springer Berlin Heidelberg, Jan. 2003.
[17]
T. Mussweiler. Comparison processes in social judgment: Mechanisms and consequences. Psychological Review, 110(3):472--489, 2003.
[18]
T. T. Nguyen and J. Riedl. Predicting users' preference from tag relevance. User Modeling, Adaptation, and Personalization, UMAP 2013, pages 274--280, 2013.
[19]
S. Nobarany, L. Oram, V. K. Rajendran, C.-H. Chen, J. McGrenere, and T. Munzner. The design space of opinion measurement interfaces: exploring recall support for rating and ranking. In Proc. of CHI '12, pages 2035 -- 2044, New York, NY, USA, 2012. ACM.
[20]
M. P. O'Mahony, N. J. Hurley, and G. C. Silvestre. Detecting noise in recommender system databases. In Proceedings of the 11th international conference on Intelligent user interfaces, IUI '06, pages 109 -- 115, New York, NY, USA, 2006. ACM.
[21]
A. Said, B. J. Jain, S. Narr, and T. Plumbaum. Users and noise: The magic barrier of recommender systems. In UMAP 2012, pages 237--248. Springer, 2012.
[22]
I. Simonson. Determinants of customers' responses to customized offers: Conceptual framework and research propositions. Journal of Marketing, 69(1):32--45, Jan. 2005.
[23]
E. I. Sparling and S. Sen. Rating: how difficult is it? In Proc. of RecSys '11, pages 149 -- 156, New York, NY, USA, 2011. ACM.
[24]
A. Tversky and D. Kahneman. Judgment under uncertainty: Heuristics and biases. Science, 185(4157):1124--1131, Sept. 1974.
[25]
A. Tversky, S. Sattath, and P. Slovic. Contingent weighting in judgment and choice. Psychological Review, 95(3):371--384, 1988.
[26]
J. Vig, S. Sen, and J. Riedl. The tag genome: Encoding community knowledge to support novel interaction. ACM Trans. Interact. Intell. Syst., 2(3):13:1 -- 13:44, Sept. 2012.
[27]
E. U. Weber and E. J. Johnson. Mindful judgment and decision making. In Annual Review of Psychology, volume 60, pages 53--85. Annual Reviews, Palo Alto, 2009.

Cited By

View all
  • (2024)On the Negative Perception of Cross-domain Recommendations and ExplanationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657735(2102-2113)Online publication date: 10-Jul-2024
  • (2023)Less Can Be More: Exploring Population Rating Dispositions with Partitioned Models in Recommender SystemsAdjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3563359.3597390(291-295)Online publication date: 26-Jun-2023
  • (2023)Lower bound estimation of recommendation error through user uncertainty modelingPattern Recognition10.1016/j.patcog.2022.109171136:COnline publication date: 1-Apr-2023
  • Show More Cited By

Index Terms

  1. Rating support interfaces to improve user experience and recommender accuracy

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    RecSys '13: Proceedings of the 7th ACM conference on Recommender systems
    October 2013
    516 pages
    ISBN:9781450324090
    DOI:10.1145/2507157
    • General Chairs:
    • Qiang Yang,
    • Irwin King,
    • Qing Li,
    • Program Chairs:
    • Pearl Pu,
    • George Karypis
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 October 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. cognitive load
    2. experimentation
    3. human factors
    4. interface design
    5. preference elicitation
    6. recommendation accuracy
    7. user experience

    Qualifiers

    • Research-article

    Conference

    RecSys '13
    Sponsor:

    Acceptance Rates

    RecSys '13 Paper Acceptance Rate 32 of 136 submissions, 24%;
    Overall Acceptance Rate 254 of 1,295 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)51
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 26 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)On the Negative Perception of Cross-domain Recommendations and ExplanationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657735(2102-2113)Online publication date: 10-Jul-2024
    • (2023)Less Can Be More: Exploring Population Rating Dispositions with Partitioned Models in Recommender SystemsAdjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3563359.3597390(291-295)Online publication date: 26-Jun-2023
    • (2023)Lower bound estimation of recommendation error through user uncertainty modelingPattern Recognition10.1016/j.patcog.2022.109171136:COnline publication date: 1-Apr-2023
    • (2023)Modeling users’ heterogeneous taste with diversified attentive user profilesUser Modeling and User-Adapted Interaction10.1007/s11257-023-09376-934:2(375-405)Online publication date: 1-Aug-2023
    • (2022)The Tag Genome Dataset for BooksProceedings of the 2022 Conference on Human Information Interaction and Retrieval10.1145/3498366.3505833(353-357)Online publication date: 14-Mar-2022
    • (2022)Rating consistency is consistently underratedProceedings of the 37th ACM/SIGAPP Symposium on Applied Computing10.1145/3477314.3507270(1355-1364)Online publication date: 25-Apr-2022
    • (2022)A Mixture-of-Gaussians model for estimating the magic barrier of the recommender system▪Applied Soft Computing10.1016/j.asoc.2021.108162114:COnline publication date: 1-Jan-2022
    • (2021)Effective Strategies for Crowd-Powered Cognitive Reappraisal Systems: A Field Deployment of the Flip*Doubt Web Application for Mental HealthProceedings of the ACM on Human-Computer Interaction10.1145/34795615:CSCW2(1-37)Online publication date: 18-Oct-2021
    • (2021)Every Cloud Has a Silver Lining: Exploring Experiential Knowledge and Assets of Family CaregiversProceedings of the ACM on Human-Computer Interaction10.1145/34795605:CSCW2(1-25)Online publication date: 18-Oct-2021
    • (2021)Task Execution Quality Maximization for Mobile Crowdsourcing in Geo-Social NetworksProceedings of the ACM on Human-Computer Interaction10.1145/34760535:CSCW2(1-29)Online publication date: 18-Oct-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media