User perception of sentiment-integrated critiquing in recommender systems

Highlights

• Development of a novel critiquing-based recommender system based on feature sentiments extracted from product reviews.

• Modeling of user preferences for both static attribute values and feature sentiments.

• Design and conduction of two experiments on the developed system.

• Empirical demonstration of the system’s effectiveness in enhancing user experiences.

Abstract

Critiquing in recommender systems has been accepted as an effective feedback mechanism that allows users to incrementally refine their preferences for product attributes, especially in complex decision environments and high-investment product domains where users’ initial preferences are usually uncertain and incomplete. However, the traditional critiquing methods are limited in that they are only based on static attribute values (such as a digital camera’s screen size, effectiveness pixels, optical zoom). Considering product reviews contain other customers’ sentiments (also called opinions) expressed on some features, in this manuscript, we propose a sentiment-integrated critiquing approach, for helping users to formulate and refine their preferences. Through both before-after and within-subjects experiments, we find that the incorporation of feature sentiments into the critiquing interface can significantly improve users’ product knowledge, preference certainty, decision confidence, perceived information usefulness, and purchase intention. The results can hence be constructive for enhancing current critiquing-based recommender systems.

Introduction

In recommender systems, critiquing has been recognized as a distinct feedback mechanism to solve the popular cold-start problem in high-investment product domains (e.g., digital cameras, laptops, cars, apartments) (Chen and Pu, 2012). As users in those domains are usually new and do not have well-defined, fixed preferences initially, the critiquing system has been targeted to elicit users’ preferences for product attributes on site and allow them to incrementally refine preferences through posting critiques on the recommended product (such as “I would like something cheaper” or “with higher optical zoom” if the product is a digital camera). In this way, the system is able to improve recommendations in the next interaction cycle. Thus, in such a system, the initial user preference model does not influence the accuracy of their decision. Rather, it is the subsequent process of incremental critiquing that assists users in making more informed and confident decisions. According to prior experiments (Chen, Pu, 2006, Chen, Pu, 2007b), for a user to finally reach her/his ideal product, a number of critiquing cycles are often required. The studies from the areas of decision theory and consumer behavior also show that users are likely to construct their preferences in a context-dependent and adaptive manner during the decision process (Payne, Bettman, Johnson, 1993, Payne, Bettman, Schkade, 1999, Tversky, Simonson, 1993), and a typical buyer has some latent constraints and preferences that s/he may only become aware as s/he sees more options (Pu, Faltings, 2000, Pu, Faltings, 2002).

However, though critiquing has been popularly adopted in preference-based recommender systems (Chen, Pu, 2007c, Chen, Pu, 2010), knowledge-based recommender systems (Burke, 2000, Burke, Hammond, Young, 1997), and conversational recommender systems (McCarthy, Reilly, McGinty, Smyth, 2005, Shimazu, 2002, Smyth, McGinty, Reilly, McCarthy, 2004), the current methods are mainly based on products’ static attribute values (such as a digital camera’s screen size, effectiveness pixels, optical zoom) to elicit users’ critiques. Little work has studied whether and how other customers’ reviews could be leveraged into the critiquing interface for aiding the current user to construct her/his preferences. For example, suppose a user initially does not know the meaning of “optical zoom” when she searches for a digital camera, but after seeing the review “Nice 38X optical zoom lens for capturing beautiful close-ups of faraway action”, she may be able to specify preference for not only the optical zoom’s static value (e.g., “$>=38X$”) but also its associated sentiment (e.g., “ > 3” if the sentiment is in the range [1,5] from “least negative” to “very positive”). It hence implies that product reviews could be potentially useful for users to learn from others customers’ experiences (Aciar, Zhang, Simoff, Debenham, 2007, Kim, Srivastava, 2007, Wu, Wu, Sun, Yang, 2013), and hence possibly increase their own product knowledge and preference certainty.

Therefore, in this article, we propose a novel critiquing method that particularly extracts feature sentiments (i.e., opinions the other customers have expressed on some specific features in their reviews) and integrates them with products’ static attribute values for users to perform critiques. The user’s preference model is hence built on both static values and sentiments, for the system to compute product utility and return that with the highest utility as the recommendation in each interaction cycle. In the experiment, we report results of two user studies: before-after and within-subjects, which compared our method with the traditional critiquing system (without considering feature sentiments) in two different experimental settings. Both studies validate the superior performance of our method in terms of improving user perceptions, such as their product knowledge, preference certainty, decision confidence, perceived information usefulness, and purchase intention.

The remainder content is organized as follows. We first introduce related work in two branches, critiquing-based recommender systems and review-based recommender systems (Section 2). We then describe our method, i.e., the sentiment-integrated critiquing, in Section 3, followed by two experiments’ setup, materials, participants, and results analysis in Sections 4 and 5. Finally, we summarize our major findings and discuss their practical implications to the research field (Sections 6 and 7).