Full length article
A concept-level approach to the analysis of online review helpfulness

https://doi.org/10.1016/j.chb.2015.12.028Get rights and content

Highlights

  • We have examined the factors that participate in online reviews helpfulness.

  • The regular, comparative and suggestive are included as qualitative aspect.

  • The numbers of concepts in opinion types are examined as quantitative factors.

  • Further challenges are discussed for coming researchers.

Abstract

Helpfulness of online reviews serves multiple needs of different Web users. Several types of factors can drive reviews' helpfulness. This study focuses on uninvestigated factors by looking at not just the quantitative factors (such as the number of concepts), but also qualitative aspects of reviewers (including review types such as the regular, comparative and suggestive reviews and reviewer helpfulness) and builds a conceptual model for helpfulness prediction. The set of 1500 reviews were randomly collected from TripAdvisor.com across multiple hotels for analysis. A set of four hypotheses were used to test the proposed model. Our results suggest that the number of concepts contained in a review, the average number of concepts per sentence, and the review type contribute to the perceived helpfulness of online reviews. The regular reviews were not statistically significant predictors of helpfulness. As a result, review types and concepts have a varying degree of impact on review helpfulness. The findings of this study can provide new insights to e-commerce retailers in understanding the importance of helpfulness of reviews.

Introduction

The development of Web 2.0 has encouraged people to express their opinions about products/services. Opinions are central to most human activities and hence, are one of the key drivers of human behaviors (Hu & Liu, 2004). These opinions can help consumers in purchase decisions (Liu, 2010). There are varieties of opinions that discuss different aspects of a purchase of a product/service. Early research on online reviews has identified and studied two types of opinions, namely (1) regular and (2) comparative (Jindal & Liu, 2006b). Witnessing exponential proliferation of reviews in recent years, along with the diversity of the uses and functions these perform, this dual classification seems too narrow. More recently, suggestive have been identified as a third type of reviews (Qazi, Raj, Tahir, Waheed, et al., 2014). In linguistic, suggestives are defined as indirect speech acts. The speech acts used to direct someone to do something in the form of a suggestion are classified as suggestives. They can be considered polite in the sense that instead of telling someone to do something directly, they present it in the form of a suggestion, which the reader is not obliged to follow (Kumar, 2011). The appearance of multiple review types (regular, comparative and suggestive) significantly contributes in making variety of consumption choices and future guidelines that enables consumers as well as retailers to make better purchase decisions and business policies.

The reviews types are defined based on their linguistic construct (Liu, 2012) that expresses different sort of information. A regular opinion is often referred to simply an opinion in the literature (Jindal & Liu, 2006b). A comparative opinion expresses a relation of similarities or differences between two or more entities (Jindal & Liu, 2006a). A suggestive opinion is defined as directing someone to do something in a polite manner (Qazi, Raj, Tahir, Cambria, & Syed, 2014). The classification of these types of reviews assigned “A” to regular, “B” to comparative and “C” to suggestive opinions (Jindal & Liu, 2006a; Qazi, Raj, Tahir, Waheed, et al., 2014). In the competitive business environment users experience difficulty in taking decisions if they only look at one aspect of a product (Ganapathibhotla and Liu, 2008, Liu, 2012). Clearly, different types of opinions carry variety of aspects, e.g. the notion of product comparisons is another aspect that is not only useful for product manufacturers, but also for potential buyers, thus helping in better decision making (Jindal & Liu, 2006b). Many studies suggest that online product reviews and related features have a significant impact on consumers' purchase decision and sales (Duan, Gu, & Whinston, 2008; Elwalda, Lü, & Ali, 2016; Forman, et al., 2008).

Among the many features associated with online product reviews, ‘review helpfulness’ is particularly important, as it represents the subjective evaluation of the review judged by others (Cao et al., 2011, Li et al., 2013). Therefore, helpful reviews improve the value of business sites, and sites containing more helpful reviews are more likely to attract buyers and consumers seeking information. Major Websites, such as Amazon.com, Tripadvisor and Yelp.com, ask readers to rate the helpfulness of the reviews of products/services and make that information available. This implies that online retail sites with more helpful reviews offer greater potential value to customers. Such reviews are useful for better and well-informed decisions, and, hence, maximize users' satisfaction (Kohli, Devaraj, & Mahmood, 2004). However, helpfulness of online reviews is a multi-faceted concept that can be driven by several types of factors based upon quantitative and qualitative measures. In the early studies, the most common practice to measure the review helpfulness was based upon the quantitative factors of reviews such as the star rating or thumbs up/down and the review length (Otterbacher, 2009, Pang et al., 2002).

More recent studies have focused on qualitative measures in addition to quantitative ones (search goods, search experience, experience, reviewer impact, reviewer and cumulative helpfulness) to explore helpfulness (Huang et al., 2015, Mudambi and Schuff, 2010). However, by looking into the multiple review types and associated vital aspects, helpfulness is quite a complex concept as one would equate quantitative measures of reviews to helpfulness, while others might consider qualitative instead. Therefore, this study was designed to extend existing research on online review helpfulness by viewing not just the quantitative factors (such as word count), but also qualitative aspects of reviews such as review types itself (including regular, comparative, suggestive reviews and cumulative helpfulness).

The study contributes to the conceptual development and understanding of the helpfulness components of reviews from a concept-level prospective. Built on the relevant online review literature, four hypotheses were proposed (H1, H1, H3 and H4) to study the proposed model for reviews' helpfulness. The dataset consisting of 1500 hotel reviews from Tripadvisor was employed to test these hypotheses. This study successfully validated the proposed model and found key factors to make an opinion helpful for readers. The results of the current research have contributed to relevant literature by providing further understanding of the morphological features (quantitative and qualitative) of reviews and their influence on helpfulness. Additionally, the findings of the paper have extended the results found in existing research (Mudambi & Schuff, 2010) by looking also at the review types (regular, comparative, and suggestive) to see whether each of those aspects influences online review helpfulness.

This paper is organized as follows: in Section 2, related work is presented; Section 3 presents the proposed model and related hypotheses; Section 4 presents the research methodology; Section 5 discusses evaluation results; Section 6 concludes the discussion; Section 7 presents conclusions and future work and Section 8 explains the implications of the study.

Section snippets

Literature review

The study of reviews is commonly termed opinion mining, defined as an interdisciplinary research field involving natural language processing, computational linguistics, and text mining (Thet, Na, & Khoo, 2010). Textual information is generally of two types: subjective and objective (Ganapathibhotla and Liu, 2008) and opinions are expressed by way of subjective expressions (Quigley, 2008).

Today opinion mining and sentiment analysis are mainly carried out at two levels: word-level and

Research model and hypotheses

The proposed research model is presented in Fig. 1. The model is based on morphological properties of reviews and review types. The model leverages on the hypothesis that different types of review are characterized by a different number of concepts, which influence their helpfulness. According to past research, wordiness can increase information diagnosticity (Johnson & Payne, 1985). Similarly, the number of concepts per review (NCR) is key in assessing the semantic information carried by a

Data collection

We collected data using the online reviews available from TripAdvisor. We retrieved 1500 customer reviews along with Author, Content, Date, Number of Reader, Number of Helpful Judgment, Overall rating from freely available data source (“The database and information system laboratory,” 2010). We parsed different sets of reviews for each hotel by removing HTML formatting and translating page contents to XML. This resulted in the separation of data into different records (reviews) and fields

Tobit analysis

We examined the effect of NCR and ANCS on helpfulness using Tobit regression. How NCR and ANCS affect helpfulness, given the type of review, was another objective of this study. To this end, we loaded the interaction terms of review type with NCR and ANCS respectively in our model. Table 3 summarizes the results for our model applied on the full sample set. Our model indicated a good fit with likelihood ratio (p = 0.000) and Efron's pseudo R2 value of 0.167.

For further analysis, we ran the

Discussion

These results are not only interesting but intuitively plausible too. By definition, comparative reviews obtained comparing two entities. Longer comparative reviews, supposedly, provide extensive information on the entity under review not only in isolation but also in relative terms by comparing it with other comparable entities (Liu, 2012). This specific feature requires them to be objectively longer than other types. Thus, the longer the comparative review the more informative it is. From the

Conclusion and future work

This study contributes to examine both qualitative and quantitative measure for their joint effects on review helpfulness. Built on relevant online review literature, four hypotheses were proposed (H1, H2, H3 and H4). Tripadvisor data set was used to test hypotheses. The data set included 1336 hotel reviews. We have tackled the problem of identifying morphological features of different types of reviews that contribute to the helpfulness of online reviews. Two quantifiable morphological factors,

Implications

The results of this study have implications for tourists, hotel managers and researchers. One practical implication is that managers and customers are able to see the most helpful evaluations of their businesses on travel blogs, websites and forums. Ideally, everyone desires to see online reviews that are perceived more helpful and useful, as such reviews add potential values to business (P.-Y. Chen et al., 2001, Zehrer et al., 2011). However, less helpful reviews can also be useful in some

Acknowledgements

This research is supported by University of Malaya under the research grant PPP for the project RP026-14AET.

References (59)

  • A. Zehrer et al.

    The perceived usefulness of blog postings: an extension of the expectancy-disconfirmation paradigm

    Tourism Management

    (2011)
  • O. Arazy et al.

    On the measurability of information quality

    Journal of the American Society for Information Science and Technology

    (2011)
  • E. Cambria et al.
  • E. Cambria et al.

    Sentic computing: A common-sense-based framework for concept-level sentiment analysis

    (2015)
  • E. Cambria et al.

    SenticNet 3: a common and common-sense knowledge base for cognition-driven sentiment analysis

  • E. Cambria et al.

    Jumping NLP curves: a review of natural language processing research

    IEEE Computational Intelligence Magazine

    (2014)
  • P.-Y. Chen et al.

    All reviews are not created equal: The disaggregate impact of reviews and reviewers at amazon. com

    (2001)
  • F.L. Cruz et al.

    A knowledge-rich approach to feature-based opinion extraction from product reviews

  • C. Danescu-Niculescu-Mizil et al.

    How opinions are received by online communities: a case study on amazon. com helpfulness votes

  • C. Forman et al.

    Examining the relationship between reviews and sales: the role of reviewer identity disclosure in electronic markets

    Information Systems Research

    (2008)
  • M. Ganapathibhotla et al.

    Mining opinions in comparative sentences

  • A. Gangemi et al.

    Frame-based detection of opinion holders and topics: a model and a tool

    Computational Intelligence Magazine, IEEE

    (2014)
  • B.A. Garner

    Legal writing in plain english: A text with exercises

    (2001)
  • S.L. Golicic et al.

    Examining market information and brand equity through Resource-Advantage theory: a carrier perspective

    Journal of Business Logistics

    (2012)
  • S. Hariharan et al.

    Opinion mining and summarization of reviews in web forums

  • W. Hart-Davidson et al.

    A method for measuring helpfulness in online peer review

  • F. Hou et al.

    Mining chinese comparative sentences by semantic role labeling

  • M. Hu et al.

    Mining and summarizing customer reviews

  • N. Jindal et al.

    Identifying comparative sentences in text documents

  • Cited by (153)

    • Review helpfulness prediction on e-commerce websites: A comprehensive survey

      2023, Engineering Applications of Artificial Intelligence
    • Exploring commonly used terms from online reviews in the fashion field to predict review helpfulness

      2023, International Journal of Information Management Data Insights
    View all citing articles on Scopus
    View full text