Skip to main content

Advertisement

Springer Nature Link
Log in

Measurement of online review helpfulness: a formative measure development and validation

  • Published:
Electronic Commerce Research Aims and scope Submit manuscript

Abstract

Existing measures of Review Helpfulness do not systematically capture the helpfulness of a review according to the content quality of the review. In a bid to answer this issue, we have developed and validated a formative measurement instrument of review helpfulness based on key review characteristics namely: (i) feature-wise information, (ii) comparison with other brands, (iii) grammar, and (iv) timeliness of the review post. This study adopted a mixed method approach to develop the construct. Focus groups were used to elicit the review characteristics that determine review helpfulness. Pre-tests were conducted to finalize the product and brand for which online reviews were to be assessed. Study has used orthogonal design for building survey questionnaires which contained example reviews with varying levels of the review characteristics. After validating the formative measure of review helpfulness, the theoretical as well as managerial implications of the measure were also discussed in detail.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://www.brightlocal.com/research/local-consumer-review-survey.

References

  1. Hu, H. F., & Krishen, A. S. (2019). When is enough, enough? Investigating product reviews and information overload from a consumer empowerment perspective. Journal of Business Research, 100, 27–37.

    Google Scholar 

  2. Soto-Acosta, P., Molina-Castillo, F. J., Lopez-Nicolas, C., & Colomo-Palacios, R. (2014). The effect of information overload and disorganisation on intention to purchase online: The role of perceived risk and internet experience. Online Information Review.

  3. Wan, Y. (2015). The Matthew effect in social commerce. Electronic Markets, 25(4), 313–324.

    Google Scholar 

  4. Mudambi, S. M., & Schuff, D. (2010). Research note: What makes a helpful online review? A study of customer reviews on Amazon.com. MIS Quarterly, 185–200.

  5. Siering, M., Muntermann, J., & Rajagopalan, B. (2018). Explaining and predicting online review helpfulness: The role of content and reviewer-related signals. Decision Support Systems, 108, 1–12.

    Google Scholar 

  6. Malik, M. S. I., & Hussain, A. (2018). An analysis of review content and reviewer variables that contribute to review helpfulness. Information Processing & Management, 54(1), 88–104.

    Google Scholar 

  7. Mariani, M. M., & Borghi, M. (2020). Online review helpfulness and firms’ financial performance: An empirical study in a service industry. International Journal of Electronic Commerce, 24(4), 421–449.

    Google Scholar 

  8. Liu, A. X., Li, Y., & Xu, S. X. (2021). Assessing the unacquainted: Inferred reviewer personality and review helpfulness. Management Information Systems Quarterly, 45(3), 1113–1148.

    Google Scholar 

  9. Filieri, R., Raguseo, E., & Vitari, C. (2021). Extremely negative ratings and online consumer review helpfulness: The moderating role of product quality signals. Journal of Travel Research, 60(4), 699–717.

    Google Scholar 

  10. Guo, B., & Zhou, S. (2017). What makes population perception of review helpfulness: An information processing perspective. Electronic Commerce Research, 17(4), 585–608.

    Google Scholar 

  11. Wan, Y., Ma, B., & Pan, Y. (2018). Opinion evolution of online consumer reviews in the e-commerce environment. Electronic Commerce Research, 18(2), 291–311.

    Google Scholar 

  12. Huang, G. H., & Korfiatis, N. (2015). Trying before buying: The moderating role of online reviews in trial attitude formation toward mobile applications. International Journal of Electronic Commerce, 19(4), 77–111.

    Google Scholar 

  13. Kwok, L., & Xie, K. L. (2016). Factors contributing to the helpfulness of online hotel reviews: Does manager response play a role? International Journal of Contemporary Hospitality Management., 28, 2156.

    Google Scholar 

  14. Wu, J. (2017). Review popularity and review helpfulness: A model for user review effectiveness. Decision Support Systems, 97, 92–103.

    Google Scholar 

  15. Chen, Z., & Lurie, N. H. (2013). Temporal contiguity and negativity bias in the impact of online word of mouth. Journal of Marketing Research, 50(4), 463–476.

    Google Scholar 

  16. Forman, C., Ghose, A., & Wiesenfeld, B. (2008). Examining the relationship between reviews and sales: The role of reviewer identity disclosure in electronic markets. Information systems research, 19(3), 291–313.

    Google Scholar 

  17. Baek, H., Lee, S., Oh, S., & Ahn, J. (2015). Normative social influence and online review helpfulness: Polynomial modeling and response surface analysis. Journal of Electronic Commerce Research, 16(4), 290.

    Google Scholar 

  18. Schindler, R. M., & Bickart, B. (2012). Perceived helpfulness of online consumer reviews: The role of message content and style. Journal of Consumer Behaviour, 11(3), 234–243.

    Google Scholar 

  19. Freixas, X., Guesnerie, R., & Tirole, J. (1985). Planning under incomplete information and the ratchet effect. The review of economic studies, 52(2), 173–191.

    Google Scholar 

  20. Fresneda, J. E., & Gefen, D. (2019). A semantic measure of online review helpfulness and the importance of message entropy. Decision Support Systems, 125, 113117.

    Google Scholar 

  21. Zhang, Y., & Lin, Z. (2018). Predicting the helpfulness of online product reviews: A multilingual approach. Electronic Commerce Research and Applications, 27, 1–10.

    Google Scholar 

  22. Luo, L., Kannan, P. K., & Ratchford, B. T. (2008). Incorporating subjective characteristics in product design and evaluations. Journal of Marketing Research, 45(2), 182–194.

    Google Scholar 

  23. Kivetz, R., & Simonson, I. (2003). The idiosyncratic fit heuristic: Effort advantage as a determinant of consumer response to loyalty programs. Journal of Marketing Research, 40(4), 454–467.

    Google Scholar 

  24. Dunning, D., & McElwee, R. O. B. (1995). Idiosyncratic trait definitions: Implications for self-description and social judgment. Journal of Personality and Social Psychology, 68(5), 936.

    Google Scholar 

  25. Ghose, A., & Ipeirotis, P. G. (2010). Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering, 23(10), 1498–1512.

    Google Scholar 

  26. Krishnamoorthy, S. (2015). Linguistic features for review helpfulness prediction. Expert Systems with Applications, 42(7), 3751–3759.

    Google Scholar 

  27. Singh, J. P., Irani, S., Rana, N. P., Dwivedi, Y. K., Saumya, S., & Roy, P. K. (2017). Predicting the “helpfulness” of online consumer reviews. Journal of Business Research, 70, 346–355.

    Google Scholar 

  28. Gan, Q., Cao, Q., & Jones, D. (2012). Helpfulness of onlines user reviews: More is less.

  29. Hong, H., Xu, D., Wang, G. A., & Fan, W. (2017). Understanding the determinants of online review helpfulness: A meta-analytic investigation. Decision Support Systems, 102, 1–11.

    Google Scholar 

  30. Wang, Y., Wang, J., & Yao, T. (2019). What makes a helpful online review? A meta-analysis of review characteristics. Electronic Commerce Research, 19(2), 257–284.

    Google Scholar 

  31. Ladeira, W. J., Dalmoro, M., Santini, F. D. O., & Jardim, W. C. (2021). Visual cognition of fake news: the effects of consumer brand engagement. Journal of Marketing Communications, 1–21.

  32. Zhang, J. Q., Craciun, G., & Shin, D. (2010). When does electronic word-of-mouth matter? A study of consumer product reviews. Journal of Business Research, 63(12), 1336–1341.

    Google Scholar 

  33. Grewal, D., Kavanoor, S., Fern, E. F., Costley, C., & Barnes, J. (1997). Comparative versus noncomparative advertising: A meta-analysis. Journal of Marketing, 61(4), 1–15.

    Google Scholar 

  34. Wöllmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L. P. (2013). Youtube movie reviews: Sentiment analysis in an audio-visual context. IEEE Intelligent Systems, 28(3), 46–53.

    Google Scholar 

  35. Rehman, A. (2018). Cultural usability in E-commerce website design: Using objective characteristics. Journal of Applied and Emerging Sciences, 7(2), 183–189.

    Google Scholar 

  36. Zhang, R., Tran, T., & Mao, Y. (2012). Opinion helpfulness prediction in the presence of “words of few mouths.” World Wide Web, 15(2), 117–138.

    Google Scholar 

  37. Diamantopoulos, A., & Winklhofer, H. M. (2001). Index construction with formative indicators: An alternative to scale development. Journal of marketing research, 38(2), 269–277.

    Google Scholar 

  38. Qazi, A., Syed, K. B. S., Raj, R. G., Cambria, E., Tahir, M., & Alghazzawi, D. (2016). A concept-level approach to the analysis of online review helpfulness. Computers in Human Behavior, 58, 75–81.

    Google Scholar 

  39. Choi, H. S., & Leon, S. (2020). An empirical investigation of online review helpfulness: A big data perspective. Decision Support Systems, 139, 113403.

    Google Scholar 

  40. Ngo-Ye, T. L., & Sinha, A. P. (2014). The influence of reviewer engagement characteristics on online review helpfulness: A text regression model. Decision Support Systems, 61, 47–58.

    Google Scholar 

  41. Kuan, K. K., Hui, K. L., Prasarnphanich, P., & Lai, H. Y. (2015). What makes a review voted? An empirical investigation of review voting in online review systems. Journal of the Association for Information Systems, 16(1), 1.

    Google Scholar 

  42. Merton, R. K. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63.

    Google Scholar 

  43. Du, J., Rong, J., Wang, H., & Zhang, Y. (2021). Neighbor-aware review helpfulness prediction. Decision Support Systems, 148, 113581.

    Google Scholar 

  44. Chua, A. Y., & Banerjee, S. (2017). Analyzing review efficacy on Amazon.com: Does the rich grow richer? Computers in Human Behavior, 75, 501–509.

    Google Scholar 

  45. Chen, C. C., & Tseng, Y. D. (2011). Quality evaluation of product reviews using an information quality framework. Decision Support Systems, 50(4), 755–768.

    Google Scholar 

  46. Liu, Y., Jin, J., Ji, P., Harding, J. A., & Fung, R. Y. (2013). Identifying helpful online reviews: A product designer’s perspective. Computer-Aided Design, 45(2), 180–194.

    Google Scholar 

  47. Zhu, Y., Liu, M., Zeng, X., & Huang, P. (2020). The effects of prior reviews on perceived review helpfulness: A configuration perspective. Journal of Business Research, 110, 484–494.

    Google Scholar 

  48. Camilleri, A. R. (2020). The importance of online reviews depends on when they are presented. Decision Support Systems, 133, 113307.

    Google Scholar 

  49. Jha, A. K., & Shah, S. (2019). Social influence on future review sentiments: An appraisal-theoretic view. Journal of Management Information Systems, 36(2), 610–638.

    Google Scholar 

  50. Churchill, G. A., Jr. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73.

    Google Scholar 

  51. Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199–218.

    Google Scholar 

  52. Rossiter, J. R. (2002). The C-OAR-SE procedure for scale development in marketing. International Journal of Research in Marketing, 19(4), 305–335.

    Google Scholar 

  53. Cadogan, J. W., & Lee, N. (2013). Improper use of endogenous formative variables. Journal of Business Research, 66(2), 233–241.

    Google Scholar 

  54. Coltman, T., Devinney, T. M., Midgley, D. F., & Venaik, S. (2008). Formative versus reflective measurement models: Two applications of formative measurement. Journal of Business Research, 61(12), 1250–1262.

    Google Scholar 

  55. Dickinger, A., & Stangl, B. (2013). Website performance and behavioral consequences: A formative measurement approach. Journal of Business Research, 66(6), 771–777.

    Google Scholar 

  56. Diamantopoulos, A., Riefler, P., & Roth, K. P. (2008). Advancing formative measurement models. Journal of Business Research, 61(12), 1203–1218.

    Google Scholar 

  57. Birnbaum, A., Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Some latent trait models and their use in inferring an examinee’s ability. Addison-Wesley

  58. Bollen, K. A. (1984). Multiple indicators: Internal consistency or no necessary relationship? Quality and Quantity, 18(4), 377–385.

    Google Scholar 

  59. Skowronski, J. J., & Carlston, D. E. (1989). Negativity and extremity biases in impression formation: A review of explanations. Psychological Bulletin, 105(1), 131.

    Google Scholar 

  60. Gershoff, A. D., Mukherjee, A., & Mukhopadhyay, A. (2003). Consumer acceptance of online agent advice: Extremity and positivity effects. Journal of Consumer Psychology, 13(1–2), 161–170.

    Google Scholar 

  61. Pan, Y., & Zhang, J. Q. (2011). Born unequal: A study of the helpfulness of user-generated product reviews. Journal of Retailing, 87(4), 598–612.

    Google Scholar 

  62. Nyumba, O., & T., Wilson, K., Derrick, C. J., & Mukherjee, N. (2018). The use of focus group discussion methodology: Insights from two decades of application in conservation. Methods in Ecology and evolution, 9(1), 20–32.

    Google Scholar 

  63. Fernandez, W. D., & Lehmann, H. (2011). Case studies and grounded theory method in information systems research: Issues and use. Journal of Information Technology Case and Application Research, 13(1), 4–15.

    Google Scholar 

  64. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

    Google Scholar 

  65. Hao, Y., Ye, Q., Li, Y., & Cheng, Z. (2010). How does the valence of online consumer reviews matter in consumer decision making? Differences between search goods and experience goods. In Proceedings of the 2010 43rd Hawaii international conference on system sciences, 1–10.

  66. Abernethy, A. M., & Franke, G. R. (1996). The information content of advertising: A meta-analysis. Journal of Advertising, 25(2), 1–17.

    Google Scholar 

  67. Fuchs, G., & Reichel, A. (2011). An exploratory inquiry into destination risk perceptions and risk reduction strategies of first time vs. repeat visitors to a highly volatile destination. Tourism Management, 32(2), 266–276.

    Google Scholar 

  68. Hu, X., & Yang, Y. (2021). What makes online reviews helpful in tourism and hospitality? A bare-bones meta-analysis. Journal of Hospitality Marketing & Management, 30(2), 139–158.

    Google Scholar 

  69. Gensch, D. H. (1987). A two-stage disaggregate attribute choice model. Marketing Science, 6(3), 223–239.

    Google Scholar 

  70. Yang, S. B., Shin, S. H., Joun, Y., & Koo, C. (2017). Exploring the comparative importance of online hotel reviews’ heuristic attributes in review helpfulness: A conjoint analysis approach. Journal of Travel & Tourism Marketing, 34(7), 963–985.

    Google Scholar 

  71. Golden, L. L. (1976). Consumer reactions to comparative advertising. Advances in Consumer Research, 3(1).

  72. Ash, S. B., & Wee, C. H. (1983). Comparative advertising: A review with implications for further research. Advances in Consumer Research, 10(1).

  73. Weathers, D., Swain, S. D., & Grover, V. (2015). Can online product reviews be more helpful? Examining characteristics of information content by product type. Decision Support Systems, 79, 12–23.

    Google Scholar 

  74. Bashir, A., & Malik, N. I. (2009). Effects of advertisement on consumer behavior of university students. Proceedings, 2, 345–363.

    Google Scholar 

  75. Treisman, A. M. (1969). Strategies and models of selective attention. Psychological Review, 76(3), 282.

    Google Scholar 

  76. Salehan, M., & Kim, D. J. (2016). Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decision Support Systems, 81, 30–40.

    Google Scholar 

  77. Jessmer, S. L., & Anderson, D. (2001). The effect of politeness and grammar on user perceptions of electronic mail. North American Journal of Psychology, 3(2).

  78. Karimi, S., & Wang, F. (2017). Online review helpfulness: Impact of reviewer profile image. Decision Support Systems, 96, 39–48.

    Google Scholar 

  79. Cheung, C. M., Lee, M. K., & Rabjohn, N. (2008). The impact of electronic word-of-mouth. Internet Research, 18(3), 229–247.

    Google Scholar 

  80. Liu, Y., Huang, X., An, A., & Yu, X. (2008). Modeling and predicting the helpfulness of online reviews. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, 443–452.

  81. Park, C., & Lee, T. M. (2009). Antecedents of online reviews’ usage and purchase influence: An empirical comparison of US and Korean consumers. Journal of Interactive Marketing, 23(4), 332–340.

    Google Scholar 

  82. Jiménez, F. R., & Mendoza, N. A. (2013). Too popular to ignore: The influence of online reviews on purchase intentions of search and experience products. Journal of Interactive Marketing, 27(3), 226–235.

    Google Scholar 

  83. Park, D. H., Lee, J., & Han, I. (2007). The effect of on-line consumer reviews on consumer purchasing intention: The moderating role of involvement. International Journal of Electronic Commerce, 11(4), 125–148.

    Google Scholar 

  84. Diamantopoulos, A., & Siguaw, J. A. (2006). Formative versus reflective indicators in organizational measure development: A comparison and empirical illustration. British Journal of Management, 17(4), 263–282.

    Google Scholar 

  85. Liu, Z., & Park, S. (2015). What makes a useful online review? Implication for travel product websites. Tourism Management, 47, 140–151.

    Google Scholar 

  86. Bergkvist, L., & Rossiter, J. R. (2007). The predictive validity of multiple-item versus single-item measures of the same constructs. Journal of Marketing Research, 44(2), 175–184.

    Google Scholar 

  87. Nunnally, J. C., & Bernstein, I. H. (1994). The theory of measurement error. Psychometric Theory, 3(1), 209–247.

    Google Scholar 

  88. Chen, M. Y. (2016). Can two-sided messages increase the helpfulness of online reviews? Online Information Review, 40(3), 316–332.

    Google Scholar 

  89. Willemsen, L. M., Neijens, P. C., Bronner, F., & De Ridder, J. A. (2011). “Highly recommended!” The content characteristics and perceived usefulness of online consumer reviews. Journal of Computer-Mediated Communication, 17(1), 19–38.

    Google Scholar 

  90. Li, C., Cui, G., & Peng, L. (2017). The signaling effect of management response in engaging customers: A study of the hotel industry. Tourism Management, 62, 42–53.

    Google Scholar 

  91. Phillips, P., Barnes, S., Zigan, K., & Schegg, R. (2017). Understanding the impact of online reviews on hotel performance: An empirical analysis. Journal of Travel Research, 56(2), 235–249.

    Google Scholar 

  92. Chen, M. Y., & Teng, C. I. (2013). A comprehensive model of the effects of online store image on purchase intention in an e-commerce environment. Electronic Commerce Research, 13(1), 1–23.

    Google Scholar 

  93. Orús, C., Gurrea, R., & Flavián, C. (2017). Facilitating imaginations through online product presentation videos: Effects on imagery fluency, product attitude and purchase intention. Electronic Commerce Research, 17(4), 661–700.

    Google Scholar 

  94. Bae, S., & Lee, T. (2011). Gender differences in consumers’ perception of online consumer reviews. Electronic Commerce Research, 11(2), 201–214.

    Google Scholar 

  95. Bollen, K. A. (1989). Structural equations with latent variables. Wiley.

  96. Spector, P. E. (1992). Summated rating scale construction: An introduction (Vol. 82). Sage.

  97. Leckie, C., Nyadzayo, M. W., & Johnson, L. W. (2018). Promoting brand engagement behaviors and loyalty through perceived service value and innovativeness. Journal of Services Marketing, 32, 70–82.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Marketing and Strategy, ICFAI Foundation for Higher Education, Hyderabad, India

    Rachita Kashyap

  2. Department of Marketing & Strategy, Indian Institute of Management Rohtak, Rohtak, Haryana, India

    Ankit Kesharwani

  3. Department of Business Analytics, Narsee Monjee Institute of Management Studies, Hyderabad, India

    Abhilash Ponnam

Authors
  1. Rachita Kashyap
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ankit Kesharwani
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Abhilash Ponnam
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Rachita Kashyap.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

The objective of this section is to provide easily understandable references to concepts/methodological approach used in this study:

Following papers clearly explicates the difference between formative measure and reflective approach to scale development

1. Diamantopoulos, A., & Siguaw, J. A. (2006). Formative versus reflective indicators in organizational measure development: A comparison and empirical illustration. British journal of management, 17(4), 263–282.

2. Finn, A., & Wang, L. (2014). Formative vs. reflective measures: Facets of variation. Journal of Business Research, 67(1), 2821–2826.

Following papers illustrate development of formative measure of scale construction

1. Diamantopoulos, A., & Winklhofer, H. M. (2001). Index construction with formative indicators: An alternative to scale development. Journal of marketing research, 38(2), 269–277.

2. Coltman, T., Devinney, T. M., Midgley, D. F., & Venaik, S. (2008). Formative versus reflective measurement models: Two applications of formative measurement. Journal of Business Research, 61(12), 1250–1262.

Following papers explain the use case for single item measures in scale development

1. Bergkvist, L., & Rossiter, J. R. (2007). The predictive validity of multiple-item versus single-item measures of the same constructs. Journal of marketing research, 44(2), 175–184.

2. Rossiter, J. R. (2002). The C-OAR-SE procedure for scale development in marketing. International journal of research in marketing, 19(4), 305–335.

Appendix 2

Steps to be followed in formative scale development process.

figure a

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kashyap, R., Kesharwani, A. & Ponnam, A. Measurement of online review helpfulness: a formative measure development and validation. Electron Commer Res 23, 2183–2216 (2023). https://doi.org/10.1007/s10660-022-09531-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10660-022-09531-1

Keywords