Skip to main content
Springer Nature Link
Log in

A BERT-based review helpfulness prediction model utilizing consistency of ratings and texts

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Predicting review helpfulness (RH) to ensure that consumers make effective purchasing decisions is a significant area of study. Many scholars have attempted to develop accurate review helpfulness prediction (RHP) methodologies. However, most previous studies have mainly focused on predictions using product review texts, and few studies have used product satisfaction as indicated by star ratings, particularly the consistency between review texts and star ratings. This study proposes a novel model called BHelP-CoRT (Bidirectional Encoder Representations from Transformers based RHP model utilizing consistency of ratings and texts) to predict RH. The proposed model consists of a review text encoder, star rating encoder, and text-rating interaction. The review text encoder was developed by applying the BERT model to extract contextual semantic features embedded in review texts. The star rating encoder was designed to embed star ratings into feature vectors. The text-rating interaction was constructed by applying an attention mechanism to extract the text-rating interaction and introduce consistency into the RHP tasks. This study conducted extensive experiments to demonstrate the effectiveness of the proposed model from multiple perspectives using real-world online reviews collected from Amazon. The experimental results show that the proposed model outperforms the state-of-the-art models, indicating that it can improve the RHP performance. Specifically, this effectiveness is reflected in the processing of reviews containing inconsistent information. This study supports the marketing efforts of the e-commerce industry by providing an RHP service to address consumer information overload.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The Amazon datasets are publicly available at https://cseweb.ucsd.edu/~jmcauley/datasets/amazon_v2/.

Notes

  1. https://cseweb.ucsd.edu/~jmcauley/datasets/amazon_v2/.

References

  1. Choi HS, Leon S (2020) An empirical investigation of online review helpfulness: a big data perspective. Decis Support Syst 139:113403

    Article  MATH  Google Scholar 

  2. Li X, Li Q, Kim J (2023) A review helpfulness modeling mechanism for online E-commerce: multi-channel CNN EndtoEnd Approach. Appl Artif Intell 37(1):2166226

    Article  Google Scholar 

  3. Li Q, Li X, Lee B, Kim J (2021) A hybrid CNN-based review helpfulness filtering model for improving e-commerce recommendation service. Appl Sci 11(18):8613

    Article  MATH  Google Scholar 

  4. Li H, Yan J (2024) Does Visual Review Content Enhance Review helpfulness? A text-Mining Approach. IEEE Access 12:27633–27647

    Article  Google Scholar 

  5. Lee M, Kwon W, Back KJ (2021) Artificial intelligence for hospitality big data analytics: developing a prediction model of restaurant review helpfulness for customer decision-making. Int J Contemp Hospitality Manage 33(6):2117–2136

    Article  MATH  Google Scholar 

  6. Qin J, Zheng P, Wang X (2022) Comprehensive helpfulness of online reviews: a dynamic strategy for ranking reviews by intrinsic and extrinsic helpfulness. Decis Support Syst 163:113859

    Article  MATH  Google Scholar 

  7. Ren G, Diao L, Guo F, Hong T (2024) A co-attention based multi-modal fusion network for review helpfulness prediction. Inf Process Manag 61(1):103573

    Article  Google Scholar 

  8. Saumya S, Singh JP, Dwivedi YK (2020) Predicting the helpfulness score of online reviews using convolutional neural network. Soft Comput 24(15):10989–11005

    Article  MATH  Google Scholar 

  9. Mitra S, Jenamani M (2021) Helpfulness of online consumer reviews: a multi-perspective approach. Inf Process Manag 58(3):102538

    Article  MATH  Google Scholar 

  10. Bilal M, Marjani M, Hashem IAT, Malik N, Lali MIU, Gani A (2021) Profiling reviewers’ social network strength and predicting the helpfulness of online customer reviews. Electron Commer Res Appl 45:101026

    Article  Google Scholar 

  11. Pashchenko Y, Rahman MF, Hossain MS, Uddin MK, Islam T (2022) Emotional and the normative aspects of customers’ reviews. J Retailing Consumer Serv 68:103011

    Article  MATH  Google Scholar 

  12. Yang S, Yao J, Qazi A (2020) Does the review deserve more helpfulness when its title resembles the content? Locating helpful reviews by text mining. Inf Process Manag 57(2):102179

    Article  Google Scholar 

  13. Aghakhani N, Oh O, Gregg DG, Karimi J (2021) Online review consistency matters: an elaboration likelihood model perspective. Inform Syst Front 23:1287–1301

    Article  Google Scholar 

  14. Al-Natour S, Turetken O (2020) A comparative assessment of sentiment analysis and star ratings for consumer reviews. Int J Inf Manag 54:102132

    Article  MATH  Google Scholar 

  15. Luo Y, Xu X (2021) Comparative study of deep learning models for analyzing online restaurant reviews in the era of the COVID-19 pandemic. Int J Hospitality Manage 94:102849

    Article  Google Scholar 

  16. Kashyap R, Kesharwani A, Ponnam A (2022) Measurement of online review helpfulness: a formative measure development and validation. Electron Commer Res 23(4): 2183-2216

  17. Kim SM, Pantel P, Chklovski T, Pennacchiotti M (2006) Automatically assessing review helpfulness. In: Proceedings of the 2006 Conference on empirical methods in natural language processing. pp 423–430

  18. Ghose A, Ipeirotis PG (2010) Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Trans Knowl Data Eng 23(10):1498–1512

    Article  MATH  Google Scholar 

  19. Krishnamoorthy S (2015) Linguistic features for review helpfulness prediction. Expert Syst Appl 42(7):3751–3759

    Article  MATH  Google Scholar 

  20. Bagherzadeh S, Shokouhyar S, Jahani H, Sigala M (2021) A generalizable sentiment analysis method for creating a hotel dictionary: using big data on TripAdvisor hotel reviews. J Hospitality Tourism Technol 12(2):210–238

    Article  Google Scholar 

  21. Malik M, Hussain A (2020) Exploring the influential reviewer, review and product determinants for review helpfulness. Artif Intell Rev 53:407–427

    Article  MATH  Google Scholar 

  22. Olmedilla M, Martínez-Torres MR, Toral S (2022) Prediction and modelling online reviews helpfulness using 1D convolutional neural networks. Expert Syst Appl 198:116787

    Article  MATH  Google Scholar 

  23. Bedi J, Toshniwal D (2022) CitEnergy: a BERT based model to analyse citizens’ Energy-Tweets. Sustainable Cities Soc 80:103706

    Article  MATH  Google Scholar 

  24. Li S, Deng M, Shao Z, Chen X, Zheng Y (2023) Automatic classification of interactive texts in online collaborative discussion based on multi-feature fusion. Comput Electr Eng 107:108648

    Article  MATH  Google Scholar 

  25. Jia K (2022) Sentiment classification of microblog: a framework based on BERT and CNN with attention mechanism. Comput Electr Eng 101:108032

    Article  MATH  Google Scholar 

  26. Bilal M, Almazroi AA (2023) Effectiveness of fine-tuned BERT model in classification of helpful and unhelpful online customer reviews. Electron Commer Res 23(4):2737–2757

    Article  MATH  Google Scholar 

  27. Lin SY, Kung YC, Leu FY (2022) Predictive intelligence in harmful news identification by BERT-based ensemble learning model with text sentiment analysis. Inf Process Manag 59(2):102872

    Article  MATH  Google Scholar 

  28. Zhao S, Zhang T, Hu M, Chang W, You F (2022) AP-BERT: enhanced pre-trained model through average pooling. Appl Intell 52(14): 15929-15937

  29. Karn AL, Karna RK, Kondamudi BR, Bagale G, Pustokhin DA, Pustokhina IV et al (2023) Customer centric hybrid recommendation system for E-Commerce applications by integrating hybrid sentiment analysis. Electron Commer Res 23(1):279–314

    Article  MATH  Google Scholar 

  30. Yu Y, Wang Y, Mu J, Li W, Jiao S, Wang Z et al (2022) Chinese mineral named entity recognition based on BERT model. Expert Syst Appl 206:117727

    Article  Google Scholar 

  31. Liu G, Guo J (2019) Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337:325–338

    Article  MATH  Google Scholar 

  32. Malik MSI (2020) Predicting users’ review helpfulness: the role of significant review and reviewer characteristics. Soft Comput 24(18):13913–13928

    Article  MATH  Google Scholar 

  33. Chen G, Xiao S, Zhang C, Wang W (2022) An orthogonal-space-learning-based method for selecting semantically helpful reviews. Electron Commer Res Appl 53:101154

    Article  Google Scholar 

  34. Mauro N, Ardissono L, Petrone G (2021) User and item-aware estimation of review helpfulness. Inf Process Manag 58(1):102434

    Article  MATH  Google Scholar 

  35. Sharma A, Jayagopi DB (2024) Modeling essay grading with pre-trained BERT features. Appl Intell 54(6):4979–4993

    Article  MATH  Google Scholar 

  36. Park J, Li X, Li Q, Kim J (2023) Impact on recommendation performance of online review helpfulness and consistency. Data Technol Appl 57(2):199–221

    MATH  Google Scholar 

  37. Bengesi S, Oladunni T, Olusegun R, Audu H (2023) A machine learning-sentiment analysis on Monkeypox outbreak: an extensive dataset to show the polarity of public opinion from Twitter tweets. IEEE Access 11:11811–11826

    Article  Google Scholar 

  38. Chakraborty K, Bhatia S, Bhattacharyya S, Platos J, Bag R, Hassanien AE (2020) Sentiment analysis of COVID-19 tweets by Deep Learning Classifiers—A study to show how popularity is affecting accuracy in social media. Appl Soft Comput 97:106754

    Article  Google Scholar 

  39. Koppel M, Schler J (2006) The importance of neutral examples for learning sentiment. Comput Intell 22(2):100–109

    Article  MathSciNet  MATH  Google Scholar 

  40. Puh K, Bagić Babac M (2023) Predicting sentiment and rating of tourist reviews using machine learning. J Hospitality Tourism Insights 6(3):1188–1204

    Article  MATH  Google Scholar 

  41. Gomes L, da Silva Torres R, Côrtes ML (2023) BERT-and TF-IDF-based feature extraction for long-lived bug prediction in FLOSS: a comparative study. ‎Inf Softw Technol 160:107217

    Article  MATH  Google Scholar 

  42. Sajjad H, Dalvi F, Durrani N, Nakov P (2023) On the effect of dropping layers of pre-trained transformer models. Comput Speech Lang 77:101429

    Article  Google Scholar 

  43. Malik M, Hussain A (2018) An analysis of review content and reviewer variables that contribute to review helpfulness. Inf Process Manag 54(1):88–104

    Article  MATH  Google Scholar 

  44. Tsai CF, Chen K, Hu YH, Chen WK (2020) Improving text summarization of online hotel reviews with review helpfulness and sentiment. Tour Manag 80:104122

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This research is supported by the BK21 FOUR (Fostering Outstanding Universities for Research) funded by the Ministry of Education (MOE, Korea) and National Research Foundation of Korea (NRF).

Author information

Author notes

  1. Xinzhe Li and Qinglong Li contributed equally to this work.

Authors and Affiliations

  1. Department of Big Data Analytics, Kyung Hee University, Seoul, 02447, Republic of Korea

    Xinzhe Li, Qinglong Li, Dongyeop Ryu & Jaekyeong Kim

  2. School of Management, Kyung Hee University, Seoul, 02447, Republic of Korea

    Jaekyeong Kim

Authors
  1. Xinzhe Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Qinglong Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Dongyeop Ryu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jaekyeong Kim
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

The authors confirm contribution to the paper as follows: study conception and design: Xinzhe Li, Qinglong Li, Jaekyeong Kim; data collection: Dongyeop Ryu; analysis and interpretation of results: Xinzhe Li, Dongyeop Ryu; draft manuscript preparation: Xinzhe Li, Qinglong Li, Jaekyeong Kim. All authors reviewed the results and approved the final version of the manuscript.

Corresponding author

Correspondence to Jaekyeong Kim.

Ethics declarations

Competing interests

The authors declare no potential conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Li, Q., Ryu, D. et al. A BERT-based review helpfulness prediction model utilizing consistency of ratings and texts. Appl Intell 55, 455 (2025). https://doi.org/10.1007/s10489-024-06100-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06100-x

Keywords