skip to main content
10.1145/3487664.3487775acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiiwasConference Proceedingsconference-collections
research-article

What Makes a Review Encouraging: Feature Analysis of User Access Logs in a Large-scale Online Movie Review Site

Published: 30 December 2021 Publication History

Abstract

This paper reveals the characteristics of the reviews that encourage readers to watch the reviewed movie by analyzing large-scale access log data. We assume that some of the reviews that users saw just before they clicked the links to a streaming site contain factors that help users decide whether they watch that movie. Our method used a random forest classifier trained to determine whether a review encouraged a movie-watching behavior. We conducted feature importance-based analysis using three types of features: review itself, item, and reviewer. We analyzed 70,000 user behaviors from Yahoo! Movies (a movie review site in Japan) and Gyao! (a movie streaming site in Japan). Through a cross-validation experiment, the classifier was able to classify encouraging reviews with an F-score of 0.78, and mainly the features about the item contributed to the classification performance. An additional subjects experiment confirmed that these features contribute to the review’s usefulness.

References

[1]
Einar Bjering, Lars Jaakko Havro, and Øystein Moen. 2015. An empirical investigation of self-selection bias and factors influencing review helpfulness. International Journal of Business and Management 10, 7(2015), 16.
[2]
Don Charlett, Ron Garland, Norman Marr, 1995. How damaging is negative word of mouth. Marketing Bulletin 6, 1 (1995), 42–50.
[3]
Pei-Yu Chen, Samita Dhanasobhon, and Michael Smith. 2008. All Reviews are not created equal: The disaggregate impact of reviews on sales on Amazon. com. In Carnegie Mellon University Working Paper.
[4]
Christy MK Cheung, Matthew KO Lee, and Neil Rabjohn. 2008. The impact of electronic word-of-mouth: The adoption of online opinions in online customer communities. Internet research 18, 3 (2008), 229–247.
[5]
Judith A. Chevalier and Dina Mayzlin. 2006. The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research 43, 3 (2006), 345–354.
[6]
Alton Chua and Snehasish Banerjee. 2014. Understanding review helpfulness as a function of reviewer reputation, review rating, and review depth. Journal of the Association for Information Science and Technology 66 (06 2014). https://doi.org/10.1002/asi.23180
[7]
L. Connors, S. M. Mudambi, and D. Schuff. 2011. Is It the Review or the Reviewer? a Multi-Method Approach to Determine the Antecedents of Online Review Helpfulness. In 2011 44th Hawaii International Conference on System Sciences. 1–10. https://doi.org/10.1109/HICSS.2011.260
[8]
Chris Forman, Anindya Ghose, and Batia Wiesenfeld. 2008. Examining the relationship between reviews and sales: The role of reviewer identity disclosure in electronic markets. Information systems research 19, 3 (2008), 291–313.
[9]
Yuanyuan Hao, Qiang Ye, Yijun Li, and Zhuo Cheng. 2010. How Does the Valence of Online Consumer Reviews Matter in Consumer Decision Making? Differences between Search Goods and Experience Goods. 1–10. https://doi.org/10.1109/HICSS.2010.455
[10]
Hong Hong, Di Xu, G Alan Wang, and Weiguo Fan. 2017. Understanding the determinants of online review helpfulness: A meta-analytic investigation. Decision Support Systems 102 (2017), 1–11.
[11]
Liqiang Huang, Chuan-Hoo Tan, Weiling Ke, and Kwok-Kee Wei. 2013. Comprehension and assessment of product reviews: A review-product congruity proposition. Journal of Management Information Systems 30, 3 (2013), 311–343.
[12]
Zhiwei Liu and Sangwon Park. 2015. What makes a useful online review? Implication for travel product websites. Tourism Management 47(2015), 140–151.
[13]
Yue Lu, Panayiotis Tsaparas, Alexandros Ntoulas, and Livia Polanyi. 2010. Exploiting social context for review quality prediction. In Proceedings of the 19th International Conference on World Wide Web. 691–700.
[14]
Jing Luan, Zhong Yao, FuTao Zhao, and Hao Liu. 2016. Search product and experience product online reviews: An eye-tracking study on consumers’ review search behavior. Computers in Human Behavior 65 (2016), 420 – 430. https://doi.org/10.1016/j.chb.2016.08.037
[15]
Julian John McAuley and Jure Leskovec. 2013. From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. In Proceedings of the 22nd international conference on World Wide Web. 897–908.
[16]
Susan M Mudambi and David Schuff. 2010. What makes a helpful review? A study of customer reviews on Amazon. com. MIS quarterly 34, 1 (2010), 185–200.
[17]
Phillip Nelson. 1974. Advertising as information. Journal of political economy 82, 4 (1974), 729–754.
[18]
Yue Pan and Jason Q Zhang. 2011. Born unequal: a study of the helpfulness of user-generated product reviews. Journal of Retailing 87, 4 (2011), 598–612.
[19]
Pradeep Racherla and Wesley Friske. 2012. Perceived ‘usefulness’ of online consumer reviews: An exploratory investigation across three services categories. Electronic Commerce Research and Applications 11, 6(2012), 548–559.
[20]
Yoshiyuki Shoji, Makoto P. Kato, and Katsumi Tanaka. 2014. Can Diversity Improve Credibility of User Review Data?Springer International Publishing, Cham, 244–258. https://doi.org/10.1007/978-3-319-13734-6_17
[21]
Jiliang Tang, Huiji Gao, Xia Hu, and Huan Liu. 2013. Context-aware review helpfulness rating prediction. In Proceedings of the 7th ACM Conference on Recommender Systems. 1–8.
[22]
Y. Zhou and S. Yang. 2019. Roles of Review Numerical and Textual Characteristics on Review Helpfulness Across Three Different Types of Reviews. IEEE Access 7(2019), 27769–27780. https://doi.org/10.1109/ACCESS.2019.2901472

Cited By

View all
  • (2024)Emotions in recommender systems for discrepant-usersKnowledge and Information Systems10.1007/s10115-024-02307-z67:1(953-976)Online publication date: 23-Dec-2024

Index Terms

  1. What Makes a Review Encouraging: Feature Analysis of User Access Logs in a Large-scale Online Movie Review Site
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        iiWAS2021: The 23rd International Conference on Information Integration and Web Intelligence
        November 2021
        658 pages
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 30 December 2021

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Access Log Analysis
        2. Feature Study
        3. Online Review
        4. Random Forest

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        iiWAS2021

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)5
        • Downloads (Last 6 weeks)2
        Reflects downloads up to 13 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Emotions in recommender systems for discrepant-usersKnowledge and Information Systems10.1007/s10115-024-02307-z67:1(953-976)Online publication date: 23-Dec-2024

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media