research-article

Generating Tips from Product Reviews

Authors:

Slava Novgorodov,

Alexander NusAuthors Info & Claims

WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

Pages 310 - 318

https://doi.org/10.1145/3437963.3441755

Published: 08 March 2021 Publication History

Abstract

Product reviews play a key role in e-commerce platforms. Studies show that many users read product reviews before purchase and trust them as much as personal recommendations. However, in many cases, the number of reviews per product is large and finding useful information becomes a challenging task. A few websites have recently added an option to post tips - short, concise, practical, and self-contained pieces of advice about products. These tips are complementary to the reviews and usually add a new non-trivial insight about the product, beyond its title, attributes, and description. Yet, most if not all major e-commerce platforms lack the notion of a tip as a first class citizen and customers typically express their advice through other means, such as reviews. In this work, we propose an extractive method for tip generation from product reviews. We focus on five popular e-commerce domains whose reviews tend to contain useful non-trivial tips that are beneficial for potential customers. We formally define the task of tip extraction in e-commerce by providing the list of tip types, tip timing (before and/or after the purchase), and connection to the surrounding context sentences. To extract the tips, we propose a supervised approach and provide a labeled dataset, annotated by human editors, over 14,000 product reviews using a dedicated tool. To demonstrate the potential of our approach, we compare different tip generation methods and evaluate them both manually and over the labeled set. Our approach demonstrates especially high performance for popular products in the Baby, Home Improvement and Sports & Outdoors domains, with precision of over 95% for the top 3 tips per product.

References

[1]

Leman Akoglu, Rishi Chandy, and Christos Faloutsos. 2013. Opinion fraud detection in online reviews by network effects. In Proc. of ICWSM.

[2]

Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2009. Multi-facet rating of product reviews. In ECIR. Springer, 461--472.

[3]

Adam Berger and John Lafferty. 1999. Information retrieval as statistical translation. In Proc. of SIGIR. 222--229.

Digital Library

[4]

Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. TACL, Vol. 5 (2017), 135--146.

[5]

BrightLocal. 2018. Local Consumer Review Survey. https://www.brightlocal.com/research/local-consumer-review-survey/

[6]

David Carmel, Erel Uziel, Ido Guy, Yosi Mass, and Haggai Roitman. 2012. Folksonomy-Based Term Extraction for Word Cloud Generation. ACM Trans. Intell. Syst. Technol., Vol. 3, 4, Article 60 (Sept. 2012), 20 pages.

Digital Library

[7]

Chien Chin Chen and You-De Tseng. 2011. Quality evaluation of product reviews using an information quality framework. Decision Support Systems, Vol. 50, 4 (2011), 755--768.

Digital Library

[8]

Judith A Chevalier and Dina Mayzlin. 2006. The effect of word of mouth on sales: Online book reviews. Journal of marketing research, Vol. 43, 3 (2006), 345--354.

[9]

Paolo Cremonesi, Raffaele Facendola, Franca Garzotto, Matteo Guarnerio, Mattia Natali, and Roberto Pagano. 2014. Polarized review summarization as decision making tool. In Proc. of AVI. 355--356.

Digital Library

[10]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[11]

Wenjing Duan, Bin Gu, and Andrew B Whinston. 2008. Do online reviews matter? An empirical investigation of panel data. Decision support systems, Vol. 45, 4 (2008), 1007--1016.

[12]

Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, Vol. 76, 5 (1971), 378--382.

[13]

Shen Gao, Xiuying Chen, Piji Li, Zhaochun Ren, Lidong Bing, Dongyan Zhao, and Rui Yan. 2019. Abstractive text summarization by incorporating reader comments. In Proc. of the AAAI Conference, Vol. 33. 6399--6406.

Digital Library

[14]

Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive summarization of product reviews using discourse structure. In Proc. of EMNLP. 1602--1613.

[15]

Ido Guy, Avihai Mejer, Alexander Nus, and Fiana Raiber. 2017. Extracting and ranking travel tips from user-generated reviews. In Proc. of WWW. 987--996.

Digital Library

[16]

Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proc. of WWW. 507--517.

Digital Library

[17]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[18]

Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proc. of KDD. 168--177.

Digital Library

[19]

Nan Hu, Indranil Bose, Noi Sian Koh, and Ling Liu. 2012. Manipulation of online reviews: An analysis of ratings, readability, and sentiments. Decision support systems, Vol. 52, 3 (2012), 674--684.

[20]

Soo-Min Kim, Patrick Pantel, Tim Chklovski, and Marco Pennacchiotti. 2006. Automatically assessing review helpfulness. In Proc. of EMNLP. 423--430.

[21]

Theodoros Lappas, Mark Crovella, and Evimaria Terzi. 2012. Selecting a characteristic set of reviews. In Proc. of KDD. 832--840.

Digital Library

[22]

Theodoros Lappas and Dimitrios Gunopulos. 2010. Efficient confident search in large review corpora. In ECML PKDD. Springer, 195--210.

[23]

Piji Li, Zihao Wang, Lidong Bing, and Wai Lam. 2019. Persona-Aware Tips Generation. In The World Wide Web Conference. 1006--1016.

[24]

Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. 2017. Neural rating regression with abstractive tips generation for recommendation. In Proc. of SIGIR. 345--354.

Digital Library

[25]

Stephen W Litvin, Ronald E Goldsmith, and Bing Pan. 2008. Electronic word-of-mouth in hospitality and tourism management. Tourism management, Vol. 29, 3 (2008), 458--468.

[26]

Linqing Liu, Yao Lu, Min Yang, Qiang Qu, Jia Zhu, and Hongyan Li. 2018. Generative adversarial network for abstractive text summarization. In Thirty-second AAAI conference on artificial intelligence.

[27]

Mitchell Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. (1993).

[28]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint, Vol. abs/1301.37810 (2013).

[29]

Thanh-Son Nguyen, Hady W Lauw, and Panayiotis Tsaparas. 2013. Using micro-reviews to select an efficient set of reviews. In Proc. of CIKM. 1067--1076.

Digital Library

[30]

Slava Novgorodov, Ido Guy, Guy Elad, and Kira Radinsky. 2019. Generating product descriptions from user reviews. In Proc. of WWW. 1354--1364.

Digital Library

[31]

Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proc. of EMNLP, Vol. 14. 1532--1543.

[32]

Irina Rish. 2001. An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence, Vol. 3. 41--46.

[33]

Ruben Sipos and Thorsten Joachims. 2013. Generating comparative summaries from reviews. In Proc. of CIKM. 1853--1856.

Digital Library

[34]

Doug Snowball. 1980. Some effects of accounting expertise and information load: An empirical study. Accounting, Organizations and Society, Vol. 5, 3 (1980), 323--338.

[35]

Cheri Speier, Joseph S Valacich, and Iris Vessey. 1999. The influence of task interruption on individual decision making: An information overload perspective. Decision Sciences, Vol. 30, 2 (1999), 337--360.

[36]

Oren Tsur and Ari Rappoport. 2009. Revrank: A fully unsupervised algorithm for selecting the most helpful book reviews. In Proc. of ICWSM.

[37]

Ingmar Weber, Antti Ukkonen, and Aris Gionis. 2012. Answers, not links: extracting tips from yahoo! answers to address how-to web queries. In Proc. of WSDM. 613--622.

Digital Library

[38]

Alfan Farizki Wicaksono and Sung-Hyon Myaeng. 2012. Mining Advices from Weblogs. In Proc. of CIKM. 2347--2350.

Digital Library

[39]

Alfan Farizki Wicaksono and Sung-Hyon Myaeng. 2013. Toward advice mining: Conditional random fields for extracting advice-revealing text units. In Proc. of CIKM. 2039--2048.

Digital Library

[40]

Qing-Song Xu and Yi-Zeng Liang. 2001. Monte Carlo cross validation. Chemometrics and Intelligent Laboratory Systems, Vol. 56, 1 (2001), 1--11.

[41]

Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In Proc. of NAACL. 1480--1489.

[42]

Qiang Ye, Rob Law, and Bin Gu. 2009. The impact of online user reviews on hotel room sales. International Journal of Hospitality Management, Vol. 28, 1 (2009), 180--182.

[43]

Di Zhu, Theodoros Lappas, and Juheng Zhang. 2018. Unsupervised tip-mining from customer reviews. Decision Support Systems, Vol. 107 (2018), 116--124.

Cited By

Allen THorwitz ASprigle S(2025)Leveraging AI and customer reviews to evaluate technology used by people with disabilitiesDisability and Rehabilitation: Assistive Technology10.1080/17483107.2025.2465603(1-9)Online publication date: 16-Feb-2025
https://doi.org/10.1080/17483107.2025.2465603
Kuchy LLevy RMejer ASegev NAgmon SFarber MFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Generating Product Insights from Community Q&AProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615480(4660-4666)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615480
Xu HLiu HLv ZYang QWang WFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Sentiment-aware Review Summarization with Personalized Multi-task Fine-tuningProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615056(2826-2835)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615056
Show More Cited By

Index Terms

Generating Tips from Product Reviews

Recommendations

The Tip of the Buyer: Extracting Product Tips from Reviews
Product reviews play a key role in e-commerce platforms. Studies show that many users read product reviews before a purchase and trust them to the same extent as personal recommendations. However, in many cases, the number of reviews per product is large ...
Scoring products from reviews through application of fuzzy techniques

e-Commerce is becoming an increasingly popular trend nowadays, where a consumer can buy a product through the internet. After buying a product, consumers can post their reviews and comments about the product in web sites and blogs. These comments can be ...
The Impact of Online Product Reviews on Product Returns

Although many researchers in information systems and marketing have studied the effect of product reviews on sales, few have looked at their effect on product returns. We hypothesize that, by reducing product uncertainty, product reviews affect the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

March 2021

1192 pages

ISBN:9781450382977

DOI:10.1145/3437963

General Chairs:
Liane Lewin-Eytan
Amazon, Israel
,
David Carmel
Amazon, Israel
,
Elad Yom-Tov
Microsoft, Israel
,
Program Chairs:
Eugene Agichtein
Emory University and Amazon, USA
,
Evgeniy Gabrilovich
Google Health, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM '21

Sponsor:

WSDM '21: The Fourteenth ACM International Conference on Web Search and Data Mining

March 8 - 12, 2021

Virtual Event, Israel

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
392
Total Downloads

Downloads (Last 12 months)35
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Allen THorwitz ASprigle S(2025)Leveraging AI and customer reviews to evaluate technology used by people with disabilitiesDisability and Rehabilitation: Assistive Technology10.1080/17483107.2025.2465603(1-9)Online publication date: 16-Feb-2025
https://doi.org/10.1080/17483107.2025.2465603
Kuchy LLevy RMejer ASegev NAgmon SFarber MFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Generating Product Insights from Community Q&AProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615480(4660-4666)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615480
Xu HLiu HLv ZYang QWang WFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Sentiment-aware Review Summarization with Personalized Multi-task Fine-tuningProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615056(2826-2835)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615056
Hirsch SNovgorodov SGuy INus A(2023)The Tip of the Buyer: Extracting Product Tips from ReviewsACM Transactions on Internet Technology10.1145/354714023:1(1-30)Online publication date: 23-Feb-2023
https://doi.org/10.1145/3547140
Xiong ZYan ZYao HLiang S(2022)Design Demand Trend Acquisition Method Based on Short Text Mining of User Comments in Shopping WebsitesInformation10.3390/info1303011013:3(110)Online publication date: 25-Feb-2022
https://doi.org/10.3390/info13030110
Gershtein SMilo TNovgorodov SRazmadze KIves ZBonifati AEl Abbadi A(2022)Classifier Construction Under Budget ConstraintsProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3517863(1160-1174)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3517863
Zhao MYang YLi MWang JWu WRen Pde Rijke MRen ZAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Personalized Abstractive Opinion TaggingProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3532037(1066-1076)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3532037
Farber MCarmel DKuchy LMejer AAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)Analyzing the Support Level for Tips Extracted from Product ReviewsProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531805(2059-2064)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3531805
Zang JDong A(2022)Stylistic Pattern Guided Tip Extraction from Music Reviews2022 4th International Conference on Data Intelligence and Security (ICDIS)10.1109/ICDIS55630.2022.00077(463-468)Online publication date: Aug-2022
https://doi.org/10.1109/ICDIS55630.2022.00077

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten