Sentiment classification of Internet restaurant reviews written in Cantonese

doi:10.1016/j.eswa.2010.12.147

Expert Systems with Applications

Volume 38, Issue 6, June 2011, Pages 7674-7682

https://doi.org/10.1016/j.eswa.2010.12.147 Get rights and content

Abstract

Cantonese is an important dialect in some regions of Southern China. Local online users often represent their opinions and experiences on the web with written Cantonese. Although the information in those reviews is valuable to potential consumers and sellers, the huge amount of web reviews make it difficult to give an unbiased evaluation to a product and the Cantonese reviews are unintelligible for Mandarin Chinese speakers.

In this paper, standard machine learning techniques naive Bayes and SVM are incorporated into the domain of online Cantonese-written restaurant reviews to automatically classify user reviews as positive or negative. The effects of feature presentations and feature sizes on classification performance are discussed. We find that accuracy is influenced by interaction between the classification models and the feature options. The naive Bayes classifier achieves as well as or better accuracy than SVM. Character-based bigrams are proved better features than unigrams and trigrams in capturing Cantonese sentiment orientation.

Research highlights

► Naive Bayes and SVM are used for Cantonese sentiment classification. ► Accuracy is influenced by interaction between classification models and features. ► Naive Bayes classifier achieves as well as or better accuracy than SVM. ► Character-based bigrams are better features than unigrams and trigrams in capturing Cantonese sentiment.

Introduction

The Internet continues to become an essential part of everyday life. people are now able to access not only opinions from family members and friends, but also from strangers located around the world who may have used a particular product, visited a certain destination, or seen a movie. Internet provides a virtual environment for consumers to share their experiences with world-wide travelers via the electronic word-of-mouth (WOM) communication channel (Cheung, Shek, & Sia, 2004). The importance of WOM has been widely documented in the existing literature (Cheung et al., 2004, Goldenberg et al., 2001). WOM not only strongly influences consumers’ decision making process (Goldenberg et al., 2001), but also has important implications for managers to consider their brand building, product development, and quality assurance (Dellarocas, 2003).

As today’s consumers are increasingly making their opinions and experiences available online (Horrigan, 2008), there have accumulated a huge amount of consumer reviews for products or service on the Web. When trying to locate user opinions of a product, a general online search will turns up millions of web pages. Getting an overall sense of those reviews can be daunting or time-consuming, however, if only few reviews were read the evaluation would be biased. Sentiment classification aims to address this problem by automatically classifying user reviews into positive or negative opinions.

Review sentiment classification has become one of the foci of recent research endeavors. Many sentiment classification techniques have been developed for English, Japanese, and Mandarin Chinese. But the interest in the sentiment analysis is worldwide to provide support for various NLP applications. Researches on automatic sentiment analysis should be conducted in more new languages such as the Cantonese.

Cantonese is an important dialect spoken in and around the cities of southern China where are typical areas with rapid development in China. In those areas, Cantonese is widely used in social settings and many native Cantonese consumers are not well literate in Mandarin Chinese. Take Hong Kong for example. According to statistics of Hong Kong Census and Statistics Department for 2006 population, Cantonese was the most commonly used language at home for about 91% of the population. Only about 40% of the population claimed to be able to speak Mandarin Chinese,¹ and the percent capable of writing would be less. Those Cantonese-speaking consumers are very likely to express themselves with written Cantonese in informal settings such as Internet forum; however, due to the difference between Cantonese and Mandarin Chinese, Mandarin speakers cannot read the online Cantonese contents (or finds it so difficult that the effort will rapidly be abandoned). Given the importance of written Cantonese (Snow, 2004), innovative techniques that can automatically detect the consumer opinions in Cantonese reviews are urgently required.

In this paper, standard machine learning techniques are incorporated into the domain of online Cantonese-written restaurant reviews to automatically classify user reviews as thumbs-up or thumbs-down. Two popular text classification algorithms – naive Bayes and SVM, and six feature presentations concerning n-gram presence/frequency are chosen to examine the effects of the classifiers and the feature options on Cantonese sentiment classification. This study seeks empirical answers to the following research questions:

1.
Dose the SVM classifier beat naive Bayes regarding Cantonese sentiment-based classification?
2.
Are high order n-grams better features than unigrams to capture sentiments in the Cantonese text?
3.
Is feature presence a better text presentation than feature frequency regarding feature selection and text classification?
4.
How dose the size of feature set affect the performance of classifiers?

Section snippets

Literature review

Sentiment classification aims to automatically classify the text of written reviews from customers into positive or negative opinions. It has emerged as a hot research area. While it is still in a preliminary stage, there has been much work related to various languages, such as English (Liu et al., 2005, Pang et al., 2002), Japanese (Fujii & Ishikawa, 2006), Mandarin Chinese (Ku, Liang, & Chen, 2006).

In this paper, we focus our interest on written Cantonese which can be viewed as a written

Data collection

Due to no benchmark data available, we created a corpus of Cantonese-written reviews by retrieving consumer reviews from a Cantonese site OpenRice (URL: http://www.openrice.com). The site allows diners to input text feedback and a three-point satisfaction rating for a restaurant located in Hong Kong. As the majority of OpenRice users are inhabitants of Hong Kong, the feedback are generally written in Cantonese with a few exceptions in English and Mandarin. A crawler was developed by Java to

Performance measures

The category assignments of a polarity classifier can be evaluated using a two-way contingency table (Table 3) which has four cells, where

–
cell a counts the documents correctly assigned to positive reviews;
–
cell b counts the documents incorrectly assigned to positive reviews;
–
cell c counts the documents incorrectly assigned to negative reviews;
–
cell d counts the documents correctly assigned to negative reviews.

The performance measures recall, precision and accuracy are defined and computed from the

Results and discussion

Three-fold cross-validation was performed for the experiments reported in this study. The experiments used our own implementation of a naive Bayes classifier and Chang and Lin’s (2001) LIBSVM implementation of a Support Vector Machine classifier with all parameters set to their default values. We ran each classifier with various-sized feature sets to examine the effects of feature size on sentiment classification performance. Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7, Fig. 8, Fig. 9, Fig.

Conclusion

This paper has shown that machine learning techniques perform quite well in the domain of Cantonese review classification. Despite its unrealistic independence assumption, the naive Bayes classifier surprisingly achieves comparable, or better performance than SVM. Interactions between classification methods and feature presentation options are observed, and bigram frequency is proved the effective feature in capturing sentiments in the Cantonese text. In addition, we look at the effects of

Acknowledgments

This study was partially funded by National Science Foundation of China (70971033, 70890082) and NCET-08-0172.

References (29)

Q. Ye et al.
Sentiment classification of online reviews to travel destinations by supervised machine learning approaches
Expert Systems with Applications
(2009)
Chang, C.-C., Lin, C.-J. (2001). LIBSVM: A library for support vector machines. Software available at...
K. Cheung et al.
The representation of Cantonese with Chinese characters
(2002)
Cheung, C. M. Y., Shek, S. P. W., Sia, C. L. (2004). Virtual community of consumers: Why people are willing to...
S.R. Das et al.
Yahoo! for Amazon: Sentiment extraction from small talk on the web
Management Science
(2007)
Dave, K., Lawrence, S., Pennock, D. M. (2003). Mining the peanut gallery: Opinion extraction and semantic...
C. Dellarocas
The digitization of word of mouth: Promise and challenges of online feedback mechanisms
Management Science
(2003)
P. Domingos et al.
Beyond independence: Conditions for the optimality of the simple Bayesian classifier
Machine Learning
(1997)
Fujii, A., Ishikawa, T. (2006). A system for summarizing and visualizing arguments in subjective documents: Toward...
J. Goldenberg et al.
Talk of the network: A complex systems look at the underlying process of word-of-mouth
Marketing Letters
(2001)

U. Gretzel et al.

Use and impact of online travel reviews

(2008)

Hatzivassiloglou, V., McKeown, K. (1997). Predicting the semantic orientation of adjectives. In: Proceedings of the...

M. Hearst

Direction-based text interpretation as an information access refinement

(1992)

Horrigan, J. A. (2008). Online shopping, pew Internet & American life project...

Cited by (187)

Sentiment analysis of financial Twitter posts on Twitter with the machine learning classifiers
2024, Heliyon
This paper presents a sentiment analysis combining the lexicon-based and machine learning (ML)-based approaches in Turkish to investigate the public mood for the prediction of stock market behavior in BIST30, Borsa Istanbul. Our main motivation behind this study is to apply sentiment analysis to financial-related tweets in Turkish. We import 17189 tweets posted as "#Borsaistanbul, #Bist, #Bist30, #Bist100″ on Twitter between November 7, 2022, and November 15, 2022, via a MAXQDA 2020, a qualitative data analysis program. For the lexicon-based side, we use a multilingual sentiment offered by the Orange program to label the polarities of the 17189 samples as positive, negative, and neutral labels. Neutral labels are discarded for the machine learning experiments. For the machine learning side, we select 9076 data as positive and negative to implement the classification problem with six different supervised machine learning classifiers conducted in Python 3.6 with the sklearn library. In experiments, 80 % of the selected data is used for the training phase and the rest is used for the testing and validation phase. Results of the experiments show that the Support Vector Machine and Multilayer Perceptron classifier perform better than other classifiers with 0.89 and 0.88 accuracy and AUC values of 0.8729 and 0.8647 respectively. Other classifiers obtain approximately a 78,5 % accuracy rate. It is possible to increase sentiment analysis accuracy with parameter optimization on a larger, cleaner, and more balanced dataset by changing the pre-processing steps. This work can be expanded in the future to develop better sentiment analysis using deep learning approaches.
Measuring service quality with text analytics: Considering both importance and performance of consumer opinions on social and non-social online platforms
2023, Journal of Business Research
Online word-of-mouth (WOM) has attracted considerable attention from researchers due to its abundant information on customer perceptions that drive product and service improvement. This study develops a novel weighted service quality (WSQ) metric derived from online customer opinions, leveraging the importance-performance analysis framework. Data collected from social and non-social online platforms confirms that this WSQ approach outperforms the widely used average sentiment score approach and significantly predicts the industry service quality standard, Airline Quality Rating (AQR). In addition, the WSQ metric derived from social media proves to be a more vital indicator for AQR than that derived from a non-social online platform. A significant difference in topic distributions was also identified between consumer opinions from social media and non-social online platforms. Our study makes several crucial contributions to the service quality literature on employing online WOM using sentiment analysis and topic modeling techniques.
Improving the accuracy of sentiment analysis using a linguistic rule-based feature selection method in tourism reviews
2023, Measurement: Sensors
Sentiment Analysis technique involves extracting the relevant information from Unstructured User Reviews (UUR) dataset fetched from online and classifying them into appropriate positive and negative comments for making decisions. In UUR, data may be in noisy state, irrelevant features exist which creates high dimensional feature space. To design an effective sentiment learning model, users are required to extract the most relevant sentiment features from UUR. To overcome the issue, we proposed a Linguistic rule based feature selection method for extracting and selecting the sentiment features for Sentiment Analysis as it improves the predictive performance of classification algorithms. The proposed novel feature selection method involves identifying the various sentiment features in the review dataset by using filtering methods such as POS tags, n-grams. In the ensemble model, where the Random Forest classification algorithm is trained for textual sentiment classification, the chosen sentiment feature sets are used. Finally, we test our approach using the real-time review dataset that was collected from a multitude of sources, and the results demonstrate prediction accuracy that is superior to that of existing Sentiment analysis techniques.
Blowing minds with exploding dish names/images: The effect of implied explosion on consumer behavior in a restaurant context
2023, Tourism Management
Dish names and dish images can be widely found online, providing consumers with important information. Meanwhile, implied explosion (i.e., the perception of explosion induced by static stimuli) is increasingly utilized by real-world restaurants. The present research thus combines dish names, dish images, and implied explosion to examine the impact of implied explosion on various aspects of consumer behavior within a restaurant context. Three experiments demonstrated that exploding dish names and exploding dish images (i.e., dish names/dish images showing implied explosion) can create a more intense taste perception and a more favorable taste evaluation. Additionally, exploding dish images can enhance perceived dish liking and increase consumers’ willingness to pay. The present research suggests that exploding dish names/dish images are subtle but effective communication tools for the tourism industry, helping to deliver a more stimulating perception and experience to consumers and to generate higher margins. By exploring the effects of implied explosion, we also introduce the implied motion concept to the tourism management literature.
An online reviews-driven large-scale group decision making approach for evaluating user satisfaction of sharing accommodation
2023, Expert Systems with Applications
To promote the development of the peer-to-peer (P2P) accommodation in sharing economy, it is important to understand and ensure the determinants of high-level user satisfaction. Focusing on factors that affect the travel experience of P2P accommodation users, this article proposes a large-scale group decision making (LSGDM) based method with online reviews to evaluate user satisfaction of sharing accommodation. Firstly, the user demands (UDs) reflecting the actual concerns of users are extracted by combining negative and positive reviews from P2P accommodation platform Airbnb, where negative reviews are classified by sentiment analysis, TF-IDF and Word2Vec technology are used for extraction and the further verification of UDs from all online reviews is performed. Secondly, the level of satisfaction is evaluated through online responses from P2P accommodation users in a large-scale group of decision makers. Thirdly, the final degrees of satisfaction are ascertained by the proposed LSGDM approach, which includes subgroup clustering and a minimal variance weights based feedback mechanism for fair weights allocation under the condition of reaching consensus. Finally, the conclusions further serve as references for improving the performance of P2P sharing accommodation.
Thematic analysis of reviews on the air quality of tourist destinations from a sentiment analysis perspective
2022, Tourism Management Perspectives
This thematic analysis examines whether reviews on transactional and social media websites can reflect the air quality of a tourist destination. We used linguistic and sentiment analysis methods to establish an analytical framework for assessing the credibility of the reviews with sufficiency and consistency analyses. We collected Ctrip and Sina Weibo reviews to analyze the sentiment values using deep learning and Baidu sentiment dictionary methods. We found that although the sentiment value of the Ctrip transactional comments on air quality was high, they hardly reflected reality. Conversely, the Sino Weibo social media comments were highly credible, despite their low sentiment values. Tourists' perception of air quality is mainly affected by intangible air factors (such as pollutants), then tangible air factors, hydrology factors and terrain factors. The study uses online reviews to analyze air quality and provides a reference for the environmental management of destinations and decision making among tourists.

View all citing articles on Scopus

View full text

Sentiment classification of Internet restaurant reviews written in Cantonese

Abstract

Research highlights

Introduction

Section snippets

Literature review

Data collection

Performance measures

Results and discussion

Conclusion

Acknowledgments

Expert Systems with Applications

The representation of Cantonese with Chinese characters

Yahoo! for Amazon: Sentiment extraction from small talk on the web

Management Science

The digitization of word of mouth: Promise and challenges of online feedback mechanisms

Management Science

Beyond independence: Conditions for the optimality of the simple Bayesian classifier

Machine Learning

Talk of the network: A complex systems look at the underlying process of word-of-mouth

Marketing Letters

Use and impact of online travel reviews

Direction-based text interpretation as an information access refinement