An Empirical Analysis on Leveraging User Reviews with NLP-Enhanced Word Embeddings for App Rating Prediction

Abstract

The growth in mobile applications (apps) is rapid and immense. What was once a technology to satisfy the basic needs of a mobile user has now become a huge source of entertainment, education, games, etc. App development is iterative and incremental; therefore, they are released in improved versions to achieve higher app success in both functional and non-functional aspects. In this generic setting, the increasing prevalence of user reviews and generated ratings offers a wealth of insights, especially for post-development analysis. However, user reviews can be highly variable and inconsistent, as the informal nature of the language allows for misinterpretation, leading to a daunting task. In this paper, an empirical analysis is conducted to assert the effectiveness of NLP techniques in modeling the semantic contexts of user reviews in embedding vectors for predicting mobile app ratings. Here, each contextual macro-category is preprocessed with SMOTE to handle skewed distribution and then subjected to 4 feature selection techniques, e.g. PCA, ANOVA, etc. In the last step, 13 different prediction models (classifiers) are fitted to predict the mobile app rating. The study confirms significantly improved predictive performance in Glove-T and GloVe-W equipped word embedding models. Among prediction models, logistic regression (LOGR) outperforms the equivalent with an overall predictive ability of a greater extent, with an average accuracy score of 67, an average AUC of 0.70, and an F-measure of 0.72, indicative of its high functioning with numerical data and word vectors.