short-paper

How features in iOS App Store Reviews can Predict Developer Responses

Authors:

Kamonphop Srisopha,

Devendra Swami,

Barry BoehmAuthors Info & Claims

EASE '20: Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering

Pages 336 - 341

https://doi.org/10.1145/3383219.3383258

Published: 17 April 2020 Publication History

Abstract

Until recently, communications regarding apps on the iOS App Store have been one-way from users to developers, with developers unable to respond to reviews directly. While studies have shown that responding to reviews improves an app's overall rating and user satisfaction, resource limitations make it so developers can usually only respond to some of the reviews. Although developers' response behavior has been studied, little is known about which features (aspects) of user reviews spur their responses. Motivated by these observations, we investigate a wide range of features that can be extracted from a user review and apply a random forest algorithm and the features it extracts to predict whether developers will respond to that review. We then determine the importance of these features in distinguishing reviews that receive a developer response from those that do not. Through a case study of three popular free-to-download iOS apps, we find that although features such as rating and review length are among the most important features for all apps, each app has its own individual feature importance ranking, indicating that developers assign different feature weights when prioritizing reviews. Our results may help guide research or the development of tools that are more in line with developers' actual response behavior.

References

[1]

Apple. Ratings, reviews, and responses. https://developer.apple.com/app-store/ratings-and-reviews/. Accessed: 2019-06--04.

[2]

Apptentive. The guide to mobile app ratings and reviews. go.apptentive.com/RatingsReviews.html. Accessed: 2019-12--04.

[3]

Bailey, K., Nagappan, M., and Dig, D. Examining user-developer feedback loops in the ios app store. In Proceedings of the 52nd Hawaii International Conference on System Sciences (2019).

[4]

Breiman, L. Random forests. Machine learning 45, 1 (2001), 5--32.

Digital Library

[5]

Di Sorbo, A., Panichella, S., Alexandru, C. V., Shimagaki, J., Visaggio, C. A., Canfora, G., and Gall, H. C. What would users change in my app? summarizing app reviews for recommending software changes. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (2016), ACM, pp. 499--510.

Digital Library

[6]

Gao, C., Zeng, J., Xia, X., Lo, D., Lyu, M. R., and King, I. Automating app review response generation. In Proceedings of the 34th ACM/IEEE International Conference on Automated Software Engineering (2019), ACM.

Digital Library

[7]

Greenheld, G., Savarimuthu, B. T. R., and Licorish, S. A. Automating developers' responses to app reviews. In 2018 25th Australasian Software Engineering Conference (ASWEC) (2018), IEEE, pp. 66--70.

[8]

Guzman, E., and Maalej, W. How do users like this feature? a fine grained sentiment analysis of app reviews. In 2014 IEEE 22nd international requirements engineering conference (RE) (2014), IEEE, pp. 153--162.

[9]

Hassan, S., Tantithamthavorn, C., Bezemer, C.-P., and Hassan, A. E. Studying the dialogue between users and developers of free apps in the google play store. Empirical Software Engineering 23, 3 (2018), 1275--1312.

Digital Library

[10]

Hu, M., and Liu, B. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining (2004), ACM, pp. 168--177.

Digital Library

[11]

Khalid, H., Shihab, E., Nagappan, M., and Hassan, A. E. What do mobile app users complain about? IEEE Software 32, 3 (2015), 70--77.

Digital Library

[12]

Kraskov, A., STögbauer, H., and Grassberger, P. Estimating mutual information. Physical review E 69, 6 (2004), 066138.

[13]

Maalej, W., and Nabil, H. Bug report, feature request, or simply praise? on automatically classifying app reviews. In 2015 IEEE 23rd international requirements engineering conference (RE) (2015), IEEE, pp. 116--125.

[14]

Man, Y., Gao, C., Lyu, M. R., and Jiang, J. Experience report: Understanding cross-platform app issues from user reviews. In 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) (2016), IEEE, pp. 138--149.

[15]

Martin, W., Harman, M., Jia, Y., Sarro, F., and Zhang, Y. The app sampling problem for app store mining. In Proceedings of the 12th Working Conference on Mining Software Repositories (2015), IEEE Press, pp. 123--133.

Digital Library

[16]

Martin, W., Sarro, F., Jia, Y., Zhang, Y., and Harman, M. A survey of app store analysis for software engineering. IEEE transactions on software engineering 43, 9 (2016), 817--847.

[17]

McIlroy, S., Shang, W., Ali, N., and Hassan, A. E. Is it worth responding to reviews? studying the top free apps in google play. IEEE Software 34, 3 (2015), 64--71.

[18]

Novak, P. K., Smailović, J., Sluban, B., and Mozetič, I. Sentiment of emojis. PloS one 10, 12 (2015), e0144296.

[19]

Omondiagbe, O. P., Licorish, S. A., and MacDonell, S. G. Features that predict the acceptability of java and javascript answers on stack overflow. In Proceedings of the Evaluation and Assessment on Software Engineering (2019), ACM, pp. 101--110.

Digital Library

[20]

Pagano, D., and Maalej, W. User feedback in the appstore: An empirical study. In 2013 21st IEEE international requirements engineering conference (RE) (2013), IEEE, pp. 125--134.

[21]

Palomba, F., Salza, P., Ciurumelea, A., Panichella, S., Gall, H., Ferrucci, F., and De Lucia, A. Recommending and localizing change requests for mobile apps based on user reviews. In Proceedings of the 39th international conference on software engineering (2017), IEEE Press, pp. 106--117.

Digital Library

[22]

Panichella, S., Di Sorbo, A., Guzman, E., Visaggio, C. A., Canfora, G., and Gall, H. C. Ardoc: App reviews development oriented classifier. In Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering (2016), ACM, pp. 1023--1027.

Digital Library

[23]

Salehan, M., and Kim, D. J. Predicting the performance of online consumer reviews: A sentiment mining approach to big data analytics. Decision Support Systems 81 (2016), 30--40.

Digital Library

[24]

Savarimuthu, B., Licorish, S., Devananda, M., Greenheld, G., Dignum, V., and Dignum, F. Developers' responses to app review feedback-a study of communication norms in app development. In Proc. of the COIN 2017 workshop@AAMAS (2017) (2017).

[25]

Schindler, R. M., and Bickart, B. Perceived helpfulness of online consumer reviews: The role of message content and style. Journal of Consumer Behaviour 11, 3 (2012), 234--243.

[26]

Strobl, C., Boulesteix, A.-L., Kneib, T., Augustin, T., and Zeileis, A. Conditional variable importance for random forests. BMC bioinformatics 9, 1 (2008), 307.

[27]

Thelwall, M., Buckley, K., Paltoglou, G., Cai, D., and Kappas, A. Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology 61, 12 (2010), 2544--2558.

[28]

Yang, L., Dumais, S. T., Bennett, P. N., and Awadallah, A. H. Characterizing and predicting enterprise email reply behavior. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (2017), ACM, pp. 235--244.

Digital Library

Cited By

Khan NAli Khan JLi JUllah TAlwadain AYasin AZhao Q(2024)How Do Crowd-Users Express Their Opinions Against Software Applications in Social Media? A Fine-Grained Classification ApproachIEEE Access10.1109/ACCESS.2024.342583012(98004-98028)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3425830
Alismail MAlbesher A(2023)Evaluating Developer Responses to App Reviews: The Case of Mobile Banking Apps in Saudi Arabia and the United StatesSustainability10.3390/su1508670115:8(6701)Online publication date: 15-Apr-2023
https://doi.org/10.3390/su15086701
Dąbrowski JLetier EPerini ASusi A(2022)Analysing app reviews for software engineering: a systematic literature reviewEmpirical Software Engineering10.1007/s10664-021-10065-727:2Online publication date: 20-Jan-2022
https://dl.acm.org/doi/10.1007/s10664-021-10065-7
Show More Cited By

Index Terms

How features in iOS App Store Reviews can Predict Developer Responses
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
  2. Information systems applications
    1. Data mining
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
      1. Software evolution

Recommendations

Learning Features that Predict Developer Responses for iOS App Store Reviews
ESEM '20: Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

Background: Which aspects of an iOS App Store user review motivate developers to respond? Numerous studies have been conducted to extract useful information from reviews, but limited effort has been expended to answer this question.

Aims: This work aims ...
How Should Developers Respond to App Reviews? Features Predicting the Success of Developer Responses
EASE '21: Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering

Context: The Google Play Store allows app developers to respond to user reviews. Existing research shows that response strategies vary considerably. In addition, while responding to reviews can lead to several types of favorable outcomes, not every ...
App store mining is not enough for app improvement

The rise in popularity of mobile devices has led to a parallel growth in the size of the app store market, intriguing several research studies and commercial platforms on mining app stores. App store reviews are used to analyze different aspects of app ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '20: Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering

April 2020

544 pages

ISBN:9781450377317

DOI:10.1145/3383219

General Chairs:
Jingyue Li
Norwegian University of Science and Technology, Norway
,
Letizia Jaccheri
Norwegian University of Science and Technology, Norway
,
Program Chairs:
Torgeir Dingsøyr
SINTEF Digital and Norwegian University of Science and Technology, Norway
,
Ruzanna Chitchyan
University of Bristol, UK

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

NTNU: Norwegian University of Science and Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

EASE '20

EASE '20: Evaluation and Assessment in Software Engineering

April 15 - 17, 2020

Trondheim, Norway

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
299
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khan NAli Khan JLi JUllah TAlwadain AYasin AZhao Q(2024)How Do Crowd-Users Express Their Opinions Against Software Applications in Social Media? A Fine-Grained Classification ApproachIEEE Access10.1109/ACCESS.2024.342583012(98004-98028)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3425830
Alismail MAlbesher A(2023)Evaluating Developer Responses to App Reviews: The Case of Mobile Banking Apps in Saudi Arabia and the United StatesSustainability10.3390/su1508670115:8(6701)Online publication date: 15-Apr-2023
https://doi.org/10.3390/su15086701
Dąbrowski JLetier EPerini ASusi A(2022)Analysing app reviews for software engineering: a systematic literature reviewEmpirical Software Engineering10.1007/s10664-021-10065-727:2Online publication date: 20-Jan-2022
https://dl.acm.org/doi/10.1007/s10664-021-10065-7
Srisopha KLink DSwami DBoehm B(2020)Learning Features that Predict Developer Responses for iOS App Store ReviewsProceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1145/3382494.3410686(1-11)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3382494.3410686

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten