skip to main content
10.1145/3530019.3535346acmotherconferencesArticle/Chapter ViewAbstractPublication PageseaseConference Proceedingsconference-collections
research-article

Better Identifying and Addressing Diverse Issues in mHealth and Emerging Apps Using User Reviews

Published: 13 June 2022 Publication History

Abstract

The COVID-19 pandemic has changed the way we live, leading to a rapid expansion of mHealth apps usage. The pandemic also led to the introduction of a large number of ”emerging apps” to the mobile app market. mHealth and emerging app users have reported a range of serious issues in their user reviews, which we identified and better understood after extracting, translating, analysing and classifying over 6 millions user reviews of these apps into different aspects. As evidenced by user reviews, many mHealth and emerging apps are plagued by major issues and problems. App developers could improve the quality and adoption of their apps if they had a better grasp of the major concerns raised by their users. We also link the findings from our user review analysis to the app version history release notes to better understand and identify what issues the developers of mHealth/emerging apps managed to solve or not. Investigating the association between user reviews and app updates will allow us to design a model for mHealth/emerging app developers to follow. We identified that our recommendation models and tools can assist mHealth/emerging app developers and designers in proactively identifying and preventing software and design issues before their final apps are deployed to mobile users. A proactive evaluation model and discovering mHealth/emerging issues early can save billions of dollars and avert millions of deaths a year. As a result, more people will download these apps when they believe that the updates are actually addressing and solving their problems, which will result in saving lives and improving the quality of life for people with disabilities or those who use these apps.

References

[1]
Rajindra Adhikari, Deborah Richards, and Karen Scott. 2014. Security and privacy issues related to the use of mobile health apps. ACIS.
[2]
Felwah Alqahtani and Rita Orji. 2020. Insights from user reviews to improve mental health apps. Health informatics journal 26, 3 (2020), 2042–2066.
[3]
Kevin Anderson, Oksana Burford, and Lynne Emmerton. 2016. Mobile health apps to facilitate self-care: a qualitative study of user experiences. PloS one 11, 5 (2016), e0156164.
[4]
Muneera Bano, Chetan Arora, Didar Zowghi, and Alessio Ferrari. 2021. The Rise and Fall of COVID-19 Contact Tracing Apps: when NFRs Collide with Pandemic. In 29th IEEE International Requirements Engineering Conference (RE). IEEE.
[5]
Oyungerel Byambasuren, Sharon Sanders, Elaine Beller, and Paul Glasziou. 2018. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ digital medicine 1, 1 (2018), 1–12.
[6]
Ning Chen, Jialiu Lin, Steven CH Hoi, Xiaokui Xiao, and Boshen Zhang. 2014. AR-miner: mining informative reviews for developers from mobile app marketplace. In Proc. 36th Int. Conf. on software engineering. 767–778.
[7]
Marco Ciotti, Massimo Ciccozzi, Alessandro Terrinoni, Wen-Can Jiang, Cheng-Bin Wang, and Sergio Bernardini. 2020. The COVID-19 pandemic. Critical reviews in clinical laboratory sciences 57, 6 (2020), 365–388.
[8]
Laura Di Domenico, Giulia Pullano, Chiara E Sabbatini, Pierre-Yves Boëlle, and Vittoria Colizza. 2020. Impact of lockdown on COVID-19 epidemic in Île-de-France and possible exit strategies. BMC medicine 18, 1 (2020), 1–13.
[9]
John Drury and S Tekin Guven. 2020. Emergencies and disasters. Together apart the psychology of COVID-19(2020), 97–101.
[10]
Mattia Fazzini, Hourieh Khalajzadeh, Omar Haggag, Zhaoqing Li, Humphrey Obie, Chetan Arora, Waqar Hussain, and John Grundy. 2022. Characterizing Human Aspects in Reviews of COVID-19 Apps. In 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems 2022 (MobileSoft 2022).
[11]
Maria Gomez, Matias Martineza, Martin Monperrus, and Romain Rouvoy. 2015. When app stores listen to the crowd to fight bugs in the wild. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 2. IEEE, 567–570.
[12]
John Grundy, Hourieh Khalajzadeh, and Jennifer Mcintosh. 2020. Towards Human-centric Model-driven Software Engineering. In ENASE. 229–238.
[13]
John Grundy, Hourieh Khalajzadeh, Jennifer McIntosh, Tanjila Kanij, and Ingo Mueller. 2020. HumaniSE: Approaches to Achieve More Human-Centric Software Engineering. In International Conference on Evaluation of Novel Approaches to Software Engineering. Springer, 444–468.
[14]
Xiaodong Gu and Sunghun Kim. 2015. ” what parts of your apps are loved by users?”(T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 760–770.
[15]
Emitza Guzman and Walid Maalej. 2014. How do users like this feature? a fine grained sentiment analysis of app reviews. In 2014 IEEE 22nd international requirements engineering conference (RE). IEEE, 153–162.
[16]
Omar Haggag, John Grundy, Mohamed Abdelrazek, and Sherif Haggag. 2022. Better Addressing Diverse Accessibility Issues in Emerging Apps: A Case Study using COVID-19 Apps. In 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems 2022 (MobileSoft 2022).
[17]
Omar Haggag, Sherif Haggag, John Grundy, and Mohamed Abdelrazek. 2021. COVID-19 vs Social Media Apps: Does Privacy Really Matter?. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). IEEE, 48–57.
[18]
Kayla Joanne Heffernan, Shanton Chang, Skye Tamara Maclean, Emma Teresa Callegari, Suzanne Marie Garland, Nicola Jane Reavley, George Andrew Varigos, and John Dennis Wark. 2016. Guidelines and recommendations for developing interactive eHealth apps for complex messaging in health promotion. JMIR mHealth and uHealth 4, 1 (2016), e14.
[19]
John P Higgins. 2016. Smartphone applications for patients’ health and fitness. The American journal of medicine 129, 1 (2016), 11–19.
[20]
Claudia Iacob and Rachel Harrison. 2013. Retrieving and analyzing mobile apps feature requests from online reviews. In 2013 10th working conference on mining software repositories (MSR). IEEE, 41–44.
[21]
Claudia Iacob, Varsha Veerappa, and Rachel Harrison. 2013. What are you complaining about?: a study of online reviews of mobile applications. In 27th International BCS Human Computer Interaction Conference (HCI 2013) 27. 1–6.
[22]
IQVIA Institute. 2020. The Growing Value of Digital Health. Retrieved Apr 10, 2022 from https://www.iqvia.com/insights/the-iqvia-institute/reports/the-growing-value-of-digital-health
[23]
Nishant Jha and Anas Mahmoud. 2019. Mining non-functional requirements from app store reviews. Empirical Software Engineering 24, 6 (2019), 3659–3695.
[24]
Timo Johann, Christoph Stanik, Walid Maalej, 2017. Safe: A simple approach for feature extraction from app descriptions and app reviews. In 2017 IEEE 25th International Requirements Engineering Conference (RE). IEEE, 21–30.
[25]
Rodney P Joseph, Nefertiti H Durant, Tanya J Benitez, and Dorothy W Pekmezi. 2014. Internet-based physical activity interventions. American journal of lifestyle medicine 8, 1 (2014), 42–67.
[26]
Shaidah Jusoh. 2017. A survey on trend, opportunities and challenges of mHealth apps.Int. J. Interact. Mob. Technol. 11, 6 (2017), 73–85.
[27]
Huiying Li, Li Zhang, Lin Zhang, and Jufang Shen. 2010. A user satisfaction analysis approach for software evolution. In 2010 IEEE International Conference on Progress in Informatics and Computing, Vol. 2. IEEE, 1093–1097.
[28]
Xiaozhou Li, Zheying Zhang, and Kostas Stefanidis. 2018. Mobile App Evolution Analysis Based on User Reviews. In SoMeT. 773–786.
[29]
Walid Maalej, Zijad Kurtanović, Hadeer Nabil, and Christoph Stanik. 2016. On the automatic classification of app reviews. Requirements Engineering 21, 3 (2016), 311–331.
[30]
Walid Maalej and Hadeer Nabil. 2015. Bug report, feature request, or simply praise? on automatically classifying app reviews. In 2015 IEEE 23rd international requirements engineering conference (RE). IEEE, 116–125.
[31]
Amy Nordrum 2016. Popular internet of things forecast of 50 billion devices by 2020 is outdated. IEEE spectrum 18, 3 (2016).
[32]
Patrick Howell O’Neill. 2020. No, coronavirus apps don’t need 60% adoption to be effective. Retrieved Apr 10, 2022 from https://www.technologyreview.com/2020/06/05/1002775/covid-apps-effective-at-less-than-60-percent-download
[33]
World Health Organisation. 2020. Physical inactivity: a global public health problem. Retrieved Apr 10, 2022 from http://www.who.int/dietphysicalactivity/factsheet_inactivity/en/
[34]
Brian A O’Shea and Michiko Ueda. 2021. Who is more likely to ignore experts’ advice related to COVID-19?Preventive medicine reports 23 (2021), 101470.
[35]
Hewlett Packard. 2015. Failing to meet mobile app user expectations: a mobile user survey. Tech. rep. (2015).
[36]
Debajyoti Pal, Vajirasak Vanijja, and Syamal Patra. 2020. Online Learning During COVID-19: Students’ Perception of Multimedia Quality. In Proceedings of the 11th International Conference on Advances in Information Technology. 1–6.
[37]
Francisca Pimenta, Laís Lopes, Frederica Gonçalves, and Pedro Campos. 2020. Designing Positive Behavior Change Experiences: a Systematic Review and Sentiment Analysis based on Online User Reviews of Fitness and Nutrition Mobile Applications. In 19th International Conference on Mobile and Ubiquitous Multimedia. 152–161.
[38]
Zhilei Qiao, G Alan Wang, Mi Zhou, and Weiguo Fan. 2018. The impact of customer reviews on product innovation: Empirical evidence in mobile apps. In Analytics and Data Science. Springer, 95–110.
[39]
Enrico Louis Quarantelli. 2000. Emergencies, disasters and catastrophes are different phenomena. (2000).
[40]
Kaavya Rekanar, Ian R O’Keeffe, Sarah Buckley, Manzar Abbas, Sarah Beecham, Muslim Chochlov, Brian Fitzgerald, Liam Glynn, Kevin Johnson, John Laffey, 2021. Sentiment analysis of user feedback on the HSE’s Covid-19 contact tracing app. Irish Journal of Medical Science (1971-)(2021), 1–10.
[41]
Marion Lara Tan, Raj Prasanna, Kristin Stock, Emma Hudson-Doyle, Graham Leonard, and David Johnston. 2017. Mobile applications in crisis informatics literature: A systematic review. International journal of disaster risk reduction 24 (2017), 297–311.
[42]
Thomas Timmers, Loes Janssen, Joep Stohr, JL Murk, and MAH Berrevoets. 2020. Using eHealth to support COVID-19 education, self-assessment, and symptom monitoring in the Netherlands: observational study. JMIR mHealth and uHealth 8, 6 (2020), e19822.
[43]
Google Translate. 2021. Python client library. Retrieved Apr 10, 2022 from https://cloud.google.com/translate/docs/reference/libraries/v2/python
[44]
Oxford University. 2020. Digital contact tracing can slow or even stop coronavirus transmission and ease us out of lockdown. Retrieved Apr 10, 2022 from https://www.research.ox.ac.uk/Article/2020-04-16-digital-contact-tracing-can-slow-or-even-stop-coronavirus-transmission-and-ease-us-out-of-lockdown
[45]
Rajesh Vasa, Leonard Hoon, Kon Mouzakis, and Akihiro Noguchi. 2012. A preliminary analysis of mobile app user reviews. In Proc. 24th Australian Computer-Human Interaction Conference. 241–244.
[46]
Phong Minh Vu, Tam The Nguyen, Hung Viet Pham, and Tung Thanh Nguyen. 2015. Mining user opinions in mobile app reviews: A keyword-based approach (t). In 2015 30th IEEE/ACM Int. Conf. on Automated Software Engineering (ASE). IEEE, 749–759.
[47]
Worldometers. 2021. COVID-19 CORONAVIRUS PANDEMIC LIVE UPDATES. Retrieved Feb 2, 2022 from https://www.worldometers.info/coronavirus/

Cited By

View all
  • (2024)How to effectively mine app reviews concerning software ecosystem? A survey of review characteristicsJournal of Systems and Software10.1016/j.jss.2024.112040213(112040)Online publication date: Jul-2024
  • (2024)Potential effectiveness and efficiency issues in usability evaluation within digital healthJournal of Systems and Software10.1016/j.jss.2023.111881208:COnline publication date: 4-Mar-2024
  • (2023)Association Between the Characteristics of mHealth Apps and User Input During Development and Testing: Secondary Analysis of App Assessment DataJMIR mHealth and uHealth10.2196/4693711(e46937)Online publication date: 22-Nov-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
EASE '22: Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering
June 2022
466 pages
ISBN:9781450396134
DOI:10.1145/3530019
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Accessibility
  2. Analysis
  3. COVID-19 Apps
  4. Emerging Apps
  5. Evaluation
  6. Guidelines
  7. Mobile Apps
  8. Privacy
  9. Recommendations
  10. User Reviews
  11. Version History
  12. mHealth Apps

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

EASE 2022

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)45
  • Downloads (Last 6 weeks)4
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)How to effectively mine app reviews concerning software ecosystem? A survey of review characteristicsJournal of Systems and Software10.1016/j.jss.2024.112040213(112040)Online publication date: Jul-2024
  • (2024)Potential effectiveness and efficiency issues in usability evaluation within digital healthJournal of Systems and Software10.1016/j.jss.2023.111881208:COnline publication date: 4-Mar-2024
  • (2023)Association Between the Characteristics of mHealth Apps and User Input During Development and Testing: Secondary Analysis of App Assessment DataJMIR mHealth and uHealth10.2196/4693711(e46937)Online publication date: 22-Nov-2023
  • (2023)A review of the factors influencing adoption of digital health applications for people living with dementiaDIGITAL HEALTH10.1177/205520762311629859Online publication date: 15-Mar-2023
  • (2023)“It’s Another Feather in My Hat”-Exploring Factors Influencing the Adoption of Apps With People Living With DementiaDementia10.1177/1471301223118528322:7(1487-1513)Online publication date: 26-Jun-2023
  • (2023)Runtime Monitoring of Human-Centric Requirements in Machine Learning Components: A Model-Driven Engineering Approach2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C)10.1109/MODELS-C59198.2023.00040(146-152)Online publication date: 1-Oct-2023
  • (2022)A large scale analysis of mHealth app user reviewsEmpirical Software Engineering10.1007/s10664-022-10222-627:7Online publication date: 1-Dec-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media