research-article

Better addressing diverse accessibility issues in emerging apps: a case study using COVID-19 apps

Authors:

Mohamed Abdelrazek,

Sherif HaggagAuthors Info & Claims

MOBILESoft '22: Proceedings of the 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems

Pages 50 - 61

https://doi.org/10.1145/3524613.3527817

Published: 17 October 2022 Publication History

Abstract

The COVID-19 pandemic resulted in introducing a large number of "emerging apps" to the mobile app market. These apps were developed and deployed quickly to address the urgency of the situation. This gave us an indication that the cycle of having new emerging apps will likely reoccur in every upcoming emergency in the future e.g. for advice and guidance during bush fires, floods, other pandemics, etc. We carried out an in-depth analysis of user reviews and version history release notes for 30 COVID-19 apps that were developed in a great hurry in 2020. We identified many diverse accessibility issues that exist, not just related to conventional challenged end-user accessibility issues, but including the ability to register, access, download, and use from different app stores in different countries and for different end-users. From this large-scale analysis, we developed a new advisory tool for software developers of emerging apps to avoid many of the wide accessibility issues presented in these COVID-19 apps. A user evaluation of our prototype tool with 13 real-world app developers indicates it will assist developers to address many of these issues prior to initial emerging app deployment.

References

[1]

Daniel Allington and Nayana Dhavan. 2020. The relationship between conspiracy beliefs and compliance with public health guidance with regard to COVID-19. (2020).

[2]

Apple. 2021. App Store. Retrieved Feb 2, 2022 from https://www.apple.com/au/app-store/

[3]

Muneera Bano, Chetan Arora, Didar Zowghi, and Alessio Ferrari. 2021. The Rise and Fall of COVID-19 Contact Tracing Apps: when NFRs Collide with Pandemic. In 29th IEEE International Requirements Engineering Conference (RE). IEEE.

[4]

Surekha Borra. 2020. COVID-19 apps: Privacy and security concerns. In Intelligent Systems and Methods to Combat Covid-19. Springer, 11--17.

[5]

Laura Bradford, Mateo Aboy, and Kathleen Liddell. 2020. COVID-19 contact tracing apps: a stress test for privacy, the GDPR, and data protection regimes. Journal of Law and the Biosciences 7, 1 (2020), lsaa034.

[6]

Eugene Y Chan and Najam U Saqib. 2021. Privacy concerns can explain unwillingness to download and use contact tracing apps when COVID-19 concerns are high. Computers in Human Behavior 119 (2021), 106718.

Digital Library

[7]

Ning Chen, Jialiu Lin, Steven CH Hoi, Xiaokui Xiao, and Boshen Zhang. 2014. AR-miner: mining informative reviews for developers from mobile app marketplace. In Proc. 36th Int. Conf. on software engineering. 767--778.

Digital Library

[8]

Marco Ciotti, Massimo Ciccozzi, Alessandro Terrinoni, Wen-Can Jiang, Cheng-Bin Wang, and Sergio Bernardini. 2020. The COVID-19 pandemic. Critical reviews in clinical laboratory sciences 57, 6 (2020), 365--388.

[9]

Laura Di Domenico, Giulia Pullano, Chiara E Sabbatini, Pierre-Yves Boëlle, and Vittoria Colizza. 2020. Impact of lockdown on COVID-19 epidemic in Île-de-France and possible exit strategies. BMC medicine 18, 1 (2020), 1--13.

[10]

John Drury and S Tekin Guven. 2020. Emergencies and disasters. Together apart the psychology of COVID-19 (2020), 97--101.

[11]

Stuart Ellis, Peter Kanowski, and RJ Whelan. 2004. National inquiry on bushfire mitigation and management. (2004).

[12]

Mattia Fazzini, Hourieh Khalajzadeh, Omar Haggag, Zhaoqing Li, Humphrey Obie, Chetan Arora, Waqar Hussain, and John Grundy. 2022. Characterizing Human Aspects in Reviews of COVID-19 Apps. In 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems 2022 (MobileSoft 2022).

[13]

Necmiye Genc-Nayebi and Alain Abran. 2017. A systematic literature review: Opinion mining studies from mobile app store user reviews. Journal of Systems and Software 125 (2017), 207--219.

[14]

Google. 2021. Google Play. Retrieved Feb 2, 2022 from https://play.google.com/store/

[15]

Australian Government. 2020. 2019--20 Australian bushfires---frequently asked questions: a quick guide. Retrieved Feb 2, 2022 from https://www.aph.gov.au/About_Parliament/Parliamentary_Departments/Parliamentary_Library/pubs/rp/rp1920/Quick_Guides/AustralianBushfres/

[16]

The Guardian. 2013. FireReady bushfire app sent out false alerts, users complain. Retrieved Feb 2, 2022 from https://www.theguardian.com/world/2013/dec/30/freready-bushfire-app-sent-out-false-alerts-users-complain/

[17]

Tobias Guggenberger, Jannik Lockl, Maximilian Röglinger, Vincent Schlatt, Johannes Sedlmeir, Jens-Christian Stoetzer, Nils Urbach, and Fabiane Völter. 2021. Emerging Digital Technologies to Combat Future Crises: Learnings From COVID-19 to be Prepared for the Future. International Journal of Innovation and Technology Management (2021), 2140002.

[18]

Emitza Guzman and Walid Maalej. 2014. How do users like this feature? a fine grained sentiment analysis of app reviews. In 2014 IEEE 22nd international requirements engineering conference (RE). IEEE, 153--162.

[19]

Omar Haggag, Sherif Haggag, John Grundy, and Mohamed Abdelrazek. 2021. COVID-19 vs Social Media Apps: Does Privacy Really Matter?. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). IEEE, 48--57.

Digital Library

[20]

Claudia Iacob, Varsha Veerappa, and Rachel Harrison. 2013. What are you complaining about?: a study of online reviews of mobile applications. In 27th International BCS Human Computer Interaction Conference (HCI 2013) 27. 1--6.

[21]

Nishant Jha and Anas Mahmoud. 2019. Mining non-functional requirements from app store reviews. Empirical Software Engineering 24, 6 (2019), 3659--3695.

[22]

Timo Johann, Christoph Stanik, Walid Maalej, et al. 2017. Safe: A simple approach for feature extraction from app descriptions and app reviews. In 2017 IEEE 25th International Requirements Engineering Conference (RE). IEEE, 21--30.

[23]

Huiying Li, Li Zhang, Lin Zhang, and Jufang Shen. 2010. A user satisfaction analysis approach for software evolution. In 2010 IEEE International Conference on Progress in Informatics and Computing, Vol. 2. IEEE, 1093--1097.

[24]

Jinfeng Li and Xinyi Guo. 2020. COVID-19 contact-tracing apps: A survey on the global deployment and challenges. arXiv preprint arXiv:2005.03599 (2020).

[25]

Mengmeng Lu and Peng Liang. 2017. Automatic classification of non-functional requirements from augmented app user reviews. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. 344--353.

Digital Library

[26]

R Harry Luke, Alan Grant McArthur, et al. 1978. Bush fires in Australia. Bush Fires in Australia. (1978).

[27]

Walid Maalej, Zijad Kurtanović, Hadeer Nabil, and Christoph Stanik. 2016. On the automatic classification of app reviews. Requirements Engineering 21, 3 (2016), 311--331.

Digital Library

[28]

Joshua C Morganstein, Carol S Fullerton, Robert J Ursano, Darrin Donato, and Harry C Holloway. 2017. Pandemics: health care emergencies. Textbook of disaster psychiatry (2017), 270--284.

[29]

Jessica Morley, Josh Cowls, Mariarosaria Taddeo, and Luciano Floridi. 2020. Ethical guidelines for COVID-19 tracing apps.

[30]

ABC News. 2021. The COVIDSafe app has cost $9m to date, but it hasn't uncovered any close contacts during the current outbreaks. Retrieved Feb 2, 2022 from https://www.abc.net.au/news/2021-09-30/covidsafe-app-cost-hasnt-uncovered-close-contacts-2021-outbreaks/100499870/

[31]

Editors of News.com.au. 2018. Residents left stranded by useless smartphone app during 'catastrophic' bush fire conditions. Retrieved Feb 2, 2022 from https://www.news.com.au/technology/environment/residents-left-stranded-by-useless-smartphone-app-during-catastrophic-bush-fire-conditions/news-story/ae48b58c62fe3cf69f35c9f2cf1346a8/

[32]

Patrick Howell O'Neill. 2020. No, coronavirus apps don't need 60% adoption to be effective. Retrieved Oct 16, 2020 from https://www.technologyreview.com/2020/06/05/1002775/covid-apps-efective-at-less-than-60-percent-download

[33]

Brian A O'Shea and Michiko Ueda. 2021. Who is more likely to ignore experts' advice related to COVID-19? Preventive medicine reports 23 (2021), 101470.

[34]

Douglas Paton, Gail Kelly, Petra T Burgelt, and Michael Doherty. 2006. Preparing for bushfires: understanding intentions. Disaster Prevention and Management: an international journal (2006).

[35]

Enrico Louis Quarantelli. 2000. Emergencies, disasters and catastrophes are different phenomena. (2000).

[36]

Kaavya Rekanar, Ian R O'Keefe, Sarah Buckley, Manzar Abbas, Sarah Beecham, Muslim Chochlov, Brian Fitzgerald, Liam Glynn, Kevin Johnson, John Laffey, et al. 2021. Sentiment analysis of user feedback on the HSE's Covid-19 contact tracing app. Irish Journal of Medical Science (1971-) (2021), 1--10.

[37]

Marion Lara Tan, Raj Prasanna, Kristin Stock, Emma Hudson-Doyle, Graham Leonard, and David Johnston. 2017. Mobile applications in crisis informatics literature: A systematic review. International journal of disaster risk reduction 24 (2017), 297--311.

[38]

Chrisa Tsinaraki, Irena Mitton, Marco Minghini, Marina Micheli, Alexander Kotsev, Lorena Hernandez Quiros, Fabiano-Antonio Spinelli, Alessandro Dalla Benetta, and Sven Schade. 2021. Mobile Apps to Fight the COVID-19 Crisis. Data 6, 10 (2021), 106.

[39]

Oxford University. 2020. Digital contact tracing can slow or even stop coronavirus transmission and ease us out of lockdown. Retrieved Oct 16, 2020 from https://www.research.ox.ac.uk/Article/2020-04-16-digital-contact-tracing-can-slow-or-even-stop-coronavirus-transmission-and-ease-us-out-of-lockdown

[40]

Rajesh Vasa, Leonard Hoon, Kon Mouzakis, and Akihiro Noguchi. 2012. A preliminary analysis of mobile app user reviews. In Proc. 24th Australian Computer-Human Interaction Conference. 241--244.

Digital Library

[41]

Kerstin Noelle Vokinger, Vasileios Nittas, Claudia M Witt, Sara Irina Fabrikant, and Viktor Von Wyl. 2020. Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Medical Weekly (2020).

[42]

Phong Minh Vu, Tam The Nguyen, Hung Viet Pham, and Tung Thanh Nguyen. 2015. Mining user opinions in mobile app reviews: A keyword-based approach (t). In 2015 30th IEEE/ACM Int. Conf. on Automated Software Engineering (ASE). IEEE, 749--759.

[43]

Michel Walrave, Cato Waeterloos, and Koen Ponnet. 2020. Adoption of a contact tracing app for containing COVID-19: a health belief model approach. JMIR public health and surveillance 6, 3 (2020), e20572.

[44]

Josh Whittaker and David Mercer. 2004. The Victorian bushfires of 2002--03 and the politics of blame: a discourse analysis. Australian Geographer 35, 3 (2004), 259--287.

[45]

Worldometers. 2021. COVID-19 CORONAVIRUS PANDEMIC LIVE UPDATES. Retrieved Feb 2, 2022 from https://www.worldometers.info/coronavirus/

[46]

Yanxiang Yang and Joerg Koenigstorfer. 2020. Determinants of physical activity maintenance during the Covid-19 pandemic: a focus on fitness apps. Translational behavioral medicine 10, 4 (2020), 835--842.

[47]

ZDNet. 2021. Australia's COVIDSafe app costs AU$100,000 per month to keep running. Retrieved Feb 2, 2022 from https://www.zdnet.com/article/australias-covidsafe-costs-au100000-per-month-to-keep-running/

[48]

Sebastian Zimmeck, Ziqi Wang, Lieyong Zou, Roger Iyengar, Bin Liu, Florian Schaub, Shomir Wilson, Norman M Sadeh, Steven M Bellovin, and Joel R Reidenberg. 2017. Automated Analysis of Privacy Requirements for Mobile Apps. In NDSS.

Cited By

Oliveira ASantos PAljedaani WEler M(2024)Exploring the Influence of Software Evolution on Mobile App Accessibility: Insights from User ReviewsJournal of the Brazilian Computer Society10.5753/jbcs.2024.432130:1(584-607)Online publication date: 2-Nov-2024
https://doi.org/10.5753/jbcs.2024.4321
Oliveira AEler M(2024)Exploring Accessibility of Mobile Applications Through User Feedback: Insights from App Reviews in a Systematic Literature ReviewProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702094(1-15)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702094
Zeng ZFang ZChen LGao YZheng KChen G(2024)FedCTQ: A Federated-Based Framework for Accurate and Efficient Contact Tracing Query2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00352(4628-4642)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00352
Show More Cited By

Recommendations

Better Identifying and Addressing Diverse Issues in mHealth and Emerging Apps Using User Reviews
EASE '22: Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering

The COVID-19 pandemic has changed the way we live, leading to a rapid expansion of mHealth apps usage. The pandemic also led to the introduction of a large number of ”emerging apps” to the mobile app market. mHealth and emerging app users have reported ...
Studying the consistency of star ratings and reviews of popular free hybrid Android and iOS apps

Nowadays, many developers make their mobile apps available on multiple platforms (e.g., Android and iOS). However, maintaining several versions of a cross-platform app that is built natively (i.e., using platform-specific tools) is a complicated task. ...
Studying the consistency of star ratings and the complaints in 1 & 2-star user reviews for top free cross-platform Android and iOS apps

How users rate a mobile app via star ratings and user reviews is of utmost importance for the success of an app. Recent studies and surveys show that users rely heavily on star ratings and user reviews that are provided by other users, for deciding ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MOBILESoft '22: Proceedings of the 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems

May 2022

154 pages

ISBN:9781450393010

DOI:10.1145/3524613

General Chair:
Leonardo Mariani
University of Milano - Bicocca, Italy
,
Program Chairs:
Gemma Catolino
Tilburg University & Jheronimus Academy of Data Science, Netherlands
,
Mei Nagappan
University of Waterloo, Canada

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council

Conference

MOBILESoft '22

Sponsor:

SIGSOFT

MOBILESoft '22: IEEE/ACM 9th International Conference on Mobile Software Engineering and Systems

May 17 - 24, 2022

Pennsylvania, Pittsburgh

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
111
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)3

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Oliveira ASantos PAljedaani WEler M(2024)Exploring the Influence of Software Evolution on Mobile App Accessibility: Insights from User ReviewsJournal of the Brazilian Computer Society10.5753/jbcs.2024.432130:1(584-607)Online publication date: 2-Nov-2024
https://doi.org/10.5753/jbcs.2024.4321
Oliveira AEler M(2024)Exploring Accessibility of Mobile Applications Through User Feedback: Insights from App Reviews in a Systematic Literature ReviewProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702094(1-15)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3702038.3702094
Zeng ZFang ZChen LGao YZheng KChen G(2024)FedCTQ: A Federated-Based Framework for Accurate and Efficient Contact Tracing Query2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00352(4628-4642)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00352
Grundy JMadugalla AMcIntosh JTran T(2023)Vision: Requirements Engineering for Software Development in Aged Care2023 IEEE 31st International Requirements Engineering Conference Workshops (REW)10.1109/REW57809.2023.00083(440-445)Online publication date: Sep-2023
https://doi.org/10.1109/REW57809.2023.00083
Alkasassbeh JAlawneh TAl-Qaisi AAltarawneh Malja'fari M(2023)The Role of AI in Mobile Apps to Combat Future Pandemics: A COVID-19 Case Study2023 2nd International Engineering Conference on Electrical, Energy, and Artificial Intelligence (EICEEAI)10.1109/EICEEAI60672.2023.10590538(1-8)Online publication date: 27-Dec-2023
https://doi.org/10.1109/EICEEAI60672.2023.10590538
Wang ZFan YDing LYang Y(2022)Research on Mobile Platform for Field Operation of Power Grid Infrastructure under the Digital Transformation of Power GridProceedings of the 2022 7th International Conference on Systems, Control and Communications10.1145/3575828.3575839(61-65)Online publication date: 14-Oct-2022
https://dl.acm.org/doi/10.1145/3575828.3575839
Haggag OGrundy JAbdelrazek MHaggag S(2022)A large scale analysis of mHealth app user reviewsEmpirical Software Engineering10.1007/s10664-022-10222-627:7Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10664-022-10222-6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten