skip to main content
10.1145/3524613.3527814acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Characterizing human aspects in reviews of COVID-19 apps

Published: 17 October 2022 Publication History

Abstract

To successfully satisfy user needs, software developers need to suitably capture and implement user requirements. A critical and often overlooked characteristic of user requirements are "human aspects", which are personal circumstances affecting the use of software (e.g., age, gender, language, etc.). To better understand how human aspects can impact the use of software, this work presents an empirical study focusing on app reviews of COVID-19 contact tracing apps. We manually analyzed a dataset of 2,611 app reviews sampled from the reviews associated with 57 COVID-19 apps. To analyze the reviews, we performed qualitative and quantitative analyses. The analyses characterize the human aspects contained in the reviews and investigate whether the apps suitably address the human aspects. We identified 716 reviews related to human aspects and grouped these into nine categories. Of these 716 reviews, 8% report bugs, 14% describe future/improvement requests, and 22% detail the user experience. Our analysis of the results reveal that human aspects are important to users and we need better support to account for them as software is developed.

References

[1]
Abdulaziz Alshayban, Iftekhar Ahmed, and Sam Malek. 2020. Accessibility issues in Android apps: state of affairs, sentiments, and ways forward. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 1323--1334.
[2]
Apple and Google. 2020. Privacy-Preserving Contact Tracing. Retrieved May 15, 2021 from https://covid19.apple.com/contacttracing
[3]
Muneera Bano, Chetan Arora, Didar Zowghi, and Alessio Ferrari. 2021. The Rise and Fall of COVID-19 Contact Tracing Apps: when NFRs Collide with Pandemic. In 29th IEEE International Requirements Engineering Conference (RE). IEEE.
[4]
Muneera Bano, Didar Zowghi, and Chetan Arora. 2020. Requirements, Politics, or Individualism: What Drives the Success of COVID-19 Contact-Tracing Apps? IEEE Software 38, 1 (2020), 7--12.
[5]
Lars Baumgärtner, Alexandra Dmitrienko, Bernd Freisleben, Alexander Gruler, Jonas Höchst, Joshua Kühlberg, Mira Mezini, Richard Mitev, Markus Miettinen, Anel Muhamedagic, et al. 2020. Mind the GAP: Security & privacy risks of contact tracing apps. In 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom). IEEE, 458--467.
[6]
Yoshua Bengio, Daphne Ippolito, Richard Janda, Max Jarvie, Benjamin Prud'homme, Jean-François Rousseau, Abhinav Sharma, and Yun William Yu. 2021. Inherent privacy limitations of decentralized contact tracing apps. Journal of the American Medical Informatics Association 28, 1 (2021), 193--195.
[7]
Petter Bae Brandtzaeg, Antoine Pultier, and Gro Mette Moen. 2019. Losingcontrol to data-hungry apps: A mixed-methods approach to mobile app privacy. Social Science Computer Review 37, 4 (2019), 466--488.
[8]
John L Campbell, Charles Quincy, Jordan Osserman, and Ove K Pedersen. 2013. Coding in-depth semistructured interviews: Problems of unitization and inter-coder reliability and agreement. Sociological Methods & Research 42, 3 (2013), 294--320.
[9]
Pew Research Center. 2019. Internet/Broadband Fact Sheet. Retrieved May 15, 2021 from https://www.pewresearch.org/internet/fact-sheet/internet-broadband
[10]
Eugene Y Chan and Najam U Saqib. 2021. Privacy concerns can explain unwillingness to download and use contact tracing apps when COVID-19 concerns are high. Computers in Human Behavior 119 (2021), 106718.
[11]
Yi Chen, Mingming Zha, Nan Zhang, Dandan Xu, Qianqian Zhao, Xuan Feng, Kan Yuan, Fnu Suya, Yuan Tian, Kai Chen, et al. 2019. Demystifying hidden privacy settings in mobile apps. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE, 570--586.
[12]
Hyunghoon Cho, Daphne Ippolito, and Yun William Yu. 2020. Contact tracing mobile apps for COVID-19: Privacy considerations and related trade-offs. arXiv preprint arXiv:2003.11511 (2020).
[13]
Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.
[14]
Corona-Warn-App. 2020. Corona-Warn-App Open Source Project. Retrieved May 15, 2021 from https://www.coronawarn.app/en
[15]
Maheswaree Kissoon Curumsing, Niroshinie Fernando, Mohamed Abdelrazek, Rajesh Vasa, Kon Mouzakis, and John Grundy. 2019. Emotion-oriented requirements engineering: A case study in developing a smart home system for the elderly. Journal of Systems and Software 147 (2019), 215 -- 229.
[16]
Merriam-Webster Dictionary. 2021. Merriam-Webster Dictionary. Retrieved May 15, 2021 from https://www.merriam-webster.com
[17]
Tayba Farooqui, Tauseef Rana, and Fakeeha Jafari. 2019. Impact of human-centered design process (HCDP) on software development process. In 2019 2nd International Conference on Communication, Computing and Digital systems (C-CODE). IEEE, 110--114.
[18]
Mattia Fazzini, Hourieh Khalajzadeh, Omar Haggag, Zhaoqing Li, Humphrey Obie, Chetan Arora, Waqar Hussain, and John Grundy. 2022. Artifacts for Characterizing Human Aspects in Reviews of COVID-19 Apps. Retrieved Mar 28, 2022 from
[19]
Vahid Garousi, David Cutting, and Michael Felderer. 2020. Mining user reviews of COVID contact-tracing apps: An exploratory analysis of nine European apps. arXiv preprint arXiv:2012.13589 (2020).
[20]
Necmiye Genc-Nayebi and Alain Abran. 2017. A systematic literature review: Opinion mining studies from mobile app store user reviews. Journal of Systems and Software 125 (2017), 207--219.
[21]
Google. 2020. Recent Advances in Google Translate. Retrieved May 15, 2021 from https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html
[22]
Googletrans. 2021. googletrans 3.0.0. Retrieved May 15, 2021 from https://pypi.org/project/googletrans
[23]
John Grundy, Hourieh Khalajzadeh, and Jennifer Mcintosh. 2020. Towards Human-centric Model-driven Software Engineering. In ENASE. 229--238.
[24]
John Grundy, Hourieh Khalajzadeh, Jennifer McIntosh, Tanjila Kanij, and Ingo Mueller. 2020. HumaniSE: Approaches to Achieve More Human-Centric Software Engineering. In International Conference on Evaluation of Novel Approaches to Software Engineering. Springer, 444--468.
[25]
The Guardian. 2019. Instagram bans graphic self-harm images after Molly Russell's death. Retrieved May 15, 2021 from https://www.theguardian.com/technology/2019/feb/07/instagram-bans-graphic-self-harm-images-after-molly-russells-death
[26]
Emitza Guzman and Walid Maalej. 2014. How do users like this feature? a fine grained sentiment analysis of app reviews. In 2014 IEEE 22nd international requirements engineering conference (RE). IEEE, 153--162.
[27]
Omar Haggag, John Grundy, Mohamed Abdelrazek, and Sherif Haggag. 2022. Better Addressing Diverse Accessibility Issues in Emerging Apps: A Case Study using COVID-19 Apps. In 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems 2022 (MobileSoft 2022).
[28]
O. Haggag, S. Haggag, J. Grundy, and M. Abdelrazek. 2021. COVID-19 vs Social Media Apps: Does Privacy Really Matter?. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). IEEE, Los Alamitos, CA, USA, 48--57.
[29]
Dongjie He, Lian Li, Lei Wang, Hengjie Zheng, Guangwei Li, and Jingling Xue. 2018. Understanding and detecting evolution-induced compatibility issues in Android apps. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 167--177.
[30]
Katie Hogan, Briana Macedo, Venkata Macha, Arko Barman, Xiaoqian Jiang, et al. 2021. Contact Tracing Apps: Lessons Learned on Privacy, Autonomy, and the Need for Detailed and Thoughtful Implementation. JMIR Medical Informatics 9, 7 (2021), e27449.
[31]
Nem ID. 2021. Nem ID. Retrieved May 15, 2021 from https://www.nemid.nu/dk-en
[32]
Instagram. 2021. Instagram. Retrieved May 15, 2021 from www.instagram.com
[33]
Steve Jacob and Justin Lawarée. 2020. The adoption of contact tracing applications of COVID-19 by European governments. Policy Design and Practice (2020), 1--15.
[34]
Nishant Jha and Anas Mahmoud. 2019. Mining non-functional requirements from app store reviews. Empirical Software Engineering 24, 6 (2019), 3659--3695.
[35]
Aria YukFan Jim, Hyun Shim, Jue Wang, Lionel Richie Wijaya, Rongbin Xu, Hourieh Khalajzadeh, John Grundy, and Tanjila Kanij. 2021. Improving the Modelling of Human-Centric Aspects of Software Systems- A Case Study of Modelling End User Age in Wirefame Designs. In Evaluation of Novel Approaches to Software Engineering (ENASE).
[36]
Timo Johann, Christoph Stanik, Walid Maalej, et al. 2017. Safe: A simple approach for feature extraction from app descriptions and app reviews. In 2017 IEEE 25th International Requirements Engineering Conference (RE). IEEE, 21--30.
[37]
johnpcarty. 2021. Thesaurus of Job Titles. Retrieved May 15, 2021 from https://github.com/johnpcarty/Thesaurus-of-Job-Titles
[38]
Bobbie Johnson. 2021. The Covid Tracing Tracker: What's happening in coronavirus apps around the world. Retrieved May 15, 2021 from https://www.technologyreview.com/2020/12/16/1014878/covid-tracing-tracker
[39]
Alita Joyce and Jakob Nielsen. 2019. Teenager's UX: Designing for Teens. Retrieved May 15, 2021 from https://www.nngroup.com/articles/usability-of-websites-for-teenagers
[40]
Douglas J Leith and Stephen Farrell. 2021. Contact tracing app privacy: What data is shared by Europe's GAEN contact tracing apps. In IEEE INFOCOM 2021-IEEE Conference on Computer Communications. IEEE, 1--10.
[41]
Huiying Li, Li Zhang, Lin Zhang, and Jufang Shen. 2010. A user satisfaction analysis approach for software evolution. In 2010 IEEE International Conference on Progress in Informatics and Computing, Vol. 2. IEEE, 1093--1097.
[42]
Li Li, Tegawendé F Bissyandé, Haoyu Wang, and Jacques Klein. 2018. Cid: Automating the detection of api-related compatibility issues in android apps. In ISSTA.
[43]
Bin Liu, Jialiu Lin, and Norman Sadeh. 2014. Reconciling Mobile App Privacy and Usability on Smartphones: Could User Privacy Profiles Help?. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW '14). ACM, New York, NY, USA, 201--212.
[44]
Mengmeng Lu and Peng Liang. 2017. Automatic classification of non-functional requirements from augmented app user reviews. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. 344--353.
[45]
Walid Maalej, Zijad Kurtanović, Hadeer Nabil, and Christoph Stanik. 2016. On the automatic classification of app reviews. Requirements Engineering 21, 3 (2016), 311--331.
[46]
Kirsten Martin and Katie Shilton. 2016. Putting mobile application privacy in context: An empirical study of user privacy expectations for mobile devices. The Information Society 32, 3 (2016), 200--216. arXiv:https://doi.org/10.1080/01972243.2016.1153012
[47]
Matthew B Miles, A Michael Huberman, and Johnny Saldaña. 2018. Qualitative data analysis: A methods sourcebook. Sage publications.
[48]
George A. Miller. 1995. WordNet: A Lexical Database for English. COMMUNICATIONS OF THE ACM 38 (1995), 39--41.
[49]
Elizabeth R Morrissey. 1974. Sources of error in the coding of questionnaire data. Sociological Methods & Research 3, 2 (1974), 209--232.
[50]
Amber L Mueller, Maeve S McNamara, and David A Sinclair. 2020. Why does COVID-19 disproportionately affect older people? Aging 12, 10 (2020), 9959.
[51]
Humphrey O. Obie, Waqar Hussain, Xin. Xia, John Grundy, Li Li, Burak Turhan, Jon Whittle, and Mojtaba Shahin. 2021. A First Look at Human Values-Violation in App Reviews. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS). 29--38.
[52]
Donatella Rita Petretto and Roberto Pili. 2020. Ageing and COVID-19: What is the role for elderly people?
[53]
Klaus Pohl. 2010. Requirements Engineering - Fundamentals, Principles, and Techniques. Springer.
[54]
Kaavya Rekanar, Ian R O'Keeffe, Sarah Buckley, Manzar Abbas, Sarah Beecham, Muslim Chochlov, Brian Fitzgerald, Liam Glynn, Kevin Johnson, John Laffey, et al. 2021. Sentiment analysis of user feedback on the HSE's Covid-19 contact tracing app. Irish Journal of Medical Science (1971-) (2021), 1--10.
[55]
Frantz Rowe. 2020. Contact tracing apps and values dilemmas: A privacy paradox in a neo-liberal world. International Journal of Information Management 55 (2020), 102178.
[56]
Marcel Salathé, Christian L Althaus, Nanina Anderegg, Daniele Antonioli, Tala Ballouz, Edouard Bugnion, Srjan Capkun, Dennis Jackson, Sang-Il Kim, James Larus, et al. 2020. Early evidence of effectiveness of digital contact tracing for SARS-CoV-2 in Switzerland. medRxiv (2020).
[57]
Victor Sucasas, Georgios Mantas, Ayman Radwan, and Jonathan Rodriguez. 2016. An OAuth2-based protocol with strong user privacy preservation for smart city mobile e-Health apps. In 2016 IEEE International Conference on Communications (ICC). IEEE, 1--6.
[58]
Thesaurus. 2021. Thesaurus. Retrieved May 15, 2021 from www.thesaurus.com
[59]
Carmela Troncoso, Mathias Payer, Jean-Pierre Hubaux, Marcel Salathé, James Larus, Edouard Bugnion, Wouter Lueks, Theresa Stadler, Apostolos Pyrgelis, Daniele Antonioli, Ludovic Barman, Sylvain Chatel, Kenneth Paterson, Srdjan Čapkun, David Basin, Jan Beutel, Dennis Jackson, Marc Roeschlin, Patrick Leu, Bart Preneel, Nigel Smart, Aysajan Abidin, Seda Gürses, Michael Veale, Cas Cremers, Michael Backes, Nils Ole Tippenhauer, Reuben Binns, Ciro Cattuto, Alain Barrat, Dario Fiore, Manuel Barbosa, Rui Oliveira, and José Pereira. 2020. Decentralized Privacy-Preserving Proximity Tracing. arXiv:2005.12273 [cs.CR]
[60]
UNWTO. 2020. 100% of Global Destinations Now Have COVID-19 Travel Restrictions, UNWTO Reports. Retrieved May 15, 2021 from https://www.unwto.org/news/covid-19-travel-restrictions
[61]
Michel Walrave, Cato Waeterloos, and Koen Ponnet. 2020. Adoption of a contact tracing app for containing COVID-19: a health belief model approach. JMIR public health and surveillance 6, 3 (2020).
[62]
Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2016. Taming Android Fragmentation: Characterizing and Detecting Compatibility Issues for Android Apps. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (Singapore, Singapore). Association for Computing Machinery, New York, NY, USA, 226--237.
[63]
Lili Wei, Yepang Liu, and Shing-Chi Cheung. 2019. Pivot: learning api-device correlations to facilitate android compatibility issue detection. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 878--888.
[64]
Zhibiao Wu and Martha Palmer. 1994. Verbs Semantics and Lexical Selection. In Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics (Las Cruces, New Mexico). Association for Computational Linguistics, USA, 133--138.
[65]
Sebastian Zimmeck, Ziqi Wang, Lieyong Zou, Roger Iyengar, Bin Liu, Florian Schaub, Shomir Wilson, Norman M Sadeh, Steven M Bellovin, and Joel R Reidenberg. 2017. Automated Analysis of Privacy Requirements for Mobile Apps. In NDSS.

Cited By

View all
  • (2025)Who uses personas in requirements engineering: The practitioners’ perspectiveInformation and Software Technology10.1016/j.infsof.2024.107609178(107609)Online publication date: Feb-2025
  • (2024)Fairness Concerns in App Reviews: A Study on AI-Based Mobile AppsACM Transactions on Software Engineering and Methodology10.1145/369063334:2(1-30)Online publication date: 29-Aug-2024
  • (2024)Requirements Elicitation in Government Projects: A Preliminary Empirical Study2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00025(146-154)Online publication date: 24-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MOBILESoft '22: Proceedings of the 9th IEEE/ACM International Conference on Mobile Software Engineering and Systems
May 2022
154 pages
ISBN:9781450393010
DOI:10.1145/3524613
  • General Chair:
  • Leonardo Mariani,
  • Program Chairs:
  • Gemma Catolino,
  • Mei Nagappan
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

  • IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

  • Facebook
  • Australian Research Council Laureate Fellowship

Conference

MOBILESoft '22
Sponsor:

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)0
Reflects downloads up to 07 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Who uses personas in requirements engineering: The practitioners’ perspectiveInformation and Software Technology10.1016/j.infsof.2024.107609178(107609)Online publication date: Feb-2025
  • (2024)Fairness Concerns in App Reviews: A Study on AI-Based Mobile AppsACM Transactions on Software Engineering and Methodology10.1145/369063334:2(1-30)Online publication date: 29-Aug-2024
  • (2024)Requirements Elicitation in Government Projects: A Preliminary Empirical Study2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW)10.1109/REW61692.2024.00025(146-154)Online publication date: 24-Jun-2024
  • (2024)How do software practitioners perceive human-centric defects?Information and Software Technology10.1016/j.infsof.2024.107549176(107549)Online publication date: Dec-2024
  • (2024)Developer and End-User Perspectives on Addressing Human Aspects in Mobile eHealth AppsInformation and Software Technology10.1016/j.infsof.2023.107353166:COnline publication date: 4-Mar-2024
  • (2024)ED&I and SE: Challenges, Progress, and LessonsEquity, Diversity, and Inclusion in Software Engineering10.1007/978-1-4842-9651-6_2(17-35)Online publication date: 21-Sep-2024
  • (2023)Supporting Developers in Addressing Human-Centric Issues in Mobile AppsIEEE Transactions on Software Engineering10.1109/TSE.2022.321232949:4(2149-2168)Online publication date: 1-Apr-2023
  • (2023)Towards Human-Centred Crowd Computing: Software for Better Use of Computational ResourcesProceedings of the 45th International Conference on Software Engineering: New Ideas and Emerging Results10.1109/ICSE-NIER58687.2023.00022(90-94)Online publication date: 17-May-2023
  • (2023)Better Understanding Diverse End User Website Usage Challenges with Browser-Based Augmented Reality ApproachesEvaluation of Novel Approaches to Software Engineering10.1007/978-3-031-36597-3_13(269-291)Online publication date: 8-Jul-2023
  • (2022)VR4HcRE: Virtual Reality Platform for Human-centric Requirements Elicitation2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00168(788-793)Online publication date: Oct-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media