Paulo Sérgio Henrique Dos Santos
ABSTRACT
Mobile applications are constantly updated to adapt to evolving user and environment requirements. As changes are successively implemented to promote user satisfaction, the complexity of mobile apps increases, and overlooked quality aspects (e.g. privacy, security, power consumption) may decline if counter-measures are not adopted. In this paper, we analyzed accessibility reviews of mobile apps to show evidence that updates may introduce barriers that make the app less accessible than its previous version according to users’ perceptions. Our results show that accessibility barriers reported by users mostly include incompatibility with screen readers, the removal of accessibility features (e.g. color scheme or font customization), small font sizes and widgets, and incompatibility with the device’s accessibility configuration. The accessibility barriers impact the users considering different levels: i) perception: an inability or difficulty to see, read or distinguish interface elements due to their small size or color; ii) understanding: the inability or difficulty to navigate, access and interpret information; iii) operation: the inability of difficulty to perform tasks such as adding items to the cart, reading or sending messages, and booking a ride; iv) and physical reactions, such as eyestrain, vertigo, and headache. The sentiments expressed by users are generally negative, including frustration, disappointment, and sometimes a sense of discrimination against people with disabilities. The results of our study raise an alert to organizations and developers that they should implement measures to avoid introducing accessibility barriers while they add new features to their mobile apps.
- Wajdi Aljedaani, Mohammed Alkahtani, Stephanie Ludi, Mohamed Wiem Mkaouer, Marcelo M. Eler, Marouane Kessentini, and Ali Ouni. 2023. The State of Accessibility in Blackboard: Survey and User Reviews Case Study. In Proceedings of the 20th International Web for All Conference (Austin, TX, USA) (W4A ’23). Association for Computing Machinery, New York, NY, USA, 84–95. https://doi.org/10.1145/3587281.3587291Google Scholar
Digital Library
- Wajdi Aljedaani, Mohamed Wiem Mkaouer, Stephanie Ludi, and Yasir Javed. 2022. Automatic Classification of Accessibility User Reviews in Android Apps. In 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA). IEEE, 133–138.Google Scholar
- Wajdi Aljedaani, Mohamed Wiem Mkaouer, Stephanie Ludi, Ali Ouni, and Ilyes Jenhani. 2022. On the identification of accessibility bug reports in open source systems. In Proceedings of the 19th International Web for All Conference. 1–11.Google ScholarDigital Library
- Wajdi Aljedaani, Furqan Rustam, Stephanie Ludi, Ali Ouni, and Mohamed Wiem Mkaouer. 2021. Learning sentiment analysis for accessibility user reviews. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). IEEE, 239–246.Google ScholarCross Ref
- Eman Abdullah AlOmar, Wajdi Aljedaani, Murtaza Tamjeed, Mohamed Wiem Mkaouer, and Yasmine N El-Glaly. 2021. Finding the needle in a haystack: On the automatic identification of accessibility user reviews. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–15.Google ScholarDigital Library
- Abdulaziz Alshayban, Iftekhar Ahmed, and Sam Malek. 2020. Accessibility issues in android apps: state of affairs, sentiments, and ways forward. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 1323–1334.Google ScholarDigital Library
- Tingting Bi, Xin Xia, David Lo, John Grundy, Thomas Zimmermann, and Denae Ford. 2021. Accessibility in software practice: A practitioner’s perspective. ACM Transactions on Software Engineering and Methodology (2021).Google Scholar
- Courtni Byun, Piper Vasicek, and Kevin Seppi. 2023. Dispensing with Humans in Human-Computer Interaction Research. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 413, 26 pages. https://doi.org/10.1145/3544549.3582749Google ScholarDigital Library
- Sen Chen, Chunyang Chen, Lingling Fan, Mingming Fan, Xian Zhan, and Yang Liu. 2022. Accessible or Not? An Empirical Investigation of Android App Accessibility. IEEE Transactions on Software Engineering 48, 10 (2022), 3954–3968. https://doi.org/10.1109/TSE.2021.3108162Google ScholarDigital Library
- Adelina Ciurumelea, Andreas Schaufelbühl, Sebastiano Panichella, and Harald C Gall. 2017. Analyzing reviews and code of mobile apps for better release planning. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 91–102.Google ScholarCross Ref
- Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46.Google Scholar
- Marcelo Medeiros Eler, Leandro Orlandin, and Alberto Dumont Alves Oliveira. 2019. Do Android app users care about accessibility? an analysis of user reviews on the Google play store. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems. 1–11.Google ScholarDigital Library
- Joseph L Fleiss, Bruce Levin, Myunghee Cho Paik, 1981. The measurement of interrater agreement. Statistical methods for rates and proportions 2, 212-236 (1981), 22–23.Google Scholar
- Jun Gao, Li Li, Tegawendé F. Bissyandé, and Jacques Klein. 2019. On the Evolution of Mobile App Complexity. In 2019 24th International Conference on Engineering of Complex Computer Systems (ICECCS). 200–209. https://doi.org/10.1109/ICECCS.2019.00029Google ScholarCross Ref
- C. Iacob and R. Harrison. 2013. Retrieving and analyzing mobile apps feature requests from online reviews. In 2013 10th Working Conference on Mining Software Repositories (MSR). 41–44. https://doi.org/10.1109/MSR.2013.6624001Google ScholarCross Ref
- Frederic B. Kraft and Charles L. Martin. 2001. Customer Compliments as More than Complementary Feedback. Journal of Consumer Satisfaction, Dissatisfaction & Complaining Behavior 14 (March 2001), 1–13. https://www.jcsdcb.com/index.php/JCSDCB/article/view/101Google Scholar
- M.M. Lehman. 1980. Programs, life cycles, and laws of software evolution. Proc. IEEE 68, 9 (1980), 1060–1076. https://doi.org/10.1109/PROC.1980.11805Google ScholarCross Ref
- M. M. Lehman. 1996. Laws of software evolution revisited. In Software Process Technology, Carlo Montangero (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 108–124.Google Scholar
- Manoel Victor Rodrigues Leite, Lilian Passos Scatalon, André Pimenta Freire, and Marcelo Medeiros Eler. 2021. Accessibility in the mobile development industry in Brazil: Awareness, knowledge, adoption, motivations and barriers. Journal of Systems and Software 177 (2021), 110942.Google ScholarCross Ref
- Tianyang Liu, Chong Wang, Kun Huang, Peng Liang, Beiqi Zhang, Maya Daneva, and Marten van Sinderen. 2023. RoseMatcher: Identifying the impact of user reviews on app updates. Information and Software Technology (2023), 107261. https://doi.org/10.1016/j.infsof.2023.107261Google ScholarDigital Library
- Stuart McIlroy, Nasir Ali, and Ahmed E. Hassan. 2016. Fresh apps: an empirical study of frequently-updated mobile apps in the Google play store. Empirical Software Engineering 21, 3 (June 2016), 1346–1370. https://doi.org/10.1007/s10664-015-9388-2Google ScholarDigital Library
- Maleknaz Nayebi, Konstantin Kuznetsov, Paul Chen, Andreas Zeller, and Guenther Ruhe. 2018. Anatomy of Functionality Deletion: An Exploratory Study on Mobile Apps. In Proceedings of the 15th International Conference on Mining Software Repositories (Gothenburg, Sweden) (MSR ’18). Association for Computing Machinery, New York, NY, USA, 243–253. https://doi.org/10.1145/3196398.3196410Google ScholarDigital Library
- Alberto Dumont Alves Oliveira, Paulo Sérgio Henrique Dos Santos, Wilson Estécio Marcílio Júnior, Wajdi M Aljedaani, Danilo Medeiros Eler, and Marcelo Medeiros Eler. 2023. Analyzing Accessibility Reviews Associated with Visual Disabilities or Eye Conditions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 37, 14 pages. https://doi.org/10.1145/3544548.3581315Google ScholarDigital Library
- Fabio Palomba, Mario Linares-Vásquez, Gabriele Bavota, Rocco Oliveto, Massimiliano Di Penta, Denys Poshyvanyk, and Andrea De Lucia. 2018. Crowdsourcing user reviews to support the evolution of mobile apps. Journal of Systems and Software 137 (2018), 143 – 162. https://doi.org/10.1016/j.jss.2017.11.043Google ScholarCross Ref
- Rahul Potharaju, Mizanur Rahman, and Bogdan Carbunar. 2017. A Longitudinal Study of Google Play. IEEE Transactions on Computational Social Systems PP (08 2017), 1–15. https://doi.org/10.1109/TCSS.2017.2732167Google ScholarCross Ref
- Emma Sabzalieva and Arianna Valentini. 2023. ChatGPT and artificial intelligence in higher education: quick start guide. UNESCO International Institute for Higher Education in Latin America and the Caribbean 5 (April 2023). https://unesdoc.unesco.org/ark:/48223/pf0000385146Google Scholar
- Mara Taynar Santiago and Anna Beatriz Marques. 2022. Are User Reviews Useful for Identifying Accessibility Issues That Autistic Users Face? An Exploratory Study. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’22). Association for Computing Machinery, New York, NY, USA, Article 6, 11 pages. https://doi.org/10.1145/3554364.3559114Google ScholarDigital Library
- Vincent F. Taylor and Ivan Martinovic. 2017. To Update or Not to Update: Insights From a Two-Year Study of Android App Evolution. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security (Abu Dhabi, United Arab Emirates) (ASIA CCS ’17). Association for Computing Machinery, New York, NY, USA, 45–57. https://doi.org/10.1145/3052973.3052990Google ScholarDigital Library
- Shunguo Yan and PG Ramachandran. 2019. The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS) 12, 1 (2019), 1–31.Google ScholarDigital Library
- Aidan Z. H. Yang, Safwat Hassan, Ying Zou, and Ahmed E. Hassan. 2022. An empirical study on release notes patterns of popular apps in the Google Play Store. Empirical Software Engineering 27, 2 (March 2022), 55. https://doi.org/10.1007/s10664-021-10086-2Google ScholarDigital Library
Index Terms
- Evolution may come with a price: analyzing user reviews to understand the impact of updates on mobile apps accessibility
Recommendations
Analyzing Accessibility Reviews Associated with Visual Disabilities or Eye Conditions
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsAccessibility reviews collected from app stores may contain valuable information for improving apps accessibility. Recent studies have presented insightful information on accessibility reviews, but they were based on small datasets and focused on general ...
The State of Accessibility in Blackboard: Survey and User Reviews Case Study
W4A '23: Proceedings of the 20th International Web for All ConferenceContext: Nowadays, mobile applications (or apps) have become vital in our daily life, particularly within education. Many institutions increasingly rely on mobile apps to provide access to all their students. However, many education mobile apps remain ...
The Current Status of Accessibility in Mobile Apps
This study evaluated the status of accessibility in mobile apps by investigating the graphical user interface (GUI) structures and conformance to accessibility guidelines of 479 Android apps in 23 business categories from Google Play. An automated tool, ...
Comments