Skip to main content
SpringerLink
Log in

Semantic web technologies applied to software accessibility evaluation: a systematic literature review

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Software accessibility is a current field of interest. Governments and organizations boost initiatives and legislations to assure universal access. On the other hand, the semantic web is the evolution of the Internet from a web of pages to a web of data. It enables the linked data engineering. It facilitates search of information, data analysis and data sharing. Some studies have applied semantic web technologies to improve software accessibility. Their methods, goals and results are heterogeneous. This systematic literature review aims to review how the application of semantic web technologies has evolved in software accessibility evaluation. First, we defined some main and secondary research questions that would set the context for the review. Then, we defined a search strategy and selection criteria which allowed us to get a set of primary studies related to our research question. We measured the quality of the studies and performed a qualitative analysis. A lack of studies in some of the sub-areas included was revealed. A low impact level was also detected in most cases, so this research field needs to be explored further. Semantic web has a direct application on software accessibility evaluation. However, this review reveals a lack of studies in some important sub-areas. A very low impact of the studies was also detected compared with other studies in the accessibility field. The evolution of this area is slow, so we recommend exploring further the different research branches and trying to generate studies with more impact.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. World Health Organization: World Bank: World report on disability (2011)

  2. ISO: ISO 26800:2011 Ergonomics—general approach, principles and concepts (2011)

  3. Torrente, J., Blanco, Á.D., Moreno-Ger, P., Martínez-Ortiz, I., Fernández-Manjón, B.: Implementing accessibility in educational videogames with. In: Proceedings of the 1st ACM International Workshop on Multimedia Technologies for Distance Learning, pp. 57–66 (2009)

  4. Alonso, J.V.G., Ratzka, A., Barnes, C., Hasler, F., Zarb, G., Evans, J., Malinga, J., Maraña, J.J., Heumann, J., Martínez, K., Cayo, P.B.L., Lobato, M., Bieler, R.B.: El movimiento de vida independiente: experiencias internacionales. Fundación Luis Vives (2014)

  5. World Wide Web Consortium: Semantic Web—W3C. https://www.w3.org/standards/semanticweb/. Accessed 2015

  6. Google; Microsoft; Yahoo; Yandex: Home—Schema.org. https://schema.org/. Accessed 2015

  7. DBpedia: DBpedia. https://wiki.dbpedia.org/. Accessed 2019

  8. World Wide Web Consortium: RDF—Semantic Web Standards. https://www.w3.org/RDF/. Accessed 2014

  9. Beckett, D., Berners-Lee, T., Prud’hommeaux, E., Carothers, G.: RDF 1.1 Turtle—Terse RDF Triple Language. https://www.w3.org/TR/turtle/. Accessed 5 Feb 2014

  10. Brickley, D.: RDF Schema 1.1. https://www.w3.org/TR/rdf-schema/. Accessed 25 Feb 2014

  11. World Wide Web Consortium: OWL—Semantic Web Standards. https://www.w3.org/OWL/. Accessed 2013

  12. Abou-Zahra, S.: Evaluation and Report Language (EARL) 1.0 Schema. https://www.w3.org/TR/EARL10-Schema/. Accessed 2 Feb 2017

  13. Abou-Zahra, S.: A data model to facilitate the automation of web accessibility evaluations. Electron. Notes Theor. Comput. Sci. 157(2), 3–9 (2006)

    Article  Google Scholar 

  14. Lopes, R., Votis, K., Carriço, L., Likothanassis, S.: A service oriented ontological framework for the semantic validation of web accessibility. In: Virtual Communities: Concepts, Methodologies, Tools and Applications, pp. 388–406 (2011)

  15. Tahir, A., Tosi, D., Morasca, S.: A systematic review on the functional testing of semantic web services. J. Syst. Softw. 86(11), 2877–2889 (2013)

    Article  Google Scholar 

  16. Elias, M., Lohmann, S., Auer, S.: Fostering accessibility of OpenCourseWare with semantic technologies—a literature review. In: International Conference on Knowledge Engineering and the Semantic Web, pp. 241–256 (2016)

  17. Insfran, E., Fernandez, A.: A systematic review of usability evaluation in web development. In: Web Information Systems Engineering—WISE 2008 Workshops. Lecture Notes in Computer Science, Berlin, vol. 5176, pp. 81–91 (2008)

  18. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering. EBSE (2007)

  19. Caldwell, B., Cooper, M., Reid, L. G., Vanderheiden, G.: Web content accessibility guidelines (WCAG) 2.0. https://www.w3.org/TR/WCAG20/. Accessed 11 Dec 2008

  20. ISO: ISO/IEC 40500 Information technology—W3C Web Content Accessibility Guidelines (WCAG) 2.0 (2012)

  21. Kirkpatrick, A., O Connor, J., Campbell, A., Cooper, M.: Web Content Accessibility Guidelines (WCAG) 2.1. https://www.w3.org/TR/WCAG21/. Accessed 5 June 2018

  22. ETSI: EN 301 549 v 3.1.1—Harmonised European Standard, Accessibility requirements for ICT products and services. https://www.etsi.org/deliver/etsi_en/301500_301599/301549/03.01.01_60/en_301549v030101p.pdf. Accessed 10 April 2019

  23. United States Access Board: Section 508 Standards for Electronic and Information Technology. https://www.access-board.gov/guidelines-and-standards/communications-and-it/about-the-section-508-standards/section-508-standards. Accessed 21 Dec 2000

  24. United States Access Board: Information and Communication Technology (ICT) Final Standards and Guidelines. https://www.access-board.gov/attachments/article/1877/ict-final-rule.pdf. Accessed 18 Jan 2017

  25. Agencia per l’Italia Digitale: Normativa|AgID. http://www.agid.gov.it/agenda-digitale/pubblica-amministrazione/accessibilita/normativa. Accessed 2017

  26. National Disability Authority: Irish National IT Accessibility Guidelines—Centre for Excellence in Universal Design. http://universaldesign.ie/Technology-ICT/Irish-National-IT-Accessibility-Guidelines/. Accessed 2012

  27. SGMAP: RÉFÉRENTIEL GÉNÉRAL D’ACCESSIBILITÉ DES ADMINISTRATIONS (RGAA). https://references.modernisation.gouv.fr/rgaa-accessibilite/. Accessed 2016

  28. DIAS GmbH: BIK BITV-Test—Verzeichnis der Prüfschritte. http://www.bitvtest.de/bitvtest/das_testverfahren_im_detail/pruefschritte.html. Accessed 2017

  29. Web Accessibility Infrastructure Comitee: JIS X 8341-3:2016. http://waic.jp/docs/jis2016/understanding/201604/. Accessed 2016

  30. Allan, J., Lowney, G., Patch, K., Spellman, J.: User Agent Accessibility Guidelines (UAAG) 2.0. https://www.w3.org/TR/UAAG20/. Accessed 15 Dec 2015

  31. Richards, J., Spellman, J., Treviranus, J.: Authoring Tool Accessibility Guidelines (ATAG) 2.0. https://www.w3.org/TR/ATAG20/. Accessed 24 Sept 2015

  32. Diggs, J., McCarron, S., Cooper, M., Schwerdtfeger, R., Craig, J.: Accessible Rich Internet Applications (WAI-ARIA) 1.1. https://www.w3.org/TR/wai-aria/. Accessed 14 Dec 2017

  33. Velleman, E., Abou-Zahra, S.: Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0. https://www.w3.org/TR/WCAG-EM/. Accessed 10 July 2014

  34. World Wide Web Consortium: Evaluation and Report Language (EARL) overview|Web Accessibility Initiative (WAI)|W3C. https://www.w3.org/WAI/intro/earl. Accessed 2012

  35. Timbi-Sisalima, C., Amor, C.I.M., Otón, S., Hilera, J.R., Aguado-Delgado, J.: Comparative analysis of online web accessibility evaluation tools. ISD (2016)

  36. Jaeger, P.: Assessing Section 508 compliance on federal e-government Web sites: a multi-method, user-centered evaluation of accessibility for persons with disabilities. Gov. Inf. Q. 23(2), 169–190 (2006)

    Article  Google Scholar 

  37. Velasco, C., Abou-Zahra, S.: Developers’ Guide to Features of Web Accessibility Evaluation Tools. https://www.w3.org/TR/WAET/. Accessed 2 Feb 2017

  38. Fiers, W., Kraft, M.: Accessibility Conformance Testing (ACT) Rules Format 1.0. https://www.w3.org/TR/act-rules-format/. Accessed 5 July 2018

  39. Horrocks, I., Patel-Schneider, P., Boley, H., Tabet, S., Grosof, B., Dean, M.: SWRLSWRL: A Semantic Web Rule Language—Combining OWL and RuleML. https://www.w3.org/Submission/SWRL/. Accessed 21 May 2004

  40. Gandon, F., Schreiber, G.: RDF 1.1 XML Syntax. https://www.w3.org/TR/rdf-syntax-grammar/. Accessed 25 Feb 2014

  41. Sporny, M., Longley, D., Kellogg, G., Lanthaler, M., Lindström, N.: JSON-LD 1.0—A JSON-Based Serialization for Linked Data. https://www.w3.org/TR/json-ld/. Accessed 16 Jan 2014

  42. Das, S., Sundara, S., Cyganiak, R.: R2RML: RDB to RDF Mapping Language. https://www.w3.org/TR/r2rml/. Accessed 27 Sept 2012

  43. Harris, S., Seaborne, A.: SPARQL 1.1 Query Language. https://www.w3.org/TR/sparql11-query/. Accessed 21 March 2013

  44. Masuwa-Morgan, K.R., Burrell, P.: Justification of the need for an ontology for accessibility requirements (Theoretic framework). Interact. Comput. 16(3), 523–555 (2004)

    Article  Google Scholar 

  45. Masuwa-Morgan, K.R.: Introducing AccessOnto: Ontology for Accessibility Requirements Specification. In: 2008 1st International Workshop on Ontologies in Interactive Systems, pp. 33–38 (2008)

  46. Abou-Zahra, S.: A common vocabulary to facilitate the exchange of web accessibility evaluation results. In: International Conference on Dublin Core and Metadata Applications, pp. 173–177 (2005)

  47. Abou-Zahra, S.: SemanticWeb enabled web accessibility evaluation tools. In: Proceedings of the 2005 International Cross-Disciplinary Workshop on Web Accessibility (W4A), pp. 99–101 (2005)

  48. Abou-Zahra, S.: Managing and monitoring web site accessibility. In: ICCHP’06 Proceedings of the 10th International Conference on Computers Helping People with Special Needs, pp. 192–198 (2006)

  49. Lopes, R., Votis, K., Carriço, L., Tzovaras, D., Spiridon, L.: The semantics of personalised web accessibility assessment. In: Proceedings of the 2010 ACM Symposium on Applied Computing, pp. 1440–1441 (2010)

  50. Google: Google Scholar Metrics Help. https://scholar.google.com/intl/es/scholar/metrics.html#metrics. Accessed 2018

  51. Computing Research and Education: CORE Rankings Portal—Computing Research and Education. http://www.core.edu.au/conference-portal. Accessed 2016

  52. Capes: Plataforma Sucupira. https://sucupira.capes.gov.br/sucupira/public/index.jsf. Accessed 2016

  53. Clarivate Analytics: Journal Citation Reports. https://jcr.incites.thomsonreuters.com/. Accessed 2018

  54. Scimago Lab: SJR Help. https://www.scimagojr.com/help.php. Accessed 2018

  55. Elsevier BV: FAQs Journal Metrics—Scopus.com. https://journalmetrics.scopus.com/index.php/Faqs. Accessed 2017

  56. Elsevier Ltd: Elsevier Journal Metrics Visualization—Helping Authors—Visualizing key metrics—Delivering journal insights. https://journalinsights.elsevier.com/journals/0969-806X/snip. Accessed 2018

  57. Wikipedia Contributors: h-index—Wikipedia. https://en.wikipedia.org/wiki/h-index. Accessed 2018

  58. Hilera, J.R., Estrada, F., Otón, S., Ingavelez, P., Timbi, C.: Formalización ontológica de reglas de evaluación de accesibilidad. In: ATICA2018: Aplicación de Tecnologías de la información y comunicaciones avanzadas y accesibilidad, Guatemala, pp. 101–108 (2018)

  59. Velasco, C., Abou-Zahra, S.: Developer Guide for Evaluation and Report Language (EARL) 1.0. https://www.w3.org/TR/EARL10-Guide/. Accessed 2 Feb 2017

  60. Squillace, M., Abou-Zahra, S.: Requirements for the Evaluation and Report Language (EARL) 1.0. https://www.w3.org/TR/EARL10-Requirements/. Accessed 2 Feb 2017

  61. Hilera, J., Otón, S., Timbi-Sisalima, C., Aguado-Delgado, J., Estrada-Martínez, F., Amado-Salvatierra, H.: Combining multiple web accessibility evaluation reports using semantic web technologies. World Acad. Sci. Eng. Technol. Int. J. Comput. Inf. Eng. 4(6), 65–78 (2018)

    Google Scholar 

  62. Hilera, J., Oton, S., Martin-Amor, C., Timbi-Sisalima, C.: Towards a Service-based Architecture for Web Accessibility Federated Evaluation. In: ACHI 2016, The 9th International Conference on Advances in Computer–Human Interactions, pp. 6–10 (2016)

  63. Ackermann, P., Vlachogiannis, E., Velasco, C.A.: Developing advanced accessibility conformance tools for the ubiquitous web. Procedia Comput. Sci. 67, 452–457 (2015)

    Article  Google Scholar 

  64. Giakoumis, D., Votis, K., Tzovaras, D.: Introducing web service accessibility assessment techniques through a unified quality of service context. Serv. Oriented Comput. Appl. 8(2), 159–174 (2014)

    Article  Google Scholar 

  65. Centelles, M., Ribera, M., Rodriguez, I.: Potential accessibility of web-based mathematical information resources (2014)

  66. Doush, I.A., Alkhateeb, F., Maghayreh, E.A., Al-Betar, M.A.: The design of RIA accessibility evaluation tool. Adv. Eng. Softw. 57, 1–7 (2013)

    Article  Google Scholar 

  67. Ponsard, C., Beaujeant, P., Vanderdonckt, J.: Augmenting accessibility guidelines with user ability rationales. In: 14th IFIP TC 13 International Conference on Human–Computer Interaction—INTERACT 2013, pp. 579–586 (2013)

  68. Chen, Y.-L., Liu, L.: The knowledge base development for the web content accessibility guidelines. Int. J. Intell. Syst. 4(1), 29–37 (2014)

    Google Scholar 

  69. Martín, Y.S., Yelmo, J.C.: Guidance for the development of accessibility evaluation tools following the unified software development process. Procedia Comput. Sci. 27, 302–311 (2014)

    Article  Google Scholar 

  70. Strobbe, C., Frees, B., Engelen, J.: An accessibility checker for libreoffice and openoffice.org writer. In: ICCHP’12 Proceedings of the 13th International Conference on Computers Helping People with Special Needs, vol. Part I, pp. 484–491 (2012)

  71. Strobbe, C., Frees, B., Engelen, J.: Accessibility evaluation for open source word processors. In: USAB’11 Proceedings of the 7th conference on Workgroup Human–Computer Interaction and Usability Engineering of the Austrian Computer Society: information Quality in e-Health, pp. 575–583 (2011)

  72. Giakoumis, D., Tzovaras, D., Hassapis, G.: Enabling accessibility characteristics in the web services domain. In: UAHCI’11 Proceedings of the 6th International Conference on Universal Access in Human–Computer Interaction: Design for All and eInclusion, vol. Part I, pp. 177–186 (2011)

  73. Chalkia, E., Bekiaris, E.: A harmonised methodology for the components of software applications accessibility and its evaluation. In: UAHCI’11 Proceedings of the 6th International Conference on Universal Access in Human–Computer Interaction: Design for all and eInclusion, vol. Part I, pp. 197–205 (2011)

  74. Mourouzis, A., Kastori, G.-E., Votis, K., Bekiaris, E., Tzovaras, D.: A Harmonised methodology towards measuring accessibility. In: UAHCI’09 Proceedings of the 5th International Conference on Universal Access in Human–Computer Interaction. Addressing Diversity. Part I: Held as Part of HCI International 2009, pp. 578–587 (2009)

  75. Vigo, M., Aizpurua, A., Arrue, M., Abascal, J.: Quantitative assessment of mobile web guidelines conformance. Univ. Access Inf. Soc. 10(1), 33–49 (2011)

    Article  Google Scholar 

  76. Fernandes, N., Lopes, R., Carriço, L.: An architecture for multiple web accessibility evaluation environments. In: UAHCI’11 Proceedings of the 6th International Conference on Universal Access in Human–Computer Interaction: Design for all and eInclusion, vol. Part I, pp. 206–214 (2011)

  77. Partarakis, N., Doulgeraki, C., Antona, M., Oikonomou, T., Kaklanis, N., Votis, K., Kastori, G.-E., Tzovaras, D.: A unified environment for accessing a suite of accessibility evaluation facilities. In: UAHCI’11 Proceedings of the 6th International Conference on Universal Access in Human–Computer Interaction: Design for all and eInclusion, vol. Part I, pp. 267–275 (2011)

  78. Lopes, R., Votis, K., Carriço, L., Tzovaras, D., Likothanassis, S.D.: Towards the universal semantic assessment of accessibility. In: Proceedings of the 2009 ACM Symposium on Applied Computing, pp. 147–151 (2009)

  79. Votis, K., Lopes, R., Tzovaras, D., Carriço, L., Likothanassis, S.D.: A semantic accessibility assessment environment for design and development for the web. In: UAHCI’09 Proceedings of the 5th International Conference on Universal Access in Human–Computer Interaction. Part III: Applications and Services, pp. 803–813 (2009)

  80. Olsen, M., Nietzio, A., Snaprud, M., Fardal, F.: Benchmarking and improving the quality of Norwegian municipality web sites. Autom. Specif. Verif. Web Syst. 115, 123–136 (2009)

    Google Scholar 

  81. Lopes, R., Carriço, L.: Querying web accessibility knowledge from web graphs. In: Web Technologies: Concepts, Methodologies, Tools, and Applications, pp. 1437–1461 (2009)

  82. Abou-Zahra, S., Cooper, M.: WCAG 2.0 test samples repository. In: UAHCI’09 Proceedings of the 5th International Conference on Universal Access in Human–Computer Interaction. Part III: Applications and Services, pp. 619–627 (2009)

  83. Velasco, C.A., Denev, D., Stegemann, D., Mohamad, Y.: A web compliance engineering framework to support the development of accessible rich internet applications. In: Proceedings of the 2008 International Cross-disciplinary Conference on Web Accessibility (W4A), pp. 45–49 (2008)

  84. Ulltveit-Moe, N., Goodwin, M., Olsen, Pillai, A. B., Thomsen, C., Gjøsæter, T., Snaprud, M.: Architecture for large-scale automatic web accessibility evaluation based on the UWEM methodology. In: Norwegian Conference for Informatics NIK (2008)

  85. Abou-Zahra, S.: Tool-supported web accessibility evaluation (2008)

  86. De Oliveira, E., Motti, V., Pimenta Freire, A., De Matos Fortes, R.: Supporting web page accessibility by using earl reports. In: Proceedings of the XIII Brazilian Symposium on Multimedia and the Web (Webmedia), pp. 32–36 (2007)

  87. Ackermann, P., Velasco, C. A., Power, C.: Developing a semantic user and device modeling framework that supports UI adaptability of web 2.0 applications for people with special needs. In: Proceedings of the International Cross-Disciplinary Conference on Web Accessibility, p. 12 (2012)

  88. Fernandes, N., Costa, D., Neves, S., Duarte, C., Carriço, L.: Evaluating the accessibility of rich internet applications. In: Proceedings of the International Cross-Disciplinary Conference on Web Accessibility, p. 13 (2012)

  89. Fernandes, N., Batista, A.S., Costa, D., Duarte, C., Carriço, L.: Three web accessibility evaluation perspectives for RIA. In: Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, p. 12 (2013)

  90. Kamoun, F., Almourad, M.B.: Accessibility as an integral factor in e-government web site evaluation: the case of Dubai e-government. Inf. Technol. People 27(2), 208–228 (2014)

    Article  Google Scholar 

  91. Suarez-Torrente, M.D.C., Conde-Clemente, P., Martínez, A.B., Juan, A.A.: Improving web user satisfaction by ensuring usability criteria compliance: The case of an economically depressed region of Europe. Online Inf. Rev. 40(2), 187–203 (2016)

    Article  Google Scholar 

  92. Hilera, J.R., Otón, S., Timbi-Sisalima, C., Aguado-Delgado, J., Estrada-Martínez, F.J., Amado-Salvatierra, H. R.: Combining multiple web accessibility evaluation reports using semantic web technologies. ISD (2017)

Download references

Acknowledgements

This study has been funded partially by the government of the Comunidad Autónoma de Madrid (Spain) via Madrid program. Thanks to EduTech Project (609785-EPP-1-2019-1-ES-EPPKA2-CBHE-JP), co-funded by the Erasmus Program of the European Union.

Author information

Authors and Affiliations

  1. Computer Sciences Department, University of Alcala, Edificio Politécnico, Campus Universitario, Ctra. Barcelona Km 33.6, 28871, Alcalá de Henares, Madrid, Spain

    Francisco J. Estrada-Martínez, José R. Hilera, Salvador Otón & Juan Aguado-Delgado

Authors
  1. Francisco J. Estrada-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José R. Hilera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salvador Otón
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juan Aguado-Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francisco J. Estrada-Martínez.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Estrada-Martínez, F.J., Hilera, J.R., Otón, S. et al. Semantic web technologies applied to software accessibility evaluation: a systematic literature review. Univ Access Inf Soc 21, 145–169 (2022). https://doi.org/10.1007/s10209-020-00759-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-020-00759-y

Keywords

Search

Navigation