Skip to main content
SpringerLink
Log in

Advance human–machine interface automatic evaluation

  • Long paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

The need for accessibility evaluation tools is motivated by several endogenous and exogenous reasons coming from the end user (the designer and the developer) and companies releasing information systems. Existing evaluation tools mainly concentrate on examining the code of Web pages: Web pages more and more frequently contain non-HTML parts that entirely escape from being treated by existing techniques. This is the case of the advanced human–machine interface (AHMI), a piece of software programmed in C/C++, used for controlling the advanced flight management system in the aircraft cockpit. Studying this new user interface (UI) requires a structured approach to evaluate and validate AHMI designs. The goal in this work is to develop an evaluation tool to automate the process of evaluating the AHMI. The method addresses: support of multiple bases of guidelines (accessibility or usability or both) on-demand (partial or total evaluation), with different levels of details (a presentation for developers and for those responsible for certifying accessibility). The method goes a step toward the automatic evaluation of UI containing non-HTML parts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Beirekdar, A., Vanderdonckt, J., Noirhomme-Fraiture, M.: A framework and a language for usability automatic evaluation of web sites by static analysis of HTML source code. In: Proceedings of 4th International Conference on Computer-Aided Design of User Interfaces CADUI’2002 (Valenciennes, 15–17 May 2002), pp. 337–348. Kluwer Academics Publishers, Dordrecht (2002)

  2. Beirekdar, A., Keita, M., Noirhomme, M., Randolet, F., Vanderdonckt, J., Mariage, C.: Flexible reporting for automated usability and accessibility evaluation of web sites. In: Proceedings of 10th IFIP TC 13 International Conference on Human–Computer Interaction INTERACT’2005 (Rome, 12–16 September 2005), Lecture Notes in Computer Science, Vol. 3585, pp. 281–294. Springer, Berlin (2005)

  3. Calvary, G., Coutaz, J., Bouillon, L., Florins, M., Limbourg, Q., Marucci, L., Paternò, F., Santoro, C., Souchon, N., Thevenin, D., Vanderdonckt, J.: The CAMELEON reference framework, deliverable 1.1, version V1.1, CAMELEON project (2002) Campbell, E., Maintaining accessible Websites with Microsoft Word and XML. In: Proceedings of XML Europe 2003 (London, 5–8 May 2003), XML Workshop Ltd., London (2002)

  4. Choong, Y., Plocher, T., Rau, P.: Cross-cultural Web design. Handbook of Human Factors in Web Design, pp. 284–300. Lawrence Erlbaum Associates, Mahwah (2005)

  5. Chisholm, W., Vanderheiden, G., Jacobs, I.: Web Content Accessibility Guidelines 1.0, W3C Recommendation, 5 May (1999). Accessible at http://www.w3.org/TR/WAI-WEBCONTENT/

  6. Cooper, M.: Evaluating accessibility and usability of web sites. In: Puerta, A., Vanderdonckt, J. (eds.) Proceedings of 2nd International Conference on Computer-Aided Design of User Interfaces CADUI’99 (Louvain-la-Neuve, 21–23 October 1999), pp. 33–42. Kluwer Academics Publisher, Dordrecht (1999)

  7. Cooper, M., Limbourg, Q., Mariage, C., Vanderdonckt, J.: Integrating universal design into a global approach for managing very large web sites. In: Kobsa, A., Stephanidis, C. (eds.) Proceedings of the 5th ERCIM Workshop on User Interfaces for All UI4ALL’99 (Dagstuhl, 28 November-1 December 1999), GMD Report 74, GMD—Forschungszentrum Informationstechnik GmbH, Sankt Augustin, pp. 131–150 (1999). Accessible at http://ui4all.ics.forth.gr/UI4ALL-99/Cooper.pdf

  8. Farenc, C., Liberati, V., Barthet, M.-F.: Automatic ergonomic evaluation: what are the limits? In: Vanderdonckt, J. (ed.) Proceedings of 2nd International Workshop on Computer-Aided Design of User Interfaces CADUI’96 (Namur, 5–7 June 1996), pp. 159–170. Presses Universitaires de Namur, Namur (1996)

  9. Gallagher, K.B., Lyle, J.R.: Using program slicing in software maintenance. IEEE Trans. Softw. Eng. 17(8), 751–761 (1991)

    Article  Google Scholar 

  10. Guerrero-García, J., González-Calleros, J.M., Vanderdonckt, J., Muñoz-Arteaga, J.: A theoretical survey of user interface description languages: preliminary results. In: Chavez, E., Furtado, E., Moran, A. (eds.) Proceedings of Joint 4th Latin American Conference on Human–Computer Interaction-7th Latin American Web Congress LA-Web/CLIHC’2009 (Merida, November 9–11, 2009), pp. 36–43. IEEE Computer Society Press, Los Alamitos (2009)

  11. Grammenos, D., Akoumianakis, D., Stephanidis, C.: Integrated support for working with guidelines: the Sherlock guideline management aystem. Int. J. Interact. Comp. 12(3), 281–311 (2000)

    Article  Google Scholar 

  12. IBM: IBM’s web design guidelines, December (1999). Accessible at http://www.ibm.com/ibm/hci/guidelines/web/web_design.html

  13. ISO: ISO 9241-10:1996. International Organization for Standardization. Retrieved 22 July (2011)

  14. Ivory, M.Y., Hearst, M.A.: State of the Art in Automating Usability Evaluation of User Interfaces, ACM Computing Surveys, Vol. 33, No. 4, December 2001, pp. 470–516 (2001)

  15. Ivory, M.Y., Mankoff, J., Le, A.: Using Automated Tools to Improve Web Site Usage by Users with Diverse Abilities, IT&Society, Special Issue on Web Navigation Skills, Vol. 1, No. 3, pp. 195–236 (2003). Accessible at http://www.stanford.edu/group/siqss/itandsociety/v01i03/v01i03a11.pdf

  16. Jacobs, I., Brewer, J.: Accessibility Features of CSS, W3C Note 4, W3C, August 1999. Accessible at http://www.w3.org/TR/CSS-access

  17. Jiang, J., Murphy, E.D., Bailin, S.C., Truszkowski, W.F.: Automating a human factors evaluation of graphical user interfaces for NASA applications: an update on CHIMES. In: SpaceOps’92: Proceedings of the Second International Symposium on Ground Data Systems for Space Mission Operations, Pasadena, CA. Jet Propulsion Laboratory (1992)

  18. Kaur, K.: Designing virtual environments for usability, Ph. D. Thesis, City University, London (1998)

  19. Kolski, C., Millot, P.: A rule-based approach for the ergonomic static evaluation of man-machine graphic interface in industrial processes. Int. J. Man Mach. Stud. 35, 657–674 (1991)

    Article  Google Scholar 

  20. Lecharlier, B.: Abstract interpretation and application to interactive system verification. In: Bodart, F., Vanderdonckt, J. (eds.) Proceedings of 3rd International Workshop on Design, Specification, and Verification of Interactive Systems DSV-IS’96 (Namur, 5–7 June 1996), pp. 46–72. Springer, Vienna (1996)

  21. Lecerof, A., Paternò, F.: Automatic support for usability evaluation. IEEE Trans. Softw. Eng. 24(10), 863–888 (1998)

    Google Scholar 

  22. Lee, A., Hanson, V.: Enhancing web accessibility. In: Proceedings of the 11th ACM International Conference on Multimedia MULTIMEDIA’2003 (Berkeley, 2–8 November 2003), pp. 456–457. ACM Press, New York (2003)

  23. Leporini, B., Paternò, F., Scorcia, A.: Flexible Tool Support for Accessibility Evaluation, Interacting with Computers, vol. 18, no 5, pp. 869–890. Elsevier, New York (2006)

  24. Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., Lopez, V.: UsiXML: a language supporting multi-path development of user interfaces. In: Proceedings of 9th IFIP Working Conference on Engineering for Human-Computer Interaction jointly with 11th International Workshop on Design, Specification, and Verification of Interactive Systems EHCIDSVIS’2004 (Hamburg, July 11–13, 2004). Springer, Berlin (2005)

  25. Löwgren, J., Nordqvist, T.: Knowledge-based evaluation as design support for graphical user interfaces. InCHI’92: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 181–188. ACM, New York, NY, USA. Mollwitz, V. AFMS Handbook for Users. Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR). June 2006

  26. Lüdtke, A., Weber, L., Osterloh, J.P., Wortelen, B.: Modeling pilot and driver behavior for human error simulation. HCI 11, 403–412 (2009)

    Google Scholar 

  27. McCathieNevile, C., Koivunen, M.-R.: Accessibility Features of SVG, W3C Note 7, W3C, August 2000. Accessible at http://www.w3.org/TR/SVG-access/

  28. Ressler, S., Wang, Q.: Making VRML accessible for people with disabilities. In: Proceedings of ACM International Conference on Assistive Technologies ASSETS’98 (Marina del Rey, 15–17 April 1998), pp. 144–148. ACM Press, New York (1998). Accessible at http://ovrt.nist.gov/projects/VRMLaccess/assets98/vrmlsigc.htm

  29. Ritsch, H., Sneed, H.M.: Reverse engineering programs via dynamic analysis. In: Proceedings of Working Conference on Reverse Engineering WCRE’93 (Baltimore, 21–23 May 1993), pp. 192–201. IEEE Computer Society Press, Los Alamitos (1993)

  30. Rukshan, A., Baravalle, A.: Automated Usability Testing: Analysing Asia Web Sites. arXiv preprint arXiv:1212.1849 (2012)

  31. Singer, G.: Methods for Validating Cockpit Design The best tool for the task, Ph.D. thesis, Department of Aeronautics of the Royal Institute of Technology, Stockholm, Sweden, March, (2002)

  32. Smith, S.L., Mosier, J.N.: Design guidelines for user-system interface software. Technical Report NTIS No. AD A 154 907 (1984)

  33. Van Sickle, L., Zheng, Y., Ballantyne, M.: Recovering user interface specifications for porting transaction processing applications. EDS Research, Austin Laboratory, 1601 Rio Grande, Suite 500, Austin TX 78701 (1993)

  34. Vanderdonckt, J.: Development milestones towards a tool for working with guidelines. Interact. Comput. 12(1999), 81–118 (1999)

    Article  Google Scholar 

  35. Vanderdonckt, J., Beirekdar, A.: Automated web evaluation by guideline review. J. Web Eng. 4(2), 102–117 (2005)

    Google Scholar 

  36. W3C: Web Content Accessibility Guidelines 2.0, December 2008. Accessible at http://www.w3.org/TR/WCAG/

  37. Wills, L.: Automated Program Recognition: A Feasibility Demonstration. Artificial Intelligence, Elsevier Science Publishers B.V., (North-Holland) (1990)

Download references

Acknowledgments

We gratefully acknowledge the support of the Human European project (Model-based Analysis of Human Errors during Aircraft Cockpit System Design, project funded by FP7-AAT-2007-RTD-1/CP-FP-211988 from European Commission), the ITEA2 Call 3 UsiXML project under reference 20080026, and Mexican PROMEP projects under reference 103.5/12/8136 and 103.5/12/4367.

Author information

Authors and Affiliations

  1. Computer Science Faculty, Benemérita Universidad Autónoma de Puebla, Av. Universidad # 940, CP 20100, Puebla, Mexico

    Juan Manuel González Calleros & Josefina Guerrero García

  2. Belgian Laboratory of Computer–Human Interaction (BCHI), School of Management (IAG), Université Catholique de Louvain, Place des Doyens, 1, 1348, Louvain-la-Neuve, Belgium

    Jean Vanderdonckt

Authors
  1. Juan Manuel González Calleros
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josefina Guerrero García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jean Vanderdonckt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Manuel González Calleros.

Rights and permissions

Reprints and permissions

About this article

Cite this article

González Calleros, J.M., Guerrero García, J. & Vanderdonckt, J. Advance human–machine interface automatic evaluation. Univ Access Inf Soc 12, 387–401 (2013). https://doi.org/10.1007/s10209-013-0310-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-013-0310-7

Keywords

Search

Navigation