Experimental validation of a set of cultural-oriented usability heuristics: e-Commerce websites evaluation
Introduction
Usability is a main concept in human–computer interaction (HCI). Among efforts to explain the term, usability has been called “the capability to be used by humans easily and effectively” [1]; “quality in use” [2]; and “the effectiveness, efficiency, and satisfaction with which specified users can achieve goals in particular environments” [3]. In general, its attributes are highly important to users when choosing a product [4].
The problem is that people differ across regional, linguistic and country boundaries and users requirements are strongly influenced by their local cultural perspective [5], [6]. Cultural-oriented usability is about making websites an effective means of communication between a global website user and a local user. However, the influence of culture on usability is a controversial issue in the field of HCI. Those that are in support of accommodating culture into the design of user's interfaces do not seem to agree on whether to incorporate objective culture, subjective culture or both, into user interface design [7].
We need to consider that users do not want to spend more than a couple of minutes learning how to use a website, and an interesting use case can be e-Commerce websites. The first impression and experience of use is critical and might determine the product’s success [8]. But, are the e-Commerce interfaces ready to accept this new group of users [7], [9], [10], [11]? Different authors base their studies on Hofstede, that identifies the cultural influence of values that people possess during their activities over cross-cultural interfaces [12].
Nowadays, considering the exponential growth of internet [13], multidisciplinary areas likes usability assumes a major role, as transactional websites must allow users to achieve their objectives efficiently, effectively and satisfactorily [14]. A way to evaluate this usability is through heuristic evaluation [15], were a small set of evaluators judge a user interface in order to determine whether or not it meets usability design principles [16]. As stated by Lewis [17], there are two major concepts of usability, commonly referred to as summative and formative. While summative usability focuses on the use of metrics associated with meeting global task and product goals (i.e., measurement-based usability), the focus of formative usability is the detection of usability problems and the design of interventions to reduce or eliminate their impact. The latter is the focus of this work, but in a local experiment we applied a summative definition too.
Heuristic evaluation is one of the most widely used usability evaluation methods. It normally requires the participation of 3–5 evaluators [15]. When selecting the set of heuristics, there are (generally) two choices: generic or specific heuristics. Specific heuristics may be comparatively difficult to understand and apply, but they can potentially detect more specific usability issues related to the application’s domain. Generic heuristics are usually easy to apply and understand, but they can miss specific (domain-related) usability issues [18].
In this paper, we present a set of 12 cultural-oriented usability heuristics, developed through an iterative process. An early (abbreviated) version was published, that includes only the name and a brief description for each heuristic [19]. We now present a set of usability heuristics using a template that includes: definition, associated cultural dimension, explanation, examples, benefits, and anticipated problems. We have performed additional validation experiments, such as heuristic evaluation and a walk-through experiment.
This paper is structured as follows: Section 2 introduces main concepts related to Hofstede’s cultural dimensions, usability, usability heuristics, and usability evaluation challenges. Section 3 describes the process of developing the cultural-oriented usability heuristics, and how we conduct the validation. Section 4 presents the results of the latest experiments. We detail our proposed usability heuristics set in Section 5. Finally, conclusions and discussion are presented in Section 6.
Section snippets
Cross cultural interaction and Hofstede's cultural dimensions
Geert Hofstede performed a study that may be the most comprehensive about the cultural influence of the values that people possess during their activities [12], [20]. From first outcomes and lately additions, Hofstede developed a model that identifies the principal six dimensions in order to help to differentiate between cultures: (1) power distance, (2) individualism, (3) masculinity and (4) uncertainty avoidance. Then he added two more dimensions, (5) long-term orientation (based on Confucian
Defining usability heuristics
The usability inspections, including heuristics evaluation, are well documented and many publications describe the use of these methods [42], [43]. Literature usually focuses on describing the advantages and disadvantages of heuristic evaluation methods, but not in developing new evaluating methods or usability heuristics.
We developed the set of cultural-oriented usability Heuristics, using a 6-steps methodology proposed by Rusu et al. [44]:
STEP 1: An exploratory stage, centered in gathering
Analysis and results
Iterations 1 to 3 were briefly described in Section 3; the outcomes were published in previous works [19], [51]. The different experiments, which have been the basis for the last iteration are summarized below.
We divided the experiments in three topics: (1) Heuristic evaluation using an experimental vs control group approach, (2) a heuristic evaluation focused on Nielsen´s proposal with cultural approach results, and (3) a usability walk-through method. We used different approaches to cover
A set of cultural – oriented usability heuristics
Based on the experimental results described in the previous section, the cultural-oriented usability heuristics were refined and presented below. Each of the 12 heuristics has a defined structure: definition, the associated cultural dimension (based on the heuristic attribute), application example, benefits, and anticipated problems associated with misinterpretation.
Regarding the methodology and experimentation process
Like other sciences and engineering disciplines, usability engineering requires a cycle of model building, experimentation, and learning. Experiments are valuable tools for all researchers who are involved in evaluating and choosing between different methods, techniques, languages, and tools. However, for our case, there is currently no clear protocol for heuristics’ validation. The whole process of developing usability heuristics is yet to be formalized.
The methodology used in the development
Acknowledgments
The authors would like to thank all the participants involved in the experiments that the present study required, especially the members of the “UseCV” Research Group. The work was highly supported by the School of Informatics Engineering (Escuela de Ingeniería Informática) of the Pontifical Catholic University of Valparaiso (Pontificia Universidad Católica de Valparaíso — PUCV) — Chile. Jaime Díaz has been granted the PUCV 2013–2014 and 2014–2015 Graduate Scholarship.
References (70)
The relative importance of antecedents to cross-cultural adjustment: implications for managing a global workforce
Int. J. Intercult. Relat.
(2004)Psychology of translation in cross-cultural interaction
Procedia - Soc. Behav. Sci.
(2014)Barriers to e-commerce and competitive business models in developing countries: a case study
Electron. Commer. Res. Appl.
(2007)- et al.
Towards effective online review systems in the Chinese context: a cross-cultural empirical study
Electron. Commer. Res. Appl.
(2013) - et al.
Cross-cultural effects in E-retailing: the moderating role of cultural confinement in differentiating Mexican from non-Mexican Hispanic consumers
J. Bus. Res.
(2013) Customer behaviour in electronic commerce: the moderating effect of E-purchasing experience
J. Bus. Res.
(2010)- et al.
Usability – context, framework, definition, design and evaluation
Hum. Factors Inform. Usability
(1991) - N. Bevan, M. Azuma, Quality in use: incorporating human factors into the software engineering lifecycle, in: Software...
- ISO 9241−11, Ergonomic requirements for office work with visual display terminals (VDTs)-Part 11, Guidance on...
- et al.
The importance of usability in product choice: a mobile phone case study
Ergonomics
(2009)