Keywords

1 Introduction

In 2010, ISO9241-210 “Human-centred design for interactive system” which is an ergonomic related standard about human centered design was published [1], discussion related human centered design (HCD) and user experience (UX) which is newly defined in this standard becomes active in IT business field.

HCD activities are applied to each development process shown in Fig. 1. HCD is a method to give better UX to stakeholders and to provide system and product with high usability for users and stakeholders [2].

Fig. 1.
figure 1

The relationship among HCD activities and development process

Full size image

On the other hand, traditionally, usability is one of the most important factors for products, system or service. Usability and User experience are also important factors for not only suppliers but also users when product, system and service are developed and operated. Though these are often confused, concepts are totally different.

The former is an index for “easy to use” when user uses product, system or service [3]. So, targets are product, system or service themselves. The latter is to experience whether user is able to do which he/she would like to really do or not. So, this targets to experience using them which includes users [4].

In case of realize “ease of use” or provide experience what user really want to do, some quantitative purpose have to be set in planning and designing phase of product, system or service. To set some concrete purpose, they have to be indices and decided as objective value.

When system and product with high usability could be developed, it is easy to verify their usability by usability test [5]. However, it is difficult to check whether these products or system achieve that a user wants to really do it.

This paper describes a trend of indexing, measurement and evaluation of usability and issues when UX is indexed and possibility of measuring UX and proposed that evaluation index of UX shall be “good”/“bad”, not “high”/“low”.

2 Usability

As described Sect. 1, usability is an index for “easy to use” when user uses product, system or service, and an index of product, system or service themselves. In this section, usability is describe from a view point of “human” and “development”.

2.1 From a View Point of Human

“Easy to use” when using product, system or service are also classified by “efficient operation”, “easy to learn”, “easy to memorize” and “less error” [3]. There is a research to quantify them directly [6]. In this case, “easy to use” is an evaluation of interaction about UI operation. In a point of index to achieve an aim, usability is defined as “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” in ISO9241-11 [7]. Finally, to realize high “extent”, “efficient operation” or “easy to learn” have to be achieved. ISO decided these elements to set easily as objective value.

Recently, there is a trend that usability is a part of quality. “Quality in Use” is defined and software quality model [8]. In this model, “freedom from risk” and “context coverage” are added to three elements in ISO9241-11. Figure 2 shows the quality in use model defined in ISO/IEC25010 [8].

Fig. 2.
figure 2

Quality in use model [8]

Full size image

2.2 From a View Point of Development

Quality model described before, not only “quality in use” but also “product quality” which has eight elements is defined (Fig. 3).

Fig. 3.
figure 3

Product quality model [8]

Full size image

In this product quality model, “usability” is also listed. This usability is different from the meaning of ISO9241-11, they are, “appropriateness recognizability”, “learnability”, “operability”, “user error protection”, and so on. These elements are related to total product quality. This means that to achieve product quality related to usage is to realize quality in use. Figure 4 shows the relationship among usability (correspond to quality in use), dialogue principle (correspond to product quality) and characteristics of information presentation [9].

Fig. 4.
figure 4

Relationship among usability, dialogue principle, characteristics of information presentation [9]

Full size image

2.3 Evaluation and Measurement of Usability

Usability evaluation has been tried to carry out conventionally. However, evaluation and measurement do not distinguish each other. In ISO, two kinds of evaluation formats, they are “summative test” [5] and “Formative test” [10] are defined, respectively. The former is not an evaluation index but a format for measuring “effectiveness”, “efficiency” and “satisfaction” defined in ISO9241-11, the latter is a format for identifying usability problem using inspection method, usability test and user observation. Three detail elements of usability are as follows [7]:

  • Effectiveness

    • Accuracy (extent to which an actual outcome matches an intended outcome)

    • Completeness (extent to which users of the system, product, or service are able to achieve all intended outcome.

  • Efficiency

    • Time used (the time expended in attempting to achieve a goal)

    • Human effort expended (the mental and physical effort expended to complete specified tasks)

    • Financial resources expended (include the costs of using the system, product or service, such as paying wages, or the cost of energy or connectivity

    • Materials expended (physical items (e.g. raw materials, water, paper) used as input to the task (including maintenance tasks) and processed by the system, product or service.

  • Satisfaction

    • Physical responses (Feelings of comfort or discomfort represent physical components of satisfaction)

    • Cognitive response (Attitudes, preferences and perceptions represent cognitive components of satisfaction)

    • Emotional response (represent affective components of satisfaction).

By using this summative test format, results of evaluation test for these items can be described. Though there are many test methods to do this, the experimenters can refer measurement method of quality in use in ISO/IEC 25022 [11].

3 User Experience

3.1 What is User Experience

User experience (UX) is a general term of experience which is provided by use and a product or service. Sometime, UX is explained as a concept which emphasize not only each function or usability but also to realize what user want to do pleasantly and comfortably. UX is defined that “To have experiences with use in “before use”, “during use”, “after use” and “through total usage” and “To be able to shape “users want to do” through experience in each step and to achieve it” in User Experience white paper [4]. This definition is concreted of concept, but it is difficult to measure UX by using this definition.

ISO 9241-210 [1] defines UX as follows: “person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service”. This definition is difficult to understand for almost engineers though this text is tried to represent that UX could be measured.

3.2 Evaluation and Measurement of UX

As described in Sect. 1, usability and UX are often confused, concepts are totally different. To distinguish them, some trial were carried out. Kurosu et al. corresponded the relationship between product quality and quality in use to UI and UX [12]. This paper explains that quality in use is evaluated by using UI. Quality in use are separated to objective quality in use and subjective quality in use. And total of them defined as UX. These index of UX are similar to usability, so, evaluation results are numeric value, and can be represented by “high” and “low”. However, does this index evaluate “experience”? An index of “experience” seems “good” of “bad”.

Firstly, “before use experience” is considered. When before use, specification according to user requirements are decided, expected value will be high because of emotional change by imagine for use, environment and promotion. Next, “during use/after use experience” are considered. This is real experience which depends on satisfaction or emotion level which are achievement level of specification or usage feeling correspond to users’ original image.

So, each expected value and real experience is represented by “high” and “low”. However, user experience is a function of these two elements.

As shown in below, expected value by before using and real experience value by during/after use are made these a function respectively.

  • Expected value (before experience)

Before using, expected value for use will be high because specification is decided by user requirements and influence by imaging usage at special environment and promotion.

$$ {\text{Exp = f(specification, environment, promotion, usage, etc}} . ) $$

  • Real experience (experience during use and after use)

By using, user receives high satisfaction because this system can be judged to meet specification originally decided. And high emotion because of enjoyable and comfort by using.

$$ {\text{Realexp = f(real measurements, environments, usability, etc}} . ) $$

And index of UX is a difference between them. That is,

$$ \begin{array}{*{20}c} {\text{UX = f(Exp, Reaexp)}} \\ {{\text{e}} . {\text{g}} . {\text{ (Exp}}\; - \;{\text{Realexp), (Exp/Realexp)}}} \\ \end{array} $$

From this, to reduce a gap between high expected value and real experience value becomes to provide good UX for developers and providers.

4 Conclusion

This paper proposed an indexing method for UX. We apply this method to secure operation system include human Fig. 5.

Fig. 5.
figure 5

Secure system operation based on behavior factor analysis of human

Full size image

From this figure, expected value and real experience are able to describe below:

$$ {\text{Exp }} = {\text{ f}}\left( {{\text{specification}},{\text{ usage}},{\text{ environment}},{\text{ possible emergency information}},{\text{ education for counterplan}},{\text{ etc}}.} \right) $$
$$ {\text{Realexp = f(real performance, usability, environments, defense against attack, etc}} . ) $$

When system are installed, expected value include emergency information and education for counterplan are raised. When system will attacked, satisfaction by experience are raised if education/provision of information are good. From this, difference between expected value and real experience are minimized.

Like this, when measuring UX, a variety of elements have to be considered.