1 Introduction

With the progress and the diffusion of Information and Communication Technology (ICT), many devices, such as computers, cell phones, smartphones and tablets, have become more convenient and are used in a variety of ways in our everyday life. At the same time, in Japan, the progress of aging is very rapid and since 2005 Japan has become the most aged society in the world. The percentage of elderly people (percentage of total population aged 65 and over) in Japan has risen up from 5 % in 1950, 10 % in 1985, and 20 % in 2005, to 25 % in 2013, and is expected to rise to 30.3 % in 2025 and 39.9 % in 2060 [1]. Against these factors, manufacturers in electronics have developed cell phones, smartphones and computers for elderly users [2, 3], but there still exists a gap between high-skilled end users and low-skilled end users depending on theie demographic traits. One of the marked differences is related to differences within the age group [4].

In previous studies [5, 6], we conducted a questionnaire survey based on a quantitative approach to grasp the overall trend of using cell phones. Answers to the questionnaire showed some variation, such as how elderly people are using afewer number of functions and how they cannot operate functions that are widely known and used among other age groups. Compared to young people, elderly people do not use cell phones actively and do not use as many functions effectively. It was also found that there are differences in the relative importance of the value criteria regarding age and sex. Younger people, ages 20s, tend to emphasize performance, functionality and design, though there is a slight disparity between male users who emphasize performance and female users who emphasize design. On the other hand, elderly people emphasize the ease of operation and the display size. Elderly people do not simply neglect the design of devices (Kansei aspect) but put more emphasis on the usability and will regard the design as important if there are no usability problems. There is a generation gap between those people in regards to the value criteria, how the design (appearance) and the usability are affecting the purchase of cell phones by adopting the research results. The usability of the device is significant to the elderly, so the usability of software such as smartphone applications should also be considered through the spread of smartphones.

ISO9241-11’s definition of usability is the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [7].” Regarding this definition, Kurosu argued that the concept of usability consists of just effectiveness and efficiency. They should be considered objective quality characteristics in the same way that reliability, cost, safety, compatibility and maintenance are. However, satisfaction is related to subjective quality characteristics such as pleasure, joy, beauty, attachment, motivation and value, as well as objective quality characteristics [8].

In this paper, satisfaction was regarded as one of the key criteria for the evaluation of artifact quality. We focused on the following points, and conducted the usability evaluation of smartphone applications as follows:

  1. (1)

    What is the most important factor affecting the satisfaction when using artifacts?

  2. (2)

    Whether the strongest determinant of satisfaction differs depending on age groups?

2 Usability Evaluation of Smartphone Application

We conducted a usability evaluation for the purpose of getting detailed information regarding the users’ subjective impressions and feelings while using a new application we developed. The application focuses on the connection between satisfaction and other items used the evaluation.

2.1 The Stress Measurement Application Overview

We developed an application that can estimate mental stress levels more easily than the conventional method of using an electrocardiograph (ECG). Figure 1 is the flow diagram of calculations for the heart rate variation (HRV) in the conventional method, ECG measurement using many devices, and from the proposed application to measure the pulse wave using only the smartphone.

Fig. 1.
figure 1

HRV calculation flow

Full size image

We confirmed the accuracy of HRV detected by our application with a verification experiment. According to the verification experiment, HRV measurement using our application was found to be valid for assessing mental stress at the same precision as the ECG method [9].

Figure 2 shows how to hold the smartphone during the HRV measurement using our application. The first step is to place a fingertip over the lens of the smartphone camera and the light-emitting diode (LED). The left index finger is preferable. The camera acquires images of blood vessels in the finger, and obtains the pulse wave data by measuring the timing of the blood flow. Using this algorithm, we can calculate the HRV from the pulse wave data using a smartphone.

Fig. 2.
figure 2

Measurement of the pulse wave with the smartphone application

Full size image

2.2 Method of Usability Evaluation

Participants were 17 young people in their 20s and 12 elderly people in their 60–70s, all living in Tokyo. The average age of the young people was 20.9 years with a standard deviation (SD) of 0.5, and the average age of the elderly users was 66.2 years with an SD of 1.9. All 17 young participants had their own smartphones and all 12 elderly people had cell phones.

We defined elderly people as those who are over 65 years of age, based on the World Health Organization (WHO) definition.

First we explained the research it is for the academic purposes and does not have anything to do with the sales or marketing. We then told the participants the expected duration of the experiment and that they were free to quit at anytime during the experiment should they feel uncomfortable with our questions or our treatment of any information gathered.

After obtaining their consent, we explained the evaluation procedure and how to use the application to each participant. Then the participants performed a trial to measure their mental stress level using the application. After the measurement, subjects answered the following questions and commented on the application.

2.3 Contents of Usability Evaluation

The usability evaluation included the following questions (Table 1): demographic characteristics (name, sex, age, experience with using ECG, etc.), number of errors during the use of the application, whether or not the measurement was successful, and their impressions after using the application.

Table 1. Contents of usability evaluation
Full size table

Description on basic characteristic of the application and comprehensive evaluation of the application were evaluated by 5 point rating scale. Only the results of the usability evaluation are described in terms of the association with satisfaction and other evaluation items in this paper.

2.4 Results

The evaluation first quantified the degree of subjective belief in the questions and then analyzed by Kendall’s rank correlation coefficient. The rank correlation coefficients were calculated using a software excel statistics: Ekuseru-Toukei 2012 (Social survey research information Co., Ltd., Tokyo, Japan). In the results of the rank correlation coefficients analysis, we found some differences in the use of the ICT devices between young people and elderly people (Table 2).

Table 2. Rank correlation coefficients between age/sex and items
Full size table

It was shown that elderly people made more positive evaluations than the youth on the following 5 items. Among the 5 items that the elderly rated positively, these 3 items showed a significant difference with a value of .001; “(6-a) Ease of understanding on the contents”, “(6-g) Responsiveness” and “(6-i) Length of measurement”. Items “(6-f) Ease of use” and “(7-e) Satisfaction” also showed a significant difference with a value of .005.

The association between evaluation of satisfaction and other items are shown in Table 3. 4 items were commonly linked to satisfaction for both age groups: “(6-g) Responsiveness”, “(6-h) Reliability of data”, “(7-c) Likability” and “(7-d) Repetitive use of the App”. On the other hand, “(6-a) Ease of understanding on the contents” had a statistically significant association with satisfaction from only young people. Additionally, 4 items: “(6-b) Ease of understanding on the operational procedure”, “(6-c) Ease of understanding on the screen structure”, “(7-a) Safety” and “(7-b) Usefulness” were statistically significant linked to the satisfaction among elderly people only.

Table 3. Rank correlation coefficients between satisfaction of 2 age-groups and items
Full size table

2 of the 5 items suggested the association with satisfaction from the youth, were linked to satisfaction at the significance level of .001: “(6-a) Ease of understanding on the contents” and “(7-d) Repetitive use of the App”. 8 items indicated the significant association with satisfaction among elderly people. 3 of the 8 items were related to satisfaction at the significant level of .001: “(6-h) Reliability of data”, “(7-c) Likability” and “(7-d) Repetitive use of the App”.

3 Discussion

This study focused on factors that influence satisfaction with the usability evaluation. The evaluation of satisfaction with the application was more positive among elderly people. Factors affecting satisfaction were different depending on the age groups.

All young participants were using smartphones in their daily lives, hence they are accustomed to using such applications. Because the application does not require any special operation, they reported boredom while staring at the screen during the use of the application. On the other hand, all member in the group of elderly users who had never used smartphones but had experienced ECG measurements in hospitals, felt that the measurement time was short and was content with the application.

The results showed that understandability of the application contents and whether to use the application repetitively were strongly linked to satisfaction among young people. Whether to use the application repetitively, in addition to reliability and likeability, had deep connections with satisfaction from the elderly. It was also found that understandability of the application contents had association with satisfaction from only the youth, and that the understandability of the application procedure and screen structure, safety and usefulness of the application were linked to satisfaction among the elderly only. It can be interpreted that only the elderly were strong satisfied with the safety and usefulness of the application based on their prior experiences of with standard ECG measurement practices.

Early in the application development process, we conducted a product test using a prototype for another paticipants and extracted the requirements from young and elderly groups [10]. Requests obtained from the youth included a demand to make application contents easy to understand, and requests from elderly people were also embraced demands to make the application procedure and the screen structure more understandable. In order to get high level of satisfaction with the quality of artifacts, it is important to ensure conformance to the requirements from each user.

4 Conclusion

In this paper, we focused on the factors influencing satisfaction with the usability evaluation of smartphone application. We conducted the usability evaluation of the application for obtaining information regarding the relationship between satisfaction and other evaluation items.

It was found that the evaluation of the application was more positive among elderly people, and the youth and elderly evaluated satisfaction of the application use by different criteria. The result showed that factors influencing to satisfaction could be determined by the conformance to requirements of each user group.