Keywords

1 Introduction

This paper presents the user evaluation (Heuristics) stage from an on-going project aiming to develop an age-friendly system design for smart homes that increases independence of older adults by enabling them to age in their own homes. The ongoing project will propose a system design that explores daily activities by monitoring, predicting and reminding functionalities, aiming for comfort and independent living for older adults ageing at home. The system is based on pressure, motion, temperature and air quality sensors that monitor the daily activities of the user and send the information to the system hub. The collected data will set alarms and reminders to assist older adults and their caregivers in predicting, preventing or providing emergency support. The project aims to take into consideration age-friendly design guidelines of a mobile application to address older adults’ users [1]. The proposed system is composed of three interactive interfaces: Smart TV app, home hub tablet and smartphone app, to work as a personal assistant at home. In this paper, the smartphone application interfaces were evaluated to apply the findings on the other system interfaces.

In 2019, as reported by the United Nations [2], the range of 65 years of age was identified as the world fastest-growing population. Therefore, by 2050, is projected that the share of older adults population will reach 28.7% of the total population in the European Union [3]. As life expectancy increase, most of older adults prefer to remain independently in their own places [4]. The concept of ageing at home enable older adults to live in their own homes as they age despite health and mobility changes [5]. As life expectancy grows, in many countries the older adults’ population is retiring later in life [6]. Therefore, it is important not to stereotype the ageing population based on common perceptions and assumptions when developing comprehensive and accessible solutions. This outdated stereotypes for older adults’ lifestyle and behavior limits the comprehension of their problems and the development of innovative opportunities for the ageing population needs [7]. A new perspective should be taken into consideration for the future ageing population. This emphasizes the importance of ageing research, especially innovative solutions (i.e., smart home systems and assistive technologies) that have significant potential to empower older adults to remain independent, safe and comfortable at home. The smart home concept can be defined as a lifestyle support that represents a house installed with sensors and control devices connected through a communication network [8]. It empowers the users to remotely control household appliances and it can provide comfort, security, convenience and energy efficiency. A smart home environment aims to assist and support residents to feel more comfortable, safe and independent at their home, using monitoring, warning and remote controlling integrated systems [9]. Smart and assistive technologies can provide older adults self-care, relieve caregivers support and also supply new opportunities for personalized healthcare monitoring [7]. Moreover, it can offer constant health and safety management enhancing their convenience and comfort in their daily activities [8]. This innovative technology can have a great impact to improve quality of life and encourage independent living at home so older adults can achieve long and healthy life [10].

Smart home systems have been highly increased to facilitate assisted living and health monitoring so older adults can live independently in their home, and also improve the relationship and proximity with their families and caregivers [11]. Currently, a large number of existing mobile applications are focused on health monitoring and assistive living; though, most of the existing systems have not been developed or adapted enough to older adults needs and abilities [12]. In this sense, the older adults still face considerable challenges in usability, feasibility and accessibility among User Interfaces (UIs), such as small fonts, low color contrasts and complex interactions [12]. It is essential to take into consideration the natural ageing declines that can potentially impact their experience with the technology and provide them accessible age-friendly products and services [12, 13]. Technologies, tools and devices, when properly designed to address older adults’ abilities, needs and preferences, can empower their sense of wellness and independence. Once they are introduced to the basic functions, they feel included to the technology, and interested in and curious about smart devices that can enhance their daily activities [12]. Although, when they can’t adapt or understand those tools and devices it can cause the sense of frustration and isolation [14, Ch. 21].

There are significant studies [1, 15, 16] about the natural ageing, cognitive, sensitive and motoric declines and how it affects the older adults learning process and experience when navigating on the web or using mobile applications. Some cognitive abilities, such as vocabulary, can be resilient to an aging brain and sometimes even improve with age. However, other abilities, such as conceptual reasoning, memory, and processing speed, decline gradually over the years. On the other hand, some activities can be associated with high cognitive function in older adults, such as intellectual engaging activities (e.g., doing puzzles, reading, using a computer, playing musical instruments), physical activities and social engagement [17]. Ageing behavior process can be hard to generalize. It always depends on the context living of the older adult. Although, to assist the development of age-friendly products and services, there is a slightly gap on recent research on usability heuristics for UIs to address older adults needs, abilities and limitations [18]. Age-friendly design aims to help older adults read, notice, scan and understand the information displayed on the interface. Age-friendly guidelines would not only help older adults, but would also be more user friendly for all users [1]. Developing an age friendly system design, to a complex smart home and health monitoring system, enables the opportunity to increase the adoption of assistive technologies by older adults. As the older adult’s population increases, they turn to be potential beneficiaries of digital products. The age-friendly design should go further off accessibility, it should also make the technology attractive, powerful, easy and enjoyable to use [1].

Usability is a key aspect of the multidisciplinary field of Human Computer Interaction (HCI) to ensure the ease of use of a system, tool or device. When designing and building interactive systems the user should be first priority. It is about to understand human capabilities and limitations [19]. In many UI designs, with low contrast colors, small fonts, small targets, it can make it difficult for older adults easily use and accept mobile technologies and other smart devices [1]. Among current studies of heuristic evaluation of mobile applications to support older adults’ users, Silva et al. [20] identified a research gap of appropriate heuristics to evaluate smartphone applications to be suitable and inclusive for such user groups. In this sense, this paper shows results of a heuristic evaluation that identified potentially usability problems which could be faced by older adults when interacting with a smartphone app for Smart Home.

2 Methodology

The expert-based method developed by Nielsen [21] is applied to evaluate the usability problems of a smart home and health monitoring prototype supporting older adults. The objective of the heuristic evaluation is to evaluate the UI usability characteristics according to the principles and guidelines. The 10 Heuristics are as follows: “Consistency and Standards” (H1), “Visibility of the system” (H2), “Match between the system and the real world” (H3), “User control and freedom” (H4), “Error prevention” (H5), “Recognition rather than recall” (H6), “Flexibility and efficiency to use” (H7), “Aesthetic and minimalist design” (H8), “Help users recognize, diagnose, and recover from errors” (H9) and “Help and documentation” (10).

According to Rogers et al. [22], if heuristic evaluation is conducted on a functioning product or a conceptual prototype, the experts need to have some specific user tasks in mind to focus the inspection. Therefore, representative user tasks were defined for the experts.

2.1 Participants

Three experts with high expertise in user experience were selected to participate in the study. The user experience experts have also relevant expertise in performing heuristic evaluation in digital products.

2.2 Procedure

The procedure was published online. Each evaluator received by email the heuristic evaluation sheet with the instructions and list of tasks with a link for the interactive prototype. These tasks were selected due to their representation of the main functions of the system related to health and home features. The procedure was done in three stages:

  • (i) Briefing Stage. This stage entailed giving the evaluators the description and objective of the project with the information in detail about the goals and procedure of the heuristic evaluation.

  • (ii) Task Analysis. In this stage, the evaluators performed tasks analysis to find usability problems they face within each interface of each given task. The flow of each task was shown in Fig. 1, 2, 3, 4, 5, 6, 7, 8 and 9.

    Fig. 1.
    figure 1

    User flow of Task 1 “You should take “Aspirina” 3 times a day. Check if you are taking it correctly”.

    Full size image
    Fig. 2.
    figure 2

    User flow of Task 2: You should take 2 pills of “Dipirona” per day, starting at 10 a.m. for one week. Add “Dipirona” to your medicine reminder list.

    Full size image
    Fig. 3.
    figure 3

    User flow of Task 3: New check-ups become available from time to time. Check if you are missing any available health checkup.

    Full size image
    Fig. 4.
    figure 4

    User flow of Task 4: You remembered that you left the living room lights on and you are in your bedroom, reading a book. Turn off the lights in the living room.

    Full size image
    Fig. 5.
    figure 5

    User flow of Task 5: You are feeling cold in your bedroom. Increase the temperature to 25 °C.

    Full size image
    Fig. 6.
    figure 6

    User flow of Task 6: It’s raining a lot lately and it causes humidity problems at home. Check the humidity of the house.

    Full size image
    Fig. 7.
    figure 7

    User flow of Task 7: You are arriving at the supermarket and it just started to rain. Check if you left your windows open.

    Full size image
    Fig. 8.
    figure 8

    User flow of Task 8: A cake is in the oven while you are watching TV in the living room. You decided to open the kitchen window to release the heat. Check if the kitchen window is open.

    Full size image
    Fig. 9.
    figure 9

    User flow of Task 9: You are not at home and you wonder if you left your TV on. Check if the TV in the living room is on.

    Full size image

  • (iii) Evaluation Stage. In this stage, the evaluators conducted the evaluation (Nielsen’s 10 Heuristics) on the usability problems that they found in the interactive prototype and prioritize them according to Nielsen’s five-point Severity Ranking scaled from 0–4, where 0- indicates no problem, 1- cosmetic problem, 2- Minor problem, 3- Major problem, and 4- catastrophic [23]. During the evaluation, evaluators also compiled a document where they reported the usability problems, the descriptions of the problems, place of occurrence on the UIs, and related to the violated heuristics and the severity ratings applied.

3 Results and Discussion

The data were reported both quantitatively and qualitatively. Quantitatively, after receiving the heuristics severity ratings by each evaluator, the results were combined. Qualitatively, the evaluators performed task analysis to identify the problems and, also, sent a report by compiling their feedback regarding the problems.

  1. a)

    Task analysis and heuristic evaluation

The UI compliance to Nielsen’s heuristics was inspected while the experts performed the representative tasks. Evaluators were asked to highlight and describe each usability problem while they were performing each task. The problems were sorted in 6 categories: labelling, visual hierarchy, visual consistency, terminology, feedback and interaction. The most common usability problems found in the task analysis is reported in Table 1.

Table 1. Common usability problems identified
Full size table

In total, there were 80 usability problems found, 8 out of 10 heuristics were violated, and 78 violations were encountered. For each usability problem one or more heuristics violations could be assigned. “Consistency and Standards” (H1), “Match between the system and the real world” (H3) and “Aesthetic and minimalist design” (H8) were the most violated heuristic, encountered in 66 usability problems, equivalent to 68% of the overall problems. “Visibility of the system” (H2), “Error prevention” (H5), “Flexibility and efficiency to use” (H7), “User control and freedom” (H4) and “Recognition rather than recall” (H6) were others violated heuristics encountered in 31 usability problems. In heuristics violations, “Consistency and standards” (H1) represents 30% of the violations, followed by “Match between the system and the real world” (H3) with 23% and “Aesthetic and minimalist design” (H8) with 15% of the heuristics violations. While “Help users recognize, diagnose, and recover from errors” (H9) and “Help and documentation” (10) had no violations identified. Heuristics violations percentages are shown in Fig. 10.

Fig. 10.
figure 10

Percentages per each Heuristics

Full size image

Experts estimated the severity rating for each Heuristics. The severity mean analysis showed that 3% of the problems were considered catastrophic (mean > 3,5) and it must be a high priority to fix them [23]. 13% were considered major usability problems (3,5 > severity rating <=2,5), according to Nielsen [21] that have the potential for confusing users and making them use the system erroneously. While 81% were minor usability problems (2,5 > severity rating >=1,5), that might slow down the interactions or cause some unnecessary inconveniences for the users [21]. Finally, 6% were considered cosmetic problems. Figure 11 illustrates the average severity rating given among Heuristics.

Fig. 11.
figure 11

Severity rating

Full size image

On the overall, the experts found eight major and two catastrophic usability problems (Table 2). At the main screens related to health (saúde) and home (casa) features, the experts found the majority of major usability problems and two catastrophic. On the tab navigation bar, the experts considered as catastrophic the unlabeled icons even when the screen is not displayed. This problem can confuse the user and impact the main features of the system. Confusing and technical language was also reported as catastrophic when using terms that might not be familiar for the older users and in generic feedback messages. As the project developed in Portuguese language, the experts recommended to avoid using English terms, highlighting that it is a more familiar language for young users than and for the older ones. As the main page of the system, these catastrophic problems can compromise the system comprehension, consequently they are urgent to be fixed. Major usability problems were encountered on medication features. The experts reported that there is too much complex information on medication details that can make it difficult for users to read and it can highly compromise the user comprehension of this important feature for the user’s health. Notifications and status of the system were also reported as major problems as experts reported that is insufficient visual identification and it can be almost unnoticed for the user.

Table 2. Catastrophic and major usability problems
Full size table

The severity given by heuristics showed that the catastrophic problems were identified in “Match between the real world and the system” (H3) and “Aesthetic and minimalist design” (H8). However, on the mean severity analysis, no catastrophic problem was encountered. Most major usability problems were considered into “Recognition rather than recall” (H6). While minor usability problems were considered mostly into “Consistency and standards” (H1) followed by “Match between the system and the real world” (H3) and “Aesthetic and minimalist design” (H8). The distribution of severity rating is shown in Fig. 12.

Fig. 12.
figure 12

Distribution of severity rating into heuristics

Full size image
  1. b)

    Experts recommendations

Overall, the health monitoring features were considered to be more important for the users than the smart home features. Therefore, experts recommended that the health monitoring screen should be displayed as a homepage by default, instead of house monitoring features.

The usage of plain, obvious and natural language is recommended to improve error prevention and ease of use of a complex system, avoiding technical and English terms, even that it seems to be a common use in other systems.

Minimalist design is also recommended to clarify and visually reinforce important information for the user. They suggested removing unnecessary visual elements (e.g., underlined text, icons with no context of actions, unnecessary pop ups) to keep relevant graphics (e.g., graphics should provide relevant and simple information for the user) and visual consistency so don’t distract the user from important information. And create an effective visual language to maintain the visual consistency of the system.

They were concerned that using only Nielsen heuristics on a design process would not improve the interface considerably and provide a granularity of findings on the performed user tasks, according to the identification discovery protocol, identifying as violated heuristics and severity level. Combining these findings with real user testing will provide a return of more discoveries which could be more valuable to improve the interface and the overall experience.

Through the heuristic evaluation method, experts identified potential usability problems that would probably impact older adults’ experience. This study showed that applying Nielsen´s broad usability rules with evaluators with high expertise in user experience can provide relevance and knowledge to improve usability and experience in age-friendly design. However, more specific existing heuristics for mobile applications (SMASH) [18] and for elderly users [24] could be explored to inspect age-friendly design guidelines.

4 Conclusions and Future Work

This paper presents the user evaluation (Heuristics) stage from an on-going project aiming to develop an age-friendly interface design for older adults to enable them age in their own homes while increasing their independence. The objective of the paper is to report the findings of a heuristic evaluation that identified potentially usability problems which could be faced by older adults when interacting with a smartphone app for Smart Home.

Through the heuristic evaluation method, experts identified potential usability problems that would probably impact older adults’ experience. This study showed that applying Nielsen´s broad usability rules with evaluators with high expertise in user experience can provide relevance and knowledge to improve usability and experience in age-friendly design. It can be noticed that Nielsen’s heuristics are broadly generic to evaluate age-friendly design guidelines. However, it is still relevant to provide substantial and important findings in the evaluation process.

In an iterative design process, these usability issues can be fixed so the smart home system can evolve into a more usable application for older adults. Despite the valuable expert’s evaluation, it is important to say that this methodology doesn’t overpass the importance of user testing studies with the real end users of the product. It is concluded that findings should be supported with more expert’s evaluation and target user testing to provide insights for designers and developers and create design guidelines for more usable interfaces to address the needs and abilities of the fast-growing population.

As a future work, besides more expert’s evaluation, the design review of the interactive prototype will be applied to individual usability tests to assess its usability and to obtain more detailed users’ feedback. The same tasks in Heuristics will be applied to the target users to evaluate the relevance and acceptance of the smart home mobile application. Furthermore, the rest of the smart home system (i.e., Smart TV app and home hub tablet) besides smart phone appl will be designed which will follow the same guidelines for an age-friendly system design. This iterative process could improve the acceptance and usefulness of assistive systems among older adults.