1 Introduction

At present we are surrounded by devices, the most common are smartphones, tablets, and laptops. However, more and more devices are being added to create a truly interactive environment: sensors, cameras, microphones, smart watches, and screens that can be found in the most unexpected places. Today the number of devices connected to the Internet surpasses 7 billion [1]. The omnipresence of these devices, especially mobile ones, is progressively changing the way people perceive, experience, and interact with products and each other [2]. This transition poses a significant challenge, since we have to design User eXperiences (UX) according to each device.

Within this context, a topic of particular interest is when a single application can be executed on multiple devices. The ability to seamlessly connect multiple devices of varying screen size and capabilities has always been an integral part of the vision for distributed UX and Ubiquitous Computing [3]. That is, the same application has to provide a similar UX regardless of the device or its environment. One way to preserve UX is to have a consistent application.

Consistency states that presentation and prompts should share as much as possible common features and refer to a common task, including using the same terminology across different inputs and outputs [4]. Several studies have shown that consistency is a crucial factor for multi-device experience, but they have also argued that it is a challenge for developers, since maintaining consistency of a multi-device system is an open problem [5,6,7,8]. Consistency is important because it reduces the learning curve and helps eliminate confusion, in addition to reducing production costs [9,10,11].

Microsoft is an excellent case to exemplify the importance of consistency. Windows 10 and Office (in its most recent versions) are two of the most important products of the company; It is notorious that both GUIs are a design statement since they follow the same layout. In both software products, we can see that their toolbars have a similar design, i.e., the grouping, positioning, and labelling of buttons and commands is identical. This is intended to allow users to focus on their productivity, without the need to learn a new tool panel for each software they use. For this reason, Microsoft developed a series of tools, including an API and design guidelines that are integrated into a framework called Ribbon [12], so that this design discourse propagates to all applications developed by third parties.

In this way, we can talk about three approaches to the design of applications which, although authors like Coutaz and Calvary [13] and Vanderdonckt [14] studied years ago, Levin [2] summarises them in her 3C framework:

  • Consistent design approach: Each device acts as a solo player, creating the entire experience on its own.

  • Continuous design approach: Multiple devices handle different pieces sequentially, driving the user toward a common goal.

  • Complementary design approach: Multiple devices play together as an ensemble to create the experience.

This framework presents a series of challenges since it involves, among other things, the fragmentation of the GUI and business logic. Thus the task for the developers is to preserve a positive UX among all the devices.

By adding consistency elements to the design of multi-device environments, usability is improved, and the possibility of a scenario with negative UX is reduced [15, 16]. The primary goal of our work is to propose a set of design guidelines that serve as a model in the creation of consistent applications. These guidelines are depicted through a case study: DistroPaint. This application has been evaluated by five UX experts, who have identified a list of consistency violations and have assessed the severity of each one.

This paper is organised as follows. First, in Sect. 2, related work is studied. Section 3 describes the research methodology that we use. Then, in Sects. 4 and 5, we respectively define and implement our set of design guidelines. Finally, in Sect. 6 we discuss the achieved work and provide some ideas for future work.

2 Related Work

This section describes some of the investigations carried out in the field of multi-device UX. They serve to support the importance and necessity of our proposal.

After having interviewed 29 professionals in the area of interactive environments, Dong et al. [3] identified three key challenges that have prevented designers and developers from building usable multi-device systems: (1) the difficulty in designing interactions between devices, (2) the complexity of adapting GUIs to different platform standards, and (3) the lack of tools and methods for testing multi-device UX.

Marcus [17] was a pioneer in the description of good practices to develop GUIs. He claims that the organisation, economisation, and communication principles help GUI design. The highlights are his four elements of consistency: (1) internal (applying the same rules for all elements within the GUI), (2) external (following existing conventions), (3) real-world (following real-world experience), and (4) no-consistency (when to deviating from the norm).

Meskens et al. [18] presented a set of techniques to design and manage GUIs for multiple devices integrated into Jelly, a single multi-device GUI design environment. Jelly allows designers to copy widgets from one device design canvas to another, while preserving the consistency of their content across devices using linked editing.

O’Leary et al. [19] argue that designers of multi-device UX need tools to better address situated contexts of use, early in their design process through ideation and reflection. To address this need, they created and tested a reusable design kit that contains scenarios, cards, and a framework for understanding tradeoffs of multi-device innovations in realistic contexts of use.

Woodrow [20] defines and contextualises three critical concepts for usability in multi-device systems: (1) composition (distribution of functionality), (2) consistency (what elements should be consistent across which aspects), and (3) continuity (a clear indication of switching interactions). The author makes a call for more active involvement by both the systems engineering and engineering management communities in advancing methods and approaches for interusability (interactions spanning multiple devices with different capabilities).

All these works show that the highly interactive environments formed by multi-device applications are a promising field with many issues to explore. However, they are also examples of a knowledge gap that we try to fill with our guidelines for consistency.

3 Research Methodology

The research methodology for the development of our design guidelines is based on the Design Science Research Methodology (DSRM) process model proposed by Peffers et al. [21] (see Fig. 1). We chose this methodology because its popularity in the state of the art, and it has proved useful in similar problems [22,23,24].

Fig. 1.
figure 1

We started from an Objective-Centred Initiation (coloured in orange) in the DSRM process model [21]. (Color figure online)

Full size image

An Objective-Centred Initiation has been chosen as a research entry point because our goal is to improve the design of multi-device applications. As for Identification & Motivation, we have already described the importance of highly interactive environments, and the role of GUI consistency in that matter. The Objective of a Solution, the second step of the process, is to develop a set of design guidelines that helps developers to create consistent applications to improve UX. The third step Design & Development is the description of our guidelines for GUI consistency (see Sect. 4). Demonstration and Evaluation are described in our case study (see Sect. 5). This is the first iteration of the process. Subsequent iterations will begin in the Design & Development stage, in order to improve said guidelines.

4 Consistency Guidelines

With the review of various works in the state of the art, and taking into account the challenges discovered and common characteristics of each one, we present our five design guidelines to maintain consistency in multi-device systems:

  • Honesty: Interaction widgets have to do what they say and behave expectedly. An honest GUI has the purpose of reinforcing the user’s decision to use the system. When the widgets are confusing, misleading, or even suspicious, users’ confidence will begin to wane.

  • Functional Cores: These are indivisible sets of widgets. The elements that constitute a Functional Core form a semantic field, out of their field they lose meaning. The granularity level of interaction for a Functional Core depends on the utility of a particular set of widgets.

  • Multimodality: Capability of multi-device systems to use different means of interaction whenever the execution context changes. In general, it is desirable that regardless of the input and output modalities, the user can achieve the same result.

  • Usability Limitations: When multimodality scenarios exist, it is possible that situations of limited usability could be reached. When the interaction environment changes and its context is transformed, the environment can restrict the user’s interaction with the system.

  • Traceability: Denotes the situation in which users can observe and, in some cases, modify the evolution of the GUI over time.

5 Case Study: DistroPaint

In order to demonstrate the proposed consistency guidelines (see Sect. 4), we developed DistroPaint, a prototype application that integrates them. We decide to create a basic graphics editor, which provides several tools that can be distributed on several devices (PC, phone, and tablet). This section describes our proof of concept (see Sect. 5.1) and the expert analysis carried out (see Sect. 5.2) based on the works by Andrade et al. [25], Grice et al. [26], and Schmettow et al. [27].

5.1 DistroPaint

DistroPaint is a Web application for basic graphic design. The user can access the application from a PC, a phone, and a tablet. They can distribute the GUI from the PC to the mobile devices, e.g., the colour pallet can be displayed on the phone, while the drawing tools are being shown on the tablet (see Fig. 2). The user can configure the GUI at any moment. Below we list how our design guidelines are reflected in the implementation of DistroPaint:

Fig. 2.
figure 2

Predominant GUIs of DistroPaint on a PC web browser: (a) GUI of the graphical editor, and (b) the distribution menu for the widgets

Full size image
Fig. 3.
figure 3

Presence system: (a) a grey box means that the device is unreachable; (b) an orange box indicates that the device is connected but it can not receive widgets; and (c) a green box expresses that the device is ready to receive widgets (Color figure online)

Full size image

  • Honesty: The part where DistroPaint’s honesty stands out most is its presence system (see Fig. 3), since it informs the user about the availability of their devices. The Honesty at this point is critical, because it allows the user to make decisions (to distribute, or not) according to the state of their interactive environment.

  • Functional Cores: The main way of interaction in our application is the toolbox (see Fig. 4), so we choose it as the main element for the DUI. The decision of how to divide the elements could seem trivial, e.g., each tool (brush, eraser, line, etc.) could be distributed individually among several devices, however, this could be a risky option, since it would bring very few benefits to the cost of generating confusion and increasing the system requirements. So we decide that the tools and the slider for the stroke thickness should form a semantic field. In the same way, another field would be occupied by the colour palette, thus, we have two Functional Cores as result.

  • Multimodality: The element for the change of context that has more repercussion in our application is the change of platform. No matter whether a user uses one element of the toolbox from the PC (by clicking with a mouse) or from a mobile device (by touching with a finger), DistroPaint has to respond seamlessly (see Fig. 5).

  • Usability Limitations: We create a synthetic limitation in our prototype (see Fig. 6). We decide that both of our Functional Cores have to be available for both the phone and the tablet, but only the tablet can display both at the same time. Although this can also be achievable for the phone, we want to demonstrate that despite the capabilities of the devices (in this case, the difference in screen sizes), it is desirable to offer alternatives, so users can accomplish theirs tasks in one way or another.

  • Traceability: Besides the already explained presence system, DistroPaint also gives feedback to the users about where the widgets are being distributed and also maintains synchronised all the values for all the widgets from the toolbox, no matter from where or when the user changes such values (see Fig. 7).

Fig. 4.
figure 4

Functional Cores division for the toolbox: (a) tools core, (b) colours core, and their respective mobile formats (a’) and (b’)

Full size image
Fig. 5.
figure 5

DistroPaint allows interaction through: (a) a mouse, and (b) with a finger; with both modalities the user can obtain the same result

Full size image
Fig. 6.
figure 6

Functional Cores can be seen: (a) one at a time on the phone; (b) both of them at the same time on the tablet. The reason to do this is that the tablet has a bigger screen, thus, it can display more widgets

Full size image
Fig. 7.
figure 7

(a) As part of the presence system, the user knows where the widgets are. When the user makes a change in a widget, the system automatically reflects such a change in all the GUIs, e.g., tool, stroke thickness, and colour are synchronised between: (b) the PC and (c) the tablet

Full size image
Fig. 8.
figure 8

Guidelines violations in DistroPaint

Full size image

5.2 Evaluation and Results

The evaluation has been worked out with the help of five UX experts. We chose the experts for their experience applying usability tests, and because they are familiar with the topics of our research. All the experts are university professors and have postgraduate studies; two of them belong to our university. Their experience comes from both work in industry and research centres. It should be noted that none is related to this work in addition to their participation in the evaluation.

Before starting the evaluation, we gathered and explained to the experts each of our design guidelines, their purpose, and discussed some examples so that everyone had a similar starting point. Each expert drafted a list of problems and violations of the guidelines that we propose. Once the evaluators have identified potential consistency problems, the individual lists have been consolidated into a single master list. The master list was then given back to the evaluators who independently have assessed the severity of each violation. The ratings from the individual evaluators are then averaged, and we present the results in Table 1. For the rating, we adapted the severity classification proposed by Zhang et al. [28]:

  • 0 - Not a consistency problem at all.

  • 1 - Cosmetic problem only. No need to be fixed unless extra time is available.

  • 2 - Minor consistency problem. Fixing this, should be given a low priority.

  • 3 - Major consistency problem. Important to fix, should be given a high priority.

  • 4 - Consistency catastrophe. Imperative to fix this before the product can be released.

Evaluators found a total of 23 usability problems using our guidelines (a mean of 4.6 problems per evaluator). The severity rating of problems had an average of 2.42. For the master list, a total of 10 problems were evaluated and guidelines were violated 18 times (see Fig. 8). Honesty and Traceability were the two most frequently violated guidelines, 6 and 4 times, respectively. In contrast, the guideline with less detected problems was Functional Cores with 2 violations.

Fig. 9.
figure 9

Severity rating of consistency problems found in DistroPaint

Full size image

With respect to the severity of the problems detected, we can see in Fig. 9 that severity level 3 - “Major consistency problem” was the most frequent with 36%, closely followed by severity level 2 - “Minor consistency problem” with 24% of occurrence. On the contrary, we can notice that the lowest classification 0 - “Not a consistency problem at all” got 8%.

Table 1. Consistency problems and its rating in DistroPaint
Full size table

In general, we can say that DistroPaint has many aspects in which to improve because several problems with severe qualifications were identified. Nevertheless, the evaluation was fruitful, as various problems could be discussed, as well as scenarios that, if neglected, could cause conflicts in the future. So our guidelines were advantageous in identifying particular conflicts in this specific case.

That Honesty was the guideline with the highest number of violations is an exciting aspect. Perhaps improving those weaknesses of design, the violation of the other guidelines disappears, or its qualification is reduced because Honesty brings with it a better workflow and a more solid GUI.

The experts concurred that the design guidelines could be a useful tool to detect consistency problems. However, they also acknowledged that in order to be more effective, they have to be refined and detailed.

6 Conclusions and Future Work

The main contribution of this paper is a series of consistency guidelines for the design of multi-device applications. This kind of applications represent the challenge of configuring the available resources and their role in the environment. When the users control the application, it allows them to explore their environment, identify the tasks and services compatible with it, and combine independent resources in a significant manner, in order to perform tasks and interact with services. Consistency is the element that maintains the users in a stable base since it is the key to assist GUI distribution. Besides, it is an essential factor in maintaining a positive UX.

We chose this type of evaluation to be able to explore our proposal in a deep way. We knew that by working with experts in the area, we could get feedback on our work, benefit from their experience, and directly observe how other people use our design guidelines.

While it is true that an expert evaluation can give good results, it is not exempt from problems. For example, we recognise that we have few points of view since we only have the participation of five experts, which can lead to misleading results. However, we considered that this was the best way to carry out an exploratory study, since with a small group we can have more in-depth discussions and work for a longer time. In this sense, the results seem promising, because we realised how the experts interpreted the guidelines. In general, our expectations were fulfilled, but we also know that we have to refine and be more specific so that each guideline is not too ambiguous.

It is also possible that expert evaluators can identify many consistency problems in multi-device applications without relying on our guidelines. However, using them provides evaluators with a structure that helps them take into account each major design dimension in turn, and to prevent them from becoming distracted by other design aspects, which could cause them to miss essential consistency problems.

As the nature of the guidelines is empirical, to prove their validity it is necessary to perform experiments, such as heuristic evaluations and UX tests with end-users.

The biggest challenge for future work involves improving DistroPaint; thus we could perform tests with end users. We plan to create an alternative version of DistroPaint that does not follow our design guidelines, so we would have two versions that users can use, compare and evaluate. In this way, we could see the effect of our design guidelines directly. In this way, we could compare the number of problems found in each case. Also, we have the intention of enriching our work through revisions of GUI design patterns [29]. Finally, we expect that our consistency guidelines to continue evolving in the future as we gain more experience and insights from using them to evaluate other applications, such as those found in the Internet of Things domain.