Keywords

1 Introduction

Users of digital systems and products such as websites and applications (apps) frequently experience a variety of emotions, both negative and positive, while interacting with apps and websites. The affective aspects of such interactions may not be effectively captured during studies of usability, which is defined as “the extent to which a product, service or environment can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” [1] User experience (UX) is a concept that goes beyond this largely system-oriented paradigm of “effectiveness, efficiency, satisfaction,” to take into account a number of user-oriented factors such as likability, pleasure, comfort and trust [2]. User experience differs from ‘experience’ in a general sense, in that it explicitly refers to the experience(s) derived from users encountering products, services or artefacts through a user interface [3]. The notion of user experience has different meanings for different professionals, with a variety of definitions across different domains [4].

UX design, as a field, is concerned with the design of enjoyable websites and apps for users [5]. There are tools used for evaluating UX during usability studies involving users but these do not consider users with disabilities. Although accessibility studies involving users with disabilities do mostly evaluate usability, they have not progressed to the stage of evaluating the emotional experiences of users with disabilities for the purpose designing websites and apps that deliver more enjoyable UX for them. The few studies that took into consideration the emotional experiences of users who are blind, or visually impaired, have primarily revealed these users’ frustration regarding the inaccessibility of the websites and apps [6, 7]. An examination of multiple studies describing UX evaluation revealed that these studies were not conducted with users with disabilities.

There are circa 50 tools available to help designers and developers measure and evaluate UX [8]. There is no published process or tool, which specifically describes, or caters to, UX evaluation for users with disabilities. In particular, for users who are blind or visually impaired, who have a distinct disadvantage because much of the content and interaction on websites and apps is visual. Designing for improving the user experience for all users, including blind and low vision users, is essential to make websites and applications more inclusive. A community of UX researchers is aiming to push for greater inclusion in design by developing a Manifesto for Accessible User Experience (AUX) where they urge for a focus of accessibility efforts on delivering quality user experiences [9]. AUX concerns the integration of accessibility into UX design and practice to create “genuinely inclusive high-quality digital experiences for everyone, regardless of disability or age.” [10] At the moment, there is insufficient research or dialogue on this topic.

To help mitigate the lack of focus, both academic and professional, on this subject, this study attempts to systematically understand the experiences of users who are blind or visually impaired and the ways in which technology mediates their digital user experience. These users rely on a screen readerFootnote 1 to interact with their digital devices. The primary research question examined by this study is: How can a tool be designed for measuring the accessible user experience (AUX) of users who are blind when interacting with websites and applications? An exploratory approach was employed since there is paucity of published information concerning this area. Borrowing methods from the usability and UX domains, interviews were conducted with six screen reader users employing a think-aloud protocol and hands-on activities using their laptop and phone. The data gathered were analyzed to identify the components of an AUX framework that could form the basis for building an accessible evaluation tool.

The objective of this study is to initiate the inclusive designFootnote 2 of a tool and document the process for evaluating the AUX of screen reader usersFootnote 3. The tool is intended for UX designers to use during usability sessions with screen reader users. To further support UX designers, the process of conducting an AUX session with screen reader users has been documented in detail [11].

2 Theoretical Framework

2.1 User Experience

UX is a multidisciplinary concept. Hassendahl, Sheldon, Mehrabian, Russell, Porat, Tractinsky, Forlizzi, Battarbee and Dewey have made key theoretical contributions to UX design. Of these, the ideas of Hassenzahl [12] and Bevan [2] have been selected because of their simplicity and precision to construct a lens for analysis. Their contributions are listed below:

Hassenzahl (2010)

  • Autonomy: Feeling that you are the cause of your own actions rather than feeling that external forces or pressure are the cause of your action.

  • Competence: Feeling that you are very capable and effective in your actions rather than feeling incompetent or ineffective.

  • Relatedness: Feeling that you have regular intimate contact with people who care about you rather than feeling lonely and uncared for.

  • Popularity: Feeling that you are liked, respected, and have influence over others rather than feeling like a person whose advice or opinion nobody is interested in.

  • Stimulation: Feeling that you get plenty of enjoyment and pleasure rather than feeling bored and under stimulated by life.

  • Security: Feeling safe and in control of your life rather than feeling uncertain and threatened by your circumstances.

Bevan (2008) [ 2 ]

  • Likability: the extent to which the user is satisfied with their perceived achievement of pragmatic goals, including acceptable perceived results of use and consequences of use.

  • Pleasure: the extent to which the user is satisfied with their perceived achievement of hedonic goals of stimulation, identification and evocation and associated emotional responses.

  • Comfort: the extent to which the user is satisfied with physical comfort.

  • Trust: the extent to which the user is satisfied that the product will behave as intended.

2.2 Non-visual Access

The term screen reader is generic for a system that reads out the contents of the display on a computer or mobile device. There are several screen reader programs available, both commercially and in the form of free, open source software. Some popular commercial screen readers used for accessing the computer/Internet are JAWS, Window-Eyes, HAL, and VoiceOver. Free, open-source screen readers such as NVDA and Orca, as well as low cost products such as System Access, are also available. However, JAWS is reportedly the most popular screen reader in use in North America [13].

There are many studies about barriers to information access when using a screen reader and factors of interface design that facilitate accessibility [14–16]. But there is very little empirical research about what emotions these factors produce in screen reader users. In particular as regards investigating features that produce positive emotions and enhancing the design for a more positive experience. This is a potential area for research.

2.3 Positive Psychology

Positive psychology involves the study of the human pursuit of individual happiness as something central to human life. Until two decades ago, psychology was confined, primarily, to the study of mending an injured or unhappy mind. But, around the turn of the century, Seligman and Csikszentmihalyi [17] started a new paradigm of study termed positive psychology or the “knowledge of what makes life worth living.” Since then, happiness has been a subject of many empirical studies [18–21].

Lyubomirsky stated that the pursuit of happiness requires the acquisition of positive experiences on a day-to-day basis with a more general assessment of life as positive and meaningful and that a good part of happiness depends on activities and is, thus, variable. This idea gave rise to the concept of “designing for happiness” in the design domains to provide a positive experience. A natural complement to this idea was the concept of evaluation of users to determine what makes them happy and how happy they become. The theory of positive UX for users with disabilities is a focus this study has drawn from positive psychology.

3 Methodology

Data for the research was primarily derived from interviews with six blind adults living in Toronto (four men and two women aged between 35 and 60 years). The technologies they used to access websites and apps were comparable. All of them primarily used a laptop with Windows Operating System, Firefox browser and JAWS screen reader to interact with websites and web-based apps. All of them used an iPhone to interact with websites and apps. Yet, their profiles showed diversity along occupation, information-seeking interests, nature of use of websites/apps and proficiency in the use of computer/phone, screen reader and websites/apps.

The interviews included semi-structured questions pertaining to the three main areas based on the critical incident technique [22] with abundant hands-on demonstration by participants to substantiate their responses. The three areas were:

Incident(s) where user felt a negative emotion such as frustration or anger while using a website or app.

Incident(s) where user experienced a positive emotion such as happiness or enjoyment while using a website or app.

User’s experiences of interacting with a website or app using the phone (with touchscreen and Voiceover) vs. the laptop (with keyboard and JAWS).

The interviews lasted around 90 min, affording adequate discussion about each of the areas noted. Each participant used their own laptop and iPhone to demonstrate their answers and, additionally, to share other related experiences. Data triangulation was achieved through audio recording of the sessions (including think-aloud protocols), video recording of the laptop/phone screen, and observational notes. An initial framework for AUX was derived through a literature survey and environmental scan. This framework comprised likability, pleasure and comfort. An inductive-deductive content analysis of the transcribed data was done using the above framework to arrive at the final AUX framework for the study (described in the next section), which will be strengthened through further inclusive design iterations. Personas were created based on the data of each of the six interviewees to illustrate their diversity as an indicator of the validity of the findings and to enhance communication during discussion of the research outcomes.

4 Findings

4.1 The Framework

Data from the user sessions were analyzed using the lens derived from secondary research based on the concept proposed by Hassenzhal and Bevan. This study resulted in a five-point framework (see Fig. 1) for Accessible User Experience consisting of the following components:

Comfort, Likability, Autonomy, Agency and Pleasure. The framework, acronymized as CLAAP, signifies the positive feeling associated with clapping of hands. Proponents of the science of positive psychology state that positive emotions serve as markers of flourishing, or optimal wellbeing [6]. Adopting this premise, the five components above are identified with, and derived through, the study of points of positive experience for the users. The first author proposes the term ‘pleasure points’ for these points, in contrast to the search for ‘pain points’ in conventional usability studies.

Based on the five components of the CLAAP framework, an AUX evaluation tool was developed in the form of a questionnaire as given below:

Fig. 1.
figure 1

Diagram of the CLAAP framework

Full size image

Imagine that you have just used the {name of website or app}. With that experience in mind, please answer the following questions:

Q1: Does the website or app exceed your expectations? If so, how? Please describe in detail.

A1:

Q2: What parts of the website or app engaged you the most?

A2:

Q3: Did you find the website or app intuitive to use? If so, describe how?

A3:

Q4: Do you feel that you are able to accomplish all the tasks on the website or app independently?

A4:

Q5: Could you describe your navigation experience on the website or app?

A5:

The tool was sent to all the participants and refined based on the responses and feedback obtained from them. Responses to the tool were sought from all the participants with a reference to “Twitter” because all of them had mentioned that website/app on their favourites list. A detailed protocol for conducting UX studies with participants who are blind was drafted based on the experiences during the interviews. The outcome of this study is thus a process and tool for AUX evaluation with users who are blind.

5 Discussion

This paper presents an exploration of the emotional experiences of persons who are blind when they use websites and apps on their computers and smart phones and develops both an empirically sound process and tool for evaluating the accessible user experience (AUX) aspects of such interactions. Some significant outcomes from the study are an AUX evaluation tool and an analysis of impact of technology on interaction and pleasure points. They are discussed in the following subsections.

5.1 AUX Evaluation Tool

The evaluation tool developed through this study is in the form of an electronic questionnaire that is accessible via screen readers. It contains five questions along the dimensions of Comfort, Likeability, Autonomy, Agency and Pleasure (CLAAP.) The dimensions were arrived at qualitatively through analysis of data gathered from six screen reader users. The tool is merely a first attempt at developing a useful AUX resource for UX designers to consider pleasurable user experiences when they work with users who have disabilities.

The evaluation tool that was created with the CLAAP framework is merely a first attempt at developing a useful AUX resource for user experience designers hoping to work with users who have disabilities. The disruptive effect that was hoped to come from this tool is in attempting to influence individuals within both the UX and AUX field to consider the pleasurable user experience for people who are disabled. Those within the accessibility movement have considered user experience for disabled users but have only done so through focusing on the negative feelings associated with pain points and user frustration. The AUX movement is still nascent, and as it has been established in the literature, thus far, very little work has been done within the conventional user experience field to consider disabled users in terms of actually developing evaluation methods to determine pleasurable user experience. The goal of the AUX evaluation tool is to contribute to the AUX field by drawing attention to this disparity and striving to provide a solution that could either inspire some designers and AUX practitioners to consider pleasurable user experience for disabled users within the field and to give other AUX practitioners a tool to assist in usability testing.

The tool was effective in capturing key insights. However, based on responses and feedback obtained from the interviewees, the following points were noted for future refinement:

  • Justification for selecting the website or app used for usability testing with the participants should be made known to them.

  • If comparison across participants is desired, then specifications must be made about the product to be used (website or app) for the service (such as Twitter) whose usability is being tested. This is because responses about experiences were found to vary across products (website or app)

  • Question about agency should be clarified further by adding components relating to adaptation and personalization of sites and apps as they are both concrete examples of user-control to change their computing environment to match their personal preferences.

  • Ways of separating pleasurable feedback from positive feedback should be documented.

  • The current long answer question format makes it difficult for cross measurement across participants. The inclusion of Likert scale questions that are accessible to screen readers could be helpful to better track measurements across participants as well to help with quantitative data analysis.

5.2 Pleasure Points

Drawing from positive psychology, this study introduced a term ‘pleasure points’ in contrast to the traditional conversation about ‘pain points’ that prevail in user testing. Components of a design that evoke positive emotional reactions from users are termed as pleasure points. Two original contributions of this research are: (1) a process to evaluate pleasure points while testing websites and apps with screen reader users and (2) a tool to facilitate the evaluation of pleasure points.

It must be emphasized that the AUX tool was not designed to solely look at pleasure points. Rather, the tool is designed to counterbalance the overemphasis on pain points in UX research. Since this tool evaluates UX and not positive UX, the responses that are resultant from the tool will vary based on the mindset of the user both during and after using the website or app.

5.3 Impact of Technology on Interaction

One of the major insight that came from observations made during the user sessions was that the technological configurations used by participants played a big role in the interaction of screen reader users with websites and apps. Furthermore, it was also seen that the mode of interaction between the technology and the user also contributed to their UX. This provided a useful insight that UX is not just about how a website or app is designed but also how it is caused by the interaction itself. The user’s electronic devices such as computer, browser, screen reader and input-output devices/mechanisms, all contribute to their user experience.

Participants were comfortable using both Windows laptop with Jaws and iPhone with Voiceover. Those with greater proficiency seemed to enjoy the experience more. Some participants’ enjoyed the experience on their phone more than on the laptop. With audio from the screen reader being common, it appeared that the touchscreen interaction on the phone (versus keyboard on laptop) made it more enjoyable. For the same product, some participants had very different experiences vising website or app on laptop or phone.

An important technology-related emotional experience that was observed in the study was the variance of feelings elicited between engaging in an identical activity on a laptop relative to a phone. This divide is evidenced by through response of the participants the user experience evaluation tool. The tool itself asked the participants for feedback on 5 key metrics of user experience. The site chosen to evaluate upon was twitter and participants had the option of preforming their evaluations on the website or app or in combination.

When the same individuals assessed the site for the same metric (expectations) the results were different. Karan noted that: “the iPhone app is very accessible and easy to use. Although I was pleasantly surprised to find it accessible and I enjoy using it.” In contrast, the same user when assessing the website mentioned: “The website is OK to use, but I rather use the iPhone app because I find it easier to use in general.” In both instances Karan was preforming the same activities, navigating the site to view content. Yet these observations were not unique. Fatima when asked if they could navigate the website independently described the user experience as “…cumbersome as it takes too much time and effort to perform simple tasks.” In contrast, Jose had overwhelming positive experience with the app noting that it had exceeded his expectations as it was: “very accessible, tweets can be sent and navigation happens easily. It should be noted that the comments about the improved user experience for mobile devices revolved less around the screen reader and more focused on the interaction model as evidenced by Jose’s describing why he prefers the iPhone to his computer: “…because you can get information with shorter journeys and less keystrokes.”

6 Conclusion

With respect to accessing and enjoying digital content, screen reader users today still face many challenges. The web accessibility community is actively working on reducing barriers faced by individuals with disabilities in accessing websites and apps. A core tenet of UX is that all users on the spectrum of human diversity deserve to have a pleasurable experience when using digital products. However, this is not followed in practice. In this context, the contributions of the study, as well as the limitations and plans for future research, are outlined below

6.1 Contributions

This study is intended to reinforce the concepts promulgated in the manifesto of AUX, namely to examine accessibility through the lens of user experience. The study also contributes to the three dimensions of Inclusive Design as identified by the Inclusive Design Research Centre [23].

First, it seeks to recognize diversity and uniqueness in individuals by examining the needs of extreme users on the spectrum of individual abilities––in this case screen reader users who are blind––who are commonly ignored as an insignificant set of outliers when considering UX in designing products or solutions for individuals.

Secondly, the study offers an inclusive process and tool for evaluating AUX, thus making these easily usable by UX designers for providing a positive experience to all users with varied abilities. Ultimately, the goal of every UX designer is to provide a pleasurable user experience, and people with disabilities should not be left behind.

Thirdly, the study promises a broader beneficial impact because the AUX evaluation tool designed to meet the needs of extreme users also has the ability to benefit users with other disabilities, as well as the general population. Even though the tool was designed though interviews with screen reader users, its components could help many other user groups who require accessibility support, such as those with hearing loss, those with low vision and seniors. Moreover, because the tool can be an effective barometer of people’s emotional states when using websites or apps, the tool can be used with the general population as well. Thus, a curb cut effect exists. This study thereby contributes to Inclusive Design.

6.2 Limitations and Future Work

This paper is based on a master’s research project that was completed within a limited time frame. Hence, this study on AUX evaluation was conducted with a small number of users. By replicating the process with screen reader users with a variety of profiles, interests and proficiency levels, the set of evaluation criteria could be expanded; and the process could be refined.

The AUX evaluation tool developed through this study was based only on interaction with users having vision impairment. To expand the tool and to develop comparable processes for AUX evaluation for users with dexterity/cognition/hearing constraints, similar studies ought to be conducted with these users.