Keywords

1 Introduction

According to the statistics of the last Census carried out in 2010 by IBGE [1], 45 million people declared to have at least one of the deficiencies investigated. At the time, the declared visual, motor, hearing and intellectual deficiencies corresponded to 23.9% of the Brazilian population. Visual impairment is the most representative and affects 18.8% of Brazilians, physical disability is the second most frequent and summarizes 7.0% of the population, followed by hearing (5.1%) and intellectual (1.4%).

Considering that people with disabilities represents the largest minority in the world and that the definition of laws that aim to promote accessibility and inclusion, as Law No. 13,136/15, Law No. 10.436/02, Law No. 12.319/10, etc. it is a reality, it is necessary to ensure for this portion of the population rights such as adequate access to services, products, information, etc. In addition, to include them in society in its various segments (educational, cultural, health, etc.), allowing them to participate actively, with as few restrictions as possible, is a necessity.

However, historically, people with disabilities (sensory, physical, cognitive, etc.) were left out of society [2], with the mistaken conclusion, even in the Middle Ages, that deaf people were not “educable” [3]. Although advances are noticeable in the way society has been dealing with the desire for inclusion, the educational gap in this portion of the population is still significant. Also according to the 2010 Census [1], the low level of education and training of people over 15 and who declared themselves to be disabled, corresponds to 61.1% of the population (have no education or have only the incomplete elementary school). On the other hand, this percentage is 38.2% for people without disabilities, in the same age group.

When the focus is directed to people with hearing impairment (deaf) it is necessary to understand that the absence of phonetic perception (sounds) hinders the process of literacy in the Portuguese language [4]. Therefore, the teaching of Libras (Brazilian Sign Language), regulated by law No. 10,436/02, makes this the mother tongue of most deaf people. When the deaf has Libras as their first language, there is a difficulty in understanding the text written in Portuguese, as the mental process for interpreting these two languages is different [2].

Therefore, when the presentation of information is primarily textual, in Portuguese, the difficulty of understanding the content, on the part of the deaf, is evident. As alternatives to minimize this problem, Assistive Technology resources have been developed to allow the translation between Portuguese and Libras, like the ProDeafFootnote 1, HandTalkFootnote 2, FaLibrasFootnote 3, VLibrasFootnote 4, etc. applications.

However, when considering the need to assess the usability of these AT resources with its target audience (deaf people), difficulties are faced by the evaluation team with regard to data collection. Usually the interaction observation sessions are supported by the application of questionnaires to obtain information about the participants’ profile, their level of subjective satisfaction and their emotional state when interacting with the product under evaluation [5, 6]. The questionnaires adopted have, for the most part, presentation of primarily textual information (questions and answer options). As examples, it is possible to mention: POCUS (Objective and Cognitive User Profile) [9]; SUMI (Software Usability Measurement Inventory) [7], SUS (System Usability Scale) [8], GEW (Geneva Emotion Wheel) [5].

In order to make usability assessments also inclusive, in which means In order to make the usability assessments of AT resources intended for deaf users inclusive, it is necessary that test participants can answer the questionnaires applied autonomously. Therefore, the questionnaires that are commonly adopted in this context of evaluation need to be adapted so that the representation of information (questions and answers) prioritizes Libras. This article describes, therefore, the planning, conduct and results of a set of Participatory Design workshops [10] conducted at the Fundação Centro Integrado de Apoio ao Portador de Deficiência (FUNAD), with the participation of two Libras interpreters and 20 deaf people, literate or in the process of literacy in Libras. The results of the workshops will serve as a basis for the collaborative design of TUTAForm (Assistive Technology Usability Test - Form). This resource consists of an online, multimedia form and adapted for deaf respondents who communicate in Libras. With TUTAForm, the aim is to allow deaf respondents to be autonomous during the data collection performed. Via form, in usability tests of targeted assistive technology resources and this audience of users. From TUTAForm it is possible to obtain data on the profile of respondents, the survey of their subjective satisfaction and their emotional state when interacting with the resource under evaluation.

This article is organized into six sections, including this one. Section 2 presents some proposals for adapting questionnaires in order to make them accessible to the public of respondents with some types of disability. Section 3 details the methodology adopted for planning, conducting and verifying the results of the Participatory Design workshops conducted within the scope of this research. In Sect. 4 there is a description of the Participatory Design workshops, including the decisions made on the creation of TUTAForm. In Sect. 5 the results of the DP workshops held with a group of potential users are presented. Finally, in Sect. 6, the conclusions and potential for expanding this work is presented.

2 Related Works

There are a number of instruments that can be used to support data collection when assessing the usability of products and systems. These are mostly self-reported (questionnaires, interviews, checklist, etc.) and seek to highlight the perception of participating users about the product being evaluated. These instruments favor the measurement of subjective satisfaction levels, or emotional state and their intensities. Questionnaires are still adopted that allow to outline the profile of the participating users.

Prietch and Filgueiras [11] carried out a mapping of data collection instruments used in usability assessments to investigate their suitability for deaf respondents. The authors highlighted, after analysis that the gap in adequate instruments for respondents with hearing impairment is evident. Since they rely on predominantly textual information and the use of value scales (Table 1). In both cases, understanding is not evident for deaf respondents, literate in Libras and illiterate in Portuguese. Although the authors have identified some initiatives that use multimedia resources, not exclusively textual ones, to represent the content of the questionnaires. None of them was developed targeting the specific audience of deaf respondents.

Table 1. Analysis of allaboutux.org UX methods.
Full size table

However, in a literature review, it was possible to identify two propositions of questionnaires adapted for deaf respondents: SignQUOTE [12] and Emotion-Libras [11].

SignQUOTE [12] is a tool to support remote usability assessment data collection. This aims to maintain the advantages of remote tests (scheduling flexibility and reduced cost) and enable the participation of deaf users. For this, all information and instructions available in the tool (from the consent form to the post-test questionnaire) are presented in American Sign Language (ASL). The questions and answers available on SignQUOTE consist of videos previously recorded by interpreters with knowledge of the sign language. Considering the particularities of each assessment and the need for specific questions, the configuration of SignQUOTE is recurrent. Adopting this strategy for the production of ASL content can be costly considering the cost of interpreter participation. Furthermore, the adaptation of this tool to be used with Libras is not evident.

Emotion-Libras [11] is a questionnaire to be used in the context of evaluating the user’s experience in the face of interaction with a specific interactive product. Its objective is to allow the deaf user to express his emotional state without the help of third parties, therefore, in a more independent way. The participant can indicate, in addition to the emotions felt, the respective intensities. At Emotion-Libras, emotions and intensities are presented from videos previously recorded with interpreters. The fact that the tool does not have an Open Source license limits its configuration, restricting the amount of emotions covered in the questionnaire. In addition, for the participants’ profile and subjective satisfaction data (commonly collected during the assessments of this nature) the deaf respondent continues to depend on third parties.

In view of the scarcity of adapted questionnaires for deaf respondents. When developers of AT resources need to evaluate their solutions with real users, they prepare adapted questionnaires, in an ad hoc manner. This strategy was adopted for the evaluation of CineLibras [4] and VLibras [13]. On both occasions, an interpreter participated to translate the questions to be recorded on video and hosted on an online form creation platform. The problem of adapting the collection instruments used in an ad hoc manner (without the participation of users who will be real respondents) is that it often leads to doubt when questionnaires are applied, and interpreter interference is still necessary, implying expenses additional. According to the Translators’ Union, hiring an interpreter costs around R$ 300.00 to R$ 3,000.00 depending on the context and the hours required.

As the elaboration of research contents carried out in an ad hoc manner also requires an interpreter to translate the contents in real-time or in previously recorded video, workshops for Participatory Design were proposed for the elicitation of the system requirements. The construction of a tool through DP workshops results in a product with the lowest possible failure rate [10].

The decisions taken to meet the requirements mentioned and the preparation of the workshops are described in the following section with the methodological steps adopted.

3 Methodology

As can be seen in the analysis of Table 1, the existing assessment methods do not include the deaf as the assessor and have no or little visual illustration, presenting most of their content still in written language. The few evaluation methods that present the possibility of using visual aids, have limitations that make it impossible to use them in the context of the proposition. To this end, the methodology of the work is divided into two stages: bibliographic survey and its data collection instruments, where an analysis of the existing data collection tools is made for possible adaptation; analysis of similar works in which a study of the existing data collection tools is carried out in order to analyze the possible improvements to apply to the proposed tool; analysis of questionnaires to survey the content, in which the analysis of contents in common between the existing questionnaires already applied is done; elaboration and execution of participatory design workshops, where the complete script of the experience is elaborated, and describes the results obtained in them.

Usability test is and what user information needs to be recorded for analysis [6, 15]. It was possible to conclude that for a good usability test, three types of information should be obtained and evaluated. They are: User Profile Characteristics, Satisfaction Test and Emotional State.

Therefore, the two proposals mentioned above are not adequate to support the data collection in Assistive Technology Usability Tests. Whose users are deaf, Brazilian, literate or in LIBRAS literacy process since neither meets the set of requirements listed below:

  1. 1.

    To be for LIBRAS, given that the target audience is deaf Brazilians literacy (or in process of) in LIBRAS;

  2. 2.

    Do not depend on the active participation of interpreters to generate the questionnaire content or its application, aiming at reducing the cost of production and the autonomy of the respondents;

  3. 3.

    Not based on primarily textual representation. However, multimedia, based on the use of images, animated gifs and videos made with an avatar in order to facilitate the understanding of the content covered;

  4. 4.

    Be made available online, allowing respondents to use the service simultaneously and not in person (if necessary);

  5. 5.

    Contemplate items related to the respondents’ profile, their level of subjective satisfaction and emotional state after interaction with the product under evaluation.

A practice that can be used to collect, analyze and project information in this process is participatory design, which covers the participation of all those interested in this process as developers, users, employees and partners [10]. The role of all those involved in the process (organizing, implementing and executing an inclusive environment) is to perceive differences, listen to the different ones and establish involvement strategies [16]. That in turn enable each one to influence the organization’s destinies, building the sense of belonging.

4 Participatory Design Workshop as an Involvement Strategy in Adapting Communication

In this stage, a survey of the number of tools was carried out from the point of view of usability that have an adaptive interface. Aiming to evaluate the existing tools and realize the possibility of taking advantage of some types of result already obtained. Subsequently, an evaluation of the usability testing methods was also carried out from the perspective of the needs of the deaf. Verifying whether the tools have more visual content than textual, in order to be able to use some devices and adapt them to the context of the proposal.

4.1 Choice of Items for the Form

As a guide for the definition of the elements that would compose the analysis section of the user profile that will use TUTAForm, the profile adopted by POCUs - Objective and Cognitive User Profile [9], was chosen, which originated from the profiles [4, 14].

In addition to the model adopted by the POCUs, others profiles was also considered [13, 14]. Since, it was noticed that some specific elements related to disability contemplated in the study should be added and that some adaptations should be made to the context. Some works were applied in tests with deaf people [13, 14].

On the other hand, the set of information requested in the POCUs contains 76 items and obtaining all this information by the user would end up being an overly tiring task, in addition to those that would be added due to the context. Thus, the inverse filtering was performed, considering in the first instance the information presented in the models [13, 14] and analyzed which ones were in the POCUs and which ones are not and are relevant to the context, without making it very large section. Due to having other questions that should be analyzed and it would not be interesting to leave the user tired from the beginning of the questionnaire.

A comparative analysis was carried out and it was noticed that some information was presented in the same way in the three profiles. Others had to be adapted, so that they were common to the analyzed profiles.

Based on this filtering, it was concluded that the user information that would be relevant to the context of the proposed form are:

  • Genre;

  • Age;

  • Deafness level;

  • Educational stage;

  • Level of understanding of Portuguese;

  • Level of understanding in LIBRAS;

  • Frequency of smart phone use;

  • Frequency of computer use.

For each of the items analyzed, some items were collected from the literature that could illustrate each of them in a self-explanatory way, either through a static or mobile figure, video, typing and also in written text.

For the items of satisfaction and usability, among the instruments analyzed, the SUMI - Software Usability Measurement Inventory, a questionnaire of 50 items of the likert type, seemed to be very complete and suitable for the context, it is online, it can be applied in systems under development: functional or market prototype and can be applied to web services. On the other hand, as it consists of a vast amount of questions, it would tire the user a lot.

Another instrument that appeared in the form of a questionnaire and seemed to fit the context of TUTAForm, was the SUS - System Usability Scale, a 10-item questionnaire, which, in addition to being quick, can be online and applied to any product.

In comparative studies, most scores on usability questionnaires are highly correlated somewhere. This means that on most items, they actually measure the same thing. Where it can be several points can be evaluated. For example, what users feel about the efficiency of a software, what they will say it is if they liked it or not. Thus, as the two types of questionnaires presented seem to fit in the context. A comparative analysis of the two was carried out in order to arrive at a set of items in common in both.

Some items are directly linked to the context, others had to have their structure adapted, in the form of a direct question and without presenting bipolar information. The emotional state in the usability analysis, the existence of bipolar emotions in opposite ends caused confusion in understanding the relationship between them [11]. And the same can be received when composing a questionnaire with bipolar reactions, where there may be results of an inverse scale.

With the result of this analysis and adaptation, the following set of items was reached:

  • Would you use this system more often?

  • Did you find the system easy to use?

  • Would you recommend the system to your friends?

  • Did you need help from someone to use the system?

  • Were you able to understand all the functions of the system?

  • Did you think that the system had many errors?

  • Did you find it easy to learn how to use the system?

  • Did you find the system organized?

  • Did you feel confident using the system?

  • Did you need to learn new things to use the system?

As for the type of response of the items, SUMI and SUS are based on scale. The SUMI scale consists of three options: Agree-Undecided-Disagree. The SUS scale, for each item, the user must evaluate on a scale of one to five how much he agrees with the statement, being 1 - Totally Disagree and 5 - Totally Agree. Among these, it was chosen to keep SUMI’s, with three options. Since in the same study by [16] it was noticed that the deaf is confused when there is a lot of choice between “Yes” and “No”, since for them there are no intermediate options.

When the answer options were presented only with the numbers five to one, there was a lack of understanding about their meaning [11]. And as a possible solution one could use illustrations indicating the meaning of the answers. It was also considered to add illustrations to the answers, whether they are textual, static and mobile figurative, typing or video.

For the items that would make up the emotional state section, from the instruments listed on allaboutux.org, none of them had adaptation for the sign language, some had images or animations and users need to check answer options informing about the emotions felt. Only the Brazilian tool Emotion-Libras provides support for the deaf to understand, taking into account that it was developed considering the deaf as their target audience.

However, for the creation of videos expressing emotions, a person is required, whether an interpreter or a deaf person. And the idea is that TUTAForm is automatic and does not depend on third parties to compose the content.

For the generation of the videos, a translation application was used. Due to experience with the tool, the application chosen was VLibras. And the emotions assessed are only positive [11], bipolar emotions represented in the same scenario, caused confusion in understanding. The emotions assessed are:

  • Cheered up;

  • Confident;

  • Scared;

  • Interested;

  • Pleased.

4.2 Workshop Planning

Before giving the workshops, there was a planning stage. And it was taking into consideration: (i) the infrastructure of the place where the workshops would be; (ii) the availability of an interpreter to accompany the sessions; (iii) the organization of the workshop; (iv) preparation of the material to be evaluated; (v) script; and, (vi) the role of the evaluator and other helpers and the evaluation method.

At the end of the planning stage, the place where the workshops would be held was defined, at FUNAD. And how the evaluation team would be composed:

  1. 1.

    An evaluator, with no experience in Libras, responsible for running the workshop in a way that followed the previously prepared script. In addition, he was responsible for distributing and collecting the questionnaires in due course during the tests.

  2. 2.

    A Libras interpreter, responsible for passing on the evaluator’s information to the participants.

  3. 3.

    An observer, responsible for recording the personal information of the deaf participants. Where such as the interpreter, the role is making notes on the behavior of the deaf and everything else that she found relevant for further analysis. He was also responsible for photographing the workshops during the experiment.

The sessions were divided into two days and subdivided into four sessions.

4.3 Conducting the Workshops

The execution of the workshop was divided into three phases and will be exhibited in just one day, they are:

  1. 1.

    Presentation of the different ways to display a question;

  2. 2.

    Representation of possible responses for collecting user profile data;

  3. 3.

    Scale representation, which aims to measure the degree of satisfaction of usability questions.

The sessions were divided into two days and had the participation of 20 deaf employees, who proposed to analyze the material voluntarily. Each of them lasts 15–30 min, depending on the number of participants in each one. All had the same infrastructure: a projector, a computer and the answer sheets that were given to the participants.

The workshop was conducted as follows:

  1. 1.

    Delivery of the answer book to the participants, where each page of the answer book has three until six options of items. These items v vary according to the objective of the question, in which he would have to choose among them which he liked best.

  2. 2.

    Previous explanation of its purpose. The options were designed and the deaf was instructed to choose the option that pleased him most among those displayed. Or maybe to suggest one that was not present in the material.

  3. 3.

    Display of each set of options presented on the slide. In a class board was presented all the available options, just like the answer book. Each option contains a corresponding color, in order to facilitate the interpretation of participants who do not have knowledge in Portuguese and keep data consistent.

The participants were between 14 and 54 years old and their educational level ranged from the 5th year of elementary school to complete higher education.

5 Results

The final result of the workshops together with the result of the interviews was analyzed. The conclusion was that to present the questions the video with an interpreter would be ideal, in the opinion of the deaf. In contrast, the pending of having an interpreter in the construction of the questionnaire is not a valid option. Due the fact the scope of the proposal is to grant greater independence while the research participants answer it. Without the need for assistance from third parties, besides, there would be no freedom to add and edit the questions and there would be additional expenses with the interpreter.

Dactylology was chosen as the best or one of the best options to represent several items of the questionnaire. However, this option was disregarded, based on the report of the teachers who participated in the workshops. In these cases, the second most voted option was chosen to compose the questionnaire.

Resulting in the following representations:

  • Personal profile:

    • Genre: Illustration similar to bathroom gender plates;

    • Age: Numerical illustration;

    • Deafness Level: Video using the VLibras automatic translator;

    • Education: Video using the VLibras automatic translator;

    • Yes or No options: Like/Dislike illustration;

  • Subjective satisfaction:

    • Agree-Undecided-Disagree option: illustration by emoji for agreement, neutrality and disagreement.

  • Emotional state:

    • Video with avatar translated by VLibras converted into gif to represent emotions (excited, confident, surprised, interested and satisfied).

6 Conclusion

After the approval of Law 13.146/15, there were many change by the public authorities in general and public service concessionaires. Wherever it became mandatory to provide institutionalized ways to support the use and diffusion of Libras as an objective means of communication.

Thus, the production of products, equipment, devices, resources, methodologies, strategies, practices and services that aim to promote functionality, related to the activity and participation of people with disabilities or reduced mobility have become more frequent.

Such products need to be evaluated by consumers. However, none of the data collection instruments is presented in a way that considers the deaf to be a potential user. They usually have content in Portuguese with little or no visual illustration. Without aiming at their autonomy, independence, quality of life and social inclusion.

Therefore, it was perceived the need to create a technological solution that enables the autonomous participation of deaf people in Usability Tests of Assistive Technology resources. Usability tests which they are potential users, regarding the filling of questionnaires/forms usually adopted. Participatory Design workshops were adopted as a strategy for obtaining system requirements. Since by involving people who can use the system later, the system can be more accepted by end users and consequently more accessible and usable.

The proposal of TUTAForm is to allow the deaf to access a data collection instrument whose information is in their natural language of communication (Libras) and with representations that they understand at first (video, images, animations), avoiding the use of Portuguese terms. And that your design is fully collaborative.

Finally, this work presented the first phase of creation of TUTAForm, which is the definition of the media items that will compose it. The media representations of the questions and answers were obtained through the Participatory Design workshops and counted with the collaboration of 20 literate deaf people or in the process of literacy in Libras and two Libras teachers. The workshops were held at FUNAD and were subdivided into four sessions.

The results of the workshops were analyzed and a set of media was reached. Which, in the opinion of the participants, would be the best representations to illustrate the topics presented. Such results are the basis for consolidating the design of the tool.

As a perspective for future work, we intend to apply the results obtained in the DP workshops in the creation of TUTAForm. Then that it can support the data collection in usability tests by deaf respondents.