1 Introduction

In recent years, there has been a growing concern in providing broad and democratic access to urban space. Through monitoring and analyzing citizen data, governments and companies have been planning on how to improve cities inhabitants’ life quality [1], as well the cities mobility design [2]. Consequently, multiple initiatives and technological aids aiming to support urban mobility frequently encourage the use of public transportation and environmentally sustainable vehicles. The users of such applications must be provided with a pleasurable experience and have their needs met when interacting with the application [3] as they commute, even when using a small screen or experiencing a limited internet connection. They need to feel encouraged to handle the applications while using private or public transportation in multiple contexts. Therefore, mobile applications to support outdoor mobility must not only work properly but also provide a positive User Experience (UX) and have a good User Interface (UI) design [4], which can be seen as the most important element of a computer-based system [5]. Such applications must be easy to use, flexible, have a simple and intuitive interface, maintain data integrity and provide an easy user adaptation, according to the usage context [6]. However, during the design process of a user interface, designers and practitioners frequently do not address the most common usability and experience issues, resulting in user’s frustration [5]. Redesigning the UI according to the users’ expectations and experience tends to positively affect the application usability [7]. Several approaches have been carried on to evaluate the usability of such applications and to assure they meet the user needs, focusing on user’s attention in the context of use [8], or in techniques performed by specialists [9]. Numerous studies focus on conceptualization, development and evaluation [4, 10, 11] of user interfaces in this context, however, the evaluation usually takes place in a laboratory, often ignoring the diverse aspects of the outdoor context of use that may affect the user experience.

In order to provide the users of such mobile applications with a positive experience and help them to engage into a pleasurable experience with the city outdoor mobility, we present a case study of the UI redesign of a bicycling mobility application based on the UX evaluation involving users in their real context of use. Our analysis is mainly based on observing the user behavior while interacting with the UI, which indicates the actual feedback of the users, and allows comparison of their experiences [4]. We address different techniques used for user interface evaluation based on a combination of methods in real context [11], whose outcome served as feedback to the redesign process, effectively covering user’s requirements. The redesign process was based on user-centered design [10] and prototype evaluation [12].

Our target application is the bike-sharing mobile application Bicicletar, which allows citizens to borrow public shared bicycles, in various Brazilian’s metropolises. The main contribution of this work is the analysis of how the aspects of UX with outdoor mobility applications may affect the design choices for the user interface. As a result, we propose the redesign of Bicicletar’s interface and discuss a set of design recommendations for outdoor mobility applications designers.

2 Methodology

2.1 Study Object

For this study, we chose an application called Bicicletar Footnote 1, which allows access to a public bike sharing system in Fortaleza, a Brazilian city with over 2 million inhabitants and with 216 km of cycling infrastructure. This application features include: registration, acquisition of usage credentials, a function to borrow bikes from a station, and the ability to locate stations on the map. We used the Android version of the app because it is the most common operating system in Brazil for mobile devices, with 92.4% of its market share [13]. In addition, in 2012, the bicycle was pointed as one of the most used vehicles for commuting in Brazil [14], which shows the relevance of this mean of transportation in the routine of the Brazilian people. In this research, we carried out a regional-focused evaluation in order to value local initiatives to support mobility. Furthermore, the results obtained will be forwarded to the responsible companies for implementation, so that the evaluation and redesign proposal can generate real improvements and benefits for the community.

2.2 Materials

The interface portion evaluated (Fig. 1) corresponds to the main functions of the application, which comprises six interfaces:

Fig. 1.
figure 1

Evaluated sections of Bicicletar application.

Full size image
  1. 1.

    acquisition of passes through the application;

  2. 2.

    location and details of stations;

  3. 3.

    information on the number of parking spaces and availability of bicycles;

  4. 4.

    choice and release of bicycles;

  5. 5.

    mailbox and

  6. 6.

    registration.

Four smartphone models were used during user testing and inspections and are listed in Table 1. To create both high-fidelity prototypes the POP 2.0 tool was used on Primary Redesign (web versionFootnote 2) and Final Redesign (mobile appFootnote 3). A digital camera was used to record the interactions during all user tests, in addition, OBS StudioFootnote 4 was used to capture the computer screen and webcam images while AZ Screen Recorder Footnote 5 app was used to capture the smartphones’ screen.

Table 1. Details of the devices used
Full size table

2.3 Stages of Research and Application of Methods

For the research planning, we followed the DECIDE [15] framework. The general objectives defined were: (i) analyze how the UX with outdoor mobility applications affect the design choices for the UI; (ii) propose the redesign of Bicicletar’s interface; and (iii) discuss the set of design recommendations for outdoor mobility application designers. Figure 2 summarizes the methods used and the results generated in each stage of the research. Four stages were defined to conduct the research, based on the objectives: (I) UX evaluation, (II) UX analysis, (III) application redesign and (IV) prototype validation, each one of those containing a step to consolidate the results based on the adequate qualitative or quantitative analysis of each set of data.

Fig. 2.
figure 2

Overview of the stages in this survey

Full size image
Fig. 3.
figure 3

Identified user profiles

Full size image
Fig. 4.
figure 4

Overall experience identified in the comparison of similar applications at Google Play

Full size image
Fig. 5.
figure 5

Distribution of responses to the checklist

Full size image
Fig. 6.
figure 6

Examples of problems identified in the Bicicletar inspection on laboratory

Full size image
Fig. 7.
figure 7

Dominance: Bicicletar vs. Primary Redesign vs. Final Redesign

Full size image
Fig. 8.
figure 8

Arousal: Bicicletar vs. Primary Redesign vs. Final Redesign

Full size image
Fig. 9.
figure 9

Pleasure: Bicicletar vs. Primary Redesign vs. Final Redesign

Full size image
Fig. 10.
figure 10

Overview of users’ opinions on general aspects

Full size image

UX Evaluation.

To get familiarized with apps that aim to facilitate urban mobility, we made a comparative analysis based on user data using 6 of the main apps that serve Brazilian metropolises. The following apps were used: Bike BH, Bike Brasilia, Bicicletar, Bike PoA, Pedala SP and Bicidade. The evaluators registered their perceptions on the apps conforming to Norman’s model [16] of interaction and the fundamentals of Information Architecture [17]. Still, in the same stage, we applied an online questionnaire to gather some data about the usage of Bicicletar. The evaluation of the application occurred in two complementary phases. We performed usability inspections of Bicicletar application in a laboratory context and user tests were carried out in the real context of use. It is important to emphasize that the usability evaluation was performed because it can contribute to the quality of the user experience indirectly, making the user feel easier and probably fulfilled [3]. The application’ inspection was based on the usability checklist proposed by [18].

The user test was planned based on the results obtained in the usability inspection and the online questionnaire, improved with items to investigate the elements of the user experience proposed by [19]. The user test counted on 10 participants, of which 5 attended Profile 1 and 5 that corresponded to Profile 2. For the accomplishment of the procedures, the trio of evaluators received different assignments (Fig. 11a).

Fig. 11.
figure 11

User tests in real and laboratory context of use

Full size image

During the execution of the tests, the users were led to a station of the Bicicletar system and the evaluators used contextual observation and Think Aloud [20] technique. The tests included performing specific tasks for the application such as getting a bike using the application and consulting a specific station and informing the number of bike spots available. The post-test questionnaire was based on Computer System Usability Questionnaire (CSUQ) [21] and Questionnaire for User Interface Satisfaction (QUIS) [22]. Finally, users responded to the Self-Assessment Manikin (SAM) [23, 24].

UX Analysis.

A Content Analysis technique [25] was used to interpret the results of the user tests. A quantitative analysis of the data generated by the questionnaires was also included. To prepare the data, audios and videos from pre and post-test interviews were organized to get an overview of application usage. In addition, the videos recorded during the tests were transcribed. Then content units were organized and specific codes were assigned to each unit. Finally, the units were categorized and analyzed.

Application Redesign.

In order to propose the redesign of Bicicletar interface, the UCD processes [10] were used. To produce the redesign of Bicicletar application the UI development [10] process was divided into two phases, conceptualization and prototyping. Two proposals of redesign were created for this work, a primary redesign and a final redesign. In both proposals, a UX evaluation was made. However, the evaluation of the primary redesign occurred in a laboratory (Fig. 11b) with 5 users [26], and the evaluation of the final redesign happened in real context of use (Figs. 11c, d) with 10 users.

Prototype Validation.

At the end of each stage, the collected data were interpreted, organized and cross-referenced to obtain answers to questions about how UI changes affected UX. The data collected in the questionnaires pre and post-test was quantitatively analyzed for the creation of comparative graphs, interview information and usage observations were analyzed qualitatively to be compared with the questionnaire data.

2.4 Ethical Issues

Four aspects are relevant to IHC research involving people: the need for informed consent of subjects, the preservation of their anonymity, the protection of vulnerable groups and the guarantee of the well-being of individuals [27]. Thus, participants in both methods voluntarily agreed to participate. Vulnerable groups such as minors and disabled people did not participate.

2.5 Limitations and Threats to Validity

The redesigns proposals of the interface were created based on User Centered Design processes [10] and tested with 15 users through a high-fidelity prototype, because many authors [28,29,30,31,32] have argued that the prototypes are primarily used for the communication, exploration, refinement, and evaluation of design ideas. During the tests with Bicicletar the performance was evaluated, but due to the use of prototypes to present the new UIs, this aspect was not considered on evaluation of redesign proposals.

As described, a Content Analysis was done, but due to the large volume of data generated - combined with the limited experience of the evaluators – the Unitarization and Categorization processes were difficult, which impacted on the execution time of the later stages, causing delays in the research schedule. For that reason, to consolidate the results of the redesign of the application only qualitative and quantitative analysis were done.

3 Results

3.1 Overview of Top Bicycling Mobility Applications in Brazil

In the first stage of the research, which sought to identify and compare the main services and applications focused on bicycling mobility, we obtained an overview, although superficial, of the applications of this type available to Brazilians. Some similarities were found between the applications, including the functions offered, such as the number of vacant spots and bicycles available and the display of stations per map, we also found similar problems, such as delays in the update of information and bad overall performance. In the opinion of users, bad experiences are common because of these problems, and their comments often associate those bad experiences with difficulties during use.

Figure 4 shows the comparison of the overall experience identified in the 6 applications. The results showed that the main problems in all applications are related to feedback, affordances, labeling, navigation, and organization, explaining several of the complaints from users on Google Play. Among those apps, Pedala SP was the most problematic. The review given by users on Google Play confirms the analysis’s findings.

3.2 Online Questionnaire

Through the data collected in the online questionnaire, we created user profiles, (Fig. 3), that identified problems that affect the user experience and outlined an overview of users’ perceptions of Bicicletar.

3.3 Usability Inspection of Bicicletar

With the inspections, we obtained two sets of results: the checklist results with the usability rate of each application specifying which heuristics were affected positively and negatively and a set of problems encountered while operating the interfaces.

To calculate the usability rate, the “yes” and “no” answers were counted and the non-applicable items were ignored. The percentage of positive responses constitutes this indicator. The resulting usability rate for Bicicletar was 57%, based on items with positive evaluation. Figure 5A represents the distribution of responses to the checklist, considering the 90 items that did not apply to Bicicletar. Figure 5B represents the distribution of 140 valid responses for analysis (i.e., disregarding non-applicable items).

The portion of the interface most affected was the stations section, with 19 of the 56 problems discovered. Many problems that affect the interface, in general, have also been detected, such as the location of the exit button and the misuse of colors, this group being the second largest concentration with 13 problems. Considering that a problem may affect more than one heuristic, the most affected ones were: (H8) aesthetic and minimalist design, (H7) flexibility and efficiency of use, (H3) user control and freedom and (H1) visibility of system status. Figure 6 exemplifies some of the violations identified.

3.4 User Tests

This section presents the results of user tests with 25 people (10 of current Bicicletar application, 5 of Primary Redesign and 10 of Final Redesign), and the comparative graphs of users’ opinions collected by SAM (Figs. 7, 8 and 9) and post test questionnaires (Fig. 10).

Bicicletar in Real Context of Use.

The sample of users of Bicicletar (Profiles 1 and 2) had 10 participants combined. 70% of the members of this group alleged they knew how to use all the functions of the app, 80% reported having their needs met by the application and 80% felt partially satisfied with it. The feeling of control divides the opinion of the participants: 40% of users rated it as partially low and the other users are divided between regular, partially high and high. A pattern was noticed in the users of the application: they feel satisfied even if they do not feel in control (Figs. 7 and 9). In the data from the analysis of the post-test questionnaire 40% of users stated that the application is partially confusing. Regarding the sequence of screens, 40% rated Bicicletar as partially clear, 10% as confusing and 10% as partially confusing. The information presented by Bicicletar on stations and use only has validity for the users if they are correct in terms of use. It was observed in the reports that the imprecision of this information frustrates the users and creates a feeling of insecurity. In addition, the break in the flow of navigation makes the interaction dull and confusing. Problems of communication and usability indicated that the conceptual model of the application does not correspond to the mental model of the users. Some users do not notice the presence of features in the application, such as (1) the time of use of shared bikes, (2) indication of the return of the bike at the station and (3) the use status of the pass. In all three cases, this happens because the information is barely visible and the changes that indicate the status update are not perceived.

Primary Redesign in Laboratory Context.

During the analysis of the data set of user tests with Bicicletar app, suggestions for improvement were identified by evaluators and users, which were met in the primary redesign proposal (Fig. 12). This evaluation was conducted in the laboratory and through a computer-based [11] high-fidelity prototype was evaluated by the users. In it, 60% of users had a partial high feeling of control, 80% presented partial satisfaction with the application and 40% affirmed to feel partially exciting during the tests. 60% of the users pointed having a high feeling of control, and their sense of satisfaction was equally divided between partial and fully satisfied.

Fig. 12.
figure 12

Primary redesign’s screens

Full size image

Final Redesign in Real Context of Use.

In general, the UI changes in the final redesign (Fig. 13) highlighted a very positive UX in this evaluation stage, for 80% of users the application became partially easy to use and presented a partial clear screen sequence as well as partially organized.

Fig. 13.
figure 13

Final Redesign’ screens

Full size image

4 Discussion

4.1 User Experience: Bicicletar vs. Primary Redesign vs. Final Redesign

To evaluate the adequacy of the functionalities and the interaction problems, 6 objectives were defined. Each of these objectives is discussed below based on the results obtained with the triangulation of data collected throughout the research.

Do the emotional aspects inherent in the use point to a positive and enjoyable experience?

On Bicicletar.

Partially. In general, users were satisfied with the application. However, feelings of frustration, confusion, fatigue and insecurity permeate the use of the application. Errors and difficulties in interaction made users feel incapable, ashamed or technically unprepared. Frustration is not just application-driven, as users take the blame for the flaws and are disappointed in themselves.

On Primary Redesign.

Partially. The level of user satisfaction remained constant, this first version of redesign obtained a positive evaluation in the aspect of the dominance of the SAM questionnaire. It was evident the need to change in the conceptual model of purchase of a new pass in the application, because in the Primary Redesign only aesthetic aspects (UI) were changed, the conceptual model and sequence of screens of the bicycle application were preserved.

On Final Redesign.

Yes. All users were satisfied, an explicit fact of 100% of positive ratings on the pleasure, arousal and dominance items of the SAM questionnaire. During the interactions, 15 reactions or indications of positive feeling were scored, as were the expressions or attitudes that demonstrate satisfaction of expectation.

Are the users able to properly detect the main features of the application?

On Bicicletar.

No. The lack of clarity in information architecture complicates user perception. Users had some difficulty in locating certain stations, but those difficulties were overcome with a closer examination of the interface.

On Primary Redesign.

Partially. There were problems in detecting the cycling display feature in the map, 40% of users believe that bike lane view functionality does not exist on the map section, while it does. In the primary redesign, this functionality was not easily identified and raised questions during interactions.

On Final Redesign.

Yes. In the quantitative analysis of the observations of use were counted 107 occurrences of executing an activity purposely, without making a mistake. Users reacted positively while executing the main features of the application. It was noticed that the perception of the users was benefited by the changes of the interface.

Are the users able to perform the main functions of the application?

On Bicicletar.

Yes. After exploring interface elements and functions. Users reported problems with: application crashes, updating real-time information, interacting with the map and organizing the list of stations. It was also noticed that the start of the borrowing process of the bicycles does not match their expectations. In addition, the lack of information or their conflict about sections and their apparent disabled status confuse users and limit their possibilities of interaction.

On Primary Redesign.

Yes. The effort required to understand the new elements of the redesign proposal was not painful. The problems of low usability were corrected, however, due to the limitations of the prototype, the performance could not be evaluated.

On Final Redesign.

Yes. Users have demonstrated high ability to perform the main functions, indicated by the 107 occurrences of running an activity purposely without making a mistake. Aspects evaluated in the SAM and post test questionnaires as control feeling, satisfaction, ease of use, compensation and organization of information had 100% of Positive ratings. There were 18 occurrences of comparison with previous experience, which indicates that users were familiar with the elements of the interface and that there is a good relationship between the conceptual model and the mental model.

Do the users properly understand the interface elements and the application concept?

On Bicicletar.

Partially. In general, users understand the elements of the interface and understand the concept of the application as a whole, however, the concept of passes is not well understood, which makes it difficult to perform the function of borrowing a bicycle, one of the main functions. The lack of integration between Bilhete Único/Student Identity and the application contributes to the existence of this gap. In the post test, 40% of users said the application is partially confusing.

On Primary Redesign.

Partially. The elements of the interface were understood by most users, as perceived in the post test questionnaire, in which 80% of users claim that the application is partially stimulant and to 60% of users organized. To allow clear communication, a brief description of the operation and rules of use were added to the interface element representing the Pass.

On Final Redesign.

Yes. In the selection section of the bike, details about the available accessories were added through visual elements in the interface, and these were also perceived during the interactions. The application presented 100% of positive ratings in the post test questionnaire, in which users claim to feel compensated. Organization of information and ease of use also presented 100% of positive ratings.

Do the users’ needs match the features available in the application?

On Bicicletar.

No. Users have some needs that are not met by the application, the main ones are updating the information, needs relating to helping section, status of bicycles and stations, notifications, display of bike paths on the map and search for stations by address. In SAM, 80% of users say they are partially satisfied with the application, and this is the one with the highest positive ratings. Of the users who feel partially satisfied, 50% rated the application as partially rewarding.

On Primary Redesign.

Yes. User needs are met, features reported as missing were made available, such as the help section, search option in the stations section, and the display of cycle paths. In the post test questionnaire, 80% presented partial satisfaction with the application. The compensation sentiment received 100% of positive ratings. 80% of users claim that the application is fully rewarding and 20% that the application is totally rewarding.

On Final Redesign.

Yes. As in Primary Redesign, the final version redesign met the users’ needs, as they stated partial or total satisfaction. For 40% of users the application is partially rewarding and 60% that the application is fully rewarding.

Can social and environmental aspects influence the experience of using the application?

On Bicicletar.

Yes. The use of mobile devices to borrow bicycles must happen at stations, a public environment, which gives users a sense of insecurity and sets up a usage situation that requires speed and convenience to perform operations. The application is expected to be easy to use, simplify access to important information, provide quick responses and clear feedback, and instantly update the necessary data.

On Primary Redesign.

Yes. The use of a larger screen of laptop during the tests and the absence of adverse conditions such as constant noise, distractions and a sense of insecurity allowed users to better understand the interface elements. Due to the good environmental conditions, such as the pleasant temperature and the use of chairs in the laboratory tests, the users felt more stimulated to express opinions and suggestions of improvement.

On Final Redesign.

Yes. In addition to the social aspects, it has been noticed that ambient lighting conditions (e.g. sunlight) can also affect the visibility of the interface elements, especially in devices that have screen protectors, because depending on the material (e.g. glass). This accessory may negatively influence the user experience. However, the use of high contrast colors reduced the negative impact of both factors.

4.2 User Interface Components Used on Redesigns Proposals

The use of the green color and bicycle icon was preserved in redesign proposals to keep the user interface associated with other components of Bicicletar, like stations and bicycles, that also have the same characteristics. Colors, typography and elements (Fig. 14) used on redesigns proposals are based on Google’s Material Design guidelines for Android applications to ensure the consistency and patterns with the Operating System, like floating action button to main functions, subheaders to group similar information on lists; fixed tabs to enable content organization at a high level, such as switching between views, toolbar, above app content highlighting important features and section name and the side navigation to allocate application’s sections, enabling quick navigation between unrelated views and reducing visibility of infrequent destinations.

Fig. 14.
figure 14

Examples of components and patterns of material design used on redesign proposals (Color figure online)

Full size image

4.3 Design Recommendations for Outdoor Mobility Applications Designers

Users need to have information reliable and constantly updated, presented in a concise and organized way, since the context of use and the nature of the application requires speed and practicality. The sequence of screens should be logical and simple, the use of wayfinding on the top of the screen is recommended. Interrupting the flow of navigation by changing unnecessary screens and organizing information in many sublayers makes it difficult to use, making it an arduous process that requires concentration, which is not recommended for outdoor mobility applications. This type of application should not present a conceptual model that differs from the mental model of users, so that users do not feel incapable or confused. This problem can be avoided by developing a clear organizational structure. Uniformity in User Interface is a good way to ensure that your design communicates effectively without confusing or overwhelming the users. The confirmation or error messages should be short and easy to interpret, keeping in mind that there is difficulty in capturing user perception due to the context of use, as external environments, moving, divided attention, and how users use the application, as brief consultations and rapid implementation of actions. The main functions of the application should be easily found and in some cases, allow more than one way of execution. Using contrast as a tool to draw the user’s attention is a good way to help the user understand the relationships between functions and elements. The frequency of use of the application should be considered in the design process, because if users do not perform long and continuous use experiences, the learning curve is compromised. The frequency of use is related to ease of understanding, learning and use.

5 Conclusion

The Bicicletar app benefitted the people who depend on the public transportation system of Fortaleza. Due to the quality and variety of the experiential aspects explained by this research, result of the association of different methods, we noted that users recognize the usefulness and relevance of the provided information, and for this reason, they act with a certain tolerance towards the problems that negatively affect their experience with the application. The identified problems are the same as other applications from several states in Brazil, as initially identified in comparative research step, which makes this research more relevant. Changes in the UI positively affected the UX. Improvements in the sequence of screens, organization of information and visual elements have generated an increase in users’ feelings of pleasure, dominance and security. The Public Transportation System in large metropolises is complex and linked to problems that affect the user’s commute. Designers of outdoor mobility applications need to consider situations of stress, discomfort and insecurity that are part of the users’ routine. The design process of these applications should be based on the understanding of mental model and context of use, meeting user’s needs, with an UI that provide easy of use and a pleasurable experience. Due to the good receptivity of UI redesign, our next steps will be to implement and test with more users.