Abstract
This paper presents fourteen useful guidelines to evaluate and determine how much does a digital interface in a public facility provide user experiences for citizens. In addition, it describes the partial outcomes of applying these guidelines to three urban interfaces in the city of Monterrey, Mexico. An ethnographic study was conducted to obtain preliminaries results where the user experience was assessed in these interfaces when applying the proper guidelines. The goal is to validate guidelines with global assessment methods to establish a protocol of design for urban interfaces; in other words, an application that provides the information required to design urban interfaces from point of view of UX. This extended ethnographic study will be implemented in different cities around the world and urban interfaces.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
- Digital interfaces
- User experience
- Smart cities
- Interaction design
- Citizen-centered design
- Design principles and usability test
1 Introduction
We live in an age of constant changes where information technology is reaching a great number of people. In this sense, the context of digital channels and technological devices use has evolved from robust and costly computers and specialized knowledge to a “massive” environment, where a person with scarcely any money or computer skills can use these devices.
Thus, daily activities are being systematized; people are using tools and computer systems aiming to improve their well-being. These automated activities take place while the cities grow, and governments establish technological infrastructure to improve their citizen’s well- being. This is how the transition from a “traditional” city to a “smart” city has occurred nowadays [6].
2 Smart Cities and Urban Interfaces
2.1 Smart Cities
A smart city is characterized by offering people different digital channels to interact with each other or with their local government. According to García [9], every time technology for residents or tourists is taken into account when the city is carefully designed, an environment of intelligent and sustainable growth is created and promoted.
2.2 Urban Interfaces
Since the 90s, with the commercial boom of the Internet, governments began to offer information and, subsequently, offered services and procedures to users through websites known as government web portals. This was the main and only digital channel of intelligent environment in the cities. Due to the increase of mobile devices, governments realized the importance of having a digital presence, ubiquitous and customized. In the last few years, other technological devices such as kiosks, sensors and ATMs allowed governments to increase their service-offer to people. Digital channels are suited to perform in urban environments such as public roads or special facilities that are part of an urban context design [9]. According to Mazhar [1] and Sakamoto [17], these channels must have a simple and easy digital interface which allows people to offer a faster and more efficient service.
Verhoeff [22] mentions that urban interfaces include an array of technologies, screen forms, formats, facilities, sensorial interfaces and architecture. Those technologies are located in public facilities; they are interactive and connect the user with his closest environment. They are important because they can be shared, are available to everyone, they are also intelligent and are crucial for the city’s social welfare.
The most common urban interfaces were not designed with a user-centered approach because it becomes an ordeal for the user to accomplish what he/she desires to do. Authors such as Sandnes [19], Prior [15], mention some aspects they bore in mind when they conducted a research on the train ticket-machines in Taiwan. Their main aspects are: the machine’s ergonomics and dimensions, vision angles on the screens and use of deflector materials, the interface’s location, data privacy when necessary, use of proper colors, iconography and significant signals, availability in different languages and structured menus.
In spite of these aspects, many existing urban interfaces are neither evaluated nor designed adequately due to a lack of knowledge about the process to validate them.
3 Usability Heuristics
3.1 Brief Review of Existing Heuristics
During the 90s, Nielsen suggested a series of usability heuristics to evaluate user-experience of digital interfaces [12,13]. These heuristics are a qualitative proposal based on the fact that the evaluator must execute an empathy process while using the system to simulate the different scenarios where the final users would be immersed.
As a result of Nielsen’s findings, other methods have been developed to evaluate the interface usability; some have had a more quantitative approach such as the Sirius method, published by Suárez, that takes into account a checklist with a set of items [20]. Other proposals such as Weinschenk’s psychological heuristics evaluate its usability from a perspective related to the user’s senses and reasoning when he/she uses an interface [18, 24].
However, all these proposals aren’t very accurate since the evaluator must simulate he/she is the final user. So, the real point of view remains unknown [15].
3.2 Usability Assessment of Urban Interfaces
Since the rise of the first digital channels which were provided by some governments, very little has been done to evaluate citizens who use digital interfaces. If there is some luck, the interfaces of the government´s web portals are evaluated by a quantitative heuristic method that only provides a general diagnosis showing how well-positioned is the portal in the user’s mind.
Aside from government web portals, there is no other practice to evaluate interfaces in other channels. This lack of assessment is the reason why many of the government’s technological initiatives fail, because they are focused on the achievement of a smarter city.
In addition, in the case of urban interfaces, many of these channels have been implemented under a technology-centered perspective or are limited to the government’s budget, rather than adopting a citizen-centered perspective [3, 4, 6, 14].
4 User Experience Guidelines to Design Urban Interfaces
4.1 Defining the Guidelines
The collected information from experts in usability, user- experience design and ergonomics was analyzed for heuristics development. The consulted authors and organizations were Nielsen [13], Weinschenk [24], Tognazzini [21], Connell [7], Russo and Boor [2], Rosenfeld [16], Web Accessibility Initiative (WAI) [23] and the Biomechanic Institute of Valencia [11].
The analysis included a heuristic revision of the consulted authors in order to assemble those that share similarities. This information was used to create a comparative table of heuristics [8].
Once the groups were identified, we created the assessment indicators giving a name and a representative description to each group. We ran a verifying process to be sure that all the revised and planned areas of the research were covered.
The final result was fourteen guidelines to be presented in the following paragraphs. The guidelines include aspects of usability, ergonomics, psychology, and accessibility among others. The suggested heuristics are the following: efficiency, assistance and direction, content-structure, environmental resemblance, context relevant information, mistake prevention, feedback, cognitive processes, internationalization, visual design, movement and perception, perception access, complementary and alternative channels [8].
4.2 Guidelines to Assess Urban Interfaces
We offer below, a detailed description of the guidelines required to judge the user-experience in an urban interface. Each guideline has its own description and criteria to enable a proper qualitative assessment.
Guideline 1. Efficiency. This guideline evaluates the extent in which a task is completed without deviations or delays. The goal is to determine whether the interface is flexible when adapted to a person’s different requirements. At the same time, we checked if the interface provides various options to perform the tasks depending on the devices where the action takes place.
The method to evaluate this guideline is based on the behavior that is observed in the following situations:
-
Does a person have the opportunity to skip a step to complete the task?
-
Is he/she able to complete the task without making a mistake?
-
Is he/she able to go one step backwards or all the way to the beginning?
-
Does a person obtain all the information from only one place?
-
Does a person capture the minimum amount of data?
Guideline 2. Assistance and direction. This guideline reviews the way in which the urban interface provides a rapid access to the information that solves all the possible doubts that a person may have.
To assess this guideline we must observe the following situations:
-
Does a person have to find information on how to perform a specific step?
-
Is there a way to contact someone in case a mistake is made?
-
Does he/she know exactly what to do to attain his/her goal?
Guideline 3. Content structure. This guideline measures whether the information is well organized in the interface. It also determines if browsing allows people to find what they are searching for.
The following situations are useful to determine the fulfillment of this guideline:
-
Once a person has completed the task, can he/she remember the required steps to complete it?
-
Does he/she make mistakes when he/she captures the information?
-
Does he/she understand what he/she will find based on the labeling?
-
Can a person access the contents in a simple and easy manner?
-
Is there a search engine tool?
-
Is there any ambiguity in the instructions?
-
Is the information grouped in such an order that anybody may comprehend how to complete the format and interpret the results?
-
Are there more ways to find the same information?
Guideline 4. Likeness with reality. To assess this guideline, we use metaphors that enable a person to identify familiar situations in a simple and natural way.
The following situation is useful to evaluate this guideline:
-
Did a person have previous knowledge of the elements included in the interface before he/she interacted with them?
Guideline 5. Relevant contextual information. It shows the necessary information for interaction. However, it is possible to present more data if required.
This guideline is accomplished by answering the following questions:
-
Does the interface show relevant information?
-
Does the interface constrain the user to memorize the previous steps?
-
Is the information clear and concise?
Guideline 6. Mistake prevention. This guideline determines whether the interface is designed to anticipate a person’s needs.
To define the extent in which this guideline is fulfilled, some questions must be analysed:
-
Does the interface allow to choose options that are not valid?
-
Did a person have to consult the same information more than once?
Guideline 7. Mistake retrieval. This guideline analyses if the interface allows you to correct mistakes and learn from them.
The following conditions must be reviewed:
-
Yes, there was a mistake. Was the person able to recognize what caused it?
-
Do error messages allow a person to know what went wrong?
-
Do error messages allow him/her to discern how to retrieve and correct the error?
Guideline 8. Feedback interface. In this case, it is important to know whether a person identifies what is taking place while the interface performs a specific action.
The following questions must be studied for the evaluation of this guideline:
-
Did he/she performed an action more than once or did he/she use an element of the interface that wasn´t required?
-
Does he/she know his/her progress percentage to complete the task?
-
Was it possible to find the desired option without making a mistake?
-
Is a person familiarized with the unresolved steps to complete the task?
-
Does the interface inform a person whether he/she is performing an action? For example: recording, consulting, information processing
Guideline 9. Cognitive processes. This guideline determines whether the interface was designed for human mental processes. It considers the required work flow and the steps that a person must follow.
The following questions must be revised to evaluate this guideline:
-
Does the interface give the impression of being user friendly? This includes not only the screen but also the hardware’s appearance.
-
Is the information difficult to remember? Does a person need to obtain these data from other sources?
-
Does a user find distractors that hinder the process to complete the task?
-
Does he/she feel safe when using the interface?
-
Does the user feel satisfied after completing the task?
Guideline 10. Internationalization. This guideline determines whether the information in the interface corresponds to the country´s cultural background and whether the information is available in different languages.
The following situations must be reviewed to determine the internationalization:
-
Is the content available in relevant languages?
-
Is the text translated properly?
-
Are there available options in the original language to change it to a foreign language?
-
Is it possible to choose the desired alphabet?
-
Are special characters (accents, ñ, among others) properly used in the text?
-
Are the formats adapted to the country? (date, currency, measure units, names, addresses, among others)
-
Does the interface use colors that correspond to the country´s cultural background and dismisses those that do not correspond?
Guideline 11. Visual design. This guideline revises the first visual appearance of the interface. In other words, it analyses whether the design is appealing, coherent and minimalist, contrasted, and space and hierarchy-designed.
The following information must be analysed to determine whether the interface includes this guideline:
-
Do the elements in the whole interface have the same visual design?
-
If there are many similar elements, can mental groups be created?
-
Is the same grid used in the whole interface and in the interfaces of other devices?
-
Can the color palette sustain contrasting colors?
-
Is a proper symbology used to distinguish the relation between sizes and objects.
Guideline 12. Movement access. This guideline determines whether the design was developed using a user-centered approach where capabilities, motion and physical limitations were taken into consideration.
The following information must be reviewed to evaluate this guideline:
-
Is little effort required to activate the keyboard or tactile screen?
-
Do the keyboard keys have the correct form, texture and space between them so fingers neither slip nor press two keys at the same time.
-
Is it possible to obtain a nearby wheelchair?
-
Does the machine’s height correspond to the standards issued by the country?
Guideline 13. Perceptive access. This guideline complements the above-guideline number 12. All possible users, with their abilities and sensorial limitations were considered as references during the design process.
The following questions must be analyzed to review and validate this guideline:
-
Is the option for the output-sound via headphones/earphones clearly visual?
-
Is the interface applicable to all people and tasks?
-
Does the interface show information in layouts that disabled people can identify?
-
Is there any way to customize a person’s preferences?
Guideline 14. Complementary and alternative channels. This final guideline shows whether there are other digital channels to complement the main channel, for example a mobile phone that accompanies the use of a kiosco, in order that a person may use different communication channels to interact with the interface.
The following questions must be studied to ensure the accomplishment of this guideline:
-
Can a person choose one or multiple outgoing information channels? For example, use the e-mail, receive a SMS with the ticket or recharge a card via his mobile device? Can he/she choose one or multiple incoming information channels? (for example, use a bill or printed ticket, use a bar-coded card, scan codes)
-
Is it possible to interact by using a mobile digital device?
-
In case that it is required, is it possible to pay using other payment procedures besides cash? (debit or credit cards, e-money)
5 Ethnographic Study to Validate the Suggested Guidelines
According to Hernandez Sampieri [10], the qualitative approach must be applied when the goal is to examine the way in which people perceive and experience nearby events and express their viewpoints, interpretations and meanings.
The ethnographic study mustn’t be descriptive only; it must make additional inquires by asking people further questions to obtain information about the meaning of things.
A research will be conducted based on a reliable theory. We will observe people’s behaviors when they use three similar urban interfaces in the cities of Monterrey and Mexico City, in Mexico.
The final goal is to calculate user-experience in each interface by applying the guidelines previously described.
As it was mentioned, we will analyze three different urban interfaces, which will result in three groups of people. They will be under observation in order to study their user-experience.
Each group will be identified by their common characteristics. To create a model of these groups, we will apply the personas technique to obtain archetypes, which are a fictional but concrete representation of the users- groups that will utilize the product. In other words, these archetypes are based on behavioral patterns recorded during the user-research; their goal is to serve as tools that enable communication during the product-design process [9].
5.1 Number of People to Observe
To determine the number of people that will be part of this research, we must bear in mind that a sample size is never established before collecting the information during the qualitative inquiring. A unit type of analysis has been set up. Thus, we shall obtain an approximate number of cases. But the final sample is only known when we reach the saturation point, where the new data no longer provides new or different information [10].
According to Nielsen [12], at least five users must be considered to get significant results from a usability test. Those five persons, will enable us to identify 85% of the problems in the evaluated system. The purpose of this research is to have eight users in each archetype so we can identify at least 95% of the problems of each interface.
5.2 Suggested Method
The field study includes the comments of each person. The researcher will participate as a full-time observer, taking photos and recording videos [10], followed by the application of an ASQ (After- Scenario Questionnaire). This type of questionnaire was developed by Lewis. The participant must answer questions after completing a scenario or task. The participant will select some sentences by choosing an answer with a grading scale from 1 to 7 (The lowest score would show the highest usability affinity with the system). It also reviews the Likert scale, which employs scales from 5 to 7 options and is mostly used to measure quality in questionnaires.
Open questions will be included in the questionnaire. The goal is to obtain information, showing the opinion, explanations and justifications of the participants [10].
To conclude, the questionnaire’s design will take into consideration the problem presented. We will try to determine the fulfillment of each guideline and evaluate the participants’ general experience. The questions must be designed in such a manner that they conduct the person through a pondering process which reflects his feelings toward the problem.
Finally, once the participant has finished the questionnaire, we will obtain a score showing the number of questions answered in order to get a “quantification” of the experience in each interface [5].
6 Future Projects
One of the most important goals of this research’s effort is to internationalize all the guidelines depicted in this document; it is one of the main challenges. Another test is the creation of an interactive tool that enables people to evaluate urban interfaces.
The urban interfaces will be assessed in Monterrey and Mexico City, in Mexico. Nonetheless, the guidelines are global assessment methods; in other words, they are applicable in different scenarios.
For this reason, in the short term, this ethnographic study can be implemented in other cities and urban interfaces. The purpose of this guide is to be put into operation in important cities of countries such as Canada, UK, Colombia and Spain.
Having obtained this information, we will compare and draw conclusions in other type of scenarios and people. We, then, will be able to create an assessment index of urban interfaces.
References
Abbas, M.: Challenges in implementation of TVM (ticket vending machine) in developing countries for mass transport system: a study of human behavior while interacting with ticket vending machine-TVM. In: Marcus, A. (ed.) DUXU 2014. LNCS, vol. 8519, pp. 245–254. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07635-5_24
Russo, P., Boor, S.: How fluent is your interface? Designing for international users. In: Proceedings of the INTERCHI ’93 on Human Factors in Computing Systems (INTERCHI 1993), Amsterdam, The Netherlands, 24–29 April 1993. IOS Press (1993). http://dl.acm.org/citation.cfm?id=164943. Accessed 12 July 2017
Martinez Ballesteros, L.G., Alvarez, O., Markendahl, J.: Quality of experience (QoE) in the smart cities context: an initial analysis. In: 2015 IEEE First International Smart Cities Conference (ISC2), pp. 1–7 (2015). https://doi.org/10.1109/ISC2.2015.7366222
Benouaret, K., Valliyur-Ramalingam, R., Charoy, F.: CrowdSC: building smart cities with large-scale citizen participation. IEEE Internet Comput. 17(6), 57–63 (2013). https://doi.org/10.1109/MIC.2013.88
de Castro, F., Silva, S.L., Barbará de Oliveira, S., Augusto, G.A.: Service desk software usability evaluation: the case of Brazilian National Cancer Institute. Proc. Comput. Sci. 100, 557–564 (2016). https://doi.org/10.1016/j.procs.2016.09.195
Concilio, G., Marsh, J., Molinari, F., Rizzo, F.: Human smart cities: a new vision for redesigning urban community and citizen’s life. In: Skulimowski, A.M.J., Kacprzyk, J. (eds.) Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions. AISC, vol. 364, pp. 269–278. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-19090-7_21
Connell, I.: Full Principles Set. http://www0.cs.ucl.ac.uk/staff/i.connell/DocsPDF/PrinciplesSet.pdf. Accessed 12 July 2017
Flores, A., Velázquez, A., Ramirez, E.: Definicion y aplicacion de un instrumento de evaluacion de experiencia de uso de interfaces digitales urbanas dirigidas a ciudadanos. Universidad de Monterrey, San Pedro Garza García (2017)
Garcia, R., Dacko, S.: Design thinking for sustainability. In: Design Thinking. Wiley, Hoboken, pp. 381–400 (2015). https://doi.org/10.1002/9781119154273.ch25
Hernández Sampieri, R., Fernández Collado, C., Baptista Lucio, P.: Metodología de la investigación. McGraw-Hill. https://books.google.com.mx/books/about/Metodología_de_la_investigación.html?id=wXMtSgAACAAJ&redir_esc=y. Accessed 12 July 2017
Insitituto de Biomecanica de Valencia.: Instituto de Biomecánica-ERGONOMÍA Y MUEBLE. Guía de recomendaciones para el diseño de mobiliario ergonómico (2017). http://www.ibv.org/publicaciones/catalogo-de-publicaciones/ergonomia-y-mueble-guia-de-recomendaciones-para-el-diseno-de-mobiliario-ergonomico. Accessed 12 Oct 2017
Nielsen, J.: Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems-CHI ’92, pp. 373–380 (1992). https://doi.org/10.1145/142750.142834
Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People-CHI ’90, pp. 249–256 (1990). https://doi.org/10.1145/97243.97281
Opromolla, A., Ingrosso, A., Volpi, V., Medaglia, C.M., Palatucci, M., Pazzola, M.: Gamification in a smart city context. An analysis and a proposal for its application in co-design processes. In: De Gloria, A. (ed.) GALA 2014. LNCS, vol. 9221, pp. 73–82. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22960-7_8
Prior, P.: Train Ticket Vending Machines: Designing for Usability (2016)
Rosenfeld, L.: Bloug: Information Architecture Heuristics (2004). http://louisrosenfeld.com/home/bloug_archive/000286.html. Accessed 12 Oct 2017
Sakamoto, M., Yoshii, A., Nakajima, T., Ikeuchi, K., Otsuka, T., Okada, K., Ishizawa, F., Kobayashi, A.: Human interaction issues in a digital-physical hybrid world. In: 2014 IEEE International Conference on Cyber-Physical Systems, Networks, and Applications, pp. 49–54 (2014). https://doi.org/10.1109/CPSNA.2014.17
Sanchez, J.: Psychological usability heuristics|UX magazine. In: UX Magazine (2011). http://uxmag.com/articles/psychological-usability-heuristics. Accessed 12 July 2017
Sandnes, F.: User interface design for public kiosks: an evaluation of the taiwan high speed rail ticket vending machine (PDF download available). J. Inf. Sci. Eng. pp. 307–321 (2010). https://www.researchgate.net/publication/220587882_User_Interface_Design_for_Public_Kiosks_An_Evaluation_of_the_Taiwan_High_Speed_Rail_Ticket_Vending_Machine. Accessed 12 Oct 2017
Suárez Torrente, M.C.: SIRIUS: Sistema de Evaluación de la Usabilidad Web Orientado al Usuario y basado en la Determinación de Tareas Críticas (2010). http://digibuo.uniovi.es/dspace/handle/10651/12866. Accessed 12 Oct 2017
Bruce, T.: Bruce Tognazzini's Tog on Interface—Developing User Interfaces for Microsoft Windows. http://flylib.com/books/en/2.847.1.19/1/. Accessed 12 Oct 2017
Verhoeff, N.: Urban interfaces: the cartographies of screen-based installations. Telev. N. Media 18(4), 305–319 (2017). https://doi.org/10.1177/1527476416667818
Web Accesibility Initiative.: WAI Guidelines and Techniques|Web Accessibility Initiative (WAI)|W3C (2015). https://www.w3.org/WAI/guid-tech. Accessed 12 Oct 2017
Weinschenk, S.: The Psychologist’s View of UX Design|UX Magazine. In: UX Magazine. https://uxmag.com/articles/the-psychologists-view-of-ux-design. Accessed 12 Oct 2017
Acknowledgments
We would like to thank Andrea Cortés, Alma Velázquez and Edgar Ramírez to their contribution to the heuristics, both in the revision of existing ones and the creation of new one, work done as undergraduates for their final project under the supervision of their thesis advisor, Marta Sylvia del Río at the Universidad de Monterrey.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Aceves Gutiérrez, L.C., Martín-Gutiérrez, J., del Rio Guerra, M.S. (2019). Guideline Definition for the Evaluation of Citizen Experience Using Urban Interfaces. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Theory, Methods and Tools. HCII 2019. Lecture Notes in Computer Science(), vol 11572. Springer, Cham. https://doi.org/10.1007/978-3-030-23560-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-23560-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23559-8
Online ISBN: 978-3-030-23560-4
eBook Packages: Computer ScienceComputer Science (R0)