Keywords

1 Introduction

Online travel agencies generate online communities. Travelers share their opinions, comment on their experiences, and quantitatively evaluate services. Their quantitative and qualitative reviews offer valuable information for other potential travelers. Their opinions also express their experiences as customers.

Usability is a basic concept in Human – Computer Interaction. A well-known definition is the one provided by the ISO 9241-210: “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [1].

User eXperience (UX) extends the usability concept beyond effectiveness, efficiency and satisfaction. The ISO 9241-210 standard defines UX as a “person’s perceptions and responses resulting from the use and/or anticipated use of a product, system or service” [1]. As usability, UX applies to software systems, but also to products and services.

Customer eXperience (CX) is a broader concept. As Lewis highlights, it addresses the growing emphasis on service design and the Service Science as discipline [2]. Service Science is an interdisciplinary area of study focused on systematic innovation in service. A compelling CX leads to enhanced customer attraction and retention.

As UX extends the usability concept, CX extends the UX concept. Service Science and CX may benefit from the adoption of lessons learned in usability engineering and UX design. However, the obvious link between Human – Computer Interaction and Service Science was not yet properly attended.

The paper analyzes customers’ (quantitative) opinions extracted from www.hotelclub.com in February 2016, before the website was closed; HotelClub is now part of Hoteles.com. Section 2 briefly describes the concepts of UX and CX, and their evaluation. Section 3 analyzes the HotelClub case study. Section 4 highlights conclusions and future work.

2 eXperiences and Evaluations

A well-known usability definition was proposed by the ISO 9241 standard back in 1998 [3]. The ISO 9241 standard was updated in 2010 [1]. Yet a new revision started briefly after, in 2011 [4]. It proves the evolving nature of the usability concept.

UX extends the usability concept beyond effectiveness, efficiency and satisfaction, dimensions recurrently referred in most of usability’s definitions. As usability, UX does not limit to software systems; it also applies to products and services. UX is usually considered as an extension of the usability concept [5]. The “User Experience White Paper” aims to “bring clarity to the UX concept” [6]. But instead of giving a unique UX definition, the document mentions the wide collection of definitions available at www.allaboutux.org [7].

As Joshi highlights, the concept of CX is increasingly discussed, but is rarely defined [8]. Laming and Mason consider CX as: “the physical and emotional experiences occurring through the interactions with the product and/or service offering of a brand from point of first direct, conscious contact, through the total journey to the post-consumption stage” [9]. Gentile, Spiller and Noci identify several CX dimensions: sensorial, emotional, cognitive, pragmatic, lifestyle, and relational [10]. Nambisan and Watt also identify CX dimensions: pragmatic, hedonic, sociability, and usability [11]. They explicitly relate CX to usability.

There are well established usability evaluation methods. If we consider UX as usability’s extension, all these methods may be used in assessing part of the UX dimensions. But evaluating all UX aspects is much more challenging. Allaboutux.org refers to almost 90 UX methods [7]. The amount of methods that are proposed is overwhelming especially for novice UX evaluators/designers.

If we consider CX as an extension of UX, that means UX and usability evaluation methods are also able to evaluate some CX aspects. But evaluating other CX aspects requires specific methods.

CX is developed through a sequence of interactions between the customer and the company (or companies) that offer the product and/or service, called customer “touch-points”. CX should be assessed at least at each touch-point, and CX evaluation methods should address the specificity of each interaction/touch-point. Applying a single evaluation method offers a limited perspective and results. If time and resources are available, several quantitative and qualitative methods should be used.

Our research work first focused on transactional websites’ usability; most of the case studies were online travel agencies. We proposed a methodology to evaluate transactional websites [12]; we also developed a set of usability heuristics for transactional web applications [13]. Later on we extended our research to UX, and recently to CX.

Researches usually focus on qualitative travelers’ comments, but we decided to take an alternative approach, focusing on quantitative data. In a previous study we analyzed quantitative data on travelers’ opinion, freely available at two online travel agencies’ websites: www.tripadvisor.cl and www.hotelclub.com [14]. Data relationships and trends were identified. The study was limited to quantitative data on hotels located in Viña del Mar, one of the most popular tourist destinations in Chile, and focused on www.tripadvisor.cl, as very few reviews were available on www.hotelclub.com.

3 Case Study: www.hotelclub.com

HotelClub was an online travel agency oriented to hotels/accommodations. Its website (www.hotelclub.com) was closed in February 2016. HotelClub is now part of Hoteles.com.

We analyzed almost 4700 travelers’ quantitative reviews on hotels from major Latin American cities: Bogota, Buenos Aires, Ciudad de Mexico, Lima, Montevideo, Panama, Quito, Rio de Janeiro and Santiago de Chile. We also analyzed almost 7700 travelers’ quantitative reviews on hotels from Sydney.

Travelers’ reviews freely available at www.hotelclub.com were both qualitative (comments) and quantitative (numeric evaluation). Quantitative evaluations were made using a 5 points scale, from 1 (worst) to 5 (best), on the following dimensions: D0 – Overall rating, D1 – Amenities, D2 – Cleanliness, D3 – Hotel staff, D4 – Comfort, D5 – Location, D6 – Value. Travelers were classified by HotelClub in 6 types: Business, Single, Family, Friends, Couple, LGBT.

As observations’ scale is ordinal, and no assumption of normality could be made, data were analyzed using nonparametric statistics tests. We used p-value ≤ 0.05 as decision rule.

We performed Spearman ρ tests to check the hypothesis:

  • H0: ρ = 0, the dimensions Dm and Dn are independent,

  • H1: ρ ≠ 0, the dimensions Dm and Dn are dependent.

We performed Kruskal–Wallis H tests to check the hypothesis:

  • H0: there are no significant differences between the opinions of different type of travelers,

  • H1: there are significant differences between the opinions of different type of travelers.

3.1 Bogota

We analyzed 202 reviews of travelers to Bogota. They identified themselves as Business (57), Single (42), Family (25), Friends (21), and Couple (51). 6 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 1. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimension D3 – Hotel staff.

Table 1. Spearman ρ test for Bogota
Full size table

3.2 Buenos Aires

We analyzed 326 reviews of travelers to Buenos Aires. They identified themselves as Business (33), Single (27), Family (44), Friends (58), Couple (145), and LGBT (1). 18 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 2. There are significant correlations between all dimensions. Most of the correlations are moderate. There are only few strong and few weak correlations.

Table 2. Spearman ρ test for Buenos aires
Full size table

3.3 Ciudad de Mexico

We analyzed 1428 reviews of travelers to Ciudad de Mexico. They identified themselves as Business (448), Single (224), Family (199), Friends (117), Couple (382), and LGBT (11). 47 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 3. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimension D4 – Comfort.

Table 3. Spearman ρ test for Ciudad de Mexico
Full size table

3.4 Lima

We analyzed 393 reviews of travelers to Lima. They identified themselves as Business (78), Single (63), Family (67), Friends (38), Couple (134), and LGBT (1). 3 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 4. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimensions D3 – Hotel staff, D4 – Comfort, and D6 – Value.

Table 4. Spearman ρ test for Lima
Full size table

3.5 Montevideo

We analyzed 69 reviews of travelers to Montevideo. They identified themselves as Business (15), Single (14), Family (5), Friends (5), Couple (29), and LGBT (1).

Spearman ρ test results are shown in Table 5. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimensions D1 – Amenities, D2 – Cleanliness, D3 – Hotel staff, and D4 – Comfort.

Table 5. Spearman ρ test for Montevideo
Full size table

3.6 Panama

We analyzed 1116 reviews of travelers to Panama. They identified themselves as Business (188), Single (210), Family (192), Friends (121), Couple (392), and LGBT (9). 4 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 6. There are significant correlations between all dimensions. Most of the correlations are strong. A few correlations are moderate.

Table 6. Spearman ρ test for Panama
Full size table

3.7 Quito

We analyzed 137 reviews of travelers to Quito. They identified themselves as Business (18), Single (36), Family (18), Friends (10), Couple (50), and LGBT (2). 3 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 7. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimensions D2 – Cleanliness, and D3 – Hotel staff.

Table 7. Spearman ρ test for Quito
Full size table

3.8 Rio de Janeiro

We analyzed 722 reviews of travelers to Rio de Janeiro. They identified themselves as Business (97), Single (109), Family (76), Friends (92), Couple (304), and LGBT (9). 35 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 8. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimensions D1 – Amenities, and D4 – Comfort.

Table 8. Spearman ρ test for Rio de Janeiro
Full size table

3.9 Santiago de Chile

We analyzed 294 reviews of travelers to Santiago de Chile. They identified themselves as Business (77), Single (35), Family (21), Friends (27), Couple (123), and LGBT (2). 9 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 9. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. The only dimension weakly correlated to all others is D5 – Location.

Table 9. Spearman ρ test for Santiago de Chile
Full size table

3.10 Sydney

We analyzed 7659 reviews of travelers to Sydney. They identified themselves as Business (662), Single (540), Family (859), Friends (462), Couple (2095), and LGBT (43). 9 travelers did not specify the group they belong to.

Spearman ρ test results are shown in Table 10. There are significant correlations between all dimensions. Most of the correlations are strong or moderate. Dimension D5 – Location is weakly correlated to dimensions D0 – Overall rating, D1 – Amenities, and D2 – Cleanliness.

Table 10. Spearman ρ test for Sydney
Full size table

3.11 Kruskal–Wallis H Tests Results

The Kruskal–Wallis H test results (p-values) for all locations are shown in Table 11.

Table 11. Kruskal–Wallis H test results (p-values)
Full size table

In general, most of the time there are no significant differences between the opinions of different type of travelers. Significant differences occur once for Bogota (dimension D1 – Amenities), twice for Rio de Janeiro (dimensions D2 – Cleanliness and D5 – Location), and four times for Ciudad de Mexico (dimensions D0 – Overall rating, D2 – Cleanliness, D5 – Location, and D6 – Value). Sydney is an exception: significant differences occur for all dimensions.

4 Conclusions

Online travel agencies generate online communities. Quantitative and qualitative reviews offer valuable information for other potential travelers. Travelers’ opinions also express their experiences as customers.

Assessing CX is more challenging than assessing UX and usability. We took a quantitative approach, analyzing travelers’ opinion, freely available at www.hotelclub.com until February 2016. We analyzed almost 4700 travelers’ quantitative reviews on hotels from major Latin American cities, but also almost 7700 reviews on hotels from Sydney. In general, most of the time there are no significant differences between the opinions of different type of travelers. Only in the case of Sydney significant differences occur for all dimensions.

In all cases, there are significant correlations between all surveyed dimensions. Most of the correlations are strong or moderate. Some weak correlations are usually related to dimension D5 – Location. That is probably because all other dimensions are intrinsically related to the hotels, but location is mainly related to the environment.

As future work, we will extend our research to other case studies. We intend to check if the preliminary conclusions are valid in new contexts.