1 Introduction

In the last two decades, information and communication technology (ICT) has become the constituent part of many different aspects of human life. Regardless of whether ICT serves as an aid in business and learning or is employed in leisure time for relaxation and fun, an interaction with it results in user’s subjective perception which is commonly referred to as user experience (UX).

When reasoning about the definition of UX, researchers can be grouped in two main streams. The first one defines UX as a synonym for usability and user-oriented design. As one of the main representatives of this stream, Bevan [3] argues that usability evaluation contributes to better understanding of users’ needs, improves software performance, and in turn creates positive UX. In addition, Alben [2] stated that successful, interesting, and valuable UX is affected by both software development and interaction with software criteria. The second stream defines UX as a concept which is separated from usability and represents added value to software in terms of fulfilling users’ needs and desires [17]. For instance, Hassenzahl and Tractinsky [16] emphasize that UX facets go beyond instrumental and are orientated on satisfying hedonic human needs related to beauty, invocation, stimulation, growth, etc. In that respect, UX monitoring and testing should be included in all phases of the software lifecycle [31].

Specific piece of information and communication technology are computer games. Today, they are used in leisure time for entertainment purposes as well for learning and gaining diverse skills. Facer [10] claims that learning through computer games is easier and more fun because they provide information through graphics, animations and videos which is more interesting than pure text used in regular learning materials. Moreover, educational computer games are challenging in a way that they enable students to solve real-life issues in a safe environment [22, 29]. By anticipating possible moves that one will make, a computer game can predict events that might occur [38] and are due to this feature used for training purposes in organizational [7] and military settings [11]. Considering that Massively Online Battle Arena (MOBA) genre of computer games have implemented all of the set forth features and can be employed in numerous contexts, they have been used as a representative sample in our empirical study on assessment of specific UX facets.

The remainder of the paper is structured in following way. In the second section, a brief overview of relevant and recent studies on UX assessment together with the proposed research framework and hypotheses is offered. Employed research design is described in the third section. Results of the empirical study are presented in the fourth section. Discussion of study findings and concluding remarks are contained in the last section.

2 Background to the Research

2.1 Literature Review

Recent research related to the UX in interaction with computer games was mainly focused on exploring aspects such as flow, fun, immersion, presence, tension, attention, frustration and addiction. Some studies also examined impact of playing with others and effect of game elements such as story, gameplay and game mechanics on UX [8, 20, 33, 34]. In the light of the aforementioned, Takatalo et al. [37] developed fifteen scaled questionnaire to measure two aspects of game playing. The first one is adaptation which is manifested by physical presence, attention, and interest while the second one is flow and quality which is represented by playfulness, game challenges, impression, and enjoyment, among others. The same authors [36] developed another questionnaire, Presence-Involvement-Flow-Framework (PIFF), which was used for evaluating the impact of presence, involvement and flow on UX. Nacke et al. [26] measured seven different dimensions (sensory and imaginative immersion, tension, competence, flow, negative affect, positive affect, and challenge) of UX with their Game Experience Questionnaire (GEQ). Desurvire et al. [8] developed Heuristics Evaluation of Playability (HEP) framework meant for evaluating game story, game play, game mechanics and game usability. Very similarly, Hunicke et al. [20] conducted a research in which they explored Mechanics, Dynamics, and Aesthetics (MDA) of a computer game. Results of a study carried out by Lazzaros [23] have shown that challenges (overcoming obstacles), grabbing attention (curiosity, excitement, and adventure), altering states (emotions and sensations) and presence of other players (competition and co-operation with others) are the main aspects of a computer game that have impact on UX. Roth et al. [32] developed a questionnaire in a form of a multidimensional self-reporting scale which purpose was to examine the influence of interactive stories on UX. The aim of the research conducted by Hannu et al. [15] was to investigate the influence of various elements of user experience such as challenge, competition and fellowship, and sympathy and thrill on playfulness and pleasure, in the context of computer games. Choi and Kim [5] carried out a research in which they analyzed the effect of personal and social interaction, flow and absorption on users’ loyalty. Finally, Lee [24] completed a study whose purpose was to examine the influence of player’s motivation to participate, addictive behavior, flow and immersion experience, role-playing and achievements on player’s loyalty.

2.2 Research Model and Hypotheses

The aim of the research presented in this paper was to identify to what extent different facets of UX affect user’s continuance intention related to playing MOBA genre games. For that purpose, the research framework that illustrates an interplay among five UX facets and user’s loyalty was designed. Four UX facets (absorption & dissociation (A&D), flow (FLW), immersion (IMS), and presence (PRS)) were adopted from Nacke et al. [26] and subsequently enhanced with questionnaire items that were proposed by Takatalo et al. [36, 37]. Social play (SPL) was measured with items designed by Isbister [21] whereas loyalty was evaluated with items that were adopted from Agarwal and Karahanna [1].

Immersion is a specific state of becoming engaged in the gaming experience while still retaining some awareness of surroundings, but in a way that surrounding distractions can be successfully and easily ignored [26]. Prior research [8] showed that immersion represents deeper engagement with the game and is result of positive experience that occurs during the game play. Study conducted by Hannu et al. [15] revealed that trait of successfully ignoring surroundings is constitutive part of playfulness. In that respect, we hypothesize the following:

H1.:

Immersion will positively influence absorption & dissociation.

Flow is a state achieved when one’s skills and capabilities are in balance with heaviness of game objectives. According to Sánchez et al. [33], flow is one of the influential factor of player’s loyalty towards online games. In addition, Lee [24] found that well-build game elements such as role-playing and achievements have great influence on flow. Considering that player will be more motivated to gain specific skills if he or she establish deeper emotional connection with an avatar, we proposed following hypothesis:

H2.:

Flow will positively influence presence.

Social play refers to any form of socialization through joint game play, including competition, collaboration and assistance [23]. Several studies [21, 24] discovered that avatars, which form one’s virtual identity and represent him or her in interaction with others, helps one to feel more connected with and included into game. Playing with others eventually results in deeper relationship between players in form of virtual friendship. Choi and Kim [5] stated that one’s desire for spending time with friends will result in continuance intention related to playing the game. In the light of the aforementioned, we proposed following hypotheses:

H3.:

Social play will positively influence presence.

H4.:

Social play will positively influence loyalty.

Presence can be described as a state of being aware of real surroundings and at the same time having experience of being inside the virtual environment. The set forth can be achieved by empathizing the plot or identifying with the avatar (its appearance, abilities and skills). By making a connection with the game, player willingly becomes present in it [36]. As a result of their research, Lombard and Ditton [25] concluded that media attributes (e.g. color and sound richness, camera angle, extent of social realism) influence to a certain degree on one’s presence perception in the context of the virtual world. Based on the aforesaid, the following hypothesis is proposed:

H5.:

Presence will positively influence absorption & dissociation.

Absorption & dissociation refers to a state in which one’s mind detaches from reality in a way that he or she identifies himself or herself with an avatar and stops having any awareness of real surroundings while playing the game. Prior work suggests that excessive immersion into the game can lead to absorption in a way that person adopts and mimics appearance, behavior, and thinking of favorite avatar in everyday life situations [4]. In their research, Poels et al. [28] discovered that dissociation can manifest as anxiety when one is not able to play the game, excessive excitement when talking or thinking about the game, and anticipation of being able to play the game again. Taking the set forth into account, following hypothesis was defined:

H6.:

Absorption & dissociation will positively influence loyalty.

3 Methodology

Research subjects were randomly selected players of MOBA games. The sample of study participants was composed of individuals who play MOBA games on regular basis as well of individuals who used to play this genre of computer games. The research included 158 MOBA players of which 91.77 % completed the questionnaire correctly. Majority (71.72 %) of study participants were male while 28.28 % of them were female. The age of players ranged from 10 to more than 50 years where majority (71.03 %) of them had between 20 and 29 years.

Data were collected by means of the post-use questionnaire that was designed with KwikSurveys online survey builder. Link to the questionnaire was published at several websites and forums dealing with popular MOBA games such as Defense of the Ancients 2 (DOTA2) and League of Legends (LOL). It was also posted to gaming community Steam and official fan pages of the aforementioned MOBA games on social networking sites Facebook and Twitter. The questionnaire was available for three weeks during August 2014.

Questionnaire was comprised of two main parts. Demographic data about MOBA players were collected in the first part of the questionnaire. The second part contained 24 statements related to six dimensions of the proposed research framework. The answers were modulated on a four-point Likert scale ranging from (1) “strongly disagree” to (4) “strongly agree”.

Considering the exploratory nature of conducted empirical study, the psychometric features of the proposed research framework and associated hypotheses were examined by means of the partial least squares (PLS) structural equation modeling (SEM) technique. The reasoning behind the choice of PLS-SEM over its covariance-based counterpart (CB-SEM) relies on the fact that PLS-SEM does not require sound theoretical foundations and achieves high level of statistical power even when the sample size is relatively small and data significantly deviate from normal distribution [14]. Data analysis was conducted with SmartPLS 2.0 M3 [30] software.

4 Results

PLS-SEM algorithm performs path analysis in two stages. The first one is iterative approximation of measurement model parameters while the second one is the estimation of standardized partial regression coefficients which are part of the structural model [9]. Consequently, the assessment of psychometric characteristics of the research framework was two-step procedure.

The quality of the measurement model was evaluated by examining the reliability of manifest variables (items), reliability of latent variables (constructs), convergent validity, and discriminant validity. Reliability of manifest variables was estimated by exploring the standardized loadings of manifest variables with their respective latent variable. The purification guidelines suggested by Hulland [19] indicate that manifest variables should be removed from the model if their standardized loadings are below threshold value of 0.707. The same author argue that exception can be made in the case of exploratory studies in which composite reliability values above 0.600 are considered acceptable. Results of the confirmatory factor analysis (CFA) presented in Table 1 indicate that standardized loadings of 23 manifest variables were greater than recommended cut-off value, except of SPL1 whose standardized factor loading was 0.6920. However, since this manifest variable presents an important dimension of social play and its omission would significantly reduce the validity of the research framework, it was retained in the measurement model. Standardized loadings of manifest variables were in the range from 0.6920 to 0.8925 which indicates that latent variables explained between 47.89 % and 79.66 % of their manifest variables’ variance.

Table 1. Standardized factor loadings and cross loadings of manifest variables
Full size table

Reliability of latent variables was tested using the Cronbach’s alpha (α) coefficient and composite reliability (CR). Opposed to Cronbach’s alpha (α) which assumes that weightings of items are equal, CR includes actual item loadings and consequently indicates better estimate of internal consistency. Hair et al. [14] recommended thresholds of 0.707 for both CR and Cronbach’s α. Data provided in Table 1 imply that estimated values for all six latent variables were above the aforementioned cut-off value.

Convergent validity was examined using the average variance extracted (AVE). According to Fornell and Larcker [12], AVE values above 0.50 are considered acceptable because they indicate that shared variance between specific latent variable and its manifest variables is larger than variance of the measurement error. As can be seen in Table 2, all constructs have met this criterion.

Table 2. Convergent validity and internal consistency variables
Full size table

Discriminant validity is defined as an extent of dissimilarity among latent variables in the measurement model. Two measures were used to evaluate discriminant validity [14]. The first one are cross loadings according to which manifest variables should load higher on latent variable they are assigned to than on any of other latent variables in the model. As depicted in Table 1, loadings of all manifest variables on their respective latent variables are higher than their loadings on all remaining latent variables which indicates that all manifest variables have met this criterion of discriminant validity. The second measure is the Fornell-Larcker criterion [12] according to which the square root of the AVE of each latent variable should be greater than its highest correlation with remaining latent variables in the model. Table 3 clearly illustrates that each latent variable shares greater amount of variance with their manifest variables than with other latent variables in the model which implies that all latent variables have met the second criterion of discriminant validity. All the aforementioned confirms reliability and validity of the measurement model.

Table 3. Discriminant validity of latent variables
Full size table

In the second step of evaluation, the quality of structural model was estimated by means of endogenous latent variables’ determination coefficient, path coefficients’ significance level, exogenous latent variables’ effect size, and exogenous latent variables’ predictive relevance.

The determination coefficient (R2) refers to the proportion of endogenous latent variables’ variance explained by the set of predictors. According to Orehovački [27], R2 values of 0.15, 0.34, or 0.46 for endogenous latent variables in the structural model can be, as a rule of thumb, interpreted as weak, moderate, or substantial, respectively. As shown in Fig. 1, 22.83 % of variance in presence was explained by flow and social play, 44.77 % of variance in absorption & dissociation was explained by immersion and presence while 43.91 % of variance in loyalty was explained by social play and absorption & dissociation. Considering the set forth, predictors of absorption & dissociation and continuance intention have moderate explanatory power while predictions of presence have weak explanatory power.

Fig. 1.
figure 1

PLS estimates for structural model

Full size image

The evaluation of path coefficients’ goodness was carried out with an aim to examine the hypothesized associations among latent variables. One-tailed t-statistics derived from a bootstrapping resampling procedure was used to determine significance of path coefficients. The number of bootstrap samples was 5.000 while the number of cases was equal to the sample size.

Results of hypotheses testing are shown in the first five columns of Table 4. Data analysis revealed that both flow (β = 0.2914, p < 0.001) and social play (β = 0.2865, p < 0.01) significantly contribute to the presence thus providing support for H2 and H3. It was also discovered that immersion (β = 0.5690, p < 0.001) and presence (β = 0.2481, p < 0.001) significantly affect the absorption & dissociation thereby supporting H1 and H5. Finally, it appeared that absorption & dissociation (β = 0.1685, p < 0.05) and social play (β = 0.5792, p < 0.001) have significant impact on players’ loyalty thus demonstrating support for H6 and H4.

Table 4. Results of testing the hypotheses, effect size, and predictive validity
Full size table

The effect size (f2) reflects the change in the endogenous latent variable’s determination coefficient. According to Cohen [6] values of 0.35, 0.15, or 0.02 imply that specific predictor has large, medium, or small influence on endogenous latent variable, respectively. As presented in the sixth column of Table 4, immersion strongly (f2 = 0.54) affects absorption & dissociation while presence has small impact (f2 = 0.10) on this endogenous latent variable. Both flow and social play have small influence (f2 = 0.09) on presence. Finally, the effect of social play on loyalty is large in size (f2 = 0.49) whereas it turned out that absorption & dissociation has small effect (f2 = 0.04) on loyalty.

The predictive validity of exogenous latent variables was tested with non-parametric Stone’s [35] and Geisser’s [13] cross-validated redundancy measure Q2 that based on blindfolding reuse technique predicts indicators of endogenous latent variable. Changes in Q2 indicate relative impact (q2) of exogenous latent variables in predicting manifest variables assigned to endogenous latent variable. Values of 0.35, 0.15, or 0.02 signify substantial, moderate, or weak predictive relevance of a specific exogenous latent variable [18]. Taking into account data presented in the last column of Table 4, immersion has moderate relevance (q2 = 0.25) while presence has weak relevance (q2 = 0.05) in predicting absorption & dissociation. Moreover, flow and social play have weak relevance (q2 = 0.07 and 0.06, respectively) in predicting presence. Finally, social play has moderate relevance (q2 = 0.30) whereas absorption & dissociation has weak relevance (q2 = 0.02) in predicting loyalty.

5 Discussion and Concluding Remarks

The objective of this paper was to identify to what extent different facets of UX (immersion, flow, presence, absorption & dissociation, and social play) have influence on users’ loyalty in terms of a continuance intention to play Massively Online Battle Arena (MOBA) games. For that purpose, a research framework which illustrates interference among aforementioned latent variables was proposed. Its psychometric characteristics were examined by means of the partial least squares (PLS) structural equation modeling (SEM) technique. Considering that measurement and structural model have met all criteria prescribed by Hair et al. [14], validity and reliability of the proposed research framework were confirmed.

Given that reported findings add to the extant body of knowledge, they offer several implications for researchers and practitioners. Researchers can use introduced framework as a foundation for future studies in the field. On the other hand, designers and developers can employ the post-use questionnaire in order to determine to what extent games they implemented have met specific criteria of user experience. In addition, the analysis of the proposed framework revealed several designing aspects which developers should take into account when developing a MOBA game. Firstly, they have to assure that the player does not feel frustration or boredom when playing a game. The set forth can be easily achieved by balancing heaviness of in-game assignments and one’s capabilities which would eventually lead player to a flow state. The attention should be also paid to the development of avatars and the plot so the player could bound with game on emotional level thus reaching a game presence. In addition, a game should be successful in attracting one’s attention in a manner that nothing can distract him or her when playing a game. The state of immersion and a feeling of a game presence together indicate that one is completely involved in a virtual world and detached from reality when playing a game. Finally, if a MOBA game supports connecting and playing with others and is capable to bring a player into a state of absorption & dissociation, he or she will be eager to play a game again and recommend it to others.

Considering that topic of this paper were MOBA games and that presented results are part of an ongoing research, in our future work we will enhance the proposed research framework with additional UX dimensions and then explore its validity and reliability on the representative sample of diverse game genres.