1 Introduction

The idea of applying the principles of game design to professional interfaces has been developed recently under the concept of gamification (for a synthesis, see: Marache-Francisco and Brangier [9]). In fact, gamification interaction is often considered as a challenge subject to conditions of gain, a points system, a leader board and bonuses linked to a rewards system (badges). However, Kim [2] underlines that, points, badges and leader boards are not sufficient to create a game like experience. They are in fact only feedback elements. According to Kim, game design is based on intrinsic motivation generated through three points: autonomy, mastery and purpose. Gamification is therefore not reduced to just applying surface elements (badge, level, reward challenge …) but requires understanding player social styles, their level of expertise and manner of engagement in the task. Several authors have provided gamification guides which go further than simply transposing game elements to a non-game context. Werbach and Hunter [11] provide a list of gamification elements grouped under the following ranked categories: dynamics, mechanics and components. They also identify six steps in gamification design: Define business objectives; Delineate target behaviours; Describe your players; Devise activity cycles; Don’t forget the fun!; Deploy the appropriate tool. As for Kumar and Herger [3] they speak about Player Centered Design and describe five design stages: Know your player; Identify the mission; Understand human motivation; Apply mechanics; Manage, monitor and measure. They also define player persona templates, as well as lists of elements to use. Finally, Nicholson [10] highlights the concept of “meaningful gamification” such as the “integration of user-centered game design elements into non-game contexts”. Here he stresses the need for a design approach adapted to the context of application and to the types of players involved.

The aim of this article is to provide an additional contribution to these defining initiatives and to help in gamification design. To this aim, Sect. 2 will introduce the gamification guide. It includes a description of the design process which enhances user-centered design through questions of emotion and motivation, a tool-box providing principles, decision trees and a design grid as well as examples illustrating the impact of design choices on users. Section 3 raises the question of the guide’s validation and related methodology. Finally, Sect. 4 will show the results of an experiment to validate a design guide involving 29 designers.

2 Design Work for a Guide to the Gamification Process

In response to the question of the poor quality of gamification, a guide to designing gamified interfaces was produced (described in Marache-Francisco and Brangier, [6]). It provides easily applicable standard approaches, fostering the creation of coherent systems adapted to the context into which they are integrated. The underlying hypothesis is that gamification design, a creative exercise, can be guided in a rational and scientific manner by a set of rules drawn from current literature and emerging practices, via a design guide which will accompany its user throughout the design process.

The guide highlights gamification design processes [3, 11] and gamification elements as well as focus points on the various negative effects encountered. In conjunction, an initial classification of gamification is included (sensory-motor modalities, motivation and persuasion, elements, elements supporting cognitive processes, [7]) and the results of a study carried out on the perception of gamification by designers (cosmetic approach with appeal and readability; implicational approach with social identity, freedom of choice and competing against oneself and others) described in [8]. The design guide was produced iteratively by identifying, in each source, the most important points to present to users. A pre-test on two subjects enabled us to fine-tune the guide.

The guide contains three parts:

  1. 1.

    A description of the design process, a two phase iterative approach: context analysis and conception

  2. 2.

    A «tool-box» supplying a set of design support elements:

    • Gamification principles: voluntary engagement, benefits both parties, personalized experience, sustain long term interaction, anticipate unwanted side effects, legality and ethics;

    • Decision Trees: questions guiding the selection of gamification elements; e.g., «Is there a large and reachable social community?»;

    • Design Grid: categories of elements (appeal, narration, avatar, task support, level, motivation, reward, gratification).

  3. 3.

    Examples illustrating the guide’s different dimensions and variables (screen illustrations, choice of the gamification experience…).

This design choice gives an overview of the approach (design process for the steps and principles for the key concepts), an inventory of available elements (design grid), help in decision-making (decision trees), and finally, illustrated examples.

The aims of this guide are to provide designers with a body of ideas (content); guidance for the design (aim, finalisation) and resources for designing (steps, process). The guide should therefore fulfil the task of stimulating designers’ creative performance.

3 Problem and Method of Validating the Gamification Guide

The guide not only aims to be educational and scientific but also features every aspect of the gamification design process. It particularly focuses on translating abstract concepts into easily understandable and operationalizable elements, the target group being an extensive one with quite varied levels of mastery of ergonomic concepts and HMI design. The designers should not be too restricted, and should be given a certain amount of freedom to ensure their compliance with the guide.

3.1 Problem

Hence the aim of this research is to determine whether the guide is useful for gamification design and how. The basic premise being that if anyone is capable of transferring game elements from existing games into a non-game system, having a guide which provides the approach to gamification and steers the designer through an extensive set of available elements would enable him/her to be more creative. For this study the general hypothesis is therefore that the guide enables interface enrichment with gamification. This is tested using the creativity indicators defined by Guilford in 1967 (cited by [5]) and Torrance in 1976 (cited by [1]), being:

  • Fluidity: Refers to the number of ideas produced;

  • Flexibility: How many categories of ideas are covered;

  • Originality: Corresponds to the ability to produce different, new ideas;

  • Elaboration: Describes the amount of detail provided by the designer.

Hence the general hypotheses are that the guide fosters simultaneously: fluidity; flexibility; originality; and elaboration.

3.2 Method

Sample.

Twenty-nine designers specialized in human-machine interaction (22 men and 7 women; average age = 31, standard deviation = 9.5; 11 Masters Students and 18 professionals) took part in the experiment. They were all more or less familiar with Business Intelligence (3 not) and gamification (1 not) and 24 were game players.

Task and Material.

The 29 participants’ task was to gamify three screens of a Business Intelligence software, with help from the software’s personas and the following material:

  1. 1.

    A 20-min oral presentation illustrated with slides on gamification (definition; objectives; limits);

  2. 2.

    For one group, guidelines to using the gamification guide (process; grid with cards and tables; examples);

  3. 3.

    For one group, the gamification guide containing: a booklet describing gamification principles; Design grid with 4 cards summarizing the elements in categories and 4 posters describing the content of the cards (description of elements and examples);

After answering a small questionnaire to sum up their profiles, they were given the following material to carry out their gamification task:

  • A size A3 sheet of paper to make sketches on and to describe gamified screens;

  • Material to be able to explain in writing the choices made for each screen;

  • For one of the groups material to show the link between personas and gamification dimensions;

  • A medium to enable a description of the gamification technique used;

  • Pencil crayons, felt-tip pens, pencils, black pen, eraser, post-its.

Procedure.

Two experimental groups were formed:

  • One was a control group (g1).

  • The other group had the gamification guide (g2).

The subjects were divided equally between the two groups according to their profiles (e.g., level of knowledge of gamification, experience or not of video games).

The procedure was as follows. Firstly the subjects (g1 and g2) were given a quick training in gamification. They were then given a questionnaire to fill in focused on their gamification and video game knowledge. The first design phase (test phase p1) lasted one hour. All the groups were given a case study which consisted in gamifying a scenario in the software. The intermediary stage was a coffee break for the control group (g1) where the subjects were asked not to discuss their ideas with each other. Group g2 were given training on the gamification guide which included: the design process, the different supports, and examples of application. During the second design phase (experimental phase p2), the groups (g1 and g2) had to go back to the initial case. They were asked to rework their gamification suggestions. Group g1 worked in the same conditions as in phase one, whereas group g2 were provided with the principles, cards and posters.

Measures.

How and what are the designers going to gamify? The results analysis seeks to identify and quantify the gamification strategies and elements used by the two groups.

The strategies and elements identified are the following: Define the functions; Plan the temporality of actions; Plan the progression; Inform on the aims pursued; Organize personalization; take into account level of experience; Develop appeal; Set the scene; Solicit other users; Enable interpersonal cooperation; Initiate social comparison; Launch individual or social competition; Motivate with high level goals; Acknowledge and reward; Contribute to group visibility; Enable self-expression; Give feedback valence; Manage perceived freedom of choice; Foster user’s ethics; Reduce harmful elements or their perception. It should be noted that the category, Define the functions, groups together all the elements which do not refer to gamification, an interesting element to take into consideration when conducting the analysis. The coding work carried out is illustrated in Table 1:

Table 1. Examples of coding of what the designers produced
Full size table

Once all the gamification elements were identified in the sketches, comparisons between phase 1 and 2 were made for the two groups:

  • The average difference (phase1–phase 2) in the number of gamified elements;

  • The average difference (phase 2–phase 1) in the number of gamification strategies;

  • The number of ideas which, in phase 2, have been re-used, changed or abandoned from stage 1 or were completely new;

  • The percentage of new ideas in phase 2 compared to the total number of ideas put forward.

3.3 Operational Hypotheses

The hypotheses are the following:

  • H1: The average difference (phase 2–phase 1) in the number of elements will be higher for the group which used the guide than in the group without the guide;

  • H2: The average difference (phase 2–phase 1) in the number of elements per strategy will be higher for the group which used the guide than for the group without the guide;

  • H3: The percentage of new ideas in phase 2 will be higher for the group which used the guide than for the group without the guide;

  • H4: The average difference (phase 2–phase 1) in the number of secondary ideas will be higher for the group which used the guide than for the group without the guide.

4 Results Analysis

All the results were analysed statistically using comparison of averages tests. Normality and homogeneity of variances tests were conducted in order to select the type of test to conduct (parametric: student’s t test; non parametric: Mann-Whitney U test).

The average difference in the number of gamified elements between phase 1 and phase 2 is much higher for the subjects who used the guide than for the others (p = 0.03 with U = 55.5 and Z adjusted = 2.14). They had on average 15.8 more gamification elements in phase 2 (standard deviation: 12.83) against 7.57 (standard deviation: 7.73) which is more than twice as many. Thus validating our hypothesis: The guide fosters fluidity.

The average difference in the number of elements per strategy is significantly different between the subjects who used the guide and the others for 7 strategies out of 20 (results in detail in Table 2). Thus validating the hypothesis: The guide fosters flexibility with enrichment in the following categories: Plan the progression; Take into account level of experience; Set the scene; Launch an individual or social competition; Enable self-expression; Manage perceived freedom of choice. Conversely, the use of elements from the strategy Define the functions increased for the subjects without the guide whereas they decreased for those with the guide.

Table 2. Intermediary results summary
Full size table

The percentage of new ideas out of all the ideas in phase 2 is much higher for the subjects using the guide than the others (p = 0.00 with U = 41 and Z adjusted = 2.83). Hence, they had on average 51.8 % of new ideas out of all the ideas put forward (standard deviation: 31.8) against 20.1 % (standard deviation: 21.8) which is more than twice as many. Thus validating our hypothesis: the guide fosters originality.

It appears finally, that the average difference in the number of secondary ideas between phase 1 and phase 2 is significantly higher for the guide users than for the others (p = 0.03 with U = 56 and Z adjusted = 2.13). Hence, they had on average 6.9 more secondary ideas in phase 2 (standard deviation: 5.4) against 2.9 (standard deviation: 3.4) which is more than twice as many. Our hypothesis is thus validated: the guide fosters elaboration. All the hypotheses have therefore been validated. The results are summarized in Table 2.

This study was initiated in order to verify the validity of the gamification guide described in [6]. Two groups of subjects had to carry out the task of gamifying interfaces of a Business Intelligence software programme. The experiment was conducted in two phases: firstly, all the subjects were given a short introduction to gamification, they then had to begin gamifying the software presented. During the intermediate phase, one group was presented with the design guide while the other group was taking a break. In the experimental phase, each group was given the task of returning to the initial case study and finishing it. The hypothesis being that the guide would enable interface enrichment with gamification, tested using four creativity indicators (fluidity; flexibility; originality; and elaboration).

All the hypotheses were validated. Hence, the subjects using the guide added more than twice as many gamification elements in phase 2 compared to the subjects without the guide (H1). Fluidity is therefore fostered by using the guide. Additionally, the subjects of the group with the guide demonstrated a more substantial enrichment in terms of elements for six gamification strategies compared to the group without the guide (H2): Plan the progression; Take into account the level of experience; Set the scene; Launch an individual or social competition; Enable self-expression; Manage perceived freedom of choice. Flexibility is therefore also fostered using the guide.

Notably, the subjects using the guide reduced the number of elements for the strategy Define the functions (which refers to the drawing for new useful functions, unrelated to gamification) while the subjects without the guide did the opposite. In the light of these results we can assume that the guide enables understanding of what gamification refers to, thus leading to the abandon of irrelevant strategies.

Moreover, in phase 2, the subjects using the guide put forward more than 50 % of new ideas than the subjects without the guide (H3). They were therefore much more original in their initial proposals. An in-depth analysis of the ideas put forward in phase 2 (re-used, modified, abandoned or new) also shows that only new ideas differentiate the groups. Both groups similarly decided to modify, re-use or abandon the primary ideas in phase 1.

Finally, the subjects using the guide added more than twice as many secondary ideas in phase 2 compared to the subjects without the guide (H4). They developed their ideas much more, detailing their production in a precise manner.

5 Conclusion

The results of this study emphasise the fact that the guide fosters interface enrichment when numerous gamification elements are brought in. Hence, the subjects who used the guide in phase 2 showed much more:

  • Fluidity: With a higher increase in gamification elements;

  • Flexibility: With a higher increase in coverage of gamification strategies;

  • Originality: With more than half as many new primary ideas;

  • Elaboration: With more detailed ideas.

However, it is interesting to note that the subjects who didn’t benefit from the design guide did manage to submit proposals which embraced gamification elements. Thus, the designers seemed to grasp the meaning of these practises intuitively. Be that as it may, the subjects were more creative and produced more, and apparently better, having used the specially developed design guide (more strategies, more new ideas, and more details). Furthermore, we also observed more relevant design with the abandon of functionalities not linked to gamification. This study is therefore considered as an initial validation of its efficacy.

However, several limitations of the study should be noted. Firstly, the subjects were not a homogenous population group. The task was complex. They had to immerge themselves into a large case control study, the concepts being, for a certain number of them, completely unknown. Furthermore, the time allowed was relatively restricted. Moreover, the subjects using the guide in the second phase didn’t get a break. They also had to learn about and understand complex material before being able to put it into application, thus shortening the time for the actual design work. The subjects in the group without the guide had time for a break and to think about what they had done. They had more time to fine-tune, and develop their ideas during the design phase. To remedy this, the subjects of the group without the guide could, for example, be trained for an unlinked task, then a break could be given for each group. Another focus of research would be to carry out an analysis of the designers’ activities when using the guide to identify the improvements to be made. Presenting the guide to gamification experts could be interesting in order to identify the areas where progress is needed.