Keywords

1 Introduction

Both toy and games industries are investing in hybrid products as connected toys. These are playful artifacts embedded with electronic components able to connect with other devices. Smart toys can connect with other toys, controllers, smartphones, tablets, and game consoles. Despite electronics, these products use passive technologies as computer vision techniques and touchpoints. Such toys can promote different play experiences, with fixed or open rules. In these, user access the system using toys as input/output (I/O), either in outdoor or indoor environments. Thus, such toys consist of playful user interfaces [1]. We refer to ‘playful system’ as play activities including hybrid games, connected toys, augmented board games, multimedia applications, interactive storytelling, and open-ended play scenarios. Such systems are complex artifacts since they use real and virtual information. Therefore, they present new challenges to both designers and developers. These challenges include defining what type of data to extract from the environment, how to present this information in the toy’s interface, and selecting appropriate technologies to collect such data. The relational model for hybrid gameplay [2], works as a tool to describe playful systems. The model can synthesize system setup information in a group of interactive aspects. Our research goal was to turn the model into a practical tool for creating hybrid game concepts. To achieve it, we included the model in a 16-week class to design hybrid games. In this paper, we presented the model and its aspects, then, we detailed its usage in the course schedule, and how students experienced it. Furthermore, we presented student’s six working prototypes, including design cycles, and playtesting sessions. After class, we conducted semi-structured interviews with student’s representatives, then, we presented data collection procedure, and interview results. Finally, we proposed improvements in model nomenclature based on student’s feedback. In addition, we recommended topics for a methodological approach to design hybrid games.

1.1 Design Practices of Hybrid Play Products

Hybrid products available on the market come from traditional toy and games industries, including robotics companies, and independent teams. Hasbro, a toy company, has several products as Furby, a stuffed toy robot able to connect to mobile devices. Other, are the Playmation connected toys, inspired by Avengers franchise. Nintendo, a game company, produces amiibo toys; these are character figurines from popular brands, including a set of cards, and plush toys. Amiibo toys use near-field communication (NFC) to transfer data to connected game consoles. In 2016, the robot company Sphero, released the wearable Force Band along with droid BB8, both inspired by Star Wars series. Wearing the smart wristband user can control, through gestures, BB8 movements. Following advances of digital prototyping, small independent teams are releasing new products. These advances include low-cost 3D printing, and electronic components as Arduino and Raspberry platforms. Moreover, small teams are opting for crowdfunding campaigns to raise resources. For example, DiDi, a teddy bear, and ROXs, a pervasive game console, have used funding platforms such as Kickstarter and Indiegogo.

Despite novelty and potential of hybrid interactions, many products have failed on the market. Tyni and Kultima [3], in their study, interviewed professionals of hybrid play products field that worked within 2012 and 2014, accordingly, the hybrid products market experienced an experimental phase. Thus, companies discontinued several products, while still looking for best design practices. Conforming to authors, professionals and companies adopt different trends in their design process. Such trends include toys as a platform for several games, (e.g., ROXs, Tiggly, and Osmo), and games associated with characters and narratives (e.g., Skylanders, Edwin: the duck, and Amiibo). Besides, interviewers claimed that companies make use of disconnect design practices from both toy and game design. Hence, we supposed both industries could benefit from hybrid design approaches since product concepts.

Developing engagement strategies is a problem that affects both narrative and platform cases. Regarding narrative products, the content design must associate a great semantic value. Such content refers to the game or playful activity aspects, including physical features of products. To meet it, larger companies adopt popular franchises such as Batman and Star Wars. Although, companies discontinued cases as Mattel Apptivity and Disney Infinity, despite their products incorporated popular brands. On the other hand, platform cases consist of generic products allowing several play modalities. Thus, such products demand regular content updates to achieve long-term engagement. For example, ROXs engagement strategy is allowing custom play modes. Another strategy is developing independent games to a single product, such as Magikbee and Tiggly. Moreover, many products use artificial intelligence resources to promote spontaneous play interactions and unexpected events (e.g., Furby, and Cozmo).

A design approach is developing such toys as meaningful interface elements. Therefore, we recommend professionals to design playful interface elements as game objects. Then, these may interact with other game objects, either physical or virtual, including the environment, and people. We expect this approach can assist designers of both narrative and platform products since it integrates both interactive and semantic values. For example, in a space themed game, a spaceship toy works as an interface, so, the toy enables player access to game world functions. Besides, viewing toy as a game object, it includes the semantic value of a spaceship. Then, the toy must assimilate game object features, such as power attack, defense, movements, and game stats. Thus, such features when incorporated during development may result in meaningful toys. Hence, we proposed investing in the relationship among interface elements during first design stages.

2 Related Work

In their study, Barba et al. [4] presented results of an experimental class for designing augmented reality (AR) board games. The class was multidisciplinary combining 19 students, from both undergraduate and graduate level. Student’s background included fields of arts, design, and technology. All developed projects used game platform Gizmodo, a handheld device similar to a portable console. To meet an agile prototyping approach, games used fiducial markers to augment virtual information. Course schedule included four prototyping cycles, exercises with selected technologies, and brainstorming sessions. Authors claimed that first prototypes were alike to existing board games. Yet, following AR technologies practices facilitated students in developing meaningful ideas. In our research, we used a comparable approach, joining students of engineering, computer science, and design. Similarly, several prototypes used computer vision techniques such as color tracking. Otherwise, due student’s technology background, we implemented projects using multiple technologies. Hence, our prototypes included electronics such as rumble and servomotors, Bluetooth and Radio Frequency modules, and LEDs. Besides, analogous to Barba et al., our games used visualization and manipulation tasks. However, one of the projects used Kinect along with wearable technology to embodiment facilitation.

Eva Hornecker [5] adapted a tangible interaction framework as a creative method to designing hybrid systems. The proposed method used a card game set during brainstorming sessions. The card set comprised four framework aspects: Tangible Manipulation, Spatial Interaction, Embodied Facilitation, and Expressive Representation. Thus, each group of cards had a color, and each card had a related question, along with representative images and subtitles. The research goal was to use cards to promote discussion among participants, in brainstorming sessions. Hornecker applied the method in 10 sessions with professionals and students. As result, subjects produced games and interactive installations, for both indoor and outdoor environments. Accordingly, the cards enabled groups to discuss the relevance of each framework aspect, facilitating ideas formulation. Similarly, our research adopted a theoretical model as a tool for the creative process. We choose to use the hybrid gameplay model since its descriptive structure enables to name several aspects of playful systems. These, include physical and social interaction, types of I/O technology, and the relationship among interface elements. Taking lessons from Hornecker and the previous study [4], we decided to insert the model as a resource in design stages of conception, ideation, and concept refinement. Furthermore, we opted for a brainstorming method including a sequence of questions. Yet, we used toys as resources rather than a set of cards.

Vissers and Geerts [6] introduced an evaluation method to distinguish tangible objects made from different materials. Authors evaluated five smart dices in augmented board games. Thus, they proposed eight guidelines for smart dice design, considering aspects of shape, material, texture, and its usage. Among these, one guideline related size of objects to player’s precaution. For example, users experienced a large dice with caution than a small sized. Similarly, Heijboer and van den Hoven [7] presented a study to analyze the level of abstraction in playful tangibles. The study compared user’s perception among adults and children, to differentiate how subjects apprehended object meanings. Such objects were a tangible interface for Totti, a collaborative game. In Totti, objects represented game characters, and each figure related to elements of Nature (e.g., water and fire). Then, each object group had eight abstraction levels, ranging its appearance resembling character’s power, to a realistic look. Results revealed a better understanding of second abstraction level in both user groups. Indicating that a few information is enough to assimilate artifacts functional design. Both studies addressed physical aspects related to appearance of objects, and in how users perceive interface elements. Hence, in our study, we explored material aspects of student’s prototypes, in addition to computational resources. Especially, on the 3D printed artifacts since students adapted existing toys as interface elements. Thus, students considered both physical and functional design aspects in their projects. For example, to build touchpoint figures, participants tested several materials, such as foil paper, copper wires, magnets, and metal. Moreover, the shape of such objects required allowing direct contact of conductive terminals with user’s hands.

3 Research Method

Our research goal was adapting the relational model for hybrid gameplay [2] as a practical tool for designing playful systems. To achieve it, we included the model in a 16-week class to design hybrid games. Hence, the first step of research method was elaborating course structure using the theoretical model. Then, we taught the course in two classes with undergraduate and graduate students. Participants of both classes had experience in fields of Design, Engineering, and Computer Science. Classes took place in March-July of 2016, at Computer Centre of Universidade Federal de Pernambuco (UFPE), Brazil. After class, we conducted semi-structured interviews with student’s representatives. The aim was to assess model usefulness in stages of the creative process. Besides, we recognized aspects of the model that required improvements.

3.1 A Relational Model for Hybrid-Gameplay

The relational model describes playful systems relating three entities: things, environment, and people (see Fig. 1). Each relation axis represents a group of interactive aspects between two entities. Besides axes, each entity relates to itself. Things and environment axis shows what interface elements do in the interactive environment. While, environment and people axis indicates where participants locate in the environment. Then, people and things axis represents how they physically interact with interface elements, including what I/O technologies enable such interactions. Finally, the relationships among things and the environment happen in a physical domain, while, people’s relations occur in a social domain.

Fig. 1.
figure 1

Relational Model for Hybrid-gameplay Interaction, current version.

Full size image

Things are the physical interface elements, so, things are toys and the auxiliary devices enabling interaction. Such devices comprehend things such as computers, tablet, cameras, monitors, and controllers. Then, things communicate with each other using active and passive technologies. Hence, we name playful things as traditional toys, smart toys, and smart playground. Traditional toys have no embedded technology in their design, yet, smart toys has electronic components. We refer as a smart playground to large installations including multiple connected toys. According to model, things relate to the environment replicating, extending, replacing, creating, destroying, updating, and augmenting both real and virtual information in the environment. Showing some mentioned aspects in an example; we start with a physical spaceship connected to a virtual environment. In this scenario, the toy can replicate its data in the game environment. Such data may relate to toy appearance and its movement, so, when a user moves the toy, a replicated virtual spaceship moves in the environment. Besides, if a user tilts the toy, it can create virtual shots, attacking enemies in the environment. Then, while flying across space, the same spaceship may collide with asteroids. Thereby, destroying them in the environment. Furthermore, to present feedback of hit damage, the toy may augment spaceship health points, flashing a LED displayed on the toy. Thus, as we presented, the interactive aspects simplified concepts in short terms. Therefore, selecting terms, the model described what the interface actions in the environment.

Concerning access to information, the environment may be private or shared. We considered access to both physical interface elements, and other environment elements (e.g., stats, scores, and items). People are co-located or remotely located in the environment. In addition, they may socially interact through competition, collaboration, or taking parallel actions. People physically interact with things in four perspectives. First, user can visualize things, and then, manipulate things or part of things. Moreover, they can interact through embodiment as moving their body, or use body information (e.g., heart rate). Finally, people can immerse in a smart playground, by interacting with its surroundings. Therefore, to enable such interactions, system make use of I/O technologies. These are, displays, such as screens, LEDs, or projectors; handheld devices, as smartphones, tablets, and smart toys itself; wearable technology, including clothing, accessories, and sensors; and through connective technologies of the Internet of Things (IoT).

3.2 Course Schedule

The relational model aims to describe playful system setups of multiple characteristics. Hence, it can describes playful activities with both fixed and open rules, located either indoor or outdoor environments, including on-screen display or none. Therefore, due its synthesis aspect, the model presented as an appropriate tool to define hybrid system concepts. Motivated to put into practice, we inserted the model in a project-based class. We taught the course in two multidisciplinary classes at UFPE campus. The undergraduate class, we called U1, joined 23 students from bachelor courses of Computer Engineering, Computer Science, and Design. The second class, we referred as G2, reunited 8 students from a graduate program in Computer Science, including both master and doctoral degrees. Students from G2 class had experience in fields of technology, design, and publicity. Besides, participants, from both classes, declared little or no experience in developing hybrid games. So far, a group of students participated in similar projects, such as 2D/3D digital games, AR applications, and Kinect games. In addition, subjects were familiar with game engines, graphic editors, digital prototyping platforms, and 3D modelling software.

The course schedule had 16 weeks, divided in 2 h meetings twice a week. The curriculum comprised initial stages of conception, ideation, selection, concept refinement, and low-fidelity prototyping. After these stages, students produced an initial documentation. Then, we started the prototyping cycles, including playtesting sessions, and a final documentation. Besides, we evaluated the prototypes every two weeks, providing individual guidance for each project. Moreover, we performed complementary lectures and workshops of several design and technology topics. A few students along with specialist guests led these complementary activities. In the first class, we introduced hybrid play products field giving examples from both market and literature. During 2nd week, we presented the model concepts and terms, so, we conducted an exercise. For this coursework, we asked students to previous research 2–3 system examples, and then, described them using the model.

Furthermore, we used Marco et al. [8] approach in the ideation stage. The authors proposed to take inspiration for hybrid games, observing how children play with both traditional and technological toys. Thus, we requested to students bring toys to class. The goal was to use them in a brainstorming session that we named ‘Brainstorm Toy’. We adapted Brainstorm Toy from a creative technique called Discussion 66 or Phillips 66 [9]. In these, participants discussed ideas following a sequence of questions; in addition, method proposes rotating subjects in small groups. The goal is to stimulate an exchange of views, to avoid participants to fixate on a single idea. We defined a sequence of questions aiming to extract from toys both physical features and their semantic value. Hence, questions included how people played with toy, what are game genres related to toy thematic, what the physical features of the toy, then how to improve toy features, and how a toy can interact with other toys.

In brainstorming, we divided students into groups of 4–5 participants each. Then, we shared a set of toys among groups. Both opening and closing sessions had 15 min, and rotating sessions last 10 min each. After every rotating session, we exchanged both students and available toys among groups. For the closing session, we reunited initial groups to compile ideas for selection. Thus, we asked students to select 2–3 ideas of their interest, to produce pitch presentations using the model. Hence, subjects used the model to refine game concepts, and to standardize pitch presentations. To assist us in joining final groups, we requested students to inform their skills and abilities in an online sheet. Later, in pitch sessions, participants voted for their three favorite ideas. Despite preference votes, we allocated students in groups based on the list of abilities. Finally, the U1 class composed four groups of 5–6 participants each, and G2 class organized two groups of four students.

At the 4th week, students refined game concepts using paper prototyping. The goal of low-fidelity prototypes was to define core game mechanics and name physical interface elements. Before practice, we reintroduced the model, detailing its individual aspects, and giving examples of each term. Then, we requested students to prototype concepts, following their selected model aspects. We guided each group, and all teams prototyped playable mechanics. In consequence of this stage, students produced the first version of their game design document (GDD). We requested GDDs in two moments, the first document needed to focus on prototyping schedule, including a list of required materials, and individual tasks of each team member. While, the final version covered updates of game balancing, setup changings, and design improvements. We provided to students a template based in a GDD structure proposed by Tim Ryam [10]. Then, the relational model appeared in opening section of document template. The section had to contain a visual representation of prototype model, along with a descriptive text of selected aspects.

The prototypes cycles started at the 5th week, in all stages, we supported students in acquiring or borrowing materials. Participants requested things such as 3D prints, displays, cameras, gaming devices, and electronic components. We evaluated three versions of functional prototypes; also, we guided students on their ongoing work presentations. The students developed their 1st. playable versions implementing features of initial GDDs. We recommended them focusing on physical interface working aspects, and in how integrate selected technologies. For the alpha prototypes, we required that games incorporated ending assets, playable gameplay balancing, and the interface in full operation. Then, students used alpha prototypes in the public playtesting sessions.

The playtesting sessions occurred in two contexts, a closed test, and a public test. Thus, initial sessions happened in class with students and a few guests. Then, in public sessions, end-users experienced the prototypes. Initial tests aimed to collect technical feedback, and to adjust game setups to avoid complications during public sessions. While, public playtests had a goal to validate game experience with players, also collecting feedback from them. Thereby, we taught lectures on how to conduct user testing, including how to elaborate data collection tools. Hence, each group prepared an approach of data gathering, and we evaluated proposed tools. Such approaches, included pre and post testing questionnaires, semi-structured interviews, and individual or collective evaluations. Afterward, students presented test results in class, thus, each group listed points for future improvement in beta versions. Therefore, our last evaluation concerned student’s beta prototypes along with their updated GDDs. Besides students produced a closing presentation, including a demo video of gameplay prototype.

3.3 Assessing Model Usefulness

One month after classes, we conducted semi-structured interviews with student’s representatives of each group. The goal was to assess model usefulness in several stages of the creative process. For selecting participants, we consulted GDDs revision history, so, we identified which of students described their prototypes in documentation. Then, we emailed them to schedule interviews, also asking students to confirm their part or to suggest another representative. We recorded all interviews to transcription; so, we performed free translation from Portuguese to English. Moreover, we used open coding procedure to analyze data on texts, enabling us to point out topics and similar views among participants. Thus, we established an interview script in 11 main questions along with auxiliary questions, or probes, to aid in collecting hidden data. We defined the script aiming to assess how students comprehended and experienced model and its terms. Thereby, we asked questions on what class materials students used to apprehend model terms, and what they think about term’s descriptions. Then, we included questions to assess model usefulness, such as how they used model during project stages, how groups communicated using its terms, and if they would recall any terms. Besides, we elaborated questions on other course methods, such as the Brainstorm Toy, paper prototyping, documentation, and playtesting sessions. Our goal was to distinguish model usefulness from other course resources.

4 Results

We divided research results into three sections; first, we presented how students used the model in initial design stages. Then, we showed results on prototyping cycles, followed by student’s outcomes of playtesting sessions. Finally, we discussed data from subject’s interviews, and presented how their feedback aided us to improve the relational model aspects.

4.1 Modelling Student’s Concepts

The initial stages of course schedule were conception, ideation, and documentation. In conception stage, students used the model in a practical exercise of describing existing systems. Overall, subjects produced 50 exercise sheets, so, U1 made 37 sheets, and G2, 13. Then, in exercise, students described 35 single cases, yet, 26 sheets presented 11 repeated systems. Among replicated cases were Pokemon Go, Cubbeto, Toymail, Amiibo, BB8, and Cognitoys. During the brainstorming session, students generated 15 ideas in U1 class, and 10 ideas in G2 class. There, participants described original concepts in a single sentence, or in a sequence of references. For idea selection, students presented 13 pitch ideas in U1 class, and 6 concepts in G2. All presentations consisted of 4–6 slides, introducing an idea using the relational model and its aspects. Hence, students presented what were the things, other setup elements, a few topics and related images, along with a figure of their concept model.

After preferential voting, students selected six game ideas. Students from the U1 class named their four concepts as Cubica, BUD Monster, Forecastle, and Legends of the World. Then, G2 students elected two ideas as Stormstone and Undercroft. Thus, in puzzle battle Cubica, to perform game actions each player uses a Rubik’s cube. BUD Monster, a Kinect game, a user wears a haptic glove to experience both tactile and visual feedback from game events. Forecastle is a nautical themed board game where three players had access to dynamic map elements using boat figurines. Legends of the World simulated team battles of several creatures, there, players used character figurines and a card set. Stormstone consisted of a hybrid platform for Role Playing Games (RPG), using metallic figures to interact with a custom game map. In the arcade game Undercroft, to avoid virtual obstacles, a player manipulates an articulated toy on an active board.

Students modelled their six concepts while preparing pitch presentations, and in both initial and final versions of documents. Thus, the initial GDDs had 4–8 pages divided into 5 main sections with several subsections. These were an introduction including concept model, tasks schedule, required materials and components, core game mechanics, and user interface elements. Then, the final version of GDDs had 30–70 pages, these incorporated updates of first GDD sections, along with complementary content. The additional sections referred to arts and design assets, audio content, summary of prototype versions, and user testing reports. In following subsections, we detailed student’s models according to records in their final documents. Yet, due to limited pages in paper format, we selected three prototypes to detail. Therefore, to presenting model aspects, we adapted and translated text fragments of student’s GDDs. Thereby, results reflected how students experienced the model in classes. Besides, since using the model was an interpretative process, we recognized a few aspects that required adjustments.

Cubica.

Cubica (see Fig. 2) is a hybrid puzzle with a turn-based battle system. There, two co-located players compete as wizards, guiding mystic creatures using passive handheld devices. Then, each player manipulates a private Rubik’s cube, and both players visualize virtual game objects through a shared on-screen display. To activate game actions, players must arrange a single cube’s face following required color combos. Thus, a webcam captures the cube’s face after players positioning it in a passive fixed base. Then, when the system recognizes a valid combo, the cubes extend player’s in-game actions. Each action is analogous to a single color, so, the system can distinguish the cube faces through its central colored square. The colors and its corresponding actions are: blue to move characters, red to melee attack, orange to ranged attack, white for special attack, green to activate shield, and yellow to energy recharge. Besides color, players need to arrange combos to complete actions. For example, to move a creature, a player selects the face with the blue central square. Then, a user must arrange an adjacent blue square, to indicate movement’s direction. Therefore, when players activate valid combos, the cube creates game elements on-screen. Players may choose among three creatures of equivalent skills and strengths. Despite the combats, characters can suffer damage through dynamic thorns appearing on the arena.

Fig. 2.
figure 2

Student’s version of hybrid-gameplay model in Cubica

Full size image

Undercroft.

Undercroft (see Fig. 3) is a hybrid arcade game inspired by the Hole in the Wall TV game show. The game world is private, consisting of a dungeon full of virtual obstacles, as moving walls and traps. Thus, player visually access the game through an on-screen display, and manipulating a passive articulated toy. The goal is to create poses using the toy that matches with silhouettes of the moving walls. Hence, the player manipulates the toy articulating its body, arms, and legs on an active board. The articulated toy has magnets located on its foot, enabling it to stand-up on the game board surface. A webcam positioned on game board terminal captures toy’s outline in each round. Then, the system identifies through images if the pose was valid. The smart board extends all game elements off-screen, so, the game scenario comprises both virtual and real scenarios. To represent virtual elements, the board has a 10 RGB LEDs to update game elements proximity. Then, LED colors will vary as a warning sequence, in green to yellow, and red. Besides, the board has a motor mechanism to open a trapdoor, so, if the player loses all health points, the toy falls into the trap. Regarding game elements, the extended moving walls are analogous to elements of Nature, as water, fire, wood, and stone. Moreover, a player can collect extended power-ups, so, a user must position any part of the toy inside a range, where the desired item appears. For example, if the item comes from the upper-left corner of the screen, the player may create a pose enabling toy’s arms to reach it. Therefore, when player catch game element, the player destroy it in the environment. Such items can confer either defensive or offensive powers, then, a user can store a single item of each type at a time. To activate them, user press buttons located on the board, therefore, creating corresponding game elements on-screen. The four buttons set includes directional arrows, to make the character jump and crouch to avoid traps.

Fig. 3.
figure 3

Student’s version of hybrid-gameplay model in Undercroft

Full size image

BUD Monster.

BUD Monster (see Fig. 4) is a hybrid Kinect game inspired by Japanese monster movies, or ‘kaiju’ genre. In a private environment, the player replaces a giant marine monster in the game world. So, players access to game through embodiment using Kinect technology. Then, to provide immersion, the user wears a monster glove to augment feedback of game actions. The active glove communicates to system using Radio Frequency technology, and provides both tactile and visual feedbacks. Thus, the glove augments character attacks vibrating a rumble motor, and special attacks flashing three LED displays. The game goal is to destroy the main enemy tower; yet, player must destroy two shield generators first. Then, all three targets locate in different spots of a virtual map visualized in an on-screen display. Meantime, the player replaces the character walking through the map, terrifying citizens, destroying buildings, and attacking army tanks. The number of enemies on-screen relate to destruction level held by the player. Hence, if the monster destroy too many buildings, a giant robot appears from the sky, and user must defeat it. The player extend game actions through gestures, these are melee punch, special punch, laser eyes, and monster roar. In the screen, player can visualize monster’s arms, and it is analogous to user’s wearable. Besides, during gameplay, player can interact with several characters on dialogue boxes, as the monster boss, the city mayor, TV reporters, and the main enemy. Thus, the game ends when the monster destroys the main tower, or if the NPCs defeat the player.

Fig. 4.
figure 4

Student’s version of hybrid-gameplay model in BUD Monster

Full size image

Discussion.

Students used the relational model in stages of conception, idea selection, and documentation. Starting with conception, while completing exercise sheets, several students requested advice to select model terms. Considering that the students accessed general information of systems through images, videos, and text, they used limited data on both interface and gameplay. Although, in overall, they had selected system aspects consistently. However, analyzing replicated system sheets, we found inconsistency among student’s interpretations, especially on things-environment axis. We expected this since it is the model axis containing original terms and concepts. Besides, students had difficulty in selecting aspects of the people-environment axis in single-player systems. Moreover, due people-things axis comprehend its aspects in two columns, several participants made a few mistakes. For example, they selected ‘display’ as an I/O technology, without, selecting ‘vision’ as a physical interaction. Similarly, happened to ‘manipulation’ and ‘handheld’, despite both terms are not inherent, they tend to appear together in several systems.

Participants described and improved their concepts using the model during pitch selection. For this step, students got access to class materials, so, they consulted presentation slides, and we distributed hard copies of model’s article [1]. Therefore, students used the model efficiently while describing their ideas. For example, participants better-selected aspects of people-things axis since they could distinguish the terms of two columns. Then, after this stage, students presented no difficulty in selecting both I/O technologies, and physical interactions modalities. In addition, several students started using the center of model’s triangle to name things present in their systems. Moreover, we noticed mistakes in things-environment aspects, yet, this time recurrent flaws related to students omitting existing aspects. For example, the concept model of Undercroft presented selected aspects of ‘create’, ‘destroy’, and ‘updated’, despite the system augment data on game board, students omitted this term.

We could recognize design changes among projects, comparing how students described their models in initial and final versions of GDDs. Thus, several prototype concepts involved multiplayer modes, and a large number of physical interface elements. Hence, the initial concepts predicted more interactive aspects than final versions. For example, the Undercroft expected to build two smart boards to promote competitive and collaborative game modes, later, they produced a single-player system. Furthermore, initially, the LEDs on smart board replicated the walls movements. In final version, they opted to extend the walls through the LEDs, duplicating available gaming area. Regarding the quality of model’s description, these were equivalent in both versions. However, the final GDDs were larger documents, and they included projects adjustments, so, during GDDs updates, a few groups made mistakes. For example, BUD Monster team selected aspects of ‘replicate’, ‘destroy’, and ‘update’ in initial GDDs. Then, such aspects described their prototype better than the final version. In final GDD, they omitted ‘destroy’ aspect, and exchanged ‘replicate’ to ‘replace’, despite the monster glove replicate on-screen. However, such mistakes not appeared in Cubica since their prototypes had minimal changes from concept to beta versions.

4.2 Prototyping Cycles and Playtesting Sessions

Prototype cycles comprised four main stages, including ongoing presentations within each step. The first version was the paper prototypes, in this stage, groups focused on core game mechanics and interface elements of systems. For example, Cubica team defined a grid map, and basic interface aspects, as for how to use the cube to move characters, and release attacks. Other three versions were the working prototypes, then, the 1st playable version required basic interface aspects and its technologies integrated to the core gameplay. In G2 class, the Undercroft presented the active board game, containing LEDs integrated to virtual obstacles, and a 3D printed articulated toy, yet, they did not implemented color detection. Afterward, we required that alpha prototypes had an interface in full operation and a playable gameplay balancing. Besides, they had to incorporate finished assets, including animation of virtual elements, graphical user interface, and visual identity. The Stormstone, for example, presented a touch sensitive monitor and metallic figurines of playable characters for both heroes and enemies. Besides, the game had a private terminal, where the RPG master would customize game elements on the map. Then, students used alpha prototypes (see Fig. 5) in the public playtesting sessions.

Fig. 5.
figure 5

The alpha prototypes: (a) Undercroft, (b) Stormstone, (c) Cubica, (d) BUD Monster, (e) Forecastle, (f) and the Legends of the World.

Full size image

The public playtesting occurred in two sessions, so, the events received about 25–35 users, in both days. At least, each prototype collected feedback from 10 players. There, users pointed aspects on fun experience, engagement level, game balancing flaws, and interface features. Overall, users evaluated positively all games and according to gathered, their favorite prototypes were the puzzle battle Cubica and the Kinect game BUD Monster. Due technical issues a few prototypes performed partial or adapted functionalities. Both Undercroft and board-game Forecastle that used computer vision techniques for color tracking, found issues on environment lighting. Then, the Undercroft tested their prototype outside the event, and Forecastle exchanged a projector by a monitor. During tests, students gathered feedback from players using different data collection tools. Besides, several groups used a demographic questionnaire to assess player’s profile, such as their game preferences. Later, students presented test results in class, so, their presentations showed a list of proposed improvements. We evaluated their presentations and helped them in selecting adjustments for beta versions. Hence, final prototypes, or beta versions, were similar to their alpha prototypes, nevertheless, including design improvements in consequence of user testing results. Moreover, students produced playtesting reports to incorporate in final GDDs.

Discussion.

The paper prototypes were essential to defining basic game design decisions and interface I/O elements. Cubica team, for example, defined both movement and battle systems relating gaming actions to the physical puzzles. Then, these initial decisions grounded mechanics to other game actions in working prototypes. During prototyping cycles, all working versions incorporated improvements, and adjustments followed two main reasons. First, aspects related to planning and development challenges, second, in response to feedbacks of both ongoing presentations and user testing. For example, BUD Monster concepts planned four wearables; these were a pair of gloves and two monster caps, each one representing a character. The idea was enabling users to exchange characters wearing different costumes. However, they implemented a single working glove in time. Concerning development issues, the Forecastle team replaced the projector and physical map with an on-screen display. This, due projector emitted a light that was interfering color tracking of boats on the map. Besides, several prototypes exchange their physical interface materials. Stormstone conductive points used paper foil, magnets, and copper wire. Finally, they replaced 3D prints with metallic figurines and insulation tape. Moreover, modifications promoted by user feedbacks, related to how system presented information in both graphical and tangible interfaces. Such improvements were changing position of visual elements, replacing pictograms of icons, and resizing text content.

Furthermore, the playtesting sessions enabled to assess fun experience promoted by each prototype. Despite functional issues, students of both classes produced playable prototypes. Thereby, their core game mechanics resulted in fun experiences for players, since test reports presented positive evaluations in this topic in all six prototypes. Cubica, BUD Monster, and Forecastle were among best-evaluated games. Thus, students were successful in creating meaningful interface elements that incorporated both interactive and semantic values. Cubica proposed a puzzle battle, where both mechanics and strategies, turned the toy fundamental to interaction. Moreover, the Undercroft team created level design based on physical features of articulated toy. Then, to define virtual wall silhouettes, they had to take pictures of the toy in several poses. Besides, the active board extended game environment, then, it duplicated available gaming area. We supposed such decisions were a consequence of model usage in early stages of development. Hence, to assess model usefulness, we interviewed students, so we presented and discussed its results in the following section.

4.3 Interview’s Results

We conducted seven semi-structured interviews with at least one representative student of each group. The interviews lasted 10–15 min, then, during transcription, we attributed codes for participants to ensure anonymity. Therefore, we referred to subjects in this paper using acronyms. According to gather, the model was useful to describe student’s prototypes. For example, participant U1A mentioned how model helped their group in determine game characteristics. Similarly, U1C claimed that model assisted in describing their project, since it was easy to visualize game aspects using its terms. Hence, for interviewers, the model facilitated in describing and in better understanding game setup. Thus, as G2G elucidated, “the model helps you to define game setup.” Besides, subject G2F commented that through the model was possible to visualize setup complexity, while recognizing the number of selected aspects in the triangle. In addition, subject asserted that the model supported in project planning such as cutting off elements and selecting both feasible and essential aspects. Analogously, U1D said, “initially, we wanted to select every aspect of the model, then, during development, we could know what elements supposed to stay, and which of them we could remove.”

According to students, the model helped them during initial stages of development. For example, participant G2F recognized model as a fundamental tool to define hybrid game concepts. Thus, subject U1E declared that model aided in better defining system requirements, before implementing game functions. Besides, subject U1B commented on things-environment axis decisions, “it was good because we had a doubt while choosing extension or replication, then using the model, we could select one of those”. Similarly, both U1C and G2G mentioned how the things-environment axis assisted their group in establishing game mechanics. Moreover, students incorporated model vocabulary, so, during interviews, several students used model’s terms while commenting on their projects. According to participants, the terms facilitated intergroup communication. For example, U1B claimed, “Since all students knew the vocabulary, made easy to use terms than formulating larger sentences to describe what interface elements would do in the system.” Analogously, U1C commented on term ‘extend’, “Just in mentioning the term ‘extend’, someone would ask, where it will extend? Will it use a screen? Will it use a projector?” In addition, interviewees claimed that vocabulary helped them to understand other group’s projects. For example, U1E said that when visualizing other group’s triangle was possible to name what they were developing. The student G2F alleged that terms were useful while talking to other teams, as on comparing how they implemented similar aspects in their projects.

Thus, when we asked on model usage in GDDs, several students mentioned its relevance in the initial documentation. They stated model usefulness to both describe and visualize system aspects. Interviewer G2F thought the model synthesis so useful, that subject suggested us to create similar mechanisms for the entire documentation. Furthermore, students cited other course resources in interviews; these were the Brainstorm Toy, paper prototyping, and playtesting sessions. Then, participants appreciated both brainstorming dynamics and pitch selection. Moreover, they considered paper prototyping a good practice to define initial requirements of their games. Subject U1D commented on the course schedule, “I really liked pitch presentations, as in using toys to generate ideas, also, I’ve enjoyed paper prototyping practice. The theoretical part was very important, but the practical stages were better. Besides, I found great that happened several playtesting sessions during course schedule.” In addition, students asserted on technology and design lectures, distinguishing topics on game engines, computer vision, and concept art.

Implications.

Based on interview results, we could recognize points of the model that required adjustments. Overall, students comprehended the three entities and its individual relationships; it is things, people, and the environment. Thereby, participants could distinguish passive from active technologies, access to information, and social interaction modalities. Despite, in their first contact with the model, students presented struggle to understand people-things axis, due it presented aspects displayed in two columns. However, while experiencing the model through the course, they could differentiate physical interactions from types of I/O technologies. For example, subject G2F commented that had difficulty in understanding such aspects since it seemed similar. However, after consulting class materials, the participant stated as “clear” the distinction among groups of aspects. Then, the same process happened with other four subjects. Concerning, people-environment axis, students presented difficulty in locating a person in single-player systems. In consequence, participants felt confused in selecting social interaction aspects of such systems. Interviewer U1B commented that their team implemented a single-player game, and they found issues since model only presented multiplayer aspects.

Regarding things-environment axis, students referred to aspects definitions as “clear”, “intuitive”, “easy to learn”, and “concise”. So far, several participants presented issues in selecting model’s terms. The aspects that caused more mistakes for students were ‘update’, ‘destroy’, and ‘augment’. For example, subject U1D commented on ‘destroy’, “At first, I did not understand if destroy was related to extinguish an object entirely, or it would apply to a simple game instance. Yet, after seen it implemented in other prototypes, it really helped me to understand its meaning”. Several students alleged that examples of aspects in existing systems assisted them in difference model concepts. For example, when G2G referred to the first model exercise, “During exercise, where you showed several examples of existing systems, it helped a lot in understanding the concepts.” The, participant U1C claimed, “Yes, the main resource was the examples. Using the examples was easy to know what to do or not, and if our game had an aspect or not”. Besides, interviewer U1D mentioned that despite information in the article, the visual material with examples was very elucidative and important. Therefore, according to students, while visualizing existing systems, the aspects presented more clearly.

To address student’s issues, we promoted adjustments in all three axes of the model (see Fig. 6). Starting in people-things axis, we replaced term ‘immersion’ with ‘pervasiveness’ since the word ‘immersion’ incorporated multiple aspects on game experience. For example, the BUD Monster team selected ‘immersion’ in their model, despite their interpretation contrasted the aspect’s meaning. Our goal was to select a term to represent systems where people interact with things and its surrounds, such as in smart playgrounds. Hence, we considered the term ‘pervasiveness’ appropriated to describe the disruptive concept of such systems. Besides, in people-environment axis, we included the term ‘single’ to describe systems with one player, therefore, representing systems without social interaction among people.

Fig. 6.
figure 6

Hybrid-Gameplay Interaction model update

Full size image

Observing student’s prototypes, we noticed that current terms were missing a few interactive aspects. Therefore, we added two new terms in the things-environment axis. Hence, we included ‘activate’ to represent when things send/receive a single action to the environment. In Undercroft, ‘activate’ would describe actions where a user presses a button on the game board. Thus, a user would both activate power-ups and activate character’s movement functions. Besides, we inserted ‘transform’ to represent when things change an existing game object in the environment. Thereby, an interface element may transform other game object identity into a new game object. Moreover, ‘transform’ can refer to changing information of an existing object, such as its position, visual features, and state. We defined such term as a complementary action to both ‘create’ and ‘destroy’. Afterward, we organized all nine interactive aspects into three groups; these are how things are self-represented, how things act on other game objects, and how things send/receive data in the environment. Thus, a physical game object, or interface element, can self-represent in the environment through replication, extension or replacement. Then, such thing may interact with other game objects creating, destroying, or transforming their information. Finally, things can send/receive data through an update, when things record such data; activate, by sending single data to the environment; and augment, when system provide data feedback.

5 Conclusion and Future Work

We considered student’s outcomes satisfactory since groups have prototyped hybrid game concepts of meaningful interface elements. All six working prototypes promoted fun experiences during playtesting sessions, using either passive or active technologies. The relational model presented as a useful tool to both describe and assimilate hybrid game concepts, in early design stages. Overall, modelling concepts during pitch selection and in initial GDDs had helped students in defining their game setup, as the interactive aspects of interface elements and the environment. Furthermore, the model established a specific vocabulary that aided students in both communications among team members and with other groups. However, we recognized points of the model that required adjustments. Therefore, we proposed improvements based on results of seven semi-structured interviews held with students.

Besides the theoretical model, students positively evaluated other course resources and methods. Hence, the Brainstorm Toy presented as an appropriate tool to generate hybrid game concepts. Yet, we must refine idea selection procedure since, through preferential voting, students discarded concepts of great potential and semantic value. To solve such issue, we would provide an additional evaluation method to analyze interactive aspects of concepts, using model information in pitch selection. Moreover, in paper prototyping, we identified that when groups prototyped interface elements and its interactive aspects, they required little change in working development stages. Thereby, to assure that subjects will prototype such aspects, we suppose that the method could incorporate pre-defined I/O technologies resources including system data flow. Finally, we expect to put in practice the updated model, to both validate improvements and refine model usage in the creative process. Therefore, we will experiment other course formats, such as short duration workshops. Furthermore, we must focus model usage in initial design stages of conception, ideation, concepts modelling, idea selection, and low-fidelity prototyping.