Keywords

1 Introduction

Outdoor play has been proven to benefit children’s development [1]. The specific features in an outdoor environment which not only be naturalized with plants, trees, flowers, water, sand, animals, and insects but also be rich with a wide range of physical objects and play equipment such as building, cars, statues, swings, slides. When outdoors, children can construct on an expanded scale, explore the world at first hand through a great deal of running and chasing experience, as well as experience natural phenomena such as the changing seasons, the light and shadows [2, 3]. Fjortofit et al. [4] pointed out that children who play in a flexible environment tend to be healthier. By playing outdoors, children engage in social and physical interaction both with competition and coordination, which develop their empathy, increase their social competence and strengthen their relationship with the physical world [5,6,7]. What’s more, these benefits are of importance as they play a significant role in children’s development of future adult-life [10].

We live in a digital-rich society where digital technology plays a curial role in our lives. These mobile applications in various forms that link the physical world with digital information are changing our practice and the way of perceiving the world. It has been proven that not only adults but also children are exposed to these technologies extensively [9]. Some research evaluated that although these digital contexts provide children various opportunities of an immersive learning experience, it also highlights the massive tension between increased digital learning and decreases physical exploration and interaction with the surrounding world [14, 15].

To solve the tension described above, the HCI community focused on the domain includes Pervasive games, Exertion games, and Heads-Up games to shed light on the decrease in children’s outdoor exploration [8]. By embedding novel technology into social and physical interaction experience, more immersive and intriguing outdoor exploration activities have great potential of changing natural outdoor play routine and facilitating children’s engagement and motivation in the outdoor experience. Augmented Reality (AR) application, which provides an interactive and digital enhanced view of the physical world, has been reported to success in shifting human mobility patterns. The main feature of AR application is the use of a visual marker, which enables users to link these markers with some specific objects or location and gain different types of information at the same time [11]. This novel technology has the potential to enhance outdoor exploration and learning is already used in the museum, art exhibition, field studies like identification of plants species and educational environment [12, 13]. This entertaining and novel way has the potential to bring real-world into digital-enhanced games that promote the outdoor experience for children.

In this paper, we designed a prototype of an AR application that utilized shadows from physical objects as AR marker and applied it in the context of children’s outdoor exploration experience in a treasure hunting mechanism game. We conducted user research to understand how children reacted to this prototype and how technology could be used to motivate children into exploring the open-air environment and strengthen children’s relationship with the real world. The main contribution of our research is utilizing more insight on using shadows from physical environment objects in marked-based AR application to enhance the connection between digital knowledge and physical world, which support naturalistic and unobtrusive play experience considering children’s perspective. This study provides insights for future research on technological-enhanced outdoor games for children, as well as inspiration and lessons for mobile augmented reality (MAR) development.

2 Related Work

2.1 Head-Up Games

Head-Up Games (HUG) are technology-based outdoor games to promote outdoor interaction with the minimal support of digital devices and screen time [8]. HUGs are a subset of pervasive games that, in contrast to games that mainly use mobile devices as gaming interfaces, does not force players to stick to screen, thus encouraging physical activity and personal engagement. Federica. et al. developed ABBOT [17], which combined a smart tangible object with a mobile application to access new content related to the discovered natural elements.

Another unique feature of HUGs is guiding children with creative thinking. RaPIDO, a sensor-based prototype, as well as an accompanying platform for rule changing [19]. By changing parameters such as buzzing duration and the number of participating teams, children could create various outdoor games from RaPIDO, based on their ideas and concepts. Admittedly, HUGs are practical approaches to enhance children’s outdoor play, especially with the feature of no-screen-based interaction and self-exploration settings. In this research, we concentrate on applying novel technology to conventional mobile applications which have the potential of blending the physical and digital world to enhance interactive experience for children, reaching to naturalistic and unobtrusive play experience.

2.2 AR Games

There is a rapidly growing interest in AR application in children’s education [18,19,20]. Research has concluded that AR technology can become innovative tools that combine physical experience, virtual content to develop children’s imagination and creativity. Since early research on MAR focusing on interaction design [11]; nowadays, these applications are widely used in various domains. Salmi et al. developed a scientific AR-application [21] which visualizes the invisible physical phenomenon like ‘Doppler Effect’ and ‘Molecule Movement in Gas,’ proved to stimulate the interest and situational motivation in the science domains. Amy M. Kamarainen et al. constructed EcoMOBILE project [22], which uses an environmental probe with AR technology in a local pond environment to aid middle schoolers’ understanding and interpretation of water quality measurement. There is a wide range of previous research focused on using MAR in nature and environment exploration areas. Oh and Byun [24] have presented an augmented reality learning system called “the interactive flower garden” that makes use of interactive agents in the augmented picture and allows learners to interact with the agents. Current research seems to validate that MAR may provide positive impacts on children’s nature subjects education. However, the definition of outdoors can be described as an open and constantly changing environments, where it is possible to experience freedom, gross and boisterous movements, and also physical elements. Given that, we focus on an interactive solution that can use MAR to combine the physical world and digital information in a playful outdoor experience for children.

2.3 Interacting with Shadows

Shadows are ubiquitous around our everyday life, where there is light, there have shadows. Shadows can be seen as a natural information affordance, providing the shape of objects surrounding us [16]. Therefore, we are used to recognizing our surroundings by glancing their projected shadows, which provides a new way of discovering nature and natural environments. What’s more, shadows are sensational which are differences from silhouettes, transforming 3D objects into 2D illumination, in consequences make shadows changeable and cunning. Meanwhile, many researchers in HCI have focused mainly on shadow interactive installations [23] such as shadows as input device and shadow transformation through various parameters, without considering about young children’s conceptions and creativity of shadows, and particularly the conceptions of very young children (aged below six) in a novel environment exploration approach. In this paper, we seek to utilize shadows as a medium to bridge the physical world and screen time.

2.4 Visual Marker

Visual markers, such as RFID, QR code, barcodes are essential graphic symbols that can be recognized with machine vision through accessing to cameras in mobile devices or tablets. Visual markers may also be utilized without graphical symbols. Ismo et al. [26] designed a mobile augmented reality application in which physical items from nature are used as AR markers to enhance children’s nature exploration. Their findings from user interviews suggested that the concept has sound potential in its mixture of physical activity and educational elements in an outdoor context. The main problems of conventional visual markers with mobile devices are their appearance which related to technical looking and has the potential to lead to be obtrusive to children. To solve this problem, design markers which use natural objects may have the chance to use it naturalistic and unobtrusive.

2.5 Main Features of Our Work

Given that our focus on creating an engaging outdoor interaction experience that engages children in observing shadow from physical objects, we concentrate on the aspects that select shadows from the real world and use them as visual markers for MAR. Despite previous research, which focuses on technical issues on image recognition with AR application, our research concentrated on the user perspective and engaging exploration experience for children. We utilize shadows as an unobtrusive and mysterious clue in a treasure hunting game mechanism to create a playful outdoor game for children. Besides, rather than in a fixed nature environment, our work seeks to expand the novel exploration towards the whole physical world, which includes schoolyards, daily routine, botanical gardens, and so on.

3 ShadowHunter

The concept of ShadowHunter is solving the tension between increased addiction to mobile games and decreased outdoor physical exploration in children. Our main goal is to design an unobtrusive and intriguing approach which uses shadows, a natural phenomenon, as physical objects markers to be able to provide contextual information on the real-world objects to children. We also seek to use this MAR way to encourage children to pay attention to their living environment and explore the real world with the mobile AR application.

3.1 Conceptual Design

The paper describes a novel application of AR concept into children’s outdoor exploration, where children can not merely observe the physical phenomenon and combine these abstract images with real-world objects to form their conceptual explanations, but also be encouraged to explore surroundings which are too familiar to ignore.

The game follows the style of treasure hunting games which players ought to find hidden treasures by following a serious of clues. The concept of ShadowHunter is that children should match given riddles, which are represented as shadows with real physical objects. Thus, they can obtain random shadow monsters as a reward (see Fig. 1).

Fig. 1.
figure 1

The game designed follow the style of treasure hunting game where children should match given riddles (shadows) with real physical objects, thus they can obtain random shadow monsters as a reward

Full size image

The designed game starts by given shadows of one familiar objects around children everyday life. Because of shadows’ unique feature that link 3D and 2D visualizations into one combined visual and information display. The given shadows are different from silhouettes, which to be riddles. Once the mysterious shadow is given on the tablet, each child embarks on treasure hunting through their familiar environment to find appropriate physical objects (facilities, buildings, sculptures, plants, etc.) by pointing the tablet camera at the object. The game identifies the object, gives information (i.e. videos/text/animation concerning that physical objects) and presents a shadow monster of this object designed by other users as a reward if the shadow matches the object. If the shadow does not match, the children may be given extra information to simplify the exploration. The players can also design their shadow monsters to gain energy for their monsters as well as enrich the game interaction. The monsters can be cultivated by energy children earned by either design popular shadow monsters or exchange shadow monsters with others. After the matching object has been found, children will be able to claim other shadow and embark on the following mission follow the style of treasure hunting game.

3.2 System Design

The prediction-observation-explanation (hereinafter called POE) model proposed by White and Gunstone [25] seeks to promote children’s conceptual changes by confronting their prior knowledge through three tasks: prediction, observation, and explanation. This system provides opportunities for children to predict circumstances, clarify discrepancies or congruencies between their prediction and observation.

The applications of the POE model have been widely implemented in a computer-based environment. However, little research has reported the integration of POE model into a game-based context concerning scientific education.

In this paper, we endeavoured to follow POE strategy in AR game design concerning outdoor exploration experience as a game mechanism. As shown in Fig. 2, according to the prediction aspect, the game provided a shadow which functions as clues for children to predict which real physical object it belongs. Considering the deformation of shadows, it can be indeed challenging and confusing. Children proceed to the observation part upon finishing the conceptual speculation.

Fig. 2.
figure 2

POE model of ShadowHunter

Full size image

The observation aspect consisted of seeking with game playing activities. In the seeking activities, the player is assigned a task that explored their outdoor environment to identify objects in a natural world environment. For example, if provided a shadow of a seesaw, the player should speculate and identify it through the tablet camera. As feedback is curial for knowledge improvement and skill acquisition in an educational context. We try to provide an exciting and emotional appealing experience, rewarding (e.g. Collect one random shadow monster/gain energy) and challenge (e.g. impose a time limitation or a competitive system) are embedded into the game.

For the explanation aspect, the players are given an attractive introduction of the physical object educational information, which includes an animation video explaining vividly on how the shadow deforms with sunlight and how this physical object functions in our daily life if they identified the object successfully. If they failed, they could replay the observation aspect with additional information to encourage their exploration.

The game application is implemented in Photoshop, Unity 3D and Qualcomm Vuforia platform. Seven image targets representing AR markers are used in the application. All of the physical objects were photographed straight from a top-down perspective and imported to Unity through Vuforia’s Target Manager. All of the markers, physical objects and black-white images were Vuforia image targets in Unity, meaning that each marker is identified only with one 2D image. The majority of the markers gained five stars in Vuforia’s target quality rating, means it can be easily recognized.

The videos designed by After Effect and exported in PNG sequential frame were imported, as well as audio resources in this unity projects and coded to follow concept setting.

4 Preliminary User Research

4.1 Research Process

With our preliminary prototype, we tried to confirm on how children reflect, feel and think about ShadowHunter and how this system impact on children outdoor exploration, as well as how can this system motivate children’s outdoor activities. Thus, we conducted a game-type experiment in September 2018 in a kindergarten (Singapore). With the cooperation of principal and teachers, the game site was chosen to be located in the yard of a kindergarten, considering the level of difficulties and original design intention of encouraging children to explore the surrounding environment. We designed seven shadow images and several shadow monsters in the yard (see Fig. 3). The yard consisted of a rural environment and natural plants.

Fig. 3.
figure 3

Game map. We chose seven objects and collected their shadows in yard at 12 am.

Full size image

We collected specified shadow images in advance into Unity database to output designated ShadowHunter application which fit our game-type experiment, regarding variable of shadow formation and practical usability.

The game-type experiment consisted of the following stages and qualitative user study methods:

  • Demonstration of ShadowHunter to children

  • Presentation of game experience

  • Direct observation of children

  • Semi-structure discussion with children (prior experience of ShadowHunter with tablets, how they feel about the game etc.)

  • Group discussion with teachers (potential use in teaching, identified problems, future work

All users showed great interest in the interaction process, some users said that when they experienced the AquaBot, they were surprised that they could control the underwater robot without touching the water. P3 said the system was very interactive and provides a new water gaming experience. P6 said that this system gave her a new understanding of underwater robots. During the test, we found that the task of controlling the robot’s movement was the most popular, and some participants experienced this task many times.

4.2 Methodology

A qualitative approach was chosen as it is a valid approach when trying to achieve a deep understanding of the behavior of individuals and groups. The data collected from semi-structure interview with children and researchers’ group discussion, which can yield in-depth information on both learning and playing perspectives.

All the discussions were recorded and transcribed for discussion and future work. Sentences were collected as the unit of analysis and identified into meaningful categories. The records and observations were cross-checked by two researchers and three teachers (Table 1) to ensure validity of the findings.

Table 1. The composition of research team
Full size table

4.3 Participants

In the consideration of the target user of ShadowHunter, six girls and four boys ranging in age from four to six years (mean age = 5.2, sd = 0.92) were selected. We were aware of differences in participant’s cognitive abilities due to age and gender. In spite of that, we wanted to gain universal feedback from the assumed potential users that are children.

4.4 Procedure

Before the experiment, a letter was sent to children’s parents informing the research study and ask for their consents. The session was started by giving a demonstration and presenting the idea of ShadowHunter to children, which last about 7 min. After that, players were given tablets running ShadowHunter application and instructed to find objects which match given shadow images and collected or designed their shadow monsters in the surrounding area.

The children were divided into four groups of two or three. Each group received a tablet and task sheet of given shadow images which need to be matched with real physical objects. Children were asked to change roles during the game experience.

The main task for children was to predict the provided shadow which functioned as a clue to find one real physical object it belongs to, then examine their prediction with a tablet. The players were a monster as a reward, as well as animation videos and the brief introduction of this object if they identified the right object successfully. If they failed, they could replay the observation aspect with additional information to encourage their exploration. Each group was accompanied by one researcher or teacher, who take charge of observing and notes taking (see Fig. 4).

Fig. 4.
figure 4

Child interact with ShadowHunter

Full size image

4.5 Semi-structured Interview

After the game-type session, the semi-structured interview was carried out with children, who shared their game experiences and individual development ideas with researchers; the whole process took about 20 min. We utilized Smileyometer published by Read et al. to measure children’s fun and satisfaction. We measured enjoyment of POE modal: Prediction (conceptual speculation), Observation (confirm predication) and Explanation (gain multimedia scientific information) with the question of “How much fun was it to do that part?” The answer on the Smileyometer is re-coded to 1 (for awful) until 5 (for brilliant). Almost all the children were satisfied with the game setting and level of difficulties (see Table 2).

Table 2. The result from Smileyometer
Full size table

Following are some of the children’s comments:

“It’s so exciting that I can catch a unique shadow monster” (P3)

“Searching for was fun and so was to design my shadow monster… It was easy; there should have more challenging tasks” (P4)

“It would be great if I can share shadow monsters with my friends” (P6)

The interviews with children indicated that gamified approach motivated them in exploring surrounding areas and engaging outdoor play. They appreciated the game mechanics about feedbacks which encourage them to design and collect shadow monsters in an entertaining way to expand physical activities.

4.6 Group Discussion

Based on the above findings, two researchers and three teachers (Table 1) who observed children and took notes on the both experiment and interview process gathered for a group discussion to form a shared understanding on all the observations and confirm the efficiency and limitation of ShadowHunter.

The discussion themes were as followed:

  • First impression and observation

  • Pro and cons of using mobile technology in children’s outdoor play

  • The meaning of physical exploration education

  • Feasibility of ShadowHunter in preschool education

  • Future exploration and development ideas

The whole discussion process lasted about 45 min and findings were sorted out in order to build practical implications and development directions.

5 Result

The overall responses about ShadowHunter during the semi-structured interview and group discussion were summarized. In general, the use of the surrounding environment with AR technology was considered to be a promising approach for children and especially for their outdoor exploration experience.

Based on the session, we found that it was essential to conduct a brief introduction before the task so that children can grasp the idea and manipulate the application.

Furthermore, our observation broke through the prior studies concerning about children may be too excited to concentrate on external reality. One recognized challenge in the study was the feature of shadow, leading to space and time limitation. An interesting finding was that the children who kept the tablet played a leading role in the group. As was well known that the combination with physical activity and cognitively demanding tasks can support learning achievements. However, it requires further studies with a multidisciplinary approach.

An important observation was that the competition mechanics in gamified approach could empower and motivate those who are not good at this part. As discussed in group discussion, we highly agreed that role-taking, social interaction and coordination needs to be taken account when designing game mechanics and applying novel technology in children outdoor interaction field.

Although teachers have used a large number of educational applications in the classroom, they failed to facilitate a personal engagement in outdoor experience for children, especially the motivation of outdoor exploration. Teachers appreciated the fact that ShadowHunter was intended for children to explore in the real world, outside of the classroom. Some of the teachers were a bit hesitant and sceptical about new technology in education. However, the ShadowHunter application was considered a playful game that could encourage children’s outdoor exploration as well as raise their awareness of their surroundings as one teacher noted.

“In general, I am not interested in technology… there is a purpose; … the opportunity to support independent learning and physical exploration, especially strengthen their emotional connection with the physical world” (T2)

Based on our above result, we draw some practical recommendations regarding edutainment application design for outdoor exploration.

  • Use real-life objects as exploring materials

  • Stimulate physical activity with scientific phenomenon interaction for a multi-sensory experience

  • Considering the gamification mechanics, which includes both competitions and reward to encourage children to explore and learn.

  • Design the game process with consideration of the interference of the external environment, including weather, temperature, etc.

6 Limitations and Future Work

The shadow markers worked well technically, albeit with certain limitations. Use of such markers would provide unobtrusive experience in outdoor exploration experience. However, the most serious technical limitation with the technology used in this study lies in the process by which the markers are created.

Every marker has to be created beforehand one by one for the Vuforia Target Manager. Also, due to the unique nature of the markers, transferring between contexts is not straightforward.

A further challenge with our solution is that each shadow is unique, and thus, the same objects in different time are not recognizable due to its slightly changes. This poses an obstacle to more extensive use of physical shadows from real-world objects and calls for other technical approaches, such as computer vision and machine learning.

Also, our work has some methodological limitations. The data were collected from one small group of children and the teachers studied were working in the same school. Thus, we cannot draw any generalizable conclusions on the long-term value of ShadowHunter applications in children’s outdoor exploration, expect their initial satisfaction about this game. Also, we fail to conduct a control group to stress on our striking findings. Furthermore, the prototype is rough, and it also contained several technical issues which led to disappointing user experience to some extent.

7 Conclusion

In general, ShadowHunter was seen as a straightforward and intuitive way to interact with real-world objects that could be applied in children’s outdoor exploration experience. The advantages of ShadowHunter application from outdoor play are its simplicity and intriguing game which follow treasure hunting game mechanism as well as using natural phenomenon- shadows as a clue in outdoor activity. Using shadows from real-world objects as markers were seen as a good direction for the gaming experience, as it created a naturalistic and unobtrusive play experience to enhance with the open-air environment.

Compared to the previous systems, we focused to utilize shadows from physical objects as a medium to bridge screen game with physical activity in familiar surroundings for children. We followed the POE modal strategy to design the game and conduct a semi-structured interview to confirm children’s satisfaction through Smileyometer strategy. We also suggested practical recommendations regarding outdoor interaction for children based on our preliminary study. However, we acknowledge limitations to our approach that our system is limited by time and space, considering the feature of shadows. The future work should address developing the concept further and integrating it with more advanced visual recognition technologies for more extensive trials in varying in-the-wild contexts for outdoor exploration.

Overall we believe this research is a ground-breaking experience towards an unobtrusive and intriguing approach that uses shadows, a natural phenomenon, as physical objects markers to be able to provide contextual information on the real-world objects to children and facilitate their outdoor exploration.