Keywords

1 Introduction

Spatial skills have been found to play critical roles in students’ success in science, technology, engineering, and mathematics (STEM) fields [1,2,3]. However, training for developing spatial skills is too often overlooked, “because spatial thinking is not a subject, not something in which children are explicitly tested” [4]. In addition, traditional print-based materials have inherent limitations for presenting three-dimensional information for supporting the development of spatial reasoning ability. Existing digital technology allows us to explore the affordances of interactive programs (e.g., animation and simulation, augmented reality (AR), and virtual reality [5]) for spatial reasoning training. In this study, we chose to modify an existing AR application to explore what benefits AR can provide and what makes an AR app usable in the context of spatial skills training in middle school. This application was originally designed to teach spatial skills to college-aged students entering a school of engineering. However, research suggests that adolescence is a critical period for developing spatial thinking, and adolescents with stronger spatial ability are more likely to choose a career in STEM fields [2]. Therefore, we believe applications such as ours (see Fig. 1) that are designed for practicing and developing spatial skills can be beneficial to younger students if appropriately modified for their abilities and approaches to learning. This study describes our design and implementation steps for modifying the AR application for use by middle school students.

Fig. 1.
figure 1

The augmented reality application on a mobile phone

Full size image

2 Original Design Considerations

The spatial skills development app we used in this study, called Spartan SR, was designed to use augmented reality for a number of reasons. First, AR was deemed to be an excellent medium for spatial skills training since the environment creates a realistic impression of being able to interact with and manipulate digital objects in one’s own physical space, giving embodiment to spatial tasks that would be awkward or difficult in real life (e.g., rotating objects in perfect 90-degree increments and cleanly around any of the three possible axes). In addition, AR allows designers to integrate spatial skills development with familiar gamification elements (e.g., unlocking challenges, leveling up, high-score competition, personal bests, etc.) for added learning and motivational incentives. Finally, while there are different types of augmented reality currently available, the choice to use a mobile phone/tablet version was due to the ubiquitous nature of mobile phones and tablets, ensuring that the majority of participants had their own device capable of running our augmented reality program. (Note that we loaned devices for students who needed them.)

In addition, the Spartan SR tool has several constraint-based affordances that traditional printed materials do not possess. As noted, learners can precisely rotate objects 90-degrees on a single axis at a time, which helps them see clear connections between different object states and abstract spatial expressions. Spartan SR was also designed to allow teachers to create their own lesson modules to better fit their class curriculum, as well as access students’ progress data for formative learning assessments.

3 Motivations for Using Spartan SR with Younger Users

We had both theory-based and practical motivations for making the application usable for younger students. First and foremost, longitudinal studies show that spatial ability plays a critical role in developing expertise in STEM fields, and that spatial reasoning skill during adolescence is a salient attribute of individuals who go on to achieve advanced educational credentials and occupations in STEM [3]. Second, students around 11 years of age have been found to start becoming interested in STEM subjects, only to often lose this interest again a few years later, which is particularly true for girls [6]. This makes interventions with interactive and playful opportunities to foster the key skill of spatial reasoning and to maintain interest in STEM particularly relevant for learners during this critical window. From a practical standpoint, we recognized spatial reasoning to be a somewhat neglected area in secondary schools. For example, most standardized tests students take, such as SAT, ACT, and GRE exams, do not typically include spatial measures. This reflects a bias in favor of formal mathematics and verbal skills that may be rooted in teachers’ and education advocates’ own strengths and backgrounds [7, 8]. It is often not until college or beyond that future architects or engineers encounter the intricacies of spatial reasoning, and future mechanics or machinists have to pick up their needed spatial skills on the job [7]. We were also eager to tap into younger learners’ often high appreciation for smartphone apps and games when used in school and in class [9], and their appreciation for gamified approaches to learning, such as unlocking new challenges and levels.

4 Differences Between College and Middle-School Learners

In contrast to college students, the middle school users in this case study reported little to no prior experience with three-dimensional coordinate systems, although they did have some basic knowledge about two-dimensional (X and Y) axes. Similarly, the younger users had no previous exposure to the concept of positive versus negative rotations around axes and the related notation (such as “+X” or “−Z”).

According to Piagetian theory of cognitive development, problem solving skills in 10–11-year-olds are in the process of becoming more logical, but still depend to some extent on concrete events and objects rather than abstract and hypothetical ones [10]. Children aged 11–15 are right at the transition point where they move from trial-and-error approaches to systematically solving a problem in a logical and methodical way [11]. Since these developments do not occur at the exact same time in every student, sixth grade classes have a broad range of abilities and maturity levels. Conversations with teachers interested in using Spartan SR in their classes also described this age range as marked by often high energy levels, as well as mixed levels of task persistence and emotional maturity.

5 Guiding Principles for Design Modifications

To modify Spartan SR for younger users, we considered the elements of direct manipulation interfaces that Hutchins, Hollan, and Norman describe as gulfs to be bridged by effective design choices for successful user interface interaction to occur [12, 13]. The gulf of execution describes the cognitive effort required to have the system do what you want it to do, whereas the gulf of evaluation refers to the cognitive effort required to make sense of the output displays. To create a sense of directness in interacting with the system, cognitive effort for both is to be minimized.

In the case of Spartan SR, we wanted users to experience various objects to be rotated and compared to a target as though they were moving them in real physical space. Our objective was to make their learning experience as authentic as possible, and to give them a sense of control over the rotation of the objects, within the intentional constraints provided. While we considered enabling direct swiping of the objects on the phone as a way reduce the gulf of execution, we chose to maintain the use of the six rotation icons (three axes in two directions) to support the deliberate practice of mental rotation between the stages of perceiving the object’s state and acting upon it.

6 Design Modifications and User Reactions

Given earlier experiences we had with college-age users [14], we felt we needed to make modifications to the app in order to make it more suitable for younger users. Our modifications were related to both evaluation and execution processes in order to maximize practice and feedback with the crucial mental rotation step in between the two. In what follows, we describe the designed changes along with student feedback regarding the efficacy of these changes.

To gather this feedback, we introduced the modified app in three six-grade classrooms over the course of three days. Across the three 50-min class periods, students were asked to work through a series of increasingly challenging games, involving the rotation of objects of varying complexity around the three axes of the coordinate system. For example, during early rounds, users had to match the rotated position of an object to that of a smaller identical target object. A more advanced game asked them to achieve a certain rotation by clicking one or two rotation buttons before the object would be responding. On the fourth day, thirty-three of the participating students (18 males, 15 females; 11–13 years old) took part in one of eight 4–5 person, 50-min focus groups. Questions centered on their experience using the app overall; on specific features, levels and games within it; and on any suggestions they had for future iterations of the design.

6.1 New Bridges Across the Gulf of Evaluation: UI Practice Opportunities with Familiar Objects

In order to use the application effectively for practicing mental rotation, a user must first perceive the initial orientation of a presented object in space. To meet the younger users’ preference for more concreteness, we redesigned the initial practice rounds to include familiar objects such as a simple house, a pickup truck, or a bench (see Fig. 2). User comments indicate that this approach was helpful in building up to more abstract, “engineering-like” random shapes:

Fig. 2.
figure 2

Everyday objects (e.g., a house as shown in the above figure) were used in our redesign to meet younger users’ need for more relatable and concrete objects.

Full size image

“I think the everyday objects (helped) because they’re just objects that you know and they’re easier to predict what the other sides would look like. Because if you’ve just got a square face on one of your shapes, there could be like a weird jagged curve on the other side of it.”

“The everyday shapes did help you out a lot because you see them every day, everywhere and but the harder ones, like the box ones and stuff, those really made you think, like think ahead, two steps ahead about what they look like in a different form, different angle.”

“Well, when you did the beginner (everyday objects), it was easier than intermediate and expert, so it got harder, but you’re already kind of ready. All the other lessons built you up.”

6.2 New Bridges Across the Gulf of Execution: Intuitive Button Design

For the younger users, we wished to scaffold a growing understanding of what rotations around the three different axes looked like. Since learning formal notation was not the goal for this age group, we replaced the original user interface buttons (see the upper-left corner of Fig. 3 for the original rotation notation) with icons of curved arrows going around small segments of the axes, color-coded to correspond with the provided coordinate system model (see the upper-left corner of Fig. 4 for the redesigned icons that correspond with the color-coded axes). Overall, these more intuitive icons used as button labels worked well for the middle school students:

Fig. 3.
figure 3

The original rotation notation (upper-left corner).

Full size image
Fig. 4.
figure 4

The redesigned rotation icons (upper-left corner).

Full size image

“The arrows helped a lot because otherwise, I wouldn’t have no… I would have no idea which way it would go.… Instead of like trying to memorize which buttons do what.”

“I used the buttons because it would say like, oh, it’s going around a red pole or over the blue bar. And I thought that was really useful.”

However, perhaps because they lacked previous exposure to the third dimension (Z-axis) of the coordinate system, some confusion remained regarding the axes over the course of four sessions:

“The Y axis was easy to figure out but I’m like I know what to do but I couldn’t figure out which one to click.”

6.3 Combining System Cues with Embodied Cognition Guidance

Anticipating we would need to explain the axis system in terms the students could relate to, we redesigned the in-class orientation to the app to connect movement in Spartan SR to movement in the real world. We did this by comparing the axis to physical, full body movements such as somersaults, cartwheels, and pirouettes (see Fig. 5). Some students found this full body embodiment explanation helpful: “Since I’m a gymnast, I really get that, so (for) the one that you are supposed to turn it like this, I would think of it as like a cartwheel or like a full turn or like a front or backward flip.” However, not all students showed equal enthusiasm for this approach.

Fig. 5.
figure 5

One of our researchers introducing the AR application to one of three sixth grade classes

Full size image

We also offered an alternative hand-only embodiment explanation, asking students to imagine picking up objects and rotating them by hand following the indicated rotation direction. In our case, there was a bit more appreciation of this approach: “If somebody put their hand along with the arrow and just tipped it, that’s the way it would go. So I used it, I used the buttons because it would say like, oh, it’s going around a red pole or over the blue bar. And I thought that was really useful.”

6.4 Providing Additional Strategies for Mental Rotation Practice

We anticipated additional challenges when working with sixth graders as compared to college students. In particular, we expected greater variety in maturity and self-regulation, and weaker problem solving strategies. To help with these challenges, one of our strategies was to include drawing activities with concrete objects. Doing so provided a buffer in terms of time (when some students completed lessons much faster than others), gave more concrete activities than just manipulating a phone interface, and different ways to engage when the phone app was too challenging. In addition, we provided more direct support for students when intervention seemed to be needed.

Direct observation led us to believe that these strategies were very important for this population of students. Some of them needed more support when encountering difficult tasks as well as guidance for how to handle times of significant frustration by particularly challenging task conditions (see Sect. 7.1).

7 Lessons Learned for App Modifications Across Age Groups

Our goal was to make the Spartan SR app more suitable for use with younger learners. We feel this goal of the modification was met, as evidenced by the positive experience and high engagement that the younger users exhibited. Focus group participants frequently mentioned appreciating the “real” appearance of objects in space and one’s ability to explore their position and control their movements.

“I liked that if you moved around the paper … you could see the different sides of the object which made it really cool.”

“It was really fun because it’s like you’re not actually like touching it in real life but you can move it and it’s like real life even though it’s not real life.”

The younger users also responded positively to the gamification elements of the app, more explicitly so than the original target group of college students. For example, one student described her response to the application’s stepwise unlocking of and checkmarks for progressively more challenging objects and tasks:

“It was addicting… Because like every time you get, beat a level, it’s like, yes, it’s going to the next one and if you mess up, you have to be like, no, I have to redo that, I have to redo and you just keep going.”

7.1 A Challenge for Design: Setting an Ideal Challenge Level

Apart from such positive feedback overall, students gave mixed responses to a particular version of a prediction game within the app. In the standard version of this game students had to decide all the moves to make prior to the object’s moving. In the advanced version, if a single part was wrong, they would have to start from the beginning. While students generally recognized the prediction game as an activity that really got them to mentally rotate objects rather than using a trial and error approach, the advanced level requiring perfect execution tested the students’ persistence and would have lowered feelings of self-efficacy without encouragement and buffering statements on the part of the research team/instructors. Practical suggestions made included allowing the user to choose a desired level of challenge (such as unlocking higher levels even if the current level was completed with errors).

“It was frustrating when I got super far and then I had to restart but whenever I messed up.”

“You had to be patient with it. You had to (…) calm yourself down or else you’d get too frustrated. Yeah. It made me frustrated a little bit but I actually, it gave me a challenge and I like challenges.”

Importantly, regarding the question of whether one can improve one’s mental rotation skills through practice, some students indicated that frustrating moments within the games and errors made throughout were useful in getting better.

“I mean, I think all of us sort of did (improve their skills) in a way, so I think if we did, then anyone could.”

“Your brain hurts, (but in) a good way.”

These responses suggest that the use of the app was supporting the constructive mindset of the targeted skill as amenable to growth through practice rather than as fixed [15], particularly with this group, which was not something we had expected.

7.2 Additional Takeaways for Future Design Modifications

For the future developments of this kind of AR application for use with middle school students, we took insights both directly from the user feedback and indirectly from overlaying relevant theory and user interface design principles with our experience in the new use context. One category of user feedback was the desire for competition:

“I liked the leader board idea… where you would compete against your classmates if you sign in on the same account and you can like, there’d be a leader board and you could try and compete for first place and you’d know who’s in first place.”

“I liked the time thing. I think what you could also do because a lot of kids our age and even older kids, they like competition, try to be like the best at something. So maybe in the corner, you could have a leader board or something.”

Another category of feedback was regarding object design. That is, the adjustment to add practice objects that users would be familiar with was not only well received, but also encouraged further in a common suggestion for what would make the application better for this age group: “If they added more realistic shapes that you’d see in real life.”

Based on theory and user interface principles in connection with our observations, we took away additional ideas: First, to provide a brief video tutorial up front that is easily accessible initially and as a reference after getting started. In Norman’s (1986) terms, “Visual presence can aid the various stages of activity. Thus, we give support to the generation of intentions by reminding the user of what is possible. We support action selection because the visible items act as a direct translation into possible actions.” Second, an application such as ours could provide a replay function that supports a learner’s self-correction, aiding “evaluation by making it possible to provide visual reminders of what was done” [13] Third, since a competitive spirit was mentioned as typical of 6th graders by many, but not all participants, we consider options based on motivation research indicating benefits of an alternative to the leader board element of gamification: Providing the goal and feedback on beating one’s own time over the course of repeated use of the application, or having leaderboards for different “leagues” to avoid constant direct comparisons between students of widely differing ability instead [16, 17].

8 Summary

We found that in general our design modification of the application was a success for the audience of sixth grade students. Particularly, we encountered benefits from the use of intuitively understandable objects, as well as from scaffolding initial evaluation processes as users needed to focus on getting oriented within the application, with the three axes and directions of possible rotation, and the act of doing so in one’s mind. However, based on feedback from the students some aspects of the application require additional changes, such as allowing for errors in completion of game levels and making interface buttons for the rotation of objects more intuitive. Overall, students reported and demonstrated that the majority of this application was motivating even in the face of the challenges the students encountered while practicing this skill that is so valuable for STEM learning. Based on our experience described here, we plan on building on the successful elements of gamification and address the problematic areas so as to improve the user experience of this kind of application for various levels of age and preexisting experience with the application’s content.