Keywords

1 Introduction and Motivation

Many efforts have been carried out to come up with new learning methodologies as well as improve upon existing ones. Moreover, researchers have been investigating what other factors play an important role in supporting the learning process. Towards this direction, the development of teaching material that better reflects the characteristics of the subject matter and thus makes the learning process more approachable has been gaining a lot of attention.

Geometry is an important branch of mathematics [3]. Throughout elementary school, students are exposed to several geometry concepts: shapes and their attributes and shape classification using lines and angles. In hopes of supporting the learning process of geometry-related concepts, researchers have developed educational applications with the goal of helping students to better explore geometry concepts through dynamic manipulation of geometric objects as lines, circles, and dots [5, 7, 18]. These applications are often termed interactive geometry software (IGS; interactive geometry, IG; dynamic geometry environments, DGEs; dynamic geometry systems, DGSs; or dynamic geometry, DG). The fundamental premise of these applications is the learning-by-doing method where, through the manipulation of a geometric space, students can construct their knowledge [7].

There is evidence of the benefits of using IG systems in classroom [4, 6, 7]. For example, several studies have shown that using IG software to teach geometry, tends to result in more committed students in comparison with students learning with traditional tools, such as rulers and compasses [4]. Moreover, current research indicates that IG software encourages students to develop their own hypotheses and find new ways to solve proposed problems [1, 6, 9]. When using IG software, students interact with the subject being presented through the graphical user interface, being able to alter the contents on the fly. This rapid feedback can play an important role in the learning process of elaborate concepts [20]. However, the majority of the IG software developed to date are desktop-based. That is, the interactions between user and the system take place through conventional input and output devices such as keyboard, mouse, and monitor. It turns out that there is a shortage of material tailored towards teaching abstract math concepts that takes advantage of more recent technological advances: for example, when teachers have to impart knowledge about three dimensional concepts, they have to resort to two dimensional illustrations. We argue that in such scenarios not all students are able to fully grasp the implications of these concepts because they are ill-represented. Hence, these students might end up with a subpar knowledge of the subject at hand.

Many of the learning initiatives have been trying to tap into technological advances such as augmented reality to support the learning process [11, 12, 22]. One of the advantages of using augmented reality is that this technology can be applied to teach students of any age. Another advantage of this technology is that it provides students with an improved depth perception, which is not possible via orthodox learning materials such as textbooks. Consequently, we conjecture that augmented reality can be used to teach concepts whose comprehension is rather difficult without visual aids. We hypothesize that solid geometry concepts fall into this category of concepts that can be better assimilated by students when the learning process is supported by technology. According to Kaleff [8], visualizing abstract concepts is key. Kaleff states that it is important for students to differentiate between the flat representation of an object and its corresponding three dimensional form.

Fig. 1.
figure 1

User investigating an augmented reality object mapped to the marker.

Full size image

Given that spatial skills can be learned, we believe that students with poor spatial skills can benefit from our augmented reality environment. To evaluate the effectiveness of augmented reality in supporting the learning process of solid geometry concepts, we created a set of learning material that capitalizes on this technology. This material is based on the requirements elicited during an interview with an elementary school teacher. Essentially, this material is presented using web-based technologies, and augmented reality is introduced to allow students to better visualize the concepts. More specifically, our application uses marker-based augmented reality, hence students have to point the camera at the underlying marker; and only after recognizing the marker (i.e., while acquiring camera data) the three dimensional objects are shown by the application. This step is shown in Fig. 1.

We set out to evaluate our hypothesis by randomly assigning 24 participants to two groups: i.e., an experimental group and a control group. The participants in the experimental group engaged in an introductory geometry lesson using augmented reality, while the students in the control group were exposed to solid geometry concepts in a traditional fashion.

The remainder of this paper is organized as follows. Section 2 describes related work. Section 3 outlines the augmented reality environment we developed to support the teaching and learning process of solid geometry concepts. Section 4 describes the experimental design we used to evaluate our augmented reality based learning environment. Section 5 presents the results, statistical analysis, and compares the performance of the participants that were exposed to solid geometry concepts in a traditional way with the scores of the participants that engaged in a geometry lesson through our augmented reality environment. Section 6 discusses the threats to validity and Sect. 7 presents concluding remarks.

2 Related Work

Computer systems and human beings interact through a graphical user interface (GUI). Ill-designed GUIs (i.e., interfaces that do not meet usability criteria) often hamper how users interact and access the underlying functionality. Thus, users may end up performing wrong operations, thus reducing their productivity. In the context of educational software for teaching geometry, the development of interfaces can play a key role in how learners explore and understand the concepts shown on the computer screen. Nevertheless, despite the importance of a judiciously designed GUI, there are not many studies exploring novel ways of interaction with the user. That is, there are many under explored technologies that can be used to improved the interaction user-computer and the learning process as a whole. In [17] Reis et al. describe the state of the art in terms of the technology used to implemented GUI for IG systems. According to Reis et al., most forms of interaction with IG systems are either keyboard or mouse based (or both), while most of the information is shown in regular monitor screens. Few efforts have been trying to take advantage of more advanced technology as augmented reality.

Meiguins et al. [14] employed augmented reality to allow students to manipulate solids and Oliveira and Kirner [15] developed an augmented reality environment within which students can create solids by specifying several properties and then these solids can be visualized after pointing the underlying cameras to the pre-configured markers. The assumptions of these studies is that augmented reality is an effective way of imparting knowledge related to shapes that are inherently three dimensional. Nevertheless, both studies fail to provide evidence of the benefits of employing augmented reality to boost the learning experience.

Silva and Ribeiro [21] implemented a tool to support the teaching of spatial geometry: this tool represents three dimensional shapes along with their definitions. Silva and Ribeiro evaluated this augmented reality environment in terms of its usability (according to the participants) and whether or not the participants were able to learn from it. Similarly to the aforementioned studies, several other studies have also investigated the application of augmented reality to support and boost the learning process [2, 9, 16].

Kirner et al. [10] proposed an interactive book where the markers in each page allow users to visualize three dimensional objects as well as information about these objects. This book, named GeoAR, was tailored to elementary school students, hence covering only the main geometric shapes and related information. According to Kirner et al., retrofitting augmented reality to textbooks promotes students’ interest and engagement. However, the authors fail to report the results of the pilot study they conducted to evaluate their interactive book. They claim that the prototype yielded “very positive results concerning the educational potential of GeoAR”, but evidence of such benefits are omitted from the study. Martins et al. [13] devised an educational application geared towards supporting the teaching and learning process of geometry. Martins et al. also carried out an usability evaluation.

Although the aforementioned studies deal with improving the learning experience by exploiting augmented reality to engage and motivate students, none of these studies carried out a randomized controlled experiment to assess whether augmented reality contributes to the learning process. To some extent, most of the previous studies evaluate their approaches based only on the feedback of the students. One of the prime contributions of our study is that, apart from developing an augmented reality environment, we also evaluated the benefits of augmented reality in educational settings.

3 An Augmented Reality Based Environment for Teaching and Learning Solid Geometry Concepts

Our learning environment was devised based on an interview with an elementary school teacher at the target school. This interview shed light on the needs of whose ranges from eight to nine years. Moreover, according to the results of the interview, there is a shortage of learning material covering solid geometry concepts for elementary school students. In addition, the available material is not interactive, i.e., in the sense that it does not give the student the ability to manipulate the content. According to the interviewed teacher, when teaching solid geometry, teachers have to resort to elementary resources such as textbooks [19] in which the shapes are represented (i.e., discretized) as flat, two dimensional images.

It was this lack of innovative learning material that motivated us to try and bridge this gap by tapping into augmented reality to foster learning. After deciding on the basic functionality, we investigated what sort of technology could be used to implement the planned application. We developed a web-based environment in HTML (HyperText Markup Language) that enables students to visualize three dimensional shapes using augmented reality. Our augmented reality application was developed using the authoring tool Flaras (Flash Augmented Reality Authoring System). The objects rendered by this tool were selected from a open repository [1]. After putting together all these technologies we were able to create an interactive, augmented reality-based learning environment for teaching solid geometry.

Figure 2 shows the web-based module of our learning environment. By clicking on the several options at the top left side of the GUI the user can browse by content or the learning experience can be guided by the teacher. The user can, for instance, access the option that renders a tetrahedron or assess their knowledge in the environment’s test mode. The center of the web page shows the object that is currently being rendered. Overall, the environment was designed to increase student engagement during the learning process by promoting active, student-centered learning. The environment features four learning modules: one covering solid geometry concepts and three involving interactive games tailored towards enforcing retention of the underlying concepts.

Fig. 2.
figure 2

Web-based module of our learning environment.

Full size image

4 Experiment Setup

This section describes the randomized experiment we carried out to evaluate the benefits of using augmented reality to teach solid geometry. More formally, we set out to answer the following research question (RQ):  

RQ: :

Is augmented reality a more effective approach to teach solid geometry than the traditional approach?

 

The aforementioned RQ outlines the issue that this study is intended to investigate. As detailed in the following subsections, the RQ was used as the basis to formulate the hypotheses we used in this study.

4.1 Scoping

Experiment Goals. Basically, defining the scope of an experiment boils down to setting its goals. Towards this end, we used the Goal/Question/Metric (GQM) template [23]. Following this template, our experiment can be summarized as follows:

figure a

As mentioned, we are particularly interested in examining whether augmented reality is beneficial in introducing solid geometry concepts to children. More to the point, we set out to find if incorporating augmented reality can yield better results than the traditional method (i.e., expositive lecture) when used to impart solid geometry concepts to students.

Variable Selection and Metrics. Here we clarify the dependent variable we set out to measure. Since it is complex to gauge the effectiveness of the teaching and the amount/quality of learning taking place, we settled on evaluating the students’ knowledge by giving them a test on solid geometry concepts. So the operational definition of our variable is the score of the subject students on a solid geometry test: we will answer our RQ by comparing the score of students that were introduced to solid geometry concepts through an augmented reality tool with the scores of students that learned the same concept via the traditional method (i.e., expositive lecture).

Hypothesis Formulation. We formalized our RQ into hypotheses so that statistical tests can be carried out. Throughout this section we refer to the approach based on augmented reality as ARBA and the traditional approach as TA.

  • Null hypothesis, H \(_0\) : there is no difference in terms of effectiveness (wherein the quality of teaching and learning are evaluated in terms of how students fare in a test) between the two approaches (i.e., expositive lecture and using an augmented reality tool to introduce concepts to students) to impart solid geometry knowledge. H\(_0\) can be formalized as follows:

    $$\begin{aligned} \varvec{H_0} = \mu _{ARBA} = \mu _{TA} \end{aligned}$$

  • Alternative hypothesis, H \(_1\) : there is a significant difference in efficiency between the two approaches (measured in terms of the students’ scores), which can be formalized as follows:

    $$\begin{aligned} \varvec{H_1} = \mu _{ARBA} \ne \mu _{TA} \end{aligned}$$

Setup. 24 subjects participated in our experiment. The ages of these subjects range from 8 to 9 years old. To evaluate our conjecture, the subjects were assigned to the two different treatments at random: 12 subjects were assigned to ARBA and 12 subjects were assigned to TA. As mentioned, in this experiment, the main dependent variable is the scores of the subjects in a solid geometry test. More specifically, this dependent variable is defined in terms of the number of questions that they correctly answered.

Our experiment was broken down into four steps. These steps are listed in chronological order:

  • Pre-test: all students answered questions on solid geometry. This step lasted 8 min.

  • Random assignment: as mentioned, subjects were assigned to treatments at random.

  • Learning step: during this step each of the groups was introduced to solid geometry concepts. Subjects in the ARBA group were introduced to these concepts through a 10-minute interaction with an augmented reality tool. Subjects in the TA group attended a 30-minute lecture on solid geometry.

  • Post-test over the course of this step, subjects had to answer questions related to the concepts they were previously introduced to. We allocated 8 min for the subjects to go over all the questions.

Fig. 3.
figure 3

Pre- and post-test scores obtained by the subjects in the TA group.

Full size image

5 Analysis of the Results

Figure 3 shows the scores of the students in the TA group in the pre- and post-tests. As indicated in Fig. 3, the traditional method seems to be an affective approach to introduce students to solid geometry concepts given that most subjects retained the information imparted to them and performed well on the post test. As shown in Table 1, during the pre-test, subjects answered on average 5 questions correctly. Five subjects (i.e., s1, s3, s4, s6, and s7) did not perform well on the pre-test. In particular, s7 answered all questions incorrectly. Almost all subjects performed better during the post-test. During the post-test, on average, subjects answered approximately 9 questions correctly (Table 1).

During the pre-test (Table 2), the subjects in the ARBA did not performed as well as the subjects in the TA group did . On average, subjects answered 4.83 questions correctly (Fig. 4). However, the high standard deviation (3.46) indicates that the scores varied widely. Our results seem to indicate that using augmented reality to introduce students to solid geometry concepts can be an effective approach. During the post-test, that is, after being exposed to solid geometry through an augmented reality tool, all but three subjects improved their scores. As shown in Fig. 4, eight of the 12 obtained perfect scores on the post-test. Similarly to the TA group, on average, subjects in the ARBA answered 8.83 questions correctly during the post-test.

Table 1. Summary of the scores obtained by the subjects in the TA group.
Full size table
Fig. 4.
figure 4

Pre- and post-test scores obtained by the subjects in the ARBA group.

Full size image

Owing to the small size of the groups and given that the results do not follow a normal distribution, in order to compare each treatment (i.e., TA and ARBA) we carried out a non-parametric test: Wilcoxon signed rank test. More specifically, we compared whether there was a significant improvement between the scores obtained during the pre- and post-test in each treatment. It turns out that both treatments showed significant improvements on the post-test. The p-value for the TA was 0.013, while the p-value for ARBA was 0.009. Since that the p-value for ARBA was even smaller we conjecture that ARBA is an approach slightly more suited for presenting solid geometry concepts than TA, which is a static, chalkboard-based approach. However, the difference between these two treatments is not statistically significant. Thus, further comparisons are necessary to increase our confidence in the results.

Table 2. Summary of the scores obtained by the subjects in the ARBA group.
Full size table

Despite the results of our statistical analysis, we argue that augmented reality can be a positive addition to the classroom. According to the teacher, some of the participants in the ARBA group outdid expectations. It seems that the main reason is that students in the ARBA group developed more positive attitudes towards the material being exposed.

6 Threats to Validity

This section discusses the threats to validity of the experiment we carried out to evaluate the advantages of ARBA over TA. A threat to the external validity of this study is that it might not be possible to generalize the results of this experiment to a broader population because the sample size used was somewhat small. A possible threat to construct validity is that the questions we used to evaluate the subjects may result in a poor measurement of the subject’s knowledge of solid geometry concepts. Another threat to the construct validity is inadequate preoperational explication of the constructs [23]: that is, in the context of our experiment it is not possible to be sufficiently clear about what being “better” means. In other words, we did not elaborate on or evaluate whether using augmented reality leads to better short- or long-term retention. Basically, the benefits of both approaches were evaluated in terms of the scores obtained by the participants: that is, the scores obtained by participants were the only concept operationalized as experimental measure. However, test scores are an incomplete gauge of how much students have learned, they can be seem as a starting point, a basic means of comparison. Therefore, to better examine the advantages of augmented reality in this context, the scores of the participants should not be the sole factor involved in the investigation.

7 Concluding Remarks

Many students struggle with solid geometry concepts due to the limitations in the way these concepts are presented to them (i.e., three dimensional content is usually presented using two dimensional images). In this direction, and based on our believe that augmented reality has the potential to provide a myriad of benefits to the learning environment, we took an initial step towards probing into these benefits. An experiment using a control and a experimental group should be conducted throughout a period of one to three months. In this experiment, the control group should be exposed to the underlying concepts only through traditional instruction while the experiment group should also receive traditional instruction along with being exposed to augmented reality based material, which should be employed in expository and hands-on learning activities. This way it would be possible to focus on how much each approach helps the students to improve, taking into account the fact that students start with different sets of knowledge. Such an experiment will yield more conclusive results concerning the pros and cons of adopting either approach in the long run: that is, how much augmented reality contributes to student growth during the time students are in the classroom. Moreover, it is worth mentioning that using multiple measures further helps ascertain the effectiveness of each approach. Hence, as mentioned in Sect. 6, follow-up experiments should also gather other information besides student achievement gains: for instance, surveys could be carried out throughout the conduction of such experiments along with systematic observations of the students’ behavior in classroom. Such information can help to shed light on the advantages of augmented reality in comparison to the traditional approach to teaching inherently three dimensional concepts.

It is also worth mentioning that all efforts in this context must emphasize and employ user-friendly technologies, which are easier to adopt. Otherwise, adopting these educational applications may be difficult even for teachers willing to adopt innovative approaches as augmented reality.