Keywords

1 Introduction

Research has shown that Educational Robotics (ER) can be used as a tool to enhance teaching [1] and learning [22], from early childhood [4] to tertiary education [3]. ER provides “an experimental platform for practice” [18] and improves students’ motivation, engagement and learning [26]. While ER can be viewed as a tool fitting many and varied disciplines [17], it is most commonly associated with Computer Science [10] and STEM related disciplines [3, 18] such as mathematics [30]. Papert, who instigated the learning theories on constructionismFootnote 1 [24], was one of the first to employ ER for mathematics, in the 80’s. He used the LOGO programming language to teach geometry and found that ER improves children’s motivation, learning and interaction in the classroom [24]. Given such premises, it is thus surprising to find that only a limited number of studies explore the benefits of introducing robotics into formal mathematics education [11, 29]. In 2019, Leoste and Heidmets [21] conducted a longitudinal study with students from 20 classes, confirming that the use of ER in mathematics lessons improved students’ learning outcomes on a national standardised assessment. Their results are coherent with recent studies on the use of the “Concreteness Fading” method in mathematics: Kim [19] found that starting with physical activities that include manipulatives (such as ER) and “gradually fading concreteness to access abstract concepts” effectively supports “students [access to] conceptual understanding in mathematics classrooms”. These findings suggest that ER could play a pivotal role in a “Concreteness Fading Strategy” to improve learning outcomes in the formal mathematics curriculum.

Despite its numerous benefits, the use of robots in formal education settings, as opposed to extra-curricular activities, is still relatively sparse [3], both in terms of research and practice. A recent review by Zhong and Xia [30] on the use of educational robots in mathematics education concluded that more research was required “to further explore the integration of robotics and mathematics education”. Progressing in such research requires facilitating the introduction of ER into regular classrooms. Unfortunately, teachers, who play a determining role in the classroom, are often preoccupied by time [6, 10] and need to be assured that the use of Educational Robots will help reach the learning outcomes without incurring in a loss of time. Furthermore, the research is well aware of the importance of providing teachers with adequate guidelines for activity design [14] and intervention [5], to support the alignment of ER learning activities with the learning outcomes of the curriculum. However the reality is often far from these principles, leaving teachers to face the difficulties of integrating ER activities in their practices alone [8].

In an effort to contribute to the study of effects and modalities of the integration of ER in mathematics formal education, in this article we specifically address the following research questions: 1) Should an ER-based theoretical lecture precede, succeed, or replace a traditional theoretical lecture? 2) What is the students’ perception of, and engagement in, the ER-based lecture and exercises? 3) Do the findings differ according to students’ prior appreciation for mathematics? I.e., can ER help broaden the number of students successfully engaged in the discipline? The methodology devised to investigate the afore-listed questions (see Sect. 2), and the results herein reported (see Sect. 3), constitute the main contributions of this article. Additionally we provide in open-source all the ER-based pedagogical resources devisedFootnote 2 and used in this studyFootnote 3.

2 Methodology

A key requirement for the assessment of ER learning activities in the context of formal mathematics education is the presence of adequate ER content. Section 2.1 outlines the learning unit considered for this study and describes the proposed ER-based theoretical and exercise activities for a geometry lecture. Details about the study participants and experimental design are provided in Sect. 2.2.

2.1 Design of the ER Content

In our study, we consider the case of the formal mathematics curriculum at the level of secondary school in Switzerland. A common practice for Swiss mathematics teachers is to start with the theoretical introduction of a concept and then proceed to paper-based exercises. Students are often free to choose the order in which to do the exercises. Based on this pedagogical approach, and existing material, we thus designed an ER-based theoretical introduction lecture for the curriculum topic of “planar geometric figures” and a set of related ER-based exercises.

Choice of the ER Platform. All the learning activities we designed rely on the Thymio II robot [23] (henceforth referred to as Thymio), which was chosen because (1) its structure and shape is well suited for attaching a pencil and making it draw geometric figures; and (2) it is presently being introduced into classrooms in the region [10] and thus already familiar for a number of teachers. Moreover, Thymio has been successfully employed as an educational tool in a variety of settings, ranging from primary school [10] to universityFootnote 4. This versatility is rendered possible by the spectrum of programming languages that can be used with it (including three block-based visual programming languages: Blockly [23], VPL [23], Scratch [25]; and a text based programming language, ASEBA [23]). Scratch was selected for the present context as it is particularly adapted for our target group (15-year-old students) [12] and has already been used in various studies to teach mathematics [12, 16].

ER-Based Theoretical Lecture. The ER lecture was designed starting from the guidelines for the theoretical introduction of planar geometric figures provided in the official regional study planFootnote 5, and adhering as closely as possible to the way the teachers in the study introduce new mathematics topics. In the designed ER lecture, the teacher programs Thymio to illustrate the various concepts students are expected to acquire. Specifically, as the students are expected to understand what a regular polygon is, know its properties and how to construct it, the teacher illustrates the construction of complex polygons (e.g. hexagons, octagons...) by programming Thymio and making it draw polygons on a sheet of paper. Additionally, as the students are also expected to recognise and name the various angles in parallel linesFootnote 6, two Thymios are concurrently used to demonstrate the relationships between these angles. The material prepared for the lecture is available at 10.5281/zenodo.4649842. The lecture was designed to last 90 min, which equals the duration of the corresponding traditional theoretical lecture.

ER-Based Exercises. Starting from the 31 exercises present in the curriculum, we designed 6 robot-based activities which, while functionally equivalent to their pen-and-paper counterparts, are centred on the programming of, and/or interaction with, the Thymio robot for their resolution. As commonly done for robotics activities, our ER exercises envision that students work in pairs, a setup known to foster collaboration and often preferred to individual settings [15]. To detach the proposed ER exercises from programming, students were provided with pre-filled code snippets, and thus required only the basic programming skills that were covered during the ER introduction. Once a code snippet is loaded for execution on the robot (i.e., assumed by students to be the right answer), its correctness is immediately and directly assessed by observing the figure drawn by the robot. This is an interesting feature of ER activities, as the benefits of feedback mechanisms allowing learners to verify the correctness of their solutions have been shown in previous studies [2] and are typically lacking in traditional pen-and-paper exercises.

Validation of the ER Content. The designed ER lecture and exercises were submitted to 5 teachers with diverse background and experience to ensure their alignment with learning objectives and methods. Specifically, our experts included one pre-service teacher, three experienced in-service mathematics teachers and a teacher who transitioned to research on digital education.

2.2 Participants and Study Design

The study was conducted in a public school in Switzerland, with two classes of grade 11 students (15 y.o.) taught by two different mathematics teachers. A total of 26 students participated in the study (16 boys and 10 girls), most of which had no prior experience in robotics, CS or Scratch-based programming. The overall outline of the study is reported in Fig. 1, with each class being split in two to avoid a confound between the teacher and the order effect. As anticipated, both the traditional and the ER-based theoretical lecture lasted 90 min, while the exercise session (in which students are presented with traditional and ER-based exercises) lasted approximately 225 min (5 teaching periods), corresponding to the time allocated to exercises in the official study plan. Details on the assessment tools used during the study are reported in Table 1. The surveys administered focused mainly on interest [27] and perceived utility [20] to gain insight into the students’ intrinsic motivation [27] to introduce ER into mathematics formal education. Each question is administered on a 7-point Likert scale (score between –3 and 3, 0 being neutral).

Fig. 1.
figure 1

Study design with each class being split in two to have half the students of each class in Condition TR and the other half in RT.

Full size image
Table 1. Summary of the data collected during the study (tests, surveys and roadmap)
Full size table

RQ1 - Should an ER-based Theoretical Lecture Precede, Succeed or Replace a Traditional Theoretical Lecture? Our hypothesis is that starting with a concrete experience (i.e. with the ER-based lecture) and moving to a theoretical lecture helps the students have a better “conceptual understanding” of geometry. To verify this hypothesis, we designed a between-subjects experiment comparing the condition Traditional-Robotic (TR) (i.e., starting with the traditional lecture and moving on to the ER-based lecture, shown in purple in Fig. 1) with the condition Robotic-Traditional (RT) (i.e., starting with the ER-based lecture and moving on to the traditional lecture, shown in orange in Fig. 1). Differences between the two conditions in terms of academic performance are assessed at three points in time (see Fig. 1). Test 1, administered prior to the start of the experiment, assesses students’ knowledge in the geometry concepts identified as pre-requisite for the considered unit. Test 2, administered at the end of the first theoretical lecture, assesses students’ understanding of the presented content, and is similar to test 3, administered at the end of the exercise sessions. All tests are based on the assessments in the official curriculum. RQ1 was therefore evaluated by checking the learning gains computed for Test 2 and Test 3 with respect to the baseline provided by Test 1, between the two experimental arms. To mitigate a possible teacher effect, half of each class was placed in condition RT and the other half in TR, with the responsible teacher giving the traditional lecture and a researcher (the same for both classes) giving the ER-based one. Students were assigned to conditions to ensure they had similar distributions of competency in the discipline based on their performance in the course, as assessed by their teachers.

RQ2 - What Is the Students’ Perception Of, and Engagement In, the ER-based Lecture and Exercises? Our hypothesis is that students would find the ER-based lecture and exercises interesting and useful, manifesting in high engagement in these activities and interest to integrate robotics into other mathematics lessons in the long term [21]. Students’ appreciation for the proposed ER-based activities was assessed with a within-subjects experiment and via two complementary approaches: an objective assessment of their behaviour during the exercise session and a subjective assessment of their perception of the ER-based theoretical lecture and exercises. Throughout the exercise sessions students are free to decide which (among the 31 traditional and 6 ER-based) exercises to address, in which order and in which manner. Students could in fact pick a traditional exercise and solve it without using the robot (referred to as “traditional”); an ER-based exercise and solve it using the robot to validate their solutions (referred to as “robotics with robot”); or, lastly, the same ER-based exercises that the students decided to solve without using the robot (referred to as “robotics without robot”). In the Roadmap, students were thus asked to report the order in which they did the exercises and, for ER-based ones, whether they used the robot or not. Students’ appreciation for the ER-based theoretical lecture and exercises was measured in terms of perceived interest and usefulness, respectively with the Robotics theory survey and the Roadmap, which also allows for the analysis of students’ behaviour. At the end of the experiment, students were administered a final Perception survey, to evaluate their perception of the proposed ER-based content from the perspectives of interest, collaboration, facility (with respect to solving the exercises), effort and future interest (with respect to including robotics in future mathematics lessons, as well as lessons of other disciplines). Each of these items corresponded to a minimum of 2 questions, to acquire a more reliable estimate of the construct from the students [13]. Internal consistency is calculated using Cronbach’s alpha [7].

RQ3 - Do the Findings Differ According to Students’ Prior Appreciation for Mathematics? Our hypothesis is that ER would interest students and help engage those that are generally less invested in the mathematics curriculum, and thus compensate for differences in terms of prior mathematics appreciation. To investigate this question, we rely on the data collected for RQ1 and RQ2, categorising students based on their reported liking of mathematics on a 7-point Likert scale (13 do, 7 don’t). More specifically, we consider differences in performance (learning gain), the level of engagement in the exercise sessions and the overall perception of the robotics-enhanced geometry lecture.

3 Results

3.1 RQ1 - Lecture Type and Students’ Learning

Test 1 was administered prior to the start of the interventions and was used to ensure that the students in both conditions had similar levels of prior knowledge (Kruskal Wallis test fails to reject \(H_0\), \(p>0.05\)). The students’ performance in Test 2 and Test 3 exceeded 80% for both experimental arms (Test 2: \(84\pm 11\)% for condition RT, and \(84\pm 15\)% for condition TR; Test 3: \(86\pm 8 \)% for condition RT and \(87\pm 11\)% for condition TR). No significant difference was found between the two conditions (Kruskal Wallis test fails to reject \(H_0\), \(p>0.05\)). The finding thus seems to suggest that the ER-based lecture is equivalent to the traditional one in terms of students’ learning, with both groups being sufficiently prepared to move on to the exercises after their first theory lecture. The lack of significant progress from test 2 to test 3 might be due to a ceiling effect in the test as to its focus on fundamentals.

Fig. 2.
figure 2

Students’ interest and utility assessment of the ER-based lecture. Significant Kruskal Wallis tests are indicated with p-value, H statistic and Cohen’s effect size (D).

Full size image

3.2 RQ2 - Students’ Perception and Engagement

Figure 2 reports the students’ perception of the ER-based theoretical lecture, in terms of interest and utility, measured via the Robotics theory survey. Students in the RT condition, who started with the ER-based lecture, perceived it as significantly more interesting (Kruskal Wallis p = 0.0073, H = 7.3, D = 1.43), and, although not significantly, also more useful than those in the TR condition, who did it after the traditional lecture. This finding supports the results of RQ1 in suggesting that the two types of lectures are equally valid in transmitting relevant knowledge to the students, with those in condition TR thus finding the ER-based lecture of little interest and utility. Conversely, one of the teachers expressed a preference towards the TR condition, described as closer to the current practice and both manifested interest towards ER-based mathematics.

Fig. 3.
figure 3

Order in which the exercises were conducted.

Full size image
Fig. 4.
figure 4

Number of exercises conducted by each student.

Full size image

To investigate students’ engagement with the ER-based exercises proposed during the exercise sessions, we extracted from their Roadmap documents the order in which the activities were done by students (shown in Fig. 3) and the number of activities each student did individually (see Fig. 4). In both analyses, we distinguish between traditional exercises, ER-based exercises solved without using the robot (“robotics without robot”) and ER-based exercises solved using the robot (“robotics with robot”). Figure 3 shows that most students started with “robotics with robot” exercises and finished with the traditional exercises, with few “robotics without robot” exercises being conducted overall. Since only 6 ER-based exercises were designed, the figure suggests that most students not only did many of them, but also did them in block, before transitioning to traditional ones. Indeed, Fig. 4 shows that all the students conducted at least one ER-based exercise with the robot (“robotics with robots”, \(\mu = 4.15 \pm 1.53\)), with only 6 students engaging in “robotics without robot” exercises. Moreover, since the students who did the largest number of exercises also did all of the “robotics with robot” exercises, it would seem that the time spent on ER-based exercises was not detrimental for their overall engagement with the exercises. It is important to note that 5 sessions were allocated to exercises (as per curriculum), and several students missed one or more of themFootnote 7, leading to lower-than-average number of exercises conducted within the allotted time.

Fig. 5.
figure 5

Students’ interest and utility assessment of the ER-based exercises. Significant Kruskal Wallis tests are reported with p-value, H statistic and Cohen’s effect size (D).

Full size image

Figure 5 reports the students’ perception of the ER-based exercises, in terms of interest and utility, comparing the ratings of “robotics with robot”, “robotics without robot” and “traditional” exercises. “Robotics with robot” exercises were perceived as interesting (\(\mu = 0.5 \pm 1.5\))), while “robotics without robot” and traditional ones were rated more negatively (\(\mu = 0.0 \pm 0.7\), \(\mu = -0.33 \pm 1.2\) respectively). This contributes to a significant difference in interest between the ER-based exercises done with the robot and the traditional ones (Kruskal Wallis test \(p=0.0011\), \(H=10.7\), \(D=0.64\)). Similarly, “robotics with robot” exercises are perceived as useful (\(\mu = 0.6 \pm 1.6\)) and significantly more so than the traditional ones (mediocre utility, \(\mu = -0.9 \pm 0.8\)) and “robotics without robot” ones (\(\mu = -0.02 \pm 1.4\)). “Robotics without robot” exercises are not only judged less favourably that their “with robot” counterparts, but also perceived as less useful than traditional activities, which suggests that the role of the robot in the ER-based exercises was meaningful, allowing for the creation of novel exercises relying on different modalities to convey and verify a same content.

Fig. 6.
figure 6

Students’ perception of the ER-based content. Cronbach’s \(\alpha \) internal consistency of the scale is calculated for each construct and shown in parenthesis.

Full size image

In the perception survey, students evaluated the ER-based content from the perspectives of interest, collaboration, facility, effort put in, and future interest (see Fig. 6). Students had a globally positive opinion of the ER-based content and reported high interest, facility and effort with respect to integrating robotics in the geometry lecture. Although collaboration and future interest obtained slightly lower scores than the others constructs, the results remained globally positive (\(\mu = 0.9 \pm 1.9\) and \(\mu = 1.0 \pm 1.6\) respectively). However, some students observed that the robot “lacked a bit of precision for the constructions”. Indeed, the robot’s motion accuracy was not-always meeting the requirements of the application, which, together with some connectivity issues (causing the robot to skip certain instructions), caused some frustration for the students.

3.3 RQ3 - Effect of Prior Appreciation for Maths

The level of engagement in the exercise sessions (extracted from the Roadmap) was compared between the students who liked and those who did not like mathematics, to verify the effect of this variable on our observed ones. No significant differences were found between the students in terms of number of “robotics with robot”, “robotics without robot”, nor traditional exercises they engaged in (Kruskal Wallis test fails to reject \(H_0\), \(p>0.05\)). Similarly, no significant differences were found between the students in terms of knowledge acquired, both at the end of the theoretical lectures and at the end of the unit (Test 2 and Test 3, see Sect. 2.2), although it is possible that they were affected by a ceiling effect. Figure 7 compares the responses given to the final Perception survey by the students who liked and those who did not like mathematics. No significant differences were found between these groups (Kruskal Wallis test fails to reject \(H_0\), \(p>0.05\)) for collaboration, facility, effort and future interest. Conversely, students who don’t like math perceive the ER-based content as less interesting than their classmates who appreciate the discipline, a finding that can be read as the proof that students were not fooled by the novelty introduced by the robot and perceived the ER-based content as geometry content. These findings (albeit limited in validity by the low number of students not liking mathematics) would seem to suggest that a hybrid between traditional and ER-based content has the potential to engage students who generally don’t like mathematics as much as those who do. As a consequence, these students might possibly improve their competence in and appreciation of mathematics. Indeed, it would be interesting to verify this hypothesis in a longitudinal study, concurrently tracking students’ perception of mathematics and of Robotics.

Fig. 7.
figure 7

Students’ perception of the ER-based content depending on prior maths appreciation. Significant Kruskal Wallis tests are reported with p-value, H statistic and Cohen’s effect size (D).

Full size image

4 Discussion and Conclusion

This article investigates modes and benefits of the introduction of Educational Robotics in the formal curriculum of secondary school mathematics, specifically focusing on the 7-hours long learning unit about planar geometric figures, that students in Swiss schools address at grade 11 (15 y.o.). The study, involving 26 students from two classes, started with the preparation of a 90-minute long ER-based theoretical lecture and 6 ER-based exercises, validated by 5 experts to ensure their alignment with the learning objectives of the unit and state-of-art teaching practices. Rather than designing a fully robotics-based geometry course, our objective was to include a limited set of activities where the robot had an added value. Indeed, robotics should be considered as a tool, an extension to traditional paper-based methods, but not a total replacement. We specifically investigated (RQ1) the role of ER-based theoretical lectures, with respect to traditional ones; (RQ2) students’ perception of and engagement in the proposed ER-based lecture and exercises; (RQ3) whether the findings of RQ1 and RQ2 differ according to students’ prior appreciation for mathematics.

To investigate RQ1, half of the students started with the ER-based theoretical lecture and the other half with the traditional lecture, and then switched. Results showed that both groups of students, after their first lecture, reached a similar (and similarly high) level of competence, suggesting that ER, more often associated with exercises than theoretical lectures, can be as effective as traditional means to introduce abstract concepts. While the order in which students received the two types of theoretical lecture had no impact on their learning, likely due to the double exposure, the students evaluated the ER-based lecture more positively when done first. As different hypotheses can be made as to what caused these differences in perception, future studies should be envisioned to further and specifically investigate this topic. Students showed a generally positive perception of the ER-based content (RQ2), both through their assessment in terms of interest, utility, collaboration, facility, effort and interest for future integration and their behaviour during the exercise sessions. Indeed, most students started with, and engaged in all of, the ER-based exercises, despite the limitations of the platform sometimes frustrating their efforts. We hypothesise that a key reason for this success, to be verified in future studies, is that ER allows the student to be actively engaged in the exercise and provides immediate visual feedback that helps avoid the fear of errors and judgement. Lastly, results were generally similar between students who had declared liking mathematics, and those who had declared not to (RQ3), encouraging us to investigate in a future long-term study whether ER could possibly have a positive impact on students’ competence in, and perception of, mathematics.

In more general terms, the findings highlight the importance of considering, and doing so as early as possible, the alignment between the learning outcomes and the robotics artefacts [14], the requirements posed by the classroom context [28], as well as discipline-specific ones (e.g., the need for precise localisation required by geometry). While our preliminary findings should be verified in broader and longer studies, the most important result of this study may be its standing as proof not only that ER can be introduced in formal education, but also, thanks to the increasing efforts to train teachers [9, 10], that this can be done in a way that allows research and teaching practices to coexist and mutually benefit from their interplay within a context of translational research.