Educational Robotics in Primary School: Measuring the Development of Computational Thinking Skills with the Bebras Tasks ^†

Abstract

Research has shown that educational robotics can be an effective tool to increase students’ acquisition of knowledge in the subjects of science, technology, engineering, and mathematics and promote, at the same time, a progression in the development of computational thinking (CT) skills in K–12 (kindergarten to 12th grade) education. Within this research field, the present study first sought to assess the effect of a robotics laboratory on the acquisition of CT-related skills in primary school children. The study also aimed to compare the magnitude of the effect of the laboratory across third- and fourth-grade students. For the purpose of the study, a quasi-experimental post-test-only design was adopted, and a group of 51 students, from third- and fourth-grade classrooms, participating in the robotics laboratories, were compared to a control group of 32 students from classrooms of the same grades. A set of Bebras tasks was selected as an overall measure of CT skills and was administered to children in both the intervention and control groups. Overall, the results showed that programming robotics artefacts may exert a positive impact on students’ learning of computational thinking skills. Moreover, the effect of the intervention was found to be greater among third-grade children.

1. Introduction

Project-based learning (PBL) is developing as a teaching and learning approach in which educators structure inquiry-based learning activities as early as in primary school [1]. Within this framework, students are involved in a series of collaborative and teacher-guided problem-solving activities that help them acquire new domain-specific knowledge and thinking skills that are useful in solving real-world problems, especially when technology is embedded to facilitate the learning process [2,3].

The use of robotics kits within educational settings allows teachers to design engaging learning activities that can stimulate students’ interest and motivation, mainly in disciplines related to science, technology, engineering, and mathematics (STEM) subjects, and their problem-solving skills, as well [4,5]. When applied to educational robotics, the PBL approach is intended to promote children’s constructionist learning through physical manipulation of artefacts that, in turn, is expected to stimulate the development of mental representations of the world around them [6]. Tasks designed according to the PBL approach allow students to create, evaluate, and revise concepts, while designing and programming the interaction with robots. More specifically, the use of robotics as an educational tool requires children to explore, observe, and manipulate their environment, and then reflect, design, and program the behavior of a robotics artefact to provide an answer to the proposed task. Indeed, the learning process develops on the basis of challenging tasks where the observation and comprehension of phenomena occur within an experimental setting rather than through traditional theoretical lessons. As a consequence, children act as active learners and are strongly encouraged to be protagonists of the learning process, while developing problem-solving and critical-thinking skills [5].

Within the broad category of robotics educational kits, some programming tools, like Scratch and Lego^® Education WeDo, introduce block programming as a visual method of coding that is particularly suitable for introductory programming lessons and the teaching of important computer science concepts, especially in the case of novice programmers, young student, and primary school education [7]. Block programming is based on drag and drop of consecutive visual tiles on the editing area to compose different instructions; first of all, it facilitates the learning of programming skills because of a reduction in the cognitive load potentially required for textual programming [8,9]. Moreover, Kelleher, and Pausch [10] recognize further advantages in the use of visual programming for novice learners. Indeed, the visual block-based instruction is built using icons that facilitate students’ recognition of its function, such as commands, conditions, and loops. Accordingly, visual programming allows students to focus their attention more on the logic and structures involved in programming tasks than in challenges related to writing code. Specifically, Lego^® Education WeDo [11] is a programmable building system that was originally designed to foster children’s learning in STEM disciplines and early development of computational thinking skills, through the implementation of a robotics artefact able to perform programmed actions.

Computational thinking (CT) was first conceptualized as a set of cognitive processes that allow people to solve problems, design systems, and model human behavior by relying on concepts pertaining to computer science [12]. Later, the author [13] specified that CT refers to those thought processes whereby solutions to problems are designed to be implemented by an information-processing agent. In order to transfer the theoretical definition of CT to teaching and learning issues in K–12 education, the International Society for Technology in Education (ISTE) and the Computer Science Teachers Association (CSTA) have recently proposed an operational definition that can be profitably applied for educational purposes [14]. According to this definition, CT is basically a problem-solving process involving several cognitive skills, such as logically organizing and analyzing data, creating abstract representations of data through the use of models or simulations, designing solutions in terms of ordered steps (algorithmic thinking), and, most importantly, generalizing the acquired problem-solving skills to a wide range of problematic situations. Similarly, the Computing at School (CAS) Institute identifies abstraction, decomposition, algorithmic thinking, evaluation, and generalization as the key CT skills to be mandatorily promoted in K–12 [15] Indeed, given the recently recognized relevance of CT as an effective additional tool for preparing students to deal with future challenges, the promotion of CT skills is now considered a pillar of the development of computer science curricula in K–12 education worldwide [16]. In this regard, the Italian Minister of Education, University, and Research has implemented a national plan for digital school [17], with Action 17, which was specifically intended to promote the introduction of CT teaching in primary education. Within this perspective, the student becomes a computational thinker who is able to apply, in an active and autonomous way, critical-thinking processes and resolve problems in a broad variety of disciplines and real-life contexts.

Robotics in education is mainly used to teach content related to nontechnical (language and sciences) and technical (robots and technology) subjects [4,18]. In addition to lesson plans focusing on computer science and programming concepts, robots are employed as mediators to foster students’ learning of science subjects, such as mathematics and physics. In this specific regard, research has shown that the use of robotics can be an effective tool in enhancing students’ learning of STEM concepts in K–12 education, with most of the studies focusing on students aged six and over [4]. In terms of individual skills, research suggests that thinking skills (e.g., observation and manipulation), problem-solving abilities (e.g., hypothesis generation and testing), and social and interactive skills are especially promoted by robotics.

A more recent review reported consistent findings about the effect of the educational use of robotics construction kits in promoting, through problem-solving activities, a specific progression in the acquisition of CT skills, generally beginning with sequencing, followed by learning of reasoning abilities, and ultimately resulting in systems thinking in K–12 learners [19]. While younger students proved to be able to use sequencing and make simple causal inferences, older elementary children progress to causal reasoning about more complex programs and conditional reasoning. Subsequent studies have provided additional evidence to support the effectiveness of robotics kits (i.e., Lego WeDo) in fostering computational thinking skills and learning of STEM content among students of primary school [20,21] and in the case of early childhood education [22]. Finally, it should be considered that lessons based on robotics kits were found to improve students’ interest and motivation in learning [23].

Within the field of research assessing the effectiveness of educational robotics kits in promoting children’s learning, our contribution sought to answer two main research questions:

• Does a basic robotics laboratory supported by the use of a Lego^® Education WeDo 2.0 robotics kit impact the development of CT skills in children aged from 8 to 10 years?
• Is there a difference in the effect of the laboratory on CT skills between children from third- and fourth-grade, respectively?

On the basis of the existing research findings, we foresaw a positive effect of the laboratory on the promotion of children’s cognitive skills. We also expected that pupils from a higher grade would benefit to a greater extent from the robotics laboratory than their younger schoolmates.

For the purposes of the study, we chose a set of Bebras Tasks (https://www.bebras.org (accessed on 28 June 2019)) [24] specifically designed for children in the target age range as a measure to assess students’ overall development of CT skills. More in detail, we used the tasks to evaluate the participants’ ability to use this set of skills for the solution of problems similar to “real-life” situations. The Bebras tasks are a wide set of activities realized for a worldwide annual challenge that aims at increasing students’ engagement and learning in informatics while stimulating the development of CT through the resolution of engaging problems among primary and secondary school students. The Bebras tasks consist of a set of multiple-choice questions, as well as interactive tasks related to problem-solving that do not strictly depend on programming. The tasks do not require prior computer programming abilities, but all questions are related to computational thinking concepts. Further details on the Bebras challenge and tasks are provided below in the text.

This contribution represents an extended version of an already published conference paper [25]. The previous paper briefly reported the overall effect of a robotics laboratory on the early acquisition of CT competence among primary school students. The current work aims to further investigate differences in the impact of the robotics laboratory between children attending third- and fourth-grade classrooms, respectively; moreover, a more in-depth discussion of the previous published preliminary results is provided.

2. Materials and Methods

2.1. Research Design

In order to answer our research questions, we used a quasi-experimental, post-test-only research design [26]. Children from three classrooms (two third-grade classrooms and one fourth-grade classroom) of a primary public school in Palermo, Italy, were chosen to be assigned to the intervention group and thus participated in the activities of a robotics laboratory. On the other hand, children from three different classrooms (of the same grades as the intervention group) of the involved primary school, with the same two informatics teachers, were assigned to the control group. Students from the matched classrooms used as the control did not participate in the laboratory activities and just attended the regular school curriculum.

2.2. Description of the Intervention

The robotics laboratory was designed and conducted within the framework of the PROMISE project activities [27]. The PROMISE was a project funded by the Italian Ministry of Education, University, and Research (MIUR) with the aim to constitute a network of national institutions whose main activity was the development of technological and scientific skills among Italian professional educators (and students, consequently), both in school and in extra-school settings. The Network comprised schools, museums, and several organizations providing technical and scientific education and training, located throughout the national territory. The stated goal of the shared activities of the network partners was to promote a multiplier effect in the diffusion of educational robotics and, more generally, technical–scientific culture.

The implementation of the PROMISE project foresaw two main phases. First of all, 105 teachers and experts in the educational field received training focused on providing them with the skills required to autonomously design and conduct cooperative learning laboratories based on the use of educational robotics. Training curricula were delivered to teachers and experts in three training schools that were opened for this purpose in three cities, including one in Northern Italy, one in Central Italy, and one in Southern Italy (i.e., Milan, Pisa, and Palermo). During the second phase of the project, teachers and experts were asked to actively experiment with their newly learned skills by individually designing and activating educational robotics workshops in primary and secondary schools. Moreover, the trained educators participated in robotics festivals, interactive exhibits during which students from participating schools showed their robotics artefacts and projects (Zoo of the Robot), and expositions of scientific and technological research.

The robotics laboratory presented in the current study was designed and implemented in the second phase of the PROMISE project, with the aim of fostering the development of computational thinking skills among students attending primary school. The laboratory was conducted in the Direzione Didattica Statale Monti Iblei, a primary school in Palermo, and involved students from four third-grade and two fourth-grade classrooms. The laboratory was designed by two school of the teachers who attended the training programs of the first phase of the PROMISE project, in collaboration with researches of the National Research Council of Italy—Institute for Educational Technologies (Consiglio Nazionale delle Ricerche-Istituto per le Tecnologie Didattiche, CNR-ITD) and University of Milan, Department of Computer Science.

A total of three parallel robotics laboratories were conducted in two third-grade classrooms and one fourth-grade classrooms; each laboratory consisted of four two-hours meetings, for a total of 8 h. The activities started on 10 May 2018 and ended on 6 June 2018. During the first meeting, the students were introduced to the Lego^® Education WeDo 2.0 kit (LEGO System A/S, DK-7190 Billund, Denmark) and the visual programming environment, as well as the hardware components of the kit. The use of sensors and actuators was also investigated. During the subsequent meetings, students experimented with the construction of new robots and the writing of codes in the context of STEM challenges. Basic concepts of coding were taught, and students were asked to build robots by following the step-by-step instructions included in the robotics kit; after that, they were asked to write simple programs to allow their robots to perform basic actions, such us move forward, rotate 90 degrees, and collect objects and then place them in a specific place. Students from the fourth-grade classroom were additionally asked to construct robots and write codes to represent the life cycle of a frog.

It should also be considered that we introduced peer-coaching in order to enhance learning opportunities in participating students. For this to happen, students from a third-grade classroom of Istituto Superiore Majorana of Palermo, Italy, a computer-science high school, supported younger peer students in building and programming educational robots. Such an approach was chosen for the well-documented positive impact of peer-mentoring on both tutors’ and tutees’ learning and skills development [28].

A typical lesson plan first included an explanation provided to students (both children and peer-mentors) of the relevant learning content, such us elements of block-based programming or functionality of sensors and actuators. After the solution of a simple concrete problem had been shown to participants, the challenge was generally introduced. Children were usually divided into groups composed on average of four children; groups were asked to autonomously work to complete the challenge, as required by the PBL methodology described in [6]. One to two peer-mentors, based on availability, were assigned to each group and instructed to support younger participants in their efforts toward the challenge solution.

2.3. Data Collection and Characteristics of Participants

Parents of all the children participating in the present study were asked to sign an informed-consent form describing the research procedure and selected measures. With regard to the data-collection procedure, children for whom informed consent was obtained were first involved in the administration of the Bebras tasks, within a few days following the completion of the laboratory. The involved teachers were asked to create their Bebras accounts to register students and print their log-in credentials. Before starting the completion of the tasks, the attendant teacher provided each child with a card containing anonymous log-in information. Once all the children of a session of administration were logged in, a researcher from the CNR-ITD provided them with instructions on how to use the Bebras platform to complete the tasks and told them the amount of time available to finish the whole challenge. Children were explicitly told to ask for help when problems or difficulties with the use of the platform arose and were also assured that the tasks were not intended for evaluation purposes. After the administration of the Bebras tasks, students were asked to complete a set of different questionnaires on the basis of group membership (intervention vs. control). The Bebras tasks and other questionnaires were administered during the informatics lessons of each participating classroom during the first two weeks following the end of the robotics laboratories. Finally, the two informatics teachers participating in the study filled in a questionnaire to evaluate students’ academic performance.

With regard to children participating in the research, 71 students attended the robotics laboratories; only 51 participants completed the outcome measures and were thus included in the present study. The ages ranged from 8 to 10 years (M = 8.57, SD = 0.70), and 18 students were female. Sixty-five students composed the three classes selected as control; only 32 students completed the outcome measures and were consequently included in the current analyses. The age of the control group ranged from 8 to 10 years, and 12 were females. Further details on participants’ characteristics are described below in the text.

2.4. Measures

The next two paragraphs contain a description of the measures selected for collection of data on pupils’ outcome.

2.4.1. Computational Thinking

A selection of 10 exercises from the Bebras tasks was used as a global measure of the level of computational thinking (CT) skills expressed by the children. The Bebras tasks are a wide set of activities designed to be used within the Bebras International Challenge on Informatics and Computational Thinking, which consists of a worldwide annual challenge aimed at enhancing the engagement in informatics of students aged from 5 to 19 and knowledge of informatics concepts and promoting the development of computational thinking skills through the resolution of real-life and engaging problems [29,30]. The Bebras challenge takes place in several countries (55 in 2018). In order to make the Bebras tasks accessible to a non-vocational audience, informatics technicalities and jargon are avoided in their texts. In fact, tasks focus on the core of computing and other specific skills that are relevant for anyone, not just computing professionals, and that are now included under the category of CT. Members of the international Bebras community meet annually to design a set of new tasks to be proposed; the national organizers in each country choose, translate, and adapt from this original pool a subset of tasks for the annual local contests. In Italy, the contest is organized by the University of Milan; it is administered via an online platform and comprises 5 categories, each aimed at pupils of different school levels [31]. Specifically, the KiloBebras category is intended for participation of pupils attending the highest grades of the Italian primary school system (third to fifth grade, aged about 8–10 years old).

For the purpose of the present study, a special session was created that included the set of ten tasks previously proposed in the KiloBebras challenge that took place in Italy in 2017/2018; the tasks can be completed within a time interval of forty-five minutes. The sets of tasks used in the Italian Bebras challenge held in 2017/2018 for the different categories, including the KiloBebras, can be visualized and solved via the Bebras website [32]. The main skills promoted and involved in the solution of the Bebras tasks in the KiloBebras category are as follows: (1) logically analyzing and organizing data; (2) representing data through abstraction and formal encodings; (3) using algorithmic thinking as a way to automatize solution; and (4) implementing simple algorithmic procedures (coding) [33]. It is noteworthy that, even if tasks are designed with an evident computational flavor, the skills they actually imply and/or promote are at least partially shared with other (not only STEM) disciplines, especially in the case of tasks designed for younger pupils.

Even considering that each task can be correctly completed using different approaches and skills, some CT skills can be easily associated with each task. Specifically, the tasks used for the purpose of the current study can be grouped according to CT skills required for their understanding and resolution as follows:

• Logically analyzing data: three tasks (parking lot, Christmas presents, and crossbred animals) subsume abilities related to logical inference, deductive reasoning, checking whether data satisfy given properties, and drawing correct conclusions.
• Logically organizing data: four tasks (crossbred animals, wallpaper, volley tournament, and ninjas) present data arranged or ask to reorganize data according to certain criteria or structures (e.g., sequences or hierarchical trees).
• Representing data through formal encoding: one task (ninjas) built on a simple cryptographic system, and pupils need to perform a decoding process.
• Algorithmic thinking: five tasks (colored paths, through the passage, wallpaper, blocks, and interactive dance) deal with procedures defined as sequences of steps/actions/instructions that need to be selected within a predefined set of “primitives”. As a consequence, the tasks require pupils to understand the role of such primitives and the effects of their combination, to correctly execute the sequence or foresee the effect of this execution, and thus to combine the primitives in order to obtain the desired outcome.
• Implementing simple algorithmic procedures: three of the tasks requiring algorithmic thinking (through the passage, blocks, and interactive dance) deal with basic programming; in other terms, the algorithmic procedures need to be implemented on the basis of simple kinds of formal syntax.

Despite the fact that the Bebras tasks have been primarily created to foster children’s learning of informatics concepts and CT skills, they have been recently used for the naturalistic assessment of participants’ ability to apply learned computational thinking skills to the solution of “real-life” problems [34]. In that study, the Bebras tasks showed a moderate correlation with a different measure of computational thinking, focused on the evaluation of computational concepts rather than their actual use in the solution of problems emulating realistic situations. In the current study, the students’ total scores on the Bebras tasks were used as an overall measure of computational thinking skills. Total scores on the selected Bebras tasks may range from 0 to 39.

2.4.2. Additional Measures

Teachers provided for each student a rating of academic achievement in all the subjects of the scholastic curriculum on a 5-point Likert scale (from 1 = insufficient to 5 = excellent). Students’ average academic performance and average performance only on STEM subjects were computed to compare the intervention and control groups. Ratings of students’ academic achievement were collected before the beginning of the intervention to obtain measures (average academic performance and average performance on STEM subjects) of similarity between the intervention and control groups. Moreover, students participating in the robotics laboratory were asked to complete a questionnaire to assess their satisfaction with the intervention (six questions on a 4-point Likert scale, from 0 = Not at all to 3 = Extremely). Additionally, students of both groups filled in a questionnaire to evaluate their experience with the Bebras tasks in terms of (1) satisfaction with the tasks, (2) perceived difficulty of the tasks, and (3) value of the tasks as learning tools (three questions on a 4-point Likert scale, from 0 = Not at all to 3 = Extremely).

2.5. Statistical Analysis

Descriptive statistics were computed as frequencies, and means and standard deviations. Further, two Mann–Whitney U tests were run to assess differences between the intervention and control groups on average overall academic performance and average performance on STEM subjects, and thus ensure similarity among the groups of participants. With regard to the first research question of the study, an independent t-test was performed to evaluate the difference between the intervention and control conditions in the Bebras total scores as a measure of computational thinking skills. Moreover, two independent t-tests were computed in order to evaluate the differences in time to complete the Bebras challenge and satisfaction with the challenge between the conditions. For the second research question, two Mann–Whitney U tests were performed to evaluate differences in Bebras total scores between the treatment conditions (intervention vs. control) separately for children from third- and fourth-grade classrooms; effect sizes were also separately computed to compare the magnitude of the intervention effect between school grades [35].

3. Results

Students form the third-grade classrooms were all 8 years old; on the other hand, students attending the fourth-grade classrooms were in the age-range from 9 to 10 years (M = 9.27, SD = 0.45).

Table 1. Students’ gender and academic performance across classrooms and by type of condition.

Variable	Intervention			Comparison
	3rd GC 1 (n = 16)	3rd GC 2 (n = 14)	4th GC (n = 21)	3rd GC 1 (n = 8)	3rd GC 2 (n = 8)	4th GC (n = 16)
	M (SD) or N	M (SD) or N	M (SD) or N	M (SD) or N	M (SD) or N	M (SD) or N
Female	5	5	8	2	5	5
AP	4.49 (0.74)	4.49 (0.60)	4.55 (0.49)	3.60 (1.05)	3.31 (0.70)	4.86 (0.31)
STEM AP	4.50 (0.73)	4.45 (0.75)	4.51 (0.55)	3.63 (1.06)	3.25 (0.71)	4.81 (0.36)

Note: GC = grade classroom; M = mean; SD = standard deviation; AP = average academic performance; STEM AP = science, technology, engineering, and mathematics average academic performance.

shows students’ gender, average overall academic performance, and average performance in STEM disciplines across the classrooms and by type of treatment (intervention vs. control).

Moreover, no significant differences between the intervention and control groups in the overall academic performance (intervention median = 5, comparison median = 4.5, n = 51, U = 650, p = 0.10) and in that related to the STEM subjects (intervention median = 5, comparison median = 4.5, n = 32, U = 655, p = 0.11) were found according to results of the two Mann–Whitney U tests. The groups were thus comparable in levels of scholastic and, specifically, STEM performance before the administration of the intervention.

With regard to the first research question of the study, the result of the independent t-test showed that students participating in the robotics laboratory (intervention group) obtained significantly higher total scores on the Bebras tasks (M = 19.84, SD = 6.69) than did pupils in the control group, i.e., those only attending regular lessons (M = 13.41, SD = 5.95), t(81) = −4.45, p < 0.001. Moreover, students in the intervention group spent more time in minutes (M = 30.63, SD = 7.90) to complete the Bebras tasks than did those in the control group (M = 22.29, SD = 6.38), according to the results of the t-test (t(81) = −5.03, p < 0.001). Students in both the intervention and control groups similarly reported a moderately to high satisfaction with the Bebras tasks and perceived the tasks as a useful tool for learning of new scholastic topics. Finally, students in the intervention group reported a high satisfaction with the activities of the robotics laboratory and considered them a motivating way to explore and learn new topics.

With regard to the second research question, the first Mann–Whitney U test showed that total scores on Bebras tasks were significantly higher for third-grade students in the intervention group (Mdn = 20.00) than for third-grade pupils in the control group (Mdn = 13.00), U = 404.00, p < 0.001, r = 0.59. Similarly, the second Mann-Whitney U test indicated a significant difference with fourth-grade students in the intervention group obtaining higher total scores on the Bebras tasks (Mdn = 15.50) than fourth-grade children in the control group (Mdn = 19.00), U = 234.50, p < 0.05, r = 0.34. The effect size of the intervention was found to be large for children attending third-grade classrooms (r = 0.59), while it was medium for children attending fourth-grade classrooms (r = 0.34). Notably, average total scores on the Bebras tasks obtained by children attending third- and fourth-grade classrooms, respectively, and assigned to the intervention group were roughly comparable (see

Table 2. Students’ means and standard deviations, and medians across classrooms and by type of condition.

Variable	Intervention				Control
	3rd GS		4th GS		3rd GS		4th GS
	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn	M (SD)	Mdn
Bebras Total Scores	19.90 (6.40)	20	19.76 (7.24)	19	11.31 (5.51)	13	15.50 (5.79)	16

Note: M = mean; SD = standard deviation; Mdn = median; GS = grade students.

).

4. Discussion

The current paper presents findings on the efficacy of a robotics laboratory, conducted using Lego^® Education WeDo 2.0, on the promotion of cognitive skills related to the construct of computational thinking (CT) in children of primary school, aged from 8 to 10 years. To achieve this purpose, a selection of Bebras tasks was used as a specific overall measure of children’s CT skills, such as logically organizing and analyzing data, representing data through formal encoding, algorithmic thinking, and implementing simple algorithmic procedure. Overall, findings from the present study are consistent with those reported in previous research in providing some evidence supporting the educational use of robotics kits to stimulate an effective expression of CT skills for problem-solving purposes. Interestingly, the present findings also highlight the greater effect of the laboratory on children attending lower primary school grades.

Specifically, the current results provide support for the benefits of using educational robotics kits to promote the development of specific CT abilities that may help children solve problematic situations that mirror real-life issues [34]. In fact, the result of the t-test comparing total scores on the Bebras tasks obtained by participants in the two groups showed a significantly higher performance in tasks solution in children attending the robotics laboratory. First of all, our findings are consistent with those reported by existing literature that confirmed the effectiveness of robotics education in supporting the acquisition of cognitive skills related to CT [19,20,36,37]. More specifically, findings of the present study are similar to those shown in a previous study evaluating the development of CT skills in a sample of fifth-grade students [38]. However, that study reported a more significant increase in children’s CT skills in the context of robotics programming than in contexts requiring everyday reasoning abilities. It should also be noted that, although the current study evaluated the development of CT skills mainly expressed in problem-solving tasks emulating real-life situations, children’s performance in programming was not specifically measured. One suggestion for future research would be to evaluate the development of CT skills via educational robotics kits by using a set of complementary tools to assess skills acquisition within different domains of application [34].

Second, this study found that pupils in the intervention group spent more time on their assignment than did those in the group attending only regular lessons. It may be argued that children participating in the laboratory activities made a greater effort to complete each proposed task, while obtaining better results, due to a more effective use of the abilities related to logically analyzing and organizing data that have been stimulated by the robotics laboratory. Moreover, it may be hypothesized that students in the control group found the tasks more difficult to complete since they haven’t had enough experience with the use of the needed CT skills; in consequence, they could have renounced finishing at least some of the Bebras tasks faster than their intervention counterpart. Notably, children in both conditions reported comparable judgements about the difficulty of the tasks, on average rating the challenge as “not at all” to “slightly” difficult.

According to the results of the current study, the laboratory activities seem to account for a better performance in the Bebras tasks among children attending both third- and fourth-grade classrooms. The results of the computed Mann–Whitney U tests showed that total scores on Bebras tasks were significantly higher for both third-grade and fourth-grade students in the intervention condition than those of their same-graded counterparts in the control condition. However, an interesting finding was that the effect of the intervention was found to be greater for younger children (r = 0.59) than for those from the higher-grade classroom (r = 0.34), with pupils from both third- and fourth-grade classrooms participating in the laboratory reaching the same average total scores on the Bebras tasks (M = 19.90 vs. M = 19.76, respectively). This finding suggests that older students in the intervention group remained at the same level of CT skills than their younger counterpart and seems to contradict results from a previous review highlighting how educational robotics can support students in developing increasingly sophisticated CT skills across school grades [19]. A possible explanation is that the robotics activities proposed during the laboratory were not fully adequate to the developmental level of the fourth-grade children. Even if the laboratory included specific activities designed in accordance with the educational objectives set by the scholastic curriculum for the fourth grade (metamorphosis of the frog), it probably failed in stimulating learning and effective expression of more advanced cognitive skills among fourth-grade students than those expressed by younger pupils. The assessment of the level of CT skills already achieved by students before the intervention would have helped educators in designing more challenging and tailored learning tasks. However, this result needs to be interpreted and generalized with caution given the small sample size and the overall measure of CT skills used in the present study. Future studies with larger sample sizes should focus on the impact of robotics education on different dimensions of CT in order to allow a more in-depth evaluation of the educational advantages of the use of robotics kits. Future studies addressing the development and validation of parallel forms of the Bebras tasks may also support researchers in implementing and adequately evaluating interventions focusing on specific CT dimensions.

A final consideration should be done with regard to the average performance on the Bebras tasks of children in the intervention group when compared to the results of the Italian Bebras contest held in 2017/2018. On average, children participating in this study obtained total scores on the Bebras tasks clearly lower than those reported for the national contest, even after the participation in the robotics laboratory [39]. This finding indicates that children enrolled in the study would have needed a more structured and in-depth educational intervention to obtain scores at least comparable to the average performance of children engaged in the Bebras contest. However, caution is needed when interpreting this result, given that the contest involves the participation of teams composed typically of four students, while children in the current study were asked to solve the tasks individually.

5. Conclusions

This current work adds some additional evidence to existing research supporting the assumption that educational robotics is an effective tool for the development of CT skills among students attending K–12 education. Specifically, third- and fourth-grade students participating in a robotics laboratory, conducted with the support of the Lego^® Education WeDo 2.0 kit, performed better on solving a set of “real-life” tasks than their control counterpart did. Additionally, the results show that the students from third-grade classrooms benefited more from the laboratory activities than their older schoolmates. The current findings suggest that educational laboratories conducted with the aid of robotics kits can facilitate the early development of CT skills that children may successfully transfer to dealing with realistic problematic situations. Our results thus encourage the systematic adoption of educational robotics in K–12 education as an engaging tool to foster the acquisition of transversal cognitive abilities required for different disciplines and domains of application. On the other hand, the current findings also suggest that attention should be paid in designing educational activities that adequately match children’s developmental level in order to provide a fully effective intervention.

Moreover, the greater amount of time spent by students in the intervention group to complete the Bebras tasks (when compared to the control group) may be related to their greater use of skills (i.e., logically analyzing and organizing data) stimulated during the participation in the robotics laboratory. However, future research should directly address this hypothesis. Additionally, due to the limited sample size of this study, we provided only a summary evaluation of CT skills. Future studies using selected Bebras tasks, and with larger sample sizes, should study which CT skills (e.g., abstraction, algorithmic thinking, decomposition, evaluation, and generalization) are specifically promoted by educational robotics, as well as the extent to which each skill is promoted. Future research should also assess the development of CT skills supported by robotics by using complementary measurement tools that can provide an assessment of the use of those skills within different domains. Moreover, the Bebras tasks, specifically the set of tasks we used in the current study, were initially designed to promote students’ acquisition of coding and CT skills and not as a measurement tool. As a consequence, future studies should directly provide systematic evaluations of the psychometric properties of specific sets of tasks.

Another aspect that must not be overlooked is the professional update of the teachers involved in this pilot. The face-to-face workshops implemented to train the teachers in the designing of PBL tasks with the use of robotics kits stimulated their interest in the integration of such kits in the traditional computer science curriculum of primary school. Finally, peer-coaching was introduced in the laboratory to increase the impact of experiential learning; unfortunately, the incremental contribution of peer-mentoring in explaining the students’ performance on the Bebras tasks was not assessed, due to both practical and resource limitations.