A Framework for Using Humanoid Robots in the School Learning Environment

Abstract

With predictions of robotics and efficient machine learning being the building blocks of the Fourth Industrial Revolution, countries need to adopt a long-term strategy to deal with potential challenges of automation and education must be at the center of this long-term strategy. Education must provide students with a grounding in certain skills, such as computational thinking and an understanding of robotics, which are likely to be required in many future roles. Targeting an acknowledged gap in existing humanoid robot research in the school learning environment, we present a multidisciplinary framework that integrates the following four perspectives: technological, pedagogical, efficacy of humanoid robots and a consideration of the ethical implications of using humanoid robots. Further, this paper presents a proposed application, evaluation and a case study of how the framework can be used.

1. Introduction

According to Oxford University researchers, many white and blue-collar jobs are at risk of the Fourth Industrial Revolution [1,2] with its increasing supply and demand of industrial robots globally [3]. According to the Economist Intelligence Unit’s recently released Automation Readiness Index, not a single nation included in the study was fully prepared to address the challenge [4]. Robotics and efficient machine intelligence are the building blocks for the coming revolution [5,6]. Countries need a long-term strategy to deal with the challenges of automation and education must be at the center of it. Countries must provide students with a grounding in certain technical skills, such as computational thinking, which are likely to be required in many future roles [4]. Many such roles will also require an understanding of robotics [4].

Humanoid robots have already been used with children to examine various phenomena [7,8,9]. However, the use of humanoid robots in classrooms is a recent development [10]. The understanding of how children use and learn with these robots is beginning to display signs of future potential [10]. Much of the research to date has focused on the technological capabilities of robots to act as educational tools, focusing for example on language acquisition, Science, Technology, Engineering and Mathematics (STEM) and the basic principles of programming [11,12].

Educational robotics (ER) offer the possibility both of the facilitation and the evaluation of learning as “pedagogical agents” [13]. Through human–robotic interactions and targeted feedback, ER can be programmed to help with learning and develop technical skills through individual and collaborative learning [14]. In particular, ER can be used to target specific learning outcomes of subject knowledge (i.e., math), skills (i.e., programming and critical thinking) [15]. A recent meta-analysis [16] has shown that ER has been shown to improve knowledge and skills, help with transferring skills to other domains, increase creativity and motivation, increase the inclusion of broad and diverse populations and have an added benefit of increasing teacher development. ER has also shown benefits in STEM subjects [17], but in general, there are mixed findings on the effectiveness of ER [18]. This may be due to methodological shortcomings in design and evaluation [19].

In the context of educational robotics, there have been many efforts made to improve the teaching work in STEM programs to aid both teachers and learners; however, there is a lack of clear-cut guidelines or standards [20]. While ER is a growing field, the benefits to learning outcomes and the evaluations of these interventions need standardized and validated frameworks to assess the efficacy of ER in schools.

Robots have also been used as educational agents with a focus on developing social psychological skills. For example, the iCat robot has been used to teach children to play chess [21] and the Keepon robot for robot-assisted therapy with children on the autistic spectrum [22,23]. Research with the NAO, RoboVie and Tiro humanoid robots have provided insights into the psychological dynamics characterizing social human-robot interaction (HRI) in educational settings [24]. However, multiple studies [25,26] have acknowledged a lack of understanding of the efficacy of humanoid robots in school learning environments (SLEs).

In recent reviews, it has been found that humanoid robots largely act as novices, tutors or peers in educational settings to support learning and that the majority of these applications are driven by technological feasibility and not grounded in didactical theory [12,26]. When theory has provided some didactical frame-working for working with robots in educational contexts, the following approaches have been used: project-based learning, experiential learning and constructionist learning [27].

From the technological perspective, the social element of the interaction between robot and human is difficult to automate and fully autonomous social tutoring behavior in unconstrained environments remains elusive [28]. The robots are limited by the degree to which they can accurately interpret the learner’s social behavior [28]. Building artificial “social interaction requires a seamless functioning of a wide range of cognitive mechanisms and their interfaces” ([28] p. 7). This social element of the interaction is especially difficult to automate [12] and needs further research.

In Reference [27], the benefits of incorporating robotics as an educational tool in different areas of knowledge are explored. Another study [29] investigated how robots in the classroom reshape education and foster learning. A recent study has reported that students are generally motivated and have a very positive reaction to the introduction of educational robotics in the academic curriculum [30]. Although humanoid robots have the potential to bring benefits, the incorporation of such technology into SLEs brings its own set of challenges for teachers. These are due to the robot’s presence in the social and physical environment and the expectations that the robot creates in the user [28]. In Reference [31], the influence of robots on children’s behavior and development and their reaction to the robot’s appearance and visual characteristics were examined. There is a call for research into people’s interactions with and social reactions towards humanoid robots as a way to shape ethical, social and legal perspectives on these issues as a prerequisite to the successful introduction of robots into our societies [32].

There is a lack of empirical research involving the use of robots in SLEs; therefore, there is a need for more effective analysis of the potential of robotics as a teaching tool for schools [27]. A recent review of the literature [16] observed that the majority of the existing studies lacked an experimental or quasi-experimental design. Another study [33] proposed having more intervention studies with focused research design in K–12 spaces. Recently emphasis has been put on the importance of conducting these interventions with effective robotic pedagogies and underlying theoretical foundations that are required for educational modules in STEM education to make robot-based pedagogies more efficient [16]. Further to this, it has been argued that educational robotics allows for an integrated, multi-disciplinary approach and it is essential to provide a more holistic portrayal of the research on educational robots [16]. In response, this article contributes to the field by presenting a multidisciplinary framework. The multidisciplinary nature of the framework acknowledges that the use of humanoid robots in SLEs must be holistic, rather than focusing on just the technical, or the pedagogical for example. As a position paper, our intention is to present the framework with a proposed application, evaluation and case study by way of an illustration. In particular, we propose that the introduction and evaluation of technology in the classroom should be explored from the following four perspectives: pedagogical, technological/human robot interaction, psycho-social development and a consideration of the ethical implications of using humanoid robots.

Firstly, from an educational perspective and in light of the United Nations Sustainable Development Goal 4 which seeks to “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all.” [34], can humanoid robots contribute anything to the promotion of quality education? Can a humanoid robot offer a learning experience tailored to the learner, supporting and challenging students in ways unavailable in current resource-limited classrooms? Can humanoid robots contribute to adapted or differentiated education? Can robots be used and thereby “... free up precious time for human teachers, allowing the teacher to focus on what people still do best: providing a comprehensive, empathic, and rewarding educational experience” [12]? What are the pedagogical and didactical foundations or frameworks for the use of humanoid robots in educational settings?

Secondly, how can Artificial Intelligence (AI) and robotic technology be integrated to develop humanoid robots to teach children in SLEs?

Thirdly, how do the human factors interaction with humanoid robotics influence psycho-social development in children (i.e., motivation, self-efficacy, resilience)?

Finally, as AI technology develops and the social interactions between robots and students become more complex, what are the ethical implications of using humanoid robots in educational settings and how do we address these?

This article firstly in Section 2 presents the multidisciplinary framework for using humanoid robots in SLEs. Section 3 includes concrete suggestions on how the proposed framework could be applied and evaluated by researchers and practitioners in different contexts and settings. Section 4 describes a case study related to the application of this framework in a real setting followed by a conclusion and future work.

2. A Multidisciplinary Framework for Humanoid Robots in School Learning Environments

In this section, we present the presuppositions upon which the framework is built. We then present an outline of the framework, including a brief description of each of the four aspects.

2.1. Presupposition

The framework is grounded in the values of inclusive education and the right to education for all. The foundations of inclusive education are built upon the principles of universal human rights and supported by international organizations, such as UNICEF, UNESCO, the Council of Europe, the United Nations and the European Union [35]. The Salamanca Declaration includes all groups of students in danger of marginalization highlighting the right to participate in common learning activities within the ordinary school system, regardless of special needs, gender, ethnicity, culture, social background, etc. [36]. If inclusive education is to become a reality, we must develop learning environments to embrace diversity. For example, some students understand quickly through images, others may prefer texts and readings. Some may deal well with theories, others may learn through experiments and examples and some may have specific learning difficulties [37]. Some learn through engaging in discussion with others, whilst some learn through having the opportunity to work alone. What are the potential ways in which humanoid robots can contribute to the development of SLEs that embrace diversity and help to promote inclusive education?

With the focus on the learning of each individual, the student is placed at the center of our proposed framework as shown in Figure 1.

In a two-way collaboration with the student, educators (teachers and assistants), technology (humanoid robots), peers and researchers contribute to the SLE. Through the development of this collaborative learning environment, we seek to explore the following areas.

2.2. Pedagogical/Didactical Development

It is proposed that the pedagogical/didactical aspect of the framework should be grounded in experiential learning theory (ELT) which defines learning as “the process whereby knowledge is created through the transformation of experience. Knowledge results from the combination of grasping and transforming experience” [38] (p. 41). With the focus on learning as a “process”, the ELT model proposes two dialectically related modes of grasping experience—Concrete Experience (CE) and Abstract Conceptualization (AC) [39]. In addition, the ELT model proposes two dialectically related modes of transforming experience—Reflective Observation (RO) and Active Experimentation (AE). The ELT model allows for a diversity of learning styles in students and acknowledges that for some, concrete experience helps them to grasp, perceive and gain new knowledge. However, for others, grasping or taking hold of new information happens through symbolic representation or abstract conceptualization. In the same way, some of us transform or process experiences by watching others and reflecting on the observation of others who are involved in the experience, whereas others actively experiment, jumping right in and doing things [39].

We propose that the ELT model be used as the theoretical foundation for the didactical approach. Further, the didactical approach must be developed as part of an iterative process in collaboration with those working in the specific SLE context.

2.3. Technological Development for Human–Robot Interaction

In order to realize a successful human–robot interaction, a key element—a spoken dialog system—needs to be implemented. A spoken dialog system consists of multiple components: speech recognition, natural language understanding, dialog management, natural language generation and speech synthesis [40]. On the other hand, social signal processing [41], social expression generation, turn-taking [42] and physical action generation including pose, hand, arm, head movements [43] are also important elements of a spoken dialog system, especially in multiparty dialogs. In order to maintain a multi-turn dialog, the robot has to maintain and understand the conversational history and context [44].

2.4. Psycho-Social Development

We propose that the individual and social behaviors, capabilities, constraints and limitations should be explored as the humanoid robot is incorporated into the SLE. The development of behavioral prediction models for user-behavior and performance outcomes can then be used to develop the framework further for human cognition in socio-technical systems. We propose the modeling of user–task interaction at the individual and group level of the SLE through systematic experimentation and naturalistic testing. The research findings then have the potential to be used in the development of evidence-based training modules that cover both the needs of the students and teachers.

2.5. Ethical Development

We recognize the need for applied ethical engagement when it comes to the use of humanoid robots in social settings such as learning environments. In particular, we wish to see research with humanoid robots that moves beyond the question of “what can we do technically?” to “what should we do, ethically?”

This framework requires a theoretical contribution by developing a didactical approach that can be used and evaluated through working with humanoid robots in SLEs. The proposed framework allows for the expansion of the boundaries of artificial intelligence by implementing various components of spoken dialog systems for humanoid robots. Further, we propose that the key performance indicators, to assess different aspects of HRI in SLEs, are identified to determine the efficacy of existing HRI metrics and propose new HRI metrics if required. Finally, we propose the development and evaluation of the humanoid robot’s efficacy to help pupils to learn. The framework enables the promotion of students and teachers learning about how robots work, but it also uses robots to help them to learn competencies needed for a future with robots. In particular, the framework incorporates applied ethical engagement as an important aspect of the competencies needed for a future with robots.

3. Proposed Application and Evaluation of the Framework

In this section, we first present our methodological standpoint for the framework followed by an outline of how the framework can be applied, evaluated and executed.

3.1. Methodology

The proposed framework requires a multidisciplinary and multiple-methods approach that will include applied, qualitative and quantitative aspects. Whilst respecting the integrity of the different paradigms, we propose the utilization of different ways of knowing to expand our understanding of the potential ways in which humanoid robots can be used in SLEs to promote student learning. With such a research design, we can expand the scope of our understanding as different methods will be used to assess different aspects of the phenomenon [45]. By combining qualitative and quantitative aspects in our evaluation of humanoid robots in the SLE, we incorporate both subjective experiences and objective observations [46,47].

3.2. Methodological Implications

Research into understanding and learning the effects of human–robotic interactions in schools is still in the early stages. The applied nature and real-world complexity of this field mean this research is multidisciplinary. The use of a mixed-methods research design that includes qualitative, quantitative and theory can lead to insights and discoveries in this novel domain. There are few existing theoretical frameworks in the literature encompassing these research questions and validated approaches. This requires using validated approaches from different disciplines, that is, psychology, human factors and educational research.

This framework also promotes using naturalistic settings over laboratory settings due to the nature of the domain studied. Socio-technical domains incorporate human-technology interactions while in social settings (i.e., classroom) but research frameworks need to be validated across domains. Experimental laboratory settings are applicable to identify the impact of variable manipulation on outcome variables and may give high internal validity, but it is limited in generalizations. Naturalistic design allows the observation of participants in their natural settings and observes for outcomes. While this approach may have low internal validity, it is high in ecological validity, therefore the findings can be generalized to other populations.

Both quantitative and qualitative approaches need to include their respective approaches to validity (See for qualitative approaches [48]). By using a mixed-methods design and triangulation methods, new insights on ELT approaches can be validated and form the foundations for future work that are applicable to all four domains (technological, psychological, educational and ethical). This approach will allow for the reflective observation and active experimentation of the ELT framework.

3.3. Preparation

We propose that the framework must be situated within the specific context and take into account the needs of the teachers and SLE. In particular, the needs of the SLE must be established regarding the identification and definition of scenarios related to existing educational contents suitable for the use of humanoid robotics, for example, grade/age, types of school, state/private, types of learning formats, group/individual/whole class. In order to complete this task, the researcher will need to engage in a period of consultation and information gathering with school teachers. This activity may take multiple sessions as the researchers learn about existing educational content to be able to develop a set of scenarios involving humanoid robots depending upon the learner profile(s) to deliver context-appropriate and tailored educational content.

This preparation stage also involves organizing information sessions for teachers and parents along with obtaining necessary permissions from relevant ethical boards and parents since these activities involve children.

In addition, in this preparation stage, the researchers must identify and design appropriate data collection tools that measure learning outcomes, performance, user interface experience and psychosocial skill development.

3.4. In-Context Development of Various Aspects and Evaluation Instruments

3.4.1. Pedagogical

As stated in Section 2.2, we propose the development of a didactical approach to working with humanoid robots in SLEs based on ELT [38]. The didactical approach should, however, be developed in collaboration with the teachers and based on the needs of the specific SLE context. We propose that this should be an iterative process to allow for the investigation of both how the development of a didactical approach can contribute to more effective working with humanoid robots in specific SLEs, and in what ways educational activities with humanoid robots can promote learning.

We propose that to evaluate the effectiveness of the pedagogical aspect of the framework the main approach should be qualitative and exploratory. Since programming robots for social interaction and for teaching is highly creative, it requires co-design and development with stakeholders, and an iterative development methodology will be highly beneficial. Semi-structured interviews could be used to evaluate humanoid robots in SLEs with respect to HRI, robot behavior, natural language understanding and social signal processing.

In addition, a qualitative approach could be used to focus on both student and teacher experiences of introducing and working with humanoid robots in the classroom. The advantage of adopting a qualitative approach is that it allows us to explore how the students and teachers interact with the humanoid robots, including feelings, strengths/challenges and ethical considerations of working with humanoid robots.

3.4.2. Technical

The main approach for technical development can be iterative, requiring continuous qualitative and quantitative evaluation. We propose to implement a spoken dialog system consisting of various components (as shown in Figure 2) to create engaging educational activities with humanoid robots.

(a)

Automatic speech recognition, natural language understanding, gesture recognition and understanding so that the robot can perceive the learning environment and human participants;

(b)

Interaction state tracking so that the robot can determine the current state comprising of the dialog act and/or gesture by maintaining a “memory” to store interaction history and contextual information;

(c)

The robot will then form an interaction strategy plan consisting of various actions with personalization;

(d)

Natural language generation, text to speech and physical action generation including gestures with personalization for adaptive learning customized according to the level and learning speed of the user.

The above-mentioned activities can be designed for two settings, individual educational activities and multi-party educational activities with group interactions and teamwork between peers.

Existing tools and libraries provided with commercially available humanoid robots can be explored for components such as automatic speech recognition and generation, natural language understanding and generation, text to speech synthesis and the main focus can be on components such as creating a knowledge base for efficient dialog management to be used with the humanoid robot in SLEs. Other available techniques and methods such as for natural language understanding, deep learning methods involving Convolutional Neural Networks [49] or Recurrent Neural Networks [50] and for leveraging external knowledge for natural language understanding [51] and natural language generation [52], knowledge graphs can also be explored.

Various metrics (e.g., cognitive interaction, degree of monotonicity, human awareness—human recognition, characterization and adaptation, robots’ self-awareness, safety) have been discussed to evaluate and assure functionality of humanoid robots [53]. However, a key factor that limits the success of human–robot teams is the lack of consistent test methods and metrics for assessing the effectiveness of HRI [54] since existing metrics are not sufficient to capture all aspects of HRI [53] in every setting [55]. Therefore, HRI metrics in conjunction with observations, quantitative (e.g., questionnaire) and qualitative methods (e.g., semi-structured interviews) can be used to evaluate humanoid robots in SLEs.

3.4.3. Psycho-Social

We propose the development of behavioral prediction models for user-behavior and performance outcomes that are situated in the specific context of the SLE. This can be achieved through the modeling of user-task interaction at the individual and group level of the SLE through systematic experimentation and naturalistic testing.

We propose that by using validated approaches from human factors and cognitive engineering, we can evaluate the efficacy of humanoid robots on the psycho-social development of learners (i.e., motivation, self-efficacy, resilience). This can be achieved by developing and validating applied interventions based on human factors and cognitive engineering aspects where the interaction of individual aspects of human behavior (microcognition; i.e., self-efficacy, resilience, metacognition) and naturalistic environments (macrocognition; i.e., shared situational awareness, communication) are considered in both human–robot interaction and human–human interactions. These measures will be analyzed using social science paradigms (i.e., statistical analysis, cognitive task analysis, qualitative interviews).

3.4.4. Ethics

Careful consideration must be given to ethics and it is proposed that these considerations are situated in the specific context in which the research is taking place. Some considerations to be taken are, first, what are the implications for the students and teachers/assistants in introducing humanoid robots into the SLE? As researchers, we have an ethical responsibility to “do no harm” to those who participate in such studies. Secondly, as the technological advancement of artificial intelligence continues and humanoid robots become more autonomous, what ethical applications apply to the robots? Thirdly, and related to the above two, how do we prepare students and teachers/assistants for a future with robots which are founded upon ethical considerations?

4. Case Study

This section presents an example of how the framework can be implemented.

Aim: To explore how humanoid robots can assist teachers to promote Mathematics and programming skills.

Sample: Grade 6 students (n = 20) and teachers (n = 2)

Preparation: Researchers have two meetings with the grade 6 teachers to prepare the content of the three-day workshop, including discussion surrounding the learning needs of the students. Ethical consent is gained from the relevant body to conduct the research. An information meeting is held for teachers and parents/guardians of participants under the age of 16. Informed consent is gained from participants and the parents/guardians of participants under the age of 16. The discussion related to the selection of evaluation methods (e.g., observations, quantitative and qualitative) and instruments is also initiated at this stage.

Didactical approach: Execution of a three-day workshop which involves the following activities for the participants:

Activity 1—Introduction to robots—including a presentation and class discussion led by the researchers. Informed consent is explained to the participants.

Activity 2—Participants complete a pre-test structured questionnaire of their metacognitive judgment on how they expect to do working with the robot, math and programming.

Activity 3—Participants are separated into groups of four or five by the regular class teachers. Each group participates in a one-hour practical session led by the researchers. The session includes basic programming and math tasks using the robot.

Activity 4—Participants complete a post-test structured questionnaire of their metacognitive judgment about how well they think they did working with the robot, math and programming.

Activity 5—The researchers conduct semi-structured group interviews with each of the four groups of grade 6 students to gather in-depth data about the experiences of working with the robot.

Activity 6—Plenary—including a presentation and class discussion surrounding the experiences of working with robots, what a future with robots looks like and the ethical considerations to working with robots, led by the researchers.

Activity 7—The researchers conduct semi-structured group interviews with the grade 6 teachers to gather in-depth data about the experiences of working with the robot.

Technical development: The robots are programmed for activities related to mathematics and programming tasks. This is done in multiple iterations so that other researchers and teachers can provide feedback in order to improve these activities before the workshop with the participants. Questionnaire and semi-structured interviews are used to evaluate human–robot interaction along with participants’ views on the current technical capabilities, limitations and potential improvements in robot activities for future workshops.

Psycho-social development: This is explored during the three-day workshop and in particular through the collecting of pre- and post-test data that explores the participants’ self-efficacy and meta-cognition.

Ethical development: This occurs primarily through the discussions during Activity 6 and in the semi-structured interviews. This is also covered through following ethical guidelines such as informed consent.

Evaluation: Both qualitative and quantitative analysis of the interviews and pre- and post-test data can be analyzed using validated methodologies. Inferential statistics can be used for quantitative data, while qualitative approaches such as Interpretive Phenomenological Analysis or Thematic Analysis can be used to analyze interview data. These approaches have been validated across social and technical domains to measure experiences, interactions and outcomes.

5. Conclusions and Future Work

This position paper has proposed a framework that addresses an under-researched and not well-understood aspect of humanoid robots in SLEs. Rapid technological progress in SLEs needs to be balanced with a holistic approach to research that attempts to support human adaptation in rapidly changing socio-technical system dynamics. With such a multidisciplinary framework, we offer the possibility to move beyond extending the technical possibilities to evaluating how technological advancements can be used in an ethical way to benefit individuals and society through education. In particular, the multidisciplinary framework presented here integrates the technological, pedagogical, psycho-social and ethical aspects of HRI. Further, this paper has presented a possible way to apply and evaluate the framework, methodologically, along with an example of a case study. It is hoped that readers will be inspired to adopt this interdisciplinary framework as their starting point for research into how humanoid robots can be used effectively in SLEs and contribute to the development of the research base within this field.

Although this study includes concrete suggestions regarding the application and evaluation of the proposed interdisciplinary framework along with a case study describing its application in a real setting with a focus on learning mathematical and programming concepts, it is beyond the scope of this paper to include empirical data. Further research is needed to empirically evaluate the framework in order to derive more grounded conclusions. Therefore, future work will report on the comparative analysis, both by longitudinal research and by comparison with the results of experiments designed within different courses and also at other schools.

If humanoid robots can contribute positively towards the SLE and increased learning opportunities (motivation, self-efficacy, resilience) then this will benefit both students in the short and long-term, and in turn society. This framework has the potential to impact the teaching and training of future generations of students that can be reached and benefit from the implementation of the proposed framework. The addition of humanoid robotics in the classroom may facilitate the learning process in students who struggle and may decrease apprehensive behaviors in students, allowing for cognitive processes to open up for more efficient learning and the promotion of inclusive education for all.