Engaging English Language Learners as Cultural Informants in the Design of a Social Robot for Education

Abstract

Background: There are 4.9 million English Language Learners (ELLs) in the United States. Only 2% of educators are trained to support these vulnerable students. Social robots show promise for language acquisition and may provide valuable support for students, especially as we return to needing smaller classes due to COVID-19. While cultural responsiveness increases gains for ELLs, little is known about the design of culturally responsive child–robot interactions. Method: Therefore, using a participatory design approach, we conducted an exploratory study with 24 Spanish-speaking ELLs at a Pacific Northwest elementary school. As cultural informants, students participated in a 15-min, robot-led, small group story discussion followed by a post-interaction feedback session. We then conducted reflexive critiques with six ELL teachers who reviewed the group interactions to provide further interpretation on design feature possibilities and potential interactions with the robot. Results: Students found the social robot engaging, but many were hesitant to converse with the robot. During post-interaction dialogue students articulated the specific ways in which the social robot appearance and behavior could be modified to help them feel more comfortable. Teachers postulated that the social robot could be designed to engage students in peer-to-peer conversations. Teachers also recognized the ELLs verbosity when discussing their experiences with the robot and suggested such interactions could stimulate responsiveness from students. Conclusion: Cultural responsiveness is a key component to successful education in ELLs. However, integrating appropriate, cultural responsiveness into robot interactions may require participants as cultural informants to ensure the robot behaviors and interactions are situated in that educational community. Utilizing a participatory approach to engage ELLs in design decisions for social robots is a promising way to gather culturally responsive requirements to inform successful child–robot interactions.

1. Introduction

English Language Learners (ELLs) are the most rapidly growing subgroup of public school students across the United States—the number of ELLs grew by roughly 60% over the past decade [1]. The growing segment of children living in non-English-speaking households creates an increasing demand for teachers prepared to serve English learners. Unfortunately, state and federal policies as well as teacher preparation programs have not sufficiently prioritized training educators for this spike of English learners, and teachers are, therefore, left unprepared in the classroom [2]. Due to insufficient professional development for classroom teachers, ELLs are more likely than any other group of students to be instructed by a teacher who lacks appropriate professional preparation. ELLs also suffer from insufficient instructional time at school. It is not uncommon for many ELLs to receive only 30 min or less of focused language instruction during a 6-h school day [3]. The lack of professional teacher preparation, the inadequate language-focused instruction, and the unexpected demands for remote learning results in an increasing unmet instructional need for a vulnerable student population.

In the current global environment, social robots can be appropriate tools to provide language support for immigrants and refugees [4] and to expand the language capacity for children who strive to be multilingual [5]. However, introducing a technology into an educational environment serving a minority population requires increased attention to the appropriateness, usability, and feasibility of the technology [6]. If child–robot interactions are not designed specifically for the students they are intended to serve, this technology could potentially do more harm than good.

Culturally responsive instruction [7,8], calls for an intentional adaptation of academic materials and instructional delivery to integrate and align with social-interactive patterns of culturally diverse learners. It is understood to be essential for ELLs and has yet to be fully considered in child–robot interactions. In fact, it has been argued that a socio-cultural dimension is necessary to reflect diverse student populations in technological interventions [9]. Therefore, this exploratory study engaged ELLs in a culturally responsive robot interaction to help us understand the experiences of ELLs interacting with educational robots. In addition, as cultural informants, ELLs provided valuable feedback regarding how they felt an educational robot should look and behave in their school setting. We also gathered reflexive critiques of these child–robot interactions from expert ELL instructors to help us better situate our findings in the ELL context. Finally, from these data, we then discuss iterations and future directions for our own project and our intention to serve ELLs through a culturally responsive lens.

2. Background

2.1. English Language Learners

Approximately 33% of students in the United States, more than 19.8 million children, speak a language other than English at home [1]. About half of these students are officially classified as English Language Learners (ELLs), which signals they have not yet met the established English proficiency requirements [10]. ELLs, a heterogeneous group of children from a range of cultural and language backgrounds, have tremendous academic, linguistic, and civic resources and potential that enriches our school system. However, glaring and persistent discrepancies exist in academic performance between ELLs and their fluent English speaking peers. While many ELLs are themselves immigrants, the majority of ELLs are second-generation immigrants who were born in the United States and do not speak English as a first language at home. It is well documented that these children of immigrants face formidable challenges inside and outside school [11,12,13,14]. These include learning a new language and grade-level content as well as adjusting to an American school environment. Very often, societal pressures to acquire the English language, coupled with the low prestige of the home language, lead to a gradual loss of ELLs’ home languages in favor of English [15]. Because language is strongly tied to identity and family, the subordination of home languages to English often causes ELLs personal, cultural, and familial tensions [16]. Many ELLs find themselves suffering from a lack of English and home language proficiency to function well at home and at school [17]. These conditions result in a negative impact on their school achievement and their motivation for learning [18,19]. Subsequently, many ELLs become more timid or reluctant to participate in class discussions and thereby assume a more passive role in classroom interactions. Educational robots, with a culturally responsive design, may increase ELL verbal interaction and participation in the classrooms. We will now argue that educational robots may be a helpful tool to facilitate English learners’ acquisition of English so that they have the language capacity and confidence to learn the academic content in various subject areas.

2.2. Children’s Perceptions of Robots

As the US student population diversifies in cultural values and home languages, it is critically important that we understand how robots are perceived by culturally diverse learners if we hope to design appropriate and engaging educational tools. Children’s perceptions of social robots are not only shaped by robotic behaviors and appearances, but also by the previous exposure to robots in the media and in their own experiences [20,21]. Lin et al. [22], conducted a survey of Taiwanese fifth graders and found that preferences in educational robots vary in appearance and 59% of the 5th graders felt a robot would be a good companion or tutor in the school setting. Woods [23] conducted a survey of 159 5th and 6th graders exploring their responses to robot images. Participants identified humanoid type robots as more likely to understand them. Hyun et al. [24], found that young children accepted a robot into their classroom and saw the robot as a peer. In addition, they found many children wanted to interact with the robot in a small group. However, how do these findings apply to very diverse classrooms that include ELLs?

In our own, previous study of perceptions of social robots in education for ELLs [25], we conducted interactive, group interviews with 95 ELLs, 39 parents, and 8 school staff to explore the perceptions of social robots in education. Overall, the majority of students and stakeholders endorsed the idea of a social robot to support English-learning language development. In addition, parents and children identified the Nao robot (a humanoid) as the most appropriate robot embodiment for this educational role. Parents and children perceived robots to be nonjudgmental and therefore, helpful in encouraging ELLs to take risks and practice language learning. Parent and teachers postulated that ELLs might be more likely to speak English with the robot, as opposed to a human, without fear of being ridiculed. Finally, school staff suggested that social robots could be essential to increasing ELLs access to language practice.

2.3. Robots in Education

Although not nearly ubiquitous in educational settings, social robots or assistive robots as tutors, peers, and even learners have been found successful in educational research studies. Currently social robots are used in elementary education to tutor [26], converse [27], and practice storytelling [28]. In a large meta-analysis of social robots in educational settings, Belpaeme et al. [4] conducted an extensive review of 101 studies including children and adults. They found that robot behaviors although diverse across numerous studies, had a significant effect on learning outcomes. They also suggested that the social robots’ physical embodiment may be related to their efficacy, but that current studies are too differentiated to make any empirical comparisons across embodiment. They point out that the roles of educational robots in most studies are that of a tutor or teacher, and in only 9% of the studies was the robot a peer or novice. Finally, they raise important barriers to implementation such as natural language processing, curricular integration, and ethical concerns about the use of social robots in education.

In another review robot studies in education, Mubin et al. [29] found robots are predominately used to provide language, technology or science education, and the robot role is often a tool, tutor, or peer in the learning activity. Mubin et al. suggested aligning social robot interactions to the attributes of learners. They also recommended further research to better understand the practical and contextual aspects of robots in educational settings. In a systematic review specific to studies exploring robots in education for young children, Toh et al. [30] called attention to the importance of robot interaction and embodiment design as key factors to increase the effectiveness of educational robots.

2.4. Robots for Language Learning

A growing body of literature suggests that social robots support language development by encouraging participation and increasing engagement in learning activities [31,32]. Specifically, social robots have been shown to increase vocabulary and language skills in young children [33]. Much research has also been done on older elementary school children, mostly in Asian countries. Kanda [34] was the first paper published on robot-assisted language learning (RALL) and conducted their field trial in an authentic social setting of two Japanese elementary schools using English-speaking robots. They offered insights for human–robot interaction (HRI) including (1) children strongly preferred interacting with the robots in the company of one or more friends, (2) children need some initial English proficiency, and (3) need qualitative data and analysis on children’s interactions with the robots.

Two large reviews of social robots in education help to illustrate the current landscape of RALL. First, in a 2019 review of 79 studies from 2004 to 2017, Randall [35] found that overall robots have been shown to support language learning activities of all ages, however it remains unclear as to whether the social component of robots exceeds other technologies such as computer-assisted language tools in terms of learning gains. Randall does suggest that RALL has a positive effect on engagement and motivation (important components in any learning environment). Similar to other reviews of robots in education, Randall emphasizes the importance of robot form and appearance as important in supporting language gains. In another review of 33 RALL studies involving mostly children, Ver den Berghe, et al. [36] found that many studies had small sample sizes and mixed results making it difficult to assess actual learning gains as a result of interactions with the social robot. However, studies that did find high word-learning gains employed the robot as a teaching assistant or peer learner rather than as an independent tutor.

Schulz et al. [37] designed a language program to use a social robot to improve immigrant children’s ability to learn Norwegian as a second language in kindergarten. Their pilot results indicated that involving parents and teachers is essential in designing lessons and interaction activities that will engage and motivate the young learners. Breazeal, et al. [33] studied how greater non-verbal contingent attentiveness shown in robotic interaction helped preschoolers retain more information. Park et al. [38] reported that an attentive listening robot elicited increased storytelling from preschool children. Although social robots show promise for improving language learning, less is known about how robots can support language learning in a diverse ELL population.

Affective factors such as anxiety, motivation, and engagement are known to have strong impact on second language learning [39]. Robots have shown to have a positive effect on language learners’ affect. Scholars have long documented the anxiety aroused when learning or using a second language [40]. Alemi et al. [41,42], found that robot-assisted learning reduced language learners’ anxiety during instructional time. Wang et al. [43], reported that the learners in the teacher with robot-assistance were less anxious and more willing to use the language skills. Furthermore placing robots into the learning environment increases language learners’ motivation even when the novelty effect wears off [44,45].

Given recent interest in social robots aimed to support learning a second language in children [4,46,47] and to ensure a pedagogically sound program, Belpaeme [48] proposed guidelines for robotic design based on principles grounded in educational psychology and observations of second language learning in human–human interaction. Their guidelines included attention to first encounters, the role of the robot, context, verbal and non-verbal behaviors, and robot feedback. Although important components in social robot design, these guidelines do not address cultural responsiveness.

If social robots are going to leave the laboratories and enter the classrooms, then social robots have to function in today’s classrooms which strongly promote curriculum and instruction responding to the learners’ cultural frame [8]. For social robots to have ecological significance and true impact on language learners’ academic growth, robotic design and interaction experts have to integrate cultural responsiveness into their guidelines.

2.5. Cultural Responsiveness

Cultural differences as evidenced by expectations and beliefs about social robots are well studied. After administering a questionnaire on users’ assumptions about robots to students in Japan, Korea, and the United States, Nomura, et al. [49] found that cross-cultural differences require designers to thoroughly survey people’s expectations towards robots in the country where the robots are to be deployed. Many additional studies also indicate cross-cultural differences in attitudes towards robots about [50,51], receptivity towards tutoring [52], expectations [49], and biases about preference [53]. However, ELLs are an immensely diverse and rapidly changing group [54] making customization for a specific culture or nationality nearly impossible. Because of the heterogeneity of the population, providing appropriate instructional supports is complex and requires getting to know the students on various levels including their cultural and linguistic background [55].

For the past three decades, educators have embraced a culturally responsive pedagogy that would enhance students’ academic growth and improve their psycho-social well-being [8,56,57]. The Center for Research on Education, Diversity and Excellence has established four elements of culturally responsive instruction: (1) connecting school to students’ lives (2) engaging students in instructional conversation, (3) clear language and literacy instructional goals, and (4) Learning through joint productive activity with teachers and peers. Culturally responsive teaching has been shown to positively impact the learning outcomes [8]. When compared to standard educational curriculum, culturally responsive pedagogy for ELLs has been found to significantly increase learning gains in computer programming [58] and language outcomes [59]. In fact, it has been shown that bringing the student’s home culture into the classroom and into instruction is the most important element when working with a culturally diverse population, such as ELLs [60].

Given the importance of cultural responsiveness for ELLs in education, child–robot interactions designed to be culturally responsive are likely to increase learning gains for ELLs, however, this has not been explored. To date, explorations of cultural issues affecting robot interactions have focused mainly on adult interactions related to culturally appropriate robot behaviors such as robot gaze and greetings [61,62] as well as cross-cultural differences in attitudes towards robots [50,51,52]. Guidelines for culturally appropriate adult-robot interactions have also been proposed. Trovato et al. [61,62] recommended alignment between the nationality of the subjects and the cultural characterization of the robot’s facial expression, gestures, and ways of speaking such as greeting.

However, designing for child–robot interactions that are culturally responsive is quite different than well-established cultural differences and appropriateness that have been explored in robotics and mostly for adults. Intentional cultural responsiveness involves engaging the student in bringing their personal culture and experience into the learning environment. It also involves creating curricula with the students’ diverse backgrounds and cultures in mind and inviting their experiences to be part of the learning. As robots are expected to interact and communicate with children of different cultural backgrounds, robot interaction design for diverse learners should model evidence-based, culturally responsive instruction similar to human–human classroom instruction.

2.6. The Current Study

Given the importance of cultural responsiveness in human–human interactions in the classroom, we began an exploration of translating a culturally responsive interaction onto a social robot design to work with ELLs. Having learned from our own study that ELLs, their teachers and families preferred the Nao robot form factor, we chose to use the Nao robot for this study. It was also imperative to conduct the study in a real-world setting to better understand how interactions might differ from expectations. According to Lee et al. [63], it is important to evaluate and develop robots in social settings due to their distinct contexts. Therefore, we investigated how ELLs perceived and interacted with a Nao robot conducting a culturally responsive discussion in the school setting. In an effort to leverage the ELLs as cultural informants in the process, we hoped to answer the following questions:

• How do ELLs experience interactions with a culturally responsive, social robot in small group discussion setting?
• After experiencing the social robot interactions, what modifications or iterations do ELLs suggest to improve their experience and make the social robot more culturally and contextually appropriate?
• How do experienced ELL teachers evaluate culturally responsive robot interactions with ELLs?

3. Methods

3.1. Participatory Design and the Clue2 Framework

One way to incorporate cultural responsiveness into robot interaction designed specifically for diverse ELLs is to use a participatory design (PD) approach. PD is an appropriate approach for designing social robots due to its commitments and meaningful engagement of people in the design process [64]. In PD, the goal is not to just understand people in an effort to build systems for them, but rather to design co-operative and collaborative design relationships that can empower users and make practical or political improvements in people lives [65]. Within PD, the method of co-design has been found successful in robot design and interaction specifically design to support children [66] and teens [67], in educational settings [68], and including vulnerable children [69,70].

A participatory, human-centered approach is appropriate given the diversity of ELLs and the need to co-design robot interactions to be responsive to their cultural backgrounds. Involving culturally-diverse users in the design process allows for improved and engaging design, improves the user experience, and results in greater academic gains and ownership of the technology-based learning. The prevailing design process involves an exploration of cultural features based on a homogeneous single national group ignoring the needs of a multicultural and diverse body of learners, typical in the US. Therefore, it is efficient and productive to involve and design for a diverse body of learners. Sun [71] developed the Culturally Localized User Experience (CLUE) framework as a design thinking protocol that prioritizes human experience through ideation, prototyping and testing and has been found successful with multicultural users. Sun places importance on including the voices of participants as cultural informants in the research process and eventual design.

Given the novelty of a social robot as an educational tool for ELLs, we wanted to ensure our participants could experience a contextually appropriate interaction with a social robot prior to eliciting their feedback and insights about their experience. Therefore, we designed a small group, story-based, social-robot interaction for ELLs based upon culturally responsive guidelines. After this interaction, we engaged ELLs to describe their experiences interacting with the robot and offer valuable feedback regarding interactions to make in the robots appearance and behavior. Finally, we conducted a reflexive critique of these interactions with expert ELL teachers inviting them to further situate and contextualize the children-robot interactions. Combining these layers of data along with valuing the ELLs as expert informants provided a holistic understanding of the child–robot interactions as well as informed future iterations in the robot design to ensure a culturally responsive design.

3.2. Robot Script Creation

Based upon the guidelines for culturally responsive conversation-based instruction to aid ELLs in connecting with the text [72], we designed a conversational script based on a culturally sensitive story for the robot interaction. Mama’s Nightingale: A Story of Immigration and Separation [73] is about a little girl helping to release her mom from immigration center housing undocumented individuals. Many ELLs have prior knowledge on this topic because they encounter similar situations in their families.

The script for the robot was constructed using culturally responsive language, meaning the robot engaged the students in discussing their home culture and language as part of the conversation. For example, after introducing itself, the robot said,

I want to start today’s conversation by talking about what we like to do with our families. I’ll go first. I don’t have a family, but I like learning about families from students and teachers. (look to first student on the left) Hello, what do you enjoy doing with your family? (wait for response)

After prompting each child individually, the robot reminded them of the story they read and said,

Now I’d like to talk with you about the story you read called, Mama’s Nightingale. Can one of you tell me, what problem does Saya face in the story? (Turn to look at the student who is speaking).

The script continued like this infusing personal and culturally sensitive questions with discussion of the story they had read. When students were quiet, the robot prompted them with questions like “Can you say a bit more?” or “Can you provide more detail?” At the end of the interaction, the robot said, “Thank you all for talking with me about this story, I really enjoyed our conversation”.

After creating the first version of the script, a researcher (B.L.) field tested the script with 25 teacher candidates studying to receive ELL certification. Based upon their feedback, the script was revised slightly to further simplify the language to better match the English proficiency of the ELLs participating in the study.

3.3. Robot and Story Introduction

Given the importance of introductions to social robots [74], a researcher (B.L.) met with the student participants in their full classrooms a few days prior to the small group robot interactions. B.L. read them Mama’s Nightingale and to let them know they would be discussing this story with a social robot a few days later. B.L. also introduced them to the robot (through images) so that the idea of the social robot and the image would be familiar.

3.4. The Nao Robot

In our previous study with ELLs, their parents and their teachers [25], Nao was chosen as an appropriate and desirable educational robot. Nao [75] is a small humanoid robot, commonly used in educational settings with elementary school-aged children [76,77] and allows for customizable programming [78]. Nao is a petite robot measuring 23 inches in height, weighing 12 pounds, and having 25 degrees of freedom. Nao’s eyes can establish the feeling of joint attention with the user during the interaction, and its arms and body can generate culturally responsive gestures based on the interaction component guidelines to be developed in this project. The robotics community support around Nao including the Robot Operating System (ROS) compatibility as well as the nodal coding system, Choreographe [79], is also a big advantage. Finally, Nao was identified as the most popular robot in educational studies and demonstrated a moderate effect size on learning gains [4].

The behavior for Nao was programmed to be appropriate for child–robot interaction. Our robot script and the robot’s arms were coded to enable teleoperation during the small group interactions. Nao was programmed to gently use its arms when speaking to ensure it appeared animated, but not aggressive. Nao’s head was operated in real time to face the participant to whom it was speaking or respond to participants speaking to it. The robot was placed in a sitting position, to reduce intimidation and appear more collaborative. See Figure 1. Using our script, Nao was programmed using Choreographe for teleoperation. Nao led the discussion and was operated to ensure that each of the three students was asked at least one question individually.

3.5. Small Group Discussions and Participatory Feedback

The ELLs were invited into small group discussions with the social robot. The discussion was captured on 360 video using a Rico Theta camera, placed on the table near the robot. The students knew ahead of time that they would get to talk about the story they read in class with a social robot. After the small group discussions, a researcher (B.L.) sat with the students in their small discussion groups to briefly discuss their experience interacting with the robot. The ELLs were asked, “How did it feel to interact with Nao?” She also asked, “What do you want me to change about Nao before I bring it back to work with you next time?” The researcher also probed them for more detail asking followup questions such as, “How did that make you feel?” and “Can you tell me more about that?”.

3.6. Reflexive Critique with Teachers

ELL Teachers were invited to review video footage from our student-robot interactions and then asked a few open-ended questions about the interactions. Teachers were asked questions such as, “When you see these interactions between the ELLs and the robot, what stands out to you?” and “What do you think about the students’ gestures and behaviors; how does that compare to a typical teacher led small group discussion?” To conclude, teachers were also asked to share any risks or benefits of these interactions for ELLs with a social-robot. The reflexive critique took about 30-min and the teachers were compensated $50 for their time.

3.7. Student Recruitment and Sample

After obtaining institutional human subjects review approval, we recruited an ELL teacher as a collaborator to volunteer to work with us. Child-robot interactions took place during the school day based upon the classroom teacher’s preference. Parents were informed of the study taking place in their child’s classroom during the school day and consented to study and the use of their child’s image for research publications.

A total of 24 Spanish-speaking English Language Learners aged 8–10 participated in the study, with 14 third-graders, 10 fifth graders, 12 boys, and 12 girls. All students spoke Spanish as their home language and were receiving special pull-out English language instruction with English proficiency levels ranging from level 1 to 3 according the state WELPA test [80]. It is important to note that although all of the ELLs had the same home language, they were still quite diverse in relation to their home cultures stemming from numerous South American countries. The researcher met with them in groups of 3 for 15-min in their regular classroom for their pull-out English language instruction. The students were reminded that their participation was voluntary and each student appeared engaged and completed the interaction and the interview that followed. During the meeting, students responded to Nao’s questions on a story read to them by the researcher before the meeting. The interaction replicated a standard story discussion session.

3.8. Teacher Recruitment and Sample

ELL teachers were recruited through a Pacific Northwest urban school district via word-of-mouth. Six school staff (teachers, instructional coaches, a bilingual liaison, and an administrator) from local area elementary schools participated in the reflexive critique interviews. Many of the instructors had additional experience up to 30 years spanning different age groups from children to adults and working in environments across the United States and abroad. All the teachers were currently teaching or directing ELL programs in a Pacific Northwest, urban public school district. See

Table 1. Demographics of English language learner teacher sample (n = 6).

ID	School	Grades Taught	Gender	Exp. (yrs)	Ethnicity
T1	1,2	K-12	F	30	Caucasian
T2	1,3	K-5, 9, 10	M	1	Hispanic/Latino
T3	1	K-5	F	30	Asian, Caucasian
T4	1	K-12	F	11	Asian
T5	4,5	K-5	F	28	Caucasian
T6	1	K-5	F	19	Caucasian

for more detail.

4. Analyses and Application of Findings

A sequential analysis was done in three phases. First we explored and analyzed the child–robot interactions and their feedback as cultural informants. Then we analyzed and integrated the reflexive critiques from teachers’ observations, and finally we applied the findings as new requirements to our own social robot interactions for future studies. In phase one, the authors conducted an applied thematic analysis as described by Guest [81]. Initially, two researchers (E.A.B. and P.W.) explored the video and transcript data using low level coding and contributing excerpts to a shared collaborative platform, Miro [82]. Codes were then refined as patterns began to emerge collaboratively during discussions of the excerpts. Once salient codes were identified the transcripts and video data were explored again to search for further supporting evidence or contradictions to the emergent codes.

In the second phase we used the data from expert ELL teachers to further refine and contextualize the initial analysis of ELL data. A relfexive critique is a process by which individuals can critique observations and claims in order to provide alternative views [83]. In our study, this felt an appropriate step to help us contextualize child–robot interactions and to insights about the context of these interactions.

Finally, we collectively gathered the findings from both ELLs and teachers to inform iterations on the social robot embodiment, behavior, and interaction design to improve the experience for ELLs in future studies.

5. Results

Our exploratory findings are presented in relation to our guiding research questions. In addition, we summarize the application of our findings in the form of recommendations in

Table 2. Summary of recommendations.

Design Category	Recommendations
Rapport	Engage students to build a background story around the robot so that it fits with their community.
Appearance	Engage students to make or suggest customization for the robot
	Engage students in programming robot gaze, gestures, and body position
Greeting	Invite students to program Nao with greetings in their home languages.
Interaction	Program the robot to facilitate the students to engage with with another, not individually with the robot.
	Reduce response time of the robot so students feel heard.
Robotics Literacy	Engage the students in discussions of the robot as a programmable machine.

in Section 5.4.

5.1. Q1: How Do ELLs Experience Interactions with a Culturally Responsive, Social Robot in a Small Group Discussion Setting?

Overall the children appeared engaged and excited to interact with the Nao robot. In each of the eight groups the students were seen smiling and sometimes even giggling, likely due to the novelty of this experience. Though none of the children seems frightened or disturbed by the robot, many were surprised especially when the robot began to speak. In addition, some seemed slightly frightened by some aspects of the robot’s appearance.

5.1.1. Speechless

Although the intention of leading small group discussion with ELLs is to encourage confidence and increase language practice, a common shyness or quietness of the students occurred during the robot interactions. The speechless pattern was primarily evidenced by many behavioral examples of students covering their mouths, or hesitating to respond during the interaction. One student jumped back a bit, as if startled, when Nao first spoke to her [P5, F]. Some students covered their mouths in response to the robot asking them to respond, possibly out of shyness or surprise. In one example [P10, P11, P12] all three students covered their mouths when Nao’s eyes and body illuminated during warm-up and began to turn its head toward each student. There was a shared excitement and curiosity among the students covering their mouths in anticipation of the robot’s next movement or response. See Figure 2.

5.1.2. Social Referencing

A second concept that emerged was social referencing. All of our ELL participants used social referencing [84] (looking to one another for confirmation or a signal) during interactions with the robot. This often resulted from the robot asking one particular student to speak. Students looked to one another for confirmation before responding. One example is a student who looked to a peer, then waited for her peer to nod to her before responding to the robot [P1,F]. See Figure 3 for an example. This social referencing behavior seemed to result from genuine uncertainty about how to interact with the robot or whether to follow the robot’s instruction to respond. In another example a female student looks to her two peers for confirmation, together they speak to one another in Spanish, before she responds to the robot [P5, F].

5.1.3. A Robot Is a Curious Machine

Not surprisingly, many students had never seen a robot in person before, so this was a very novel experience, as one 3rd grade boy said, “I’ve never had a robot before. I’ve never seen one ever.” [P22,M] During our interactions with ELLs, the Nao battery became low causing Nao’s eyes to change to a purple color. When this happened, we explained to the students what was happening and that the robot needed to be charged. However, for many students this was surprising and concerning. For other students this battery event elicited curiosity. A student who had been quiet at first, felt compelled to dismiss the story discussion activity and ask about the robot, “It’s not about the [story] but how do you know if the batteries are down?” [P22, M]. In these interactions, it appeared that for some ELLs their curiosity around the robot and its functions led to increased verbal engagement.

5.2. Q2: After Experiencing the Social Robot Interactions, What Modifications or Iterations Do ELLs Suggest to Improve Their Experience and Make the Social Robot More Culturally and Contextually Appropriate?

Although fairly quiet during the robot activity itself, when asked to describe their experience interacting with the social robot, most students immediately became more verbally engaged. Students described their overall impressions of the robots appearance and behavior, their expectations for the robot’s role, as well as the potential for the robot to support them in language learning.

5.2.1. Robot Appearance and Behavior

Overall students seemed to like the robot’s general appearance. One student commented, “He’s cute” [P24, M] another said, “He’s gorgeous!” [P23, F]. Students also had suggestions that Nao should have clothing, such as a t-shirt or a hat in order to be more appropriate for school. Many students were very drawn to the robot’s eyes. This is not surprising given the eyes are an important factor in a social robot. Several students avoided eye contact while interacting with Nao, especially when Nao turned to look directly at them and invited them to speak. A couple of students avoided Nao’s gaze by tilting their head or looking away to avoid eye contact with Nao. One said, “It’s like he’s staring at me” [P9, M] which may have signaled discomfort. A female student actually articulated her fear regarding Nao’s eyes, “The only thing that I got scared of [was] his eyes.“[P19, F] See Figure 4 for an illustration.

Finally, in a nice illustration of supporting one another in sharing feedback, one student interpreted for another student in order to share her feedback which allowed a third student to chime in with his agreement.

4th grade girl (interpreting for another student who only spoke in Spanish): She got scared. It was looking at her like that several times.

4th grade boy: Yeah it’s haunted.

4th grade girl: Yeah staring like that.

At least one student was not even clear if the glowing circles were supposed to be the robot’s eyes, “Are those his eyes on the front?” [P24, F].

The position in which Nao sat was also identified as uncomfortable for some students. One said, “Yeah. I also feel scared when he sits like that. I think I like him more sitting in the chair I think” [P22, F]. Another student explicitly said, I put my hands like that [when I’m frustrated] or I’m sometimes mad and stuff. or when people would have to bother us and trying to see what our secret is” [P22, M]. “He looks kind of weird sitting like that. It would be better in a chair.” [P23, F].

Some students articulated the idea of Nao as a stranger during these interactions. A few students suggested that Nao was “strange” or described the strangeness of the robot, articulated mostly by its behaviors. A couple of students made comments suggesting that they expected Nao to recognize them or respond to them individually. One student suggested that the robot saw him as a stranger, “It’s like he’s staring at me. It’s like he doesn’t even recognize me.” [P9, M]. Another student exclaimed, “It’s like he doesn’t even see me”[P4, F].

5.2.2. The Role of a Helper

The concept of the social robot as a helper quickly emerged among the ELLs. They understood that Nao could help them to learn English. Many were excited about that opportunity. They also suggested that the role of the robot as a peer or even a learner was appropriate and desirable. In one small group, the ELLs even made the explicit suggestion that they could tell the robot a story to practice learning English.

4th grade girl 1: She wants it to talk more in English

Researcher: You want it to talk more in English?

4th grade girl 2: (spoke in Spanish)

4th grade girl 1: Because she talks a lot in Spanish. She wants to learn more English.

4th grade girl 2: (Spoke in Spanish to her peers.)

4th grade girl 1: She said for it to help her talk in English. You can make it read a story to us.

Researcher: You want Nao to tell you a story.

4th grade girl: Or we can tell her [the robot] a story.

ELLs suggested that it might be helpful for Nao to speak their language as well as help them with English. For example, one boy said, “have Nao speak Spanish and English” [P9, M] and a girl suggested maybe not just Spanish but other languages as well, “Maybe make him talk different languages” [P8, F].

5.3. Q3: How Do Experienced ELL Teachers Evaluate Culturally Responsive Robot Interactions with ELLs?

During the reflexive critiques, teachers overall saw huge potential for the role of the robot as a classroom aid for ELLs. All teachers suggested ways in which the robot could support their language development and increase their confidence. For example, one teacher said, “So I think, I think the benefits [of having an educational robot to work with ELLs] are endless” [T2, M].

Teachers suggested the student behaviors of hesitancy that we saw in speaking to the robot in addition to seeking help from one another were common and not necessarily negative. One teacher suggested those behaviors would go away with increased interactions with the robot. “I saw a lot of those gestures the first or second week I was teaching. As they began to get more comfortable, you saw those mannerisms to go away and to be themselves”. [T2, M]. Another teacher suggested the hesitancy might be culturally appropriate for initial meetings. She said, “Some of them are from different cultures and they focus on respectful ways for the elders so those kids want to listen first. They want to listen and wait…there are definitely some cultural differences” [T4, F].

We noticed ELLs social referencing and even translated for one another during interactions and feedback interviews. One teacher emphasized the importance of this collective communication as integral to culturally responsive instruction, “She looks to her other students for help. So they are guiding her and instructing her on what is going on. With other students you see that quite a bit. They are helping one another” [T2, M].

A teacher articulated this behavior as similar to have a visitor in the classroom. “You get a similar response from having a guest in the classroom because it is unfamiliar, it’s new. They are unsure if their responses are correct or if it’s what you want them to say. Additionally, a combination of this piece of technology that they are not familiar with.” [T2, M]. Another teacher made it more clear that the robot was more of a “stranger” rather than a guest. “The robot is a stranger: You know, they’re just like they would be if a stranger just started asking them questions though” [T1, M].

Cultural Responsiveness

Teachers recognized the ELLs curiosity about the robot during interactions and most felt this was a very positive thing. One teacher said, “Oh, they’re curious about the robots. I think it’s good …like a hook…for them to talk about what to change about the robot.” Another teacher suggested that the social robot interactions could continually engage ELLs as cultural informants as that engagement invites them to practice their language skills.

[T5] Teachers agreed that a social robot could exhibit cultural responsiveness. When discussing the idea of the robot speaking the ELLs home language, one teacher said, “That would be cool. And they’d be like robot greeted them and two languages or, you know, Vietnamese, or Spanish or Samoan” [T1]. Another teacher also suggested that the learning benefit of language practice with the robot could build ELL confidence and help them prepare for class discussions. He said, ELLs might feel like, “...When I go to the group, you know, like, okay, I got to practice it, and I feel more confident, you know, and that that is so the learning side of it is big.”

5.4. Application of Findings to Social Robot Interactions for ELLs

From these data we have several findings that will be immediately applied to our social robot interactions for future studies. See Table 2.

Rapport: although the ELLs were introduced to Nao prior to the story discussion robot interaction, it appears that on boarding must include a live robot interaction. ELLs could help us to co-design a class greeting in which each student has an opportunity to say hello and introduce themselves to the robot, ideally days before sitting down for the story discussion. In addition, Nao could be part of the larger classroom on a couple of occasions, again to increase familiarity. Appearance: As robot interaction designers working with a vulnerable population, it is most important to first take into consideration any components of the robot behavior or interaction that invited discomfort or fear. For this reason, Nao’s eyes are a top priority for iteration. As cultural informants, it may be most efficient and effective to engage the ELLs directly in the iteration of Nao’s eyes and gaze behavior to ensure we are modifying the robot in a way that is more appropriate. ELLs also had valuable feedback about Nao’s clothing and position. These are easy iterations that can be explored and tested in future interactions. We suspect that Nao sitting on the table detracted from our intention for Nao to be in the role of a peer. By placing Nao in a chair at the table, its role of a peer and facilitator of conversation will likely be more appropriate.

Greeting: both ELLs and teachers suggested that the robot could speak multiple languages with students in order to illustrate cultural responsiveness. Future interactions will include the robot saying hello in multiple student languages.

Interaction: through the observations of ELLs and teacher evaluations, it became clear that although common in human–human culturally responsive interactions, having Nao direct questions to individual students, may not be culturally appropriate. Therefore, future interactions will be designed to allow for Nao to be a facilitator of peer-to-peer verbal interactions, prompting students to discuss topics amongst themselves and ask questions of each other. This iteration will hopefully further encourage community building and belonging, both important factors is cultural responsiveness.

Robotics Literacy: our research team and teachers noted that many ELLs were motivated to speak and to help each other give feedback when discussing the robot and its behavior. Teachers felt confident that eliciting feedback and encouraging discussions about the robot itself might stimulate more communication and engagement from ELLS. Therefore, future interactions will include discussing the robots behavior and illustrating how the behavior is programmed may encourage and sustain engagement. By applying these iterations to our social robot interactions, we feel confident that we are one step closer to a culturally responsive interaction with a social robot for ELLs.

6. Limitations and Further Research

This particular study explored a real-world, small group child–robot interactions designed to be culturally responsive. ELL participants in the current study each come from a non-American culture, therefore these data must be interpreted using an appropriate cultural lens. This study was limited in scope to only include data from 24 children and six teachers. However, given its qualitative design, we gathered rich data embedded in a real world context enhancing its ecological validity. These data may be appropriately transferred to other similar studies and environments. The children in this study were from multiple South American cultures, but were homogeneous in their home language of Spanish. Therefore, it is imperative to gather further data from children with more diverse home languages. In addition, our participants were in a small age range, meaning child–robot interactions may vary greatly in children much younger or older. Further research is needed to explore the ways in which cultural responsiveness can be translated into child–robot interactions. In particular, further research regarding the robot’s appearance and behaviors. Utilizing ELLs as experts in their own context and participants in co-design of appearance and behavior is likely to be successful.

Future studies with a more diverse range of ELL students should also be explored to understand additional home languages and cultures (i.e., Korean, Russian, Vietnamese, etc.) Additionally, to avoid using this tool as generalizing cultural differences, it is important to combine these dimensions with additional qualitative and participatory research to understand specific contexts around various student interactions with different languages.

7. Discussion

Our study explored a culturally-responsive, child–robot interaction with ELLs in order to elicit further data about embedding cultural responsiveness in child–robot interactions with ELLs. Although the interaction was based in a culturally responsive design, ELLs were not verbally engaged with the robot until we engaged them as cultural informants. The social robot interaction did however, provide an experiential context (interacting with a social robot) from which they could provide valuable feedback.

7.1. ELLs as Cultural Informants

Returning to our CLUE methodology [71] as a process, we found this an effective and successful approach by which to gather interaction design requirements from ELLs. Perhaps not surprising, students were more verbally active in providing their feedback about the robot than they were in the robot interaction itself. This suggests that engaging them in the design and customization of the robot, may be both valuable to inform interaction design, but also to support their language gains. During the reflexive critiques, a couple of the teachers reiterated this idea of the social robot as an experiential context from which to express their experience verbally. Teachers also suggested that this feedback process elicited strong engagement and therefore could also be used to promote verbal engagement from ELLs by asking for feedback and improving the interaction design after each robot interaction. Our team plans to work with the school personnel to engage children in the customization of Nao robot to improve acceptance and belonging.

7.2. Belonging

ELLs and their teachers suggested the social robot, despite its culturally responsive dialogue, was still a “stranger.” Culturally variable attitudes and preferences toward robots may not simply be reducible to some specific factors, rather they relate to more specific social dynamics and norms [12]. Something that was evident from our current study, was a need for onboarding or rapport building with the Nao robot prior to the small group discussion. Children’s social bonds with their teachers has been shown to influence their performance. The greater rapport children have with their teachers, the more engaged they are during instructional time, resulting in higher retention of the lesson content [85]. Rapport has been identified as an important factor in successful human–robot interactions [74,86] and likely even more important in child–robot interactions with ELLs. Finally, research in implementation science suggests that adoption of new technologies requires explicit boarding and integration into the community culture [87]. Onboarding and introductions are key components to a successful implementation. A many-to-one set up will reduce the initial resistance of some children when first introduced to robots [31].

7.3. Guidelines for Interaction Design for ELLs

Belpaeme’s existing guidelines specific to robots intended to support language learning [48] mainly focus on human–robot differentiation with the goal of enhancing robotic design to make the robot behave more like humans. We propose that cultural responsiveness should also be included in guidelines for ELLs. According to Vygotsky [88], language development and leaning originate in social and cultural interactions. As instructional designers, we should be more sensitive to diversity in learners and recognize that a large amount of research has been done on white, middle-class individuals associated with Western tradition, and the resulting understanding of development and learning often incorrectly assumes universality [89,90].

Although a large area of exploration in social robotics, cultural responsiveness includes body language and must be considered in interaction design. It has been suggested that language teachers need to orchestrate combinations of gaze [91], gesture [92], and body movement [93] when interacting with their students. The non-verbal means of communication takes on a more important role when language is the object of instruction. Cultural responsive teaching behaviors involve adapting the non-verbal means of communication to align them to the cultural preferences of the language learners [94]. For example, students should not be asked to look at the teachers during conversation practice because many non-Western learners prefer not staring at each other when speaking.

7.4. Robotics Literacy and Participatory Design

The students level of surprise and almost disbelief during the robot interactions was likely a result of the pure novelty of the scenario. Therefore allowing students to have more understanding of the robot and how it operates would not only increase their robotics literacy, which would be valuable for them in terms of STEM education, but also likely increase their comfort level when interacting with the robot. In addition the ELLs provided immediate and valuable feedback for iterations on the Nao robot to make it more comfortable and appropriate. Participatory design (PD) is an approach to design and development that is most appropriate for vulnerable populations including children [95]. Co-Design, is about providing scaffolding and structure for participants to engage in the creative process of design, thereby embedding appropriateness and usability into a new technology or tool [96]. Both Belpaeme [4] and Randall [35] found robot appearance and embodiment were correlated with language learning gains. Therefore, using PD may be an effective and efficient way to gather data to inform customizations for a specific community as the culture of that community will be embedded within those data.

Co-Design and customization of the social robot might also be a protective factor in these scenarios and allow the students to have some ownership over their experience with the social robot. Co-design is a participatory approach to design that specifically minimizes the power differential in typical design, making it a successful and useful approach for working with children [97] vulnerable populations [98]. Co-Design has been shown effective in the design of social robots for children [99] and adolescents [64]. The data regarding customizing the robot to be more like a student, supports this theory.

Our exploration of cultural responsiveness in ELLs speaks to this specific population of ELLs. However, our process of engaging ELLs as cultural informants, reflecting on their direct experience of a social robot interaction may be transferable to other settings (schools and research studies) if the intention is to design culturally responsive interactions for a vulnerable ELL population. Future studies are needed to explore other home language populations as well as diverse home language groups. In addition, studies exploring this approach in older ELL populations are also needed.

8. Conclusions and Future Research

Although cultural appropriateness and cultural norms in robot interactions have been explored, cultural responsiveness has largely been ignored. Our preliminary exploration of ELL interactions with a culturally responsive social robot provides two valuable insights into how to better support social technologies for vulnerable ELLs. First, culturally responsive robot appearance and behavior in addition to dialogue may be important components to ensure successful interactions. Second, using a participatory approach such as including ELLs as cultural informants in the research and design process will ensure that social robots are designed as appropriate and valuable supports for ELLs and their communities.

As the ELL population increases, social robots may be a promising tool to improve student learning and engagement while increasing resources for ELL instruction. This study provides a process by which to use a cultural lens to explore social robot interactions as well as preliminary data suggesting that collective instruction, building rapport, and leveraging customization and co-design may be especially important components for successful and culturally sensitive instruction for English Language Learners.