The Influence of Feedback Type in Robot-Assisted Training

Abstract

Robot-assisted training, where social robots can be used as motivational coaches, provides an interesting application area. This paper examines how feedback given by a robot agent influences the various facets of participant experience in robot-assisted training. Specifically, we investigated the effects of feedback type on robot acceptance, sense of safety and security, attitude towards robots and task performance. In the experiment, 23 older participants performed basic arm exercises with a social robot as a guide and received feedback. Different feedback conditions were administered, such as flattering, positive and negative feedback. Our results suggest that the robot with flattering and positive feedback was appreciated by older people in general, even if the feedback did not necessarily correspond to objective measures such as performance. Participants in these groups felt better about the interaction and the robot.

1. Introduction

The growing field of socially assistive robotics (SAR) aims at social assistance for users on specific tasks. In human–robot interaction (HRI) literature, socially assistive robots for older people are receiving particular attention to meet the increasing healthcare needs of the aging population. Research on SAR covers many application areas, one of them being robot-assisted training where the robot provides physical or cognitive assistance. Despite the accelerating trend in robot-assisted training, little is known about the interaction dynamics of these technologies with older people. However, if robots are to be deployed in health care centers and homes, it is important to understand these interaction dynamics when we design interactive experiences for older people.

Despite the positive effects of cognitive and physical interventions for older people [1], implementing and sustaining such interventions requires a huge amount of resources and health care professionals due to the increase in the aging population [2]. Besides lacking health care professionals, SAR systems are promising tools for cognitive and physical interventions for older people, due to their physical embodiment. The interactions with a physically embodied robot were favored over virtual ones [3]; additionally, the existence of a collocated physical robot in a person’s space was found to be more persuasive, perceived more positively and result in better task performance than the robot that was shown on a screen [4]. SAR systems have been employed as exercise coaches or exercise assistants for older people [5,6]. The goal of our research is to develop SAR aids as an effective tool for cognitive and physical training for older people. We aim to investigate each aspect separately, therefore, in this paper, we explore the type of feedback as an isolated concept.

This article presents the results of a short-term study that explores the influence of feedback type (flattering, positive and negative) of a social robot in the context of physical and cognitive training along with acceptance, sense of safety and security, and performance of the participant. To assess how the type of feedback affected the various facets of participant experience in the study, we asked a series of questionnaires. In our scenario, the Pepper robot assists users in training where users need to remember (cognitive component) a sequence of arm motions and repeat after the robot (physical component).

The results show that the feedback type affects the usability related constructs of robot acceptance questionnaire (e.g., perceived ease of use, perceived usefulness and intention to use). The flattering robot, i.e., the robot that praises its users regardless of the actual performance of the users in a task, produced similar effects to flattery from computers, as presented in [7]. The authors reported that the participants who received flattery from the computer had more positive affect, better performance, more positive evaluations of the interaction and a more positive regard for the computer. In our study, while positive feedback produced a similar effect with flattering feedback, our results show that the participants who received flattering feedback from the robot had positive opinions of the robot and the interaction, and appreciated the robot more.

In the remainder of this paper, an overview of related studies regarding feedback in SAR systems and robot-assisted training is given in Section 2. The method, experimental design and measures are described in Section 3. The experimental results and data analysis are given in Section 4. Furthermore, we discuss the implications, observations and limitations of the study in Section 5 and we conclude the paper in Section 6.

2. Related Work

We present the studies investigating the effect of different feedback types in HRI in Section 2.1, and we summarize robot-assisted training studies that focus on robotic exercise coaches or exercise assistants for older people in Section 2.2.

2.1. Feedback from SAR

The impact of different feedback types in SAR has been investigated in diverse HRI settings. For instance, Midden and Ham [8] explored the effect of positive and negative social feedback on users’ energy consumption behavior. Specifically, a scenario in [8] involved participants using a simulated washing machine, and the iCat robot gave positive and negative feedback via various facial expressions or speech utterances; the results show that negative social feedback had the strongest influence on behavior change. Another study presented the effect of positive, negative, and neutral feedback of a robotic teaching assistant on perceived attraction and acceptance [9]. In the experiments in [9], participants answered a short quiz based on a short lecture given by a Nao robot and received verbal feedback. Participants who received positive feedback were more appreciative of the robotic instructor. Aside from positive and negative feedback, the relational and nonrelational feedback was investigated in a seated aerobic exercise scenario in [10]. The relational feedback involved praise of the robot for accurate completion of an exercise, reassurance in the case of failure, and personalization (e.g., referring to the user by name and referencing past experiences). In the case of nonrelational feedback, the robot did not provide positive feedback or empathy, and the robot was not personalized. The authors of [10] reported that older participants preferred the relational robot over the nonrelational one as an exercise coach. Another study presented the impact of emphatic and supportive feedback in a tutoring application, which was positive on the students’ learning performance and motivation [11]. Similarly, in [12], the effects of emphatic feedback were explored in a long term study where the robot was a chess partner for children.

There are studies investigating the motivational aspects of SAR systems in different HRI scenarios, for example, as an indoor cycling instructor [13], in cardiac rehabilitation [14], and a robot exercise buddy for overweight adolescents [15]; all three studies using the Nao robot in their experiments. Casas et al. [14] presented an architecture that provided social support and assistance during cardiac rehabilitation. Süssenbach et al. [13] developed an interactive, action-based motivation model including a feedback mechanism based on observations and analysis from human fitness instructors. They reported that participants who trained with a robot instructor had a more intensive workout and higher motivation as opposed to participants who trained without a robot. The same research group further investigated the effect of feedback in a long period of consecutive indoor cycling training over 18 days, where the robot gave social assistance and information about training performance, which resulted in higher user compliance and engagement [16]. In another study, Fridin and Belokopytov [17] presented an SAR system for physical therapy of children with cerebral palsy. Similar to the previous studies, the Nao robot was employed in the experiments and provided positive feedback and verbal encouragement accompanied by body expressions. Guneysu and Arnrich [18] presented a SAR system where a Nao robot was employed as a physical exercise coach for children. In the experiments, the robot performed basic arm motions and provided positive feedback by praising the child or gave corrective feedback by explaining verbally what was wrong or by showing the correct motion. They reported that children engaged in physical exercise and enjoyed interacting with the robot.

As these studies suggest, feedback of a SAR system is beneficial in various use cases. In this paper, we further investigate the effects of feedback type in a training robot for older people. We think that the results of this study will be helpful for providing insights about the design of SAR systems for older people.

2.2. Robot-Assisted Training

SAR systems as exercise coaches or exercise assistants have been getting particular attention in the recent decade [5,6,10,19,20,21,22,23]. Gadde et al. presented an interactive humanoid robot trainer for older people where the robot monitored the participant’s progress and gave positive feedback based on the participant’s correct arm motions [19]. Similarly, in [5], the Nao humanoid robot was employed as a training coach with positive or corrective feedback mechanisms for older people. Another study presented an engagement-based robotic coach system for one-on-one and multi-user interactions [20]. The authors of [20] reported that the robotic coach was positively accepted by older adults regardless of whether they had cognitive impairment or not. Other research has shown that physically embodied robotic coaches are preferred over virtually embodied coaches regarding their enjoyableness, helpfulness, and social attraction among older people [6]. Lotfi et al. presented a SAR system in the form of a robot exercise trainer, which was tested with 17 older people; the system provided feedback in the form of facial expressions and voice based on the participants’ performance [21]. In the same paper, the researchers reported that the SAR system was indeed found to be stimulating for older people to perform the regular exercise correctly. The acceptance of a robot as a partner in dance-based exercises among older people has been investigated in [22]; the results show that participants perceived the robot as useful, easy to use, and enjoyable. In [23], the authors aimed to assess the acceptability of a robotic exercise coach among older people. The experiments were carried out in a senior living community and the results show that older residents moderately accepted robots, while nurses and administrators were enthusiastic about them. Overall, the existing research on robot-assisted training suggests that interactive robots can engage older people. SAR systems, therefore, could potentially offer a good way of administering health interventions and health maintenance for older people.

3. Method

To explore the influence of different feedback types on robot-assisted training, we conducted a user study with 23 participants. The participants performed physical and cognitive training, which involved remembering and performing random combinations and sequences of basic arm motions shown by the Pepper robot (shown in Figure 1). The training included both a memory component (the participants needed to remember a sequence of exercises) and a physical component (the physical movements).

3.1. Participants

Twenty-three participants, 9 males and 14 females ranging from 66 to 88 years of age ($\mu$ = 73.95, $\sigma$ = 5.6) took part in the experiment. All participants’ mother tongue was Swedish. The experiment and the administered questionnaires were in Swedish. The education level was as follows: six participants had elementary school education, four had high school education, one did not answer the question. The remaining twelve participants had university education. The participants rated their technology experience, i.e., how often they use a computer or a smart phone on a scale ranging from 1 to 5 (1 = never, 5 = often (daily)). All participants use smart phones daily. The average for the scale was 4.80. The average for familiarity with robots was 1.95 (scale ranges from 1 to 5; 1 stands for not familiar at all and 5 stands for very familiar). The study designed as a between-subjects study design (except for the attitude towards robots questionnaire where all participants filled out the same questionnaire before and after the experiment) and the participants were randomly assigned to one of the feedback types. Eight participants received flattering feedback, seven participants received positive feedback, and eight participants received negative feedback from the robot (for details, see Section 3.3.1).

The participants were not informed that the robot’s feedback would be based on their performance, however almost all participants assumed that the feedback was based on their performance. Only one participant asked whether the feedback was based on performance or something else. It is also worth mentioning that all participants received feedback based on their performance at random times to keep the robot feedback realistic. In that sense, the feedback could be negative, positive or flattering regardless of the participants’ assigned group (flattering, positive, or negative); however, the majority of feedback was based on their assigned group. The experiment took between 30 min and 1 h, depending on the participant.

3.2. The Robot

The robot used in our study was Pepper (see Figure 1), a humanoid robot with 20 degrees of freedom (DOF), a height of 1.2 m, and a weight of 28 kg. There are two DOFs in the head (pan and tilt), two DOFs in the hips, one DOF in the knee, and three DOFs in the base. Each arm has six DOFs: two DOFs in the shoulder, two DOFs in the elbow, one DOF in the wrist, and one DOF in the hand. The robot is equipped with three multidirectional wheels, four directional microphones, six touch sensors, several infrared sensors, laser sensors and sonar sensors, as well as two loudspeakers. There are three cameras: two RGB cameras (forehead and mouth) and one 3D camera located behind the eyes [24].

3.3. Experimental Design

Our methodology consists of physical and cognitive training at three levels of difficulty: easy, medium and hard. The level of difficulty is based on the number of arm motions in a sequence.

Our user studies began with informing the participants about the experimental procedure and the robot. Before starting, each participant read and signed the consent form. Thereafter, they filled out the pre-experiment questionnaire including demographics (age, gender, and technology experience), the SF-12 health survey and the attitude towards robots questionnaire (details are given in Section 3.4). The training started with the robot welcoming the participant and providing information about the training. Then, the participants started with a warm-up session to familiarize themselves with the experimental procedure. Since people tended to exercise during the demonstration, the warm-up session helped them to understand that they were supposed to wait until the robot had completed all movements. Next, the training continued with the easy level.

The easy level included 10 exercises, 5 of them had one (arms up, arms side, etc.) and 5 of them had two successive arm exercises (arms up and arms to the side, arms in front and arms up, etc.). At the beginning of each exercise, there was a beep sound indicating that a new exercise was about to start. After the robot finished demonstrating the exercise, the robot informed the participant that it was her/his turn. When the participant finished repeating the exercise, the participant received feedback based on her/his assigned group (i.e., flattering, positive or negative). The robot gave feedback by using verbal utterances in Swedish; these utterances were recorded sound files by a native speaker (the second author of this paper). The experiment continued with the medium and hard level consecutively. In between levels, there was a 30 s break. There were 8 exercises at the medium level comprising three (e.g., arms up and arms in front and arms to the side, etc.) and four successive arm motions. The number of exercises at the hard level was 6, comprising five and six successive arm exercises. Each level was approximately 4 min long. Our scenario included both physical and cognitive training since all participants needed to both remember and physically perform the sequence of arm motions. As detailed above, the scenario starts with one arm motion and increases to up to six arm motions based on the average human memory span, $7\pm 2$ [25]. A picture from the interaction is given in Figure 2. The participants’ acceptance, sense of safety and security, and attitude towards robots were measured through questionnaires administered after having exercised with the robot.

3.3.1. Conditions

Although the effects of various types of feedback (corrective, empathetic, supportive, etc.) from SAR have received considerable attention, as mentioned in Section 2.1, the studies in SAR literature mostly focused on positive feedback from SAR. However, different feedback types can have different influence on different types of tasks [26] and user groups. As an example, in [8], negative feedback from an SAR was found to have the strongest influence on behavior change. Both negative and positive feedback can have an increasing or decreasing effect on performance and motivation [27]. Furthermore, we aimed to investigate the feedback mechanism as an isolated concept. Since the type of feedback can have different effects on various tasks, we started with the two extremes (i.e., flattery and negative) in order to understand the reactions of older user groups.

In this between-subjects design study, the robot gave feedback about the participants’ performance, which was flattering, positive, or negative after each exercise. Note that an exercise in our scenario consists of several successive arm motions, as explained in the previous section. In order for type of feedback to have some realism, the robot was also programmed to interleave feedback based on participants’ actual performance. This was a design decision, as a robot which consistently gives positive feedback or consistently gives negative feedback regardless of performance may be viewed as scripted and less interactive.

Flattering feedback: In [7], the authors used the definition “communicating positive things about another person without regard to that person’s true qualities or abilities” for flattery. We adopt the same definition in our study. In this condition, the participants received flattering feedback (“You are really good!”, “Excellent”, etc.) regardless of their performance. There were 8 participants in this group.

Positive feedback: Participants in this group received positive feedback (“Good”, “You did well”, etc). There were 7 participants in this group. Both flattering and positive feedback praised the participant, but, unlike the flattery, positive feedback was less exaggerated.

Negative feedback: In this case, the robot was never satisfied with the performance of the participants. Participants in this group received feedback like “Ok”, “You have been better”, “Come on!”, etc. Note that, even though we considered this case as negative feedback, the robot never used any negative word (e.g., “bad”) to avoid diminishing the motivation of the participants. There were 8 participants in this group.

3.4. Measures

In this section, we explain the administered questionnaires in detail. In the first part of the pre-experiment questionnaire, the participants were asked about their demographic information (age, gender, and technology experience), SF-12 health survey and their attitude towards robots. After the interaction, they answered the post-experiment questionnaire including a slightly modified version of the Almere robot acceptance questionnaire, the sense of safety and security questionnaire, a task evaluation questionnaire and once again the attitude towards robots questionnaire. In the robot acceptance questionnaire, the attitude towards robots questionnaire and the task evaluation questionnaire, the participants indicated their level of agreement on a five-point Likert scale ranging from “1 = strongly disagree” to “5 = strongly agree”. The sense of safety and security questionnaire was a five-point semantic differential scale (for details, please see [28]). Finally, the participants were asked about their opinions of the interaction and the robot; however, this part was not an interview, but more similar to a little talk.

SF-12 Health Survey: The 12-item Short Form Survey (SF-12) is one of the most widely used questionnaires for assessing self-reported health. The questionnaire comprises a selection of 12 items from the Medical Outcomes Study (MOS) 36-Item Short-Form Health Survey (SF-36) [29]. Two summary scores; a mental component score (MCS-12) and a physical component score (PCS-12), can be reported from the questionnaire. This questionnaire allows investigating the perception of the state of physical and mental health. The perceived health of older people is related to their sense of safety and security [30]. In that sense, we analyzed the correlation between SF-12 summary scores and the sense of safety and security questionnaire.

Robot Acceptance Questionnaire: Technology acceptance models focus on people’s perceptions of technology attributes to predict acceptance. Heerink et al. presented the Almere model on acceptance of assistive social technology [31]. The Almere model was adapted from the Unified Theory of Acceptance and Use of Technology (UTAUT) [32] and was designed specifically for use by older people. We adapted the Almere questionnaire to our study with some slight modifications. In particular, we excluded the facilitating conditions construct and we redesigned the intention to use construct by adding two items ourselves and including two items from other studies. In our scenario, the participants interacted only once with the robot. Thus, we excluded the questions that require multiple interactions for obtaining a proper idea of the robot. The ad hoc version is given in

Table 1. The used acceptance questionnaire [31].

Construct	Item
ANX	If I should use the robot, I would be afraid to make mistakes with it. *
	If I should use the robot, I would be afraid to break something. *
	I feel relaxed interacting with the robot (taken from [33]).
	I find the robot scary. *
	I find the robot intimidating. *
ATT	I think it’s a good idea to use the robot.
	The robot would make life more interesting.
	It’s good to make use of the robot.
ITU	If the robot was available, I would use it (taken from [34]).
	I would recommend the robot to my friends (we added this item).
	I would choose to interact or communicate with the robot outside of this study (taken from [35]).
	I would exercise with the robot again (we added this item).
PAD	I think the robot would be adaptive to what I need.
PENJ	I find it enjoyable interacting with the robot (taken from [36]).
	I enjoy exercising with the robot (modified item for our scenario).
	I find the robot enjoyable.
	I find the robot fascinating.
	I find the robot boring. *
PEOU	I think I can use the robot without any help.
	I think I can use the robot when there is someone around to help me.
	I think I can use the robot when I have a good manual.
PS	I think the robot is nice.
	I found the robot pleasant to interact with
PU	I think the robot is useful to me.
	It would be convenient for me to have the robot.
	I think the robot can help me with many things.
	Using a robot would enhance my effectiveness in my daily life (taken from [37]).
	Using a robot would improve my daily life (taken from [37]).
SI	I think it would give a good impression if I should use the robot.
SP	When interacting with the robot I felt like I’m talking to a real person.
	It sometimes felt as if the robot was really looking at me.
	I can imagine the robot to be a living creature.
	I often think the robot is not a real person.
	Sometimes the robot seems to have real feelings.
Trust	I would trust the robot if it gave me advice.
	I would follow the advice the robot gives me.

* Reverse item. ANX, Anxiety; ATT, Attitude Towards Technology; ITU, Intention To Use; PAD, Perceived Adaptiveness; PENJ, Perceived Enjoyment; PEOU, Perceived Ease of Use; PS, Perceived Sociability; PU, Perceived Usefulness; SI, Social Influence; SP, Social Presence.

.

Attitude Towards Robots Questionnaire: To investigate older people’s attitudes towards robots, we used a questionnaire that is a combination of the Negative Attitudes Toward Robots Scale (NARS) [38] and attitude towards robots (ATT) construct (four items) in the robot acceptance questionnaire presented in [34]. NARS has been developed to gauge changes in a person’s general attitude towards robots over time based on prolonged robot interaction studies with the individual [38]. The scale consists of three subscales, but we only used S1: “Negative Attitude toward Situations of Interaction with Robots” (six items). The items of the questionnaire are given in

Table 2. The used attitude towards robots questionnaire.

ATT from [34]	It is a good idea to use the robot to help me with everyday tasks in the future. *
	The robot would make life more interesting and stimulating in the future. *
	It is good to make use of the robot to help me with everyday tasks today. *
	The robot would make life more interesting and stimulating today. *
NARS S1 [38]	I would feel uneasy if I was given a job where I had to use robots.
	The word “robot” means nothing to me.
	I would feel nervous operating a robot in front of other people.
	I would hate the idea that robots or artificial intelligences were making judgments about things.
	I would feel very nervous just standing in front of a robot.
	I would feel paranoid talking with a robot.

* Reverse item.

. The participants were asked to fill out the questionnaire twice (before the interaction and after the interaction) to allow us to study how a one-time short interaction may influence potential users’ attitudes towards robots.

Task Evaluation Questionnaire: The participants evaluated the training with the Pepper robot by answering an 8 item task evaluation questionnaire. All items were rated between 1 and 5, which correspond to “Difficult” and “Easy”, “Very little” and “Very much” or “Strongly disagree” and “Strongly agree”, respectively. The questionnaire items are given in

Table 3. The used task evaluation questionnaire.

1	How easy/difficult was it to follow the training? 1
2	How easy/difficult (in terms of physical effort) were the exercises you were asked to do?
3	How easy/difficult were the exercises you were asked to remember?
4	To what extent did you feel stressed? 1
5	How hard was it to understand the robot? 1
6	How much did you like this? 1
7	I can see myself getting used to exercising with the robot on a daily basis.
8	The robot looks appropriate as a training coach.

¹ These items are taken from http://pages.cs.wisc.edu/~bilge/private/RobotEvaluationQuestionnaire.pdf. Questions 1–3 range Difficult–Easy, Questions 4–6 between Very little–Very much, and Questions 7–8 range between Strongly disagree–Strongly agree.

.

Sense of Safety and Security Questionnaire: We also investigated the participants’ level of comfort and feeling of safety and security during the interaction by using a sense of safety and security questionnaire. We believe that sense of safety and security should be considered as one of the dimensions of robot acceptance. We define sense of safety and security as “the user’s perceived state of being protected from physical and psychological harm during the interaction with a robot”. We reviewed the studies about older people’s sense of safety and security in gerontology literature and designed a questionnaire. This questionnaire was evaluated in several user studies [28,39]. It is a semantic differential scale questionnaire containing sets of adjective pairs (insecure–secure, anxious–relaxed, uncomfortable–comfortable, lack of control–in control, unsafe–safe, unfamiliar–familiar, unreliable–reliable, and scary–calming).

Performance of the Participants: We collected data from 25 persons; each person repeated each arm motion 5 times. We recorded a video of each repetition, and extracted images from these videos. These images constituted our dataset for arm exercise recognition. We used transfer learning for for exercise recognition by retraining the final layer of Google’s Inception-v3 model [40], a deep convolutional architecture trained on the Imagenet dataset. The final layer of the Inception-v3 model was trained by using 80% of the data of all the subjects for training, 10% for validation and 10% for testing. We have 6 different arm motions: idle, arms-up, arms-side, arms-front, arms-front-bending, and arms-side-bending (see Figure 3).

The test accuracy of the exercise recognition on 10% of the self-built dataset is 95.6%. The trained model and the code for the exercise recognition are given in the following link: https://github.com/neak26/exercise-recognition.

Due to privacy concerns given that the collected data are RGB videos, the dataset is not made publicly available. The trained model was used in real-time exercise recognition and provided feedback at random times based on the actual performance of the user. As mentioned above, it was provided through a realistic feedback mechanism. We also kept track of each participant’s performance (the total number of correctly remembered and performed motions). We explored the relation between performance and other measures.

4. Results

We conducted a reliability analysis to check the internal consistencies within the items in the used questionnaires. Cronbach’s $\alpha$ was used for measuring the internal consistencies within the items of a test [41]. The $\alpha$ coefficient ranges from 0 to 1 representing the overall assessment of a measure’s reliability. A value over 0.7 is considered as acceptable. Throughout this section, we used Analysis of Variance (ANOVA) to test the effect of feedback in different measures. Norman [42] provided compelling evidence that parametric statistics can be used with ordinal data such as Likert scales and small sample sizes without “fear of coming to the wrong conclusion”. However, we also checked the non-parametric equivalent of ANOVA: the Kruskal–Wallis test results. The results of ANOVA and Kruskal–Wallis test were consistent in terms of being statistically significant or not, here we reported only the ANOVA results.

4.1. SF-12 Health Survey

The results of SF-12 showed that the majority of our participants are in a good health condition (PCS-12: $\mu$ = 51.27, $\sigma$ = 8.44, min–max: 26.78–61.19 and MCS-12: $\mu$ = 56.51, $\sigma$ = 5.65, min–max: 41.02–62.20). Note that higher scores indicate a better health condition. We analyzed the correlation between SF-12 scores and the sense of safety and security of the participants during the interaction. The participants interacted with the robot only once and it was a relatively short interaction. We did not find any significant correlation between the SF-12 scores and the sense of safety and security. However, we will investigate this relation in long-term studies by comparing the baseline and follow-up assessments to observe the changes over time. We believe that robot-assisted training would improve the physical and mental components in regular usage.

4.2. Robot Acceptance Questionnaire

We used a slightly modified version of the Almere [31] questionnaire, since not all the questions were suitable for our scenario. We tested internal consistencies of the constructs by calculating Cronbach’s $\alpha$. The Cronbach’s $\alpha$ value for the Anxiety (ANX) construct was 0.27, which indicates that the $\alpha$ value for this construct is unacceptable. Therefore, we revised the items by computing the correlation of each test item and deleting the items with low correlations, as suggested in [43]. Once we had removed poorly correlated items, the Cronbach’s $\alpha$ value increased to 0.74. The remaining items were “I find the robot scary” and “I found the robot intimidating”. The Cronbach’s $\alpha$ value for the construct Perceived Sociability (PS) was 0.048, which was not acceptable, therefore, we discarded this construct in our analyses (Cronbach’s $\alpha$ and ANOVA). The Cronbach’s $\alpha$ values for the remaining constructs had good internal consistencies (see

Table 4. Cronbach’s α values for Almere constructs.

Construct	Cronbach’s α
ANX	0.74
ATT	0.88
ITU	0.93
PENJ	0.9
PEOU	0.73
PS	0.048
PU	0.94
SP	0.87
Trust	0.95

).

After we discarded some of the items and constructs, we tested the internal consistency of the questionnaire by calculating Cronbach’s $\alpha$. The Cronbach’s $\alpha$ value for the modified version of the Almere [31] questionnaire was 0.94, which indicates that the questionnaire items had a good internal consistency. A one-way ANOVA was conducted to compare the effect of feedback type on the remaining constructs in the Almere questionnaire of robot acceptance, and no significant results were found. Further analyses were then conducted for each adopted construct of Almere questionnaire.

Anxiety There was no statistically significant difference between groups for the remaining items of the ANX construct [F(2.20) = 2.67, p = 0.09], as determined by the one-way ANOVA.

Attitude Towards Technology There was a statistically significant difference between groups for the ATT construct as determined by the one-way ANOVA [F(2,20) = 4.18, p < 0.05]. A Tukey HSD post-hoc test indicated that the mean score among those receiving positive feedback ($\mu$ = 3.86, $\sigma$ = 0.69) was significantly higher than in the negative feedback condition ($\mu$ = 2.75, $\sigma$ = 1.04), p < 0.05. However, the flattering feedback condition ($\mu$ = 3.75, $\sigma$ = 0.70) did not significantly differ from positive and negative feedback conditions. Taken together, these results suggest that positive feedback leads to a better attitude towards technology.

Intention To Use A statistically significant difference was found for the ITU construct of the questionnaire with [F(2, 20) = 8.54, p < 0.001]. Post-hoc comparisons using the Tukey HSD test indicated that the mean score for the flattering feedback condition ($\mu$ = 3.75, $\sigma$ = 0.70, p < 0.05) and positive feedback condition ($\mu$ = 3.71, $\sigma$ = 0.75, p < 0.05) was significantly higher than the negative feedback condition ($\mu$ = 2.12, $\sigma$ = 1.13). Taken together, flattering feedback and positive feedback positively influenced users’ intention to use.

Perceived Enjoyment There was no statistically significant difference between groups for the PENJ construct [F(2,20 )= 1.21, p = 0.31].

Perceived Ease Of Use A statistically significant difference was found for the PEOU construct with [F(2,20) = 4.67, p < 0.05]. Tukey HSD post-hoc analysis revealed that the flattering feedback condition score ($\mu$ = 3, $\sigma$ = 0) was significantly lower than the negative feedback condition score ($\mu$ = 4.12, $\sigma$ = 0.83), suggesting that the participants who received a negative feedback perceived the use of the robot easier. The definition of perceived ease of use was given as “the degree to which one believes that using the system would be free of effort” [31]. Interestingly, the participants who received negative feedback perceived the use of robot easier. This may be explained by humans’ tendency to anthropomorphize non-human creatures [44], and perception of the robot as a social actor with a higher agency, which might be the case in flattering feedback condition. However, should be further investigated.

Perceived Usefulness A statistically significant difference was found between groups for the PU construct for the three feedback types [F(2, 20) = 8.39, p < 0.001]. Post-hoc comparisons using the Tukey HSD test indicated that the mean score for the flattering feedback condition ($\mu$ = 3.88, $\sigma$ = 0.35) and positive feedback condition ($\mu$ = 3.57, $\sigma$ = 0.97) were significantly higher than the negative feedback condition ($\mu$ = 2.25, $\sigma$ = 1.04) (p < 0.001). This construct was considered as one of the determinants of the technology acceptance [45]. Taken together, these results suggest that the type of feedback has an effect on the perceived usefulness of the robot.

Social Influence The Social Influence (SI) construct of the Almere questionnaire has two questions, but we have only used one question (“I think it would give a good impression if I should use the robot”) and we discarded the other question (“I think the staff would like me using the robot”) since it was not suitable for our scenario. There was a statistically significant difference between groups for this question [F(2, 20) = 9.63, p < 0.001]. A post-hoc Tukey HSD test showed that the participants in the flattering feedback group ($\mu$ = 3.87, $\sigma$ = 0.35) differed from participants in the negative feedback group ($\mu$ = 1.87, $\sigma$ = 1.24) significantly (p < 0.001). However, the positive feedback condition ($\mu$ = 2.86, $\sigma$ = 0.9) did not significantly differ from flattering and negative feedback conditions. Therefore, we can conclude that the type of feedback has an influence on participants’ perception of the social influence of the robot.

Social Presence, Perceived Adaptiveness, and Trust There was no statistically significant difference between groups according to one-way ANOVAs for the Social Presence (SP) [F(2,20) = 0.26, p = 0.77], Perceived Adaptiveness (PAD) [F(2.20) = 0.15, p = 0.86], and Trust [F(2,20) = 1.6, p = 0.22] constructs of the Almere robot acceptance questionnaire.

In summary, this research has established that older people evaluated the robot with flattering and positive feedback more positively. The attitude towards technology, intention to use, perceived usefulness, and social influence constructs were rated significantly higher in the flattering and positive feedback groups. Conversely, the perceived ease of use construct was rated significantly higher in the negative feedback group. There was no statistically significant difference for the other constructs of the robot acceptance. The box-plots of Almere constructs are given Figure 4.

4.3. Attitude Towards Robots Questionnaire

We calculated the internal consistency of the items in the attitude towards robots questionnaire by calculating Cronbach’s $\alpha$, which was 0.89. It is important to note that we reversed the first four items, since they were positive expressions (see Table 2). This questionnaire was administered twice, before the experiment and after the experiment, to see if the interaction with the robot had any effect on participants’ attitude towards robots. We performed a paired t-test to test the difference, which indicated that there was no significant difference. We further analyzed the post-experiment questionnaire by applying one-way ANOVA to reveal the effect of feedback on the attitude towards robots. There was no significant difference between the groups.

We also analyzed the gender effect, however, there was no statistically significant difference between males and females on both the before and after questionnaires. A Pearson analysis was run to determine the relationship between the responses of post attitude towards robots questionnaire and robot acceptance questionnaire. There was a moderate, negative correlation between attitude towards robots and acceptance (r = −0.57, p < 0.05). Since we reversed the positive items in the attitude towards robots questionnaire, as expected, the people who have a negative attitude towards robots tend to be less accepting the robots.

4.4. Task Evaluation Questionnaire

The Cronbach’s $\alpha$ for the task evaluation questionnaire was 0.74. We applied one-way ANOVAs on each item in the task evaluation questionnaire. There was a statistically significant difference between groups for the question “How much did you like this?” [F(2, 20) = 5.45, p < 0.05]. Tukey HSD post-hoc test revealed that the score for the flattering feedback group ($\mu$ = 4.12, $\sigma$ = 0.64) was significantly higher than the score for the negative feedback group ($\mu$ = 2.87, $\sigma$ = 0.64) with p < 0.05, indicating that participants in the flattering feedback group liked exercising with the robot more than the participants in the negative feedback group. For the question “The robot looks appropriate as a training coach.”, there was a statistically significant difference between groups with [F(2, 20) = 5.46, p < 0.05]. Tukey HSD post-hoc test showed that the response from the flattering feedback group ($\mu$ = 3.87, $\sigma$ = 0.35) and the response from the negative feedback group ($\mu$ = 2.5, $\sigma$ = 1.30) as well as the response from the positive feedback group ($\mu$ = 3.71, $\sigma$ = 0.75) and the response from the negative feedback group, respectively, differed significantly. The mean score of the flattering feedback group was the highest, showing that more positive feedback affected participants to perceive the robot as more appropriate to be as a training coach.

4.5. Sense of Safety and Security Questionnaire

The Cronbach’s $\alpha$ for the sense of safety and security questionnaire was 0.88. The feedback type had no significant influence on the sense of safety and security. However, regarding the question “I think robot is safe-unsafe”, the groups differed significantly with [F(2, 20) = 3.91, p < 0.05] and the Tukey HSD post-hoc test showed that the mean score of the flattering feedback group ($\mu$ = 4.37, $\sigma$ = 0.74) was significantly higher than the mean score of the negative feedback group ($\mu$ = 3.12, $\sigma$ = 1.12) with p < 0.05, indicating that the flattering feedback indeed affects as how safe participants perceived the robot. The question “I think the robot is scary-calming” was again significantly different between groups [F(2, 20) = 3.55, p < 0.05]. Once again, the flattering feedback group ($\mu$ = 4.25, $\sigma$ = 0.70) was significantly different from the negative feedback group ($\mu$ = 3, $\sigma$ = 1.06). Hence, the participants in the flattering feedback group perceived the robot as more calming.

There was a moderate positive correlation between the robot acceptance and the sense of safety and security (Pearson correlation with r = 0.57, p < 0.05). This result supports our claim that the sense of safety and security should be considered as one of the dimensions of acceptance and a key requirement for successful HRI.

4.6. Performance

The total number of arm motions during each interaction between the robot and each participant was 78 of which they performed on average 57 correctly. There was no statistically significant difference in the performance between the different feedback groups. However, interestingly, the average performance of the participants in the flattering feedback group was the highest (59.5), followed by the positive feedback group (55.85) and the average performance of the participants in the negative feedback group was the lowest (54.37). We might conclude that more positive feedback of the robot produces a higher task performance, however, this needs to be further tested with more participants. It is worth to mention that the correlation between the mental component of SF-12; PCS12 was close to be moderately correlated (Pearson correlation test, r = 0.41, p = 0.051). This can be explained by the prominent role of the cognitive component in our scenario. After all, while the execution of arm motions is simple, participants had difficulties remembering the exercise sequence. As the number of arm motions increase, the number of correctly remembered exercises decrease dramatically (

Table 5. Average performance for each difficulty level.

Difficulty	Flattering Feedback Group	Positive Feedback Group	Negative Feedback Group	Total Arm Motions
Level 1	16.9	16.7	16.6	17
Level 2	23.9	23	21.4	28
Level 3	19.4	16.1	16.4	33

). In our future studies, the maximum number of arm motions at the hard level will be five since none of our participants performed well with six arm motions in a sequence.

5. Discussion

The presented study investigated whether the type of feedback (flattering, positive, and negative) given by a socially assistive robot influence the various facets of participant experience in physical and cognitive training. The research is motivated by the importance of cognitive and physical interventions for older people and urged by the shortage of health care professionals as well as the importance of physically embodied and physically collocated robots. In the study, 23 participants received negative, flattery, or positive feedback during robot-assisted training. The training was conducted at three consecutive levels: easy, medium and hard. This study conveys some insights about the effects of the feedback type on robot acceptance, sense of safety and security, attitude towards robots, task evaluation and performance, and their interrelations. Our results suggest that the robot with flattering and positive feedback was appreciated by older people in general.

The studies investigating the type of feedback is summarized in Section 2.1. Although there are studies investigating positive and negative feedback in SAR [8,9], our study focused on a particular user group (older people) and a particular scenario (physical and cognitive training). Since our purpose is to design an interactive training robot for older people, the experiment presented in this paper provide insight into our future work. We will avoid negative feedback. As mentioned above, both negative and positive feedback can have an increasing or decreasing effect on performance and motivation [27]. Besides, there is evidence that negative feedback of a robot can be effective in behavior change [8], however, in our scenario and user group, this was not the case.

In our experiment, the two feedback types flattering and positive feedback were rather similar, but flattering feedback was an exaggerated version of positive feedback. We would expect that the users in the flattering group would appreciate the robot more and perceive the robot differently based on the administered questionnaires. However, the responses in flattering and positive feedback group were similar. However, our results coincide with previous findings by Fogg and Nass [7], who found that participants receiving flattery feedback from a computer were more positive. The results show that the participants who received flattering or positive feedback rated the attitude towards technology, intention to use, perceived usefulness and social influence constructs of the robot acceptance questionnaire higher than the participants in the negative feedback group. These results provide an indication that older individuals would be more eager to use a robot with a positive attitude. Thus, especially with regard to deployment of robots in homes and health care centers, HRI designers can enhance user experience with generous positive feedback of robots. It should be emphasized that the feedback in this experiment did not consider the performance of the participants. Hence, how the participants rated the experience may be influenced by how well they think that the feedback matches their performance. In addition, while one might imagine that the difficulty level would remain constant or decrease when a person has difficulties following the training or is experiencing the sense of fatigue, the robot in this experiment increased the difficulty level.

Even though all of our participants use smartphones daily, they do not use new technology to the same extent as younger adults. However, older people may adopt social robots better if we could discover interaction dynamics such as the type of feedback. The participants in the flattering and positive feedback group liked exercising with the robot more and perceived the robot appropriate as a training coach. Furthermore, in the sense of safety and security questionnaire, the participants in the flattering feedback group perceived the robot as safer and calmer.

The performance-related feedback of a robot had been found to have an impact on the users’ task performance, enjoyment, engagement and motivation [46]. In our study, the average performance was the highest in the flattering feedback group. This finding is again consistent with flattery from a computer, which resulted in better performance [7]. In the post-experiment attitude towards robots questionnaire, interestingly, the response in the positive feedback group was the highest towards the idea of robots and artificial intelligence making a judgment about things. Nuanced responses would be interesting for future work. Conversely, the perceived ease of use construct was rated the highest in the negative feedback group, the participants in this group perceived the use of the robot easier. One reason for that perceived ease of use was statistically significantly higher in negative feedback group than others might be the matching between their performance and the feedback they received from the robot. Hence, the participants in the positive and flattering feedback groups may have had problems understanding how the robot works. To enhance the robot acceptance, people should feel safe around the robot. We claim that the sense of safety and security should be considered as one of the dimensions of robot acceptance. This claim is supported by a moderate correlation between acceptance and the sense of safety and security. Anxiety toward the robot and the interaction with it was minimal, and the type of feedback had no influence on the participants’ anxiety toward the robot. It is an important construct when designing SAR systems aimed to be used by older people every day. The majority of the participants expressed negativity toward the notion of having a robot trainer in daily use and in the home environment during our short informal conversations after the experiment. Nevertheless, regardless of the feedback type, the majority of the participants commented that they would be open to using a robot at later stages if they were dependent and alone in their homes. They would prefer to have a robot instead of being alone or being visited by different health care personnel. We observed that some of the participants attributed the robot as a “social actor”, as in the Computer as Social Actor (CASA) paradigm (the perceptions of computers as playing social roles) [47]; they displayed politeness towards the robot by thanking it for the flattering and positive feedback. As mentioned above, the robot asked questions as feedback at random times, and all of the participants answered questions as well as smiled to the robot. Another interesting observation was that participants in the negative group tended to have doubts about themselves thinking that they were underperforming because of the robot’s negative feedback even though they were performing well. They started to copy the robot and tried to imitate the robot’s arm and hand motions exactly. They also made comments such as “It is more difficult than I thought”. Thus, this study shed light on some interesting interpretations, e.g. a robot can manipulate its interactant in objective task measures.

It is important to note that, in SAR systems, we should not only focus on good impressions about robots but also focus on making the participants feel good about themselves. In our study, flattering and positive feedback from the robot was appreciated by older participants. Further, the average for the number of correctly performed motions was higher in these groups. Our study had some limitations. Except for one participant, all participants interacted with the Pepper robot for the first time, therefore, the study had a novelty effect. Peek et al. [48] pointed out that the predictive factors of acceptance can change over time of use, thus future research would include long-term trials. Most of our participants were healthy, independent, and physically active individuals; we think that dependent older people would appreciate the robot more. Another limitation was that all of our participants were Swedish, thus, for generalizing the results, a cross-cultural study would be needed.

6. Conclusions

This paper reports the results of the influence on feedback type on older people’s interaction experience. The flattering and positive robot was well received, and the participants in these groups evaluated the robot more favorably. Our experiment hence suggested that the positive and flattering robot being accepted better by older adults. Even though negative feedback does not include any negative words, the robot was not appreciated by older people in this group. Our study included only verbal feedback and it affected many facets of the interaction experience with a robot. As human–robot interaction designers, we should focus on a robot with more positive attitude for being accepted by older people. We also believe that will lead to a better sense of safety and security. This study provided insight into our future work: our training robot will avoid negative verbal feedback. We will also consider varying the level of difficulty, which would better fit the particular participants. In our future work, we will conduct longer-term studies, as well as investigate the relationship between interaction-related objective aspects of the interaction (facial features of participants) and the participants’ subjective evaluations (e.g., questionnaires), in particular the relationship between objective findings and evaluations.