A Robot Has a Mind of Its Own Because We Intuitively Share It

Abstract

People perceive the mind in two dimensions: intellectual and affective. Advances in artificial intelligence enable people to perceive the intellectual mind of a robot through their semantic interactions. Conversely, it has been still controversial whether a robot has an affective mind of its own without any intellectual actions or semantic interactions. We investigated pain experiences when observing three different facial expressions of a virtual agent modeling affective minds (i.e., painful, unhappy, and neutral). The cold pain detection threshold of 19 healthy subjects was measured as they watched a black screen, then changes in their cold pain detection thresholds were evaluated as they watched the facial expressions. Subjects were asked to rate the pain intensity from the respective facial expressions. Changes of cold pain detection thresholds were compared and adjusted by the respective pain intensities. Only when watching the painful expression of a virtual agent did, the cold pain detection threshold increase significantly. By directly evaluating intuitive pain responses when observing facial expressions of a virtual agent, we found that we ‘share’ empathic neural responses, which can be intuitively emerge, according to observed pain intensity with a robot (a virtual agent).

1. Introduction

“Does a robot (a virtual agent) have a mind of its own?” This question has long been debated. Supported by remarkable improvements and advances in artificial intelligence, the answer “Yes” has recently become the dominant view, mainly in engineering. Researchers in this field have eagerly adopted neuro-computing to understand learning systems, model networks, and transform information from various modalities with respect to computational neuroscience. Such engineering applications have resulted in the distribution of autonomous robots like Siri (Apple Inc., Cupertino, CA, USA), Alexa (Amazon.com, Seattle, WA, USA), and Pepper (Softbank Robotics.com, Tokyo, Japan) in our everyday life, and these robots can provide reasonable responses to requests and inquiries by people. Furthermore, autonomous robots have achieved the ability to understand and respond to semantic tasks, resulting in a “theory of mind” [1]. People certainly perceive the mind of a robot through their semantic interactions with it; however, such minds simply reflect the intellectual and cognitive processes of robots, which are autonomously intended and acted. As an alternative to the intellectual mind, people perceive the affective mind in a wide variety of entities, including humans and animals, without any intellectual actions or semantic interactions (for example, when just observing a baby giant panda playing with a toy, people intuitively perceive the panda as feeling happy). Thus, people perceive the mind in two dimensions: One is intellectual, and the other is affective [2]. From the standpoint of the affective mind, many psychological researchers still do not agree that a robot (a virtual agent) has minds of its own. Furthermore, specifying the intellectual mind, some researchers argue that people merely create semantic meanings for actions by a robot on their own and it is undefined whether a robot really has a mind of its own. According to this argument, the affective mind of a robot might be also semantically created through people’s cognitive processes based on their experiences to date. Modelling multimodal expressions of affective minds in a robot has shown that people perceive it as having affective minds [3]. However, it still has not been confirmed whether the minds are created semantically through people’s own cognitive processes or a robot itself has such minds. We investigated whether pain experiences are modulated intuitively and unconsciously when observing the facial expressions of a virtual agent modeling affective minds.

2. Materials and Methods

2.1. Participants

Nineteen healthy subjects (31.4 ± 5.9 years, 9 male) gave their informed consent and participated in this study. The institutional local ethics committee approved this study (#2831-1).

2.2. Measurement

Cold pain detection threshold was measured using a computerized thermal tester (Pathway; Medoc, Israel) with a Peltier device measuring 32 × 32 mm. Baseline temperature was set at 32 °C. Cold pain detection threshold testing was performed on the central volar part of the hand. The temperature of the thermode gradually decreased (1 °C/s) until the subject pressed a response button at a specific thermal sensation, which was defined as detecting a painful sensation of cold. This returned the stimulus intensity to baseline (3 °C/s) and recorded the temperature at which the subject responded. Cold pain detection thresholds were calculated as the average for three stimuli.

First, the cold pain detection threshold was measured when the subjects were asked to watch a black screen (defined as the default cold pain detection threshold). Cold pain detection thresholds were then measured while watching three different facial expressions of a virtual agent (see below). The presentation of the three different facial expressions was randomized for each participant to prevent order effects. After cold pain detection thresholds had been measured for all facial expressions, the subjects mandatorily rated the pain intensities of the three different facial expressions on an 11-point numerical rating scale (NRS).

2.3. Facial Expressions of a Virtual Agent

Each facial expression of a virtual agent consisted of a 20-s looped video clip. The video continuously restarted without pause until cold pain detection thresholds have been measured for each facial expression. The video clips contained dynamic facial expressions intended to be painful, unhappy, and neutral, all generated with three-dimensional computer graphing software for the creation of virtual characters (People Putty, Haptek, Santa Cruz, CA, USA). This software autonomously creates moving facial expressions. Using a picture of a gender-neutral adult, the facial expressions of a virtual agent were systematically manipulated based on the Facial Action Coding System (FACS), an objective, anatomically based system that permits a full description of the basic units of facial movement associated with private experience, including pain. Forty-four different action units (AUs) have been identified. Core action units representing the facial expression of pain in adults are: brow lowering (AU4), tightening of the orbital muscles surrounding the eye (AU6&7), nose wrinkling/upper lip raising (AU9&10), and eye closure (AU43) [4]. The pain expression can be differentiated from other facial expressions like neutral and unhappy [5,6,7].

2.4. Statistical Analysis

Differences between the average default cold pain detection threshold of all participants and their cold pain detection thresholds while watching a black screen and the three facial expressions were compared using analysis of covariance (ANCOVA) with the respective pain intensities as covariates. We set NRS = 0 for pain intensity while watching a black screen. Holm’s multiple comparison procedure was used to compare the pair-wise differences in this ANCOVA model.

3. Results

The default cold pain detection threshold of 19 participants watching a black screen was 18.0 ± 8.2 °C. When watching a virtual agent with painful, unhappy, and neutral facial expressions, cold pain detection thresholds were 21.1 ± 5.7 ℃ 19.4 ± 6.9 ℃ and 18.5 ± 7.7 °C, respectively (

Table 1. Background Information and Measuring Data of Individual Participants.

Subject	Age	Gender	Pain Intensity Rating #			Cold Pain Detection Threshold (°C) *
			Neutral	Painful	Unhappy	Control	Neutral	Painful	Unhappy
1	37	male	1	5	8	17.5 ± 0.75	11.6 ± 0.90	13.4 ± 3.18	13.3 ± 2.35
2	24	female	1	8	5	10.2 ± 1.15	11.1 ± 0.72	12.4 ± 0.95	9.50 ± 0.55
3	26	female	7	8	5	17.9 ± 1.99	20.7 ± 1.32	20.3 ± 0.35	16.5 ± 0.31
4	35	male	8	8	4	28.5 ± 0.25	29.8 ± 0.12	29.0 ± 1.24	29.4 ± 0.68
5	28	female	6	7	7	23.0 ± 2.78	23.0 ± 0.97	22.7 ± 3.10	23.4 ± 1.99
6	34	male	3	8	1	28.4 ± 0.32	28.7 ± 0.36	29.6 ± 0.58	28.3 ± 0.15
7	31	female	1	5	3	16.7 ± 4.60	18.4 ± 1.10	23.7 ± 1.03	20.2 ± 1.22
8	26	male	1	3	5	24.4 ± 1.45	25.2 ± 0.38	24.3 ± 1.12	26.6 ± 0.81
9	34	male	1	8	3	16.3 ± 1.40	16.2 ± 1.50	20.5 ± 0.81	13.3 ± 3.86
10	33	male	7	1	8	19.5 ± 0.44	23.2 ± 0.93	16.4 ± 0.32	23.4 ± 1.10
11	25	female	2	6	5	5.90 ± 4.40	10.1 ± 2.40	16.5 ± 2.06	13.9 ± 1.64
12	44	male	1	9	6	1.20 ± 1.22	1.1 ± 1.15	11.0 ± 0.87	5.90 ± 3.97
13	25	female	1	7	4	12.6 ± 3.25	16.8 ± 1.87	15.3 ± 1.56	19.6 ± 1.81
14	35	male	1	9	4	4.50 ± 2.57	6.5 ± 1.21	22.7 ± 1.46	10.8 ± 0.30
15	27	female	6	10	8	21.8 ± 0.10	22.1 ± 2.13	20.6 ± 2.51	19.3 ± 2.25
16	27	female	1	6	3	28.1 ± 0.20	28 ± 0.91	29.4 ± 0.25	28.5 ± 0.71
17	27	female	2	5	5	27.2 ± 1.59	21.9 ± 3.47	26.0 ± 1.73	22.6 ± 1.82
18	36	female	1	4	8	20.3 ± 0.50	20.2 ± 0.47	22.5 ± 0.80	24.3 ± 0.50
19	42	male	1	9	7	18.3 ± 2.20	17.6 ± 0.90	23.9 ± 1.60	20.3 ± 1.96

# Pain intensity is rated on an 11-point numerical rating scale; * Mean ± standard deviation.

). Cold pain detection thresholds were thus quite-variable. Changes of cold pain detection thresholds watching a virtual agent with painful, unhappy, and neutral facial expressions were 3.1 ± 5.4, −1.7 ± 4.2 and −0.9 ± 2.6, respectively. Pain intensities of the different facial expressions (painful, unhappy, and neutral) were rated as 6.6 ± 2.3, 5.2 ± 2.0 and 2.7 ± 2.6, respectively. These pain ratings for three different facial expressions were varied among the participants (Table 1). Following facial expressions were rated in order of severity: Painful > unhappy > neutral (n = 10); Unhappy > painful > neutral (n = 3); Painful = unhappy > neutral (n = 2); Painful > neutral > unhappy (n = 2); Unhappy > neutral > painful (n = 1); and painful = neutral > unhappy (n = 1). Adjusted by the respective pain intensities, only when watching the painful expression of a virtual agent did the cold pain detection threshold increase significantly (Figure 1).

4. Discussion

Perception of given behaviors in another individual automatically activates one’s own representation of those behaviors, including facial expressions [8]. This phenomenon supports recognizing not only motor contexts but also affective minds through facial expressions and/or affective behaviors, which is called empathy [9]. Pain is an unpleasant sensory and emotional experience and thus is considered as one embodiment of affective minds. Empathy is of course observed in pain [10] and most previous studies reported as follows: (1) Pain rating of another individual by the observer is influenced by amounts of the facial expressions and/or behaviors of another individual. (2) The perception of pain in another individual taps into the affective component of pain processing of the observer, notably through activation of neural substrates like the anterior cingulate cortex and anterior insular cortex. Further, the activities of these neural substrates are correlated with the intensity of the pain observed, as descriptively rated by the observer [11]. Empathic neural responses thus share neural circuits between the observer and another individual, and the observer’s neural circuits are activated according to another individual’s pain intensity. Therefore, empathic neural response is suggested to be an isomorphic experience of the observer and the affective minds “shared” between the observer and another individual. In the present study, we confirmed the participants were healthy, but we did not collect data on their psychological conditions like as state anxiety and depressive mood. Psychological conditions might affect empathic behaviors when observing facial expressions. We should raise this as a limitation of this study.

Empathic understanding of another individual’s pain intensity does not dispel skepticism about the semantically created meanings indicated by describing pain ratings on the basis of the observer’s painful experiences to date. In fact, the social situation and reasoning are most efficiently triggered in a context where the observed behaviors of another individual are highly relevant to the observer’s own intentions and actions [12]. This would indicate that empathy for the intellectual mind is reflected in conscious experiences and modulated by top-down contextual factors. Apart from the intellectual mind, our present study directly evaluated intuitive pain responses when observing facial expressions of a virtual agent and found that empathic neural responses can intuitively emerge according to observed pain intensity. This finding would suggest that empathy for affective minds is developed automatically and without conscious awareness of the observer’s own affective state. That is, our subjects intuitively shared pain, one embodiment of affective minds, with a virtual agent. The term “share” means to have something at the same time as someone else. If one has something and another individual has it at the same time, they can share it. Considering this, one can “share” an affective mind regardless of one’s intellectual cognitive processes when a virtual agent has it at the same time although our sample size was limited. Consequently, we can respond to the classical question, “Does a robot (a virtual agent) have a mind of its own?”, that “Yes, a robot (a virtual agent) has a mind of its own because we intuitively share it.”