The Effects of Motion Artifacts on Self-Avatar Agency

Abstract

One way of achieving self-agency in virtual environments is by using a motion capture system and retargeting user’s motion to the virtual avatar. In this study, we investigated whether the self-agency is affected when motion artifacts appear on top of the baseline motion capture data assigned to the self-avatar. For this experiment, we implemented four artifacts: noise, latency, motion jump, and offset rotation of joints. The data provided directly from the motion capture system formed the baseline of the study. We developed three observation tasks to assess self-agency: self-observation, observation through a virtual mirror, and observation during locomotion. A questionnaire was adopted and used to capture the self-agency of participants. We analyzed the collected responses of participants to determine whether the motion artifacts significantly altered the participants’ sense of self-agency. The obtained results indicated that participants are not always sensitive to the motion artifacts assigned to the self-avatar, but the sense of self-agency is dependent on the observation task they were asked to perform. Implications for further research are discussed.

1. Introduction

Numerous new devices and interfaces for displaying virtual information and capturing user motion have been developed in recent years. The growth of such devices suggests immersive virtual reality is becoming increasingly popular, and immersive virtual reality interaction has become easily accessible for many. Besides the development of head-mounted displays (HMDs), devices such as smart phones, sensors, and motion capture systems provide users with an affordable capability of interacting with virtual environments.

The use of motion capture technology in order to enhance the levels of embodiment is quite common. It has been shown that giving the participants freedom to control their self-avatar using motion capture can have a positive impact on their experience [1,2]. Exploring virtual environments using motion capture can be characterized as the most realistic and natural interface that can be used for virtual reality experiences, since users embody human-like virtual avatars [3]. Motion capture systems for virtual reality interaction can also be described technically as the most challenging due to either hardware issues (systems selection, sensors that needs to be worn, calibration, etc.) [4] or quality issues (offset estimation of arm and hand rotation, inability to capture finger motion, self-occlusion causes body parts not to move properly, etc.).

While various motion capture systems have been developed over the past years to allow an increased sense of embodiment, less attention has been given to how motion artifacts affect the sense of self-agency when controlling a virtual avatar within a virtual environment. In this paper, self-agency, which is also known as the phenomenal will, is achieved by providing users agency over the virtual body. In our case, we are trying to explore whether motion artifacts, assigned to the motion of the self-avatar on top of the data provided directly by the motion capture system, affect the sense of agency when a user is embodied to a virtual avatar and is asked to interact within a virtual environment. In addition, to generalize our findings, viewing variability of the self-avatar (self-observation, observation through a virtual mirror, and observation during locomotion) was also considered as a parameter that was worth exploring.

For the purposes of this study, we first consider the data from the motion capture system as ground truth and then we investigate whether the addition of motion artifacts would affect the sense of self-agency. Thus, we considered four different motion artifacts that can impact the participants’ sense of self-agency. For the needs of this experiment, the following motion artifacts were developed:

• Noise: A wave or vibration artifact that is produced by the hardware during the motion capture process. Many motion capture systems (both optical and inertial) produce this artifact, and in most cases smoothing algorithms are used to eliminate them.
• Latency: The delay of the motion data to be transmitted and rendered into the computer screen or HMD. All motion capture systems have latency but, in most cases, it is within the acceptance spectrum.
• Motion jump: The inability of the motion capture sensor/system to estimate fast movements. This also occurs with body-occlusion. This can be found in optical motion capture systems when they cannot capture the retro-reflective markers attached to a participant’s body-parts, or in low-cost RGB-D sensors due to the low frame rate.
• Offset rotation of joint: The inability of the motion capture device/system to properly estimate the rotation of the user’s body-parts. This can also be caused by inaccuracies in the calibration of the motion capture system.

Based on the motion artifacts and the viewing variability developed for the purpose of this experiment, this study tries to answerer the following two research questions:

• RQ1: Do motion artifacts affect the sense of self-agency?
• RQ2: Do the observation tasks affect the sense of self-agency?

Such artifacts can be found in a number of motion capture setups. Thus, we chose to investigate the way they can possibly affect the sense of self-agency when compared with the baseline data provided directly by the motion capture system. It should be noted that not all of these artifacts apply to commercial low-accuracy and/or low-cost motion capture systems. For the authors of this paper, it is important to understand the way that participants perceive their sense of self-agency when embodied in a virtual avatar whose motion is augmented by noticeable motion artifacts. This information could help us understand further how virtual reality users perceive self-motion when embodied in virtual avatar’s as well as how and if the examined artifacts affect the self-agency.

2. Related Work

Embodiment (or the sense of embodiment) has not been conceptually clarified because the properties and the experiences have not been specified extensively [5]. The experience concerning the body can be classified in three to six main categories [5,6]:

• body ownership [7] (present in case there is a virtual body or body part that represents the user’s body or body part);
• agency and motor control of the virtual body [8] (the participant should be able to control parts or the whole body of the virtual avatar);
• tactile sensations [9] (enhance embodiment illusion by using tactile or haptic stimulation devices);
• location [10] (a virtual body or body part might be either collocated or not);
• appearance [11] (the self-avatar might be a lookalike avatar, or not, in terms of size, gender, race, clothing, etc.); and
• response to external stimuli [12] (events that modify or threaten the appearance of the self-avatar).

Clearly, embodiment is not a single entity; many subareas can influence it and multiple parameters should be considered in order to provide virtual reality users with an enhanced embodied experience within a virtual environment.

According to Blanke et al. [13], the sense of agency, which is an embodiment subarea that concerns this paper, is related to the sense of having “global motor control, including the subjective experience of action, control, intention, motor selection and the conscious experience of will.” Self-agency is one of the most important factors that provide consistency to a participant’s body [14]. According to Spanlang et al. [11] and Gonzalez-Franco and Peck [6], to achieve a higher level of embodiment in virtual environments, the use of motion capture devices/systems that transfer user movements to the virtual avatars they control with their bodies is crucial.

When the body obeys the user’s intentions, there is a sense that this can be the user’s body and vice versa. Kokkinara and Slater [9] found that enhanced embodiment is achieved when the real and the virtual bodies move synchronously. Newport et al. [8] found that a lack of agency reduces the sense of embodiment and David et al. [15] suggested that agency is the result of the difference between the predicted output of the user’s motion from the neuromotor output and the actual sensory output. The development of agency depends on the synchronicity of visuomotor correlations. Several previously conducted studies indicate that variations between the visual feedback of the action and the actual actions negatively affect the sense of agency [16,17,18]. Contrarily, when the predicted and the actual output of the action match (e.g., the feeling of synchronous visuomotor elation based on user movement), virtual reality users feel they are controlling those actions. Recent studies also support this trend and it has been shown that, when there is synchronization and correlation between the visual and motor abilities of participants, a stronger sense of embodiment is achieved [9]. However, when participants are deprived of the ability to properly control the self-avatar, they fail to experience high levels of embodiment [5]. Therefore, it can be stated that the concurrent use of visuomotor stimulation has a positive impact on the sense of embodiment since users are able to control the virtual body that embodies them. Finally, such a mechanism can also be applied to a number of tools (e.g., controllers) that we can use to interact with virtual environments and tasks located within them [19,20]. This study tried to expand on these findings, on both the predicted and actual output of the action, by proposing an additional dimension, which is the noticeable motion artifacts of the matching motion assigned to the self-avatar.

Agency has been studied in correlation with other subareas of embodiment in the past, and several studies indicate a correlation between the sense of agency and body ownership. According to previously published work, self-agency and motor control experimentation in virtual reality is increasingly aligned with the appearance of body [11]. Tsakiris et al. [21] stated that body ownership is not always enough to express the sense of agency but the sense of agency most likely expresses body ownership. A number of studies that used fake hands to induce a body ownership illusion show positive evidence for agency-embodiment illusion [14,22,23,24,25]. Additionally, there are cases where no correlation is found between agency and embodiment, such as in the work of Kalckert and Ehrsson [26]. Finally, in the study conducted by Longo et al. [27], participants reported a sense of agency even when seeing a rubber hand moving on its own (no self-motion). Taking into account a number of studies [14,21,23,24,25,26,27], it can be said that there is mixed evidence that associates agency with embodiment or other subareas of embodiment.

Given the technological growth and the affordability of motion capture devices/systems as well as the arguments made by Spanlang et al. [11] and Gonzalez-Franco and Peck [6], the sense of agency can be induced to a certain extent in a virtual body by capturing and retargeting the motion of the user to the virtual body in real-time. Motion capture systems (optical, inertial, etc.) can be used to retarget motion to animate a virtual avatar [28]. Among others, inverse kinematics [29], motion reconstruction [30,31], and activity recognition [32] techniques are used to animate a virtual avatar.

Considering motion artifacts, Sato and Yasuda [18] developed an experiment where a delay between the artificial actions and feedback was added to investigate the agency in relation to the predicted and actual feedback. They concluded that the agency is independent of body ownership since the agency was rated negatively compared to self-ownership by the participants. In another study, van den Bos and Jeannerod [33] concluded that there is an independence between agency and self-ownership. This conclusion was the result of an experiment that examined whether participants had difficulty assessing the ownership of a virtual hand model when they were not responsible for performing the actions they saw the hand do. Additionally, studies concerning the observation tasks have shown that the illusion of embodiment can be increased when a virtual mirror is placed in front of the self-avatar [34]. Moreover, it has been found [35] that participants consciously monitor the location and their locomotion with low precision when being embodied into a virtual environment.

Most studies have used questionnaires to understand agency. Kalckert and Ehrsson’s [26] questionnaire about assessing agency over the rubber hand is the most common, and it was used in the study conducted by Longo et al. [27]. Gonzalez-Franco and Peck [6] reported on questionnaires for all subareas of embodiment. In this experiment, a standard questionnaire, which originated in Longo et al. [27] and Kalckert and Ehrsson [26] and was altered to suit the purposes of this study, was used to assess self-agency.

The aforementioned studies provide interesting findings and insights about the way users perceive their embodiment through agency and motor control in the virtual environment. However, these studies focus mainly on the way that body ownership is affected without considering the artifacts of the motion data assigned to the self-avatar. This experiment tried to assess the sense of self-agency in terms of the motion artifacts assigned to the self-avatar to further understand the impact of the latter on the former. We assumed that such an experiment would help us better understand the sense of self-agency of users when observable motion artifacts are put on top of the data provided by the motion capture devices. To the best of our knowledge, no similar study has been conducted in the past.

3. Methodology and Implementation

This section presents the methodology followed, the implementation of the conditions, and the three tasks of the virtual reality experiment that were developed to investigate whether the motion artifacts that augment the motion capture data assigned to the self-avatar affects the participants’ sense of self-agency.

3.1. Participants

In this study, the participant group was comprised of 75 volunteers, both undergraduate and graduate students. Of the sample, 32 were female (age: $M=21.83$, $SD=1.72$) and 43 were male (age: $M=22.04$, $SD=1.98$). All of the participants were majoring in STEM fields. The participants were recruited through in-class announcements and posters placed at various locations in the department. As a reward, all of the participants got an extra course credit and were put in a drawing for ten €20 bonus cards. The user study took no more than 60 min per participant, and all participants completed the study. All participants gave written consent prior to the beginning of the study. Approval for this study was granted by the Institutional Review Board of the University of the Aegean.

3.2. Physical Environment

The study took place in the department’s lab space, which is 30 ft (≈9 m) long, 20 ft (≈6 m) width, with a ceiling height of 10 ft (≈3 m). The lab was almost free of obstacles and objects, so the participants were able to walk and move freely. The lab space is equipped with the necessary hardware for this study.

3.3. Virtual Reality Application and Equipment

The virtual reality application was developed in Unity3D version 2018.2.8. The Oculus Rift was used as the virtual reality hardware to display the content. In addition, an inertial measurement unit (IMU) motion capture system (the 32 Neuron Edition V2 of Perception Neuron) was used to capture and retarget the participants’ motions to the self-avatar. Each IMU houses a gyroscope, accelerometer, and magnetometer. The provided data from the motion capture system were used as a baseline condition. Here, it should be noted that the accuracy of the IMUs motion capture systems as the one used for the purpose of the study is generally lower compared to an optical tracking system, as shown by Filippeschi et al. [36]. However, since the scope of this study was to investigate whether the motion artifacts altered the self-agency of participants, it was speculated that similar effects would be found even when the motion artifacts are applied on top of an optical motion capture system.

We decided to assign gender to a self-avatar as the one specified by the participants when filling in the demographic questionnaire form. Since we wanted to provide an embodied experience and make the participants feel that the body that represents them is their own, we decided to allow participants to choose a self-avatar that most closely represented their skin color and dimensions. To do this, we designed twelve male and twelve female avatars with variations in skin color (light, medium, brown, and black), and size (skinny, regular, and fat). We developed a simple interactive application in Unity3D that showcases these avatars. All 24 avatars were designed in Adobe Fuse, and a simple idle motion was assigned to them. Each participant was able to zoom in and out and rotate the avatar in order to observe it from different perspectives. Participants were able to choose the avatar that best represented them, and the selected one was later used as the self-avatar. Figure 1 shows all 24 avatars that were used in this study.

Please note that it has been shown that self-avatars that mimic the representation of the participant enhance the sense of body ownership [37,38]. Since the developed experiment investigated the impact of motion artifacts of the self-motion assigned to the self-avatar instead of its appearance, we decided to provide a number of customized avatars for the participants to eliminate the effects of body ownership on self-agency. According to Steed et al. [39], the use of a self-avatar to represent participants also provides a positive impact on presence. This finding was supported in a number of studies that examined embodiment [34,37,40,41]. The use of the motion capture system and the HMD gave participants the ability to navigate within the virtual environment as well as to immerse themselves within it. According to Slater [42], virtual reality experiences that provide users the ability to interact with the virtual environment have a positive impact on the feeling of presence. Finally, following Slater and Wilbur’s [43] definition of presence, it can be said that both the self-avatar and the freedom to control the motion of it provided a positive sense of presence in participants.

The developed application used for this experiment consisted of a single scene with available buttons for the experimenter in order to switch the conditions of the experiment. The buttons were available from the Unity3D inspector panel, and each made one of the following motion artifacts appear: no artifact (baseline motion retargeted to the self-avatar directly from the motion capture system), noise, delay, motion jump, and offset rotation of joints.

A single virtual environment (scene) was developed for this experiment (see Figure 2); however, the experimenter was able to switch on and off the component of the scene. The baseline virtual environment was used for the self-observation task. The experimenter was able to switch on a virtual mirror that was used by the participants to observe themselves during the second task. We decided to switch off the mirror during the self-observation and the observation during the locomotion task since we did not want it to distract the participants. Finally, during the third task, the experimenter was able to switch on the four target positions participants were asked to pass by. The switching between the components of the experiment was performed by using two buttons on a Unity3D inspector. Finally, it should be noted that before a condition began and after it ended, the participants would see a black screen.

3.4. Motion Artifacts

Four motion artifacts were developed for this experiment. Before detailing them, it is worth mentioning that the motion data provided directly from the motion capture system were considered as the baseline and the developed motion artifacts were implemented on top of them. The motion artifacts were developed in a way that could be noticeable by the participants. We did so as in the current study we wanted to investigate whether such motion artifacts alter the self-agency of participants.

For the noise artifact, the well-known Perlin noise [44] was used. Since we wanted to ensure that the noise motion artifact was noticeable, a value of 16 Hz for the frequency and 20 cm for the amplitude were used. These values provided the desired results. For the latency artifact, we developed a simple buffer that stores the input signal from the motion capture device for a certain time-period, and then the motion captured data is sent to the self-avatar. We used the noticeable number of 150 ms as the time-period of latency. The 150 ms latency was also found by Franck et al. [17], who also suggested that visuomotor correlations can be granted by keeping the necessary noticeable time boundaries between the user’s action and visual feedback. It should be noted that another study investigated the effect of latency on perceptual and motor tasks during full-body action inside a cave automatic virtual environment (CAVE). In particular, Waltemate et al. [45] found that both the sense of agency and ownership decline at a latency higher than 125 ms.

For the motion jump artifact, a simple implementation that freezes the input signals from the motion capture system was developed. Specifically, the freezing function stops the transmission of the input motion to the self-avatar, and after the defined period, the freezing functions stops, and the motion capture signal is transmitted properly. Since we wanted to provide a “natural” way with which such a motion artifact might occur, we randomized how often it should appear as well as the exact time period it should last. Therefore, the motion artifact appeared between 5 and 10 s intervals and lasted between 0.5 and 1 s. It should be noted that the randomization process was performed in a preprocessing stage, and the resulting time-steps and durations of the motion jumps were stored. The computed values that were used for the motion jumps were similar for all participants, which means that all participants experienced the exact same motion jump artifact. The motion jump artifact was implemented to appear only on the arms and legs of the virtual avatar. It should be noted that the root joint of the self-avatar was excluded from the motion jump, since we realized that such a motion artifact might cause nausea, especially during the third task when participants were asked to perform locomotion.

The last motion artifact implemented is the offset rotation of joints. For this artifact, we simply overrode the input rotation of the arm and feet joints with an additional value of 45 degrees. It should be noted that this rotation override was made to a single axis of each body-part, $x-axis$ for the arms and $y-axis$ for the hips. Note that the $x-axis$ and $z-axis$ denote the plane $(x,z)$, and $y-axis$ denotes the up vector. As before, the 45 degrees were found to be the noticeable offset rotation of self-avatar joints.

3.5. Experiment Procedure

The participants came to the lab in which the experiment was conducted and were asked to complete a demographic questionnaire. Then, they observed the twelve designed virtual avatars and were asked to choose the one that most closely represented them. Only the virtual avatars that corresponded to the participant-specified gender were shown.

Table 1. The choice of participants regarding skin color and weight of the self-avatar for both male (M) and female (F).

	Light		Medium		Brown		Black
Skinny	M: 1	F: 3	M: 9	F: 6	M: 1	F: 2	M: 0	F: 0
Regular	M: 4	F: 5	M: 17	F: 12	M: 1	F: 2	M: 0	F: 0
Fat	M: 2	F: 2	M: 5	F: 2	M: 2	F: 0	M: 1	F: 0

summarizes the choices of participants for each skin color and weight variation of the avatars.

The participants were asked to wear the motion capture system, which was subsequently calibrated. Then, the participants were asked to wear the HMD. Figure 3 shows a participant wearing the motion capture suit and the HMD and observing its virtual body. Before starting the experiment, the participants were informed about it, as well as what they would be asked to perform when immersed into the virtual environment. Participants were also given the option of terminating the experiment at any time if they felt any kind of discomfort (nausea, motion sickness, etc.). After the short briefing, the experimenter started the application. It should be noted that the order of the tasks and the five motion artifacts were randomized. Counterbalance among the five conditions for all three tasks was also maintained. Finally, it should be noted that we chose a within-subjects design for this study in order to allow for model comparison despite the large differences among participants.

The participants were asked to observe the body of the virtual avatar that represented them for a period of one minute and then respond to Questions Q1–Q4 (see

Table 2. The questionnaire used during the experiment.

Label	Question	Options
Q1	How strong was the feeling that the body you saw was your own?	1 being not at all, 7 being very much
Q2	How much did you feel that you were looking at your own body?	1 being not at all, 7 being very much
Q3	How strong was the feeling that your real body was becoming virtual?	1 being not at all, 7 being very much
Q4	How strong was the feeling that the virtual body was beginning to look like your real body?	1 being not at all, 7 being very much
Q5	How strong was the feeling that you could control the virtual body as if it was your own body?	1 being not at all, 7 being very much
Q6	How strong was the feeling that the movements of the virtual body were caused by your movements?	1 being not at all, 7 being very much
Q7	How strong was the feeling that your virtual body moved just like you wanted it to?	1 being not at all, 7 being very much
Q8	How strong was the feeling that you were causing the movement of the virtual body you saw?	1 being not at all, 7 being very much

and Section 3.6) for each motion artifact assigned to the self-avatar. At each condition of this part of this study, the participants were asked to take off the HMD and to report on their sense of body ownership in a computer-based environment. For this part of the questionnaire only self-observation was considered. We decided to not ask the body ownership questionnaire for all three developed tasks for two reasons. Firstly, we wanted to capture the general sense of body ownership when the participants self-observed their virtual body. Secondly, we realized that asking questions about the body ownership for all motion artifacts and for all three developed tasks would significantly increase the duration of the experiment, causing fatigue and loss of motivation.

After the end of the preliminary study regarding body ownership, the main part of this experiment that concerns the sense of self-agency started. For the self-observation task, the experimenter asked the participants to observe the motion of the self-avatar for two minutes. For the observation through the mirror task, participants were asked to observe their motion through a virtual mirror that was placed in front of them, again for two minutes. It should be noted that participants were able to move around and perform small steps near the virtual mirror in order to observe their motion from different angles and distances. Participants were also instructed to focus on the avatar’s body excluding the face since no motion was assigned to it. This instruction was provided since it was assumed that the face of the virtual avatar might affect the results obtained during the observation through a virtual mirror task. Finally, for the self-observation during locomotion task, participants were asked to walk to target positions in the virtual environment. For the observation during locomotion task, four simple indicators with numbers (1–4) were put into the virtual environment (see Figure 2). Participants were asked to pass by each indicator twice. Participants were informed that there were no obstacles in the path they should follow, which allowed them to move freely without thinking that they might hit something and hurt themselves. After the end of each condition, participants were asked to take off the HMD and to self-report their sense of self-agency in a computer-based environment by responding to Questions Q5–Q8 (see Table 2 and Section 3.6). When the participants were finished, the experiment continued to the next condition. Finally, before each new condition started, the participants were asked whether they would like to have a short break.

3.6. Questionnaire

Two sets of questions were used for this study. The complete questionnaire is shown in Table 2. All participants answered the whole set of questions. Q1–Q4 utilized are from Steed et al. [39] and are based on the body ownership questionnaire of Botvinick and Cohen [46] and altered to suit the virtual environments Slater et al. [47]. The body ownership questionnaire was used to understand whether participants had the illusion of owning the virtual body that represented them. Q1–Q4 questions were asked in a preliminary part of this study when the participants were first exposed to the virtual environment. These questions were asked for all five conditions of the experiment but only during the self-observation task. The second part of the questionnaire on agency and motor control (Q5–Q8) originated in Longo et al. [27] and Kalckert and Ehrsson [26] and was slightly altered to suit this experiment. This is the main questionnaire used to assess the participants’ sense of self-agency. It was used in all three tasks for each of the five developed motion artifacts. It should be noted that only four questions were utilized from the aforementioned sources for both questionnaires.

4. Results

To analyze the collected data, we used a one-way repeated-measures analysis of variance (ANOVA) with Greenhouse–Geisser corrected estimates of sphericity using the four motion artifacts and the baseline motion from the motion capture system (five conditions) as the independent variable, and the obtained results from the questionnaires as dependent variables. The post hoc comparisons were performed using Bonferroni corrected estimates. Before the analysis of the obtained data, a reliability analysis test was performed by computing the Cronbach’s alpha coefficient for each questionnaire separately (body ownership and self-agency). The obtained results from the reliability test are summarized in

Table 3. Reliability test for the questionnaires that were used in this study for each of the five conditions.

	Baseline	Noise	Latency	Motion Jumps	Offset Rotation
Ownership: Self-observation	0.893	0.879	0.797	0.858	0.841
Agency: Self-observation	0.792	0.819	0.723	0.708	0.830
Agency: Observation through mirror	0.903	0.829	0.716	0.797	0.762
Agency: Observation during locomotion	0.870	0.827	0.878	0.850	0.866

. The four-item scales for both body ownership and self-agency questionnaires yielded acceptable to excellent reliability coefficients. It should also be noted that no removal of items from either questionnaire would enhance these reliability measures. We would also like to note that we conducted analyses regarding gender differences, virtual reality and games experiences, and prior experiences of virtual reality and game interaction with motion capture systems, but we were not able to identify differences. For the statistical analysis, the IBM SPSS [48] statistical package was used.

4.1. Body Ownership

We start by presenting the results on participants’ sense of body ownership. Body ownership measures were based on questions Q1–Q4, which were asked only once for each motion artifact. By analyzing the four-item scale, we found that there was no significant impact of the motion artifacts on body ownership for all examined conditions (baseline motion capture, noise, latency, motion jump, and offset rotation of joints) of the experiment [$\mathsf{\Lambda }=0.885$, $F(4,71)=2.318$, $p=0.065$, ${\eta }_p^2=0.115$]. This was an expected result since it is common for participants to perceive a virtual body as their own even if it does not match their appearance exactly [39,49]. The obtained results concerning body ownership are presented in Figure 4.

4.2. Self-Agency

Here, we present the results concerning self-agency in the three conducted tasks individually. The obtained results concerning the self-agency for all three tasks are presented in Figure 5.

4.2.1. Task 1: Self-Observation

During the first task, participants were asked to self-observe the motion assigned to the self-avatar. The obtained results indicate that there was a significant impact of the five conditions on the self-observation task [$\mathsf{\Lambda }=0.231$, $F(4,71)=59.215$, $p<0.01$, ${\eta }_p^2=0.769$]. Post hoc comparisons indicated that participants’ sense of self-agency during the baseline condition ($M=5.39$, $SD=1.16$) was significantly higher than the noise ($M=3.13$, $SD=1.31$; $p<0.01$), latency ($M=3.04$, $SD=1.08$; $p<0.01$), motion jump ($M=2.96$, $SD=1.02$; $p<0.01$), and offset rotation or joints ($M=3.11$, $SD=1.14$; $p<0.01$) conditions. Taken together, these results suggest that the motion artifacts assigned to the captured motion really did have an impact on the sense of self-agency. This means that, when no motion artifacts appeared in the retargeted motion, a higher level of self-agency was achieved.

4.2.2. Task 2: Observation through Virtual Mirror

For the second task, participants were asked to observe their motion assigned to the self-avatar through a virtual mirror. The obtained results indicate that there was a significant impact of the five conditions on the observation through mirror task [$\mathsf{\Lambda }=0.180$, $F(4,71)=80.761$, $p<0.01$, ${\eta }_p^2=0.820$]. Post hoc comparisons indicated that participants’ sense of self-agency during the baseline condition ($M=5.48$, $SD=1.23$) was significantly higher than the noise ($M=2.66$, $SD=1.27$; $p<0.01$), latency ($M=2.48$, $SD=1.11$; $p<0.01$), motion jump ($M=2.45$, $SD=1.15$, $p<0.01$), and offset rotation or joints ($M=2.24$, $SD=1.33$; $p<0.01$) conditions. Similar to the results obtained from the self-observation task, these results suggest that the motion artifacts assigned to the self-avatar altered the sense of self-agency.

4.2.3. Task 3: Observation during Locomotion

This task examined the sense of self-agency during a locomotion task. Interestingly, there was no significant impact of the motion artifacts on self-agency for all examined conditions [$\mathsf{\Lambda }=0.943$, $F(4,71)=1.071$, $p=0.377$, ${\eta }_p^2=0.057$]. This result indicates that participants had a similar sense of self-agency in all five conditions of the task regardless of the motion artifact assigned to the motion that animates the self-avatar. This is an unexpected result that needs to be discussed, and also revisited in the future.

4.3. Observation Impact

Three observation tasks were used as an independent variable, and the composite scores from the five motion conditions (four motion artifacts and the baseline motion) were used as dependent variables. We decided to use the composite scores since we wanted to understand in a general way the impact of the observation type on the sense of self-agency.

The obtained results indicate that there were significant impacts of three observation tasks on the self-agency of participants [$\mathsf{\Lambda }=0.575$, $F(2,73)=26.970$, $p<0.01$, ${\eta }_p^2=0.425$]. Post hoc comparisons indicated that the sense of self-agency was significantly lower during the observation through mirror task ($M=3.06$, $SD=0.79$) compared to the self-observation ($M=3.53$, $SD=0.81$; $p<0.05$) and the observation during locomotion ($M=4.77$, $SD=1.38$; $p<0.01$) tasks. In addition, post hoc comparison indicated that the sense of self-agency during the self-observation task was significantly lower compared to the observation during locomotion task ($p<0.01$). The obtained results concerning the observation impact on the sense of self-agency are presented in Figure 6.

Taken together, these results suggest that the way that participants observed their motion affected the way that they perceived it. Participants were more sensitive when they observed their self-motion through a virtual mirror compared to self-observation and observation during locomotion. This is an interesting finding that should be explored further.

5. Discussion

It is widely known that the visual system dominates other senses [50]. When we wear the HMD, we rely on the provided virtual information to make our choices on the ways we interact within a virtual environment. As a result, we unambiguously place ourselves in scenarios where we are aware of the impact between what we see and what we know from real-world experiences. A typical example of a conflict between the real world and the virtual reality experiences is the motion artifacts that might appear in the motion assigned to a self-avatar, the avatar that represents us in the virtual world. As humans we are aware of our motions and the way that our body moves, but when our motion is retargeted to a virtual avatar and augmented with motion artifacts, we might perceive our motion differently. Thus, we studied whether the motion artifacts assigned to a self-avatar affected participants’ sense of embodiment, and specifically their sense of self-agency.

Interesting results were found when analyzing the collected data from this experiment. We feel that the obtained result from the conducted experiment is good evidence that the motion artifacts assigned on top of the captured motion data reduces the self-agency, which is an important factor for enhancing the levels of embodiment. Moreover, we are also positive on the idea that the conducted experiment can return useful insights and results in our future work. However, we also believe that there is a need for additional experimentation to further understand the effect of motion artifacts on self-agency.

The obtained results indicate that the motion artifacts do not affect the sense of body ownership, which is in line with previously published work on the self-motion of the participant being applied to the self-avatar [26,51]. This is a positive finding since it indicates that participants tend to perceive the virtual body as their own even if the assigned motion to their virtual body is augmented with motion artifacts. This finding also means that the use of a low-accuracy motion capture device can provide participants with quite reasonable body ownership levels. Most likely this result was exhibited because participants could choose a virtual avatar that best represented them (matching their gender, appearance, and size).

Interesting results were also obtained when analyzing the collected data for the self-agency questions. Before discussing the obtained results, it should be mentioned that, since body ownership was granted to participants, it can be said that the sense of self-agency was not affected by the body ownership but only by the artifacts of the motion assigned to the self-avatar. The obtained results indicate that, depending on the observation task, the impact of the motion artifacts on self-agency changed. We found that, when the participants were asked to either self-observe themselves or self-observe themselves through a virtual mirror, the motion artifacts assigned to their virtual body affected their sense of self-agency negatively. This is an interesting finding since it indicates that participants are sensitive to the way they perceive their motion, and that the motion artifacts can be a factor in reducing the self-agency and their sense of embodiment. Thus, it can be said that the self-agency might not be only related to asynchronous control conditions [39,47] but also to other motion artifacts, similar to those implemented for the purpose of this experiment.

Here, we would also like to note that, after having short discussions with participants at the end of the experiment, some of them indicated that the avatar’s movements were asynchronous, exaggerated, shaky, or wrong when compared with their own movements, which means that participants were fully aware of the motion assigned to them. A few more participants also stated that some of the motion artifacts were annoying and unnecessarily intense, and this made them focus more on the motion instead of the body of the self-avatar.

For the third task, participants were asked to walk within the virtual environment and observe themselves. In this case, a quite interesting result was obtained. Participants indicated that the motion artifacts did not affect their sense of self-agency. The lack of any effect on this task was disappointing given that we were expecting some significant impact of motion artifacts on self-agency. In retrospect, two observations can be made that result from the discussions we had with the participants. Firstly, because participants were performing locomotion, they were targeting particular areas (the areas they were instructed to pass by) within the virtual environment with their gaze. Thus, participants were distracted from observing the motion assigned to the self-avatar. On the one hand, this can be considered as a positive outcome since the sense of self-agency was not taken that much into account during a locomotion task, which is partially supported by Kannape et al. [35]. On the other hand, we should not get complacent. There are a lot of virtual reality applications in which the user should perform both locomotion and non-locomotion tasks. In this case, even if it can be claimed that during the locomotion tasks the artifacts of the motion data does not affect the sense of self-agency, this sense is affected during the non-locomotion tasks. A second observation is related to the virtual environment designed for this experiment. We decided to design a virtual environment with enough empty space to allow the participants to easily move within it without thinking about encountering obstacles and collisions. However, it seems this was a wrong decision. According to discussions we had with some of the participants, since the path they had to follow (both the physical and the virtual) was free of obstacles, they were not focusing much on their virtual body and consequently on their motion since they were aware that no collision would happen. Based on this, it can be said that obstacles and narrow passes need more attention during locomotion in the virtual environment. In such cases, participants tend to focus more on their virtual body and possible collisions.

An additional observation that can be made is based on the results related to the way that the sense of self-agency changes based on the observation task. The obtained results indicated that participants rated differently their sense of self-agency for the three tasks (self-observation, observation through mirror, and observation during locomotion). Lower levels of the sense of self-agency were found for the observation through mirror and higher levels were found for the observation during locomotion. The self-observation was rated in-between for the other two observation tasks but closer to the observation through mirror. This indicates that, when participants observe themselves through a virtual mirror, they become more sensitive of their motion that is applied to the self-avatar. One interpretation of this finding is related to the actual viewing variability the participants had when observing their own motion. During the self-observation through mirror, participants were able to move and rotate and observe themselves from different distances and angles, compared to the self-observation when participants were able to observe themselves from a single point of view. Therefore, participants had a generalized view of the motion assigned to the self-avatar and were able to judge this motion and the corresponding motion artifacts more strictly. Contrarily, considering the discussion we had with participants regarding the observation during locomotion task, participants did not focus much on the self-motion since they were focusing more on the target positions they should reach. Therefore, they decided to judge the assigned motion less strictly. In any case, it can be said that the obtained results seem to go further and suggest that self-agency is dependent not only on the artifacts of the motion assigned to the self-avatar but also on the observation task that the users are performing.

Another observation we would like to discuss based on the obtained results is related to the way that participants ranked the four developed motion artifacts on the sense of self-agency. By looking at the mean values of the obtained results, the baseline motion received the highest scores. Regarding the four motion artifacts that were developed for this study the following results were obtained. For the self-observation task, the noise received the second highest score, followed by the offset rotation, the latency, and the motion jump artifacts. For the observation through mirror task, the noise received the second highest score, followed by the latency, the motion jumps, and the offset rotation artifacts. Finally, for the observation during locomotion task, the offset rotation received the second highest score, followed by the latency, the noise, and the motion jumps. Based on this observation, it can be said that there is no specific pattern in the way that each motion artifact affected the sense of self-agency for the three examined observation tasks. Therefore, it can be further said that the way that participants observe their motion altered the way that they rated their sense of self-agency for each of the motion artifacts. Lastly, it can be finally said that in the self-observation and the observation during locomotion task, the motion jump was the one that most affected the sense of self-agency and for the self-observation and observation through mirror task, the noise was the one that least impacted the sense of self-agency.

Although the results obtained from this experiment are diverse, we would like to present a few observations, and we would also like to make a few suggestions for future studies that intend to investigate the impact of motion artifacts on the sense of self-agency. The self-observation and observation through mirror tasks indicated that indeed there is an impact on the sense of self-agency when comparing the baseline motion with the four developed motion artifacts. On the other hand, participants indicated that for the third task, observation during locomotion, the motion artifacts did not affect the sense of self-agency. The observation during locomotion tasks is quite complex since participants tend to focus on the tasks they perform or the target positions they should reach. Thus, it can be said that self-agency during locomotion tasks may differ when compared to self-observation and observation through a mirror in which participants concentrate on their own body and observe the assigned motion with more attention and precision.

The motion artifacts were designed so that they would be noticeable by the participants and therefore do not represent the way that such artifacts appear when using low-accuracy motion capture devices. In addition, in this implementation, we strictly forced the artifacts to appear in a specific way and on specific joints of the self-avatar’s body (e.g., offset rotation of joints only appears on the arms and legs). However, when using low-accuracy motion capture devices, such artifacts might appear under specific circumstances and conditions in a single body part and not in several body parts simultaneously. Considering that this is the first study that tries to assess the self-agency through the artifacts of the motion assigned to the self-avatar, and by taking into account that the motion artifacts might not represent the exact way in which they should appear, we view the results obtained from this study as good evidence that the motion artifacts alters the sense of self-agency in immersive virtual reality experiences.

6. Conclusions and Future Work

Depending on the level of embodiment that we are trying to achieve, it is possible that motion artifacts that are applied on top of the baseline motion provided by the motion capture system affect the way that we experience embodiment. To understand the impact of these artifacts on the sense of self-agency, being a subarea of embodiment, three tasks were used, and four motion artifacts were implemented and compared with the baseline motion provided by a motion capture system.

The motion artifacts were designed to be noticeable by the participants of this study based on our previously conducted study. However, we believe that new directions for future studies are emerging in regards to the motion artifacts. In our future work, we would like to individually investigate the effects of motion artifacts on self-agency. We assume that finding the acceptable critical numbers for each of the mentioned motion artifacts would help us better understand the way virtual reality users perceive the sense of self-agency. It should also be mentioned that combinations between artifacts are also common and can be found in low-accuracy motion capture systems, and the impact of these combinations on self-agency are worthy of study. Thus, it is of the authors’ interest to examine the combination of motion artifacts on a participant’s sense of self-agency, especially for locomotion-related tasks. Additionally, developing experimental scenarios in which participants have to perform more complex locomotion tasks, e.g., side stepping or passing through narrow hallways, might be necessary to further understand the impact that the motion artifacts have on the sense of self-agency. Finally, in our future work, we would like to explore whether the sense of self-agency is affected when providing tactile feedback [52,53] to participant’s body as well as how participants perceive self-agency when embodied to a virtual character with variations on his/her appearance [54,55].

In conclusion, we believe that the presented results can be very important when developing virtual reality applications in which the user is represented with a self-avatar that is controlled by his/her own motions. For example, by knowing that participants did not pay much attention to their motion when performing locomotion tasks, developers might consider making participants focus more on the virtual environment instead of the self-avatar, and excluding from their virtual environments narrow passes and obstacles that might make the users focus on their virtual bodies. Moreover, useful consideration can be made when conducting virtual reality experiments in which the participants were asked to perform locomotive tasks [56]. We also believe that this study makes an important contribution to the virtual reality community by confirming that self-motion and specifically the artifacts that can possibly augment the baseline motion provided by the motion capture device have an important impact on self-agency and consequently on embodiment. Finally, according to the obtained results, it can be said that, when the user’s motion retargeted at a self-avatar does not contain motion artifacts, this enhances the sense of self-agency, and consequently the sense of embodiment within virtual reality environments.