Technological Immersion and Delegation to Virtual Agents

Abstract

Interacting with virtual agents in immersive settings is becoming increasingly common thanks to the proliferation of dedicated media devices, such as consumer-grade virtual and augmented reality headsets. These technologies offer many advantages, e.g., in terms of presence and engagement, and can impact user behavior and attitudes toward virtual agents. Recognizing and understanding these effects is essential, especially in critical contexts involving the delegation of high-stake decisions to virtual agents. This article presents two experiments that explore users’ delegatory behavior toward virtual agents experienced via different media devices that vary in their technological immersion, i.e., a device’s technical capacity to deliver immersive experiences. The experiments’ results suggest that technological immersion is not a significant factor in users’ delegation decisions. Thus, for virtual agents designed to carry out critical tasks, developers may focus on other relevant factors, such as agents’ trustworthiness or performance.

1. Introduction

Since the advent of graphical user interfaces, virtual agents (VAs) have been the subject of lively discussion, research, and development [1,2]. VAs are software-based agents, often represented with a virtual body, and employing human communication such as gestures (e.g., Ref. [3]), speech (e.g., Ref. [4]), or facial expressions (e.g., Ref. [5]). From early attempts (e.g., Ref. [6]) to today’s modern virtual humans (e.g., Ref. [7]), the goal in VA design has always been to interact with users more naturally and personably than the traditional WIMP (windows, icons, menus, and pointer) interface allows [8]. Besides their advanced communication capabilities, VAs have become increasingly autonomous and accessible in the last decades due to progress in artificial intelligence, cloud services, and others [9]. As a result of their increasing technological capabilities, interaction with VAs has gradually evolved from a traditional user–tool relationship to one resembling interpersonal delegation, where a manager (user) empowers a subordinate (VA) to carry out specific tasks on the manager’s behalf [10]. On the one hand, the emerging delegatory relationship with VAs significantly facilitates the interaction in terms of, e.g., efficiency and convenience, as users can assign their tasks to VAs without the need to tender strategies and initiatives. On the other hand, this type of relationship entails problems and challenges because users have to delegate their authority to VAs, which makes them vulnerable to the actions of VAs [11]. These problems can be exacerbated when developers exploit the potential of VAs to manipulate users through affection ruthlessly [12]. Therefore, knowing and understanding the factors that govern the delegatory relationship and users’ decisions on delegation to VAs is crucial.

As later discussed in Section 2.1, the literature has documented many relevant factors, most of which are related to users (e.g., age [13] or gender [13,14]), contexts (e.g., task criticality [15]), and VA properties (e.g., capabilities [16,17] or controllability [16]). However, less attention has been paid to hardware-related, technological aspects, which have seen considerable progress over the past years in terms of ergonomics, interactivity, tracking, visual and auditory fidelity, etc. Due to the lack of research on this topic, it remains unclear what influence these technological advances have on users’ delegation behavior towards VAs. To this end, this article examines the influence of an often neglected but increasingly relevant factor, technological immersion, i.e., the capacity of a media device to mediate and represent an environment in a way that matches human perception of the physical world.

VAs are inherently simulated, so a technical device must mediate the interaction. Commonly employed devices include desktop computers, laptops, tablets, and smartphones [18,19]. These devices mainly feature a two-dimensional display to (mostly visually) mediate and present an environment as if users were viewing it through a window. However, due to this limitation, these devices are arguably not ideal for delivering immersive interactive experiences with a strong sense of presence, despite being widely used and offering various advantages [20,21]. To overcome this issue, researchers and companies have devoted a substantial amount of effort over the past decades to developing immersive media devices, particularly head-mounted displays for virtual reality (VR) and augmented reality (AR). Featuring stereoscopic displays or panoramic fields of regard, this class of devices is capable of immersing users in a “technology-driven environment with the possibility to actively partake and participate in the information and experiences dispensed by the generated world” [22]. Evidence shows that the user experience of the same environment or interaction varies depending on the media device’s technological immersion level [23,24,25,26,27,28,29]. Based on these previous studies showing the impact of technological immersion, we formulate the positive hypothesis that users are more likely to delegate to VAs experienced via technologically more immersive devices.

The hypothesis was tested via two experiments employing different methods and contexts to obtain robust and generalizable findings. Experiment 1 (cf. Section 3) is a within-subjects experiment comparing three different levels of technological immersion in a game setting, whereas Experiment 2 (cf. Section 4) is a between-subjects experiment focusing on two distinctive levels of technological immersion through a hypothetical investment scenario. The findings derived from the experiments help explain the impact of technological aspects on delegation to VAs and contribute to the scarce research that directly and systematically compares user–VA interaction across desktop, AR, and VR settings.

In the remainder of this article, Section 2 first reviews related work on delegation to software agents and introduces the notion of technological immersion and other related concepts. Section 3 and Section 4 detail the two experiments, methodologies, and results. Finally, Section 5 contextualizes and discusses the experiments’ overall findings and limitations.

2. Background

This section discusses the two central concepts underlying this study, beginning with related work on delegation to software agents in Section 2.1, then clarifying the notion of technological immersion and its many effects on user experience in Section 2.2.

2.1. Delegation to Software Agents

Delegation is a common practice in human societies, typically within hierarchical structures and organizations such as companies and governments. Despite its prevalence, taking the right delegation decision is challenging; even managers of successful companies make mistakes in that regard [30,31]. Due to its significance and complexity, delegation has been the subject of much debate across several disciplines as diverse as economics, management, and politics [15,32]. One of the most influential contributions on delegation is agency theory, which mainly addresses the delegatory relationship between two rational entities—often referred to as a principal and an agent—with each entity having its own belief, desires, and intentions. The theory posits that this relationship is subject to so-called agency problems, which often arise in two situations: (1) when the principal and the agent have conflicting goals or different attitudes toward risks; and (2) when it is difficult for the principal to verify what the agent is actually doing [15]. To resolve these problems, agency theory takes an approach from economics and advocates using a set of rules—i.e., a contract—to regulate the principal–agent dyad so as to minimize the negative influence emanated from the agency problems and maximize the dyad’s efficiency. Besides this economic perspective, some studies take more psychological approaches and focus on factors underlying principals’ delegation decisions. For instance, Leana [13] identified several relevant factors, including task criticality, agents’ age, gender, trustworthiness, capabilities, and principals’ workload.

While the term “delegation” has been predominantly used and studied for interpersonal contexts, recent years have seen the term gradually adopted by the computer science research community as interaction with software agents becomes increasingly similar to delegation. In their work on delegation to software agents, Milewski and Lewis [11] raised their concerns about software agents being increasingly mediated by delegation-oriented rather than tool-oriented user interfaces. The rapid change may inflict issues common to interpersonal delegation similarly on interaction with software agents and thus merit attention. To “overcome well-established drawbacks in delegation”, they proposed guidelines covering several aspects of user–agent interaction, including trust, communication, performance control, users’ demographics, and cost–benefit analysis.

Another early study formalized delegatory relationships within a multi-agent system [33]. Although targeting inter-agent instead of user–agent delegation, their theory is still interesting in a broader human–computer interaction context as it offers a different perspective on delegation to software agents. Different to other approaches, Castelfranchi and Falcone [33] regarded delegation as a state of principals, where “an agent A needs or likes an action of another agent B and includes it in its own plan”, removing the element of responsibility and accentuating principals’ demand or preference for specific actions from agents. Based on this definition, delegation is classified into three categories—weak delegation, mild delegation, and strict delegation—and further derived related concepts such as adoption and subdelegation.

After these pioneering works, delegation to software agents as a concept has been sporadically explored, with some relevant studies emerging only in recent years [17,34,35,36,37,38,39]. Many of these studies investigated factors underlying users’ decisions on delegation to software agents. The identified factors so far include, for example, agents’ capabilities [17,40], trustworthiness [11], the predictability of agents’ actions [41], the extent to which users can control the situation [16], and the extent to which users are accountable or responsible for task outcomes [16]. Among these factors, trust is often studied and found to be highly relevant. Theoretically, delegation entails two essential components of trust, defined in [42], including uncertainty (agents’ actions and the ensuing outcomes are neither entirely predictable nor completely unknown to users) and vulnerability (users are accountable or responsible for the outcomes). The relevance between trust and delegation has also been demonstrated empirically in management [13,43,44] and human–computer interaction research [16,17,39]. Other studies have focused on the effect of delegation on user–agent interaction instead of investigating individual factors [34,45,46]; for instance, in the public goods game, cooperation increases when individual players delegate the playing of the game to software agents [45].

2.2. Technological Immersion

In the literature, the term “immersion” is commonly used to describe an individual being enveloped by or absorbed in an experience, an interaction, or a mediated reality, as in the following quote:

“Immersion is a metaphorical term derived from the physical experience of being submerged in water. We seek the same feeling from a psychologically immersive experience that we do from a plunge in the ocean or swimming pool: the sensation of being surrounded by a completely other reality, as different as water is from air, that takes over all of our attention, out whole perceptual apparatus.”

[47] (p. 90)

Although the general idea behind the term is clear, its practical definition has been diverging in academia. A recent review article identified numerous definitions, including system immersion (i.e., technological immersion), sensory immersion, perceptual immersion, fictional immersion, psychological immersion, narrative immersion, imaginative immersion, systemic immersion, strategic immersion, tactical immersion, ludic immersion, and challenge-based immersion [48]. Among these, technological immersion constitutes a unique perspective that views immersion objectively as a description of technologies, whereas other definitions predominantly take a psychological stance regarding immersion as a subjective construct, such as feeling, attention, or mental experience. The prevailing standard for measuring technological immersion is the five dimensions proposed by Slater and Wilbur [49], i.e., inclusiveness, extensiveness, surroundingness, vividness, and matching:

“Inclusive (I) indicates the extent to which physical reality is shut out. Extensive (E) indicates the range of sensory modalities accommodated. Surrounding (S) indicates the extent to which this virtual reality is panoramic rather than limited to a narrow field. Vivid (V) indicates the resolution, fidelity, and variety of energy simulated within a particular modality (for example, the visual and colour resolution) […] Matching requires that there is match between the participant’s proprioceptive feedback about body movements, and the information generated on the displays.”

[49]

The more capable a media device is in each dimension, the more immersive the device is, and the more the device “delivers displays (in all sensory modalities) and tracking that preserves fidelity in relation to their equivalent real-world sensory modalities” [50]. By these definitions, reading a compelling comic magazine is psychologically more immersive than watching a boring movie on an 85-inch screen. Yet, the screen is technologically more immersive than the magazine because of its wide field of regard, surround sound, and more.

Various effects of high-level technological immersion have been documented in the literature. For instance, there is ample evidence that people experience a stronger sense of presence when using head-mounted displays rather than desktops [23,24,25,26,27,28,29]. Immersive media devices can also better support 3D-intensive tasks, such as spatial learning [51,52] and object detection [23,53,54], when compared with less immersive devices, as in desktops or smartphones. Psychological studies often report a positive correlation between technological immersion and stronger emotional responses [28,55,56] or social effects [20,24,57,58]. In education, students can benefit from being taught via technologically immersive devices with better knowledge retention and transfer [21]. An overview of the effects of technological immersion is provided in

Table 1. Effects of high-level technological immersion.

Positive (+)

Negative (−)/Neutral (o)

+ Higher level of perceived presence and co-presence [24]. + More positive emotions in video game playing [56]. + Higher-level emotional arousal when carrying out the Stroop task [28]. + Higher states of physiological and sexual arousal when viewing pornographic video materials [55]. + Improvement of social skills in autistic individuals after virtual social skill interventions [58]. + Lower-level implicit bias toward persons with disabilities after experiencing a disability simulation [20]. + Better knowledge retention and transfer in students educated using generative learning strategies [21]. + Higher recall accuracy in using a virtual memory palace [51]. + Higher accuracy of detecting assembly errors, for example, a misplaced piece in a model [54]. + Better performance in object recognition and discrimination after spatial learning in a virtual environment [52]. + Better performance in an object-identification task [23].

- Higher probability of the Uncanny Valley effect [59]. - Higher probability of inflicting fatigue on users [60,61]. - Higher cognitive load and thus worse learning outcome after learning science through virtual learning simulations [25]. ○ Similar performance in navigating a virtual environment [62,63]. ○ Similar levels of engagement but higher probability of the simulator sickness in video game playing [64]. ○ Similar performance in manipulating a virtual object’s basic spatial properties such as its position, orientation, and scale [60]. ○ Similar performance in memory rehabilitation in an after-stroke virtual exercise [65] ○ Similar performance in spatial understanding of a complex 3D virtual model, such as livers or pyramids [66]. ○ Similar levels of intention to reuse a virtual shopping environment after the initial use [67]. ○ Similar performance in knowledge acquisition after learning digitally [26,68].

.

Apart from technological immersion, according to [48], the other immersion definitions can be grouped into three categories:

• Sensory and perceptual immersion are closely related to technological immersion because both regard immersion as the subjective experience of an individual’s senses enclosed by a technology-driven environment. According to these two definitions, immersion increases, for example, when “large screens close to the players’ eyes and powerful sounds easily overpower the sensory information coming from the real world” [69] or when the physical world is hidden from users “by the use of goggles, headphones, gloves, and so on” [70]. The major difference between technological and sensory/perceptual immersion is that technological immersion objectively describes the technical configuration of a hardware device. In contrast, sensory/perceptual immersion refers to the extent to which users feel like their senses are enveloped by the hardware device.
• The technology-driven definitions mentioned above cannot explain why reading a compelling comic magazine feels more immersive than watching a boring movie, despite books being a relatively primitive form of media. This question is answerable if one follows another category of immersion definitions, including psychological, narrative, fictional, and imaginative immersion. These definitions are highly similar, and all emphasize the extent to which individuals are involved or absorbed in a narrative. Technology degrades to an insignificant factor, whereas storytelling elements such as plots and characters become paramount for immersion.
• Immersion may also occur without narratives and advanced technologies, such as when playing Tetris on one of the early portable devices. This type of immersion has been named systemic, strategic, tactical, and challenge-based, but is more widely known under the term “engagement”. These definitions argue that immersion originates from individuals trying to utilize their mental or sensorimotor skills intently to achieve a certain performance in a task. During this process, individuals are highly attentive to the task at hand and thus become absorbed in the task.

Returning to technological immersion, a within-subjects experiment and a between-subjects experiment were devised to investigate its effects on delegation.

3. Experiment 1—Within-Subjects

Following a within-subjects design, Experiment 1 ($N=30$) compared delegation to VAs across three different levels of technological immersion.

3.1. Method

During the experiment, participants were tasked to play the Colonel Blotto game [71]. The original Blotto game involves a martial scenario between two warring countries. The game was recontextualized with a commercial scenario, where two beverage companies compete for three cities’ beverage markets. Participants acted on behalf of one company and played against a VA that controlled the other company. Each company initially had ten liters of concentrate of its beverage product and could distribute the concentrate to the three cities (with the minimal division unit being one liter). The concentrate allocated for each city would be further diluted, bottled, and sold in the city. A company wins a city’s market if the company has distributed more concentrate to the city than the other company. A player wins the game if the player’s company has won two markets. Players had unlimited time to set their distributions, which were not shown until all the players confirmed theirs.

The number of distributions that a player could make in this game was limited within a suitable range that was neither too big (e.g., 100 L of concentrate would allow for thousands of possible distributions) nor too small (e.g., the Prisoner’s Dilemma provides a player with only two available options). Findings derived from such a moderate-complexity setting are arguably more generalizable than low- or high-complexity settings since, in real-world cases, available strategies are often not binary but constrained to a finite set.

3.1.1. Procedure

Participants were asked to finish three consecutive game sessions throughout the experiment. The three game sessions followed the same procedure illustrated in Figure 1. In each game session, participants played six rounds of the game described above. To render the game critical, based on a fabricated dynamic rewarding mechanism, participants were informed that their monetary reward depended entirely on their game performance. In the first three rounds of a game session, participants played against a trial agent (cf. Figure 2a) to practice the game, and the outcomes of these three rounds had no impact on their reward. In the remaining three rounds of a game session, participants played against an opponent agent (cf. Figure 2b) whose strategy was claimed to be different from the trial agent but, in fact, was the same (cf. Section 3.1.3). The participants’ performance in the remaining three rounds pertained to their reward: for each round they won or lost, the participants’ reward would be increased or decreased by a small amount of money, respectively. If a round ended with a draw, the reward would not change. The reward could not be deducted to negative and was minimally zero. The deception was disclosed and thoroughly explained by the end of the experiment, and all the participants received the same standard compensation. In between the first and remaining three rounds, participants were administered a brief questionnaire (cf. Table 2) within the mediated environment and then offered an opportunity to delegate the playing of the remaining three rounds to the trial agent. If a participant chose to delegate, the trial agent would take over complete control and autonomously play the remaining rounds on the participant’s behalf.

Following a within-subjects experimental design, the three game sessions were played using different media devices that varied in technological immersion (cf. Figure 3 and Section 3.1.2). The within-subjects design helps to isolate the impact of individual differences in, for example, demographics or experience with similar technologies. To mitigate potential carryover effects, participants were told that the agents in each game session—including both the trial and opponent agents—were unique and had different behaviors and strategies from the agents in other game sessions. Furthermore, for each participant, the order of the media devices was randomized using the Latin Square design. Every three participants constituted a square, and each participant in a square followed one of the following unique orders: (1) device A, device B, device C; (2) device B, device C, device A; or (3) device C, device A, device B. After finishing the three game sessions, participants completed a post-experiment questionnaire (cf. Table 3).

3.1.2. Variables, Conditions, and Measures

The independent variable was the media device’s technological immersion level. Based on the criteria (inclusiveness, extensiveness, surroundingness, vividness, and matching) mentioned in Section 2.2, three distinctive types of devices—a laptop, a Microsoft HoloLens 2, and a Meta Quest 2—were selected as the three conditions of the independent variable.

Laptop (low-immersion condition). Environments mediated by the laptop are only marginally inclusive because, in typical cases, its 17-inch screen can only occlude a fraction of the physical reality. The laptop’s level of extensiveness is high since it utilizes both visual and auditory channels to communicate with users. The screen can display 81 pixels per inch at the Full-HD level, offering a relatively high level of vividness but falling short of surroundingness due to its limited field of regard. Compared with the other two devices, the laptop is also inferior in matching as it only tracks some simple user inputs, such as mouse movement and keyboard strokes.

HoloLens (medium-immersion condition). The screen of the HoloLens is transparent, due to which environments mediated by the HoloLens are minimally inclusive. The HoloLens has a similar level of extensiveness as the laptop but displays less vivid images rendered at 42 pixels per degree. Nevertheless, the HoloLens has a much higher level of surroundingness than the laptop, thanks to its panoramic field of regard and stereoscopic display. The HoloLens also outperforms the laptop in matching with its higher-resolution head and hand tracking.

Quest (high-immersion condition). Environments mediated by the Quest are highly inclusive as the Quest has a dedicated design for hiding the perception of real-world environments from users. The Quest is comparable to the HoloLens in terms of extensiveness and matching. The Quest also has a panoramic field of regard and stereoscopic display, but it offers a higher level of surroundingness with its much larger field of view at 97 degrees compared with the 37 degrees provided by the HoloLens. As a side effect of its wide field of view, images displayed by the Quest are rendered less vividly at only 19 pixels per degree. Given that the field of view is generally more critical to the sense of presence than image quality [27], the Quest arguably has a higher level of technological immersion than the HoloLens.

The dependent variable was participants’ delegatory decisions in the game sessions.

In addition to the independent and dependent variables, some other factors were also measured to learn about participants’ decision-making processes and to check for potential alternative explanations. These factors were measured using questionnaires (cf. Table 2 and Table 3) on a 101-point scale that ranged from 0 to 1, with 0 and 1 denoting “strongly disagree” and “strongly agree”, respectively. The condition-wise questionnaire inquired about three factors related to delegation, including the trustworthiness and competence of the trial agent and the perceived workload during the game. Notably, the condition-wise questionnaire also probed participants’ subjective feeling of immersion, allowing us to examine its relationship with technological immersion. Since participants needed to repeat answering the condition-wise questionnaire three times in a short period of approximately 5–10 min, each of the four factors mentioned above was measured using a single item for simplicity and avoiding boredom and frustration in participants, though at the cost of decreased reliability and validity. Nevertheless, the four factors are relatively straightforward as opposed to more complex constructs such as presence, owing to which the single-item measure is arguably sufficient for comparing these factors across the three conditions [72,73]. The post-experiment questionnaire checked on some other delegation-related factors and asked participants to compare directly the trustworthiness of the trial agent in different game sessions.

Table 2. Condition-wise questionnaire (Experiment 1).

#	Item
1	The agent in the last rounds was trustworthy.
2	The agent in the last rounds was competent in the game.
3	I felt cognitively overloaded in the last rounds.
4	I felt immersed or involved in the game over the last rounds.

Measured factors: trustworthiness (#1), competence (#2), workload (#3), and immersion (#4). Regarding the last item, research suggests that people indeed can “reliably reflect their own immersion in a single question” [74].

Table 2. Condition-wise questionnaire (Experiment 1).

#	Item
1	The agent in the last rounds was trustworthy.
2	The agent in the last rounds was competent in the game.
3	I felt cognitively overloaded in the last rounds.
4	I felt immersed or involved in the game over the last rounds.

Measured factors: trustworthiness (#1), competence (#2), workload (#3), and immersion (#4). Regarding the last item, research suggests that people indeed can “reliably reflect their own immersion in a single question” [74].

Table 3. Post-experiment questionnaire (Experiment 1).

#	Item
1	The game outcome is important to me.
2	The agents felt controllable.
3	I must account for the game outcome.
4	Which agent did you find most trustworthy?

Measured factors: criticality (#1), controllability (#2), accountability (#3), and trustworthiness (#4).

Table 3. Post-experiment questionnaire (Experiment 1).

#	Item
1	The game outcome is important to me.
2	The agents felt controllable.
3	I must account for the game outcome.
4	Which agent did you find most trustworthy?

Measured factors: criticality (#1), controllability (#2), accountability (#3), and trustworthiness (#4).

3.1.3. VA Design

An individual’s decision on delegation can be significantly influenced by performance-related information [13,40,75]. Thus, to control for the impact of performance, an agent (either the trial agent or the opponent agent) was allowed to cheat when the agent played the role of the participants’ rival in the game. When cheating, the agent was secretly informed of the concentrate distribution on the participants’ side. With the information, the rival could win or lose certain rounds deliberately. For every three rounds in a game session, the rival randomly chose one of the following pre-defined sequences to enact: (1) win, lose, lose (the rival deliberately wins the first round and then loses the second and third rounds); (2) lose, win, lose; (3) lose, lose, win; (4) win, draw, draw; (5) draw, win, draw; or (6) draw, draw, win. For example, within the first three rounds of a game session, the trial agent may choose to win the first round and lose the second and third rounds, whereas, within the remaining three rounds, the opponent agent may deliberately lose the first and second rounds and win the third round. The different sequences all result in the same payoff: participants receive the money equivalent to winning one round.

The VAs were embodied in virtual humans in light of the evidence that users prefer to interact with visually human-like VAs in critical tasks [76]. The virtual human characters used in the experiment were generated using Character Creator 4, a software application that provides an integrated solution for creating high-quality digital characters. The characters were animated to make the interaction with VAs natural for participants. The animation combined a full-body idle animation clip with randomly generated simple facial motions, such as eye blinks or subtle movements of facial muscles. The trial and opponent agents were invariably embodied in a virtual woman and man across the three conditions, respectively. Although an agent’s gender can bias people’s delegation to the agent [13,14,77], its effect is arguably not potent enough to overshadow the effect of technological immersion. The appearances of the VAs in different game sessions were identical, except that the color of the VAs’ shirts was varied to emphasize visually that they were not the same agent. For example, the trial and opponent agents in one game session may both wear yellow shirts (cf. Figure 2), whereas, in another game session, they may both wear blue shirts.

3.2. Results

The participants (N = 30, 15 males and 15 females, mean age = 26.4) were recruited by disseminating a call to the student body at the University of Luxembourg. The exclusion criteria were that participants must be healthy adults without any motor handicaps and previous symptoms of seizures. The experiment results showed that participants were generally reluctant to delegate to the trial agent. Only four participants chose to delegate in the low-immersion condition, whereas this number increased to seven in the medium- and high-immersion conditions. There was no statistically significant difference in participants’ delegatory decisions among the three conditions (Cochran’s Q test; $Q\left(2\right)=2.571$, $p=0.276$). These results were unable to support the hypothesis that technological immersion influences users’ decisions on delegation to VAs.

Figure 4a summarizes participants’ responses to the condition-wise questionnaire. The assessments of the trial agent’s trustworthiness were almost identical under different conditions (Friedman test; ${\chi }_F^2\left(2\right)=0.019,p=0.990$). A similar pattern can be found in the assessments of the trial agent’s competence, except for a slightly higher variance (${\chi }_F^2\left(2\right)=4.056$, $p=0.132$). The perceived workload was generally low during the game sessions (${\chi }_F^2\left(2\right)=3.217,p=0.200$), whereas the subjective feeling of immersion was generally high. Participants felt less immersed in the low-immersion condition than in the other two conditions (no statistical significance; ${\chi }_F^2\left(2\right)=5.556,p=0.062$), while the medium- and high-immersion conditions were psychologically immersive to similar extents.

The responses to the post-experiment questionnaire showed that participants generally considered the game outcome critical: the mean value of their assessments of the task criticality was 0.705 ($sd=0.209$; Shapiro–Wilk test, $W=0.903,p=0.010$). Corresponding to the high-level criticality, the perceived accountability for the game outcome was also high, with the mean value at 0.731 ($sd=0.208$; $W=0.905,p=0.011$). The controllability assessments were positive (i.e., the trial agent was controllable) but had a relatively low mean value of 0.630 ($sd=0.226$; $W=0.929,p=0.047$). When participants were asked to pick the most trustworthy trial agent among the different conditions, eight of them picked the trial agent in the low-immersion condition, whereas this number in the medium- and high-immersion conditions grew to 12 and 10, respectively. This result conformed to participants’ tendencies to delegate, where more participants chose to delegate in the medium- and high-immersion conditions than in the low-immersion condition.

Of all the participants, 17 were non-delegators, i.e., those who did not delegate in any of the game sessions. The other 13 participants chose to delegate in at least one of the game sessions (10 participants delegated in one session, 1 participant delegated in two sessions, and 2 participants delegated in all three sessions). Delegators and non-delegators exhibited some notable differences in their questionnaire responses (cf. Figure 4b). Delegators reported a considerably higher level of perceived accountability for the game outcome than non-delegators (Mann–Whitney U test: $U=62.5,p=0.022$). This result corroborates an earlier finding that there is a positive correlation between users’ willingness to delegate a task to software agents and users’ perceived accountability for the task outcome [16]. Furthermore, when averaging each participant’s assessments of trustworthiness in the three conditions, delegators were found to put a higher level of trust in the trial agent than non-delegators ($U=68.0,p=0.039$), which is also consistent with other studies showing a positive correlation between trust and delegation [16,40]. Most delegators (9 out of 13) picked the trial agent in the medium-immersion condition as the most trustworthy one, whereas only 1 and 3 delegators picked the trial agent in the low- and high-immersion conditions, respectively. The preference for medium-level immersion pointed to the possibility of using mixed-reality environments to increase users’ trust in VAs.

A correlation analysis of the collected data showed that the perceived trustworthiness and competence of the trial agent were positively correlated, except that the correlation in the medium-immersion condition was only close to statistical significance (for the low-immersion condition, $r=0.374,p=0.042$; for the medium-immersion condition, $r=0.352,p=0.055$; for the high-immersion condition, $r=0.466,p=0.010$). The analysis also identified some other inconsistent correlations across the three conditions. For example, there was a positive correlation between the perceived competence and the subjective feeling of immersion in the high-immersion condition ($r=0.388,p=0.034$), yet this correlation was absent in the low-immersion condition ($r=0.167,p=0.376$) and the medium-immersion condition ($r=0.023,p=0.902$). The correlation between perceived competence and workload was statistically significant in the medium-immersion condition ($r=0.435,p=0.016$) but not in the low-immersion condition ($r=0.117,p=0.538$) and high-immersion condition ($r=0.335,p=0.071$). When data from the three conditions were aggregated, a positive correlation was found between the perceived trustworthiness and competence ($r=0.399,p<0.001$), which is consistent with other findings on the relevance between trust and competence-related factors, such as ability [42] or performance [78]. The analysis also revealed a positive correlation between the perceived workload and competence ($r=0.305,p=0.003$), suggesting that participants tended to consider the trial agent more competent when they felt like having a heavy workload.

3.3. Limitations

Experiment 1 has several limitations, one of which concerns the measure of the dependent variable that only comprises a binary choice on delegation. More fine-grained measures could be employed, such as letting participants make a series of delegatory decisions or using self-report measures to assess participants’ intention to delegate, as in the case of [16]. Similar to the issue of measurement, the questionnaires were overly simplified in order to pack the repeated measures in a short period of time. A few more questions could be added to the questionnaires to improve their validity and reliability without significantly increasing the risk of boring or frustrating participants. The within-subjects experimental design also constitutes a limitation because participants’ delegatory decisions could be influenced by their experience of previous game sessions. Although some measures have been taken to minimize potential carryover effects, its influence arguably cannot be eradicated, especially when examining high-level cognitive processes such as trust or decisions to delegate. More between-subject experiments would be helpful here to understand and confirm the relationship between technological immersion and delegation to VAs.

4. Experiment 2—Between-Subjects

Considering the limitations of Experiment 1 and aiming to test the hypothesis again using a different approach, Experiment 2 ($N=30$) was conducted following a between-subjects approach to compare delegation to VAs in two conditions of distinctive technological immersion levels.

4.1. Method

Unlike Experiment 1, where the participants’ reward was spuriously claimed to be contingent on performance, Experiment 2 involved no deception and induced the sense of criticality differently by giving participants the following brief at the beginning of the experiment:

Imagine you are a high-ranking manager of a large private fund. You are tasked with predicting the long-term price trends of 15 individual stocks. Your predictions are essential because subordinate traders will manage the fund and make investments based on your predictions in the coming years. For each stock, you can make the prediction yourself or let a virtual agent make the prediction for you.

4.1.1. Procedure

After the brief, participants interacted with the VA (the virtual robot in Figure 5) in a way that emulated a face-to-face conversation, where participants and the agent alternately spoke to each other. The interaction served as an opportunity for participants to learn about and familiarize themselves with the agent. To ensure the same interactive experience for all participants, the conversation was limited to a pre-defined dialogue (cf. Appendix A), where the agent always spoke first and then participants could only pick one from several provided options as their response to the agent. For example, during the conversation, the agent asked participants if they had any questions about itself, after which participants were presented with three options regarding the agent’s capabilities, developers, and hobbies, as Figure 5a depicts.

Once the conversation was over, participants were shown the price chart of each stock individually (cf. Figure 5b) and tasked to predict whether the stock price would increase or decrease based on its historical price trend. There were three options available to participants: increase (predicting that the stock price would increase), decrease (predicting that the stock price would decrease), or delegate (letting the agent make the prediction). After the decision was made, participants would immediately proceed to the next price chart without being informed of whether their prediction was correct or which prediction the agent had made for them. These pieces of information were concealed to prevent performance-related factors from influencing participants’ decisions on delegation. The experiment ended when participants finished predicting the last stock’s price trend. Participants were compensated with the standard monetary reward for their participation.

4.1.2. Variables, Conditions, and Measures

The independent variable was the media device’s technological immersion level. A low-immersion condition and a high-immersion condition were compared between subjects. In the low-immersion condition, the experiment was conducted on a laptop (the same one used in Experiment 1), whereas in the high-immersion condition the media device was a Meta Quest 2. The dependent variable was the number of times participants delegated to the agent. Compared with Experiment 1, where only a single binary choice was recorded, the measure of delegation in Experiment 2 was more fine-grained, allowing us to capture more nuanced changes in participants’ delegatory attitudes.

Before starting the experiment, participants filled in a short pre-experiment questionnaire inquiring about some basic demographic information and their experience in using VR and investing in stocks or other financial assets. A different questionnaire (cf.

Table 4. Post-experiment questionnaire (Experiment 2).

#	Item
1	The agent felt deceptive.
2	The agent behaved in a dishonest manner.
3	I was suspicious of the agent’s intent, action, or outputs.
4	I was wary of the agent.
5	I thought that delegating to the agent would lead to a negative outcome.
6	I was confident in the agent.
7	The agent has integrity.
8	The agent felt dependable.
9	The agent felt reliable.
10	I can trust the agent.
11	I felt emotionally close to the agent.
12	I like the agent.
13	I felt that I had a connection with the agent.
14	The agent felt warm and caring.
15	The agent appeared unattractive.
16	Predicting stock price trends felt difficult or challenging.
17	It was important for me to retain control of the prediction decision.
18	I can make better predictions than the agent.
19	Sometimes I chose to delegate simply because the game was boring.

Measured factors: trust (#1–10), rapport (#11–15), task difficulty (#16), control desire (#17), self-confidence (#18), and boredom (#19).

) was administered after the experiment to measure other relevant factors, including trust, rapport, boredom, self-confidence, task difficulty, and the desire to take control. Trust was measured using an adapted version of the Trust in Automation Questionnaire [79]. The rapport measurement (Cronbach’s $\alpha$ = 0.89) consisted of an Inclusion of Other scale [80] and several items used in other questionnaires on rapport [81,82,83,84,85]. Each of the remaining factors was assessed through a single item as they are more specific than high-level mental constructs, such as trust and rapport. Participants’ responses to both questionnaires were collected in the form of a 7-point Likert scale, except for the Inclusion of Other scale.

4.1.3. VA Design

The agent was embodied as a human-like virtual robot. Apart from a minimal level of full-body animation to make the interaction feel more natural, the agent was also equipped with three facial expressions that conveyed different signals, as illustrated in Figure 6. During the conversation, the agent displayed the friendly face and gazed at participants to appear personable. After the conversation, the agent showed the neutral face and focused on the price chart to indicate its concentration. When delegated, the agent would briefly switch to the thinking face as if it were making predictions. Since performance-related information was hidden from participants, the agent did not have to actually make the prediction and thus had no algorithm running behind.

4.2. Results

Participants ($N=30$) were recruited employing the same method and criteria used in Experiment 1 (cf. Section 3.2). They were evenly divided into two groups, which, as mentioned previously in Section 4.1.2, included a low-immersion group (nine males and six females, mean age = 27.8) and a high-immersion group (nine males and six females, mean age = 25.8). According to the pre-experiment questionnaire results, the two groups had highly similar backgrounds: most participants had very limited experience in using VR devices and some knowledge or experience about investing in stocks or other financial assets.

Participants in the low-immersion group on average delegated 4.06 times to the agent ($sd=2.87$; Shapiro–Wilk test, $W=0.945,p=0.444$), whereas the delegation frequency in the high-immersion group was almost the same but slightly higher at 4.20 times per participant ($sd=2.08$; $W=0.941,p=0.389$). The similarity in delegation frequencies was corroborated by a Mann–Whitney U test that showed no statistical significance ($U=103.5$, $p=0.722$). These results were unable to support the hypothesis that technological immersion can impact users’ delegation to VAs.

As Figure 7 illustrates, participants’ responses to the post-experiment questionnaire were generally consistent; for example, both groups considered the agent to be trustworthy. Nevertheless, the high-immersion group exhibited a higher level of trust than the low-immersion group, though the difference was only close to statistical significance ($U=67.0,p=0.061$). Notably, the perceived rapport in the high-immersion group was higher than in the low-immersion group with statistical significance ($U=59.0,p=0.027$). However, no statistically significant difference was found for the remaining four factors, including boredom ($U=127.5,p=0.529$), task difficulty ($U=108.0,p=0.861$), self-confidence ($U=122.5,p=0.680$), and the desire to take control ($U=102.5,p=0.683$).

A correlation analysis of the entire data set revealed that participants’ delegation decisions were not correlated with any of the factors measured in the pre- or post-experiment questionnaires. Nevertheless, the agent’s trustworthiness was found to be correlated with several other factors, including a positive correlation with the perceived rapport ($r=0.401$, $p=0.028$), a negative correlation with self-confidence ($r=-0.476,p=0.008$), and a negative correlation with task difficulty ($r=-0.384,p=0.036$).

A stock-wise view of the data revealed that most of the stocks were delegated to and predicted by the agent eight or nine times ($mean=8.27,sd=2.40$; Shapiro–Wilk test, $W=0.898,p=0.086$), which indicated that the difficulties in predicting each stock were generally at the same level.

5. Discussion

This article presents two experiments for investigating whether technological immersion is a relevant factor of delegation to VAs. Experiment 1 followed a within-subjects approach and compared three different levels of technological immersion. Each level was operationalized as a unique type of media device, including a laptop (low immersion), a Microsoft HoloLens 2 (medium immersion), and a Meta Quest 2 (high immersion). Experiment 2 employed a between-subjects design focusing on two distinctive conditions: low immersion (the laptop) and high immersion (the Meta Quest 2). Participants made a series of delegatory decisions in a new context, where the task and VA were different from those used in Experiment 1. Overall, neither experiment supported the hypothesis that technological immersion influences users’ decisions on delegation to VAs.

Nevertheless, within the samples collected, it can still be observed that participants were more likely to delegate when using immersive media devices. Although the difference was not statistically significant, it remains indicative of the connection between technological immersion and delegation. Such relevance was not observed in the experiments, possibly because its effect on delegation is limited and can be easily overshadowed by other influential factors. For example, in both experiments, the interaction with the agents lasted only for a short period of time, due to which there might have been a paucity of information (e.g., related to the agents) that made participants inclined to retain control rather than delegate. This issue manifested itself in Experiment 2, where the post-experiment questionnaire results showed that participants generally considered it important to retain control and made most of the predictions themselves. Controllability was also found to be an issue in one of our previous studies [86].

A noteworthy element of the experiments is the comparison between participants who were willing to delegate and those who were reluctant. In Experiment 1, which contained a one-off decision on delegation, there were some differences between delegators’ and non-delegators’ attitudes regarding the trial agent’s trustworthiness. The difference agrees with the literature, where trust and delegation were found to be highly relevant and positively correlated [13,16,17]. However, there was no correlation between trust and delegation in Experiment 2, which can be attributed to differing settings. In Experiment 1, the one-off decision on delegation was highly critical, as participants would lose all of their control once they delegated. Given that only limited information about the trial agent was provided, the high-criticality setting might have rendered the feeling of trust a central element in the participants’ decision-making process. In Experiment 2, participants were tasked with 15 consecutive delegatory decisions, which were comparatively failure-tolerant and created space for tactics such as a half–half strategy (i.e., participants predict half of the stocks and the agent predicts the other half) or simply delegating all the difficult stock predictions to the agent. Trust, in such cases, may degrade to a less important role in the decision on delegation, yielding to more strategic solutions.

Moreover, the agents’ trustworthiness was found to be correlated with several factors, including perceived rapport, task difficulty, participants’ self confidence, and the agents’ competence. These correlations demonstrate the complex and multi-faceted nature of trust, but meanwhile they also enrich empirical evidence to derive theories and practices for designing trustworthy VAs. Conversely, the relevance between these factors and delegation was not clearly indicated by the experiment results, despite the often-reported connection between trust and delegation. Trust, therefore, may play a critical but not decisive role in delegation. Nevertheless, it should be noted that the causality between trust and delegation remains unclear, despite evidence showing their relevance. Trust may be the product of users’ commitment to delegation rather than an influential factor that causes users to delegate. However, further studies on their causality would be necessary.

The experiments showed that technological immersion has the potential to modulate the intensity of the subjective feeling of immersion (cf. Experiment 1) and rapport (cf. Experiment 2). Both constructs can be useful in improving the user experience of interaction with VAs. Developers may consider tapping into this advantage and deploying VAs on immersive media devices in scenarios favored by social connection, such as education and healthcare.

To the authors’ best knowledge, currently there is no study that directly and systematically compares interaction with VAs between desktop, AR, and VR settings. It is crucial to know and understand the differences in user–VA interaction across these settings, since more and more immersive media devices—such as the already popular Meta Quest series and the anticipated Apple Glasses—are involved in human societies and activities. Based on the results and discussion presented in this article, developers may consider putting less effort in accommodating VAs into platforms of different technological immersion and, instead, focus more on other more salient factors, like agents’ performance. As exemplified by today’s proliferating generative artificial intelligence (e.g., ChatGPT), their near-human performance has made many people willing—sometimes blindly—to delegate critical tasks (e.g., thesis writing, exam essays, medical consultation) to them. Despite the fact that many of these GPT-based VAs are still using textual interfaces, their capability of natural and personable communication with users is no lower—arguably stronger—than embodied VAs.

5.1. Limitations

Due to the difficulty in recruiting enough participants in a reasonable period of time, the sample sizes of both experiments were small. As a result, the power of the statistical analysis in both experiments was generally below 80%, the commonly accepted value in experimental research. Thus, our findings are only indicative and require more samples to be fully validated. Given that the effect of technological immersion on delegation and other constructs (e.g., trustworthiness) might be small, one might consider carrying out similar experiments with a much larger sample size—if time and finances allow—to achieve high power and consequently derive more reliable findings. Alternatively, it is advisable to change the experiment design to increase the effect size, for example, by employing the latest VR headset (e.g., Meta Quest Pro) as the high-immersion condition so as to increase the immersion contrast between different conditions.

Another limitation of the experiments was that both were conducted in a laboratory, where the sense of criticality was induced through emulated (Experiment 1) or hypothetical (Experiment 2) contexts. Participants might lack the subjective feelings (such as pressure or anxiety) entailed in these critical situations. This issue is potentially more prominent in Experiment 1, where the performance-dependent rewarding mechanism might have gone awry and made the game simply more fun and thrilling instead of critical. Thus, an interesting future research avenue is to utilize immersive media technologies fully and envelope users in a virtual environment that replicates critical scenarios to fine details. Participants may react more authentically in virtual environments that appear realistic.

To obtain robust and generalizable findings, the hypothesis was tested in two experiments of different designs (e.g., contexts, settings), but at the price of reduced comparability. Since it remains unclear to what extent technological immersion is relevant to delegation, it is advisable to emphasize robustness and generalizability over comparability at this stage to gather sufficient evidence to clarify whether and when the relevance exists. Experiments with similar settings can be conducted later to explore more subtle aspects governing the potential effect of technological immersion on delegation.

6. Conclusions

With VAs becoming increasingly intelligent and autonomous, delegation to VAs has been gaining relevance and become more common in recent years. Although the emerging delegatory relationship with VAs can considerably benefit users in terms of, e.g., convenience and efficiency, it also comes with issues and challenges that have not been sufficiently studied and deserve further exploration. Various factors potentially underpinning delegation to VAs have been identified in the literature. Yet, little research has been dedicated to exploring the impact of hardware-related technological aspects on users’ delegatory behavior.

To close the gap on this research topic, this article examined whether technological immersion can impact user decisions on delegation to VAs. The hypothesis—users are more likely to delegate to VAs experienced via technologically more immersive devices—was tested through two experiments that compared participants’ delegatory behaviors toward VAs under different settings and conditions that varied in their technological immersion levels. The results indicate that technological immersion may not be a significant factor in users’ delegation decisions. Consequently, when designing VAs for critical tasks, developers need to focus on other more salient factors, such as agents’ trustworthiness or performance, rather than the influence of media devices on VA perception.

Following this work, many possible future research avenues are opening up. Apart from those already mentioned in Section 5.1, one may consider investigating less critical scenarios, such as letting VAs carry out online grocery shopping for users. Performance-related factors may have a weaker impact in such scenarios, which allows other factors—such as technological immersion—to unfold their influence on delegation to VAs. Alternatively, one may also consider integrating and comparing other mediating technologies, such as handheld video see-through AR devices or CAVE-like systems.