On the Synergy between Virtual Reality and Multi-Agent Systems

Abstract

Multi-agent systems integrate a great variety of artificial intelligence techniques from different fields, these systems have made it possible to create intelligent systems more efficiently. On the other hand, virtual reality applications are accepted as viable techniques in different areas such as visualization, simulation, design, and research. The combined use of these two technologies has led to the development of realistic and interactive applications. This work aims to do a Systematic Mapping Study (SMS) relying on the guidelines of Kitchenham and Petersen to analyze the state of the art of VR applications using multi-agent systems. Inclusion and exclusion criteria have been applied to identify relevant papers, 82 articles were selected and categorized according to the publication type, the research type, the asset type, and the purpose of the work. A complete review of the 82 selected articles was performed, based on the research questions that were established. This review made it possible to clarify the open lines of research that exist and to know where research in this field can be directed.

1. Introduction

In recent years, multi-agent systems (MAS) have gained strength, systems that integrate a great variety of Artificial Intelligence techniques from different fields (reasoning, knowledge representation, machine learning, planning, coordination, communication, etc.). These systems have made it possible to create intelligent systems more efficiently and naturally, allowing them to solve complex problems of the real world [1,2,3,4,5].

A complex system can be decomposed into components, these into sub-components, and thus successively until some primitive entities are obtained. These entities can be seen as agents that solve local problems and interact with other agents, so they can achieve the system’s objectives [1]. An agent is an autonomous and adaptive entity capable of communicating and taking action [6]. According to Balaji et al. [7] some important characteristics that differentiate an agent from a simple controller are:

• Situatedness: interaction between an agent and the environment through sensors and the resulting actions of the actuators.
• Autonomy: the ability of an agent to make its decisions about its actions independently without any external intervention.
• Inferential capacity: the ability of an agent to perform its tasks on abstract objective specifications.
• Responsiveness: ability to perceive the environment and respond taking into account any changes that may occur in the environment.
• Pro-activeness: agents must display opportunistic behaviors in which actions are aimed at achieving a goal rather than simply being actions in response to a change in the environment.
• Social behavior: although the agent’s decisions must be free from any external intervention, they must be able to interact with external sources when the situation requires it to achieve their objectives.

A MAS is a system composed of agents, in most cases these agents are computational processes (the agents can be distributed on different computers), but they also could be a robot, a human, etc. In a multi-agent system, a group of autonomous agents acts in an environment to accomplish an objective, which is achieved by cooperating or competing, sharing or not sharing knowledge among them.

For example, imagine that you want to simulate the situation in which a group of predators (a pack of wolves) is trying to hunt prey (a moose). In a MAS, each wolf in the pack and the moose are represented as agents with abilities and objectives. The wolf agents must be able to interact with their environment to decide which is the best way to achieve their objective (hunting the moose) but they must also be able to interact with the rest of the members of the pack. The moose agent, for his part, must also interact with his environment to know which way to take so as not to be reached and remain safe (for example, crossing a river that it knows that wolves cannot cross).

On the other hand, in recent times there has been a boom in the development and use of virtual reality (VR) applications [8,9,10]. In the beginning, 3D virtual environments were used almost exclusively for military purposes, however, lately, a new software tool to develop virtual 3D environments called game engine has been developed. These can maintain high visual quality and allow high interactivity with the user [11] while the environment is being built.

With virtual reality, the user is placed in a scene (sometimes as close as possible to reality) where he/she can interact with the objects in it. Nowadays, this technology is mostly used for videogames development, nevertheless, it also has begun to be used in many other fields (education [8,9], architecture [12,13], engineering [14], medicine [15], emergency services [16], tourism [17], marketing [18], etc.).

Today, VR is accepted as a viable technique in different areas such as visualization, simulation, design, and research. Virtual reality enables the user to interact with a three-dimensional scenario where the reality that is simulated may not be accessible to users in the real world due to being dangerous or physically inaccessible [19,20].

Nowadays, the combined use of these two technologies has led to the development of increasingly realistic and interactive applications in different areas such as videogames [21], education [6], or human behavior [21,22,23].

Taking this into account, it can be said that the use of VR combined with multi-agent systems makes available to anyone a useful and safe environment for the development of different practices without incurring costly testing, and also allows the development of powerful and versatile applications, useful in countless fields.

This work aims to analyze the state of the art of VR applications using multi-agent systems. Systematic mapping study (SMS) methodology was used, relying on the guidelines given by Kietchenham et al. [24] and Petersen et al. [25]. Thanks to this kind of study, it is possible to categorize existing studies in a specific research area through a scheme and structure indicating the frequency in which an investigation is carried out in the area [26,27].

2. Methods

A SMS based on the methodology and process proposed by Kitchenham et al. [24] and Petersen et al. [25] is implemented in this paper. Furthermore, an investigation following this methodology will be carried out. The procedure will consist of three phases: Planning, Development, and Report.

2.1. Planning

Using as reference the work of Kitchenham et al. [24], Petersen et al. [25], and Sittón-Candanedo et al. [27], three activities were included within the planning phase: motivation, objective, and research questions.

2.1.1. Motivation

The motivation for this Systematic Mapping Study is to get a general idea of the evolution of the use of MAS in Virtual Reality applications in different fields. The main objective of this research is to review the state of the art of studies that propose the combined use of multi-agent systems and VR to provide new and more effective solutions in different areas. Likewise, it is intended to find new lines of research in the area of MAS and VR.

2.1.2. Objective

In recent times, it has been seen that the use of MAS in different fields allows the creation of more complete and efficient applications. On the other hand, Virtual Reality has been gaining importance thanks to the fact that it allows simulating situations that if done in real life can be dangerous or/and expensive. Besides, it appreciates the great functionality that the multi-agent systems provide in VR applications in different areas such as education, human behavior, staff training, videogames, etc. Due to the importance that both MAS and VR have gained, the prime objective of this research is to identify the trend, evolution, and main research on the use of multi-agent systems within virtual reality applications, so that a fairly comprehensive view of current research on MAS and VR is obtained.

2.1.3. Research Questions

The research questions used as the basis for the Systematic mapping study are:

• What applications have been developed with Multi-agent systems in the field of Virtual reality?
• How does Virtual Reality benefit from the use of Multi-agent systems?

Answering these questions will allow categorizing the works found up to 18 November 2020, and present the results obtained in Section 3.

2.2. Development of the Study

During the development of the study, all the tasks associated with the SMS will be performed in this phase. As in the article of Sittón-Candanedo et al. [27], both the strategy and the study’s inclusion and exclusion criteria must be established.

2.2.1. Search Strategy

The Search strategy refers to the establishment of search guidelines to conduct the study. Population, Intervention, Comparison, and Outcomes (PICO), proposed by Kitchenham et al. [28], was chosen as a search strategy. This strategy was developed to identify keywords and formulate search guidelines taking into account the research questions [25]:

• Population: everything that the study covers, in this case, research articles.
• Intervention: refers to the methods, models, architectures, etc. of MAS that are used within VR solutions.
• Comparison: a comparison of the different solutions find during the process.
• Outcomes: not applicable to this SMS.

After applying the PICO search strategy, the keywords that were initially considered for the database search were Virtual Reality and Multi-agent systems. However, to increase the results obtained during the search, other keywords were also included: Multi-agent architecture, 3D application, 3D model, and 3D simulation (Figure 1). The Web of Science and Scopus were used for the research and the deadline for the article search was 18 November 2020. In

Table 1. Search strings used.

Database	Search String
Web of Science (WoS1)	(TS=(virtual reality AND multi-agent systems)) Bases de datos=WOS, CCC, DIIDW, KJD, MEDLINE, RSCI, SCIELO Período de tiempo=2016–2020 Idioma de búsqueda=Auto
Web of Science (WoS2)	(TS=((virtual reality OR 3D applications OR 3D model OR 3D simulation) AND multi-agent systems)) Bases de datos=WOS, CCC, DIIDW, KJD, MEDLINE, RSCI, SCIELO Período de tiempo=2016-2020 Idioma de búsqueda=Auto
Web of Science (WoS3)	(TS=((virtual reality OR 3D applications OR 3D model OR 3D simulation) AND (multi-agent systems OR multi-agent architecture))) Bases de datos=WOS, CCC, DIIDW, KJD, MEDLINE, RSCI, SCIELO Período de tiempo=2016–2020 Idioma de búsqueda=Auto
Scopus (S1)	TITLE-ABS-KEY (“virtual reality” AND “multi-agent systems”) AND PUBYEAR > 2015
Scopus (S2)	TITLE-ABS-KEY ((“virtual reality” OR “3D applications” OR “3D model” OR “3D simulation” ) AND “multi-agent systems”) AND PUBYEAR > 2015
Scopus (S3)	TITLE-ABS-KEY ((“virtual reality” OR “3D applications” OR “3D model” OR “3D simulation”) AND (“multi-agent systems” OR “multi-agent architecture”)) AND PUBYEAR > 2015

search strings are detailed, for each database, three search strings were created thinking of maximizing the number of works obtained in the preliminary results.

2.2.2. Inclusion and Exclusion Criteria

Regarding the inclusion and exclusion criteria, according to the guidelines set by Petersen et al. [25], it is important to establish the characteristics that the selected studies must have to be included or excluded from the SMS. Using this guide and the SMS developed by Sittón-Candenado et al. [27], the following inclusion and exclusion criteria were considered to select a paper:

• Inclusion criteria:

  -

  IC1: Studies about multi-agent systems and virtual reality.

  -

  IC2: Studies published between 2016 and 2020.

  -

  IC3: Articles published in conferences or journals and peer-reviewed book chapters.

• Exclusion criteria:

  -

  EC1: Duplicated papers.

  -

  EC2: Papers that do not address the topic of MAS and VR.

  -

  EC3: Papers in which the authors doubt whether there were contributions to the field or not.

  -

  EC4: Papers that are not in English or Spanish.

2.3. Report

To make this study a complete work, the report includes an outline of the information extracted, which includes the asset and the type of investigation. While the study filtering process was being performed, there were doubts about the relevance and/or fit of 43 studies within this SMS. To solve this issue, the other authors were asked about these studies, and from this discussion, it was decided whether or not a study should be included, this was an iterative process that was carried out in different stages of the filtering of studies.

2.3.1. Filtering Studies

In the first phase, a total of 572 studies were found. The process of filtering studies applying the aforementioned inclusion and exclusion criteria was carried out by the first author. In case it was not clear, the other authors were consulted if a paper should be included or excluded.

2.3.2. Classification Process

Taking as reference the work of Sittón-Candanedo et al. [27], when the filtering process has been concluded, the papers obtained were classified taking into account the following categories: Author, Title, Year, Publication type, Research type, and Asset type. The Publication type can be a Journal article, a Conference article, or a Book chapter. The Research type can be Conceptual, Experimental, or Validation, being non-exclusive so that a study can be classified into different types. In the Asset type category, it is intended to determine the main contribution of a work: Model, Framework, Architecture, Platform, etc. As in the previous category, the types are not exclusive. The results of this work can be seen in Appendix A.

2.3.3. Validation Process

For the validation of Systematic Mapping Studies, Petersen et al. [25] established guidelines that indicated a series of activities that should be accomplished when performing a study of this type. This evaluation consists of 26 activities that should be performed when conducting an SMS, also, it is indicated that a well-developed SMS must carry out at least 33% of the activities. Taking into account this validation system, it is necessary to calculate the relation between the number of activities and the 26 activities proposed by Petersen et al. to measure the validity of an SMS. Taking as an example the SMS conducted by Sitton-Cadenado et al. [27], we elaborated the

Table 2. Evaluation realized in this systematic mapping study (SMS), taking into account the guidelines proposed by Petersen et al. [25]

	Research Process		Applied (Yes = ✓, No = ✗)	Defined in:
Planning	Need for the map	1. Motivate the need and relevance	✓	Introduction (Section 1), Motivation (Section 2.1.1), Discussion (Section 4)
		2. Define objective and questions	✓	Objective (Section 2.1.2), Research Questions (Section 2.1.3)
		3. Consult with target audience to define questions	✗
Development	Choosing search strategy	4. Snowballing	✗
		5. Manual search	✗
		6. Database search	✓	Search Strategy (Section 2.2.1), Results(Section 3)
	Develop the search	7. PICO	✓	Search Strategy (Section 2.2.1)
		8. Consult experts	✗
		9. Iteratively improve search	✓	Report (Section 2.3)
		10. Keywords from known papers	✓	Table 1
		11. Use standards, encyclopedias	✗
	Search evaluation	12. Paper test-set	✓	Validation process (Section 2.3.3)
		13. Expert evaluation	✗
		14. Author’s web pages	✗
		15. Test-retest	✗
	Inclusion/Exclusion	16. Identify objective criteria for decision	✓	Search Strategy (Section 2.2.1), Inclusion and Exclusion Criteria (Section 2.2.2)
		17. Resolve disagreements among multiple researchers	✓	Report (Section 2.3)
		18. Decision rules	✓	Inclusion and Exclusion Criteria (Section 2.2.2)
Report	Extraction Process	19. Identify objective criteria for decision	✓	Inclusion and Exclusion Criteria (Section 2.2.2)
		20. Obscuring information that could bias	✗
		21. Resolve disagreements among multiple researchers	✓	Report (Section 2.3)
		22. Test-retest	✗
	Classification scheme	23. Research type	✓	Methods (Section 2)
		24. Research method	✓	Methods (Section 2)
		25. Venue type	✓	Results (Section 3)
	Validity and discussion	26. Validity discussion / limitations provided	✓	Validation process (Section 2.3.3)

where it is seen the mapping phases (Planning, Development, and Report) along with the activities that were executed (based on Petersen’s guidelines) to ensure the validity of our study, 16 activities were performed, constituting 61% of the total, an amount that allows us to say that it is a well-developed SMS.

3. Results

This section includes the Systematic Mapping study report. As detailed above, Virtual Reality and Multi-agent Systems were used at first as keywords for searching in databases, both for Web of Science and Scopus, but to increase the number of works obtained, the search was extended using the keywords seen in Table 1 (the preliminary results per database are shown in

Table 3. Search strings used.

Database	Search String
Web of Science (WoS1)	171
Web of Science (WoS2)	296
Web of Science (WoS3)	299
Scopus (S1)	246
Scopus (S2)	272
Scopus (S3)	273
Total	572

), finally obtaining 572 studies that were reviewed and filtered applying the inclusion and exclusion criteria.

This process of filtering studies is detailed in Figure 2, which includes the activities performed.

The results of the categorization of the works according to the Research type can be seen in Figure 3 and according to the Asset type in Figure 4.

As the SMS progressed, it became necessary and natural to carry out a new categorization based on the purpose of each of the works included in this study. This category will later allow making a classification that will be shown in the Discussion (Section 4). The works, according to the research carried out, may be directed to the development of videogames, the human behavior simulation, the dissemination of cultural heritage, urban planning, the development of autonomous vehicles, the machinery simulation, the development of intelligent virtual environments (IVEs) and robot simulation.

4. Discussion

A SMS is a systematic review of the literature that aims to identify and categorize the research works available in a specific area [24,29]. The main goal of this type of study is to determine the scope of a specific research field, analyzing the frequency of publications in different categories in a schematic way [27].

In the field of computer science, at first, Systematic mapping studies were used mainly in the area of software engineering. These allowed researchers to reach a fairly comprehensive understanding of the field of study and helped to refine research questions and topics of interest [27]. Today, this type of study has begun to be used in more specific areas, as in this research work, which focuses on the use of multi-agent systems in virtual reality applications.

As the study progressed, it became very clear that the form of categorization of the works that had to be followed to capture the studies found was according to their purpose (Figure 5), then, taking into account the keywords and titles of each work, it was decided to do the following categorization: videogames development, robot simulation, interactive environment development, human behavior simulation, dissemination of cultural heritage, urban development, development of autonomous vehicles, and simulation of machinery. Some works have been identified and are presented below.

4.1. Videogames

As is known, the development of videogames using virtual reality has become quite popular in recent times, so it was to be expected to find studies in the field of VR and MAS that would get involved in videogames [30,31,32,33,34,35,36]. Within the area of virtual reality videogames, MAS are used to improve the user experience during the game. In the study of Lakshika et al. [30], it is proposed a multi-objective evolutionary framework to develop low-complexity, high fidelity MAS through the use of machine learning trained taking into account human aesthetic judgment to achieve a realistic animation of non-player characters (NPCs). García-Ortega et al. [31] proposed a three-layered system: evolutionary computation, agent-based model, and logical reasoner, that is applied to the promotion of the myth of the hero’s journey to improve the experience of videogames users by creating more complex and interesting backstories. Narang et al. [32] developed a real-time algorithm based on a multi-agent simulation that allows inferring the intention of the user’s avatar, it is intended to improve the response of virtual agents to make the experience more realistic and comfortable for the user.

On the other hand, in the study of Makarov et al. [33], a first-person shooter game with advanced and intelligent NPCs was presented, the authors developed an intelligent agent that derives its skills from the experience in the game and makes decisions through the same cognitive patterns as humans. Likewise, they created smart teams in which each player can make their own decisions under the supervision of an expert team goal system.

In the paper of Seele et al. [34], it was highlighted that populating virtual worlds with virtual agents with realistic behavior improves the user’s sense of presence and therefore their experience inside those worlds. Taking this into account a framework for improving the user’s synthetic perception is proposed, and another work of Seele et al. [35] also presented a framework to address this issue, the main objective of this framework is to achieve a natural behavior for the agents and to get real-time performance and controllability. There is also a framework proposed by Nunnari et al. [36] for the creation of real-time interactive virtual humans, it allows the deployment of new characters in a short time of work.

Interactive Narratives

Matthew et al. [37,38] go beyond the subject of videogames and address the subject of Interactive Storytelling, specifically the way narrative agents are visualized in complex scenes. They proposed a framework to automate aspects of a staging of the activities of narrative agents and their interactions. Along the same lines, Porteous et al. [39] studied the use of interactive narratives to support psychology experiments in understanding stories. For this, they developed an interactive narrative generation mechanism that allows variations of the original story that facilitates the understanding of the story.

4.2. Robots

In the field of robot simulation, these technologies have also begun to be used, where the MAS agents serve as a representation of robots. Raza et al. [40] presented an imitation-based solution that learns a weighted Bayes structure, to learn collaborative strategies between multiple agents through imitation. The applicability of this approach was evaluated in the RoboCup Soccer 3D simulation environment.

Seghour et al. [41] investigated the control of multiple mobile robots based on a consensus approach, ensuring collision avoidance. For this, a consensus algorithm was developed, including a fuzzy logic system to calculate the translation and the angular velocity of the robot to maintain the formation and obtain the direction of the leading robot.

4.3. Intelligent Virtual Environments (IVEs)

For the development of IVEs, MAS have also been used within VR. Rincon et al. [42] presented the JaCalIVE framework design for the execution and adaptation of Intelligent Virtual Environments (IVEs). This framework is based on the MAM5 ontology (also proposed by Rincón et al. [43]) that describes the environment for MAS as populated not only by agents but also by other entities called artifacts. This ontology improves the knowledge representation characteristics and reasoning capabilities of an agent-based IVE. Continuing with this line, as an improvement of the previous ontology, Okreša et al. [44] proposed MAMbO5 ontology that allows all agents to reason and deal with the elements of the environment, and it is also integrated with the JaCalIVE framework.

Subagdja et al. [45] proposed a framework for the creation of interactive systems where the system components are considered autonomous agents that the user can configure. This framework allows non-expert users to configure all components at run time.

4.4. Human Behavior

Likewise, there are some studies in the field of human movement and behavior that combine both technologies [46,47,48,49,50,51,52,53,54,55,56]. Within this line, the field of crowd simulation has also gained strength, and we also find some studies on it using VR and MAS [57,58,59,60,61,62,63,64,65,66,67,68]. This type of simulation has multiple purposes: planning emergency plans [69,70,71,72], improvement of the work environment [73,74,75,76,77,78], improvement of the educational environment [79,80,81,82,83,84,85,86], people training [87,88,89,90,91,92,93,94,95], interactions between humans and avatars [96,97], smart buildings and commerce simulation [98,99], etc.

Narang et al. [46] proposed an algorithm called Body Aware Movement (BAM) to generate plausible movements for human-like agents. They integrated BAM with the Unreal game engine, so they could demonstrate the benefits of the algorithm generating agent-agent and avatar-agent interactions. Likewise, Cafaro et al. [47] presented an interactive system for generating plausible non-verbal behaviors realized by virtual agents in a small group. Song et al. [48], for their part, proposed a form of simulation of individual behavior combining Geographical Information Science (GIS) and MAS, thus creating Grid Object and Agent Model (GOAM). In like manner, Starzyk et al. [49] presented the development of a motivated learning agent that is equipped with impulses and pains that establish their motivations to learn to respond to events and create related goals. This agent was implemented within a virtual environment to explore the autonomous development of motivations and memory in agents.

Bönsch et al. [50] devise a multi-agent system that models the transition between an anonymous and an individual agent when the user approaches.

Zhang et al. [51] proposed a form of simulation of virtual humans behaviors in built environments, in which task-based behaviors are simulated. They integrated the traditional approaches of behavior authoring frameworks into a single framework, which allows satisfying the temporal variation, the specification of the level occupancy of buildings, the distribution of the behavior in rooms, and the motivation of the agents. In the same manner, the objective of Puig et al. [52] is to model activities that occur at home, so it proposes to use programs as a high-level representation of complex tasks.

In the paper of Andelfinger et al. [53], it is proposed an incremental calibration of pedestrian models using virtual environments where computational agents and human-controlled agents interact, with each calibration cycle, the computational agents adapt to previously observed human behavior. Bera et al. [54] emphasize that large and cohesive groups of people can be threatening to observers, therefore they propose an algorithm to predict the socio-emotional impact of these groups on observers. To validate this algorithm, they presented an interactive multi-agent simulation that models this type of group with VR.

Ranjbartabar et al. [55] investigated the usefulness of a virtual agent performing therapy on a user using the Emotional Freedom Technique (EFT) (psychological acupressure technique to optimize physical and mental health), comparing two types of therapeutic agents (one empathic and the other non-empathic).

Ohmoto et al. [56] proposed an agent behavior model that induces intentional stance in multiple to multiple human-agent interactions.

4.4.1. Crowd Simulation

On the other hand, Kim et al. [57] and Bera et al. [58] proposed a crowd simulation approach that combines data-driven methods with the interactivity of multi-agents techniques. Multi-agent models are used to change each individual’s behavior in response to an external event such as an external stressor resulting in a panic situation. Phone-Amnuaisuk et al. [59] also proposed a form of crowd simulation with realistic animations for the agent’s activities in a 3D environment. They used a multi-agent approach to simulate crowd movements, where an agent can perceive and interact with other agents and their environment.

To try to solve the problem of team-building with realistic crowd behaviors, Wang et al. [60] proposed a crowd formation framework through hierarchical planning (CFHP), consisting of three layers of automatic and intelligent planning: cooperative-task, coordination behavior, and action-control planning.

For their part, Agil et al. [61] proposed a model that allows virtual agents to produce plausible gaze behaviors within crowds, predicting the points of interest of the characters at each timestamp, the generated framework can be added as a plugin in game engines. Narang et al. [62] also presented an interactive approach to generate realistic movements for virtual agents, such as looking and gesturing and allowing a more natural interaction between humans and agents. Following this line, Novick et al. [63] presented a virtual reality scene where the user interacts with different virtual agents and initiates conversations with them either by approaching or gazing at them, likewise, the agent can initiate a conversation if the user approaches, a model was used for examining interpersonal distances. Likewise, Randhavane et al. [64] showed an approach to multi-agent navigation that facilitates face-to-face interaction between a user and a virtual agent that is part of a crowd. The authors described a model of approach behavior to predict if the user is approaching to have a face-to-face interaction or not, besides, they presented an interaction velocity prediction (IVP) algorithm that combines with the restrictions of movement of the human body and facial actions to improve the realism of the behavior.

Dickinson et al. [65] explored the impact of crowd density on a person’s experience and behavior, for which they created an agent-based simulation using VR in which participants walk freely and perform some task while interacting with agents controlled by a social simulation model. A form of crowd simulation is also proposed by Montana et al. [66], it is suggested a way to control pedestrian real-time simulations in which non-expert users can interact with the simulation by creating actions and objects from the scene.

Jayalath et al. [67], more than a form of crowd simulation, showed a form of simulation of individuals within a crowd, they presented an agent-based behavior model that simulates the decision-making abilities of individual characters within social groups in crowd simulations, including a decision function for each character to select the most appropriate target. Also, Chen et al. [68] proposed a system that simulates agents transporting objects in virtual environments with pedestrians. They proposed techniques to achieve different kinds of cooperation behaviors between agents in crowd simulations.

4.4.2. Emergency Plans

In the case of emergency plans, Yi Li et al. [69] studied the human environment during flooding and promoted a simulation model that combines cellular automata and a MAS to simulate crowd evacuations in flood disasters. The model proposed to integrate the evolution of water, land cover, objective domain, crowd, and individual movements data. For their part, Wang et al. [70] proposed a way to model the behavior characteristics of objects in a scene applying behavior tree and multi-agent model in-game artificial intelligence. The objective is to improve and serve as a guide for emergency plans and public safety deployments at different kinds of events.

Mao et al. [71] presented a form of simulations for emergency evacuations, also simulating emotional contagion in groups during these events. The authors presented an agent-based framework to simulate these evacuations in a virtual environment. Montecchiari et al. [72] also presented an agent-based evacuation tool that enables human interaction through immersive VR.

4.4.3. Work Environment

Barriuso et al. [73] proposed a multi-agent system called MOVICLOUD based on virtual organizations (VO), with the ability to perform social simulations in 3D environments. This study focuses on the integration of disabled people in workspaces, trying to improve the building’s accessibility and adapting work processes so that they continue to be efficient while allowing these people integration. Also to improve the work environment, Zeng et al. [74] proposed a platform that simulates the activity of an office building to determine and improve sedentary lifestyle at work, it is believed that the improvement in the design of these buildings can promote an active and healthy workforce. Antakli et al. [75,76] proposed a framework for the 3D simulation of hybrid equipment (robots, humans, etc.) in production scenarios to evaluate the critical properties of such scenarios and the assignment of tasks to team members. Antakli et al. [77] proposed an agent platform (HumanSim), a combination of Belief, Desire, Intention (BDI) paradigm and Answer Set Programming (ASP) to simulate worker agents in a shop floor domain. In the paper of Cai et al. [78], the use of an interactive virtual environment is proposed for the simulation of the behavior of miners inside an underground coal mine, for which a behavioral framework based on emotional states and motivational needs is proposed. The main objective of this system is to be able to simulate human behaviors that can generate accidents inside the mine.

4.4.4. Educational Environment

In the educational field, Calvo et al. [79] developed the Deployment Representation Model to work with MAS in immersive virtual environments systems applied to educational projects. Baierle et al. [80] proposed a logic programming environment with BDI and multi-agent concepts integrated with VR used to create an interactive virtual environment, focused on teaching the beginnings of Industrial Evolution. There is also a project proposed by Tazouti et al. [81] and Boulaknadel et al. [82], whose objective is Amazigh language learning in a virtual environment, using 3D serious games combine with MAS to improve interactivity.

Within this field, but aimed at teacher training, Nilsson et al. [83] proposed the simulation of human behavior but to train the gestures and body language of teachers in front of an audience. Therefore, they presented a MAS that uses the BDI model that emulates human behavior with a group of agents that would constitute a virtual audience. Likewise, Lugrin et al. [84] proposed a system for teacher training that simulates virtual classes with different levels of discipline. Their contribution was a method for the development and systematic evaluation of virtual human groups in VR and their ability to simulate specific atmospheres. In the same way, Barange et al. [85] proposed the design of an agent architecture for the generation of reactive, proactive, and pedagogical behaviors for agents. The project aims to support instructors as well as improve the learning experience for trainees. The proposed architecture is based on the BDI model and allows the coordination of activities between agents using communication in natural language. Moreover, Fukuda et al. [86] created a virtual classroom using VR where virtual students (agents) can interact through the atmosphere, then teachers can improve their class by taking into account this atmosphere.

4.4.5. Training

Within the scope of staff training, Blankendaal and Bosse [87] carried out a study which it is discussed how biofeedback can be shown and used in the context of virtual training based on social skills agents, it is simulated an adaptive scenario where the user interacts with a threatening virtual interlocutor. In the same sense, Feng et al. [88] intend to examine the effects of negative feedback on users, with virtual instructors, taking into account the gender and non-verbal behavior of those instructors. Johnson et al. [89] proposed a way to learn negotiation skills using virtual role-playing agents, they focused on the post-negotiation feedback, arguing that is the most crucial part of a training experience. They used an agent called Conflict Resolution Agent (CRA) that allows conversing with simulated negotiation partners.

Tavcar and Gams [90] proposed an approach in which the simulated agents are virtual copies of humans being trained. These agents were built and tested to provide feedback and decision support in future rounds of human learning. The objective of these simulations is to facilitate training in simulated environments dealing with both cooperative and adversarial agents.

Barthès et al. [91] discussed the design of training environments using a VR space to train medics to organize a medical post in a combat area so that all wounded can be treated efficiently. Also in the field of medicine, De Lima et al. [92] presented a complementary environment for teaching medicine in which clinical cases are simulated to evaluate the knowledge of the students, this system uses a MAS and machine learning for the disease classification of virtual patients. In the paper of Benkhedda et al. [93], to teach first aid in the event of cardiac arrest, serious games and multi-agent systems are combined to create an agent-based simulation that allows simulating this situation, integrating BDI agents into the NPCs.

In the article of Ooi et al. [94], a VR training system focused on fire prevention was proposed, it is said that VR provides an experience closer to reality. It uses different types of agents: fire smoke agent showing the details of the fire over time and indicating whether or not the player can evacuate, and student agent showing an agent occupying an area.

Tianwu et al. [95] proposed a travel training system for children with intellectual disabilities that would allow them to achieve a slightly more independent life. To simplify the interaction between all traffic entities and the virtual traffic environment, they implemented a MAS with a centralized data manager.

4.4.6. Interactions between Humans and Avatars

Best et al. [96] proposed an approach for generating realistic interactions between virtual agents and user avatars, they called it Sense-Plan-Ask (SPA) and enables virtual agents to achieve their individual goals with uncertain information in complex multi-agent and multi-avatar environments. Braz et al. [97] described the development of a cognitive simulation game that integrates a group of agents that manage and monitor user performance as they move around the environment, the main goal is to improve the attention and memory of patients with neuropsychiatric disorders.

4.4.7. Smart Buildings and Commerce

Christian et al. [98] presented a prototype of a multi-agent simulation tool of a shopping center together with a fuzzy logic algorithm that uses the basic knowledge of the agents as input to determine the behavior of the agents in the system so that this behavior is the most realistic possible.

Zhao et al. [99] proposed a multi-agent human activity simulator for the development and evaluation of activity analysis algorithms in smart buildings. Their proposal integrates the Building Information Modeling (BIM) with Unity 3D where they simulate multiple agents with different abilities and tasks.

4.5. Cultural Heritage

3D simulations of human behavior have even been used for the dissemination of intangible cultural heritage as in the study of Vosinakis et al. [100], where the ancient agora of Athens is reconstructed in a virtual world and an interactive scenario is created, where user can see what life was like there at that time. They used MAS to simulate people’s behaviors using NPCs and create a world where users can explore the environment, observed the activities performed by NPCs, and discuss with them to learn more about their life and customs.

On the other hand, Kiourt et al. [101] presented a framework for linking up Agents and Multi-agents Systems with virtual environments and cultural heritage.

4.6. Urban Development

Urban development also makes use of these two technologies because (i) it allows reducing costs when planning, designing, and building, (ii) it allows simulating different scenarios without putting anyone’s life at risk, and (iii) it allows efficiently simulating different scenes.

Ren et al. [102] presented an approach that combines physics-based simulation with data-based techniques to compute trajectories and behaviors of entities, this approach can simulate heterogeneous agents such as human crowds, traffic, or vehicles.

Rivalcoba et al. [103] presented a system for simulate and visualize pedestrians inside urban environments, the system includes a MAS that assigns physical and psychological characteristics to the virtual agents.

Okamoto et al. [104] also proposed a virtual simulator that optimizes the design factors of a lane from the point of view of reducing pedestrian stress to reduce accidents in the bicycle and pedestrian lanes. Also, in the work of Chen et al. [105], VR is used for the operational simulation of urban rail traffic, it is required that pedestrians act realistically in the simulation, so this model was developed based on a multi-agent system that allows imitating the behavior of individuals and their interaction with the traffic system. For their part, Garg et al. [106] developed a tool for traffic simulation that, through an interactive environment, allows a rigorous simulation that is close to reality. The idea is that this simulation serves to carry out an implementation in the real world.

4.7. Autonomous Vehicles

Chen et al. [107] proposed a simulation platform for the validation of autonomous vehicles while using a multi-agent interaction system, it evaluates the perception of the scene, planning algorithms, decision-making, and control of vehicles. Elmquist et al. [108] proposed a framework also related to the validation of autonomous vehicles, the idea is to evaluate the piloting control programs and improve them in a safe environment, one of the main components of this framework is a multi-agent simulated communication support.

4.8. Machinery

For the application considered in the study of Ďurica et al. [109], a continuous and decentralized route planner is necessary for a modular manufacturing line, therefore they proposed an approach in which a delegate multi-agent system (D-MAS) is used and this approach verified using a simulation with VR.

Xie et al. [110] said that the existing control program for the machines in a mechanized coal mining face doesn’t satisfy the dynamic environment of coal mines. Then the authors established a collaborative mathematical model that takes into account the effects of the dynamic environment. They also proposed a collaborative planning simulator in VR with MAS-based methods for the machines.

Wang et al. [111] proposed a Multi-agent based virtual maintenance simulation system (MVMSS) where the virtual human, the tools, and the device can be considered agents. This system would allow a more realistic and natural interaction of the user with the environment.

4.9. Resolution of Research Questions

This section will summarize the responses to the research questions posed in Section 2.1.3.

4.9.1. What Applications Have Been Developed with Multi-Agent Systems in the Field of Virtual Reality?

In the Discussion (Section 4), a description of the different applications found while conducting this study was made. These applications are related to different purposes that are described and in which that section has been divided: videogames development [30,31,32,33,34,35,36,37,38,39], robot simulation [40,41], interactive virtual environments development [42,43,44,45], human behavior simulation [46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99], dissemination of cultural heritage [100,101], urban m development of autonomous vehicles [107,108], and machinery simulation [109,110,111] (Figure 5).

Also, simulation of human behavior can be divided into subcategories: crowds simulation [57,58,59,60,61,62,63,64,65,66,67,68], emergency plans [69,70,71,72], work environment [73,74,75,76,77,78], educational environment [79,80,81,82,83,84,85,86], staff training [87,88,89,90,91,92,93,94,95], interactions between humans and avatars [96,97], and smart buildings and commerce [98,99].

As shown in Figure 6, most of the works found during this study are aimed at simulating human behavior (65.9%). This is because it has applications in different areas, as seen in the diagram (Figure 5), and those that have as-purpose crowd simulation, are the ones that represent the majority within the works about human behavior simulation (22.2%).

4.9.2. How Does Virtual Reality Benefit from the Use of Multi-Agent Systems?

In general, Virtual Reality applications benefit from Multi-Agent Systems since they allow the development of high-quality applications in a more efficient way. But according to the purpose of the works, Virtual Reality applications benefit in different ways from the use of Multi-Agent Systems, in the case of videogames, MAS serves to improve the user experience, improving the quality of the games [30,32] and creating more realistic stories [31] and characters [33,34,35,36]. On the other hand, in the case of robot simulation, the agents are used to make simulations closer to the reality that allows the user to see the movement of the robot when programming them without the need for the physical robot [40,41].

In the case of the development of IVEs, frameworks were presented for their creation in which MAS serve to represent all the objects in the environment [42,43,44,45]. In the human behavior simulation, multi-agent systems are used to create different types of movements [46,47] and behaviors [48,49,50,51,52,53,54,55,56] for avatars and NPCs that are appreciated by the user as realistic so that they can be used for more serious simulations such as emergency plans [69,70,71,72], improvements of the labor [73,74,75,76,77,78] and educational [79,80,81,82,83,84,85,86] environment, personnel training [87,88,89,90,91,92,93,94,95], etc. For the dissemination of cultural heritage, MAS serve to represent people in the past as well as objects and artifacts [100,101], so that users could learn about it.

Regarding urban development, MAS are used to make realistic simulations of pedestrians, traffic objects, cars, etc., in this way improvements can be made in urban planning [103,104,105,106]. In the development of autonomous vehicles, multi-agent systems are used to create an interactive system to validate vehicles in the real world [107,108], in a way that reduces costs and avoids accidents. Likewise, in the simulation of machinery, the MAS are used to simulate the objects that make up the machinery [109,110,111] in a way that serves as a form of training for the personnel who have to use it.

4.10. Future Research

To talk about future research in this field it is necessary to discuss the research gap. First of all, it can be seen that VR and MAS are technologies with a lot of potentials that, combined, can be very useful in different fields, as has been seen throughout Section 4.

Now, although in all fields it is possible to find ways to improve what already exists and create new applications, here the importance of the simulation of human behavior is highlighted. This study made it possible to demonstrate the simulation of human behavior is useful in many directly associated fields such as education, training, emergency plans, work environment, etc., but these simulations are also useful in areas in which the main thing is not human behavior such as videogames, dissemination of cultural heritage, urban development, etc.

Many works focus on the relationship between the user and avatars. An important part for the improvement of the user-avatar relationship is the gestural response of the agents, however, this topic has rarely been addressed and less in simulations where there are a large number of agents. There are different studies in which an attempt is made to infer the user’s intention, based on that, the agents make a decision. However, these behaviors do not usually also occur between agents, this makes the simulations lose realism since the agents only interact with the user and never between them. Also, this type of inference usually only generates two intentions: avoiding the agent as if it were an obstacle or having face-to-face interaction, a situation that makes simulations unrealistic by not being able to emulate other types of interactions.

Following this line, some studies focus on the changes in the agents’ gaze behavior after the inference of the user’s intention. However, most of the time this intention has no impact on other aspects of the agent such as changes in their movements or objectives. Likewise, in the real world, gaze behavior is usually accompanied by a change in steering behavior, and this is not usually taken into account in simulations. In the same way, changes in behavior due to events that occur in the environment such as people running or loud noises are rarely taken into account.

On the other hand, when creating human simulations, only the environment is usually taken into account both for decision-making and for the movement of agents, but if what is sought is to make simulations close to reality, it would be necessary to include the mental state of agents (emotions, personality, etc.) as part of the agent’s perception process, since these factors influence human attention processes in the real world.

As it is seen, there are quite a few future research lines, here some related to improving the simulation of human behavior were highlighted.

5. Conclusions

Advances in both technologies, MAS and VR, have allowed the development of applications of higher quality and realism in different fields. These applications have allowed advances in different areas that are exposed in this paper: videogames development, robot simulation, IVEs development, human behavior simulation, dissemination of cultural heritage, urban development, autonomous vehicles development, and simulation of machinery; either by allowing the simulation of real situations that can become dangerous and expensive to emulate in real life, or allow more realistic simulations and more interesting characters.

To carry out this study, two research questions were established: What applications have been developed with Multi-agent systems in the field of Virtual reality? and How does Virtual Reality benefit from the use of Multi-agent systems? The answers to these questions made it possible to demonstrate the magnitude of research that has been carried out so far in this area.

In the area of development of videogames with virtual reality, multi-agent systems are used mostly to improve the user experience during the game, either by achieving more realistic animations, creating more realistic and complex characters, improving communication between the user and the avatars and characters of the game, among others. In the field of robot simulation, the MAS agents serve as representations of the robots, so that collaboration and movement strategies between multiple agents can be simulated. Within the development of intelligent virtual environments, frameworks were proposed that describe the environment for multi-agent systems, environments composed of agents and entities configurable by the user.

It was found that most of the research in this area focuses on the simulation of human behavior, either studying the generation of plausible humans movements for the agents, creating realistic interactions of the human agents with the user, among others. Now, within the simulation of human behavior different fields are addressed: simulation of crowds, simulation of emergency plans, simulation of the work environment, simulation of the educational environment, simulation for the training of personnel, interactions between humans and avatars, and simulation of smart buildings and commerce.

These technologies have also been used to disseminate knowledge about cultural heritage, for this purpose scenarios of the past are usually reconstructed in virtual environments and an interactive environment is created where the user can appreciate all the agents and components of the scenario. Likewise, it can be seen that different research works were carried out in the field of urban development, which in general try through simulations to improve urban planning and then implement it in the real world.

In the development of autonomous vehicles, VR and MAS have also been used, generally to validate, through simulations, the algorithms and other components of these vehicles before putting them into operation in the real world. In the same way, in the field of machinery development, simulations have been created that allow to validate and improve the processes carried out by these machines and can also serve as training for the personnel in charge of putting this machinery into operation.

Performing this study made it possible to demonstrate the magnitude of research that has been carried out on this topic, in addition to highlighting that the majority of research attempts to solve the problem of conducting realistic human behavior simulations, and especially in the simulation of crowds and training staff.

Developments in these areas open the possibility of creating virtual reality applications that use multi-agent systems to train personnel in the field of driving, in an environment in which all objects typical of a city and necessary for training are included as agents, in this way, a realistic environment could be established that serves as a complement to driver training in an environment in which nobody’s life is put at risk.