Immersive Virtual Reality Enabled Interventions for Autism Spectrum Disorder: A Systematic Review and Meta-Analysis

Abstract

Autism spectrum disorder (ASD) is characterized by persistent deficits in social communication and interaction, which can have significant impacts on daily life, education, and work. Limited performance in learning and working, as well as exclusion from social activities, are common challenges faced by individuals with ASD. Virtual reality (VR) technology has emerged as a promising medium for delivering interventions for ASD. To address five major research questions and understand the latest trends and challenges in this area, a systematic review of 21 journal articles published between 1 January 2010 and 31 December 2022 was conducted using the PRISMA approach. A meta-analysis of 15 articles was further conducted to assess interventional effectiveness. The results showed that most studies focused on social and affective skill training and relied on existing theories and practices with limited adaptations for VR. Furthermore, the enabling technologies’ affordances for the interventional needs of individuals with ASD were not thoroughly investigated. We suggest that future studies should propose and design interventions with solid theoretical foundations, explore more interventional areas besides social and affective skill training, and employ more rigorous experimental designs to investigate the effectiveness of VR-enabled ASD interventions.

1. Introduction

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by one’s persistent deficits in social communication and interaction across multiple contexts as well as restricted and repetitive patterns of behavior that could cause significant impairments in social and occupational functioning [1]. According to recent estimates from the Autism and Developmental Disabilities Monitoring (ADDM) network of Centers for Disease Control and Prevention (CDC) in the United States, about 1 in 54 children aged 8 years has been identified with ASD [2]. Epidemiological studies suggested that the prevalence of ASD does not vary significantly in different geographic regions or among different ethnic groups [2,3]. Theory of Mind (ToM) deficit, meaning the limited or absent ability of perspective-taking and understanding other people’s beliefs, was originally proposed to explain deficits of the ASD population in social communication and interaction [4,5,6]. Later, weak central coherence and executive dysfunction were identified and accounted for other ASD symptoms such as perceptual abnormalities and repetitive patterns of behaviour [7,8,9,10]. Currently, there is no clinical evidence that fully supports a particular treatment for ASD. However, a great number of evidence-based interventions have been developed and delivered to help children with ASD at a young age. The most widely delivered evidence-based interventions include Applied Behavior Analysis (ABA) [11], Picture Exchange Communication System (PECS) [12], Treatment and Education of Autistic and Related Communication Handicapped Children (TEACCH) [13], and Social-Communication, Emotional Regulation and Transactional Support (SCERTS) [14]. Besides evidence-based interventions, recent studies suggested that virtual reality (VR) technology could be a promising medium to deliver interventions for ASD [15,16].

VR technology is a combination of software and hardware technology, which aims to create an interactive and vivid computer-generated virtual environment, where people can experience “the sense of presence” [17,18]. VR can provide a safe and controllable environment, showing characteristics that are important when delivering ASD interventions. Features of VR enable the transfer of knowledge and skills gained in the environment to real-life scenes when some resemblance is realized [19,20,21]. This transfer used to be challenging for the ASD population in conventional interventions due to their rigid thinking and weakness in generalization [22,23]. Moreover, prior studies suggest that the ASD population primarily relies on visual thinking [24]. Computer-generated visual stimulations during VR exposure can not only attract attention with their visual cues but also help individuals with ASD think structurally [25,26,27].

VR comes in many forms, from virtual environments displayed on a desktop computer to immersive VR that one has to engage with using head-mounted displays (HMDs) (also known as, VR headsets). HMDs can enable highly immersive VR experiences but have to be worn directly on the user’s head, which can be a challenging procedure for children and young adolescents with ASD. The exploration of the use and effects of immersive VR on individuals with ASD started in the early 1990s. Early studies focused on the acceptance of VR devices and the comprehension of computer-generated visual stimulations by the ASD population. Strickland et al. reported the use of HMDs on two school-aged children with ASD, a 7.5-year-old girl and a 9-year-old boy [28]. Despite the poor display quality and the far-from-ideal ergonomics of the HMDs used for this study, the two children were found to be able to accept the HMDs with preliminary efforts. Moreover, there was strong evidence showing that both of them were able to comprehend and respond to computer-generated visual stimulations. Recent studies that utilized more user-friendly off-the-shelf HMDs demonstrated similar results—individuals on the spectrum, in general, can accept wearing HMDs with minimal preliminary efforts and are able to initiate meaningful interactions with the virtual objects as well as navigation in the simulated scenes [29,30]. Besides HMDs, surround-screen projection, such as the Cave Automatic Virtual Environment (CAVE) [31,32], can also provide a highly immersive VR experience. Although costly to build and maintain, the surround-screen projection that was originally designed to be a one-to-many presentation installation has its own advantages over HMDs when being used for delivering ASD interventions [27,30]. For example, during interventions, it is much easier for psychologists, psychiatrists, and other professionals to provide facilitation in this particular setting than for someone whose vision is completely blocked by HMDs. Compared to conventional desktop VR environments, both types of immersive VR technologies can deliver a higher sense of presence in the computer-simulated scenes [18,33], which is believed to be critical for easing the generalization process of individuals with ASD [19,20,21]. In this paper, we only include studies that utilize immersive VR to deliver ASD interventions, meaning that we do not look into work utilizing desktop VR environments.

In the past decade, the enabling software and hardware for VR have become more capable. Related studies show the potential of immersive VR-enabled ASD intervention, with a number of empirical studies in this period covering topic areas such as social skills [27,34,35,36], emotion recognition [27,34], daily living skills [21], vocational skills [37], and phobias [38,39]. In regard to the fast growth of applying VR for ASD intervention, a few review articles have been published, covering various aspects of this interdisciplinary research area. For example, the systematic review published by Bozgeyikli et al. mainly focused on the design considerations of the VR-enabled interventions for the ASD population [40]. It covered the task design, information presentation, and the VR system, but no findings were reported regarding the effectiveness of the interventions reviewed. Bradley and Newbutt also conducted a systematic review covering the area of application and the participant characteristics but limiting the enabling technology to HMDs [41]. However, more recent publications in this area of research reporting more rigorous experiment designs (e.g., randomized controlled trials) with a higher number of citations, such as [27,39], were not included in the most recent reviews (e.g., [42,43]). In addition, although previous reviews have highly praised VR as a promising medium for ASD intervention, the remaining challenges, current research, and practical gaps have yet to be clearly identified with the support of data. Hence, the present study aims to further contribute to this area by providing a systematic review and a meta-analysis of high-quality empirical studies on using immersive VR for ASD intervention. Five research questions to be addressed are listed as follows: (1) what are the areas of intervention in previous empirical studies? (2) What are the affordances of immersive VR in ASD intervention? (3) What are the theories applied to the design of the VR content? (4) How effective are these immersive VR-enabled interventions in various topic areas? (5) What are the latest research trends and the challenges that need to be further addressed in future studies? These questions have been chosen as they address the main issues around using immersive VR for ASD interventions. We believe that the answers to these questions will allow us to assess the value of immersive VR for individuals with ASD, as well as identify research gaps and directions for future research and development. These answers will contribute to the further development of this line of research, supporting the well-being of individuals on the spectrum.

2. Methods

The Preferred Reporting Items for Systematic Reviews (PRISMA) guidelines were applied to guide the review process of this study [44]. PRISMA provides a standard peer-accepted methodology that uses a guideline checklist to contribute to the transparency, quality, and replicability of systematic reviews and meta-analyses. The detailed protocols are as follows.

2.1. Search Procedure and Criteria

A systematic literature search was conducted from four online databases, which are PubMed (https://pubmed.ncbi.nlm.nih.gov/), IEEEXplore (https://ieeexplore.ieee.org/), Education Resources Information Center (ERIC) (https://eric.ed.gov/), and Web of Science (https://www.webofknowledge.com/), in order to cover literature published under different disciplines. PubMed mainly comprises literature related to medical sciences. IEEEXplore was searched for technology-related publications. ERIC focuses on education and educational technology. Web of Science is believed to be an interdisciplinary database. The search was limited to journal articles published in English between 1 January 2010 and 31 December 2022. The search query included “autism” or “autism spectrum disorder” or “Asperger”, “virtual reality”, and “intervention” or “training” or “education” or “learning” or “rehabilitation” or “intervention”, but “survey” or “review” was explicitly excluded. The decision of having 1 January 2010 as the cut-off year of inclusion is mainly due to the lack of studies on this research topic before 2010. Although this research area can be traced all the way back to 1996 when Strickland et al. studied the use of HMDs on two children with ASD [28], there is a lack of follow-up studies from 1996 to 2010 [42]. The decision of including only journal articles is mainly because most of the conference papers had a clear focus on the technology and less emphasis on the psychological, psychiatric, or psychoeducational aspects of the studies. For example, many of these conference papers mentioned the use of VR as stimuli for capturing participants’ electroencephalogram or eye gaze data during VR exposure but did not provide any interventional procedures or did not mention the studies’ implications for ASD intervention. As mentioned above, this study aims to cover a broader spectrum of concerns in this research area; if such conference papers were included in the analysis, the interdisciplinary nature of this area might be overlooked. Although they are not included in the systematic review or the meta-analysis, those conference papers will contribute to this review and will be cited individually, especially when discussing the technical aspects of the research as well as the latest trends and challenges.

2.2. Selection Procedure

Each of the articles found during the literature search was reviewed by two independent evaluators (i.e., the first and second authors of this article) to determine whether it should be included or excluded by following pre-defined inclusion criteria. The inclusion criteria were (1) the interventions were delivered through immersive VR devices, (2) the participants of the studies were formally diagnosed with ASD according to DSM-V [1] or ICD-10 [45], (3) the interventions were assessed through repeated measures, and (4) the study should not be a single-case study. If the two evaluators could not agree on whether an article should be included, a third evaluator (i.e., the fourth author of this article) would make the decision.

3. Results

3.1. Search Results

The initial search yielded 108 articles from PubMed, 21 from IEEEXplore, 40 from ERIC, and 191 from Web of Science. After removing the duplicate records and retracting publications and manually identifying records from prior systematic reviews, 252 records were included in the initial screening (see Figure 1). The initial screening was performed using the titles, abstracts, and metadata of the records. During the initial screening, a total of 190 records were excluded. Specifically, two conference papers and four book chapters were first removed to ensure only rigorously peer-reviewed journal articles were taken into account for the analysis; 45 articles were excluded due to the fact that the technologies applied in the studies were not related to VR; 21 articles were excluded because they were either reviews or otherwise conceptual commentaries with no empirical information; 40 articles were removed because the participants of those studies were not formally diagnosed with ASD as formulated in our selection criteria or the participants had conditions other than ASD; in addition, three records were removed because the studies reported were neither related to VR nor ASD intervention. Next, the remaining 62 articles were further assessed for eligibility by full-text screening. In this step, the evaluators agreed to further remove 22 articles from the synthesis, because the VR technologies employed in those articles were desktop VR and could not be considered immersive VR; one article was excluded since the VR device described in the article was incomplete and thus unidentifiable; nine articles were removed due to the employment of the single-case design; two articles were excluded because the participants were not confirmed to have an ASD diagnosis; four articles were excluded due to evaluations that were either not aligned with the intervention objectives or no evaluation results were reported. In addition, three articles were removed due to the lack of interventions delivered to the participants. In the end, 21 articles were included in the following analysis (see

Table 1. Characteristics of the interventions covered by included publications.

Authors	Year	Theoretical Foundation	VR Device
Lorenzo et al. [34]	2013	Task-based learning	L-shaped screen
Maskey et al. [38]	2014	GE	Blue room
White et al. [46]	2016	CBT with mindfulness	Desktop, tablet, and HMD a
Lorenzo et al. [47]	2016	N/P	L-shaped screen
Cox et al. [48]	2017	Learn from simulation	Curved screen
Amaral et al. [49]	2018	N/P	HMD (Rift DK2)
Ip et al. [27]	2018	Situated learning	CAVE
Yuan & Ip [50]	2018	N/P	CAVE
Simões et al. [51]	2018	Gamification	HMD (Rift DK2)
Jacques et al. [52]	2018	N/P	CAVE
Politis et al. [53]	2019	N/P	N/P b
Ravindran et al. [54]	2019	N/P	Cardboard VR
Maskey et al. [39]	2019	CBT with GE	Blue room
Herrero & Lorenzo [55]	2020	Situated learning	HMD (Rift)
He et al. [56]	2021	N/P	HMD (VIVE)
De Luca et al. [57]	2021	N/P	Multiple screens
Baker-Ericzén et al. [58]	2021	Simulation and CBT	Customised driving simulator
Frolli et al. [59]	2022	N/P	Stereoscopic screen
Ip et al. [60]	2022	Experiential learning	HMD (Rift)
van Pelt et al. [61]	2022	Situated learning	HMD (Rift S)
Hocking et al. [62]	2022	N/P	HMD (VIVE Pro)

GE-graded exposure; CBT-cognitive behavioural therapy; HMD-head-mounted display; CAVE-cave automatic virtual environment; N/P-not presented in the article. ^a The authors stated that the intervention could be delivered in HMDs but did not explicitly report what devices were used for the experiment. ^b The devices used for the experiment were not explicitly reported in the article. By examining the software provider’s website, we found that the contents could be delivered across multiple devices, including HMDs. An email was sent to the corresponding author for clarification but no reply was received.

). We have conducted both descriptive analyses of the articles, as well as quantitative meta-analyses. Below, we present the results of the two analytical steps.

3.2. Descriptive Analysis

In the descriptive analysis of the articles, we present the vital topics that are discussed within the included studies, including the areas of intervention, the characteristics of the participants, the technology used, the theoretical foundation that the included studies build upon, as well as the evaluation approach and measures.

3.2.1. Areas of Intervention

The interventional focus of the included studies was on skills individuals diagnosed with ASD might lack, such as social skills, attention deficits, and communication skills. In detail, 11 articles focused on social and/or affective skill training, of which one article also covered executive functioning training [34] and one article also covered physical coordination [56]. It is observed that different articles may have different definitions of social skills. Defining social skills is considered to be challenging in either theoretical or practical terms because of the variety of behaviors subsumed under the rubric [63]. Besides social and/or affective skills, two articles focused on the intervention of phobias [38,39], three articles focused on daily life skill training, such as driving [48,58] and using public transportation [51], one article focused on social attention training [49], one article focused on joint attention training [54], one article focused on conversation skill training [53], one article focused on the training of motor skills [62], and one article focused on enhancing the participants’ cognitive functions [57].

3.2.2. Characteristics of Participants

A total number of 613 participants with ASD were included in the studies reported in the selected articles, covering an age range of 6 to 62 years. If the areas of intervention are specifically taken into consideration when analyzing the ages of participants, it is observed that studies focusing on social and/or affective skill training covered a narrower age range of 7 to 23 years old excluding participants of four studies where details of the participants’ ages were not reported [34,47,52,56]. Male participants represent a larger group than female participants. Specifically, 485 of the 584 participants (83.05%) were male, excluding 29 participants from [53,55,56] where participants’ gender was not reported in detail. This generally reflects the prevalence and characteristics of ASD; the estimated prevalence is around four to five times higher among males than among females [64]. However, this also introduces concerns over the gender imbalance and whether the approaches should be further evaluated among females with ASD.

3.2.3. Enabling Technology

Surround-screen projection is the most widely adopted enabling technology among the included studies. CAVE and one of its variants called the “Blue Room” [38,39] were used in five of the analyzed studies. L-shaped projection installations or display installations consisting of multiple screens with external sensors for tracking and data collection were used in three studies [34,47,57]. Ip et al. discussed the benefits of using CAVE for ASD intervention [27]; while being fully immersed in the VR experience, the surround-screen projection enables easier interaction between the participants and their trainers in the physical space, resulting in more effective and prompter facilitation if needed. However, this shortcoming of HMDs in the context of ASD intervention had been partially addressed by introducing program-controlled virtual agents with pre-scripted dialogues in the authors’ latest work [60]. Besides surround-screen projections, HMDs were also widely used in the included studies, and we found that HMDs were employed more in the latest studies published after 218. For example, Amaral et al. and Simões et al. both used the second generation of the Oculus Rift Development Kit HMDs [49,51], the devices that later evolved into the Oculus Rift and Oculus Rift S HMDs used in [60,61], while Ravindran et al. reported the use of Google Cardboard [54], a more primitive but much cheaper type of HMDs, in their study. However, there is generally a lack of comparison of different HMDs given the ASD population as the end users. Moreover, the acceptance of HMDs among the ASD population, especially children and young adolescents, was not thoroughly discussed in the nine studies that employed HMDs as the enabling technology.

3.2.4. Theoretical Foundations

The design of the VR scenarios, interventional protocols, and corresponding assessments all require a solid theoretical foundation. Surprisingly, 10 of the 21 analyzed articles did not explicitly report the theoretical foundations for the design of the reported immersive VR-enabled interventions. By examining the descriptions of the methods, especially the design of the VR scenarios, we identified that all of these 10 studies to some degree borrowed the idea of simulating real-life situations using VR, which generally matches the characteristics of the situated approach [65]. For the rest of the 11 analyzed articles in which the theoretical foundations for design were explicitly reported, one article reported the adoption of the combination of cognitive behavior therapy (CBT) [66] and mindfulness-acceptance based approach [46], one article reported the combination of CBT with the learning from simulation approach [58], one article reported the adoption of a combination of CBT and the graded exposure therapy approach [39]. One article reported the adoption of the graded exposure therapy approach only [38]; one article reported the adoption of gamification [51]; one article report the adoption of experiential learning theory [60]; and the remaining three articles all adopted one or more learning theories and/or approaches, such as task-based learning, adaptive learning, situated learning, and learning from simulation [27,34,48,55,61].

3.2.5. Evaluation Approach and Measures

Among the 21 included articles, 14 articles (66.67%) reported the use of standardized tests in their studies to measure the effectiveness of the immersive VR-enabled intervention. The use of computerized assessments and the assessment results were reported in nine (42.86%) included articles.

The use of computerized assessments seems to be straightforward since the enabling technology of the intervention, immersive VR systems, is computerized. For example, multiple sensors were used in [34,47] to capture interaction data during the interventions, allowing performance monitoring and analysis as the intervention was delivered. Similarly, the dialogue choices made by the participants when interacting with the virtual agents during interventions were captured for analysis in [56]. Baker-Ericzén et al. used a customized driving simulator to deliver the cognitive behavioral intervention for driving for teenagers and adults with ASD [58]. As the participants drove in the simulator, their performance was automatically recorded and quantified as the number of off-road crashes, vehicle collisions, pedestrian collisions, tickets, speeding, and centerline crossing by the computerized system. Hocking et al. used the Kinect sensor, a low-cost motion tracking device originally designed for playing motion-sensing games, to monitor the participants’ motor proficiency as the immersive VR-enabled exercise intervention progressed [62].

Physiological data seemed to be another good source for evaluating the effectiveness of immersive VR-enabled interventions. For example, Ravindran et al. used video recording for analyzing the participants’ joint attention during VR exposure [54]; the video clips were then analyzed manually by the researchers. On the other hand, eye tracking techniques were used in [49] for measuring joint attention automatically. Cox et al. also reported the use of eye-tracking techniques in their study; the driving performance was evaluated using the data captured directly in the virtual cockpit [48]. This approach greatly reduced the risk of assessment in this particular use case scenario. Simões et al. captured the electrodermal activity (EDA) for measuring the participants’ anxiety level during the VR exposure [51]. The use of electroencephalography-based brain–computer interfaces (BCI) was seen in [46] for measuring the attention level of the participants, but the results were not reported in the article. Electroencephalography data were also captured by De Luca et al. to study the potential effects of their immersive VR-enabled intervention on the participants’ brain functional connectivity [57].

3.3. Meta-Analysis on the Effectiveness

Among the 21 included articles, effect sizes and 95% confidence intervals (CI) were calculated for a total of 76 outcome measures reported in 15 articles, of which 20 outcome measures pertain to controlled trials reported in 5 articles [27,46,56,59,60]. Five articles were excluded from the analysis due to the reported data being either incomplete or insufficient for calculating the effects sizes [34,47,48,51,55]. Trial results were excluded if they were not directly for measuring the effectiveness of the intervention. Examples of such trial results include the measuring of system usability and cybersickness in [62]. In addition, the results of [50] were also excluded from the analysis since the raw data reported was highly similar to [27]; new data analysis tools were used in [50] so the focus of the article was solely on data analysis and presenting the data from a new perspective. Hence, to prevent misestimating the overall effectiveness, we decided to exclude these four articles from the analysis. Five articles reported the measurements of both the immediate and the retained interventional effects [38,39,49,57,61]. Since the relapse and retention of the interventional effects are beyond the scope of this work and the waiting time for the follow-up evaluations varies greatly from study to study, we only included the immediate effects in the meta-analysis. The methodological characteristics of the 15 included studies are summarized in

Table 2. Methodological characteristics of studies employed uncontrolled trials and included them in the meta-analysis.

Article	Characteristics of Participants		Interventional Area	Sample Size	Measures
	Age Mean (SD)	IQ Mean (SD)
[38]	11.2 (2.0)	N/P	Specific Phobia	4	SCAS; Confidence rating
[49]	22.17	102.53 (11.64)	Social Skills	15	HADS; BDI; POMS; JAAT; ATEC; VABS
[52]	N/P	N/P a	Social Skills	3	Social decoding; CPS; CPI; SISST
[53]	N/P	N/P	Social Skills	3	PESE; PSSE; GAD-7
[54]	13.5	N/P	Social Skills	12	JAA
[39]	N/P	N/P	Specific Phobia	8	PHQ-9; BAI; GAD-7
[57]	11 (3)	26 (1.7) b; 31 (1.7) c	Cognitive Functions	20	RCPM; RSPM; VMI; BAI-Y
[58]	20.53 (4.4)	N/P d	Life Skills-Driving	17	STAI; DCQ; DAS; Computerized DSE e
[61]	27.62 (11.50)	103.00 (14.39)	Social Skills	22	MASC; FEEST; TMT; SIAS; BFNE; SRS; EQ
[62]	14.0 (2.6)	N/P	Motor Skills	10	BOT-2; DCCS

N/P-Not presented in the article; SCAS-Spence Children’s Anxiety Scale (parent version); HADS-Hospital Anxiety Depression Scale; BDI-Beck Depression Inventory; POMS-Profile of Mood States; JAAT-Joint-attention Assessment Task; ATEC-Autism Treatment Evaluation Checklist; VABS-Vineland Adaptive Behavior Scale; CPS-Behaviors involved detecting social cues; CPI-Perception of social decoding and social skills; SISST-Social Interaction Self-Statement; PESE-Perceived Empathic Self-Efficacy Scale; PSSE-Perceived Social Self-Efficacy Scale; GAD-7-General Anxiety Disorder Questionnaire-7; JAA-Joint Attention Assessment; PHQ-9-Patient Health Questionnaire-9; BAI-Beck Anxiety Inventory; RCPM-Raven’s Colored Progressive Matrices; RSPM-Raven’s Standard Progressive Matrices; VMI-Developmental Test of Visual-Motor Integration; STAI-State-Trait Anxiety Inventory; DCQ-Driving Cognitions Questionnaire; DAS-Driving Attitude Scale; MASC-Movie for the Assessment of Social Cognition; FEEST-Facial Expressions of Emotions: Stimuli and Tests; TMT-Trail Making Test; SIAS-Social Interaction Anxiety Scale; BFNE-Brief Fear of Negative Evaluation Scale; SRS-Social Responsiveness Scale; EQ-Empathy Quotient; BOT-2-Bruininks–Oseretsky Test of Motor Proficiency Second Edition; DCCS-NIH Dimensional Change Card Sort Test. ^a The IQ of the participants with ASD was reported as “without intellectual disability” but detailed statistics were not reported. ^b As measured using RCPM. ^c As measured using RSPM. ^d IQ of the participants was reported to be “normal”, but detailed numbers were not presented in the article. ^e Computerized approaches were used to measure the participants’ driving simulator experience.

and

Table 3. Methodological characteristics of studies employed controlled trials and included them in the meta-analysis.

Article	Characteristics of Participants		Interventional Area	Sample Size	Measures
	Age Mean (SD)	IQ Mean (SD)
[46]	CTRL: 20.25 (1.71)	CTRL: 115.75 (22.28)	Social Skills	CTRL: 4	I-CLE; SACQ
	EXP: 20.75 (1.71)	EXP: 126.75 (5.62)		EXP: 4
[27]	8.86 (1.13)	95 (17.79)	Social and Affective Skills	CTRL: 36	PEP-3; Faces test; Eyes test;
				EXP: 36	ABAS-II
[56]	N/P a	N/P	Social Skills b	CTRL: 6	Computerized evaluation c
				EXP: 6
[59]	CTRL: 9.4 (0.49)	CTRL: 103.13 (2.04)	Affective Skills	CTRL: 30	SE; ESPE; ESSE
	EXP: 9.3 (0.63)	EXP: 103 (1.70)		EXP: 30
[60]	CTRL: 112 (19.5) d	CTRL: 92.5 (16.5)	Social and Affective Skills	CTRL: 59	PEP-3
	EXP: 101 (19.9) d	EXP: 93 (15.6)		EXP: 48

CTRL-Control group; EXP-Experiment group; N/P-Not presented in the article; I-CLE-College Living Experience Satisfaction Scale; SACQ-Student Adaptation to College Questionnaire; PEP-3-Psychoeducational Profile Third Edition; ABAS-II-Adaptive Behavior Assessment System Second Edition; PE-Recognition of primary or basic emotions; SE-Recognition of secondary emotions; ESPE-Emotions and Situations for Primary Emotions. ^a The age range was reported as six to eight years. ^b Hands-on ability and physical coordination were also covered but are not unique to ASD intervention. ^c The computerized evaluation assessed the participants’ social ability, hands-on ability, and physical coordination. ^d The age of the participants was reported in months.

.

The effect sizes and 95% CI were calculated according to [67,68]. Specifically, for trials that utilized only the within-subject design (i.e., uncontrolled trials) and trials that utilized the between-subject design but also reported the pre- and post-test results of the intervention group, the effective sizes d are calculated using the formula

$$d=\frac{{\overline{Y}}_{pre}-{\overline{Y}}_{post}}{S_{within}}$$

(1)

where ${\overline{Y}}_{pre}$ and ${\overline{Y}}_{post}$ are the sample means in the pre- and post-tests, respectively. The denominator $S_{within}$ is the standard deviation within groups, which can be calculated using the formula

$$S_{within}=\sqrt{\frac{S_{pre}^2+S_{post}^2}{2}}$$

(2)

where $S_{pre}$ and $S_{post}$ are the standard deviations in the post- and pre-tests, respectively. For trials that utilized the between-subject design with a control group (i.e., controlled trials), the effective sizes d are calculated using the formula

$$d=\frac{{\overline{Y}}_e-{\overline{Y}}_c}{S_{pooled}}$$

(3)

where ${\overline{Y}}_e$ and ${\overline{Y}}_c$ are the sample means of the post-tests from the experiment (i.e., intervention) group and control group, respectively. The denominator $S_{pooled}$ is the standard deviation pooled across two groups, which can be calculated using the formula

$$S_{pooled}=\sqrt{\frac{(n_e-1)S_e^2+(n_c-1)S_c^2}{n_e+n_c-2}}$$

(4)

where $S_e$ and $S_c$ are the standard deviations in the post-tests from the two groups, respectively, and $n_e$ and $n_c$ are the sample sizes of the two groups, respectively. The 95% CI of the effect sizes are calculated through

$$CI=d\pm 1.96S{E}_d$$

(5)

where $S{E}_d$ is the standard error of the effective size d. For trials that utilized only the within-subject design and trials that utilized the between-subject design but also reported the pre- and post-test results of the intervention group, $S{E}_d$ can be calculated by

$$S{E}_d=\sqrt{(\frac{1}{n}+\frac{d^2}{2n})2(1-r)}$$

(6)

where n is the sample size, and r is the correlation between pre- and post-tests. When r is not reported or cannot be directly calculated, r is estimated using the formula

$$S_{within}=\sqrt{\frac{S_{pre}^2+S_{post}^2}{2}}\approx \frac{S_{Diff}}{\sqrt{2(1-r)}}$$

(7)

where $S_{Diff}$ is the standard deviation of the difference. Thus, we can then calculate the estimated r as $r\approx 1-S_{Diff}^2/(S_{pre}^2+S_{post}^2)$. If the standard deviation of the difference $S_{Diff}$ is not reported in the original article, we calculated $S_{Diff}$ based on the reported t-test results using the formula

$$S_{Diff}=\frac{{\overline{Y}}_{pre}-{\overline{Y}}_{post}}{t}\sqrt{n}$$

(8)

where ${\overline{Y}}_{pre}$ and ${\overline{Y}}_{post}$ are the sample means in the pre- and post-tests, respectively, t is the t-score, and n is the sample size. For trials that utilized the between-subject design with a control group, $S{E}_d$ can be calculated using a more straightforward way by using the formula

$$S{E}_d=\sqrt{\frac{n_e+n_c}{n_e{n}_c}+\frac{d^2}{2(n_e+n_c)}}$$

(9)

where $n_e$ and $n_c$ are the sample sizes of the two groups, respectively. Note that for some of the measures such as the General Anxiety Disorder Questionnaire-7 (GAD-7) [69], the higher the scores, the worse the interventional effects. To make the results easier to understand and compare, the corresponding effect sizes and 95% confidence intervals have been reversely coded here. Hence, a positive d value always means a positive interventional effect in our analysis. Next, the combined effect sizes T were calculated through

$$T=\sum \frac{w_i{d}_i}{\sum w_i}$$

(10)

where $d_i$ is the Cohen’s d of the ith trial and $w_i$ is the weight of the ith trial, which is defined as

$$w_i=\frac{1}{v_i}=\frac{1}{S{E}_{di}^2}$$

(11)

where $S{E}_{di}$ is the standard error of the effective size d of the ith trial. Combined Cohen’s d for all uncontrolled trials is $0.125$ with a 95% CI of $[0.085,0.165]$. Combined Cohen’s d for all controlled trials is $0.285$ with a 95% CI of $[0.132,0.438]$. Figure 2 and Figure 3 show the forest plots of the meta-analysis results.

4. Discussion

4.1. Area of Intervention

With respect to research question (1), areas of intervention of the studies analyzed most often involve the improvement of social and/or emotional skills. This result is expected because deficits in social and emotional skills are the two most important diagnostic criteria of ASD [1] and the deficits can greatly hinder one’s participation in various social functions. The use of immersive VR can help create a safe, controllable, and embarrassment-free environment, in which social and emotional skills can be practiced repeatedly across a variety of social contexts. Participants of such interventions are expected to gain the ability to better judge a given context and respond to it in a socially appropriate way. However, due to the broadness of both terminologies, there seems to be a lack of a common definition of the term social skills. For example, Politis et al. investigated conversation skill training using VR [53]. Seemingly, social conversation requires one to understand the social context as well as the emotion of other parties involved in the conversation—the ability of social perception [70]. This also requires emotional skills, such as recognition of others’ facial expressions and comprehension of emotions and the implicit reciprocal, which were not discussed explicitly in the article. Further, the studies analyzed dealt with a variety of interventions including the improvement of daily life skills, social and joint attention training, and facing general or specific phobias. All these areas of intervention could, following the articles looked into, significantly improve the life quality of individuals with ASD and help them better engage in social life.

4.2. Affordances of Immersive VR

With regard to the research question (2), the affordances of immersive VR in ASD intervention design have been partially discussed, but have yet to be fully investigated. The affordance of immersive VR that was most commonly mentioned in the articles included in our analysis was simulation fidelity, allowing participants to practice the skills and generalize to real-life situations. Unfortunately, generalization is difficult to measure and there was little evidence reported in the included articles. Another affordance that was not covered to a large extent but could potentially change the landscape of ASD intervention is the ability to deliver intervention remotely, especially when considering the current Coronavirus disease pandemic that is shaping the methods of intervention delivery. Moreover, measures of effectiveness can greatly benefit from immersive VR-enabled approaches. The computerized nature of VR allows for the collection of an enormous amount of data that can be analyzed to measure the interventional effectiveness as the intervention happens. The collection of driving performance data in the simulated cockpit in [48] is a great example of utilizing this affordance. Lorenzo et al. used computer vision algorithms to automatically recognize the participants’ facial expressions during the intervention [47], which aimed to assess the participants’ performance by examining the number of inadequate behaviors and/or facial expressions performed given the social situation. Physiological data, such as EDA [51] and electroencephalogram [46], was also captured in some of the studies, providing another way of measuring the participants’ responses to the interventions. It is expected to see such approaches being further developed to provide performance-based personalized intervention programs, given the great flexibility and customizability of VR scenarios.

4.3. Theoretical Foundation

Regarding research question (3) and the theoretical foundations of the studies investigated, clinical approaches such as graduated exposure and CBT have been adapted for guiding the design of the immersive VR-enabled interventions reported in [38,39]. CBT has shown to be effective in treating anxiety disorder, phobia, and depression among individuals with ASD by identifying unrealistic feelings or unhelpful thoughts, and thus changing their way of thinking and behaving [71]. In the case of applying CBT in an immersive VR environment, the computer-generated content can help the participants to better relax and break down the challenges into thoughts, feelings, and actions under the guidance of a therapist. Besides CBT, there are other clinical approaches, which may require a method of relaxing or the presentation of certain stimuli, and that can benefit from the utilization of immersive VR. Although not explicitly mentioned in many other included studies, the design of the ASD interventions clearly borrowed the idea from conventional learning and education. For example, Ip et al. created various scenarios that a primary school student will encounter on a school day [27], in which challenging situations would be introduced as the intervention progresses. The participants need to interpret the social contexts and choose a strategy to respond to these situations. The VR scenarios, as well as the humanoid virtual agents, were programmed to respond to the participants’ actions, providing instant feedback, which prompts the participants to reflect and rethink a new strategy if their actions were not socially appropriate in the contexts. These approaches of using immersive VR to allow free practicing and strategic reflection are very similar to Kolb’s experiential learning model (KELM) [72]. However, this might also bring questions regarding such approaches when being applied to individuals with ASD. Specifically, the ASD population may need more facilitation and scaffolding in the process, especially when they face a challenging situation. Adopting KELM requires learners to self-initiate the rethinking and reflecting process, which might not happen among individuals with ASD if there is insufficient facilitation. Ip et al. tried to address this issue by emphasizing the role of the trainers in their study [27]. Cox et al. adopted a similar approach; the participants were always accompanied by a driving coach in the virtual cockpit [48]. However, in some cases, facilitation from trainers might be limited by the VR environment. If HMDs were used as the enabling device, the participant would be totally immersed in the VR scenario and isolated from the surrounding. Whether the facilitation from trainers can still be effective in this case is yet to be fully investigated. Ip et al. tried to address the issue by placing program-controlled virtual agents with pre-scripted dialogues at the locations where the participants might need facilitation [60]. With the recent advancements of large language models (LLMs) and the development of ChatGPT (https://openai.com/blog/chatgpt (accessed on 26 May 2023))-like services, the solution can be further refined and is worth further investigation.

4.4. The Effectiveness of Intervention

With regard to research question (4), we can summarize that the effectiveness of immersive VR-enabled interventions for individuals with ASD needs to be further examined and investigated in future studies. The limitations of the included articles are obvious; most of the studies reported involved a rather low number of participants and the experimental designs were not rigorous enough to draw a solid conclusion regarding the interventional effectiveness. Among the 15 articles included for the effect size calculation, five articles reported controlled trials [27,46,56,59,60]. The combined effect size ($d=0.285$, 95% CI $=[0.132,0.438]$) of all controlled trials suggested a small-to-medium combined effect of VR-enabled interventions over the controlled conditions. Because the interventional areas covered in all controlled trials had a focus on social and/or affective skills, it is safe to conclude that VR-enabled interventions are generally effective on social and/or affective skills. However, there is a lack of evidence to conclude whether the VR-enabled approaches are effective in other areas of ASD interventions. Furthermore, the long-term effectiveness and the ability to generalize seem to be the two major issues that need to be further addressed in future studies as well. Among all fifteen included articles, only three articles reported results from follow-up assessments regarding the lasting of the interventional effects [38,39,49]. As relapse is often observed in conventional ASD intervention, it would be interesting to see whether the interventional effects of an immersive VR-enabled program can last. Moreover, the technology’s ability to simulate real-life situations with great fidelity could potentially be extremely helpful for individuals with ASD, who are widely considered visual thinkers. Unfortunately, there is no in-depth research reported in the 15 included articles on the generalization of virtual scenarios to real-life situations.

4.5. Research Trends and Challenges

In examining the results presented, conclusions can be drawn concerning research question (5), identifying the latest research trends and challenges that need to be addressed in the area of immersive VR-enabled interventions for individuals with ASD.

With respect to research trends, it is evident that recent publications show a gradual increase in sample sizes and the implementation of more rigorous experimental designs. This trend suggests that the research area is maturing, particularly in the realm of social and affective skill training for individuals with ASD. Furthermore, the advancements in and accessibility of enabling hardware and software have facilitated the delivery of interventions in various settings and the development of interventional materials at reduced costs. For instance, Ip et al. initially developed a social and affective skill training program in a CAVE environment [27], but as HMDs became more accessible and ergonomic, later publications from the authors reported delivering interventional sessions in school settings [60]. In fact, a significant increase in the use of off-the-shelf HMDs was observed in publications after 2018, with nearly all VR content and experiences developed using readily available game engines such as Unity and Unreal Engine. It is anticipated that there will be continued growth in this research area as VR and Metaverse technologies become even more accessible.

Regarding the challenges that remain, the most pressing issue is the integration and adaptation of existing theories and practices to suit both VR and the specific interventional needs of individuals with ASD. As demonstrated in Table 1, 10 out of the 21 publications included in our systematic review did not clearly report the theoretical foundations of their studies. Furthermore, even among those publications that did report the utilization of existing theories, the manner in which these theories were adapted for VR-enabled ASD interventions remains unclear. In addition, the affordances of enabling technologies for the interventional needs of individuals with ASD have not been thoroughly investigated.

In conclusion, while the field of VR interventions for individuals with ASD is experiencing growth and maturation, there are still challenges to overcome. Further research must focus on the integration and adaptation of existing theories and practices for VR-enabled ASD interventions, as well as investigating the affordances of enabling technologies for the specific needs of individuals with ASD.

5. Limitations

This study has several limitations that should be acknowledged. First, despite using the Web of Science, an interdisciplinary and comprehensive database, in combination with PubMed, IEEEXplore, and ERIC, as well as manually searching previous systematic reviews to identify additional records, it is possible that some important articles were not included in the analyses. Second, the selection criteria included immersion as an objective capability of the enabling technologies [33], which led to the exclusion of some studies that employed less capable technologies. However, it is possible that these excluded studies still provided a high sense of presence, which is a key and desirable feature of the VR experience [18]. Unfortunately, few studies reported on the evaluation of a sense of presence, and to ensure objectivity and reproducibility, immersion had to be used as a selection criterion.

6. Conclusions

While our quantitative meta-analysis did not provide a clear indication of the effectiveness of immersive VR technology for ASD interventions other than social and/or affective skill training, there is promising growth and maturation in this research field. The systematic review results suggest a gradual increase in sample sizes and more rigorous experimental designs being employed in the latest publications. Furthermore, there is a greater focus on social and/or affective skill training for individuals with ASD. The advancements in and accessibility of enabling hardware and software have also facilitated the delivery of interventions in various settings and the development of interventional materials at reduced costs. Therefore, it is anticipated that there will be continued growth in this area of research and practice as VR and Metaverse technologies become even more accessible.

However, challenges remain in the integration and adaptation of existing theories and practices for VR-enabled ASD interventions, as well as investigating the affordances of enabling technologies for the specific needs of individuals with ASD. To address these challenges, future studies should focus on better utilizing the affordances of VR for the interventional needs of individuals with ASD, reporting the design of intervention procedures clearly, and implementing more solid theoretical foundations. With the increasing accessibility of off-the-shelf HMDs and other enabling hardware and software technologies, we expect to see more studies that utilize the enabling technologies in the near future. Ideally, these studies should connect the maturity level of technology to theoretical and methodological considerations, leading to valid research results.

In conclusion, while there is still much work to be conducted in the field of immersive VR-enabled interventions for individuals with ASD, the promising growth and maturation of the field suggest that more effective interventions can be developed for the specific needs of individuals with ASD by addressing the challenges that remain. Ultimately, the goal is to improve the quality of life for individuals with ASD by utilizing the enabling technologies of immersive VR to provide more accessible and effective interventions in various settings.