Virtual Reality Assessment of Attention Deficits in Traumatic Brain Injury: Effectiveness and Ecological Validity

Abstract

Early detection is crucial for addressing attention deficits commonly associated with Traumatic brain injury (TBI), informing effective rehabilitation planning and intervention. While traditional neuropsychological assessments have been conventionally used to evaluate attention deficits, their limited ecological validity presents notable challenges. This study explores the efficacy and validity of a novel virtual reality test, the Computerized Battery for the Assessment of Attention Disorders (CBAAD), among a cohort of TBI survivors (n = 20), in comparison to a healthy control group (n = 20). Participants, ranging in age from 21 to 62 years, were administered a comprehensive neuropsychological assessment, including the CBAAD and the Attention Related Cognitive Errors Scale. While variations in attentional performance were observed across age cohorts, the study found no statistically significant age-related effects within either group. The CBAAD demonstrated sensitivity to attentional dysfunction in the TBI group, establishing its value as a comprehensive test battery for assessing attention in this specific population. Regression analyses demonstrated the CBAAD’s effectiveness in predicting real-life attentional errors reported by TBI patients. In summary, the CBAAD demonstrates sensitivity to attentional dysfunction in TBI patients and the ability to predict real-world attentional errors, establishing its value as a comprehensive test battery for assessing attention in this specific population. Its implementation holds promise for enhancing the early identification of attentional impairments and facilitating tailored rehabilitation strategies for TBI patients.

1. Introduction

1.1. Motivation

Traumatic brain injury (TBI) refers to a clinical condition of any post-natal brain damage resulting from an external, violent factor. Its two main forms are blunt and penetrating traumas. TBI represents the greatest risk factor for death and long-term disability worldwide. The incidence of TBI is estimated to be 1.6 to 1.8 million people per year worldwide [1], with 16.3% of those cases requiring hospitalization, and 69 million people suffering worldwide [2,3]. TBI results mostly from motor vehicle accidents, falls, assaults, combat traumas, and sports injuries [4]. Attention and memory deficits are among the most commonly observed complications of TBI. Cognitive impairment following a TBI has a great impact on the quality of life of patients, affecting all aspects of their daily routines, such as the ability to work, live independently, maintain or develop social relationships, study, and interact, with a potentially adverse impact on the economy [5]. The research shows that a significant proportion, ranging from 26% to 45%, of individuals with a severe TBI experience challenges with long-term social integration [6]. The primary objective of rehabilitation is to facilitate patients’ effective reintegration into society, enabling them to function optimally in their daily lives [7]. Hence, the assessment of the range and severity of attentional difficulties and their potential influence on daily functioning shortly after a TBI assumes paramount importance. Such a timely evaluation holds the potential to aid clinicians in formulating suitable intervention strategies to address patients’ challenges effectively. Consequently, the selection of appropriate rehabilitation approaches considerably relies on a personalized assessment of the individual’s attentional capabilities [2].

According to the majority of contemporary neuropsychological theories, attention is a multidimensional system composed of several functions, components, or processes (for a review, see [8]). Most attention models (i.e., [9]) converge with those attentional processes that can be divided into several distinct components, which are related to specific attentional behaviors and strategies. This subdivision comprises selective attention (target selection among distractor stimuli, i.e., selecting the desired product among others in a supermarket), sustained attention (maintaining attentional focus during a prolonged time period in order to detect a rarely observed target stimulus, i.e., when driving a car over a long distance), divided attention (performing two or more tasks simultaneously in an effective manner, i.e., listening to the radio or talking on one’s mobile phone while driving, or listening to a phone call and watching television), and switching attention (alternating one’s attention between two or more tasks consecutively but not simultaneously, i.e., a secretary who has to write a letter and stops to pick up the phone or talk to a client and then continue with the letter) [9,10]. Patients with TBI are reported to exhibit deficits in most of these attention functions. For example, a well-known area of impairment regarding TBI is sustained attention, where deficits arise due to neuropathology or secondary factors, such as fatigue, a depressive mood, or sleep abnormalities. Traditionally, the assessment of sustained attention involves the use of Continuous Performance Tests (CPTs), in which examinees are presented with a repetitive task requiring vigilance and are asked to maintain their focus in order to respond as quickly as possible to certain targets. Patients with TBI present a poor performance for these tasks compared to the healthy controls, specifically, slower reaction times and increased attentional lapses and response variability [11,12,13]. Manifestations of selective attention dysfunctions are also demonstrated in some TBI patients, including a slow processing speed, immediate attentional overload, decreased efficiency in target processing, difficulty detecting targets among distractors, and an inability to shift response strategies [14]. TBI patients also exhibit deficits in more complex attention functions, such as divided and switching attention, which both rely on other higher-order prefrontal functions, including working memory and executive control. In fact, although some research has suggested that patients may perform normally on complex attention tasks that can be carried out relatively automatically, they have also shown difficulties in performing more demanding and complex tasks that are performed under high time-pressure situations and require executive control, or that include a substantial working memory load [15]. In conclusion, the investigation of attentional dysfunction in TBI patients holds significant importance, not only for elucidating the severity and extent of attention impairments, but also for comprehending the consequential limitations in other cognitive domains (e.g., memory and language) that rely on attention for optimal functioning.

To date, the cognitive assessment of TBI relies mainly on clinical interviews and neuropsychological tests in the form of school- or laboratory-based paper-and-pencil or computerized tasks. Most of the existing attention tests, however, have been found to have relatively low ecological validity [8,16] (ecological validity refers to the degree to which test requirements resemble the demands of common situations in daily living). An illustrative instance is the widely employed Sustained Attention to Response Task (SART, a two-dimensional CPT), which, despite its good psychometric characteristics, is rated as monotonous and detached from the demands of daily life by the participants [17,18]. Indeed, in recent literature reviews [19,20], a growing consensus emerges regarding the necessity for innovative computerized attention assessment tools that utilize virtual environments to mimic the real-life demands of daily living, especially for clinical populations experiencing attention problems, i.e., people with TBI and other attention disorders [21]. According to previous studies [21,22], appropriately and carefully designed computerized batteries can enrich clinical assessments with flexible diagnostic tools.

Virtual reality (VR) technology is a promising alternative to the traditional paper-and-pencil and laboratory two-dimensional (2D) computer-assisted testing techniques. The main reason is that such technology can provide a more stimulating and vivid environment with increased ecological validity, simulating real-life situations [23,24], where TBI patients predominantly experience their attention problems, which did not induce cybersickness. Moreover, multiple benefits from VR neuropsychological tests are reported in the literature, such as, for example, the increased precision of behavioral data for the identification of attentional deficits and proneness to distractibility than conventional tests [25].

1.2. VR Technology

In recent times, there has been a rapid proliferation of technological advancements, particularly in the realm of “immersive” technologies, like virtual reality (VR). These improvements can be attributed to the development of diverse computing systems and mobile devices, enabling users to engage with and manipulate visual elements in a highly satisfactory manner. VR enables each user to be immersed in a three-dimensional (3D) computer-simulated reality or a 360° human-recorded real-world environment, giving the impression of realness, spatial presence, and engagement in a first-person form. The factors affecting the users’ perceptions are the “sense of presence” when they have the perception of “being there” and have the chance to view and/or manipulate any changes made. Using handheld controllers or eye movements connected with VR applications, users can achieve a greater understanding of the surrounding digital environment and a subjective sense of immersion that they are enclosed. In addition to the use of different VR devices, an interactive and immersive 3D user interface (UI) is displayed in front of the user’s eyes to “cut” from view anything else from the real world and be able only to view everything that is exhibited by a computing device combined with control mechanisms. VR applications are currently generated by standalone computers and head-mounted display (HMD) mobile devices [26].

Desktop-based VR applications are projected by large computer monitors, which allow users to view and control the objects’ mechanisms utilizing a keyboard and mouse; thus, their eye contact can increase the mimicry of intransitive body movements. In HMD devices, such as Oculus Rift, VIVE, users have headsets with a small display optic in front of their eyes and use handheld controllers for a vivid and plausible interaction with the surrounding environment [27]. For the latter case, however, previous literature reviews [25,26] indicate that VR usage can negatively affect users’ exposure to simulation-based tasks and rehabilitation, such as cybersickness headaches, seizures, fatigue, disorientation, and dizziness.

1.3. The Use of VR for Neuropsychological Assessment

Implementing VR techniques represents another promising approach to both VR assessment and rehabilitation. Utilizing the intuitive interface and vivid graphics of immersive VR projects, along with the potential for experimenting with 3D digital objects, users can actively participate in hands-on practices and tasks within simulations. These simulations boast high representational fidelity and realism, closely approximating natural depth perception (Ref. [27]). Moreover, the limited measures of neuropsychological function, and the need to develop more ecologically valid measures due to the technological advances that are available at present, have turned researchers’ interests to the use of VR as it offers an alternative option to these assessment problems [21,28].

The widespread development and utilization of various VR applications has gained researchers’ interests for assessment purposes. For instance, Kourtesis [29] developed a VR-supported battery, the Virtual Reality Everyday Assessment Lab (VR-EAL), to measure users’ everyday cognitive functions. The same authors advocate that VR technology improves ecological validity in contrast to the conventional procedure (paper-and-pencil neuropsychological battery), resulting in a pleasant testing experience that does not induce cybersickness. In regard to the assessment of cognitive functioning in patients, neuropsychological tests use using VR technology to evaluate performance in specific domains. A meta-analysis conducted by Corti et al. [30] provided an overview of all VR cognitive assessment tools that were developed for patients with acquired brain dysfunctions in 2010–2019, a total of 31 tools being identified. Half of these tools assessed both executive functions (EFs) and prospective memory (PM), while the other half assessed visuospatial abilities. More tools were developed so as to assess PM compared to EF. Even the tools that were created for EF assessment focused on planning ability (i.e., organizing a business meeting, performing shopping tasks, preparing a dinner, and purchasing items in a supermarket) rather than specific attention aspects. For example, Hogan et al. [21] presented the VR Prospective Memory Shopping Task (VRPMST) as a sensitive task for the assessment of time- and event-based prospective memory impairments in stroke patients when compared to their healthy counterparts. The same study also demonstrated that VRPMST was characterized both by good ecological and convergent validity aspects to conventional prospective memory measures. Pieri et al. [31] created a virtual adaptation (named ObReco-360°) of the Picture Recognition subtest from a well-known traditional memory test, the Rivermead Behavioral Memory Test-III (RBMT-III). The authors concluded that the use of 360° immersive technology actually enhanced the ecological value of standard memory assessment tests. Lesk et al. [32] used a VR spatial memory recognition task to detect early signs of Alzheimer’s disease in patients with mild cognitive impairments.

On the contrary, in the field of the VR assessment of attention dysfunction, less is known about the utility and validity of the respective tools. Attention displays a core parameter of executive functioning, which is severely affected after a TBI [33] and constitutes the main underlying deficit putting them at high risk when high concentration and speeded reactions are required, specifically for everyday tasks [34]. Additionally, Park [35] assessed the driving simulation performance and explored any possible relationship with the performance that TBI patients displayed on neuropsychological tests of attention. The VR-supported simulation driving assisted individuals with TBI to better coordinate and monitor their actions, whereas users’ previous experience and age were reported as the most influential factors influencing their driving errors. Furthermore, Bock et al. [36] assessed the switching attention outcomes for driving and street-crossing VR tests, but only in healthy participants. Finally, Voinescu et al. [37] assessed sustained attention by using a VR adaption of the classical Continuous Performance Test (CPT) paradigm, the Nesplora Aquarium, which showed adequate convergent validity to traditional CPT measures of attention and inhibition and predictive validity of depression and anxiety symptoms in healthy participants, too.

To summarize, the use of VR is rapidly gaining ground in the field of neuropsychological assessment. A possible explanation for using VR rehabilitation treatments for participants of different genders and socio-cognitive backgrounds is referred to as follows: (a) the first-person perspective of users with the illusion of possible movements that are projected in simulated realistic contexts [35,37] and (b) the ergonomic/naturalistic simulated interactions among users and digital elements that approximate real-life physical interactions, which can mitigate users’ movement omission errors in real time [38,39]. With that in mind, 3D visualization modality has not been found to significantly impact users’ cognitive overload, whereas, from a technological perspective, all studies utilize HMD devices, and thus, little is known regarding the effects of desktop VR systems on non-patients and patients with TBI.

1.4. Problem Statement

A TBI poses a significant public health concern, often leading to attentional impairments that can profoundly impact daily functioning. Addressing these impairments necessitates accurate and ecologically valid assessment tools for the effectiveness of implemented rehabilitation strategies. Ecological validity, signifying how closely test requirements align with the demands of everyday situations, is crucial for ensuring the relevance of assessments in real-life scenarios. In this study, ecological validity specifically pertains to how well the CBAAD reproduces the challenges of daily life, ensuring the assessment faithfully captures the obstacles individuals encounter in their natural environments. Establishing this validity is essential for a more meaningful interpretation of the assessment outcomes and, consequently, for guiding rehabilitation planning and interventions tailored to the specific needs of TBI patients. Consequently, there is an urgent requirement for assessment tools that not only accurately measure attentional functions, but also exhibit a high degree of ecological validity.

Despite the advancements in VR technology, the realistic simulation of specific core cognitive domains, such as attention, for patients with TBI remains limited. While an extensive body of relevant literature [1,35,40] highlights the significance of early neuropsychological rehabilitation targeting attention functions in TBI patients, incorporating ecologically valid VR neuropsychological tasks into assessment procedures has presented little progress. This deficiency is critical as it hampers the timely detection and prediction of attentional dysfunction in real-life scenarios, as well as the selection of appropriate rehabilitation approaches. Therefore, the present study aims to achieve two objectives. This study aims to investigate potential dysfunction in four attentional domains among a cohort of TBI survivors, comparing them to a control group of healthy individuals. This exploration utilizes a recently developed VR assessment tool known as the Computerized Battery for the Assessment of Attention Disorders (CBAAD) [41]. Subsequently, we investigate whether the CBAAD can effectively predict patients’ everyday attention errors, as assessed by the Attention Related to Cognitive Errors Scale (ARCES) [42]. Therefore, the present study is motivated by the recognition that traditional neuropsychological assessments, while conventionally used, can fall short in capturing the intricacies of attentional deficits in real-life situations.

2. Materials and Methods

2.1. Research Context and Participants

The research employed a cross-sectional comparative study design to assess the efficacy and validity of a computerized battery for the assessment of attention deficits (CBAAD) in individuals with TBI compared to a healthy control group. This design allowed for the examination of the differences in attentional performance between the two groups at a single point in time [37,41].

The sample size was determined based on a power analysis to detect significant differences in the attentional performance between the TBI and control groups. The calculation considered the expected effect size, significance level, and statistical power to ensure the study’s robustness. The total sample comprised 40 adult participants with a mean age of 42.30 years (SD = 11.79, range: 21–62 years old), with an average of 14.20 years of education (SD = 4.30). Twenty of them were patients with closed-head TBI (10 of whom were women) from the Papageorgiou General Hospital of Thessaloniki, and the other 20 were healthy controls (10 of whom were women). The TBI group included individuals meeting the following conditions: the absence of severe comorbid neurological or psychiatric conditions (such as dementia or schizophrenia) not directly stemming from the TBI, no history of substance abuse or dependence within the last six months, the ability to engage in virtual reality tasks without hindrance from physical limitations or severe cognitive impairment, and a lack of current participation in any other clinical trials or rehabilitation programs.

All patients received appropriate medical care following the accident, with the time since injury ranging from 3 to 29 years and hospitalization durations ranging from 6 days to 1 year. Sixteen participants attributed their injuries to vehicle/motor accidents, while the remaining four experienced severe falls from great heights, as indicated in

Table 1. Demographic characteristics of the participants.

	TBI		Control
	(n = 20)		(n = 20)
Variable	M	SD	M	SD	t	df	p
Age (years)	42.20	1.90	42.40	14.90	−0.053	38	0.958
Education (years)	13.50	4.47	14.90	4.13	−1.03	38	0.310
Time since injury (years)	11.50	7.25	-	-
Days of hospitalization (days)	99.60	95.77

M = Mean, SD = Standard Deviation, t = t-value, df = degrees of freedom, and p = p-value.

. Healthy participants were excluded from the study if they had a history of neurological or psychiatric disorders, closed head injuries, or any other medical conditions potentially impacting the central nervous system. To ensure comparability, the patients and controls were selected and matched based on age, sex, and years of education. Notably, the two groups did not exhibit any significant differences in demographic variables, as presented in Table 1.

Ethical considerations were strictly adhered to, and all participants were provided with comprehensive information regarding the study’s objectives. Prior to participation, each individual signed an informed consent form, acknowledging their understanding of the study’s ethical guidelines and content. According to the federal regulations for data protection, ethical review and approval matters were of utmost importance, even though the primary consideration was administrative convenience for the training sessions, where no personal data were collected. Safeguarding the well-being of participants was also taken seriously due to the ethical complexities of using them as human subjects in the research [43].

All participants were required to provide informed consent in a hard copy, which included information on (a) the potential side effects that could arise during this study, (b) data collection and usage in accordance with the Data Protection Regulation (GDPR) guidelines, and (c) the right to withdraw from the study at any time. Both groups received comprehensive details about the study’s purpose. All experimental protocols included in this study received approval from the Ethics Committee of the Third Health Region of Macedonia (protocol number: Δ3β/5225), which operates under the Greek Ministry of Health.

To achieve the study’s objectives, several ethical considerations were followed, as outlined by White [43]. First, the personal information of the participants was not gathered, and they had the freedom to withdraw at any point. Additionally, there were no rewards or penalties associated with participation or withdrawal, and participant identities remained anonymous in all the collected data. Second, participation in the study was voluntary, and written consent was obtained from all the participants. Each participant was identified through a unique code to maintain data confidentiality. Third, the study did not pose significant risks or offer notable advantages to those taking part.

2.2. Apparatus

We used one desktop computer and a large screen alongside a keyboard and a mouse for each participant. The four VR applications were generated in a 27-inch computer monitor with a resolution of 1080 by 1920 pixels, with a refresh rate of 60 Hz and a field of view of 110. Integrated headphones provided real-time 3D audio effects. All VR applications were developed in Unity 3D and the code was written in C#. The data of VR events were stored in a local SQL database. The operating system of the desktop computer was Windows 10.

2.3. Materials

2.3.1. Computerized Battery for the Assessment of Attention Disorders

The Computerized Battery for the Assessment of Attention Disorders (CBAAD) [41], comprised four main attentional functions (selective, sustained, divided, and switching attention) using daily life scenarios (i.e., visiting a supermarket, watching sports, and driving a car). CBAAD consists of two processes. The first process (Player Executables) is compiled by Unity 3D, a well-known game engine, which provides an integrated development environment (IDE) for designing and developing interactive video games. The Unity platform was utilized for the user’s test behavior based on specific script components attached to game elements. Script components emit events that are captured and collected as an event-stream and persist as .xml. The second process (Battery Catalog) is a GUI (graphical user interface) based on the Windows Presentation Framework (WPF) that imports the .xml event stream and saves it in a proprietary database. Events collected in the database are then interpreted into observations that build an abstract report of measures. In the following step, the abstract report is transformed into various application formats (.sav, .xlsx, .docx, .csv). The script components are attached to game elements that are projected in each scene, working together to resemble the workflow for the test. All digital elements were coded in the C# programming language, which was compiled with the Mono-framework to the .NET platform (ver. 3.5).

To record a new test session, the examiner must open the Battery Catalog application (platform) of the CBAAD and register him/herself on the examiner form and the examinee on the participant form. The participant form requires entering the following field values: name, birth date, gender, years of education, profession, hand preference, and, additionally, in the case of patients (diagnosis, medication, and other psychiatric or neurological disorders). When the form is validated, the examiner selects and launches one of the four CBAAD subtests. The instructions for each subtest are presented by an audio–visual instruction guide (Unity 3D scene on its own) displaying the original scene. Prior to commencing with the test session, the participants were given a training session to acquaint themselves with the demands of the specific subtest. The training session can be repeated as many times as needed in order to ensure that the workflow is understood. After a subtest is completed, performance indices are calculated with respect to the performance accuracy (correct responses, errors, omissions, and false alarm errors) and response speed (mean, median, and standard deviation of the response time, RT).

As it was designed for patients with attention problems, instructions and reaction demands were kept as simple as possible (either a mouse scroll and click or a simple one-button press for each task) to purely assess attention and not comprehension or the ability to produce multiple and complex motor responses on the keyboard, domains that could be additionally impaired in some patients. The latter raised the validity of CBAAD and its compliance with well-established clinical assessment procedures used for neuropsychological assessment. Such tools, despite being relatively easy in their administration to adjust to the patients’ level, are very strict, similar to CBAAD for the definition of the desired behavior or response, the scoring rules, and the precise calculation of multiple performance indices. Respective findings were reported in a psychometric study where CBAAD tests were highly correlated (Pearson’s r > 0.80) to a well-known and widely used paper–pencil attention test: the Trail-Making Test (TMT) [44]. The authors concluded that the CBAAD could serve as a virtual analog of traditional paper–pencil attention testing or even replace it. Each of the four CBAAD subtests is described below.

• The Supermarket-selective attention subtest was performed in a 3D scene resembling a real visit to a supermarket. The examinee moved (either in a forward or backward direction) through a corridor by scrolling the mouse wheel. The task required the examinee to scan the scene visually and select (mouse click) all objects/products that match a predefined target (i.e., all the kettles with a certain pattern) as quickly as possible. The subtest consisted of six levels with different objects for each level (plates, toothbrushes, kettles, briefcases, and televisions; see Figure 1a–e) and the examinee proceeded to the next level when he/she thought that he/she had found all the level’s objects. The number of all selectable objects was 351, while the targets were 83. The total duration of the entire subtest depended on the examinee’s reaction speed and usually did not take more than 10 min to complete. Performance indices were calculated separately for each level and the total performance for all levels.

2.

The Car driving-sustained attention subtest involves the examinee participating in a monotonous driving task, where they follow another car (lead car) on a highway (Figure 2). The lead car was automated using a script component, relieving the examinee of navigation responsibilities. Notably, this task extensively employed the physics engine of Unity, integrated with CBAAD script components that applied forces to both the leader and follower cars, effectively controlling their movements and wheel colliders. This incorporation of a physics engine ensures a more realistic driving experience. Throughout the task, the lead car driver intermittently applies brakes at certain positions on the highway, signaled by the activation of red brake lights. The examinee’s objective was to promptly press the brake button (Spacebar) upon observing the brake lights, thereby avoiding a collision with the lead car. The response time, indicating the speed of reaction, was recorded when the brake button was pressed within a minimum time frame of 5 s. Failure to press the brake button within this interval resulted in a recorded error, signifying a crash. Irrespective of whether a crash occurred or not, the driving task continued until the subsequent breakpoint. Any inadvertent pressing of the brake button outside the brake phase was categorized as a false alarm error. The entire duration of this task was set at 3 min.

3.

The Car driving while listening to music-divided attention subtest is a dual task consisting of a visual component (which is based on the Car driving subtest) and an audio component, which resembles the everyday habit of listening to music. More specifically, the latter uses the Unity timeline component (see Figure 3) and involves listening to different song clips. The composition, orchestration, and production of the song clips playing in the subtest were delegated to a music professional. In between songs, he/she hears the tuner searching for the next song. The examinee was asked to press a “buzzer key” as quickly as possible (response time) only for songs including a female voice, which represented the correct response to the task. For songs with a male voice or instrumental pieces, the examinee had to withhold his/her response by avoiding pressing the button; in case he/she hit the button, this event was recorded as a false alarm error. Moreover, if he/she forgot to press the button to indicate hearing a female-voiced song, an omission error was recorded. The second (audio) task was independent of the driving task since the need for a response to a female-voiced song could occur either during driving or a brake phase. The whole test duration was approximately 10 min.

4.

The TV sports watching–switching attention subtest is a task designed to measure one’s ability to alternate his/her attention between two same modality tasks (visual). The 3D environment here was very similar to a real café and the examinee sat in front of a monitor panel watching track and field sports (Figure 4a,b). In particular, he/she watched athletes perform high and long jumps. The player showed clips from two video sources (high and long jumps) without splitting the video sources via the VideoClipInfo-type, which were references and denoted the start and end of the clips, and also the start and end of the jump. Many VideoClipInfo objects were used to form a sequence of clips, which represented the clip sequence for the examinee responding to the jumps. For each type of jump, the examinee had to press as quickly as possible (response time) the correct jump-type button (there were two buttons), as the athlete was airborne (correct response). If either of the buttons was pressed when the athlete was grounded, the response was recorded as an error. If the wrong jump-type button is pressed for the jump (key error), the response was recorded as a false alarm error. In case someone forgot to press any key as the athlete was airborne, this constituted an omission error. The whole test duration lasted no more than 5 min.

2.3.2. Attention Related Cognitive Errors Scale

The Attention Related Cognitive Errors Scale (ARCES) was created by Cheyne et al. [42] to measure the frequency of cognitive errors due to attention slips. It is a 12-item questionnaire that reflects cognitive errors in daily life. This is a self-report scale employing a 5-point Likert scale, ranging from 0 (never) to 4 (very often), to evaluate the frequency of cognitive errors occurring over the past 6 months.

A high score for the ARCES reveals a larger number of errors. The scale was adjusted for the Greek population by Malegiannaki and Metallidou [17], and the structure validity measurement revealed the existence of two factors (one main factor and a second one with a minor contribution): the errors of attention distraction factor (Cronbach’s a = 0.82) and the errors of automated action factor (Cronbach’s a = 0.78). ARCES’s internal reliability for all the items was also adequate (Cronbach’s a = 0.87).

2.4. Procedure

To conduct this study, the authors received institutional approval, and the treatment followed the guidelines for participants in compliance with the Declaration of Helsinki.

Upon enrolling patients and healthy controls in the study, a member of the research staff reviewed the records of participating health facilities to assess their eligibility for inclusion in the project. Initial contact was made to determine the willingness of selected patients and the healthy controls to participate. Subsequently, an interview was conducted to provide a brief introduction and explanation of the research study’s purpose as part of the informed consent process. Participants were informed about the potential implications of the research on practice and their right to withdraw from the study at any point voluntarily. The examination procedure was scheduled to begin and end at an agreed-upon time, with a total duration not exceeding two hours, and breaks were provided if needed to prevent fatigue.

During the interview, necessary data for each participant were collected and the utmost care was taken to ensure the confidentiality of all the results and information provided. Informed consent forms were signed by all the participants, and they completed a demographic questionnaire covering age, gender, education, profession, and health information. Subsequently, all the participants underwent examinations with the CBAAD. Tests were administered only when the participants demonstrated an error-free performance during the training trials, ensuring their comprehension of the instructions and a familiarity with the subtest requirements for the optimal baseline performance. TBI patients also completed the ARCES with the assistance of the examiner. The interviews and examination procedures were conducted in person by the second author of the study, individually, in a suitable room at each health facility for patients and at a convenient location for healthy controls. Particular attention focused on ensuring that all the participants had a private and quiet environment thereby guaranteeing an uninterrupted experience. Standardized conditions were maintained across all participants, including uniform levels of lighting, noise, and temperature. Assessments were scheduled in the morning hours to mitigate the influence of fatigue on the participants’ performances. None of the participants were taking medications that could affect their cognition. All participants were provided with identical equipment and recorded and standardized oral instructions before each examination task, ensuring a uniform understanding of the study requirements and minimizing any experimenter effects. The material was arranged at a consistent distance of 40 inches from the examinee and the equipment (the laptop) of the attention battery test to reduce eyestrain and promote an optimal performance [45].

2.5. Data Analysis

The data from the current study were subjected to analyses using the Statistical Package for the Social Sciences (SPSS ver. 29). Cases that were deemed outliers or did not complete the entire battery were excluded from the analyses. Additionally, our data met the normality requirements. The descriptive statistics of the demographic variables are presented in the section concerning the participants. To assess the CBAAD’s ability to detect attentional dysfunctions in TBI patients, we compared the patient group’s performances on the four subtests with those of their matched controls using two-tailed independent sample t-tests. Pearson’s r correlational analyses were employed to examine the relationship between patients’ performances on the CBAAD subsets and their factor scores in the ARCES questionnaire. Finally, to investigate whether the participants’ performances on the CBAAD could predict their reports of everyday attention failures, stepwise linear regression analyses were used with independent variables for the performance scores on the CBAAD and dependent variables for the factor scores on the ARCES questionnaire, respectively.

3. Results

The results presented in

Table 2. Comparison of mean performance and standard deviation between TBI patients and controls on the CBAAD subtests.

	TBI		Control
	(n = 20)		(n = 20)
CBAAD Subtests	M	SD	M	SD	t	df	p
Supermarket-selective attention subtest
Corrects	67.90	19.11	79.30	9.10	−2.41	27.19	0.023
Errors	8.00	8.35	3.45	7.58	1.81	38.00	0.079
Omissions	9.05	12.15	2.25	3.04	2.41	21.37	0.024
Total duration (sec)	503.51	334.66	267.31	99.58	3.03	22.34	0.006
Mean level duration (sec)	83.92	55.78	44.55	16.60	3.03	22.34	0.006
Median level duration (sec)	78.14	52.58	43.02	16.02	2.86	22.50	0.009
SD level duration (sec)	28.10	32.87	11.08	7.09	2.26	20.76	0.034
Car driving-sustained attention subtest
Corrects	4.85	1.23	5.70	1.34	−2.09	38.00	0.430
Errors	0.65	0.67	0.00	0.000	4.33	19.00	0.000
Omissions	0.50	0.61	0.00	0.000	3.68	19.00	0.002
Mean RT (msec)	855.24	300.37	602.27	182.70	3.216	31.37	0.003
Median RT (msec)	776.35	319.04	534.70	153.81	3.051	27.38	0.005
SD RT (msec)	0.27	0.20	0.21	0.15	1.128	38.00	0.266
Car driving while listening to music-divided attention subtest
Audio corrects	6.50	1.638	8.55	5.10	−5.34	38.00	0.000
Audio errors	1.00	1.214	0.30	0.47	2.41	24.58	0.024
Audio omissions	1.40	1.00	0.15	0.37	5.47	24.02	0.000
Audio mean RT (msec)	3685.90	2323.45	1747.65	1028.99	3.41	26.18	0.002
Audio median RT (msec)	2615.58	2237.11	1324.70	723.45	2.46	22.18	0.002
Audio SD RT (msec)	3.23	3.38	1.27	1.37	2.40	25.07	0.024
Visual corrects	11.65	2.72	14.25	1.02	−4.00	24.24	0.001
Visual errors	0.95	1.00	0.35	0.59	2.32	30.73	0.027
Visual omissions	2.35	2.52	0.35	0.93	3.33	24.12	0.003
Visual mean RT (msec)	790.40	211.53	588.88	123.46	3.68	30.60	0.001
Visual median RT (msec)	746.53	228.79	568.05	126.31	3.05	29.60	0.005
Visual SD RT (msec)	0.21	0.10	0.13	0.06	3.07	38.00	0.004
Sports watching-switching attention subtest
Corrects	25.15	3.63	29.20	0.89	−4.84	21.30	0.000
Errors	2.80	2.35	0.55	0.76	4.07	22.91	0.000
False alarms	1.35	1.76	0.25	0.55	2.67	22.70	0.014
Omissions	0.75	1.59	0.00	0.00	2.12	19.00	0.048
Mean RT (msec)	1686.46	2339.39	1668.35	2297.61	0.03	38.00	0.980
Median RT (msec)	1592.75	2337.90	1071.43	18.79	1.00	19.00	0.331
SD RT (msec)	0.28	0.02	0.27	0.01	2.06	21.52	0.052

Note: SD = standard deviation, RT = reaction time, sec = seconds, msec = milliseconds.

reveal that patients score significantly lower than the healthy control group across all tasks (p < 0.05). Time indices (mean/median/SD of the reaction time or completion duration) proved to be as equally effective as the accuracy indices (errors, corrects, omission, and false alarms) in distinguishing between the two groups, with the exception of the switching attention subtest. Both groups exhibited a higher frequency of errors in the selective and switching attention tasks, where the reaction time did not differentiate the two groups. Notably, in the healthy participant, a ceiling effect was observed in the switching attention subtest, as they displayed no errors of omission, but only errors of commission.

Also, it is worth mentioning that the reaction time increases for older adults, suggesting that response speed worsens with age. From the data given below (Table 2), it can be assumed that the Car driving subtest is an easy task, as ceiling effects can be observed for all age groups; this was expected, as the test was developed for clinical purposes.

Pearson’s r correlations among the scores for the ARCES and CBAAD subtests revealed that specific subtest indices (both accuracy and time-based) were significantly correlated with the ARCES factors (

Table 3. Pearson’s r correlations of patients’ scores among the ARCES and CBAAD subtests.

CBAAD

Supermarket-selective attention subtest

Car driving-sustained attention subtest

ARCES

COR

ERR

OM

MTD

MdLD

SDLD

COR

ERR

OM

MRT

MdRT

SDRT

Errors of attention distraction

−0.570 *

-

0.584 *

-

0.448 *

-

-

-

-

0.477 *

0.464 *

-

Errors of automated action

−0.466 *

0.487 *

-

0.521 *

0.457 *

0.477 *

-

-

-

0.446 *

-

-

Total errors

−0.625 *

0.557 *

0.587 **

0.584 *

0.557 *

-

-

-

-

0.564 *

0.548 *

-

CBAAD

Car driving while listening to music-divided attention subtest

Audio

Visual

ARCES

COR

ERR

OM

MRT

MdRT

SDRT

COR

ERR

OM

MRT

MdRT

SDRT

Errors of attention distraction

−0.520 *

-

0.447 *

0.565 *

0.556 *

0.456 *

−0.565 *

-

0.657 *

0.621 *

0.606 *

-

Errors of automated action

-

-

-

-

-

-

-

-

-

-

-

-

Total errors

-

-

-

0.580 *

0.458 *

-

−0.502 *

-

0.552 *

0.593 *

0.551 *

-

Note: * p < 0.05. ** p < 0.01. COR = correct response, ERR = error, OM = omission, MTD = mean total duration (sec), MdLD = median-level duration (sec), SDLD = standard deviation of level duration (sec), MRT = mean response time (msec), MdRT = median response time (msec), SDRT = standard deviation of response time (msec).

). Among all the subtests, the performance parameters of the selective and divided attention subtests, and secondly the mean RT of the sustained attention subtest, were highly correlated with the ARCES variables. On the switching attention subtest, only false alarm errors were correlated with errors of distraction (r = 0.501, p < 0.05).

The high correlations that emerged from the CBAAD and ARCES tests were further explored with respect to the predictive validity of the CBAAD subtests. The results of the stepwise regression analyses are displayed in

Table 4. Stepwise linear regression analyses displaying the validity of CBAAD subtests for predicting patients’ attention-related errors on the two ARCES score factors.

ARCES Scores (Dependent Variables)	CBAAD Subtests (Independent Variables)	Beta	t	p	R2	F	df	p
ARCES errors of attention distraction	Supermarket-selective attention subtest
	Omissions	0.584	3.053	0.007	0.341	9.322	1.18	0.007
	Car driving-sustained attention subtest
	Mean RT (msec)	0.477	2.301	0.034	0.227	5.293	1.18	0.034
	Car driving while listening to music-divided attention subtest
	Visual correct responses	−0.639	−4.873	0.000	0.736	23.724	2.17	0.000
	Visual mean RT (msec)	0.408	3.113	0.006
	Sports watching-switching attention subtest
	False alarms	0.501	2.459	0.024	0.251	6.048	1.18	0.024
ARCES errors of automated action	Supermarket:-selective attention subtest
	Mean total duration (sec)	0.521	2.586	0.019	0.271	6.690	1.18	0.019
	Car driving-sustained attention subtest
	Mean RT (msec)	0.446	2.113	0.049	0.199	4.466	1.18	0.049

. Specific CBAAD task indices from each subtest predicted errors of attention distraction in the ARCES. However, regarding the second factor of the ARCES, the automated errors of action, only the mean total duration of the selective attention subtest, and the mean RT of the divided attention subtest predicted patients’ reports.

4. Discussion

The findings of the present study offer promising insights into the efficacy and ecological validity of a novel VR assessment tool, the CBAAD, for TBI patients. Primarily, the CBAAD performance discriminated TBI patients from the matched healthy control group. Moreover, particular scores obtained from the test served as predictors of patients’ everyday attention errors. These results have important implications for the diagnosis and treatment of TBI survivors, which are discussed below.

4.1. The CBAAD’s Ability to Differentiate TBI Patients from Healthy Controls

Regarding the first point, TBI performance on the CBAAD aligned with the previous research indicating a pattern of multiple weaknesses in attention and executive functions [46,47]. Given the involvement of the frontal lobe regarding attention and the damage to the frontal subcortical circuits, it was unsurprising that our TBI group exhibited slow reaction times and poor accuracy on the CBAAD [12]. Their impaired performances across all tasks suggested a comprehensive dysfunction of the attentional system [48]. Commonly available rehabilitation programs often emphasize the remediation of sustained attention, assuming it to be the most affected function in TBI patients [49]. However, the findings of the present study highlight that TBI patients experience a broader range of attention impairments, necessitating careful consideration when devising rehabilitation interventions to cater to their actual needs.

Beyond delivering a thorough depiction of the patient’s impairments via quantitative performance recordings, the attention assessment utilizing the CBAAD presents a distinct advantage compared to alternative neuropsychological tools. The tool incorporates qualitative and descriptive aspects of the patient’s daily life behavior, specifically focusing on evaluating particular elements of their car driving behavior or their effectiveness in buying products in a supermarket. This qualitative insight into the patient’s behavior allows clinicians to gain a better understanding of their real-life functioning and adaptability. By assessing these practical aspects, the clinician can take pre-emptive measures to safeguard the patient from engaging in activities or situations for which they may not be adequately prepared or that may pose potential risks [50]. For instance, in the case of a patient who demonstrates delayed reaction times or frequently neglects pressing the car brake, the clinician can proactively advise the patient to delay their return to driving [51]. The incorporation of such qualitative information complements the quantitative data, enriching rehabilitation planning and intervention for TBI patients with a more comprehensive and personalized approach. Moreover, Tamietto et al. [52] suggested that the utilization of post-injury driving fitness measures, which possess significant ecological and external validity, would be promising for evaluating real-world driving abilities.

From a psychometric standpoint, the CBAAD proved highly sensitive in detecting attention dysfunction in TBI patients. Both time- and accuracy-based indices were equally valuable in revealing patients’ attention impairments [53]. In contrast, previous clinical studies using traditional or two-dimensional computer-assisted tools have not presented encouraging results regarding the diagnostic usefulness of accuracy scores when compared to speed-based performance indices [54]. The accuracy of detecting static visual stimuli in conventional assessments, while essential for monitoring performances during neuropsychological assessments, was considered less informative than the time-based scores [55]. This phenomenon can be attributed to the prevalence of ceiling effects on accuracy-based scores, where a significant number of participants achieve the highest score, limiting the test’s ability to accurately measure an individual’s true baseline performance [56]. Ceiling effects are known to mislead neuropsychologists in their diagnostic interpretations of individuals with brain damage [56]. In contrast, the use of virtual reality allows participants to navigate through complex and vivid environments, resembling real-life scenarios and physical movements. Accordingly, the performance variability in VR accuracy scores is more likely to occur due to the perceptual complexity of 3D real-life stimuli compared to 2D stimuli, which places higher demands on the cognitive system, especially when motion is involved. The results of the present study support this assumption. For instance, in the CBAAD switching attention subtest, the participants’ reaction to moving targets (athletes) within a short and predefined time frame (milliseconds) rendered the number of errors even more informative than the speed indices [54]. Therefore, VR assessment represents a new era of neuropsychology, enabling studies to identify performance indices that are clinically valuable, especially for specific populations.

4.2. The CBAAD’s Predictive Ability concerning Errors in Daily Life Activities

Regarding the second objective of our study, we observed a strong correlation between patients’ performances on the CBAAD subtests and the ARCES factor scores. Specifically, omission errors in the Supermarket selective attention subtest served as predictors for the patients’ self-reported everyday errors.

Selective attention, by definition, involves focusing and concentrating on a target stimulus or a specific activity while inhibiting the processing of other distracting stimuli. Thus, when the ability to ignore external or internal distractors diminishes, individuals can become easily distracted. This distraction can lead to omission errors, where tasks or information are missed. For instance, it is common for patients with TBI to read a page but lose concentration midway due to distractions, resulting in the need to reread the page and potentially causing frustration [57]. These challenges of managing distractions can also impact patients’ ability to complete their studies or work effectively in noisy office environments [58]. A VR test that can predict a patient’s difficulty in handling such situations is invaluable for designing appropriate interventions. Such interventions can focus on training patients to utilize effective attention strategies to reduce distractions in their working environments. Furthermore, conducting repeated attention and executive function assessments during rehabilitation sessions can enable clinicians to make more informed and beneficial decisions regarding interventions for each patient’s unique needs. Previous research [59,60,61] has shown that the timeliness and preparedness of a patient’s return to work play a vital role in their successful reintegration, resulting in smoother transitions and reduced depressive symptoms. A recent study by Aliaga et al. [62] validated a strong correlation between executive functioning and the reintegration of TBI patients into the workforce. Therefore, using VR assessments to predict attention-related difficulties can significantly aid in optimizing rehabilitation strategies and ultimately improve the patient’s outcomes in their daily life activities and work environments.

Regarding the divided attention subtest, it was found to predict the ARCES errors related to the attention distraction factor. The divided attention function pertains to the capacity to simultaneously concentrate on two distinct tasks or activities, such as driving while listening to music or breaking news.

This type of attentional demand necessitates a substantial amount of selective attention and an executive component that, as proposed by the central capacity theory [63], manages the allocation of attentional focus to effectively complete the tasks. It seems that patients also associate their experiences of getting distracted with their performances on either one or both tasks simultaneously. The difficulties they encounter in divided attention tasks likely contribute to the errors they report in everyday situations requiring multitasking or concurrent for on multiple stimuli. This finding underscores the significance of assessing divided attention abilities in TBI patients, as it directly correlates with their real-life challenges in managing distractions and multitasking effectively.

With reference to the sustained attention subtest, correlations were only observed for the mean performance time, which could be attributed to the relatively simpler nature of this task compared to the other subtests. Although patients made a few errors on this task, the reaction times were found to be correlated with everyday attention errors on the ARCES. This can be explained by the fact that sustained attention tasks require examinees to maintain their attentional focus (i.e., selective attention ability) for an extended period [63]. For patients with TBI, sustained attention tasks can lead to fatigue, resulting in increased reaction times and making them more susceptible to distractions, to “behavioral microsleeps” and errors of automated action or commission errors [64,65]. In a previous study by Malegiannaki and Metallidou [17] on a healthy adult student population, mean reaction times on a two-dimensional electronic task, the Sustained Attention to Response Task (SART), predicted everyday attention errors. Thus, the CBAAD behaves similarly to traditional and well-established 2D computerized tools used for attention assessment while retaining all the advantages that 3D technology offers [66,67,68].

The switching attention subtest demonstrated a relatively weaker correlation with the ARCES compared to the other subtests. One plausible explanation is that this task involves alternating attentional focus, which is often considered an executive ability strongly dependent on working memory. Therefore, switching attention tests are presumed to assess both attention and executive functioning [67]. However, reports of false alarm errors, such as pressing the button for a high jump when a long jump is performed and vice versa, predict everyday distraction errors, which is sensible as both are considered errors. Consequently, a correlation with automated errors would also be expected. It is worth noting that the automated error factor of the ARCES, as mentioned in the material description section, includes fewer items and has a minor contribution to the overall variance [12]. This might account for the fewer correlations observed between performances on the CBAAD subtests and the scores for this particular factor.

Furthermore, in line with the previous studies involving a healthy population [41], the CBAAD proved to be a motivating, enjoyable, and time-efficient battery that closely replicated real-life situations. More than 90% of the participants rated it as pleasant, 74% felt that it did not require any specific computer proficiency, and nearly 80% of the participants expressed their willingness to recommend the CBAAD examination to a friend or consider repeating the examination themselves. The findings of the present study extend the previous research by indicating that the CBAAD serves as a suitable and valid instrument for conducting a comprehensive assessment of attention disorders in TBI patients. Notably, its capability to discern finer differences in attention functions aligns with the primary goal of CBAAD’s construction. Additionally, the CBAAD proves to be a valuable tool for clinicians to explore their patients’ capabilities and design personalized interventions, as it offers rich quantitative and qualitative information. By using the CBAAD, clinicians can gain insights into the specific attentional challenges faced by TBI patients, enabling them to tailor rehabilitation strategies to address their unique needs and enhance the overall treatment outcomes.

The present study has notable limitations that warrant consideration. Firstly, the generalizability of the findings based on Greek participants may be challenging, given the global variations in their socio-cognitive backgrounds and demographics. Despite including both younger and older individuals with and without TBI, this study’s relatively small sample size limited result robustness and generalizability. The cross-sectional nature of the study precluded our understanding of the progression of attentional deficits, suggesting the need for longitudinal data. This study’s limitations extended to the scarcity of concurrently administered standardized measures assessing functions, such as semantic memory. Additionally, the correlations of the CBAAD test’s performance with a self-report measure of daily functioning could be biased, and, as such, not reflect true attentional difficulties in daily functioning. The adoption of innovative approaches, like the CBAAD, could incur additional costs and time investments. Clinical integration challenges also exist, requiring additional training and support for effective CBAAD administration and interpretation in routine clinical practice. To establish the CBAAD’s reliability in tracking attentional improvements over time, its test–retest reliability needs confirmation through consistent results in longitudinal assessments on the same individuals.

5. Conclusions

The current study highlighted the critical importance of the early detection and assessment of attentional impairments in individuals with TBI. Traditional neuropsychological assessments, while commonly used, have limitations with respect to ecological validity, underscoring the need for more innovative and sensitive approaches. The newly developed VR test, the CBAAD, emerged as a valuable tool for addressing these limitations. The findings also demonstrate that the CBAAD effectively identifies attentional dysfunctions in TBI patients, showing significantly lower performance scores in comparison to the healthy controls across all subtests. Moreover, the CBAAD’s ability to predict real-life attentional errors, as evidenced by the significant correlations with the ARCE, underscores its potential as an essential asset in the rehabilitation process.

By leveraging the CBAAD, clinicians and researchers can gain valuable insights into the specific attentional deficits experienced by TBI patients, paving the way for personalized and tailored rehabilitation strategies. The early identification of attention impairments can lead to timely interventions, optimizing the recovery process and improving the overall quality of life for TBI survivors. The incorporation of VR testing, such as the CBAAD, in routine clinical practice holds promise for enhancing the precision and effectiveness of attention assessments and rehabilitation strategies. Its implementation can empower healthcare professionals to better understand the complexities of attentional impairments and offer more targeted interventions, thus fostering improved outcomes for TBI patients. To summarize, this study contributed to the advancement of assessment tools and the comprehension of attentional deficits in TBI patients. The practical implications center on early detection, personalized rehabilitation planning, and enhanced functional outcomes, facilitated by the implementation of the CBAAD in clinical settings.

6. Theoretical and Practical Implications

Based on the present findings, there are several implications that need to be mentioned. The most important theoretical implications are the following:

• Advancement in assessment tools: the study highlights the potential for innovative neuropsychological assessment tools, using VR to enhance ecological validity and sensitivity for measuring cognitive impairments.
• Understanding attentional deficits in TBI patients: the research deepens our theoretical understanding of attentional impairments in TBI patients, examining specific components of attention through the CBAAD.
• Linking cognitive performance to everyday functioning: establishing a connection between performance on the CBAAD and real-life attentional errors in TBI patients bridges the gap between cognitive assessments in controlled settings and practical implications for patients’ daily lives.
• Potential for individualized rehabilitation: the study suggests that the CBAAD’s effectiveness in predicting attentional errors opens avenues for personalized rehabilitation planning, addressing specific challenges faced by each patient.

In addition to the abovementioned ideas, the most significant practical implications are as follows:

• Early detection and intervention: the CBAAD offers a practical tool for the early detection of attentional impairments in TBI patients, enabling timely intervention strategies and facilitating interactions through well-presented visual elements in VR applications.
• Enhanced rehabilitation planning: implementing the CBAAD in clinical settings aids in formulating effective rehabilitation plans by identifying specific attentional domains of difficulties, with VR applications enhancing the participants’ comprehension of real-life scenarios.
• Objective measurement of progress: the CBAAD provides a standardized way to objectively measure the progress of TBI patients during rehabilitation, allowing clinicians to track their improvements over time and adjust treatment plans accordingly.
• Facilitating therapeutic feedback: the CBAAD’s ability to identify attentional strengths and weaknesses facilitates the therapeutic feedback, helping patients gain insights into their difficulties and motivating their engagement in rehabilitation efforts, with VR-supported assessments offering multisensory exploratory contexts.
• Supporting long-term functional outcomes: addressing attentional deficits early on with the CBAAD has the potential to improve long-term functional outcomes for TBI patients, positively impacting various aspects of their lives.