Evaluation of a Human–Machine Interface for Motion Sickness Mitigation Utilizing Anticipatory Ambient Light Cues in a Realistic Automated Driving Setting

Abstract

Motion sickness (MS) is a syndrome associated with symptoms like nausea, dizziness, and other forms of physical discomfort. Automated vehicles (AVs) are potent at inducing MS because users are not adapted to this novel form of transportation, are provided with less information about the own vehicle’s trajectory, and are likely to engage in non-driving related tasks. Because individuals with an especially high MS susceptibility could be limited in their use of AVs, the demand for MS mitigation strategies is high. Passenger anticipation has been shown to have a modulating effect on symptoms, thus mitigating MS. To find an effective mitigation strategy, the prototype of a human–machine interface (HMI) that presents anticipatory ambient light cues for the AV’s next turn to the passenger was evaluated. In a realistic driving study with participants (N = 16) in an AV on a test track, an MS mitigation effect was evaluated based on the MS increase during the trial. An MS mitigation effect was found within a highly susceptible subsample through the presentation of anticipatory ambient light cues. The HMI prototype was proven to be effective regarding highly susceptible users. Future iterations could alleviate MS in field settings and improve the acceptance of AVs.

1. Introduction

With the introduction of the SAE Level 4 (L4) of automated driving [1], the role shift for the human from driver to passenger is inevitable. An L4 automated vehicle (AV) is defined to be capable of performing dynamic driving tasks without any expectation on the user to intervene. One of the anticipated changes in passenger behavior is the frequent engagement in non-driving related tasks (NDRTs). These tasks include sleeping, relaxation, reading, consuming display-based media, and engaging in social interaction [2]. The vehicle’s interior configuration is expected to adapt accordingly, with design concepts considering flexible seating arrangements to enhance the passenger experience. This mode of transportation could increase quality of life for users that commute daily, travel long distances, or have busy schedules.

While the introduction of AVs as the new standard of individual mobility is objectively desirable, a major obstacle for the acceptance of this innovation is motion sickness [2,3,4,5]. Motion sickness (MS) is a condition characterized by symptoms of nausea, dizziness, and other forms of physical discomfort. AVs are considered to be potent in the evocation of MS because of three factors: (1) the transition of the driver to a passenger and the resulting reduced vehicle control; (2) the engagement of the driver in non-driving tasks; and (3) rearward-facing seating arrangements [6]. While all of these factors are related to increased MS, they are also part of the innovations that make L4 automated driving desirable. There is therefore a great need for a solution that preserves the benefits of automated driving but also relieves users from MS. Additionally, a potential solution for MS mitigation needs to not only be effective but also tailored to the psychological needs of the passengers.

The aim of this paper was to present the current state of MS basic research, as well as its application in HMI development. HMI conception, an experimental methodology, and its challenges, are illustrated. The presented experimental study investigated the efficacy of an HMI mitigating MS in a highly automated vehicle. The HMI was tested in a realistic driving environment based on self-reports of participants in regard to MS. Additionally, the user experience, predictability, and perceived safety of the system were evaluated.

2. Background

This section provides a general overview of the current state of research regarding MS (Section 2.1), specifically the effect of anticipation on its mitigation (Section 2.2). Finally, based on the insights of previous research and general acceptance criteria, an HMI prototype for MS mitigation is introduced (Section 2.3).

2.1. Motion Sickness in Automated Vehicles

2.1.1. Motion Sickness

MS describes a syndrome characterized by different symptoms that occur during movement or perceived movement in an environment [7,8]. MS is considered a complex phenomenon because no single cause or simply defined mechanism explaining the occurrence of MS has been identified so far. A wide range of stimulations can evoke the phenomenon so that it occurs in many different environments and conditions [9]. Some common forms of MS in dynamic environments are sea sickness, air sickness, and car sickness [10,11,12]. However, not everyone is equally susceptible to MS. The reported prevalence varies strongly between different sources. In a survey by Schoettle and Sivak, for example, about half of the participants reported that they had experienced MS during a car ride [13]. However, although a significant proportion of the population seems to be susceptible to the syndrome, highly susceptible individuals should be given special consideration in the development of vehicle solutions because the burden of MS symptoms is higher for them [12].

The vestibular system plays an important role in MS. It is sensitive to acceleration, with the semi-circular canals being stimulated by angular acceleration and the otoliths being stimulated by linear acceleration. Both types of acceleration can potentially cause MS [14,15]. In addition, the visual system is also considered to be strongly involved in the pathology of MS. Since the visual and vestibular systems are strongly interfering, it has been suggested that visual stimulation usually associated with a matching vestibular stimulus increases central vestibular activity [16,17]. However, visual input is not essential to develop MS, as blind participants have been shown to be equally susceptible to MS as participants with vision [18]. Nevertheless, the manipulation of vision can be utilized to modulate MS [19,20,21].

2.1.2. Impact of High MS Susceptibility for AV Users

MS susceptibility describes the individual response to nauseagenic stimulation and the intensity of the elicited MS symptoms [17]. It is determined by three factors [22]: firstly, the individual’s initial sensitivity to motion varies between individuals and is influenced by sex [23], age [22], genetics [24], and other factors [25]. Women have been indicated to be generally more susceptible to MS with research, suggesting a connection to the female hormonal cycle [26,27]. Regarding age, children around 9–10 years old show a peak in susceptibility [28]. While MS susceptibility is thought to gradually decline throughout the life span, an increase during older ages has been suggested [27]. As the elderly are a relevant and growing demographic part of our society that could profit from AVs, this trend further increases the need for an MS mitigation strategy. Secondly, the individual rate of adaptation, meaning the habituation to specific nauseagenic stimuli through exposure, influences MS susceptibility [14,28]. Sustained exposure to a nauseagenic environment leads to a reduction of MS symptoms. Depending on the duration and repetition of exposure, the sensitivity towards the stimulus is reduced or even fully eliminated. Most individuals show no more MS symptoms after two-to-three days of exposure within a specific environment [17]. The exposure history of an individual has a high impact on its MS susceptibility in general [29]. Finally, the ability to retain the adaptation to nauseagenic stimuli determines an individual susceptibility to MS in future encounters with the environment. Repeated exposure to the same stimulus is associated with a lowered response intensity [28,30].

About 5–10% of the population are considered “highly susceptible” and are therefore likely to show more severe expressions of MS symptoms [29]. The majority of the population reacts to nauseagenic stimuli with mild symptoms (e.g., stomach awareness, drowsiness, and headache). Highly susceptible individuals might show more intense symptoms like nausea, apathy, non-vertiginous dizziness, and decreased cognitive performance [17,25,31]. Additionally, these individuals also report fewer encounters with nauseagenic environments, which reflects an avoidance behavior towards situations that cause severe MS symptoms [23]. The combination of high sensitivity, severe symptom expression, and avoidance behavior limits highly susceptible individuals’ exposure to nauseagenic environments and habituation to them.

AVs are a novel but potently nauseagenic environment [6]. Their novelty means that most users will not have an exposure history, which is hypothesized to lead to an increase in MS incidences [2]. Through exposure and (finally) full habituation, most individuals will be able to use AVs without suffering severe MS symptoms. However, highly susceptible individuals might suffer highly aversive symptoms when using AVs for the first few times. This could lead to not only a negative and skeptical attitude towards the innovation but also cause avoidance patterns that inhibit exposure and therefore adaptation to the environment. Without a strategy to support the transition for highly susceptible individuals during the adaptation process, this specific population could be excluded from using AVs. Developing an effective mitigation strategy that allows for highly susceptible users specifically to adapt to AVs promotes the inclusivity of AVs.

2.1.3. Pathogenesis of MS: Sensory Rearrangement Theory and Postural Instability Theory

Motion sickness and its pathogenesis have been explored and discussed for over a century. Multiple theories have been discussed in literature. Thus far, none of them has been able to fully explain the phenomenon. The two most influential theories are the sensory rearrangement theory by Reason and Brand [22] and the postural instability theory by Riccio and Stoffregen [32]. First introduced in 1975, the sensory rearrangement theory (SRT) specifies the cause of MS at the interference of different sensory channels, specifically the vestibular and visual channels. If motion-related signals transmitted by these channels are discordant, sensory rearrangement ensues. All environments that elicit MS are characterized by sensory rearrangement [22]. Sensory signals are considered discordant if they are in conflict with each other or the sensation expected based on previous interactions with a similar spatial environment. The latter conflict and its somatic symptoms are reduced through continued interaction with a nauseagenic stimulus or habituation. The SRT was expanded to a structural component model: the neural mismatch model [33]. The neural store, a neurophysiological module, retains sensory information patterns previously experienced under similar circumstances (reafferences). Current and previous sensory information are compared in another module. If incongruent, the mismatched signal is sent along reflex pathways and MS symptoms ensue. With a longer exposure to the mismatched stimuli, the content of the neural store is updated, the conflict is being decreased or resolved, and the symptoms fade. These mechanisms explain the effects of habituation in the reduction of MS [14,30,34].

Additionally, Reason and Brand postulated that the vestibular system must be implicated for MS to occur, either directly via acceleration or indirectly via the visual illusion of movement [22]. This premise is supported by research indicating that an intact vestibular system is necessary to be susceptible to MS [35]. Additionally, since the vestibular organ is only sensitive to angular and linear acceleration, constant movements should not be nauseagenic [33]. In addition, Reason postulated the extend of rearrangement to be dependent on the number of discordant sensory channels, the magnitude of discrepancy, and the duration of the conflict [33].

Engaging in NDRTs and the introduction of novel flexible seating arrangements are expected to increase the incidences of MS in AVs. Engaging in NDRTs or sitting backwards to the direction of acceleration is predicted to cause discordance between the visual and vestibular sensory channels, thus leading to MS symptoms [2,10].

The postural instability theory (PIT) focuses on the phenomenon of postural instability that has been shown to precede MS [32,36,37]. Postural sway occurs in situations that constrain the control of posture—the coordinated stabilization of all body segments [37] (p. 277). In a novel situation like riding an AV, regaining control over posture is difficult because an adequate stabilization strategy is either not available to the organism or very challenging to enact. This causes prolonged postural instability. It is hypothesized that the prolonged exposure to postural instability is the cause of MS and necessary to elicit its associated symptoms [32]. Additionally, failure to detect or ignorance towards changing dynamics can have a negative impact on postural stability. Information about possible constraints is hypothesized to support adequate reactions to nauseagenic situations. The severity of MS is proportional to the duration and magnitude of instability. Posture and postural control are strongly connected to the visual [38] and vestibular system [39]. Both modalities support the body by detecting information relevant to postural control [32].

2.2. Effect of Anticipation on Motion Sickness

Anticipation in a driving context is the cognitive ability to identify cues that could cause potential conflict and to act accordingly [40] (p. 604). In the context of AVs, acceleration, turns, and other changes in the vehicle dynamics can be sources of sensory conflict or postural instability. Providing reliable cues to enable passenger anticipation has been considered to be a potentially protective factor against MS [6,32,33]. Two properties of a system that allows a user to anticipate its actions are controllability and predictability. Understanding their relationship to MS will provide insight on how anticipation can be modified in AVs to benefit the user.

While traveling via vehicles, most passengers have made the following observation: out of all passengers, the driver is least likely to develop MS. After investigating this phenomenon, Rolnick and Lubow came to the conclusion that controllability was the key advantage drivers had over passengers [41]. As controllability provides users with reliable information about a vehicle’s behavior, the authors hypothesized that as drivers are in control, they are also able to anticipate the movement of the vehicle. Participants within the group with controllability developed 35% less intense MS than participants in the no-controllability group [41]. Similar effects were reported in an experiment by Levine, Stern, and Koch using opto-kinetic stimuli [42]. Being in control during nauseagenic stimulation, like a car ride, has a positive impact on the user’s well-being. However, utilizing the benefits of L4 AVs no longer allows a user who is no longer a driver and now a passenger to be in charge while the automation is active.

Predictability is commonly defined as the capacity to accurately anticipate future events [42] (p. 2676). It has been shown that increased predictability via reliable information about the trial progression reduces MS during opto-kinetic stimulation. Additionally, it was reported that the effect of predictability was distinct from controllability in the attenuation of perceived MS [42]. In a different experimental design that manipulated the predictability of both movement direction and intervals in forward and backward linear acceleration, similar results were found. The condition with both direction and interval timing being predictable resulted in 52% lower MS scores after 15 min compared to conditions with neither direction nor interval timing being predictable [43]. In contrast to controllability, predictability can be increased by HMIs in AVs, like in our study, potentially offering a remedy to MS.

Anticipation as Motion Sickness Prevention

The anticipation of motion patterns has been suggested to have a protective effect on a user against MS. SRT and other models attributing sensory discordance as the cause of MS hypothesize a discrepancy between sensed and expected sensory information to be a main cause of MS [22,33,44,45]. Reason theorized that passengers in a car enduring passive movements are unable to create an anticipative efference to the neural store [33]. Therefore, the activation of an associated reafference trace, which contains information about the expected sensory input and appropriate adaptation patterns, is delayed. The sensed gravitational and inertial forces on the passenger and the expected forces are in conflict with each other. This is caused by the reafference trace not being activated in time. It is assumed that being able to anticipate passive movement can counteract the delay of retrieval, reduce sensed conflict, accelerate adaptation, and finally, reduce the intensity and duration of MS symptoms [33]. Anticipation is also considered beneficial by the PIT, because being able to anticipate the perturbating effects of motion on posture enables the user to more effectively coordinate their postural control. This is theorized to reduce the duration of postural instability and therefore the intensity of MS symptoms [32,46]. Enabling a user to anticipate a vehicle’s movement trajectory both in theory and practice appears to be a promising path in the development of a mitigation strategy against MS [6].

2.3. Development of an HMI for MS Mitigation

2.3.1. Requirements for HMI Acceptance

In the context of today’s in-vehicle HMIs, acceptance is defined as the intent to use the system and incorporate it into the user’s driving behavior. Acceptance can be achieved through a pleasant user experience. User experience includes different aspects of interaction between user and system, like usability [47], as well as the hedonic quality of interaction [48]. Additionally, acceptance can be created by providing the user with reliable information according to its needs [49]. As L4 is still a relatively new mode of transportation, its lack of controllability for users might increase the need for information regarding the vehicle’s safety and driving behavior [50,51]. Therefore, the HMI should ideally increase the predictability of vehicle dynamics [52] and enhance the perceived safety of the vehicle to accommodate for the user’s loss of vehicle control.

Creating an HMI that is acceptable for users has an important impact on the ecological validity of the system and the problem it is intended to solve: if users do not accept the HMI and, as a consequence, do not utilize it, the effectiveness of the HMI is limited. Therefore, an HMI should not only be evaluated on its efficacy but also on parameters regarding its acceptability. As the system should be designed with the intent to support the user’s transition to L4, an HMI that not only is effective in the mitigation of MS but also accommodates the user’s psychological needs will probably be more accepted.

2.3.2. HMIs Utilizing Anticipation of Vehicle Dynamics

Because L4 limits a user’s need to actively control the vehicle’s movements, the need for predictability increases [10]. Research on HMIs has explored different modalities to provide users with information supporting anticipation. Presenting the vehicle’s trajectory allows passengers to anticipate turns or pitching. This concept was investigated in an aviation simulator setting [53]. The visual projection of a 4-m-trajectory on the windshield, similar to roller coaster rails, significantly reduced the average reported MS intensity. A similar visualization of the vehicle trajectory could be implemented in AVs. However, as users of an AV shift visual attention to NDRTs, the effect of the projected information might decline. Utilizing peripheral vision, both Bloch [54] and Karjanto et al. [55] explored the effect of visual cues provided by a vehicle in anticipation of a driving maneuver. Bloch’s HMI displayed color-coded cues on video panels positioned on the car doors inside the vehicle. The stimuli would change color whether the car would accelerate or decelerate 0.5 s in advance. The participants, engaged in NDRTs, would detect the cues only via their peripheral vision. While the results did not yield significant effects, the study provided insights into potential parameters to be considered [54]. A similar HMI introduced by Karjanto et al. [55] also presented visual cues in the periphery of the visual field. The cues signaled the initiation of a turning maneuver 3 s in advance. Participants of this study showed a significantly smaller increase in MS during the AV ride when receiving visual cues compared to not receiving them. Considering the mixed evidence on visual HMIs, more evaluation is needed to develop a system optimized for the user enhancing its effectiveness.

The effect of auditory anticipation cues was investigated by Kuiper et al. [56]: the participants were accelerated in a linear forward or backward motion on a rail. Visual information was negated. Auditory cues were either presented 1 s in advance containing information about the direction of acceleration or in the control condition presented at random 2–6 s after acceleration and with no reliable information about the direction. Reported MS was significantly different between the anticipatory cue condition and the control condition. The anticipatory cues with reliable information about the acceleration direction were accompanied by a 17% decrease in average reported MS.

Using the haptic sensory channel to present anticipatory information was suggested by Yusof et al. [57]: through vibrating devices attached to the participant’s forearms, a tactile stimulus was presented to inform the participant of oncoming changes in the direction of the vehicle’s movement. An increase in situation awareness and a decrease of MS was reported, but, given the exploratory nature of this study and with no reported inference statistics, these findings should be interpreted with caution.

Even though different modalities can be used to transmit anticipatory information and should theoretically be able to prevent MS, visual cueing is considered to be a very promising approach, more so than auditory or haptic cues when considering the predominant theories on MS pathology [6,56]. HMIs utilizing different modalities have shown the first promising results. However, visual and vestibular cues have been shown to be interactive [58]. Manipulating visual input directly, in theory, might be more effective than communicating via other modalities. Therefore, a visual communication strategy for an HMI was evaluated in this study.

2.4. Prototype of an HMI for MS Mitigation in This Study

Following both theoretical and empirical considerations, an HMI prototype utilizing anticipatory ambient light cues was developed for this study. The HMI prototype promoting MS mitigation was designed with the user in mind and evaluated in a previous online study on its communication strategy, perceptual properties, and user experience [59].

Utilizing anticipation of vehicle dynamics as the mitigation strategy, the HMI provides the user with anticipatory visual cues via ambient lighting as part of an integrative assistance system [60]. The provided light cues allow the user to anticipate obtuse curves and their turn direction (see Section 4.8.1). The source of the light-based cues are LED light bands installed in the interior of the AV. The system resembles the integrative assistance system proposed by Wilbrink, Schieben, and, Oehl [61] utilizing a 360-degree LED light band in the vehicle interior [60]. In a traffic environment, the positioning does not obstruct the user’s view and allows them to scan the outside environment. Being positioned within the field of view, it enables stimuli detection within peripheral vision whether sitting with or against driving direction or engaged in an NDRT (as recommended by [62]). The chosen design color was cyan, as it was deemed to be a neutral color in the traffic context [63]. Considering that not only passengers of the AV but also other traffic participants might be exposed to the stimuli, choosing a neutral color is important to reduce confusion and ensure road safety.

This HMI was designed for the specific use case of driving maneuvers involving obtuse angle curves, focusing on the use case of driving on rural roads. The use case of 90-degree curves, which are common in urban settings, was already successfully addressed in a previous simulator study [64].

3. Research Question and Hypotheses

In a realistic driving study, the abovementioned HMI utilizing anticipatory ambient light cues was examined regarding its ability to mitigate MS. Based on the aforementioned considerations, we assumed that the HMI would effectively mitigate MS in highly susceptible individuals, and we therefore tested the following hypotheses:

Hypothesis 1 (H1).

Highly susceptible participants will report a significantly lower increase in MS symptoms when presented with anticipatory ambient light cues by the HMI compared to being provided with no cues.

Highly susceptible participants in the context of this study were defined as the upper 50% percentile of the convenience sample at hand regarding MS susceptibility. Hypothesis 1 (H1) was limited to this specific population, as participants with an inherent low susceptibility were not expected to develop severe MS. Additionally, as this HMI was developed with the aim to specifically support highly susceptible individuals (see Section 2.1.2), its efficacy regarding this population was of main interest. Therefore, H1 was tested on a subsample of the participants with the highest susceptibility of the sample (see Section 5.2). This, however, also limits the application of the research results to this specific population, which will be discussed in more detail in Section 6.1.

Hypothesis 2 (H2).

Participants will report a significantly higher user experience of the system in the condition, during which the HMI provides anticipatory ambient light cues compared to conditions during which no light cues are provided.

With every feature added to a system, the corresponding user experience needed to be evaluated. A well-designed HMI incorporates both functional and hedonic qualities, indirectly supporting its efficacy and the acceptance of the user (see Section 2.3.1). User experience was expected to increase for all participants, as not only highly susceptible users would be exposed to the system in a real-life setting. Therefore, this hypothesis was tested on the total sample of this study.

Hypothesis 3 (H3).

Participants will report significantly higher predictability and perceived safety of the system in the condition, during which the HMI provides anticipatory ambient light cues compared to conditions where no light cues are provided.

As a result of the passenger being enabled to anticipate the AV trajectory, the predictability was expected to increase. The user could ensure themselves that the automation and its communication were reliable, which was hypothesized to result in a higher perceived safety. Perceived safety, in this context, was defined as perceived safety from driving accidents [50,51]. Predictability and perceived safety are both predictors for acceptance and thus influence the intention to utilize a system while driving [65]. Being accepted by the user and utilized willingly is a desirable quality for an HMI (see Section 2.3.1). Additionally, the acceptance of an HMI also positively reflects the AV itself, possibly evoking more acceptance for L4 automated driving.

4. Materials and Methods

This section provides a detailed description of all aspects regarding the realization of the realistic driving study. Firstly, the design and variables of the experiment are presented, and their operationalization is explained (Section 4.1, Section 4.2 and Section 4.3). The research setting and sample (Section 4.4 and Section 4.5), as well as the experimental set-up of the AV and the integrated HMI in the vehicle interior (Section 4.6, Section 4.7, Section 4.8 and Section 4.9), are explained component by component. Finally, the procedure of data collection is illustrated chronologically (Section 4.10).

4.1. Design

The performed experiment was constructed as a single factor (communication strategy: HMI vs. baseline) within-design, resulting in two experimental conditions: the independent variable communication strategy (IV) was varied between both trials to either the treatment condition (HMI) or the control condition (baseline). During the treatment condition, the participants were presented with anticipatory ambient light cues by the HMI. Additionally, the automation mode was presented (see Section 4.8). During the control condition, only the automation mode was presented to the participants. The order of presentation was counterbalanced between the two conditions.

4.2. Dependent Variables

4.2.1. Motion Sickness

MS increases over the duration of the trials was measured using the German translation of the Simulator Sickness Questionnaire (SSQ) [66]. The state of MS of the participants was measured before being exposed to the nauseagenic stimulus (SSQ_pre) and after exposure (SSQ_post) [67]. The questionnaire consisted of 16 items, each evaluating the intensity of an MS symptom on a 4-point Likert scale (0 = “none;” 1 = “slight;” 2 = “moderate;” and 3 = “severe”).

Even though the SSQ was developed to measure simulation sickness, the questionnaire was chosen because it measures the same characteristic symptoms as by established MS measures (e.g., Motion Sickness Questionnaire [68]) but is more sensitive to lower symptom expressions [66]. The nauseagenic stimulus in this study was chosen to be of moderate intensity for ethical reasons and to prevent a high dropout rate. Even highly susceptible participants were expected to show only light-to-moderate MS symptoms. Additionally, the questionnaire was deemed suitable for measurements before and after the nauseagenic stimulation [69].

4.2.2. User Experience, Predictability, and Perceived Safety

User experience was measured using the German version of the User Experience Questionnaire (UEQ) [48]. The questionnaire consisted of 26 items, each regarding an attribute of the system. Each item was scored on a 7-point Likert scale (min = −3 (e.g., “complicated”) and max = +3 (e.g., “easy”)). Predictability was measured using a single self-devised item (“During the drive, I was constantly aware of what the system did or would do.”) scored on a single choice, metric scale (min = 0 (“fully disagree”) and max = 100 (“fully agree”). Perceived safety was also measured with a single self-devised item (“During the drive, did you perceive the system to increase your safety?”) scored on a single choice metric scale (min = 0 (“fully disagree”) and max = 100 (“fully agree”).

4.3. Control Variables

Affinity for technology interaction, defined as personal resources for successfully coping with technology, was measured using the German Affinity for Technology Interaction Scale (ATI) [70]. The scale consisted of 9 items evaluated on a 6-point Likert scale (min = 1 (“completely disagree”) and max = 6 (“completely agree”). Additionally, individual MS susceptibility was measured utilizing the Motion Sickness Susceptibility Questionnaire (MSSQ) [71]. The questionnaire was constructed from six subscales regarding the occurrence of MS in nauseagenic environments during childhood and adulthood. The subscales measure the exposure frequency on a 4-point Likert Scale (min = 0 (“Never”) and max = 3 (“11 or more trips”)) and symptom occurrence on a 5-point Likert scale (min = 0 (“Never”) and max = 4 (“Always”).

4.4. Sample

Twenty participants (11 male and 9 female) between the ages of 22 and 59 (M = 33.75 and SD = 11.05) took part in the experiment. The sample was recruited via the institute’s participant database. A self-devised questionnaire was utilized to determine the participant’s ability to participate. The preselection questionnaire examined age, gender, and pre-existing health to ensure ensuring the physical safety of the participants. Participants were excluded from participation if they reported preconditions endangering their health during the participation of this study (e.g., pregnancy, vertigo, preconditions regarding the spine, heart, metabolism, or vestibular organ). Additionally, participants were excluded if they reported being infected with the SARS-CoV-2 virus or having close contact with an infected person during the last 14 days prior to the experiment.

The experiment was conceptualized and realized in accordance with the Declaration of Helsinki. Informed consent was obtained from all participants before the experiment. The participants were allowed to stop the experiment at any point without justification or consequence. The participants volunteered, but they were financially compensated.

4.5. Setting

The study was conducted on a former airport runway with dimensions of 895 by 29.5 m. The runway is currently used solely for scientific research.

4.6. Apparatus

4.6.1. Automated Experimental Vehicle

The German Aerospace Center (DLR) runs experimental vehicles with the ability to drive highly automated (L4). The used vehicle in this study called FASCar^®-II is a part of the large-scale research facility FASCar^®. The basic car is a Volkswagen Passat. It was modified with an interface that translates motion commands from the automation software (Section 4.6.2) to the longitudinal acceleration, deceleration, and steering angle of the vehicle (Figure 1).

4.6.2. Automation Software

To achieve a maximum intervention, the FASCar^®-II is equipped with a throttle paddle and a prototype of a brake booster that support full longitudinal control without any restrictions. For lateral control and new HMI concepts, a steer-by-wire system is integrated into the vehicle. It allows for the active control of the vehicle’s wheels without a turning of the steering wheel as well as the turning on the steering wheel without turning the vehicle’s wheels. This advantage could be used for new HMI concepts and automated security interventions, and it enables FASCar^®-II to be used not only on test sites but also in simulator settings. FASCar^®-II, due by its hardware, has no road approval and can only be driven on test sites. In exchange, it offers a higher level of active interventions and a futuristic HMI.

For closing the loop of controlling the car’s movements, a high precision GNSS location system is installed onboard. The GNSS System uses a differential GPS together with a precise inertial measurement unit (IMU). The vehicle position is forwarded to the automation software, which runs on the car-PCs in the trunk. The interface to the car commands is realized with a dSPACE MicroAutoBox [72] which translates the commands from the automation software to the car internal CAN protocol. On the dSPACE MicroAutoBox, safety checks are executed before setting the car in motion.

For perceiving the environmental situation, FASCar^®-II is equipped with four laser scanners and three long-range radars that are mounted in front and rear bumpers of the vehicle. A C2X-System could be used for vehicle-to-infrastructure and vehicle-to-vehicle communication.

The automation software used in this study is a proprietary development of the DLR’s Institute for Transportation Systems and is named Cooperative Safe Automation (CSA) [73]. The main parts are represented in different libraries and are shown in Figure 2:

• ENV—classes and functions for environment representation.
• VIEW—different views of the environment data for different purposes.
• FUN—a library of classes and functions for autonomous driving functions.
• MAD—a library of mathematical functions and representations used by the other libraries.
• Ifmiddleware—an interface specific to an IPC middleware to decouple the automation framework from the middleware in use.
• The IPC middleware used is named Dominion [74], also developed at DLR. The logical functions are ordered into different modules and are called apps in Dominion.

To guarantee a precise reproduction for any repetition of the scenario, it was decided to use a static trajectory for the test scenario. This resulted in a reduction to a minimum set of automation apps. The following apps were used for automated driving:

• StaticTrajectoryManager: Thus reads an offline generated trajectory file (Section 4.7) and translates the information to Dominion data structure for vehicle control. Furthermore, the app generates and sends the trigger signal for the LED light band to the input port of the Arduino (Section 4.6.3).
• HighlevelController: This generates motion commands based on the current vehicle position to follow the offline generated trajectory.

4.6.3. HMI—LED Light Band

The LED light bands (Figure 3) were built up using RGBW (red, green, blue, and white)-light stripes. This stripe consists of single, individually-controllable light points. Each of the light points consists of one red, one blue, one green, and one phosphor-converted white emitter. Each of the emitters is individually controllable so that the light color and the brightness can be adjusted for each light point. Thus, it is possible to create any lighting scene whether static or dynamic. The light stripes were placed in an aluminum profile and covered with a diffuse cover lens to generate a homogeneous lighting surface. The LED light bands each were positioned between the B- and C-pillar on each side of the AV, emulating the positioning of a 360-degree LED light band (Figure 3) [61].

The LED light bands were controlled using a micro board from “Arduino” and were driven by a 12 V power supply (modulated by a 60 W DC/DC converter). To adjust the lighting scenes, a user interface was programmed to give the experimenter the possibility to set different parameters of the cue like frequency, brightness, and the time before the event starts. For triggering the programmed light scenes, an interface to the vehicle was provided (Section 4.6.2).

4.7. Automation Scenario

For generating the static trajectory in the test scenario, i.e., the concrete AV driving track, an open-loop single-track model of the experimental vehicle modeled in MATLAB was used. With a set of time-based input variables (u1: acceleration; u2: steering angle rate), the track shown in Figure 4 and the according trajectory were calculated offline. The pictured track represents only a section of the total track. Different variations were connected to a track with a driving time of 15 min in total. The AV drove at a maximum speed of 13 m/s. By calculating and using a static trajectory, it was possible to easily define and plan the maximum lateral acceleration for each curvature of the automated driven track in advance (shown in Figure 4; illustrating sketch in Figure 5).

4.8. Stimulus

During the treatment condition (HMI), the HMI provided the participants with anticipatory visual light cues. Additionally, the LED light band was continuously illuminated in cyan to signify to the participants that the AV was driving in L4 mode (mode awareness) [61]. The communication strategy for mode awareness is a feature of the 360-degree light band previously introduced by Wilbrink, Schieben, and Oehl [61]. During the control condition (baseline), the HMI was solely illuminated in cyan (for mode awareness). The two conditions could be differentiated based on the quality and quantity of information provided by the HMI. During the baseline condition, the HMI provided only the information about the AV’s automation mode. During the treatment condition, besides the automation mode, additional information (quantity) was provided, namely by providing anticipatory information (quality) to provide an MS mitigation effect.

4.8.1. Communication Strategy: Anticipatory Ambient Light Cues

In addition to displaying the mode of automation, the LED light bands provided the user with anticipatory visual light cues. The visual light cues were presented by the LED light band on the side consistent with the turning motion. The chosen cue onset was 2 s before the initiation of a steering maneuver by the AV. This cue onset was chosen because previous studies operating with an anticipative cue onset between 1 and 3 s successfully reduced MS [55,56].

The cue duration lasted from the standardized onset, 2 s before a driving maneuver, to the first steering motion of the maneuver. The LED light band was fully illuminated for 0.5 s and then dimmed to the base luminance. This illumination pattern was repeated at a 1 Hz frequency for 2 s. The design was chosen because of its analogy to the turn signal, which was very familiar to most users. Additionally, this communication strategy was rated most favorably out of four other strategies evaluated in an online survey [59].

4.9. Operationalization of NDRT

NDRTs in this study were operationalized by limiting the participant’s view outside the AV sitting in the back seat (see [57]). The rear windshield and side windows were covered. The back seat was separated from the front seats with a divider. The only light source in the rear department of the AV was the ambient lighting by the HMI. NDRTs are thought to increase MS because they limit the outside view and therefore the anticipation of vehicle dynamics [2,21], so limiting the outside view emulates this effect. The chosen operationalization is commonly used in this field of research [43,56,57,75] and has a high degree of standardization. While NDRTs can be abandoned by the participant at random to compensate for the effects of the nauseagenic stimulus, blocking the outside view is easy to control. This interior set-up of the AV was consistent throughout both experimental conditions.

4.10. Procedure

The experiment was conducted during two separate sessions, at least 7 days apart from each other. During each session, one trial was conducted. Demographic data were only assessed during the first session. Depending on the balancing of the conditions, the order of the two conditions varied between participants. As the data collection of this study took place during the COVID-19 pandemic in Germany, the experiment was conducted under hygienic precautions issued by the DLR. The involved participants and staff wore FFP2-grade protective respirator masks for the duration of the experiment. Participants and staff were instructed to disinfect their hands and practice distancing upon arrival. The AV was aerated for 30 min before each session. Participants arrived via private transportation at the test field.

Upon arrival at the first session, the participants were welcomed and informed about safety requirements and the procedure. The participants were informed about the research subject, as MS research ethically requires full disclosure about potential risks during participation [69]. Informed consent was obtained. The participants completed a demographic questionnaire regarding age, gender, driving and simulator experience, and alcohol consumption during the last 24 h leading up to the experiment. Affinity for technology was measured using the ATI [70]. MS susceptibility was measured utilizing the MSSQ [71].

Following demographic data collection, the first experimental trial was initiated. Before being exposed to stimuli, the participants reported their current state of MS using the SSQ_pre [66]. According to the condition, the participants were introduced to the AV, the HMI, and its communication strategy. This also included the type of information it provided depending on the condition. They were instructed to get accustomed to the HMI because they would evaluate it after the trial. The participants sat down on the middle seat of the rear bench. They were instructed to sit back and inform the staff immediately if MS symptoms reached their individual threshold of discomfort. Throughout the experimental trial, the AV was supervised by a safety driver on the driver’s seat and an automation supervisor on the front passenger seat. The participant was manually driven to the starting location of the automation scenario. The automation and HMI scenarios were started by the staff. The driving scenario duration was about 15 min. After completing the scenario, the participants reported their current state of MS (SSQ_post).

After the driving scenario was completed, the user experience of the system was evaluated using the UEQ [48]. Predictability was measured with a single self-devised item. Perceived safety was also measured using a single self-devised item. After the completion of the experimental trial, the participants were instructed to rest until they felt fully recovered. The participants were allowed to leave the property after stating their capability to handle their vehicle safely.

During the second session, the other experimental trial was conducted following the same procedure as before but under a different experimental condition. After completing the second trial, the participants were fully debriefed and financially compensated for their participation.

5. Results

Four datasets had to be excluded for suspicion of confounded data. Participants that presented a decrease in MS during the baseline condition were excluded. Since items of the SSQ test for symptoms that are not exclusive to MS, participants might have arrived at the experiment with symptoms present (e.g., headache, exhaustion, and cold sweating caused by nervousness). If they recovered from the symptoms during the experiment, this could have resulted in a measured decrease of MS, confounding the effect of MS induction. Participants with reported decreases of MS beyond their pre-measurement during the treatment condition were included because this effect could be attributed to the manipulated mitigation effect of the HMI compared to an uncontrolled factor during the baseline condition. The data analysis was performed with n = 16 datasets (9 male and 7 female).

5.1. Descriptive Analysis

The ages of participants ranged from 22 to 59 years, and the mean age of the sample was M(age) = 32.94 years (SD = 10.34). The sample had a mean ATI score of M(ATI) = 4.67 (SD = 1.15) [70]. MSSQ raw scores (M(MSSQ) = 78.36; SD = 29.25) ranged from min = 25.08 to max = 113.52.

5.2. Sample Selection for Inferential Statistics

In order to perform inferential tests for H1, a median split was performed based on the MSSQ raw scores of the participants (Mdn = 87.29), separating the sample in two subsamples. Subsample High (n = 8; 5 male and 3 female; M(age) = 32.30 and SD(age) = 8.81) had MSSQ raw scores ranging from min = 92.07 to max = 113.52 (M(MSSQ) = 101.94 and SD = 7.77). Subsample Low (n = 8; 4 male and 4 female; M(age) = 33.60 and SD(age) = 12.27) had MSSQ raw scores ranging from min = 25.08 to max = 82.50 (M(MSSQ) = 54.97 and SD = 12.27).

As subsample High represented the user group described in the hypothesis, the inferential data analysis to test for H1 was performed on this subsample.

5.3. Inferential Statistics

5.3.1. Effect of MS Mitigation

The MS increase (∆SSQ) during the trial was defined as the difference between SSQ_pre and SSQ_post.

$$\Delta SSQ=SS{Q}_{post}-SS{Q}_{pre}$$

∆SSQ was computed for both experimental conditions. Paired sample Student t-tests between the HMI and baseline condition were performed for both subsamples. The test computed for subsample High reached significance (t(7) = 2.53; p = 0.039) detecting a high effect (d = 0.895) [76], indicating that participants developed fewer MS symptoms during the treatment condition and thus supporting a mitigation effect by the HMI. Hence, H1 was confirmed (Figure 6).

Subsample Low presented an M(∆SSQ) = 4.43 (SD = 3.05) for the baseline condition and an M(∆SSQ) = 6.42 (SD = 7.23) for the HMI condition. Explorative inferential testing for differences did not reach significance.

5.3.2. User Experience

The UEQ subscales of Attractiveness, Efficiency, Perspicuity, Dependability, Stimulation, and Novelty were calculated for both experimental conditions [70]. Because the normality assumption for the factor Perspicuity was not met, non-parametric Wilcoxon rank tests were performed for all subscales. All six tests computed reached significance and detected intermediate-to-large effect sizes (

Table 1. Wilcoxon rank test for UEQ subscales of Attractiveness, Efficiency, Perspicuity, Dependability, Stimulation, and Novelty.

UEQ	HMI	Baseline
Subscale	M (SD)	M (SD)	W	Mean Difference	p	r
Attractiveness	0.92 (0.86)	−0.29 (1.05)	116.50	1.17	0.013	0.713
Efficiency	1.61 (0.65)	0.31 (1.02)	117.50	1.37	0.001	0.958
Perspicuity	2.53 (0.69)	1.33 (1.57)	89.50	1.25	0.002	0.967
Dependability	1.89 (0.70)	0.42 (1.28)	115.50	1.50	0.002	0.925
Stimulation	0.66 (1.04)	−1.30 (0.96)	128.50	2.12	0.002	0.890
Novelty	0.80 (1.07)	−0.20 (1.11)	110.00	1.00	0.005	0.834

) [76]. H2 was confirmed, meaning that the treatment condition was rated better on all six factors of user experience.

5.3.3. Perceived Safety and Predictability

A paired sample Student t-test was computed to analyze the variable predictability (HMI: M = 75.00 and SD = 22.97; baseline: M = 20.43 and SD = 27.98). The test reached significance and detected a large effect size (t(15) = 7.52; p < 0.001; d = 1.880).

For the variable of perceived safety, a nonparametric Wilcoxon rank test was computed between the HMI (M = 71.19 and SD = 23.94) and baseline conditions (M = 26.69 and SD = 24.27) because the normality assumption was not met. The difference between both conditions reached significance (W = 91.00; p = 0.002; r = 1.000). H3, predicting higher ratings for both predictability and perceived safety during the treatment condition, was confirmed regarding both the effects. These results indicated that the HMI was perceived as supportive regarding the predictability and road safety of the AV during the treatment condition.

6. Discussion and Future Research

This section is dedicated to a differentiated interpretation of the results of the presented study. Its methodological strengths and weaknesses are discussed, as are its implications for future research in this field.

The realistic driving study investigated the efficacy of a prototype for an HMI for MS mitigation in AVs by utilizing anticipatory ambient light cues provided by a LED light band as the HMI. The cues provide users with reliable anticipatory information about the onset and direction of the AV’s obtuse-angle turns. Data analysis revealed a significant MS mitigation effect regarding highly MS-susceptible individuals, confirming the hypothesis that anticipatory cues regarding vehicle dynamics are an effective countermeasure. Participants developed significantly fewer MS symptoms during experimental trials in which they were provided with anticipatory cues compared to when they were provided with no cues. Additionally, the effect corroborated previous findings regarding the effect of anticipatory HMIs on MS mitigation [43,55,56]. As the ultimate goal of MS research is to mitigate the symptoms in nauseagenic environments, the results of this study prove to be a step towards reaching this goal, at least regarding AV settings. This study specifically focused on highly susceptible individuals, the users being most negatively affected by MS and who are therefore an important population to be considered for AV implementation in traffic. The development of an MS mitigation strategy that is effective for them could make a noticeable difference in the inclusivity and therefore acceptance of AVs. Considering that the prototype investigated in this study was evaluated in its first iteration, the confirmation that the HMI was able to mitigate MS is the first step in the development of a possible solution for highly susceptible individuals in AVs. Additionally, H2 and H3 were confirmed by the experiment. In the HMI condition, in which the HMI presented anticipatory ambient light cues, the system was rated to be significantly more attractive regarding user experience (H2), predictability, and perceived safety (H3) compared with the baseline condition without anticipatory ambient light cues. User experience, increased predictability, and perceived safety might influence the intention of a user to utilize an HMI. By designing an HMI to be attractive, purposeful, and informative, the utility of the HMI and its efficacy are enhanced. The new insights regarding these qualities of the HMI will be helpful when tailoring future iterations to the psychological needs of the user.

The HMI was evaluated in a realistic driving setting including a naturalistic passenger environment and a L4 AV. Thus far, MS mitigation strategies for L4 have mostly been assessed in driving simulators [43,54,56] or by utilizing the Wizard-of-Oz method with partially automated driving support systems [55,57]. As the automated driving style itself and the increased autonomy of the vehicle are unique aspects of L4, the ecological validity of this experimental setup enhances the impact of the MS mitigation effect reported in this study. The validity and standardization of the method at hand are superior compared to a Wizard-of-Oz set-up. Firstly, the method is not dependent on participants believing in the illusion of an AV to create a valid simulation. Secondly, the driving style of an AV is more consistent than a Wizard-of-Oz-simulated automated driving style because the variance of the human driving behavior does not have to be accounted for [77].

This study provides a successful proof of concept for the first iteration of an ambient light-based anticipatory onboard HMI for MS mitigation. The results comprise a promising basis for future iterations of this HMI with potential for field application in the future.

6.1. Limitations

While the highly susceptible subsample of this study benefited from the anticipatory cues provided by the HMI, it is to be determined how reliable this effect is in regard to different stimulus intensities. Based on previous research, 10–20 min of moderate stimulation is considered sufficient to induce MS but not to the point of strong physical discomfort [69,71,75]. However, the intensity and quality of nauseagenic stimulation are difficult to quantify, and in a real-life setting, users might be exposed to more intense stimuli. The manipulation of stimulation intensity could be increased or implemented as a dependent variable to control for the effect of different intensities. However, with increased stimulation and more trials, the burden on the participants increases because MS symptoms are considered to increase linearly with intensity and time of exposure [75]. Considering research ethics in regard to human subjects and the potential increase of dropouts, finding a balance between sufficient stimulation depending on the subject of research and the compliance and well-being of the participants will be a continuous challenge. The high variation in responses by individuals to the same stimulus is another aspect to be considered when testing for interaction between mitigation effect and stimulus intensity.

The stimulus in this study was chosen to be of moderate intensity, as the main objective was a proof-of-concept of the HMI. The stimulus was sufficient to induce enough MS that the mitigation effect was detectable. However, we had no dropouts caused by MS symptoms, and all participants recovered from the stimulus within minutes.

For the test of the main hypothesis of this study (H1), which required a highly susceptible sample, a subsample was created post hoc out of the convenience sample. The data exclusions and division of the convenience sample reduced the number of analyzed cases. The test power provided by a smaller sample limits its possibilities to detect the investigated effect. Recruiting a sample that fulfils the criteria of being highly susceptible for MS and being possibly more homogeneous in its reaction to nauseagenic stimuli may be realized by performing a screening with an MS susceptibility measure [55].

The sample size case of this study was sufficient to detect the predicted effect. Coincidently, the subsample created by the median split was rather representative of the highly susceptible population, presenting with MS susceptibility scores within the upper 10th percentile [71].

The chosen operationalization that emulates the effect of NDRTs by limiting visual information is commonly used in this field of research, but its ecological validity is limited [43,56,57,75]. As operationalization has a high degree of standardization, the influence of different NDRTs could have an effect on the efficacy of an HMI. If an HMI’s efficacy is hindered by factors like visual attention, cognitive workload, or other uncontrolled factors, its usability in a natural traffic setting may be reduced. Future research should evaluate the HMI, its efficacy on MS mitigation, and its usability during the performance of realistic NDRTs [55,78,79].

6.2. Implications for Future Research

The investigation and evaluation of this novel HMI prototype demonstrated a promising solution to reduce MS in AVs and, therefore, to create more inclusivity and acceptance for L4 (and future L5) automated driving. This HMI significantly mitigates MS by allowing passengers to anticipate vehicle behavior. Thus far, only the anticipation of obtuse-angle turns is supported by the HMI. Both SRT and PIT postulated improvements in MS mitigation related to the more accurate anticipation of vehicle dynamics [32,33]. Additional features like specific ambient light cues for 90-degree curves and changes in acceleration could further increase the mitigation effect of the system as the HMI could provide the user with more anticipatory information [54,64]. Additionally, optimizing the HMI further to the perceptual and hedonic needs of potential users considering aspects like cue onset time, cue duration, and other aspects of the stimulus presentation may further improve the efficacy of the system.

With the proven efficacy of anticipation as a mitigation strategy, as well as the replication of MS mitigation effect of other anticipatory HMIs [55,56], the importance of the modulating factor increases. At this point, the pathology of MS and its connection to cognitive processes like anticipation is still not fully understood. Basic research on the pathological mechanisms of MS would improve our understanding of the syndrome and how to more effectively mitigate it.

Finally, once the system itself is optimized, its interaction with other features and information strategies proposed for an HMI utilizing a 360-degree light band needs to be further investigated [60,61,64]. This HMI for MS mitigation is intended to be part of an integrative information system for AVs. Investigating and optimizing its compatibility with other features that provide the user with information will be necessary to tailor the system to the needs of future AV users.

7. Conclusions

In the present study in a realistic AV setting with 16 participants, an HMI utilizing anticipatory ambient light cues for MS mitigation was found to have a mitigating effect on participants with high MS susceptibility. The HMI prototype was developed by utilizing anticipation as the main moderating factor on MS. In the current study, the MS mitigation effect—previously mostly evaluated under simulator conditions—was detected in a realistic L4 driving environment. In addition, increases in user experience, predictability, and perceived safety were associated with the presentation of anticipatory ambient light cues by the HMI. The results of this study support previous findings of the mitigating effect of anticipatory cues on MS and propose a promising first iteration of a potential MS mitigation strategy with a high ecological validity. Future research should focus on potential interactions of the HMI with aspects like stimulus intensity, cognitive workload, compatibility with NDRT, and other on-board HMIs, as well as potential improvements in the fit between user and system.

The study emphasized the importance of creating inclusivity for different populations in regard to using AVs without suffering from MS symptoms. Highly susceptible individuals might be especially in need of support during the adaptation to L4 (and L5). Therefore, developing a countermeasure to MS for AVs is predicted to increase the inclusivity of AVs for all demographics. This may be an important step towards the societal acceptance of AVs.