Data-Driven Scenario Specification for AV–VRU Interactions at Urban Roundabouts

Abstract

Detailed specifications of urban traffic from different perspectives and scales are crucial for understanding and predicting traffic situations from the view of an autonomous vehicle (AV). We suggest a data-driven specification scheme for maneuvers at different design elements of the built infrastructure and focus on urban roundabouts in Germany. Based on real observations, we define classes of maneuvers, interactions and driving strategies for cyclists, pedestrians and motorized vehicles and define a matrix for merging different maneuvers, resulting in more complex interactions. The sequences of these interactions, which partially consist of explicit communications, are extracted from real observations and adapted into microscopic traffic flow simulations. The simulated maneuver sequences are then visualized in 3D environments and experienced by bicycle simulator test subjects. Using trajectory segments (in fictional space) from two conducted simulator studies, we relate the recorded movement patterns of test subjects with observed cyclists in reality.

1. Introduction

Autonomous driving in the city requires the understanding of complex relationships between used infrastructure, traffic rules, performed maneuvers of various road users and the overall contextual assessment of specific traffic situations. As field tests of autonomous driving functions are not always able to achieve the complexity of specific interaction and communication processes at selected urban locations, microscopic traffic flow simulations and driving simulator studies are required to gain further insight into this area. More specifically, the performed simulator studies investigate the perspective of vulnerable road users (VRUs).

We follow up on enriching complex road intersections from OpenStreetMap (OSM) with traffic-related behavioral information [1] and continue with collecting information on test subjects traveling through these modeled virtual-reality (VR) simulation environments.

By using an established system of a physical bicycle simulator [2] that has been evaluated [3] and further developed, we are able to recreate both common and unusual behaviors of various road users from the perspective of VRUs, more precisely cyclists. By extracting gestures from data coming from a stationary depth camera, we are able to reconstruct the sequence of signaling and communication of every bicycle simulator test subject. We simulate road users based on the defined classes resulting from real traffic observations of respective locations and recreate selected real maneuver sequences in a VR environment.

This research focuses on contributing to the understanding of VRU maneuvers from different perspectives, their sequences and the factors influencing their compositions. We focus on a small roundabout, modeled within an artificial bicycle simulator network and on one real roundabout in Germering, Germany. Both roundabouts have the same diameter of the central island.

After we introduce state-of-the-art simulator studies and scenarios between autonomous vehicles (AVs) and VRUs in Section 2, we explain our methodology in Section 3. We conclude in Section 4 with an explanation of preliminary results from the first bicycle simulator study with roundabouts.

2. Bicycle Simulator Studies and Interactions at Roundabouts

Specific studies with driving simulators address the perception of, understanding of and communication with VRUs [4]. Besides evaluating the resulting trajectories of test subjects, qualitative data are also acquired via questionnaires after the test rides.

Besides focusing on road user interaction, there are bicycle simulator studies designed for evaluating novel bicycle infrastructure such as bicycle highways [2] or the perception of countdown timer displays [5].

Other simulator studies with physical bicycle simulators focus on the evaluation of bicycle traffic control [6] and facilities design [7], understanding and evaluating behavioral reactions of cyclists to street design interventions [8,9] and evaluating navigational concepts and perception of infrastructure. In general, we can say that previously investigated bicycle simulator scenarios focus less on the interactions with other road users. Novel approaches for bicycle simulator studies investigate conflicts with other road users [10] or the reactions of cyclists to external HMIs of simulated autonomous vehicles [11].

One example, where very specific interaction patterns occur on the part of the cyclist are roundabouts. Roundabout scenarios are commonly used in driving simulator studies [12,13,14,15], mainly focusing on perception of the vehicle driver approaching the design element. Other research on roundabouts suggests optimal maneuvers for autonomous driving using reinforcement learning [16], adaptive game-theoretic decision making [17] or classification possibilities for driver intentions based on vehicle trajectories resulting from real observations [18,19].

Besides research on calculating roundabout entry capacities [20,21,22], only a few contributions [23] are concerned with interaction and communication between motorized and vulnerable road users. Nevertheless, there are approaches focusing on computer-vision-based techniques for the ego views of vehicle drivers at every entry to the roundabout, which enable predictive models for autonomous driving [24], as well as approaches for simulating road user conflicts and subsequent analysis of safety indicator measures at roundabouts [25,26]. The AV driving strategy starts with roundabout specifications with varying levels of detail in early research [27,28] referring to the detection of roundabout centers, the number of circulatory lanes and of entries.

3. Methodological Approach

One part of our approach is based on the usage of a physical bicycle simulator allowing test subjects to interact with a simulated virtual environment, which is usually based on available geodata of selected locations. Due to frequent exchange with international research facilities and TESIS GmbH, which is part of Vector Informatik GmbH, and the evaluations by test subjects of previous studies [5], we successively improve the immersion and handling of the current bicycle simulator. A specific feature of the system is a functioning interface to the open traffic flow simulation tool SUMO, allowing test subjects to interact with simulated road users. Another advantage of the used bicycle simulator [29,30] is the possibility to connect the sensors to most currently available physical bicycles, which comes together with adjusting virtual bicycle sizes and cycling dynamics within the simulator software.

In our approach, we first define flows for a traffic simulation network that can optionally be a depiction of real-world infrastructure [30] and base on real-world observations at traffic counts or extracted information from video data acquisitions. With the collected data, we can then classify observed maneuvers based on specifying their appearances and trajectory shapes together with variations in speed and their frequencies within the observation period [1]. With these collections, it is possible to design scenarios for bicycle simulator studies [2]. These scenarios can depict infrastructural design elements of the real world or fictional (via VR visualization) [5]. After validating the output data from the simulator studies, which are in most cases test-subject trajectories enriched with behavioral information (as gestures and head movements), we extend the maneuver catalogs. As these catalogs are classified by type of infrastructural design element, every maneuver has a dependency on these. Additionally, the interaction within every maneuver is specified from a bird’s eye view depicting the movements of every road user in the investigation area in an interaction catalog. The driving strategy from an ego perspective is described in a driving strategy catalog.

3.1. The @CITY Scenario Specification Method—An Overview

Based on a specification scheme from the project UR:BAN [31], partners within the publicly-funded, project @CITY have extended the specification methodology for depicting urban built environments (static features), movements of different types of road users, predicted AV trajectories and the view of the passenger within the ego vehicle.

The whole specification procedure by using the @CITY specification methodology consists of detailed descriptions of different catalogs (and specification tables) and workflows for defining scenarios with a high level of detail. For the present work, we focus only on selected aspects of the static branch of the used specification table.

Each scenario has a fixed number of properties, which are described via attributes. In the static branch of the specification table, we have 12 attributes describing each scenario at two levels. This means of the 12 attributes there are 3 attributes consisting of sub-attributes providing depictions of micro-properties. An example of those for the static branch is shown in the Other dynamic attributes which include weather and daylight conditions. These are connected to the specification of the attribute visibility, which might be introduced for human road users and for every sensor attached to a motorized vehicle. Despite the visibilities, there is an important relation between weather and flow rates of VRUs [32]. Another aim of this partial project was to aim to specify in the open standard OpenDRIVE [33], with extension toward different types of landmarks. Therefore, the last attribute of the 12 static attributes is for landmarks and consists of numerous sub-attributes.

In addition to the specification tables, there are phase models, which originate from the research of Gstalter and Fastenmeier [34]. We use this method for including situation-related details only describable from certain perspectives of ego vehicles. The variation of every specified scenario consists of the number of phases within every maneuver of the ego vehicle. This means it might consist only of 3 phases for performing right turning at a non-signalized intersection or of 10 to 12 phases for an urban roundabout with 3 entries. The latter might take into account that the ego vehicle misses exiting and continues on the circulatory roadway until it reaches again the same position.

In addition to these phases, it is possible to define a number of nominal behavior trajectories as representations for options of the ego vehicle (together with specific behavior of other road users), which serve as the usual options of maneuvers during normal conditions: during a weekday with daylight at a time outside the peak and with other road users behaving compliant to traffic rules. This selection of nominal behavior trajectories originates from a selection of decision trees mainly resulting from different types of infrastructure (crossing, roundabout, connection road), where the input information comes from the entries of the specification tables and the phase model for every scenario.

Our idea is to extend these catalogs with behavior trajectories originating from video observations (and later observations from simulator studies) represented as archetypes of maneuver trajectories.

3.2. Traffic Observation and Documentation of Maneuvers and Gestures

In a first step of the approach, we extract trajectory data from videos gathered with installed cameras at fixed locations showing the entirety or parts of a specific design element. We use motion capture and computer vision methodologies for defining the type of road user and computing the respective center points of the detected moving objects in every frame. Due to privacy issues, only the extracted trajectories of classified visible road users are kept for further analyses. This is the base for achieving a detailed representation of the complex behavior of VRUs, the goal of which is to improve the understanding of interactions and communication patterns of VRUs with respect to motorized vehicles in a given static environment.

As stated in previous research [1], we already identified numerous insights from a case study at a small roundabout in Germering, Germany, which is pictured in Figure 1. There are complex interaction and communication patterns between the 3 types of vehicle drivers, bicyclists and pedestrians which are directly dependent on the infrastructural design and the changing visibilities of every road user.

3.3. Designing Data-Driven Specification Catalogs for Autonomous Driving in the City

Due to the highly accurate representations of static and dynamic components, we are able to map and represent a broad selection of possible interactions between different road users. Resulting from one selected trajectory, we can acquire sequences of interactions from the perspective of every visible road user. In the case of roundabouts, we are able to follow up on 5 different categories defined in the German guidelines for the design of roundabouts [35]. We use the extraction techniques for acquiring the spatial dimensions and configurations from digitized road segments of the OpenStreetMap project and enrich the selected design element with observed maneuvers of 3 types of road users.

The methodology for the data-driven maneuver classification relies primarily on the properties of the collected underlying data sets and secondarily on the investigation area and focus of inspection (such as simplifications in classifying road-user types or exclusion of specific road users). In general, we are interested more in the entire trajectories (respective shapes and lengths) of individual road users within the view of a given static sensor (e.g., a pole- or building-mounted camera) and less so in the individual records that they comprise. A precondition for ensuring that entire trajectories are recorded intact is the avoidance of visual obstructions between the sensor and the area of interest. Adjusting sensor placements and fields of view according to these restrictions, however, also restricts the average trajectory lengths and results in trajectories that might cover only a part of complete infrastructural design element (e.g., crossing, T-junction, roundabout or connecting road segment), especially at locations where many visual obstructions are present.

Additionally, the maneuver type classification is dependent on the number of defined road-user classes for every study, for example, whether there is a differentiation between vehicle or bicycle type classes or if there are only the generalized three types of road users such as cyclists, pedestrians and vehicle drivers. One related restriction that is common for trajectory extraction from video observations is the video resolution, which might restrict the possibilities of identifying the type of road user.

A more general factor influencing the manner of data-driven maneuver classification and maneuver archetype definition is the duration of the traffic observation. We introduce a threshold of 6 h for deciding between applying different trajectory clustering techniques (for data sets longer than 6 h) and the manual selection of observed trajectories (for data sets shorter than 6 h). Both approaches lead to the definition of maneuver archetypes with varying reliability of occurrence. Another additional restriction is the temporal resolution, namely the number of frames per second, which can influence the accuracy of the maneuver archetype definition.

We classify, for the case of cyclists, different trajectory shapes together with behavioral information (as attributes) and define typical and highly recurrent maneuvers represented in

Table 1. Possible behavior of cyclists at a small roundabout in Untere Bahnhofstraße, Germering (Germany), with typical and highly recurrent maneuvers, based on [1].

No	Route	Description
10.1		- Typical maneuver - Entering, bypassing and exiting along circular roadway at cycleway
10.2		- Typical maneuver - Entering, bypassing and exiting along circular roadway at roadway
10.3		- Typical maneuver - Entering and bypassing at cycleway - Changing at exit to roadway, since no cycleway present
10.4		- Typical maneuver - Entering and bypassing at roadway - Changing at eastern exit to cycleway, since cycleway present in the north
11.1		- Highly recurrent maneuver - Distance reduction by using the pedestrian crossing - Cyclist changes before crosswalk to sidewalk
11.2		- Highly recurrent maneuver - Changing shortly before southern crossing to pedestrian crossing and then again to cycleway

and unusual maneuvers in

Table 2. Possible behavior of cyclists at a small roundabout in Untere Bahnhofstraße, Germering (Germany), with unusual maneuvers, based on [1].

No	Route	Description
12		- Unusual behavior - Cyclist turns left by driving along circular roadway in the opposite direction
13		- Cyclist rides behind vehicle at pedestrian crossing via eastern road segment - Vehicle stops at bicycle and pedestrian crossing, since circular roadway is occupied
14		- Cyclist traverses southern road segment via pedestrian crossing in opposite direction
15		- Cyclist rides on roadway into roundabout, even though cycleway is present - Changing at western exit to cycleway in southerly direction - Choosing radii with shortest paths
16.1		- Taking a shortcut at eastern exit - Usage of side space for shortest path in easterly direction - Selection of route with least conflict potential with pedestrians
16.2		- Taking a shortcut at western entry - Usage of side space to avoid riding along circular roadway - Usage of sidewalk as shortest path to the south
17		- Cyclist enters cycleway of roundabout in wrong direction - Cyclist avoids forthcoming cyclists by accessing empty sidewalks - Usage of sidewalk in the south for achieving shortest paths

based on an observation of a small roundabout in Germering (Germany). We distinguish between these terms by obtaining the first classification results and segment between those trajectories with similar shapes with high appearance frequencies and those classified as being compliant with the German traffic rules.

In general, cyclists, if present, play an important role in the usual interactions in daily traffic. The defined catalog partition for small roundabouts consists of 13 defined maneuvers in Table 1 and Table 2. It is to state that the maneuvers, ranging from numbers 10.1 to 10.4 in Table 1, are compliant with German traffic rules.

Maneuvers 11.1 and 11.2 in Table 1 can be classified as highly recurrent maneuvers that, in many cases, include misuse of pedestrian infrastructure such as pedestrian crossings or sidewalks. The unusual behavior in Table 2 can be described as part of complex situations that are caused by numerous different road users.

In general, when observing real maneuvers, there is a detectable aim to achieve the shortest path, which comes along with violating assigned space of pedestrians and vehicle drivers.

When specifying cyclist maneuvers for the case of urban roundabouts, we define 5 different bicycle maneuver classes, including traversing the roundabout, usage of sidewalks, usage of pedestrian crossing, traversing in wrong directions and the usage of side space. Depending on the number of types per each class, which ranges from 2 to 4, it is possible to define static and dynamic attributes of the @CITY specification methodology of respective components [1]. We have to state that the collections of abnormal behavior would not cover all possible at one specific intersection or intersection type. The reason for this is, on the one hand, the lack of longer-term video data and, on the other hand, the generally lower probabilities of their appearances. Here, it might be helpful to obtain additional data sets of historical traffic accidents (involving VRUs), which might indicate selected patterns of appearing abnormal behavior for a specific investigation area. Resulting from this fact, the prediction of abnormal behavior remains complex due to the general lack of information on recorded abnormal maneuvers.

3.4. Implementation of Bicycle Simulator Studies

The main reason for implementing physical simulator studies is evaluating the defined maneuver catalogs from real observations. With regards to cyclists, we are able to record head movements, gestures and postural changes of every test subject. Additionally, we can acquire body sizes and classify the style of cycling. The latter is achieved using the speed and steering sensors of the simulator. These sensors are connected to a microcontroller which collects and transmits the data in real time. Based on the two inputs, we also project every movement of the test subject into a position in the VR environment.

3.5. Definition of Application-Specific Driving and Interaction Strategies

The aforementioned maneuver catalog can be extended (in content) and subsequently validated. Recent extensions that are still under rework, especially due to numerous observed sub-classes of observed maneuvers, include non-signalized intersections, static and dynamic bottlenecks and two additional roundabout designs with different category types (“mini roundabout” and “large roundabout”) [35].

The design of a maneuver specification catalog for different types of road users facilitates understanding complex behavior by highly automated vehicles. However, the matching of maneuvers has many complexities, due to variations not only in the shapes of resulting trajectories but also in temporal intervals for maneuvers and episodes that include an unspecified number of maneuvers of interacting road users. Furthermore, depending on the data collection methodology, there might be a specified spatial restriction such as, for example, restricted views from the cameras.

When defining application-specific driving and interaction strategies, we currently document three catalogs:

• A maneuver catalog based on real observations divided into highly recurrent, typical and abnormal maneuvers by cyclists, pedestrians and drivers;
• An interaction catalog with semantics for the specified maneuvers, i.e., logical connections of all tracked road users based on the exact sequence;
• A driving strategy catalog of the autonomous vehicle depending on the predicted interaction processes.

As our approach is fully data-driven, there has to be a testing phase where possible but not yet included behavior will be defined as an extension of the catalogs. This is of manifold importance due to misinterpretation possibilities and the option of having specific gaps during the appearance of selected VRU behavior. For the case of roundabouts, this should be additionally a relation of the different entries and exits connected to the circular roadway. Based on these extensions, a potential AV system should include specified options for all these extensions of the catalogs.

4. Preliminary Results, First Insights and Discussion

In a first study on acquiring gestures from bicycle simulator test subjects, we provided maneuver suggestions via spatial trigger locations with custom triggering functions built on top of TraCI [36]. Test subjects were also instructed at the beginning of the test study to perform behavioral signals as they would do in reality [37]. With 31 test subjects, we tested gesture applications on numerous non-signalized design elements coming from archetypes of German-categorical classified infrastructures [38]. A modeled, but generalized, virtual depiction of the real roundabout in Figure 1 is pictured in Figure 2. The diameter of the circular island is the same as at the roundabout in Germering, and only the entries are adjusted to be symmetrical for complying with the most common German roundabouts as it is named as small roundabout in the German guidelines on designing roundabouts [35].

As pictured in Figure 3a, the TUM-VT bicycle simulator has been extended from its state in 2018 [2]. We extended the setup with a depth camera of the type Intel^® RealSense™ D435, which allows us to record the movements of every test subject. As pictured in Figure 3b, we are able to estimate the skeletal pose in two-dimensional pixel space via the pre-trained PoseNet model [37]. By attaching a helmet marker, we can additionally record the head movements as pictured in Figure 3b. The extracted key points of the skeletons are then classified into several classes of gestures.

Subsequently, we can associate different performed gestures, based on the key points of detected skeletons, with acquired trajectories of every test subject. Previous video observations of real traffic can then be validated on the level of comparing selected maneuvers or on the level of comparing sequences of maneuvers resulting from specific interactions with other road users.

A map view in Figure 4b shows the distribution of trajectory partitions where arm signals were performed by test subjects. The test subjects enter the roundabout from the south and exit it in the north. Five test subjects gave a right arm signal before entering the roundabout, which contradicts the required behavior of German regulations [37].

In general, signaling at roundabouts can differ from country to country; however, no correlation between behavior and the home countries of the test subjects was found. By comparing the roundabout scenario with right-turning scenarios at non-signalized intersections, we see a significant difference: only 50% give a signal at the roundabout, and of those, only around 80% of the test subjects give a signal before turning right at the four-way intersection. This means five test subjects signal before entering the roundabout and perform hand signals against conformity to German traffic rules.

The application of the defined behavior in Table 1 and Table 2 for bicycle simulator studies occurs with explicit instructions for every test subject as no instructed studies would result in low comparability between every simulator run. The relationship between instructions at every scenario and to-be-performed maneuver is implemented via pictograms that are being triggered at specific locations throughout the study for every scenario. Those relate to defined maneuvers in the catalogs at different levels of details, as some of the scenarios are generalized to comply with the most common infrastructural element types in Germany. From the point of view of a bicycle simulator test subject, there have to be explicit instructions for every scenario. Only if this is guaranteed it is possible to reproduce trajectory outcomes that are similar to the archetypes defined in Table 1 and Table 2. These instructions appear at trigger locations and have a high level of detail with the indicated lane, which should be used for performing the respective maneuver. This is a possibility to focus on the selected behaviors in the catalogs in Table 1 and Table 2.

Within the bicycle simulator studies with the roundabouts, we give the test subjects instructions to take the first exit in mixed traffic. This relates to the typical behavior 10.3 and the highly recurrent behavior 11.1. Nevertheless, the roundabout pictured in Figure 2 is a generalization of the real roundabout pictured in Figure 1. Via instructing every bicycle simulator test subject with a cycling task and a respective pictogram, it is possible to restrict the behaviors for the few selected cases in Table 1 and Table 2. Nevertheless, this refers mainly to the maneuver shapes of the listed behaviors and not necessarily to individual velocities. Depending on individual cycling styles of every test subject, there is not only difference in average maneuver speeds but also in the sequences of gestures and head movements.

The testing of a specific roundabout scenario with simulated road users from a bicycle simulator ego perspective serves more as an enrichment procedure for the classified trajectories. The reason for this is that, in our current case, there is often no possibility of extracting skeletal information from the video data due to the long distance of the mounted camera to the roundabout, which is also dependent on the possible video acquisition resolutions of the used video devices. Subsequently, this comes often together with a trade-off of acquiring traffic information at the whole roundabout and discarding information on VRU gestures and head movements and of observing only a part of a roundabout (for example, just one specific entry) and having the option of estimating skeletal information. Additionally, the simulator study shows that via explicit instructions, we can mostly achieve similarly shaped trajectory outcomes. Based on the trajectory shapes and velocities, we try to relate the two trajectory data sets (from video observations and from bicycle simulator test rides) and subsequently provide an enrichment with averaged gestures. This requires a bigger sample size from the bicycle simulator experiments for gaining potential patterns in the communication sequences.

The study participants were university students at the undergraduate and graduate level interested in testing the bicycle simulator, as most of them were passionate urban cyclists. As this study continues, we want to gather data of elder and younger participants due to the motivation of depicting a distribution along most age classes (in relation to other city-wide statistics of Munich, Germany). Nearly all test subjects in the first trial behaved familiar with steering the physical bicycle but showed partially unusual behavior at the roundabouts. This might result from the fact that the test subjects are more urban cyclists and there are only a few roundabouts in the central parts of Munich. Another interpretation of not properly using hand gestures to indicate turnings from the circular roadway might result from the fact that the test subjects cycle for the first time with a bicycle simulator. A larger sample size would deliver more insights concerning the mentioned assumptions.

Relating Test-Subject Trajectory Segments at Roundabouts with Varying Spatial Compositions—The Case of Single and Double Cycle Lanes

Future research will focus on a further investigation of arm signaling patterns. Therefore, we will include two other German categories of roundabouts into the network of an upcoming bicycle simulator study. A potential indicator of the absence of arm signals might come from the distraction of performing a curve on the simulator while planning to execute a signal.

Figure 5 shows another roundabout from Traunstein, Germany, a roundabout with a two-lane circular road. Due to the higher surface areas, cyclists in the VR bicycle simulator environment are confused with its appearance while entering the circular road. Numerous cyclists performed left-arm signaling while entering the entry coming from the east, as well as when exiting the roundabout at the second exit as pictured in Figure 5b.

This shows that local knowledge on the transportation infrastructure is of high importance and misleading signalization via gestures by cyclists can lead to not definite understanding by AV systems.

5. Conclusions

The most significant contribution of the current and ongoing work is the methodological approach of a data-driven specification procedure for complex interactions in urban traffic, which is reproducible in simulation environments. Depending on the fields of views and their variations of the selected road-user type, we can approve different maneuver sequences with test subjects riding or driving with physical simulators.

The suggested methodology of observing, specifying, modeling and simulating maneuvers of different road users allows one to overcome selected restrictions during the traffic data acquisition of real traffic in the field, especially the temporal restrictions, due to the reproduction of a specific traffic situation in a virtual bicycle simulator environment. We can gather trajectories of test subjects for simulated traffic situations that are based on a cutout of an observation or on joint maneuver archetypes.

With these data, we can reconstruct communication processes and understand specific reactions to various simulated events.

While the classification delivers catalogs of similar trajectory archetype classes, it requires an observation of the whole scene from a bird-eye angle to guarantee that specific decision points will be detected. This is of manifold importance as AV sensors might not cover entire trajectories of VRUs due to an obstacle in visibilities from ego perspectives or from positions at the ego vehicle. As the decisions of an AV might depend on specific VRU gestures or head movement, there has to be an extension of only respecting VRU trajectory shapes and velocities.

The testing of a specific roundabout scenario with simulated road users from a bicycle simulator ego perspective serves more as an enrichment procedure for the classified trajectories. The reason for this is that, in our current case, there is often no possibility of extracting skeletal information from the video data due to the long distance of the mounted camera to the roundabout, which is also dependent on the possible video acquisition resolutions of the used video devices. Subsequently, this comes often together with a trade-off of acquiring traffic information at the whole roundabout and discarding information on VRU gestures and head movements and of observing only a part of a roundabout (for example, just one specific entry) and having the option of estimating skeletal information. Additionally, the simulator study shows that, via explicit instructions, we can mostly achieve similarly shaped trajectory outcomes. Based on the trajectory shapes and velocities, we try to relate the two trajectory data sets (from video observations and from bicycle simulator test rides) and subsequently provide an enrichment with averaged gestures. This requires a bigger sample size from the bicycle simulator experiments for gaining potential patterns in the communication sequences.

6. Future Work

Representing real traffic in VRU simulators might be incorporated into a testbed for autonomous driving functions and sensor simulations. Future studies will focus on defining visibility fields of all observed road users, simulated AVs and the dynamic restrictions and obstacles from different fields of view. The reason for this focus is aiming to create an understanding of the abilities of a complex sensor setup of various AVs to obtain information via data matching or extended machine learning techniques. On the other hand, it is important to understand visibilities of VRUs, especially if they are restricted in their communication possibilities (such as children, elder or disabled people). These might be introduced as means of implicit communication strategies. Additionally, it is possible to estimate visibility fields or volumes of every VRU related to respective trajectories. Via the enclosing by static and dynamic obstacles within real and virtual investigation areas, we can obtain probabilities of seeing and not seeing other road users at specific time windows or moments. Understanding of what restricted visibility fields or volumes can cause for which group of road users would be essential for autonomous driving systems or for their functionalities. This might be tested or designed in virtual test environments.

By implementing interconnected simulator studies, we want to observe head movements of test subjects in different vehicles for gaining knowledge on the communication sequences and their relation to specific view angles and visible road users. Here we want to further expand the idea of simulating sensors attached to autonomous vehicles and compare their functions to obtain data used to classify behavioral information of VRUs. Additionally, we want to inspect the influences of sounds, may it be signals or the surrounding sounds within traffic coming from motorized road users.