Cooperation between Roads and Vehicles: Field Validation of a Novel Infrastructure-Based Solution for All Road Users’ Safety

Abstract

Cooperative intelligent transport systems (C-ITS) are expected to considerably influence road safety, traffic efficiency and comfort. Nevertheless, their market penetration is still limited, on the one hand due to the high costs of installation and maintenance of the infrastructures and, on the other hand, due to the price of support automated driving functions. A breakthrough C-ITS technological solution was studied, designed, built and tested that is based on the implementation of custom low-cost on-road platforms (named “strips”) that embed micro/nano sensors, communication technologies and energy harvesting to shift intelligence from the vehicle to the road infrastructure. The strips, through a V2X and LTE communication gateway, transmit real-time, reliable and accurate information at lane level about the environmental and road condition, the traffic and the other road users’ position and speed. The exchanged information supports a series of C-ITS functions and services extending equipped vehicles capabilities and providing similar functions to non-equipped ones (including powered two wheelers). The general framework and the technological solution proposed is presented and the results of the field trials, conducted in three pilot sites around Europe, quantify the promising system performance as well as the positive effects of the C-ITS applications developed and tested on driver/rider’s behavior.

1. Introduction

According to most recent reports [1,2], advanced driver assistance systems (ADAS) showed a maximum penetration rate that was less than 10% in 2015 in Europe. Thus, the impact of cooperative intelligent transport systems (C-ITS) on road safety, traffic efficiency and comfort, is drastically limited. The main reason is the high cost implied, especially due to specific infrastructure required for C-ITS. For instance, a simple virtual message sign (VMS) infrastructure, that is not even cooperative neither personalized, may cost EUR ~24–90 K each. Still, in Europe, critical infrastructure situations are responsible for the 35% of the road accidents [3,4], (i.e., low visibility, unknown road surface conditions). The road surface conditions are particularly relevant in the case of powered two wheelers (PTW) accidents. The Motorcycle Accident In-Depth Study (MAIDS) [5] makes evident that road conditions and wear should be considered in advanced road safety systems. The analysis reports, respectively, that the road surface was described as “wet” in 7.9% of all collected cases (total cases n = 921). Water was classified as a roadway contamination, due to its detrimental effect on PTW handling and braking capabilities. Ice, snow and mud were reported in 2.5% of the cases. Direct surface deterioration or damage was found on 26% of all roadways (i.e., separated asphalt). On the other hand, the variable speed limit (VSL) at intersections/merging yields an accident reduction cost that ranges from 30% to 40% [6]. As seen from a typical VMS application in Sweden, benefits could be shown in terms of safety, traffic efficiency and time gains.

A thorough analysis of the accident statistics [4,7] in terms of numbers, severity of injuries, type of road users, road layout and location, shows that the downward trend, which started in the early years of the second millennium, has slowed down and reached a plateau. Actually, for some road users, numbers have even worsened. Within urban areas, with almost 40% of fatalities, pedestrians account for the largest share of victims. Pedestrian fatalities add up to 12% of cyclists’ fatalities and 18% of users of PTWs to reach the 70% of total fatalities in urban areas for vulnerable road users (VRUs). Outside of urban areas, this percentage is still high (32%). Additionally, road accidents at junctions show a large percentage (about 20%) of total accidents and weather conditions represent a major impact factor. Although there is a significant decrease noticed in motorway accidents in the last decade, in absolute numbers they are still high, which implies the need for safety initiatives in this direction. There are also accident scenarios not covered by specific safety applications. For instance, level crossings (rail crossings) are the most critical parts of the railway’s infrastructure in terms of hazards of collisions between the train and the road users. A significant number of level crossings in Europe (47% of all, [8]) are not provided with any infrastructure to protect it (such as barriers or even traffic lights). The level crossing-related accidents are about 25% of the accidents related to the railways and are quite severe. Similarly, temporary maintenance interventions such as work zones represent a big concern for road safety, particularly at the beginning of roadwork zones and carriageway changeover points where inattention mainly causes rear-end accidents [9,10]. In these critical zones, not only drivers are at risk but also road workers that could be struck by motor vehicles [11]. Additionally, roadworks also have a strong impact on traffic flow, especially affected by how the physical layout changes and the length of the roadwork site [9].

To face the aforementioned challenges, SAFE STRIP H2020 EU funded project [12] proposed a revolutionary C-ITS technological solution. The proposed system is based on custom low-cost on-road platforms, named “strips”, on which micro and nano sensors are deployed. The strips are supported by infrastructure to vehicle (I2V) and vehicle and infrastructure (V2I) communication technologies and energy harvesting. The strips are installed on each lane and they are covered by a road marking paint.

The strips, through a communication gateway, transmit static and dynamic information in real-time. The information relates to road conditions, traffic (for each vehicle passing), and environmental attributes. Equipped and non-equipped (meaning non-C-ITS equipped) vehicles communicate with the strips, that support C-ITS functions with reliable, accurate and lane specific information. Interactions are personalized via the implementation of a negotiations-based human machine interface (HMI). To prove the benefits brought by the “strips” solution, a number of C-ITS safety functions have been developed and tested to address the most safety critical scenarios, aiming to improve the performance of existing applications or even create new ones.

With this innovative solution, the vision was to make roads self-explanatory (thanks to personalized in-vehicle messages) and forgiving (due to the advanced cooperative functions introduced) for any type of vehicle and user (non-C-ITS equipped and C-ITS equipped vehicles, autonomous vehicles including PTW as well as pedestrians). Additionally, all this is with reduced cost and reduced negative environmental impact, supporting, in addition, value-added services and real-time predictive road maintenance functions. The target has been to offer a solution that all vehicles’ generations will benefit from; specifically, C-ITS equipped vehicles are expected to benefit through the upgrade of their intelligence, whilst non-C-ITS equipped vehicles will be enriched with intelligent functions not yet available for them. Last but not least, autonomous vehicles are assisted through the advancement of lane localization and the creation of virtual lanes, corridors, etc. to fulfil the gaps of on-board systems that otherwise require human handovers. While benefits are anticipated for infrastructure operators, drivers and VRUs, the most important contribution of the innovative solution lies in the vision to replace mainstream surveillance systems, VMS and toll stations, resulting in a drastic reduction in installation and maintenance costs with a corresponding increase in traffic safety.

This manuscript presents the innovative technological solution proposed, focusing on presenting the results of field trials conducted in the three European pilot sites of the project across two evaluation rounds regarding the system performance as well as the effects on drivers/riders’ behavior of the C-ITS proof-of-concept applications tested. The manuscript starts by describing, in short, the implementation objectives, the novel solution that has been developed and the driver/rider C-ITS applications that are conceived as proof of concept of the integrated technological solution (Section 2). In Section 3, the validation materials and methods are presented; in Section 4, the system and driver performance results emerging are summarized, which are further discussed in Section 5, to conclude in Section 6 with the recognition of the near future steps and potential extensions of current work.

The approach and results presented in this manuscript are deemed of high importance as, on one hand, they refer to a truly novel C-ITS infrastructure-based solution, but, also, on the other hand, because they may provide valuable reference for future implementation and validation initiatives in the field of C-ITS.

2. The SAFE STRIP Solution

2.1. Implementation Objectives

The integrated solution consists, on one hand, of the infrastructure-based platform and the custom peripheral equipment for sensing, communication, and energy harvesting and, on the other hand, of the driver/rider safety and value-added applications that are accommodated through the infrastructure-based solution in the C-ITS context. The technological platform developed aimed at:

• embedding and transmitting to vehicles updated static info (i.e., speed limit, enhanced map data, asphalt characteristics, curvature, etc.);
• receiving, processing and transmitting to the passing vehicles dynamic personalized info (e.g., from TMC—traffic management center messages);
• measuring dynamic environmental parameters (i.e., temperature, gas, ambient light, humidity, ice) and accurately estimating each vehicle’s dynamic friction coefficient by fusing road sensors data with those coming from vehicles’ intelligent tires;
• sensing passing vehicles, including non-equipped ones, to obtain the speed and lateral position in the lane, transit time and providing basic classification of the vehicle type and, submit key circulation and road load data to the TMC;
• sensing pedestrian crossings, railway crossings work zones, and other critical areas to warn the drivers/riders well ahead of them;
• enabling high accuracy and automatic tolling/parking/insurance practices of low-cost;
• defining and managing lane-level virtual corridors for automated driving.

Section 2.2 below presents, in short, the solution developed, whilst Section 2.3 presents the driver/rider applications that have been also developed in the project to prove the potential of the infrastructure solution, constituting themselves part of the integrated cooperative solution.

2.2. The Technological Cooperative Solution

From an architectural point of view, the platform consists of four interconnected parts, namely the on-road unit (ORU), a road-side unit (RSU) (in the specific system architecture, the naming convention selected, respectively, is road-side bride—RSB), the equipped and non-equipped vehicles with their on-board units (OBUs), being the final recipients of the information originated and classified and the message queuing telemetry transport (MQTT) (MQTT Broker) adopted to deliver the information via the cloud servers to non-equipped vehicles and road operators or service providers (Figure 1).

The core novelty of the solution lies in the custom on-road unit (ORU) developed in the project—the so-called “strip”—which was designed to fulfill three critical requirements: (1) low energy for sustainability for the ORU due to space limitations, (2) low height dimension (for the ORU form) and (3) robustness as the application concerns high criticality. The ORU consists of the following elements [15,16] all hosted and integrated on one single component:

• The sensorial board: it gathers all the data from the various sensing enablers, creates and finally sends in the appropriate format the corresponding message information to the RSU, via Bluetooth low energy (BLE) wireless communication technology. The implemented version integrates a humidity and temperature sensor and a gas sensor, as well as a custom vehicle detection and identification system (VDIS) described below. The gas sensor has been developed in the project and is based on the odorant binding protein (OBP). The sensor is intended to infer the presence of gasoline or lubricants on the street, near the ORU’s location. The key target gas is benzene but other aromatic hydrocarbons (VOCs) are involved. Benzene is found in the emissions from coal, oil and gasoline. Due to the C6H6 molecule’s volatile nature and the high relative weight, this gas stands close to the soil for the ORU’s benefit. The sensing principle is the electrical impedance change using odorant-binding proteins as molecular recognition elements. The environmental data collection (temperature, humidity and gas) takes place every 60 s while an event triggered message is produced whenever the system interacts with a passing vehicle, enabling the VDIS system.
• The vehicle detection and identification system (VDIS): it is also a custom system based on mechanical switches manufactured to detect a vehicle’s (1) position in lane, (2) speed, (3) direction, (4) longitudinal angle of velocity (5) vehicle type (two-wheel, four-wheel, etc.) and also identification through a radio–frequency identification (RFID) technology implementation.
• The energy management module: it is mainly responsible for powering all the individual modules in the ORU by integrating energy harvesters and batteries. Harvesters and batteries (rechargeable and non-rechargeable are supported at the same time) are connected over an energy management board that incorporates the energy management ICs that hierarchically use the available power sources. In particular, energy is consumed, if available, with the following order: (1) directly from the harvesters (any energy surplus is used to charge the batteries), (2) rechargeable batteries and, (3) non-rechargeable batteries. The energy management module can potentially support different configurations regarding the number and type of the energy harvesters (solar, piezo, rf, etc.), different number of batteries per type, external powering and an optional cabling in which two ORUs can share a common energy storage pool in cases that the operational load is not balanced, meaning if one out of two ORUs is overloaded by passing vehicles.
• Strain gauges: a strain gauges module is hosted in the ORU to monitor the strain profile of the pavement through two strain gauges per lane, properly oriented at the right angle to each other. Strain gauges accommodate the road predictive maintenance application that has been developed.
• The encapsulation layer: the encapsulation layer on top of the ORU aims to protect the ORU electronic components, batteries, sensors and antennas from: (1) vehicle loads, (2) temperature extremes, and (3) corrosive agents (gas, oil) and water. The integrated encapsulated unit has a maximum height of 8.5 mm and outer dimensions of 1000 × 500 mm, allowing the placement of cold plastic material of 1.5 mm—that is finally applied for integration on the road infrastructure—leading to a total maximum height of 10 mm of the whole strip from the road surface. Its cross-section has a curved shape to minimize passing vehicles’ discomfort.

The “strip” consists of four discrete parts (Figure 2), hosting the elements described above. Those are installed orthogonally with respect to the road lanes direction, with each lane hosting one strip at the end. A similar but simpler configuration of the strip has been also created and added to accommodate vulnerable road user (VRU) scenarios; in this case, the strip is installed on the road pavement area that precedes the zebra crossing and the sensorial part consists only of a corresponding detection system; this time of the pedestrians.

Overall, it needs to be made clear that the specific solution aims to constitute a low-cost solution that will place the key intelligence and source of safety critical information on the road infrastructure directly and not peripheral to the road infrastructure elements (e.g., cameras and other surveillance systems) exactly to fill in the gaps—if not replace—and increase the reliability of the information that is provided already by the latest as well as the on-board solutions. The proposed solution is operating in any environmental condition (rain, fog, snow), which is not the case for the alternative systems in place, while it also provides accurate lane position, that has been one of the key technical objectives of the solution.

The peripheral equipment refers basically to the communication gateway of the system, the road-side unit (RSU). The RSU is custom-made and it is powered by energy harvesters but also remains connected to the power grid. The specific design favors energy being consumed primarily from harvested energy to sustain the system operations [15]. The RSU is also equipped with an additional multi-sensor, achieving in this way the provision of environmental information related to the existence and density of fog, rain, and the combination of snow and rain; these readings play a critical role in the decision-making system to produce an environmentally related alert message [13,15].

Collected and processed data are finally communicated to the drivers/riders in the form of ITS messages via ITS-G5 technology (Cohda Wireless MK5-RSU). For the vehicles that are not equipped with the corresponding on-board ITS-G5 module, data are transmitted in the format of modified ITS messages through the RSU to a central ITS station (C-ITS-S), operating on the cloud, via long-term evolution (LTE). C-ITS-S stands also as the gateway of the system to the third parties (e.g., TMC and parking operators). The MQTT protocol was selected to provide an interface via LTE for cooperative standard messages to be exchanged among the vehicles, the RSUs and the other users. The same protocol implements an interface for a web service that exchanges cooperative messages with the mobility management centers. A manager of cooperative messages converts data originating from the different control centers into adequate messages ready to be forwarded to vehicles, orchestrating, also, the geo-dispatching of the information and the administration of the message validity such as deleting old messages in the queue.

The OBU in the equipped vehicles embeds all the necessary communication modules and the software stack that pertains to vehicle to everything (V2X) communications and interfaces to connect with the vehicle’s CAN bus system and sensors. The OBU hosts a decision support system (DSS) that produces warning messages shown to the driver through the on-board warning system. For non-equipped vehicles, the decision-making runs on the C-ITS-S.

Whilst the equipped vehicles receive information on-board, non-equipped vehicles rely on a mobile application purposely developed. The mobile application addressing drivers and riders uploads continuously to the C-ITS-S platform a variety of information (e.g., id, coordinates, speed, etc.). The C-ITS-S platform stores the information (for historic purposes) and also processes it to finally publish the results by push back notifications/warnings/recommendations information to the application. This message pushing is made available only to entities registered to this specific notification service (MQTT) of the C-ITS-S platform. Each entity (RSU, DSS, HMI and the Co-Driver that is explained below) is considered for the system as a software module that communicates with each other via the MQTT protocol.

A common personalized human machine interface (HMI) supports in the same way both on-board and mobile configurations of the end-user applications [17]. Finally, a tire pressure monitoring system (TPMS) by the company Continental is also part of the integrated solution.

The proposed hardware platform was iteratively optimized to comply with the initial requirement set. In summary, the expected lifetime of the solution is estimated to last at least 500 days of operation. The RSU and the ORU cover the 20% of the corresponding energy needs from renewable energy sources (RES) and achieve reliable energy provisioning in standard ambient conditions. The system has proved to fully operate in field in an (external) temperature range of −5 °C to 40 °C. The encapsulation and the plastic material applied on top protects the solution from adverse weather conditions (e.g., snow ploughs), mechanical load (˂44,000 kg subdivided by vehicle axes), thermal loads and solvents such as calcium chloride and sodium chloride as well as salt. According to the strictest regulations for lane markings [18], the system protrusion from the paving should not be more than 3 mm. Nevertheless, the prototype system developed and presented herein protrudes 10 mm, due to the selected electronics and batteries’ size. Still, on one hand, other, smaller ones can be selected to lead to a different configuration that will comply with the strictest rules, and, on the other hand, the system in its current configuration can be already applied in countries with more flexible regulations such as Greece, Spain and France, among others. In addition, the proposed strip element cannot be classified as a typical road marking element—for which those regulations apply; it is more a novel C-ITS road element, only interfaced by road markings. The color of the strip paint is similar to the pavement for not being distractive for the drivers/riders. The system does not affect the driving comfort of the driver (noise, bump, etc.), as it is proved by the field results. The compliance with the above requirements implies that the proposed solution is adequate to be installed and operate under realistic environmental and operational conditions.

2.3. The C-ITS Proof of Concept Applications

2.3.1. SAFE STRIP Proof of Concept Applications

The cooperative solution has been implemented to accommodate a series of C-ITS driver/rider applications as well as an application for infrastructure operators that serves as a proof of concept of the solution and to reflect the project Use Cases [19].

The sensorial and communication architecture has been designed to provide both personalized and accurate information about the state of the vehicles and of the road surface conditions. The information is provided at the lane level to improve the performance of existing safety and service applications or to build new ones not possible with the current available technology. One of the main targets of the proposed solution has been to include, in the information exchange, not only the connected vehicles (here named “equipped”) but, potentially, also all other vehicles (named “non-equipped”) and all potential service providers.

In order to maximize interoperability and transparency to future application developers, everything was implemented using standard V2X and limited cost for the user. To this end, the most important messages defined by the V2X communication (i.e., CAM, DENM and MAP) have been mapped into corresponding messages implemented with the MQTT protocol that has been selected to communicate with the non-equipped vehicles over LTE/4G channel. The MQTT messages (or topics) are named “virtual” and are an exact copy of the original messages of the standard V2X communication, as defined by ETSI ITS-G5 or SAE J2735. With this solution, non-equipped vehicles can receive and send information to connected vehicles and cloud services with the RSU that behaves as a gateway, therefore becoming, finally, connected vehicles, even if not in the traditional sense. Therefore, not only the non-equipped vehicles could benefit from applications developed for connected ones, but they can become “visible” road users to connected vehicles, implicitly improving the safety level provided by the applications.

Actually, every vehicle detected by the strips is available in the dynamic local map used by the safety applications, broadening the scenarios they can cover (for instance, intersection support application for connected vehicles will also handle the presence of non-equipped vehicles even those without the application installed in a mobile device).

Additionally, from the developers’ perspective, the solution simplifies the application design and maintenance since a common interface can be adopted both for equipped and non-equipped vehicles as well as for cloud services. Moreover, the same safety application can be hosted in the equipped vehicle or on the cloud, to serve non-equipped vehicles, even if with some limitations due to communication delay and lower update rate of the information.

The lane level layout of the sensorial system and the richness of the sensors embedded in the strips make it possible to collect more accurate and specific, localized, information about the vehicles and the road surface conditions that allows a fine personalization of each application response and functioning and how it communicates to the driver/rider.

A diverse number of applications have been designed in order to prove the benefits and the potential brought by the proposed technological solution, which exploits the available sensorial information and the communication architecture not only to improve already existing functions but also to design new ones. The applications developed are herein gathered and described based on their typology and objectives:

• Road surface condition and friction estimation functions that, thanks to the sensors embedded in the strips installed on the road surface and the cooperation with vehicles mounting intelligent tires, can estimate the integrity of the road to be used by maintenance prediction programs of road operators and the actual friction conditions of a specific road section to be exploited by the ADAS and advanced rider assistance systems (ARAS) functions.
• Autonomous vehicle support applications to create virtual corridors in case of a missing lane marking or clear route or lane to follow to reach toll gates or navigate through a road works area. Situations that may confuse autonomous vehicles and require human takeovers.
• Supportive added value services to implement “virtual toll” and “parking service” applications (“virtual toll collection application” and “parking booking and charging”, respectively). The first one is a new application that allows one to remove the toll gate infrastructure and tries to harmonize and simplify, with a limited cost, the automatic payment system for all classes of vehicles. The second one is an example of how the information provided by the strip can be integrated into an already existing service, to manage parking requests and corresponding transactions, and improve the efficiency and user satisfaction.
• A VMS application with its two counterparts—“In-vehicle VMS and Traffic Center Information” and “Mobile VMS and Traffic Centre Information”—that offers lane level personalized VMS services to the road users without the need of real VMS infrastructure installed on the road infrastructure with a great reduction in installation and maintenance costs and personalized series of information referring to the specific profile of the user, their location and driving intentions.
• C-ITS safety applications, which implement advanced driving support functions in a cooperative framework for a variety of scenarios such as intersection and merging, road works, unprotected railway crossings, crosswalks and wrong way driving. Some of the applications are an improvement of already existing ones, benefitting from more detailed and accurate information provided by the on-road sensorial system. Others are almost new applications that are possible thanks to the sensorial and cooperative framework proposed. The applications are designed both for equipped and non-equipped vehicles in a seamless way and in a way to consider the information of the actual friction of the road surface and the local dynamic map built from the information provided by the solution [20]. Additionally, a unifying underling working principle (named the Co-Driver concept [13]) is used by all safety applications to evaluate, with a common metric, the warning level across all scenarios. The unifying approach is also pursuit through the common HMI designed to visualize the information to the driver/rider and rate in a homogenous way the different danger causes (see Section 3.2.2).

As mentioned above, all information provided in output by the above applications or services are prioritized by a DSS and visualized through a common HMI (on mobile device or on-board display). The DSS module manages and prioritizes warnings and information produced by all applications co-running in the same vehicle or in the user mobile device in order to minimize as much as possible the level of distraction and overloading of the driver/rider which constitutes one of the negative possible side effects of intelligent systems during the on-trip use. Apart from the autonomous functions (that follow their own perception and decision making) and the road predictive maintenance application for infrastructure operators, all applications are subject to the central DSS and share the same parameterized HMI strategy.

The present manuscript focuses on the performance results and effect on driver behavior only for the VMS application and the C-ITS safety applications, that are further discussed in the sections following. All the other applications results are out of the scope of the article. Nevertheless, the number of applications herein analyzed is still high (eight out of twelve in total). The next subsections describe in detail the structure and concepts of the VMS and C-ITS applications.

2.3.2. VMS Application

The VMS application aims to exploit the benefits of the installation of strips and RSU infrastructure, in order to produce accurate VMS messages specific to current on-site conditions (even at lane level) and to convey these messages to the proper destination vehicle, regardless of if it is an equipped or a non-equipped one. The main purpose of the application is to “alert” a vehicle driving in the area of interest when there is an on-going special condition and to clear-up this warning if and when the initial critical condition ceases to exist. In general, the application is a service that is “running” on the cloud and keeps an up-to-date image of the prevailing environmental conditions surrounding the site area by listening to environmental data coming from that site. In addition, the VMS also listens for vehicle detection and identification messages and when a vehicle is detected in an area where there is currently an active condition, a warning is issued destined for that particular vehicle. By learning about successfully detected vehicles, VMS application also obtains knowledge about the traffic conditions at the site of interest, therefore creating the opportunity for messages regarding not only environmental issues, but traffic congestion conditions also. In order to evaluate the usability of the VMS applications, three VMS-relevant scenarios were tested, namely: “low visibility” (based on sensor information from the RSU), “liquid on road” and “heavy traffic conditions” (based on sensor information and vehicles detection from the strips).

2.3.3. C-ITS Applications

All safety applications are based on the same common Co-Driver concept [20], which is a virtual agent used to understand what the driver is doing with respect to the potential incoming danger (e.g., pedestrian on cross walk, wrong way driving vehicle, etc.) and issuing a warning if a safe maneuver is not available to achieve the goal required by the driving scenario (e.g., stopping at a crosswalk). The underling idea is that a human driver/rider can handle an incoming danger or a critical situation selecting among a number of feasible maneuvers the one that best fits his/her final goal associated with the type of danger (for instance, stopping at a shorter distance than a pedestrian crossing, or passing an intersection at different speeds) [13]. It is assumed that the criterion used for the maneuver selection is rating the possible maneuvers based on the effort necessary to change the current vehicle state (i.e., acceleration, speed and location) to end with the final desired safe state (i.e., the goal of the maneuver). The maneuver with the minimum effort is the one that most probably the human will select [21]. A physical indicator of the effort needed to change the vehicle dynamics is the rate of the change of longitudinal acceleration (i.e., the jerk) at the initial phase of the maneuver. It can be demonstrated that the initial jerk can be directly mapped to the driver/rider’s rate of change of gas/brake pedal [22] and, thus, the necessary force. Therefore, the initial jerk of the maneuver assumes the meaning of common metric space to evaluate all longitudinal maneuvers (independently from the type of danger).

Thus, the above process can be distinguished in two phases: maneuver generation with respect to the type of danger and associated goal and the action selection. The Co-Driver algorithm is based on the above concept and is composed of two hierarchical layers [23,24]. The outer layer depends on the specific danger to be considered (e.g., road works, intersections, etc.). The outer layer objective is the interpretation of the specific scenario trying to estimate the behavior of other actors (i.e., vehicles and road users) to set the safe maneuver goals. This is carried out by using the geometry of the road, the right of way rules to be satisfied when interacting with other vehicles, estimating the intentions of the other vehicles. The inner layer is dedicated to the generation of human-like longitudinal maneuvers consistent with the situation derived from the higher layer. In essence, they represent predictions of possible maneuvers compatible with the goal of the specific scenario (e.g., stop at a pedestrian crossing). The resulting maneuver (i.e., speed and acceleration profiles) depends on the actual vehicle state and the desired goal (i.e., final vehicle state). All generated maneuvers are then rated based on the initial jerk to form the metric space of the specific scenario. The metric space is partitioned into three zones, defined by two thresholds. If at least one of the computed maneuvers has the initial jerk in the first zone (above the first threshold), it is assumed that the driver/rider has at least the chance to safely and conformably stop, and no warning is provided (level 0: only informative). If all the feasible maneuvers have the initial jerk below the first threshold, and above the second, a yellow (level 1: medium severity) warning is issued, otherwise the red one (level 2: high severity) is produced. It is interesting to mention that those thresholds can be also associated with the probability that the driver chooses one of the maneuvers in the computed set. The concept has its foundations in biology, where in the “basal ganglia” brain structure of vertebrates, a selection phase of all maneuvers is carried out based on the metric space of motor actions [13,21].

From the implementation point of view, a basic Co-Driver object class with the above-described structure is specialized for each considered scenario via the inheritance mechanism to implement each application with all the necessary properties and methods. Those are namely the “mobile cooperative safety function” and the “in-vehicle cooperative function” (for non-equipped and equipped vehicles, respectively), the “mobile application for rail crossing and road works safety function” and the “in-vehicle application for rail crossing and road works safety function” and the “mobile application for merging and intersection support” and the “in-vehicle application for merging and intersection support”. Then, a specialized Co-Driver object is dynamically instantiated at runtime for each vehicle that has to be “protected” by the safety application. In the case of the equipped vehicle, there will be one Co-Driver in the main program of the OBU while for the non-equipped vehicles, a Co-Driver for each vehicle that has subscribed to the service will be instantiated in the cloud server application. The latter solution allows one to monitor all the vehicles that enter in the zone of interest that have the application installed in their portable device or infotainment system to create a full cooperative safety application both for equipped and non-equipped vehicles.

3. Materials and Methods

3.1. Evaluation Framework

An iterative implementation and validation plan was developed, consisting of four (4) successive rounds [25]. The first two validation rounds focused on technical validation on the component level of the solution. Due to the novelty of the solution, the performance of each component was assessed through various subsequent steps with increasing levels of integration before moving to the validation of the integrated solution. An enhanced and optimized version of the solution was derived based on the outcomes of the first two testing iterations. In the third validation round, and after the final technical verification of the integrated system on infrastructure, application, communication and demonstrator ends, the first round of user trials took place. The following fourth round corresponded to the final real-life trials of the project. The current manuscript focuses on the results emerging with regard to system performance and driver behavior in the user trials held in the third and the final round of the iterative evaluation scheme.

The trials with users held can be seen as small-scale field operational trials (FOTs) in a controlled environment, as although two out of the three pilot sites were motorways, they were closed to/separated from real traffic for safety reasons. The evaluation framework and experimental plans were based on the FESTA methodology (see FESTA handbook [26]) and the CONVERGE (TR 1101) methodology, as adapted and extended within the INSAFES project [27].

The focal point of the validation phase was the performance evaluation of the developed functions/applications, as described in Section 2.3, to prove the potential of the whole cooperative solution that has been implemented.

The research questions/evaluation objectives were determined, further mapped to the developed functions under assessment as well as the key performance indicators (KPIs), the testing rounds and the involved user groups. The real-life experimental scenarios were structured in a stepwise format for each proof-of-concept application (on the basis of the project Use Cases) and the experimental conditions were defined (see Section 3.6). Specific hypotheses that corresponded to KPIs for each function were recognized, along with the metrics and the measuring tools for their assessment. Legal, ethical and data management principles governed the trials [13].

The stakeholder chain of the proposed solution includes road users of all types (pedestrians, riders and drivers), OEMs, Tier 1 suppliers but also authorities and infrastructure operators. The latest are the main target of the proposed infrastructure-based cooperative solution as it is a cost-efficient solution that primarily aims to positively impact traffic efficiency and the environmental status of the traffic network and the overall safety. Therefore, in the last round of real-life field testing, those stakeholder clusters were represented directly or indirectly.

Among the 13 research questions (RQs) determined, the ones that constitute the objective of the current manuscript are as follows:

RQ1: Does the system work as expected under realistic conditions (in compliance with specifications)?

RQ2: What are the impacts on safety and mobility?

All research questions were rendered in desired effects for all the different stakeholders in the short and medium/long term and complemented with a list of unintended effects for all stakeholder clusters, being a novel technological solution recognized following the Michon [28] layers (strategic, tactical and operational) and correlations. The desired and undesired effects guided the generation of a specific experimental hypothesis that is described per experimental scenario. The most significant intended effects are the ones that yield an increase in traffic safety, “equity” in roads and the reduction in manufacturing, operational and maintenance cost for operators. Excessive information/warning workload and overconfidence of road users due to overreliance to the system during driving are the most important unintended effects. Others worthy of mention are non-smooth interaction with other vehicles of surrounding traffic, especially due to the creation of new lanes/corridors for automated vehicles and potential changes in the business routines of infrastructure operators [13].

3.2. Demonstrators and Terminals

3.2.1. Vehicle Demonstrators

The goal of the project was to widely test the solution and its general applicability as much as possible. Hence, the project tackled different types of vehicle demonstrators: passenger cars and PTWs equipped with ETSI ITS-G5, autonomous cars and finally drivers with only smartphone applications available driving non-equipped vehicles and PTWs where cellular connectivity was available. Primarily, the demonstrators were divided in two categories, namely equipped and non-equipped.

Equipped demonstrators can exchange messages between themselves and infrastructure via V2X communication. A dedicated V2X OBU was equipped in the vehicle in compliance to the IEEE 802.11p standard (ETSI ITS-G5 or SAE J2735). The demonstrators received V2X messages (CAM, DENM and MAP) and forwarded them to the local Co-Driver-based application through a local MQTT service. A common library was adopted in order to encode and decode the messages between all the components (the V2X board, the application module, the DSS and the HMI). The V2X board also generated a CAM message for the ego-vehicle and published the relevant content (CAN plus GNSS content) to the Co-Driver module in the same format and frequency of the other V2X messages. In such a way, the Co-Driver could correlate the dynamic of the ego-vehicle as it was an additional V2X station. All the messages in the equipped vehicles were published at 10 Hz following the recommendations of the ITS-G5 standard.

Non-equipped vehicles exploited the very same library, but in this case the application evaluating the risks ran on a cloud service. The communication with the remote Co-Driver was based on the remote MQTT service with the broker running on the cloud. The Co-Driver-based application collected the information from all the non-equipped vehicles subscribed to the service and, after fusing with data coming from the road sensors, injected notifications and warnings only to the vehicles that require a corrective maneuver. In such a way, it was possible to minimize the power consumption and the computational power required by the personal devices of the users and fully exploit the cooperative exchange of information. The cellular connectivity was based on point-to-point communication (in contrast to the IEEE 802.11p broadcast communication); therefore it is more convenient that all the vehicles communicate with a centralized server rather than enabling the direct communication between vehicles. The main drawback of this approach is the communication delay: all the messages were collected at 1 Hz in order to avoid message congestion due to the cellular communication channel. For non-equipped vehicles, no other integration with the vehicle architecture took place; all was needed was the smartphone application and the RFID tag to apply to the front bumper. An extended version of the smartphone application was implemented to include additional vehicle information such as vehicle acceleration, lean angle, throttle position and other information coming from the in-vehicle sensors. This was carried out by integrating the Piaggio Multimedia Platform, which makes motorcycle data available via BLE communication.

It is stressed that the very same information and protocols were used for both the equipped and non-equipped applications, aiming to provide the same experience to both types of users. The only difference was the update frequencies of the messages that impact on the overall performances of the applications in the vehicles.

Three (3) equipped demonstrators were deployed for field trials, namely two passenger cars, a Fiat 500L (provided by CRF) and Lancia Thesis (provided by CERTH/HIT; Figure 3), and one PTW, Piaggio MP3 (Figure 4). Four (4) non-equipped vehicles also participated in the field trials: two passenger cars, a Fiat 500X and Lancia Thesis (acting in this case as non-equipped) and three (3) PWTs, namely two Piaggio MP3s (one provided by CERTH/HIT—same as before, acting as non-equipped in this case—and one provided by PIAGGIO) and one Piaggio Beverly 300, provided by PIAGGIO. In addition to the internal vehicle demonstrators, a series of other passenger cars and PTWs participated in the Greek and Spanish trials as “non-equipped” vehicles.

3.2.2. The Driver/Rider Application

The mobile application receives the data presented to the driver/rider by the applications running on the back-end through the DSS. The applications providing data to the mobile application are the Cooperative Safety, the Road Works Safety and Rail Crossing, the Merging and Intersection Support, the VMS/VDS, the Virtual Toll and the Parking applications. As already mentioned, the communication between the mobile application and the back-end of the system is realized with the use of the MQTT protocol.

An Android [29] and an iOS version of the application (downloadable by Apple Store typing “SafeStrip”) were developed in each case. The installation process in both operating systems is quite simple, whilst English, Spanish and Italian languages are supported in both.

When the user launches the application, the user is required to fill in the vehicle station id, the application of interest, his/her name and the password (Figure 5). The password is used to connect safely to the MQTT server, avoiding unauthorized connections. From the menu, the user may select which of the application(s) that they wish to be active, as concurrent running of applications has been anticipated. The possibility to change the active application status gives to the user the control on the desired output, considering that only for the active applications, the mobile application will display the warnings receiving. When an event occurs, the mobile application displays the appropriate icons, textual messages and audio messages (Figure 6). The textual message contains detailed information about the event. However, in most cases the driver must not be visually distracted during driving. Thus, the mobile application also provides a supporting audio message in order to inform the driver. This feature has been crucial, specifically for PTW riders. Specifically, for road safety reasons, while the same HMI is also used for riders, its application is quite different. The TTS application runs simultaneously to read the textual messages that pop-up in the different occasions for the rider, while headsets are used for the sounds and the TTS output to eliminate the effect of the ambient noise the rider is exposed to. Still, among others, the user can enable or disable the sound and text-to-speech (TTS) operations through the application’s settings area (Figure 5).

The mobile application was designed and implemented for vertical and horizontal orientation of the device; the manufacturers of the equipped vehicles can use the vertical/horizontal orientation according to the orientation of the in-vehicle monitor (Figure 7). The Android mobile application was designed and implemented according to the Web Content Accessibility Guidelines (WCAG) 2.0 [30] and WCAG 2.1 [31], which have been adapted to mobile applications. For the iPhone application, the respective accessibility guidelines were followed [32,33]. Further details on the development of the mobile applications and the HMI are out of the scope of the current manuscript.

3.3. SAFE STRIP Mobile Test-Bed and Test Sites Set-Up

The field trials were conducted in one closed test track in CIDAUT premises in Spain and two motorways in real traffic conditions, namely on the A22 motorway in Trento, Italy and on the Attiki Odos motorway, in Athens, Greece. Field trials lasted from December 2019 to August 2020 (facing COVID-19 in between).

The infrastructure-based solution manufactured was transferred back and forth to the test sites for the two phases of the field trials, as shown in Figure 8. We started from Athens in ATTD (beneficiary running the Attiki Odos motorway), then moved to Trento in Italy for the field trials on the A22 motorway, then to Spain in CIDAUT premises (after an interruption due to COVID-19), then to Thessaloniki premises of CERTH/HIT for fine-tuning, and, finally to Athens, ATTD and back to Thessaloniki to execute the intersection scenario in a closed road segment (in the parking lot of the shopping center “Praktiker Hellas”).

The vehicle demonstrators provided by CRF and PIAGGIO were deployed for the field trials on the A22, whilst the CERTH/HIT demonstrators moved back and forth from Spain to Greece for the field trials taking place in both countries.

For the pilot needs in Italy, Spain and Greece, different approaches for the ORUs and the powering of the RSUs were adopted, in order to reduce the equipment volume during transport, to allow fix interventions in failure cases, and to minimize potential errors by applying simpler solutions. To this end, the electronic parts of the ORU that initially were placed inside the strip were moved in a small box (ORU box) located next to the strips at the roadside and a compact version of the RSU powering pillar was provided. Hence, for the three pilot locations and the different application scenarios, all the integrated platform elements were constructed and travelled back and forth.

A specific set-up of the anticipated testing areas and the creation of system topologies was required prior to the tests. Those topologies were prepared initially on the demonstration scenario level (see Figure 9 and Figure 10 and

Table 1. Parameters for VRU protection experimental scenario topology (for all topologies that follow, 200 m were anticipated in advance in addition for acceleration and speed stabilization and 100 m at the end for deceleration and stopping).

Parameters	Units
Vehicle velocity	50 km/h for dry road 40 km/h for wet road
Number of Strips	4
Number of RSUs	1–2
Number of Lanes	1
Lane index	1
A	32 m
B	34 m
C	34 m

below as an example) and then clustered for each test site as a whole, based on the demonstration scenarios that were implemented and complied with the following principles:

• Execution of field trials with 100% safety and reliability;
• The greatest possible proximity to the original scope;
• Minimization of the field trials’ duration and required resources (number of strips and RSUs);
• Minimization of installation and integration effort;
• Minimization of travel/mobilization costs.

The lane width was about 3.5 m in all test sites. Lane markings were present or not, depending on the scenario. For instance, lane markings had to be covered at some points for the evaluation of the automated functions. In addition, a full installation and operation manual was prepared for the whole of the integrated solution.

Attiki Odos in Greece is a 70 km long urban motorway, bi-directionally separated by dual carriageways of three lanes each, for the majority of their length, and an emergency lane (hard shoulder). The location that was chosen was between the P10.6 and P11.1 km of the Imittos Western Peripheral Motorway, in the direction towards Rafina, just before the Y8 (Pallini) exit (Figure 10). The designated area was closed to traffic (right lane and emergency lane) using traffic cones to separate the trials lanes from the traffic. Proper signage and VMS notifications for speed reduction were also in operation, while a press release provided wide-spread information to the motorway users regarding the tests at the specific location.

The A22 has a length of 314 km and connects Brennero (km 0) to Modena (km 314), passing through four regions: Trentino—Alto Adige/South Tyrol, Veneto, Lombardy and Emilia Romagna. The field tests in Italy occurred within a closed section of the A22 motorway, once used as a motorway exit, named Trento Centre (Figure 11). Nowadays, the exit—that is adjacent to the correspondent fully operational entry—is permanently closed to traffic. All trials were conducted on this stretch of motorway. Thus, although performed within a real and adequately equipped motorway environment, the tests did not have to deal with traffic-related dangers.

The Spanish trials were carried out at CIDAUT’s facilities in Spain and, particularly, in the locality of Mojados in the province of Valladolid (Figure 12). CIDAUT’s test track is a restricted access area dedicated for vehicle and infrastructure testing. The test site consists of two main roads with one km length each, which cross each other at 16°. In addition, there is a service road interconnecting all the track and making the movement around the roads easy. The Spanish trials employed the central road.

Finally, after the latest trials in ATTD were conducted, all the system parts were again transferred to Thessaloniki in order to test the intersection scenario that required an urban-line context and, actually, a specific setting (the highway context of ATTD was not applicable for this reason). For the sake of this scenario conduct, the parking lot of the shopping center “Praktiker Hellas” was used (Figure 13).

3.4. Test Subjects, Ethics and Data Management Issues

The number of drivers, riders and infrastructure operators that tested the safety and added value applications in the two rounds of user trials across the different test sites were as follows:

• A22 motorway, Italy: 12 drivers, 3 riders;
• CIDAUT closed test track, Spain: 10 drivers, 6 riders, 4 infrastructure operators;
• ATTIKI ODOS motorway, Greece: 20 drivers, 10 riders, 4 infrastructure operators;
• Thessaloniki test site (closed road segment): 11 drivers.

The drivers, riders and infrastructure operators who participated in the trials were recruited from the personnel of the participating entities. In addition to that, due to the safety criticality and novelty of the solution, the field trials were decided not to be open to external participants. Therefore, drivers and riders were either professional ones or had had a driving license more than 10 years. Specifically, 80% of the drivers participating were male, 35% of them had been driving for less than 10 years, 37% between 11 and 20 years, 16% between 21 and 30 years, while 12% more than 30 years. Moreover, 2% drove less than 5000 km per year, 17% drove between 5000 and 10,000 km per year, 43% drove between 10,000 and 15,000 per year, 8% 15,000 to 20,000 km, 14% 20,000 to 25,000 km, 2% 25,000 to 30,000 km, 12% more than 30,000 km and 2% stated “don’t know”. Most participants drove almost every day in the city and less often on the highway and in rural areas. A percentage of 95% of the riders participating were male. A total of 11% drove less than 5000 km per year, 16% drove between 5000 and 10,000 km per year, 26% drove between 10,000 and 15,000 per year, 5% 15,000 to 20,000 km, 11% 20,000 to 25,000 km, 5% 25,000 to 30,000 km and 26% more than 30,000 km. Most participants drove almost every day in the city and less often on the highway and in rural areas. The participating drivers’ and riders’ insurance was covered either by their business contracts or by ad hoc insurance contracts. Half of the operators participating in the trials were civil engineers who work in the operation and maintenance of highways and the other half of them were employees working in the same field. Their average professional experience was 13.7 years. The operators did not run traffic scenarios; they monitored the trials, and they rated the user acceptance through subjective monitoring tools, the results of which are out of the scope of the current manuscript.

The users participating in the trials were “blind to the project”, meaning that they were involved in the definition of tests’ objectives, design and procedure but not directly involved in the project implementation. All participants were required to be experienced and active drivers. Additionally, participants were selected among those that had a normal or corrected vision, no psychiatric or substance abuse medical history and at least moderate experience with advanced in-vehicle systems and average ICT literacy. A specific ethics code of conduct was established from the beginning of the project and also followed, and primarily for the field trials, an encompassing signature of informed consent forms of the participants, the application of a specifically designed ethics controlling process, adherence to general data protection regulation (GDPR) processes, confidentiality and anonymization of data collected and approvals by Ethics Committees whenever applicable [13].

3.5. Measuring Tools

A series of subjective and logging tools were developed and applied during the trials. In order to log the metrics that were determined for system and driver performance, which is the current manuscript objective, specific ITS log modules (ITS-LM) were implemented in each main end of the system (i.e., mobile devices, vehicles, infrastructure, the cloud) upon a common structure that allowed a consistent processing and consolidation of the test results analysis phase. Driver performance data were logged either on-board or on mobile terminals (for equipped and non-equipped vehicles, respectively). System performance data, on the other hand, were logged in different phases of the scenario for each unit of the system, namely in the RSU, in the C-ITS-S and the application servers that communicate with it, in vehicles’ OBUs and in mobile terminals.

To guarantee an accurate time synchronization for the logging mechanism, the GPS timestamp was selected as the time reference for all the ITS devices and modules having access to GPS signal. For the modules without GPS reference, the network time protocol (NTP) synchronization mechanism was used. The logging mechanism was designed in a way to log only the data that were useful for the evaluation process, discarding the redundant ones. Data were stored in encoded (binary) format, to reduce the filing overhead. The logging was triggered prior to each single experiment when the user pressed the START button in the HMI application. At the start event, for each test, the scenario ID, the user ID running the test and the station ID (i.e., the vehicle driven by the user) were recorded with time stamps in order to ease the post processing of the logging files.

Every log file contained entries with information data combined with a UTC (Coordinated Universal Time) timestamp (in the form of milliseconds since 1 January 1970—Unix epoch) that marks the exact time of the corresponding message logging.

A specific logging service was developed, connected to the MQTT broker to log all relevant/subscribed topics transferred via MQTT messages. The RSU was responsible for logging all ORU transmitted messages since the ORU devices did not have the processing/storage capabilities to do that. The HMI application was responsible for logging all its received messages (i.e., warnings). In addition, and to complement the logging mechanism, event diaries were also kept by the test conductors.

3.6. Experimental Plans

The full experimental plans are out of the scope of the current paper; only the part relevant to the specific objective of the manuscript is presented.

3.6.1. Evaluation Scenarios

The current manuscript focuses on the field trials that aimed to assess the potential of the VMS and C-ITS safety functions, as presented in Section 2.3.2 and Section 2.3.3. Specific test scenarios were built for the assessment of each application, as shown in the following table. In reality, those scenarios were structured in a stepwise format, where user–system interaction is described in each; still, herein, only a summary is provided (

Table 2. Test scenarios and experimental conditions.

Function	Test Scenario	Short Description
Mobile cooperative safety function	ES1.1: Virtual VRU protection	Alerting the driver about pedestrian prompt to cross the zebra crossing with stopped vehicle.
	ES1.2: Wrong way driving	Alerting the driver about wrong way driving vehicle ahead.
In-vehicle cooperative safety function	ES2.1: Virtual VRU protection	As above. However, in this case, the terminal is the on-board terminal and not the mobile application.
	ES2.2: Wrong way driving
In-vehicle application for rail crossing and road works safety function	ES4.1: Work zone detection	Alerting the driver about work zone ahead.
Mobile application for rail crossing and road works safety function	ES5.1: Work zone detection	As above. However, in this case, the terminal is the mobile application and not the on-board terminal.
In-vehicle application for merging and intersection support (e2Call)	ES6.1: Urban intersection	Alerting the driver of intersection ahead and incoming vehicles from all possible directions.
Mobile application for merging and intersection support (e2Call)	ES7.1: Urban intersection	As above. However, in this case, the terminal is the mobile application and not the on-board terminal.
	ES7.2: Motorway exit	Alerting the driver of not visible motorway exit ahead.
In-vehicle application for personalized VMS and traffic center information	ES8.1: Heavy traffic detection	Alerting the driver about the presence of heavy traffic ahead.
	ES8.2: Fuel/oil on the road	Alerting the driver of the presence of dangerous fluids (typically oil or fuel) on the road (simulated for safety reasons).
	ES8.3: Low visibility scenario	Alerting the driver of low visibility ahead.
Mobile application for personalized VMS and traffic center information	ES9.1: Heavy traffic detection ES9.2: Fuel/oil on the road ES9.3: Low visibility scenario	As above. However, in this case, the terminal is the mobile application and not the on-board terminal.

).

3.6.2. KPIs and Metrics

The KPIs, testing hypotheses, metrics and success targets that correspond to each research question are depicted in the following tables (

Table 3. KPIs, hypotheses, metrics and success targets for RQ1 (Section 3.1).

KPIs	Testing Hypotheses	Metrics	Success Targets
False/missed alarms	The function issues the warning correctly.	Detection of pedestrian. Detection of vehicle (non-equipped) Warning issued.	<2–3% false alarms <2–3% false negative
Communication time (overall)	The system will detect the pedestrian in due time and acknowledge it in time to the user. The system will detect the vehicle in due time.	Timestamp of pedestrian and vehicle detection. Timestamp of HMI warning received by the terminal.	1 s overall
Accuracy of vehicle positioning	Distance from the zebra crossing is accurate.	Vehicle position (mean and std).	<5 m
Accuracy of vehicle speed	Speed of ego-vehicle.	Ego vehicle speed (mean and std).	<1 m/s
Correctness/accuracy/reliability of warnings (in terms of content and time)	The driver has enough time to react and in the correct way.	Maneuver time. Time to collision @ warning (mean and std). Deceleration (mean and std). Reaction time (sec).	>3 s >3 s 1.2 m/s2 1 < 1.5 s

,

Table 4. KPIs, hypotheses, metrics and success targets for RQ2 (Section 3.1).

KPIs	Testing Hypotheses	Metrics	Success Targets
Driving performance/behaviour	The driver is expected to decelerate as soon as the warning is dispatched as long as the warning is active. Expected deceleration has to be compatible with comfort values.	Deceleration (mean, std) during warning period. Reaction time @ warning.	<1.2 m/s2 <1.2 s
Readiness to unexpected traffic incidents	The user is efficiently supported by the system to react to unexpected traffic incidents in the safest possible way.	Acceleration (stream). Velocity (stream).	The user stops smoothly before the collision point Deceleration < 1.2 m/s2
Impacts on “non-users” (i.e., VRU)	No near-end collisions with VRUs.	Position.	No collision

).

The frequency of logging was determined to be 10 Hz.

Two of the most important metrics listed above are the time to collision (TTC) that represents the time that it takes to the vehicle to cover, at constant speed, the distance from the actual position, when the warning is first issued, to the danger ahead. This indicator is used to quantify if the warning anticipation is long enough to take a comfortable and safe corrective action.

$$TTC=\frac{v\left(t_0\right)}{s\left(t_0\right)}$$

(1)

where $t_0$ is the time instant at which the first warning is issued (red or yellow), $v$ is the actual speed and $s$ is the distance to the danger. For the TTC indicator, a threshold of 3 s has been defined as a minimum value of anticipation.

The second indicator reported is the average acceleration (AA) (evaluated only in terms of deceleration to correct the actual speed) to measure the smoothness of the driver response. It is calculated as:

$$a_a=\frac{v\left(t_0\right)-v_{end}\left(t_e\right)}{T_m}$$

(2)

where $t_e$ is the time when the maneuver ends, i.e., the vehicle passes the obstacle/danger location (if possible), or the vehicle stops (or reach a low velocity), $v_{end}$ is the speed at time $t_e$ and $T_m=t_e-t_0$ is the maneuvering time. Low average acceleration means that the system warns the driver with a sufficient anticipation in order give him/her enough time to execute a smooth and comfortable safe maneuver to comply with the danger. This usually has an impact to the general traffic safety and mobility, avoiding late and abrupt response that may cause rear-end collisions. For the AA indicator, a threshold of 1.2 m/s² has been defined as upper maximum average deceleration.

The information necessary to evaluate the above indicators are collected, via the communication system, from the on-board CAN bus data for the equipped vehicles and from GPS and accelerometer sensors embedded in the mobile devices installed on non-equipped vehicles. The results of driver performance analysis consider only data related to experiments that successfully logged all the necessary information.

3.6.3. Experimental Plans

Each of the drivers and riders participating in the trials ran 10 repetitions of each of the scenarios assigned to them.

The drivers were instructed to drive normally and in the safest way possible—as they would in real life—with only two recommendations:

• To enter the test area with the recommended speed for each scenario and keep it as their average speed as much as possible—unless something prevents it.
• To react normally to the notifications, warnings and recommendations provided by the system—as they would in a real-life situation.

Prior to the tests, users signed the consent form, they were familiarized by the test conductors about the scope of the testing and they completed a series of pre-test questionnaires. They were also given the opportunity to drive freely in the test area to familiarize themselves with the generic context.

The experimental design was a 2 × 4 within-subjects design, to let all drivers/riders try all scenarios assigned to them and run the same traffic scenarios with the in-vehicle applications and the mobile applications.

There were 2 independent variables (i.e., user performance and technical performance) and 3 levels (i.e., autonomous, non-equipped and equipped) for the two independent variables common across testing all services. Users were clustered in six (6) experimental groups, namely: Experimental Group 1: drivers of passenger demonstrators that are equipped; Experimental Group 2: drivers (same) of passenger vehicles but using their mobile terminal (smart phone); Experimental Group 3: riders of PTW demonstrators that are equipped; Experimental Group 4: riders (same) of PTW’s demonstrators; Experimental Group 5: drivers (3 with one in control, by rotation) driving the automated functions; Experimental Group 6: infrastructure operators involved in the evaluation of scenarios related to their operational filed only to oversee the process and provide their feedback through the subjective measuring tools. Not all users would test all applications. As such, they were matched to the applications on the basis of age group, gender, literacy and driving/riding profile criteria and driving/riding experience such as kilometers driven per year and frequency on weekly basis. To avoid order effects and over-familiarization, and the consequent desensitization of participant, the order of trips, regardless of condition, was counterbalanced [13].

4. Results

4.1. System Performance Results

4.1.1. Telecommunications Deployed in the Solution

The developed solution combines various technologies and types of telecommunications. Figure 14 presents the system’s simplified communication architecture.

Table 5. User selection fields.

Communication Technology Type	Usage
Bluetooth Low Energy (BLE)	Transfer vehicle detection, identification (VDIS) and environmental data from ORU to RSU
LTE/4G	Long range communications between the cloud MQTT broker, applications and edge devices (RSU and smartphones)
802.11p ITS-G5 V2X	Direct communications between RSU and neighbor equipped vehicles

summarizes the three basic communication technologies used, each of them having different characteristics with respect to reliability, packet loss, mean delay and each contributing differently to the end-to-end system performance.

Besides telecommunications, the system also contains several software applications running in various places such as the cloud, vehicles, edge devices, etc. Theoretically, software applications also affect the overall system performance. It should be noted that from the system’s performance analysis point of view, the applications performance indicator was the execution time of various algorithms and procedures and not the time required for, i.e., a state machine to generate an output. The contribution of the applications’ execution time in the system’s performance was found to be negligible compared with telecommunications latencies, because modern computers, smartphones and, in general, the embedded devices have strong processing power that can easily handle the specific solution requirements.

4.1.2. Vehicle Detection System Performance

In the specific solution, the vehicle detection is crucial and a key element for the performance of the overall system. Additionally, the vehicle’s motion parameters such speed, direction and position in lane are significant for all applications. In order to properly evaluate the performance of the detection system in the pilot sites, additional expensive perception sensors, such as LIDARs, RADARs and cameras were required to be installed in the field, which was impossible. However, these tests had already been conducted during the development and validation of the detection system in previous test rounds. As such, there was no reason to expect different behavior of the system in the pilot sites. The performance of the system regarding the velocity detection is shown in Figure 15.

At the early stage of the project, the vehicle’s detection rate was in the range of 70% accuracy. The main reason for this problematic performance were the manufacturing tolerances, problematic installation, bad cable joints and environmental conditions. By continuous rework and fine tuning, the system performance reached the 95% performance accuracy in the final field trials.

4.1.3. Performance Analysis of Equipped Vehicles

From the telecommunications point of view, the solution contributes to the equipped vehicles with V2X communications quite differently compared with the non-equipped ones that use 4G/LTE communications. It can be also considered as a type of a perception system that can detect and track at certain positions the neighbor vehicles and can also sense the environmental conditions. Equipped vehicles have the ability to calculate risks themselves by gathering information from other equipped vehicles or road infrastructure equipment that are in range. The solution’s contribution to equipped vehicles is the additional provided information for their neighborhood in terms of possible dangers’ sources (such as roadworks) and other vehicles. For example, an equipped vehicle cannot detect a non-equipped vehicle coming from wrong direction without proper installed sensors on board. This type of information is provided by the system. The system feeds the C-ITS V2X ecosystem, by injecting perceptual and environmental information to the gateway points, which are the RSBs. This information is then translated to the proper V2X messages and it is transferred to the equipped vehicles via dedicated direct communication channels.

The DSRC type of direct V2X communications was used. It is a CSMA-CA type of wireless communications, heavily based on the WiFi 11a/ac standards. CSMA-CA types of communications are asynchronous, and all the client nodes share a common medium. DSRC V2X communication is standardized by ETSI to a specific frequency, bandwidth, data rate, QoS and mechanism to resolve contentions and to avoid channel flood. Thus, the latency injected by the V2X communication channel is well predetermined and stable over time, with a mean time around 150 milliseconds, which has been reconfirmed from logged measurements during the field trials.

It is reminded that the strips were used, among others, to detect and identify vehicles on the road. This information is passed from the strips (ORU) to the RSB, via BLE communication channels in advertisement mode of operation. As in the V2X case, the latency in the BLE communication path is practically always the same and stable over time, with a mean time around 75 milliseconds. The ORU, due to limited hardware resources, does not implement any kind of on-board logging mechanism. The RSB is the device where the vehicle detection events are actually logged. In some equipped scenarios, there is also a communication path from the TMC to the RSB unit. This is used by the operator to inject non-safety critical information to the V2X system, such as VMS, or virtual toll billing information. This communication path is based on 4G/LTE technology. The latency caused by this path does not affect the system performance (in the case of the equipped vehicles), because this information is not an action trigger, but rather the payload of V2X messages.

4.1.4. Performance Analysis of Non-Equipped Vehicles

As opposed to the equipped vehicles, non-equipped ones do not incorporate themselves with the capability to evaluate risks and potential dangers. The solution fills in this gap by gathering all the required information from the road infrastructure that feeds the applications running on cloud, by evaluating potential risks and, finally, by communicating the application results to the corresponding non-equipped vehicles. In order to evaluate the system performance in this case, the scenarios that were tested were divided in two major groups according to their common characteristics. Group A scenarios included cases that combine the vehicle detection events with fast varying notable incidents in the neighborhood environment. These type of incidents are necessary preconditions to trigger the execution of the scenario and the moment of their detection is critical for the vehicle’s safety. Typical such cases include wrong way driving, work zone detection, pedestrian presence in crossing, etc. A typical event timeline during a group A test scenario is presented in Figure 16.

The time marks and time periods shown in the timeline are further explained below:

• t1: DENM event detection denoting the detection of the corresponding event for the specific test scenario (e.g., the existence of a pedestrian or a vehicle moving in the wrong way);
• t2: time of the DENM event reception by the cloud application;
• t3: time of the vehicle detection event that is sent by the RSU to the cloud;
• t4: time of the vehicle detection event reception by the cloud application;
• t5: time of an output message generation by the application. For example, time of generation of a collision warning message, a warning clear message, a warning level change message, etc.;
• t6: time of the visualization of the message in the HMI driver’s display;
• Δt12, Δt34: these time differences contain the telecommunications delay caused by the LTE/4G network and the delay from the MQTT broker processing time;
• Δt56: this time difference contains the telecommunications delay caused by the LTE/4G network (starting from the cloud), the delay from the MQTT broker processing time, the HMI application processing delay and the delay on the visualization device;
• Δte2e: this is the time difference between the detection of the trigger incident and the warning of the driver. This is practically the key performance indicator of the system, that denotes the overall end-to-end delay.

Figure 17 contains a graph of the telecommunication latencies for group A scenarios with non-equipped vehicles observed during the test trials.

Group B scenarios encompass cases that are based mainly on vehicle detection events and may consider at most messages with environmental measurements or map messages with non-critical information. For these types of scenarios (for example VMS), the environmental condition event usually pre-exists in the system for a long time before the detection of the ego-vehicle and it does not practically affect the system performance from the latency point of view. This does not mean that these scenarios are not critical. For example, oil on road is a safety critical condition, but this information exists in the system for quite some time before the vehicle approaches the dangerous area. A typical event timeline during a group B test scenario is presented in Figure 18 and contains fewer elements than in the group A cases.

The time marks and time periods shown in this timeline are further explained below:

• t1: time of the vehicle detection event that is sent by the RSU to the cloud;
• t2: time of the vehicle detection event reception by the cloud application;
• t3 time of an output message generation by the application. For example, time of generation of a warning message, a warning clear message, a warning level change message, etc.;
• t4: time of the visualization of the message in the HMI driver’s display;
• Δt12: this time difference contains the telecommunications delay caused by the LTE/4G network and the delay from the MQTT broker processing time;
• Δt34: this time difference contains the telecommunications delay caused by the LTE/4G network (starting from the cloud), the delay from the MQTT broker processing time, the HMI application processing delay and the delay on the visualization device;
• Δte2e: this is the time deference between the detection of the trigger incident and the warning of the driver. This is practical the key performance indicator of the system, that denotes the overall end-to-end delay.

Figure 19 contains a graph of the telecommunication latencies for group B scenarios with non-equipped vehicles observed during the test trials.

It is obvious that the dominant factor that affects and determines the overall system performance for the non-equipped case of vehicles, is the telecommunications latency. The most indicative performance metrics with respect to system latency are indicated in

Table 6. Overall system end to end latency.

Δte2e (ms)	Group A	Group B
Minimum	429	341
Maximum	3025	2475
Average	1126	1299
Deviation	370	347

and

Table 7. System single hop latency, from RSU to cloud MQTT broker.

Δte2e (ms)	Group A	Group B
Minimum	85	100
Maximum	1214	1200
Average	625	595
Deviation	270	251

.

4.2. Driver Behaviour Results

In this section, the main aggregated results from the user trial tests of VMS and C-ITS applications are illustrated. For the discussion, one can refer to the next section. The results are divided by the type of vehicles (i.e., equipped and non-equipped) and indicators (time to collision (TTC) and average acceleration (AA)). The box plots of the indicators are shown in Figure 20. The box plot represents the second and the third quantiles in the box and the median in red, as well as the outliers as crosses.

In the above figure, the different applications (test scenarios) are defined by different symbols:

• VRU stands for vulnerable road users and indicates the pedestrian test scenario, consisting in the issuing of a warning, if necessary, when a pedestrian is crossing the road (at the zebra crossing) detected by a specific strip.
• WWD stands for wrong way driving; this test scenario occurs close to a ramp in which a vehicle is coming in the wrong way. The vehicle that receives the warning is expected to slow down and pay attention. The wrong way driving vehicle is detected at the starting point of the ramp.
• RDW stands for roadworks; this test scenario involves the roadworks presence in which a lane is closed. This is an example of lane-wise personalization. The warning is differentiated for vehicles travelling in the closed and open lanes. The cases in which the lane is closed are excluded from these statistics since the driver does not receive yellow or red warnings but only an informatory one (blue warning).
• INT stands for intersection; this is the most complex scenario since it involves other vehicles in an active manner (in the WWD, the other vehicle is considered as a mere source of danger in a precise point). In this case, the warning level changes if the other vehicles in the intersection represents a danger or not.

Specific trials were made with the PTWs. Those vehicles were not provided with V2X communication; they used LTE technology. Despite this, they were not considered non-equipped vehicles because the state (position, velocity and acceleration) was retrieved from the vehicle itself by means of a connection with the smartphone. The results are depicted in Figure 21 and discussed in the next section. For non-equipped vehicles, VMS functions were also tested, consisting in personalized messages once a vehicle passes over a strip. Several types of messages were tested, divided into the three scenarios of the application (see Section 3.6.1). For these applications, the TTC is not defined since the position of the danger is not provided. The messages are not designed to convey this numerical data. Therefore, what is used as indicator for the driver behavior in this case is the average acceleration computed in the next five seconds after the message is received. This indicator is shown in Figure 22 for all the VMS messages.

5. Discussion

5.1. Discussion on System Performance Results

The services and applications for the non-equipped vehicles applications are based on LTE/4G networks. It is shown that from the telecommunications point of view, there is no practical difference between the safety critical applications and the non-safety applications. However, the impact on the service quality of these applications is very different. For the non-critical applications (e.g., VMS) even the maximum end-to-end latency is acceptable with no actual impact on the provided service quality. For the safety critical applications such as WWD or VRU, the case is different. This type of application requires stricter telecommunications latencies with lower maximum values and more deterministic characteristics with a stable and predictable quality of service.

By careful examination of the presented graphs, it can be noted that almost all the peak values of end-to-end latency occur in a condition where both RSU to cloud and cloud to HMI display (smartphone) communication latencies have high values. Both paths of communication are affected by the fact that they use LTE channels in one of their sections. The RSU and the HMI on the smartphone are both connected to the internet via LTE and were located in roughly the same area during the field trials. This means that most probably they are being serviced by the same LTE provider base stations and a possible network traffic overload, even a momentary one, can affect both of them for the same incident detected, leading to unacceptable end to end latencies for at least the safety applications.

Another factor that affected the overall system performance was the wide geographical spread of the servers hosting the applications. The integrated system is a heavily distributed system and many modules and sub systems have been developed by different partners. All these modules have been virtually integrated under a common MQTT broker running on a single cloud server. This feature proved to be very helpful and productive during the design and development of the system and also during the pilot tests, but it has the drawback that injects delays to the overall system due to the addition of unnecessary communication hops. It is clear that when the system will be integrated in a single or a more physical concentrated place, the overall latency will be significantly reduced, and it will converge to latencies comparable to a single hop.

The system performance results and analysis have showed that the developed solution can definitely support all the types of equipped services and applications. Furthermore, it enhances the perceptual ability of equipped vehicles by providing information about detected vehicles in their neighborhood. Perception sharing is a key concept that ETSI invests a lot of effort in. Especially the cooperative perception service (CPS) with the upcoming cooperative perception message (CPM), is the area where the solution presented herein may have valuable contribution.

For the non-equipped type of applications, the analysis has shown that not all types of the selected proof-of-concept applications can be directly supported in all cases. It is obvious that non-safety applications which are not sensitive to communications latencies are supported adequately and can serve vehicles, drivers and road operators. On the other hand, safety critical applications suffer from the unpredictable and unstable latencies of the 4G/LTE carrier network. It is also important to mention that future field trials have to include the assessment of traffic scenarios with varying traffic density in order to evaluate the specific impact to 4G/LTE and C-ITS V2X latencies, which has not been explored in our current tests.

At the specific moment of development, 4G/LTE was the only available network that could be used for pilot test implementation. The 4G/LTE network is a fundamental part of the current system, but it is not a prerequisite or a structural component of it. The transparent upgrade of 4G/LTE to the upcoming 5G network may well overcome the problems caused by network latencies. The slicing capability of the 5G networks can easily support all the safety applications and services in an efficient and cost-effective way. Additionally, considering the support of 5G for direct V2X communications (via PC5 channels), the equipped and non-equipped application versions of the solution can be merged into a common application framework in the C-V2X ecosystem.

5.2. Discussion on Driver Behaviour Results

5.2.1. Equipped Vehicles, Time to Collision

In Figure 20, the two box plots on the left show the indicators (TTC and AA) of the equipped vehicles for all user trials. The resulting TTCs are entirely above the expected threshold (3 s). In the case of equipped vehicles, the communication is reliable, and the delays are very small thanks to the ITS-5G technology (also known as V2X) and the TTC is consistent because the position is continuously updated.

The few outliers coincide with a more conservative TTC. They come from the cases in which the initial acceleration of the driver was significatively high. The Co-Driver takes into account the driver intention and the initial high acceleration increases the effort of the driver to reduce the speed to put the vehicle into a safe state. Therefore, those outliers mark dangerous maneuvers that were warned quite in advance to the drivers.

A closer look is needed for the intersection application; in this case, the TTC is higher in general and also more variable. The first aspect is due to the fact that, instead of a point, a whole area is considered as a source of danger (gaining some meters on the time to collision computation). The second aspect regards the behavior of other vehicles. In fact, the warning can be issued in different instants if other vehicles (there was only one other vehicle in the experiments) manifest a change of intention; for example, accelerating towards the intersection.

5.2.2. Equipped Vehicles, Average Acceleration

While the TTC is more related to the performance of the system (the driver can only influence it by increasing/decreasing the vehicle acceleration), the average acceleration is more related to the driver behavior. This indicator represents how the driver reacts to the warning provided by the system. If the driver is warned ineffectively or too late, this indicator will exceed the proposed threshold. The driver’s acceleration in absolute value is higher in a significant number of cases (especially in the pedestrian application); despite this, the average acceleration mean is below the threshold in all cases. This means the danger is properly perceived and the driver has the time to conformably react.

The outliers here represent the over-reaction of the drivers in few cases for WWD and RDW applications. Specifically, in these two functions/scenarios, the source of danger cannot be immediately visualized by the driver. Especially in the WWD, the driver does not see the wrong way vehicle immediately and sometimes the reaction is conservative, and the driver slows down as soon as possible. In the roadworks instead, the closed lane point was not immediately visible because it was signed only by a single cone (on purpose).

5.2.3. Non-Equipped Vehicles Results

The non-equipped vehicles represent the true challenge of the system since with this type of vehicle, all the applications evaluate the warning only when the vehicle passes over the strip located before the danger. Therefore, it is not a continuous evaluation also using the vehicle on-board data as for the equipped vehicles. As anticipated above, the position, velocity and acceleration provided by the smartphone sensors are only logged to evaluate the performance of the driver. In principle, they could be used by the applications and provided by proposed technology, but here, the application for non-equipped vehicles are stimulated only by the minimal set of information.

It is evident from Figure 20 that the results for the non-equipped vehicles present greater variability, made evident by the width of the boxes. Non-equipped vehicles rely on the accurate vehicle position only when it passes over the strip. Additionally, the LTE/4G introduces a larger delay in the communication (as it is also proved in the system performance results section). The two factors together are responsible for the variability of the TTC since the warning evaluation cannot be updated continuously. Nevertheless, the third quantile of TTC across all experiments is well above the threshold for all applications, except VRU. Furthermore, the limitation factors can be overtaken in future implementation. Specifically, the communication delays can be reduced by the upcoming 5G technology, while the limitation of the state update can be compensated with a proper increase in density of the strips in the layout.

The fact that the results are comparable with the equipped ones even with this limitation strengthens the potential of the system.

5.2.4. Non-Equipped Vehicles, Time to Collision

As stated before, the distribution of the indicators tends to be wider for the non-equipped vehicles. For the TTC, the two reasons for the higher variability lie in the communication and in the presence of the strips, just as stated above. The first is about delays in the communication; a delay necessarily shortens the TTC since the warning arrives later. The second reason is more related to the architecture of the system itself. With non-equipped vehicles, the system cannot benefit from a continuous update of the vehicle state and does not have the acceleration value (as it is the case in the equipped vehicles). The position and the velocity of the vehicle are retrieved only when the vehicle passes over a strip. Therefore, while in the equipped case, the warning was issued as soon as the state of the vehicle triggered the system; in the non-equipped case, it can be issued only when the vehicle passes over a strip. This makes the TTC more variable since it is directly related to the velocity of the vehicle while the distance is fixed by the strip position itself.

One can notice that the threshold of 3 s is violated in some experiments for the pedestrian use case (VRU in the figure), while in the other three applications it is not. This happens for a significant number of experiments: only the 60% percentile of the experiments is above the threshold. For this reason, after this use case, for the other three, the strip layout was changed in order to compensate for delays of communication. The series of three strips did not change, but the point of the danger was moved, since it was a static datum for the WWD, and in the position of a cone for the RDW. In the case of the intersection (INT), it was not necessary to modify the layout, since it was already more conservative than the other use cases (the whole area of the intersection is considered as the source of danger).

For each case, there are some values below the threshold. This happens when the first strip is crossed at a safe velocity, and the warning is triggered only from the second. This problem in real applications must be faced with a reasonable density of strips; that was not the case of these experiments, in which the number of strips was limited by the effort and cost to build the prototypes.

5.2.5. Non-Equipped Vehicles, Average Acceleration

The average acceleration in these experiments was comparable with the one in the equipped vehicles cases. Having a closer look at the box plots, one can notice that the variability is still higher (as expected as consequence of the variability of the TTC), but most of the values lie above the threshold as expected.

For this indicator, there is a significant number of outliers with higher decelerations. These cases correspond to the lower values of TTC that are not present in the equipped case. A low TTC means that the warning arrived too late, and the driver must decelerate more to avoid the danger that triggered the warning.

5.2.6. PTW Results

The results are very similar to the ones of the equipped vehicles. The TTC, as well as the average acceleration, are always above the threshold. Such good indicators were not expected using LTE technology; the delays were not very high during such experiments, and therefore the results resemble the ones of the equipped vehicles. The specific trials also made evident that the problem of the density of the strip affects the performance of the system; therefore, making them cheap and easy to install would be a certain requirement for future implementations.

5.2.7. VMS Trials Results

The expected results in terms of driving are similar to the other safety applications. The average acceleration is expected to be higher (therefore lower in terms of absolute value) than the threshold. The first message came from the hardest detection. The simulation of heavy traffic was difficult because of the limited availability of actual vehicles for the testing. This message returned the mildest reaction, probably due to the fact that the heavy traffic was not really there, and the driver did not feel the danger. The second message was different, and it generated stronger responses—the presence of fuel and oil cannot be immediately verified at a glance. Drivers decelerated as expected, without too much urgency. The last case returned decelerations even if it was not meant to; drivers turned on the lights but they decelerated also, manifesting a sense of danger anyway.

6. Conclusions

The current manuscript presents the system and driver performance results as emerging from the field trials with users that tested an innovative C-ITS infrastructure-based solution aiming primarily at road safety and at replacing key state of the art infrastructure elements such as VMS and toll stations. The research took place in the context of the SAFE STRIP EU funded project. The results that are discussed in the current manuscript reveal the potential of the solution that addresses all types of vehicles. The benefits for the non-equipped vehicles are even greater, promising a future equity in the roads in terms of road safety and driver experience regardless of intelligent on-board systems and costly infrastructures. C-ITS applications developed and tested on the rich and accurate set of information, provided by the “strip” technology, have proved to issue proper warnings on-time and efficaciously, increasing the driver/riders awareness of potential dangers with consequent smoother and more comfortable reactions, which is also a benefit for traffic safety and flow. Limitations and further implementation grounds have been recognized towards the future advancement and deployment of the solution.