Pedestrian Collision Avoidance in Autonomous Vehicles: A Review

Abstract

Pedestrian collision avoidance is a crucial task in the development and democratization of autonomous vehicles. The aim of this review is to provide an accessible overview of the pedestrian collision avoidance systems in autonomous vehicles that have been proposed by the scientific community over the last ten years. For this purpose, we propose a classification of studies in the literature in terms of the following: (i) pedestrian detection methods, (ii) collision avoidance approaches, (iii) actions, (iv) computing methods, and (v) test methods.

1. Introduction

Collision Avoidance (CA) is an essential module of an Autonomous Vehicle (AV). Indeed, one of the main criteria for the social acceptance of autonomous vehicles is the trust that other road users place in them, especially in Vulnerable Road Users (VRU). The authors of this review decided to focus on the Pedestrian Collision Avoidance (PCA) systems in autonomous vehicles. This topic represents a crucial issue in the improvement in and automation of vehicles when, at present, the World Health Organization (WHO) estimates the death of one pedestrian in a car accident every 1 min and 41 s [1]. Moreover, more than half of all the road traffic deaths are vulnerable road users such as pedestrians, cyclists, and motorcyclists [2]. Therefore, equipping partially autonomous vehicles with effective and robust PCA systems is a prime solution through which the safety of pedestrians on the road can be improved while moving toward the full automation of road traffic.

At present, self-driving vehicles belong to one of the five levels of autonomy described by the Society of Automotive Engineers (SAE). These levels can vary from level 0 (being a nonautomated vehicle) to level 5 (being a fully autonomous vehicle) [3]. In this review, not only were thePCA approaches in the context of fully autonomous car studied, but also the approaches that concern partially autonomous cars such as Advanced Driver Assistant System (ADAS) or approaches that are applied on autonomous robots and shuttles (as all these approaches could be applied in the context of autonomous cars). In addition, although PCA is mainly related to a pedestrian crossing the road, this review covers as wide a range of studies as possible through an analysis of the currently available approaches for avoiding collisions of autonomous vehicles with individual pedestrians, as well as with groups of pedestrians—both crossing and non-crossing, on signalized pedestrian crosswalks or not—in different road configurations.

In this paper, a thematic literature review of pedestrian collision avoidance in autonomous vehicles is presented. The main objective and aim of this review are to give an overview of the state-of-art approaches used for the development and the implementation of these collision avoidance systems. The rest of this paper is structured as follows: Section 2 presents the systematic process that was followed to conduct this literature review. Section 3 describes the classification of the papers included in this literature review into five main categories; therefore, the following five sections are dedicated to the discussion of one of the themes proposed in the classification. Finally, the concluding remarks for this review are presented in Section 10.

2. Review Process

In this section, we analyze the process that was applied to carry out this literature review. It was based on a systematic literature review process and consisterd of three different steps:

• Identifying the relevant papers in the literature for the review topic;
• Reading and summarizing the selected papers;
• Interpreting the findings and proposing a well-structured classification of the papers.

In order to identify the relevant work in the literature for the topic of this review, we decided to apply the following constraints:

• The research must have been carried out using the Google Scholar database.
• The selected articles must have been published within the last ten years.

Therefore, the first step was to conduct some research on the Google Scholar search engine, where the aim was to find relevant search strings to obtain results relevant to the topic. Below are some of the search strings that yielded the most relevant results:

• Pedestrian collision avoidance in autonomous vehicle;
• Pedestrian accident avoidance in autonomous vehicle;
• Pedestrian collision prediction in autonomous vehicle;
• Pedestrian accident prediction in autonomous vehicle;
• Pedestrian safety in autonomous vehicle.

As a result, we selected the 40 most relevant papers related to pedestrian collision avoidance in autonomous vehicles based on their abstracts.

Finally, we produced a pie chart of the distribution of the papers selected for this review according to their publication date, as shown in Figure 1. This made it possible to better understand how recent the analyzed approaches were.

3. Thematic Classification

This section presents the classification of the papers that were selected and analyzed for this review. A tree visualization of this classification is presented below. The classification in Figure 2 is made up of five main parts, which include pedestrian detection methods, collision avoidance approaches, actions triggered by collision avoidance systems, their computation, and the methods of testing these approaches.

4. Pedestrian Detection Methods

In this section, the different detection methods used by PCA systems are analyzed. The main objective of these methods is to determine the position of pedestrians around the autonomous vehicle as accurately as possible. An important challenge for an AV is to be able to see and to be clearly visible to agents present in its vicinity. Whereas mutual visibility can be relatively easily established between a human driver (driving a conventional vehicle) and a pedestrian, this is non-rivial when there is no driver in the vehicle. To carry out this recognition of the environment from the perspective of an AV, the detection methods can be divided into two categories:

• Line-Of-Sight (LOS) detection methods;
• Non-Line-Of-Sight (NLOS) detection methods.

It is also interesting to note that some of the approaches studied for this review combine both of these detection techniques.

4.1. Line-of-Sight Detection Methods

LOS detection methods include the methods that are applicable in the situations where an AV is able to detect pedestrians that are in its field of view. Pedestrian detection is thus carried out using the AV’s on-board perception sensors. A paper [4] describes an example AV, the autonomous-driving test vehicle of University of Tartu, along with its sensor set. Usually, AVs employ all or a subset of the following sensors:

• Cameras;
• Light Detection And Ranging (LiDAR) sensors;
• Radio Detection And Ranging (RADAR) sensors;
• Infrared (IR) sensors;
• Temperature sensors;
• Humidity sensors;
• Ultrasonic sensors;
• Road and traffic detectors.

The early works on PCA systems were based on this type of detection method. Moreover, a significant portion of the papers analyzed for this review used this type of detection method. The references in

Table 1. Works froinm the literature that employed line-of-sight detection methods.

Line-of-Sight Detection Methods
[5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29]

indicate approaches that use or could use, LOS detection methods.

A technique that is frequently used in multisensor approaches for perception is sensor fusion. Instead of relying on a single sensor, which might be prone to failures and whose quality may deteriorate over time, sensor fusion is used for merging the perception data that have been acquired using several sensors in order to produce more reliable information about an AV’s surroundings.

Although expensive sensors such as LiDAR are used in some of the studies that were included in this review, it is important to note that in industry, there is no consensus on their viability for consumer AVs of the future. For example, while companies like Waymo use LiDAR sensors in their AVs [30], other vehicle manufacturers like Tesla prefer to rely as much as possible on vision-based sensors [31].

4.2. Non-Line-of-Sight Detection Methods

More recently, NLOS detection methods have also been investigated by the research community that is working on PCA systems. This category refers to the methods that address the detection of pedestrians who are not in an AV’s field of view. Such situations are very common in traffic. Some examples include bad weather conditions such as rain, fog, or snow, and situations where the vehicle’s field of view is obstructed such as when overtaking another vehicle parked on the roadside and a pedestrian steps onto the road from behind the parked vehicle. In these situations, the LOS detection methods (described in the previous subsection) become much less reliable and effective. In order to overcome this problem, alternative detection methods have been proposed in the literature, which can be broadly divided into the following two categories:

• Wireless communication;
• Probabilistic methods.

Table 2. Works from the literature that employed Non-Line-Of-Sight detection methods.

Non-Line-of-Sight Detection Methods
Wireless Communication	Probabilistic Methods
[6,10,16,19,32,33,34,35,36,37,38,39]	[21,24,25,40]

presents studies in the literature that employed NLOS pedestrian detection methods.

4.2.1. Wireless Communications

NLOS detection methods based on wireless communication between an AV and a pedestrian (within the context of PCA systems) are referred to as Vehicle-To-Pedestrian (V2P) and Pedestrian-To-Vehicle (P2V) communication methods. In the literature, however, it is also possible to find the term Vehicle-To-Everything (V2X), which refers to the communication between an autonomous vehicle and all other agents that might be present in its vicinity such as other vehicles, pedestrians, and even the infrastructure. These methods can be based on different protocols such as:

• Wi-Fi-based communication;
• Communication using cellular networks (3G, 4G, and 5G).
• Dedicated Short Range Communications (DSRC)

In any NLOS PCA system, the main objective is to enable an AV to obtain information about any pedestrians that might be occluded from its field of view. This information can usually be obtained from the pedestrians’ mobile phones and might be more or less detailed and include features such as the location of the pedestrians, their age, and their sex. Some studies, such as [32], went one step further and proposed a classification of the VRU-contextual information into five categories: VRU-location context, physical activity, high-level activity, level of distraction, and personal information. They discussed how this information is helpful for facing the different challenges imposed by the use of smartphones, which include the energy consumption of a PCA running on small-battery devices such as mobile phones, the network load generated by the PCA, and the accuracy of the positioning information provided by smartphones (which depends on the quality and consequently the cost of the device and is usually far from being comparable to the accuracy of specialized or military equipment).

Additionally, as stated in [41], the privacy of pedestrians’ data and the security of the communications are crucial challenges when designing a V2P or P2V system for PCA. In fact, ensuring secure and authenticated communication between a pedestrian and an AV is essential for guaranteeing the integrity and privacy of the pedestrian’s personal information. Without this protection of communications, it would be quite possible for a malicious agent to steal the information of pedestrians or to modify it and cause serious accidents.

Finally, detection methods base on wireless communication may suffer from network uncertainties and issues such as communication delay or packet loss. These issues are discussed in [34]. When a delay is introduced in P2V communication or if packet loss rate increases to a certain value, an AV might not be able to determine the pedestrian’s correct position and velocity; therefore, a collision might occur. Therefore, minimizing communication delays and packet loss is of paramount importance for this type of pedestrian detection method.

Different communication protocols used in NLOS PCA systems are presented in more detail below.

Wi-Fi-Based Communication

Two articles, Refs. [35,39], which were analyzed in the context of this review, present PCA systems that are partially and fully based on Wi-Fi communication between a pedestrian’s smartphone and the AV On-Board Unit (OBU), respectively.

The authors of [39] tried to overcome the usual limitations of Wi-Fi such as the long connection time (which undermines the low latency requirement of PCA systems) and proposed an approach that is compliant with the IEEE 802.11p standard.

However, the authors of [38] pointed out that it is difficult to overcome certain limitations of Wi-Fi communications such as the problems caused by the interference with other networks (which are particularly problematic in urban areas), the limited communication range of Wi-Fi (which is about 100 m), and its weak mobility support due to its sensitivity to the Doppler effect.

Communication Using Cellular Networks

In [35], the authors propose an alternative to WiFi-based communication using communication through cellular networks, using an external server. Among the papers studied for this review, Ref. [38] also proposed an approach based on the use of cellular networks and a cloud-based server. The authors highlighted the fact that cellular networks have an interesting potential owing to their high mobility support, high bit rate, and wide communication range and capacity.

However, despite their potential, the requirement for communication with an external server in these approaches makes them more susceptible to performance degradation. (This is discussed in more detail in Section 7.)

Dedicated Short-Range Communication

DSRC is a one-way or two-way, short-range to medium-range, wireless communication channel, operating in the 5.8 or 5.9 GHz wireless spectrum, specifically designed for automotive use [42]. DSRC is widely used in V2X communications. This protocol is based on the IEEE 802.11p standard, which defines a method of exchanging data in high-speed motion without the need to wait for the association and authentication to a target to complete prior to exchanging data. In addition, currently, DSRC is the only short-range wireless technology whose performance is not degraded by bad weather conditions. Using DSRC, the information transmitted between pedestrians and an AV take the form of standardized messages named Cooperative Awareness Message (CAM). Among the papers studied in this review, Refs. [35,36,37] propose PCA approaches based on DSRC.

However, even if the DSRC communication seems effective and promising for PCA systems, it is currently difficult to envision this solution in the real world since no smartphone currently offers the possibility of embedding a DSRC module, even though efforts to embed them have recently started.

4.2.2. Using Probabilistic Models

Several approaches analyzed in this review use probabilistic methods in order to help the AV to pay attention to any occluded areas. An interesting method is presented in [21], in which the authors propose a way to find the distribution of emerging pedestrians from occluded areas. This solution is based on contextual information collected with LOS sensors such as the presence of parked cars, crosswalks, and any visible pedestrians. The authors also stated that this type of pedestrian detection method has not been investigated very much in the literature in comparison to other pedestrian detection methods in the NLOS category. We indeed observed the same while conducting our review. As the direct transmission of information between a vehicle and a pedestrian is more reliable as long as the communication is stable, it is surely interesting to further investigate the combination of NLOS and probabilistic methods.

5. Collision Avoidance Approaches

In this section, we present the main types of collision avoidance approaches employed in the papers we reviewed. These approaches can be classified into four types:

• Rules-based approaches;
• Force-field-based approaches;
• Model-based approaches;
• AI-based approaches.

The approaches that are discussed in this section employ different input (in terms of pedestrian detection) methods than those presented in Section 4. And, the actions that these approaches output are discussed in Section 6. One key aspect to consider while studying these approaches is to determine which seem to be the most suitable for real-world conditions, i.e., whose calculations can be performed in real time. In the following subsections, the four above-mentioned categories are presented. It is important to note that some of the methods that were reviewed combine several types of approaches in their collision avoidance system.

5.1. Rules-Based Approaches

This subsection presents rules-based approaches. These approaches are based on the calculation of risk assessment indicators to trigger certain actions.

Table 3. List of works that employed rules-based approaches for PCA.

Rules-Based Approaches
[9,14,16,21,33,34,35,38,39]

lists studies in the literature that employed rules-based methods for PCA.

In PCA, one of the most widely used metrics is the Time To Collision (TTC), which (at any given time) indicates the time remaining before collision if the two entities involved in the process (i.e., the AV and the pedestrian) maintain their respective speeds. As mentioned in [41], the TTC is easy to calculate as long as the speed of the vehicle and the distance between the vehicle and the pedestrian is known.

Several other indicators have been derived based on the TTC in the literature. For example, Ref. [9] presents the Time-To-Collision Range (TTCR) metric. The authors [9] argued that the TTC does not take into account the potential collisions that may be dangerous for a pedestrian. As a result, they proposed an improvement to the TTC metric, i.e., the TTCR, which takes into account the potential collision and is therefore more reasonable and safe for pedestrian collision avoidance, they argued.

In general, the metrics used in these approaches can be quickly calculated from the information collected during the pedestrian detection stage. Therefore, this type of approach seems feasible for application in real time.

5.2. Force-Field-Based Approaches

In this subsection, the force-field-based approaches are presented. These approaches can be further divided into two main branches:

• Potential field methods;
• Elastic band methods.

Table 4. Classification of studies the literature that employed force-field-based approaches.

Force-Field-Based Approaches
Potential Field	Elastic Band
[22,24,25]	[20,36,37,43]

presents the two classes within the force-field-based approaches.

5.2.1. Potential Field

Potential fields have been used in robotics for a long time (and have also been more recently used in autonomous driving) for the purpose of path planning. In such methods, an artificial potential field of attractive and repulsive forces is generated by the factors that affect the path planning of a vehicle. For example, the destination constitutes an attractive force, and the obstacles, such as pedestrians or road boundaries, are considered as repulsive forces. Based on the created force field, the idea is then to find a collision-free path within this force field and in turn to determine the trajectory and the velocity of the vehicle.

However, even if these approaches are interesting, as mentioned in [20], they suffer from a high computational cost and are therefore difficult to apply in real time. As a result, the research community has recently come up with a new force-field-based method that is better suited for real-time computation, i.e., the elastic band method.

5.2.2. Elastic Band

The elastic band method is a local path planning method. Path planning, on a higher level, can be divided into two categories, which are global path planning methods and local path planning methods. The global path planning methods are responsible for planning the path for an AV from point A to point B, on a relatively larger scale. Generally speaking, such a path takes the form of a series of waypoints represented by GPS coordinates. On the other hand, the local path planning methods are responsible for modifying a small portion of a predefined path (which is usually defined by the global path planning module). Figure 3 illustrates and compares the goals of global and local path planning methods.

The elastic band method uses a predefined path that is locally modified as a result of the action of internal and external forces. The part of the path affected by the forces is called the band. Internal forces keep the band together, while external forces, such as the potential artificial force, keep the band clear of obstacles.

As stated in [20], in comparison to the potential field methods, the elastic band collision avoidance methods do not take into account all the possible trajectories of the AV in the force field. Because of this reason, the elastic band methods are naturally faster to compute than the potential field ones.

In addition to better performance, these methods also make it easy to incorporate the notion of respecting the social distance of the pedestrian, as presented in [43]. Indeed, during the computation and execution of collision avoidance with a pedestrian, their personal space must be taken into account and respected by the AV. This corresponds to adding an extra safety distance around the pedestrian in the calculations. Compliance with this condition echoes the so-called “socially acceptable” criterion, which has sometimes been discussed in the literature.

Finally, another interesting point regarding these methods is that they can handle the case of a group of pedestrians by considering the group as a single entity that is to be avoided. The recognition of the group of pedestrians is also dependent on the detection method employed by the specific PCA system in question.

5.3. Model-Based Approaches

This subsection presents the model-based collision avoidance approaches. These approaches can also be described as algorithms that are based on the mathematical modeling of an AV and its environment.

Table 5. List of studies that investigated model-based approaches.

Model-Based Approaches
[13,15,17,18,22,23,25,40,44,45]

lists studies in the literature that investigated model-based approaches.

As mentioned in [15], the advantage of this type of approach in comparison to learning-based approaches is that these approaches do not require the construction of a reliable training dataset, which may struggle to cover all PCA scenarios. In addition, the solutions that are developed on the basis of a mathematical model are optimal. Thus, there are no cases of false positive actions, which is quite possible in learning-based models (which can in turn lead to ethical and legal disputes in the event of an accident).

However, according to the authors of this review, this type of approach also has its shortcomings. The effectiveness and ability of the model to interact with the environment are completely dependent on how it is designed. In addition, it seems difficult to imagine being able to incorporate all possible situations in the environment into a mathematical model. This type of approach seems to be inflexible in the face of unexpected changes in the environment. In fact, the approaches that are flexible and adaptable perform better under real-life conditions, such as the approaches that are based on the elastic band method or AI-based approaches. Finally, the power of model-based approaches is tightly related to the computing power that is available to find the solutions. Several computing solutions are possible with such approaches, such as calculations on the vehicle’s OBU or on external servers. As discussed in the Section 7, computing on external servers currently seems impractical for this use case, which would in turn mean that the calculations need to be performed on the AV’s OBU. Therefore, it is possible to imagine that the most powerful model-based autonomous vehicles will also be the most expensive.

5.4. AI-Based Approaches

In this subsection, the AI-based PCA approaches are presented. The use of such approaches is widespread in the literature on autonomous vehicles in general, and this is also the case for the literature on PCA systems. In the works that we reviewed, the approaches related to Artificial intelligence (AI) can be classified into two subcategories:

• Approaches based on Neural Networks;
• Approaches based on Fuzzy Controllers.

Table 6. Classification of studies that employ AI-based approaches.

AI-Based Approaches
Neural Networks	Fuzzy Controller
[5,6,7,8,10,12,20,26,29]	[5,11,19,28,29]

shows the classification of methods into these two categories.

5.4.1. Neural Networks

The PCA approaches that use neural networks as the solution can be further divided into two main categories, i.e., the approaches that use Reinforcement Learning (RL) and those that employ Fuzzy Neural Network (FNN).

Reinforcement Learning

RL is a well-known method in autonomous vehicles, which is often used in order to learn a driving policy. So, it makes intuitive sense to use RL to learn a driving policy to avoid collisions with pedestrians. The problem is usually formulated as anMarkov Decision Process (MDP) before using a Deep Q-Network (DQN) to solve it and learn a near-optimal policy. The main components of an MDP that may vary from one approach to another are as follows:

• State space: the space where one state describes the state of the own vehicle and the pedestrians.
• Action space: the set of actions that can be chosen by the autonomous vehicle.
• Reward function: the function that attributes a numerical score based on the state of the environment after the chosen action has been completed.

A crucial part in RL is the design of the reward function. It is based on several criteria that can positively or negatively impact the final score. For example, hitting a pedestrian, not respecting traffic regulations, or going outside the limits of the road can negatively impact the final score. On the other hand, reaching the destination in a reasonable time can positively impact the final score. For each approach, it is therefore necessary to find the function that allows the most efficient training of the DQN according to the modeling previously conductedx.

Such a Neural Network is made up of three main parts:

• Input layer: there are as many neurons as variables needed to describe a state;
• Hidden layers: these are usually several fully connected layers, the number of which may vary depending on the approach;
• Output layer: there are as many neurons as possible actions (defined in the MDP model).

The Stochastic Gradient Descent (SGD) algorithm is usually used to update the network parameters while minimizing the loss function. Furthermore, a DQN is generally accompanied by experience replay memory to speed up and stabilize the learning process. The agent’s experiences at each time step are kept in a data set called experience replay memory. During the training of the DQN, the goal of the replay memory technique is to break the correlation between consecutive samples that could be fed as input to the neural network by randomly selecting and inputting experiments from the replay memory.

This replay memory can also be optimized according to the proposed approaches. For example, Ref. [10] proposes a double DQN with a Prioritized Experience Replay (PER) buffer. Indeed, it is possible to order the experiences stored in the experience replay memory depending on their relevance. For example, an experience that leads to a collision with a pedestrian may be more relevant to replay than the other ones. Therefore, instead of picking experiences randomly in the replay memory during the training phase, they are picked from the highest to the lowest priority.

Another example is presented in [27], where two experience replay memories were used. The authors argued that there are usually insufficient negative experiences available in the common experience replay memory. In their approach, negative experiences refer to experiences that lead to a collision state. In the case of a lack of negative transitions in the replay memory, the learning process of the driving policy is more time consuming. As a result, they decided to create a second replay memory dedicated to negative experiences and to keep the first one for the other experiences. During the training process, experience is uniformly picked up from the first and second replay memories. This allowed the authors to achieve faster training times compared to a single traditional replay memory.

Fuzzy Neural Network

A FNN is used to find the outputs of a fuzzy controller. It uses the training and learning algorithms from neural networks to find the values of the output parameters. The fuzzy controllers used inside the neural network are described in the next subsection.

5.4.2. Fuzzy Controllers

Some of the approaches studied in this review base their collision avoidance systems on Fuzzy Controllers. This type of controller allows an AV to operate quite closely to the functioning of human decision making. Fuzzy Controllers are a good solution for building PCA systems because they are very flexible and adapt well to noisy environments. The activity of a Fuzzy Controller can be summarized in the following three main phases, as presented in detail by [19]:

• Fuzzification: The controller receives crisp input values corresponding to the different input variables. This phase converts the crisp input values into fuzzy values using a membership function.
• Inference engine: The controller makes fuzzy inferences on the inputs using predefined inference rules (if-then rules). The result of this phase is the fuzzy values of the output variables.
• Defuzzification: The controller converts the fuzzy output values into crisp output values. This is a reverse process of the fuzzification phase.

In the approaches that we studied within the context of this review, the number and definition of input and output variables vary. The input variables depend on what information is collected about the environment of the AV. The output variables correspond to the values of actions that will be carried out by the AV, e.g., braking the vehicle by half its speed. Finally, the last component that can vary between the approaches is the inference method used in the inference engine. There are different methods of inference including Mamdani’s inference method, which is the most used in the Fuzzy Controllers among those that were studied in this review. Studies such as [5,19,29] used this inference method. One other well-known inference method is the gravity center method, which was used in [28].

6. Actions

In this section, we present the possible actions that are triggered by different PCA systems. The actions triggered by these systems can have an impact on two different parties:

• The AV itself;
• The pedestrian(s) involved.

6.1. Involving the Autonomous Vehicle

Intuitively, a PCA system can trigger actions that are related to the AV itself. In the different PCA approaches that have been proposed in the literature, the following are the main types actions that can be undertaken by the vehicles themselves:

• Longitudinal control (action on throttle or brake);
• Lateral control (action on the steering wheel);
• Both.

Table 7. Classification of studies based on the triggered actions in relation to the AV itself.

Actions
Longitudinal Control	Lateral Control	Both
[5,6,21,24,28,29,34,38]	[25,36]	[8,9,10,11,12,13,14,15,17,19,20,22,23,26,27,33,35,39,40,44]

shows the distribution of these types of actions that were studied for this review.

In a PCA system, the numerical values corresponding to the throttle, the braking, and the steering angle are usually produced by a “high-level” controller. In order to be applied on the AV, these values must be translated into throttle or brake pedal forces or steering wheel angles, respectively. This translation is usually performed by a “low-level” controller. One example of a low-level controller is a Proportional–Integral–Derivative (PID) controller, which was employed in [29]. Figure 4 shows the common architecture of a PCA system, along with the autonomous vehicle components involved at each stage.

6.1.1. Longitudinal Controller

In terms of the longitudinal control, the actions that are triggered by a PCA system can be applied to the throttle or the brake of an AV.

Throttling Action

In some situations, acceleration is a useful action for the vehicle to overtake a pedestrian who might otherwise collide with it. However, these situations are quite rare compared to the situations that involve vehicle braking.

Braking Action

In relation to the braking action of a vehicle, there are two types of situations.

The first type includes the situations where the PCA system has detected a possible collision sufficiently in advance. In such situations, the vehicle can gradually reduce its speed while respecting the comfort of the passengers. In fact, passenger comfort is a major consideration when designing the actions the can be taken by an autonomous vehicle. These actions must be as smooth as possible so as to not disturb the passengers.

The other type includes the situations where the AV may not have had enough time to anticipate the collision with the pedestrian. This is sometimes the case in certain occlusion situations, where a pedestrian may suddenly come onto the road. In this case, the braking action can often take the form of an Automatic Emergency Brake (AEB) system. The approaches presented in [5,13,14,29,40] involve AEB systems. Such systems slow down the vehicles as quickly as possible when a collision is imminent and cannot be avoided otherwise. Here, it is obviously pedestrian safety that takes precedence over passenger comfort.

6.1.2. Lateral Controller

Based on the reviewed literature, the use of lateral control depends on the speed of the AV at the time when a possible collision with a pedestrian is detected. Intuitively, depending on the speed of the vehicle, the steering angle applied to the wheel has a larger or a smaller impact on the vehicle. It is important to note that in emergency situations where a vehicle has a relatively high speed, the relatively high impact of an action on the steering wheel, and in turn on the movement of the vehicle, may aggravate the accident. Thus, it is clear that the more in advance a possible collision can be detected by a PCA system, the more time the vehicle has to decelerate sufficiently to avoid a collision using longitudinal control action, without having anything to do with the steering wheel.

Additionally, in situations that involve multilane roads, any lateral control action must also take into account the environment before passing a pedestrian. It is necessary to take into account the other vehicles in an AV’s vicinity in order not to collide with them and cause an accident. Systems based on sensors or intervehicle communication that try to mitigate such situations do exist.

6.2. Involving the Pedestrian

Some PCA systems are able to send a warning message to the pedestrians that are involved in a potential collision situation at hand. This type of action concerns approaches directly involving the pedestrian, notably through NLOS detection methods with wireless communication. The aim of this type of approach is to be able to warn both parties involved in the potential collision, namely, the autonomous vehicle and the pedestrian.

The challenge here is to be able to adapt the alert message sent to the pedestrian according to their profile. As stated in [32], in low-risk scenarios, inappropriate warnings could surprise and frighten a pedestrian and even result in an collision, for example with an alarm sound that is excessively loud. In order to best adapt the type of alert message to pedestrians, one solution is to use the pedestrian’s contextual information such as age or distraction level. In this regard, attention should be paid to all kinds of scenarios, including, for instance, the case where a pedestrian’s phone is in active use and is, for example, in their pocket.

7. Computing Methods

In this section, we review different computing methods that are used in different PCA systems. Naturally, PCA systems need computing resources to perform calculations that can vary depending on the approaches that a PCA system is based on. In terms of the computing methods, PCA systems can be divided into three subcategories:

• Computations performed on the vehicle OBU;
• Computations performed on the pedestrian’ smartphone;
• Computations performed on external servers.

It is important to highlight here that almost all the approaches that we studied perform their computations on-board the vehicle OBUs.

7.1. Performed by Vehicle’s On-Board Unit

Performing the computations required by an autonomous vehicle on the OBU is the most common approach. An OBU can be operational both when the vehicle is moving and when it is stationary. In the context of computations performed on the vehicle OBU, the computing resources available vary. In fact, in the literature, depending on whether an approach is being tested on a real vehicle or in a simulator, the hardware that is performing the computations can vary and thus impact the performance of a PCA system.

7.2. Performed on Pedestrian Smartphone

Two of the approaches [16,35] that were studied within the context of this review propose performing computations on the pedestrian’s smartphone. These approaches are based on NLOS pedestrian detection methods using wireless communication.

The pedestrian collision avoidance systems that are loaded on pedestrian smartphones usually take the form of a mobile application. For example, in [16], when the pedestrian’s mobile phone is in use, the collision prediction algorithm is implemented as a background-running mobile application. It accesses the pedestrian’s current location using the GPS localization sensor embedded inside the smartphone. Based on this information, the system updates the algorithm with pedestrian position data, and then it waits to receive the location information of nearby autonomous vehicles. Finally, the application computes the collision prediction directly on the smartphone according to the information collected by it and displays a warning to the pedestrian if a collision may happen.

This type of approach is questionable as it relies heavily on the pedestrian’s phone, which may provide poor-quality information or run out of battery while running the application in the background. With regard to the use of the battery, different solutions have been investigated, such as the one proposed in [38]. It proposes tackling the energy limitation of the battery by using an adaptive multilevel application that can switch to an energy-saving mode in risk-free situations. Additionally, the computing resources of smartphones depends entirely on the quality of these devices, which varies greatly from one pedestrian to another. Therefore, this type of computing method seems impractical in real-world conditions.

7.3. Performed on External Servers

An approach presented in [38] proposes the use of a cloud-based server to carry out the computations required by the collision avoidance system. This approach employs a NLOS pedestrian detection method that is based on wireless communication. Again, the reliability of such approaches may be debatable. Even if the progress in the broader field of cloud computing makes it possible to offer powerful servers that are accessible at short distances, the use of such an architecture multiplies the number of communications through the network, which may not only increase the total computation time of the collision avoidance system (with the addition of extra communication time in comparison with a direct communication approach) but may also make the system less resilient. This is because all additional communications with the servers are susceptible to failure due to delays or packet loss.

In addition, this type of architecture requires the system to be fault-tolerant on the server side by requiring, for example, replication of the server, which is more expensive and more complex to deploy and maintain than an architecture that only involved the AV and the pedestrian smartphone. Furthermore, the issues of security and authentication of communications are also important in this context. As it is the case for V2P or P2V wireless communications, it is crucial to carry out security authentication on the external servers in the communication process with the AV in order to avoid malicious communications that could be the cause of serious accidents.

8. Test Methods

In this section, we present different test methods that have been employed in the literature in order to test and validate different PCA systems that have been proposed. These test methods can be divided into two main categories:

• Test methods using a real car;
• Test methods using simulators.

8.1. Using Real Car

A small proportion of the approaches that were studied in this review underwent full-scale testing with real autonomous vehicles. Naturally, in most cases, these test vehicles belonged to the laboratories of the researchers who proposed these PCA systems. It can be argued that this type of real-world testing could make it easier to integrate the proposed PCA approaches into vehicles that are or will be produced by vehicle manufacturers for the general public.

Table 8. List of works that used a real car for the testing of PCA systems that they proposed.

Test Approach Using Real Car
[9,15,19,20,33]

lists the works that used real cars for their testing.

In general, the proposed approaches in these works were not tested on a real vehicle as the first stage of testing but rather underwent tests in simulators before being tested on real vehicles. Deploying a PCA approach directly (i.e., without first testing it in simulation) on a real-life AV for tests in real traffic could entail risks for the pedestrians involved in the tests as well as for the equipment used, even if there is usually a safety driver in the vehicles. A common way to reproduce the physical conditions of a real autonomous vehicle under simulated conditions is to use a Hardware-in-the-loop (HiL) simulation. HiL simulations include real sensors and actuators in an electrical emulation to verify their behavior under the tested approach.

8.2. Using Simulators

The testing approaches that are based on simulators can be divided into two broad categories:

• Custom simulators (software and prototypes);
• Open-source simulators.

Table 9. Classification of works in the literature based on the type of simulator that they used.

Simulators
Custom Simulators	Open-Source Simulators
[7,8,12,13,16,17,21,23,24,26,32,34,35,36,37,38,39,43,44,46]	[5,6,10,11,14,18,20,22,25,27,28,29,40,47]

presents this classification of the works from the literature based on the type of simulator that they used.

From Table 9, it can be observed that almost all the approaches that were studied in this review were tested using simulators. It is indeed difficult to implement real-world test scenarios involving collisions with pedestrians. Pedestrians (as well as the equipment) could be put at risk in such experiments, especially when trying to test the limits of a PCA system. However, the transition from simulation to the real world, for a system that has only been tested in a simulation framework, can be challenging. As a result, the use of prototypes can be seen as a midway solution between software simulations and real-world tests.

Two interesting approaches in the works that we reviewed focused on the generation of test scenarios. The authors in [46] provided a method to extract the most relevant test cases from a map given as an input, considering that the system tested should avoid pairwise collisions. To reduce the involved search space, i.e., the number of test cases that are redundant, and guarantee test case diversity, the proposed system analyzes the topology of the map for extracting test scenarios. Then, it uses a fuzzy model to evaluate the own vehicle’s motion, based on which the test cases can be classified.

The authors in [47] proposed diversifying test cases by automatically generating safety-critical scenarios. As mentioned in this article, there are currently several datasets available for developing and testing collision avoidance systems. However, the variety of test cases is naturally constrained by manually gathered datasets, and acquiring data from challenging scenarios, such as those involving collisions with pedestrians, is particularly problematic. Therefore, the authors came up with the Adversarial Test Synthesizer (ATS), which is an RL agent that positions pedestrians in a given scene, thus allowing the generation of critical test cases.

The subsections below describe the two subcategories of simulators that were encountered during our review in more detail.

8.2.1. Custom Simulators

We found that, in general, the simulators that have been developed were most often written in Python. This allows for high flexibility in terms of the implementation of the simulation and simplifies the integration of the corresponding proposed approach in the simulation environment. However, the scenarios evaluated by such simulators are likely to be limited and less comprehensive than those offered by open-source simulators. Here, the use of prototypes is one way that can improve the quality of testing by being closer to real-world conditions, while remaining on a small scale where everything is under control.

8.2.2. Open-Source Simulators

A significant number of approaches that were studied in this review were tested using open-source simulators. Among theses simulators are, for instance, Car Learning to Act (CARLA) [48], CarSim [49], and PreScan [50]. Such simulators allow authors to configure test scenarios with fine granularity.

A common approach in several studies that originated from Europe, such as [40] and [14], is to use scenarios from the European New Car Assessment Program (Euro NCAP) [51] in order to test their PCA systems. This program provides a comprehensive collection of tests cases for assessing new vehicles and contains a subcollection of scenarios dedicated to AEB system testing involving pedestrians.

9. Discussion

Below, we comment on the benefits and limitations of the different methods in the categories presented above. We also discuss the avenues for future research in pedestrian collision avoidance.

In terms of pedestrian detection methods, LOS methods are naturally intuitive to humans and will continue to be investigated and deployed in future autonomous vehicles. We expect that cameras and RADAR will continue to be used and LiDAR to see a rise in use as in situ perception sensors with the advancement in solid-state LiDAR technologies (which are designed with the specific aim of deployment in automated driving). At the same time, arguably [52], the autonomous driving community will move more and more toward the paradigm where vehicles and infrastructure work together in order to achieve high levels of vehicle autonomy. With that in mind, we expect infrastructural sensors to be investigated more and more for providing assistance to autonomous vehicles and for tasks such as pedestrian collision avoidance and pedestrian motion prediction. This shift in paradigm also relates to the NLOS methods. In the future, it is expected that, with advancements with privacy-preserving communication, wearable devices like smartphones and smartwatches will be investigated more and more for inclusion in the traffic infrastructure, where such devices can anonymously indicate the presence of road agents (including pedestrians) to vehicles in their vicinity. Because of the coverage and availability (and their ever-increasing speed, bandwidth, and stability), we expect cellular networks to be the dominant medium for communication between the infrastructure and vehicles in the future.

The above-mentioned shift in paradigm, where vehicles and the infrastructure cooperate for achieving high levels of autonomy, will also have an impact on collision avoidance approaches. We expect rules-based and AI-based approaches to be the main approaches that will be investigated by the research community in the near future. In relation to the same shift in paradigm, it is expected that future infrastructural devices will be able to share the computation needed for autonomous operation of AVs rather that all the computation occurring in situ on AVs.

In terms of the actions to be taken by an AV for implementing a collision avoidance maneuver, all the kinds of approaches listed in Section 6 have their own significance based on the context and scenario in which an AV finds itself. Therefore, it is expected that all the methods (including their combinations) will continue to be investigated and deployed in future AVs.

In terms of the test methods in autonomous driving, one aspect that remains largely underinvestigated is the use of small-scale vehicles. Small-scale robotic platforms are aplenty [53], but the limiting factor in their effective use in autonomous-driving research is systematically studying how and for what aspects of autonomous driving they can be meaningfully used. While some preliminary studies such as [53] have been conducted that try to systematically investigate this question, this remains an open research area. We believe that once the suitability of small-scale vehicles for use in autonomous driving has been systematically established, they have the potential be be used in investigating many different aspects of autonomous driving, as indicated by studies such as [54]. At the same time, advancements in autonomous driving simulators (indicated by studies such as [55]) are making them more effective for autonomous driving research. In the future, it is expected that simulators will continue to be used for autonomous driving research, and small-scale vehicles will be more systematically investigated for their possible use in autonomous driving research.

10. Conclusions

To conclude, in this review, an overview of pedestrian collision avoidance systems in autonomous vehicles was presented. This review is based on a classification of the relevant literature into five main categories. These categories represent the pedestrian detection methods, the collision avoidance approaches, the actions triggered, the computation methods, and the methods employed for the testing of PCA systems.

There are also some limitations of this work. On the one hand, not all the papers published on this topic over the selected time period could be studied in this review due to time constraints. Another limitation is the use of Google Scholar only for searching the relevant literature. In the future, inclusion of databases such as Web of Science and Scopus should also be considered for such reviews. On the other hand, the assumptions made about the applicability of some approaches to autonomous vehicles and others to pedestrian avoidance rather than obstacle avoidance also constitutes a limitation of this work.

Finally, over the past 10 years, the systems proposed to meet this major challenge have evolved greatly. Future approaches proposed by the research community will be all the more interesting as the challenge is important and topical, as we are gradually making the transition to a world of fully autonomous transport.