Object Detection in Autonomous Vehicles under Adverse Weather: A Review of Traditional and Deep Learning Approaches

Abstract

Enhancing the environmental perception of autonomous vehicles (AVs) in intelligent transportation systems requires computer vision technology to be effective in detecting objects and obstacles, particularly in adverse weather conditions. Adverse weather circumstances present serious difficulties for object-detecting systems, which are essential to contemporary safety procedures, infrastructure for monitoring, and intelligent transportation. AVs primarily depend on image processing algorithms that utilize a wide range of onboard visual sensors for guidance and decisionmaking. Ensuring the consistent identification of critical elements such as vehicles, pedestrians, and road lanes, even in adverse weather, is a paramount objective. This paper not only provides a comprehensive review of the literature on object detection (OD) under adverse weather conditions but also delves into the ever-evolving realm of the architecture of AVs, challenges for automated vehicles in adverse weather, the basic structure of OD, and explores the landscape of traditional and deep learning (DL) approaches for OD within the realm of AVs. These approaches are essential for advancing the capabilities of AVs in recognizing and responding to objects in their surroundings. This paper further investigates previous research that has employed both traditional and DL methodologies for the detection of vehicles, pedestrians, and road lanes, effectively linking these approaches with the evolving field of AVs. Moreover, this paper offers an in-depth analysis of the datasets commonly employed in AV research, with a specific focus on the detection of key elements in various environmental conditions, and then summarizes the evaluation matrix. We expect that this review paper will help scholars to gain a better understanding of this area of research.

1. Introduction

The World Health Organization (WHO) releases statistics each year on the number of people who have been hurt in traffic accidents. Around 1.3 million people worldwide die each year, and 20 to 50 million suffer serious injuries [1], with young males under 25 constituting the majority, 73%, of fatal traffic accidents. Extensive research on the worrying prevalence of traffic fatalities has led to the development of upgraded intelligent transportation systems (ITS) with expanded capabilities. The cutting-edge of contemporary automotive technology is represented by automated driving systems (ADS) and autonomous vehicles (AVs). AVs, especially self-driving cars, have generated a great deal of interest in the fields of intelligent transportation, computer vision, and artificial intelligence [2]. No other invention since the creation of vehicles has had such a dramatic impact on the automotive industry as self-driving vehicles. As per the forecasts presented in Figure 1 [3], the market for self-driving cars is expected to grow from 23.80 million units in 2022 to 80.43 million units by 2032. These vehicles primarily rely on cameras and sensors, and examples include LiDAR (light detection and ranging), radar, ultrasonic, etc., which have a number of autonomous features, including reduced use of fuel, cutting carbon dioxide emissions by up to 10%, decreased traffic congestion [4], decreased road accidents, and utilizing roads more efficiently to optimize the transportation system. AVs can assist in managing and studying traffic by gathering effective and valuable traffic data. Along with other road-related capabilities, they are capable of recognizing static and non-static objects such as vehicles, pedestrians, and road lanes. Object detection (OD) is critical for AVs as it ensures safety by identifying and avoiding moving obstacles in their path. Numerous research works have investigated methods for identifying objects in AVs, broadly divided into manual, semi-automated, and fully automated systems [5], as shown in Figure 2. Roadside artifacts are visually assessed by human inspectors using manual procedures; nonetheless, subjectivity and labor intensity are drawbacks. While semi-automated methods gather data from moving cars and process them afterward, they are still labor-intensive [6]. High-resolution cameras and sensors on cars record pictures and videos in completely autonomous systems. Software-based models that have been trained to detect objects evaluate the data either after they have been collected or in real time. These techniques seek to solve safety problems while improving the precision and effectiveness of item identification on roadways and streets. Vehicle-based surveillance is a widely used technique since it is a quick and efficient way to examine objects. However, it has always been difficult for autonomous programs to identify objects under adverse weather conditions such as rain, fog, snow, haze, storms, and low lighting effects.

The weather has a number of negative consequences for mobility and traffic. Precipitation is recorded roughly 11.0% of the time worldwide on average [7]. Studies have unequivocally shown that, in comparison to typical weather, rainfall can cause a 70% increase in the likelihood of accidents [8]. Moreover, snowfall occurs in 77% of the countries worldwide. For example, according to US national data, annually, 24% of vehicle accidents related to weather conditions take place on slippery, snowy, slushy, or icy roads, while 15% occur when snow is actively falling or mixed with sleet [9], highlighting the real risks associated with snowy weather. Visibility is greatly reduced by environmental variables such as fog, haze, sandstorms, and strong sunlight, which presents real difficulties for drivers [10].

Traditional and deep learning (DL) algorithms are crucial for enhancing OD in autonomous vehicles, especially in adverse weather. While traditional AI, based on statistical learning, has been foundational, it faces limitations due to manual feature engineering and limited adaptability to dynamic environments, requiring extensive retraining and hindering its effectiveness in real-time applications.

DL offers a powerful alternative to traditional machine learning methods for OD in AVs, addressing the limitations of manual feature engineering and adaptability in dynamic environments. DL’s multi-layer neural networks automatically extract features and learn from data, providing a more flexible and adaptable solution that enhances the performance of surveillance systems, self-driving vehicles, and smart city applications [11]. DL’s neural network-based approach is particularly adept at handling complex models that traditional techniques cannot [12]. In AV object detection, hybrid models combining one-stage and two-stage detectors are effective, with two-stage models focusing on accuracy and one-stage models on speed, although the reasons for the latter’s lower accuracy are not fully understood.

The research significance of this paper lies in its comprehensive analysis of the challenges and advancements in OD for AVs in adverse weather conditions. It provides a critical review of both traditional and DL approaches, offering insights into the limitations and potential improvements of the current detection algorithms. The paper contributes to the broader field of intelligent transportation systems by emphasizing the need for robust and reliable detection systems that can operate effectively in a variety of weather scenarios, which is crucial for the safe deployment of AVs in real-world conditions. Building on this significance, this paper also contributes to the discussion regarding AV architecture and challenges in adverse weather, and reviews the literature on detecting pedestrians, vehicles, and road lanes using traditional and DL methods. It also summarizes common evaluation metrics for OD. In this paper, we contribute to the field by examining the fundamental architecture of AVs and the specific challenges they face in adverse weather conditions. We have compiled comprehensive datasets, leveraging real-world statistics from LiDAR and camera sensors, to provide a robust foundation for our analysis. We detail the core structure of OD systems and elucidate both traditional and DL methodologies for AVs. Building upon these approaches, we provide a critical review of the existing literature, focusing on the detection of three primary objects—pedestrians, vehicles, and road lanes—under challenging weather conditions. The structure of this paper is shown in Figure 3.

2. Related Survey Papers

This section critically evaluates the existing literature on autonomous vehicle (AV) sensor performance under challenging weather conditions, with a focus on the integration of traditional and DL detection techniques. One study [13] presents an in-depth analysis of the impact of inclement weather on AV sensors, particularly the attenuation and backscatter effects on millimeter-wave radar in heavy rain. Despite the study’s novel modeling approach, it does not comprehensively address the multifaceted challenges posed by other weather phenomena, such as fog and snowfall, which is crucial for a holistic sensor performance evaluation. Research [14] offers a comprehensive review of sensor performance in adverse weather conditions, emphasizing the importance of sensor fusion for improved vehicle perception. However, the study’s focus on range and resolution does not fully explore the nuanced challenges of object classification and sensor reliability in cold weather conditions, which is critical for AV operation. The study in [15] aims to develop a robust operational system for AVs in various weather conditions, advocating for sensor fusion and infrastructure support. Yet, the study’s discussion on sensor reliability does not adequately address the practical considerations of sensor installation costs and benefits, which are significant factors in the practical implementation of AV technology. The study in [16] investigates the effects of adverse weather on LiDAR and camera lenses in AVs, suggesting improvements in sensor hardware and the application of sophisticated machine learning methods. While the study’s focus on sensor fusion and network infrastructure is commendable, it does not provide a comprehensive evaluation of these technologies across different weather conditions. The research in [17] emphasizes the role of weather recognition and categorization systems in AV decisionmaking, focusing on the detection of road lanes, vehicles, and pedestrians. While the study provides valuable insights, it acknowledges that detecting smaller, obscured, or partially visible objects remains a challenge, indicating a need for further research in this area. The study in [18] serves as a tutorial on advanced techniques addressing the impact of rainy conditions on AV object detection. The study evaluates the performance of detection methods under clear and rainy scenarios, highlighting the efficacy of deraining methods, DL-based domain adaptation, and image translation frameworks. However, it does not comprehensively address the limitations of these methods in other adverse weather conditions. The study in [19] examines vehicle detection approaches in video-based traffic surveillance, highlighting the challenges posed by dim illumination, variable weather, occlusion, and shadows. The study’s exploration of traditional approaches is valuable, but it does not fully integrate the advancements in DL methods for improved detection. Other [20] research focuses on DL methods for detecting on-road vehicles, providing a valuable list of works. However, the study’s limited scope does not encompass the full range of challenges faced by AVs in adverse weather conditions. Another paper [21] exclusively examines 3D methods for vehicle detection, offering insights into a specific detection approach. Yet, the study’s narrow focus does not address the broader context of AVs and their performance under various weather conditions.

In summary, while the surveyed literature provides a valuable foundation for understanding AV sensor technologies and detection techniques, there is a critical gap in the holistic examination of both traditional and DL approaches under adverse weather conditions. This survey paper focuses on traditional approaches, deep learning approaches, and their applications in object detection in AVs under adverse weather conditions. Furthermore, we have detailed our literature review methodology, which involves a systematic search across reputable digital repositories such as Springer Link, Elsevier, IEEE, MDPI, and Wiley Online Library. We utilized targeted keywords to identify relevant research articles and focused on studies published in peer-reviewed journals and international conferences from 2001 to 2023. This approach ensured a comprehensive and high-quality selection of literature for our review.

3. An overview of AVs

The Society of Automotive Engineers (SAE) has established a classification system that divides Autonomous Driving Systems (ADS) into six levels of automation, ranging from Level 0 (No Automation) to Level 5 (Full Automation), as illustrated in Figure 4 [22]. This system is pivotal in understanding the evolution of autonomous vehicles (AVs), which are equipped with advanced sensors and software to navigate independently, thereby enhancing safety technologies and autonomy. Such a progression necessitates a collaborative effort among scientists and engineers across various disciplines to address the complex challenges associated with AV development [23]. The SAE automation levels provide a standardized framework that is critical for evaluating the capabilities and limitations of AVs at different stages of their development. By categorizing AVs into these distinct levels, it becomes possible to systematically assess the technological advancements and hurdles encountered at each level. This structured approach is instrumental in setting realistic expectations about AV performance, shaping regulatory frameworks, and educating the public on the operational capabilities of AVs under various scenarios. Thus, the classification of AVs according to SAE levels is essential for advancing the field of autonomous driving, guiding its regulatory landscape, and informing societal understanding of these emerging technologies.

Initially, in the 1920s, AVs were referred to as “phantom autos” due to their lack of a human driver, relying on remote control [24]. Significant advancements in AV technology emerged in the 1980s, notably with Pomerleau’s “Autonomous Land Vehicle in a Neural Network” project [25]. This project demonstrated the potential of neural networks to significantly enhance autonomous navigation systems. The Defense Advanced Research Projects Agency (DARPA) initiated the inaugural Grand Challenge in 2004 with the primary aim of stimulating research and the advancement of AVs. Subsequent to the 2004 challenge, DARPA hosted additional grand challenge events in 2005 and 2007, famously recognized as the Urban Challenge [25]. Developments in recent times have resulted in the global implementation of AVs. Some manufacturers have achieved Level 4 SAE standard [26] autonomy through the use of LiDAR technology. Google started secretly building its first AV in 2009, and, on May 1, 2012, in Las Vegas, they successfully completed their first autonomous driving test [27]. The UK government launched a competition in 2014 with the goal of encouraging and developing AVs [28]. The Mcity driverless shuttle project was started recently by the University of Michigan. It is the first Level 4 automated shuttle project in the US [29]. However, due to safety considerations, AVs cannot operate in severe rain or snow. Although testing and research have been carried out in inclement weather, problems such as the persistent usage of windshield wipers may result in an operational stoppage. Low temperatures impacted charging during the night and limited daily operating hours for the Sohjoa Baltic Project [30]. Tesla’s autopilot works well in mild rain and snow and has obvious lane markers, but it struggles in inclement weather [31]. Regulation (EU) 2019/2144 was introduced by the European Parliament and Council in 2019. This was the first time that guidelines pertaining to automated and fully autonomous vehicles had been established [32]. When Waymo (formerly Google’s self-driving division, rebranded as Waymo in 2016) announced that their “Waymo Driver” had successfully completed 20 million self-driven miles and carried out simulations that equated to 15 billion miles, it marked an important turning point in the growth of AVs [33]. While researchers are striving to develop sensors for different weather scenarios, thorough studies that address every facet are still absent. The AVs use environment-sensing skills to function with the least amount of human interaction [34]. Using an autonomous driving framework, the weather has a significant impact on the data obtained from car sensors and the general state of the surroundings. Adverse weather conditions create obstacles to object identification, monitoring, and localization, necessitating modifications to control and planning techniques. Furthermore, the efficiency of the vehicle can be directly impacted by meteorological factors like wind, general weather, and road conditions.

This interaction results in an ongoing cycle whereby changes in the environment and vehicle states impact one another, highlighting the weather’s crucial role in autonomous driving. Figure 5 shows the architecture of AVs.

3.1. Sensor Technologies

In the pursuit of improving OD in inclement weather, selecting the right sensor technologies is essential to guaranteeing precise and dependable results. Unpredictable natural events, like adverse weather, might have an impact on the operating environment of AVs. These changes in the surrounding environment bring about differences in how AV sensor systems, which are the fundamental parts of ADS, operate. The main sensory elements used in AVs for perception are described in the section that follows, and the comparison of sensors is shown in

Table 1. Comparison of sensors [35].

Sensors	Advantages	Disadvantages
LiDAR	High resolution Long range Wide FOV	Sensitive to weather Expensive
Radar	Long range Detection of velocity Suitable for all types of weather	Low resolution Very sensitive
Camera	High resolution Detection of colors	Sensitive to weather Sensitive to lighting
Ultrasonic	Non-InvasiveReal-time feedback	Low resolution Expensive

.

3.1.1. LiDAR

LiDAR is considered the primary perceptive sensor in the self-driving car industry. Even though 3D-LiDAR has only been used in automobiles for slightly over ten years, it has already been shown to be crucial to the development of AVs and ADS [36]. Its exceptional ability to measure things precisely and its resilience to changing lighting conditions highlight how vital it is. Critical features of 3D scanning using lasers include its form aspect, affordability, tolerance to external variables, measuring range, precision, pinpoint density, scanning acceleration, configurational versatility, and spectral qualities [37]. These sensors work well in a range of weather conditions by using laser beams to measure distances and produce accurate 3D point clouds. However, there could be problems when there is a great deal of rain, snow, or fog since laser beams can disperse and potentially skew the data. Older types, such as the Velodyne HDL64-S2 [38], continue to function better in dense fog, while the performance of more modern laser scanners decreases. While making manual modifications can increase performance, it is still difficult to perceive reliably in deep fog. For this reason, alternative technologies, including gated imaging, should be considered for better performance in adverse conditions [39].

3.1.2. Radar

The fundamental components of ADS and self-driving automobiles are automotive radars and a variety of additional sensors. The successful use of these technical advances depends on complex systems with long signal-processing chains that link the radars and sensors to the controller. The vital tasks of identifying objects and obstructions, pinpointing their exact location, and evaluating their relative speed in relation to the vehicle fall to automotive radar equipment. Advances in millimeter-wave (mm-wave) semiconductor technology, along with the development of signal processing techniques, form the basis for the progress of car radar systems. A variety of signal processing methods have been developed to improve resolution and reliability in a range of measuring parameters, including target velocities in the vicinity, azimuth elevation perspectives, and the radius of the vehicle [40]. Frequency Modulated Continuous Wave radar, or FMCW radar, modifies the broadcast signal’s frequency continually at a predetermined pace. By measuring the gap between the sent and reflecting signals, which is inversely related to the amount of time required for the electrical signal to make its way to and from the object being targeted, we can determine the range due to this modulation of frequency. FMCW radar has excellent accuracy and range resolution, as well as speed measurement features, which make it especially useful for autonomous driving applications [41].

3.1.3. Ultrasonic

One typical vehicle sensor that is frequently left out of discussions regarding ADS is the ultrasonic sensor. Despite this error, it is positioned strategically on the vehicle’s bumpers and throughout its body, making it indispensable for parking assistance and blind spot surveillance. Surprisingly, ultrasonic sensors have proven to be dependable and reasonably priced over a long period of time, which makes them a useful sensor choice [42]. The frequency range of ultrasound sounds, which are audible only to humans, is normally between 30 and 480 kHz. In the field of ultrasonic sensing, the frequency range that is most frequently utilized is between 40 and 70 kHz. The resolution and sensor range are highly dependent on the selected frequency. Longer sensing ranges are correlated with the lower ones. For example, the measuring range is up to 11 m, and the accuracy is one centimeter (cm) at the commonly used frequency of 58 kHz. Conversely, higher frequencies such as 300 kHz provide amazing resolution, maybe down to one millimeter, but at the penalty of a shorter range, capped at about 30 cm [43]. Ultrasonic sensors are useful for close-quarters applications like parking because their normal operating range is 11 m [44]. However, they can be used in autonomous driving; for example, Tesla’s “summon” capability can be used to navigate garage doors and parking lots [45].

3.1.4. Camera

As the eyes and storytellers of automobiles, cameras are essential components of ADS. They are a crucial component of ADS, capturing the dynamic story of the surroundings, even though they are technologically more advanced than LiDAR. Installed on Windows, dashcams record continuously and have made a substantial contribution to the initial ADS datasets. Fisheye-lens professional camera setups nowadays increase the volume of data that may be collected. On the other hand, adverse weather can cause visual problems for cameras due to rain, snow, and fog. In low light, specialized cameras improve visibility, such as night vision versions. The potential to overcome weather-related constraints in sensor fusion methods through the integration of these cameras is encouraging. The function of cameras will change further as autonomous driving technology develops, influencing sensor technologies and how they interact with the outside world. In summary, cameras are the storytellers of autonomous cars; they are sensitive to bad weather yet versatile enough to be essential to safer autonomous systems.

3.2. Actuators

AVs use data from the planning stage to carry out the vehicle’s motions in the control stage. This entails giving actuators instructions for steering, braking, accelerating, and signaling. A drive-by-wire method is frequently used to transmit commands effectively. The control system is in charge of creating and monitoring trajectories, making sure that intended routes are taken. Methods for generating trajectory include sensor-based, which is appropriate for robotics, and dynamics-based, which is more appropriate for vehicles. The majority of the time, trajectory tracking uses geometrical or model-based techniques. Although they have limits and only react to errors as they happen [46], feedback controllers like the proportional-integral-derivative (PID) are employed to prevent deviations from intended trajectories. Two-degree independent controllers combine feedforward and feedback controllers to circumvent these restrictions.

3.3. Perception

To highlight the importance of perception for AVs, it is important to emphasize how it helps the AV determine its location and its surroundings. In order to extract road features, detect objects, anticipate their behavior, and carry out Simultaneous Localization and Mapping (SLAM) activities, the perception module uses data from sensors and communication equipment [47]. For detecting the surroundings, a variety of active and passive sensors, such as cameras, LIDAR, and RADAR, can be employed; each has advantages and disadvantages of its own, as shown in Table 1. Emphasized are the benefits and drawbacks of various sensors, illustrating the compromises made in terms of resolution, weather sensitivity, and accuracy. One way to overcome the limits of individual sensors is to introduce sensor fusion, a technique that combines data from several sensors. There are two main types of AV systems discussed: one is sensor fusion, which uses various sensors such as LIDAR, RADAR, and cameras, and the other is pure vision, which uses cameras and computer vision algorithms. It is mentioned that a pure vision-based system has advantages like affordability and simplicity of implementation. In the parts that follow, this research concentrates on the benefits of a pure vision-based method [48].

3.4. Planning and Control

Global route planning and local path planning are the two main planning challenges in autonomous driving. When given an origin and destination, the global planner’s job is to come up with possible road options and routes. Local path planning, on the other hand, concentrates on carrying out the chosen path without running into obstructions or breaking any traffic laws. Other modules, such as OD, which guarantees safe navigation, frequently supplement this role. Further details regarding motion planning can be found in [49] for a more thorough explanation. Like traditional cars, steering, throttle, and brake inputs are the main controls for AVs [50]. These inputs control the AV’s direction, speed, and acceleration, which in turn control its driving, under the supervision of the ADS mainframe’s judgments based on sensor data and analysis. Low-level safety precautions can also be put in place using sensors like sonar and electrical control units. ADS sensors are challenged by weather conditions, which interfere with perception functions and have an impact on detection, planning, and control, among other factors. Weather factors like wind and variations in the road surface have an immediate effect on the vehicle and its state, in addition to sensor-related problems. Environmental state changes result from these state changes, which also have an impact on other nearby cars and sensor operations.

3.5. Challenges for AVs in Adverse Weather

Automation parts up to Level 3 have been added by automakers, and they rely on sensors and cameras. These devices function best in optimal conditions; thus, unfavorable weather presents obstacles. Weather has a major impact on AV performance, regardless of whether it is road-related or generic. A few of these effects are shown in Figure 6, and these are explained in more detail below.

3.5.1. Performance of Sensors

Adverse weather conditions on roads provide serious difficulties for sensors in AVs. The current sensing technologies, including LiDAR and cameras, perform well in clear weather but have trouble when the roadway is covered with snow and may contain erroneous reflections caused by rain or snow, making it difficult to discern between different objects [13]. Even though forward radar is good at spotting large objects and piercing through weather, it cannot improve safety applications on its own. In snowy circumstances, ultrasonic detectors may set off false alarms, and glare and surface reflection are just two examples of weather-related variables that might affect sensor accuracy.

3.5.2. Performance of Cameras

While camera systems are crucial for applications involving driverless vehicles, unfavorable weather conditions such as fog, intense rain, snow, and dust reduce their effectiveness. Although sophisticated techniques for image processing try to solve these problems, accuracy and dependability issues still exist, which presents significant obstacles to the advancement of autonomous driving [51]. In the right circumstances, glare can potentially affect functionality. However, these systems may be able to identify hazardous weather conditions immediately.

3.5.3. Technical Parameters of the Vehicle

Adverse weather affects a vehicle’s functioning factors. Slow speeds and longer following distances are necessary for safety due to decreased road friction and slick conditions. As was previously mentioned, during bad weather, driver behavior can also have a parallel impact on vehicle operating parameters.

3.5.4. Behavior of Driver

Driver conduct is still crucial once AVs are deployed, particularly in situations with mixed traffic. In order to ensure operational safety, AV applications need to take into account the existence of other drivers and environmental elements. These factors include driver-state surveillance and the ability to react to different weather conditions.

3.5.5. V2V, V2I, and VLC Communications

The influence of meteorological conditions on network latency can sometimes prevent vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications from connecting in a direct line of sight. For safety applications, minimal latency is essential. While radio-based communications are less impacted by snow, they are nevertheless susceptible to problems like solar flares and thunderstorms. Communication interruptions can result from bad weather such as fog, heavy rain, snow, and hail obstructing the visible light communication (VLC) line of sight.

The process of OD in AVs unfolds in several key steps, beginning with the collection of image data by the sensor system. Following this, a Region of Interest (ROI) or regional proposal strategy is employed, marking a critical phase in object tracking. This approach is commonly utilized in devices such as cameras and stereoscopic systems, where ROIs serve as the foundational elements for tracking objects. The subsequent step involves the use of classifiers to meticulously analyze and refine shapes, edges, and lines. This analysis aids in the classification of targets and their identification based on human categories, embedding these ROIs within the image for object recognition.

Various techniques, including sliding windows, Locally De-correlated Channel Features (LDCF), and constrained searches, are applied to locate ROIs efficiently. The extraction of features from these ROIs represents the third stage of the process. At this juncture, either DL-based methods or traditional manual OD algorithms can be employed for the extraction and classification of features. While manual feature extraction techniques can be advantageous for models that rely on lower-level features by facilitating manual ROI suggestions [52], their efficacy diminishes when dealing with complex features, rendering them less reliable. However, to derive useful and relevant data from video or photographs using a computing device, a great deal of computation has to be completed [53]. In contrast, DL methodologies enable systems to autonomously learn feature representations, thereby enhancing the accuracy and adaptability of feature extraction in dynamic and complex driving environments.

The evolution of research on OD in bad weather depends heavily on datasets. They act as the building blocks for creating, refining, and testing algorithms and models. In this part, we are going to discuss a few of the datasets that are frequently utilized in this field of study. One of the most often used benchmarks for autonomous driving research is the KITTI [54] dataset. It features a range of various metropolitan settings. The collection offers LiDAR data, excellent-quality pictures, and annotations for cars. A2D2 [55] is a sizable dataset and delivers extensive annotations for several items, including vehicles, together with high-resolution camera and LiDAR data. Waymo’s dataset [56], which is renowned for its comprehensive and varied gathering of data from AVs and includes information from a range of weather and lighting circumstances, is a useful tool for research on inclement weather. Among the noteworthy datasets, NuScenes [57], BDD100K [58], ApolloScape [59], and SYNTHIA [60] datasets offer diverse and extensive resources for exploring vehicle detection in challenging conditions. A quick overview of the weather conditions considered and the particular sensors utilized for gathering data for each dataset can be found in

Table 2. Publicly available AVs datasets.

Ref.	Year	Dataset	LiDAR	Radar	Camera	Adverse Weather
[37]	2020	LIBRE	✓	-	✓	Rain, fog, Daytime
[51]	2013	Kitti	✓	-	✓	-
[54]	2021	ACDC	-	-	✓	Rain, fog, snow
[55]	2020	A2D2	✓	-	✓	Rain
[56]	2020	Waymo	✓	-	✓	Rain, night
[57]	2020	NuScenes	✓	✓	✓	Rain, night
[58]	2020	BDD100K	-	-	✓	Daytime, night
[59]	2020	ApolloScape	✓	-	-	Rain, Night
[60]	2016	SYNTHIA	-	-	-	Snow
[61]	2016	NCLT	✓	-	✓	Snow
[62]	2021	CADCD	✓	-	✓	Snow
[64]	2020	End of the earth	✓	-	✓	Snow
[65]	2020	nuImages	-	-	✓	-
[66]	2019	Argoverse	✓	-	✓	-
[67]	2019	Astyx	✓	✓	✓	-
[68]	2020	DENSE	✓	✓	✓	Rain, snow, fog, night
[69]	2018	Foggy Cityscape	-	-	-	Fog/haze
[70]	2020	Berkley DeepDrive	-	-	✓	Rain, fog, snow, night
[71]	2017	Mapillary	-	-	✓	Rain, fog, snow, night
[72]	2019	EuroCity	-	-	✓	Rain, fog, snow, night
[73]	2017	Oxford RobotCar	✓	✓	✓	Rain, snow, night
[74]	2020	A* 3D	✓	-	✓	Rain, night
[75]	2021	4Seasons	-	-	✓	Rain, night
[76]	2018	WildDash	-	-	✓	Rain, fog, snow, night
[77]	2018	KAIST multispectral	-	-	✓	Daytime, night
[78]	2021	Radiate	✓	✓	✓	Rain, fog, snow, night
[79]	2020	EU	✓	-	✓	Snow, night
[80]	2022	Boreas	✓	✓	✓	Rain, snow, night
[81]	2020	DAWN	-	-	-	Rain, snow, fog, sandstorm
[82]	2021	PVDN	-	-	✓	night

. The University of Michigan used a Segway robot to collect four seasons of LIDAR data for the NCLT [61] on campus. The first AV dataset, called the Canadian Adverse Driving Conditions (CADC) [62] dataset, focuses on snow conditions. The Challenging Conditions Dataset with Correspondences (ACDC) [54], created for training and testing lexical techniques for segmentation in difficult visual conditions, was later introduced by the same team. ACDC includes a high-quality pixel-level resolution collection of well-labeled photos encompassing various categories of mist, at night, rain, and snowfall. The public’s Digital Video Collection now includes the IUPUI [63] Driving Video/Image Benchmark. Example views of varied lighting and roadway conditions captured by in-car webcams are included in this baseline. In these movies, patrolling and disaster response situations involving safety automobiles are shown. Together, these datasets provide priceless resources for developing the discipline and promoting creativity in the creation of reliable and weather-adaptive vehicle detection systems.

4. Overview of Object Detection in AVs

Most OD algorithms primarily adhere to a common framework, as shown in Figure 7.

In OD, we have considered the following three issues:

• Vehicle Detection: Vehicle detection is the process by which AVs identify and locate other vehicles on the road. This capability is crucial for AVs to make informed decisions about their own movement, such as maintaining safe distances, changing lanes, or responding to traffic situations.
• Pedestrian Detection: Pedestrian detection involves the recognition and tracking of people walking near or crossing the road. This is a vital safety feature for AVs as it enables the vehicle to anticipate and react to the presence of pedestrians, preventing collisions and ensuring the safety of both the vehicle’s occupants and those outside.
• Road Lane Detection: Road lane detection is the ability of AVs to identify and understand the position and orientation of road lanes. This information is essential for the vehicle to navigate within its designated lane, follow traffic rules, and make correct turns, ensuring a smooth and safe driving experience.

5. Overview of Traditional and DL Approaches for Object Detection in AVs

The detection techniques are composed of three parts: the DL approach, the traditional technique, and a hybrid approach that utilizes both. The DL and traditional techniques are explained in detail in the section that follows. The performance graph of both approaches is shown in Figure 8.

5.1. Traditional Approach

Various algorithms were developed for the purpose of OD. Prominent traditional feature extraction methods include Scale Invariant Feature Transform (SIFT) [83], Viola–Jones rectangles [84], Haar-Like-wavelets and Histogram of Oriented Gradient (HOG) [85], Adaptive Boosting (AdaBoost), non-linear Support Vector Machines (SVMs), and linear SVMs, which are common classical approaches for object classification. A number of steps are involved in the SIFT algorithm [83], which is well known for its ability to determine scale and rotation-invariant features, making it resistant to partial occlusion, clutter, noise, and lighting variations. These include the detection of scale-space extrema, the localization of key points, the assignment of orientations, the creation of keypoint descriptors, and the final step of using key point matching to recognize objects. Descriptors are used to compare and identify items in a database using methods like nearest neighbor indexing and the Hough transform. Nevertheless, SIFT has some drawbacks, including the absence of ongoing key point consistency in dynamic objects, the significant dimensions of feature descriptors that can affect matchmaking, and some restrictions related to patents.

The Viola–Jones approach (Viola and Jones, [84] was developed for human face identification using Haar-like feature extractors. It consists of four primary parts; the primary image algorithm for effective image participation is AdaBoost. Haar-like feature extraction is used to uncover biased features. Cascade classifier implementation is used to filter out background data and concentrate on regions that are more likely to contain the object of interest. The algorithm performs remarkably well, taking only 0.067 s to process a 384 × 288 pixel image. Its detection rate is 91.4%, and it produces 50 false positives. Nevertheless, it is not well adapted for the generic recognition of objects and has slow periods for training. The HOG method was created for identifying humans in digital photos and was first presented in [85]. In order to determine the gradient’s length and direction, this method examines input images employing gradient filters, both horizontal and vertical. The filtered images are divided into 8 × 8 pixel cells, which are then further divided into 2 × 2 cell blocks that have 50% overlap. For every cell, orientation histograms are produced, quantizing gradient directions into nine bins ranging from $0°$ to $180°$. The magnitudes of these bins are used as votes. A vector representing the HOG is produced by concatenating these histograms for every cell in a block. To take into consideration changes in illumination and contrast, these vectors are standardized. The final HOG descriptor, which consists of all the normalized blocks, is fed into a Support Vector Machine (SVM) to determine if an image is human or not. Although the HOG algorithm’s dense grid method results in a larger processing burden and less efficient performance in cases involving obscured objects, it is effective in decreasing false positives when compared with Haar wavelets.

These AI techniques, which have their roots in statistical principles, have traditionally formed the basis of machine learning models. These techniques enable computers to recognize trends and forecast outcomes based on past data. However, even with their historical importance, traditional methods have clear drawbacks. Reliance on manual feature engineering, as shown in Figure 9, in which experts carefully craft features for the model necessitates domain knowledge and frequently fails to capture the nuances of intricate datasets. Furthermore, these models’ ability to adjust to unfavorable weather or dynamic surroundings is hindered by their inability to quickly incorporate new data without requiring significant retraining, which limits their usefulness in situations that change quickly. The development of more sophisticated methods, or DL approaches, to overcome these obstacles and improve the power of AI systems has been made possible by this realization of their limitations.

5.2. DL Approaches

DL emerged as a subfield of machine learning and artificial intelligence in the 1990s [86]. Unlike traditional methods, DL offers distinct advantages, including the ability to achieve higher levels of abstraction, improved precision, and faster execution. These advantages make DL a valuable choice for OD. The OD algorithm employing DL typically consists of three core components: a recurrent neural network (RNN), a depth belief network (DBN), and a convolutional neural network (CNN). Overall, object detectors based on CNN [87] can be categorized into two primary types.

1. Single or one-stage detectors are known as the “non-regional proposal method” and “dense prediction”.

2. A two-stage detector is known as the “regional proposal method” and “sparse prediction,” as shown in Figure 10. In a single-stage detector, all the tasks are integrated into a unified system structure. Conversely, a two-stage detector separates the system into distinct stages for region selection, classification, and localization. Some regional proposal techniques consist of region-based CNN (R-CNN) [88], Fast R-CNN [89], Faster R-CNN [90], Spatial Pyramid Pooling Networks (SPPNet) [91], and Feature Pyramid Network (FPN) [92]. On the other side, the non-regional category includes Single Shot Multibox Detector (SSD) [93], You Only Look Once (YOLO 1-8) [94], EfficientDet [95], DEtection TRansformer (DETR) [96], and Fully Convolutional One-Stage (FCOS) [97]. Since these methods constitute the foundation of CNN, they have emerged as the standard for OD. The amalgamation of one-stage and two-stage OD techniques has gained prominence in the field of OD within AVs. These hybrid approaches have demonstrated effectiveness across diverse scenarios, achieving promising results in precise OD and localization. Two-stage algorithms tend to deliver superior accuracy, while one-stage algorithms offer faster processing speeds. Notably, the reasons behind the lower accuracy of one-stage algorithms remain unclear. A study examining the drawbacks of one-stage algorithms, especially those with dense sampling, was conducted in Ref. [98]. The study found that there was a major problem with the unbalanced background samples (negative instances) and foreground values (positive examples).

To solve this problem, the authors created Focal Loss, which alters the traditional cross-entropy loss function to give problematic cases more weight and thereby puts the emphasis on positive samples. After this adjustment, the RetinaNet network was created, showing better accuracy and quicker detection times than one- and two-stage methods. It should be mentioned too that the approach necessitates adjusting the focusing value, an extra hyperparameter [99]. However, the two primary state-of-the-art methods for DL-based detection these days are the YOLOv8 from the YOLO series (YOLOv1–YOLOv8) and the Faster R-CNN from the R-CNN family.

In order to speed up the region proposal step and allow the system to acquire knowledge of optimal region proposals, Ren et al. [90] proposed Faster R-CNN as an improvement to Fast R-CNN. The architecture of Faster R-CNN is described in Figure 11. This was achieved mainly by integrating the region proposal network (RPN). RPN’s main objective is to produce proposals with various ratios of aspects and sizes. Similar to Fast R-CNN, it employs high-resolution feature maps as input for RPN to identify the regions of interest in an image rather than depending on selective search for region recommendations. While Faster R-CNN improves the precision of detection and reaches an evaluation time of 0.2 s (five frames per second), it still requires a large amount of processing power and cannot fully meet the real-time system demands. It is also more complex because it requires training two interconnected neural networks, which makes system optimization difficult [100]. While Faster R-CNN algorithms demonstrate satisfactory accuracy on COCO and ImageNet datasets, they encounter difficulties when it comes to detecting small objects and objects at varying scales. This approach, while being more memory-efficient and faster, faces challenges in achieving consistent performance during both training and test-time inferences.

Jocher [100] implemented YOLOv8; however, that work has not yet been published in a journal. The structure of YOLOv8 is shown in Figure 12. Different versions are available with different scales of YOLO (N, S, M, L, and X) according to the scaling coefficient. The two main differences are the variety of kernels at different locations in the network and the number of feature extraction modules. These modifications are intended to meet the unique requirements of various situations. The backbone, neck, and head are the three primary modules that make up the YOLOv8 networks. First, the backbone module extracts features from the input photos. The neck component then performs feature fusion to create features in three different sizes: large, medium, and small. The final output is the detection results. Eventually, these characteristics of various sizes are sent to the head section for identification of the object. To improve deep image extraction features, the YOLOv8 system uses a layered C2f technique inside the backbone section. The main function of the SPPF layer, which is the tenth layer in the backbone module, is to transform input feature maps—regardless of their original size—into a fixed-size feature vector. The goal of this change is to increase the responsiveness of the network. YOLOv8 uses the traditional top-down as well as bottom-up feature fusion technique referred to as PAN (Path Aggregation Network) in the neck component. This tactic successfully combines elements from several scales. YOLOv8 adopts the widely used decoupled head architecture in the head section, which divides the categorization head from the identification head. Furthermore, there is a noticeable change in the network’s OD technique: the anchor-free strategy replaces the anchor-based method.

The distinction between traditional and DL approaches is justified by their fundamentally different methodologies and performance in various weather conditions. Traditional methods, while effective in ideal conditions, struggle with adverse weather due to their sensitivity to lighting, occlusion, and the need for precise feature engineering. DL methods, on the other hand, have shown superior performance in handling these challenges by learning robust representations from large datasets, which include diverse weather conditions. This allows DL-based systems to generalize better and maintain high detection accuracy even under challenging circumstances.

6. Traditional and DL Approaches for Object Detection in AVs

AVs need to be able to identify moving items like cyclists, livestock, pedestrians, and different kinds of automobiles, as well as stationary elements like traffic signals, road signs, and parked vehicles because pedestrians are especially vulnerable and because AVs frequently and critically engage with other types of vehicles. This section focuses only on the previous literature using traditional and DL approaches for vehicles, pedestrians, and road lane detection.

6.1. Traditional Approaches for Object Detection in AVs

In this section, we will summarize all the works on traditional approaches for OD in AVs.

6.1.1. Traditional Approaches for Vehicle Detection

Traditional vehicle detection techniques, which rely on appearance- and motion-based algorithms, have drawbacks, including being sensitive to changes in the camera’s position, having trouble telling moving objects apart, and requiring a large number of photos to accurately identify motion. Appearance-based algorithms that make use of features, such as HOG, Haar-Like, and SIFT, can improve identification, but they are vulnerable to exterior objects, adverse weather, and lighting variations. Even though they help to identify objects, biased classifiers like SVM and ANN might perform poorly in scenarios involving complicated traffic, which can result in false positives and make them susceptible to changes in the background, obstacles, lighting, and dimensions.

One study [101] investigated issues with physical detection techniques and equipment maintenance by examining the application of SVM and Pyramid Histogram of Words (PHOW) for traffic and road condition detection. The experimental findings show an accuracy of more than 82.5% on average under different weather circumstances. The suggested technique efficiently divides photos into four categories based on the type of traffic and road conditions. However, the paper does not detail the dataset’s diversity, which could impact the model’s robustness and generalizability. The research in [102] tackled the requirement for a reliable traffic surveillance system that takes into consideration variables like headlights, reflections, and shadows and operates well in a variety of weather and lighting circumstances. The suggested method is based on pattern classification and uses a distinctive image’s histogram as a distinguishing characteristic for classification. Because of its excellent generalization skills, SVM is used as a classifier. In tests, the system can discriminate between headlights and shadows with good accuracy and recognize moving cars in a variety of weather and lighting scenarios. However, this paper does not address the system’s performance under high-density traffic or its computational efficiency, which could be critical limitations for real-time applications. Using webcams as weather sensors, researchers [85] came up with a novel technique that allowed them to identify over 180,000 annotated photographs depending on indoor or outdoor circumstances with a 98% accuracy rate using SVM. The results of the research [103] indicate that DPM effectively addresses occlusion difficulties, even though it might not be required for ordinary detection settings. Accuracy is improved by adding image context information, and system speed is increased by adding other features. On the other hand, the multi-scale system only slightly improves accuracy, and the classifier has little effect on detector quality [21,104,105].

The vehicle detection method in [106], which utilizes pseudo-visual pursuit and HOG-LBP feature fusion, showcases the superiority of traditional machine learning techniques in achieving high detection accuracy and speed in severe weather conditions. This approach, with its reliance on radar information, demonstrates a clear advantage over methods that do not incorporate such advanced sensor data, particularly in scenarios where visibility is compromised. In contrast, the work by Abdullah et al. [107], while also effective, highlights the limitations of traditional methods when solely relying on MOG 2 background subtraction and histogram equalization. Their system’s high effectiveness rate of 96.4% across various weather conditions suggests that traditional techniques can still perform well. The vehicle detection system by Wu and Juang [108], which uses histogram extension and the Gray-level Differential Value Method (GDVM), shows that traditional methods can be effective in complex traffic conditions. However, the system’s reduced accuracy in rainy and dark conditions suggests a limitation of traditional approaches when dealing with significant environmental variations. The authors in [109] developed a system for concurrent vehicle and weather detection using optical flow and color histogram approaches. While this system demonstrates the utility of traditional methods in distinguishing between raindrops and normal conditions, it also indicates that these methods may struggle with the nuanced recognition tasks required in adverse weather.

Tian et al. [110] proposed a method for vehicle detection and tracking at night using HOG features and SVM for object recognition, effectively leveraging traditional machine learning techniques to enhance video surveillance in low-light conditions. Kuang et al. [111] achieved a high detection rate of 95.82% with minimal false positives in their multiclass system for nighttime conditions, employing tensor decomposition and object proposal methods. Furthermore, the research in [112] presents a system that integrates rule-based algorithms with a shallow neural network for nighttime vehicle detection, demonstrating significant improvements over traditional methods in terms of detection speed and proactive capabilities. These studies highlight the strengths of both traditional and hybrid approaches in the realm of nighttime vehicle detection, providing a foundation for future advancements in this field.

A vehicle tracking and counting system with camera shaking was presented in [113] for difficult sandy weather conditions. The method used particle filter tracking for vehicle detection, integrated headlight analysis, and background subtraction. While the method worked well in difficult situations, such as evenings and sandy weather, with a vibrating camera, it had trouble identifying white automobiles in snowy settings. Similarly, the background subtraction method encountered difficulties when there was movement in the backdrop or vibrations from the camera. A vehicle detection and tracking system for complex traffic was described in [114]. It used a background reduction technique with “low rank + sparse” disassembly and performed well in both qualitative and quantitative evaluations.

Table 3. Vehicle detection previous work with traditional approaches.

Ref.	Year	Technique	Weather Classes	Accuracy
[101]	2014	SVM+PHOW	Rain, Fog, Snow, Low light	82.5%
[106]	2022	HOG–LBP	several weather	96.4%
[108]	2012	histogram GDVM	Rain, Night	92.4%
[111]	2019	tensor decomposition	Night	95.82%
[113]	2017	Particle Filtering	Dust, Snow	94.78%

tabulates these works.

6.1.2. Traditional Approaches for Pedestrian Detection

This section delves into traditional techniques for pedestrian detection, comparing their effectiveness and limitations in various conditions. A notable study [115] presents a method that leverages an infrared camera for nighttime pedestrian detection, combining Haar-Cascade and Histogram of Oriented Gradients-Support Vector Machines (HOG-SVM) to reduce false alerts, demonstrating the superiority of this approach in low-light conditions. Another study [116] employed LiDAR point clouds and SVM for object clustering and classification in snowy environments, showcasing the robustness of this method for sidewalk snowplow machines at night. The real-world validation of this approach underscores its practical advantages over traditional vision-based systems in adverse weather. The work in [117] stands out for its real-time pedestrian detection system (PDS) that uses stereo-based segmentation and SVM to distinguish pedestrians from objects with vertical edge features. The high speed and detection rate of this method highlight its effectiveness, particularly in dynamic scenarios where camera motion is a factor. Research [118] addresses the challenge of detecting pedestrians from a moving vehicle in inclement weather, using SURF key points to compensate for camera motion and SVM for classification. The impressive performance of this approach, as evidenced by its speed and detection rate, underscores the potential of hybrid feature extraction methods in complex environments. Another study [119] combines Haar and HOG features to improve multi-person recognition and tracking, particularly in occluded and congested settings. The success of approaches in detecting both the side and frontal faces of pedestrians, as demonstrated through experiments on various datasets, highlights the limitations of single-feature approaches. The work in [120] introduces a novel method for pedestrian recognition in hazy environments by integrating a dark channel prior haze removal model with HOG descriptors. The improved performance of this method over conventional HOG-based techniques on the ’INRIA’ dataset indicates the benefits of adaptive preprocessing for robust detection in challenging conditions. With an emphasis on various weather situations, the study in [121] presents a robust vision-based pedestrian detection system for intelligent transportation systems. It incorporates feature extraction and SVM-based classification and addresses issues like variability and occlusions in real traffic photos. In a more constrained environment, the work in [122] demonstrates a pedestrian recognition method using SVM and HOG features, achieving an impressive 98.00% accuracy in counting individuals in a limited field of view. This high accuracy, achieved through training on a dataset of 2000 road photos, underscores the effectiveness of the model in controlled settings, potentially outperforming other approaches that do not account for such precision. Another study [123] provides a two-phase video recording pedestrian recognition method that combines local and global data with HOG features. The system’s efficacy in pedestrian detection is demonstrated by increased detection rates and decreased false positives obtained through testing with the PETS 2009 dataset.

6.1.3. Traditional Approaches for Road Lane Detection

In the study in [124], lane detection and sliding-mode control (SMC) were employed to demonstrate a novel approach for autonomous tracking control in intelligent electric vehicles (IEVs). This method, incorporating an optimal preview linear quadratic regulator (OP-LQR) with SMC and a 2-DOF vehicle model, outperforms traditional controllers in route tracking and lane marking recognition. Additionally, it enables efficient allocation of unequal braking force to all four wheels, showcasing superiority over conventional methods. Addressing the challenge of adverse weather conditions, Ref. [125] proposed a weather-resistant lane recognition method using horizontal optical flow, highlighting its capability for reliable obstacle assignment and lateral control essential for driver assistance and autonomous driving. Moreover, Ref. [126] presents an adaptive lane marking identification method for low-light conditions, demonstrating its accuracy and robustness compared to existing techniques. Similarly, Ref. [127] developed an advanced driver assistance system (ADAS) focusing on nighttime conditions, showcasing its effectiveness through successful highway tests. Lastly, Ref. [128] introduced a real-time lane detection technique for intelligent vehicles, highlighting its performance in complex road conditions and its potential for improving driving safety.

6.2. DL Approaches for Object Detection in AVs

DL has become much more popular and prominent than regular algorithms in recent times. This change in direction is attributed to DL’s exceptional work, reliable outcomes, and breadth of industry experience. Jones and Viola introduced the well-known VJ infrared technology, which improved immediate detection efficacy and capabilities [84]. In their research [129], Romero and Antonio mainly concentrated on defining DL methods. Previous relevant research on OD, e.g., vehicles, pedestrians, and road lanes, is examined in this section under adverse weather.

6.2.1. DL Approaches for Vehicle Detection

A key element of ITS is vehicle identification, which is used in many applications such as traffic monitoring, ADAS, AVs, and traffic data. Robustness, acceleration, precision, and cost-effectiveness are the hallmarks of an intelligent system. For this, a variety of imaging sources are used, including satellite imagery, in-car cameras, UAV cameras, and traffic monitoring equipment. This study is mostly focused on on-road vehicle surveillance, wherein in-car cameras are essential. Nonetheless, there are a variety of difficulties in the field of vehicle detection. There are several different classifications of vehicles, each having a distinct size, color, structure, and design. Additionally, they can be viewed at a variety of scales and positions, frequently in intricate traffic scenarios. Adverse weather conditions create extra challenges since they distort visibility and add noise to the sensors, which can result in missed objects and false alarms. This section will examine the major research works and methodologies that have influenced DL approaches for vehicle detection under adverse weather conditions, offering a basis for the discussion of novel strategies and developments in this crucial domain. Table 4 summarizes the previous work on vehicle detection under adverse weather conditions.

One paper [130] underscores the efficacy of CNNs in image processing through a tailored methodology for multiclass weather image categorization. Experimentation with various batch sizes (128, 256, 512, and 1024) is recommended to enhance accuracy and generalization. Testing on larger datasets enhances classifier accuracy and widens learning scope. In the realm of two-stage detectors, Ref. [131] introduces a practical approach for vehicle detection in challenging conditions like fog, low light, and sun glare, leveraging three trained CNNs (ResNet, GoogleNet, and SqueezeNet) with transfer learning. Notably, ResNet achieves a validation accuracy of 100%, while GoogleNet and SqueezeNet achieve 65.50% and 90.0%, respectively. Additionally, Ref. [132] employs Fast R-CNN for day and night vehicle detection, yielding impressive results under adverse conditions, validated by high recall (98.44%), accuracy (94.20%), and precision (90%). Furthermore, the on-road vehicle detection method proposed in [133] utilizes multiple region proposal networks (RPNs) in Faster R-CNN to identify vehicles across diverse sizes and weather scenarios, outperforming the existing techniques with high average precision rates on various datasets, including DAWN (89.48%), CDNet 2014 (91.20%), and LISA (95.16%). On the other hand, regarding one-stage detectors, another study [134] introduces an improved SSD-based front vehicle recognition algorithm for smart automobiles. Tested on the KITTI dataset, it achieves a high mAP of 92.18% and processes frames quickly at 15 ms per frame. The system enhances smart car safety in congested traffic and adverse weather conditions, prioritizing both accuracy and real-time performance. Another study [135] enhances a YOLO-based algorithm for precise vehicle detection in foggy weather by integrating a dehazing module with multi-scale retinex. This enhanced model, trained with augmented data, surpasses traditional YOLO in foggy conditions. Additionally, Ref. [136] proposes a modified YOLO model for on-road vehicle detection and tracking across various weather conditions. Utilizing a single CNN, it exhibits robustness and outperforms the existing techniques in intelligent transportation systems. Furthermore, Miao et al. [137] developed a nighttime vehicle detection method using fine-tuned YOLOv3, outperforming Faster R-CNN and SSD in terms of accuracy and efficiency. They achieved an average precision of 93.66%. Another study [138] utilized YOLOv4 with SPP-NET layers for vehicle detection, achieving an 81% mAP. In contrast, the study in [139] focused on harsh weather conditions, introducing YOLOv4 with an anchor-free and decoupled head, albeit achieving a 60.3% mAP and focusing exclusively on a single class. Moreover, the goal of [140] was to enhance self-driving vehicle detection in adverse weather using YOLOv5 with Transformer and CBAM modules, achieving an impressive mAP of 94.7% and FPS of 199.86. The DL approach proposed in [141] for nighttime vehicle detection in autonomous cars, combining a Generative Adversarial Network for image translation and YOLOv5 for detection, achieved a high accuracy of 96.75%, significantly enhancing the reliability of AV recognition models for night conditions. This study in [12] presents a DL-based intelligent AV weather-detecting system. Using a combined dataset from MCWDS2018 and DAWN2020, the performance of three deep convolutional neural networks was evaluated by categorizing six weather categories: overcast, rainy, snowy, sandy, sunny, and sunrise. The CNNs are SqueezeNet, ResNet-50, and EfficientNet-b0. The ResNet-50, EfficientNet-b0, and SqueezeNet models achieved 98.48%, 97.78%, and 96.05% accuracy, respectively, in the experiments, demonstrating remarkable accuracy rates while preserving quick inference times using GPU acceleration. By combining previously disparate datasets, the study’s novel methodology makes a significant addition to DL applications for autonomous systems’ weather detection.

In simple traffic situations, the aforementioned extraction and classification algorithms have shown themselves to be successful at detecting vehicles. However, issues including scale sensitivity, occlusion, and a high number of false positive results limit their efficacy in more complex traffic conditions [142]. These algorithms perform well in simple traffic scenarios, but they are limited in complex traffic conditions due to a significant number of false positive detections. As a result, an explanation connected with vehicle detection is intended for this part, and the difficulties associated are also highlighted.

Table 4. Vehicle detection previous work with DL.

Ref.	Year	Technique	Weather Classes	Accuracy
[12]	2022	SqueezeNet, ResNet-50, EfficientNet-b0	cloudy, rainy, snowy, sandy, shine, and sunrise	98.48% 97.78% 96.05%
[137]	2020	YOLOv3	night	93.66%
[140]	2022	YOLOv5	Fog, rain, snow, sand	94.7%
[130]	2023	CNN	Cloudy, Fog, Rain, Shine Sunrise	98%
[131]	2022	ResNet, GoogleNet, SqueezeNet	Fog, Low light, sun glare	100% 65.50% 90.0%
[133]	2021	Faster R-CNN	Rain, Fog, Snow, Sand	95.16%
[134]	2020	SSD	Sunny, overcast, Night, rain	92.18%
[141]	2022	YOLOv5	night	96.75%

Table 4. Vehicle detection previous work with DL.

Ref.	Year	Technique	Weather Classes	Accuracy
[12]	2022	SqueezeNet, ResNet-50, EfficientNet-b0	cloudy, rainy, snowy, sandy, shine, and sunrise	98.48% 97.78% 96.05%
[137]	2020	YOLOv3	night	93.66%
[140]	2022	YOLOv5	Fog, rain, snow, sand	94.7%
[130]	2023	CNN	Cloudy, Fog, Rain, Shine Sunrise	98%
[131]	2022	ResNet, GoogleNet, SqueezeNet	Fog, Low light, sun glare	100% 65.50% 90.0%
[133]	2021	Faster R-CNN	Rain, Fog, Snow, Sand	95.16%
[134]	2020	SSD	Sunny, overcast, Night, rain	92.18%
[141]	2022	YOLOv5	night	96.75%

6.2.2. DL Approaches for Pedestrian Detection

A useful deformation model algorithm for managing occlusions is the Deformable Part Model (DPM), which scores blocks or sections [143]. It prevents objects from obscuring pedestrians. Classification algorithms use object features to determine which objects are pedestrians and which are not. A DL- and feature-selection-driven pedestrian detection system (PDS) is presented in [144]. The way the technology works is that a camera installed on the car’s dashboard records the surroundings as a video. After that, this video data are put through a number of preprocessing stages until pedestrians are finally classified and localized. One of the paper’s key findings is that the composition and quality of the datasets for training and testing that are used in the detection phase have a direct impact on the system’s efficacy.

The research presented in [145] demonstrates a significant advancement in nighttime human detection using visible light cameras, particularly through the application of convolutional neural networks (CNNs). The method’s effectiveness is evident in its superior performance across various contexts, including open and self-constructed databases such as DNHD-DB1, KAIST, and CVC. This achievement is a notable improvement over previous techniques and is crucial for the development of intelligent surveillance systems. Another study [146] addresses the challenges of pedestrian identification in adverse weather conditions by introducing a novel DL network framework. This framework’s ability to adjust to different lighting conditions through multispectral fusion and illumination perception is a clear advantage. The integration of cloud computing and the Internet of Vehicles (IoV) ecosystem further expands its potential for real-time applications in autonomous vehicles, offering a versatile and adaptable solution. Furthermore, Ref. [147] introduces an image fusion module (MSRCR-IF) that significantly enhances the Mask R-CNN-based pedestrian identification in low-light conditions. The 4.66% improvement in detection accuracy, reaching 85.05% on a self-built dataset, is a substantial gain over the existing mainstream techniques. This advancement is particularly valuable for surveillance systems operating in challenging lighting conditions.

Table 5. Pedestrian detection previous work with DL.

Ref.	Year	Technique	Weather Classes	Accuracy
[141]	2023	YOLOv3	several weathers	74.38% (AP)
[145]	2017	CNN	night time	98.3%
[147]	2021	Mask R-CNN	lowlight	85.05%
[152]	2022	YOLOv5	Day, Night	96.6%
[153]	2023	YOLOv7, CNN, Faster R-CNN, SSD, HOG	Rain, Fog, Snow, Sand	0.73, 0.883, 0.71, 0.657, 0.70

summarizes the work. Another study [148] provides valuable insights into the challenges of data collection across diverse meteorological conditions and introduces the ZUT dataset, which boasts over 122k annotations. The research emphasizes the potential benefits of integrating Automobile CAN data with ADAS systems and suggests temperature-based normalization as a strategy for improvement. The recommendation to use 16-bit images for a 10.67% increase in detection accuracy and the consideration of car speed for parameter adjustments are noteworthy. The study also acknowledges the need for onboard precipitation sensors and a more extensive dataset that spans multiple seasons to enhance pedestrian detection accuracy, as demonstrated by the comparison with the SCUT dataset. The study in [149] focuses on metropolitan areas with a speed restriction of 48–70 km/h and showcases impressive performance on both rainy and dry roads. The method utilizes the SMP technique for depth calculation and stereo-vision technology with two cameras. The integration of data from various sensors, such as radar or LIDAR, is proposed to enhance autonomous driving performance. The research also advocates for the application of DL methods to refine feature extraction and categorization, achieving an AP of 89.85% and a mAP of 90% on the CVC-09 and LSIFIR databases. In the realm of nighttime pedestrian recognition, Ref. [150] introduces an enhanced Faster R-CNN architecture, which delivers significant improvements, particularly for distant pedestrians. The study in [151] tackles the issue of optical camera-based pedestrian recognition under adverse weather conditions by employing a cascaded classification technique that incorporates both local and global features, resulting in increased detection rates. The effectiveness of this strategy is demonstrated in challenging weather conditions such as snow or rain. The use of YOLO-v5 in [152] introduces a novel multispectral method for pedestrian detection, combining independent subnetworks for RGB and IR images to create an effective detector. The method’s performance on open databases, with high mAP scores and IoU of 0.5 across various datasets, is encouraging. and another study [153] evaluates YOLOv7, Faster R-CNN, SSD, and HoG for both vehicle and pedestrian detection in different weather conditions. YOLOv7’s superior performance in terms of accuracy, precision, recall, and mAP underscores the critical role of weather conditions in selecting appropriate DL methods for autonomous driving.

6.2.3. DL Approaches for Road Lane Detection

A color-based road lane-detecting system was used in research [154,155,156,157]. In favorable weather, Refs. [154,155] achieved above 90% accuracy. However, in adverse weather conditions, the accuracy considerably decreased. Some researchers used LiDAR technology for lane detection in order to reduce the impact of ambient influences on the detection algorithm. With the use of both images and LiDAR data, Shin et al. [158] developed a lane detection algorithm that combines camera and LiDAR information, improving the capacity to detect lanes in difficult environmental situations. By combining data from cameras, GPS, and LiDAR, the authors in [159] were able to develop a precise distance estimation between the vehicle and lateral lane that was accurate to the centimeter. However, the high cost of LiDAR and its limited sensitivity to unfavorable weather conditions make it difficult to apply to large-scale lane detection systems.

A different method [160] uses the HSV color space to achieve an accuracy of 86.21% regarding the rate of identification. This method works best in daylight or with white light because it takes advantage of the near-color correspondence between white and yellow lane markers. Consequently, global thresholding is necessary for precisely dividing color planes and obtaining lane markers in both the YCbCr and HSV color models. In contrast, in tunnel environments with distinct color illumination, the lane markers diverge from their real hues, making reliable detection difficult. Sattar et al. [161] developed a different technique called “SafeDrive,” which identifies the lane marking in areas with poor lighting. They located other distinct images of highways at the same place using the vehicle’s location information, and they used those images to identify the lanes. Other researchers [162] also carried out an additional study that, because of the solid and easily visible lane markings, was able to identify lanes in several types of weather conditions.

Using already installed roadside webcams, real-time road weather and road state detection algorithms were developed in [163]. Transfer learning was utilized to train detection models utilizing three previously learned CNN architectures. When it came to accuracy, ResNet18 outperformed the other models, scoring about 97% for weather and 99% for surface conditions. These models can be useful in improving road safety since they can automatically recognize and relate real-time circumstances to road networks without the need for human intervention. They can be incorporated into advanced traveler information systems (ATIS) for drivers and utilized to optimize winter maintenance routes. Additionally, the models could replace manual reporting in snowplows, increasing driver safety and accuracy. Research [164] presents a novel approach that combines computer vision and DL to extract meteorological data from street-level photos without any image restrictions. With recognition accuracy ranging from 91% to 95.6% across several categories, the study presents four deep CNN models designed to identify a variety of visibility circumstances, including dawn and dusk, daytime, evening, glare, and weather elements like rain and snow. A study [165] presents a DL-based method for lane marker detection in order to address the difficulties associated with classifying road markers during rainy weather. Specifically designed for bad weather, the approach gives priority to the best feature selection from video frames in order to counteract rain-induced blurriness, leading to adequate classification precision even in difficult weather circumstances. In other research [166], two different models for AV lane prediction in highway traffic situations are presented. The study’s main focus is on the use of AI for lane prediction using a sizable dataset from NGSIM. To showcase the effectiveness of these models, two distinct subsets with 5000 car samples each were used. The strategy employed a range of classifiers in the Identification Learner and different methods in Neural Net Fitting, emphasizing the methodology’s importance in accomplishing good lane prediction without diving into particular accuracy measures.

Due to different illumination circumstances, global thresholds for lane marker identification frequently produce disappointing outcomes. An efficient lane departure (LD) technique is presented, enabling lane marker identification that may be used in daylight highway scenarios as well as tunnel situations with artificial lighting. In order to obtain lane features, the usual LD technique begins by performing pre-processing to eradicate perspective-related distortion and isolate an ROI. In order to identify suitable lane markers using color, shape, alignment, or geometrical measurements using the road scenario data, two categorization methods, model-based and feature-based, are used [17]. To minimize false positives, prospective lane markers are further refined and validated using curved or linear lane fitting. This helps with features like adaptive speed control, automatic lane focusing, and lane departure alert. LD devices may not be as successful due to a number of issues, such as road obstructions, ambient lighting sources, and noisy weather. To address these issues, a shadow-resistant methodology that makes use of Fourier transformation and the maximally stable extreme region (MSER) approach in the blue color stream is used as a reference [167]. The research findings [168] suggest that the minimal safe separation between the self-absorbed car and the vehicle should be used to estimate the extraction of ROI. According to the study, a car traveling at 110 km/h may cover a distance of 35 m with a height of just 150 pixels. Usually, a predetermined model or set of features is used to construct lane markers using the ROI.

In [169], the authors modified YOLOv3 and presented a brand-new BGRU-Lane (BGRU-L) model; the method integrates spatial distribution with visual data. High accuracy (90.28 mAP) and real-time detection speed (40.20 fps) are achieved in difficult settings through integration utilizing the Dempster–Shafer algorithm, as demonstrated by datasets from KIT, Toyota Technological Institute, and Euro Truck Simulator 2.

The dynamic environment of research and innovation in the field of road lane detection for autonomous cars is shown by our thorough analysis of the literature. The abundance of information and developing techniques indicates the ongoing commitment to increasing the potential of self-governing systems.

6.3. Practical Implications

The practical implementation of DL models for pedestrian and vehicle detection in AVs is a critical area of research as it directly impacts the safety and reliability of these systems. Studies such as Kim et al. [145] have demonstrated the effectiveness of CNNs in low-light conditions, a significant advancement for nighttime AV navigation. Lai et al. [147] have further optimized the Mask R-CNN algorithm for low-light environments, enhancing detection rates and computational efficiency. Montenegro et al. [152] have shown that YOLO-v5 can be fine-tuned to maintain high detection accuracy across varying lighting conditions, ensuring consistent performance for AVs. Zaman et al. [153] have addressed the challenge of adverse weather conditions, a common obstacle for AVs, by developing DL models that can adapt to weather-induced distortions, thereby improving system reliability. These studies have identified key performance indicators (KPIs) such as detection accuracy, false negative rate, computational efficiency, and real-time processing capabilities, which are essential for evaluating the practicality of DL models in AVs, as shown in

Table 6. Practical implications of DL approaches.

Ref.	Approach	Practical Implications	KPIs	Challenges	Addressing Challenges
[145]	CNN-based Human Detection at Nighttime	Enhances pedestrian detection in low-light conditions, improving safety and navigation for AVs.	Detection Accuracy, False Negative Rate, Computational Efficiency	Limited visibility, background noise interference	Utilizes CNNs trained on nighttime images to recognize pedestrian features, potentially reducing false negatives.
[147]	Optimized Mask R-CNN for Low-Light Pedestrian Detection	Tailors the Mask R-CNN algorithm for low-light environments, maintaining high detection rates.	Detection Accuracy, Precision, Recall, Real-time Processing	Optimization for low-light conditions, computational complexity	Adjusts model parameters, uses data augmentation, and employs hardware acceleration for real-time processing.
[152]	YOLO-v5 for Pedestrian Detection in Daytime and Nighttime	Demonstrates the versatility of YOLO-v5 across different lighting conditions, ensuring consistent performance.	Detection Speed, Accuracy, Robustness	Balancing speed with accuracy in varying lighting	Fine-tunes the YOLO-v5 model on diverse datasets, ensuring generalization across different lighting scenarios.
[153]	DL Approaches for Adverse Weather Detection	Improves detection reliability in adverse weather, crucial for AV safety and functionality.	Detection Accuracy, Robustness, System Reliability	Degradation of performance due to weather conditions	Uses DL models that learn to recognize patterns despite weather distortions and integrate multi-sensor data for enhanced detection.

. The challenges faced, including limited visibility and dynamic lighting conditions, have been mitigated through model optimization, data augmentation, and sensor fusion techniques. The insights from these studies are invaluable for AV developers as they provide a roadmap for creating systems that can operate effectively in a wide range of environmental conditions, bringing us closer to a future where autonomous vehicles are a safe and integral part of our transportation infrastructure.

7. Discussion, Limitations, and Future Research Trends

7.1. Discussion

Recent literature highlights the increasing use of DL approaches for OD, yielding promising results under typical conditions. However, there is a pressing need for further advancements, particularly in adverse weather and complex scenarios. Enhancing OD in such conditions is crucial, especially for AVs, to prevent accidents and ensure safety. In the following discussion, we delve into the key insights and implications drawn from our research on OD in adverse weather conditions for AVs. Our research has explored both traditional and DL methods for AV object detection, focusing on vehicles, pedestrians, and road lanes. Traditional methods, despite their foundational role, struggle with high computational demands, slow processing, and occasional misidentification. DL, on the other hand, excels by learning complex patterns from data, offering faster and more precise detection, especially in challenging weather conditions. This makes DL a more effective and adaptable solution for AV systems. Traditional vehicle detection techniques, which rely on appearance- and motion-based algorithms, face significant challenges in adverse weather conditions. Appearance-based algorithms like HOG, Haar-like, and SIFT are sensitive to exterior objects, adverse weather, and lighting variations, making them vulnerable in situations like heavy rain or fog. These algorithms often struggle to maintain accuracy due to the reduced visibility of vehicle features and the difficulty in differentiating vehicles from their surroundings. Motion-based detection methods, which track objects based on movement relative to the camera, also encounter issues in adverse weather. They can have trouble distinguishing between moving vehicles and other dynamic elements, such as pedestrians or debris, especially when motion cues are obscured by weather conditions. DL approaches are more accurate and perform better, as shown in Table 4 and Table 5. A comparison of traditional and DL approaches is shown in

Table 7. Comparison of traditional and DL approaches.

Ref.	Approaches	Technique	Superiority	Limitations	Additional Considerations
[154,155]	Color-based Lane Detection	Traditional	Simple to implement, low computational cost	Limited in adverse weather, reliance on good weather conditions for high accuracy	Research into adaptive color models or fusion with other sensor data could improve performance in challenging conditions.
[158,159]	LiDAR Integration	Traditional	Reduces reliance on visual data, robust in various weather	High cost, sensitivity to unfavorable weather conditions.	Cost reduction and miniaturization of LiDAR sensors could make this technology more accessible for widespread use.
[160]	HSV Color Space	Traditional	Works well in daylight or with white light; uses color correspondence	Requires global thresholding, less effective in tunnel environments with distinct color illumination.	Enhancing the color model with machine learning could improve its adaptability to different lighting conditions.
[161]	SafeDrive Technique	Traditional	Utilizes historical data and vehicle location for lane detection in low-light areas	May not generalize well to all environments, relies on available historical data.	Incorporating real-time weather data and vehicle dynamics could improve the technique’s robustness.
[163]	CNN-based Real-time Road Condition Detection	DL	High accuracy with transfer learning automates real-time condition recognition	Requires pre-trained models, may not adapt quickly to new conditions	Continuous learning and model updating could help maintain high accuracy as conditions change.
[165]	Lane Marker Detection in Rainy Weather	DL	Prioritizes feature selection to counteract rain-induced blurriness	May require extensive training data, complex model architecture	Using smaller, more specialized models could reduce computational demands and improve real-time performance.
[141]	Generative Adversarial Networks (GANs)	DL	Can generate realistic data for training, improve feature extraction	Requires significant computational resources, may struggle with certain types of adverse weather.	Research into GANs for adverse weather simulation could provide more robust training data for DL models.
[152]	Multispectral Imaging	DL	Combines different imaging modalities for improved detection	Requires specialized hardware; may be complex to integrate	Further development of multispectral imaging techniques could lead to more reliable detection systems.
[163]	Transfer Learning	DL	Allows models to adapt to new tasks with fewer data	May not perform as well as task-specific models if the transfer is not well-aligned.	Fine-tuning transfer learning models for specific AV tasks could enhance their effectiveness.

.

Figure 13 presents the overall percentage distribution of papers, while Figure 14 describes the papers of traditional and DL approaches related to the studied three object detection issues. From Figure 13, one can see that vehicle detection is frequently studied using traditional and DL approaches, followed by pedestrian and road lane detection. As shown in Figure 14, DL approaches were more frequently used for all three issues in AVs compared to traditional approaches. Prior to 2008, traditional feature extraction methods were prevalent for detection and classification but had limitations in adverse conditions. Manual feature extraction made them less suitable for complex applications. DL has become much more popular and prominent than regular algorithms in recent times. This change in direction is attributed to DL’s exceptional work, reliable outcomes, and breadth of industry experience. Romero and Antonio [129] also mainly concentrated on defining DL methods. Peer review has been applied to DL one-stage and two-stage detectors from the investigations. Currently, the leading methods are YOLOv8 from the YOLO series and Faster R-CNN from the R-CNN family, renowned for their superior accuracy and performance. Figure 15 shows the number of summarized papers for one-stage and two-stage detectors, which shows that one-stage detectors are frequently used for vehicle detection, while, for pedestrian detection, both are equally applied. Two-stage detectors are frequently used for road lanes compared with single-stage detectors.

In order to improve OD, Faster R-CNN was integrated into the RPN. This resulted in shorter evaluation times, but it also required a significant amount of processing power and had difficulties maintaining consistent performance for small and variously scaled objects.YOLOv8 is designed to cater to specific requirements. The YOLOv8 structure comprises backbone, neck, and head modules, enhancing feature extraction with C2f and employing the PAN feature fusion technique. It also adopts a decoupled head architecture and introduces an anchor-free strategy for OD. Furthermore, we analyzed high-quality datasets related to AVs, and some standard evaluation metrics were also reviewed and summarized. Our discussion underscores the advancements in OD for autonomous vehicles under adverse weather, paving the way for enhanced safety and reliability in ADs.

7.2. Limitations

• Our literature review primarily focuses on the detection of pedestrians, vehicles, and road lanes, which may not encompass all possible objects and scenarios relevant to AVs in adverse weather conditions. There may be other critical elements, such as traffic signals or animals, that warrant further investigation.
• The detection algorithms discussed in our review may have inherent limitations, such as difficulties in detecting small or occluded objects, which could impact the accuracy and reliability of AVs in certain situations.
• Our study primarily considers a range of adverse weather conditions, but there may be other environmental factors, such as dust storms or heavy fog, that were not extensively covered in the reviewed literature.
• The field of AV technology is rapidly evolving, and new detection algorithms and sensor technologies may emerge that could significantly impact the findings of our review. Our study may not capture the most recent advancements in this area.
• While our study includes datasets that simulate adverse weather conditions, the simulation environments may not perfectly replicate real-world driving scenarios. The complexity of real-world conditions, including unpredictable human behavior and dynamic traffic patterns, can introduce additional challenges not fully captured in simulated datasets.
• The ethical considerations and societal acceptance of AVs, especially in challenging conditions, are not addressed in our study. Public trust and the ethical use of AV technology are essential factors for their successful integration into smart cities.

7.3. Future Research Trends

• It has become more important to address the real-time requirements for OD in real-world applications. Deep neural networks, however, frequently require large amounts of computational power, which presents difficulties for embedded systems. To properly fulfill these objectives, resource-efficient technique development has become essential. To ensure the practical usefulness of the suggested methodologies, future research should focus heavily on their computational components, offering both quantitative and qualitative analysis.
• The existing deep neural network techniques for difficult item detection mainly depend on large-scale annotated datasets. However, creating these databases is expensive and time-consuming. Consequently, there is an urgent need to create OD algorithms that can train with little to no labeled information.
• The employment of various evaluation metrics and IoU criteria for OD in difficult situations has resulted in the absence of a clear benchmark standard, which makes comparing methods difficult. For future research in this area to be uniform and comparable, a global baseline must be established.
• Creation of extensive simulation environments should occur that imitate inclement weather to thoroughly test and improve object identification algorithms.
• To develop comprehensive solutions for adverse weather OD, researchers, engineers, and policymakers should collaborate more closely.
• It is necessary to study the psychology and behavior of human drivers in adverse weather, with an emphasis on developing successful communication and trust with AVs.
• Creation of novel tactics for the real-time modification of OD algorithms for AVs in response to changing environmental circumstances should be achieved.
• Investigation into cutting-edge techniques for combining current weather data with weather forecasts to enable proactive decisionmaking during unfavorable weather conditions is necessary.
• Improvements in sensor fusion methods should be attained, which integrate information from several sensor types to provide more accurate and dependable identification in adverse weather.
• To develop behavior prediction models for AVs, leveraging machine learning and deep learning to forecast the actions of vehicles, pedestrians, and cyclists should occur. These models will operate effectively in adverse weather, improving AV decisionmaking for enhanced safety and efficiency.

8. Conclusions

This paper reviewed the traditional and DL approaches for vehicles, pedestrians, and road lane detection in AVs under adverse weather conditions. We first studied the architecture of AVs with sensor technologies and other components and also discussed the challenges for AVs in adverse weather. After an overview of almost all the datasets related to AVs covering different weather conditions, we explained the basic structure of OD and the evaluation matrices used for it. Then, we explained the traditional approaches and DL approaches and discussed the traditional feature extraction methods that were prevalent for detection and classification but had limitations in adverse conditions. Manual feature extraction made them less suitable for complex applications. DL has become much more popular and prominent than regular algorithms in more recent times. DL approaches explain the structure of YOLOV8 (one-stage detectors) and Faster R-CNN (two-stage detectors). Two-stage algorithms tend to deliver superior accuracy, while one-stage algorithms offer faster processing speeds. Notably, the reasons behind the lower accuracy of one-stage algorithms remain unclear. In addition, the statistics about the status quo of traditional and DL approaches for OD in AVs were provided based on the works collected in this survey paper. We found that DL was intensively used for vehicles, pedestrians, and road lane detection in AVs compared with the traditional approaches. Specifically, one-stage detectors were frequently used for vehicle detection compared with two-stage detectors. In addition, vehicle detection was frequently studied using both traditional and DL approaches followed by road lane and pedestrian detection. Finally, we presented a useful discussion along with some future research directions.