A Taxonomy for Autonomous Vehicles Considering Ambient Road Infrastructure

Abstract

To standardize definitions and guide the design, regulation, and policy related to automated transportation, the Society of Automotive Engineers (SAE) has established a taxonomy consisting of six levels of vehicle automation. The SAE taxonomy defines each level based on the capabilities of the automated system. It does not fully consider the infrastructure support required for each level. This can be considered a critical gap in the practice because the existing taxonomy does not account for the fact that the operational design domain (ODD) of any system must describe the specific conditions, including infrastructure, under which the system can function. In this paper, we argue that the ambient road infrastructure plays a critical role in characterizing the capabilities of autonomous vehicles (AVs) including mapping, perception, and motion planning, and therefore, the current taxonomy needs enhancement. To throw more light and stimulate discussion on this issue, this paper reviews, analyzes, and proposes a supplement to the existing SAE levels of automation from a road infrastructure perspective, considering the infrastructure support required for automated driving at each level of automation. Specifically, we focus on Level 4 because it is expected to be the most likely level of automation that will be deployed soon. Through an analysis of driving scenarios and state-of-the-art infrastructure technologies, we propose five sub-levels for Level 4 automated driving systems: Level 4-A (Dedicated Guideway Level), Level 4-B (Expressway Level), Level 4-C (Well-Structured Road Level), Level 4-D (Limited-Structured road Level), and Level 4-E (Disorganized Area Level). These sublevels reflect a progression from highly structured environments with robust infrastructure support to less structured environments with limited or no infrastructure support. The proposed supplement to the SAE taxonomy is expected to benefit both potential AV consumers and manufacturers through defining clear expectations of AV performance in different environments and infrastructure settings. In addition, transportation agencies may gain insights from this research towards their planning regarding future infrastructure improvements needed to support the emerging era of driving automation.

1. Introduction

Autonomous vehicles (AVs) have garnered substantial attention from various sectors, including government agencies and policymakers, the automotive and technology industries, and academia, due to their potential to enhance road safety, improve travel efficiency, and reduce energy consumption [1,2,3,4]. In several countries, such as China, Japan, the United Kingdom, Sweden, Germany, and Japan, AVs have been permitted or are in the process of being permitted for deployment in specific locations such as experimental test tracks, academic campuses, and demarcated urban zones, with restrictions to avoid mass public exposure [5,6,7]. In the U.S., in 2021, President Biden signed the USD 1 trillion Infrastructure Investment and Jobs Act (IIJA), which commits $110 billion to roads, bridges, and other significant projects, and mandated the United States Department of Transportation (USDOT) to develop the Automated Vehicles Comprehensive Plan (AVCP) to ensure American leadership in autonomous vehicle technologies [8]. The Federal Highway Administration (FHWA) has developed the innovative CARMA Platform to foster collaboration aimed at enhancing transportation efficiency and safety [9]. In addition, the American Association of State Highway and Transportation Officials (AASHTO) has published a document titled “Connecting on CAVs”, featuring newly established policy principles for connected and autonomous vehicles. These principles are intended to facilitate the advancement of connected and autonomous vehicle (CAV) technology development [10].

Despite the growing interest in AVs, there seems to exist a lack of consensus on the definition of the term AV. This conundrum relates to the purpose of this paper and motivates a closer study of the reasons for such lack of taxonomical consensus. Automation generally refers to the use of control systems for operating equipment or performing human tasks. Therefore, vehicle automation is part of a broader trend toward the replacement of humans in various functions, including machine operation. This trend is driven by reasons that include (i) the rapid growth of information and communication technologies (ICT), which has led to increased computing power, and (ii) the need for greater efficiency and improved operator safety. Cognitive psychologist Lisanne Bainbridge pointed out that, paradoxically, with automation, human involvement becomes more critical, even as humans become less involved in the operation of automated systems [11]. Automation involves not only control systems, computer engineering, and ICT, but also psychology, social sciences, and business. As such, it is not surprising that pilot research efforts on the deployment efficacy of automated driving systems and their policy development are quite multidisciplinary in nature as they involve knowledge from these diverse fields. It appears that the multidisciplinary nature of automation has been a curse as much as it has been a blessing, because to date, there seems to be a longstanding lack of consensus on the definition of the term “autonomous vehicles” [12].

The Society of Automotive Engineers (SAE) has developed a taxonomy scale to standardize the levels of driving automation and related terminology for the benefit of the automotive industry and AV policymakers. The scale consists of six levels of vehicle automation, ranging from Level 0 to Level 5 [13]. Level 0 vehicles are fully controlled by the driver, while Level 1 vehicles have a single automated system for assisting with either steering or cruise control. Level 2 vehicles, such as Tesla Autopilot and Cadillac Super Cruise, feature automation for both steering and cruise control. In general, drivers still perform a significant portion of the driving tasks in Level 0 to Level 2 vehicles. However, from Level 3 to Level 5, the automated driving systems take over all driving tasks when engaged. Level 3 vehicles can make informed navigation decisions and undertake driving tasks, but still require human intervention in certain situations. Level 4 vehicles can intervene, if necessary, but are limited to a specific geographic area. Finally, Level 5 vehicles represent the pinnacle of automation and do not require any human intervention under any circumstances.

The SAE taxonomy provides a clear and comprehensive description of the driving automation levels based on the capabilities of the automated system. However, it lacks information on the necessary infrastructure to support each level of automation [14,15]. In reality, the road network is composed of diverse road types, requiring AVs to seamlessly transition between them during their operation. Our main insight is that for each 4-x level, different types of infrastructure, i.e., capabilities, are required. In this paper, we first define the different types of infrastructure, which are classified based mainly on the characteristics of the infrastructure/environment:

(1)

Dedicated Guideways:

• Lanes are exclusive and fully controlled;
• Intelligent and complete infrastructure is accessible;
• Other road users such as pedestrians seldom occur.

(2)

Expressways:

• Reliable V2I and V2V communication is accessible;
• Surrounding vehicles are moving in the same direction;
• Other road users seldom occur. Wild animals may suddenly appear but at a relatively low frequency.

(3)

Well-Structured Roads:

• Clear lane markers and complete traffic signals are accessible;
• Smart and communicable infrastructure may be inaccessible;
• A large number of other road users such as pedestrians and bicycles exist.

(4)

Limited-Structured Roads:

• Road lane markers and traffic signs are incomplete or even unavailable;
• Intelligent infrastructure is usually inaccessible;
• The road may be covered by flood, ice, or dirt such that lane markers are invisible;
• Some wild animals, pedestrians, vehicles, and other road users exist in the surroundings.

(5)

Disorganized Areas:

• The surroundings are constituted by huge crowds of people, bicycles, motors, and other road users;
• Space suitable for driving is usually limited;
• Assistance from nearby intelligent infrastructure is inaccessible.

Figure 1 depicts a bird’s-eye view of a hypothetical road network that comprises a variety of road types: dedicated guideways for AVs (labeled with ‘①’ at the top right), expressways (labeled with ‘②’ at the top left), well-structured roads (labeled with ‘③’ in the middle), rural roads as limited-structured roads (labeled with ‘④’ at the bottom left), and disorganized areas in communities (labeled with ‘⑤’ at the bottom right). The tunnel in ‘①’ is an example of a dedicated guideways for AVs, with advanced infrastructure that is capable of guiding AVs and protecting them from unexpected interactions with pedestrians, bicyclists, animals, and so on. When driving in this tunnel, the vehicle does not need navigation capabilities or real-time object detection capabilities. Since the vehicle does not require human take-over, vehicles without any capabilities of navigation capabilities or real-time object detection but have simple self-driving functions (e.g., lane-keeping) to drive in such tunnels can be identified as L4 in the SAE classification framework. However, such vehicles (without any capabilities of navigation capabilities or real-time object detection) will not be able to realize L4 automation on the roads labeled with ‘③’, which are well-structured urban roads with mixed traffic. An SAE L4 vehicle must have the capabilities of real-time object detection and collision avoidance when driving in such an area (due to complex interactions with human-driven vehicles, pedestrians, etc.). Obviously, there is a clear distinction between these two types of vehicles to realize L4 automation in areas ‘①’ and ‘③’.

Now, consider the expressways labeled as ‘②’; these roadways can have higher speed limits and fewer unexpected obstacles compared to urban areas. However, they may still encounter unexpected situations, like sudden traffic congestion or a vehicle breakdown. Thus, an SAE L4 vehicle on an expressway still needs to be capable of real-time object detection, collision avoidance, and the ability to navigate through the potentially high-speed traffic. While both the ‘②’ expressways and ‘③’ well-structured urban roads require an SAE L4 vehicle to have the capabilities of real-time object detection and collision avoidance, the primary difference lies in the traffic dynamics. Thus, an L4 AV must be capable of handling high-speed decision-making on expressways, and at the same time, it must deal with the complexity and diversity of urban traffic scenarios. Roads labeled as ‘④’ represent rural roads as an example of limited-structured roads, where there may not be clear road markings or signals, and the vehicle may encounter unexpected obstacles like wildlife or agricultural vehicles. In such scenarios, an SAE L4 vehicle needs robust navigation capabilities and advanced perception to handle less predictable road conditions. Further, consider an AV classified as L4 (represented in red in the figure): while the red vehicle exits the tunnel and navigates urban roadways, particularly in the disorganized area labeled as ‘⑤’, it must be aware of the changes in surrounding infrastructure to take appropriate actions accordingly; otherwise, catastrophic traffic accidents may occur.

Clearly, the full potential of automated driving systems can only be realized when accompanied by infrastructure that possesses the appropriate level of technological advancement or “smartness”. Hence, it is vital to understand the crucial role that infrastructure plays in autonomous driving [16,17]. Without appropriate infrastructure support, an AV that is considered to be at a certain SAE level in certain areas may not realize a similar level of autonomous driving performance in other areas.

2. Study Objectives and Organization of the Paper

It is expected that the deployment of AVs will follow a gradual progression, commencing with simple operational domains such as dedicated lanes, access-controlled freeways, and rural arterials, before moving on to more complex domains including urban arterials, intersections, and city/town streets. Both prior to the onset of this progression and at each subsequent stage, there is a need to comprehend the crucial role that infrastructure plays in supporting automated driving systems. This can be achieved through incorporating the requisite level of environmental infrastructure support required for each level of automation in the established SAE taxonomy.

However, as previously discussed in this paper, the current definition of SAE Level 4 lacks distinction between the varying capabilities of vehicles with this level of automation across different environment–infrastructure domains. To throw more light and stimulate discussion on this issue, this study proposes an enhancement of the current taxonomy through incorporating the role played by infrastructure in automated driving systems, specifically with regard to Highly Automated Driving Systems (HADS)—Level 4. The focus on Level 4 systems is due to their greater potential for near-future implementation compared to Level 5, as well as the recognition that HADS can only attain full operational capability under specific and limited conditions that require a clear understanding of the necessary infrastructure.

The major contributions of this work are as follows:

• With our proposed supplemental taxonomy, various driving conditions can be classified, enabling the vehicle to fully understand its ambient driving environment. This extends beyond alerting the vehicle when the level of infrastructure advancement changes from Level 4-A to Level 4-B. Indeed, it underscores the essential need for an AV to be adaptable across all Level 4-x infrastructures. In this context, our proposed taxonomy does not merely serve as a warning system but also functions as a foundation upon which AVs can evaluate and adjust their capabilities accordingly. This adaptability is crucial in ensuring that human intervention is only sought when necessary, thereby maximizing the autonomy of these vehicles. Furthermore, the proposed taxonomy will also create an industry standard that helps manufacturers and vehicle sellers clearly demonstrate the capabilities of their AVs. Instead of simply advertising a “Level 4” vehicle, manufacturers should explicitly inform consumers about the specific levels of road infrastructure environments where the Level 4 vehicle is capable of operating safely and effectively.
• The proposed supplement to the SAE taxonomy incorporates the role of the environment–infrastructure domain, offering a more comprehensive approach to characterizing Level 4 automated driving systems. This supplement aims to provide clarity regarding potential subsets of Level 4 automation, enhance the implementation of the SAE taxonomy, and serve as a reference for the design, testing, and evaluation of high-level AVs for the benefit of all stakeholders. Through directly characterizing the operations of HADS in terms of both automation level and infrastructure smartness, this supplement has the potential to create realistic expectations, increase confidence, and enhance the credibility of autonomous vehicle operations.
• This study has the potential to provide a robust foundation upon which AV manufacturers can precisely delineate the operational capacities of their vehicles. Moreover, it can enable AV users to attain a more accurate understanding of their vehicle’s capabilities. Furthermore, the proposed supplement can inform government regulators and policymakers in formulating suitable policies and regulations that consider the infrastructure–environment domain. Additionally, it can provide road agencies with the necessary information to make informed decisions regarding investments in infrastructure to support AV operations. Ultimately, the proposed supplement can offer infrastructure managers, investors, and policymakers stronger justifications for policies, initiatives, and investments aimed at preparing infrastructure for AVs.

The organization of the rest of this paper is as follows: Section 3 provides a brief overview and critique of the existing taxonomy system for AVs. In Section 4, we examine the various factors that impact AV operations. The supplementary taxonomy is presented in Section 5, followed by a consideration of the future development of AVs in Section 6.

3. Review of the Current SAE Taxonomy and Its Limitations

3.1. Existing SAE Taxonomy

Several institutions have established automated vehicle classifications, including the International Organization of Motor Vehicle Manufacturers (OICA) [18], the Germany Federal Highway Research Institute [19], and the SAE [13]. Of these, the SAE’s taxonomy, consisting of six levels of vehicle automation, has garnered the most widespread acceptance and has been adopted as the industry standard. This system has been endorsed by the US Department of Transportation [13,20]. The current SAE taxonomy, although widely adopted and recognized as the industry standard, does have certain limitations. One such limitation is that the taxonomy primarily focuses on the technical aspects of vehicle automation, such as the level of driver involvement and the extent of automation, without considering the variability in driving conditions and environments across different regions, which can have a significant impact on the performance and reliability of automated systems [21,22,23,24,25,26].

To effectively characterize and evaluate the complex nature of automated driving systems and related issues, various stakeholders, including the SAE and the USDOT, have played a crucial role in shaping key terms and concepts. One such concept is the operational design domain (ODD), which serves to describe the specific conditions and scenarios under which an automated system or feature is designed to operate. The ODD incorporates environmental, geographical, time-of-day, and roadway restrictions and characteristics, as outlined in the literature [4,13]. It is worth noting that the elements of the ODD can be classified into two broad categories: those that are within the direct control of the relevant agency, such as infrastructure quality, and those that are beyond the agency’s control, such as weather conditions (e.g., ice, wind, fog, smoke) and traffic conditions (e.g., occlusion by large vehicles). The dynamic driving task (DDT), as defined by the SAE [13], encompasses all the functions required to operate a vehicle on-road in traffic. As detailed in

Table 1. SAE Taxonomy [13].

Level	Name	Narrative Definition	DDT		DDT Fallback	ODD
			SLLVMC	OEDR *
Driver Performs Part or All of the DDT
0	No Driving Automation	The performance by the driver of the entire DDT, even when enhanced by active safety systems.	Driver	Driver	Driver	n/a
1	Driver Assistance	The sustained and ODD-specific execution by a driving automation system of either the lateral or the longitudinal vehicle motion control subtask of the DOT (but not both simultaneously with the expectation that the driver performs the remainder of the DDT.	Driver and System	Driver	Driver	Limited
2	Partial Driving Automation	The sustained and ODD-specific execution by a driving automation system of both the lateral and longitudinal vehicle motion control subtasks of the DDT with the expectation that the driver completes the OEDR subtask and supervises the driving automation system.	System	Driver	Driver	Limited
ADS (“System”) performs the entire DDT (while engaged)			System	System	Fallback-ready user (becomes the diver during fallback)	Limited
3	Conditional Driving Automation	The sustained and ODD-specific performance by an ADS of the entire DDT with the expectation that the DDT fallback-ready user is receptive to ADS-issued requests to intervene, as well as to DDT performance-relevant system failures in other vehicle systems and will respond appropriately.
4	High Driving Automation	The sustained ODD-special performance by an ADS of the entire DDT and DDT fallback without any expectation that a user will respond to a request to intervene.	System	System	System	Limited
5	Full Driving Automation	The sustained and unconditional (i.e., not ODD-special) performance by an ADS or the entire DOT and DDT fallback without any expectation that a user will respond to a request to intervene.	System	System	System	Unlimited

Note: SLLVMC—sustained lateral and longitudinal vehicle motion control. ‘*’ denotes the OEDR—object and event detection and response. n/a—not applicable.

, each level of automation within the SAE taxonomy has a distinct set of requirements that a vehicle must meet before it can be considered operational at that level.

3.2. Role of Road Infrastructure

The role of infrastructure in supporting automated driving is significant [27,28,29]. It is expected that road infrastructure in the AV era will be equipped with smart features to augment the AV’s capabilities. First, the infrastructure should enhance the sensing abilities of AVs through collecting, analyzing, and transmitting data regarding the driving environment, including the characteristics of roadways and other vehicles in the traffic stream. This information can be used to inform the strategic, tactical, and operational driving decisions of the AV [30]. Secondly, the infrastructure should provide support to AVs in navigating through challenging driving conditions, such as periods of impaired sensing or navigation capabilities caused by adverse weather conditions, occlusion by larger vehicles, or lack of connectivity [31]. The USDOT, in its Automated Vehicle Comprehensive Plan document, duly recognizes these prospective roles of AV infrastructure and the environment, and the document mentions that modernizing the regulatory environment and preparation of the existing transportation infrastructure is critical for successful automated transportation [8,32]. Also, the American Association of State Highway and Transportation Officials indicated that keeping a safe driving environment for AVs through defining regulations and constructing smart infrastructure is a major issue [33]. Notably, McAslan et al. [34] found that many regional planning agencies in the United States have implemented policies aimed at enhancing infrastructure maintenance to facilitate the testing and deployment of AVs. Thus, the importance of smart infrastructure in ensuring the safe and efficient operation of AVs cannot be overstated.

The significance of infrastructure support for the successful operation of AVs is highlighted by the fact that several AV manufacturers and technology companies invest in the development of dedicated infrastructure and accompanying hardware. One such example is the tunnel system developed by Elon Musk’s The Boring Company in Los Angeles, known as the “Test Tunnel”. This 1.2-mile-long tunnel, shown in Figure 2a, was designed for the purpose of research and development for Musk’s innovative vision of a network of underground highways [35]. Within this controlled environment, autonomous vehicles have the potential to reach speeds of up to 25 mph. The “Future Bus” project developed by Mercedes-Benz involves the operation of buses on dedicated bus-only lanes that are equipped with vehicle-to-infrastructure (V2I) connectivity facilities, providing information on the bus route and station locations. This allows the bus to operate at a speed of 43 mph and make precise stops at designated stations, as shown in Figure 2b. Another example is the 3.9 km guideway used by the Heathrow Airport Authority in London for elevated automated personal transportation pods that utilizes a sled guideway, as shown in Figure 2c [36]. Even where testing is carried out on surface roads, the urban–rural nature of the highway is of paramount importance in designing road infrastructure to support AV operations. For example, for Volvo’s autonomous truck, which runs on rural highways, there is access control infrastructure, well-marked pavements, and a relatively uniform traffic environment. In contrast, Google’s self-driving cars were tested in urban areas that had no access control and had more dynamic and challenging driving scenarios including several traffic signals, dense traffic, and traffic jams. As we can see, the role of infrastructure is not only to provide a physical platform for AVs, but also to enable the achievement of autonomous driving in specific areas through reducing the need for complex real-time environmental perception and decision-making capabilities.

Another noteworthy example is the Ultimate Urban Circulator Program (U2C) in Jacksonville, Florida [37]. Through repurposing the existing downtown circulator guideway for autonomous vehicles and combining this with flexible routes outside the downtown area, Jacksonville showcases a real-world example of how cities can effectively adapt existing infrastructure to usher in an era of autonomous transportation.

Figure 2. Infrastructure and supporting hardware for AVs invested by technology companies. (a) The Test Tunnel [35]; (b) Mercedes-Benz Bus of the Future [38]; (c) Heathrow ULTra pods [39].

Figure 2. Infrastructure and supporting hardware for AVs invested by technology companies. (a) The Test Tunnel [35]; (b) Mercedes-Benz Bus of the Future [38]; (c) Heathrow ULTra pods [39].

It is interesting to note that both Volvo’s autonomous trucks and Google’s self-driving cars both fall into Level 4 according to SAE’s definition. However, their infrastructure requirements for their successful operation are very dissimilar: Volvo’s autonomous trucks cannot operate in the driving environment of Google’s self-driving cars, and vice versa. The current SAE taxonomy for Level 4 does not address the distinction between these infrastructures and environments. In describing an automated driving system, it seems reasonable to mention the functionality and requirements of off-board infrastructure conditions and the corresponding driving environment. Scholars argue that a fully automated driving system is attainable only in the context of their constraints and that the current SAE taxonomy system for vehicle automation should be further described with specificity to a prevailing condition, instead of broad categorization that is irrespective of infrastructure conditions [23]. Ran et al. [40] proposed a definition of intelligence in smart road infrastructure called connected automated highway (CAH). They defined CAH using five levels and illustrated how the levels function with the CAV in collaborative automated driving system (CADS). Tengilimoglu et al. [26] interviewed 168 experts from 29 countries to determine the infrastructure-related requirements for safe operation of SAE Level 4 autonomous vehicles regarding (1) deployment paths, (2) autonomous driving road certification, (3) key infrastructure elements, and (4) factors impacting safe operation. Saeed et al. [41] developed a road infrastructure classification and discussed the challenges and opportunities related to the readiness of infrastructure for connected and autonomous vehicles (CAVs). Soteropoulos et al. [42] proposed a framework and relevant metrics for evaluating autonomous driving performance based on the relationship between the current technological state of autonomous driving systems (ADSs) and various domains.

Moreover, it is crucial for prospective users of AVs of a certain level of automation, to be aware of the level of infrastructure advancement that is consistent with their AV. If this is not done, the high-level AV user may be operating their vehicle in a low-level environment/infrastructure, thereby jeopardizing the travel efficiency and safety of the AV user and other road users. For example, a purchaser of a “Level 4” autonomous vehicle should be made aware that the vehicle can attain its full potential only under specific strictly designed infrastructure which matches that level of automation and not in any normal road environment–infrastructure domain. The questions that arise, then, are as follows: realizing that the capabilities of highly automated driving systems are different under different levels of infrastructure advancement, does the specification of the autonomous vehicle’s level of automation alone suffice in characterizing its operational capabilities? Should a complete characterization of operational capability not depend on both the level of vehicle automation as well as the level of the environment–infrastructure domain?

3.3. Limitations of SAE Taxonomy

The SAE taxonomy system continues to be the most prevalent and cited reference in the field of automated driving. However, the authors of this paper believe are a few opportunities to improve the SAE taxonomy.

3.3.1. From the ODD Perspective

The SAE taxonomy does not incorporate any ODD elements to accompany the levels of automation, particularly the higher levels. The extensive variety of the infrastructure and environmental conditions (including the level of connectivity, the level of intelligence of supporting infrastructure, and the quality levels of AV-supporting road facilities including lane markings) are not addressed in the SAE’s taxonomy. In this regard, it can be argued that the level of infrastructure advancement or “smartness” needed to support any given level of automation is a key consideration in assessing the capability of an autonomous vehicle [43].

As discussed in a previous Section 3.2 of this paper, facilities such as the Test Tunnel [35], Mercedes-Benz Future Bus, and Heathrow ULTra pods [36] all enable Level 4 automated driving but accomplished this only under very specific infrastructure conditions. In such carefully designed environments and infrastructure, Level 4 performs well. However, the same Level 4 vehicles are unlikely to exhibit such performance at other less-defined conditions of the environment and infrastructure, for example, a typical urban environment with errant HDVs, pedestrians, and other uncertainties in the traffic stream and the roadway environment in general. Therefore, the performance of Level 4 autonomous vehicles is expected to be different in different environments.

3.3.2. From the DDT Perspective

There is also an argument to be made for supplements to the SAE taxonomy, from the DDT perspective. The infrastructure required to support an automated driving system is not described when defining a high-level automated driving system. One fundamental assumption adopted by SAE is that DDT is performed by either in-vehicle systems or human drivers. Nevertheless, supported by the rapid advancement of wireless communication, intelligent infrastructure has played a significant role in DDT performance. For example, recent research studies have extensively deployed vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication (or V2X in short) to facilitate the monitoring of the roadway environment, trajectory planning, and decision-making in automated driving [14,15,44,45]. A practical illustration is the PATH project, where cars are driven automatically at close spacing on a freeway with magnetic markers embedded under the road. Through decreasing the spacing of cars, the density of traffic on a highway can be increased without requiring additional lanes [46]. In more recent studies on platoon-based cooperative driving, a vehicle collects kinetic data on neighboring vehicles through V2V communication and maintains its state in a cooperative way [47,48]. This is evident not only in the key role of infrastructure in achieving highway performance but also in the need to recognize the limitations of road infrastructure in automated driving and accounting for such limitations in characterizing the capabilities of automated driving systems.

Last but not least, policymakers and the public need more explicit information regarding scenarios in which a specific type of HADS can operate safely and regarding the infrastructure requirement associated with a given level of performance of the automated system [26,34,41]. Several companies have stated that they have developed vehicles with Level 4 automation [49,50]. However, as discussed previously, some projects are launched only in strictly controlled conditions, and significant variations in the capability of Level 4 automated driving systems discrepancies have been observed. This would not only lead to public confusion about the capability of their high-level autonomous vehicles but also increase the agency’s cost of road systems management because software and hardware requirements vary widely for successful automated driving in different environments. In making new policies or passing new traffic legislation that duly recognizes the limitations and capabilities of automated driving systems in different environments and infrastructures, policymakers need clearer ways to communicate with AV companies. For these reasons, there seems to be some merit in the notion that it is useful to identify the requisite infrastructure conditions to support each level of automated driving, and that such a description could serve as a supplement to the existing SAE taxonomy.

4. Roadway Conditions and Facilities That Influence the Performance of Highly Automated Driving Systems

Given the highly dynamic nature of not only traffic operations and conditions but also the road environment, safe driving can be a demanding task. The operator needs to be able to interact effectively with other vehicles in its proximity and elements in the wider roadway environment including roadside units, traffic signs, pedestrians, two-wheelers, road surface conditions (ice, potholes), atmospheric conditions (fog, rain), and so on. For these reasons, driver licensing departments in most countries require that the driver should have adequate mental and visual acuity, and intuitiveness. In this emerging age of autonomous transportation where humans prepare to hand over the driving task to automation, one of the dilemmas that remain is how to train the automated systems to acquire the full capabilities of a “good” human driver [51,52,53,54,55,56]. Fortunately, researchers have identified a number of roadway conditions and facilities that influence the performance of highly automated driving systems. With this knowledge, efforts can be made to identify the extent to which an environment and infrastructure combination is deemed deficient for automated driving and to serve as a basis for investments to upgrade the environment and/or infrastructure. Also, such knowledge can help support the argument that automated systems classifications should address not only the vehicle capabilities but also the reinforcing or extenuating conditions of the environment and/or infrastructure in which the system is in operation. In Section 4, we delve into these various aspects under three subsections: Section 4.1, Traffic Conditions; Section 4.2, Cyber Infrastructure; and Section 4.3, The Language of the Road. Each subsection aligns with our proposed Level 4-x infrastructure sublevels and details the specific infrastructure prerequisites required for each. It is important to note that our intent here is to provide an integrated understanding of the complex interactions between automated vehicles, traffic conditions, cyber infrastructure, and road environments, and how these interactions relate back to our proposed Level 4-x taxonomy.

4.1. Traffic Conditions

The safety of automated driving is particularly challenging in complex and dynamic roadway traffic conditions [57]. The Waymo Safety Report underscores this perspective [58]. It discloses findings from one million miles of autonomous driving tests conducted on public roads in California and Arizona, emphasizing the profound influence of environmental and infrastructure factors on the overarching safety of autonomous driving systems. Roadway environments that are auspicious for automated driving are rare in certain areas where traffic flow is non-uniform, traffic composition is heterogenous, and the flow of multitype traffic is chaotic. For example, in several cities in developing countries, two-wheelers constitute over half of the vehicle distribution, resulting in unstable traffic flow [59,60]. Also, there is the issue of non-uniform traffic regulations across jurisdictional boundaries (for example, switching between left-hand vs. right-hand traffic) to which human drivers generally adapt quickly. In such situations, a highly automated driving system may encounter significant difficulty and, therefore, will require advanced sensing and tracking capabilities. Furthermore, urban roadway environments are typically characterized by non-verbal communication among the users of the roadway space. There exists a myriad of movement-related cues and gestures that automated systems may not comprehend [61]. For example, an AV will need to recognize a person standing at any part of the roadway waving their hands and ascertain whether that person is a traffic police official directing traffic, a passenger hailing a taxi, or a pedestrian requesting vehicles to stop so that other vulnerable road users may cross the street.

The implementation of automated driving in complex and intricate urban environments presents considerably more formidable challenges than doing so in well-defined and relatively conducive environments. Therefore, it is obvious that the automated system cannot reach its full potential in a non-conducive environment. To date, there is no agreed standard regarding the order of traffic conditions for automated driving. Moreover, significant variations in such conditions have been observed in pilot AV projects in recent decades, as shown in

Table 2. Comparison of traffic conditions among AV projects (most up to date).

Project Name	Traffic Condition of Driving Scenario	Launch Year	Country	Latest Progress (2023)
Benz Future Bus	A dedicated bus-only lane of length is approximately 12 miles. Pedestrians seldom enter the lane.	2017	Germany	The Mercedes-Benz “Drive Pilot” system can only be used during daytime on the highway at speeds of up to 40 mph.
Volvo autonomous truck ‘Vera’	A predefined and fixed route between logistic ports. Most of the roads are public roads with limited traffic.	2017	Sweden	The company’s business remains focused on transporting products from logistics centers to ports
Robot Taxi	A 5 km road between Tokyo station and the Roppongi area. The on-road traffic is light and stable, but pedestrians are present at intersections and crossways.	2019	Japan	Autonomous vehicles intended for use as delivery robots or tour buses on routes in sparsely populated areas
Pony autonomous taxi	Crowded and busy roads in Guangzhou. Copious amounts of pedestrians and bicycle traffic at intersections.	2019	China	Received permission to run a fully automated driverless ride-sharing service in Guangzhou, China
Waymo self-driving	Waymo operates commercial self-driving taxi services in Phoenix, Arizona, and San Francisco, CA.	2020	USA	Waymo is now permitted to begin driverless taxi service in San Francisco, California, after receiving permission from the California Public Utilities Commission.

. The interested reader can find many more recent projects in [62]. Therefore, it can be argued that a classification system that specifies the conditions under which the AV will realize its full potential will be beneficial.

4.2. Cyber Infrastructure

Intelligent equipment installed on the infrastructure at or near the roadway (for example, on the roadway pavement, roadside structures, nearby buildings, drones, guardrails, and traffic signals) are typically capable of V2X communication and provide information that can influence the performance of the automated driving system. They have been used widely to facilitate localization, construct mapping, characterization of the roadway environment, and other functions for purposes of automated driving [63,64,65,66]. For example, a “bottleneck manager” typically installed at freeway locations with recurrent bottleneck congestion receives and prioritizes requests from AVs and optimizes their trajectories to reduce congestion and smooth traffic flow [53,67,68,69,70,71]. In addition, the concept of V2X communication has been extended to connected vehicle-to-pedestrian communication (V2P) via smartphone applications. V2P has been applied in several contexts of automated driving particularly, pedestrian safety enhancement through broadcasting locations and potential movements of pedestrians to the AVs in the disorganized area [72,73,74]. Also, Nikola Motor Company attempted to establish a connection between self-driving vehicles and pedestrians using the cell phones of individuals [75].

Highway agencies seek guidance on the changes in infrastructure design and management needed for AV operations because a robust network of physical and cyber infrastructure can help resolve several obstacles to automated driving. Ref. [41] discussed the challenges and opportunities associated with infrastructure preparation for AVs, identified stakeholder roles regarding AV infrastructure provision, and discussed uncertainties regarding AV market penetration and level of autonomy during the AV transition period. Concerning cyber infrastructure, there exist concerns about their practical deployment. First, the wide implementation of cyber infrastructure is not always feasible because these technologies hinge on the availability of wireless communication and big data computing capabilities, which are costly [76,77,78]. Second, effective communication hinges on the proper functioning of all nodes in the communication process. Malfunction of any component (for example, the communication device or a stable internet connection) could cause mischaracterization of the traffic environment, and in extreme cases, traffic accidents [61]. Therefore, HADS that depend partially on supporting infrastructure would be inadequate in areas where the quantity or quality of such infrastructure (software and hardware) is inadequate.

4.3. The Language of the Road

The “language of the road” is a term that collectively represents traffic lights, signals, signs, and road markings (road markings here exclude the physical characters painted on the road pavement surface). These infrastructures play an important role in traffic operations as they provide information to drivers, provide alarm of impending hazards or special attention downstream, and help in the navigation task [79,80]. AVs that can understand the language of the road will be able to enhance their traffic movements and trajectory planning [81]. From the literature, it is shown that attempts have been made to enable AVs to recognize road signs [82,83,84]; however, at the current time, the reliability of the developed recognition systems seems to be far from perfect. First, as the figures and text in Figure 3 show, the shapes and appearances of road language can be intricate and elaborate. Therefore, an AV may encounter difficulty distinguishing, for example, the road markings in the first two rows of Figure 3, which hold similar profiles but different meanings. Stop signs, which are typically encoded in the local language, have characters that are dissimilar across countries. Furthermore, the units of speed are not the same across countries: most countries, including China, Australia, and Singapore, use the metric system (i.e., kilometers per hour), while others such as the United States use the imperial system (i.e., miles per hour).

Secondly, in cases where the AV depends partially on mapped information established based on road inventory or where AV maneuvering algorithms are based on prior knowledge of infrastructure locations [85], any deviations from the mapped inventory and actual ground-truth inventory can threaten the AV’s safety and travel efficiency. Mapping updates may be infrequent enough to capture some changes in the road inventory. For example, it can be challenging for automated systems facilities to comprehend or recognize the existence of temporary traffic control and management installed by the road agency or the police.

Thirdly, the AV’s ability to interact with supporting infrastructure may be jeopardized by a non-favorable locational or functional relationship with the infrastructure. For example, it may be difficult for AV sensors and image-detection algorithms to read traffic signs due to adversities related to the angle position of the signs, the sign retro-reflectivity [86,87,88], time of day (daytime glare or nighttime obscurity), inclement weather, and the sun’s position (the AV’s camera view may be positioned towards or away from the sun).

Finally, the typical urban skyline is strewn with several objects of various sizes, shapes, and colors, and it may be difficult to distinguish between these objects and those that were installed to support traffic operations, such as traffic lights and road signs. This places a large burden on sign-detection algorithms [89]. As such, understanding the language of the road can be generally more challenging in metropolitan areas compared with rural areas that have clear skylines.

4.4. Physical Characters “Of the Ground”

The physical characters often painted on the road pavement surface can influence the effectiveness of not only normal human driving [90] but also, automated driving. These include lane and road appearance and clarity, visibility, road curvature, and other characteristics. Most known AV demonstrations have been carried out on simple roads that offer a conducive environment for AV operations, for example, well-marked lanes and high standards of road design [89]. Recent research studies have adopted assumptions regarding the consistency of lane/road texture, lane/road width, alignment, and surface markings [91,92]. However, these conditions do not exist on relatively less structured roads where there is greater uncertainty in the design and operational conditions of roadways. Automated driving on unstructured roads is therefore challenging, and the specification of highly automated systems must duly account for the conditions under which HADS realize their full potential.

5. The Proposed Supplement for Classifying Highly Automated Driving Systems

As the preceding sections have established, the capability of automated driving systems is not the same in all environment/infrastructure domains, and therefore the HADS user must be aware of the capabilities of their system when it is operating in each specific environment/infrastructure domain to avoid unexpected problems that may imperil traffic safety and travel efficiency. Such capability awareness could be realized if the SAE level of autonomy is reported together with a specification of the infrastructure/environment domain needed for the smooth operation of that level of autonomy. In other words, the SAE level of autonomy could be reported with sub-levels within each level; sublevels corresponding to specified levels of infrastructure/environment are needed for the smooth operation of that sub-level of autonomy. Therefore, this paper proposes that the taxonomy should address not only the vehicle capability at each level of automation (as the current SAE taxonomy does) but rather the capabilities of the duo (the vehicle and its infrastructure/environment domain).

In this section of the paper, we propose a supplement that is intended to serve as an addition to the existing SAE taxonomy. We particularly focus on only Level 4. Therefore, the supplement is developed for Level 4 HADS only.

Table 3. The proposed supplement.

Sub-Level of Level 4	Characteristics of the Infrastructure/Environment Domain	Capabilities of the HADS
Level 4-A (Level 4 Vehicles on a Dedicated Guideway)	Lanes are exclusive and fully controlled Intelligent and complete infrastructure is accessible Other road users such as pedestrians seldom occur	Guideway following Roadside parking
Level 4-B (Level 4 Vehicles on an Expressway)	Reliable V2I and V2V communication is accessible Surrounding vehicles are moving in the same direction Other road users seldom occur. Wild animals may suddenly appear but at a relatively low frequency	Trajectory planning Mapping and localization at the lane level Road language detection Roadside parking
Level 4-C (Level 4 Vehicles on a Well-structured Road)	Clear lane markers and complete traffic signals are accessible Smart and communicable infrastructure may be inaccessible A large number of other road users such as pedestrians and bicycles exist	Trajectory planning Mapping and localization at the lane level Detect complex and multi-road language Quick object and event detection and response Parking at the parking lot and roadside
Level 4-D (Level 4 vehicles on a Limited-Structured Road)	Road lane markers and traffic signs are incomplete or even unavailable Intelligent infrastructure is usually inaccessible The road may be covered by flood, ice, or dirt such that lane markers are invisible Some wild animals, pedestrians, vehicles, and other road users may exist in the surroundings	Trajectory planning and updating Mapping and localization at centimeter-level Detect complex and multi-road language Quick object and event detection and response Pass intersections without traffic signals Parking
Level 4-E (Level 4 Vehicles in a Disorganized Area)	The surroundings are constituted by huge crowds of people, bicycles, motors, and other road users Space suitable for driving is usually limited Assistance from nearby intelligent infrastructure is inaccessible	Trajectory planning and updating Mapping and localization more precise than centimeter-level Detect complex and multi-road language Object and event detection and response in a very dynamic and timely manner Travel with the crowd at a low speed Pass intersections without traffic signals Parking

, which presents a summary of the proposed supplement, shows the sublevels of Level 4 automation, the supporting infrastructure/environment associated with that sublevel, the features of the infrastructure/environment at each sublevel, and the capabilities of HADS corresponding to that level of the infrastructure/environment domain.

The supplement is motivated by the need for specifying the levels of autonomy together with physical or cyber infrastructure and environmental conditions. The Level 4 vehicle indicated in the table has capabilities consistent with any SAE Level 4 vehicle. It is important to note that this table is intended to provide a broad overview of possible scenarios and is not an exhaustive guideline for every possible circumstance. To bring the proposed levels of automation to life, consider the following examples: For Level 4-A, which necessitates a dedicated guideway for the normal operation of Level 4 HADS, a city bus running a regular route could be a practical example. The dedicated bus lane, equipped with traffic signals that prioritize the bus, can function as the dedicated guideway. The bus, outfitted with Level 4 automation, can operate safely within this controlled environment. For the case of Level 4-B, imagine an emergency vehicle such as an ambulance. These vehicles often benefit from traffic priority systems that can modify traffic signals to allow them to pass through intersections more quickly. However, the presence of these examples does not mean that all Level 4-A vehicles will be buses, or that all Level 4-B vehicles will be emergency vehicles. These are simply illustrative examples, intended to provide context and clarity for our proposed supplement.

The proposed supplement accounts for a key part of the ODD for automated systems classification. The ODD identifies specific limitations on the physical, cyber, environmental, geographical, and time-of-day restrictions and roadway characteristics. In the supplement, we focus only on the ODD elements that are within the control of the agency; therefore, weather-related elements such as ice, fog, heavy snow, or rain are excluded. The supplement is motivated by the need for specifying the levels of autonomy together with physical or cyber infrastructure and environmental conditions. Unlike physical and cyber conditions, weather-related conditions are not under the direct control of the stakeholders of the automated system, namely, the vehicle manufacturer, the agency, and the user.

The Level 4 vehicle indicated in the table has the following capabilities, consistent with any SAE Level 4 vehicle: (a) capable of adaptive cruise control, lane following, and lane transition; (b) capable of the basic detection of objects and events and the formulation of a response; (c) requires neither a conventional nor remote driver during route operation. It is assumed that there is no hardware or software system failure. The proposed sublevels of automation in the supplement are Levels 4-A, 4-B, 4-C, 4-D, and 4-E.

The designation of a specific sublevel of automation indicates its required operating condition. For example, Level 4-A depicts that a dedicated guideway is necessary to enable the normal operation of the Level 4 HADS. From Level 4-A to Level 4-E, there is a decrease in the quality requirement for the infrastructure/environment domain, for example, smart technology for road infrastructure.

5.1. Level 4-A (Level 4 Vehicles on a Dedicated Guideway)

Dedicated guideway conditions refer to lanes exclusively designed for AVs or areas with full access control. Examples of dedicated guideway conditions include automated bus-only lanes [93], Heathrow ULTra pods [36], and Tesla’s AV-dedicated tracks [94]. Figure 2a shows autonomous pods at Heathrow Airport operating on a 3.9 km track that connects the terminals. These pods are only capable of unmanned driving on the dedicated tracks and are expected to park at pre-defined locations for passenger pick-up/drop-off. In Level 4-A, the entire automated system (vehicle and infrastructure/environment domain) is capable of guideway following and roadside parking. For the guideway-following task, the vehicle uses only the trajectory directed by the guideway and cooperates with enroute traffic lights, signs, and other roadside equipment. Such navigation may be supported by intelligent V2I communication. For the roadside parking task, the vehicle can perform temporary parking for passenger pick-up/drop-off or emergency and send out failure information when necessary.

5.2. Level 4-B (Level 4 Vehicles on an Expressway)

On a typical expressway, driving is characterized by reliable and communicable roadway infrastructure featuring high-quality, clear, and recognizable lane markers and legible signs, as shown in Figure 4. Surrounding vehicles move in the same direction, and platoons may be formed. When an AV joins a platoon, its automated driving system may use V2V communications to supplement its sensing capabilities. Similar to Level 4-A, Level 4-B automated systems (vehicle and infrastructure/environment domain) permit the AV to perform roadside parking. On an expressway, vehicles constitute the only road users; however, it is still possible for the AV to encounter anomalous environmental conditions including crossing animals or unruly pedestrians or two-wheelers. In encountering such situations, the AVs’ HADS adjust their speed accordingly to maintain safe operations. In addition, in Level 4-B systems, HADS carry out the tasks of trajectory planning, mapping and localization, and road language detection. In planning a trajectory, the HADS decides the optimal route to the destination to minimize travel costs or travel distance using optimal route planning algorithms. For example, on a multi-lane expressway, optimal route planning could mean choosing the lane with the least congestion to minimize time, or selecting the lane that requires fewer lane changes to optimize fuel consumption and reduce wear. Furthermore, in cases where there are multiple routes available to reach the destination, such as with expressway junctions or interchanges, the HADS would determine the most efficient route based on current and predicted conditions. As a specific example, Magsino et al. [95] proposed an intelligent highway tollgate queue selector using fuzzy logic. Its purpose was to automatically select the most appropriate tollgate server for a vehicle to ensure the shortest waiting time. In performing the mapping and localization task, the AVs’ HADS are expected to recognize their lane position accurately. This process might be supported by referential lane markers and frequency selective strips (FSSs) via radar detection or magnetic markers installed on the road surface [96]. Regarding road language detection, HADS are capable of detecting enroute traffic lights, signs, and other road language and interpreting the information from these facilities, often with the help of V2I communication.

5.3. Level 4-C (Level 4 Vehicles on a Well-Structured Road)

This infrastructure/environment domain is characterized by a road with clear lane markers and complete traffic signals and signs. These include urban arterials and collectors. However, smart and communication infrastructure may be absent or inaccessible. Also, GPS signals may not be strong enough to provide localization support, particularly in tunnels or occluded areas. Also, there is a large volume of road user classes other than vehicles, particularly at intersection points in the corridor. Yet still, in Level 4-C, vehicles, and pedestrians use separate facilities. Figure 5 presents an example of a well-structured infrastructure/environment domain.

In operating safely at Level 4-C, the AVs’ HADS detect objects and respond accurately and timely. During driving, HADS should identify all nearby road users using computer vision, predict their behavior, and take actions to avoid collision accordingly. This task is challenging because there are traffic lawbreakers who may either suddenly cross the road or violate traffic signals. Apart from timely object detection, HADS at this level have the capabilities of Level 4-B but offer these capabilities at a higher precision and with less external support. For the trajectory planning task, the HADS of this level decide the optimal route to the destination with considerations of real-time road traffic. The decision can be updated using enroute real-time traffic information to avoid potential congestion [99,100,101]. For the mapping and localization task, the vehicle should be able to recognize its location with at least lane-level accuracy using its on-board vision system. Compared to the Level 4-B expressway level, the localization task is much more challenging as the road may be narrow on minor streets. Also, as supporting infrastructure such as magnetic lane markers may be absent, the HADS detect lanes solely using their onboard sensing features. About the capability to detect road language, the HADS identify not only permanent traffic signals and signs but also temporary signals including traffic police hand directions and short-term detour signs at construction work zones. The AV is able to pass through an intersection safely with guidance from traffic signals and signs.

5.4. Level 4-D (Level 4 Vehicles on a Limited-Structured Road)

At level 4-D, the infrastructure/environment domain is characterized by undeveloped off-road conditions/limited-structured roads. Examples of limited-structured roads include rural gravel roads (Figure 6), dirt tracks, intersections and roads without traffic signs, deserts, and frozen lakes. Road lane markers, traffic signs, and intelligent infrastructure are incomplete or even unavailable in these areas. The off-road nature of the driving environment and lack of infrastructure severely impair the capability of the Level 4 vehicle from realizing the full potential capabilities of SAE-defined automation at Level 4. In this case, the entire system (the Level 4 vehicle and its infrastructure/environment domain) is described as Level 4-D. In such environments, HADS plan their trajectory with little or no support from surrounding infrastructure and do this largely using onboard sensors to delineate the geometry of the road space, including boundaries of the pavement and vegetation, and to recognize pedestrians, bicycles, and other vehicles that may intrude their trajectory and adjust accordingly. At an unsignalized intersection, the HADS use their sensors to establish or predict the movements of other road users and to negotiate with them in order to pass the intersection safely.

5.5. Level 4-E (Level 4 Vehicles in a Disorganized Area)

The most challenging infrastructure/environment domain for an automated driving system is one where there is an absolute lack of structure and organization in the way the road space is used (Figure 7). Often, such road spaces are dominated by pedestrians and two-wheelers without clear paths milling around the road space in a seemingly random manner [51,103]. Road lane markers and traffic signs are often not visible or non-existent. Examples include pedestrian zones in urban areas, outdoor markets, sports squares, and crowded plazas. The highly dynamic nature of this infrastructure/environment domain severely impairs the capability of a Level 4 vehicle to operate in the manner expected of an SAE Level 4 vehicle. The vehicle is expected to travel within the crowd at low speed and to continuously assess the proximity of the surroundings even though the view of vehicles might be partially blocked by moving humans. Assistance from nearby intelligent infrastructure is neither expected nor provided due to pedestrian overcrowding and the highly dynamic nature of the environment.

As Figure 7 suggests, driving in disorganized environments can be even more complex compared to limited-structured roads. In the task of trajectory planning in such an environment, HADS should be capable of not only evaluating the spatial limitations and choosing the safest and most route for navigating through the crowd (e.g., the route with the lowest density of pedestrians) but also responding to changes in that environment in a timelier manner compared to the HADS of Level 4-D (Level 4 Vehicles on a Limited-Structured Road).

To facilitate their movement in the crowd, the HADS construct a real-time profile of the moving crowd, estimate their position in the crowd, and calculate relative distances to surrounding objects (individual pedestrians and other moving and stationary features) using their onboard sensors. The surrounding traffic consists of automobiles, two-wheelers, and people; therefore, mapping and localization requirements are high. The HADS are able to recognize the type, size, locations, directions, and intentions of other road users. Extremely dynamic disorganized scenes also raise challenges in driving strategies: the vehicle is supposed to travel through the crowd at a safe speed. The vehicle may need to perform stop-and-go behavior frequently, detour to bypass pedestrians, or inch forward to make space in a dense crowd [105]. At unsignalized intersections, similar to Level 4-D, Level 4-E’s HADS predict the actions of other road users and negotiate with them to pass the intersection safely. It is worth mentioning the parking task for this proposed level: the vehicle should be able to park itself on roadside or parking spots inside the crowd even when the boundaries of the parking place are not visible. Compared to other levels, parking is more demanding at this level because the HADS may be expected to be able to find a parking spot in an area that is crowded with milling pedestrians.

6. Discussion

6.1. The Proposed Supplement

Section 5 documents the establishment of a five-level classification supplement for Level 4 SAE automation, through an examination of the different infrastructure/environment domains and the capabilities of Level 4 HADS under those domains. It is noteworthy that the necessary capabilities for HADS differ across various levels. Some levels may not necessitate certain advanced capabilities as the driving scenarios at that level do not require such capabilities. On the other hand, certain levels may require more demanding capabilities for specific functions owing to the nature of the driving conditions at that particular level. For example, trajectory planning functions are not needed for Level 4-A because the HADS are generally guided by the guideway. The requirement of object recognition is relatively low for HADS of Level 4-A and Level 4-B because strict access control is applied for their driving scenes and road users other than vehicles are rare. In contrast, a fast and accurate vision system is necessary for HADS of Level 4-E due to the existence of a large number of pedestrians.

Each level of the proposed supplement indicates a mandatory minimum rather than maximum capabilities for HADS operations at that level. It is not possible to specify a complete set of features for each level of HADS since ADS-dedicated vehicles have not been extensively implemented. The details of each sublevel may be expanded in the future to reflect technology-driven advancements, and new sublevels may be added in the future.

6.2. The Future of HADS and Infrastructure Development

Using the proposed classification supplement, it is easier to characterize the capability of automated driving systems in a wide range of operating conditions. Accessibility to cyber infrastructure, the complexity of driving algorithms, and the domain of driving tasks vary across the different infrastructure conditions. For example, accurate digital maps with high definition which assist localization and detection are not available for all locations, particularly those in rural areas. On the other hand, the traffic conditions in urban areas are significantly more complex compared to rural areas. As a result, autonomous vehicles rely not only on the on-board sensors for navigation, but also heavily depend on infrastructure [29]. In this regard, certain studies focus on researching the optimal deployment of roadside units (RSUs) [106], the use of fog computing for efficient data exchange [68,88], and the identification of crucial intersections [107], aiming to expedite the integration of autonomous vehicles in urban environments.

In recognition of the nature of different levels of HADS, we provide Figure 8 to visualize the concept of potential differences among levels of HADS regarding their dependence on intelligent infrastructure and in-vehicle artificial intelligence (AI) systems. The AI systems carried by vehicles are expected to perceive the surrounding driving environment and make informed decisions in a timely manner. This is particularly critical for HADS of Level 4-D and Level 4-E, where the vehicle needs to drive in highly dynamic and complicated situations and operate in areas with inadequate supporting infrastructure and/or challenging environments. A powerful onboard AI system would make the AV capable of navigating challenging and diverse driving scenarios. However, a flexible and robust system is still far from reality. Ref. [108] reviewed state-of-the-art computer vision models used in automated driving, and pointed out that existing models are still inferior to human perception and reasoning. Ref. [109] suggested that such decision-making algorithms lack extensive real-world tests.

Infrastructure plays a crucial role in certain levels of HADS such as Level 4-A, Level 4-B, and Level 4-C through providing reliable traffic information, guiding the vehicle through the high-quality network, and even sending advisory instructions to AVs. These levels of AVs are easier to reach compared to other levels since existing research has studied how infrastructure can be built into applications [93,110,111,112]. For example, traffic lights were used to assign priority when more than two vehicles enter an intersection in UK’s Autodrive Project [111], and traffic accidents were avoided. Ye and Yamamoto also investigated the positive impact of setting dedicated lanes for AVs on traffic flow throughput [112]. These studies pointed out that superior system efficiency, a lower fatality rate, and improved fuel economy could be expected when infrastructure is made to play a key role in automated driving, which is likely to be the case for HADS of Level 4-A and Level 4-B.

7. Concluding Remarks

This study is based on the premise that high-level AVs, in the future, will not only be deployed in areas where they have been tested to exhibit full capabilities of Level 4 vehicle automation, but also in areas where the infrastructure and environment may not be so conducive to support such full capabilities. In that case, the existing SAE taxonomy may be inadequate for characterizing the capability of these systems and their ODD. To throw more light and stimulate discussion on this issue, this study proposed a five-level (SAE sublevels) classification framework to serve as a supplement to the current SAE taxonomy for highly automated driving systems (HADS). The study identified features of the infrastructure and environment as well as the minimum capabilities of HADS at each sub-level. In contrast to a school of thought which postulates that the capabilities of HADS should be viewed only against the functions of human drivers in the automated system, the proposed supplement emphasizes the inclusion of the infrastructure and corresponding driving environment in the classification of automated systems. The study recognizes that different capability levels of high-level automated driving are realized under different infrastructure and environmental domains, and therefore, the classification should be based not only on the capability of the vehicle (and thus, the functions of the human driver) but should be based on both the vehicle capability and the infrastructure–environment within which it operates. Through examining the significance of different infrastructure conditions, we proposed five sublevels of Level 4 HADS: Level 4-A (Dedicated Guideway Level), Level 4-B (Expressway Level), Level 4-C (Well-Structured Road Level), Level 4-D (Limited-Structured Road Level), and Level 4-E (Disorganized Area Level). From Level 4-A to Level 4-E, (a) the environment becomes less orderly and increasingly chaotic, and (b) support from nearby intelligent infrastructure decreases, and therefore, onboard navigation systems play an increasingly greater role.

The proposed supplement is descriptive rather than normative. The supplement will clarify the realistic capabilities of the ADS, the condition of use, and the requisite supporting infrastructure. AV developers and manufacturers are herein provided with a basis upon which they can provide the more accurate specification of the capabilities of their products from the ODD perspective; that way, user expectations regarding travel efficiency and safety of the ADS at different locations and environments can be clearer. In other words, the supplement will help clear up any confusion regarding the capabilities and limitations of AVs.

In future studies, work can be carried out to further improve certain aspects of the proposed supplement. First, to enhance further the applicability of the supplement, future efforts could examine the classification of infrastructure intelligence, and incorporate the dynamics of traffic scenes and road conditions. Second, the framework is based on the current state of the AV market, where the automation level is close to the SAE’s Level 3. However, it can be envisioned that various levels of current roadway design features including lane and shoulder width, horizontal and vertical alignment, and cross-section will need to be evaluated to facilitate AV traffic operations. Therefore, with the rapid development of automated driving technologies, potential adjustments could be applied to improve the feasibility and compatibility of our proposed supplement. In the future, a well-defined scientific framework that addresses the issue of varying capabilities required at distinct levels of HADS is imperative. Such a framework should provide a normative and quantitative specification of the essential capabilities for the different levels of HADS while additionally clarifying the respective degree of dependence on intelligent infrastructure or artificial intelligence. This paper’s Figure 8 may serve as a guiding framework for such discussions. Third, using real-world (and possibly in-service) data, quantitative comparisons between the capabilities of different HADS could be carried out in order to avoid descriptive comparisons as done in this paper.

Overall, the supplement presented in this paper is expected to be beneficial in clarifying potential sublevels of Level 4 automated driving systems and enhancing the SAE’s taxonomical classification. Reference to the proposed supplement is expected to benefit prospective AV consumers and vehicle manufacturers in terms of setting a clear and readily understandable bar of performance expectations in different environments and infrastructure settings. In addition, government road agencies will be placed in a better position to justify investments in infrastructure geared towards improving their infrastructure to support the coming age of driving automation.