A Novel Ramp Metering Approach Based on Machine Learning and Historical Data

Abstract

The random nature of traffic conditions on freeways can cause excessive congestion and irregularities in the traffic flow. Ramp metering is a proven effective method to maintain freeway efficiency under various traffic conditions. Creating a reliable and practical ramp metering algorithm that considers both critical traffic measures and historical data is still a challenging problem. In this study we use simple machine learning approaches to develop a novel real-time ramp metering algorithm. The proposed algorithm is computationally simple and has minimal data requirements, which makes it practical for real-world applications. We conduct a simulation study to evaluate and compare the proposed approach with an existing traffic-responsive ramp metering algorithm.

1. Introduction

Reducing traffic congestion and maintaining the flow within the freeway capacity is necessary for the safe and proper operation of any urban traffic system [1,2,3,4,5,6,7]. Ramp metering (RM) is one of the most important strategies for traffic control to reduce freeway delay and improve safety [8,9,10,11,12,13,14]. A ramp meter is a traffic light installed on a ramp to control traffic flow entering a freeway. RM is an effective way to reduce traffic congestion and maintain capacity flow on a freeway. It regulates the access of ramp traffic to the mainline [8]. Figure 1 shows a RM system which allows on-ramp vehicles to enter the freeway only if the ramp signal is green. Therefore, the vehicles need to wait behind the stop line for a green light to enter the freeway.

One of the most important impacts of RM is to prevent freeway breakdown phenomenon. This phenomenon often occurs at freeway entrance ramps where platoons of vehicles entering the congested freeway create a bottleneck and reduce service capacity. The shockwave created by a sudden drop in speed may propagate many miles upstream causing unsafe conditions. RM can minimize the shockwave effect and prevent freeway breakdown. In addition, it improves safety and reduces travel time and environmental pollution [15]. Field evaluations have proven the positive impact of RM on the traffic conditions on freeways [16].

RM methods are classified into two primary categories of fixed-time or pre-timed control and adaptive or traffic-flow responsive control. Fixed time methods, which are the simplest, consider historical traffic information to determine the metering rates and establish the rates on a time-of-day basis [17]. Therefore, these systems do not perform effectively in the presence of severe traffic fluctuations.

Traffic-responsive approaches are more advanced and, therefore, complex. They benefit from the data collected by sensors installed on freeways to calculate metering rates [15]. Local traffic-responsive methods control traffic conditions on the freeway by considering online traffic measures, such as on-ramp demand and occupancy in the vicinity of the ramp. Traffic data is typically collected using a vehicle detection system, such as inductive loop detectors in the vicinity of the ramp. ALINEA (3 Linéaire d’Entrée Autoroutière) [8] is an example of a promising traffic-responsive algorithm which is currently used in some ramp meter systems to reduce traffic congestion and maintain the desired freeway occupancy [15]. The inherent dynamic nature of traffic flow renders the traffic control a challenging problem that still requires more research [18].

According to the Ramp Management and Control handbook [19] most of the major traffic-responsive algorithms currently in use in the United States (four out of five) control the ramp signal only based on the current traffic volume. Many intelligent traffic-responsive algorithms based on advanced machine learning techniques, such as deep learning, have been discussed in the literature, which are reviewed in the background section. While such methods have proven effective in simulated test platforms, issues such as excessive programming complexity, long and difficult training procedures, and high data requirements limit their practicality in real-world scenarios [20].

In this paper, we use machine learning to develop a smart RM method. The proposed method relies uses common traffic metrics such as current flow, occupancy, and speed in real-time, as well as historical traffic data. The computational simplicity of the proposed method allows it to run on extremely low-end hardware, such as microcontrollers and legacy embedded systems. The machine learning models incorporated in the proposed algorithm can be practically trained with a few weeks of traffic data.

We use real traffic data collected from Stafford Road to the I-205 northbound ramp [21] in Oregon state to compare the performance of the proposed algorithm with ALINEA ramp metering method in a simulation study.

2. Background

A taxonomy of conventional ramp metering algorithms based on a study by Papageorgiou and Kotsialos [1] is presented in Figure 2. Fixed time metering is the oldest and simplest off-line strategy which is usually adjusted based on historical data and applied during particular times of day. Obviously, fixed time metering has very limited application due to its major shortcoming of not reacting to any real-time traffic metrics.

In contrast to fixed time methods, reactive ramp metering techniques are based on real-time traffic metrics. Local ramp metering uses traffic measures collected form the ramp vicinity. Demand, capacity, and occupancy based strategies allow as much traffic inflow as possible to reach the freeway capacity. ALINEA offers a more complex and responsive strategy that unlike capacity and occupancy strategies generates smoother responses to changes in metrics. While ALINEA performs well in maintaining traffic flow at the freeway, it may create long queues at the on ramp which leads to bottlenecks [16]. Different modifications are proposed to improve ALINEA algorithms to improve its performance in the actual scenarios including UP-ALINEA (Upstream-Occupancy Based ALINEA), FL-ALINEA (Flow-based ALINEA), and X-ALINEA/Q, AD-ALINEA [1,22,23]. More recent extensions to ALINEA include ALINEA with Speed Discovery [24], Data-Driven Iterative Feedback Tuning Approach of ALINEA [25], Proportional Integral ALINEA (PI-ALINEA) [26], Feed Forward (FF-ALINEA) [27], parameterized ALINEA with variable speed limit strategy [28], and CS-ALINEA [20].

Multivariable regulator strategies perform the same as local strategies, but more comprehensively and independently on a set of ramps and usually outperform local strategies. METALINE is a more general and extended form of ALINEA for coordinated control of ramp meters [29,30]. Other examples of multivariable methods include zone algorithm [31], bottleneck algorithm [32], System-Wide Adaptive Ramp Metering (SWARM) [33], and fuzzy ramp metering (which uses fuzzy logic algorithms) [34,35]. Some of the more recent fuzzy logic algorithms are based on genetic fuzzy logic control (GFLC) and swarm optimization [36,37,38,39]. Reactive ramp metering strategies are helpful to some extent if their target parameters (capacity, occupancy, etc.) are set to appropriate values. They also have a local nature which may cause some control inconsistencies in a larger scale.

Nonlinear optimal control strategy considers local traffic parameters and metrics, as well as nonlinear traffic flow dynamics, incidents, and demand predictions in a freeway network and outputs a consistent control strategy. This introduces many challenges including quite high computational load and complexity in design and application of corresponding models.

Knowledge based control systems are developed based on historical data and human expertise. They usually rely on heuristics and ad-hoc procedures for traffic control to provide a good and not necessarily optimal output. Integrated freeway network traffic control is a more general approach to nonlinear control that extends application of optimal control strategies to all forms of freeway traffic control.

All mentioned strategies have their own strengths and weaknesses, but in general, more complex models were introduced to address simpler models’ shortcomings. A common drawback of model based controls is their sensitivity to the model’s accuracy. Practically, mathematical traffic models can seldom represent the full, real-world traffic dynamics [30]. Additionally, models rely on accurate parameters and full information regarding the system they represent, which is not always easy to achieve. On the other hands, almost all approaches that apply control theory are computationally demanding and, therefore, impractical for real-world applications [40].

In case of knowledge based systems, inability to learn and adapt to temporal evolution of the system being controlled can be an issue, so knowledge based systems need to be periodically updated to remain efficient [40].

All these issues together, urge the quest to develop an intelligent algorithm which can be adapted to different settings and scenarios at a reasonable cost, without the need of very accurate parameters and enormous processing infrastructure. Artificial intelligence and machine learning approaches seem to be eligible candidates for this purpose. Several intelligent methods have been proposed for ramp metering including reinforcement learning (RL) [41] and artificial neural networks (ANN) [42]. Research presented in [18,40] and [43,44,45,46,47,48,49,50,51,52,53,54] describe some of the models that use reinforcement learning. Most of the reinforcement learning models use Q-learning [49,50,51] or some methods based on Q-learning. Models in [52,53,54] are based on ANN. There are several studies involving metering methods based on deep neural networks [55] and deep reinforcement learning [56,57,58].

For most of machine learning techniques, training phase is very critical and final performance and efficiency of the algorithm strongly depends on that. In some cases, researchers tried to introduce incidents and accidents to gain a well-prepared model. However, there can still be many unpredictable situations for which the proposed model would not be trained for. A possible solution we are proposing is to use real historical data captured from different ramp locations to train the models instead of or along with the use of simulation models. Additionally, we focus on integrating regression and clustering, two effective machine learning approaches, to come up with a framework to control ramp signal in an accurate and efficient way.

3. Methodology

The proposed ramp metering algorithm is visualized in Figure 3. Our method consists of four main modules: (1) Data refinement and feature selection; (2) regression; (3) clustering; and (4) ramp metering algorithm. In the following, we will discuss our proposed approach for each module.

3.1. Data Refinement and Feature Selection

We chose a freeway section of I-205, in the state of Oregon, and analyzed real traffic data collected at this station for a working week [21]. As shown in Figure 4, the number of vehicles entering the freeway through the ramp counted at five-minute intervals forms an M-shaped pattern with two peaks on working days, while on Saturdays and Sundays it generates a pattern with one peak. Therefore, we use data from the working days (Monday to Friday) to train and evaluate the algorithm in this study. The same procedure can be followed to generate results for weekends as well, but it is important to use weekend data to train the algorithm.

The other data source is the real-time traffic data which includes timestamps, volume, speed, and occupancy for two lines from the same location recorded at 5-min intervals. For speed and occupancy, we computed the average for the two lines, and for volume, we summed up the values for the two lines.

3.2. Regression

In the regression stage, we used linear regression to predict traffic volume Vol(t*) based on Time(t), Occupancy(t), Speed(t), and Vol(t) data. To validate the regression results, we compared predicted data to real data as shown in Figure 5. Despite the fact that perfect conformance is not possible with regression, we look for similarity in trends. The results show that the fitted model can consistently replicate the real data trend (i.e., the M-shaped pattern) to a great extent. The fitted model is used along with other components in the ramp metering algorithm.

3.3. Clustering

In this step, we implement two k-means clustering approaches. First, we clustered traffic volume data based on Time and ∆vol/∆t to identify traffic phases capturing the fluctuations in traffic volume based on the time of day. Identifying the traffic phase helps the ramp metering algorithm (Figure 8) to compare the measured traffic volume with the expected behavior and detect anomalies such as accidents.

The second cluster is only based on Δvol/Δt to identify the traffic type. Traffic type characterizes the rate by which the traffic is expected to change. Detecting the traffic type helps the ramp metering algorithm to choose the appropriate model and values for the ramp signal state. Both clustering schemes are presented in

Table 1. Two cluster sets and their definitions.

Cluster	Cluster 1: Traffic Phases	Cluster 2: Traffic Types
1	0:00–4:40 AM (early morning)	Sharp negative slope
2	4:45–9:35 AM (morning)	Moderate negative slope
3	9:40 AM–2:25 PM (afternoon)	Small slope or constant
4	2:30–7:10 PM (evening)	Moderate positive slope
5	7:15–23:55 (night)	Sharp positive slope

.

The five traffic phases (1–5) are shown in Figure 6. After some experimentation, setting the number of clusters (k) in the K-means algorithm to five resulted in the best classification of the M shaped daily traffic volume behavior.

Traffic types identified in the second clusters are shown in Figure 7. Two data points which could not be fit into any of the five clusters were classified into the ‘other’ category.

3.4. Proposed Ramp Metering Algorithm

The flowchart view of the ramp metering algorithm (RMA) is presented in Figure 8. The algorithm uses the regression and clustering model outputs.

The RMA starts by predicting the traffic volume for the next time interval V_t* and applies linear interpolation to calculate the volume for the next time interval V(t + Δt). The reason for the discrepancy between the time intervals is to allow the RMA to update the green phase duration and light status in real-time (e.g., every second) using the live sensor data despite the training dataset being aggregated over longer periods of time (e.g., 5 min). As shown in Equation (1), the current traffic volume V_t measured by the sensors is used along with the predicted volume in the linear interpolation:

$$V\left(t+\mathsf{\Delta }t\right)=\frac{V_{t^{*}}-V_t}{t^{*}-t}·\mathsf{\Delta }t+V_t$$

(1)

Next, we compare the estimated traffic volume for the next time interval V(t + Δt) with the volume threshold. Volume threshold is traffic volume at which the ramp signal is activated. Threshold volume needs to be set to a value below the highway breakdown capacity.

If the estimated volume is less than the threshold, the ramp signal is disabled (always green) by setting the green phase duration (T_Green) equal to the traffic cycle (T_Cycle). In case of one of the 2, 3, or 4 traffic phases, we raise the UAF (upstream accident flag) since traffic volumes less than the threshold are not expected at these phases and can be caused by an upstream accident. Similarly, if the estimated volume is greater than threshold while in traffic phases 1 or 5, we set the DAF (downstream accident flag) since high traffic volume at these phases are not expected and can be caused by an accident downstream.

To calculate the green phase duration for the ramp signal, highway capacity needs to be calculated. Highway capacity is a fundamental parameter in traffic studies. In simple words, capacity is defined as the maximum sustained 15-min flow rate observed in one direction in a uniform highway segment. Aside from speed and road characteristics, capacity also depends on vehicle and driver behavior. Zunhwan et al. [59] developed a dynamic method to estimate capacity based on vehicle speed and time headway. They collected vehicle speed, time headway, and flow data and developed three models based on the upper 5%, 10%, and 15% time headway values sorted in ascending order referred to as 95%, 90%, and 85% models, respectively.

Table 2. Polynomial dynamic speed time headway models [59].

Model	Equation	Relative Time Headway
95%	$Y=32.793\times X^{-0.9916}$	Shortest
90%	$Y=32.532\times X^{-0.9682}$	Medium
85%	$Y=32.537\times X^{-0.9524}$	Longest

shows the three models used to estimate time headway in seconds denoted by Y based on speed in km/h denoted by X.

The 95% model captures the lowest time headway values (i.e., very dense traffic). We use this estimate when traffic type 3 is detected since drivers tend to maintain shorter time headways during constantly high traffic flows [60]. The 85% model however generates the longest time headways which is applicable to traffic types 4 and 5 representing sharply increasing traffic flow. The 90% model is, therefore, used during the decreasing flow periods (i.e., traffic types 1 and 2) in which the drivers tend to increase their time headway as the flow decreases. Hourly capacity (vehicles per hour) can be calculated by dividing 3600 by the estimated headway values in seconds as shown in Equation (2):

$$Cap=\frac{3600}{H}$$

(2)

At the end, the green phase duration for the ramp signal is computed based on the estimated capacity and predicted traffic volume for the next time interval using Equation (3). The signal cycle time T_cycle is the summation of green and red phase durations in seconds while $T_{Green}^{Min}$ is the minimum green phase duration:

$$T_{Green}= T_{Cycle}-\left(\frac{V\left(t+\mathsf{\Delta }t\right)}{Cap}\right).\left(1-\frac{T_{Green}^{Min}}{T_{Cycle}}\right), T_{Green}^{Min}\le T_{Green}\le T_{Cycle}$$

(3)

The calculated green phase duration is checked to ensure the result is always between minimum green phase duration and traffic cycle.

4. Evaluation and Results

In this section, we compare the proposed algorithm with the most popular traffic-responsive algorithm ALINEA in a simulation study. We used MovSim, which is a microscopic traffic simulator [61]. The source code for the JavaScript version of the simulation software is available on GitHub [62].

Figure 9 shows a screenshot from the ramp metering scenario tested on MovSim. The software had to be customized to accommodate the ramp metering scenario with upstream and downstream sensors allowing to capture real-time speed, flow, and occupancy data. Both ALINEA and the proposed ramp metering algorithms were implemented in the software. Additionally, incoming onramp and mainline traffic flows were generated based on the real traffic data. Data from two traffic sensors on the mainline before and after the ramp (referred to as upstream and downstream sensors) as well as ramp queue size and ramp signal status were recorded in a comma separated value (CSV) data files for visualization and analysis.

4.1. No Control Scenario

The first scenario being discussed involves no ramp metering to form a baseline for comparison. We used the real mainline and onramp traffic data for a weekday from Stafford Road to the I-205 northbound ramp [21] in Oregon state in the simulation model. We experimented with the incoming flow to ensure breakdown happens in the absence of ramp control. To effectively create breakdown during the entire peak hours, both onramp and mainline flows were multiplied by 24 since the original data was far below the breakdown capacity. The same multiplier is used in the two scenarios involving ALINEA and the proposed ramp metering method to evaluate their effectiveness in preventing breakdown and handling extremely high traffic flow.

Hourly traffic flows upstream and downstream of the ramp are presented in Figure 10. As the results suggest, breakdown happens when the flow exceeds 2500 vehicles per hour at 6:50 and continues until 22:30. The same situation is visible in Figure 11 with respect to the speed falling from 88 km/h (speed limit) to about 44 km/h during the same period.

The blue line in Figure 12 indicates the green phase duration of 40 s which is the traffic cycle value in the two next scenarios to simulate disabled (always green) ramp signal. The length of the ramp queue fluctuates between 33 and 41 during the rush hours.

4.2. ALINEA Ramp Control Scenario

ALINEA employs closed-loop feedback for green-phase duration as follows:

$$g\left(k\right)=g\left(k-1\right)+K_R\frac{C}{r_{sat}}\left(\widehat{O}-O_{out}\left(k\right)\right), g_{min}\le g\le g_{max}$$

(4)

• g(k): Green phase duration at time interval k
• g(k − 1): Green phase duration at time interval k − 1
• C: Traffic cycle (red phase + green phase duration)
• $r_{sat}$: The ramp capacity flow (vehicles/hour)
• $K_R$: Regulator parameter (vehicles/hour)
• $\widehat{O}$: Critical occupancy (%)
• $O_{out}\left(k\right)$: Occupancy downstream of the merge area at time interval k (%).

In this study, we set K_R = 70, C = 40 s, r_sat = 2766 vehicles/hour, and $\widehat{O}$ = 13%. The values for the ramp saturation capacity r_sat and critical occupancy $\widehat{O}$ were calculated based on the ramp characteristics and results from the first scenario. Moreover, K_R = 70 is the recommended value in the majority of the analytical studies using ALINEA in the literature.

Hourly traffic flow plots for the ALINEA scenario are presented in Figure 13. As the results suggest, ALINEA kept the flow below the breakdown capacity and handled the excessive flow during the majority of the peak hours. The fluctuations in the flow indicate frequent breakdowns which have recovered quickly.

The same fluctuations caused by the periodic breakdowns is visible in Figure 14. The average speed is 88 km/h (speed limit) for the majority of the peak hours, falling to about 40 km/h during the breakdown episodes.

Green phase duration and ramp queue length plots in Figure 15 explain why the periodic breakdowns happened. As the ramp flow reaches the ramp capacity saturation flow r_sat, ALINEA extends the green phase duration to prevent excessive traffic forming on the ramp. The onramp traffic flows instantly to the mainline and increases downstream traffic volume which results in another long red phase cycle and possible breakdown. This cycle keeps repeating during peak hours. In addition to adversely affecting the mainline flow and speed, this cyclic behavior is a serious safety concern increasing the chance of risky merge maneuvers during peak hours.

When the peak hours start, the green phase duration is clamped down to the lowest allowed value (10 s) until the saturation on the ramp causes the algorithm to increase the green phase duration to over 20 s. Another notable observation is the periodic spikes in the ramp queue length reaching 35 vehicles or more in some instances.

4.3. The Proposed Ramp Control Scenario

The proposed ramp metering method was run with the same parameters as the previous scenarios. Traffic cycle and minimum green phase duration were set to 40 and 10 s, respectively. The volume threshold was set to 2000 vehicles/hour, which is slightly below the breakdown capacity of 2500 vehicles/hour observed in Figure 10. Real data for one work week (Monday–Friday) from Stafford Road to the I-205 northbound ramp [21] in Oregon state was used to train the regression and K-means algorithms.

The traffic flow in Figure 16 is stable around the breakdown capacity during peak hours. The fluctuations in the flow are less severe than those observed in the previous scenario.

The speed remained quite steady around 88 km/h during the peak hours according to Figure 17. Although there are several instances of abrupt decrease, the lowest speed is about 60 km/h which is significantly higher than the breakdown speed of 40 km/h observed in the two previous scenarios. According to the flow and speed plots, no breakdowns have occurred.

The green phase duration plot (blue line) in Figure 18 indicates that the proposed method actively adjusted the green phase duration to keep the traffic flow and speed within the capacity. The green phase duration started at about 20 s and varied between 15 and 20 s during the majority of the peak hours.

The ramp queue length (orange line) fluctuated between 20 and 25 (with occasional lower values) during the peak hours.

4.4. Ramp Signal State and Queue Length

The percentage of ramp signal green and red states for ALINEA and the proposed algorithm are compared in Figure 19. The green and red ratios are calculated based on the ramp signal status recorded every second in the simulation model. The average green phase durations for ALINEA and the proposed method are 28 and 26 s, respectively, which might sound contradicting with the pie charts below. However, in both ramp metering scenarios, green phase duration was constantly calculated and adjusted based on the live data from the downstream sensor. Therefore, comparing the actual read and green signal states is a better indicator of ramp control compared to the green phase duration.

According to Figure 19, ALINEA generates 8% more red lights compared to the proposed algorithm. This indicates that the proposed algorithm is more permissive in allowing vehicles to enter the freeway compared to ALINEA.

The comparison of the ramp queue lengths in Figure 20 indicates that, while the average queue length is slightly higher for the proposed algorithm, the maximum length of the ramp queue for ALINEA is higher.

5. Conclusions and Future Work

This study investigated the potential of using simple machine learning techniques to develop an algorithm for ramp signal control. The proposed method incorporates linear regression and clustering approaches to learn the traffic flow trend over time. We compared the proposed algorithm with the widely used traffic-responsive algorithm ALINEA on a real traffic dataset.

The results of the comparison confirm that the proposed algorithm can effectively maintain freeway traffic flow at reasonable levels while allowing the on ramp traffic into the mainline as much as possible. However, ALIENA demonstrated cycles of long red phases followed by overcompensation and brief breakdowns.

While linear regression and K-means are not sophisticated machine learning methods, the proposed method showed promising results in the simulation study. Minimal computational complexity of the proposed method makes it a perfect choice for implementation on embedded systems and other low-end hardware for cost effective implementation on existing ramp signal controllers.

The proposed algorithm generates a green phase duration based on the discrepancy between the real-time capacity dynamically calculated using Zunhwan et al.’s [59] polynomial time headway models and the current flow from the downstream sensor. This allows the proposed algorithm to exploit the existing capacity to allow as much traffic as safely as possible into the mainline without causing breakdown.

Training and fine tuning the algorithm is simple and requires a few weeks of traffic data. Another advantage of the proposed algorithm is the capability to detect irregularities in traffic flow and react by raising possible upstream or downstream accident flags which can help with early detection of accidents and alleviating the consequences.

An area for future work is to explore the application of dynamic regression models. Dynamic models use current data to improve the accuracy of the estimates which results in a better overall performance. This can also be used as a mechanism to update the training dataset over time to avoid periodic updates. Another possible improvement is to use video data to update the dynamic time headway model parameters. Zunhwan et al. [59] used video data from a highway in South Korea to extract the drivers’ time headway behavior. While this can be a costly endeavor, it improves the accuracy of the capacity estimates, which results in a more efficient usage of the highway capacity.