Enhancing Model-Based Anticipatory Traffic Signal Control with Metamodeling and Adaptive Optimization

Abstract

Traffic signal control is one effective way to alleviate traffic congestion. Anticipatory traffic signal control determines signal settings from a network planning perspective, which takes into account the influence of travelers’ route choice response and triggers better equilibrium flow patterns for better network performance. For the route choice response, it is usually predicted by a response function known as traffic assignment model. However, the response behavior can never be precisely modeled, leading to a mismatch between the modeled and real traffic flow patterns. This model-reality mismatch generally contributes to suboptimal control performance and hence brings unexpected congestion in real-life traffic operations. This study aims to address the model-reality mismatch and proposes an effective anticipatory traffic control for real operations. A metamodel is introduced that serves as a surrogate of the unknown structural model bias. Then an iterative optimizing control scheme is applied to correct the model bias by learning from observations. By integrating the model-based control design with data-driven learning techniques, the metamodeling framework is able to enhance the control performance. Moreover, the analytical model bias formulation allows theoretical investigation of the model approximation error. To further improve the control performance, a joint traffic model parameter estimation is developed, hence achieving a better model calibration jointly with the model bias correction. The proposed control method is examined on a test network. Numerical examples confirm the effectiveness of the proposed method in improving control performance despite the model-reality mismatch. Comparison results show that the proposed method outperforms the traditional model-based control method and an improvement of 14.8% in total travel time is achieved in the example network.

1. Introduction

Traffic signal control is one important means of traffic management in urban road networks. Optimizing signal timings is considered a cost-effective way to reduce congestion and improve urban mobility. Local control strategies usually assume given traffic arrivals, and they are not efficient regarding network performance which relies on the flow patterns. Although many control measures and strategies have been developed over the past decades [1], the effective design of network-wide signal control that incorporates travelers’ route choice response remains a challenge.

It has long been recognized that route choice and signal control are closely connected. Traffic control can be used to affect the route choice behavior and hence the resulting flow pattern so as to achieve better network-wide performance. An anticipatory traffic control (ATC) was proposed for the combined traffic assignment (route choice) and signal control problem [2,3,4]. In general, ATC is a specific instance of the more general network design problem (NDP), in terms of signal setting design.

A comprehensive design of ATC is typically based on the availability of an accurate flow response function (i.e., traffic assignment model) and uses a model-based optimization for setting signal plans [3]. This signal plan is kept fixed when implemented on a real-life network. However, traffic managers often face model uncertainty about the real response of road users. In general, the actual route choice behavior is usually approximated by traffic assignment models, which can never be precisely modeled, leading to a mismatch between the modeled and real traffic flow patterns. The presence of model-reality mismatch, due to structural model bias or inaccurate model parameters, could result in suboptimal control performance, for instance, unexpected delay and congestion in real-life traffic operations. How to address this model-reality mismatch becomes a key question for practical applications of ATC.

This paper focuses on designing an effective anticipatory traffic control that can perform well for real operations. In order to achieve optimal control associated with real-life network performance, a data-driven iterative learning technique is integrated with the model-based design. The objective is to iteratively amend model bias and enhance ATC by learning from observing the real system response. We consider the fact that routine traffic operation is repetitive and we can observe traffic flows during one time period, defined as an epoch. Then we can learn from the observations how to implement better control in the next epoch. We assume that traffic flow stabilizes in an epoch and we can obtain the equilibrium flow measurements from observations. Therefore, the epoch refers to a period that is long enough, e.g., the epoch of days or weeks of traffic operations [5], in order to allow the system to settle in equilibrium every epoch. For instance, an empirical study has found that after the occurrence of a bridge collapse, individual route choice behavior changed and the aggregate traffic stabilized in about six weeks [5]. An adaptive fine-tuning activity is therefore straightforward, which is likely to be done in practice by traffic engineers: they implement the optimal control and observe the users’ response; then they may learn from the data, e.g., the aggregate traffic flows, obtained on the previous epoch, and calculate optimal control for the current epoch.

Whereas heuristic approaches, such as the trial-and-error procedure, have been investigated in the literature, this paper adopts a more systematic approach based on the metamodeling technique. The approximation or metamodeling technique has been developed for simulation-based optimization or large-scale problems. The main motivations of metamodeling include: for large-scale problems, it is computationally expensive to evaluate the performance function, hence a metamodel is usually used as a surrogate of the expensive simulation process; moreover, the metamodel can be also used as a surrogate of performance functions without closed-formed formulation, or unknown model.

In this regard, this study introduces a metamodel as a surrogate of the unknown process model through the analysis of the structural model bias. The metamodel-based optimization is then performed, which incorporates the effect of the model bias. To improve the metamodel performance, one can learn from the actual observations of the real system. An iterative improvement scheme is straightforward, which has been applied in traffic perimeter control [6] and traffic signal control [7]. Therefore, this study further introduces an iterative learning scheme and the iterative learning is performed on the metamodel to elevate the control algorithm to its best achievable performance. By integrating the model-based control design with data-driven learning techniques, the metamodeling framework is able to enhance the model-based control performance. Moreover, the analytical formulation allows theoretical investigation of the model approximation error.

In addition to the structural model bias that cannot be captured by model parameters, this paper also addresses model inaccuracy due to parametric errors. It is known that traffic model parameters can be quite sensitive to the topology and characteristics of a specific network, for instance, the free-flow speed and the jam density of the network, which need to be calibrated before their applications for the specific system. This study further performs model parameter estimation jointly with model bias correction. Hence, a better model calibration is also achieved, which may be applied to other traffic management measures than signal control design, for instance, travel information or route guidance.

The main contributions of this study are summarized as follows:

• This paper develops an effective anticipatory traffic signal control that tackles the model-reality mismatch in the equilibrium flow response function.
• This paper introduces a metamodeling framework that integrates the model-based control design with data-driven iterative learning optimization.
• This paper performs traffic parameter estimation jointly with model bias correction to achieve a better model description.

The rest of the paper is organized as follows. Section 2 provides an overview on the anticipatory traffic control as well as the problem of enhancing the model-based control. In Section 3, the mathematical formulation is elaborated. The model-based anticipatory control optimization problem is first formulated. An iterative learning scheme is then proposed to enhance the model-based design. Furthermore, the iterative model bias correction is jointly performed with a choice behavioral parameter estimation as shown in Section 3. The proposed control method is tested in an example traffic network in Section 4. Section 5 presents conclusions and discussions on the future study.

2. Literature Review

2.1. The Anticipatory Traffic Control Problem

In anticipatory traffic control (ATC), the controller anticipates the travelers’ route choice response to the implemented signal settings, aiming for the resulting flows to achieve the network-wide objective, e.g., total network travel time. It is important to model the interaction between traffic assignment and signal control as depicted in Figure 1. A mutual decision-making procedure is presented, which assumes the controller to respond to the travelers’ route choice; in turn, travelers make a route choice based on the travel cost which accounts for the signalized delay. The controller may play a leader’s role and anticipate the route choice response, while the travelers act as followers, who follow the signal timing and make route choices according to the corresponding travel cost. As such, ATC is characterized by this leader-follower structure. In general, ATC is a specific instance of the more general network design problem (NDP), in terms of signal setting design. In literature, the NDP is typically formulated as a bi-level optimization problem, or a mathematical program with equilibrium constraints [8,9,10,11,12]. Comprehensive overviews have also been conducted. For instance, reference [13] was among the first to provide an extensive introduction to the interaction between traffic assignment and signal control. Reference [14] presented a review on the simulation-based dynamic traffic assignment models with urban traffic control systems.

However, it is impossible to precisely model the complex traffic system. There are exit errors in the underlying models. Reference [15] has pointed out that there exist errors in the traffic assignment models; these errors influence the network design decisions and they should be addressed in the network planning procedures. Reference [16] takes into account the actual stochastic demand in designing an optimal signal setting. To achieve an effective ATC that can perform well for real traffic operations, it is necessary to tackle the mismatch between the prediction model and the process model.

2.2. The Problem of Enhancing Model-Based Control

To enhance the model-based ATC control, adaptive fine-tuning is incorporated. A trial-and-error approach has been investigated for other control measures, e.g., tolling. For example, a trial-and-error iterative tolling scheme was proposed in the absence of an exact demand function [17,18,19]. After imposing the toll, link flows are observed. Based on the observations, traffic managers can apply the trial-and-error scheme to determine optimal road pricing, taking account of travelers’ response to the toll. Furthermore, this fine-tuning scheme was extended to the application of dynamic congestion pricing taking into account different vehicle types [18], as well as the application of a tradable credit scheme [20]. Reference [19] combined link capacity constraints with the trial-and-error approach for congestion pricing under elastic demand. Whereas the trial-and-error tolling procedure follows a heuristic approach, a more systematic approach based on the metamodeling technique is proposed. Reference [21] provided a review of the state-of-the-art metamodel-based techniques according to their role in supporting engineering design optimization. For traffic applications, reference [22] developed a metamodel method that integrates a physical model component and a general-purpose component to optimize urban transportation problems. In literature, the metamodeling approach has been widely adopted to address large-scale traffic simulation calibration [22,23,24].

For further improvement of the model-based control performance, a joint model parameter estimation is applied, to achieve a better model calibration jointly with the model bias correction. Based on the available measurement data, a model calibration procedure is usually conducted offline, and the model parameter values are identified properly via the offline calibration [25,26,27]. In general, a least square error problem is usually utilized to formulate the model calibration problem, whereby the discrepancy between the real process and the model is minimized, using a certain quadratic error function. Other approaches that extract information from data have also been extensively investigated, for instance, the statistical methods. Reference [28] provides a thorough overview of the system identification methods considering the characteristics of the models to be estimated. For joint estimation, reference [29] proposed a joint estimation of traffic flow variables and important traffic model parameters, so as to achieve better adaptive capability for the traffic state estimator.

3. Mathematical Formulation

3.1. Model-Based Anticipatory Traffic Control Optimization

Anticipatory traffic signal control optimizes signal settings taking into account travelers’ route choice behavior in response to signal changes. The route choice response, as well as propagations of flows over the network, can be captured by a traffic assignment model. This study focuses on the static traffic assignment. It is described as a mutually consistent system formulated by a fixed-point model, which usually combines the link cost function and the flow function [30,31]. Let $\mathbf{c}$ denote the link cost, which is a function of link flows $f$ and signal settings $g$ through the function form $\mathbf{C}(.,.)$, for instance the Bureau of Public Roads (BPR) function. In general, different signal control parameters can be incorporated, e.g., cycle length, signal green split, offset, depending on the specific function considered. Let denote the link flow operator $F(.)$. The link cost function (i.e., traffic supply function) and the flow function (i.e., traffic demand function) are formulated as follows.

$$\begin{array}{c}\mathbf{c}=\mathbf{C}(f,g)\\ f=F(\mathbf{c})=Bh(\mathbf{c}) \end{array}$$

(1)

Solving the solution of the system (1) represents a classic fixed-point problem. Assuming that the link cost function is continuous and strictly increasing, and the link flow function is continuous and monotonically decreasing, the fixed-point solution is unique [3,31]. This solution is used to describe the equilibrium flow response which is then written as follows.

$$f=F(\mathbf{C}(f,g))$$

In this study, we assume that the traffic system stabilizes on the study horizon and we can obtain the equilibrium flow measurements. The equilibrium flows $f$ is represented as a function of signal settings $g$. Now reformulate the fixed-point solution in the following Equation (2).

$$f^{real}=f^r(g)$$

(2)

The superscript real means equilibrium flows in the real-life network. $f^{\mathit{r}}(.)$ refers to the real flow response function. Equation (2) indicates that the real equilibrium flow depends on signal control decisions.

In general, it is impossible to accurately describe the reality system in mathematics, especially regarding the complexity of modeling human response behavior. It is usually approximated by an equilibrium traffic assignment model.

$$f^{real}\approx f^{Eq}(g,\mathsf{\mu })$$

(3)

in which, $f^{Eq}(.)$ is the equilibrium flow model, and $\mathsf{\mu }$ represents a set of model parameters, which is adjustable to increase modeling accuracy. Two types of model inaccuracy are addressed in this paper: one is the imperfect calibration of $\mathsf{\mu }$ and the other is the structural model bias $f^{Eq}(.,.)\ne f^{\mathit{r}}(.)$.

By such a model approximation, one can formulate a model-based anticipatory traffic control optimization problem as follows.

$$\underset{g}{\min } z(g,f)$$

(4)

$$s.t. f=f^{Eq}(g,\mathsf{\mu })$$

(5)

$$f\ge 0$$

(6)

$$\chi (g)=0$$

(7)

$$g^L\le g\le g^U$$

(8)

here, $z(.,.)$ is the objective function of the optimization problem, for example, network total travel time. Equation (5) is the equilibrium flow model and (6) is a non-negativity constraint on the link flows. Equation (7) represents constraints on dependent signal timings. (8) represents the boundary of the signal control variables.

By solving the model-based optimization problem (4)–(8), an optimal signal control setting $g*$ is derived. Due to the mismatch between model and reality, $g*$ can differ significantly from the real optimum denoted as $g^{real}*$, leading to suboptimal control performance, and even worse, resulting in unexpected congestion and spillback.

3.2. Model Bias Correction Using Iterative Learning

3.2.1. Model Formulation

Motivated by an iterative learning control (ILC) technique, which was originally presented for robotic control by [32], this paper applies an iterative learning scheme to compensate for the unknown structural model error. An iterative improvement on control settings is performed, by using the link flow measurements $f^{mea}$. In this study, we assume that we can observe the link flows and that the flow measurements are noise-free, thus $f^{real}=f^{mea}$. The basic idea behind ILC is the iterative improvement by learning from observed errors. Whereas the conventional ILC usually follows pure data-driven approaches [33,34,35], this paper integrates iterative learning to enhance a model-based design.

A model bias term $b$ is first introduced as in Equation (9), which describes the error between real measurements and model prediction.

$$b=f_{}^{mea}-f^{Eq}(g,\mathsf{\mu })$$

(9)

It is important to correct the model bias for the model-based anticipatory control optimization. It is known that in the bi-level optimization problem, the derivative of equilibrium flows to design variables, in our case the signal settings, is a crucial element in recognizing the leader-follower structure of the bi-level problem, hence using this derivative information to develop solution methods could provide sufficient solution optimality [8]. In this study, a simple polynomial function based on a first-order approximation is applied for the model bias correction, which updates both the value and the sensitivity around the current operating point $g_k$.

$$\begin{array}{l}b=b_k+{\delta }_k(g_{k+1}-g_k)\\ {\delta }_k=\frac{\partial f^r}{\partial g}|{}_{g_k}-\frac{\partial f^{Eq}}{\partial g}|{}_{g_k}\end{array}$$

(10)

in which, $b_k$ is the model bias observing at the current operating point, ${\delta }_k$ denotes the Jacobian error between reality and model. A one-step prediction in model bias correction is captured by Equation (10). It means that during the calculation of the new signal settings, traffic managers take into account the impact of a new signal setting on correcting the model bias. Whereas a finite different method can be applied to calculate the derivative of modeled equilibrium flows with respect to signal settings, calculating the derivative of the real equilibrium flows is not trivial. Following Brdyś’s method [36], a different way of implementing the finite different approximation has been adopted to determine the real flow derivative, which uses measurements observed in the previous iterations instead of additional perturbations.

$$\frac{\partial f^r}{\partial g}|{}_{g_k}=F(g_k)G^{-1}(g_k)$$

(11)

$G(g_k)=[\begin{array}{ccc}{g}_k-g_{k-1}& \dots & g_k-g_{k-n_g}\end{array}]$ is a matrix of signal changes and $F(g_k)=[\begin{array}{ccc}{f}_k^{mea}-f_{k-1}^{mea}& \dots & f_k^{mea}-f_{k-n_g}^{mea}\end{array}]$ is the corresponding matrix of measured flow changes, in which $n_g$ is the number of signal control variables. Thus the real flow derivative is calculated based on a flow set containing ($n_g$ + 1) flow measurements {$f_k^{mea}$, $f_{k-1}^{mea}$, …, $f_{k-n_g}^{mea}$}, as well as the past ($n_g$ + 1) signal settings. Equation (11) can estimate the real flow derivative at a sufficient level of accuracy in the case of few variables and measurement noise-free. In the presence of measurement noise, and considering multiple decision variables for a large-scale problem, inverting the matrix in Equation (11) is not trivial, which is left for our future study. Methods of inverting the matrix in the presence of measurement noise have been discussed in [37].

Based on the model bias correction, a metamodel of the real system, denoted as $f^{meta}(g,\mathsf{\mu })$, more specifically the real equilibrium flows, is derived.

$$f^{meta}(g,\mathsf{\mu })=f^{Eq}(g,\mathsf{\mu })+b_k+{\delta }_k(g-g_k)$$

(12)

The model-based anticipatory control optimization problem (4)–(8) is then integrated with iterative learning on model bias correction. After the completion of the kth iteration, the enhanced control calculates, an optimal signal setting $g_{k+1}*$ for the next iterative k + 1. This is derived by solving the following optimization problem.

$$g_{k+1}*=\underset{g}{\arg \min } z(g,f_{k+1}^{meta}(g,\mathsf{\mu }))$$

(13)

$$s.t. f_{k+1}^{meta}(g,\mathsf{\mu })=f^{Eq}(g,\mathsf{\mu })+(f_k^{mea}-f^{Eq}(g_k,\mathsf{\mu }))+{\delta }_k(g-g_k)$$

(14)

$$f_{k+1}^{meta}(g,\mathsf{\mu })\ge 0$$

(15)

$$\chi (g)=0$$

(16)

$$g^L\le g\le g^U$$

(17)

Equation (14) represents that the metamodel modifies the model prediction at each iteration.

Using the newly obtained $g_{k+1}*$ as the optimization direction for the next iterate, we then choose a step size $\mathrm{{\rm K}}$ along the optimization direction. Then we can derive the new control setting:

$$g_{k+1}=(I-\mathrm{{\rm K}})g_k+\mathrm{{\rm K}}{g}_{k+1}*$$

(18)

here $\mathrm{{\rm K}}$ is a gain matrix representing a suitable step from $g_k$ to $g_{k+1}*$ and usually takes a value of $\mathrm{{\rm K}}=diag({\lambda }_1,\dots ,{\lambda }_{n_g})$, in which ${\lambda }_1,\dots ,{\lambda }_{n_g}\in [0,1]$, hence allowing in principle different step sizes for the different dimensions of $g$. In many cases, $\mathrm{{\rm K}}$ is regarded as a design parameter and serves as a practical setting for regulating convergence.

Algorithm 1 presents the procedure to calculate the optimal control solution.

Algorithm 1 Enhanced anticipatory traffic signal control algorithm

Step 1: Initialization. Set initial value for signal setting $g_0$, traffic model parameter $\mathsf{\mu }$, and design parameter $\mathrm{{\rm K}}$. Step 2: Solve the model-based control optimization. Calculate the initial $f^{Eq}(g_0,\mathsf{\mu })$ and measure $f_0^{mea}$ based on $g_0$. Then solve $g_1*$ from the control optimization problem (13)–(17), derive $g_1$ and set k = 1; implement $g_k$, calculate the equilibrium flow by model prediction $f^{Eq}(g_k,\mathsf{\mu })$ and obtain the flow measurements $f_k^{mea}$. Step 3: Perform the model bias correction. Calculate the model bias $b_k$, at the current operating point, calculate both the model Jacobian for and the reality Jacobian, then derive the Jacobian error ${\delta }_k$. Step 4: Update the metamodel. Update the metamodel (12) with the model bias correction, derive a prediction flow for designing the next optimal signal control; Step 5: Solve the enhanced control optimization. Calculate the optimal signal setting based on the updated metamodel, derive an optimization direction; design appropriate step size and update signal control with Equation (18). Step 6: Convergence check. If the predefined termination condition is satisfied, then stop; otherwise go to step 2, set k = k + 1.

3.2.2. Solution Property

As discussed, this study develops an enhanced anticipatory traffic control with iterative learning. The ultimate goal is to elevate the model-based solution to the real optimal point by learning from observations. It indicates that the optimal solution derived from the enhanced control method should be consistent with the solution to the real optimization problem.

The model bias formulation allows explicit analysis of the solution property (solution optimality). This can be addressed via a general analysis of the necessary optimality conditions (NOC) of the optimization problem. It needs to prove that the NOC of the enhanced control method matches with the NOC of the real optimization problem. The first-order NOC is usually known as the Karush–Kuhn–Tucker (KKT) conditions [38]. The NOC point is defined as when the KKT conditions of the optimization problems are satisfied. By applying iterative learning to the model bias correction, i.e., iteratively correcting the model error, it ensures that the necessary optimality conditions of the model-based optimization match with the necessary optimality conditions associated with the real-life traffic system. Upon convergence, the optimal solution by solving the model-based design is consistent with the real optimal point (the proof follows a general analysis of KKT conditions and is not elaborated in this paper). This is guaranteed by the following condition: at the final converged point, the metamodel derivative exactly matches the real derivative.

$$\frac{\partial f_{\infty }^{meta}}{\partial g}|{}_{g_{\infty }}=\frac{\partial f^r}{\partial g}|{}_{g_{\infty }}$$

(19)

Therefore, a data-driven iterative model bias correction improves the solution optimality of the model-based design. By implementing the proposed enhanced control method, the real optimal solution can be achieved.

3.3. Jointly Model Parameter Estimation

A joint model parameter estimation is further proposed in addition to iterative model bias correction. The added value is that, upon convergence, the discrepancy between modeled and measured output is also reduced.

Since model parameters are also adjusted now, for the kth iteration, the model bias is formulated with the parameter ${\mathsf{\mu }}_k$ at the current iteration:

$$b_k=f_k^{mea}-f^{Eq}(g_k,{\mathsf{\mu }}_k)$$

(20)

The model bias update should simultaneously consider the impact of the parameter modification.

$$f^{meta}(g_{k+1},{\mathsf{\mu }}_{k+1})=f^{Eq}(g_{k+1},{\mathsf{\mu }}_{k+1})+b_k+(\frac{\partial b}{\partial g})|{}_{(g_k,{\mathsf{\mu }}_k)}\Delta g+(\frac{\partial b}{\partial \mathsf{\mu }})|{}_{(g_k,{\mathsf{\mu }}_k)}\Delta \mathsf{\mu }$$

(21)

This study focuses on invariant parameters, which means that in reality, the parameters are not varying; hence they are not affected by the signal control. In this regard, the signal control update will not affect the parameter update, hence the adjustment of $\mathsf{\mu }$ does not respond to signal changes. Substituting $\Delta \mathsf{\mu }={\mathsf{\mu }}_{k+1}-{\mathsf{\mu }}_k$ and $\partial f^r/\partial \mathsf{\mu }=0$ in Equation (21), it can be viewed as using a better model calibration with a modified parameter value to predict the model bias.

$$b_{k+1}=b_k^{\mathrm{mod}}+(\frac{\partial b}{\partial g})|{}_{(g_k,{\mathsf{\mu }}_{k+1})}(g_{k+1}-g_k)$$

(22)

in which $b_k^{\mathrm{mod}}=f_k^{mea}-f(g_k,{\mathsf{\mu }}_{k+1})$.

Remark 1.

The value of the model parameter does not affect the solution optimality of the iterative learning optimization problem. However, it affects the model bias. More accurate model parameters can reduce the discrepancy between model and real system. Hence, a better model parameter estimation is also achieved by jointly adjusting the parameter value during the iterative learning process.

Remark 2.

The jointly control scheme can be extended to include estimation of varying parameters. For instance, the parameters to be identified may change due to environmental conditions in reality and are sensitive to signal setting changes. In this circumstance, estimation is correlated with control optimization, and hence response of parameter estimation to control changes should be considered in control optimization as well.

We adopt a common approach for parameter estimation which follows a method of least squares error. By minimizing the discrepancy between the model output and the measurement with respect to the 2-norm, we can obtain an optimal parameter value ${\mathsf{\mu }}_{k+1}*$:

$${\mathsf{\mu }}_{k+1}*=\arg \underset{\mathsf{\mu }}{\min } {\Vert f_k^{mea}-f^{Eq}(g_k,\mathsf{\mu })\Vert }_2^2$$

(23)

$$s.t.f^{Eq}(g_k,\mathsf{\mu })\ge 0$$

(24)

$${\mathsf{\mu }}^L\le \mathsf{\mu }\le {\mathsf{\mu }}^U$$

(25)

Equation (23) represents a minimization criterion of vector 2-norm:

$${\Vert f_k^{mea}-f(g_k,\mathsf{\mu })\Vert }_2^2={\displaystyle \sum {(f_k^{mea}-f(g_k,\mathsf{\mu }))}^2}$$

Non-negativity and boundary constraints are included in (24) and (25) respectively.

The model bias is then calculated using the updated parameters. Based on the modified model bias correction, the control optimization problem (13)–(17) is solved. The control optimization can be reformulated as:

$$g_{k+1}*=\arg \underset{g}{\min } z(g,f_{k+1}^{meta}(g,{\mathsf{\mu }}_{k+1}))$$

(26)

$$s.t. f_{k+1}^{meta}(g,{\mathsf{\mu }}_{k+1})=f^{Eq}(g,{\mathsf{\mu }}_{k+1})+b_k^{\mathrm{mod}}+{\delta }_k(g-g_k)$$

(27)

$$f_{k+1}^{meta}(g,{\mathsf{\mu }}_{k+1})\ge 0$$

(28)

$$\chi (g)=0$$

(29)

$$g^L\le g\le g^U$$

(30)

4. Numerical Examples

4.1. Simulation Setting

In this section, we conduct a case study to validate the effectiveness of our proposed control method. In the numerical example, the ‘virtually’ real measurements are assumed to be obtained from the computer simulations. By the numerical tests, the purpose is to provide a conceptual proof of the proposed control method before its deployment using field data.

The enhanced anticipatory control scheme is tested on the network shown in Figure 2 as illustrated in [39]. There is one OD pair, i.e., node 1 to node 6 in Figure 2, nine links and five routes. The Bureau of Public Roads (BPR) function is used to calculate the link travel time [40]. For the demand side, the route choice is formulated by the Logit route choice model with the dispersion parameter. For the purpose of illustration and without loss of generality, we consider two control variables in this case study, i.e., two signalized intersections, which locate at node 4 and node 5. We further assume that the traffic signal timings operate in a two-phase signal plan. The signal green split is taken as the decision variable and the signal loss time is not considered in this case. Since green splits of the two phases (assumed as g(1) and g(2)) are dependent variables, i.e., g(1) + g(2) = 1, therefore, one independent decision variable of signal setting is defined for one signalized node. To this end, signal settings at nodes 4 and 5 are denoted as g¹ and g², respectively. A control optimization problem with two decision variables (g¹, g²) is studied in this numerical case.

This case study simulates the model-reality mismatch by applying different route choice models. In reality, it is assumed that travelers make route choice decisions following a Nested Logit (NL) structure [31]. The implication is that travelers have a different amount of information at different levels, i.e., the upper choice level located at node 1 and the lower choice level located at nodes 2 and 3. The probability of choosing route j can be written as:

$${\rho }^{real}[j]=\frac{\exp (-{\mathit{c}}_j\theta )}{{\displaystyle {\sum }_{i\in I_k}\exp (-{\mathit{c}}_i\theta )}}\times \frac{\exp (\zeta Y_k)}{{\displaystyle {\sum }_h\exp (\zeta Y_h)}}$$

in which $\mathit{c}$ is the Link travel time. Route choice set can be divided into subsets $I_1$,…, $I_k$,…, known as nests. The ratio of dispersion parameters ${\theta }_0$ and $\theta$ is denoted by $\zeta =\frac{{\theta }_0}{\theta }$, which reflect the features of the first and second choice level, respectively. $Y_k=\ln {\displaystyle {\sum }_{j\in I_k}\exp (-{\mathit{c}}_j\theta )}$ is the logsum variable.

In general, it is impossible to precisely model the real route choice response. It is assumed that we consider only one choice level for modeling the choice behavior. Hence, we adopt the multinomial logit (MNL) model with the dispersion parameter $\overline{\theta }$. The calculation of the model predicted probability, i.e., by the MNL model, is written as:

$$\rho [j]=\frac{1}{1+{\displaystyle {\sum }_{i\ne j}\exp [\overline{\theta }({\mathit{c}}_j-{\mathit{c}}_i)]}}$$

This case mainly focuses on demand side uncertainty, assuming the link travel cost is accurately modeled by the BPR function just as ‘reality’. The link cost is a function of link flow and signal setting.

$$\mathit{c}=\mathit{C}(f,g)={\mathit{c}}^0(1+\alpha {(\frac{f}{gs})}^{\beta })$$

in which ${\mathit{c}}^0$ is the free-flow travel time, $s$ is the saturation flow, g is the signal green split, $\alpha$ and $\beta$ are coefficients of the BPR function. For the non-signalized links, signal spits equal 1. For the test network, the total travel time $z$ can be also precisely formulated, which is derived as a function of equilibrium flows and signal settings.

$$z=z(g,f^{mea})={\displaystyle \sum _l\mathit{C}(f^{mea},g).f^{mea}}$$

An equilibrium flow model is calculated by the MNL structure. As such, the dispersion parameter $\overline{\theta }$, which is an important choice behavioral parameter capturing the route choice response, is taken as the model parameter to be estimated.

$$f^{Eq}=f(g,\overline{\theta })$$

Based on the equilibrium flow model, signal control decisions are to be optimized, and the objective is the minimization of the network’s total travel time. In this numerical example, all optimization problems are solved using the MATLAB optimization toolbox©.

Table 1. Characteristics of the example network.

OD Demand (veh/h)	3000
BPR Function Parameters	$\mathit{\alpha }$$=0.15, \mathit{\beta }=4$
		Saturation Flow (veh/h)	Free-Flow Travel Time (h)
	Link 1	1000	0.3
	Link 2	1000	0.1
	Link 3	1000	0.2
	Link 4	1200	0.3
	Link 5	1800	0.3
	Link 6	1800	0.2
	Link 7	1800	0.3
	Link 8	1800	0.3
	Link 9	2500	1.2
Dispersion parameters in reality	${\theta }_0$	0.8
	$\theta$	1.2

lists the characteristics of the test network.

Regarding the equilibrium flow model, a nominal value of $\overline{\theta }=10$ is taken for the dispersion parameter.

4.2. Simulation Results Analysis

We first compute the real optimal solution. The real optimal results are derived by solving the model-based anticipatory control optimization (4)–(8) using the exact NL model (i.e., the real route choice model). The results are listed in

Table 2. Real optimal solution, optimal solution obtained from the model prediction, and implement the model-based optimal solution in reality.

		Real Optimal Solution	Modeled Optimal Solution	Implement the Model Predicted Optimal Solution in Reality
Traffic signal green split		(g1, g2) = (0.44, 0.53)	(g1, g2) = (0.10, 0.90)
	Link 1	511	131	218
	Link 2	677	1394	929
	Link 3	654	1264	909
	Link 4	584	175	259
	Link 5	1188	1525	1147
	Link 6	1238	1439	1168
	Link 7	1165	1395	1128
	Link 8	1261	1569	1188
	Link 9	574	36	685
Total travel time (veh/h)		2724.1	2599.2	3066.0

. Note that the real optimum is used for the purpose of illustration, as well as for validating our proposed control method and solution algorithm. In actual applications, we cannot obtain the exact formula of the real system model, and hence we cannot derive the real optimum either. In view of the model inaccuracy, our proposed control method aims to drive the system towards the real optimal performance despite the model bias. This may be achieved by iteratively learning from the real system responses (through flow measurements). This small test network allows us to derive the real optimum and use it as benchmarking for validating our proposed solution method.

Then a traditional model-based traffic control without learning is applied. The MNL structure is used to model the route choice and the approximated equilibrium flow model is obtained. By solving the optimization problem (4)–(8) with the approximated equilibrium model, we derive the optimal signal splits (g¹, g²) = (0.10, 0.90). Table 2 also lists the results of the modeled flows and the total travel time. Due to the inaccurate equilibrium flow model, when implementing the optimal signal splits in the real-life traffic system, it obviously cannot achieve the real optimal performance, as indicated by the resulting link flows and total travel time in Table 2. The real control performance is significantly different from the model calculation. It is observed that there is an increase of 17.9% in total travel time in reality compared with what the controller calculates based on the inaccurate model. Moreover, compared with the real optimum, the traditional control method leads to an increase of 12.6% in total travel time.

Starting from the nominal optimal point of (g¹, g²) = (0.10, 0.90), the enhanced model-based control method generates convergent control to the real optimal point. Figure 3 illustrates the trajectory of the proposed control method over iterations. To validate the control performance, we depict the trajectory within the total travel time contours of the real control optimization problem, which indicates that the proposed control method is able to improve the control performance towards the real optimum. In addition to the optimal control trajectory, we also illustrate the convergence of the proposed method in terms of total travel time, as shown in Figure 4. The control method converges to the optimal solution in 16 iterations, i.e., after 16 epochs of traffic operations. Upon convergence, the optimal signal setting (g^1*, g^2*) = (0.45, 0.53) is obtained, with the corresponding total travel time of 2724.3 veh/h, which is quite closed to the real optimum.

Figure 5 compares the performance of two control methods, i.e., our proposed enhanced ATC with iterative learning and the traditional model-based control method. Generally, the structural modeling error is not addressed in the traditional control method. Moreover, to test the control performance under different traffic conditions (different congestion levels), we analyze the performance comparison under five demand levels, from 2000 veh/h to 4000 veh/h with an increase of 500 veh/h. Compared with the traditional model-based control method, the enhanced method is superior in improving the total travel time. As indicated by the comparison results, the enhanced ATC can improve the total time at least by 6.1% in this example. An improvement of 14.8% in total travel time is achieved for the case of demand = 2000 veh/h.

Regarding the traffic model parameter estimation, by solving the joint optimization problem, the optimal parameter value is obtained as ${\overline{\theta }}^{*}=0.388$. Figure 6 illustrates the convergence procedure of the model parameter estimation. Since the parameter estimation is performed locally at each signal control point, it determines a local optimal parameter at the optimal control point. Again, this example network allows us to enumerate the relation between the model parameter and the objective of the estimation problem. This helps to determine the global solution which can be used to validate the performance of the local solution. To validate the optimal model parameter, we calculate the objective function of the parameter estimation problem, i.e., the root squared error between measured and modeled flow, by enumerating the parameter values. Figure 7 demonstrates the summation of root squared errors under all feasible signal settings. The optimal parameter value is derived at ${\overline{\theta }}^{*}=0.44$. In this test example, the local optimal parameter estimation differs slightly from the global optimum. Intuitively, the optimality of the estimation solution can be improved by incorporating more measurement information under different signal settings.

Furthermore, we examine the impact of the initial conditions and step sizes K on the control trajectories. Sensitivity analysis is conducted with respect to initial points and step sizes. We compare three initial points: (g¹, g²) = (0.50, 0.50), (g¹, g²) = (0.20, 0.80) and (g¹, g²) = (0.70, 0.30). Figure 8 shows the evolution of total travel time. As indicated, the transient behavior of the convergence is affected by the initial settings, while in this test network, they both converge to the real optimum. For general applications, however, it is well recognized that because of the non-convexity due to the complexity involved in the equilibrium problem, local optimal solutions are generally obtained. We should take into account the role of initial settings in designing a successfully operating control scheme.

We further analyze the impact of the step size K. As mentioned in Section 3.2, the step size is a design parameter that affects the convergence and optimal performance of the control algorithm. We compare five step sizes, including four fixed values and a step size of K = 1/k, in which k is the number of iterations. As shown in Figure 9, the step size highly affects the control performance; different values result in different control trajectories and a larger step could lead to fluctuated procedures and hence non-convergent solutions. The method of successive averages with K = 1/k generally provides a stable control scheme, which prevents excessive changes to traffic signal settings especially when we obtain more information with the increase of iteration. Therefore, it is obvious that the step size is crucial in smoothing the control trajectory. Similarly, for the application of the control method, it is important to select a proper step size for designing a better operating control scheme.

5. Conclusions

Traffic signal control strategies have been developed over the past decades, with recent efforts using emerging techniques such as reinforcement learning [41] and metaheuristics [42]. However, the effective design of network signal control that takes account of travelers’ route choice response remains a challenge. This study proposes a metamodeling approach for an effective design of anticipatory traffic signal control, which is typically based on an equilibrium flow response function. It is known that inaccurate modeling usually contributes to sub-optimal performance when the model-based control system is implemented in reality. This paper addresses two sources of model inaccuracy: structural model bias and imperfect model parameter calibration.

First, a model-based anticipatory traffic control optimization problem is formulated. To address the inherent model-reality mismatch, a metamodel is then introduced as a surrogate of the unknown process model (the real system) through the analysis of model bias. The metamodeling framework integrates the model-based design with a data-driven iterative learning technique, and provides a closed-form formulation for theoretical analysis of solution optimality. A simple first-order approximation is adopted for iterative learning on the model bias correction. To further improve the control performance, a model parameter estimation is proposed jointly with the iterative model bias correction. In this study, a choice behavioral parameter estimation is performed. Hence, a better model calibration is achieved, which could be applied to other traffic management measures than signal control design, for instance, travel information or route guidance. Numerical examples confirm that enhancing the model-based anticipatory traffic control with iterative learning is able to elevate the control algorithm to the optimal solution that is associated with the real-life system. Compared with the traditional no-learning model-based control method, an improvement of 14.8% in total travel time is achieved. Furthermore, sensitivity analysis is performed to test the impact of initial points and step sizes on the control performance. For general applications, we should take into account the role of initial settings and step sizes in order to design a successfully operating control scheme.

This study focuses on deterministic model approximation errors and applies a linear function for model bias correction. Incorporating other random disturbances and investigating other metamodeling structures are worthy of further exploration. Moreover, we have focused on numerical experiments and the measurements are obtained from simulations. This is an important intermediary step of methodology development before its actual deployment. The practical implementation issues, for instance, processing of field data, and model validation, need further research efforts before linking to the eventual field deployment.