Transportation Asset Management Decision Support Tools: Computational Complexity, Transparency, and Realism ^†

Abstract

Asset management decision support tools determine which action (maintenance, rehabilitation, or reconstruction) is applied to each facility in a transportation network and when. Sophisticated tools recognize uncertainties and consider emerging priorities. However, these tools are often computationally complex and lack transparency, the models are difficult to evaluate, and the outputs are challenging to validate. This paper explores computational complexity, transparency, and realism in transportation asset management decision support tools to better understand how to select the right tools for a particular context. Descriptions of how state departments of transportation in the United States make use of optimization in their mandated transportation asset management plans to make decisions are used to understand the needs of states. This qualitative analysis serves as a review of the goals and practices of state agencies. An existing asset management tool is then used to demonstrate the tradeoffs involved in accurately capturing the decision-making process and complexity. The results provide examples of strategies that agencies can use when selecting decision support tools and for researchers and tool developers working toward developing the right tool for an application.

1. Introduction

Asset management decision support tools are used by transportation agencies to determine which action (maintenance, rehabilitation, or reconstruction) is applied to each facility in a transportation network and when. These decisions are made considering the existing and predicted condition and performance, asset life cycle costs, and user costs. Sophisticated tools recognize uncertainties and consider emerging priorities, such as resilience and sustainability. However, these tools are often computationally complex and lack transparency, the models are difficult to evaluate, and the outputs are challenging to validate. As a result, true optimization tools are not widely used by state agencies in the United States.

The objective of this paper is to better understand how to select or develop the right tools for a particular context. To address this objective, three questions are posed: (1) How are different tools characterized? (2) How are different contexts characterized? (3) What criteria can be used to select the right tool? In answering these questions, this paper explores computational complexity, transparency, and realism in asset management decision support tools. The analysis and results are intended to provide examples of strategies that agencies can use when selecting decision support tools.

This paper reviews different types of decision support tools (ranking, prioritization, thresholds, and optimization), how tools are set up, alternative solution methods, and the different contexts for decision making. Based on a review of the transportation asset management plans that are required by state departments of transportation in the United States, the goals and needs of state agencies in making decisions are summarized. Using a multi-asset roadway improvement scheduling tool [1] as a case study and a series of computational experiments, complexity, transparency, and realism are explored. To accomplish this, the analysis compares the computational burdens, the parameters involved, and the range of outcomes for different scenarios.

The results demonstrate four issues in the selection of decision support tools in the context of state agencies in the United States. The first issue is computational burden: running the decision support tool for a simple network requires several hours. The second issue is the sensitivity of the results to the input parameters. The results help to validate the outputs and show the relative importance of different parameters. The third issue is the differences between simple and complex decision support tools and generalizing the circumstances in which to use one versus another. Some simple heuristics for selecting tools are identified. The fourth issue is the validation of the results. Strategies for qualitative validation are explored.

2. Materials and Methods

2.1. Background

Transportation asset management involves the systematic process of managing transportation assets, such as roads, bridges, and other infrastructure elements, to provide efficient and reliable transportation services. In recent years, optimization techniques have emerged as valuable tools in developing effective asset management plans. Optimization allows transportation agencies to make informed decisions regarding resource allocation, maintenance schedules, and infrastructure investments.

Despite the numerous benefits, incorporating optimization techniques into transportation asset management plans presents challenges that need to be addressed. One challenge is the availability and quality of data. Optimization models heavily rely on accurate and up-to-date data on asset conditions, performance, and costs. However, many transportation agencies struggle to collect and maintain comprehensive data sets. Efforts must be made to improve data collection methods and establish reliable data management systems to effectively support optimization analyses.

Another challenge addressed in this paper is the complexity and variability of transportation systems. Transportation networks are intricate, comprising various asset types, geographical areas, and user demands. Developing optimization models that capture the diverse characteristics of transportation systems can be challenging. Models must consider different asset types, their interactions, and the potential impacts of interventions on overall network performance. Collaboration among different stakeholders, including transportation planners, engineers, and policymakers, is crucial to ensure that optimization models account for the complexities of real-world transportation systems.

Asset management processes were adopted by state departments of transportation in the United States in response to aging infrastructure, traffic growth, unanticipated damage to infrastructure assets due to natural hazards, higher performance expectations of users, declining funding bases, and legislative requirements. Decision support tools are a key element of asset management.

The simplest tools rank and prioritize activities [2]. More sophisticated and complex asset management tools use optimization to make decisions that select and prioritize activities. Formulations, which define the decision variables, objective function, and constraints, are classified as project or network level, single or multiple asset types, single objective or multi-objective, and deterministic or stochastic [3]. Objective functions often minimize cost by consisting of different components of cost, including agency cost, user costs, disruption costs, and external costs. Another classification also considers whether the decision variables are discrete or continuous and whether uncertainty is considered or not. Optimization problems can also be classed as activity selection, scheduling, or both selection and scheduling. Some seek to identify policies. Invariably, optimization problems focus on a discrete set of locations and activities, a finite period (planning horizon), the condition of the asset, and the costs including agency and user costs. To simplify the solution process or reduce the size of the solution space, assumptions are usually made. Solution methods range from the enumeration of alternatives through heuristics, simulation, algorithmic solutions, transformation, and combinations of these methods. Most recently, many tools use artificial intelligence including machine learning (for example, [4,5,6]).

There is a large body of the literature on decision support tools for asset management. Building from early work in pavement management, the seminal work of Golabi, Kulkarni, and Way [7] that recognized deterioration and uncertainty and then extended the work to bridges is a foundation for advanced and more sophisticated optimization of maintenance and resurfacing decisions for pavements. Reviews of the state of the art provide context. Chen and Bai [3] review over 300 papers on optimization in asset management. Chen et al. [8] provide a review of optimization in transportation asset management for roads and bridges. Chen et al. [9] focus on multi-objective optimization for maintenance decisions. Other papers address the changing needs for asset management tools that embrace resilience, sustainability, and uncertainty [10,11,12]. Together, these papers provide a clear picture of the variety of approaches to the problem formulation and solution methods, both of which are tailored to a particular application.

Furthermore, the details of the formulation being solved vary with the context. This context can be characterized in terms of the objectives being considered, the important of network versus project solutions, the time frame being considered, and the jurisdictions involved in the solution.

Table 1. Attributes describing context.

Attribute	Description	Examples
Cost components	Agency (maintenance, repair, and rehabilitation) User (travel time and vehicle operating costs) Disruption (delays due to maintenance, repair, and rehabilitation) External (environmental impacts and disruption to non-users)	DCMAC 1 [1]; MOO 2 [9] DCMAC [1]; MOO [9] DCMAC [1] MODAT 3 [13]
Other objectives (in addition to minimizing cost)	Functionality (safety and comfort) Condition (distress and integrity) Structural (remaining life)Equity Resilience (ability to withstand and recover from external events) Sustainability (economic, social, and environmental)	[8] * [8] * [8] * [14] [15] * MODAT [13]
Spatial representation	Project (specific location) Network (interconnected assets) Corridor (parallel assets serving an origin and destination)	PAVER [16] DCMAC [1]; BrM 4 [17] MCDM 5 [18]
Assets	Network characteristics (redundancy and connectedness, congested/uncongested, and overall condition) Single modes/multi-modal (motorized transportation of people and goods, transit, rail, pedestrians, and bicycles) Single type/multiple types (roads, bridges, and ancillary assets)	Network size [19] BrM [17]; scenario analysis of auto, bike and ped [20] DCMAC [1]; roads and bridges [8]; MODAT [13]; cross-asset tradeoffs [21]
Time frame	Planning horizon (decisions for a prespecified planning horizon) Life cycle (decisions over the life cycle of an asset)	DCMAC [1] [22] *
Jurisdiction or geographical area	Local (typically a city or town), county (often includes both urban and rural areas), or regional (for example, an area designated as a Metropolitan Planning Organization) State (commonly used in the United States, as states are responsible for most roads) Federal (with designations by subsystems such as the National Highway System) Combinations	[23] * BrM [16] HERS 6 [24]

* Review articles; acronyms: ¹ DCMAC—Deep Centralized Multi-Agent Actor Critic; ² MOO—multi-objective optimization; ³ MODAT—Multi-Objective Decision Analysis Tool; ⁴ BrM—bridge management; ⁵ MCDM—Multi-Criteria Decision Making; ⁶ HERS—Highway Economic Requirements System.

presents one approach to characterizing these contexts. Table 1 shows attributes, a description of the attributes, and examples from the literature of where these attributes were used or applied in that context. The review articles are designated with “*”. These are in addition to the already-cited review articles.

In this paper, we focus on understanding issues related to the computational complexity, transparency, and realism of tools.

2.2. Overview

To assess computational complexity, transparency, and realism, this paper uses both quantitative and qualitative methods. This paper first reviews the goals and practices of state agencies related to asset management decision making. The review is based on the transportation asset management plans submitted by each state department of transportation in 2019 and 2023 to the Federal Highway Administration, as required by the Moving Ahead for Progress in the 21st Century Act (MAP-21) [25].

A series of computational experiments are then conducted using a multi-asset roadway improvement scheduling tool [1] as a case study. The analysis compares the computational burdens, the parameters involved, and the range of outcomes for different scenarios.

2.3. Review of Transportation Asset Management Plans

In the United States, departments of transportation are required to submit transportation asset management plans to the Federal Highway Administration. Guidance for the development and certification of TAMPs was published in the Federal Register [26]. Every four years, FHWA is required to certify a state DOT’s processes for developing its TAMP. Every year, FHWA is required to complete a consistency determination.

The certification focuses on the applicable requirements for TAMPs. Specifically, a state DOT’s TAMP development processes must include (1) performance gap analysis and strategies to close gaps; (2) lifecycle planning; (3) risk analysis and a risk management plan; (4) financial plan covering at least a 10-year period; (5) investment strategies; (6) obtaining necessary data from NHS owners other than the state DOT; and (7) using the best available data and bridge and pavement management systems to analyze NHS bridge and pavement conditions.

Consistency determination focuses on the plan developed using the certified processes and its implementation. Evidence of implementation is described as actual investment levels using the work types of initial construction, maintenance, preservation, rehabilitation, and reconstruction that are close to planned investment levels for each work type. Other evidence or explanations of deviations are considered.

Penalties for non-compliance either reduce the federal share of Nation Highway Performance Program (NHPP) project or FHWA would not approve any further project involving NHPP funds.

To support TAMP processes, plan development, and consistency determination, the Federal Highway Administration developed guidance documents including transportation asset management plan templates [27]; Using a Lifecycle Planning Process to Support Asset Management [28]; Developing TAMP Financial Plans [29]; Incorporating Risk Management into Transportation Asset Management Plans [30]; and Transportation Asset Management Plan Annual Consistency Determination Final Guidance [31].

Plans are downloadable from the American Association of State Highway and Transportation Official (AASHTO) Asset Management Portal (https://www.tam-portal.com/ accessed on 9 October 2023). Plans from 2019 and 2022 are reviewed to explore how states are implementing their investment decision-making processes. Consistency determination also provides some insight into the validation process. While consistency determination reports are not widely available, discussion between peer exchanges and state DOTs is used to provide some insights into the decision-making processes for NHS pavements and bridges [32].

Plans from 2019 and 2022 are reviewed to understand how states are using optimization in their asset management decision-making process. The 16 plans from 2019 (Alabama, Alaska, California, Connecticut, Delaware, District of Columbia, Hawaii, Minnesota, New Hampshire, North Carolina, Pennsylvania, Rhode Island, South Carolina, Virginia, Washington, and West Virginia) are selected because they included “optimization” or “optimal” in their goals or objectives. The 32 plans (Alaska, Arkansas, California, Colorado, Connecticut, Delaware, District of Columbia, Georgia, Hawaii, Illinois, Indiana, Iowa, Michigan, Mississippi, Montana, New Hampshire, New Jersey, New Mexico, North Carolina, North Dakota, Ohio, Oregon, Pennsylvania, Rhode Island, South Dakota, Tennessee, Texas, Utah, Vermont, Virginia, Washington, and Wisconsin) from 2022 represent all the plans posted to the AASHTO TAM Portal as of July 2023.

2.4. Case Study

There are three elements to the case study: (1) the tool, (2) the application network, and (3) a series of computational experiments using the tool and application to explore the challenges with implementation and the attributes of the solution. Each element is briefly described.

The tool [1] was developed to fill gaps in the existing methodologies reported in the literature by including the disruption caused by maintenance and rehabilitation activities, along with stochastic deterioration models, and network models that include multiple classes of assets (pavements and bridge). The tool is structured as a bi-level optimization program that prioritizes and schedules roadway improvement activities, recognizing users’ costs and disruption. The upper level of the program minimizes improvement costs and traffic delays across the network for all years in the planning horizon. Inputs are the condition states, actions, Markov transition probabilities, and costs for all pavements and bridges in the network. The implementation involves a Markov decision process (MDP) to identify and prioritize potential roadway improvement actions. The lower level seeks to determine traffic flows based on a network user equilibrium solution across paths that is affected by capacities determined through actions determined at the upper level. Inputs are the network connectivity, travel time functions for each link, and demand. The solution of each level is repeated until the results converge. The problem is solved using a reinforcement learning method referred to as a Deep Centralized Multi-Agent Actor Critic (DCMAC) method. Zhou [1] provides background on this machine learning method and its advantages. Alternative solution methods are also documented by Zhou [1]. For comparison purposes, the tool also includes an enumeration of outcomes using thresholds to determine optimal thresholds and solutions excluding traffic. Thresholds are commonly used by agencies as simple rule-based decisions.

A hypothetical network drawn from the literature [33] and modified by [1] is used to illustrate the concepts. The code functions as a black box with hard-coded inputs and parameters. Working with the code provides us with some insights into the challenges that state DOTs face in adopting tools that were developed by researchers. Our computational experiments focus on the exploration of opportunities to reduce complexity, add transparency, and assess how realistic the tool is.

Two types of computational experiments are conducted. The first set of experiments center around local and global sensitivity analysis. Local sensitivity methods, commonly used in transportation and asset management, quantitatively analyze the changes in model outputs due to small variations in input around a fixed value in such a way that each variable is varied one by one, while setting others to some fixed values. Global sensitivity analysis methods, which are powerful-but-less-common methods, quantitatively analyze the variability of the model outputs to variations in the inputs by varying all the inputs over the entire ranges [34]. We use these methods to check the logic (an important step in validation) and explore the relative importance of the parameters. The second set of experiments explores an alternative solution method using the simple threshold with different levels of traffic. We hypothesize that this method gives close-to-optimal results at low traffic volumes and is computationally more efficient, simpler, and more transparent than the case study tool.

3. Results

3.1. State Perspectives

A review of 16 state DOTs’ transportation asset management plans from 2019 revealed that most states aim to optimize their investments but do not optimize in the mathematical sense of the word [35]. At best, the states optimize investment in their bridge program or pavement program. Most conduct scenario analysis and explore alternative strategies. However, the TAMPs do recognize the value of optimization, the potential gains, and the importance of good models and reliable data.

A similar analysis of 32 2022 TAMPs was conducted. This analysis suggested that the 2022 TAMPs include more specificity with respect to optimization but still have a heavy reliance on optimization within the pavement management systems and bridge management systems.

Table 2. Analysis of plan content.

Year	2019	2022
Number of plans reviewed	16	32
Plans including optimization as a goal or objective (%)	100	71.9
Plans using optimization for pavements (%)	31.3	81.3
Plans using optimization for bridges (%)	12.5	53.1

is a quantitative analysis of the content analysis. While the 2019 plans were selected because they include optimization as a goal or objective, the analysis of the 2022 plans shows that, in general, these plans include more details in terms of the optimization of pavements and bridges. Of the 2022 plans, 50% of the reviewed TAMPs discuss optimization in the context of both pavements and bridges. However, in several states, the level of sophistication of the analysis presented in the plans differs between pavements and bridges.

Furthermore, some plans recognize the need for integrating the different types of analysis (lifecycle planning, investment, and risk). For example, the Connecticut DOT’s (CTDOT) mission states [36] that “CTDOT uses a risk-based, data-driven process to maximize transportation performance and user experience, to prioritize resources, and to optimize treatments and costs over the life cycle of an asset for the state’s multimodal transportation system”. Likewise, the New Jersey DOT’s plan integrates optimization with the plan’s elements but explicitly uses its pavement and bridge management systems to make the connection [37]: “The NJDOT pavement and bridge management systems use analytical software to support life cycle planning to develop investment strategies. The investment strategies optimize planned funding and forecast expenditures to make progress toward the TAMP state of good repair objectives at the lowest life cycle cost, while incorporating the consideration of risk”.

A small number of states provide detailed descriptions of objectives, constraints, and solution methods, but most reference commercially available software such as dTIMs, AASHTOWare Bridge Management (BrM), AgileAssets, AssetWorks, and MODAT.

Overall, the states are aiming to develop optimal plans that deliver the best serviceability given the budget constraints. Invariably, the implementations focus on independently reached optimization decisions for pavements and bridges based on a predefined set of scenarios. Essentially, the objective function is computed for each scenario that meets the constraints and the chosen “optimal” solution. Given the fact that there are many tradeoffs in terms of actions, timing, and location, it is possible for an optimal solution to be overlooked. However, given that scenarios are developed based on experience and data, the selected optimal solution is likely to be very desirable and, for the given problem and objective, either optimal or near optimal.

Thresholds for determining when to undertake a maintenance or improvement activity, decision trees, a simulation, and a scenario analysis are widely used in practice. While the strategies proved to be effective for pavements and bridges independently, the need to consider cross-asset tradeoff and integrate more complex objectives, such as users’ costs, disruption, and sustainability, adds to the complexity. In contrast, “black box syndrome” means that agencies are skeptical of outputs from elaborate mathematical models. Wang and Pyle [38] recommend engaging users, verififying results, and continuing validation.

Another important gap in developing and implementing optimal decisions is the difficulty in assembling the required data. This is important because data collection is costly, and resources for maintaining and improving roads are scarce. Taking advantage of innovative data collection methods as well as more accurate and more timely data and making better use of resources are important. Although agencies may not implement an “optimal” solution, exploring alternative solution methods provides insight into the factors that influence these decisions and ultimately helps agencies to deliver better transportation services.

In some sense, the consistency determination is a high-level validation of the decisions made in the TAMPs. In addition to the certifying process-related elements of a TAMP, the annual consistency determination answers the following question [31]:

Has the State DOT documented evidence that the State DOT is using the TAMP investment strategies? …. The best evidence is that, for the 12 months preceding the consistency determination, there was alignment between the actual and planned levels of investment (in the TAMP) for various work types as defined in 23 CFR 515.5 (i.e., initial construction, maintenance, preservation, rehabilitation, and reconstruction)?

States are also provided with the opportunity to document why deviations occur (for example, the allocation of special funds and natural disasters). Most importantly, this process encourages states to review past decisions and consider why the models and processes do not reflect actual spending. Despite some challenges due to mismatches between reporting requirements and state fiscal years as well as the use of categories that do not match current accounting practices, the consistency determination appears to raise awareness, as evidenced in the peer exchanges conducted in 2021 and 2022 [32].

3.2. Case Study

Using a simple network consisting of 10 nodes and 11 links with some redundancy, a case study is used to demonstrate the computational complexity of the problem and the sensitivity to input parameters, to compare the solution using simple thresholding, and to briefly discuss the challenges involved in validating the results.

3.2.1. Computational Complexity

A single run of the simple case study on a Windows computer (with a 16 GB RAM CPU and a 2.20 GHz Intel Core i5 processor) takes approximately 3 h. While the network structure is simple, the bi-level optimization program (described in Section 2.4) requires a solution from thousands of problems for training, with each problem representing a realization of the transition probabilities and the solution requiring a network equilibrium. Once the DCMAC is trained, the solution for that network at any point in time is very efficient. However, any changes to inputs other than the condition state or changes to any parameters require training the solution. The exploration of alternative inputs adds transparency. Therefore, the computational complexity reduces transparency. Alternatives include using parallel and high-performance computing to complete multiple runs and explore multiple solution methods or to explore options for simplifying the code or the decision process.

In the next subsection, we explore options for simplifying the code using sensitivity analysis to identify parameters that may not be needed and more important parameters. The sensitivity analysis also plays a critical role in validation.

In the subsequent subsection, we explore alternative solution methods, specifically thresholds and decision trees, to understand the contexts in which these methods are more appropriate.

3.2.2. Sensitivity Analysis

The case study model requires seven input parameters, namely, deterioration rate, user cost rate, maintenance cost rate, discount factor, traffic factor, pavement inspection accuracy, and bridge inspection accuracy, in addition to some hard-coded variables, to solve the optimization problem. The model generates four cost components over the planning horizon, namely, maintenance cost, user cost, traffic delay cost, and total cost.

The local sensitivity analysis examines the contributions of the input parameters to the variations in the output (the total cost, for simplicity) [39]. For two of the parameters, namely, user cost rate and maintenance cost rate, the initial results are illogical in terms of the relationship between the output (total cost over the planning horizon) and the parameters values. The code is modified, and the analysis rerun shows results that are consistent with the expectations. These results underscore the importance of conducting a local sensitivity analysis when using complex code.

The global sensitivity analysis based on 1024 simulations reveals that some parameters contribute little to the variability of the outputs [39]. Both bridge and pavement inspection accuracy have little impact on the total cost over the planning horizon. Either these parameters can be hard-coded or alternative representations of the inspection accuracy should be considered. In contrast, the discount rate and traffic are most influential, so care must be exercised in choosing the appropriate values.

Given that the model is a black box and lacks transparency, sensitivity is particularly important if users are to understand the relative importance of the parameters and other inputs as well as the parameters’ relationship to the inputs. Being able to explore and see whether the changes in parameters and other inputs result in changes in the outputs or decision (either in terms of the absolute or relative value) helps to build confidence in the model.

3.2.3. Alternative Solution Methods

Zhou et al. [1] compared the results using the DCMAC with seven different methods. The alternative methods were characterized by the inclusion or exclusion of traffic (reducing the complexity if traffic is excluded). Zhou et al. found an alternative machine learning method to be inferior, both with and without traffic, so only five methods are discussed here. The five methods are optimal thresholding (using complete enumeration) both with and without traffic, the DCMAC excluding traffic, and two methods treating pavements and bridges independently (one separately training each asset class and the other sequentially training the asset classes) with traffic. These results are summarized in

Table 3. Results using alternative solution methods (modified from [1]).

Traffic	Solution Method	Costs			Difference with Optimal (%)
		Agency and User	Traffic Delay	Total	Agency and User	Traffic Delay	Total
Excluded (single-level)	Optimal threshold (complete enumeration)	9.58	32.02	41.6	−28.8%	41.9%	15.5%
	Optimal prioritization via DCMAC	9.51	30.56	40.07	−29.3%	35.4%	11.2%
Included (bi-level)	Optimal threshold (complete enumeration)	12.23	24.92	37.15	−9.1%	10.4%	3.1%
	Optimal prioritization via DCMAC	13.45	22.57	36.02	0.0%	0.0%	0.0%
Included (bi-level)	Single-asset management (separately trained by asset class)	13.55	23.73	37.28	0.7%	5.1%	3.5%
	Single-asset management (sequentially trained)	13.91	23.94	37.85	3.4%	6.1%	5.1%

. While the results are more efficient using the optimization methods, the differences in total costs are modest, suggesting that complex tools are not needed for uncongested networks or a network with a high degree of redundancy. However, the costs do transfer from the agency to the users, raising equity concerns.

To explore this option, the plan is to solve the problem using different levels of demand using the DCMAC (including traffic) and optimal thresholds (excluding traffic) and then compare the total cost over the planning horizon for the two methods. The case study network is modestly congested, so it is anticipated that the two methods will yield similar results for low traffic volumes, and, as the traffic delays will be modest, equity is not expected to be an issue. Figure 1 presents a hypothetical solution. In addition, we expect the computational effort using the optimal thresholds to be significantly less than that using the DCMAC. These and other experiments will be conducted when the code becomes accessible.

3.2.4. Validation

Validation is also challenging. The results are optimal given the inputs. However, other parameters or omitted variables may influence the solution. The most common assessment is based on logic: the outputs reflect the appropriate order of magnitude, and changes in the outputs reflect changes in the inputs in the right direction. Our sensitivity analysis demonstrates the importance of this analysis to understand any errors in logic. However, a true test needs to build on the concepts in the consistency evaluation and ask the question:

Has the organization entity documented evidence that the entity is using the investment strategies recommended by the asset management tool? Is the best evidence that, for the specified period of time (say five years), there was alignment between the actual and planned levels of investment (based on the asset management tool) for various asset types and work types? And do the actual and predicted conditions of the assets align?

These questions go beyond the consistency determination. They recognize the need for a longer period of time, given that the asset condition often lags behind the expenditure or the lack of expenditure, and there is the importance of comparing the actual expenditures and conditions with the planned expenditures and predicted asset conditions to validate model outputs and outcomes.

4. Discussion

The background information includes documentation of the different types of tools and how they are characterized and the different contexts in which tools are used. The qualitative and quantitative analyses presented in this paper provide some insights into the experiences of state DOTs with and the needs of state DOTs for tools to support their asset management processes and demonstrate the challenges in selecting the appropriate tools. Theses analyses provide insights into the issues related to complexity, transparency, and realism.

First, complexity is considered. The fact that many tools function as a black box mean that it is difficult to modify the tools for a particular context and that the interactions among the parameters and inputs are poorly understood. Furthermore, complexity is often associated with computational burden. If the tool is infrequently needed, then this is less about limitation. However, if frequent runs are required, is there then a tradeoff between complexity and accuracy?

Second, transparency is closely linked to complexity. Black boxes have no transparency. However, logical evaluation, sensitivity analysis, and the documentation of assumptions can help provide a peek into a complex tool. Such analysis helps to build confidence in the results from the tool confronting some of the concerns raised in the literature [37]. However, given the dominance of simple decision-making tools in state DOTs, as evidenced by the qualitative analysis of TAMPs, tool developers need to integrate strategies to provide some of this required transparency into asset management tools. For example, sensitivity analysis (both local and global) with enhanced visualization and graphics can provide insights into the relationships among the inputs and decisions and present opportunities to explore the impact of the changes.

Finally, realism is even more challenging, as validation is simply not feasible. There are two opportunities for considering how realistic the models are. First, users must ask questions regarding how insightful or useful the results are. For example, even if a decision that is the output of a complex optimization model is not implemented, analysis may help to identify unacceptable decision and actions that are not modeled. Second, in situations where decision support tools are used, a retroactive analysis of the actual costs and conditions over the planning horizon is valuable. This goes far beyond the consistency determination, as this type of analysis requires tracking the actions, conditions, and costs.

Based on the consideration of complexity, transparency, and realism, the possible actions are to simplify codes, if appropriate; document and communicate the process, algorithms, variables, and parameters so that they are appropriately used; and evaluate, calibrate, and assess all available tools. The criteria used to select a tool from a set of available options include (1) the method matched to the context, that is, the tool demonstrates the appropriate level of complexity; (2) the model is understood, that is, there is transparency; and (3) the results are useful and insightful, that is, the model demonstrates an appropriate level of realism.

In summary, the results demonstrate four issues in the selection of decision support tools in the context of state agencies in the United States. The first issue is the computational burden: running a decision support tool for a simple network requires several hours. The second issue is the sensitivity of the results to the input parameters. The results show the relative importance of the different parameters. The third issue is the differences between simple and complex decision support tools and generalizing the circumstances in which to use one versus another. Some simple heuristics for selecting tools are identified. The fourth issue is the validation of the results. Strategies for qualitative validation are explored.

5. Conclusions

This paper explores the opportunities and challenges involved in using sophisticated optimization tools to support transportation asset management. A review of state TAMPs indicates that most states aim to optimize the use of resources, but the tools used and the levels of integration significantly vary state to state. Furthermore, many of the tools simply optimize the scenarios or options based on decision rules. A few states are using multi-objective optimization or more elaborate scoring methods, but discussions with states suggest that states are reluctant to adopt complex tools that are black boxes. This paper specifically looks at complexity, transparency, and realism.

Using a case study and the DCMAC tool, the analysis suggests that the local and global sensitivity analysis provides insight into model behavior and direction for agencies when selecting parameters for models and that complex decision support tools are not always warranted. The results also serve as a reminder to researchers and tool developers of issues that must be considered when developing the right tool for an application. Further research can provide a more specific direction.

Our exploratory research suggests that the selection of an appropriate decision support tool for transportation asset management requires matching the tool to the context, avoiding unnecessary complexity; providing as much transparency as feasible through documentation and local and global sensitivity analyses to understand the relationships among input parameters and variables and model outputs; and exploring the realism of tools to capture the context and provide meaningful results. This work is a preliminary exploration of how the tool selection process might be structured.

Additional research opportunities include exploring alternative network configurations and scales, asset attributes (such as the initial condition and capacity) and levels of congestion, and alternative solution methods, such as decision trees. The development of guidelines for state DOTs to help them select the right tool given their context would also be valuable. Also, a more detailed analysis of data from the annual consistency determination of state TAMPs paired with performance data is likely to provide further insights into strategies to support validation.