A Methodology for Predicting Ground Delay Program Incidence through Machine Learning
Abstract
:1. Introduction
2. Description of Ground Delay Programs
3. Methods
3.1. Data Set
3.2. ATMAP Algorithm
- Parsing METAR messages and extracting the 5 elements: visibility, wind, as well as precipitation, freezing, and hazardous weather phenomena;
- An indicator of the weather class status is expressed as a severity code, which ranges from 1 for good weather to a maximum of 5 for bad weather;
- For each specified severity level code, discrete values (0 to 30) called synoptic score coefficients are given a value. This enables the description of the non-linear behavior of several weather events. Examples of ATMAP weather algorithms for scoring precipitation phenomena are given in Table 2.
3.3. GDP Classification Models
3.3.1. Evaluation Indicators
3.3.2. Support Vector Machine
3.3.3. Random Forest
3.3.4. XGBoost Classification Algorithm
3.4. GDP Departure Delay Time Models
4. Results and Discussion
4.1. ATMAP Score Analysis
4.2. GDP Classification Models
4.2.1. Parameter Selection
- Establish the model function, determine the parameters of the model, and set the hyperparameter interval to be optimized;
- Bayesian optimization is performed, using the model as the objective function for optimization and the AUC value as the evaluation function to maximize it;
- The model is trained with a 5-fold cross-validation method under the current combination of parameters, and the value of the model evaluation function is calculated under the current parameters. After completion, the model is returned to Bayesian optimization and the next set of parameters is selected for a new round of training based on the probabilistic model acquisition function until the number of iterations is reached;
- Output the optimal set of parameters for model performance after Bayesian optimization.
4.2.2. Model Evaluation
4.3. Departure Delay Time Models
4.3.1. Performance Measures
4.3.2. Example Validation
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Civil Aviation Administration of China. Development Statistics Bulletin of Civil Aviation Industry in 2018; Civil Aviation Administration of China: Beijing, China, 2019. [Google Scholar]
- Grabbe, S.; Sridhar, B.; Mukherjee, A. Clustering Days and Hours with Similar Airport Traffic and Weather Conditions. J. Aerosp. Inf. Syst. 2014, 11, 751–763. [Google Scholar] [CrossRef]
- Liu, Y.; Hansen, M.; Zhang, D.; Liu, Y. Modeling Ground Delay Program Incidence Using Convective and Local Weather Information. In Proceedings of the Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, USA, 2 May 2020; pp. 1–8. [Google Scholar]
- Ball, M.O.; Hoffman, R.; Odoni, A.R.; Rifkin, R. A Stochastic Integer Program with Dual Network Structure and Its Application to the Ground-Holding Problem. Oper. Res. 2003, 51, 167–171. [Google Scholar] [CrossRef]
- Mukherjee, A.; Hansen, M. A Dynamic Stochastic Model for the Single Airport Ground Holding Problem. Transp. Sci. 2007, 41, 444–456. [Google Scholar] [CrossRef]
- Kuhn, K.D. Ground Delay Program Planning: Delay, Equity, and Computational Complexity. Transp. Res. Part C Emerg. Technol. 2013, 35, 193–203. [Google Scholar] [CrossRef]
- Smith, D.A.; Sherry, L. Decision Support Tool for Predicting Aircraft Arrival Rates from Weather Forecasts. In Proceedings of the 2008 Integrated Communications, Navigation and Surveillance Conference, Bethesda, MD, USA, 5–7 May 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 1–12. [Google Scholar]
- Chen, Y.; Xu, Y.; Hu, M.; Yang, L. Demand and Capacity Balancing Technology Based on Multi-Agent Reinforcement Learning. In Proceedings of the 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC), San Antonio, TX, USA, 3–7 October 2021; pp. 1–9. [Google Scholar]
- Wang, Y.; Kulkarni, D. Modeling Weather Impact on Ground Delay Programs. SAE Int. J. Aerosp. 2011, 4, 1207–1215. [Google Scholar] [CrossRef]
- Vossen, T.W.M.; Ball, M.O. Slot Trading Opportunities in Collaborative Ground Delay Programs. Transp. Sci. 2006, 40, 29–43. [Google Scholar] [CrossRef]
- Ball, M.O.; Hoffman, R.; Mukherjee, A. Ground Delay Program Planning Under Uncertainty Based on the Ration-by-Distance Principle. Transp. Sci. 2010, 44, 1–14. [Google Scholar] [CrossRef]
- Yan, C.; Vaze, V.; Barnhart, C. Airline-Driven Ground Delay Programs: A Benefits Assessment. Transp. Res. Part C Emerg. Technol. 2018, 89, 268–288. [Google Scholar] [CrossRef]
- Jacquillat, A. Predictive and Prescriptive Analytics Toward Passenger-Centric Ground Delay Programs. Transp. Sci. 2022, 56, 265–298. [Google Scholar] [CrossRef]
- Ikram, R.M.A.; Goliatt, L.; Kisi, O.; Trajkovic, S.; Shahid, S. Covariance Matrix Adaptation Evolution Strategy for Improving Machine Learning Approaches in Streamflow Prediction. Mathematics 2022, 10, 2971. [Google Scholar] [CrossRef]
- Kalita, J.; Balas, V.E.; Borah, S.; Pradhan, R. (Eds.) Recent Developments in Machine Learning and Data Analytics: IC3 2018; Advances in Intelligent Systems and Computing; Springer Singapore: Singapore, 2019; Volume 740, ISBN 9789811312793. [Google Scholar]
- Grabbe, S.R.; Sridhar, B.; Mukherjee, A. Clustering Days with Similar Airport Weather Conditions. In Proceedings of the 14th AIAA Aviation Technology, Integration, and Operations Conference; American Institute of Aeronautics and Astronautics, Atlanta, GA, USA, 16 June 2014. [Google Scholar]
- Liu, Y.; Liu, Y.; Hansen, M.; Pozdnukhov, A.; Zhang, D. Using Machine Learning to Analyze Air Traffic Management Actions: Ground Delay Program Case Study. Transp. Res. Part E Logist. Transp. Rev. 2019, 131, 80–95. [Google Scholar] [CrossRef]
- Klein, A.; Craun, C.; Lee, R.S. Airport Delay Prediction Using Weather-Impacted Traffic Index (WITI) Model. In Proceedings of the 29th Digital Avionics Systems Conference, Salt Lake City, UT, USA, 3–7 October 2010; pp. 2.B.1-1–2.B.1-13. [Google Scholar]
- Liu, J.; Li, K.; Yin, M.; Zhu, X.; Han, K. Optimizing Key Parameters of Ground Delay Program with Uncertain Airport Capacity. J. Adv. Transp. 2017, 2017, 1–9. [Google Scholar] [CrossRef]
- Chawla, N.V.; Bowyer, K.W.; Hall, L.O.; Kegelmeyer, W.P. SMOTE: Synthetic Minority Over-Sampling Technique. Jair 2002, 16, 321–357. [Google Scholar] [CrossRef]
- Kuhn, K.D. A Methodology for Identifying Similar Days in Air Traffic Flow Management Initiative Planning. Transp. Res. Part C Emerg. Technol. 2016, 69, 1–15. [Google Scholar] [CrossRef]
- EUROCONTROL. Algorithm to Describe Weather Conditions at European Airports; Technical Report; Eurocontrol: Brussels, Belgium, 2011. [Google Scholar]
- Schultz, M.; Lorenz, S.; Schmitz, R.; Delgado, L. Weather Impact on Airport Performance. Aerospace 2018, 5, 109. [Google Scholar] [CrossRef]
- Hazarika, B.B.; Gupta, D. 1-Norm Random Vector Functional Link Networks for Classification Problems. Complex Intell. Syst. 2022, 8, 3505–3521. [Google Scholar] [CrossRef]
- Kutty, A.A.; Wakjira, T.G.; Kucukvar, M.; Abdella, G.M.; Onat, N.C. Urban Resilience and Livability Performance of European Smart Cities: A Novel Machine Learning Approach. J. Clean. Prod. 2022, 378, 134203. [Google Scholar] [CrossRef]
- Hazarika, B.B.; Gupta, D.; Natarajan, N. Wavelet Kernel Least Square Twin Support Vector Regression for Wind Speed Prediction. Environ. Sci. Pollut. Res. 2022, 29, 86320–86336. [Google Scholar] [CrossRef] [PubMed]
- Borah, P.; Gupta, D. Affinity and Transformed Class Probability-Based Fuzzy Least Squares Support Vector Machines. Fuzzy Sets Syst. 2022, 443, 203–235. [Google Scholar] [CrossRef]
- Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef]
- Wang, Y.; Xia, S.-T. Unifying Attribute Splitting Criteria of Decision Trees by Tsallis Entropy. In Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA, 5–9 March 2017; pp. 2507–2511. [Google Scholar]
- Chen, T.; Guestrin, C. XGBoost: A Scalable Tree Boosting System. In Proceedings of the Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13 August 2016; ACM: New York, NY, USA, 2016; pp. 785–794. [Google Scholar]
- Guo, M.; Yuan, Z.; Janson, B.; Peng, Y.; Yang, Y.; Wang, W. Older Pedestrian Traffic Crashes Severity Analysis Based on an Emerging Machine Learning XGBoost. Sustainability 2021, 13, 926. [Google Scholar] [CrossRef]
- Carvalho, L.; Sternberg, A.; Maia Gonçalves, L.; Beatriz Cruz, A.; Soares, J.A.; Brandão, D.; Carvalho, D.; Ogasawara, E. On the Relevance of Data Science for Flight Delay Research: A Systematic Review. Transp. Rev. 2021, 41, 499–528. [Google Scholar] [CrossRef]
- AlKhereibi, A.H.; Wakjira, T.G.; Kucukvar, M.; Onat, N.C. Predictive Machine Learning Algorithms for Metro Ridership Based on Urban Land Use Policies in Support of Transit-Oriented Development. Sustainability 2023, 15, 1718. [Google Scholar] [CrossRef]
- Shahriari, B.; Swersky, K.; Wang, Z.; Adams, R.P.; de Freitas, N. Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proc. IEEE 2016, 104, 148–175. [Google Scholar] [CrossRef]
Category | Attribute Name (Abbr.) | Attribute Description |
---|---|---|
Weather elements | ATMAP scores | Score obtained by ATMAP algorithm |
Visibility | Surface visibility (miles) | |
Ceiling | Cloud height (miles) | |
Wind direction (WD) | 1 if headwind, 0 otherwise | |
Wind speed (WS) | Surface wind speed (m/s) | |
Traffic features | Scheduled arrivals (SA) | Hourly scheduled arrival counts (aircraft/h) |
Scheduled departures (SD) | Hourly scheduled departure counts (aircraft/h) | |
Actual arrivals (AA) | Hourly actual arrival counts (aircraft/h) | |
Actual departures (AD) | Hourly actual departures counts (aircraft/h) |
Precipitations Severity Code | Type of Precipitations | Coefficient |
---|---|---|
1 | No precipitation | 0 |
2 | RA, UP, DZ, IC | 1 |
3 | −SN, SG, +RA | 2 |
4 | FZxx, SN, +SN | 3 |
MEATR | METAR ZSNJ 150800Z VRB01MPS 3500 -SHRA BR FEW033TCU OVC033 18/17 Q1011 | |||||
---|---|---|---|---|---|---|
Weather Class | Visibility | Wind | Precipitation | Freezing | Dangerous | Sum |
3500 | 01MPS | −SHRA | - | −SHRA FEW033TCU | ||
Quantitative results | 0 | 0 | 1 | 0 | 12 | 13 |
Model | Parameter | Parameter Search Range |
---|---|---|
SVM | [0,1] | |
[0,1] | ||
kernel | [linear,poly,rbf,sigmoid] | |
Random Forest | n_estimators | [0,100] |
max_depth | [0,30] | |
min_samples_split | [0,50] | |
min_samples_leaf | [0,10] | |
XGBoost | n_estimators | [10,300] |
[0,10] | ||
learning_rate | [0.05,0.5] | |
max_depth | [3,30] | |
min_child_weight | [0,10] |
Model | Parameters | Optimal Values |
---|---|---|
SVM | 0.1 | |
0.029 | ||
kernel | rbf | |
Random forests | n_estimators | 20 |
max_depth | 3 | |
min_samples_split | 10 | |
min_samples_leaf | 5 | |
XGBoost | n_estimators | 200 |
3.54 | ||
learning_rate | 0.05 | |
max_depth | 10 | |
min_child_weight | 1.3 |
Prediction Model | Accuracy | Cohen Kappa | Precision | F1 Score | AUC | |||||
---|---|---|---|---|---|---|---|---|---|---|
Train | Test | Train | Test | Train | Test | Train | Test | Train | Test | |
SVM | 0.838 | 0.722 | 0.676 | 0.445 | 0.780 | 0.683 | 0.853 | 0.749 | 0.901 | 0.792 |
Random Forest | 0.812 | 0.748 | 0.595 | 0.497 | 0.733 | 0.707 | 0.822 | 0.771 | 0.871 | 0.821 |
XGBoost | 0.986 | 0.902 | 0.962 | 0.800 | 0.991 | 0.977 | 0.986 | 0.891 | 0.998 | 0.961 |
Model | Data Set | MSE | MAE | |
---|---|---|---|---|
Ridge | Without ATMAP score | 472 | 16.6 min | 0.51 |
With ATMAP score | 430 | 15.71 min | 0.55 | |
LASSO | Without ATMAP score | 462 | 16.89 min | 0.52 |
With ATMAP score | 418 | 15.92 min | 0.56 | |
Decision tree | Without ATMAP score | 302 | 12.03 min | 0.68 |
With ATMAP score | 244 | 10.9 min | 0.74 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Dong, X.; Zhu, X.; Hu, M.; Bao, J. A Methodology for Predicting Ground Delay Program Incidence through Machine Learning. Sustainability 2023, 15, 6883. https://doi.org/10.3390/su15086883
Dong X, Zhu X, Hu M, Bao J. A Methodology for Predicting Ground Delay Program Incidence through Machine Learning. Sustainability. 2023; 15(8):6883. https://doi.org/10.3390/su15086883
Chicago/Turabian StyleDong, Xiangning, Xuhao Zhu, Minghua Hu, and Jie Bao. 2023. "A Methodology for Predicting Ground Delay Program Incidence through Machine Learning" Sustainability 15, no. 8: 6883. https://doi.org/10.3390/su15086883
APA StyleDong, X., Zhu, X., Hu, M., & Bao, J. (2023). A Methodology for Predicting Ground Delay Program Incidence through Machine Learning. Sustainability, 15(8), 6883. https://doi.org/10.3390/su15086883