Natural Language Processing of Aviation Safety Reports to Identify Inefficient Operational Patterns

Abstract

With the growth in commercial aviation traffic and the need for improved environmental performance, strategies to lower emissions that can be implemented in the near term are necessary. Since novel technology takes time to enter the market, operational improvements that employ existing aircraft and require no new infrastructure are fit for this goal. While quantified data collected throughout aviation, such as arrival/departure statistics and flight data, have been well-utilized, text data collected through safety reports have not been leveraged to their full extent. In this paper, a methodology is presented that can use aviation text data to identify high-level causes of flight delays and cancellations, using delays as a metric of operational inefficiency. The dataset is extracted from the Aviation Safety Reporting System (ASRS), which includes voluntary safety incident reports in text narrative and metadata formats. The methodology uses natural language processing tools, K Means clustering, and dimensionality reduction by t-Distributed Stochastic Neighbor Embedding (t-SNE) to categorize and visualize narratives. The method identified 7 major clusters and a total of 23 sub-clusters. A comparison between the subclusters’ topics and the causes of flight delays revealed by the quantified data shows that the ASRS database provides a unique safety perspective to delay cause identification, as illustrated by the method’s identification of maintenance as the main cause of delays, rather than weather.

1. Introduction and Background

In the face of rapid climate change, commercial aviation must mitigate its environmental impact. The European Commission’s Flight Path 2050 has set emission reduction targets of 75% CO₂ reductions, 90% NOX reductions, and 65% perceived noise reductions [1]. Similar emission reduction targets have been put forth by ICAO and IATA, such that the net aviation CO₂ emissions from 2020 would be carbon-neutral and a 50% reduction of CO₂ emission would be sustained until 2050 [2]. Improvements in airframe and engine, the deployment of sustainable aviation fuels, the introduction of government regulations and economic measures to discourage activities that produce CO₂ emissions, and improved operations and air traffic management are the four main research areas regarding attempts to attain net-zero CO₂ emissions by 2050 [3]. Improvements in aircraft fuel efficiency can be made by designing more-electric architectures [4,5] or hybrid-electric powertrains [6,7]. However, these concepts are likely to take decades, due to issues with battery energy densities [8] and safety and reliability considerations [9,10]. Due to the urgency with which humanity must address global warming, waiting for new ultra-efficient aircraft or novel propulsion technology to enter the market is not sufficient. As improvements in operations and ATM can be implemented more quickly, requiring no new infrastructure and employing existing aircraft, research in this area has the potential to improve aviation’s environmental performance within a shorter timeframe. The present work develops a new approach to identify areas of operational inefficiency, which may reduce emissions and costs while improving safety and passenger satisfaction.

Ongoing operations and ATM research include arrival profile optimization, improvements in navigational technology, and weather-related improvements. For example, continuous descent arrival procedures encourage a continuous descent with no intermediate level-offs upon arrival to the destination airport. As a result, longer portions of the flight are at cruise altitude, which is more efficient and requires lower power levels, thus leading to lower emissions. Navigational improvements include the deployment of Automatic Dependent Surveillance-Broadcast (ADS-B) technology, which enables the more precise control of aircraft. Finally, the FAA’s Next-Generation Air Transportation System’s (NextGen) network-enabled weather technology provides advanced, real-time weather data to reduce weather-related delays, with the potential to take advantage of existing weather conditions to improve efficiency [11]. Flight delays is one metric that could be used to identify such operational inefficiencies. The mitigation of flight delays is interesting from an environmental and financial perspective. Environmentally, a reduction in flight delays will optimize and minimize aircraft operation time, which will reduce operational emissions from aviation. From a financial perspective, a study that created a statistical cost estimation model of the effect of delays on airline cost found that about 20% of airline flights arrive more than 15 min late, which costs billions of dollars annually [12]. For example, in 2007, the cost of flight delays for the U.S. economy was estimated to be $32.9 billion, for which more than half was charged to passengers [12]. To compound this problem, it is expected that delays will increase as air traffic demands grow, as it is recognized that delays nonlinearly increase as demand approaches the capacity of the air transportation system [12]. Growing delays will impair the passenger experience of air travel, worsen aviation’s environmental impact, and place a financial burden on airlines, customers, and the global economy.

The aviation industry collects vast amounts of data daily, both qualitative and quantitative, from various sources and in multiple formats, including voice recordings of air traffic control, written reports from aviation personnel, and flight data from on-board measurement devices. Research utilizing these aviation quantitative data to analyze aircraft operations abounds [13,14]. Relatively little research has been conducted on aviation text data for delay-cause-identification, as they are incompatible with conventional computational methods of analysis [15]. The use of aviation text data is a novel approach to the investigation and mitigation of flight delays. With the use of machine learning methods such as Natural Language Processing (NLP), text data can be used with minimal manual labor, using computational data analysis. This study addresses the gap in the application of machine learning to aviation text data to identify the causes of flight delays.

The present work answers two research questions. First, what are causes of operational inefficiency, specifically flight delays? Second, do textual data hold information that cannot be observed through quantified data? The final products of this study are a repeatable NLP-clustering methodology, which can be expanded to any text-based data source, and the identification of causes of flight delays based on information that is available in the selected text-based dataset. The aviation database chosen for this project was selected based on accessibility, the inclusion of narrative-style text data, and the inclusion of operational information. The operational information desired for this study is a detailed report of the events leading to a flight delay. Narrative accounts of aviation events can be found in databases such as the FAA Accident and Incident Data System, NTSB Report Database, and Aviation Safety Reporting Systems (ASRS). The ASRS database is selected for this study (Available online: https://asrs.arc.nasa.gov/ (accessed on 16 June 2022)). ASRS holds voluntary, confidential safety information from frontline aviation personnel, including pilots, controllers, mechanics, flight attendants, and dispatchers in the form of text narratives, along with various metadata characterizing the flight. These narratives are collected for the purpose of policy development, human factors, education, and training. ASRS offers an extensive filtering system, allowing the user to extract the reports which fit the user’s criteria of date, environment, aircraft, reporter, event assessment, etc. This includes a filter for all events resulting in “Flight Cancelled/Delayed”. Additionally, ASRS reports are publicly available and the selected ASRS reports can be extracted in HTML, Word, Excel, and CSV formats.

2. Literature Review

Known causes of flight delays have been identified using quantitative metadata or surveillance records. A study of flight delays in the U.S. found bad weather, carrier equipment, or technical airport problems to be the main causes [16]. Delay statistics for the National Airspace System (NAS), based on data from the Post Operations Evaluation Tool database, found that weather accounted for 69% of arrival and departure delays causes in 2000, followed by traffic volume, runway delays, ATC equipment problems, and other causes [17]. A study focusing on the delay characteristics at Newark International Airport similarly found that convective weather was the main cause for delays averaging longer than 15 min per arrival [18]. A European study found that longer delays were due to technical maintenance issues or aircraft defects while shorter delays were due to operational control, crew duty norms, ATC, and airport limitations, with the chain effects of delays seen as delays to previous flights of the same plane resulted in delays to later flights [19]. Considering the pre-COVID pandemic departure cause analysis from Eurocontrol [20], the largest causes for delays were reactionary, followed by airline, ATC, and weather. Mid-pandemic, the total number of delays went down, but delays due to government causes went up [21]. A study conducted by the Civil Aviation Administration of China found that nearly 50% of delays are caused by severe weather, while roughly 20% of delays are caused by air route problems [22]. The studies used a statistical analysis of quantitative metrics such as average departure delay per flight and percentage of en-route and airport delays or creating predictive models of flight delays using aviation big data and machine learning. The flight delay prediction models were produced using quantitative data including ADS-B surveillance data, date and time, air route, airport. The model was used to identify the most influential factors determining if a flight would experience a delay.

In addition to the causes of delays, there has been interest in the research of delay propagation. For example, if flight A’s arrival is delayed at its destination airport, flight B, which uses the same aircraft as flight A, may also be delayed. This allows for a chain of delays to occur, leading to congestion in the airspace. Congestion can spread through airline fleets and airport systems, and congestion is expected to grow as leisure and business flying demands grow in parallel with economic growth. The Department of Transport (DOT) in the US reported that American Airlines experienced 5.6 million minutes of delays in the month of March 2018 due to traffic growth, resulting in increased costs to cover longer working hours, host passengers during the delay, and other issues [23]. Subject matter experts studying the challenges that the FAA’s NextGen system must address have identified improvements in system efficiency and robustness for the reduction in flight delays as a major focus for NextGen. Limited runway capacity has been identified as a cause of flight delays, which has the potential to be mitigated, in addition to disruptions by weather, which have been identified in previous studies [24]. Runway capacity is the main limiting factor to consider when regulating the queues of departing aircraft and ensuring the orderly and well-timed landings of arriving aircraft. Of the main delay causes that were identified, carrier equipment and maintenance issues are operations-related issues that may be mitigated.

A review of the literature on aircraft maintenance planning and maintenance-related delays suggests that unscheduled maintenance can lead to costly delays or flight cancellations. Of the different types of maintenance, line maintenance, which includes routine checks, post-flight inspections, and malfunction ratification, has the power to decide GO/NOGO for the next flight, making line maintenance an operational phase of interest [25]. In the case of unscheduled maintenance due to issues found in line maintenance, the probability of delays caused by maintenance strongly correlates with the availability of resources at each airport. Research into aircraft maintenance records and maintenance employee interviews to identify the causes of delays in line maintenance corroborate the findings on the importance of resource availability. Maintenance reports identified that poor logistics processes (i.e., availability of spares) accounted for nearly 70% of causes of maintenance delays during line maintenance, followed by unscheduled maintenance and defects found during pilot reports, accounting for 29% of maintenance delays, and poor planning, accounting for 2% [26]. Improved logistics processes were suggested to ensure the availability of spares and better planning of line maintenance activities to organize work packages and manage human resources.

The implementation of TF-IDF and K Means algorithms for document clustering has been investigated by the computing industry, providing insights into the effectiveness of this methodology in non-aviation-specific applications. For the clustering of text data, the text can be represented as a binary vector or a more refined weighted method such as TF-IDF [27]. Singh et al. [28] compared the clustering results of K Means, Heuristic K Means and Fuzzy C Means for documents represented in TF, TF-IDF, and Boolean representations with different feature schemes. For a dataset of standard newswire articles, the study found that the use of TF-IDF with stemming resulted in the most successful clustering, and fuzzy clustering was found to perform better than K Means and heuristic K Means. Khan et al. [29] investigated the efficacy of a TF-IDF/K Means methodology to summarize texts and found that the careful preprocessing of all unnecessary characters, keywords, tags, and punctuations is vital. Gowtham et al. [30] applied TF-IDF and Boolean methods to documents and clustered these using K Means for document classification. Similar to the findings of Khan et al., the importance of proper pre-processing was stressed. The study concluded that the combination of TF-IDF with K Means achieved the best results, as TF-IDF performed better than Boolean and K Means was found to be more efficient than other clustering algorithms.

Finally, the literature on the use of NLP in the ASRS dataset is limited in its scope and volume. Tanguy et al. [31] presented a support vector-machine-based automatic classification task as well as a topic modeling task using ASRS reports to identify safety needs for experts. Robinson [32] implemented topic modeling on fourteen years of ASRS reports to identify changes in safety trends over time. Subramanian and Rao [33] analyzed and classified the key factors contributing to go-around and missed-approach reports from the ASRS database and trained a model to forecast the number of incidents over a given period of time. Ghaoui et al. [34] utilized a term-frequency, inverse-document frequency (TF-IDF) technique to discover the four main tasks that pilots perform during flight, which can aid in understanding the causal and contributing factors to runway incursions and other drivers for aviation safety incidents. Ref. [35] applied the Bidirectional Encoder Representations from Transformers (BERT), an attention-based language model, to determine the answer to the question “When did the incident happen?” for a set of ASRS reports. As noted earlier, little to no literature was found by the authors on the utilization of NLP on aviation text data to determine the causes of flight delays.

In summary, general flight delay causes in the literature were found to be weather, runway capacity, and maintenance issues. Within maintenance-related delays, most delay causes stem from line maintenance, in which resource availability and unscheduled maintenance are key factors. Preventing one delay-causing event can prevent multiple flights from being delayed, as delay propagation from one flight to another is a known problem in aircraft maintenance routing. However, most of the above studies utilize different types of quantitative data to analyze flight delays, leaving a gap in the analysis of qualitative text-based data. The present work seeks to address this gap by analyzing aviation text data.

3. Methodology

To answer the research questions, this study applies NLP and computational data analysis tools to analyze aviation text data and compares the insights drawn from the text data to the causes of flight delays identified in the literature review. While deep learning techniques that better utilize contextual information exist (e.g., BERT [36]), the present work focuses on utilizing a Bag-of-Words (BoW) and a TF-IDF technique to process the ASRS text narratives. Transformer-based deep-learning techniques that better capture the context will be considered in future work. An overview of this methodology is found in Figure 1.

3.1. Data Preprocessing

The dataset is selected by filtering for FAR part 121 and events resulting in “Flight Cancelled/Delayed”, resulting in 4195 reports from January 1999 to January 2022 from ASRS. This work focuses on commercial operations only by filtering for FAR part 121 operations, as the financial motivations for reducing flight delays are based on the commercial aviation market. The narratives are pre-processed to address two points. The first is to clean this in preparation for NLP and the second is to address the inconsistent use of abbreviations. The cleaning of the narratives is completed in four steps: tokenization, lower-casing, stop-word removal, and stemming. Tokenization takes the sentences contained in the narrative and separates them by word, so that each word is a token used to vectorize the narrative. Lower-casing ensures that the capitalization of letters does not interfere with identical words, which are considered distinct. For example, tokens such as “Flight” and “flight” are combined, ensuring that word counts are not affected by differing the capitalization and reducing the dimensionality of the dataset. Stop-word removal removes words that provide little insight into the characterization of the document, such as “the” or “and”. The final step reduces words to their root forms to combine similar words, such as different tenses of the same verb, for the same objectives as lower-casing. This step can be accomplished using stemming or lemmatization. Stemming trims a word to its absolute root, resulting in a token that is not a real English word, while lemmatization converts this to the most closely related existing English word. As the narratives in the ASRS dataset lack speech tags which lemmatization often requires, stemming is chosen as the final preprocessing step for NLP [38]. ASRS narratives often include abbreviations, but the abbreviations are not constant across multiple reports. For example, the word “takeoff” may be written as “tkof” or “takeoff” depending on the reporter. As the NLP process used in this methodology checks for identical words, “tkof” and “takeoff” are clustered separately. A preprocessing step to identify and fix the 17 most common abbreviations is implemented to minimize the effect of this issue. After tokenization, lower-casing, and removing stop words, the word or the root form of each word is checked against a list of English words in Python’s NLTK library. If a word, or the word with suffixes such as ‘s’, ‘ed’, ‘er’, etc., removed, is not recognized by the list of English words, it is flagged as an abbreviation. The average frequency of abbreviations per report was calculated to be 11%. The list of 17 edited abbreviations are in

Table 1. Preprocessed abbreviations list.

Abbreviation	Full Word
acft	aircraft
eng	engine
flt	flight
rptr	reporter
capt	captain
lndg	landing
rwy	runway
emer	emergency
kt	knot
tkof	takeoff
gnd	ground
apch	approach
chk	check
pwr	power
evac	evacuation
hyd	hydraulic
mech	mechanic

. Abbreviations that held more significance than the full word were kept in their original form. This included terms such as air traffic control (ATC), Quick Reference Handbook (QRH), auxiliary power unit (APU), and minimum equipment list (MEL).

3.2. Natural Language Processing

NLP is a machine learning technique that allows for a computer to handle human language. The purpose of employing NLP is to sort large quantities of text data without manual sorting. The preprocessed narrative vectors are used to generate a bag-of-words (BoW) matrix, in which each row represents a narrative, and the columns are stemmed words. For each narrative, the word count for each word in the columns are created, resulting in a numerical representation of the words of which it is composed. The numerical matrix form reduces dimensionality and can be manipulated in the subsequent clustering algorithm step [37]. The Term Frequency and Inverse Document Frequency (TF-IDF) method is employed to identify the characterizing words of each report. The TF-IDF method is a two-step process. First, in the TF step for a given report, the frequency of each term is normalized using the word count of the most frequent word in the given report. In the IDF step, the inverse document frequency is calculated for each word of each report based on Equation (1) [38]. The result is a matrix of a TF-IDF score for each word in each report, such that words that are commonly seen across all reports are given scores close to 0, while unique words that frequently appear in select reports are given higher scores. The high-scoring words can be interpreted as the key words that characterize the report. The inherent normalization when computing the TF-IDF score allows for the scores to be compared across reports, regardless of the varying lengths of the reports. Readers interested in better understanding the TF-IDF algorithm are directed to Ref. [39].

$$tfidf(t,d,D)=tf(t,d)·idf(t,d)$$

(1)

where,

$$tf(t,d)=\frac{f_{t,d}}{max\{f_{t,d}^{\prime }:t^{\prime }\in d\}}$$

(2)

$$idf(t,D)=log\frac{N}{\left|\right\{d\in D:t\in d\left\}\right|}$$

(3)

3.3. Clustering

To categorize the reports based on the characteristic words of each report, we employ K Means clustering. Clustering is a machine-learning technique to categorize data based on similar features. In this case, reports with similar language and words are grouped. While there are many clustering techniques, K Means was chosen as prior work using TF-IDF and K Means found success in an aviation-based application [37]. The dimensionality of this task was reduced to 150 components using principal component analysis (PCA), a linear mathematical algorithm which reduces the dimensionality of a dataset while retaining variation in the dataset [38]. The number of clusters that the reports will be grouped into is user-defined and determined by visually checking a color-coded scatterplot visualization of the clusters and determining if the boundaries are precise and the colored groups are distinct groupings. This visualization was created using t-Distributed Stochastic Neighbor Embedding (t-SNE). t-SNE is an algorithm developed by van der Maaten and Hinton in 2008 to scale high-dimensional data to lower dimensions [40], and will be used to reduce the high-dimensional TF-IDF matrix to two dimensions. Traditional metrics of success for K Means clusters, the Silhouette score and Calinski–Harabasz (CH) score, both encountered issues reflecting the success of clustering text-based data in previous studies [37,41], and therefore will not be reported in this work.

3.4. Observations

For each cluster, a bar chart is produced of the 20 most frequent words among the reports. This is used to manually determine the topic of each cluster that describes the characteristics of the flight delays represented in the cluster. The topic of a cluster was found by identifying the most characteristic or unique words describing a cohesive story. In this process, common words such as ’aircraft’ or ’flight’, which are not useful in distinguishing one cluster from another, are disregarded. The categorized flight delay events described in the text data were compared to the known causes of flight delays found in the literature. If the findings match, we will have shown the validity of using aviation text data and ASRS for determining flight delay causes. If the findings do not fully match, we will have shown the potential for text data to provide new insights to the sources of operational inefficiency.

4. Results

4.1. Overall Clustering

The 4195 event narratives extracted from ASRS were first preprocessed to make the use of abbreviations consistent. This step resulted in 1.78% of the total words across all reports being edited. The NLP, TF-IDF, and K Means clustering yielded the seven clusters shown in Figure 2, illustrated using t-SNE. Seven clusters were chosen after manually evaluating different numbers of clusters and weighing the costs of increasing model complexity against the benefits of obtaining more unique clusters. Each narrative corresponds to a point in the t-SNE plot. The axes of the plot, ‘Dimension 1’ and ‘Dimension 2’, are a result of reducing the high-dimensional matrix of the number of words and number of reports to two dimensions, and thus have no physical meaning.

The topics of each cluster can be discerned from the most frequent words in each cluster, as shown in Figure 3. The main cluster topics were as follows:

0.

Engine;

1.

Hydraulics;

2.

Taxi/Pushback;

3.

Landing/Approach;

4.

Takeoff;

5.

Cabin odor;

6.

Maintenance.

Note that the cluster numbers range from zero to six. The proportion of reports placed in each cluster is shown in Figure 4. Distinct cluster themes were manually identified based on the most frequently seen words in each cluster, but further investigation is necessary to identify more specific event causes, which may be specifically prevented. The clustering algorithm was applied again to each cluster for the subclustering effort.

4.2. Sub-Clustering

The seven main clusters were further subclustered, providing a total of 23 clusters. The hierarchical structure of the main clusters and the subclusters they were divided into is illustrated in Figure 5 with the number of event narratives in each cluster/subcluster shown on the bottom right.

Subclustering yielded more specific safety events that lead to flight delays. However, Cluster 6, the largest main cluster with the main cluster topic of ‘Maintenance’, resulted in vague subcluster topics. The largest subcluster, accounting for 54% of the reports in Cluster 6, had a generic topic of “Engine Maintenance”, as seen in Figure 6. The most frequent words in that subcluster are given in Figure 7. Due to the most frequent words in the subcluster being commonly found words, such as ‘aircraft’, ‘mainten’, and ‘flight’, it appears that Cluster 6 is the catch-all cluster, which captures reports with less defining characteristics or outliers. Other subcluster topics, such as “Odor”, have some specificity but raise questions regarding why these reports were placed in Cluster 6 rather than Cluster 5, with the topic of “Cabin Odor”. The inherent limitation in identifying cluster topics from individual words is the method’s inability to identify key functions, such as dispatchers, pilot, or technicians, or the specific aircraft parts involved in the scenario. Such information would be necessary to use these insights to prevent future flight delays.

Another limitation of this method was its dependency on the use of a consistent vocabulary across all narratives to achieve the best results. This was illustrated in the subclusters found in Cluster 4. The overall cluster topic was identified as “Takeoff”, with two subclusters: “Takeoff rejected” and “Takeoff aborted”. Rejecting and aborting takeoff have the same meaning but were clustered separately due to different writers using varying vocabulary to describe the same event. The abbreviation removal step in Section 3.2 aimed to address this issue, which is inherent to the TF-IDF approach to employing NLP.

An additional observation was made regarding the effect of ASRS’s voluntary reporting structure on an aviation safety study. The reports filed to ASRS are voluntary, meaning that reports are submitted only when the reporter deems the event to be report-worthy. The determination of report-worthiness can differ between reporters, suggesting that the database is not a comprehensive collection of all instances of a given event. Furthermore, the determination of which details of an event should be included or omitted is made by the reporter, which can be affected by factors such as the reporter’s function or background. Reports in ASRS are written by pilots, cabin crew, ATC, dispatchers, technicians, UAS crew, and more. Due to the diversity in the function of writers, having an event described from multiple perspectives would produce a comprehensive understanding of the event, but having only one report of an event would result in a biased or partial description.

4.3. Correlation of Clustering Results to Metadata

An investigation between the a metadata type provided in ASRS was conducted to identify trends in the clusters that are not observable through the t-SNE visualizations and the most frequent words in the cluster. ASRS reports include metadata such as date, location, flight phase, aircraft make model, primary problem, contributing factors, etc., in addition to the text-based narratives. As the cluster topics that were identified were often part-specific, such as engine, landing gear or wing, the “Aircraft Component” metadata was selected for further study. Trends in the clusters were identified by plotting bar charts of the “Aircraft Component” metadata input for each cluster and comparing the results to the cluster’s most frequent words and other clusters. While such metadata may be added as features to the clustering algorithm in addition from NLP outputs, they are not always available in text databases. Hence, they are used to validate the clustering outcomes in the present methodology.

At both the cluster and subcluster levels, the manually identified cluster topics were consistent with the most frequently associated aircraft components. For example, the topic for Cluster 5 was identified as “Cabin Odor” (

Table 2. Most frequently associated aircraft component metadata for each major cluster.

Cluster	Topic	Top Aircraft Component
0	Engine	Turbine Engine
1	Hydraulics	Hydraulic Main System
2	Taxi/Pushback	Nosewheel Steering
3	Landing/Approach	Turbine Engine
4	Takeoff	Turbine Engine
5	Cabin Odor	Air Conditioning and Pressurization Pack
6	Maintenance	Turbine engine

). At the cluster level, the top aircraft component was the Air Conditioning and Pressurization Pack. The subcluster topics were identified as “Smell/Odor” for Subcluster 0 and “Smoke/Fire” for Subcluster 1. For Subcluster 0, the top aircraft components were (1) Air Conditioning and Pressurization Pack, (2) APU, and (3) Coalescer Bag, as seen in Figure 8. The coalescer bag is a filter in the air-conditioning system that collects mist from the air, which can emit a strong odor if it is not cleaned, directly relating to the subcluster topic of smell. For Subcluster 1, the top 2 aircraft components were the same but the third was Electrical Distribution, which relates to the subcluster topic of smoke, as electrical wiring can be a source of smoke or fire. The consistency of the event topic found through NLP and analysis of the most frequently associated aircraft components verifies that the NLP methodology can identify relevant and important information from the safety narratives.

While the observations made in Section 4.2 found that the NLP methodology used in this study cannot identify specific aircraft components that relate to the safety event described by itself, it can provide context that cannot be gleaned from the selected metadata. This is well-illustrated by Cluster 1: “Hydraulics”. The subcluster topics were found to be (1) landing gear hydraulics—maintenance, (2) landing gear hydraulics—emergency, and (3) hydraulics leak. Determining the reason for the three subclusters would be difficult based on the aircraft component metadata alone, as the metadata for the subclusters are similar, as seen in Figure 9. Despite the vastly different circumstances in which the landing gear incidents occurred for subclusters 1.0 and 1.1, the contextual information would be lost in this metadata type, giving value to the NLP’s ability to successfully identify the circumstances in which the safety events took place.

4.4. Comparison to Insights from Quantitative Data

The literature review on the known causes of flight delays revealed that weather accounts for a significant portion of delays. However, the NLP-clustering methodology used on the ASRS database resulted in there being little mention of weather, aside from a deicing subcluster under Cluster 2: Taxi/Pushback. This illustrates the unique safety perspective that ASRS narratives bring to delay-cause identification. The events reported in ASRS are safety-oriented events, which exclude normal operations that are captured well by quantified statistical data. This observation is confirmed by comparing the number of reports related to flight delays and cancellations to the reported number of flight delays and cancellations. The Bureau of Transportation Statistics [42] provides the number of flights which were reported as delayed/cancelled or on time per year. While not all airlines report their statistics to BTS and, therefore, the values do not fully represent all flight operations, this value provides a notional understanding of how many delayed/cancelled flights are reported to ASRS. Comparing the number of relevant ASRS reports that we extracted each year to the statistics in BTS for 2013 to 2021, ASRS captures roughly 0.06% of all flight delay/cancellation instances. As a result of ASRS’s specialization in safety-related incidents, non-safety related causes, such as weather and runway capacity, which have received attention in the literature, are not present in the results. Instead, the reports from ASRS emphasize the maintenance aspect of delays. These findings suggest that improvements in NAS safety could also prevent delays and thereby improve emissions and operational costs.

Of the 23 subclcusters, 12 included ‘mainten’ in the top 20 frequently seen words. Due to the abundant mention of maintenance in the ASRS records, maintenance-related delay causes were investigated. The literature review revealed that unscheduled maintenance and defects found during pilot reports and the unreliable availability of spares account for many maintenance delays during A-check maintenance. To search for clusters related to unscheduled maintenance and spares, the frequency of key words such as ‘mechan’, ‘mainten’, ‘spare’, ‘wait’, and ‘inspect’ in each cluster was calculated and ranked. The results of the rankings for each keyword are shown in

Table 3. Subclusters ranked by frequency of maintenance-related keywords.

Keyword	Cluster–Subcluster Topic with Most Frequent Use of Keyword	Cluster–Subcluster Topic with Second Most Frequent Use of Keyword
Inspect	Maintenance–Maintenance Inspection	Taxi/Pushback–Deicing
Mechan	Maintenance–Minimum Equipment List	Hydraulics leak
Mainten	Maintenance–Minimum Equipment List	Hydraulics leak
Spare	Term not used in dataset
Wait	Taxi/Pushback–Deicing	Maintenance–Odor

. The Maintenance–Minimum Equipment List subcluster being most highly ranked for the keywords ‘mechan’ and ‘mainten’ suggests that maintenance requests relating to the minimum equipment list are a frequent cause of unscheduled maintenance, leading to delays.

5. Conclusions

This work implemented a framework to identify high-level causes of flight delays by applying NLP and clustering algorithms to aviation text data. Seven major categories and a total of 23 more-detailed topics were identified for ASRS reports of events resulting in flight delays or cancellations, each of which represent a circumstance in which the delay occurred. These seven major circumstances were engine, hydraulics, taxi/pushback, landing/approach, takeoff, cabin odor, and maintenance. A comparison between the 23 ASRS narrative topics and the causes of flight delays identified in the literature revealed the unique benefits and limitations of using an aviation safety database for operations research. While ASRS narratives failed to capture the characteristics of normal operations, such as the prevalent effect of weather on the flight delays highlighted in the literature, it provides a safety perspective to the identification of causes of delays. The usefulness of ASRS in cause identification suggests that improvements in NAS safety can also prevent flight delays. The present work has shown the an NLP-clustering method’s ability to identify high-level causes and the circumstances in which delays occur. The methodology can also be extended to any narrative-style data. However, more specific insights, such as specific aircraft parts that require attention or the procedure that directly leads to delays, cannot be identified from this method alone. Future work will include an exploration of other clustering algorithms such as Agglomerative Hierarchical to compare the clustering results to that of the present work. Further investigation of the subclusters by looking for correlations with more metadata types, such as reporter, primary problem, and contributing factors, could lead to more specific insights into clusters with vague topics, such as the Maintenance cluster in Cluster 6. Additionally, a comparison of the frequency of words across clusters could shed light on the value of words that were commonly used and served little purpose in the determination of cluster topics in the present methodology. Finally, subdividing the reports by flight phase before clustering to reflect the five categories of delays that the FAA uses (gate, taxi-out, enroute, terminal, and taxi-in) or the four categories of delays that the DOT uses (gate, taxi-out, airborne, taxi-in) may help to obtain clarity regarding the cluster topics.