Risk Management of Safety for Flight Training in Air Forces

Abstract

Risk management has been an essential issue in the evolution of air-force flight safety. In this paper, the investigated risk management of air-force flight training in the Gangshan airbase, Republic of China (ROC) is the main field of study. The main goal of this paper is to conduct a series of risk identification and assessments of the flight training. Firstly, the 16 risk factors (RFs) of flight training were identified according to the related studies of flight safety and risk and three experts’ interviews. Then, we created a fuzzy-analytic hierarchy process questionnaire and interviewed 20 flight instructors to obtain the weight of likelihood and consequence of the 16 RFs. Furthermore, a sequential assessment of the risk matrix was constructed to classify the 16 RFs into four groups, namely, extreme risk, high risk, medium risk, and low risk. As the results of the revised risk matrix, we provided four suggestions for the improvement of flight-training policy. These suggestions not only can facilitate the Gangshan airbase to smoothly transfer and reduce the deadly risk of flight training, but also provide exemplary risk management for other similar airbases.

1. Introduction

With the advancement of science and technology and the increasingly intertwined international economic, trade, and tourism exchanges, airplanes have become a prevalent and essential means of transportation in daily human lives. Meanwhile, the safety and risk issues that arise from airplanes have been widely discussed [1,2,3,4,5]. Generally speaking, the aircraft, by different uses, can be classified into two types, namely, commercial aircraft and military aircraft. The former typically refers to passenger or cargo transportations, while the latter, usually associates with flight training as the primary mission and becomes an essential weapon to safeguard the country. Currently, most globally published statistical information on flight safety is mainly focused on commercial aircraft. Few relevant articles reveal and investigate the accidents in military aircraft due to the sensitive and confidential features toward defense security and national safety.

Every flight accident is accompanied by a massive loss to society, so having efficient measures and effective management of risk prevention becomes a critical issue. In Taiwan, the Republic of China (ROC), commercial aircrafts experienced 83 total flight accidents that caused 117 deaths in 2010–2019 [6]; while, in the same period, 13 serious flight accidents with 15 pilots’ deaths were caused by military aircraft accidents. Among the 13 accidents, five of them (38%) and seven pilots’ death (47%) had occurred in the Gangshan airbase located in the south of Taiwan. Since Gangshan airbase is the largest among nine airbases air force in ROC, the detailed investigations of Gangshan airbase’s risk factors can not only rectify its operating procedures for further strengthening its risk prevention and management procedures but also provide an exemplary demonstration for other airbases in Taiwan.

Apart from a few studies on aviation safety for military aircraft, the daily operation and mission of military aircraft that train for engaging an enemy are entirely different from commercial aircraft. In ROC, when any invading aircraft are discovered, the Tactic Air Traffic Command Center will promptly guide the jet-fighter pilots of ROC Airforce (ROC-AF) to intercept them. Once engaging, the pilot has to proactively make a quick decision to dodge any hostile lock-on and take counterpart action to attack the invader. To increase the survivability of the engagement at such high-speed maneuvers, pilots have trained hard via a series of aerial tactics daily. By contrast, a commercial aircraft’s daily mission is only to transport passengers and goods safely. Since the flight route of military aircraft is much more dynamic, irregular, and uncertain than commercial aircraft, the flight safety issues in military aircraft should be paid much more attention in research; however, the issues have been widely ignored.

Within the domain of aviation safety, risk management has been an essential issue in the evolution of flight safety in modern aviation [7,8,9,10,11]. Risk management is a systematic procedure, usually including sequential steps: hazard identification, estimation, assessment, and the transferring of risk factors. In the procedure, the risk factors (RFs), i.e., the hazardous elements, are first identified. The risk levels of the RFs are then classified by measuring their likelihood and consequence. According to those risk levels, some subsequent executions of risk control and mitigation measures that ignore the unimportant risks and put more resources and attention on the prevention of highly dangerous RFs are finally implemented to reduce the hazards. In practice, a risk matrix is commonly used to classify the RFs’ risk levels in the procedures of risk management [12,13]. In the traditional risk matrix (TRM), those risk levels are determined by two discrete measures of consequence and likelihood. However, relevant studies have indicated the TRM has some limitations to its practical applications, including (i) the consistency of the quantitative measures in the risk matrix, (ii) the subjective ratings of consequence and likelihood, and (iii) the definition of risk measures and their scales and categories [12]. To tackle these drawbacks, some revised risk matrices (RRMs) with continuous measures of consequence and likelihood were thus proposed [13].

The main goal of this paper is to execute the risk management of flight training in the ROC Gangshan airbase. First, we identified the main RFs according to the references for flight safety and risk and experts’ interviews. Second, we made the fuzzy-analytic hierarchy process (AHP) questionnaire by applying these RFs, and conducted a questionnaire survey to the flight instructors. Finally, the fuzzy-based risk matrix was built to assess the risks of flight training. The results can be used as a reference for managers to formulate flight training safety policies.

2. Reviews of Literature and Methodology

2.1. Flight Safety and Risk

With the advent of aviation prosperity, numerous theories have been proposed to model the flight accidents’ factors including the Domino sequence theory [14], the Swiss cheese model [11], the threat and error management model [15], SMS (safety management system) [16], SHELL (software, hardware, environment, and liveware) model [1], and HFACS (human factors analysis and classification system) [17]. These theories all indicate that a flight accident was not caused by one factor but several factors.

Many studies have discussed flight safety and risk from different perspectives. Goode [7] conducted a study that discovers a discernible pattern of the increased likelihood of an accident occurring in the greater the hours of duty time for commercial aircraft pilots in the United States; the result concluded that the degree of pilot fatigue significantly correlates with the probability of an accident. The analysis further suggested establishing limits on-duty time for commercial pilots to reduce the risk of deadly tragedies. Lee et al. [18] employed a machine learning method to analysis leverage aviation data collected from commercial airline operations to help them increase flight safety. Fala [19] investigated the accident stall-type statistics in aggregate over the past 50 years and reported that stall-type accidents are more than twice as fatal as an average accident.

In the ROC between 1999 and 2006, Li et al. [8] analyzed 41 civil aviation accidents occurring to aircraft registered by using the HFACS framework. Statistical relationships that link fallible decisions in upper-management levels were found to affect supervisory practices directly, thereby creating the psychological preconditions for unsafe acts and, hence, indirectly impairing the performance of pilots, ultimately leading to accidents. Moreover, these outcomes are similar in civil aircraft accidents and military accidents. Bazargan [9] indicated that more experienced pilots are less likely to be involved in an accident caused by pilot error. However, more experienced pilots are more likely to be involved in a fatal accident, probably, due to the higher risk environment they fly in and the more challenging flights they perform. Boyd [10] found the high percentage of fatal accidents attributed to a malfunction irrespective of visibility conditions was surprising, and the lethality rate of multi-engine aircraft is higher than for their single-engine counterparts.

Furthermore, Wang [1] reported the ROC Aviation Divisions of the Ministry of National Defense had established internal, independent, safety audit systems across various divisions to improve aviation safety performance and maintain safety records. The safety audit systems include a pilot-reporting system, operation risk management, operation information resource management, flight and base service reporting system, etc. that are similar to the critical elements of SMS. Oliver et al. [3] indicated that loss of control incidents currently comprise the single most significant cause of accident fatalities in commercial aviation. The study drew on ideas from human-automation interactions, organizational limits, mindful organizing, and sense-making to explore how systems that are very safe by design may subtly undermine mindful organizing, reducing the ability of operators to handle unusual and expected situations. Kelly and Efthymiou [11] used the HFACS framework to determine the factors involved in 50 CFIT (controlled flight into terrain) accidents from 24 counties over ten years, i.e., 2007–2017. The investigation found that CFIT occur across a range of pilot experience and 44% of accidents occurred in cruise flight. Distraction, complacency, and fatigue are all elements that flight crews may experience as contributors to CFIT during cruising.

2.2. Risk Matrix

Generally, risk assessment is used based on fusing the likelihood of an event and the associated consequences of its occurrence [20]. Risk likelihood is the probability of the specified incident happening, which can also be described by a category scale such as “rare, unlikely, possible, likely, and almost certain [21]”. However, a risk consequence is the negative effect of the specified incident. To describe the severity of consequences, some categorized scaling and rating such as “insignificant, minor, moderate, major, and catastrophic” or “very low, low, medium, high, and very high” have been proposed [21,22].

A risk matrix providing a mapping of risk consequence and likelihood is one of the most popular tools for risk assessment [12]. In the pioneer studies of the risk matrix, each pair of the consequence category and likelihood category can be assigned a different risk attribute. Moreover, for easy identification, the risk categories usually use green for low risk, yellow for medium risk, and red for high risk. For instance, International Civil Aviation Organization (ICAO) [23] introduced a TRM with five consequence (severity) categories, i.e., extremely improbable, improbable, remote, occasional, and frequent, and five likelihood (probability) categories, i.e., negligible, minor, major, hazardous, and catastrophic, in Figure 1 as follows. We can observe that this TRM discriminates 25 different, discrete risk categories.

However, Cox [24] proposed six drawbacks of the TRM, which are (a) low resolution, (b) errors, (c) suboptimal resource allocation, and (d) ambiguous inputs and outputs. Subsequently, Levine [25], Chang et al. [20], Duijm [12], and Hsu et al. [13,26,27] proposed some different RRMs based on a continuous probability consequence map. These improvements provide continuous risk scales to identify RFs instead of the discrete risk scales.

Consequently, in this paper, we adopt the RRM proposed by Hsu et al. [13] to assess the risk of flight training. In the construction process of the RRM, the identification of RFs is based on the fuzzy AHP, so respondents can compare which RF occurs more likely for the RFs rather than score each of the RF’s likelihood directly. Moreover, due to the RRM having continuous risk scales, the efficiency of the RFs’ classification is superior to the discrete risk scales of TRM. Figure 2a,b profile an example of the difference between TRM and RRM. Suppose two RFs, which are denoted as A and B, lie into the TRM of Figure 2a, they will be classified into medium risk and low risk, respectively. However, in the RRM of Figure 2b, they will be classified into high risk and medium risk, respectively. As a result, we can observe the performance of classification in the RRM is better than TRM.

Before constructing the RRM of flight training, the RFs should be first identified. In the next section, we briefly review the prominent theories and studies in the field of flight safety and risk control and prevention to ensure the accurate identification of RFs.

3. Research Method

For clarity, the framework of this research method is shown in Figure 3. It begins from the RFs identification of flight training and follows by a fuzzy AHP approach employed for weighting both the RFs’ consequence and likelihood. Based on those weights, a sequential assessment of the risk matrix is finally proposed to classify the RFs’ risk levels that enable the commanders to implement risk control and prevention guidance for improving the flight-training safety operation and performances.

3.1. Identification of Risk Factors

According to the suggestion of The International Air Transport Association and the aforementioned relevant literature, we identified four RFs constructs of the flight training as follows:

I.

Human factors (HFs). This construct includes six RFs: (i) distracted or weary, (ii) unskilled of operational procedures, (iii) violation of standard operation procedure, (iv) misunderstanding of air traffic control instructions, (v) unskilled in using equipment, and (vi) misunderstanding of airport facilities or runway.

I.

II. Mechanical factors (MFs). This construct includes four RFs: (i) failure of the engine system, (ii) failure of the flight control system, (iii) failure of the landing gear system, (iv) failure of the aid navigation equipment, (v) failure of the radio, and (vi) failure of the dynamic and static pressure system.

I.

III. Environmental factors (EFs). This construct includes four RFs: (i) bird strike, internal object damage or foreign object damage of the engine, (ii) low visibility, (iii) slippery runway, (iv) sudden change of weather, (v) low ceiling, and (vi) error of air traffic control causing air miss.

I.

IV. Organizational factors (OFs). This construct includes four RFs: (i) no revised the flight training manual according to the current situation, (ii) no executed the assessment on the training of flight instructor, (iii) no executed the flight dynamic monitoring mechanism, (iv) no care about the subject and simulator training of trainee, (v) conservative leadership thinking of the flight administrator, and (vi) improper flight training arrangements.

Based on the above identifications, a two-layer system of hierarchical RFs for flight training was created. To improve the practical validity of the RFs, three senior flight instructors who have over 2000 hours of flight experience were invited to revise those RFs and check if any crucial RFs were missing. Additionally, they were also asked to confirm the independence among all RFs. After three rounds of revisions with two RFs combined and six RFs deleted, the final version of RFs is shown as

Table 1. The RFs of flight training.

Layer1: Construct		Layer 2: RFs
Human Factors (HFs)	HF1	Distracted or weary
	HF2	Unskilled in using equipment or standard operation procedure
	HF3	Misunderstanding of air traffic control instructions
	HF4	Misunderstanding of airport facilities or runway
Mechanical Factors (MFs)	MF1	Failure of the engine system
	MF2	Failure of the flight control system
	MF3	Failure of the landing gear system
	MF4	Failure of the dynamic and static pressure system
Environmental Factors (EFs)	EF1	Bird strike, internal object damage, or foreign object damage to the engine
	EF2	Low visibility
	EF3	A sudden change of weather
	EF4	Low ceiling
Organizational Factors (OFs)	OF1	No revised flight training manual according to the current situation
	OF2	No executed the assessment on the training of flight instructor
	OF3	No executed the flight dynamic monitoring mechanism
	OF4	No care about the subject and simulator training of trainee

, including four constructs of RFs in the first layer and 16 RFs in the second layer.

3.2. Questionnaire Design

To measure the subject’s perceived consequence and likelihood toward each RF, we designed two nine-point rating scale AHP questionnaires according to the hierarchical structure of RFs in Table 1, where one is for likelihood survey and another one is for consequence survey. Subsequently, we pre-tested three senior flight instructors to validate the scale, by which several statements’ descriptions were revised.

3.3. Research Sample

Since each subject was asked to answer both perceived Likelihood and Consequence and satisfaction on RFs, the total samples were 20 (20 for Likelihood measures and 20 for Consequence measures). For verifying the consistency of the 40 measures, both indexes Consistency Index (CI) and Consistency Ratio (CR) were used to test the consistency of each sample’s pairwise comparison matrix:

$$\mathrm{CI}=\frac{{\lambda }_{\max }-n}{n-1}$$

(1)

and

$$\mathrm{CR}=\frac{\mathrm{CI}}{\mathrm{RI}}$$

(2)

where ${\lambda }_{\max }$ is the maximum eigenvalue for each matrix, n is the number of criteria in the matrix and RI represents a randomized index shown in

Table 2. The values of the RI corresponding to a variety of n.

n	3	4	5	6	7	8	9	10	11	12
RI	0.525	0.882	1.115	1.252	1.341	1.404	1.452	1.484	1.513	1.535

(e.g., Hsu et al. [13]). In practice, Saaty [28] suggested that the $\mathrm{CR}\le 0.1$ is an acceptable range.

In this paper, the software package Expert Choice 11.5 was used to find the CI, by which the CR can then be obtained by Equation (2). The results showed that six samples’ CI or CR > 0.1, which meant that they were inconsistent [28]. Therefore, these questionnaires’ subjects were asked to modify their questionnaires until they passed both the consistency tests of CI and CR.

Table 3. Profile of the respondents.

Features	Range	Frequency	Percentage (%)
Age (years)	20–30	5	25%
	31–40	6	30%
	41–50	5	25%
	51–60	4	20%
Military rank	Lieutenant	1	5%
	Captain	3	15%
	Major	6	30%
	Lieutenant Colonel	4	20%
	Colonel	6	30%
Experience for flight (hours)	300–800	3	15%
	800–1300	3	15%
	1300–1800	4	20%
	1800–2300	3	15%
	Above 2300	7	35%
Flight qualification	Level D	2	10%
	Level C	7	35%
	Level B	5	25%
	Level A	6	30%

profiles the validated 20 respondents’ characteristics; we can discover easily that all of the subjects have flight instructor qualifications with at least 300 h of flight experience. It is noteworthy that the reliability of the survey findings can be endorsed due to the experienced qualifications of the respondents.

3.4. The Weights of RFs

From the sample data, we have 40 positive reciprocal matrices (20 likelihood measures and 20 consequence measures). For considering the linguistic fuzziness of respondents in answering surveys, a fuzzy AHP approach was then developed to weight the RFs from the matrices, including both the measures of likelihood and consequence. For the convenience of explanation, we take the second layer of RFs with likelihood measures in the HF construct as an example to explain the process of the fuzzy AHP approach. The RFs in the HF construct, shown in Table 1, include HF1, HF2, HF3, HF4.

3.4.1. The Aggregated Positive Reciprocal Matrix

Let P be a positive reciprocal matrix with n elements (i.e., RFs) for each respondent:

$$P={[p_{ij}]}_{n\times n},i=1,2,\dots ,n,j=1,2,\dots ,n$$

(3)

In this paper, for each element $p_{ij}$, we use a triangular fuzzy number parameterized by ${\tilde{p}}_{ij}=[a_{ij},b_{ij},c_{ij}]$ to combine the measures from m respondents by: [$a_{ij}$ = minimum, $b_{ij}$ = geometric mean and $c_{ij}$ = maximum], as follows (e.g., Hsu et al. [13]; Huang et al. [29]):

$$\begin{array}{c}{\tilde{p}}_{ij}=[a_{ij},b_{ij},c_{ij}]=\left[\underset{1\le k\le m}{\min }\left\{p_{ij}^{(k)}\right\},{\left(\underset{k=1}{\overset{m}{\mathsf{\Pi }}}{p}_{ij}^{(k)}\right)}^{1/m},\underset{1\le k\le m}{\max }\left\{p_{ij}^{(k)}\right\}\right]\\ i=1,2,\dots ,n,j=1,2,\dots ,n\mathrm{and}k=1,2,\dots ,m\end{array}$$

(4)

Based on the elements in Equation (4), an aggregated positive reciprocal matrix can be created as:

$$\tilde{P}={[{\tilde{p}}_{ij}]}_{n\times n}=\left[\begin{array}{l}1{\tilde{p}}_{12}\dots {\tilde{p}}_{1n}\\ {\tilde{p}}_{21}1\dots {\tilde{a}}_{2n}\\ :::\\ {\tilde{p}}_{n1}{\tilde{p}}_{n2}\dots 1\end{array}\right]$$

(5)

where ${\tilde{p}}_{ij}$ are triangular fuzzy numbers related to different sets of parameters:

$${\tilde{p}}_{ij}=\{\begin{array}{cc}\begin{array}{l}[a_{ij},b_{ij},c_{ij}],\mathrm{if}i>j\\ [1,1,1],\mathrm{if}i=j\end{array}& \\ \left[\frac{1}{c_{ji}},\frac{1}{b_{ji}},\frac{1}{a_{ji}}\right],\mathrm{if}i<j& \end{array}$$

As an example, since the RFs in HF were constructed with likelihood measures, based on Equations (4) and (5), we have the aggregated fuzzy matrix ${\tilde{P}}_1$ as:

$${\tilde{P}}_1=\left[\begin{array}{l}[1.000,1.000,1.000][0.111,0.473,6.000][0.200,0.782,6.000][0.167,1.012,8.000]\\ [0.167,2.113,9.000][1.000,1.000,1.000][0.111,1.691,8.000][0.250,2.076,8.000]\\ [0.167,1.278,5.000][0.125,0.591,9.000][1.000,1.000,1.000][0.333,1.311,5.000]\\ [0.125,0.988,6.000][0.125,0.482,4.000][0.200,0.763,3.000][1.000,1.000,1.000]\end{array}\right]$$

3.4.2. The Fuzzy AHP Approach

Theoretically, the weights of RFs can be determined from the eigenvectors of the fuzzy matrix $\tilde{P}$. Let $\tilde{P}$ be a positive reciprocal matrix as shown in Equation (5), For determining the eigenvectors of $\tilde{P}$, Saaty [28] suggested four simplified methods: Normalization of the Geometric Mean of the Rows (NGMR), Normalization of the Reciprocal of Columns Sum, Normalization of the Row Average, and Average of Normalized Columns; in this study, the NGMR was adopted.

For the ith RF ($i=1,2,\dots ,n$) in matrix $\tilde{P}$, its geometric means ${\tilde{g}}_i$ can be obtained by operating as follows:

$${\tilde{g}}_i={\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{\tilde{p}}_{ij}\right)}^{1/n}=\left[{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{a}_{ij}\right)}^{1/n},{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{b}_{ij}\right)}^{1/n},{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{c}_{ij}\right)}^{1/n}\right],i=1,2,\dots ,n$$

(6)

$$\Rightarrow {\displaystyle \sum _{i=1}^n{\tilde{g}}_i}=\left[{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{a}_{ij}\right)}^{1/n}},{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{b}_{ij}\right)}^{1/n}},{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{c}_{ij}\right)}^{1/n}}\right]$$

(7)

As a result, the weight ${\tilde{w}}_i$ for the ith RF ($i=1,2,\dots ,n$) can then be further formulated as:

$${\tilde{w}}_i={\tilde{g}}_i/{\displaystyle \sum _{i=1}^n{\tilde{g}}_i}=\left[\frac{{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{a}_{ij}\right)}^{1/n}}{{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{c}_{ij}\right)}^{1/n}}},\frac{{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{b}_{ij}\right)}^{1/n}}{{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{b}_{ij}\right)}^{1/n}}},\frac{{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{c}_{ij}\right)}^{1/n}}{{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{a}_{ij}\right)}^{1/n}}}\right],i=1,2,\dots ,n$$

(8)

Since the example ${\tilde{P}}_1$ is a positive reciprocal matrix, the NGMR method can thus be used to find its eigenvectors. Based on Equation (6), the geometric mean ${\tilde{g}}_i$ (i = 1,2,…,4) can be found as:

$$\left[\begin{array}{l}{\tilde{g}}_1\\ {\tilde{g}}_2\\ {\tilde{g}}_3\\ {\tilde{g}}_4\end{array}\right]=\left[\begin{array}{l}[0.2467,0.7852,4.1195]\\ [0.2608,1.6503,4.8990]\\ [0.2887,0.9977,3.8730]\\ [0.2364,0.7762,2.9130]\end{array}\right]$$

By referring to Equation (7), we have:

$${\displaystyle \sum _{i=1}^4{\tilde{g}}_i=[1.0327,4.2066,15.8044]}$$

Finally, from Equation (8), we can acquire the weight ${\tilde{w}}_i$ for the ith RF (i = 1,2,…,4) as:

$$\left[\begin{array}{l}{\tilde{w}}_1\\ {\tilde{w}}_2\\ {\tilde{w}}_3\\ {\tilde{w}}_4\end{array}\right]=\left[\begin{array}{l}[0.0156,0.1860,3.9893]\\ [0.0165,0.3923,4.7441]\\ [0.0183,0.2372,3.7505]\\ [0.0150,0.1845,2.8208]\end{array}\right]$$

3.4.3. The Defuzziness Process

Since the weight ${\tilde{w}}_i$ of the ith RF ($i=1,2,\dots ,n$) in $\tilde{P}$ is fuzzy, Yager’s index (e.g., Hsu et al. [13]) was adopted to defuzzify the ${\tilde{w}}_i$ into a crisp number $w_i$ ($i=1,2\dots ,n$). For the convenience of explanations, let ${\tilde{w}}_i=[a_i^w,b_i^w,c_i^w]$, where

$$[a_i^w,b_i^w,c_i^w]=\left[\frac{{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{a}_{ij}\right)}^{1/n}}{{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{c}_{ij}\right)}^{1/n}}},\frac{{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{b}_{ij}\right)}^{1/n}}{{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{b}_{ij}\right)}^{1/n}}},\frac{{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{c}_{ij}\right)}^{1/n}}{{\displaystyle \sum _{i=1}^n{\left(\underset{j=1}{\overset{n}{\mathsf{\Pi }}}{a}_{ij}\right)}^{1/n}}}\right],i=1,2,\dots ,n$$

The Yager’s index [30] of the ${\tilde{w}}_i$,$i=1,2,\dots ,n$, is defined as:

$$w_i=(a_i^w+b_i^w+c_i^w)/4,i=1,2,\dots ,n$$

(9)

Normalizing the $w_i$ ($i=1,2\dots ,n$), the crisp weight ${\omega }_i^{}$ of the ith RF can then be obtained as:

$${\omega }_i^{}=w_i/{\displaystyle \sum _{i=1}^n{w}_i},i=1,2,\dots ,n$$

(10)

For the example ${\tilde{P}}_1$, by manipulating both Equations (9) and (10), the $w_i$ and ${\omega }_i$ (i = 1, 2, …, 4) for the under GE construct can be obtained as:

$$\left[\begin{array}{l}{w}_1\\ w_2\\ w_3\\ w_4\end{array}\right]=\left[\begin{array}{l}1.0942\\ 1.3863\\ 1.0608\\ 0.8012\end{array}\right]\Rightarrow \left[\begin{array}{l}{\omega }_1\\ {\omega }_2\\ {\omega }_3\\ {\omega }_4\end{array}\right]=\left[\begin{array}{l}0.2520\\ 0.3192\\ 0.2443\\ 0.1845\end{array}\right]$$

Thus, we have the weights of (HF1, HF2, HF3, HF4) to be (25.20%, 31.92%, 24.43%, 18.45%). Likewise, by following the above steps 4.4.1~4.4.3, the RFs weights in the first layer and second layer can be found and listed in the second and fourth fields of

Table 4. The likelihood weights of RFs.

Layer 1 of RFs	The Global Weights of Layer 1 (%)	Layer 2 of RFs	The Local Weights of Layer 2 (%)	The Global Weights of Layer 2 (%)
HF	38.34	HF1	25.20	9.66
		HF2	31.92	12.24
		HF3	24.43	9.37
		HF4	18.45	7.07
MF	23.34	MF1	26.91	6.28
		MF2	19.80	4.62
		MF3	34.40	8.03
		MF4	18.89	4.41
EF	21.85	EF1	24.18	5.28
		EF2	38.10	8.33
		EF3	10.13	2.21
		EF4	27.60	6.03
OF	16.46	OF1	14.11	2.32
		OF2	12.70	2.09
		OF3	25.72	4.23
		OF4	47.47	7.81

, respectively.

3.4.4. The Global Weights of RFs

Finally, via Table 4, the global weights of the RFs in the second layer can be obtained by multiplying their local weights in the fourth field by their corresponding weights in the second field. The global weights of the RFs in the second layer can be found as the last field of Table 4. Similarly, the global weights of the RFs for the consequence measures can be, subsequently, found and exhibited in

Table 5. The consequence weights of RFs.

Layer 1 of RFs	The Global Weights of Layer 1 (%)	Layer 2 of RFs	The Local Weights of Layer 2 (%)	The Global Weights of Layer 2 (%)
HF	27.95	HF1	37.06	9.15
		HF2	28.48	7.03
		HF3	24.58	6.07
		HF4	9.88	2.44
MF	30.97	MF1	34.94	10.05
		MF2	30.56	8.79
		MF3	21.51	6.19
		MF4	12.99	3.74
EF	28.33	EF1	29.67	7.47
		EF2	17.97	4.52
		EF3	22.77	5.73
		EF4	29.59	7.45
OF	12.76	OF1	15.77	3.37
		OF2	25.05	5.35
		OF3	37.04	7.92
		OF4	22.14	4.73

.

3.5. The RRM of Flight Training

A widely used method to measure the impact level of an RF is the product of the risk likelihood and the risk consequence, which can be denoted as a risk significance index score (RSIS) [31]. Let ${\mathrm{CW}}_i$ and ${\mathrm{LW}}_i$ be the consequence weight (CW) and likelihood weight (LW) of i-th RF, respectively. Then, the RSIS of i-th RF is defined as:

$${\mathbf{RSIS}}_i={\mathbf{CW}}_i\times {\mathbf{LW}}_i,i=1,2,\cdots ,n$$

(11)

Finally, the RSIS can be normalized as:

$${\mathbf{RSIS}}_i=\frac{{\mathbf{CW}}_i\times {\mathbf{LW}}_i}{{\displaystyle \sum _{i=1}^n\left({\mathbf{CW}}_i\times \mathbf{LW}\right)}}\times 100\%,i=1,2,\cdots ,n$$

(12)

According to Equation (12) and the weights in the last fields of Table 4 and Table 5, the RSIS for each RF can be given. As results shown in the fourth field of

Table 6. The classification of RFs.

RFs	LW (%)	CW (%)	RSIS (%)	Risk Level
HF1	9.66	9.15	13.75	ER
HF2	12.24	7.03	13.39
MF1	6.28	10.05	9.82
HF3	9.37	6.07	8.85	HR
MF3	8.03	6.19	7.73
EF4	6.03	7.45	6.99
MF2	4.62	8.79	6.32
EF1	5.28	7.47	6.14	MR
EF2	8.33	4.52	5.86
OF4	7.81	4.73	5.75
OF3	4.23	7.92	5.21
HF4	7.07	2.44	2.68	LR
MF4	4.41	3.74	2.57
EF3	2.21	5.73	1.97
OF2	2.09	5.35	1.74
OF1	2.32	3.37	1.22

, they indicate the RFs with the higher risk are HF1 (13.75%), HF2 (13.39%), MF1 (9.82%), HF3 (8.85%), and MF3 (7.73%).

Moreover, by referring to Figure 2b, the sequential assessment of the risk matrix with four risk zones is constructed to rank the RFs’ risk levels. The risk matrix shown in Figure 4 consists of the x-axis by consequence weight and the y-axis by likelihood weight. According to Equation (12), the matrix is allowed to divide into four risk zones by three decreasing curves with different RSIS means. Firstly, by averaging the RSISs of all RFs in Table 5, we have the middle curve with RSIS = 6.25%, by which all RFs can be divided into two groups. Group one contains 7 RFs (HF1, HF2, MF1, HF3, MF3, EF4, and MF2) and group 2 includes 9 RFs. Averaging the 7 RFs’ RSISs in group one, the second curve with mean RSIS = 9.55% can be obtained. Similarly, the third curve with RSIS = 3.68% can be found by averaging the 9 RFs’ RSISs in group two.

The results, shown in Figure 4, indicate three RFs (HF1, HF2, and MF1) classified as extreme risk, four RFs (HF3, MF3, EF4, and MF2) as high risk, four RFs (EF1, EF2, OF4, and OF3) as medium risk, and five RFs (HF4, MF4, EF3, OF2, and OF1) as low risk. In this case, commanders are notified to pay more attention to the first two critical classes of RFs (i.e., HF1, HF2, MF1, HF3, MF3, EF4, and MF2) to prioritize the safety improvement in the fighter training.

4. Discussions

In this paper, we identified 16 RFs of flight training and constructed an RRM by using the fuzzy AHP to assess these RFs. The assessment of these 16 RFs shows three RFs were classified as extreme risk, which are HF1 (Distracted or weary), HF2 (unskilled in using equipment or standard operation procedure), and MF1 (failure of the engine system). Furthermore, four RFs were classified as high risk, which are HF3 (misunderstanding of air traffic control instructions), MF3 (failure of the landing gear system), EF4 (low ceiling), and MF2 (failure of the flight control system). These outcomes indicate HFs, i.e., human factors, are the highest risk construct and followed by MFs, i.e., mechanical factors.

Subsequently, we interviewed three senior flight instructors to provide some advice for the improvement of flight training policies and guidelines, as follows:

I.

Adjusting the schedule of some training subjects and duty—

In practice, the flight trainees have a heavy training load because they have to learn numerous subjects in a short period, e.g., technical order of equipment, flight operational procedures, the principle of flight, tactics of flight, etc. Besides, there are lots of training duties that have to execute every day. For example, runway foreign object removal, on duty in the mobile control room, simulator training, write flight plan before every flight training, write a flight review report after every flight training, etc. Such heavy flight training loading lets flight trainees lack adequate rest. Therefore, we suggest allocating some necessary duties as the extracurricular learning subjects at the college level; in-advance education and training can help flight trainees reduce the loading, thereby concentrating on flight training.

II.

Purchasing new training equipment—

The operational procedures of flight are very complicated with many details. Moreover, the operational procedures of the radio also require a great deal of learning time to become savvy, because many of aviation’s technical terms and terminologies are used. However, the rehearsal time on the real airplane is limited, so the simulator or other training equipment is needed for the trainee to execute more practice. With the rapid development of technology, technologies such as augmented reality (AR) and virtual reality (VR) have been applied in practice. Especially in the training area, the AR and VR technologies have been demonstrated their improvement of learning efficiency [32,33,34]. Consequently, we suggest purchasing new AR and VR training equipment to build an immersive learning environment, thereby helping the trainee learn quickly and efficiently.

III.

Purchasing the training airplane with two engines—

The engine is one of the most important pieces of equipment in the airplane. Generally, the failure of the engine system may follow by serious damage. The risk of engine failure in the training airplane with only one engine is higher than the training airplane with two engines. Due to the training airplane is driven by the trainee who is lack flight experience, the safety requirement should be stricter.

• IV. Strengthening the practice of emergency operational procedures—

Generally, there is no way to 100% avoid an emergency, such as the failure of the engine system, failure of the landing gear system, or failure of the flight control system. Hence, the trainees have to exercise the emergency operational procedures more strictly to deal with these emergencies proficiently. Moreover, using the AR and VR training equipment is a good way to rehearse these emergency operational procedures because they can immerse themselves in more emergency practices without endangering their lives. This result can provide the director of flight training more motivation for the purchasing of new training equipment.

5. Conclusions

Within the domain of aviation safety, risk management has been an essential issue in the evolution of flight safety in air forces. In this paper, the risk management of air-force flight training investigated in the Gangshan airbase, Republic of China (ROC) is the main field study since it has borne over one-third of the flight accidents and pilot deaths of the ROC air force during the years from 2010 to 2019.

After a series of risk management investigations, we identified the 16 RFs of flight training and constructed a continuous risk matrix to classify the 16 RFs into four groups, i.e., extreme risk, high risk, medium risk, and low risk, by a fuzzy AHP questionnaire and 20 flight instructors interviews. Subsequently, we provided four suggestions for the improvement of flight-training policy according to the results of the RRM, which are (i) adjusting the schedule of some training subjects and duty, (ii) purchasing new training equipment, (iii) purchasing the training airplane with two engines, and (iv) strengthening the practice of emergency operational procedures. The results can be used as a reference for managers to formulate flight training safety policies.

Finally, the RFs in this paper were only obtained from the flight instructors’ perspectives. However, flight trainees control the aircraft during most of the flight time. Especially, flight trainees are required to fly alone in some flight areas. Therefore, identifying RFs from flight trainees’ perspectives is a valuable area for future research.