Analysis of Aircraft Maintenance Related Accidents and Serious Incidents in Nigeria

Abstract

The maintenance of aircraft presents considerable challenges to the personnel that maintain them. Challenges such as time pressure, system complexity, sparse feedback, cramped workspaces, etc., are being faced by these personnel on a daily basis. Some of these challenges cause aircraft-maintenance-related accidents and serious incidents. However, there is little formal empirical work that describes the influence of aircraft maintenance to aircraft accidents and incidents in Nigeria. This study, therefore, sets out to explore the contributory factors to aircraft-maintenance-related incidents from 2006 to 2019 and accidents from 2009 to 2019 in Nigeria, to achieve a deeper understanding of this safety critical aspect of the aviation industry, create awareness amongst the relevant stakeholders and seek possible mitigating factors. To attain this, a content analysis of accident reports and mandatory occurrence reports, which occurred in Nigeria, was carried out using the Maintenance Factors and Analysis Classification System (MxFACS) and Hieminga’s maintenance incidents taxonomy. An inter-rater concordance value was used to ascertain research accuracy after evaluation of the data output by subject matter experts. The highest occurring maintenance-related incidents and accidents were attributed to “removal/installation”, working practices such as “accumulation of dirt and contamination”, “inspection/testing”, “inadequate oversight from operator and regulator”, “failure to follow procedures” and “incorrect maintenance”. To identify the root cause of these results, maintenance engineers were consulted via a survey to understand the root causes of these contributory factors. The results of the study revealed that the most common maintenance-related accidents and serious incidents in the last decade are “collision with terrain” and “landing gear events’’. The most frequent failures at systems level resulting in accidents are the “engines” and “airframe structure”. The maintenance factors with the highest contribution to these accidents are “operator and regulatory oversight”, “inadequate inspection” and “failure to follow procedures”. The research also highlights that the highest causal and contributory factors to aviation incidents in Nigeria from 2006 to 2019 are “installation/removal issues”, “inspection/testing issues”, “working practices”, “job close up”, “lubrication and servicing”, all of which corresponds to studies by other researchers in other countries.

1. Introduction

Aircraft maintenance is an important part of the global aviation industry. It sometimes entails complicated tasks to be carried out by Aircraft Maintenance Engineers (AME) often with considerable time constraints [1]. In recent times, the aviation industry has experienced a rapid advancement in technology such as highly automated and integrated systems, which increases the burden on the AMEs to maintain both old and new fleets. AMEs also need to constantly improve their knowledge compared to the AMEs in previous times [1]. These advancements in technology have the tendency to introduce new types of maintenance errors, this is because hindsight cannot always be used to assess the system safety of new and complex designs. However, it should also be noted that other advancements such as fail-safe systems, enhanced hardware and in recent decades the use of health monitoring technologies on engines, systems and even structures have contributed to the reduction in maintenance and inspection workload [2].

1.1. Study Background, Accidents and Maintenance-Related Events

The Nigerian Civil Aviation Regulations (Nig. CARs) [3] define commercial air transport and general aviation as: “an aircraft operation involving the public transportation of passengers, cargo or mail for remuneration or hire” and “an aircraft operation other than a commercial air transport operation or an aerial work operation”, respectively.

Aviation activities in Nigeria increased post-independence in 1963 after the Federal government of Nigeria obtained all Nigerian Airways shares [4] and the increase was especially observed in commercial air transport. In 2006, by the Civil Aviation Act, the Nigerian Government established Nigerian Civil Aviation Authority (NCAA) as an autonomous regulatory body and the Accident Investigation Bureau which has the sole responsibility of independently investigating accidents in Nigeria in accordance with International Civil Aviation Organisation (ICAO) Annex 13.

Nigeria has experienced various accidents and serious incidents over the years. The most recent fatal accident occurred when a Sikorsky S−76C + operated by Bristow Helicopters (Nigeria) Limited crashed at Oworonsoki area of Lagos in 2015 and unfortunately claimed all 12 souls on board [5]. Daramola [6] analysed all accidents that occurred in Nigeria from 1985 to 2010 using the Human Factors Analysis and Classification System (HFACS) which is a taxonomy framework widely used in aviation. Findings from Daramola’s research showed that skill-based error, inadequate supervision and environment were the three most contributory factors.

While this study focuses on the analysis of accidents, serious incidents and occurrences that must be reported to the regulatory authority, e.g., “Mandatory Occurrence Reporting” in Nigeria, the previous studies focusing on European and global datasets enabled us to use the previously developed taxonomies based on the analysis of such data. Therefore, understanding the trends outside of Nigeria is also important. The European Aviation Safety Agency [7] Annual Safety Review (ASR) identified aircraft maintenance as a safety issue affecting Key Risk Areas (KRA). The KRAs affected are aircraft upset, runway excursion, and aircraft environment in commercial and non-commercial operations. Aircraft maintenance also had higher risk occurrence in comparison to other safety issues identified [7]. Analysis of 120 accidents and incidents which occurred from 2014 to 2018 revealed that aircraft maintenance contributed to 17 of them [7]. Unfortunately, maintenance errors were not considered in European Aviation Safety Agency (EASA)’s ASR prior to 2014.

The UK Air Accidents Investigation Branch (AAIB) includes aircraft maintenance and inspection as one of the safety recommendation topics in their ASR. The maintenance- and inspection-related recommendations are shown in

Table 1. UK Air Accidents Investigation Branch (AAIB) maintenance- and inspection-related safety recommendations, adapted from AAIB [5,8,9].

Year	Aircraft Mx/Inspection Recommendation	Total Number of Recommendations	Maintenance Recs/Total Recommendations
2018	4	37	10.80%
2017	3	66	4.55%
2016	1	125	0.80%

.

Top findings from the International Air Transport Association [10] safety report for 2018 show that maintenance events were the sixth most significant threat by 13% contribution to the total number of accidents between 2014 and 2018 but they contributed to only 7% of the fatal accidents during the same period.

The same report also showed an interesting discrepancy about the contribution of maintenance events to total number of accidents between IOSA-registered operators (17%) and non-IOSA-registered operators (10%) revealed an interesting fact. This is rather contradictory to the difference in overall safety performance between the IOSA-registered and non-IOSA-registered operators as the latter’s performance is significantly poorer and this is used by International Air Transport Association (IATA) to promote the IATA Operational Safety Audit (IOSA) programme.

A total of 586 lives out of 4.3 billion passengers were lost in 2018, due to events in commercial air transport. This is in contrast to the 67 fatalities recorded out of 4.1 billion passengers in the year 2017—tagged the “safest year ever” [7]. In the general aviation category, EU-registered aircraft with Maximum Certificated Take-Off Mass (MCTOM) above 2250 kg experienced events that saw the loss of 12, while EU-registered aircraft with MCTOM under 2250 kg experienced the loss of 159 lives [11]. Maintenance contribution to flight safety has been identified as one of the current and emerging safety risks by IATA in the 2018 annual review [12]. Details can be found in Appendix A.

In Africa, the average fleet age for different operators is in the range of 6 to 28 years and Nigerian based airlines were found to operate the oldest aircraft in comparison to air operators based in Ethiopia, South Africa and Rwanda [13]. One of the key challenges the airlines operating aging fleet face is the additional maintenance tasks such as corrosion prevention and control tasks which aim to ensure the airworthiness of the aircraft [14].

African carriers are consistently banned by the European Commission partly due to inadequate authority oversight [15]. In spite of this, a review of various academic journals via Google Scholar, Scopus, Science Direct, Elsevier and Emerald found very limited literature presenting the analysis of aircraft accidents and incidents in Nigeria or Africa. Additionally,

Table 2. Departures and accidents rates by Regional Aviation Safety Group (RASG) region of occurrence. Adapted from International Civil Aviation Organisation (ICAO) (2018).

Regional Aviation Safety Group	Estimated Departures (Millions)	Number of Accidents	Accident Rate (per Million Departures)
Africa	1.3	7	5.3
Asia Pacific	11.8	20	1.7
Europe	8.7	12	1.4
Middle East	1.3	2	1.6
Pan America	13.5	47	3.5
Worldwide	36.6	88	2.4

The data for the African continent are in bold to highlight higher than average accident rate, in spite of the lower estimated departures compared to other regions.

shows the departures and accidents rate by the International Civil Aviation Organisation (ICAO) Regional Aviation Safety Group (RASG) region of occurrence.

1.2. Aim and Objectives

The aim of this research is to identify the most significant aircraft-maintenance-related causal and contributory factors to accidents and serious incidents in Nigeria. Additionally, the study also aims to highlight the importance of utilising taxonomies for data analysis and in order to identify mitigation strategies.

To achieve this aim, the following objectives were developed:

• Identify and validate maintenance-related accidents in commercial aircraft category and general aviation category aeroplanes, which were published in the last 10 years, i.e., 2009−2019.
• Identify and validate all maintenance-related occurrences in commercial aircraft category and general aviation category aeroplanes that occurred from 2006 to 2019.
• Qualitative analysis of the data using Insleys’s [16] MxFACS taxonomy (Appendix B) for the accidents and Hieminga’s [17] taxonomy (Appendix C) for the serious incidents.
• Collect and analyse data from Subject Matter Experts (SMEs) in Nigerian Accident Investigation Bureau (AIB), Nigerian Civil Aviation Authority (NCAA) and the maintenance engineers practising in Nigeria.
• Identify root causes of the analysed accidents, serious incidents and occurrences via survey capturing the views of Aircraft Maintenance Engineers in Nigeria about the potential mitigating measures to prevent recurrence.

1.3. Research Structure

A research structure as demonstrated in Figure 1 was set out with all the challenges and previous studies listed above. It details the steps that were followed in order to achieve clarification on different issues previously raised and to possibly reveal new information from incidents analysed.

This structure shows the process necessary to be followed in order to achieve the research aim and objectives. Following a literature review, a background study was carried out on previous studies showing existing taxonomies, their advantages and disadvantages. The choice to select taxonomies developed by Insley [16] and Hieminga [17] was due to their acceptable level of inter-rater concordance. It was also to check if the results would coincide with research previously carried out in other countries.

Finally, it was to determine if developing a taxonomy in Nigeria for the identification of maintenance-related incidents and accidents, with the contribution of stakeholders in the industry would aid in identifying and predicting future events.

Responses from SMEs were collected via online questionnaires (Appendix D) in order to further understand the results of the accident/serious incident and occurrence data analysis and discover the root causes behind some discrepancies noted. The responses from the SMEs aided in gathering more detailed information of the method used and what is currently occurring in the industry. After the analysis of SME data, a survey was carried out to capture the views of maintenance engineers to further understand the issues raised by SMEs.

2. Literature Review

2.1. Errors, Classifications and Taxonomies

The term “Error” has been defined in various ways and while there is a general understanding of the term, there is no universal definition [18]. For the purpose of the present study, one applicable definition of error as defined by Reason (1995) is:

“An error is the failure of planned actions to achieve their desired goal”.

Errors do not occur randomly and can be controlled effectively. Deviations of different kinds are involved in all errors. These deviations may either be connected to an adequate plan with unintended associated actions or adequate actions with inadequate planning for the outcome intended [19]. The starting point of an investigation is human error, where the investigation highlights what errors should be focused on [20]. Nevertheless, the ultimate focus of any investigation should be not to apportion blame but to identify the organisational, environmental causes of errors so that mitigation measures can be put in place.

The term classification can be defined as a “spatial, temporal or spatio-temporal segmentation of the world”. A classification system can be described as a set of metaphorical or other kinds of boxes that things can be put in, to do some kind of work, either bureaucratic or knowledge production [21].

Lambe [22] defines the term taxonomy as the rules or conventions of order or arrangement, where an effective taxonomy has key attributes of being a classification scheme, semantic and a knowledge map.

Various error classifications are designed based on what is in need [19]. Classification systems are expected to meet criteria such as classifying according to origin, mutually exclusive categories and completeness. No working classification as accurately met all these requirements at once [23].

According to Dekker [20], the intent of error classification tools is as uncomplicated in principle as the tools are laborious in implementation. Their simplicity is due to the ease at which humans can manipulate them as they are basic to the consciousness of humans, their complexity can be attributed to the various ways in which they can be used, making the outcomes subjective and inconsistent. The main purpose of these tools should go beyond focusing on the peripheral error and further probe the system for root cause of the occurrence [20]. Although the intent of error classification is understood, Dekker argues that there is limited clarification as regards to reasons behind choices made by an investigator when using error classification tools to analyse accidents or incidents.

To develop an extensive accident or incident reporting system, a taxonomy that takes various causes of human errors into consideration must be provided [24]. The context in which these events occur should also be taken into consideration. In this case, aviation-maintenance-related events are the areas of application.

2.2. Taxonomies Currently in Use

According to ICAO, development of common terms, definitions and taxonomies for safety reporting systems in aviation would bring about worldwide coordination. It would also remove the constraints of aviation safety analysis that are caused by lack of common global descriptors. Without data standards, the value of safety information would be diminished, and different descriptions would result in creation of various contrasting efforts [25].

The most widely used taxonomies in the aviation industry are ICAO Accident/Incident Data Reporting (ADREP), Maintenance Error Decision Aid (MEDA), and Human Factors Analysis and Classification System Maintenance Extension (HFACS-ME). The validity of the categorisation using these taxonomies is highly dependent on gathering key information in detail.

2.2.1. ICAO ADREP

The Accident/Incident Data Reporting program (ADREP) taxonomy was developed by ICAO in 2000 and combined with European Coordination Centre for Accident Incident Reporting Systems (ECCAIRS) taxonomy in 2013. It is used globally for safety related events categorisation and description [26].

It is a combination of several taxonomies, some of which are descriptive factors, events, phases, occurrence category, organisation, category of aircraft, etc. However, there are seven basic categories in ADREP [26].

In Europe, the combined taxonomy is used for the Mandatory incident reporting which is ultimately managed by ECCAIRS. However, it does not contain a structure for initial and continuing airworthiness data collection [27].

It is very broad and known to be complex and sometimes, difficult to use when categorising data, however, Cheng et al. argue that it is the most complete aviation safety event taxonomy ICAO has developed [28].

2.2.2. HFACS

The framework for the Human Factors Analysis and Classification System (HFACS) is used for the classification and identification of contributory factors to accident occurrences. It has been widely used in various industries such as marine, rail, road and healthcare [29].

The HFACS-ME taxonomy includes the maintenance extension which is used to classify causal factors contributing to maintenance-related aviation occurrences [30]. It is a very useful taxonomy for identifying maintenance-related incidents, especially where there is sufficient human factors details.

2.2.3. MEDA

The Maintenance Error Decision Aid (MEDA), though reactive in nature, is widely accepted by aviation personnel and used to investigate factors contributing to maintenance-related accidents and incidents [30].

It was developed by Boeing along with stakeholders in the industry in 1992 to further understand issues related to maintenance [31].

2.2.4. MxFACS Taxonomy

In 2018, Insley reviewed and analysed 112 aircraft maintenance-related accidents and serious incidents which occurred between 2003 and 2017. The data were obtained from the Aviation Safety Network database and aimed to provide a better understanding of the causal and contributory factors. This study enabled the development of a new taxonomy (MXFACS) and the structure of the taxonomy was based on Bowtie methodology and included three levels. The first level descriptors indicated the outcome of the event, the second level focused on system/component failures while the third level aimed to identify causal factors in the maintenance environment. Some accident reports contained obvious maintenance-related errors [16].

2.2.5. Hieminga’s Taxonomy

In 2018, 1232 mandatory occurrence reports from the European Central Repository were analysed to develop a “two-level taxonomy”. The structure of the taxonomy included two levels. Level 1 was based on high-level overview of the maintenance processes while the level 2 descriptors included more granular coding to aim to determine potential causal and contributing factors [17].

2.3. Previous Studies Trends

The most noticeable and reoccurring issue throughout the previous studies is the presence of omission errors in all events that have been analysed. In particular, installation error has been highlighted frequently throughout historical studies [10,16,17,32,33,34].

Although different phraseologies have been used in the various studies, the installation errors are attributed to incomplete maintenance, incorrect maintenance or inadequate installations. According to Johnston et al., installation errors are the most prevalent type of maintenance error [35].

2.4. Research Rationale

Although aircraft maintenance errors do not account for a large portion of aviation-related accidents and fatalities, there is still a highly visible contribution to various events. Some of these events have been fatal, leading to the loss of lives, property and confidence in the industry [36]. Taxonomies are used to highlight and categorise event outcomes as well as causal factors to identify trends and focus on key areas to prevent recurrence [37].

We believe this study provides new knowledge about the potential measures to prevent aircraft maintenance-related accidents, serious incidents and occurrences in the future for the following reasons. Firstly, there is no publicly available literature that specifically focuses on the analysis of aircraft maintenance-related events in Nigeria. Secondly, the most recent analysis of accidents in Nigeria, which was carried out by Daramola [6], covered events from 1985 to 2010. Therefore, this study offers an up to date analysis of events since 2010 as well. Thirdly, the study also offers the analysis of occurrence reports which are received by the Nigerian CAA and are not publicly available. Therefore, this study would also help future researchers gather information on occurrence reports in Nigeria, which are not publicly available.

The most common taxonomies being used globally can sometimes be complicated when trying to analyse events in Nigeria. Furthermore, it is beneficial for the Nigerian aviation industry and relevant stakeholders to be aware about the importance of analysing events over a period of time in the most suitable way they can. This would aid in identifying trends and preventing future events.

3. Method

The data used in this study were gathered from three sources, they are:

(a)

Accident investigation reports available to the public via the NAIB website.

(b)

Mandatory occurrence reports which are only available to NCAA staff.

(c)

Survey responses from Subject Matter Experts (SMEs).

3.1. Accident Analysis with MxFACS

Insley’s MxFACS was selected to analyse the accident reports. The taxonomy consists of a three-level hierarchy:

(a)

First level—Event Outcome

(b)

Second level—System/Component Failure

(c)

Third level—The maintenance contributing factors that led to the system/component failure and the ultimate event.

The MxFACS taxonomy makes use of the Bowtie Risk Assessment Model to identify risks, causal and contributory factors. The three levels are derived from the “top event” element, “consequence” element and “threats” element, respectively, as shown in Figure 2. The maintenance error is taken as an equivalent of the “hazard” in each accident. It aids in identifying the action, the outcome and the context which are three of four basic elements of error [16].

The data contained in the Accident Investigation Bureau (AIB) publications were in PDF. Therefore, each report published in the last decade was downloaded. The documents were thoroughly reviewed in order to identify the maintenance-related accidents. After identification, each accident report was analysed and coded by using the MxFACS taxonomy structure.

3.2. Incident Analysis with Hieminga’s Maintenance Incident Taxonomy

Hieminga’s taxonomy was selected to analyse the reports which were sourced from the Safety Deficiency Incidence Analysis (SDIA) mandatory occurrence report database. The taxonomy consists of a two-level hierarchy:

• First level: This level follows a logical maintenance process, i.e., from planning to preparation. It also considers general category issues which are excluded from other categories. It follows through to different tasks and concludes with a job close up.
• Second level: This level is also a logical practical maintenance process and is comprised of as many substructures present as possible.

Although it gives the reporter the opportunity to select a descriptor that is as precise as possible due to its broad spectrum [17], it is a very broad coding system and could confuse maintenance personnel. This is because one incident could be coded into more than one first and second level category. This taxonomy utilises familiar words for classification which makes it easier for maintenance personnel to report incidents appropriately.

The data contained in the SDIA dataset were thoroughly analysed. All maintenance-related incidents were identified and classified in accordance with Hieminga’s template.

3.3. Collection of Data: Accident Investigation Reports Available to the Public

The process of collecting maintenance-related accident data involved downloading all accident reports available to the public. All commercial aircraft category and general aviation category accident reports published in the last decade, i.e., 2009–2019, were identified and downloaded.

There was a total of 70 accidents published in Portable Document Format (PDF). In order to identify the maintenance-related accidents, each document was analysed by the lead author. The next step was to identify the maintenance-related accidents and compile them in a dataset. All accident investigation reports that occurred in Nigeria were sourced from Nigeria’s Accident Investigation Bureau’s (AIB) publication database.

The AIB’s database contains official documented reports of all previous civil accidents in Nigeria. It is the only aviation parastatal in Nigeria with the principal responsibility of performing independent investigations into aviation accidents and making safety recommendations to the relevant agencies [38] and this satisfies the standards and recommended practices defined in ICAO Annex 13.

3.4. Collection of Data: Mandatory Occurrence Reports (MORs)

The MORs were sourced from the Nigerian Civil Aviation Authority (NCAA). In particular, it was received from the Safety Deficiency Incidence Analysis (SDIA) unit under the Directorate of Airworthiness Standards (DAWS).

The NCAA was established in 1999 to comply with ICAO’s requirements. ICAO required every member state to set up an organisation with the responsibility of ensuring compliance with air navigation rules [6]. NCAA is responsible for the safety oversight of the aviation industry in Nigeria.

All civil aviation events that meet the criteria for reportable occurrences defined in the NCARs are reported to the NCAA through an MOR form. These incidents must be reported within 72 h of its occurrence [3]. The incidents are then analysed, uploaded to SDIA’s Google Drive database and monitored till closure.

A total of 2530 incidents from 2006 to 2019 were stored in the SDIA MOR database. The data were de-identified before analysis for the purpose of confidentiality. All the incidents were analysed to discern maintenance-related incidents. They were compiled in a dataset and reviewed to ensure that only maintenance-related events were considered.

3.5. Collection of Data: Subject Matter Experts (SMEs)

Feedback, clarification and recommendation were sought from three different groups of SME. This was to acquire relevant information for recommending next steps of the data analysis. Another reason was to create awareness of the process of developing customised taxonomy for maintenance-related accidents and incidents in Nigeria. The three different groups of SMEs were;

(a)

Four Aviation Safety Inspectors at the NCAA SDIA unit;

(b)

Seven Air Safety Investigators from the NAIB;

(c)

Twenty-five Aircraft Maintenance Engineers (AMEs) practising in Nigeria.

The data were collected by sending links of the three different surveys to three different groups. This was distributed via online questionnaires (shown in Appendix D). These were administered by the Qualtrics software.

The first questionnaire contained four open-ended questions. This was responded to by NCAA SME participants. Their experience of collecting and analysing the MORs in Nigeria made them adequately qualified to contribute to this study. The questions covered the following topics:

• Experience and view on current taxonomy being used;
• Assessment of the study’s methodology and data output;
• Suggestions of other taxonomies to be used and possibility of developing customised taxonomy in Nigeria;
• Recommendation of adequate methods to further predict and make adequate safety plans by using the results of this data.

The second questionnaire contained three open-ended questions. This was responded to by AIB SME participants. Their wealth of experience in carrying out accident investigations, developing the reports and publishing them made them adequately qualified to contribute to this study. The questions covered the following topics:

• Experience on carrying out long-term reviews of previous accident reports and views about the benefit of such reviews.
• The depth of human factors training received and the availability of human factors experts within the AIB.
• Assessment of the study’s methodology, data output and recommendation for improvements.

The third questionnaire contained three open-ended and five multiple-choice questions. It was distributed to AMEs practising in Nigeria. A total of 25 responses were received. The question covered topics related to:

• Type and years of experience;
• Prioritising the identified maintenance contributory factors likely to cause future accident, by using a scale;
• Experience, challenges faced and opinion of following maintenance instructions;
• Experience, challenges faced and opinion of inspection instructions adequacy.

All the questionnaires were sent along with a PowerPoint presentation containing the aims, objectives and methodology. The first two surveys were accompanied by an Excel worksheet containing the taxonomies used and sample data to enable them to code the data using the taxonomies. The data output of this study was also presented to the first two SME groups.

These questionnaires were also sent out due to the limited data available on Nigeria aviation industry. An inductive approach was used to gather the information for this qualitative aspect [39].

3.6. Data Analysis

Although there are various well-known taxonomies currently being used to code accidents and incidents datasets, Insley’s MxFACS taxonomy and Hieminga’s taxonomy were used to code the accidents and serious incidents dataset. According to Hieminga (2018), having reviewed the taxonomies used in “Maintenance Error Decision Aid” (MEDA), which is an investigation tool developed by Boeing, HFACS-ME Framework, CAA Paper 2009/05 [33] and CAP 1367 [32]—none of these appeared to inhibit the two scales of adequate “usability” and “comprehensiveness” at the same time. This led to the solution of developing a different taxonomy to aid in coding incident events.

Another reason why a new taxonomy was not developed for the analysis of accidents and incidents in Nigeria was due to inadequate standard phraseology present in the dataset analysed. It can be argued that although the thematic analysis provides flexibility for the researcher, that same flexibility could lead to inconsistency and a lack of coherence when developing themes from a dataset.

Thematic analysis is a suitable qualitative research method that can be used in a wide range of analysing large qualitative datasets. Its advantage is that trustworthy and insightful findings can be produced using this method [40]. It has also been described as a method used to identify, organise and describe reporting themes within a dataset [41]. This was the main method of qualitative analysis used for the SME’s survey.

3.7. Analysis of SME Survey

The NVivo 12 plus is a qualitative data analysis software and it was the primary software used for survey analysis. All the survey questions were downloaded from Qualtrics in Microsoft Word format. These were then uploaded to the software as separate projects. These projects were analysed individually for identification of reoccurring words and themes in each response to the question. The responses were to aid in the methodology used, provide in-depth information and guidance in the study. The analysis did not focus only on responses tallied with the questions asked. This was to avoid missing out on new and important information. This is an inductive analysis method which allows researchers to code data without bias [41].

3.8. Evaluation of Research Rigour

According to Brink, a valid study demonstrates what is in existence, a valid measurement and measures what it was created to measure. A reliable study should produce the same results consistently when repeated by a different researcher [42]. However, these terms are difficult to apply to qualitative research methods when compared to quantitative research methods. Rigour is a more suitable term to measure validity and reliability of a quantitative research method [43].

Although Liamputtong and Ezzy argue that it does not perfectly verify the reliability and validity of qualitative research as stated above, coherence between different researchers is important to provide meaningful information. Inter-rater reliability or inter-rater concordance can be used to assess the level of coherence between two or more researchers. Cohen’s kappa is the most commonly used measure for assessing this match [44].

To assess the proportion of coherence and corrected for chance, Cohen’s Kappa was used in this study. Equation (1) shows how it is derived, Equations (2) and (3) show how formula components are determined.

$$\kappa =\frac{P_o - P_e}{1-P_e}$$

(1)

$$P_o=\frac{{{\displaystyle \sum }}_{i=1}^{n}R }{n}$$

(2)

$$P_e=\frac{{{\displaystyle \sum }}_{i=1}^n\frac{c_i \times r_i}{n}}{n}$$

(3)

where $\kappa$ = Cohen’s Kappa, $P_o$ joint probability of agreement, $P_e$ = chance agreement, $R$ = rater agreements, $n$= total number of ratings, $c$ = column marginal and $r$ = row marginal.

To evaluate the research rigour, SMEs from the NCAA coded a sample dataset using Hieminga’s maintenance incident taxonomy. This dataset was selected from one year and all the information was cleaned. The SMEs from the AIB coded all the maintenance error accidents identified using the MxFACS taxonomy. The researcher and SME’s coding were compared to determine Cohen’s Kappa using IBM SPSS V.25 statistics software.

3.9. Ethical Considerations

To protect the rights of participants, especially when conducting a qualitative research, certain moral and ethical problems could be encountered [45]. The study included collection of data from subject matter experts. It was therefore crucial to initially seek participants consent, inform them about their participation being voluntary and ensure their anonymity and confidentiality. Only relevant components were assessed. The university granted research ethical approval after it was requested for.

4. Results

4.1. Reliability of Taxonomies Used

McHugh (2012) interprets the level of agreement of Cohen’s Kappa [46]. This is shown in

Table 3. Cohen’s Kappa Level of Agreement. Adapted from: McHugh (2012).

Value of Kappa	Level of Agreement	Percentage of Data that Are Reliable
0–0.20	None	0−4%
0.21–0.39	Minimal	4−15%
0.40–0.59	Weak	15−35%
0.60–0.79	Moderate	35–63%
0.80–0.90	Strong	64–81%
Above 0.90	Almost Perfect	82−100%

below.

Table 4. Derived agreement statistics in all levels for researcher and AIB Subject Matter Expert (SME).

Researcher and SMEs	$\mathit{\kappa }$	${\mathit{P}}_{\mathit{o}}$	${\mathit{P}}_{\mathit{e}}$
Level 1	0.70	0.70	0.001
Level 2	0.80	0.80	0.002
Level 3	1	1	0.0001

below shows the Kappa value of this research’s inter-rater concordance with the AIB SME.

Table 5. Derived agreement statistics in all levels for researcher and AIB SME.

Researcher and SME	$\mathit{\kappa }$	${\mathit{P}}_{\mathit{o}}$	${\mathit{P}}_{\mathit{e}}$
Level 1	0.70	0.70	0.001
Level 2	0.80	0.80	0.002

shows the Kappa value of this research’s inter-rater concordance with the NCAA SME.

The values showed a moderate to strong level of coherence between the researcher and SME when the MxFACS taxonomy was used to categorise maintenance-related accidents. This suggests that there is a strong research rigour when compared to the levels as shown by McHugh [46].

The values also show a moderate to strong level of coherence between the researcher and SME when Hieminga’s maintenance incidents taxonomy is used to categorise maintenance-related incidents. This suggests that there is a strong research rigour when compared to the levels as shown by McHugh [46].

4.2. MxFACS Level 1—Event Outcome

A total of 70 accident reports were analysed, however, only 11 of them were identified with maintenance-related causal or contributory factors. These 11 events were then categorised in accordance with the MxFACS taxonomy to highlight the nature of the event. A new coding of “other” was created and added to the taxonomy to categorise maintenance-related events that did not match any of the themes.

Table 6. Level 1 (Event Outcome).

Event Outcome	n	% Fatality
Cabin fume event
Insulation blanket fumes
Collision
Collision with building
Collision with terrain	2	18
Collision with water
Collision with another aircraft
Depressurisation
Progressive depressurisation
Rapid depressurisation
Diversion or air turn back
Air turn back
Diversion	1
Fire
In-flight fire	1
On-ground fire	1
In-flight shut down
In-flight engine shut down
Landing-related event
Approach and landing without auto flight assistance
Degraded hydraulic system functionality during landing
Ditching
Engine failure upon landing
Forced landing	3
Hard landing
Landing short of runway
Wheels-up landing	2
LandingGear-related event
In-flight LG-related event
On-ground LG-related event
Runway-related event
Runway excursion
Structural damage
Empennage damage
Lower fuselage structural damage
Wing structural damage
Other	1

Shows level 1 (Event Outcome) in detail

The data highlight that in all maintenance-related Event Outcomes, only terrain collision led to a fatal accident, which was 18% of all maintenance-related accidents in the last decade [47]. This correlates with EASA [7] which identified terrain collision as one of the KRAs, further supported by the global accident/serious incident review in 2018 [36].

Surprisingly, runway excursion and ground collision, which have a high potential to be caused by maintenance error, were not affected in the last decade [7]. Only 16 percent of all accidents in the last decade are related to maintenance error. This could be due to inadequate focus given to this safety critical aspect of aviation.

4.3. MxFACS Level 2—System/Component Failures

All 11 accident reports were further coded to identify the top-level system/component affected. This is shown in

Table 7. Level 2 System/Component Failures.

System/Component Failures	n
Electrical power	0
Engine	3
Flight controls	0
Fuel	0
Instrumentation and indication	0
Insulation	0
Landing gear	0
Pressurisation	0
Steering	0
Structure	6
Windscreen	0
Other	2

.

A new coding of “other” was also created in level 2 and added to the taxonomy to categorise other maintenance-error-related events that did not match any of the themes. Level 2 (System/Component Failures) shows that engine, aircraft structure and “other” were directly affected by maintenance issues.

The UK Civil Aviation Authority [32] identified Air Transport Association (ATA) Specification 100 Chapter 32 “Landing Gear” as the second most affected system out of the top six events related to maintenance error. However, no landing gear system was identified as being affected by maintenance error in the last decade. This may be caused by the limited attention being given to maintenance errors in Nigeria. This can be improved upon by the AIB targeting maintenance, airworthiness and human factors causal and contributory factors when carrying out investigations. It also highlights why NCAA should include maintenance-related causal and contributory factors in data analysis. It may also be due to adequate inspection and testing carried out on such a visible area as the landing gear.

The accidents were further coded in accordance with MxFACS level 3 to identify the nature of aircraft maintenance errors that contribute to accidents in Nigeria.

4.4. Level 3—Maintenance Errors

All 11 accident reports were further coded to identify the top-level system/component affected and the codes can be seen in

Table 8. Level 3 Maintenance Errors.

Maintenance Factors	n
Airworthiness directive	1
AMM (Aircraft Maintenance Manual)	3
Check	0
FOD (Foreign Object Debris)	1
Human Factors	1
Inadequate maintenance	0
Incorrect maintenance	0
Inspection	3
Organisational	1
Overhaul	0
Oversight	7

.

Inadequate and incorrect maintenance were not identified as causal or contributory factors in any of the accidents analysed. However, research has highlighted that these types of omission and commission errors are common in aviation maintenance [19,48]. The absence of these errors may be due to inadequate information provided to the AIB by the personnel involved. Another plausible reason may be due to the difference in phraseology used in taxonomies.

The results show a high presence of Failure to follow procedures which can be classified under “AMM” and “Airworthiness Directive”. Failure to follow procedure is a maintenance issue as well as an organisational issue. This is because every team member, including managers who mount pressure on AMEs, are involved in the maintenance chain [49]. This is also in agreement with organisational factors being one of the maintenance contributory factors. This is a growing concern in the aviation maintenance area.

There was a surprisingly low number of human-factor-related errors in the accident events. During the categorisation, no assumptions were made, hence where there were no human factors mentioned, it was not categorised. According to Sarter and Alexander, human error contributed to 70% of major aviation accidents [50]. The analysis highlights discrepancies in limited human factors considerations during investigations.

Oversight by operators and authorities was identified as the highest causal/contributory factor to maintenance-related events. Research by Dhillon showed that operators and regulator’s oversight led to some fatal maintenance-related accidents [51]. Oversight should be a critical aspect of aviation as it aids all personnel involved to obtain second views and opinions.

4.5. Analysis of Mandatory Occurrence Report by Using Hieminga’s Taxonomy

All the incidents in the SDIA MOR database were coded in accordance with Hieminga maintenance incidents taxonomy. The results show that a total of 588 incidents were related to maintenance error. This is about 23% of all incidents which occurred from 2006 to 2019. The breakdown of total number of MORs, total number of maintenance-related MORs and the number of most frequent Level 1 category events is shown in Figure 3. There are several findings which are related to the scope of this study.

Firstly, the total number of MORs fluctuates and does not follow a pattern. When such statistics are presented, one important consideration is to normalise the numbers by presenting them as rate of occurrences based on traffic numbers. Nevertheless, it is not always possible to determine the cause of certain data points such as 2014 being the lowest number of mandatory occurrence reports the Nigerian CAA has received.

Secondly, the rate of the total number of maintenance-related MORs has increased in recent years. Particularly, as shown in

Table 9. Analysis of Mandatory Occurrence Reports (2006–2019).

Year	Total Number of MORs	Total Number of Mx MORs	The Rate of Mx MORs	Highest Number of Level 1 Coding	Highest Number of Level 1 Coding
2006	175	18	10%	6	Inspection/testing issue
2007	147	11	7%	8	Inspection/testing issue
2008	185	21	11%	9	Removal/Installation issue
2009	266	51	19%	18	Inspection/testing issue
2010	216	66	31%	14	Removal/Installation issue
2011	190	40	21%	14	Working practices
2012	188	38	20%	12	Removal/Installation issue
2013	267	31	12%	12	Removal/Installation issue
2014	55	12	22%	7	Removal/Installation issue
2015	272	56	21%	26	Removal/Installation issue
2016	97	31	32%	20	Removal/Installation issue
2017	191	82	43%	42	Removal/Installation issue
2018	188	90	48%	59	Removal/Installation issue
2019	93	40	43%	28	Removal/Installation issue

, the maintenance-related MORs were more than 40% of the total number of MORs within the last three years the data set covered. While such statistics may be concerned, it is important to understand the further analysis of the data and identify the causal and contributory factors.

Thirdly, one clear finding from this analysis was that “removal/installation issues” have been the most frequent event category. For example, out of 588 maintenance-related MORs in total, 259 of these events were caused by “removal/installation issues”. Furthermore, this category appeared as the “most frequent” event within the last 8 years and ten times within the 14 years the dataset covered. This is certainly in alignment with several other studies including the ones which led to the development of two taxonomies used in this study [34].

The “removal/installation issues” category was followed by working practices which contained most errors related to accumulation of dirt and contamination. This may be caused by the dry season weather conditions in Nigeria. Studies have shown that dust would contribute to aviation safety through corrosion, blockage of Pitot-static tube, etc. [52]. Inspection and testing issues also had a high contribution to maintenance-related incidents, followed by job close up, lubricating and servicing. This correlates with previous studies carried out by Latorella and Prabhu [53].

4.6. Results of the SME Survey

Information was gathered from the responses to the questionnaires. Themes were identified and nodes created to extract the most important elements of the survey. The information identified from three different groups is written below along with brief discussions on them.

4.6.1. NCAA SDIA SME RESPONSE

• Experience and view of current taxonomies

The respondents gave information related to the process of receiving, uploading, investigating, monitoring and closing incidents reported to them.

All the respondents stated that the ADREP taxonomy is currently being used to analyse mandatory occurrence reports. It is not suitable for capturing maintenance errors because it is very broad.

2.

Assessment of study methodology

The respondents were in support of the methodology used, however, some of them suggested that the taxonomy be narrowed down to avoid a cumbersome process of classifying incidents

3.

Suggestions on other taxonomies or developing one and recommendations for safety plans

Some of the respondents suggested that MEDA or bowtie analysis with a risk matrix be used to identify maintenance errors. Some respondents agreed to the development of maintenance error taxonomies for Nigeria.

There was information related to authenticity of reports sent to the NCAA, the phraseology being used by the maintenance engineers in the industry, insufficient data provided to the SMEs and reluctance to report incidents.

The respondents recommended a standardised online reporting system, organising safety symposiums, workshops and creating awareness of the presence of just culture.

4.6.2. AIB SME RESPONSE

• Experience of analysis previous reports within a period, e.g., a decade

All the AIB respondents agreed that analysis was carried out on previous reports, however, they have not experienced such analysis being carried out since 2009. The previous analysis carried out are not available to the public.

2.

Depth of human factors knowledge and availability of human factors personnel

The respondents stated that although their AIB commissioner is focused on human capacity building and ensures they are adequately trained, there is a need for human factors training for the AIB investigators. There is currently one human factors expert who was recruited in July 2019.

3.

Assessment of study methodology and recommendation for improvements

4.6.3. AME SME RESPONSE

• Experience and challenges faced following maintenance instructions

Seventy-five percent of the respondents suggested maintenance instructions were not always practicable to follow. Some of the challenges faced by them were:

Some manufacturer’s manuals and some aircraft type were more complicated than normal.

Time constraint, human factors involvement, inadequate special tools and inadequate access to internet to update manuals.

2.

Experience and challenges with inspection instruction adequacy

Eighty-five percent of the respondent stated that some maintenance instructions are inadequate. They face challenges such as ambiguity and incomplete instructions.

They recommended developing a process of sending feedback to the manufacturers through the most appropriate means.

5. Discussion

This research has demonstrated the nature of aircraft maintenance errors that significantly contribute to or the cause accidents and serious incidents in Nigeria. Factors such as failure to follow procedures in manuals, human factors, foreign object damage, inadequate inspection and operator and regulatory oversight were identified as the nature of maintenance errors that lead to accidents, while job close up, installation/removal, lubrication/servicing, working practice and maintenance control were identified as the nature of events leading to serious incidents in Nigeria. The SME surveys highlighted various challenges faced and recommendations. Considering that the scope of this study is the nature of maintenance errors, these are hereby analysed critically, arguments are developed and comparison with previous studies carried out.

5.1. MxFACS—All Levels Discussion

In the accident analysis, 16 percent of all the accidents in the last decade are maintenance-error-related. This is slightly higher than the previous studies carried out that show that 12% of major aircraft accidents are caused by maintenance discrepancies [54,55,56,57].

A probable explanation for this could be due to the type of maintenance culture that exists in Nigeria. A study carried out by Olufunke revealed that there is a need to emphasise the importance of maintenance culture to Nigerians in every industry [58]. She also highlighted how maintenance personnel should be highly valued and importance be given to them to motivate them.

Another probable reason for this could be the high rate of traffic in Nigeria in the last decade [55]. This corresponds with findings of studies carried out by Saleh et al. which revealed that from 2005 to 2015, 14–21% of helicopter accidents in the US were related to flawed maintenance and inspection [59].

Airworthiness directive and AMM were identified as contributory factors. These could be classified under failure to follow procedures in manuals. A probable explanation for this which is taken from the SME survey, could be due to some of these documents being complex or cumbersome. According to Drury and Johnson, “procedures not followed” is now frequent in incident and accident reports in aviation. Findings from an Federal Aviation Administration (FAA) study by Johnson and Watson revealed that during a heavy maintenance check carried out within 90 days, the number one factor that caused major malfunctions was failing to comply with maintenance documents [60]. This corresponds with the findings of this study.

Another probable cause could be loss of confidence in the document as any error found by the end user would decrease the user’s confidence in the document [61]. It could also be over-confidence of highly experienced maintenance engineers in carrying out simpler tasks.

Inadequate maintenance and incorrect maintenance had no contribution to any of the accidents analysed. This is surprisingly one of the common commission errors in aircraft maintenance (Reason and Hobbs, 2003) which does not correspond to this study. A probable cause could be the terms and phrases used in the reports. Another probable cause could be that aircraft maintenance errors did not receive adequate attention [53].

Inadequate inspection being identified as one of the causal or contributory factors to the accidents is not surprising. A probable cause for this could be lack of required special tools as suggested by one of the SME respondents or improper use of tools provided. Another probable cause could be fatigue [62]. This correlates with this study’s findings. Without a fatigue risk management system in place for maintenance engineers in Nigeria, some organisations may tweak the laws regarding rest and duty limitations which do not consider commute time. This fatigue can be classified under human factors which was identified as one of the factors contributing to accidents.

Foreign Object Damage (FOD) was identified as one of the factors contributing to accidents. Studies carried out by Hussin et al. revealed FOD is a rising concern in the aviation industry [63]. An analysis of events that occurred from 1998 to 2008 was carried out by Australian Transport Safety Board (ATSB); the results show that 116 events were caused by FOD. This corresponds with the findings of this research that FOD can contribute to events.

Operator and regulatory oversight were identified as the nature of maintenance errors that lead to accidents. A plausible explanation may be due to the low attention given to aircraft maintenance errors by both operator and regulatory body. Accident Investigation Bodies around the world regularly issue recommendations for the regulatory authorities to consider taking action in many different areas. Effective oversight can be one of these areas when the investigation identifies clear evidence of ineffective oversight the regulator or the operator. According to Drury, a report by the Federal Aviation Administration (FAA) recommended increased regulatory oversight for repair stations [64]. This corresponds with the findings of this study. Another plausible explanation for operator oversight could be that the management is not balancing safety goals with production goals, which could lead to events [25].

5.2. Hieminga Maintenance Incidents Taxonomy Discussion—All Levels

In all the serious incidents analysed, 23% were attributed to maintenance errors. An analysis by Marais and Robichaud of 3242 incident reports showed that 10% can be attributed to maintenance error, which has remained constant in the past decade [65]. This correlates with the findings of this study about how maintenance-related errors cause or contribute to serious incidents.

Job close up, i.e., close up not performed correctly, was identified as one of the natures of maintenance errors. An example of this type of error identified was engine cowls not latched. This is an omission error which is common as stated earlier. An analysis of accidents cause by Cowan et al. revealed that maintenance errors such as leaving engine cowl unlatched could lead to separation during flight, this causes structural failures. This corresponds with the findings of this study. A probable cause for this could also be fatigue as explained earlier [66].

Installation and removal was also identified as one of the natures of maintenance errors contributing to incidents. A plausible explanation for this could be carrying out tasks without the approved document as discussed earlier. In recent years, “failing to follow procedures” has been identified by the FAA as a consistent causal factor and as a result an “online training programme” was developed to look at this issue holistically. Another plausible explanation could be the presence of an aging aircraft being operated as discussed earlier, which would require additional maintenance. This corresponds to the result of this finding. Inadequate oversight from operators could also lead to this type of error because quality control on aircraft maintenance helps to highlight discrepancies during audits [53].

Inspection testing was identified as a nature of maintenance-related error leading to accidents. A plausible cause for this could be inspection overdue or inadequate tools to carry out inspections. A study by Boeing revealed that 16% of hull loss and 20% of accidents that occurred from 1982 to 1991 could have been prevented by a change in maintenance inspection [54].

Working practice. A probable reason for working practice being a maintenance-related contributory factor may be due to organisation culture. According to Pettersen and Aase, operational work practice is part of the safety and regulatory systems of the industry but can be highly influenced by organisational framework [67]. This means that the personnel tend to formulate how tasks should be carried out and formulate grey zones within themselves. This eventually becomes normal especially during time pressure.

A probable cause for identifying lubrication and servicing as one of the natures of maintenance error could be using the wrong fluid, insufficient lubricant or servicing overdue. An example of a fatal accident related to this took place on the Alaska Airlines Flight 261. Insufficient lubrication of the jackscrew assembly led to thread failure; further contributing to this was extending the lubrication interval, which was approved by the FAA [40].

6. Conclusions

Prior to concluding this study, a number of challenges and limitations faced are addressed.

6.1. Challenges and Limitations of the Study

One key challenge faced during the study was gathering the mandatory occurrence report data because it was not available in the public domain and furthermore the protecting the identity of individuals and organisations involved are vitally important.

Another challenge was inadequate information in the incidents and accident reports. So many incidents had to be omitted due to insufficient information related to the scope of this study.

Gathering all three questionnaires in a short time was very challenging, however, it was possible to finish most of the analysis in time. Some parts of the data from the SMEs were not analysed due to time constraint.

The most important challenges faced during the study was the unavailability of adequate literature related to the aviation industry in Nigeria. This is one of the problems this study aims to solve.

The most significant limitation of this study is the fact that the coding of accidents/serious incidents as well as MORs was only validated by one SME for each category of data. Therefore statistical significance cannot be claimed; however, the nature of the study never aimed to be a quantitative approach and it is believed that the results can still provide real insight into the maintenance-related events and their potential causes and contributing factors.

6.2. Conclusion of the Study

The study shows the nature of aircraft maintenance errors that contribute to or cause accidents and incidents in scheduled commercial, non-scheduled commercial and general aviation category. The utilisation of the MxFACS and Hieminga’s maintenance incidence taxonomy yielded similar results with existing global research. It also highlighted how one maintenance error could be caused by another maintenance error within the same taxonomy. It also shows that utilising taxonomy can aid in predicting some future accidents. It may, however, not predict some occurrences as these taxonomies were created using hindsight.

The results of the study revealed that the most common maintenance-related “Event Outcomes” in the last decade are “collision with terrain” and “landing gear events”. The systems of components that were affected the most during accidents are the aircraft engine and structure. The maintenance factors with the highest contribution to these accidents are operator and regulatory oversight, inadequate inspection and failure to follow procedures.

The research also highlights the highest causal and contributory factors to aviation incidents in Nigeria from 2006 to 2019 are installation/removal issues, inspection/testing issues, working practices, job close up, lubrication and servicing. All of which corresponds to studies by other researchers in other countries.

The trend over the years revealed that an increase in air traffic in Nigeria led to an increase in the number of maintenance-related incidents, however, it is worthy to note that data on air traffic from 2018 to 2019 was not available for comparison.

The study’s findings could contribute to the limited literature related to maintenance errors and incidents in Nigeria. It would also aid all relevant stakeholders in understanding the nature of errors that pose a threat to the safety performance in Nigeria.

7. Recommendations

With a deeper understanding of the challenges and suggestions provided by the SMEs, it can be recommended that a less complex taxonomy be developed for the identification and categorisation of maintenance-related events in Nigeria.

Oversight should target human error as much as they do for technical failures. This can help in predicting possible events and identifying trends to aid in implementing a risk-based oversight approach by the regulatory authority.

Additional Human Factors training would help particularly the inspectors and data analysts in the regulatory authorities and the accident investigators to focus on key human performance issues during the performance of their duties.