Learning from Incidents: A Qualitative Study in the Continuing Airworthiness Sector

Abstract

Learning from incidents (LFI) is a useful approach when examining past events and developing measures to prevent ensuing recurrence. Although the reporting of incidents in the aircraft maintenance and continuing airworthiness domain is well appointed, it is often unclear how the maximum effect of safety data can be efficaciously applied in support of LFI in the area. From semi-structured interviews, with thirty-four participants, the gathered data were thematically analyzed with the support of NVivo software. This study establishes a relationship between an incident in its lifecycle and the learning process. The main aim of this work is to elucidate factors that enable LFI. The analysis of the data revealed, for example, the benefits of a just culture and the use of formal continuation training programs in this respect. Moreover, it identified limitations inherent in current processes such as poor event causation and poorly designed learning syllabi. Additionally, aspects such as a lack of regulatory requirements for competence in the areas of learning for managers and accountable persons currently exist. This thematic analysis could be used in support of organizations examining their own processes for learning from incidents. Additionally, it can support the development of terms of reference for a continuing airworthiness regulatory working group to examine, strengthen and better apply LFI in the aviation industry.

1. Introduction

If it were possible for all organizations to learn effective lessons from the past, the effects of future unwelcome events might be limited [1]. Aviation safety depends to a large extent on the efficacious efforts of all involved in the system [2]. Research has acknowledged the importance of event information when it comes to learning and preventing recurrence [3]. Thankfully, major events such as accidents are becoming less frequent and generate less points for learning [4]. In contrast, there are numerous incidents with less severe consequences and if appropriately considered, these could offer an earlier insight into the circumstances that enable unwelcome events. Predefined and relevant information harvested from incident reporting systems is a major element of learning and preserving acceptable levels of safety. Hobbs and Williamson [5] highlight the importance of aircraft maintenance staff being aware of the cumulative effect of “seemingly insignificant” incidents as this amplifies the need to be proactive when it comes to learning from incidents. This research undertook a qualitative examination of staff involved in aircraft maintenance and continuing airworthiness operations in order to identify factors that could augment learning from incidents within this industry sector.

In the areas of continuing airworthiness and aircraft maintenance, safety management systems include incident and occurrence reporting [6] as an obligation. It is common for incidents to be discovered within organizations and reported with the assistance of such “systems of systems” [7]. On an operational level, initial training on human factors and company procedures is intended to specify and re-affirm the category and type of occurrence and incident that should be reported. Recent developments in European Union (EU) regulations [8] empower voluntary and confidential reporting and are independent of all other individual obligations. Detecting and identifying hazards highlighted through incident reporting systems is also recommended by the International Civil Aviation Organization (ICAO) standards and recommended practices as an effective means of augmenting levels of safety. However, Gerede [9] strongly suggests that a failure to foster a just culture is considered to have a negative impact upon effective data collection (reporting), organizational learning and the subsequent ability to learn from incidents.

Drupsteen and Wybo [10] reaffirm organizations use experience gained from past events in order to improve safety. Effective learning can be considered as a successful translation of safety information into knowledge. Utilizing information from events with learning potential can actively improve the operating environment and help prevent recurrence. Learning in this context can often be experienced as modifying or implementing new knowledge where cultural, technical or procedural elements are integrated. Therefore, when learning is transformed into measures to prevent re-occurrence, an organization often has a reasonable means of mitigating future similar events. Argyris and Schön [11] highlight the importance of learning to detect and address effective responses to errors. Their “theory in action” concept is the focal point for this determination. The first of its two components, “theory in use” is one that guides a person’s behavior. It is often “tacit” and is how people behave routinely. Very often these observed “habits” are unknown to the specimen. The second element is known as “espoused theory”, namely what people say or think they do. Drupsteen and Guldenmund [12] mention that espoused theory comprises of “the words we use to convey what we do, or what we like others to think we do”.

However, it is important to re-affirm the linkages that exist between individuals and organizational learning. The introduction of safety management systems (SMS) has initiated a shift in how organizational errors are viewed. Firstly, equipment has become increasingly more reliable, but the human form has not displayed the same response. In the second instance, the impact of complexities associated with an increasing cognitive load for staff is just beginning to be realized. The existence of a potential for blaming an individual is now being aligned with organizational responsibilities. Prior to this, event causation was often misrepresented or even over quantified the human input as organizational factors were not always considered. They offer an insight into the connection between individual actions and organizational initiatives designed to secure the best safety outcomes. Fogarty et al. [13] also recognize the role that both individual factors have on human error and the inputs both can have on preventing recurrence.

ICAO Doc 9859 ICAO [14] defines a template for aviation operators and regulators to support the application of a variety of proactive, predictive and reactive oversight methodologies. In addition to routine monitoring schemes, voluntary and mandatory reporting, post incident follow-up, there are also regular safety oversight audits. These audits and inspections often set out to establish if there is a difference between espoused theory and the theory in use (e.g., is the task being correctly performed in accordance with the documented procedure/work instruction or is there a deviation from approved data and practice?). However, Drupsteen and Guldenmund [12] caution auditors not to “focus too much on the documentation of procedures” alone. In such cases, the oversight audit may be ineffective because of its sole focus on espoused theories of the organization only and not the theory-in-use. These authors translate this idea of poor focus on theory in action, into a valid learning component arising from incidents. They also highlight the “espoused” aspect where those attempting to learn from incidents often fail to experience the desired learning because outcomes are not fully aligned with the practical objectives of a learning from incidents (LFI) initiative. For learning to be most effective, espoused theory and theory in use should be reasonably well aligned. Ward et al. [15] propose it is necessary to further develop an operational model that can account for “what is meant to happen and what actually happens”.

Continuing airworthiness and aircraft maintenance and activities performed in EU member states are subjected to rules that mandate reporting of defined issues. Repositories of reported data tend to be populated by sources that are predominantly the subject of mandatory reporting requirements. Conventional safety oversight models also only verify the presence of reporting media and repositories in this segment of the industry. Jacobsson et al. [16] avow the degree of interest invested in learning from incidents but question its efficiency in some organizations. Although unwelcome events are less prevalent, less severe events still provide learning opportunities. There is often only a primary focus for organizations upon reporting in line with each state’s own reporting obligations. Unfortunately, a narrow focus on this single element of an incident in its lifecycle can negate the potential benefits of learning from incidents at an organizational level. The absence of clearly defined competency requirements [6] that support a pedagogy for learning from incidents for continuing airworthiness staff could also be considered an impediment to effective learning in the domain.

The featured industry sector is regulated by the application and upkeep of numerous requirements in the jurisdictions of operation. In general, a costly regulatory overhead tends to be carried by regulating states and operators to support safe and viable activity. However, a growing tendency to increase regulatory requirements in pursuance of safer activity across the segments may not always offer the same returns as previously realized by states. Brunel [17] (p.45) suggests, “…it is impossible to make men perfect: the men will always remain the same as they are now and no legislation will make him have more presence of mind…”. Furniss et al. [18] reviewed the Hollnagel [19] Functional Resonance Analysis Method (FRAM) which explores how functional variability resonates within systems, i.e., how well comprising elements function in a system. They also consider how FRAM can be modified to support complex socio technical system improvements. Perhaps as the paradigm supporting the linearity of regulatory oversight shifts, proactive regulatory inputs will also influence more effective safety outputs as intricacy increases.

1.1. Systematic Literature Review

The primary reason for conducting a systematic review was to examine how learning from incidents occurs in aircraft maintenance and continuing airworthiness management. Other sectors and the issues impacting learning in these areas were also considered. The literature review sets out to establish factors that contribute to or potentially constrain learning from incidents in the subject domain. Applying a qualitative research approach is advantageous as it can provide a deeper contextual understanding of the literature and can assist with better research integration. The application of rigor and comprehensiveness can assist with advancing knowledge and identifying research gaps and aspects for further research in this area. Okoli and Schabram [20] suggest “a dedicated methodological approach is necessary in any kind of literature review”. A preliminary search of literature highlighted a scarcity of best-practice guidelines for conducting systematic literature reviews in this area.

Qualitative research involves handling considerable volumes of data and a degree of discipline is required so that search results, decisions regarding subject inclusions and exclusions are recorded and references are well managed. Endnote was used in support of the literature review during this research. An electronic database is useful for supporting a search strategy, arranging publications and storing references [21]. The qualitative data analysis software NVivo (NVivo 12, QSR International, Melbourne, Australia) was used to augment the data management, storage and analysis associated with the literature review. NVivo possesses many functions, such as facilitating the synthesis of a review [22]. A systematic search of in excess of 1000 publications was performed in the following databases: Web of Science, Scopus, IEEE Xplore, ProQuest and EBSCO. The following predefined search terms were applied: “learning from incidents”, “learning from experience”, “aircraft maintenance”, “aircraft management” and “safety management systems”. A practical screening of title and abstract was applied to each manuscript using predefined terms (e.g., subject, setting, publication, year). This part of the process had to be broad enough to create a sufficient number of applicable publications but also had to be practically manageable. The following criteria were implemented for the practical screening of the source bibliographic details, title and abstract:

• Subject—Related to learning from incidents and past experiences.
• Setting—Any high reliability industry or sector where learning from incidents is critical.
• Publication—Journal or peer-reviewed conference proceedings.
• Date range—published post 1992. The year 1992 was the starting point for the screening process, since at the time of planning the research project, 25 years was considered to be a reasonable timespan to include material pertaining to learning from incidents.

The output of the practical screen step produced a list of publications denoted as the screened set of publications. An Endnote library was then created to store and manage the full text of the retrieved publications. The next step involved filtering the publications into primary and secondary publication subsets using only primary research manuscripts in the next phase. Applying a set of criteria helps to reduce any researcher bias in the screening system. A set of inclusion and exclusion criteria [23] was developed in accordance with the guidelines included in [24,25], listed in

Table 1. Inclusion and exclusion criteria used for the filtering of the subset of primary publications.

Included	Excluded
Research studies	Literature reviews
Qualitative and mixed methods	Quantitative methods
Perceptions and experiences	Focused on decision-making and legislative requirements
Reference to just culture	Not about “no blame” or a punitive approach
High reliability settings	Non high reliability settings
Published post 1992
Peer-reviewed publications
Industry based settings
Original studies

. Two researchers were involved in the screening process.

The final set of 18 papers was imported into NVivo and the following analysis approach, as defined by Bandara et al. [22], was used for the selection of the codification themes:

• Deductive—themes reported on are predetermined to some extent. In this case, these predetermined themes were the output of a focus group process.
• Inductive—themes reported are derived from analysis of the literature.

NVivo is limited in terms of providing thematic classifications based on the occurrence of key words but can assist with identifying relationships between words and phrases amongst publications. It also provided thematic classifications of data based on the occurrence of key words and phrases. The coding process consisted of selecting relevant passages of text that were captured in one or several of the framework nodes. Maykut and Morehouse [26] defines a propositional statement as “a statement of fact the researcher tentatively proposes, based on the data”. Memos were used to draft these summary statements which formed part of the literature review. Central to the idea of learning is how an incident is generally moderated during its useful existence. Section 1.2 documents this approach.

1.2. The Notion of a Generic Incident Lifecycle

Figure 1 illustrates how an incident tends to be managed through its quiddity. This view is one possible way of representing the elements comprising a lifecycle view. Cooke and Rohleder [27] suggest it should also be evident that an incident system will operate most effectively when a safety management system has already been put in place and avoidable risks are addressed. They propose an effective system that addresses: identification and response, reporting, investigation, identifying causal structure, making recommendations, communicating and recalling incident learning, and implementing corrective actions. Drupsteen et al. [28] also consider an incident from a learning perspective in its cycle. Their main constituents are investigating and analyzing incidents, planning interventions, intervening and evaluating (each of these four stages are further sub-divided into eleven sub-components). Continuing airworthiness-related incidents are notified by way of a formal mechanism of reporting. During the data gathering phase of this research, the steps outlined in Figure 1 were found to be dictated by regulatory requirements [6,8]. Once the incident enters its lifecycle, it ideally transverses a process that transforms the information gathered into knowledge. Figure 1 and the contiguous paragraph offer an overview of how the capture and processing of the incident information occurs in practice.

Continuing airworthiness-related serious incidents are rare but often due to environmental, cognitive and mechanical demands, reportable and unreportable events do occur. All organizations in the industry segment subscribe to a reporting system and reports can be made electronically or in paper form in smaller organizations. The main underpinning regulation in Europe, EU regulation 2018/1139 [29], refers to a management system and mandates an organization to implement and maintain such a system to ensure compliance with these essential requirements. In practice, although a reporter can report events directly to an aviation authority, all organizations are required to have an internal reporting system also. A focal point/gate keeper will process these reports either internally and/or inform third party stakeholders such as aviation authority or aircraft manufacturer as required by procedure. Depending on the event, technical management may determine there are immediate actions required to recover a situation or restore serviceability. While a small number of scenarios will require an event to be investigated fully before an aircraft returns to service, many incidents are investigated post event. As soon as causation is established, if accepted by the relevant technical function, the report is closed. This management system is strongly influenced by regulatory requirements and procedural form and is a pre-eminent influence on how an incident and its actors behave from the time a report is made to the time its impact has been terminated. One of the limitations inherent in this cycle is that lessons tend to be delivered at a later point in time mostly through the medium of recurrent training programs such as continuation and human factors training. Therefore, there is often a hiatus in the feedback cycle. However, the effectiveness of the process and the perceived contribution to learning are not fully reflected in this view.

1.3. A Potential Learning Cycle Emerges

According to Lindberg et al. [30], in order to prevent accidents, it is essential to learn from previous accidents and incidents. Lukic et al. [31] suggest that in order to increase the effectiveness of learning from incidents, it is necessary to understand who should be included in the learning process. In Figure 2, the incident lifecycle is aligned with the learning process in order to highlight where potential improvements might be made. As the incident is managed and causation is established, there are potential avenues open for learning. The ultimate desired outcome is that adequate measures are put in place to prevent a recurrence of the event. However, the lessons available in a potential learning product are not always used to best effect when considering the Figure 1 process. Drupsteen et al. [28] state that “many incidents occur because organizations fail to learn from past lessons”, because the traditional approach often stops short of preventing future incidents. Their research examines: investigating and analyzing incidents, planning and prevention, intervening and evaluating steps in a learning process. Ward et al. [15] found that the resulting relationship between the individuals and the systems have a direct impact upon the system and prevailing environment. Silva et al. [3] examined how organizations use accident information to reduce the occurrence of unwelcome events. Drupsteen and Wybo [10] found that hindsight can determine if an organization did learn from an event but there are no models to assist with gauging the “propensity” of an organization to learn. Drupsteen and Hasle [1] suggest that learning can be improved if limiting factors are addressed.

The proposed enhancement (shown in Figure 2) to the generic lifecycle in the “traditional” approach represents a novel view and brings the learning product into focus. This figure highlights the benefits of ensuring the feedback loop of an incident is centered on the learning product. Treating its development as an iterative process ensures all steps in the cycle are included and where deficiencies are noted, they can be identified and communicated during the iterations. This can assist with delivery of timely and sustainable learning and help prevent an inability to think, talk and see what actions are proper in specific situations [32]. According to Drupsteen et al. [28], it is necessary to gain an insight into the steps of the process to identify factors that hinder learning in order to make improvements. The research suggests an emphasis on accessible, timely and appropriate learning content could provide all stakeholders in the process with better value for their efforts. Perhaps one reason that the customary incident lifecycle and its limitations prevail is related to management theory. While innovators like Taylor [33] are responsible for advances in management, such theories have not always fully considered safety and learning. The early 1900s witnessed a time when it was necessary to inaugurate efficiencies in production by initially decomposing tasks in order to introduce liner efficiencies. The limitations experienced in incident learning processes today may relate to this circumscribed tradition.

2. Materials and Methods

2.1. Philosophical Underpinnings

The fields of science and philosophy consider ontology and epistemology in terms of What is the nature of reality? and How is human knowledge constructed? The ascendant ideologies of positivism and interpretivism can be applied in support of these philosophical differences [34,35,36,37]. Hirschheim [38] puts forward the aim of positivism to, “seek to explain and predict what happens in the social world by searching for irregularities and causal relationships between its constituent elements”. In contrast to this stance, Schwandt [39] suggests the aim of interpretivism is to gain understanding. Interpretive research seeks to develop a richer understanding of the complex world of lived experience from the point of view of those who live in it. “This goal is variously spoken of as an abiding concern for the life world, for the emic point of view, for understanding meaning, for grasping the actor’s definition of a situation, for verstehen” [39] (p.118).

The intent of this qualitative study was to understand how various situations impact on learning from incidents in the aircraft maintenance and continuing airworthiness management domain by interacting with the participants on a social plane. Thus, in order to gain an empathetic understanding of the participants and their actions, the pursuit of “verstehen” considers adopting an interpretive paragon as an approach. This approach is not initiated with the aid of a hypothesis intended for testing but rather using a lodestar that guides the researcher to a point of discovery supported with an inductive modus operandi. The study is unwavering in its support for the view that (individual and combined) qualitative and quantitative approaches possess equal value in terms of their investigative potential in this area of focus. In summary, the project employs a qualitative research methodology in an effort to generate “rich” findings in support of gaining a good understanding of the learning environment in the featured domain. According to Maykut and Morehouse [26], the purpose of qualitative research is to discover the inner world of perceptions and meaning-making in order to gain an understanding to describe and explain certain social phenomenon from participants’ perspectives. In order to accomplish this, focus group activity was managed concurrently with the literature review. These activities cumulatively generated five themes which were used as the basis for a semi-structured interview template. The project employed a qualitative research methodology in an effort to generate “rich” findings in support of gaining a good understanding of the learning environment in the featured domain. The outcome of a qualitative research initiative was contextual findings as opposed to broad generalizations.

2.2. Focus Group

According to Kitzinger [40], “focus groups are group discussions organized to explore a specific set of issues such as people’s views and experiences…”. The idea of conducting group interviews is not a new one. Bogardus [41] is an early example of a reference to utilizing the group interview. Frey and Fontana [42] suggest that group interviews can be formally structured for a specific purpose or can be performed in a more informal setting where a researcher can “stimulate a group discussion”. Specific examples in the literature of focus groups being developed systematically within the area of aircraft maintenance and management are scarce. Frey and Fontana [42] state that although group interviews have implicitly informed research, often they are not formally acknowledged as part of the process. Powell and Single [43] remind us that when recruiting focus group participants, one must be mindful of systemic biases. Averting this was ensured by being careful to enlist the participants from different organizations and different positions of responsibility. Three sessions comprising of three industry professionals within each group were successfully moderated by the researcher. During the three phases of working with the focus group, statements and terms were recorded as dialogue amongst the members and observed. The second meeting of the focus group developed four codes (safety, regulatory compliance, root cause, reporting) that had emerged from the group’s earlier outputs. These four codes were further distilled during the focus group activity and were consolidated into two themes (reporting, root cause) that were to eventually form part of the piloted semi-structured interview instrument.

Reporting and root-cause themes were the result of the draft consolidation of the comments and emerging codes. In concert with the focus group activities, a literature review was performed by the researchers and this generated three further themes as reflected in

Table 2. Codification themes used in the NVivo analysis of the final set of publications.

Codification Theme	Description	Origin
Root Cause	Reason to establish causation	Focus Group
Reporting	Value of reporting to learning from incidents	Focus Group
Learning from Incidents	Outcomes of learning from incidents	Literature Analysis
Just culture	Impact of just culture on learning from incidents	Literature Analysis
Precursors	Contribution of precursors to learning from incidents	Literature Analysis

.

The resulting draft semi-structured template (provided in Appendix A) containing the five themes was scrutinized by the focus group. The constituent questions relating to each theme and the running order of the document was subject to many minor changes during the individual piloting of the instrument with the three group members.

2.3. Data Collection

Data were gathered from seven organizations using a semi-structured interview template. The participating organizations were involved in aircraft maintenance and continuing airworthiness activities. Building trust and commitment, as proffered by Chatzi [44] and Chatz et al., [45] was deemed to be a necessary tenet of a successful data collection exercise. Managing the interview process with the support of senior staff complimented visible top-down support for the research and ensured there would be no confusion regarding access to what some organizations often classify as sensitive commercial data. The aim was to explore how learning from incidents occurs and what can constrain learning in the area of focus. The pilot phase ensured the desired outcome of the main data collection phase would be congruent with the aims of the study. The interviews were recorded, transcribed and participants could not be identified from the recordings or transcripts. Full ethical approval for the data gathering was granted by the University.

2.3.1. Instrument

Data were collected using semi-structured interviews, lasting on average sixty minutes. The “aide memoir” was arranged so that participants could offer a flexible response and any emerging themes could be identified. The semi-structured approach facilitated emphasis being placed upon any points that warranted further focus or examination by the researchers. An example of the interview template is included in Appendix A. Interviewees were asked to give an example of a recent incident they were familiar with. The structure of the template, (a) probed process around reporting and (b) elicited the participants perception of learning from incidents within their organizations. Following on from the initial contact on reporting, the participants discussed just culture, learning, root-cause and incident precursors during their individual engagements with the researchers.

2.3.2. Participants

The “key issue in selecting and making decisions about the appropriate unit of analysis is to decide what it is you want to be able to say something about at the end of the study” [46] (p.168). The objective of this study was to investigate individuals’ perceptions of how learning from incidents takes place and the obstacles present in the maintenance and continuing airworthiness management domain of the aviation industry segment. There were thirty-four (34) participants in total, as presented in

Table 3. Participants in the study (n = 34).

Participant Roles	Number
Category B1 Engineer	4
Supervisor	3
Category A Mechanic	3
Quality Assurance Engineer	3
Category B2 Engineer	2
Shift Controller	2
Contract Composite Inspector	1
Inspector	1
Aeronautical Engineer	1
Category B1/B2 Engineer	1
Maintenance Manager	1
Technical Safety Manager	1
Technical Services Manager	1
Line Maintenance Manager	1
Deputy Quality Manager	1
Maintenance Control Manager	1
Maintenance Planner	1
Maintenance Safety Officer	1
Apprentice Technician	1

.

Each of the organizations maintained between 6 and 300 aircraft at the time of the study. While traditional reporting and learning themes were evident outputs from the focus group meetings, it was decided that the data would be collected through one-to-one semi-structured interviews. Semi-structured interviews permitted the researchers to get a deeper understanding of complex organizational and social interactions and at the same time follow a construct. The participating organizations were selected based upon them being accredited to perform aircraft maintenance and continuing airworthiness activities since the inception of EU regulation 1321/2014 [6]. Within this domain, there are categories of staff that are required to be aware of incident reporting and make a report as necessary (e.g., technical managers, certifying staff, quality assurance staff, stores personnel, technical services). Each organization is required under legislation to employ a satisfactory level of staff regardless of their aviation activities. As a minimum, at least one of each of these roles was represented in the study. It was ensured that at least one staff member from each discipline was included in the study and had made a report in the previous twelve months. As certifying staff, technical managers and quality assurance staff are by virtue of their position active reporters (due to their exposure to active operations), staff in these disciplines were well represented in the study’s cohort. Participation in the study was on a voluntary basis and all who participated were acquainted with the project prior to performing the interviews. All participants signed consent forms.

2.4. Data Analysis

Thematic analysis was the method chosen to support the analysis of the study’s data. The Braun and Clarke [47] six-step proposition, which consists of eight discreet cycles, in conjunction with the QDA Training [48] material, formed the basis of the analysis technique. A practical iterative approach was adopted throughout the analysis where the data were formally arranged into discrete phases. The eight individual stages of analysis distributed over the six phases were designed to support a robust and rigorous analysis of the data.

Table 4. Stages and Process Involved in Qualitative Analysis. Adapted from Braun and Clarke [47] and QDATRAINING Training [48] material.

Analytical Process (Braun and Clarke, 2006) [47]	Practical application of Braun and Clarke in Conjunction with NVivo	Strategic Objective	Iterative Process Throughout Analysis
1.Familiarizing yourself with the data	Transcribing data (if necessary), reading and re-reading the data, noting down initial ideas. Import data into the NVivo data management tool	Data Management (Open and hierarchal coding through NVIVO) Descriptive Accounts (Reordering, ‘coding on’ and annotating through NVIVO) Explanatory Accounts (Extrapolating deeper meaning, drafting summary statements and analytical memos through NVIVO)	Assigning data to refined concepts to portray meaning Refining and distilling more abstract concepts Assigning data to themes/concepts to portray meaning Assigning meaning Generating themes and concepts
2. Generating initial codes	Phase 2. Open Coding:Coding interesting features of the data in a systematic fashion across the entire data set, collecting data relevant to each code
3. Searching for themes	Phase 3. Categorization of Codes:Collating codes into potential themes, gathering all data relevant to each potential theme
4. Reviewing themes	Phase 4.Coding on:Checking if the themes work in relation to the coded extracts (level 1) and the entire data set (level 2), generating a thematic “map” of the analysis
5. Defining and naming themes	Phase 5. Data Reduction:On-going analysis to refine the specifics of each theme, and the overall story (storylines) the analysis tells, generating clear definitions and names for each theme
6. Producing the report	Phase 6. Generating Analytical Memos. Phase 7. —Testing and Validating. Phase 8. Synthesizing Analytical Memos. The final opportunity for analysis. Selection of vivid, compelling extract examples, final analysis of selected extracts, relating back of the analysis to the research question and literature, producing a scholarly report of the analysis [47,48]

below illustrates the stages and processes outlined and performed in NVivo and links this to the practical guidelines set out in Braun and Clarke [47]. Their six-step approach that supports the application of thematic analysis is shown in column one and the corresponding application in NVivo is shown in column two. The third column features the strategic elements of coding as the researcher moved from the initial participant-led descriptive coding, to the secondary coding which was more interpretative in nature indicating this phase of coding was both researcher- and participant-led. The final abstraction to themes was researcher informed only. This phase was designed to allow the researchers to engage the participant in direct dialogue with a wider arena such as literature and policy or strategy for example. The fourth and final column illustrates the more iterative nature of the coding, analysis and reporting of proceedings that terminate in a conclusion.

Phase 1 activity involves familiarizing oneself with the transcribed data. In this first phase, the data were loaded into NVivo. It was checked and re-read several times to ensure accuracy of the uploaded transcripts. At the end of the phase activity, initial codes were noted down and retained.

Generating initial codes (open coding: phase 2)—According to Lincoln and Guba [49] (p. 345), a data unit can be defined as the “smallest piece of information about something that can stand by itself, that is, it must be interpretable in the absence of any additional information other than a broad understanding of the context in which the inquiry is carried out”. The open coding is intended to systemically organize the data and uncover the essential ideas found in the data [50]. Each discrete unit of data is labelled in line with the phenomenon it represents. The second phase required broad participant-driven open coding of the interview transcripts recorded during the data gathering step of the research study. Features of interest were coded in a systematic way across the complete dataset where data relevant to each code were collected. Clear labels were allocated to these codes and definitions to serve as rules for inclusion [26].

A set of provisional categories was generated for the segmented data to be coded to. These categories were descriptions of concepts and themes in broad terms. They took two forms: researcher-driven and participant-driven. The former was derived from a theoretical framework underpinning the study and the latter from the knowledge gained of the participants’ language and customs. Hammersley and Atkinson [51] (p.153) consider the importance of participant-driven categories: “the actual words people use can be of considerable analytic importance as the ‘situated vocabularies’ employed provide valuable information about the way in which members of a particular culture organize their perceptions of the world, and so engage in the social construction of reality”.

Searching for themes—In phase 3, codes from phase 2 were collated into categories of codes by structuring all the data relevant to each potential category into a framework that could be used in support of further analysis. This phase also included distilling, re-labelling and merging common codes that were generated in phase 3 to ensure the labels and definitions for inclusion were an accurate reflection of the coded content. These first-round categories are best described as broad descriptions of concepts and themes. During the analytical process they underwent content and definition change and the existence of the two forms of category provides an important means of traversing between “natural” and “theoretical” discourses. Araujo [52] (p.68) suggests that “codes should be viewed in two ways: as part of the analyst’s wider theoretical framework and as grounded in the data.; the process of coding data should be regarded as an important intermediary step in translating social actors’ frames of meaning into the frame of theoretical discourse; coding frames therefore, mediate between the ‘natural’ everyday discourse and the theoretical discourses in social science”.

Reviewing themes (coding on) in phase 4 required further decomposition of the study units of data identified in phase 1. This activity was intended to support a greater understanding of the highly qualitative elements and gain a deeper appreciation of the meanings contained within. It should be noted that not every task could be further broken down and this meant that the activity was performed only as required. Restructured codes were broken down into further sub-codes in order to augment a greater understanding of the meanings embedded within them. These distinctive aspects included communication with management, discovering latent issues, just culture, learning lessons, reporting, root causes and story of an incident.

Defining and naming themes in phase 5 of the data analysis was concerned with analyzing the tentative categories identified in phase 2 for their properties and characteristics. This is a pre-cursor to drafting a propositional statement for each category. Developing analytical memos moves the process beyond identification and description of broad categories to a position of analyzing and fusing meanings in the data under each category. This progressed to drafting a statement that aspires to illustrate the concerted meaning of the segments of data coded to each category. Maykut and Morehouse [26] (p.140) defines a propositional statement as, “a statement of fact the researcher tentatively proposes, based on data”. This phase in addition to further data analysis to refine the specifics for each theme, generated clear definitions and a name for each theme. It also involved data reduction by consolidating categories from all three cycles into a more abstract, philosophical and literature-based thematic framework and conceptually mapping and exploring their relationships with one another for reporting purposes.

Producing the report in phase 6 required analytical memos to be written against the higher-level themes to present an accurate summary of the content of each category and its codes and to also propose findings. The tasks associated with phase 6 included (i) generating analytical memos, (ii) testing and validating and (iii) synthesizing the memos coherently and cohesively, and were performed simultaneously. Writing the analytical memos against the higher-level codes (i.e., learning from incidents, learning process and learning product) required an accurate summary of each category and its codes and findings against categories. These memos considered a few key areas:

• The content of the cluster of codes which were being reported on.
• Patterns where relevant.
• Considering background information noted against participants and examining any patterns relating to participants’ profiles.
• Considering any relationship between codes and their importance in relation to the research questions.
• Noting any primary sources relating to the context of the relationship with the literature in addition to highlighting any gaps in the literature.

Testing, validating and revising analytical memos was performed in phase 7. The purpose of this was to provide a self-audit of the proposed findings by soliciting evidence in the data beyond just textual quotes in support of the recorded findings and to also expand on deeper meanings within the data. This required the data to be interrogated, not only relying on relationships across and between categories, but also a degree of cross tabulation with demographics, observations and the literature. The outcome of this phase was evidence-based findings as each proposed finding was validated by being rooted in the data themselves and was reliant on the creation of reports in support of substantiated findings.

The discipline of writing analytical memos was used during the data analysis process. Birks et al. [53] believe “memoing serves to assist the researcher in making conceptual leaps from raw data to those abstractions that explain research phenomena in the context of which it is examined”. In general, memos were employed at the “ideation” stage when the researcher was developing thought processes and early in the data capture phase. As decisions were made, the early processes and rationale for final analysis iterations were recorded using this medium. Memos were further employed to preserve an objective closeness to the harvested data and to maintain the context of each semi-structured interview at the participating individuals’ level. Developing ideas, reasons for considering possible category relationships and connections was also possible through the application of the analytical memo process. The rigorous support memoing offered served to guide the analysis of the data through different levels of abstraction [54]. The rule of this activity served to ensure a high degree of continuity between the outputs of ideation and the evolving interpretation that were honed through the researchers’ articulation, exploration and their iterations of the data. Overall, this drew out the meanings in the data through the increased sensitivity the researchers were offered by applying the memoing process [53].

In phase 8, the analytical memos were synthesized into a coherent and cohesive report with the findings well supported. The final phase involved the assembly of the narrative with the data extracts while appreciating the product of this amalgam in the context of the related literature. The example features the finding, clear links to the interview data and literature and an explanatory narrative in the form of a memo. This finally resulted in the compilation of the report which contained the results and discussion elements of the body of work.

In summary, this study adopted an interpretative approach pivoting on the fact that it was of an exploratory nature. The study performed thirty-four interviews in eight aircraft maintenance and management organizations based in Ireland. An analysis of various potential research methods and means of data collection resulted in the following research design being implemented. A thematic analysis approach was employed as a research methodology:

• Unit of analysis is an individual;
• Semi-structured interview guide was constructed following a systematic analysis of literature and the use of a focus group;
• Data were collected through qualitative interviews;
• Thirty-four interviews were collected in locations endorsed by eight organizations;
• Qualitative analysis based on the guidelines from Braun and Clarke [47] (thematic analysis) employing a six-phase approach was used in the study.

3. Results and Discussion

3.1. Framework

Figure 3 presents a framework that offers an insight into how the present study applied the research inputs and produced the results.

The top layer reflects the five themes that formed the basis for the data gathering template. These themes were developed through an iterative process of conducting focus group sessions with two themes emerging, i.e., root cause and reporting. Concurrently, a systematic literature review was performed using NVivo software to assist the researchers manage over 1000 screened publications. Following a thematic analysis of the data, three main themes (Appendix B) emerged from a final cache of 18 publications, i.e., learning from incidents, precursors and just culture. The five themes informed the structure of a data gathering instrument that supported 34 semi-structured interviews in the continuing airworthiness segment of the industry. Following transcription, the data were uploaded to NVivo where they were thematically analyzed using the Braun and Clarke [47] framework. The outputs from the thematic analysis distilled the interview analysis into three main outputs, i.e., learning from incidents, learning process and learning product. The lower tier represents the elements the themes were comprised of and the findings are presented under these headings (

Table 5. Summary of results. ¹ Learning from incidents (LFI) is a safety management activity with a desired outcome of preventing unwelcome event recurrence.² A learning process facilitates a change in knowledge and behavior intended to support LFI.³ Safety related information arising from the LFI process.

Learning from Incidents 1	Learning Process 2	Learning Product 3
The decision to report an incident can be impacted by the perceived commercial pressure and the potential for embarrassment associated with making a mistake, amongst front line maintenance staff.	The release of a safe aviation product is the primary goal all operational maintenance and management staff espouse to.	In the organizations supporting the study, it was apparent that incidents are managed with the support of a consistent life-cycle methodology.
Identifying and understanding organizational behavioral and human factors are important elements affecting decisions to report.	Single-loop learning is a level of learning that can exist in a dynamic operational environment where a “find and fix” ethos exists.	Learning products that arise from the managed lifecycle of an incident are intended to impart sufficient learning to prevent recurrence or occurrence of same or similar events.
Inadequately resourced investigation and follow up of incidents does not support the determination of accurate event causation and measures to prevent similar incidents reoccurring.	The mandatory human factors continuation training program is considered by study participants to be an effective enabler of double-loop learning.	While aircraft manufacturers generally provide feedback on notified incidents, component manufacturers provide less feedback with little or no feedback arising from aviation authorities on submitted reports in the jurisdiction of the study.
The recognition of the extended impact of under-reporting on “levels of learning” is not always a priority in some organizations.	Evidence amongst study participants where a review of single and double-loop learning within organizations was not available during the study.	The cost of classroom delivered continuation training is a primary consideration for most organizations.
The absence of a potential learning product that results from effective reporting is an impediment when attempting to gauge the effectiveness of learning.	No formal requirement for competence in the areas of learning for managers and accountable persons exists in EU regulation 1321/2014.	Computer based training is an option that is under trial by some organizations but there are concerns amongst operational staff regarding its overall effectiveness in its current form.
Pressure to prematurely close incident reports does not promote thorough event causation and measures to prevent similar incidents reoccurring.	No competence requirements for staff involved in the development or delivery of formal human factors continuation training programs.	Just culture has a positive impact on reporting rates.
		Feedback to staff on incident causation factors from an information and learning perspective is important.
		Poorly designed continuation training syllabi do not support effective learning.
		Timely follow up to incident reports supports more effective learning outputs from the reporting process.

).

3.1.1. Learning form incidents—Acquiring, Processing and Storing Data

Incident reporting is accepted as a worthwhile activity amongst those participating in the study. This is based on the collective notion that the initiative raises awareness of incidents and potential hazards and can therefore help prevent event recurrence. The authors recognize that awareness is an important component of learning from incidents. Situations do arise where due to lack of report data, it is questionable if all the necessary reports are being submitted as required. Amongst the constraints to making a report are perceived production pressures and the potential embarrassment that could arise from making a mistake and highlighting it [5]. There are just culture concerns amongst some staff because they do not always know what the impact for them personally will be if they submit an incident report [44].

A dedicated focal point in organizations is essential for the systematic management of reported incidents. Where this discipline is applied, the process owner is responsible for highlighting reported issues and raising the necessary awareness amongst operational staff. Once an incident is acquired through the efforts of a reporting system, some form of processing and analysis is necessary. The availability of adequate resources for determining causation and implementing measures to prevent recurrence was identified as a primary point of concern. Perceived premature closure of reports was also highlighted amongst participants. There was a call for improved accountability and transparency on decisions relating to some closure actions. Respondents associate the practice of applying commercial key performance indicators to safety management as shallow efforts are sometimes made by organizations to expeditiously and prematurely close reports on occasion. Incident reporting and safety management initiatives have been in existence for some time. Large repositories of associated safety data are stored in many organizations. Although entities are mandated to inform key stakeholders, there is a strong opinion amongst some participants that the data repositories could be aggregated and put to better use in support of learning amongst all operators.

3.1.2. Learning Process—Single-Loop, Double-Loop and Deutero Learning

The interview data confirms that safety is a primary underpinning value in the organizations that participated in the study. The release of a safe product, i.e., an aircraft or component, is a formative pursuit and measure of learning. In organizational environments where a “find and fix” ethos may prevail, single-loop learning [11] is evident in the examples presented.

A desired outcome of double-loop learning [11] is often witnessed for example through the adjustment of environmental, behavioral and procedural norms. Instances of double-loop learning can be evident following unsuccessful attempts through single-loop learning where causation is then adequately understood and actioned. Continuation (mandatory in-service) training was considered by study participants as an effective mechanism that enables double-loop learning. During the study, it was apparent that single and double learning loops are recognized amongst many participants as having differing capabilities in terms of delivering an effective learning product. However, there was no evidence of formal reviews of single and double-loop learning being performed within the participants’ organizations. Although deutero-learning [11,55] may be considered as a natural extension of other levels of learning, the concept did not feature strongly amongst the participants. A review of the EU1321/2014 [6] implementing requirements confirms an absence of any mandatory requirement to review learning processes.

3.1.3. Learning Product—Effectiveness and Types of Knowledge

Continuation training is a mandated European requirement [6] for all aircraft maintenance and continuing airworthiness management organizations. It is a product as well as a medium for imparting learning from incidents and safety related hazards. It was identified during the study that the learning product is shared amongst staff through three primary means of distribution: formally delivered continuation training, tool-box talks and safety briefings and electronic, paper, notice board and “read and sign” safety publications. The study suggests a learning product can arise as a result of an output from an incident lifecycle. Feedback from submitted occurrences to stakeholders varies from very good to poor. Cost is seen as a major consideration in some of the participating organizations when planning continuation training delivery. Although computer-based training is being considered in some companies as a viable option to class-room delivery, concerns are evident in respect of effectiveness of this medium in its current form. Bedwell and Salas [56] suggest computer-based training (CBT) can be used as a methodology for providing, “systematic, structured learning; a useful tool when properly designed”.

The perceived overburdening of operational staff with complex learning products and excessive cognitive loads was recorded as an impediment to learning during the study. Participants suggested this can arise from poorly designed training syllabi delivered during periods of high operational activity.

Four knowledge types were identified and relate to: conceptual, dispositional, procedural and locative knowledge forms [57]. One of the key objectives of learning from incidents is to identify the type of knowledge needed to prevent an issue recurring. When a reportable issue, for example, is discovered, the submitted report will identify “what” happened. Subsequent follow up will set out to determine “why” the issue occurred. The guiding principles of “how” to perform the task or operation are often contained in procedures or data particular to the task. The information contained in procedures will enable a person to utilize other forms of knowledge. Prevailing safety culture within an organization will have an impact on learning from incidents. If a strong commercial/production culture exists, this may have an impact on, for example, the depth and breadth of learning from incidents within the company. Induction and initial training are important when accessing information for new staff. Accident data repositories contain well-documented human factor-related examples often relating to access to approved data and consequently resulting in potentially preventable incidents. Examining the limitations of each type of knowledge when continuation training programs are being developed was flagged as important by some participants. During the study, no discernible differences were recorded in how the types of knowledge were differentiated in participant organizations. A review of the EU 1321/2014 [6] human factors syllabus requirements did not highlight a need to appreciate or account for these human centered limitations when designing and delivering training lessons. Improved regulatory guidance on the design of effective human factor related material should therefore be developed. Information on how training should be structured in order to appreciate types of knowledge and capitalize on it as a minimum are required to ensure the most efficacious outcome from incident-related training.

4. Conclusions

An ameliorating feature of learning from incidents is the potential to effect sustainable improvements in aviation safety. A review of safety from the perspective of maintenance and continuing airworthiness staff is key to understanding the relationship between safety and the concept of learning from incidents [31]. From the study’s qualitative data, we were able to identify how learning occurs in the airworthiness segment, and issues that support and constrain learning. Recurrent mandated training initiatives such as continuation training were found to be pivotal in enabling learning. Aspects such as prevailing culture and poor event causation were noted to have a negative impact on learning. Our proposed incident learning process (Figure 2) offers a panoramic of where potential learning opportunities and procedural improvements can arise within the lifecycle of an incident. This perspective could be applied in support of developing regulatory working group specifications and validating continuation training initiatives. In addition, it could also be used to develop a holistic review approach to learning from incidents within other organizations both in the aviation industry and outside. Two notable limitations to our research arise. First, the scarcity of prior studies capable of supporting the basis for the research was pronounced. However, prior studies in parallel domains were successfully leveraged in support of the literature review. Second, the study’s population (n = 34) was relatively small. As the study participants were representative of all affected domain functions and a point of saturation was reached, it was deemed adequate.

This research is capable of supporting other papers on additional benefits associated with learning from incidents (LFI). Notably, with the imminent implementation of a safety management (SMS) requirement for continuing airworthiness organizations, potential improvements to hazard identification arising from learning from incidents (LFI) could be highlighted.