Air Transport System Agility: The Agile Response Capability (ARC) Methodology for Crisis Preparedness

Abstract

Aviation is a highly inter-connected system. This means that a problem in one area may cause effects in other countries or parts of the Air Transport System (ATS). Examples range from local air traffic disruptions to the 2010 volcanic ash crisis. Agility, like resilience, refers to the ability to cope with dynamics and complexity in a flexible manner, by adjusting and adapting performance and the organization of work to fit changing demands. The aim of this work is to help ATS organizations with increasing their agility in the face of crises and challenges. To this end, this article presents the Agile Response Capability (ARC) guidance material. ARC was developed from a literature study and a number of case studies that combined past event analysis, interviews, focus groups, workshops, questionnaires, and exercise observation methodologies. ARC aims to help aviation organizations to set up, run, and evaluate exercises promoting agility to handle disturbances and crises, and to enable structured pro-active and retrospective analysis of scenarios and actual events. The elements and steps of the ARC approach are illustrated and exemplified with data from three case studies. The ARC methodology facilitates more agile and resilient ways of responding to the fundamental and novel surprises that have become almost commonplace in the past decade, and are likely to continue to do so.

1. Introduction

Aviation is a highly inter-connected system of systems. This means that a problem in one area may not be confined to the local system. Instead, it may cause effects in other countries or parts of the Air Transport System (ATS), for example a fire in an airport area may lead to the shutdown of the airport, and if it is a major hub, this can cause disruptions over a large part of Europe. Additionally, there is the potential for massive system-wide events such as volcanic ash.

Agility and resilience refer to the ability to cope with dynamics and complexity in a flexible manner by adjusting adapting performance and/or the organization of work to better fit changing demands, both pro-actively as a way of preventing unwanted events and re-actively as a way of coping with unwanted events.

The aim of this work is to help ATS organizations with increasing their agility in the face of crises and challenges. The objective is to support ATS organizations in the design of exercises for the organization(s) and roles that need to prepare for providing an agile response during crises or otherwise challenging situations. The research question that the work reported here aims to address is how to design scenarios that challenge exercise participants and their organization(s) in their agility to an appropriate degree, i.e., so that intended learning outcomes can be achieved. This includes enabling organizations to identify challenges from actual past events and potential future events, and articulate why these are challenging, in order to generate scenario elements and “what-if” courses of action, and identifying assessment points to assess the exercising organizations’ responses.

Our approach combines three major research strands into the Agile Response Capability (ARC) methodology: organizational agility and resilience, crisis exercises and scenario-based training, and functional modeling of complex events. Each of these are briefly outlined as a background to the research.

1.1. Organizational Agility and Resilience

The primary source of inspiration for this work on organizational agility in crisis management is recent military research on agility and Command and Control (C2) agility, as defined by several NATO research task groups [1,2]. In the military context, Alberts proposes the following definition: “Agility is the ability to successfully effect, cope with, and/or exploit changes in circumstances” [2] (p. 54), [3] (p. 190). In the agility concept developed by Alberts and the NATO SAS-085 group, agility is a multi-faceted concept, which includes the following components: responsiveness, versatility, flexibility, resilience, innovativeness, and adaptability [3] (p. 204). To highlight a few of these components of agility, NATO SAS-085 [2] defines versatility as “the ability to maintain effectiveness across a range of tasks, situations, and conditions” (p. 205), flexibility as “the ability to employ multiple ways to succeed and the capacity to move seamlessly between them” (p. 203), adaptability as “the ability to change the organization and/or work processes” (p. 199), and resilience as “the ability to recover from or adjust to misfortune, damage, or a destabilizing perturbation in the environment” (p. 204).

Agility can in part be achieved by being C2 agile, meaning that the organization of work and coordination is rearranged in order to better fit the current or foreseeable future situations. A conceptual tool developed in the NATO SAS work is the C2 approach space, a three-axis model presenting an organization’s approach to C2 (C2 approach) in terms of “information dissemination” (who gets to know what?), “allocation of decision rights” (who has the mandate to take action?) and the “interactions” (who is interacting with whom?) [1]. Hierarchical, formal bureaucratic organizations with limited capability to disseminate information are on the lower end of the three dimensions, while more networked, distributed organizations with a high degree of allocation of decision rights are further out on the three dimensions. The appropriateness of a C2 approach can only be evaluated in light of a specific situation and problem in which it is applied—one C2 approach is not in principle better than the other, it depends on the context. The space in which the problem is described is called the endeavor space [1].

The conceptual model shows that endeavor space monitoring, as well as self-monitoring of the own organization and response, is necessary for successful C2 agility, but it does not provide guidance on how to implement these concepts. Some preliminary answers to how to explore this issue can be found in the C2 by Design Handbook [4], which suggests a three-stage process by addressing three questions: what prevents us from going where we want to go? (the problem), what should we be doing? and what are we doing—is it working?, and presents a number of guiding questions for identifying an appropriate C2 approach.

Agility is a term also used in the literature on organizational theory [5,6] and crisis management [7]. Resilience as in Resilience Engineering is used mostly as a new perspective by scholars connecting to safety science and closely related communities (e.g., [8]) focusing on adaptive capacity [9]. Disaster management literature has also used the concept of resilience for some time [10,11]. In these research fields, agility and resilience have a similar bearing on the management of complex safety- and security-critical operations in terms of adaptability of operations in the face of change and unforeseen circumstances that are not fully avoidable. A large part of this research has emerged as a reaction to earlier reductionist and linear approaches to safeguard against failure in a desire to understand emergent properties resulting from growing complexity and unpredictability in multiple-stakeholder activities. The concepts of both resilience in Resilience Engineering [8,9] and disaster management [10] and agility in military command and control [2,3] consider adaptive capacity as the primary way to cope with the kind of events that emerge from the complexity of today’s operational and technological settings. They both consider learning as an important source for improving the ability to cope with challenges. Further similarities are that both emphasize the need for pro-active (anticipatory action before actual events unfold) and re-active action (responsiveness to disturbances and actual changes).

However, there are some important distinctions (see also [12]). Resilience Engineering, and safety science, does not aim to model an intelligent adversary or a competitive environment explicitly, and therefore does not need to “exploit changes in circumstances” in Alberts’ sense of agility. Instead, resilience according to Hollnagel aims to “sustain required operations” [8] (p. xxxvi), reflecting a more industrial setting. The International Civil Aviation Organization’s (ICAO) definition of Air Navigation Service (ANS) expectations [13], however, seems to fit well with the agility definition containing the exploitation of opportunity: “Flexibility addresses the ability of all airspace users to modify flight trajectories dynamically and adjust departure and arrival times, thereby permitting them to exploit operational opportunities as they occur” [13] (p. D-2).

Furthermore, the NATO work on agility seems in its scope not to distinguish between whether circumstances or changes were expected or not, similar to Hollnagel’s [8] inclusion of expected and unexpected conditions in the scope of resilience, whereas Woods [9] and Boin, Comfort, and Demchak [10] see the need for resilience particularly when challenges go beyond the set of events that the organization has prepared for. On resilience, ICAO states: “The ATM system must be resilient to service disruption and the resulting temporary loss of capacity” [13] (p. D-1).

Comparing the four characterizations of resilience for the future of Resilience Engineering by Woods [9] with resilience as described by Alberts and NATO [2,3] (quoted above), i.e., resilience as a component of agility, Alberts’ resilience is similar to Woods’ [9] descriptions of rebound and recovery. Instead, Alberts’ description of agility, cited above, seems closer to Woods’ descriptions of resilience as graceful extensibility, or how systems stretch to handle surprise, and, mostly, the ability to manage adaptive capacity in layered networks to produce sustained adaptability. Operationally, these two forms of adaptive capacity highlight the importance of controllers and managers recognizing surprise and adapting to novelty, as well as the Air Transport System developing more and more towards increased interdependency between nodes in a layered network. ATM units and aircraft are becoming more interconnected (e.g., through technologies such as 4D trajectory management) and aviation stakeholders are more linked than ever before (e.g., through collaborative decision-making processes). Woods [9] challenges the Resilience Engineering community to find ways to support adaptation at and between these networked layers over longer time periods, similar to the aims of the NATO agility work.

Research on both C2 agility and Resilience Engineering has taken inspiration from general systems theory and cybernetics, which focus on how a system (e.g., a team, or set of organizations, in the ATS) retains requisite variety [14] to meet the variety of a process that is attempted to be controlled. To understand the functions necessary for organizational agility, we may project the five systems of Beer’s management-cybernetic Viable System Model [15] on crisis management, with its various systems for taking action in primary activities and planning (at tactical, operations, and strategic/policy levels), as well as for monitoring the environment to determine how the organization needs to adapt. The ARC approach aims to aid organizations to train for requisite variety by providing brainstorming questions on what factors challenge agility, and where applicable, an analytic method, facilitating requisite imagination [16].

1.2. Crisis Exercises and Scenario-Based Training for Experiential Learning

The event-based training and assessment technique (EBAT) is a methodological approach to designing and performing simulation-based exercises. Events are directly linked to training and learning objectives are implemented in training scenarios [17]. This provides controls so that an exercise includes necessary elements of training on identified skills or competencies, which also facilitates relevant observation, data collection, analysis, and feedback [18]. Even though EBAT was created to support simulator-based exercises, it can also be used to structure more traditional training, and especially concerning assessments of the training [19]. Thus, its methodological approach can also provide structure to more basic types of training exercises, such as, for example, tabletop exercises.

Although EBAT contributes to the explicit design of training opportunities, including learning objectives, critical tasks, and performance measurement, its guidance remains at a high level of abstraction, not particularly addressing agility, resilience, or adaptive capacity, and not providing analytical guidance on how aspects in scenarios may be designed to meet high-level learning objectives. The ARC training approach presented in this article takes inspiration from EBAT, but goes beyond it in the sense that it provides a method and analytical support during the various stages of providing training.

It combines this with an experiential approach, where participants gain experience from exercises and train the ability to be adaptive and respond to a range of partly unexpected circumstances. There has been a growing research interest in training organizations for agility or resilience, mostly focusing on which competencies or abilities should be trained and how training or exercises can stimulate these using simulation [20,21,22,23,24,25], rather than the scenario-based analytical approach that ARC constitutes.

1.3. Functional Modeling and Analysis of Complex Systems’ Variability

Analysis methods are commonplace in the safety domain, but structured methodologies for designing, managing, and analyzing crisis exercise scenarios are not. The analysis method of ARC takes inspiration from several safety analysis methods, both traditional as well as recently developed. Two main influences are HAZOP guidewords and CREAM/FRAM variability modes.

Hazard and operability study (HAZOP) originated in the 1960s in the chemical industry as a method for analyzing a physical and functional design of a chemical plant for hazards. The method applies guidewords (e.g., more, less, early, late, before, after, reverse, not) to the various nodes that represent a design of a system to identify whether alternative outcomes of the performance parameters (e.g., flow, pressure, temperature, time) of the nodes reveal hazards that need to be managed. HAZOP has been standardized as British Standard BS:IEC61882:2002.

The Cognitive Reliability and Error Analysis Method (CREAM [26]) was developed as a response to first-generation HRA techniques. CREAM describes a number of failure modes, which were further developed into the variability modes of the Functional Resonance Analysis Method (FRAM). Hollnagel [27] defines ten modes along which variability of functions can occur: timing, duration, distance/length, speed, direction, force/power/pressure, magnitude, object, sequence, and quantity and volume. These variability modes are used in FRAM, one of the Resilience Engineering methods that may be used to analyze the way socio-technical functions, or activities, can vary in their performance over time [27,28,29].

2. Materials and Methods

The central tenet of the military C2 agility work that is used here in the ARC approach is that when the crisis at hand changes over time (for example, it escalates in scale or rate of change), or the understanding of the crisis changes (for example, uncertainty develops into knowledge as crucial facts become known), the management of the situation likely also needs to change in terms of actions and management of various dimensions of the crisis. We will call these, simplified from the NATO C2 Agility work [2], the problem space and the solution space of a crisis, as a starting point for the development of Agile Response Capability in aviation. The exploration of problem and solution spaces has been influenced by a number of guiding questions for identifying an appropriate C2 approach [4].

A number of workshops, meetings, exercise observations, literature, and document analyses have been performed as the methodology and studies to establish the ARC methodology to explore the ATS crisis problem and solution spaces. At the beginning of ARC development, the main stakeholders in the European Air Transport System within the scope of the investigation of ARC were identified, such as the Network Manager (EUROCONTROL) and its European Aviation Crisis Coordination Cell (EACCC), Air Navigation Service Providers (ANSPs), aircraft manufacturers, airport authorities and other infrastructure managers, airlines, regulatory agencies and National Supervisory Agencies (NSAs), and other national and international agencies, expert groups, and interest organizations, depending on the kind of crisis.

Expert input was obtained through workshops and interviews with project partners and other experts investigating a number of scenarios such as the volcanic ash crisis, an ANSP situation display and traffic management system blackout, and three airline crisis exercise scenarios (water landing after engine flame-out during final approach, accident upon landing involving three airlines, and runway excursion after drone collision at diversion destination), and several EACCC European multi-stakeholder exercise planning and exercise execution meetings were observed or analyzed in retrospect (see [30] for a number of past EACCC exercises). These activities (see project deliverables D5.3 and D5.7 linked in the Supplementary Materials) enabled the exploration of the wide-ranging aspects that span the problem space of agility for the Air Transport System. The questions raised during these activities were combined with the questions from the literature on the endeavor space of C2 agility [4].

Combined, these formed hypotheses about the ways in which a crisis could be a challenge for an agile response by various stakeholders in the Air Transport System, to be studied further in more specific case studies. Soon, these aspects also encompassed challenges for responding in an agile way, orienting ARC towards an exploration of challenges in both (a) monitoring various aspects of the crisis and the environment of the organizations attempting to manage it (i.e., the problem space), and (b) intra- and especially inter-organizational aspects spanning the various efforts of management and taking action (i.e., the solution space).

Developing a list of questions spanning the problem and solution spaces of the Air Transport System (illustrated in Section 3.1.4), the research transitioned into developing a method with the objective of methodologically investigating the problem and solution space of the ATS more generally. Since the case studies performed throughout ARC development were examples of analyses that now are part of the larger ARC methodology, the cases are presented as examples of the steps of ARC in this article.

The cases presented in this article are thus some of the studies of actual events as well as studies into exercises and exercise scenarios performed as part of the ARC methodology development. Furthermore, these studies illustrate the phases before, during, and after the event or exercise. Three case studies are presented here to outline the ARC approach and present part of the empirical work behind its development.

Case study 1 concerns a workshop with a European Air Navigation Service Provider (ANSP) that combined experiences of challenging actual events with potential what-ifs in future events, resulting in the suggestion of concrete future exercise scenarios. Three Area Control Centre (ACC) and one tower (TWR) air traffic controllers (ATCOs) and watch supervisors (WS) (all highly experienced and with additional roles in training and/or safety management) participated in a 2-day workshop that consisted of several focus group (see [31]) sessions based on the ARC. Overall, eight “major challenge” or “crisis” situations were explored in detail, two of which were developed into high-level crisis exercise scenarios.

Case study 2 analyzed the exercise scenario of another European ANSPs training exercise before exercise execution, included observation of the exercise, and also included the debriefing observation and a questionnaire using ARC. The training exercise is a reduced modes training, where different kinds of failures of the situation display system (for example, various kinds of radar and data processing failures) and related technical systems are simulated. The training is given to ATC watch supervisors (WS) and technical watch supervisors (TWS) and is organized by a WS (focusing on operational aspects) and a systems operator (SysOp) (focusing on technical aspects). The organizing WS and SysOp together with another WS were presented with the ARC questions exploring problem and solution spaces and asked how they apply to their reduced modes training during a 2.5 h workshop. The ARC questions on difficulty factors were then applied directly after the exercise in a questionnaire that was answered by 20 exercise participants.

Case study 3 entailed an analysis of the published literature on the 2010 volcanic ash air transport crisis using the ARC questions.

Several other workshops, interviews, and exercise observations were used to develop the ARC method, but not all of these can be described as part of this article. The case studies were analyzed qualitatively with a thematic analysis combining a deductive theory driven by major themes in agility research with an inductive data-driven analysis of the workshop and exercise participants’ statements [32]. The cases where scenario design, exercise planning, or exercise management were studied (cases 1–2 reported here, as well as observations in other exercises) also functioned as a way to capture how experienced teams of exercise managers perform their tasks, to learn from and describe the process they follow, and to align the ARC method with current state-of-the-art practice.

The resulting analyses have been condensed into a method that can be applied to both exercise planning and management and the analysis of anticipated or past actual events. The next section describes the resulting method as the organizing structure for the results of the study, exemplifying some of the major steps in the method using data from some of the case studies that were used as the foundation for developing the method.

3. Results

A way to increase the agile response capability within or between organizations is to perform regular joint exercises. Such exercises must, however, challenge the participating organizations in such a way that they actually have to produce agile responses. To facilitate meeting this challenge, this work proposes an approach to scenario development called the Agile Response Capability Method for EXercise planning (ARC-MEX), as described in Section 3.2. The ARC methodology (described in Section 3.1) is the analytical method that underlies ARC-MEX. The related and conceptually similar activities of preparedness planning and analysis of actual events, crises, and incidents may thus also benefit from the ARC approach, although in a different form: The Agile Response Capability Crisis Operations and Plan Enhancement (ARC-COPE), as described in Section 3.3. The ARC approach may thus be applied to actual past or future events and scenarios, as well as to simulated events and scenarios in exercises. Furthermore, ARC may be applied to the before, during, and/or after phases of event/scenario/crisis management in either exercises or actual events. Thus, a 2-by-3 matrix may be formed, outlining the ARC approach and the foci of ARC-MEX and ARC-COPE in each phase, as shown in

Table 1. Purposes and aims of ARC methodologies ARC-MEX and ARC-COPE per crisis phase and type of event.

Purposes/Aims of ARC Methodology Per Crisis Phase and Type of Event	Exercises (Simulated Events) (E) (ARC-MEX)	Actual Events (A) (ARC-COPE)
Before (b)	Exercise design support to generate scenarios that challenge agility and design exercises that “raise the game”	Enhance preparedness through supporting requisite imagination during preparedness planning
During (d)	Provide means for “controlling the heat” and collecting data	Through exercises and planning, improve agility during actual events
After (a)	Exercise analysis support to inform lessons to be learned through analysis, debriefing, after-action review	Supporting retrospective analysis, and what-if analysis, informing lessons to be learned

.

3.1. ARC Methodology

The central analytical tool as part of the ARC approach that is applied in the various phases of analysis of ARC-MEX and ARC-COPE is the ARC methodology, inspired by safety-analytical methods such as HAZOP and CREAM/FRAM. It consists of a combination of parameters, guidewords, and active verbs, combinable into brainstorming and analysis questions, as explained in this section.

3.1.1. Parameters

The ARC methodology uses a set of parameters to describe both the problem space and the agile response space. These parameters can be used as an analytical tool to increase the resolution of the scenario description and provide methodological support to the analysis of both triggering events and stakeholder actions. The following parameters are defined:

• State: This typically describes the state of the crisis, or the state of the crisis organization. Examples are that the organization is formally in “crisis mode” or any other alert status that a team, organization, or set of organizations may have defined. The state often classifies the severity or scale of the crisis.
• Information: This parameter describes information aspects about the information that is known or not known, or needs to be known, in order to take action on the crisis. This can mean any information that in the context of the crisis is relevant, which may be information that is collected or monitored on a regular basis (e.g., an airline’s information about their flights, aircraft, passengers, and cargo), as well as information not commonly available (e.g., ash particle density during the first volcanic ash crisis).
• Resources: This parameter is about what the organizations meeting the crisis have to work with in terms of materiel, personnel, time, money, etc., that are applied, used, or consumed for resolving the crisis. It thus concerns questions such as what the crisis teams have to work with, what they need, and how to obtain it.
• Goals: This parameter describes in which direction the organizations want the crisis to develop, and what desired and undesired states are. For example, these could be to reduce flight delays below an acceptable number of delay minutes, saving lives, restarting traffic as usual, etc.
• Coordination: This parameter describes inter-organizational and collaborative aspects, as well as information exchange and collaboration between organizations, teams, and roles. Thus, issues such as who needs to communicate with whom, how actors work together, how responsibilities and mandates are arranged, and which information is exchanged are considered.
• Expertise: This parameter describes the expertise of personnel that is available and/or needed. For example, volcanic ash experts of various kinds were necessary to understand the potential impact of volcanic ash on air traffic, which was not readily available to all actors that needed to make decisions to cope with the first volcanic ash crisis.

3.1.2. Guidewords

Guidewords that can be used to express variations or variability of the parameters described in Section 3.1.1 of a particular Object (see examples below) are the following:

• Magnitude: How much/serious? (e.g., the organization’s crisis state, the magnitude of closed-off airspace, the capacity set for an airspace sector or airport).
• Timing: When? What is “the right time”, what is early/late? (e.g., of a decision).
• Availability: Is it available? (e.g., availability of information, resources, expertise).
• Uncertainty: Is it certain? (e.g., uncertainty of information, unclarity of crisis state).
• Duration: How long has it lasted, or is it going to last? (e.g., duration of technical failure, duration of crisis state).
• Rate of change: How can the situation change and how fast does it change? (e.g., how does the crisis state change, how fast do actors act, how does the weather change, how does a nuclear spill cloud travel?).

While applying the guidewords to several exercise scenarios, the FRAM guidewords have been reduced to the minimum necessary to span the variability of scenario elements from a pragmatic perspective: The ‘magnitude’ guideword combines FRAM’s [28] modes of magnitude, length, force/power/pressure, quantity, and volume into a single more abstract guideword, i.e., varying the magnitude of various parameters in a scenario. The other guidewords are used as in FRAM [28]. The guidewords were initially reviewed by three Human Factors and Safety experts, contrasting in particular HAZOP and FRAM guidewords according to their meaning and utility in the context of identifying vulnerabilities in crisis planning and response. The resultant set was later verified by two personnel from the EACCC who run crisis exercises at a European level (see [30]).

In a way similar to HAZOP, the guidewords can be used to examine variability of parameters, e.g., more/less magnitude, earlier/later timing, lower/higher rate of change, etc. These guidewords are interpreted somewhat differently depending on the parameter they are associated with. It should be noted that not all guidewords are applicable to every parameter, depending on the type of scenario and events in question.

3.1.3. Parameters and Active Verbs

The ARC analysis processes may, in addition to the parameters and guidewords, be supported by using active verbs. Such active verbs are, just like the guidewords, associated with the parameters, and may either be used as a complement to, or instead of, the guidewords. The active verbs can be seen as a way to reason about the possible solutions or ways to cope with an event, and thus refer mostly to the solution space of the situation, i.e., in what way teams or organizations can act in order to change the state of the parameters. The active verbs in

Table 2. Illustration of how example active verbs may be used in connection to the parameters.

Parameter	Active Verbs
State	assess, define, revise, upgrade, downgrade, communicate, predict, anticipate, activate, declare, establish
Information	collect, monitor, define, assess, share, dismiss, restrict, deny, receive, transmit, broadcast, delay, confirm, request, analyze
Resources	assess, receive, maintain, deploy, mobilize, dismiss, share, activate, switch, request
Goals	define, set, revise, remind, prioritize, communicate, share, agree, reduce, downgrade, re-establish, maintain
Coordination	define, maintain, activate, revise, share, communicate, agree, remind, update, brief, publish
Expertise	recruit, maintain, consult, mobilize, request, dismiss, engage

are examples, as the context of the specific crisis determines which active verbs span the solution space, or possible activities that are available to be undertaken. Thus, the active verbs are not an exhaustive or constraining list of activities, instead they are intended to start discussions on which actions could be taken as various ways of responding to the changing problem space.

3.1.4. Using Parameters, Guidewords, and Active Verbs for ARC Analysis Questions

The ARC methodology combines parameters, guidewords, and active verbs into questions and discussion points (

Table 3. Parameters, guidewords, and active verbs transformed into example questions exploring the problem space (crisis characteristics).

Question: Was the Situation Challenging Because …	Parameters	Guidewords
… it was difficult to classify what kind of ”alert/crisis state” my organization was in?	State	availability, uncertainty
… it was difficult to know which information I should monitor to be well-informed about the development of the ongoing crisis situation?	Information	magnitude, availability, uncertainty
… of the uncertainty in the scenario?	Information	uncertainty
… information that I needed was unavailable?	Information	unavailable
… it was difficult to estimate how long the crisis situation would last?	Time	duration
… it could easily escalate into a much more severe crisis situation?	State	rate of change, magnitude
… I needed information or resources from other actors and I didn’t know how to contact them?	Information, Resources, Coordination	availability, uncertainty
… it was difficult to know what the long-term effects of the crisis would be?	Time	magnitude, duration, uncertainty
… it was difficult to know when the situation would be considered “under control”?	State, Goals	uncertainty
… we have not experienced or exercised this kind of crisis before?	Expertise	availability

and

Table 4. Parameters, guidewords, and active verbs transformed into example questions exploring the solution space (response characteristics).

Question: Was the Situation Challenging Because …	Parameters	Guidewords
… it was difficult to know what our goals were?	Goals	uncertainty
… it required us to organize our work (roles, tasks, processes, etc.) in a new way?	Coordination	availability
… it was not clear if we should act immediately or wait for the situation to develop before taking action?	Status, Goals	timing, uncertainty
… the actors involved were not working towards the same goals?	Goals, Coordination	availability
… roles and their responsibilities were not clearly defined?	Goals, Coordination	uncertainty
… the media was difficult to handle in this scenario?	Information	magnitude, timing
… difficulty with getting access to appropriate expertise?	Expertise	availability
… of limitations in the usefulness of the tools we have available for analysis and decision support?	Resources, Information	availability
… of complex interdependencies between tasks of the various roles and actors involved?	Coordination	uncertainty
… the resources available were not sufficient?	Resources	availability, magnitude
… it was difficult to find and use resources that we do not normally have available?	Resources	availability, magnitude
… we had small margins or redundancies and overlaps in available resources?	Resources	availability, magnitude

) that have been used for brainstorming and analysis, which forms the analytical ARC methodology, that can be applied in various ways in both ARC-MEX for exercises and ARC-COPE for actual events. The examples can also be used as a set of statements for retrospective analysis (“The situation was challenging, because…”) that can be used in the after phase, both for exercises (in debriefing or after-action review) and for actual events (cells Ea and Aa in Table 1). The questions may also be framed to guide future-scenario brainstorming in the before phase of an exercise or actual event (cells Eb and Ab in Table 1) (“The situation would be (made) challenging if/by…”). The example questions indicate that several parameters and guidewords may be combined, and that combinations of the same parameter and guidewords may lead to multiple questions. These questions may be used directly for a focus group or workshop (see Cases 1 and 2), or the analytical approach using parameters, guidewords, and active verbs explicitly. Thus, the degree of analytical effort may be managed, depending on time available. Importantly, the method does not replace the expert exercise scenario or event analysis competency, but aims to guide the structuring and extension of requisite imagination in order to span both problem and solution spaces in analysis, and ultimately learning.

3.2. The ARC Approach to Exercises: ARC-MEX

The Agile Response Capability Method for EXercise planning (ARC-MEX) aims to create scenarios that support event-based training for ATS-wide crisis response. The purpose is to identify scenarios that include events that fundamentally challenge the involved organization(s) ability to act in a purposeful and coordinated fashion, forcing the participants to reflect upon and work with their ability to formulate goals in the face of uncertainty, coordinate assets, and improve information exchange under pressure.

Training for agile response capability is challenging as it differs from traditional exercise approaches in the sense that it is difficult or impossible to state what the ‘correct’ solution to the challenge posed by the training scenario is/should be. Rather, the purpose of such training must be to have the trainees reflect upon the way they organize themselves and how they conduct work. Scenario design includes, thus, from an agility perspective, elements/injects that:

• Force participants to collaborate with partners they do not normally work with.
• Create a need for information that is not available from single sources but must be aggregated from different sources and stakeholders.
• Create situations that challenge prioritization so that different organizations must negotiate, compromise, or otherwise focus on global rather than local goals.
• Create situations where the chain of command is ambiguous, encouraging self-synchronization and coordination.
• Create situations where responsibility for handling the critical event may be unclear, encouraging initiative and assuming responsibility when facing uncertainty.
• Create situations that challenge information management in the involved organizations.
• Create situations that demand an understanding of the collective of organizations involved in the crisis response, in order to respond rapidly and efficiently.

Six steps of exercise design and management are proposed as part of ARC-MEX, which are discussed and exemplified in the remainder of this section. An overview of the method is presented in Appendix A, Figure A1.

3.2.1. ARC-MEX Step 1: Focus Groups and Workshops: Theme, Objectives, Participants

Figure 1 below shows the details of the first step of the ARC-MEX, the initial scenario design, which is based on focus groups (see, e.g., [31] for a detailed description) and workshops guided by brainstorming questions, such as the questions illustrated in Table 3 and Table 4. The aim of step 1 is to provide answers to the questions of what the scenario theme is, i.e., roughly what problem space is to be presented, what the learning objectives should be, and who the participants are. The primary purpose of the focus group is to identify potential scenarios at a general level, where challenging events can be introduced. The second purpose of the focus group is to explore the potential problem and solution spaces, i.e., which problems that the participants will be faced with.

3.2.2. Case 1: ARC-MEX Step 1: ANSP1 Focus Group Crises/Challenges

A total of eight hypothetical “major challenge” or “crisis” scenarios were described first individually by the three participating expert practitioners, based on operational experience and brainstorming triggered by the ARC questions of step 1, and subsequently discussed by the group to reach agreement and address multiple perspectives of all experts involved, e.g., the ACC and TWR perspectives (two different types of Air Traffic Service Units, ATSUs). Summarized workshop notes were communicated between researchers and participants for checking completeness, correctness, and understanding.

The eight scenarios were:

• General airport accident
• Unknown radar target in airspace
• Big VIP (Very Important Person) event, e.g., many heads of state
• One runway available and landing gear problem
• Radar problem at adjacent ATSU (TWR-ACC or ACC1-ACC2)
• Major disturbance in part of the ACCs own airspace, for example industrial accident
• Severe weather
• Emergency evacuation of ACC

Challenging situation No. 3 “Big VIP event” as an example of a scenario discussion is presented in

Table 5. Challenging situation No. 3 “Big VIP event” scenario discussion summary from Case 1.

Summary of Focus Group Discussion on Situation No. 3 “Big VIP Event”	Parameters	Guidewords
A major international event where a lot of VIPs are arriving at the airport at the same time (for example, heads of state) may be challenging. Flow of traffic may be heavily affected by the fact that there are constraints on how many VIPs can be handled by the airport in a certain amount of time, and there may be special restrictions for higher separation needs, in which order and at which part of the airport the VIPs need to taxi or be towed, and which stands and taxiways to take due to ceremony planning, the use of holding patterns or other delays, among other challenges.	Resources	magnitude, availability, timing, duration
In turn, these factors are all causing demands on separation and sequencing in TWR control and back to ACC sectors. Thus, constraints are put on several ATC units in TWR, TMC (Terminal Control), and ACC sectors (airspace sectors and corresponding controlling ATSUs at increasing distance from the airport) because of unusual ATC and non-ATC restrictions. Understanding of the various stakeholders’ challenges between ACC and TWR control and airport operations is critically important.	Coordination Expertise
Generally, this kind of event needs a lot of real-time problem solving. The event contains too many dependencies to be possible to fully plan beforehand. This could partially be solved by good planning in the strategic phase, but this can be difficult to achieve fully because of complexity and dynamics in the actual traffic situation.	State, Information, Coordination	uncertainty, magnitude, rate of change
Follow-up requires an explicit effort after such an event, partly because if no safety incidents or big problems occur, this may not be seen as a crisis or an incident at all, but just as an unusual or high-workload situation. It is also very hard to evaluate an event like this as there are no everyday procedures describing how details should be handled and it affects many airport functions.	State, Goals, Expertise	Uncertainty availability

, with related parameters/guidewords added during analysis.

Three more situations were discussed, that do not correspond to a specific crisis or incident directly but more to circumstances that are creating a generally more challenging situation for an ATSU (Air Traffic Service Unit) and which may become challenging in combination with other demanding situations:

9.

Difficulty to close the sector at the end of the shift due to lack of personnel, in response to unexpected traffic demands

10.

Changes in new IT systems at ATSUs

11.

Diversity of tasks within roles and shared responsibilities between several different roles in the ops room

The following list of difficulty factors (potential challenges for the Air Traffic System) was compiled at the end of the focus group to summarize all event/situation/scenario discussions ((Parameters, guidewords) added after analysis):

Problem space difficulty factors (Parameters, guidewords):

• Time pressure

  ∘

  No time for procedure as planned (Goals, timing)

  ∘

  Little time for coordination (Coordination, timing)

  ∘

  No time for brainstorming/coordination before decision (Coordination, timing)

• Limitations of redundancy of ops equipment (Resources, availability)
• Priorities between flights uncertain (Goals, uncertainty)
• Challenges in (ATS) systems understanding (Expertise, availability/uncertainty)
• Uncertainty of duration, managing expectations (State, uncertainty/duration)
• Large consequences of classification of ”small” event (State, magnitude/uncertainty)
• Lack of reliable information and prognosis difficulty (Information, uncertainty)

Solution space difficulty factors (Parameters, guidewords):

• New coordination paths between stakeholders (Coordination, availability)
• Challenges of information dissemination, uncertainty of how information is interpreted (Information/Coordination, uncertainty)
• Overlap between perceived coordination responsibilities (Coordination, uncertainty)
• Role/responsibility uncertainty (Resources/Goals/Coordination, uncertainty)
• Time delay in coordination between multiple stakeholders (Coordination, timing)
• Propagation of expectation and prediction needs across stakeholders (Information/Coordination, uncertainty/timing)
• Limited training and joint exercises (Expertise/Coordination, availability)
• Legal requirements on scarce or unexpected personnel involved (Resources/State, availability)
• Old procedures that are not updated for changing operational reality (Resources, rate of change)
• Limitations of planning/preparation/procedures, only handling based on experience (Expertise, availability)
• Delays due to propagating unexpected circumstances, affecting capacity (Coordination/Resources, timing/uncertainty)
• Differences in traffic handling between units, need for harmonization (Coordination, availability/uncertainty)
• Unexpected interdependencies (Coordination, uncertainty)

3.2.3. Case 2: ARC-MEX Step 1: ANSP2 Workshop on “Reduced Modes” Training

Considering the exercise on reduced modes and its scenarios (that were already defined before the ARC workshop), step 1 was applied in order to describe what makes the scenarios of the different runs (testing different technical “reduced modes” of the situation and traffic management system) exercise particularly challenging. The workshop was a discussion about how 22 ARC statements about scenario difficulty, and potentially other challenges, occurred in the exercise. The two exercise managers (a WS with long operational and technical experience, and a SysOp with long technical experience), and another operational WS, participated in the 2.5 h workshop. Summarized workshop notes were communicated between researchers and participants for checking completeness, correctness, and understanding. Example results are described in

Table 6. ARC questions, and how they apply to ANSP2 reduced modes training and its scenarios.

Examples of the 22 ARC Challenging Aspects (Problem and Solution Space) and How They Apply to Case 2
1. Classification of what kind of ”alert/crisis state” the organization is in. Generally, there are three main categories where a WS classifies the situation in a reduced mode: green (continue to handle traffic as usual), yellow (reduce capacity to a certain degree), red (evacuation of the airspace—‘clear the skies’). These categories are relatively well-defined since they occur in checklists, where separate technical failures are associated with suggestions of category and capacity. However, actual situations can have high complexity and contain several simultaneous failures (and checklists), so it may be difficult to determine the category/capacity. Further, assessments are changing over time when the situation is changing. WS and TWS will try to agree on a common assessment, but there can be differences in opinion. WS will in most cases determine category and capacity. (State, availability/uncertainty)
3. The uncertainty in the scenario. The scenarios in all runs of this exercise include uncertainties. What aspects of the situation that are known, unknown, or uncertain are discussed for each run/scenario. Examples are access to spare parts internally or from a supplier, and in which order actions should be performed to reach desired results (there may be complex system dependencies with uncertain consequences). Participants discuss what each checklist aims to accomplish, or why the checklist is written in a specific way, aiming to contribute to an understanding for checklists’ applicability. (Information/Resources, uncertainty)
5. Estimation of duration of the crisis situation. It is important for TWS to assess the duration of time that it can take to fix the failures and regularly communicate reassessments to WS. This is a part of the exercise, for both TWS and WS to think about what is most important (technically and operationally), to assess the impact of the duration of repairs and being in reduced modes, and regularly communicate about this. WS has a strong mandate and backing from management to put capacity restrictions in place and when needed, to demand more personnel. (State/Goals, availability/uncertainty; Collaboration, magnitude)
6. Escalation into a much more severe crisis situation. During the exercise, it is discussed how several modes can become combined modes and how subsequent failures can arise, which can be regarded as an escalation of the situation. In practice and during the exercise, it may be that WS has decided on a reduced capacity and then that something more happens, with the result that WS needs to reassess the situation and revise the action taken. TWS and WS need to constantly assess if the situation gets worse (or better). (State/Information, rate of change)
9. Knowing the long-term effects of the crisis. Long-term effects are not explicitly included as an aspect of this reduced modes exercise because of the practically limited duration of the exercise. In actual operational situations, this is, however, definitely a factor that WS and TWS must consider, for example when technical failures in the Aeronautical Fixed Telecommunication Network (AFTN) occur. (State, duration)
10. Knowing if and when a situation is considered “under control”. Among the first steps to get control of the situation in this exercise (and in an actual operational situation) is to sort out which checklists to use and what the technical failure(s) is/are and how long repairs will take. In turn, this information can generate several work tasks in order to get the situation under control. In addition, in many cases, personnel must be called in or rearranged in order to perform these work tasks. In a way, the situation is not fully under control before the situation is “green” again (no restrictions in traffic). This means that WS/TWS are not comfortable with the situation before all functionality is restored. (State/Goals, uncertainty; Resources, availability; Collaboration)
18. Limitations in the usefulness of the tools available for analysis and decision support. WS can see much less of the technical information of the system failure than TWS, so WS is dependent on TWS to interpret status and error messages from the technical system. TWS and SysOp are the roles that monitor the system, therefore WS must blindly trust TWS. The central question is which systems are still running, and how the ACC in this situation can be run without risking safety. It is important to as accurately as possible understand the technical failure and redundancy in the functioning systems. Since there are many connected systems, it is difficult to understand all sub-systems and it may thus take time to understand the error and its consequences. (Resources, availability)

.

3.2.4. ARC-MEX Step 2: Workshops: Detailed Scenario

Step 2 consists of three interrelated parts (see Figure 2): Further developing the challenges developed in step 1 using the ARC method, specifying what-if situations that would require an agile response from the participants, determining what participants’ potential actions could be, and determining performance measures according to which stakeholders’ actions can be evaluated (these steps are in line with the EBAT approach but provide more structure to the steps). The main purpose of step 2 is to generate triggering events in the scenario that will force organizations/actors that are exposed to the scenario to reflect upon and act on these events. Triggering events, or injects, should include properties that challenge participants’ standard way of operating and interacting, and thus trigger responses necessary to cope with these challenges. The outcome of step 2 is an exercise playbook that can be used as a basis for the planned exercise.

Additionally, the realism of the exercise needs to be decided to decide on the exercise type. A functional exercise is a common specific type of exercise that presents considerable complexity to the participants: Functional exercises select one or a few functions as a focus, may involve one or more crisis management agencies, and are usually conducted in real-time, by operational personnel with appropriate equipment, in the field and under realistic conditions [33]. Another often used type of exercise is the table-top, that puts both activities and time scales at a higher level of abstraction. On the other extreme, a full-scale exercise implements all functions and simulates physical processes in the actual operational and management context. The issues of realism and validity are important exercise design factors and relate to the purpose of the exercise and the educational (learning) goals. Feinstein and Cannon [34] discuss validity as the relationship between simulation development and educational processes. According to their model, internal validity relates to representational and educational validity in that participants cannot insightfully engage in a simulation if it does not behave sufficiently like a phenomenon from the real world that they can recognize or understand. External validity relates to whether (1) the simulation represents real-world phenomena (external representational validity) and (2) the simulation has the desired learning effect (external educational validity).

3.2.5. Case 1: ARC-MEX Step 2: ANSP1 Exercise Scenario What-If Elements

As part of step 2 of ARC-MEX, a discussion using the ARC questions on problem and solution spaces was applied to two of the eleven scenarios from step 1 (see Section 3.2.2) in order to discuss hypothetical exercise scenario elements: Scenarios 3: ”multiple VIPs event” and 8: ”sudden ACC closing”. The challenges in relation to the ARC questions that were discussed are presented in

Table 7. Hypothetical exercise scenario elements. Examples of questions that may be used to generate scenario injects (what-if’s) to “control the heat” before/during the exercise are indicated with *.

ARC Questions (Selection)	Hypothetical Exercise Scenario No. 3 “Big VIP Event” Elements—Problem Space
Main operational location? Who?	At least TWR and one adjacent ACC, possibly several ACCs.
General challenge? What?	Major event with many incoming VIPs (e.g., Heads of State) at airport. Amount of traffic involved—type of aircraft—and special requests for the VIPs that lay outside normal procedure that also can differ from VIP to VIP. For security reasons, classification “head of state” is used, which complicates prioritization.
Timing of scenario? When?	Daytime, peak hour.
What “data/variables” are monitored to detect if there is a crisis?	Information that this kind of event is coming a few days ahead. However, no details will be revealed.
What aspects are difficult to capture into “data”? *	Dynamics of arriving VIP flights, late changes, interdependencies of VIP activities around and from the airport.
What are the current and expected effects on own and other resources and assets (people, functions, material, etc.)? *	Only one runway and limited parking area. An additional complication can be that VIPs need further transportation, such as helicopter. Will cause traffic delays for other non-VIP traffic. Larger separation than normal will also create problems en route. Could be stressful on personnel, possible fatigue, due to sudden high demand and duration of possibly several days.
What “category of crisis” is the own organization in? Others?	May not be crisis or incident at all, but just a “high workload event”.
What are the uncertainties and unknowns about the situation? *	If it is unknown when which actors are arriving exactly, if flights are announced with short notice.
What is the potential for events to escalate in scale or severity? *	Low Visibility Procedures (LVPs) or marginal weather (conditions close to LVPs). Unknown target visible on the situation display. Activities in other sectors. Special demands from the VIPs. Simultaneous medical transports (which also have high priority).
Which stakeholders could become affected by the crisis? *	TWR, ACCs, all airport actors, police, security officers, event organizers, etc.
When is the situation considered “under control”? What does “return to normal operations” mean?	Under control = Orderly flow of VIP and other traffic. Return to normal = VIP flights handled and back to normal traffic mix.
ARC Questions (Selection)	Hypothetical Exercise Scenario No. 3 “Big VIP Event” Elements—Solution Space
What pre-defined and exercised organizational structures exist?	Some related procedures, an exercise could be used in evaluating a way of handling these kind of events, a type of new “multiple VIP” procedure exercise.
What information needs to be gathered before taking action?	Interdependencies can be difficult as some information may not be disseminated to the right persons due to lacking understanding.
Are goals, roles, responsibilities, and accountabilities clearly defined, within organizations and between organizations?	Time delays in coordination between multiple stakeholders. Decisions could be changed in the last minute between different airport stakeholders. Could lead to role confusion.
What expertise is necessary to be adequately informed about the crisis? Where is it available?	Knowledge about the actual state of flights’ interdependencies, other than ATC constraints (airport security, event organizers, etc.).
What are the main tasks and in what way are they interdependent?	Main challenge is multiple interdependencies between actors that need to be coordinated. The other main challenge is dynamics, for example security may decide on sequence of landing, etc., without adjusting to air traffic control reality.
What resources are available? Can they be re-allocated? Are there enough margins?	Towing tractors can be a problem unless they have the right configurations. Refueling can be challenging as some parking spaces cannot be used for fueling and the aircraft must therefore be moved before fueling. Possibly need to call in additional personnel.

.

As part of this step, indicators of performance and agility, qualitative and quantitative, were discussed (the exercise could for practical reasons unfortunately not be run within the project). Proposed resulting indicators (requiring evaluation by experts) were:

• Time needed to organize, considering all actors involved
• Quality of abstract plan of action
• Ask, what was the most difficult experience in the exercise?

  ∘

  Ask what information would have helped

  ▪

  Trace back how this information would have helped

• Communications trace
• Checklist use
• Relay of non-standard information, considering access restrictions
• Effectiveness of coordination

  ∘

  Time

  ∘

  Quality of coordination answer

• Follow-up of timetable (plan)

3.2.6. ARC-MEX Step 3: Running the Exercise

Step 3 of the ARC-MEX approach is running the exercise generating exercise data, using the playbook and performance measures generated in step 2. Performance measures identified in step 2 can also be used to assess the progress of participants during an exercise and thereby allow exercise managers to “control the heat” in the sense that scenario difficulty can be adjusted depending on the progression of an exercise and participant experience. The purpose is to keep the scenario sufficiently challenging while not overwhelming the participants. Thus, “the heat”, the difficulty of the events presented to the participants, may need to be adjusted both upward and downward regularly during the exercise, to be appropriate for the exercise and learning goals.

The data collection during the exercise is ideally linked to the goals of the exercise, as well as the challenges that were generated during step 1 (what do we expect to be difficult, for whom, and why?) linked to the events and injects generated in step 2. Here, observations and documentation can focus on what participants actually find challenging, or surprising, and whether these observations match hypotheses generated in earlier steps.

3.2.7. ARC-MEX Steps 4–6: Debriefing and After-Action Review, Analysis, Lessons

The fourth step of debriefing and after-action review (AAR) aims to present observations and documentation to participants in order to have them reflect on events in the scenario, actions, and outcomes, and in turn link back to learning and training objectives. The purpose of after-action review (AAR) is to elicit participants’ views on key events (injects and other scenario aspects), and to facilitate discussion about adaptations and their consequences, or other considerations and thoughts that participants have about actions they took or did not take, as well as hypothetical what-ifs that may have to be dealt with in real future events (see [20,35,36]).

Steps 5 and 6 entail further analysis and identification of lessons to be learned, as well as reporting. As steps in the process, these are not unique to the ARC approach, but the structure and analytical method of ARC can provide a structure for these activities.

3.2.8. Case 2: ARC-MEX Steps 4, 5: ANSP2 Post-Exercise Questionnaire on “Reduced Modes” Training

As a test of the ARC questions’ use as a questionnaire querying scenario challenges, a simplified version of the 22 questions was compiled into statements, similar to Table 4, so that participants could indicate their agreement on a 5-point Likert scale. The exercise manager selected a subset of 13 of these questions for inclusion in the questionnaire, as some questions were expected to be more applicable than others. At the end of each of the two exercise days, training two groups of WSs and TWSs, before the overall debriefing, participants were asked to fill out the questionnaire, for the whole day of exercises, all scenarios considered together (instead of for each scenario, because of limited exercise time). Most runs consisted of two WSs and one TWS as the main trainees in each scenario, and remaining WSs acted as ATCOs and remaining TWSs as technical support. During one training day, all participating WSs and TWSs were present during and discussed all scenarios as a group. For practical reasons, the simulated failures and reduced modes were not exactly the same during the two days. Participation in the questionnaire was voluntary and anonymous, no personal data were collected (only role in the exercise, WS/TWS). Twenty answers were received, fourteen in the WS role and six in the TWS role.

It should also be noted that the answers to such a questionnaire are not measures of objective or subjective individual performance (highly challenging scenarios can be met with varying performance, and performance is experienced differently individually—participants’ performance was out of the scope of this study), nor an objective measure of scenario difficulty, but instead are an indication of how the participants report subjective difficulty. The ARC method focuses on the qualitative description and discussion of challenges for learning purposes, instead of quantifying challenges and evaluating performance. Due to these aspects and practical limitations (relatively low n), a full statistical analysis is beyond the purpose and scope of this test of ARC questions as a questionnaire tool.

The results in

Table 8. Top 7 factors (out of 13 queried) by decreasing difficulty, 14 watch supervisors’ (WS) and 6 technical watch supervisors’ (TWS) averages. Table 6 describes the factors from the exercise managers’ perspective.

Top 7 Factors by Decreasing Difficulty, 14 WS Average	Top 7 Factors by Decreasing Difficulty, 6 TWS Average
#5 duration	#5 duration
#6 escalation potential	#6 escalation potential
#3 uncertainty	#9 long-term effects
#1 state	#10 under control
#18 tool constraints	#3 uncertainty
#9 long-term effects	#18 tool constraints
#10 under control	#1 state

illustrate the top 7 challenges or difficulty factors, as well as differences between WS and TWS, to show that the ARC questionnaire can be used by exercise managers to obtain feedback on how different groups of participants experienced a scenario. WS and TWS seem to share similar main challenges in this exercise, with the same top 2. WS and TWS share the same items challenging factors as the top 7, which is why this selection is explained in detail in Table 6. Notable differences are that long-term effects, knowing when the situation is under control, and having (little) margins were ordered higher in the experienced difficulties by TWSs, and for WSs, uncertainty and determining the state of crisis scored higher on difficulty, compared to the top difficulty factor within each role. A discussion resulting from showing results to the exercise managers seemed to resonate well with their expectations and observations.

Some participants gave feedback that the questions were difficult to answer with respect to this exercise, and some said that they had no problem associating the questions to the exercise scenarios. This indicates that the questions may need to be tailored to the specific terminology of the exercise (which was not done here) in order to avoid multiple possible interpretations of the questions and results, although diverse interpretations of each question may actually be useful for debriefing discussions. The questions recurred to some extent during the debriefing, while participants reflected on scenario challenges.

3.3. The ARC Approach to Planning for and Understanding Actual Events: ARC-COPE

The method for Agile Response Capability is also applicable to the analysis of preparedness plans and to the analysis of actual crisis events. In this format, the method is called Agile Response Capability Crisis Operations and Plan Enhancement (ARC-COPE). ARC-COPE relates to the phases before (preparedness planning) and after (retrospective analysis) actual events, as illustrated in Table 1, see Figure 3.

In the analysis of preparedness plans before an actual crisis, the ARC method can contribute to the assessment of crisis plans through structured what-if analysis using parameters, active verbs, and guidewords, described in Section 3.1. In the analysis of actual events, the collection of data and their analysis may in a similar way be informed by ARC methodology. The aspects in the fictional or actual course of events and circumstances can thus be understood and analyzed for challenges and needs for agility. These events may in turn be simulated and exercised using ARC-MEX for further experiential learning.

Case 3: ARC-COPE: Literature Analysis of the 2010 Volcanic Ash Cloud Crisis

The ARC-COPE method was also used as part of the research to establish, develop, and test the applicability of the ARC methodology. An event analyzed through a literature search and ARC questions was the volcanic ash cloud crisis that resulted from the Eyjafjallajökull eruptions in 2010.

On 20 April 2010, the volcano Eyjafjallajökull in Iceland erupted, which produced an ash cloud that caused serious disruptions on civil air traffic over northern Europe. This was a cascading crisis, with the cancellation of more than 100,000 flights in the period between 14 and 21 April 2010, which in principle paralyzed the European Air Transport System [37].

The case is an example of many of the challenging factors identified for the ATS crisis problem and solution spaces and how agile response capability was built up in various stages. A study from the ARC perspective was therefore warranted. Through a directed search in the Scopus database and snowballing following up on references, 23 publications, published until December 2016, were used in the analysis. A selection of literature found is briefly summarized here, using a selection of the ARC questions of Table 3 and Table 4, in order to provide an example of the application of ARC to a retrospective case analysis.

Particularly, the following questions are addressed:

• Had stakeholders experienced a similar event before?
• What were the new demands on information exchange between stakeholders?
• What “data/variables” are monitored to detect if there is a crisis? What aspects are difficult to capture into “data”? What defines a “crisis state”?
• From whom is data and other input available and necessary? What are the uncertainties and unknowns about the situation?
• What were the main tasks that needed to be performed and in what way are they interdependent (preconditions, timing, etc.)?
• When was the crisis considered “under control”?

Had Stakeholders Experienced A Similar Event Before?

In 1982, British Airways flight 009, a Boeing 747–200, experienced a quadruple engine flameout after entering an ash cloud that emanated from the Indonesian volcano Galunggun. Even though the crew managed to land safely, this is one of the most well-known serious ash cloud incidents [37,38]. This event and other ash encounters due to volcanic eruptions have received increased attention by the aviation community. For example, nine Volcanic Ash Advisory Centres (VAACs) have been established to inform and advise international aviation of occurrence, location, and movements of volcanic ash clouds [39]. Nevertheless, when the first Eyjafjallajökull eruption in Iceland 2010 occurred, authorities and the aviation community were largely unprepared for the extent of the ash cloud and the following crisis. Since few European nations had experience of ash clouds, when Eyjafjallajökull erupted, several governments did not have prepared plans for how to handle this type of event (e.g., [40]). The regulations when the Eyjafjallajökull incident occurred were based on the assumption that ash clouds could be avoided by simply flying around them. Participating organizations were therefore unprepared to assess and evaluate any gradual risks. The regulations when the Eyjafjallajökull incident occurred did not provide any freedom for air navigation service providers or airlines to decide when to allow aircraft to fly in ash-contaminated airspace.

What were the new demands on information exchange between stakeholders?

Several organizations were involved in the crisis, for example VAACs, volcano observatories, air traffic management, airlines, and airports. Under normal circumstances, these organizations do not work together on a daily basis. Even after a few weeks of cooperation during the crisis, they were still seeking means for efficient communication [41]. For example, the communication and information flow between aviation and scientific communities were identified as a major concern [42].

Although established procedures were used for informing relevant parties about the volcanic eruption and ash cloud dispersion, new means had to be found for information exchange in flexible risk management, where aviation authorities, air navigation service providers, and aircraft manufacturers assess potential risks based on particle concentration levels and known effects of particles on aircraft engines.

What “data/variables” are monitored to detect if there is a crisis? What aspects are difficult to capture into “data”? What defines a “crisis state”?

The ash cloud caught the authorities by surprise and revealed a need to improve existing crisis preparedness and regulations for flying in ash-contaminated air. At the start of the crisis, the ash concentration threshold for permitted flying was, according to ICAO (International Civil Aviation Organization) guidelines, 200 μg/m³ [43]. The following standstill of aviation, by strict application of the ICAO guidelines, led to enormous economic consequences, which, for example, led to protests among the airlines. This in turn led to demand of extended information of tolerance levels from engine manufacturers and issuing of test flights [44]. A European Commission-requested solution finding led to the decision to impose a red zone for ash concentrations of 200 to 2000 μg/m³, and a no-fly zone for higher densities, implemented in the morning of 20 April 2010. By this measure, air traffic was back to normal on the evening of 21 April, and by 28 April 2010, the ash crisis was considered over [39].

However, on 1 May 2010, a second eruption of Eyjafjallajökull occurred that resulted in a new ash cloud, which on 3 May 2010 led to resumed safety procedures with closures of airspace. This time, the institutions involved in the volcanic ash procedures were collaborating to find better solutions to handle the crisis [39,41]. By 21 May 2010, EASA officially announced a grey zone: The no-fly threshold was increased to 4000 μg/m³ directly after the acute phase of the crisis. Aircraft were then allowed to fly in concentration levels of 2000–4000 μg/m³ provided that airlines could present a safety case that included the agreement of their aircraft and engine manufacturers [43]. This meant that there were now four classification zones [39,43].

From whom is data and other input available and necessary? What are the uncertainties and unknowns about the situation?

The main reason that the authorities found it difficult to agree on a common approach to the crisis was their absence of shared understanding of the situation and how it could be solved. The three main factors of uncertainty were [44]:

• Uncertainty concerning the ash cloud. Experts were unsure about its exact dispersion and content. No one knew which sectors might be closed, when, or for how long. The lack of relevant guidelines created a deadlock among the stakeholders.
• Uncertainty concerning the consequences for jet engines at different ash concentrations.
• Uncertainty of responsibility, e.g., concerning who was in charge/who could be held accountable for the economic consequences of the standstill.

The most important factors to reduce the uncertainties about how to act were the redefinition of the acceptable risk level, information from engine manufacturers, and data from test flights in ash-contaminated air [44].

What were the main tasks that needed to be performed and in what way are they interdependent (preconditions, timing, etc.)?

When the effects of zero-tolerance for flying in ash-contaminated air became clear during the Eyjafjallajökull crisis, there was increased pressure to find ways to ease the threshold without compromising safety. Due to a lack of information regarding effects of volcanic ash on aircraft engines, several airlines and air forces performed test flights through the ash cloud. Pressure was also placed on engine manufacturers to present available data on tolerance levels for ash contamination [44]. Additionally, the agreement to new operating thresholds that distinguished between different degrees of ash concentration levels offered member states greater flexibility in deciding how to manage their airspace, allowing for less flight disruption while still ensuring the highest level of safety [45]. Thus, the implementation of these procedures eased the crisis. In this respect, in spite of lacking preparedness, the aviation community had the capacity to be both innovative and flexible when facing a novel situation [40]. The initiative of the European Commission to bring together all stakeholders, which meant that they could work together to develop a strategy for management of aviation safety in ash-contaminated air, was crucial to solve the crisis [44].

When was the crisis considered “under control”?

By 24 May 2010, Eyjafjallajökull entered a paused stage and the crisis was over. Already before the crisis ended, on the 19 May 2010, the European Aviation Crisis Coordination Cell (EACCC) was established, with the purpose to facilitate management of all types of crises affecting aviation in Europe, having since organized regular volcanic ash exercises (VOLCEX) [30]. On 21 May 2011, the volcano Grímsvötn in Iceland erupted, with the result that an ash cloud was formed, which in turn led to the cancellation of 900 flights [46]. This eruption was larger than the Eyjafjallajökull eruptions in 2010, but due to different weather conditions and the revision of the contingency plan and guidelines, the consequences for aviation were smaller compared to the Eyjafjallajökull eruptions in 2010 [40]. In April 2012, ICAO also issued regulations that provide more freedom to airline operators to decide whether or not to fly in contaminated airspace, based on safety risk assessment accepted by the relevant national authorities [40].

4. Discussion

The results of applying the ARC-MEX in Case 1 through brainstorming on challenging factors in various scenarios, including various types and kinds of crises in step 1, show that the ARC method contributes to eliciting what makes scenarios difficult and to an understanding of the need for an agile response in these scenarios. The factors found resonate well with the factors determined in the ARC questions. Application of ARC-MEX step 2 shows that the exercise scenario design can benefit from the ARC method and the results of step 1 in order to build agility-challenging scenario elements into the scenario, including generating initial performance indicators based on the discussions of challenges/difficulty generally and challenging elements in the specific scenarios in steps 1 and 2.

The results of applying the ARC in Case 2 show first of all that many of the factors can be tied to the reasoning about the exercise scenario design, pre-exercise, and second, that these can be used post-exercise to guide a discussion on the challenges in the exercise, using a questionnaire to provide feedback to exercise management and to trigger reflection in debriefing. Both pre- and post-exercise, the challenge factors (in this case in the form of statements or discussion questions) seem to support discussion from both an exercise management (design and evaluation) and an exercise participant perspective, although tailoring of the questions to the specifics of the exercise appears necessary to be directly addressed by participants.

ARC (both its variants ARC-MEX and ARC-COPE) does not replace experts in their exercise and event analysis roles, but rather enables organizations to allocate expert resources to exercise and event management processes, channeling expert judgment, experience, and requisite imagination around potential disturbances, challenges, and crises in their operations into structured and methodological exercise design and event evaluation. ARC provides support to the imaginative as well as analytical process of running exercises and preparing for and learning from actual events. The ARC methodology allows the structured investigation of various permutations of events and circumstances, but may also be used selectively and in a “light” version, for example by directly using the ARC questions when analytical resources or time are not available, or some aspects of scenarios are already decided by other means. This is why the method consists of various components, from high-level focus group discussion questions (see Section 3.1.4), to detailed analytical methods using parameters, active verbs, and guidewords (see Section 3.1.1, Section 3.1.2 and Section 3.1.3) to be applied in specific scenarios, which have been outlined in this article in increasingly detailed steps.

5. Conclusions

The Agile Response Capability (ARC) methodology presented here aims to aid Air Transport System stakeholders to increase their agile response capability. Based on concepts from organizational agility and resilience, ARC provides a structured approach to characterize crisis situations, their evolution, and possible response options. The ARC approach consists of the Agile Response Capability Method for EXercise planning (ARC-MEX) for exercise design, management, and analysis, and can also be applied to actual operations (planning prospectively and analyzing retrospectively) as the Agile Response Capability Crisis Operations and Plan Enhancement (ARC-COPE). Both approaches have been iteratively developed through various case studies in the ATS domain.

The intended increase in agile response capability from using ARC facilitates making ATS stakeholders better prepared to manage disturbances from unexpected and dynamic crisis situations. Future work is suggested to test the methodology on further empirical cases and throughout the entire exercise planning, execution, and evaluation cycle. All types of ATS stakeholders, such as airlines, air navigation service providers, and authorities for international coordination, may benefit from using ARC since crisis situations have cascading effects throughout the Air Transport System.

In a time of pandemics, volcanic eruptions, increased terrorism activities, and cyberattacks, as well as extreme weather disturbances due to climate change, there is a need to go beyond the typical pre-planned crisis management thinking to instead develop a more agile wide-scale reaction capability. The ARC methodology is one possible approach to provide better and more resilient ways of responding to the fundamental and novel surprises that have become almost commonplace in the past decade, and are likely to continue to do so.