Towards a Framework of Operational-Risk Assessment for a Maritime Autonomous Surface Ship

Abstract

Global research interest in the domain of maritime autonomous surface ships (MASS) is dramatically increasing. With new prototypes planned to be set to the seas where various operational modes (OMs) are claimed, the issue of the safety evaluation of an MASS, and criteria for selecting the appropriate OM for given conditions remain open questions. This paper proposes a four-step risk-informed framework to assess risk in a scenario for an MASS operating at one of three OMs: manual control (MC), remote control (RC), and autonomous control (AC). To this end, the concept of risk priority numbers (RPNs), adopted from failure mode and effects analysis (FMEA), is utilized. The required parameters to defined RPNs are obtained in the course of analyzing a model MASS accident with expert knowledge. The applicability of the proposed framework is demonstrated via a model MASS case study. Results reveal that, in the same scenario, the risk of MASS varied across the analyzed OMs. On the basis of the aggregated results for each operational mode, suggestions for OM switching are put forward.

1. Introduction

Autonomous ships were termed as maritime autonomous surface ships (MASS) with four types of degree of autonomy (DoA) by the Maritime Safety Committee (MSC) of International Maritime Organization on its 99th session. An MASS could be operating at one or more DoA for the duration of a single voyage [1]. That is, a different DoA indicates different operational modes. For example, an MASS and its remote-control center synchronously go through a number of different operational modes following different DoA.

In the literature, operational modes (OMs) for a MASS are categorized into four types, namely, manual control, autonomous control, remote control, and fail to safe [2,3,4,5,6,7,8]. First, manual control represents the mode where the ship is handled by the crew on board, as is, e.g., the case during berthing and approaching port, passing navigationally demanding waters, and dealing with emergency situations [3,9]. Second, autonomous control indicates that the ship handles itself with an advanced control system onboard, e.g., an autonomous navigation system [6]. In this mode, there may be two submodes, autonomous execution, and autonomous control or autonomous problem solving [3,10]. In the former submode, the ship follows a predefined track, and fully automatically performs navigational and lookout tasks without more advanced reasoning and decision making or guidance from the shore, but with periodic and brief status reports sent to the shore operators [7,10]. In the latter submode, the ship may make intelligent decisions on its own, e.g., on the basis of calculated algorithms or preprogrammed behavior patterns [11]. Third, remote control means that the ship is taken over by a remote-control center, e.g., a shore control center (SCC). In this mode, remote control can be divided into three submodes, namely, direct control, indirect control, and situation handling [10,12]. Lastly, fail-to-safe mode is one of the main safety principles (see [13,14]), similar to the minimal risk conditions (MRCs) proposed by the DNV [15]. The specific fail-to-safe mode depends on what problem the ship encounters and other environmental or ship parameters [3]. Porathe [4] roughly provided three submodes, i.e., drifting, anchored, and station keeping, while DNV [15] listed 10 potential MRCs. Three conventional operational modes are the interest of this study, as shown in

Table 1. Three operational modes (Oms) in this study.

Operational Modes	Description	Source	Definition by the International Maritime Organization (IMO) [1]
Manual control (MC)	The MASS is handled by onboard crew. This is similar with that of conventional ships.	[3,9]	Ship with automated processes and decision support
Remote control (RC)	The MASS is controlled by an operator in a remote-control center, e.g., shore-based control center. In this mode, an operator can directly control, indirectly control, and remotely handle a situation.	[10,12]	Remotely controlled ship without seafarers on board
Autonomous control (AC)	The MASS controls itself, e.g., with the autonomous navigation system onboard. In this mode, the MASS autonomously controls its behavior or solves problems if any exist.	[3,10]	Fully autonomous ship

.

This paper focuses on the risk assessment of potential OMs to change that may provide risk-based support for switching OM. In other words, how to switch among the three operational modes of manual control, remote control without crew onboard, and autonomous control to reduce risk. However, few studies were conducted on providing support for changing OMs. To our best knowledge, there are no standard and criteria to measure risk level for MASS. Hence, to determine the complete risk level for MASS with different OMs is very challenging [8]. To fill this gap, we propose a four-step framework to quantify such risks in these three OMs on the basis of traditional failure mode and effects analysis (FMEA). Failure modes (FMs) in the case study were derived from model MASS tests using the 24 Model for accident analysis.

The rest of this paper is organized as follows. Section 2 introduces the FMEA and 24 Model. Section 3 illustrates the framework for risk assessment of MASS in a given operational mode. Section 4 presents a case study. Contributions and limitations of this study are discussed in Section 5. Section 6 concludes the paper.

2. Methods

2.1. FMEA

FMEA was first developed as a formal design methodology in the 1960s by the aerospace industry with obvious reliability and safety requirements [16]. In the past several decades, FMEA was applied for risk assessment on manufacturing processes [16], electronic systems [17,18], the design of aircraft engines [19], and the propulsion systems of ships [20,21]. Risk assessment in FMEA conventionally develops an RPN, which is obtained by finding the multiplication of the probability, the severity, and the probability of not detecting failure mode. For obtaining the RPN of a potential failure mode, the three risk factors are evaluated using crisp values [16,22,23]. Similarly, the integer numbers, 1 to 10, as ratings are designed for this study, described in

Table 2. Ratings for occurrence of a failure mode for a MASS system.

Rating	Description of Failure Occurrence (O)
10	Mean time between failures (MTBF) is less than 2 h.
9	MTBF < 3 h
8	MTBF < 8 h
7	MTBF < 24 h
6	MTBF < 1 week
5	MTBF < 1 month
4	MTBF < 6 months
3	MTBF < 1 year
2	MTBF < 5 years
1	MTBF < 10 years

,

Table 3. Ratings for severity of a failure mode for a MASS system.

Rating	Description of Effect Severity (S)
10	Failure onboard or onshore is hazardous and occurs without warning. It suspends the operation of the system and/or involves noncompliance with international or national regulations.
9	Failure onboard or onshore involves hazardous outcomes and/or noncompliance with international or national regulations or standards.
8	Onboard or onshore system is inoperable with loss of primary function.
7	Performance of onboard or onshore system is severely affected, but still functions. The onboard or onshore system may not operate.
6	Performance of the onboard or onshore system is degraded. Comfort or convince functions may not operate.
5	Moderate effect on performance of onboard or onshore system. Onboard or onshore system requires repair.
4	Small effect on performance of onboard or onshore system. The system does not require repair.
3	Minor effect on the performance of onboard or onshore subsystem or system.
2	Very minor effect on the performance of onboard or onshore subsystem or system.
1	No effect.

and

Table 4. Ratings for the detection of a failure mode for a MASS system.

Rating	Description for Likelihood of Detection (D)
10	Onboard or onshore subsystem or system does not detect a potential cause of the failure or subsequent failure mode, or there is no system or subsystem for such detection.
9	Very remote chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
8	Remote chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
7	Very low chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
6	Low chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
5	Moderate chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
4	Moderately high chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
3	High chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
2	Very high chance that onboard or onshore subsystem or system detects a potential cause of failure or subsequent failure mode.
1	Onboard or onshore subsystem or system almost certainly detects a potential cause of failure or subsequent failure mode.

.

2.2. 24 Model

The 24 Model is a linear and systematic accident-causation model [24,25]. This method analyzes an accident from two levels, i.e., individual and organizational, and from four causes, i.e., immediate cause, indirect reasons, radical cause, and root cause. Additionally, it accounts for the effect of external factors. The 24 Model is also applied in rail transportation [26] and the maritime industry [27].

Given an accident, we adopted the 24 Model to analyze the immediate cause(s) and external factors of the accident, which are identified as failure modes for developing a scenario. In this context, we assumed that a scenario was caused by one or mutual FMs that were unsafe or by unsafe conditions related to the onboard side, the onshore side, or external environmental factors.

3. Framework

The proposed framework consists of four steps:

3.1. Step 1: Identify Potential Failure Modes

We used the accident-analysis approach for the 24 Model to identify potential failure modes related to MASS on the basis of historical data, e.g., accident reports.

3.2. Step 2: Evaluate Three RPN Parameters of FM in Given Operational Mode

An FM’s three RPN parameters, i.e., O, S, and D in a given operational mode are evaluated by expert judgements using the crisp numbers in Table 2, Table 3 and Table 4. For all identified FMs, such evaluations of expert p construct evaluating matrices O_p, S_p, and D_p, respectively, as follows:

$$O_p={\left(O_{ij}^p\right)}_{m\times n}=\left(\begin{array}{cccc}{O}_{11}^p& O_{12}^p& \cdots & O_{1n}^p\\ O_{21}^p& O_{22}^p& \cdots & O_{2n}^p\\ ⋮& ⋮& ⋮& ⋮\\ O_{m1}^p& O_{m2}^p& \cdots & O_{mn}^p\end{array}\right)$$

(1)

$$S_p={\left(S_{ij}^p\right)}_{m\times n}=\left(\begin{array}{cccc}{S}_{11}^p& S_{12}^p& \cdots & S_{1n}^p\\ S_{21}^p& S_{22}^p& \cdots & S_{2n}^p\\ ⋮& ⋮& ⋮& ⋮\\ S_{m1}^p& S_{m2}^p& \cdots & S_{mn}^p\end{array}\right)$$

(2)

$$D_p={\left(D_{ij}^p\right)}_{m\times n}=\left(\begin{array}{cccc}{D}_{11}^p& D_{12}^p& \cdots & D_{1n}^p\\ D_{21}^p& D_{22}^p& \cdots & D_{2n}^p\\ ⋮& ⋮& ⋮& ⋮\\ D_{m1}^p& D_{m2}^p& \cdots & D_{mn}^p\end{array}\right)$$

(3)

where i is the number of FM, i = {1, $\cdots$, m}, j is the number of OM, i = {1, $\cdots$, n}.

Given k experts involved in the evaluation, we have k evaluating matrices for each parameter. Then, we constructed average evaluating matrix $\overline{O}$, $\overline{S}$, and $\overline{D}$ as follows:

$$\overline{O}={\left(O_{ij}\right)}_{m\times n}$$

(4)

$$\overline{S}={\left(S_{ij}\right)}_{m\times n}$$

(5)

$$\overline{D}={\left(D_{ij}\right)}_{m\times n}$$

(6)

where $O_{ij}=\left({\displaystyle \sum }_{p=1}^k{O}_{ij}^p\right)/k$, $S_{ij}=\left({\displaystyle \sum }_{p=1}^k{S}_{ij}^p\right)/k$, $D_{ij}=\left({\displaystyle \sum }_{p=1}^k{D}_{ij}^p\right)/k$.

The averaging results are inputs to calculate the RPNs of each FM in the three OMs, as follows:

$$RP{N}_q=O_{qj}{S}_{qj}{D}_{qj}$$

(7)

where $RP{N}_q$ is the RPN of FM q. Subscript q is the number of FMs. Subscript j is the number of the operational mode, i.e., j = 1 for manual control, j = 2 for remote control, j = 3 for autonomous control. O_qj is the probability of FM q in operational mode j, S_qj is the severity of FM q in operational mode j, and D_qj is the probability of not detecting FM q in operational mode j.

3.3. Step 3: Calculate RPN of a Scenario in a Given Operation Mode

To calculate the RPN of a scenario in a specific OM, a formula is proposed as follows.

$$RP{N}_j={\displaystyle \sum }_{q=1}^{t}RP{N}_q$$

(8)

where t is the total number of FMs in the scenario. RPN_j is the RPN of a scenario in the operational mode j.

3.4. Step 4: Analyze Results and Provide Suggestions

From Step 3, the results could be further analyzed to provide feasible solutions for decision or policy makers. To this end, we used the number of FMs in the scenario to normalize the RPN of the scenario; the resulting sRPN is shown in Equation (9). We also set sRPN ranges to provide corresponding suggestions on switching OM as shown in

Table 5. Suggestions for changing OM.

Range	MC	RC	AC	Suggestions
Range 1	(1, 125]	(1, 64]	(1, 27]	S1: no change, but attention.
Range 2	(125, 512]	(64, 343]	(27, 216]	S2: if changing OM is possible, switch to an OM in which the sRPN is less than Range 2; if changing OM is impossible or unnecessary, control involved FM(s) by reducing occurrence or severity, or improving detection in the current OM.
Range 3	(512, 1000]	(343, 1000]	(216, 1000]	S3: if changing OM is possible, switch to an OM with less autonomy, e.g., switching from AC to RC or RC to MC; if changing OM is impossible, control involved FM(s) by reducing occurrence or severity, or improving detection in the current OM.

.

$$sRP{N}_j=\frac{1}{t}RP{N}_j$$

(9)

4. Case Study

In this case study, we present how to apply the proposed framework in Section 2 to assess a MASS’s operational risk in a given scenario. Results are expected to guide decision or policy makers in selecting the appropriate OM or taking measures to reduce risk in the current OM. The scenario was derived from a model MASS test using the 24 Model. In order to evaluate the three RPN parameters of FMs with respect to the three considered OMs, nine seafarers were interviewed.

4.1. Step 1: Identify Potential Failure Modes

A model MASS has been trialed in Qinhuai River, Nanjing since October 2019. Figure 1 shows a potential accident of this model MASS trail in Qinhuai River. For more details on this model ship, please refer to [28,29]. During its voyages, a ship–bank allision accident was recorded in the OM of AC (see Figure 2).

On the basis of the accident recorded in this trial, we applied the 24 Model to identify FMs. The identified accident paths and elements are shown in Figure 3, in which A1, A2, A3, B1, and B2 are immediate causes, C1 is an indirect reason, and F1 and F2 are external factors. However, from the accident, there were neither radical (safety-management system), nor root (safety culture) causes.

On the basis of the above accident analysis, five FMs were identified: FM₁, GPS information loss; FM₂, improper assessment of ship position; FM₃, deviation in course; FM₄, improper sensing; FM₅, negligence of watchkeeping.

4.2. Step 2: Evaluate Three RPN Parameters of FM in Given Operational Mode

The three RPN parameters of the five FMs identified in Step 1 with respect to the three OMs were evaluated by nine seafarers, whose profiles are shown in

Table 6. Profiles of interviewed seafarers.

No	Professional Position	Professional Experience (Years)	Educational Qualification	Type of Sailed Vessel with the Most Time
1	Chief officer	10	College	Container
2	Second officer	8	MSc	Bulk carrier
3	Captain	12	MSc	Container
4	Chief officer	10	BSc	Container
5	Third officer	7	MSc	Bulk carrier
6	Third engineer	5	PhD	Bulk carrier
7	Pilot	10	BSc	Container/bulk carrier
8	Captain	16	MSc	Bulk carrier
9	Captain	18	MSc	Bulk carrier

. In the MUNIN project, Kretschmann et al. [30] assumed that the employment plan for a shore control center involves 5 to 8 people for a remotely controlled MASS in one shift, in which 5 people for 24/7 operation and 3 people for one shift operation are needed. Hence, the number of seafarers was assumed to be reasonable to judge OM switching for a MASS. The evaluations on the three RPN parameters of the five FMs were averaged using Equations (4)–(6), and results are presented in

Table 7. RPNs of identified FMs in the three OMs.

FM		Oqj			Sqj			Dqj
	q	j = 1	j = 2	j = 3	j = 1	j = 2	j = 3	j = 1	j = 2	j = 3
$F{M}_1$: GPS information loss	1	3.78	4.67	5	5.22	6.44	6.44	4.78	4.78	4.11
$F{M}_2$: improper assessment of ship position	2	5.56	5.33	5.11	6	6.11	7	5.11	4.78	4.44
$F{M}_3$: deviation in course	3	5.78	4.56	4.56	4.78	4.89	6.22	4.44	4.11	5.56
$F{M}_4$: improper sensing	4	5.22	5.56	5.89	4.78	6	6.44	5.33	5	4.33
$F{M}_5$: negligence of watchkeeping	5	7.22	5.78	3.89	7	6.44	5.67	5.22	5.33	4.89

. According to Equation (7),

Table 8. RPNs of five FMs in three operational modes.

FM		RPN of FM
	q	j = 1	j = 2	j = 3
$F{M}_1$: GPS information loss	1	94.26	143.69	132.47
$F{M}_2$: improper assessment of ship position	2	170.37	155.72	159.01
$F{M}_3$: deviation in course	3	122.69	91.56	157.48
$F{M}_4$: improper sensing	4	133.07	166.67	164.45
$F{M}_5$: negligence of watchkeeping	5	264.01	198.58	107.74

shows their RPNs in the three OMs.

4.3. Step 3: Calculate RPN of a Scenario in Given Operation Mode

According to Equation (8), we obtained the RPNs of the scenario with respect to three OMs, i.e., 784.4 in MC, 756.22 in RC, and 721.15 in AC, as shown in Figure 4. According to Equation (9), Figure 4 also presents the sRPN of the scenario with respect to the three OMs, i.e., 156.88 in MC, 151.24 in RC, and 144.23 in AC.

4.4. Step 4: Result Analysis and Suggestions

RPNs in Figure 4 show that, under a certain scenario, an MASS is expected to encounter different levels of risk when operating under three considered OMs. On the basis of the sRPN results, suggestions are presented.

On the one hand, accounting for FMs, e.g., $F{M}_1$—GPS information loss, and $F{M}_4$—improper sensing, changing from AC to RC immediately is highly unlikely. $F{M}_1$ means that the communication link via GPS satellites is cut off. Even if the communication is acceptable, $F{M}_4$ indicates that the information of the perception sent from the onboard side to the onshore side may be a challenge to generate sufficient situation awareness. Maintaining such situational awareness may be a challenge if communication quality is unsatisfactory, which, in turn, increases the operational risk in RC. This means that changing from AC to RC may not reduce risk, but also bring new risk.

On the other hand, although there is no nearby crew who can embark on the model MASS in a timely manner, some onboard experimenters can immediately take over the ship and quickly handle the emergency. Hence, changing from AC to MC seems possible for the model MASS. In the scenario, the current OM is AC. Figure 4 shows that the sRPN in AC is 144.23, which belongs to Range 2. According to Table 5, if changing OM is possible, the onboard side should switch to an OM in which the sRPN is less than Range 2. However, values of sRPN in the two other OMs were also in the corresponding Range 2, i.e., 156.88 ∈ (125, 512], 151.24 ∈ (64, 343]. This means that, even though changing from AC to MC or RC is possible, the sRPN would still be within Range 2.

On the basis of the above analysis, changing from AC to RC or MC was unavailable. In this context, according to Table 5, the model MASS should control the involved FM(s) by reducing occurrence or severity, or improving detection capability in the current OM, i.e., AC. For example, when $F{M}_1$—GPS information loss occurred, the model MASS should have used input from another satellite, e.g., the BeiDou Navigation Satellite System, to obtain the ship position. When $F{M}_4$—improper sensing happened, the model MASS could have automatically adjusted the gain in radar to obtain proper quality in perception. These emergency responses may be feasible for onboard crew who could take measures in a timely manner to reduce or eliminate the consequence(s) of these FMs, which is the disadvantage of MASS in AC. Therefore, the proper postaccident behavior of MASS within AC is vital to overall safety (Wróbel et al. [31]).

5. Discussion

5.1. Study Contributions

This study contributes in the following three aspects.

First, the proposed four-step framework may not only be helpful for a MASS system to judge OM switching to reduce operational risk, but also be used as a demonstration to guide or train future operators in SCC to evaluate operational risk with respect to the three considered OMs.

Second, in MASS, there are no criteria based on operational risk in terms of RPN to generate corresponding suggestions for risk mitigation. Although any elaboration on MASS safety levels are inevitably incomplete due to the current lack of historical or empirical data (Wróbel et al. [32]), it is essential to develop risk-acceptance criteria, especially for MASS (Utne et al. [33]). In this study, the values of the criteria for evaluating the RPN of a scenario were different in terms of different OMs. These different ranges for the same suggestion in three OMs indicate that the AC criteria are the strictest, followed by those for RC and MC, which may provide insight for developing future risk-acceptance criteria for MASS.

Third, mitigation measures were proposed according to the sRPN range in each OM. If the obtained sRPN is located in Range 1, we suggest to keep the OM but pay attention on the increased risk. If the obtained sRPN is located in Range 2, we suggest to switch to an OM in which the sRPN is lower than Range 2. If the obtained sRPN is located in Range 3, we suggest to switch to an OM in which the DoA is less than the current DoA. However, if changing OM is not possible due to, e.g., communication delay, technical impossibility, time limitation, or because it is not necessary, measures may be taken to mitigate or even eliminate some or all involved FMs to reduce the RPN or risk in the current OM. In this sense, two directions to reduce the RPN of the involved FM are also given. One way is to reduce the occurrence and/or control the severity of the involved FM, and another is to improve the capability of detection of the involved FM.

5.2. Study Limitations

This study has two main limitations of the study.

The first is the shortcoming of evaluating three RPN parameters of FMs. The evaluation of O, D, and S of the FM in a given OM relies on expert elicitation, which may bring about interpersonal and intrapersonal uncertainty. In the future, rather than fully relying on subjective data, the three RPN parameters of some FMs, e.g., the occurrence of $F{M}_1$ (GPS information loss) and $F{M}_4$ (improper sensing) could be objectively obtained.

The second is the deficiency in the suggestions for switching OM. There are three main reasons for this deficiency: first, in the context of autonomous shipping, there exists no standard or regulation to determine whether the operational risk in terms of RPN is acceptable. If the RPNs in the three considered OMs are acceptable, it is not necessary to change OM even if there are differences among them. Drawing up and carrying out such regulations internationally rely on the International Maritime Organization. On the other hand, if their RPNs are unacceptable, it could consider other models, e.g., fail to safe. Second, it does not consider factors affecting such changes, e.g., communication, time, crew, and cost (Ewelina and Montewka [34]). If communication between the ship and SCC fails, it is hard to imagine that remote control can successfully or immediately take over the ship. Third, even though one OM is determined to change, this does not consider whether such change brings new risks or not. When changing from AC into RC, does the operator onshore quickly and correctly become acquainted with the situation? Does this process result in a greater risk of cyberattacks? These are still open issues.

6. Conclusions

This paper proposed a four-step framework to quantify the operational risk of a maritime autonomous surface ship (MASS) in three operational modes (OMs), i.e., manual., remote, and autonomous control. In this framework, FMs were identified from accident analysis using the 24 Model. Then, three RPN parameters of the identified FMs were evaluated with respect to three considered OMs by experts using crisp values. The average of the experts’ judgement was generated as input for calculating RPN of FM in a given OM. The RPN for the scenario in the given OM was calculated by adding the RPN of the involved FMs. That is, the RPN for a scenario in an OM was the summation of the RPN of FMs involved in that OM. On the basis of RPNs in three OMs, we provided some suggestions on OM switching.

In the case study, five failure modes (FMs) were defined using 24 Model on a model MASS accident: GPS information loss, improper assessment of ship position, deviation in course, improper sensing, and negligence of watchkeeping. These five FMs as potential FMs for MASS accident scenario were further analyzed following the proposed framework. Results showed that switching to manual or remote control is not useful to reduce risk. Identified FM(s) should be controlled by reducing occurrence or severity, or detection in the current OM should be improved, i.e., autonomous control. Nevertheless, for MASS with onboard crew, once the two failure modes of GPS information loss and improper sensing occurred, changing to manual control would reduce such a risk in a timely manner.