Situation Awareness-Based Safety Assessment Method for Human–Autonomy Interaction Process Considering Anchoring and Omission Biases

Abstract

Autonomy is being increasingly used in domains like maritime, aviation, medical, and civil domains. Nevertheless, at the current autonomy level, human takeover in the human–autonomy interaction process (HAIP) is still critical for safety. Whether humans take over relies on situation awareness (SA) about the correctness of autonomy decisions, which is distorted by human anchoring and omission bias. Specifically, (i) anchoring bias (tendency to confirm prior opinion) causes the imperception of key information and miscomprehending correctness of autonomy decisions; (ii) omission bias (inaction tendency) causes the overestimation of predicted loss caused by takeover. This paper proposes a novel HAIP safety assessment method considering effects of the above biases. First, an SA-based takeover decision model (SAB-TDM) is proposed. In SAB-TDM, SA perception and comprehension affected by anchoring bias are quantified with the Adaptive Control of Thought-Rational (ACT-R) theory and Anchoring Adjustment Model (AAM); behavioral utility prediction affected by omission bias is quantified with Prospect Theory. Second, guided by SAB-TDM, a dynamic Bayesian network is used to assess HAIP safety. A case study on autonomous ship collision avoidance verifies effectiveness of the method. Results show that the above biases mutually contribute to seriously threaten HAIP safety.

1. Introduction

1.1. Background

Nowadays, autonomy systems have been widely applied and succeeded in improving efficiency in domains such as medical, civil, aviation, and maritime domains [1]. However, autonomy technologies are currently not yet mature and have caused some serious accidents like the crash of Boeing 737 MAX. Learning from other industries, most of the maritime autonomy concepts consider human–autonomy interaction (HAI) as a means of compensating the defects of autonomy and ensuring safety [2]. During the HAI process (HAIP), a human in a Shore Control Centre (SCC) is required to act as a supervisor, and initiates takeover when an autonomy decision is incorrect [3]. Their situation awareness (SA) about correctness of autonomy decisions is the primary rationale of takeover decisions [4]. A well-accepted model by Endsley [5] describes SA as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status. This paper adheres to this definition and specifies SA about correctness of an autonomy decision as the perception of information reflecting the autonomy decision process, comprehension about the correctness of the autonomy decision, and prediction about future states if takeover behavior or the autonomy decision was executed. Based on SA prediction, humans can also predict and compare the utility of takeover and autonomy decisions. In the maritime domain, some capabilities of autonomy have been explored to support SA, such as autonomy transparency, which exposes the autonomy decision process to humans [1,6]. Nevertheless, humans seem not yet adapted to the role change from manipulators to supervisors and correct takeover is difficult for them [2]. It can be attributed to cognitive biases that exist in the takeover decision process. (Cognitive biases are systematic and universally occurring tendencies, inclinations, or dispositions that distort the cognitive process and adversely affect decision making [7]. In existing studies, cognitive biases are widely found in the fields of management, medical treatment, and emergency response [8,9,10]. Currently, commonly discussed cognitive biases include anchoring bias, omission bias, availability bias, base rate neglect, gambler’s fallacy, hindsight bias, and others [11]).

During HAIP in risk scenarios, humans, such as operators in SCC, could have achieved correct SA and takeover decisions based on transparency information and information from other sources (e.g., cameras, GPS, and many other types of sensors onboard the ship) [12]. However, in critical or urgent risk scenarios, humans tend to use the simple and quick heuristic cognitive mechanisms, which introduces cognitive biases into takeover decisions [13]. Among many cognitive biases, anchoring bias and omission bias are two common types that affect SA and takeover decisions [14]. Anchoring bias is the phenomenon that humans’ preference for a previous opinion (i.e., anchor) makes them ignore information disproving the anchor and tend to keep comprehension consistent with the anchor [15]. During HAIP, if the anchor is set to believing that an autonomy decision is correct, anchoring bias leads to varying degrees of neglect of transparency information, and biases their comprehension about correctness of the autonomy decision, which makes humans overlook errors of the autonomy decision. Omission bias is a phenomenon that a human is more averse to loss caused by action than loss caused by inaction, and thus they prefer inaction [16]. During HAIP, omission bias makes humans more averse to the loss caused by takeover than loss caused by autonomy decisions, which leads to the overestimation of the former. It makes humans tend not to take over. Therefore, in HAIP, anchoring bias and omission bias, when they coincidentally occur, generate an increased negative effect, which prevents the correct takeover. This paper aims to develop a safety assessment method for HAIP considering anchoring and omission biases. The proposed method is designed for use during the design and development phases of human–autonomy systems. It is intended to assist designers in evaluating the risk-avoidance capabilities of autonomy systems when integrated with human operations and to provide technical support for improving system design.

1.2. Literature Review

The effects of anchoring bias and omission bias have been qualitatively explored in many studies. According to Endsley [15], anchoring bias guides perception and comprehension in a way that confirms prior opinions. If previous opinions are incorrect, this tendency has negative effects on subsequent SA [17]. An experiment by Walmsley and Gilbey [10] demonstrated that operators affected by anchoring bias tend to ignore information disproving previous opinions and misinterpret the newly perceived evidence. Regret theory [18] and loss-aversion theory [9,19] are often used to explain the mechanism of omission bias. They argued that omission bias leads to more pessimistic subjective cognition to the predicted loss caused by action and the overestimation of it. The above studies indicate that anchoring bias and omission bias are critical causes of SA errors. Considering that SA is the critical rationale of correct takeover decisions, it is significant to incorporate effects of anchoring bias and omission bias into HAIP safety assessment.

Many methods have been developed to assess HAIP safety. In studies related to maritime autonomous surface ships (MASSs), traditional risk assessment methods are often used to assess HAIP safety, including a fault tree analysis (FTA) [20], Bayesian network (BN) [21], Systems Theoretic Process Analysis (STPA) [22], and others. For example, Li et al. [23] used FTA to model the causal relationship among the risk factors and mapped FTA into BN to assess risk of MASS. Zhang et al. [24] used the hybrid causal logic (HCL), which integrates FTA and BN, to quantify accident probability of MASS. Li et al. [25] utilized STPA to analyze risk of MASS navigation under the human–machine co-driving mode. Sumon et al. [26] used STPA to analyze safety of hydrogen-driven MASS. The above methods focused on the causal relationships between accidents and factors like autonomy failures or human errors, and aimed to find critical risk-influencing factors or propagation paths. They successfully assessed HAIP safety from the system level. Nevertheless, they are inadequate to dig deep into the takeover decision process to analyze mechanisms of accidents.

In studies related to human–robot collaboration and autonomous vehicles, HAIP safety assessment is often performed based on trust quantification to analyze the mechanism of incorrect takeover decisions. They used trust as the main rationale of takeover decisions and treated trust as the mediator between HAIP safety and its affecting factors [27]. For example, Yang et al. [28] treated trust as the probability to accept autonomy and established a hidden Markov model to assess HAIP safety. In their model, a trust update affected by autonomy reliability was quantified with Beta distribution [29]. Moreover, some research utilized decision field theory to quantify the update of trust and self-confidence and predicted the occurrence of takeover when their difference evolves below a given threshold [30,31,32]. Based on this method, Gao and Lee [33] analyzed the effects of transparency levels on trust and HAIP safety. However, the above methods ignored the SA, which is a main rationale for takeover decisions.

The key to solve this problem is to quantitatively model SA and incorporate it into the takeover decision model. As SA is the widely used framework describing human cognition, its quantification has been extensively researched. For example, the well-known Adaptive Control of Thought-Rational (ACT-R) theory quantified perception probability of information with its activation level [34]. Zhao and Smidts [35] extended the activation level of information to two dimensions, including its saliency and value. To quantify SA comprehension, a Bayesian network, which has an advantage in encoding human knowledge, was used to model the human reasoning process [36]. For example, Naderpour et al. [37,38] used a Bayesian network to construct operators’ mental model and quantified their comprehension of the system state. To quantify behavioral utility prediction based on SA, Prospect Theory is a commonly used method [39]. Prospect Theory is often used to develop autonomous systems to make them predict the utility of behavior or decisions like humans [40,41].

1.3. Gaps and Contribution

Quantitative modeling methods of SA provide a chance to overcome the problem that trust-based HAIP safety assessment methods ignore SA. Nevertheless, existing quantitative modeling methods neglect the effects of anchoring bias on SA perception and comprehension, as well as the irrational preference in behavioral utility prediction caused by omission bias. Qualitative research clarified how anchoring bias and omission bias lead to incorrect takeover decisions. They provided sufficient theoretical guidance to fulfill the above gaps.

This paper proposes a novel HAIP safety assessment method, which considers the effects of anchoring bias and omission bias. Firstly, we investigate the HAIP affected by anchoring bias and omission bias, and identify its characteristics. Based on the analysis, an SA-based takeover decision model (SAB-TDM) is developed to analytically describe the takeover decision based on SA. In SAB-TDM, SA perception and comprehension affected by anchoring bias are represented by ACT-R theory [42] and the Anchoring Adjustment Model (AAM) [43], respectively. Meanwhile, Prospect Theory [44] is used to describe the behavioral utility prediction affected by omission bias. Then, a dynamic Bayesian network (DBN) is used to model HAIP and assess safety, due to its ability to encode human knowledge and describe time dependencies [45,46]. The structure establishment and parameter determination of DBN are supported by the proposed SAB-TDM. The applicability condition of our method is that the autonomy system interacts with a single operator provided with multiple information sources and the takeover decision is made by this operator only. A case study on multi-round collision avoidance for an autonomous ship is used to illustrate the effectiveness of the proposed method.

The contributions and innovations of this paper are as follows: (1) this paper provides a new perspective to analyze HAIP safety by incorporating SA about correctness of autonomy decisions into takeover decisions; (2) to the best of the authors’ knowledge, this is the first work to consider anchoring bias and omission bias in HAIP safety assessment; (3) the SAB-TDM is proposed to quantify SA perception and comprehension affected by anchoring bias with ACT-R and AAM, and to quantify utility prediction affected by omission bias with Prospect Theory; and (4) a DBN-based method is proposed to model the HAIP and assess its safety.

1.4. Article Structure

The remainder of this paper is divided into five sections. Section 2 describes the research problem and the framework of the method. The proposed method is detailed in Section 3 and Section 4. Section 3 proposes the SAB-TDM considering anchoring and omission biases, and Section 4 develops the DBN-based method to model HAIP and assess its safety. Section 5 presents the method application and result analysis. Finally, Section 6 discusses the major conclusion of this paper.

2. Research Problem and Framework of Method

This section firstly analyzes the characteristics of HAIP affected by anchoring bias and omission bias. Secondly, the framework of the proposed method is introduced. In HAIP, based on transparency information and information from other sources (e.g., cameras, GPS, and many other types of sensors onboard), humans could have achieved correct SA and takeover decisions. However, anchoring bias leads to neglect of transparency information and miscomprehending correctness of autonomy decisions; and omission bias makes humans overestimate the loss caused by takeover. These two cognitive biases mutually contribute in HAIP to impede correct takeover decisions. Based on these characteristics, SAB-TDM is developed to support the construction of DBN, which is used to model the HAIP and assess its safety.

2.1. Research Problem Statement

HAIP in risk scenarios is shown in Figure 1. An autonomy system firstly makes a risk-avoidance decision. At the same time, humans achieve SA about the correctness of the autonomy decision. This SA can support the takeover decision that is to dictate whether to take over. If humans do not take over within defined time, the autonomy system will automatically act out its risk-avoidance decision. When the autonomy decision is incorrect, human takeover is critical to ensure safety. However, anchoring bias leads to errors in SA and omission bias distorts the takeover decision, which both impede correct takeover.

To support SA and the takeover decision, autonomy transparency is developed to expose the autonomy decision process to humans. This paper follows the framework of the Situation awareness-based Agent Transparency (SAT) model [47] and specializes it to meet the needs of HAIP in risk scenarios. Specifically, SAT information includes SAT1, autonomy’s inputs and the proposed actions; SAT2, autonomy’s reasoning of current risk; and SAT3, autonomy’s utility prediction for its risk-avoidance decision. The above SAT information effectively supports HAIP.

To complete risk avoidance, multi-round HAIP is often needed [48]. The multi-round HAIP is shown in Figure 2. In each round, due to the faults of sensors, the autonomy may make wrong decisions along with errors in SAT information. Humans firstly perceive SAT information through oriented attention allocation [38]. According to information from other sources (e.g., visual scene or communication), the correctness of perceived SAT information can be verified [49]. Any error of SAT information indicates error of autonomy decisions. When humans cannot clearly comprehend the correctness of an autonomy decision, they need to predict the future state if takeover behavior and the autonomy decision was performed. Only based on this prediction can the utility of takeover and autonomy decisions be predicted and compared. When finding errors of autonomy decisions or predicted utility of autonomy decisions is not optimal, humans will initiate takeover. If the risk is avoided successfully or an accident occurs, the HAIP in this risk scenario is finished. Otherwise, the next round of HAIP is needed.

This paper assumes that a human anchor is set to believing that an autonomy decision is correct and the initial trust level on the anchor (i.e., anchor belief) equals initial trust. Guided by the anchor whose belief updates at the end of each round, anchoring and omission bias lead to causing SA errors and impede correct takeover during multi-round HAIP. The effects of anchoring bias and omission bias are indicated by the arrow lines with different numbers in Figure 2, and introduced as follows:

① SA perception is round-dependent due to anchoring bias. Affected by anchoring bias, humans tend to perceive information supporting the anchor updated in the last round [50]. SAT information is set to help humans find the errors of autonomy decisions, which has the natural quality of opposing the anchor. Therefore, affected by anchoring bias, SAT information will be assigned with a smaller value and allocated with less attention. This will result in the inadequate perception of SAT information.

② SA comprehension is round-dependent due to anchoring bias. SA comprehension is to judge the correctness of autonomy decisions. Affected by anchoring bias, humans have preference for the anchor updated in the last round, which will distort the estimation of evidence’s causal strength [51,52]. Specifically, the causal strength of evidence indicating autonomy decisions without errors is overestimated and the causal strength of evidence indicating autonomy decisions with errors is underestimated. It makes humans overlook the errors of autonomy decisions and tend not to take over, which means the promotion of anchoring bias on omission bias.

③ Behavioral utility prediction has irrational preference due to omission bias. Behavioral utility prediction is based on SA prediction. Affected by the omission bias, humans more readily accept losses caused by inaction compared to equal or even less loss caused by action [16]. It leads humans to have a more pessimistic subjective cognition to losses caused by takeover and overestimate it. That will make humans fail to identify the condition that an autonomy decision is not optimal and inclined not to take over. It also can improve anchor belief and promote effects of anchoring bias.

A human’s takeover behavior in the current round will update anchor belief. If they choose takeover, anchor belief will decrease. Otherwise, anchor belief increases. After the generation of a new autonomy decision and SAT information, the updated anchor will continue to bias SA perception and comprehension, which makes them round-dependent. Therefore, under the mutual promotion of the anchoring bias and omission bias, a human is prone to ignore critical SAT information, overlook errors of the autonomy decision, and believe that utility of a wrong autonomy decision is optimal. These errors impede the correct takeover decision, and lead to an accident when autonomy makes wrong risk-avoidance decisions.

2.2. Framework of Proposed Method

Based on effects of anchoring bias and omission bias on HAIP, an HAIP safety assessment method is proposed. The overall framework of the method is shown in Figure 3. Firstly, the SAB-TDM is developed to analytically represent a takeover decision based on SA and quantify the effects of anchoring and omission biases. In SAB-TDM, the perception of SAT information and its round dependency caused by anchoring bias are quantified with ACT-R [53]; comprehension about the correctness of the autonomy decision and its round dependency caused by anchoring bias are quantified with an intuitionistic fuzzy set (IFS) [54] and AAM [43]; and the irrational utility prediction of takeover behavior is quantified with Prospect Theory [9]. Secondly, DBN is used to model multi-round HAIP and assess its safety. In DBN, SAT information perception, comprehension about the correctness of the autonomy decision, and utility prediction affected by omission bias guide the structure establishment and parameter determination of a single Bayesian network. The SA perception and comprehension round dependencies caused by anchoring bias guide the structure establishment and parameter determination of adjacent networks. The above methods are detailed in the following two sections.

3. SA-Based Takeover Decision Model Considering Anchoring and Omission Biases

According to the analysis about the characteristics of multi-round HAIP, this section develops the SAB-TDM to represent a human’s takeover decision based on SA. The core is to quantify the negative effects of anchoring bias on SA achievement and the negative effects of omission bias on behavioral utility prediction.

3.1. Overview of SA-Based Takeover Decision Model

A takeover decision in multi-round HAIP is based on SA, which takes SAT information as inputs and is affected by anchoring bias and omission bias. Figure 4 illustrates the logic of takeover decisions based on SA. Humans firstly perceive SAT information under the effects of anchoring bias. According to information from other sources, humans verify the perceived SAT information to comprehend the correctness of autonomy decisions. The comprehension is distorted by anchoring bias and has three result states, that is, without errors, with errors, and hesitancy. Only if humans cannot clearly identify whether errors exist in an autonomy decision is when they need to predict and compare the utility of takeover and the autonomy decision, which is affected by omission bias. If the autonomy decision has errors or takeover is optimal, humans will choose to take over.

According to Figure 4, the SAB-TDM can be expressed as

$$\left\{\begin{array}{l}{p}_{(TO{D}_r=a)}=(p_{(C{O}_r=y|P{E}_r,A_{r-1})}+p_{(P{R}_r=ad|E_r)}{p}_{(C{O}_r=h|P{E}_r,A_{r-1})})p_{(P{E}_r|A_{r-1})}{p}_{(A_{r-1})}\\ p_{(TO{D}_r=t)}=1-p_{(TO{D}_r=a)}\end{array}\right.$$

(1)

where $TO{D}_r\in \{a,t\}$ represents different states of the takeover decision, including accepting or taking over autonomy in round $r$; $A_{r-1}\in \{b,u\}$ represents belief or unbelief on the anchor updated in round $r-1$; $P{E}_r\in \{o,no\}$ is the successful observation and neglect of information; $C{O}_r\in \{y,n,h\}$ represents three states of correctness judgment, that is, without errors, with errors, and hesitancy; $P{R}_r\in \{ad,hd\}$ means that the predicted utility of the autonomy decision or takeover is optimal; and $E_r$ is the evidence from other sources.

$p_{(TO{D}_r)}$ is decided by SA about the correctness of the autonomy decision and utility prediction. Affected by anchoring bias, SA perception and comprehension are guided by the belief level of anchor $p_{(A_{r-1})}$. In the initial round, $p_{(A_{r-1}=b)}=Trust,(Trust\in [0,1])$ and in the following round, $p_{(A_{r-1}=b)}=p_{(TO{D}_{r-1}=a)}$. Conditional probability $p_{(P{E}_r|A_{r-1})}$ and $p_{(C{O}_r|P{E}_r,A_{r-1})}$ reflect perception and comprehension round dependency caused by anchoring bias, respectively. $p_{(P{E}_r|A_{r-1})}$ indicates the perception probability of information determined by anchor $A_{r-1}$. $p_{(C{O}_r|P{E}_r,A_{r-1})}$ indicates that perceptions of SAT information $P{E}_r$ and anchor $A_{r-1}$ decide a human’s belief on correctness of the autonomy decision. When $C{O}_r=h$, the utility of takeover and the autonomy decision needs to be predicted and compared. Conditional probability $p_{(P{R}_r|E_r)}$ represents the behavioral utility prediction affected by omission bias. These conditional probabilities can be determined as follows.

3.2. Quantification of SA Perception Affected by Anchoring Bias

The determination of $p_{(P{E}_r|A_{r-1})}$ is to quantify the perception of SAT information and the perception round dependency caused by anchoring bias. It essentially is the quantification of attention allocation guided by the anchor. Related studies often divide the displaying interface into several areas of interest (AOIs) and uniformly measure the perception probability of information in the same AOIs [55]. This paper divides the SAT information at the same level into an AOI and quantifies its perception with ACT-R theory [53]. ACT-R theory uses the salience and value of information to measure its activation [35]. A higher activation level means more attention and higher perception probability.

Affected by anchoring bias, humans adjust their attention allocation for SAT information according to their attitude to the anchor updated in the last round. The adjustment is reflected in value change in SAT information. Specifically, belief of the anchor will decrease the value of SAT information and unbelief of the anchor leads to an increase in the value. It can be expressed as

$$A_{ri}^V=\left\{\begin{array}{ll}{A}_{ri(A_{r-1}=b)}^V& Believe Ancho{r}_{r-1}\\ A_{ri(A_{r-1}=u)}^V& Unbelieve Ancho{r}_{r-1}\end{array}\right.$$

(2)

where $A_{ri}^V$ is the subjective value of $\mathrm{SAT}{i}_{(i\in \{1,2,3\})}$ information in round $r$; $A_{r-1}\in \{b,u\}$ indicates belief and unbelief of the anchor in round $r-1$. $A_{ri(A_{r-1}=b)}^V$ and $A_{ri(A_{r-1}=u)}^V$ represent the value of $\mathrm{SAT}i$ information in round $r$ under the condition that humans believe and unbelieve $A_{r-1}$, respectively. Obviously, $A_{ri(A_{r-1}=b)}^V<A_{ri(A_{r-1}=u)}^V$.

Then, the perception round dependency $p_{(P{E}_r^i|A_{r-1})}$ caused by anchoring bias can be calculated with

$$p_{(P{E}_r^i=o|A_{r-1})}={\displaystyle \frac{1}{1+e^{-(A_{ri}-\tau )/s}}}$$

(3)

$$A_{ri}=\gamma A_{ri}^S+\lambda A_{ri}^V$$

(4)

where $P{E}_r^i\in \{o,no\}$ is the successful observation and neglect of $\mathrm{SAT}i$ information; $A_{ri}$ is the activation of $\mathrm{SAT}i$ information in round $r$; $\tau$ is the threshold of activation; and $s$ is the influence of noise in SAT information perception. $A_{ri}^S$ is the salience of AOI containing $\mathrm{SAT}i$ information; $\gamma$ and $\lambda$ are the weights of the salience and value, and $\gamma +\lambda =1$.

3.3. Quantification of SA Comprehension Affected by Anchoring Bias

The determination of $p_{(C{O}_r|P{E}_r,A_{r-1})}$ is to quantify the SA comprehension based on SAT information and the comprehension round dependency caused by anchoring bias. In HAIP, SA comprehension is to judge the correctness of autonomy decisions, according to the correctness of perceived SAT information. The comprehension is distorted to different degrees according to anchor belief updated in the last round, which makes comprehension round-dependent.

(1) Quantify SA comprehension based on SAT information. The correctness judgment of SAT information and autonomy decisions is benchmarked on information from other sources. $T_r^i=\{t_r^{i1},\dots ,t_r^{ij},\dots ,t_r^{iN}\}$ and $B_r^i=\{b_r^{i1},\dots ,b_r^{ij},\dots ,b_r^{iN}\}$ are used to represent the transparency information set at level $\mathrm{SAT}i$ and its benchmark information set, where $N$ is the amount of information at level $\mathrm{SAT}i$. SAT information $t_r^{ij}$ and its corresponding benchmark information $b_r^{ij}$ form an information tuple, $x_r^{ij}=<t_r^{ij},b_r^{ij}>$. Correctness judgment of $t_r^{ij}$, i.e., $C{O}_r^{ij}$, depends on a human’s cognition about the consistency of elements in $x_r^{ij}$, which is often fuzzy. This paper uses IFS to model the human cognition, which is fuzzy in nature [54,56].

Let $X_{ij}$ be a universal set. The IFS of $t_r^{ij}$ correctness (i.e., $A_{ij}$) on $X_{ij}$ can be expressed as

$$A_{ij}=\{<x_r^{ij},{\mu }_{A_{ij}}(x_r^{ij}),{\nu }_{A_{ij}}(x_r^{ij})>|x_r^{ij}\in X_{ij}\}$$

(5)

where ${\mu }_{A_{ij}}(x_r^{ij}):X_{ij}\to [0,1]$ and ${\nu }_{A_{ij}}(x_r^{ij}):X_{ij}\to [0,1]$ are the membership function and non-membership function, respectively, which follow $0\le {\mu }_{A_{ij}}(x_r^{ij})+{\nu }_{A_{ij}}(x_r^{ij})\le 1$. The hesitancy function (i.e., ${\pi }_{A_{ij}}(x_r^{ij})$) of $x_r^{ij}$ in $A_{ij}$ is defined as

$${\pi }_{A_{ij}}(x_r^{ij})=1-{\mu }_{A_{ij}}(x_r^{ij})-{\nu }_{A_{ij}}(x_r^{ij})$$

(6)

Then, a human’s belief on $t_r^{ij}$ without error, with error, and with hesitancy can be represented as $p_{(C{O}_r^{ij}=y)}={\mu }_{A_{ij}}(x_r^{ij})$, $p_{(C{O}_r^{ij}=h)}={\pi }_{A_{ij}}(x_r^{ij})$, and $p_{(C{O}_r^{ij}=n)}={\nu }_{A_{ij}}(x_r^{ij})$, respectively. The following reasoning rules are set to judge the correctness of $T_r^i$ (i.e., $C{O}_r^i\in \{y,n,h\}$, representing without error, with error, and hesitancy):

(i)

IF $P{E}_r^i=no$, THEN $C{O}_r^i=y$;

(ii)

IF $\exists j\in \{1,2,\dots ,N\},C{O}_r^{ij}=n$, THEN $C{O}_r^i=n$;

(iii)

IF $\forall j\in \{1,2,\dots ,N\},C{O}_r^{ij}=y$, THEN $C{O}_r^i=y$;

(iv)

IF $\exists j\in \{1,2,\dots ,N\},C{O}_r^{ij}=h\&\forall j\in \{1,2,\dots ,N\},C{O}_r^{ij}\ne n$, THEN $C{O}_r^i=h$;

(v)

IF $C{O}_r^{i-1}=n,i\in \{2,3\}$, THEN $C{O}_r^i=n$.

The correctness of an autonomy decision (i.e., $C{O}_r\in \{y,n,h\}$, representing without error, with error, and hesitancy) can be reasoned with

(vi)

IF $\exists i\in \{1,2,3\},C{O}_r^i=n$, THEN $C{O}_r=n$;

(vii)

IF $\forall i\in \{1,2,3\},C{O}_r^i=y$, THEN $C{O}_r=y$;

(viii)

IF $\exists i\in \{1,2,3\},C{O}_r^i=h\&\forall i\in \{1,2,3\},C{O}_r^i\ne n$, THEN $C{O}_r=h$.

(2) Quantify comprehension round dependency. According to the above rules, a human’s belief on the correctness of an autonomy decision (i.e., $p_{(C{O}_r|P{E}_r)}$) can be reasoned. $p_{(C{O}_r=y|P{E}_r)}$ represents the initial causal strength that perceived SAT information supports the anchor and can be distorted by anchoring bias. This comprehension round dependency is represented by AAM [57]. It classifies evidence into positive and negative and adjusts their causal strength with different mechanisms. The classification is based on the reference point $R$ (can be set to 0.5 [43]). The perceived SAT information is considered positive if $p_{(C{O}_r=y|P{E}_r)}\ge R$ and negative when $p_{(C{O}_r=y|P{E}_r)}<R$. Then, the causal strength adjustment of perceived SAT information guided by the anchor can be expressed with Equations (7)–(9):

$$p_{(C{O}_r=y|P{E}_r,A_{r-1}=b)}=\left\{\begin{array}{cc}\begin{array}{l}1+\alpha (p_{(C{O}_r=y|P{E}_r)}-R),\\ 1,\end{array}& \begin{array}{l}{p}_{(C{O}_r=y|P{E}_r)}<R\\ p_{(C{O}_r=y|P{E}_r)}\ge R\end{array}\end{array}\right.$$

(7)

$$p_{(C{O}_r=y|P{E}_r,A_{r-1}=u)}=\left\{\begin{array}{cc}\begin{array}{l}0,\\ \beta [p_{(C{O}_r|P{E}_r)}-R],\\ 1,\end{array}& \begin{array}{l}{p}_{(C{O}_r=y|P{E}_r)}<R\\ R\le p_{(C{O}_r=y|P{E}_r)}<1\\ p_{(C{O}_r=y|P{E}_r)}=1\end{array}\end{array}\right.$$

(8)

$$\left\{\begin{array}{l}{p}_{(C{O}_r=n|P{E}_r,A_{r-1}=b)}=(1-p_{(C{O}_r=n|P{E}_r,A_{r-1}=b)}){\displaystyle \frac{p_{(C{O}_r=n|P{E}_r)}}{1-p_{(C{O}_r=y|P{E}_r)}}}\hfill \\ p_{(C{O}_r=n|P{E}_r,A_{r-1}=u)}=(1-p_{(C{O}_r=n|P{E}_r,A_{r-1}=u)}){\displaystyle \frac{p_{(C{O}_r=n|P{E}_r)}}{1-p_{(C{O}_r=y|P{E}_r)}}}\hfill \\ p_{(C{O}_r=h|P{E}_r,A_{r-1}=b)}=(1-p_{(C{O}_r=h|P{E}_r,A_{r-1}=b)}){\displaystyle \frac{p_{(C{O}_r=h|P{E}_r)}}{1-p_{(C{O}_r=y|P{E}_r)}}}\hfill \\ p_{(C{O}_r=h|P{E}_r,A_{r-1}=u)}=(1-p_{(C{O}_r=h|P{E}_r,A_{r-1}=u)}){\displaystyle \frac{p_{(C{O}_r=h|P{E}_r)}}{1-p_{(C{O}_r=y|P{E}_r)}}}\hfill \end{array}\right.$$

(9)

where $\alpha$ and $\beta$ ($0<\alpha <\beta <1$) are constants reflecting a human’s sensitivity to negative and positive evidence. According to the gap between $\alpha$ and $\beta$, the anchoring bias can be divided into three levels: the high level (HAB $<\alpha ,\beta >=<0.2,0.8>$), middle level (MAB $<\alpha ,\beta >=<0.5,0.8>$), and low level (LAB $<\alpha ,\beta >=<0.8,0.8>$) [58].

3.4. Quantification of Utility Prediction Affected by Omission Bias

The determination of $p_{(P{R}_r|E_r)}$ is to quantify the utility prediction and comparison of an autonomy decision and takeover behavior, which is based on SA prediction and has irrational preference due to omission bias. Affected by omission bias, a human is more averse to loss caused by takeover. The irrational utility prediction of takeover behavior is quantified with Prospect Theory.

The finite risk-avoidance decision space is formulated as $D=\{d_1,\dots ,d_K\}$. The prerequisite to predict the utility of risk-avoidance decision $d_i\in D$ is to predict the future state if $d_i$ was executed. Based on the SA prediction, the actual utility can be measured. The above cognition process can be expressed as

$$\left\{\begin{array}{c}PRS=f_P(E,d_i)\\ x_i=f_U(PRS)\end{array}\right.$$

(10)

where $PRS$ is the future state prediction; $E$ is the evidence observed from other sources; $f_P(·)$ is the SA prediction function; $x_i$ is the actual utility of $d_i$; and $f_U(·)$ is the actual utility function.

For the autonomy decision $d_{AD}\in D$, $D_H=\{d_H^1,\dots ,d_H^{K-1}\}=D-\left\{d_{AD}\right\}$ is the takeover behavior space. Prospect Theory uses two sigmoidal curves to represent the subjective utility functions of takeover and the autonomy decision, as shown in Figure 5 [9].

Then, the utility of autonomy decision $u_{AD}(x_{AD}^r)$ and takeover utility ($u_H(x_H^i)$) can be expressed as

$$u_{AD}(x_{AD})=\left\{\begin{array}{cc}{(x_{AD})}^{{\alpha }_{PR}}& x_{AD}\ge 0\\ -{\lambda }_{AD}{(-x_{AD})}^{{\alpha }_{PR}}& x_{AD}<0\end{array}\right.$$

(11)

$$u_H(x_H^i)=\left\{\begin{array}{cc}{(x_H^i)}^{{\alpha }_{PR}}& x_H^i\ge 0\\ -{\lambda }_H{(-x_H^i)}^{{\alpha }_{PR}}& x_H^i<0\end{array}\right.$$

(12)

where $x_{AD}$ and $x_H^i$ are the actual utility of the autonomy decision and takeover behavior $d_H^i$, respectively; ${\alpha }_{PR}\in (0,1)$ is the risk preference index; and ${\lambda }_{AD}$ and ${\lambda }_H$ are the loss averse indexes of the autonomy decision and takeover, respectively. A human is more averse to loss caused by takeover, which means $1<{\lambda }_{AD}<{\lambda }_H$. According to the gap between ${\lambda }_{AD}$ and ${\lambda }_H$, the omission bias can be divided into three levels: the high level (HOB $<{\lambda }_{AD},{\lambda }_H>=<2.25,18.9>$), middle level (MOB $<{\lambda }_{AD},{\lambda }_H>=<2.25,9.45>$), and low level (LOB $<{\lambda }_{AD},{\lambda }_H>=<2.25,4.72>$) [9,59].

Then, predicted utility of an autonomy decision ($U_{AD}$) and takeover behavior ($U_H^j$) can be calculated with

$$\left\{\begin{array}{l}{U}_{AD}={\displaystyle {\sum }_j{u}_{AD}(x_{AD}^j)w_{AD}^j}\\ U_H^i={\displaystyle {\sum }_j{u}_H(x_H^{ij})w_H^{ij}}\end{array}\right.$$

(13)

where $x_{AD}^j$ and $x_H^{ij}$ are the $j$th utility of the autonomy decision and takeover behavior $d_H^i$, respectively, whose weights are $w_{AD}^j$ and $w_H^{ij}$, respectively. The utility prediction space of takeover can be expressed as $U_H=\{U_H^1,\dots ,U_H^{K-1}\}$.

The belief that the predicted utility of an autonomy decision is optimal, i.e., $p_{(P{R}_r=ad|E_r)}$, can be expressed as

$$p_{(P{R}_r=ad|E_r)}=\left\{\begin{array}{ll}\begin{array}{l}1,\\ 0,\end{array}& \begin{array}{l}If U_{AD}\ge \max (U_H)\\ Else\end{array}\end{array}\right.$$

(14)

when humans choose to takeover, the option in $D_H$ with the highest utility will be selected to be executed, that is, $hd=\arg \underset{d_i\in D_H}{\max }(U_H^i)$.

4. Dynamic Bayesian Network for HAIP Safety Assessment

A novel DBN-based safety assessment method for multi-round HAIP considering the effects of anchoring bias and omission bias is developed in this section. The structure establishment and parameter determination of DBN are guided by SAB-TDM. The flowchart of the proposed method is shown in Figure 6 with four major steps explained as follows.

4.1. Analysis of Multi-Round HAIP (Step 1)

This preparatory step is to obtain knowledge about multi-round HAIP in risk scenarios. It includes (i) the inputs and various algorithms that autonomy uses to make a risk-avoidance decision; (ii) the SAT information generated with an autonomy decision to support a human’s takeover decision; and (iii) the rules for HAI to avoid risk. This knowledge is important input to HAIP modeling and safety assessment.

4.2. Qualitative Modeling (Step 2)

Considering the abilities to encode human knowledge and describe time dependency, DBN is selected to model the HAIP affected by anchoring and omission bias. Figure 7 shows a schematic diagram to clarify the modeling concept. As shown in the example, HAIP based on SA is considered as the important elements in DBN. The interpretation and establishing approach of the network are introduced in the following three sub-steps.

4.2.1. Identify the Critical Nodes in DBN

To model HAIP, this sub-step identifies DBN nodes to represent an autonomy decision and human takeover decision in multi-round interaction. As shown in Figure 7, nodes $E$ and Sensor FI are used to describe critical inputs and the bias caused by functional insufficiency of sensors. Moreover, nodes $T_r^1$, $T_r^2$, and $T_r^3$ are identified to represent SAT information at three levels. They also indicate the generation of the autonomy decision.

To model the takeover decision based on SA, nodes $P{E}_r^1$, $P{E}_r^2$, and $P{E}_r^3$ are identified to represent SAT information perception. Moreover, benchmark information (nodes $B_r^1$, $B_r^2$, and $B_r^3$), SAT correctness (nodes $C{O}_r^1$, $C{O}_r^2$, and $C{O}_r^3$), and autonomy decision correctness (node $C{O}_r$) are used to represent SA comprehension. Nodes $P{R}_r$, omission bias, and $D$ are identified to represent utility prediction affected by omission bias. Node $TO{D}_r$ is identified to represent a human’s takeover decision. It also works as an updated anchor affecting SA perception and comprehension in the next round. These nodes will be extended and assigned specific meaning when applied to an actual risk scenario.

4.2.2. Establish Causalities Within a Single Round

The causalities in the single time slice of DBN describe the HAI in a single interaction round. The causality arcs describing an autonomy decision start from $E$ and the Sensor FI node, and end with $T_r^1$, $T_r^2$, and $T_r^3$ nodes. In humans’ takeover decision process, the causality arcs start from the nodes related to SAT information perception and correctness judgment, followed by nodes related to autonomy decision correctness and takeover utility prediction, and end with a node representing the takeover decision. Additionally, causalities between the Trust node and SA perception and comprehension nodes (i.e., $P{E}_1^1$, $P{E}_1^2$, $P{E}_1^3$, and $C{O}_1$) reflect effects of trust (as the initial anchor) and anchoring bias in the initial round.

4.2.3. Establish Causalities Between Adjacent Rounds

This sub-step aims to establish connections between adjacent DBN slices, which reflects the perception and comprehension round dependency caused by anchoring bias. The connections between $TO{D}_{r-1}$ and SAT information perception nodes (i.e., $P{E}_r^1$, $P{E}_r^2$, and $P{E}_r^3$) are used to represent perception round dependency (blue lines in Figure 7). The connections between $TO{D}_{r-1}$ and the autonomy decision correctness node (i.e., $C{O}_r$) are used to represent comprehension round dependencies (red lines in Figure 7). The above nodes and connections are combined with that in single time slices to describe the dynamic multi-round HAIP.

4.3. Quantitative Modeling (Step 3)

Based on the construction of DBN structure, another critical step in the proposed method is to determine the parameters in DBN. The key is to determine the conditional probability distribution (CPD) of nodes.

4.3.1. Allocate Priors for Root Nodes

As shown in Figure 7, the root nodes in DBN include the Environment, risk-avoidance decision space $D$, trust level, and Sensor FI. Environment nodes serve as the input variables for both an autonomy decision and human takeover decision, varying according to the current risk scenarios. Risk-avoidance decision space $D$ depends on the capability of the human–autonomy system to handle risk. The trust level can be determined by the actual situation of the analyzed object or different analysis purposes. Sensor FI nodes represent the biases of inputs detected by sensors, which typically follow a normal distribution and can be determined by relevant industry standards [60].

4.3.2. Determine CPD for Nodes Without Round Dependencies

The goal of this sub-step is to determine CPDs for nodes without round dependencies. For nodes related to the correctness judgment of SAT information, the key of CPD determination is to construct IFS representing correctness of SAT information, as seen in Equations (5) and (6). The IFS can be determined based on relevant industry standards. Alternatively, effective methods include conducting questionnaire surveys and using expert judgment [61]. After that, CPD of nodes $C{O}_r^1$, $C{O}_r^2$, and $C{O}_r^3$ can refer to reasoning rules (i)–(v) in the SA comprehension quantification of SAB-TDM. The CPD of nodes $P{R}_r$ representing takeover utility prediction affected by omission bias can refer to Equations (11)–(14) in the utility prediction quantification of SAB-TDM. The CPD of nodes $TO{D}_r$ refer to Equation (1).

4.3.3. Determine CPD for Nodes with Round Dependencies

This sub-step aims to determine the CPD for nodes with round dependencies including nodes $P{E}_r^1$, $P{E}_r^2$, $P{E}_r^3$, and $C{O}_r$. Nodes $P{E}_r^1$, $P{E}_r^2$, and $P{E}_r^3$ represent the SAT information perception, which has round dependencies caused by anchoring bias. The determination of their CPD can refer to Equations (2)–(4) in the SA perception quantification of SAB-TDM. To determine that CPD of nodes has comprehension round dependencies (i.e., nodes $C{O}_r$), the initial causal strength that perceived SAT information supports the anchor should be determined according to reasoning rules (vi)–(viii) in the SA comprehension quantification of SAB-TDM. After that, the CPD determination of nodes $C{O}_r$ can refer to Equations (7)–(9).

4.4. HAIP Safety Assessment (Step 4)

According to the determination of the CPDs of all nodes, the joint probability distribution of DBN modeling multi-round HAIP can be calculated as follows:

$$P(Z_{1:Rn})={\displaystyle \prod _{r=1}^{Rn}{\displaystyle \prod _{i=1}^{n}P(Z_r^i|Pa(Z_r^i))}}$$

(15)

where $P(Z_{1:Rn})$ represents the joint distribution; $Z_r^i$ is the node $i$ in interaction round $r$; $Pa(Z_r^i)$ is the parent node set of $Z_r^i$, which may contain nodes in round $r$ or $r-1$; $n$ is the node number in a single round slice; and $Rn$ is the total number of interaction rounds. Subsequently, the HAIP safety can be expressed as

$$S_{HAIP}=P(Safety=safe)$$

(16)

5. Case Study

The proposed multi-round HAIP safety assessment method is demonstrated with multi-round collision avoidance for autonomous ships. In an encounter situation, an autonomous navigation system (ANS) and operator often need to interact for multiple rounds to complete collision avoidance. During multi-round HAIP, an operator’s SA is significant to make a correct takeover decision and ensure safety. However, anchoring and omission bias will lead to errors in the takeover decision, which are the significant causes of an accident. Therefore, this risk scenario is selected as the research object to illustrate the effectiveness of the proposed method.

5.1. Case Description

5.1.1. Encounter Situation Design

We construct a typical two-ship encounter situation, where the initial motion parameters of our own ship (OS), i.e., an autonomous ship, and target ship (TS) are shown in

Table 1. The settings of a two-ship encounter situation.

Ship	$\mathbf{Position}$(n mile)	$\mathbf{Course}$(°)	$\mathbf{Velocity}$(kn)	$\mathbf{Ship} \mathbf{Length}$ (m)	$\mathbf{Ship} \mathbf{Domain}$(m)
OS	(0, 0)	0	11.1	171	1000
TS	(4.02, 6.55)	230.8	15.9	154	500

. There is collision risk between OS and TS, which is a crossing situation. The ship domain is a concept encompassing safe distance, whose violation signifies an imminent collision. Existing studies suggest employing a circular shape for ship domains in open seas [62].

5.1.2. Introduction of HAIP in Autonomous Ship Collision Avoidance

As shown in Figure 8, the process of HAI to complete collision avoidance can be divided into three stages according to the relative distance of two ships [62,63]. In each stage, ANS decides whether to change course, while an operator achieves SA based on SAT information to make a takeover decision. Therefore, three stages of collision avoidance correspond to three HAIP rounds.

Among many algorithms, the Velocity Obstacle (VO) Algorithm has been widely used for its strong interpretability and graphical expression ability [64]. In the risk scenario of this case study, ANS is assumed to apply VO. The SAT information of ANS is shown in Figure 9. The inputs of VO (i.e., SAT1) are detected by radar including velocities, courses, and the relative distance and bearing of the two ships. According to these inputs, VO constructs a velocity obstacle cone (VOC) of OS, which represents the velocity space of the two ships’ collision. Subsequently, VOC can be divided into more subspaces to assess the current severity degree (SD) and urgency degree (UD) (i.e., SAT2) [65,66]. The predicted utility of ANS’s decision (i.e., SAT3) is indicated by the distance to the closest point of approach (DCPA) and time to the closest point of approach (TCPA), if ANS’s decision is executed (formulated as $d_{cpa}^{ad}$ and $t_{cpa}^{ad}$) [67]. Limited by space, details of VO can be found in [65,66].

In each round, to achieve SA about correctness of ANS’s decision, an operator firstly perceives the SAT information of ANS. Then, the correctness of SAT information can be judged with benchmark information from other sources, like an optical camera onboard the ship. The correctness of SAT information reflects that of ANS’s decision. If the operator cannot clearly identify correctness of ANS’s decision, the benchmark information can also support the operator to predict the utility of different collision avoidance decisions. When finding the ANS’s decision has error or is not optimal, the operator will take over. The takeover decision process outlined above is distorted by anchoring bias and omission bias. In the initial round, the operator is assumed to anchor in trusting ANS. Cognitive biases may make the operator ignore SAT information, miscomprehend the correctness of ANS’s decision, and overestimate the loss caused by takeover. Therefore, the operator will tend not to take over.

5.2. Application of Proposed Method

5.2.1. Analysis of Multi-Round HAIP

To model multi-round HAIP, the knowledge about HAIP in autonomous ship collision avoidance should be analyzed first, including critical inputs and the algorithm of ANS, the SAT information, the division of collision avoidance rounds, and the takeover decision process of operators. This knowledge has been introduced in Section 5.1.2.

5.2.2. Qualitative Modeling

(1)

Identify the critical nodes in DBN

The HAIP to complete collision avoidance is identified as nodes in DBN. ANS’s collision avoidance decision and SAT information are represented by sets of ANS-related nodes. SA, behavioral utility prediction, and the takeover decision are represented by sets of operator-related nodes.

Table 2. Description of nodes in DBN.

Nodes	Description	Nodes	Description
RB	Relative bearing	RD	Relative distance
VOS	Velocity of OS	COS	Course of OS
VTS	Velocity of TS	CTS	Course of TS
Sensor FI 1~6	Input data bias of sensors	AD	ANS decision
RB/RD/VOS/COS/VTS/COS + A	Input data detected by ANS	UD/SD + A	UD/SD from ANS
RB/RD/VOS/COS/VTS/COS + B	Benchmark of SAT1 information	UD/SD + B	Benchmark of SAT2 information
RB/RD/VOS/COS/VTS/COS + C	Correctness of input data	UD/SD + C	Correctness of UD/SD
$d_{cpa}^{ad}$$/t_{cpa}^{ad}$ + A	$d_{cpa}^{ad}$$/t_{cpa}^{ad}$ predicted by ANS	$P{E}_r^i,(r,i\in \{1,2,3\})$	Perception of each SAT level
$d_{cpa}^{ad}$$/t_{cpa}^{ad}$ + B	Benchmark of SAT3 information	$C{O}_r^i,(r,i\in \{1,2,3\})$	Correctness of each SAT level
$d_{cpa}^{ad}$$/t_{cpa}^{ad}$ + C	$\mathrm{Correctness} \mathrm{of} d_{cpa}^{ad}$$/t_{cpa}^{ad}$	$C{O}_r(r\in \{1,2,3\})$	ANS decision correctness
$P{R}_r(r\in \{1,2,3\})$	Utility prediction	$D$	Decision space
Omission bias	Existence of omission bias	Trust	Trust on ANS (initial anchor)
$TO{D}_r(r\in \{1,2,3\})$	Takeover decision	Safety	Result of collision avoidance

lists nodes mentioned above along with their descriptions.

(2)

Establish causalities within a single round

Within a single round, the causality arcs are established to represent the generation process of ANS’s collision avoidance decision and SAT information, and the operator’s SA achievement and takeover decision.

(3)

Establish causalities between adjacent rounds

Perception and comprehension round dependencies are described by the causality arcs from the operator’s takeover decision node to perception nodes of ANS’s SAT information, and the causality arcs between the operator’s takeover decision node and the node related to correctness judgment of the ANS decision, respectively.

Combining the connections within and between rounds, the construction of the qualitative DBN model can be completed, as shown in Figure 10. Sub-models are used to simplify the network structure. The connection related to sub-models means that the nodes in parent sub-models are all the parent nodes of nodes in child sub-models.

5.2.3. Quantitative Modeling

(1)

Allocated priors for root nodes

The priors for nodes describing the risk scenario can be determined by the designed encounter situation shown in Table 1. Secondly, the collision avoidance decision space is set as $D=\{0°,\pm 5°,\pm 10°,\pm 20°,\pm 30°\}$. Thirdly, the trust level is initialized as 0.8. At last, the bias space for input information is detailed in

Table 3. Bias distribution of input information [68].

Parameters	Distribution	Parameters	Distribution
Sensor FI 1 ($°$)	$\mathcal{N}(0,{1.5}^2)$	Sensor FI 4 ($°$)	$\mathcal{N}(0,{2.5}^2)$
Sensor FI 2 ($\mathrm{n} \mathrm{m}\mathrm{i}\mathrm{l}\mathrm{e}$)	$\mathcal{N}(0,{0.15}^2)$	Sensor FI 5 ($\mathrm{k}\mathrm{n}$)	$\mathcal{N}(0,{0.25}^2)$
Sensor FI 3 ($\mathrm{k}\mathrm{n}$)	$\mathcal{N}(0,{0.25}^2)$	Sensor FI 6 ($°$)	$\mathcal{N}(0,{2.5}^2)$

[68].

(2)

Determine CPD for nodes without round dependencies

To determine the CPDs of nodes related to SAT information correctness, we construct the membership and non-membership function describing IFS about the correctness of each piece of SAT information, as shown in Figure 11. Based on the above membership and ono-membership function, CPDs of $C{O}_r^1$, $C{O}_r^2$, and $C{O}_r^3$ can refer to reasoning rules (i)–(v) in the SA comprehension quantification of SAB-TDM.

The CPD determination of the $P{R}_r$ node needs to investigate the way that the operator predicts behavior utility. The behavior utility prediction is based on SA prediction about the DCPA if $d_i$ was executed. It means that the SA prediction function is the calculation of DCPA. Based on SA prediction, utility indexes including risk and efficiency are used to model the actual utility prediction of a collision avoidance decision [41,62]. For collision avoidance decision $d_i\in D$, its risk index $x_i^R$ and efficiency index $x_i^E$ can be represented as

$$x_i^R=\left\{\begin{array}{cc}\begin{array}{l}{d}_{cpa}^i/1500-1,\\ 0,\end{array}& \begin{array}{l}{d}_{cpa}^i<1500\\ d_{cpa}^i\ge 1500\end{array}\end{array}\right.$$

(17)

$$x_i^E=-\left|d_i\right|/\max \left|D\right|$$

(18)

where $d_{cpa}^i$ is the DCPA if $d_i$ is executed in the current situation. The weight of $x_i^R$ and $x_i^E$ are set to $w_R=0.8$ and $w_E=0.2$, respectively. Then, Equations (11)–(14), ${\alpha }_{PR}=0.88$, and parameter tuple $<{\lambda }_{AD},{\lambda }_H>$ are used to describe subjective utility prediction affected by omission bias.

(3)

Determine CPD for nodes with round dependencies

The perception round dependency is considered in the CPDs of those SAT perception nodes (i.e., $P{E}_r^1$, $P{E}_r^2$, $P{E}_r^3$). Constants ($\gamma =1/3$, $\lambda =2/3$, $\tau =1$, $s=0.4$ [69]) and parameters in

Table 4. The saliency and value of each SAT level.

SAT Information	Saliency (${\mathit{A}}_{\mathit{r}\mathit{i}}^{\mathit{S}}$)	Value (${\mathit{A}}_{\mathit{r}\mathit{i}}^{\mathit{V}}$)
		${\mathit{A}}_{\mathit{r}-1}=\mathit{b}$	${\mathit{A}}_{\mathit{r}-1}=\mathit{u}$
SAT1	3	0.5	4
SAT2	4	0.5	4
SAT3	4	0.5	4

are used in Equations (2)–(4) to calculate the CPDs of $P{E}_r^1$, $P{E}_r^2$, and $P{E}_r^3$. The comprehension round dependency is considered in CPD of the $C{O}_r$ node. The rules (vi)–(viii) in the SA comprehension quantification of SAB-TDM and parameter tuple $<\alpha ,\beta >$ in Equations (7)–(9) are used to calculate the CPD of the $C{O}_r$ node.

5.2.4. HAIP Safety Assessment

After the determination of priors and CPDs of all nodes, the multi-round HAIP safety can be calculated with Equations (15) and (16). In the case study, HAIP safety is the probability that the collision risk is avoided successfully after three-round HAIP. To simplify the calculation, the joint distribution of DBN is computationally implemented using MATLAB R2023b.

5.3. Result and Discussion

5.3.1. Result Analysis

In the case study, the operator is assumed to be under the middle anchoring bias level (MAB) and omission bias level (MOB). The safety of collision avoidance for autonomous ships is illustrated in Figure 12. To illustrate the effects of anchoring bias and omission bias, the safety without HAI and without considering the effects of these biases is plotted in Figure 12, too. It can be observed that collision risk is effectively reduced with the existence of the operator. It means that the operator who takes over when ANS makes a wrong collision avoidance decision can compensate for the defects of ANS. Nevertheless, the significant gap between safety with and without biases suggests that anchoring bias and omission bias hinder the advantage of HAI in reducing the collision risk of autonomous ships.

To explain the above phenomenon, the takeover probabilities with and without considering anchoring bias and omission bias are plotted in Figure 13. Based on the comparison, the above phenomenon can be explained as follows:

Due to perception round dependency caused by anchoring bias, the operator is prone to ignoring the SAT information used to judge the correctness of ANS’s decision. Even if those pieces of evidence are perceived, their causal strength to indicate the errors in ANS’s decision will be underestimated due to the comprehension round dependency caused by anchoring bias. These round dependencies impede the operator to find the errors of ANS’s decision. Although the operator can compare the predicted utility of ANS’s decision and takeover, omission bias leads to the overestimation of loss caused by the latter, which will make the operator believe that ANS’s wrong decision is optimal. The above effects of anchoring and omission biases impede the operator’s takeover in the current round. They also strengthen anchor belief, which affects the SA in the next round. Hence, in each round, the operator’s takeover probabilities are lower than that without considering anchoring and omission biases. It makes the defects of ANS unable to be compensated for effectively, resulting in accidents.

5.3.2. Sensitivity Analysis of Anchoring Bias Level

In this section, a sensitivity analysis is performed to investigate the effects of anchoring bias levels on HAIP safety. During risk scenarios, the anchoring bias level is the critical parameter that adjusts the strength of perception and comprehension round dependencies. We assess and compare HAIP safety and average takeover probability (i.e., ${\overline{p}}_{(TO{D}_r=t)}={\displaystyle {\sum }_{r=1}^3{p}_{(TO{D}_r=t)}}/3$) under other anchoring bias levels, including HAB and LAB. Results are shown in Figure 14. Obviously, HAIP safety and takeover probability reduce significantly with the increase in the anchoring bias level, which also reflects the promotion of anchoring bias on omission bias. These phenomena can be explained as follows:

With the increase in the anchoring bias level, the strength of perception and comprehension round dependencies is strengthened. It makes the operator tend to ignore SAT information. Even if some SAT information is perceived, their causal strength in indicating errors of ANS’s decision will be further underestimated. It becomes harder for the operator to identify the correctness of the autonomy decision and they tend not to take over, when ANS makes a wrong collision avoidance decision. Therefore, the increase in the anchoring bias level can significantly reduce takeover probability and HAIP safety.

To further explain the effects of anchoring bias levels on perception, we compare the average perception probability of three-level SAT information (i.e., ${\overline{p}}_{(P{E}_r^i)}={\displaystyle {\sum }_{i=1}^3{p}_{(P{E}_r^i)}}/3$) under different anchoring bias levels. The comparison results are illustrated in Figure 15. It can be observed that under the different anchoring levels, SAT information in the initial round has the same perception probability. This is because the perception probability of SAT information in the initial round is only related to the trust level (i.e., the initial anchor belief). In other rounds, the perception probability reduces with the increase in the anchoring bias level, which reflects the strengthening of perception round dependency. This phenomenon can be explained as follows:

The perception probability of SAT information depends on the subjective value assigned by the operator. Due to its natural quality to deny the initial anchor (i.e., trust ANS), SAT information will be assigned with a smaller value with the increase in anchor belief. Affected by comprehension round dependency, the strength of the same evidence to strengthen anchor belief increases with the anchoring bias level. The higher anchor belief makes the operator possibly assign a smaller value to SAT information in the next round, which means the strengthening of perception round dependency. Hence, perception probability of SAT information reduces with the increase in the anchoring bias level.

To further explain effects of anchoring bias levels on comprehension, we compare the operator’s comprehension about the correctness of ANS’s decision under different anchoring bias levels. It is formulated as the probability that ANS’s decision process is believed to be without error (i.e., $p_{(C{O}_r=y)}$). The comparison results are illustrated in Figure 16. It can be found that $p_{(C{O}_r=y)}$ increases with the anchoring bias level. It means that it is harder for the operator to find the errors of the ANS decision with the increase in the anchoring bias level, which reflects the strengthening of comprehension round dependency. The phenomenon can be explained as follows:

Comprehension about correctness of ANS’s decision is achieved based on SAT information. Affected by perception round dependency, less SAT information can be perceived with the increase in the anchoring bias level. The causal strength of perceived SAT information to indicate the errors in ANS’s decision will be underestimated due to anchoring bias. The underestimation degree increases with the anchoring bias level and anchor belief. It means that it is harder for the operator to find errors of ANS’s decision with the increase in the anchoring bias level. That will increase the belief of the anchor (i.e., ANS’s decision without error) and further strengthen the comprehension round dependency. Therefore, the probability of the operator believing ANS’s decision to be without errors increases with the anchoring bias level.

5.3.3. Sensitivity Analysis of Omission Bias Level

In this section, a sensitivity analysis is performed to investigate effects of different omission bias levels on multi-round HAIP safety. In multi-round HAIP during risk scenarios, the omission bias level is a critical parameter that adjusts the overestimation degree of loss caused by takeover. We assess and compare multi-round HAIP safety and takeover probability under other omission bias levels including high and low omission bias levels (i.e., HOB and LOB). The comparison results are shown in Figure 17. Obviously, multi-round HAIP safety and takeover probability decrease significantly with the increase in the omission bias level. These phenomena can be explained as follows:

With the increase in the omission bias level, the operator can more irrationally predict and compare the utility of takeover and ANS’s decision. It makes the operator less likely to identify the condition where ANS’s decision is not optimal and tend not to take over. The not taking over in the current round can also increase the belief of the anchor (i.e., ANS’s decision without error), thereby reinforcing the effects of anchoring bias, thus contributing to the incorrect takeover decision in the next round. Therefore, takeover probability and HAIP safety significantly reduce with the increase in the omission bias level.

To further explain the effects of anchoring bias levels on behavioral utility prediction, we compare the behavioral utility prediction results under different omission bias levels. It is formulated as the probability that the ANS decision is considered optimal (i.e., $p_{(P{R}_r=ad)}$). The comparison results are illustrated in Figure 18. It can be found that in each interaction round, $p_{(P{R}_r=ad)}$ increases with the omission bias level. This phenomenon can be explained as follows:

Omission bias makes the operator more averse to the loss caused by takeover. It increases the widening gap between aversion to losses from takeover and aversion to losses from ANS’s decision. The overestimation degree of loss caused by takeover will be strengthened. The operator tends to use a more pessimistic attitude to predict the utility of takeover. It makes ANS’s wrong decision more likely to be considered optimal. Hence, the probability that the ANS decision is considered optimal increases with the omission bias level.

Moreover, to investigate the promotion of omission bias on anchoring bias, we analyze and compare the SAT information perception and comprehension about correctness of ANS’s decision under different omission bias levels. The former is formulated as the average perception probability of three-level SAT information in each round (i.e., ${\overline{p}}_{(P{E}_r^i)}={\displaystyle {\sum }_{i=1}^3{p}_{(P{E}_r^i)}}/3$), and the latter is formulated as the probability that ANS’s decision is believed to be without error (i.e., $p_{(C{O}_r=y)}$). The comparison results are plotted in Figure 19. It can be found that with an increase in the omission bias level, perception probability of SAT information reduces and $p_{(C{O}_r=y)}$ increases. It means that the increase in the omission bias level can strengthen the strength of perception and comprehension round dependencies caused by anchoring bias. The main reason for this phenomenon is as follows:

In the initial round, SAT perception and comprehension are only related to the initial anchor belief (i.e., trust level) and anchoring bias level; thus, ${\overline{p}}_{(P{E}_r^i)}$ and $p_{(C{O}_r=y)}$ are both the same under different omission bias levels. Nevertheless, with the increase in the omission bias level, the operator tends to believe that ANS’s decision is optimal, which will increase the anchor belief. The higher anchor belief will make the operator more likely to assign a smaller value to SAT information and more likely to underestimate the causal strength of evidence indicating errors of ANS’s decision. Therefore, the increase in the omission bias level reduces perception probability of SAT information and probability that the operator finds errors of the autonomy decision, which both reflect the strengthening of anchoring bias.

5.3.4. Analyze the Encounter Situation of Two Autonomous Ships

In this section, such an encounter situation is discussed, in which two ships are both autonomous ships. We assume that settings for OS and TS in Figure 8 are identical. The HAIP safety and some of its details in such an encounter situation are illustrated in Figure 20. As shown in Figure 20a, when the collision avoidance behaviors of two autonomous ships are not limited by clear rules, the HAIP safety is slightly reduced. This is because in some conditions, the two autonomous ships choose to give way to each other but change courses to the same side. This kind of risky movement may result in collision risk not being effectively mitigated, and even increased. Figure 20b plots the probability of such risky movement in each round.

To avoid such risky movement and ensure the success of collision avoidance, clear collision avoidance rules should be provided to determine priority clearly. In fact, the International Regulations for Preventing Collisions at Sea (COLREGs) have given some rules to identify the give-way ship and stand-on ship, while these rules may not be strictly followed by humans. In this case study, according to Rule 15 in COLREGs, OS is the give-way ship and TS is the stand-on ship. Nevertheless, according to Rule 17, both of the two ships should undertake collision avoidance behavior when collision risk cannot be avoided by a single ship (i.e., round 3). We assume that the two autonomous ships both strictly follow COLREGs and assess the HAIP safety. The result in Figure 20a shows that HAIP safety effectively increases. It indicates that providing clear collision avoidance rules, which are uniform for the whole world, for autonomous ships is an effective means to improve safety of ocean shipping.

5.3.5. Validation of Proposed Method

To validate the proposed method, we analyze such HAIP in which ANS uses an alternative collision avoidance algorithm described in [70]. This SA-based algorithm complies with COLREG rules. It extended VO with the ship domain (SD) and integrated the fuzzy inference system based on near-collision (FIS-NC) [71]. This integrated algorithm is referred to as SDVO + FIS-NC. We assess safety and analyze the details in such HAIP, as shown in Figure 21.

As shown in Figure 21, the phenomena in HAIP using SDVO + FIS-NC are similar with that in HAIP using VO. Specifically, (i) safety in collision avoidance increases with the existence of HAI; (ii) HAIP safety and takeover probability reduce with the existence of anchoring and omission biases; (iii) HAIP safety and takeover probability decrease with the improvement of the anchoring bias level and omission bias level. HAIP safety assessments of HAIP using VO and SDVO + FIS-NC both indicate that anchoring bias and omission bias impede the advantage of HAIP to compensate for the defects of autonomy. The consistency between the conclusions of HAIP using different algorithms validates the effectiveness of the proposed method.

6. Conclusions

To better understand the performance of human–autonomy systems under risk scenarios, effective HAIP safety assessment should dig deep into the takeover decision process based on SA and consider the effects of anchoring bias and omission bias. To meet these requirements, we first developed an SAB-TDM to analytically represent takeover decisions based on SA. The SAB-TDM focuses on quantifying how round dependencies caused by anchoring bias impede SAT information perception and distort comprehension about the correctness of autonomy decisions, and how irrational utility prediction preference caused by omission bias leads to the overestimation of loss caused by takeover. Subsequently, a novel method based on DBN for HAIP safety assessment is proposed. In the proposed method, the multi-round HAIP affected by anchoring bias and omission bias is modeled. Finally, the HAIP in a typical risk scenario, that is, multi-round collision avoidance for autonomous ships, is analyzed to exemplify the proposed method.

The proposed method was successfully applied for the first time to assess the HAIP safety under a risk scenario. The assessment result provides some details about the SA and takeover decision in multi-round HAIP. As seen from the case study, anchoring bias and omission bias mutually contribute to cause SA errors and impede correct takeover, which seriously threaten HAIP safety. With the increase in the anchoring bias level, perception probability of SAT information decreases, and humans are less likely to find the errors of an autonomy decision. The increase in the omission bias level makes humans less likely to identify the condition that the autonomy decision is not optimal, which also strengthens the strength of perception and comprehension round dependencies caused by anchoring bias. The increase in the anchoring bias and omission bias level reduces HAIP safety by 0.01316 and 0.01587, respectively. Therefore, to fully paly the advantage of HAI in risk avoidance and ensuring safety, it is necessary to take some effective measures to decrease the effects of anchoring bias and omission bias, such as enhancement of de-biasing training and improvement of autonomy transparency.

This paper proposes an HAIP safety assessment method considering effects of anchoring bias and omission bias. The purpose is to provide technical support for system improvement aimed at promoting correct takeovers. For future work, to further improve HAIP safety, research efforts should also be focused on enhancing the capabilities of autonomous systems. Prospective methods include improving algorithms and increasing the precision and reliability of sensors.