Geometric Analysis of Conditional Bias-Informed Kalman Filters

Abstract

This paper presents a comparative geometric analysis of the conditional bias (CB)-informed Kalman filter (KF) with the Kalman filter (KF) in the Euclidean space. The CB-informed KFs considered include the CB-penalized KF (CBPKF) and its ensemble extension, the CB-penalized Ensemble KF (CBEnKF). The geometric illustration for the CBPKF is given for the bi-state model, composed of an observable state and an unobservable state. The CBPKF co-minimizes the error variance and the variance of the Type-II error. As such, CBPKF-updated state error vectors are larger than the KF-updated, the latter of which is based on minimizing the error variance only. Different error vectors in the Euclidean space imply different eigenvectors and covariance ellipses in the state space. To characterize the differences in geometric attributes between the two filters, numerical experiments were carried out using the Lorenz 63 model. The results show that the CBEnKF yields more accurate confidence regions for encompassing the truth, smaller errors in the ensemble mean, and larger norms for Kalman gain and error covariance matrices than the EnKF, particularly when assimilating highly uncertain observations.

1. Introduction

With highly uncertain observations and model dynamics, KF [1] and EnKF [2] estimates tend to be conditionally biased. The conditional bias (CB)-informed Kalman filter (KF) is designed to improve the estimation of extreme states in geophysical data assimilation, particularly at the onset of extreme phenomena. The CB-informed KFs developed to date include the CB-penalized Kalman filter (CBPKF; [3,4]) and its ensemble extension, the CB-penalized Ensemble Kalman filter (CBEnKF; [5]), and aim at addressing CB [3,4,5] to improve estimation of extreme states. With extreme precipitation, droughts, and floods occurring more frequently in many parts of the globe, the estimation and prediction of extremes is an increasingly important topic in hydrology. With the increasing availability of diverse sources of observations, hydrologic data assimilation has a large role to play in improving state estimation. The CB-informed KF is an effort to help address both.

There are two types of CB, Type I and Type II. The Type-I CB is defined as E[X|X* = x*] − x* where X, X*, and x* represent the unknown truth, the estimate, and the realization of X*, respectively. The Type-II CB is defined as E[X*|X = x] − x where x represents the realization of X. The Type-I CB is associated with false alarms which can be reduced by calibration. The Type-II CB is associated with failure to detect an event and cannot be reduced by calibration. The CBPKF minimizes a linearly weighted sum of the error covariance and the expectation of the Type-II error squared. With skillful specification of the weight for the latter, the CBPKF improves estimation of extremes over the KF while slightly increasing the unconditional MSE [3,4].

Though extensively studied in statistics, econometrics, meteorology, and hydrology [6,7,8,9,10,11,12,13,14,15,16], CB has gained interest only recently in data assimilation. Relevant studies to date include the CBPKF [3,4] and the CBEnKF [5]—the CBEnKF application to flood forecasting [5], adaptive filtering for the CBPKF [17] and the CBEnKF [18], and the variance-inflated KF-based approximation of the CBPKF for reduced computation [17,18]. Improving understanding of how the CB-informed KF compares with conventional data assimilation techniques, such as the KF, is crucial for interpreting and analyzing its application results, as well as identifying areas for further development. To that end, this study provides a geometric analysis of the CB-informed KF and compares performance with the EnKF via numerical experiments.

Stochastic variables may be expressed as vectors in the Euclidean domain, which can be used to geometrically illustrate vectors of innovation, observation errors, and forecast and analysis errors in the Euclidean domain [19]. The geometric interpretation of state estimation thus helps understand how Kalman gain shape to analysis error vectors and assimilation solutions for observable as well as unobservable states [19]. Error covariance matrices may also be analyzed geometrically in the state space and characterized with eigenvalues, eigenvectors, and associated confidence regions.

The purpose of this work is to gain additional insight into intuitive understanding of how the CBPKF solution differs from the KF, by casting them in error and state spaces using a bi-state model, and to advance understanding of the comparative performance of the CBEnKF with the EnKF by identifying and characterizing the representative geometric attributes in the state space via eigenvalue analysis. The new and significant contributions of this work are: comparative geometric analysis of the CBPKF solution with the KF, development of a set of geometry-based relationships for improved understanding of the CBPKF solution, and geometric characterization of ensemble analysis for improved understanding of the comparative performance of the CBEnKF with the EnKF. The comparative performance is based on numerical experiments with the Lorenz 63 model, chosen in this study for familiarity and simplicity. For real-world flood forecasting applications of the CBEnKF, the interested reader is referred to Lee et al. [5] and Shen et al. [18].

This paper is organized as follows. Section 2 describes the state updating problem, and the two solution approaches, the KF and the CBPKF. Section 3 presents the geometric analysis of the CB-informed KF in relation to the KF and the EnKF. First, Section 3.1 describes how to use vectors to represent stochastic variables in the Euclidean domain, and Section 3.2 expresses the filter equations in terms of state and observation error terms. Section 3.3 geometrically illustrates the filter equations for a low-order model based on the visualization of error vectors and their relations to Kalman gain and covariance matrices in the Euclidean space [19]. Section 3.4 describes geometric analyses in the state-space with eigenvalues and eigenvectors of error covariance matrices and associated confidence regions. In Section 3.5, an example is given for geometric analysis in the state space in which the CBEnKF, the EnKF, and Open Loop (OL) are compared for the Lorenz 63 model via numerical experiments. Finally, Section 4 describes conclusions.

2. Methodology

2.1. State Updating Problem

The nonlinear dynamical model is written as Equation (1).

$$X_k=M\left(X_{k-1}\right)+W_{k-1}$$

(1)

In the above, $X_k$ denotes the ($n_c\times 1$) model state, or control vector, where $n_c$ denotes the number of variables in the control vector, M( ) denotes the dynamical model for state variables, and W_k−1 denotes the dynamical model error at a time step k − 1 with a mean of zero and a covariance of $Q_k$. The nonlinear observational model is written as:

$$Z_k=H_k\left(X_k\right)+V_k$$

(2)

In the above, Z_k denotes the (n $\times$ 1) observation vector, where n denotes the total number of observations. V_k denotes the (n $\times$ 1) observation error vector at a time step k with a mean of zero and a covariance of $R_k$. $H_k$() denotes the nonlinear observation operator that maps $X_k$ to $Z_k$. This study solves the linear observation model which renders $H_k\left(X_k\right)$ in Equation (2) linear, i.e., $H_k{X}_k$:

$$Z_k=H_k{X}_k+V_k$$

(3)

In the case of a nonlinear observation operator, e.g., soil moisture–streamflow transformation, one can still render Equation (2) linear via state augmentation [5,20,21], which is beyond the scope of this study. Interested readers are referred to Lee et al. [5], where state augmentation is used to solve a highly nonlinear flood forecasting problem.

Equation (4) shows the state updating equation where $X_{k|k}^{}$, $X_{k|k-1}^{}$, and $K_k$ represent updated state at a time step k, state forecast from k − 1 to k, and Kalman gain, respectively.

$$X_{k|k}^{}=X_{k|k-1}^{}+K_k\left[Z_k-H_k{X}_{k|k-1}^{}\right]$$

(4)

To find $K_k$, the KF and the EnKF minimize the error variance (${\mathsf{\Sigma }}_{EV}$) in Equation (5). The CBPKF and the CBEnKF minimize the weighted sum of ${\mathsf{\Sigma }}_{EV}$ and the expectation of the Type-II CB squared (${\mathsf{\Sigma }}_{CB}$) in Equation (6), or ${\mathsf{\Sigma }}_{EV}+{\alpha \mathsf{\Sigma }}_{CB}$ where α is the weight given to the CB penalty term. The α can be estimated using an iterative method that yields the error covariance within theoretically expected bounds (see [5] for details). Since the iterative method is computationally expensive, the adaptive filtering method has been developed, and the results are reported in Shen et al. [17].

$${\mathsf{\Sigma }}_{EV}=E\left[\left(X_k-X_{k|k}\right){\left(X_k-X_{k|k}\right)}^T\right]$$

(5)

$${\mathsf{\Sigma }}_{CB}=E[(X_k-E_{X_{k|k}}[X_{k|k}|X_k]){(X_k-E_{X_{k|k}}[X_{k|k}|X_k])}^T]$$

(6)

The following sections present expressions for Kalman gain ($K_k^{}$) and covariance ($P_{k|k}$) matrices in the case of the KF and the CBPKF.

2.2. Kalman Filter, KF

Equation (4) is rewritten as Equation (7) but with the superscript K to denote the KF.

$$X_{k|k}^K=X_{k|k-1}^{}+K_k^K\left[Z_k-H_k{X}_{k|k-1}^{}\right]$$

(7)

Minimizing ${\mathsf{\Sigma }}_{EV}$ in Equation (5) results in Kalman gain $K_k^K$ in Equation (8) and the covariance analysis matrix $P_{k|k}$ in Equation (9) [1].

$$K_k^K=P_{k|k-1}{H}^T{\left[H{P}_{k|k-1}{H}^T+R\right]}^{-1}$$

(8)

$$P_{k|k}=\left[1-K_{k}H\right]P_{k|k-1}=P_{k|k-1}-K_k\left[H{P}_{k|k-1}{H}^T+R_k\right]K_k^T$$

(9)

2.3. Conditional Bias-Penalized Kalman Filter, CBPKF

State updating equation (Equation (4)) can be rewritten into Equation (10) with the superscript C denoting the CBPKF.

$$X_{k|k}^C=X_{k|k-1}^{}+K_k^C\left[Z_k-H_k{X}_{k|k-1}^{}\right]$$

(10)

Minimizing ${\mathsf{\Sigma }}_{EV}+{\alpha \mathsf{\Sigma }}_{CB}$ produces the CBPKF gain $K_k^C$ in Equation (11) and the covariance matrix ${\mathsf{\Sigma }}_{k|k}$ in Equation (12) [3,4,17].

$$K_k^C={\left[{\varpi }_{1,k}{H}_k+{\varpi }_{2,k}\right]}^{-1}{\varpi }_{1,k}$$

(11)

$${\mathsf{\Sigma }}_{k|k}={\left[{\varpi }_{1,k}{H}_k+{\varpi }_{2,k}\right]}^{-1}\left({\varpi }_{1,k}{R}_k{\varpi }_{1,k}^T+{\varpi }_{2,k}{\mathsf{\Sigma }}_{k|k-1}{\varpi }_{2,k}^T\right){\left[{\varpi }_{1,k}{H}_k+{\varpi }_{2,k}\right]}^{-1}$$

(12)

In Equations (11) and (12), the (m $\times$ n) and (m $\times$ m) weight matrices for the observations and model prediction, ${\varpi }_{1,k}$ and ${\varpi }_{2,k}$, respectively, are given by:

$${\varpi }_{1,k}={\widehat{H}}_k^T{\Gamma }_{11,k}+{\Gamma }_{21,k}$$

(13)

$${\varpi }_{2,k}={\widehat{H}}_k^T{\Gamma }_{12,k}+{\Gamma }_{22,k}$$

(14)

In Equations (13) and (14), the (m $\times$ n) modified observation matrix, ${\widehat{H}}_k^T$, and the (n $\times$ n), (n $\times$ m), and (m $\times$ m) matrices, ${\Gamma }_{11,k}$, ${\Gamma }_{12,k} \left(={\Gamma }_{21,k}^T\right)$, and ${\Gamma }_{22,k}$, respectively, are given by:

$${\widehat{H}}_k^T=H_k^T+\alpha C_k^T$$

(15)

$${\Gamma }_{22,k}^{-1}={\mathsf{\Lambda }}_{22,k}-{\mathsf{\Lambda }}_{21,k}{\mathsf{\Lambda }}_{11,k}^{-1}{\mathsf{\Lambda }}_{12,k}$$

(16)

$${\Gamma }_{11,k}={\mathsf{\Lambda }}_{11,k}^{-1}+{\mathsf{\Lambda }}_{11,k}^{-1}{\mathsf{\Lambda }}_{12,k}{\Gamma }_{22,k}{\mathsf{\Lambda }}_{21,k}{\mathsf{\Lambda }}_{11,k}^{-1}$$

(17)

$${\Gamma }_{12,k}=-{\mathsf{\Lambda }}_{11,k}^{-1}{\mathsf{\Lambda }}_{12,k}{\Gamma }_{22,k}$$

(18)

In Equations (15)–(18), the (n $\times$ m) CB gain matrix for the observation vector, $C_k$, and the (n $\times$ n), (n $\times$ m), and (m $\times$ m) modified error covariance matrices, ${\mathsf{\Lambda }}_{11,k}$, ${\mathsf{\Lambda }}_{12,k} \left(={\mathsf{\Lambda }}_{12,k}^T\right)$, and ${\mathsf{\Lambda }}_{22,k}$, respectively, are given by:

$$C_k=\left[H_k{\mathsf{\Sigma }}_{k|k-1}{G}_{2,k}^{-1}+R_k{H}_k\right]\left[G_{2,k}^{-1}{\mathsf{\Sigma }}_{k|k-1}{G}_{2,k}^{-1}+2(H_k^T{R}_k{H}_k+{\mathsf{\Sigma }}_{k|k-1}\right){]}^{-1}{G}_{2,k}^{-1}$$

(19)

$${\mathsf{\Lambda }}_{11,k}=R_k+\alpha \left(1-\alpha \right)C_k{\mathsf{\Sigma }}_{k|k-1}{C}_k^T+{\mathsf{\Lambda }}_{12,k}{H}_k^T+H_k{\mathsf{\Lambda }}_{21,k}$$

(20)

$${\mathsf{\Lambda }}_{12,k}=-\alpha C_k{\mathsf{\Sigma }}_{k|k-1}$$

(21)

$${\mathsf{\Lambda }}_{22,k}={\mathsf{\Sigma }}_{k|k-1}$$

(22)

In Equations (19)–(22), $R_k$ and ${\mathsf{\Sigma }}_{k|k-1}$ denote the (n $\times$ n) observation error covariance matrix, $Cov\left[V_k,V_k\right]$, and the (m $\times$ m) forecast error covariance matrix, respectively, and $G_{2,k}^{-1}=H_k^T{H}_k+I$.

If $\alpha =0$, the CBPKF solution is reduced to the KF, i.e., ${\mathsf{\Sigma }}_{k|k}={\left[H_k^T{R}_k^{-1}{H}_k+{\mathsf{\Sigma }}_{k|k-1}^{-1}\right]}^{-1},$ and $K_k={\mathsf{\Sigma }}_{k|k}{H}_k^T{R}_k^{-1}$ [5].

3. Geometric Analysis of the CB-Informed KF

3.1. Error Representation of Filter Equations

Since the error covariance of stochastic variables plays the key role in geometric analyses in the Euclidean space, this section defines errors in states and a residual in measurement in order to rewrite state updating and observation equations with error terms.

$${\epsilon }_k^C=X_k-X_{k|k}^C$$

(23)

$${\epsilon }_k^K=X_k-X_{k|k}^K$$

(24)

$${\epsilon }_k=X_k-X_{k|k-1}$$

(25)

$$y_k=Z_k-H_k{X}_{k|k-1}$$

(26)

In the above, ${\epsilon }_k^C$, ${\epsilon }_k^K$, and ${\epsilon }_k$ denote the CBPKF analysis error, the KF analysis error, and the forecast error, respectively; $y_k$ represents innovation; $X_k$, $X_{k|k}^C$, $X_{k|k}^K$, and $X_{k|k-1}$ represent the truth, the CBPKF-updated state, the KF-updated state, and the state forecast, respectively.

State update equations for the KF and the CBPKF are rewritten into Equations (27) and (28), assuming the same a priori states used in both filters.

$${\epsilon }_k^K={\epsilon }_k-K_k^K{y}_k$$

(27)

$${\epsilon }_k^C={\epsilon }_k-K_k^C{y}_k$$

(28)

where $K_k^K$ and $K_k^C$ are Kalman gains from the KF and the CBPKF, respectively.

Updated states from both filters have the following mathematical relationship.

$${\epsilon }_k^C={\epsilon }_k^K-\left(K_k^C-K_k^K\right)y_k$$

(29)

The observation model in Equation (3) can be rewritten as follows:

$$y_k=H_k{\epsilon }_k+V_k$$

(30)

From Equation (3), the covariance matrix of $y_k$, or $D_k$, can be written as

$$D_k=H_k{P}_k^{-}{H}_k^T+R_k$$

(31)

where $P_k^{-}$ is the covariance matrix of the state forecast.

3.2. KF and CBPKF Solutions for a Bi-State Model

A bi-state model ($X_k={\left[x_{1,k} x_{2,k}\right]}^T$) is used to illustrate the geometric relation of errors in states and observations from the CBPKF and the KF. Observation $z_{1,k}$ exists for a state $x_{1,k}$ but not for $x_{2,k}$ to investigate how state updating can be illustrated in the two-dimensional (2-D) Euclidean space for observable as well as unobservable states, i.e., $Z_k=\left[z_{1,k}\right];$ $H=\left[1 0\right]$ is used for simplicity. Kronhamn [19] used the same $X_k, Z_k$ and $H$ matrices as those described in this Section for the geometric illustration of the KF, whereas we focus on the CBPKF in reference to the KF. For the bi-state model with a forecast error covariance $P_{f,k}=\left[\begin{array}{c}{\sigma }_{1,k}^2{\sigma }_{12,k}\\ {\sigma }_{12,k}{\sigma }_{2,k}^2\end{array}\right]$ and the observation error $R_k=\left[{\sigma }_{z,k}^2\right]$, matrices of Kalman gain (Equations (32) and (33)), analysis error covariance (Equations (34) and (35)), and correlation (Equations (36)–(38)) for the CBPKF and the KF are evaluated in the equations below where ${\sigma }_{12,k }$=${\rho }_{12,k}{\sigma }_{1,k}{\sigma }_{2,k}$ with ${\rho }_{12,k}$ representing the correlation between $x_{1,k}$ and $x_{2,k}$.

$$K_k^C=\left[\begin{array}{c}{K}_{1,k}^C\\ K_{2,k}^C\end{array}\right]=K_k^K+\alpha \left[\begin{array}{c}\frac{2{\sigma }_{1,k}^2{\sigma }_{z,k}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}\\ \frac{2{\sigma }_{12,k}{\sigma }_{z,k}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}\end{array}\right]$$

(32)

where

$$K_k^K=\left[\begin{array}{c}{K}_{1,k}^K\\ K_{2,k}^K\end{array}\right]=\left[\begin{array}{c}\frac{{\sigma }_{1,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}\\ \frac{{\sigma }_{12,k}}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}\end{array}\right]$$

(33)

Equation (32) indicates that for an observable state $x_{1,k}$, $K_k^C$ is always larger than $K_k^K,$ but for an unobservable state $x_{2,k}$, the sign of $K_k^C-K_k^K$ depends on the sign of ${\sigma }_{12,k}$. The CBPKF covariance matrix $P_{a,k}^C$ and its relation to the KF-equivalent $P_{a,k}^K$ are given in Equation (34).

$$P_{a,k}^C=P_{a,k}^K+\alpha \left[\begin{array}{c}\frac{\alpha {\left(2{\sigma }_{1,k}^2{\sigma }_{z,k}^2\right)}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2} \frac{\alpha {\sigma }_{12,k}{\left(2{\sigma }_{1,k}^{}{\sigma }_{z,k}^2\right)}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2}\\ \frac{\alpha {\sigma }_{12,k}{\left(2{\sigma }_{1,k}^{}{\sigma }_{z,k}^2\right)}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2} \frac{\alpha {\left(2{\sigma }_{12,k}{\sigma }_{z,k}^2\right)}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2}\end{array}\right]$$

(34)

where

$$P_{a,k}^K=\left[\begin{array}{c}\frac{{\sigma }_{1,k}^2{\sigma }_{z,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2} \frac{{\sigma }_{12,k}{\sigma }_{z,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}\\ \frac{{\sigma }_{21,k}{\sigma }_{z,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2} {\sigma }_{2,k}^2-\frac{{\sigma }_{12,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}\end{array}\right]$$

(35)

In Equation (34), variances of both updated states $x_{1,k}$ and $x_{2,k}$ are larger than KF-equivalents, and the sign of covariance terms of $P_{a,k}^C-P_{a,k}^K$ depend on the sign of ${\sigma }_{12,k}$. Equations (34) and (35) imply that the CBPKF-updated state ensembles have larger spreads in the state space than the KF-updated, and that the matrix norm of $P_{a,k}^C$ is larger than that of $P_{a,k}^K$. From covariance matrices, Pearson product-moment correlation matrix $C_{a,k}^C$ can be computed by Equation (36).

$$C_{a,k}^C={\left(diag\left(P_{a,k}^C\right)\right)}^{-\frac{1}{2}}{P}_{a,k}^C{\left(diag\left(P_{a,k}^C\right)\right)}^{-\frac{1}{2}}=\left[\begin{array}{c}1 C_{a,12,k}^C\\ C_{a,12,k}^C 1\end{array}\right]$$

(36)

where diag( ) denotes a diagonal matrix. Equation (37) shows that the correlation coefficient of the CBPKF-updated states $C_{a,12,k}^C$ can be either larger or smaller than the KF-equivalent $C_{a,12,k}^K$.

$$\begin{array}{ll}\hfill C_{a,12,k}^C& =\frac{{\sigma }_{12,k}{\sigma }_{z,k}^{}\sqrt{\left({\left(1+2\alpha \right)}^2{\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}}{{\sigma }_{1,k}^{}\sqrt{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)\left({\sigma }_{2,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)-{\sigma }_{12,k}^2\right)+4\alpha \left(1+\alpha \right)\left({\sigma }_{1,k}^2-{\sigma }_{z,k}^2\right)\left({\sigma }_{1,k}^2{\sigma }_{2,k}^2-{\sigma }_{12,k}^2\right)}}\hfill \\ & =C_{a,12,k}^K\hfill \\ & +\frac{{\sigma }_{12,k}{\sigma }_{z,k}^{}\sqrt{A_1-A_2}}{{\sigma }_{1,k}^{}\sqrt{{\sigma }_{2,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)-{\sigma }_{12,k}^2}\sqrt{\left({\sigma }_{2,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)-{\sigma }_{12,k}^2\right)\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)+4\alpha \left(1+\alpha \right)\left({\sigma }_{1,k}^2-{\sigma }_{z,k}^2\right)\left({\sigma }_{1,k}^2{\sigma }_{2,k}^2-{\sigma }_{12,k}^2\right)}}\hfill \end{array}$$

(37)

where

$$A_1=\left({\sigma }_{2,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)-{\sigma }_{12,k}^2\right)\left({\left(1+2\alpha \right)}^2{\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)$$

$$A_2=\left({\sigma }_{2,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)-{\sigma }_{12,k}^2\right)\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)+4\alpha \left(1+\alpha \right)\left({\sigma }_{1,k}^2-{\sigma }_{z,k}^2\right)\left({\sigma }_{1,k}^2{\sigma }_{2,k}^2-{\sigma }_{12,k}^2\right)$$

$$C_{a,12,k}^K=\frac{{\sigma }_{12,k}{\sigma }_{z,k}^{}}{{\sigma }_{1,k}^{}\sqrt{{\sigma }_{2,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)-{\sigma }_{12,k}^2}}$$

(38)

From Equations (32)–(38), if $\alpha =0$, then $K_k^C=K_k^K$, $P_{a,k}^C=P_{a,k}^K$, and $C_{a,k}^C=C_{a,k}^K$.

3.3. Geometric Representation of KF and CBPKF Solutions

This Section begins with describing a relation between stochastic variables and vectors in the Euclidean domain at Equations (39)–(41), and then describes the geometric representation of KF and CBPKF equations. The covariance of stochastic variables $x_s$ and $y_s$ can be used to compute the scalar product of two vectors $\overrightarrow{x_E}$ and $\overrightarrow{y_E}$ in the Euclidean domain (Equation (39)). The vector norm $||\overrightarrow{x_E}||$ corresponds to the standard deviation of $x_s$ (Equation (40)). The angle of $\overrightarrow{x_E}$ and $\overrightarrow{y_E}$ can be computed from the correlation of $x_s$ and $y_s$ (Equation (41)) [19,22]. Figure 1 shows the vector representation of stochastic variables in the Euclidean space.

$$\overrightarrow{x_E}·\overrightarrow{y_E}=||\overrightarrow{x_E}|| ||\overrightarrow{y_E}||\cos \theta =Cov\left[x_s,y_s\right]={\sigma }_{x_s{y}_s}$$

(39)

$$||\overrightarrow{x_E}||=\sqrt{\overrightarrow{x_E}·\overrightarrow{x_E}}=\sqrt{Cov\left[x_s,x_s\right]}={\sigma }_{x_s}$$

(40)

$$\theta ={\cos }^{-1}\left(\frac{\overrightarrow{x_E}·\overrightarrow{y_E}}{||\overrightarrow{x_E}|| ||\overrightarrow{y_E}||}\right)={\cos }^{-1}\left(\frac{{\sigma }_{x_s{y}_s}}{{\sigma }_{x_s}{\sigma }_{y_s}}\right)={\cos }^{-1}\left({\rho }_{x_s{y}_s}\right)$$

(41)

Figure 2 shows the error vectors from (a) the KF and (b) the CBPKF for the observable state $x_{1,k}$ of the bi-state model, where $||{\overrightarrow{\epsilon }}_{1,k}||={\sigma }_{1,k}$, $||\overrightarrow{v_{1,k}}||={\sigma }_{z,k}$, and ${||{\overrightarrow{y}}_{1,k}||}^2=\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right),$ where the last equality is from Equation (30). In Figure 2a, the state forecast error vector ${\overrightarrow{\epsilon }}_{1,k}$ is orthogonal to the observation error vector $v_{1,k}$ owing to the independence assumption. Being a minimum variance solution,${\overrightarrow{\epsilon }}_{1,k}^K$ is orthogonal to ${\overrightarrow{y}}_{1,k}$ [19,23]. Figure 2a also shows that the forecast error is a vector sum of the gain-weighted innovation and the analysis error, i.e., ${\overrightarrow{\epsilon }}_{1,k}^{}=K_{1,k}^K{\overrightarrow{y}}_{1,k}+{\overrightarrow{\epsilon }}_{1,k}^K$ as expected from Equation (27). The KF analysis error variance may be obtained in Figure 2a via the Pythagorean theorem as ${||{\overrightarrow{\epsilon }}_{1,k}^K||}^2={||{\overrightarrow{\epsilon }}_{1,k}^{}||}^2-{||K_{1,k}^K{\overrightarrow{y}}_{1,k}||}^2={\sigma }_{1,k}^2-{\left(\frac{{\sigma }_{1,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}\right)}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)=\frac{{\sigma }_{1,k}^2{\sigma }_{z,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}$ where $K_{1,k}^K=\frac{{\sigma }_{1,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}$ as expected from Equation (35). Since ${\overrightarrow{\epsilon }}_{1,k}^C$=${\overrightarrow{\epsilon }}_{1,k}^K-\left(K_{1,k}^C-K_{1,k}^K\right){\overrightarrow{y}}_{1,k}$, we may write via the Pythagorean theorem $||{\overrightarrow{\epsilon }}_{1,k}^C||=\sqrt{{||{\overrightarrow{\epsilon }}_{1,k}^K||}^2+{||\left(K_{1,k}^C-K_{1,k}^K\right){\overrightarrow{y}}_{1,k}||}^2}=\sqrt{\frac{\left({\left(1+2\alpha \right)}^2{\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\sigma }_{1,k}^2{\sigma }_{z,k}^2}{{\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2}}$ which is expected from Equation (34). In Figure 2, the inequality, $||K_{1,k}^K{\overrightarrow{y}}_{1,k}||<||K_{1,k}^C{\overrightarrow{y}}_{1,k}||$, arises due to the fact that the CBPKF solution minimizes not the error variance but a weighted sum of the error variance and the variance of the Type-II CB.

Figure 3 is the same as Figure 2 but for the unobservable state $x_{2,k}$ where $||{\overrightarrow{\epsilon }}_{2,k}^{}||={\sigma }_{2,k}$; $||{\overrightarrow{\epsilon }}_{2p,k}^{}||=||{\overrightarrow{\epsilon }}_{2,k}^{}||\cos \theta =\frac{{\sigma }_{12,k}^{}}{{\sigma }_{1,k}^{}}$; $||{\overrightarrow{\epsilon }}_{2o,k}^{}||=\sqrt{{||{\overrightarrow{\epsilon }}_{2,k}^{}||}^2-{||{\overrightarrow{\epsilon }}_{2p,k}^{}||}^2}=\sqrt{{\sigma }_{2,k}^2-\frac{{\sigma }_{12,k}^2}{{\sigma }_{1,k}^2}}$; and $||{\overrightarrow{\epsilon }}_{2c,k}^{}||=\sqrt{{||{\overrightarrow{\epsilon }}_{2p,k}^{}||}^2-{||K_{2,k}^K{\overrightarrow{y}}_{1,k}||}^2}=\sqrt{\frac{{\sigma }_{12,k}^2{\sigma }_{z,k}^2}{{\sigma }_{1,k}^2\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}}$. In Figure 3a, the proportionality, $||{\overrightarrow{\epsilon }}_{2p,k}^{}||:||K_{2,k}^K{\overrightarrow{y}}_{1,k}||=||{\overrightarrow{\epsilon }}_{1,k}^{}||:||K_{1,k}^K{\overrightarrow{y}}_{1,k}||$, gives $K_{2,k}^K=K_{1,k}^K\frac{||{\overrightarrow{\epsilon }}_{2p,k}^{}||}{||{\overrightarrow{\epsilon }}_{1,k}^{}||}=K_{1,k}^K\frac{||{\overrightarrow{\epsilon }}_{2,k}^{}||\cos \theta }{||{\overrightarrow{\epsilon }}_{1,k}^{}||}=K_{1,k}^K\frac{{\sigma }_{2,k}^{}}{{\sigma }_{1,k}^{}}\frac{{\sigma }_{12,k}}{{\sigma }_{1,k}^{}{\sigma }_{2,k}^{}}=K_{1,k}^K\frac{{\sigma }_{12,k}}{{\sigma }_{1,k}^2}$ as expected from Equation (33). In Figure 3a, the orthogonality, ${\overrightarrow{\epsilon }}_{2,k}^K\perp {\overrightarrow{y}}_{1,k}$, and the equality ${\overrightarrow{\epsilon }}_{2,k}^K={\overrightarrow{\epsilon }}_{2,k}^{}-K_{2,k}^K{\overrightarrow{y}}_{1,k}$, yield ${||{\overrightarrow{\epsilon }}_{2,k}^K||}^2={||{\overrightarrow{\epsilon }}_{2,k}^{}||}^2-{||K_{2,k}^K{\overrightarrow{y}}_{1,k}||}^2={\sigma }_{2,k}^2-\frac{{\sigma }_{12,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}$ in agreement with Equation (35). In Figure 3b, ${||{\overrightarrow{\epsilon }}_{2,k}^C||}^2$ may be written via the Pythagorean theorem as:

$${||{\overrightarrow{\epsilon }}_{2,k}^C||}^2={||{\overrightarrow{\epsilon }}_{2,k}^K||}^2+{||\left(K_{2,k}^C-K_{2,k}^K\right){\overrightarrow{y}}_{1,k}||}^2$$

(42)

Using Equations (35), (32) and (31) for ${||{\overrightarrow{\epsilon }}_{2,k}^K||}^2$, $K_{2,k}^C-K_{2,k}^K$, and ${||{\overrightarrow{y}}_{1,k}||}^2$, respectively, we may rewrite Equation (42), in agreement with Equation (34), as:

$${||{\overrightarrow{\epsilon }}_{2,k}^C||}^2={\sigma }_{2,k}^2-\frac{{\sigma }_{12,k}^2}{{\sigma }_{1,k}^2+{\sigma }_{z,k}^2}+\frac{{\left(2\alpha {\sigma }_{12,k}{\sigma }_{z,k}^2\right)}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2}$$

(43)

Figure 2 and Figure 3 show that the KF- and the CBPKF-updated state error vectors point to different directions in the state space. Figure 4 shows the updated state error vectors in Figure 2 and Figure 3 to visually compare the differences in the angle, the magnitude, and the direction. The angles of the two-state error vectors for the CBPKF (${\theta }^C$ in Equation (44)) and the KF (${\theta }^K$ in Equation (45)) can be computed from the correlation $C_{a,12,k}^C$ and $C_{a,12,k}^K$ in Equations (37) and (38), respectively:

$${\theta }^C={\cos }^{-1}{C}_{a,12,k}^C={\cos }^{-1}\left(\frac{{\overrightarrow{\epsilon }}_{1,k}^C·{\overrightarrow{\epsilon }}_{2,k}^C}{||{\overrightarrow{\epsilon }}_{1,k}^C|| ||{\overrightarrow{\epsilon }}_{2,k}^C||}\right)={\cos }^{-1}\left(\frac{{\sigma }_{x_{1,k|k}^C{x}_{2,k|k}^C}}{{\sigma }_{x_{1,k|k}^C}{\sigma }_{x_{2,k|k}^C}}\right)$$

(44)

$${\theta }^K={\cos }^{-1}{C}_{a,12,k}^K={\cos }^{-1}\left(\frac{{\overrightarrow{\epsilon }}_{1,k}^K·{\overrightarrow{\epsilon }}_{2,k}^K}{||{\overrightarrow{\epsilon }}_{1,k}^K|| ||{\overrightarrow{\epsilon }}_{2,k}^K||}\right)={\cos }^{-1}\left(\frac{{\sigma }_{x_{1,k}^K{x}_{2,k}^K}}{{\sigma }_{x_{1,k|k}^K}{\sigma }_{x_{2,k|k}^K}}\right)$$

(45)

Below, we develop a set of geometric expressions in the 2-D state space for the analysis error covariance via eigenvalue decomposition (EVD, [24]).

3.4. Geometric Analysis in the State Space

Geometric characteristics of state ensembles in the 2-D state space can be quantified by confidence regions (CR), eigenvectors, eigenvalues, and the angle between the eigenvector and the basis vector of the x-axis. Assuming normal distributions for state ensembles in a 2-D state space, a CR, or so-called a covariance ellipse, can be constructed based on the EVD of a covariance matrix as well as the Chi-Square probability table. The presence of the CB results in different variances (eigenvalues) and directions (eigenvectors) of updated state ensembles in a 2-D state space. The major and minor axis lengths of the covariance ellipse are 2$\sqrt{s{\lambda }_1}$ and 2$\sqrt{s{\lambda }_2}$ where ${\lambda }_1>{\lambda }_2$ and the value of $s$ is from the Chi-Square probability table for a given confidence region, e.g., $s$= 4.605 for a 90% confidence region given the Chi-Square probability $P\left(s<4.605\right)=0.9$ in the case of degrees of freedom of 2. In a 2-D state space, the error in the orientation of the covariance ellipse with respect to the truth can be estimated by the angle, θ, between the largest eigenvector ${\overrightarrow{u}}_1$ and the vector $\overrightarrow{a}$ connecting the truth and the ensemble mean, i.e., $\theta \left({\overrightarrow{u}}_1,\overrightarrow{a}\right)={\cos }^{-1}\left(\frac{{\overrightarrow{u}}_1·\overrightarrow{a}}{||{\overrightarrow{u}}_1|| ||\overrightarrow{a}||}\right)$.

The EVD of the CBPKF analysis covariance $P_{a,k}^C$ may be written as:

$$P_{a,k}^C=U\mathrm{E}{U}^T$$

(46)

where

$$U=\left[\begin{array}{c}\cos \theta \sin \theta \\ -\sin \theta \cos \theta \end{array}\right]=\left[{\overrightarrow{u}}_1 {\overrightarrow{u}}_2\right], \mathrm{E}=\left[\begin{array}{c}{\lambda }_1^C 0\\ 0 {\lambda }_2^C\end{array}\right],$$

(47)

In the above, U is the eigenvector matrix which rotates the white data (W), or uncorrelated standard normal variates by $\theta$. The eigenvalue matrix E explains the variance along the principal error direction, or the direction of the eigenvector. In a 2-D state space, $\sqrt{\mathrm{E}}$ is a scale factor applied to W. The dataset (D) resulting from scaling W by $\sqrt{\mathrm{E}}$ and rotating by $U$, i.e., $D=U\sqrt{\mathrm{E}}W$, has the covariance matrix of $P_{a,k}^C=U\sqrt{\mathrm{E}}{\left(U\sqrt{\mathrm{E}}\right)}^T$. Below we apply EVD to the CBPKF analysis error covariance, $P_{a,k}^C$, from the bi-state model in Section 3.2.

With ${\lambda }_1^C>{\lambda }_2^C>0$, and $P_{a,k}^K=\left[\begin{array}{c}{p}_{1,k}^2 p_{12,k}\\ p_{12,k} p_{2,k}^2\end{array}\right]$, $P_{a,k}^C$ in Equation (34) may be rewritten as:

$$P_{a,k}^C=P_{a,k}^K+\zeta \left[\begin{array}{c}{\sigma }_{1,k}^4 {\sigma }_{12,k}{\sigma }_{1,k}^2\\ {\sigma }_{12,k}{\sigma }_{1,k}^2 {\sigma }_{12,k}^2 \end{array}\right] \mathrm{where} \zeta =\frac{{\left(2\alpha {\sigma }_{z,k}^2\right)}^2}{\left({\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right){\left(\left(1+2\alpha \right){\sigma }_{1,k}^2+{\sigma }_{z,k}^2\right)}^2}$$

(48)

Appendix A describes how eigenvalues and eigenvectors may be evaluated for a $2\times 2$ covariance matrix. Using Equation (A11) in Appendix A, we have for ${\lambda }_1^C$:

$$\begin{array}{cc}\hfill {\lambda }_1^C& =0.5\left({\left(p_{1,k}^2+\zeta {\sigma }_{1,k}^4\right)}^2+{\left(p_{2,k}^2+\zeta {\sigma }_{12,k}^2\right)}^2\right)\hfill \\ & +0.5\sqrt{{\left(p_{1,k}^2-p_{2,k}^2+\zeta {\sigma }_{1,k}^4-\zeta {\sigma }_{12,k}^2\right)}^2+4{\left(p_{12,k}+\zeta {\sigma }_{12,k}{\sigma }_{1,k}^2\right)}^2}\hfill \end{array}$$

(49)

The difference in the largest eigenvalue between the KF and the CBPKF analysis error covariance is given by:

$$\begin{array}{cc}\hfill {\lambda }_1^C-{\lambda }_1^K& =0.5\left(2p_{1,k}^2\zeta {\sigma }_{1,k}^4+2p_{2,k}^2\zeta {\sigma }_{12,k}^2+{\zeta }^2{\sigma }_{1,k}^8+{\zeta }^2{\sigma }_{12,k}^4\right)\hfill \\ & +0.5\sqrt{{\left(p_{1,k}^2-p_{2,k}^2+\zeta {\sigma }_{1,k}^4-\zeta {\sigma }_{12,k}^2\right)}^2+4{\left(p_{12,k}+\zeta {\sigma }_{12,k}{\sigma }_{1,k}^2\right)}^2}\hfill \\ & -0.5\sqrt{{\left(p_{1,k}^2-p_{2,k}^2\right)}^2+4{\left(p_{12,k}\right)}^2}\hfill \end{array}$$

(50)

where ${\lambda }_1^K$ denotes the largest eigenvalue of $P_{a,k}^K$. Using Equation (A12), we may write ${\lambda }_2^C$ and ${\lambda }_2^C-{\lambda }_2^K$:

$$\begin{array}{cc}\hfill {\lambda }_2^C& =0.5\left({\left(p_{1,k}^2+\zeta {\sigma }_{1,k}^4\right)}^2+{\left(p_{2,k}^2+\zeta {\sigma }_{12,k}^2\right)}^2\right)\hfill \\ & -0.5\sqrt{{\left(p_{1,k}^2-p_{2,k}^2+\zeta {\sigma }_{1,k}^4-\zeta {\sigma }_{12,k}^2\right)}^2+4{\left(p_{12,k}+\zeta {\sigma }_{12,k}{\sigma }_{1,k}^2\right)}^2}\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill {\lambda }_2^C-{\lambda }_2^K& =0.5\left(2p_{1,k}^2\zeta {\sigma }_{1,k}^4+2p_{2,k}^2\zeta {\sigma }_{12,k}^2+{\zeta }^2{\sigma }_{1,k}^8+{\zeta }^2{\sigma }_{12,k}^4\right)\hfill \\ & -0.5\sqrt{{\left(p_{1,k}^2-p_{2,k}^2+\zeta {\sigma }_{1,k}^4-\zeta {\sigma }_{12,k}^2\right)}^2+4{\left(p_{12,k}+\zeta {\sigma }_{12,k}{\sigma }_{1,k}^2\right)}^2}\hfill \\ & +0.5\sqrt{{\left(p_{1,k}^2-p_{2,k}^2\right)}^2+4{\left(p_{12,k}\right)}^2}\hfill \end{array}$$

(52)

Similarly, using Equation (A14), we may write ${\theta }_{a,k}^C\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)$ and ${\theta }_{a,k}^C\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)-{\theta }_{a,k}^K\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)$ as:

$${\theta }_{a,k}^C\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)=\frac{1}{2}{\tan }^{-1}\left(-\frac{2\left(p_{12,k}+\zeta {\sigma }_{12,k}{\sigma }_{1,k}^2\right)}{p_{1,k}^2+\zeta {\sigma }_{1,k}^4-p_{2,k}^2-\zeta {\sigma }_{12,k}^2}\right)$$

(53)

$${\theta }_{a,k}^C\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)-{\theta }_{a,k}^K\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)=\frac{1}{2}{\tan }^{-1}\left(-\frac{2\left(p_{12,k}+\zeta {\sigma }_{12,k}{\sigma }_{1,k}^2\right)}{p_{1,k}^2+\zeta {\sigma }_{1,k}^4-p_{2,k}^2-\zeta {\sigma }_{12,k}^2}\right)-\frac{1}{2}{\tan }^{-1}\left(-\frac{2p_{12,k}}{p_{1,k}^2-p_{2,k}^2}\right)$$

(54)

where the vector $\overrightarrow{i}$ is the basis vector of the x-axis and ${\theta }_{a,k}^C\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)-{\theta }_{a,k}^K\left({\overrightarrow{u}}_1,\overrightarrow{i}\right)$ is equal to ${\theta }_{a,k}^C\left({\overrightarrow{u}}_1,\overrightarrow{a}\right)-{\theta }_{a,k}^K\left({\overrightarrow{u}}_1,\overrightarrow{a}\right)$. With the geometric attributes established above, we now carry out the comparative geometric analysis of the KF and the CBPKF analysis results using the Lorenz 63 model [25].

With the EVD of $P_{a,k}^C$ and the ensemble mean $\left(\overline{x_1},\overline{x_2}\right)$, the minimum percentage confidence $C{R}_{MIN}$ to contain the verifying truth $\left(x_{1,T},x_{2,T}\right)$ within the confidence region can be computed by $C{R}_{MIN}=100\times P\left(s<\tilde{s}\right),$ where $P\left(s<\tilde{s}\right)$ is the Chi-Square probability with degrees of freedom of 2; $\tilde{s}$ satisfies Equation (55).

$$\frac{{\left\{\left(x_{1,T}-\overline{x_1}\right)\cos {\theta }_{a,k}^C+\left(x_{2,T}-\overline{x_2}\right)\sin {\theta }_{a,k}^C\right\}}^2}{\tilde{s}{\lambda }_1^C}+\frac{{\left\{\left(x_{1,T}-\overline{x_1}\right)\sin {\theta }_{a,k}^C-\left(x_{2,T}-\overline{x_2}\right)\cos {\theta }_{a,k}^C\right\}}^2}{\tilde{s}{\lambda }_2^C}=1$$

(55)

Figure 5 shows an example of computing $\overrightarrow{a}$, $\theta \left({\overrightarrow{u}}_1,\overrightarrow{a}\right)$, $C{R}_{MIN}$, and eigenvalues and eigenvectors of a covariance matrix.

3.5. Numerical Experiment with the Lorenz 63 Model

In the sections above, a linear two-state model was used for theoretical simplicity. In this section, we use the three-state Lorenz 63 model to illustrate the differences between the EnKF and the CBEnKF solutions in terms of the geometric attributes introduced above. In this experiment, synthetically generated observations of all three states in the Lorenz 63 model were assimilated at every time step using the EnKF and the CBEnKF [26]. Preliminary experiments suggested observation error variances (${\sigma }_z^2$) of 10 or 400 can be used for the cases of assimilating less uncertain or largely uncertain observations, respectively, based on the ensemble spread. To render the assimilation problem more challenging, ${\sigma }_z^2=400$ is used to compare the performance of the CBEnKF to that of the EnKF in Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11, where the ensemble size ($n_S$) used is 2000 to minimize filter performance degradation owing to a small ensemble size. Figure 12 and Figure 13 compare assimilation results from the two filters in the case of ${\sigma }_z^2=10$ or 400 as a function of an ensemble size, where $n_S$ values used include 10, 20, 30, 50, 70, 100, 200, 300, 500, 700, 1000, and 2000.

Figure 6 presents the results for states $x_1$ and $x_2$ of the Lorenz 63 model in a 2-D state space. The attributes shown include $|{\overrightarrow{a}}_h|$, $|{\overrightarrow{a}}_v|$, $C{R}_{MIN}$, $\sqrt{{\lambda }_1}$, $\theta$, $\rho$, and $\sqrt{{\lambda }_1}{\overrightarrow{u}}_1$; $|{\overrightarrow{a}}_h|,$ and $|{\overrightarrow{a}}_v|$ reflect errors in the ensemble mean measured along horizontal or vertical directions, respectively; large dots overlaid in the scatter plots represent mean values of the samples in each of the ten bins equally dividing the entire state space. Figure 7 and Figure 8 are the same as Figure 6 but for the state spaces of $\left(x_1,x_3\right)$ and $\left(x_2,x_3\right)$, respectively. The following summarizes general observations from Figure 6, Figure 7 and Figure 8. The spread of $|{\overrightarrow{a}}_h|$ and $|{\overrightarrow{a}}_v|$ in the scatter plots shows that the CBEnKF reduces CBs more effectively than the EnKF or the OL, particularly in the extremes. In some cases, however, their mean values appear similar, e.g., $|{\overrightarrow{a}}_h|$ in the state spaces of $\left(x_1,x_3\right)$ or $\left(x_2,x_3\right)$. The OL results for $|{\overrightarrow{a}}_h|$ and $|{\overrightarrow{a}}_v|$ show that their patterns appearing in the state space are similar to those of the state space plot in the top left of Figure 6, Figure 7 and Figure 8; this indicates the notable dependency of the amount of ensemble mean errors on model dynamics, e.g., the larger ensemble mean errors at the extreme—this is also seen in EnKF solutions but less so in the CBEnKF. This may be explained by the CBEnKF with a larger weight to observations than the EnKF in the case of largely uncertain observations (${\sigma }_z^2=400$), which reduces the reliance of CBEnKF solutions on the model dynamics. Based on $\sqrt{{\lambda }_1}$, CBEnKF covariances are generally larger than the EnKF at all state spaces. Larger $\sqrt{{\lambda }_1}$ and smaller $|{\overrightarrow{a}}_h|$ and $|{\overrightarrow{a}}_v|$ of the CBEnKF than the EnKF yield consistently smaller $C{R}_{MIN}$ than the EnKF at both extremes and the median of all three variables. This signifies the benefit of using the CB-informed KF for the estimation of extremes given that the EnKF’s $C{R}_{MIN}$ quickly increases towards extremes, i.e., the EnKF is less confident in estimating extremes than the CBEnKF. For example, 3% confidence regions for selected extreme values presented in the bottom right plots show the truth (green dots) contained within the CBEnKF’s confidence regions (red ellipses) but not within the EnKF’s (blue ellipses). At the plots, arrows represent $\sqrt{{\lambda }_1}{\overrightarrow{u}}_1$. $\theta$ and $\rho$ do not clearly indicate differences between the two filters.

Figure 9 shows $C{R}_{MIN}$, $|\overrightarrow{a}|$, $\sqrt{{\lambda }_1}$, $\theta$, and $\rho$ of Figure 6, Figure 7 and Figure 8 but as a function of exceedance probabilities to highlight CB-informed KF performances at extremes. At extremes with low exceedance probabilities, differences between the CBEnKF and the EnKF are vivid in the case of $|\overrightarrow{a}|$ and $C{R}_{MIN}$. On the other hand, $\sqrt{{\lambda }_1}$ of the CBEnKF is consistently larger than those of the EnKF and the OL across exceedance probabilities. As exceedance probabilities increase, the EnKF’s $|\overrightarrow{a}|$ becomes similar to the CBEnKF’s, implying unconditionally less biased. The CBEnKF keeps consistently low $C{R}_{MIN}$ at all exceedance probabilities owing to small $|\overrightarrow{a}|$ and large $\sqrt{{\lambda }_1}$, compared to the EnKF or the OL. Since the EnKF seeks orthogonal solutions to minimize analysis covariances, its $\sqrt{{\lambda }_1}$ is always smaller than the OL’s as well as the CBEnKF’s. On the other hand, the CBEnKF increases $\sqrt{{\lambda }_1}$ to address CBs which helps keep $C{R}_{MIN}$ low to contain the truth. Both $\theta$ and $\rho$ show no consistent patterns across different state spaces as well as exceedance probabilities.

In Figure 10, the ensemble mean error time series indicates that among the three variables, CBEnKF’s improvement is the largest for $x_1$. On the other hand, for the state $x_3$, the CBEnKF mainly remedies the underestimation of $x_3$ compared to the EnKF. In the case of $x_2$, the CBEnKF slightly outperforms the EnKF. These observations may imply the different amounts of CBs present in different states, hence the need of applying a separate weight $\alpha$ to the CB penalty for the individual state, which warrants a future effort. To compare $P_{a,k}^{}$ and $K_k^{}$ from the two filters, the time series of Frobenius norm of $P_{a,k}^{}$ and $K_k^{}$ is computed by Equations (56) and (57), respectively. Compared to the EnKF, the CBEnKF yields ${||K_k^{}||}_F$ and ${||P_{a,k}^{}||}_F$ consistently larger at all assimilation cycles, and the mean values of ${||K_k^{}||}_F$ and ${||P_{a,k}^{}||}_F$ are five and three times larger, respectively.

$${||P_{a,k}^{}||}_F=\sqrt{{{\displaystyle \sum }}_{i=1}^3{\lambda }_i^2}$$

(56)

$${||K_k^{}||}_F=\sqrt{{{\displaystyle \sum }}_{i=1}^3{\lambda }_i^2}$$

(57)

Figure 11 shows mean ${||K_k^{}||}_F$, and ${||P_{a,k}^{}||}_F$ as a function of exceedance probabilities. At extremes, both the CBEnKF and the EnKF show that mean ${||K_k^{}||}_F$ and ${||P_{a,k}^{}||}_F$ are larger than those at high exceedance probabilities, and that large differences in mean ${||K_k^{}||}_F$ and ${||P_{a,k}^{}||}_F$ between the CBEnKF and the EnKF are consistent across exceedance probabilities.

Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11 are based on the case of uncertain observations (${\sigma }_z^2=400$) where the CBEnKF may supposedly outperform the EnKF. To explore the CBEnKF performance with less uncertain observations (${\sigma }_z^2=10$) and also to see the sensitivity to the ensemble size ($n_S$), Figure 12 presents results from the combination of $n_S$ = 10, 20, 30, 50, 70, 100, 200, 300, 500, 700, 1000, and 2000, and ${\sigma }_z^2=10$ and 400. In Figure 12, $|\overrightarrow{a}|$ plots indicate that with ${\sigma }_z^2=10$, the accuracy of the ensemble mean continuously increases with an increase of $n_S$ at both cases of extremes (an exceedance probability of 0.1; red and blue dots for the CBEnKF and the EnKF, respectively) and all data (red and blue lines for the CBEnKF and the EnKF, respectively). When ${\sigma }_z^2=10$, the EnKF’s $|\overrightarrow{a}|$ is slightly smaller than the CBEnKF’s, but the CBEnKF’s $\sqrt{{\lambda }_1}$ is slightly larger than the EnKF’s. The resulting $C{R}_{MIN}$ from both filters are very similar. This implies when observations are less uncertain, the EnKF solutions are as accurate and as confident as the CBEnKF solutions at extremes as well as the whole range. When $n_S\ge$ 200 and ${\sigma }_z^2=10$, mean $C{R}_{MIN}$ maintains ~1%. When $n_S<$ 200 and ${\sigma }_z^2=10$, $C{R}_{MIN}$ quickly increases with a decrease of $n_S$ because of inaccurate error covariance estimates with an insufficient ensemble size. When observations are largely uncertain (${\sigma }_z^2=2000)$, the CBEnKF clearly shows more accurate ensemble means (smaller $|\overrightarrow{a}|$) and higher confidence in covariance estimates (smaller $C{R}_{MIN}$) than the EnKF, particularly at extremes. Compared to ${\sigma }_z^2=10$, assimilating largely uncertain observations (${\sigma }_z^2=2000$) reduces accuracies in covariance estimates, resulting in larger $\sqrt{{\lambda }_1}$ in both filters, although the CBEnKF’s $\sqrt{{\lambda }_1}$ addressing the CB is larger than the EnKF’s. When ${\sigma }_z^2=2000$, $|\overrightarrow{a}|$ and $C{R}_{MIN}$ tend to be less sensitive to $n_S$ than the case of ${\sigma }_z^2=10$. Both $\theta$ and $\rho$ show neither any consistent patterns nor sensitivities to $n_S$, but are included in Figure 12 for completeness.

Finally, Figure 13 presents mean ${||K_k^{}||}_F$ and ${||P_{a,k}^{}||}_F$ as a function of $n_S$. Compared to the results from ${\sigma }_z^2=2000$, ${\sigma }_z^2=10$ results in larger ${||K_k^{}||}_F$ in both filters due to bigger weights to the observations. When ${\sigma }_z^2=2000$, the CBEnKF maintains relatively large ${||K_k^{}||}_F$ to account for the CB; however, the EnKF’s ${||K_k^{}||}_F$ is conspicuously small. Both ${||K_k^{}||}_F$ and ${||P_{a,k}^{}||}_F$ tend to be little sensitive to the ensemble size $n_S$, except the all data case of the CBEnKF with ${\sigma }_z^2=2000$ (pink line). With uncertain observations (${\sigma }_z^2=2000$), the CBEnKF’s ${||P_{a,k}^{}||}_F$ becomes large at extremes (pink dots) as well as all data (pink line) at all $n_S$ values used to reflect CBs in all states.

4. Conclusions

Error covariance and gain matrices of two CB-informed KFs, i.e., the CBPKF and the CBEnKF, are geometrically illustrated and compared with the KF equivalents [19] for a bi-state model using error vectors in the Euclidean space. Geometric illustration and analysis offer an intuitive understanding of the relationship between the two filters. Unlike the KF, the CBPKF solution is not orthogonal to its error, which renders its error covariances and gains to be larger than the KF’s. The above differences result in different confidence regions and principal error directions in the state space. Synthetic sensitivity experiments with the Lorenz 63 model showed that the CBEnKF solutions have generally smaller errors in the ensemble mean, larger eigenvalues in the error covariance matrix, more accurate confidence regions for encompassing the truth, and larger Frobenius norms of the error covariance and gain matrices than the KF. The above differences are particularly pronounced when the observations are highly uncertain.

Future research recommendations include applying the CBPKF and the CBEnKF to diverse geophysical problems of estimating and predicting extremes, e.g., extreme precipitation or floods. The bi-state model was used in this work for a comparative geometric analysis of the CBPKF and the KF. Possible extension to an arbitrary number of states poses an interesting research topic.