Two Novel Two-Stage Direction of Arrival Estimation Algorithms for Two-Dimensional Mixed Noncircular and Circular Sources

Abstract

This paper addresses the two-dimensional (2D) direction-of-arrival (DOA) estimation problem with two novel methods for mixed noncircular and circular signals. The first proposed method is named the two-stage direction-of-arrival matrix (TSDOAM) method, and the other is called the two-stage rank reduction (TSRARE) method. The proposed methods utilize both the circularity and the direction-of-arrival differences between the noncircular and circular sources to estimate the 2D directions-of-arrival (DOAs). The maximum detectable 2D angle parameters of the TSDOAM and TSRARE methods are twice those of the existing methods. Moreover, the TSRARE method can detect more incident signals than the TSDOAM method due to the array aperture of two parallel uniform linear arrays (ULAs) being fully utilized. Simulation results show that compared to the existing methods for the small angle separation of 2D directions-of-arrival, the two proposed methods perform well in terms of the signal-to-noise ratio (SNR) and snapshots.

1. Introduction

Recently, the noncircularity of incident signals has been widely reported in the field of array signal processing, including direction-of-arrival (DOA) estimation [1,2,3,4,5,6,7,8,9,10] and beamforming [11,12,13] to improve the performance of direction-of-arrival estimation accuracy and beamformers. The aforementioned direction-of-arrival estimation algorithms are mainly focused on the one-dimensional (1D) domain. However, in practice, two-dimensional (2D) direction-of-arrival estimation with various array structures, such as two-parallel arrays [14,15,16,17,18,19], L-shaped arrays [20,21,22,23,24,25,26,27], and a uniform rectangular array [28,29,30], are closer to the actual situation.

In order to improve the direction-of-arrival estimation performance, many effective noncircular algorithms for 2D directions-of-arrival have been presented in [31,32,33]. In [31], Liu et al. proposed an extended rank reduction (ERARE) method with noncircular information exploited for two-parallel uniform linear arrays (ULAs) which achieved an improved estimation accuracy compared to [15]. Based on [25], the conjugate information of the observed data was utilized to realize a better 2D direction-of-arrival estimation [32]. use of the conjugate information of the observed data to realize a better 2D direction-of-arrival estimation. A method that applied noncircular direction finding to the hexagonally-shaped electronically steerable parasitic antenna radiator (ESPAR) array was presented in [33], and the Cramér–Rao bound (CRB) was analyzed for comparison. However, the aforementioned algorithms cannot cope with the direction-of-arrival estimation problem for the mixed noncircular (e.g., binary phase shift keying, BPSK) and circular (e.g., quaternary phase shift keying, QPSK) signal scenario. Although Yin’s method [15] and Xia’s method [16] could be applicable to the above-mentioned mixed signal scenario, the distinguishable signals were less than the array elements.

In [34,35,36], direction-of-arrival estimation schemes for joint noncircular and circular signal estimation were proposed for 1D direction finding. In [34], Gao. et al. proposed a method that constructed a new data vector with the original data and the conjugate ones to form two estimators for noncircular and circular signal finding, respectively. However, the method in [34] cannot deal with the coincident directions-of-arrival of noncircular and circular signals, and its estimation performance degraded severely in small angle separation . In addition, the maximum number of detected signals was still limited. An improved method was presented in [35] to solve the above problems, which adopted the direction-of-arrival circularity difference rather than the direction-of-arrival difference between the noncircular and circular signals to estimate the direction-of-arrival. Nevertheless, the direction-of-arrival estimation performance dropped due to few observed data being available. In [36], a sparse representation method for mixed signals was proposed by exploiting overcomplete dictionaries that were subject to the sparsity constraint to jointly represent the covariance and elliptic covariance matrices of the array output. However, for 2D situations, there are few research works for joint noncircular and circular signal direction finding. Additionally, much of the work in array processing has also been focused on optimization problems, such as genetic algorithms [37,38,39,40].

In this paper, inspired by the method in [35], two novel 2D direction-of-arrival estimation algorithms using two parallel ULAs with a two-stage direction-of-arrival matrix (TSDOAM) method and a two-stage rank reduction (TSRARE) method, separately, are proposed for mixed noncircular and circular signals estimation. The direction-of-arrival circularity difference rather than the direction-of-arrival difference between the noncircular and circular signals for the 2D directions-of-arrival’s estimation is utilized in the two proposed methods. The maximum number of distinguished mixed signals of the two proposed methods are identified compared to the conventional methods, which show that the detected number of the signals is more than that of the array elements. Moreover, when both of the 2D directions-of-arrival are incoming from small angle separation—even when both of them are from the overlapping direction—the estimation accuracy of the two proposed methods is better than Yin’s and Xia’s method.

The rest of this paper is organized as follows. The array signal model is introduced in Section 2. The TSDOAM and TSRARE methods are described in detail in Section 3. The maximum number of detective mixed signals is analyzed in Section 4. Simulation results are presented to verify the performance of the two proposed methods in Section 5. Conclusions are drawn in Section 6.

Throughout this paper, the following notations are used. ${(·)}^{*}$, ${(·)}^T$, ${(·)}^{-1}$, ${(·)}^{+}$, and ${(·)}^H$ represent conjugation, transpose, inverse, pseudo-inverse, and conjugate transpose, respectively. $E(·)$ indicates the expectation operator; $\mathit{arg}(·)$ is to get the phase; $\mathrm{diag}(·)$ stands for the diagonalization operation of a vector.

2. Array Signal Model

As shown in Figure 1, suppose that there are $K=K_n+K_c$ (assume the number of mixed signals is known to the receiver) uncorrelated far-field sources that are mixed $K_n$ noncircular sources $s_{n,k}\left(t\right)$ and $K_c$ circular sources $s_{c,k}\left(t\right)$ from direction $({\theta }_k,{\beta }_k),k=1,2,\dots ,K$, impinging on two parallel ULAs with each array having M elements. The distance between the two arrays is $\lambda$/2, denoted as $d_y$, and the interelement spacing $d_x$ on each array is also $\lambda$/2, where $\lambda$ is the wavelength of the incident waves. The additive noises of two ULAs are circular Gaussian noises, which are uncorrelated with the incoming signals.

The observed data vectors $\mathbf{X}\left(t\right)={[x_1\left(t\right),x_2\left(t\right),\dots ,x_M\left(t\right)]}^T$ and $\mathbf{Y}\left(t\right)={[y_1\left(t\right),y_2\left(t\right),\dots ,y_M\left(t\right)]}^T$ from two parallel ULAs are given by:

$$\mathbf{X}\left(t\right)=\mathbf{A}\mathbf{S}\left(t\right)+{\mathbf{N}}_x\left(t\right)={\mathbf{A}}_n{\mathbf{S}}_n\left(t\right)+{\mathbf{A}}_c{\mathbf{S}}_c\left(t\right)+{\mathbf{N}}_x\left(t\right)$$

(1)

$$\mathbf{Y}\left(t\right)=\mathbf{A}\mathbf{B}\mathbf{S}\left(t\right)+{\mathbf{N}}_y\left(t\right)={\mathbf{A}}_n\mathbf{B}{\mathbf{S}}_n\left(t\right)+{\mathbf{A}}_c\mathbf{B}{\mathbf{S}}_c\left(t\right)+{\mathbf{N}}_y\left(t\right)$$

(2)

where $\mathbf{A}$ is called the steering vector with each column denoted $\mathbf{a}\left({\theta }_k\right)$ and $\mathbf{a}\left({\theta }_k\right)={[a_0\left({\theta }_k\right),\dots ,a_{M-1}\left({\theta }_k\right)]}^T$, whose element can be expressed as $a_i\left({\theta }_k\right)=e^{-j\frac{2\pi }{\lambda }{d}_x\left(i\right)cos{\theta }_k}$. $\mathbf{B}\left(\beta \right)$ is termed as the steering element matrix with the expression $\mathbf{B}=\mathrm{diag}[v\left({\beta }_1\right),v\left({\beta }_2\right),\dots ,v\left({\beta }_K\right)]$, and the element $v\left({\beta }_k\right)$ has the form of $e^{j\frac{2\pi }{\lambda }{d}_{y}cos{\beta }_k}$. $\mathbf{S}\left(t\right)=[s_1\left(t\right),s_2\left(t\right),\dots ,s_K\left(t\right)]$ denotes the radiating signal vector. ${\mathbf{N}}_x\left(t\right)={[n_{x,1}\left(t\right),\dots ,n_{x,M}\left(t\right)]}^T$ and ${\mathbf{N}}_y\left(t\right)={[n_{y,1}\left(t\right),\dots ,n_{y,M}\left(t\right)]}^T$ represent the circular Gaussian noise vectors of the two arrays, respectively.

3. The Two Proposed Algorithms

3.1. The TSDOAM Method

A novel method called the TSDOAM method with the two-stage direction-of-arrival matrix (DOAM) method and direction-of-arrival circularity difference, is proposed in this part. According to Equations (1) and (2), and based on the assumption that the noise and the signals are uncorrelated and that the mixed signals are also uncorrelated, the auto-covariance matrix ${\mathbf{R}}_{xx}$ and cross-covariance matrix ${\mathbf{R}}_{yx}$ can be written, respectively, as follows.

$${\mathbf{R}}_{xx}=E\left[\mathbf{X}\left(t\right){\mathbf{X}}^H\left(t\right)\right]={\mathbf{A}}_n{\mathbf{R}}_n{\mathbf{A}}_n^H+{\mathbf{A}}_c{\mathbf{R}}_c{\mathbf{A}}_c^H+{\sigma }^2{\mathbf{I}}_{xx}$$

(3)

$${\mathbf{R}}_{yx}=E\left[\mathbf{X}\left(t\right){\mathbf{Y}}^H\left(t\right)\right]={\mathbf{A}}_n{\mathsf{\Phi }}_n{\mathbf{R}}_n{\mathbf{A}}_n^H+{\mathbf{A}}_c{\mathsf{\Phi }}_c{\mathbf{R}}_c{\mathbf{A}}_c^H$$

(4)

where ${\mathbf{A}}_n$ and ${\mathbf{A}}_c$ denote the steering matrices associated with noncircular and circular signals, separately. ${\sigma }^2$ is the variance of the circular Gaussian noises, and ${\mathbf{R}}_n$ and ${\mathbf{R}}_c$ have the form of ${\mathbf{R}}_n=\mathrm{diag}\{E[s_{n,1}{s}_{n,1}^{*}],...,E[s_{n,K_n}{s}_{n,K_n}^{*}]\}$ and ${\mathbf{R}}_c=\mathrm{diag}\{E\left[s_{c,1}{s}_{c,1}^{*}\right],...,E\left[s_{c,K_c}{s}_{c,K_c}^{*}\right]\}$, respectively.

In practice, non-circularity and circularity are important properties of a random variable; their concept comes directly from the geometrical interpretation of a complex random variable. The source would be called a circular source if its statistical characteristics have a rotational invariance characteristic; otherwise, it would be called a noncircular source. Based on this, we only consider the rotational invariance characteristic of the first- and second-order statistical properties of the sources. For a complex random source $s_k$, we define $E\left[s_k\right]$, $E\left[s_k{s}_k^{*}\right]$, and $E\left[s_k^2\right]$ as the mean, the covariance, and the elliptic covariance of the source $s_k$, respectively. For an arbitrary phase ${\phi }_k$ as follows:

$$E[s_k{e}^{j{\phi }_k}]=E\left[s_k\right]$$

(5)

$$E[s_k{e}^{j{\phi }_k}{\left(s_k{e}^{j{\phi }_k}\right)}^{*}]=E[s_k{s}_k^{*}]$$

(6)

$$E[s_k{e}_k^{j\phi }\left(s_k{e}_k^{j\phi }\right)]=E[s_k^2]$$

(7)

If the source’s first- and second-order statistical properties are rotationally invariant, the source is identified as circular; otherwise, it is determined to be noncircular. Therefore, the elliptic auto-covariance matrix ${\mathbf{R}}_{xx}^{\prime }$ can be expressed as follows:

$${\mathbf{R}}_{xx}^{\prime }=E\left[\mathbf{X}\left(t\right){\mathbf{X}}^T\left(t\right)\right]={\mathbf{A}}_n{\mathbf{R}}_n^{\prime }{\mathbf{A}}_n^T+{\mathbf{A}}_c{\mathbf{R}}_c^{\prime }{\mathbf{A}}_c^T+E\left[{\mathbf{N}}_x\left(t\right){\mathbf{N}}_x^T\left(t\right)\right]$$

(8)

Notice that for a complex circular random variable h, $E\left[hh\right]=0$ [6]. Therefore, the circular component and the circular Gaussian noise component of Equation (8) both equal zero, and the elliptic auto-covariance matrix ${\mathbf{R}}_{xx}^{\prime }$ can be rewritten as:

$${\mathbf{R}}_{xx}^{\prime }={\mathbf{A}}_n{\mathbf{R}}_n^{\prime }{\mathbf{A}}_n^T$$

(9)

where ${\mathbf{R}}_n^{\prime }=\mathrm{diag}\{E\left[s_{n,1}{s}_{n,1}\right],...,E\left[s_{n,K_n}{s}_{n,K_n}\right]\}$.

Similarly, the elliptic cross-covariance matrix ${\mathbf{R}}_{yx}^{\prime }$ is computed as follows:

$${\mathbf{R}}_{yx}^{\prime }={\mathbf{A}}_n{\mathsf{\Phi }}_n{\mathbf{R}}_n^{\prime }{\mathbf{A}}_n^T$$

(10)

We then estimate the 2D directions-of-arrival of noncircular signals with Equations (9) and (10). First, let $\{{\eta }_{n,1},...,{\eta }_{n,K_n}\}$ and $\{{\mathbf{v}}_{n,1},...,{\mathbf{v}}_{n,K_n}\}$ be the eigenvalues and corresponding eigenvectors of ${\mathbf{R}}_{xx}^{\prime }$, respectively, namely:

$${\mathbf{R}}_{xx}^{\prime }={\displaystyle \sum _{k=1}^{K_n}}{\eta }_{n,k}{\mathbf{v}}_{n,k}{\mathbf{v}}_{n,k}^H$$

(11)

The pseudo-inverse of ${\mathbf{R}}_{xx}^{\prime +}$ is:

$${\mathbf{R}}_{xx}^{\prime +}={\displaystyle \sum _{k=1}^{K_n}}{\eta }_{n,k}^{-1}{\mathbf{v}}_{n,k}{\mathbf{v}}_{n,k}^H$$

(12)

Due to ${\mathbf{R}}_n^{\prime }$ being a diagonal matrix and ${\mathbf{A}}_n$ a column full-rank matrix, we attain the following formula with Equation (9):

$${\mathbf{R}}_n^{\prime }{\mathbf{A}}_n^T={\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H{\mathbf{R}}_{xx}^{\prime }$$

(13)

Combining Equation (13) with (10), we obtain an alternative expression of ${\mathbf{R}}_{yx}^{\prime }$:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{yx}^{\prime }& ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\mathbf{R}}_n^{\prime }{\mathbf{A}}_n^T\hfill \\ & ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H{\mathbf{R}}_{xx}^{\prime }\hfill \end{array}$$

(14)

Right-multiplying both sides of Equation (14) by ${\mathbf{R}}_{xx}^{\prime +}{\mathbf{A}}_n$, we get:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}{\mathbf{A}}_n& ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\mathbf{R}}_n^{\prime }{\mathbf{A}}_n^T{\mathbf{R}}_{xx}^{\prime +}{\mathbf{A}}_n\hfill \\ & ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H{\mathbf{R}}_{xx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}{\mathbf{A}}_n\hfill \end{array}$$

(15)

Substituting Equations (11) and (12) into Equation (15),

$$\begin{array}{cc}\hfill {\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}{\mathbf{A}}_n& ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H\left({\displaystyle \sum _{k=1}^{K_n}}{\eta }_{n,k}{\mathbf{v}}_{n,k}{\mathbf{v}}_{n,k}^H\right)\left({\displaystyle \sum _{k=1}^{K_n}}{\eta }_{n,k}^{-1}{\mathbf{v}}_{n,k}{\mathbf{v}}_{n,k}^H\right){\mathbf{A}}_n\hfill \\ & ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H\left({\displaystyle \sum _{k=1}^{K_n}}{\mathbf{v}}_{n,k}{\mathbf{v}}_{n,k}^H\right){\mathbf{A}}_n\hfill \end{array}$$

(16)

From Equations (1) and (2), we can see that the dimensions of the observed data vectors $\mathbf{X}\left(t\right)$ and $\mathbf{Y}\left(t\right)$ are both $M\times 1$. Then, based on Equations (3) and (4), it is easy to know that the dimensions of the auto-covariance matrix ${\mathbf{R}}_{xx}$ and cross-covariance matrix ${\mathbf{R}}_{yx}$ are both $M\times M$. From Equation (8), we can see that the dimension of the elliptic auto-covariance matrix ${\mathbf{R}}_{xx}^{\prime }$ is also $M\times M$. Therefore, ${\mathbf{R}}_{xx}^{\prime }$ is a square matrix. Specifically for the first proposed method in this paper, the ${\mathbf{R}}_{xx}^{\prime +}$—which is the pseudo-inverse of ${\mathbf{R}}_{xx}^{\prime }$—is equivalent to the inverse of ${\mathbf{R}}_{xx}^{\prime }$. That is to say, the ${\mathbf{R}}_{xx}^{\prime }$ and ${\mathbf{R}}_{xx}^{\prime +}$ in the first proposed method are both square matrices. Moreover, the diagonal elements of ${\mathbf{R}}_{xx}^{\prime +}$ are nonzero elements , and ${\mathbf{R}}_{xx}^{\prime +}$ is full rank. Therefore, ${\displaystyle \sum _{k=1}^{K_n}}{\mathbf{v}}_{n,k}{\mathbf{v}}_{n,k}^H$ is an identity matrix, and Equation (16) can be simplified as:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}{\mathbf{A}}_n& ={\mathbf{A}}_n{\mathsf{\Phi }}_n{\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H{\mathbf{A}}_n\hfill \\ & ={\mathbf{A}}_n{\mathsf{\Phi }}_n\hfill \end{array}$$

(17)

From Equation (17), 2D directions-of-arrival of noncircular signals—which are obtained by performing eigenvalue decomposition (EVD) of ${\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}$, denoted as the direction-of-arrival matrix, lie in ${\mathbf{A}}_n$ and ${\mathsf{\Phi }}_n$, respectively.

$${\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}={\displaystyle \sum _{k=1}^{K_n}}{\xi }_{n,k}{\mathbf{u}}_{n,k}{\mathbf{u}}_{n,k}^H$$

(18)

where ${\xi }_{n,k}$ and ${\mathbf{u}}_{n,k}$ are the eigenvalues and the corresponding eigenvectors of ${\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}$, respectively. It can be verified that the spanned subspace from the steering matrix ${\mathbf{A}}_n$ and the signal subspace ${\mathbf{U}}_n=[{\mathbf{u}}_{n,1},\dots ,{\mathbf{u}}_{n,K_n}]$ are the same.

Unlike Yin’s method obtaining the 1D angle by the spectrum search with a certain region (which entailed high complexity), here, define ${\mathbf{h}}_{n,k}={\mathbf{u}}_{n,k}/{\mathbf{u}}_{n,k}\left(1\right)$; we can get:

$${\kappa }_{n,k}=\frac{1}{M-1}{\displaystyle \sum _{i=1}^{M-1}}arg\left[\frac{{\mathbf{h}}_{n,k}(i+1)}{{\mathbf{h}}_{n,k}\left(i\right)}\right].$$

(19)

Integrating the expression $a_i\left({\theta }_{n,k}\right)$ and $v\left({\beta }_{n,k}\right)$ with Equations (18) and (19), the estimated 2D directions-of-arrival of noncircular signals are achieved as follows:

$${\theta }_{n,k}=arccos(-\frac{\lambda }{2\pi d_x}{\kappa }_{n,k}),$$

(20)

$${\beta }_{n,k}=arccos\left[\frac{\lambda }{2\pi d_y}arg\left({\xi }_{n,k}\right)\right].$$

(21)

In the next stage, the 2D direction-of-arrival of the circular signals can be obtained with the estimates ${\theta }_{n,k}$ and ${\beta }_{n,k}$ above.

With Equation (9), ${\mathbf{R}}_n^{\prime }$ can be estimated as:

$${\mathbf{R}}_n^{\prime }={\mathbf{A}}_n^{+}{\mathbf{R}}_{xx}^{\prime }{\left({\mathbf{A}}_n^T\right)}^{+}$$

(22)

where ${\mathbf{A}}_n^{+}={\left({\mathbf{A}}_n^H{\mathbf{A}}_n\right)}^{-1}{\mathbf{A}}_n^H$ and ${\left({\mathbf{A}}_n^T\right)}^{+}={\mathbf{A}}_n^{*}{\left({\mathbf{A}}_n^T{\mathbf{A}}_n^{*}\right)}^{-1}$ [41].

Let the kth diagonal element of ${\mathbf{R}}_n^{\prime }$ be ${\mathbf{R}}_n^{\prime }(k,k)$; we get:

$${\mathbf{R}}_n^{\prime }(k,k)=E\left[s_{n,k}{s}_{n,k}\right].$$

(23)

here, we assume that the noncircular signals are BPSK-modulated signals. Therefore, $E\left[s_{n,k}{s}_{n,k}\right]={\sigma }_{n,k}^2{e}^{j{\phi }_{n,k}}$, where ${\sigma }_{n,k}^2=E\left[s_{n,k}{s}_{n,k}^{*}\right]$ and ${\phi }_{n,k}$ are the noncircular phases. It is easily deduced that ${\sigma }_{n,k}^2=\left|{\mathbf{R}}_n^{\prime }(k,k)\right|$. Due to ${\mathbf{R}}_n$ being diagonal, it follows that:

$$\begin{array}{cc}\hfill {\mathbf{R}}_n& =\mathrm{diag}[{\sigma }_{n,1}^2,\dots ,{\sigma }_{n,K_n}^2]\hfill \\ & =\mathrm{diag}\left[\right|{\mathbf{R}}_n^{\prime }(1,1)|,\dots ,|{\mathbf{R}}_n^{\prime }(K_n,K_n)\left|\right].\hfill \end{array}$$

(24)

Then, let ${\mathbf{R}}_{xx,1}={\mathbf{A}}_n{\mathbf{R}}_n{\mathbf{A}}_n^H$ and ${\mathbf{R}}_{yx,1}={\mathbf{A}}_n{\mathsf{\Phi }}_n{\mathbf{R}}_n{\mathbf{A}}_n^H$; we attain:

$${\mathbf{R}}_{xx}-{\mathbf{R}}_{xx,1}={\mathbf{A}}_c{\mathbf{R}}_c{\mathbf{A}}_c^H+{\sigma }^2{\mathbf{I}}_{xx}.$$

(25)

$${\mathbf{R}}_{yx}-{\mathbf{R}}_{yx,1}={\mathbf{A}}_c{\mathsf{\Phi }}_n{\mathbf{R}}_c{\mathbf{A}}_c^H.$$

(26)

Define:

$${\mathbf{R}}_{xx,2}={\mathbf{R}}_{xx}-{\mathbf{R}}_{xx,1}-{\sigma }^2{\mathbf{I}}_{xx}={\mathbf{A}}_c{\mathbf{R}}_c{\mathbf{A}}_c^H$$

(27)

$${\mathbf{R}}_{yx,2}={\mathbf{R}}_{yx}-{\mathbf{R}}_{yx,1}={\mathbf{A}}_c{\mathsf{\Phi }}_c{\mathbf{R}}_c{\mathbf{A}}_c^H$$

(28)

We have another direction-of-arrival matrix ${\mathbf{R}}_{yx,2}{\mathbf{R}}_{xx,2}^{+}$ that is related to circular signals.

Just as the way of attaining the 2D directions-of-arrival of noncircular signals, the 2D directions-of-arrival ${\theta }_{c,k}$ and ${\beta }_{c,k}$ of circular signals are obtained with the direction-of-arrival matrix ${\mathbf{R}}_{yx,2}{\mathbf{R}}_{xx,2}^{+}$ using the same direction-of-arrival matrix (DOAM) method.

Until now, the TSDOAM method is summarized as follows.

Step 1:

Calculate ${\mathbf{R}}_{xx}$, ${\mathbf{R}}_{yx}$, ${\mathbf{R}}_{xx}^{\prime }$, and ${\mathbf{R}}_{yx}^{\prime }$ from Equations (3)–(10);

Step 2:

Execute the EVD of ${\mathbf{R}}_{xx}^{\prime }$ to get its pseudo-inverse matrix ${\mathbf{R}}_{xx}^{\prime +}$;

Step 3:

Perform the EVD of ${\mathbf{R}}_{yx}^{\prime }{\mathbf{R}}_{xx}^{\prime +}$ with Equation (18);

Step 4:

Attain ${\theta }_{n,k}$ and ${\beta }_{n,k}$ using Equations (20) and (21);

Step 5:

Construct the matrix ${\mathbf{R}}_n^{\prime }$ with the estimate ${\mathbf{R}}_{xx}^{\prime }$ and ${\mathbf{A}}_n$;

Step 6:

Construct the direction-of-arrival matrix ${\mathbf{R}}_{yx,2}{\mathbf{R}}_{xx,2}^{+}$ with Equations (27) and (28);

Step 7:

Repeat Step 2 to Step 4 for the ${\theta }_{c,k}$ and ${\beta }_{c,k}$.

3.2. The TSRARE Method

In this section, in order to make full use of array elements of two ULAs, another novel method called the TSRARE method—which is based on the direction-of-arrival circularity difference and the rank reduction (RARE) method—is proposed in the two-stage estimation procedure.

By concatenating the observed data vectors $\mathbf{X}\left(t\right)$ and $\mathbf{Y}\left(t\right)$, we get:

$$\begin{array}{cc}\hfill \mathbf{Z}\left(t\right)& =\left[\begin{array}{c}\mathbf{X}\left(t\right)\\ \mathbf{Y}\left(t\right)\end{array}\right]=\left[\begin{array}{c}\mathbf{A}\\ \mathbf{A}\mathbf{B}\end{array}\right]\mathbf{S}\left(t\right)+\left[\begin{array}{c}{\mathbf{N}}_x\left(t\right)\\ {\mathbf{N}}_y\left(t\right)\end{array}\right]\hfill \\ & =\mathbf{C}\mathbf{S}\left(t\right)+\mathbf{N}\left(t\right).\hfill \end{array}$$

(29)

where $\mathbf{C}=[\mathbf{c}({\theta }_1,{\beta }_1),\dots ,\mathbf{c}({\theta }_K,{\beta }_K)]$ is termed as the extended steering matrix, and:

$$\begin{array}{cc}\hfill \mathbf{c}({\theta }_k,{\beta }_k)& =\left[\begin{array}{c}\mathbf{a}\left({\theta }_k\right)\\ \mathbf{a}\left({\theta }_k\right)v\left({\beta }_k\right)\end{array}\right]\hfill \\ & =\left[\begin{array}{cc}\mathbf{a}\left({\theta }_k\right)& \mathbf{0}\\ \mathbf{0}& \mathbf{a}\left({\theta }_k\right)\end{array}\right]\left[\begin{array}{c}1\\ v\left({\beta }_k\right)\end{array}\right].\hfill \end{array}$$

(30)

As the radiating mixed signals are uncorrelated with each other, the conjugated covariance matrix $\mathbf{R}$ of $\mathbf{Z}\left(t\right)$ can be written as:

$$\mathbf{R}=E\left[\mathbf{Z}\left(t\right){\mathbf{Z}}^H\left(t\right)\right]={\mathbf{C}}_n{\mathbf{R}}_n{\mathbf{C}}_n^H+{\mathbf{C}}_c{\mathbf{R}}_c{\mathbf{C}}_c^H+{\sigma }^2\mathbf{I},$$

(31)

and the elliptic covariance matrix ${\mathbf{R}}^{\prime }$ of $\mathbf{Z}\left(t\right)$ is as follows:

$${\mathbf{R}}^{\prime }=E\left[\mathbf{Z}\left(t\right){\mathbf{Z}}^T\left(t\right)\right]={\mathbf{C}}_n{\mathbf{R}}_n^{\prime }{\mathbf{C}}_n^T$$

(32)

where ${\mathbf{C}}_n$ and ${\mathbf{C}}_c$ denote the noncircular and circular extended steering matrix, respectively.

Next, we perform singular-value decomposition (SVD) of ${\mathbf{R}}^{\prime }$ to estimate the 2D directions-of-arrival of noncircular signals as follows:

$${\mathbf{R}}^{\prime }=\left[{\mathbf{U}}_{n,1}{\mathbf{U}}_{n,2}\right]\left[\begin{array}{cc}{\mathbf{\Lambda }}_n& \mathbf{0}\\ \mathbf{0}& \mathbf{0}\end{array}\right]{\mathbf{V}}_n^H$$

(33)

where ${\mathbf{\Lambda }}_n$ denotes the $K_n\times K_n$ diagonal matrix containing $K_n$ nonzero singular values in the diagonal. It is verified that all of the columns of ${\mathbf{C}}_n$ are orthogonal to all of the columns of ${\mathbf{U}}_{n,2}$; that is,

$${\mathbf{C}}_n^H({\theta }_{n,k},{\beta }_{n,k}){\mathbf{U}}_{n,2}{\mathbf{U}}_{n,2}^H{\mathbf{C}}_n({\theta }_{n,k},{\beta }_{n,k})=0,k=1,\dots ,K_n.$$

(34)

Associated with Equation (30), Equation (34) can be rewritten as:

$${\mathbf{r}}^H\mathbf{T}\mathbf{r}=0$$

(35)

where $\mathbf{r}={\left[1v^T\left({\beta }_{n,k}\right)\right]}^T$, and:

$$\mathbf{T}\left({\theta }_{n,k}\right)=\left[\begin{array}{cc}{\mathbf{a}}^H\left({\theta }_{n,k}\right){\mathbf{U}}_{n,1}^{\prime }\mathbf{a}\left({\theta }_{n,k}\right)& {\mathbf{a}}^H\left({\theta }_{n,k}\right){\mathbf{U}}_{n,2}^{\prime }\mathbf{a}\left({\theta }_{n,k}\right)\\ {\mathbf{a}}^H\left({\theta }_{n,k}\right){\mathbf{U}}_{n,3}^{\prime }\mathbf{a}\left({\theta }_{n,k}\right)& {\mathbf{a}}^H\left({\theta }_{n,k}\right){\mathbf{U}}_{n,4}^{\prime }\mathbf{a}\left({\theta }_{n,k}\right)\end{array}\right]$$

(36)

where ${\mathbf{U}}_{n,1}^{\prime }={\mathbf{U}}_{n,21}{\mathbf{U}}_{n,21}^H$, ${\mathbf{U}}_{n,2}^{\prime }={\mathbf{U}}_{n,21}{\mathbf{U}}_{n,22}^H$, ${\mathbf{U}}_{n,3}^{\prime }={\mathbf{U}}_{n,22}{\mathbf{U}}_{n,21}^H$, and ${\mathbf{U}}_{n,4}^{\prime }={\mathbf{U}}_{n,22}{\mathbf{U}}_{n,22}^H$, ${\mathbf{U}}_{n,21}$ and ${\mathbf{U}}_{n,22}$ are obtained by dividing ${\mathbf{U}}_{n,2}$ into the same two-dimensional matrices—namely ${\mathbf{U}}_{n,2}={\left[{\mathbf{U}}_{n,21}^T{\mathbf{U}}_{n,22}^T\right]}^T$.

In order to use the RARE method, we define $\mathbf{a}\left({\theta }_{n,k}\right)$ as:

$$\mathbf{a}\left(b_{n,k}\right)={[1,b_{n,k},b_{n,k}^2,\dots ,b_{n,k}^{M-1}]}^T$$

(37)

where $b_{n,k}=e^{-j\frac{2\pi }{\lambda }{d}_{x}cos{\theta }_{n,k}}$, and the matrix $\mathbf{T}$ is a function of $b_{n,k}$. Then, the 1D directions-of-arrival ${\theta }_{n,k}$ can be obtained by finding the values of $b_{n,k}$ such that $det\left[\mathbf{T}\left(b_{n,k}\right)\right]=0$. Additionally, the polynomial of $b_{n,k}$ has the following form:

$$det\left[\mathbf{T}\left(b_{n,k}\right)\right]=m_{n,1}{m}_{n,4}-m_{n,2}{m}_{n,3}$$

(38)

where $m_{n,p}={\mathbf{a}}^T(1/b_{n,k}){\mathbf{U}}_{n,p}^{\prime }\mathbf{a}\left(b_{n,k}\right),p=1,2,3,4$. $m_{n,p}$ is the polynomial of $b_{n,k}$ whose lth coefficient is given by the sum of the elements of the lth diagonal of ${\mathbf{U}}_{n,p}^{\prime }$, where $l=-M+1,\dots ,M-1$. Collecting the coefficients of the polynomial $m_{n,p}$ as a column vector denoted as ${\mathit{\mu }}_{n,p}={[{\mu }_{np,1},\dots ,{\mu }_{np,l},\dots ,{\mu }_{np,2M-1}]}^T$, we get $m_{n,p}={\mathit{o}}_n{\mathit{\mu }}_{n,p}$, where ${\mathit{o}}_n=[b_{n,k}^{-M+1},\dots ,1,\dots ,b_{n,k}^{M-1}]$, ${\mu }_{np,l}={\displaystyle \sum _{i=max[1,M-l+1]}^{min[M,2M-l]}}{[\mathbf{U}}_{n,p}^{\prime }{]}_{i,l+i-M}$.

Thus, we have that $m_{n,1}{m}_{n,4}={\mathit{o}}_n{\mathit{\mu }}_{n,1}{\mathit{\mu }}_{n,4}^T{\mathit{o}}_n^T$ and $m_{n,2}{m}_{n,3}={\mathit{o}}_n{\mathit{\mu }}_{n,2}{\mathit{\mu }}_{n,3}^T{\mathit{o}}_n^T$, and the coefficients of the polynomials $m_{n,1}{m}_{n,4}$ and $m_{n,2}{m}_{n,3}$ equal the sum of the antidiagonal elements of the matrix ${\mathit{\mu }}_{n,1}{\mathit{\mu }}_{n,4}^T$ and ${\mathit{\mu }}_{n,2}{\mathit{\mu }}_{n,3}^T$, respectively. Let ${\mathit{\delta }}_{n,f}={[{\delta }_{nf,1},\dots ,{\delta }_{nf,l},\dots ,{\delta }_{nf,4M-3}]}^T,f=1,2$ be the column vectors of the $4m-3$ coefficients of the polynomials $m_{n,1}{m}_{n,4}$ and $m_{n,2}{m}_{n,3}$, where ${\delta }_{nf,l}={\displaystyle \sum _{i=max[1,l-2M+2]}^{min[2M-1,l]}}[{\mathsf{\Phi }}_{n,f}{]}_{i,l-i+1}$, ${\mathsf{\Phi }}_{n,1}={\mathit{\mu }}_{n,1}{\mathit{\mu }}_{n,4}^T$ and ${\mathsf{\Phi }}_{n,2}={\mathit{\mu }}_{n,2}{\mathit{\mu }}_{n,3}^T$.

Hence, Equation (38) can be expressed as:

$$det\left[\mathbf{T}\left(b_{n,k}\right)\right]={\displaystyle \sum _{l=1}^{4M-3}}({\delta }_{n1,l}-{\delta }_{n2,l})b_{n,k}^{l-(2M-1)}=0.$$

(39)

The roots of the polynomial $det\left[\mathbf{T}\left(b_{n,k}\right)\right]$ can be computed by exploiting the computationally-efficient polynomial root multiple signal classification (MUSIC) algorithm, and the 1D noncircular angles ${\theta }_{n,k}$ are obtained as:

$${\theta }_{n,k}=arccos\left[\frac{\lambda }{2\pi d_x}arg\left(b_{n,k}\right)\right].$$

(40)

Substituting the estimated ${\theta }_{n,k}$ into Equation (35), we then seek out the minima of the following function [42]:

$${\beta }_{n,k}=\underset{{\beta }_n}{\min }{\mathbf{r}}^H\mathbf{T}\mathbf{r}$$

(41)

From Equation (41), we obtain ${\beta }_{n,k}$ that are given by the eigenvector corresponding to the smallest eigenvalue associated with $\mathbf{T}\left({\theta }_{n,k}\right)$ as:

$${\beta }_{n,k}=arccos\left[\frac{\lambda }{2\pi d_y}arg[-{\mathbf{a}}^H\left({\theta }_{n,k}\right){\mathbf{U}}_{n,3}^{\prime }\mathbf{a}\left({\theta }_{n,k}\right)]\right].$$

(42)

Next, we estimate the 2D directions-of-arrival of circular signals based on the estimates ${\theta }_{n,k}$ and ${\beta }_{n,k}$. With Equations (30) and (32), ${\mathbf{R}}_{n,1}^{\prime }$—denoted as the estimated ${\mathbf{R}}_n^{\prime }$ in the TSRARE method—can be expressed as:

$${\mathbf{R}}_{n,1}^{\prime }={\mathbf{C}}_n^{+}{\mathbf{R}}^{\prime }{\left({\mathbf{C}}_n^T\right)}^{+}$$

(43)

where ${\mathbf{C}}_n^{+}={\left({\mathbf{C}}_n^H{\mathbf{C}}_n\right)}^{-1}{\mathbf{C}}_n^H$ and ${\left({\mathbf{C}}_n^T\right)}^{+}={\mathbf{C}}_n^{*}{\left({\mathbf{C}}_n^T{\mathbf{C}}_n^{*}\right)}^{-1}$ [41].

Denoting the kth diagonal element of ${\mathbf{R}}_{n,1}^{\prime }$ as ${\mathbf{R}}_{n,1}^{\prime }(k,k)$, we have:

$${\mathbf{R}}_{n,1}^{\prime }(k,k)=E\left[s_{n,k}{s}_{n,k}\right].$$

(44)

Similarly, the noncircular signals here are BPSK-modulated signals. It is easily deduced that ${\sigma }_{n,k}^2=\left|{\mathbf{R}}_{n,1}^{\prime }(k,k)\right|$. Because ${\mathbf{R}}_n$ is diagonal, it follows that ${\mathbf{R}}_{n,1}$, which is the estimate of ${\mathbf{R}}_n$ as follows:

$$\begin{array}{cc}\hfill {\mathbf{R}}_{n,1}& =\mathrm{diag}[{\sigma }_{n,1}^2,\dots ,{\sigma }_{n,K_n}^2]\hfill \\ & =\mathrm{diag}\left[\right|{\mathbf{R}}_{n,1}^{\prime }(1,1)|,\dots ,|{\mathbf{R}}_{n,1}^{\prime }(K_n,K_n)\left|\right].\hfill \end{array}$$

(45)

Then, let ${\mathbf{R}}_1={\mathbf{C}}_n{\mathbf{R}}_{n,1}{\mathbf{C}}_n^H$; we attain:

$$\mathbf{R}-{\mathbf{R}}_1={\mathbf{C}}_c{\mathbf{R}}_c{\mathbf{C}}_c^H+{\sigma }^2\mathbf{I}.$$

(46)

Perform the EVD of $\mathbf{R}-{\mathbf{R}}_1$ as follows:

$$\mathbf{R}-{\mathbf{R}}_1={\mathbf{U}}_{c,x}{\mathbf{\Lambda }}_{c,x}{\mathbf{U}}_{c,x}^H+{\mathbf{U}}_{c,z}{\mathbf{\Lambda }}_{c,z}{\mathbf{U}}_{c,z}^H$$

(47)

where ${\mathbf{U}}_{c,x}$ and ${\mathbf{U}}_{c,z}$ are called the signal and noise subspaces associated with the signal eigenvalue matrix ${\mathbf{\Lambda }}_{c,x}$ and noise eigenvalue matrix ${\mathbf{\Lambda }}_{c,z}$, respectively. Similarly, we get the following relationship:

$${\mathbf{C}}_c^H({\theta }_{c,k},{\beta }_{c,k}){\mathbf{U}}_{c,z}{\mathbf{U}}_{c,z}^H{\mathbf{C}}_c({\theta }_{c,k},{\beta }_{c,k})=0,k=1,\dots ,K_c.$$

(48)

and estimate the 2D directions-of-arrival (namely ${\theta }_{c,k}$ and ${\beta }_{c,k}$) of circular signals in the same way as the 2D directions-of-arrival of noncircular signals.

At this point, the 2D directions-of-arrival of noncircular and circular signals have been achieved by the TSRARE algorithm. The simple summary of the TSRARE algorithm is as follows.

Step 1:

Calculate $\mathbf{R}$ and ${\mathbf{R}}^{\prime }$ with the observed data $\mathbf{Z}\left(t\right)$ with Equations (31) and (32);

Step 2:

Perform the SVD of ${\mathbf{R}}^{\prime }$ to get ${\mathbf{U}}_{n,2}$ using Equation (33);

Step 3:

Calculate the roots of $det\left[\mathbf{T}\left(b_{n,k}\right)\right]$ associated with noncircular signals from Equation (34) to Equation (39);

Step 4:

Attain ${\theta }_{n,k}$ and ${\beta }_{n,k}$ using Equations (40) and (42);

Step 5:

Construct ${\mathbf{R}}_1$ with the estimate ${\mathbf{R}}_{n,1}$ and ${\mathbf{C}}_n$;

Step 6:

Perform the EVD of $\mathbf{R}-{\mathbf{R}}_1$ to get ${\mathbf{U}}_{c,z}$ using Equation (47);

Step 7:

Repeat Step 3 to Step 4 for the ${\theta }_{c,k}$ and ${\beta }_{c,k}$.

Remark 1: In practice, it can be noted that only a finite number of observed data is available. Thus, ${\mathbf{R}}_{xx}$, ${\mathbf{R}}_{yx}$, ${\mathbf{R}}_{xx}^{\prime }$, ${\mathbf{R}}_{yx}^{\prime }$, $\mathbf{R}$, and ${\mathbf{R}}^{\prime }$ must be estimated by ${\widehat{\mathbf{R}}}_{\mathit{xx}}$ = $\frac{1}{L}{\displaystyle \sum _{l=1}^L}\mathbf{X}\left(l\right){\mathbf{X}}^H\left(l\right)$, ${\widehat{\mathbf{R}}}_{\mathit{yx}}$ = $\frac{1}{L}{\displaystyle \sum _{l=1}^L}\mathbf{Y}\left(l\right){\mathbf{X}}^H\left(l\right)$, ${\widehat{\mathbf{R}}}_{\mathit{xx}}^{\prime }$ = $\frac{1}{L}{\displaystyle \sum _{l=1}^L}\mathbf{X}\left(l\right){\mathbf{X}}^T\left(l\right)$, ${\widehat{\mathbf{R}}}_{\mathit{yx}}^{\prime }$ = $\frac{1}{L}{\displaystyle \sum _{l=1}^L}\mathbf{Y}\left(l\right){\mathbf{X}}^T\left(l\right)$, $\widehat{\mathbf{R}}$ = $\frac{1}{L}{\displaystyle \sum _{l=1}^L}\mathbf{Z}\left(l\right){\mathbf{Z}}^H\left(l\right)$, and ${\widehat{\mathbf{R}}}^{\prime }$ = $\frac{1}{L}{\displaystyle \sum _{l=1}^L}\mathbf{Z}\left(l\right){\mathbf{Z}}^T\left(l\right)$. When less observed data are available, such that the estimated covariance matrices are no longer strictly diagonal matrices, this leads to the estimation performance reduction in the small 2D angle separation condition.

Remark 2: As we know, one of the awkward problems about 2D direction-of-arrival estimation is the pair situation, which causes severe estimation error without exact pair process. However, the two proposed methods can pair the 2D directions-of-arrival of mixed signals automatically. This is because, in the two-stage estimation procedures, the TSDOAM method pairs the 2D directions-of-arrival by performing one EVD whose eigenvalues and eigenvectors are a one-to-one correspondence relationship, while the TSRARE method pairs them by decoupling the 2D directions-of-arrival into two successive 1D processes.

4. Location Discussion and Analysis

The maximum number of sources to estimate is analyzed in this section by the two proposed methods. Since the two proposed methods estimate the mixed signals separately, and the number of sources can be resolved is related to the dimensions of the elliptic auto-covariance matrix ${\mathbf{R}}_{xx}^{\prime }$ (see Equation (8)) and the conjugated covariance matrix $\mathbf{R}$ (see Equation (31)), it follows that $M>max\left\{K_n,K_c\right\}$ with the TSDOAM method and $2M>max\left\{K_n,K_c\right\}$ with the TSRARE method must be satisfied, respectively, so as to resolve all noncircular and circular signals, while Yin’s [15] and Xia’s [16] method estimate them simultaneously, so $M>(K_n+K_c)$ and $2M>(K_n+K_c)$ should be satisfied, respectively. In other words, the maximum detectable signals of the TSDOAM method that are detected by two ULAs with $2M$ elements are $(M-1)$ noncircular signals plus $(M-1)$ circular signals, namely $(2M-2)$, and the TSRARE method can detect $(2M-2)$ noncircular signals plus $(2M-2)$ circular signals, namely $(4M-4)$; while Yin’s and Xia’s method can distinguish $(M-1)$ and $(2M-2)$ mixed signals, respectively. Therefore, the TSDOAM and TSRARE methods can detect twice the mixed signals as Yin’s and Xia’s method, respectively. In addition, the TSRARE method can identify twice the signals as the TSDOAM method.

5. Simulation Results

In this section, some simulation results are presented to show the performance of the TSDOAM and TSRARE methods, compared with the existing methods, which include Yin’s method and Xia’s method. Assume two ULAs with each array consisting of omnidirectional sensors spaced by a half wavelength of the mixed noncircular and circular signals; the distance between the two ULAs is spaced by a half wavelength as well. Additionally, the noncircular signals employ BPSK-modulated sources, while the circular signals are QPSK-modulated sources for the simulation.

5.1. 2D Direction-of-Arrival Estimation Performance

In this part, the maximum detectable 2D directions-of-arrival are investigated. Four BPSK signals with ${\theta }_{n,k}$ and ${\beta }_{n,k}$ impinge from $\{{70}^{\circ },{60}^{\circ },{85}^{\circ },{100}^{\circ }\}$ and $\{{65}^{\circ },{80}^{\circ },{85}^{\circ },{65}^{\circ }\}$, separately, and four QPSK signals are emitted from the same 2D directions-of-arrival. The number M of isotropic sensors of each array is three. Figure 2 plots the paired results of eight radiating signals from 50 Monte Carlo trials with the signal-to-noise ratio (SNR) set at 30 dB and snapshots L = 2000, which show that the 2D directions-of-arrival of eight ($4M-4$ = 8) signals are paired correctly with the TSRARE method; even a common $\beta$ is shared in both noncircular and circular signals. However, the TSDOAM method and the methods in [15,16] can detect up to four ($2M-2$ = 4) signals, four ($2M-2$ = 4) signals, and two ($M-1$ = 2) signals, respectively, which fail to distinguish the mixed eight signals due to the limited array elements.

5.2. The Effect of SNR

In this subsection, we compare the 2D direction-of-arrival performance of the TSDOAM and TSRARE methods versus SNR with existing methods in [15,16]. Furthermore, the average root mean square error (ARMSE) is defined for precision evaluation as:

$$\mathrm{ARMSE}=\sqrt{{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{q=1}^{Mc}}\left[\right({\widehat{\zeta }}_{qk}-{\zeta }_k{)}^2]}$$

(49)

where ${\zeta }_k$ stands for ${\theta }_k$ or ${\beta }_k$, and ${\widehat{\zeta }}_{qk}$ is the parameter to be estimated for ${\widehat{\theta }}_k$ or ${\widehat{\beta }}_k$, while $M_c$ denotes the number of Monte Carlo runs.

Two BPSK signals together with two QPSK signals incoming from $({75}^{\circ },{50}^{\circ })$, $({100}^{\circ },{65}^{\circ })$ and $({75}^{\circ },{50}^{\circ })$, $({100}^{\circ },{65}^{\circ })$, respectively, impinge onto the ULAs with each array having five omnidirectional sensors. The snapshots of this test are set to 500, and the variable SNR of the four incident signals varies from −10 dB to 30 dB. The ARMSE of the four methods derived from the 2000 trials are given in Figure 3. It can be seen from Figure 3 that Yin’s method and Xia’s method fail to work; however, the two proposed methods perform well with increasing SNR. This is because the two proposed methods estimate the 2D directions-of-arrival of noncircular and circular signals separately based on the circularity difference between noncircular and circular signals rather than the direction-of-arrival difference utilized in Yin’s and Xia’s methods. Moreover, the TSRARE method has a lower ARMSE than the TSDOAM method, which results from the fact that the array aperture is fully utilized with the TSRARE method during the two-stage estimation for the BPSK and QPSK signals.

5.3. The Effect of Snapshots

The 2D direction-of-arrival performance of the TSDOAM and TSRARE methods versus snapshots with Yin’s and Xia’s methods is verified in this part. The simulation parameters are the same as Experiment 2, except the SNR is fixed at 5 dB, and the number of the collected snapshots varies from 10 to 490. At each snapshot setting, 2000 independent runs are executed for each method to obtain their 2D direction-of-arrival estimation in the statistical sense, whose results are given in Figure 4. Similar conclusions and reasons can be drawn from Figure 4 that as snapshots are increasing, the curves of the two proposed methods work well, while those of Yin’s and Xia’s methods remain unchanged. In addition, the TSRARE method has better estimation performance than the TSDOAM method in both 2D directions-of-arrival.

5.4. The Effect of Angle Separation

In this part, we testify to the 2D direction-of-arrival estimation performance of the two proposed methods coupled with Yin’s and Xia’s methods versus angular separation. The number of each array of the two ULAs is also five. Additionally, four signals consist of two BPSK signals and two QPSK signals with ${\theta }_k$ and ${\beta }_k$ incoming from $\{{70}^{\circ },{100}^{\circ },{(100+\mathsf{\Delta })}^{\circ },{(70+\mathsf{\Delta })}^{\circ }\}$ and $\{{50}^{\circ },{85}^{\circ },{(85+\mathsf{\Delta })}^{\circ },{(50+\mathsf{\Delta })}^{\circ }\}$, separately, and $\mathsf{\Delta }$ is varied from 0 to 12. In addition, the SNR is fixed at 15 dB, and the snapshots L = 300.

The ARMSE versus angular separation is shown in Figure 5 with 2000 Monte Carlo trials used. From Figure 5, it can be seen that the two proposed methods outperform Yin’s and Xia’s methods in both 2D small angle separations; however, with the angle separation increasing, the 2D direction-of-arrival estimation performance achieved by the two proposed methods is inferior to that by Yin’s and Xia’s methods. This is because in small angle separation,

the 2D direction-of-arrival information of circular signals included in the elliptic covariance matrix can be equivalent to that of noncircular signals, due to the fact that the 2D direction-of-arrival of the circular signal is close to that of the noncircular signal; thus, the estimation precision of the noncircular signals can be improved. A similar reason is suitable for the improved 2D direction-of-arrival estimation accuracy of circular signals; while Yin’s and Xia’s methods estimate the 2D direction-of-arrival of noncircular and circular signals simultaneously, which are based on the direction-of-arrival difference of noncircular and circular signals that inevitably gives rise to the degradation of the performance in small 2D direction-of-arrival separation [35]. As for large 2D direction-of-arrival separation, the estimation performance of the two proposed methods behaved with lower accuracy than Yin’s and Xia’s methods. The reason is that in the condition of a few snapshots, the incident mixed signals’ covariance matrix and elliptic covariance matrix are not strictly diagonal matrices, which results in making it difficult to separate the mixed signals from the two-stage estimation procedures. Furthermore, the TSRARE method performs better than the TSDOAM method all the way, with the full usage of array elements in the two-stage estimation procedures.

6. Conclusions

Two novel 2D direction-of-arrival estimation methods, named, respectively, as the TSDOAM and TSRARE methods for mixed noncircular and circular signals’ estimation with two parallel ULAs are presented in this paper. The direction-of-arrival circularity difference rather than the direction-of-arrival difference between the noncircular and circular signals for the 2D directions-of-arrival’ estimation is utilized in the two proposed methods. The explicit derivation of the two proposed methods is described, and the maximum number of incident signals of the two proposed methods is analyzed, which shows that the detected number of signals is more than that of the array elements compared to the conventional methods. Simulation results demonstrate the usefulness of the two proposed methods.