A Correntropy-Based Proportionate Affine Projection Algorithm for Estimating Sparse Channels with Impulsive Noise

Abstract

A novel robust proportionate affine projection (AP) algorithm is devised for estimating sparse channels, which often occur in network echo and wireless communication channels. The newly proposed algorithm is realized by using the maximum correntropy criterion (MCC) and the data reusing scheme used in AP to overcome the identification performance degradation of the traditional PAP algorithm in impulsive noise environments. The proposed algorithm is referred to as the proportionate affine projection maximum correntropy criterion (PAPMCC) algorithm, which is derived in the context of channel estimation framework. Many simulation results were obtained to verify that the PAPMCC algorithm is superior to early reported AP algorithms with different input signals under impulsive noise environments.

1. Introduction

A class of adaptive filtering (AF) algorithms are extensively considered in use in channel estimation (CE), echo cancellation, noise elimination, etc. [1,2,3,4,5,6,7,8,9,10]. For example, the well-known least mean square (LMS), normalized LMS (NLMS) and recursive least square (RLS) algorithms were used in various systems. Although the LMS algorithm has a simple principle and a small amount of computation in practice, it might converge slowly in low signal-to-noise ratio (SNR) scenes. In contrast, the RLS converges faster than the basic LMS. However, it is proved to have high cost of increased computational complexity, which will use more computing resources when the order of the AF is large. In addition, if the input signal is a speech signal, the convergence speed for the basic LMS algorithm becomes very slow as the eigenvalue distribution range for the input signal autocorrelation matrix is large [11]. To enhance the identification behaviors of the RLS and NLMS algorithms in practical engineering and to obtain high accuracy and fast convergence, the affine projection (AP) algorithm is proposed by reusing latest input signals to improve the NLMS’s performance [12]. The computation burden of the AP is between the LMS and RLS algorithms, and the AP algorithm has a fast convergence, especially for colored or speech signal input signals [13].

In many engineerings, such as speech signal processing and real-time traffic predictions, noise often exhibits strongly impulsive characteristics [14,15]. Traditional NLMS and AP algorithms, which use the minimum-mean-square-error (MMSE) criterion to construct an expected cost function, will suffer from performance degradation in those impulsive noise environments. To find the solution for handling these problems, the maximum correntropy criterion (MCC) and the minimum error entropy criterion (MEEC) have been proposed to give resistance to the impulsive noise [16,17]. Although the MEEC is a robust criterion, its computational complexity is very high, while the MCC algorithm whose computational complexity is comparable to the LMS has been widely used to resist the impulsive noise [18,19,20,21].

On the other hand, scholars found that the sparse characteristics are existing in a great number of scenarios such as network echo channels and underwater acoustic communication channels [22,23,24,25]. However, classical LMS, AP and MCC algorithms cannot take advantage of the sparse structures of these sparse channels. Then, the proportionate AF algorithms have been proposed to make use of the sparse information in the mentioned channels [26]. For example, the proportionate NLMS (PNLMS) combines the proportionate scheme into the NLMS to reassign the gains to each channel coefficients [26]. Then, proportionate-type AF algorithms were widely realized and utilized for channel estimation as well as the echo cancellation [27,28,29,30]. For the sake of comparison with the traditional NLMS, the PNLMS suffers from slow convergence if the input signal is driven by colored or speech signals, resulting in that steady-state error might be worse than that of the NLMS. Inspired by the PNLMS, the proportionate AP (PAP) algorithm has been proposed by using the idea in PNLMS to fully use the sparse structure-information of the echo channels [31] based on the data reusing principle. Then, various proportionate-type AP algorithms have been proposed [32,33,34,35,36,37]. However, the PAP-type algorithms have performance degradation in impulsive noise environments because of the MMSE scheme. Thus, the sign algorithms, such as affine projection sign (APS) algorithm and proportionate APS (PAPS) algorithm [38,39], are successfully used for dealing with impulsive noise. Additionally, another kind of sparse-aware APs have been reported and analyzed by taking the consideration of the compressed sensing (CS) theory [40]. With the help of the concept of the CS, a series of sparsity-aware AF algorithms, such as zero-attracting LMS (ZA-LMS), reweighted ZA-LMS (RZA-LMS), ZA-AP, and RZA-AP algorithms have been proposed within the AF [41,42,43,44,45,46,47].

In this paper, the AP scheme and MCC are considered together to construct a new cost function to enhance the PAP algorithm in impulsive noise environments, which is denoted as proportionate affine projection maximum correntropy criterion (PAPMCC) algorithm. The proposed PAPMCC algorithm is investigated by using $\alpha$-stable distribution as the impulsive noise model. Experimental results verify that the PAPMCC provides a lower steady state error than AP, ZA-AP, RZA-AP, and PAP algorithms with different inputs.

2. Review of the PAP Algorithm

In the range of AF, the implementation schematic diagram for CE is presented in Figure 1. Assume that the input signal $\mathbf{x}\left(m\right)=\left[x\right(m),x(m-1),\cdots ,$$x(m-K+1){]}^T$ is used in this paper, and the channel impulse response (CIR) is modeled as $\mathbf{w}\left(m\right)={\left[w_0\left(m\right),\cdots ,w_{\mathrm{K}-1}\left(m\right)\right]}^T$, where K denotes as the total length and m denotes the time slot. Then, the received signal $d\left(m\right)$ is

$$d\left(m\right)={\mathbf{x}}^T\left(m\right)\mathbf{w}\left(m\right)+r\left(m\right),$$

(1)

in which $r\left(m\right)$ represents the additive impulsive noise that is usually independent of $\mathbf{x}\left(m\right)$, and ${}^T$ represents the transposed operation. The gotten CIR is given as $\widehat{\mathbf{w}}\left(m\right)={\left[{\widehat{w}}_0\left(m\right),{\widehat{w}}_1\left(m\right),\cdots ,{\widehat{w}}_{K-1}\left(m\right)\right]}^T$, resulting in

$$y\left(m\right)={\mathbf{x}}^T\left(m\right)\widehat{\mathbf{w}}\left(m\right).$$

(2)

The estimation error at m is expressed as

$$e\left(m\right)=d\left(m\right)-y\left(m\right).$$

(3)

2.1. AP Algorithm

To the best of our knowledge, the AP algorithm reuses the current and previous input signal information, which achieves faster convergence compared with the NLMS when the input signal is colored. The input matrix for the AP algorithm is

$$\mathbf{X}\left(m\right)=\left[\mathbf{x}\right(m),\mathbf{x}(m-1),\cdots ,\mathbf{x}(m-M+1\left)\right].$$

(4)

where M is a projection order. Due to the reuse of data, $\mathbf{y}\left(m\right)$ and the estimated error $\mathbf{e}\left(m\right)$ are expressed as

$$\mathbf{y}\left(m\right)={\mathbf{X}}^T\left(m\right)\widehat{\mathbf{w}}\left(m\right),$$

(5)

$$\mathbf{d}\left(m\right)={[d\left(m\right),d(m-1),\cdots ,d(m-M+1)]}^T,$$

(6)

$$\begin{array}{c}\mathbf{e}\left(m\right)=\mathbf{d}\left(m\right)-\mathbf{y}\left(m\right).\hfill \end{array}$$

(7)

The iteration equation for the standard AP algorithm is given by

$$\widehat{\mathbf{w}}(m+1)=\widehat{\mathbf{w}}\left(m\right)+{\mu }_{\mathrm{AP}}\mathbf{X}\left(m\right){\left({\mathbf{X}}^T\left(m\right)\mathbf{X}\left(m\right)+{\delta }_{\mathrm{AP}}{\mathbf{I}}_M\right)}^{-1}\mathbf{e}\left(m\right),$$

(8)

in which ${\mu }_{\mathrm{AP}}$ denotes the step size, ${\delta }_{\mathrm{AP}}>0$ is to prevent the matrix to be inverted to singular, and ${\mathbf{I}}_M$ is a M-order identity matrix.

2.2. PAP Algorithm

From the inspiration of the well-known PNLMS algorithm, the PAP algorithm integrates the proportionate idea into the AP algorithm to modify the gain allocation method, and realizes a dynamic step size (STS) based on the magnitudes of the channel coefficients that are included in the unknown channels. The iteration equation of the PAP is modified to be

$$\widehat{\mathbf{w}}(m+1)=\widehat{\mathbf{w}}\left(m\right)+{\mu }_{\mathrm{PAP}}\mathbf{G}\left(m\right)\mathbf{X}\left(m\right){\left({\mathbf{X}}^T\left(m\right)\mathbf{G}\left(m\right)\mathbf{X}\left(m\right)+{\delta }_{\mathrm{PAP}}{\mathbf{I}}_M\right)}^{-1}\mathbf{e}\left(m\right),$$

(9)

where ${\mu }_{\mathrm{PAP}}$ is still used as a STS, ${\delta }_{\mathrm{PAP}}$ denotes the regularization factor in the PAP, and $\mathbf{G}\left(m\right)$ acts as the gain controlling matrix, which is written as

$$\begin{array}{c}\mathbf{G}\left(m\right)=\mathrm{diag}\left\{\left.g_0\left(m\right),g_1\left(m\right),\dots ,g_{K-1}\left(m\right)\right\}\right.,\hfill \end{array}$$

(10)

where

$$g_k\left(m\right)=\frac{{\phi }_k\left(m\right)}{{\displaystyle \sum _{i=0}^{K-1}}{\phi }_i\left(m\right)},$$

(11)

and

$$\begin{array}{c}{\phi }_k=max\left\{pmax\left\{\left.q,\left|{\widehat{w}}_0\right|,\left|{\widehat{w}}_k\right|,\cdots ,\left|{\widehat{w}}_{K-1}\right|\right\}\right.\right.,\left.\left|{\widehat{w}}_k\right|\right\}.\hfill \end{array}$$

(12)

In Equation (12), parameters $p>0$ and $q>0$ are used to prevent the update process from stalling. In practice, $p=\frac{5}{K}$ is usually chosen [26].

3. Proposed PAPMCC Algorithm

The PAP can provide amazing convergence performance in Gaussian noise environments, but the performance will degrade under the impulsive noise environments. To take full use of the sparse characteristics of the CIRs, a robust PAP algorithm is realized by combining the proportionate idea with the basic AP and MCC together to construct the PAPMCC algorithm. As a result, the proposed PAPMCC algorithm solves the minimization problem given by

$${∥\widehat{\mathbf{w}}(m+1)-\widehat{\mathbf{w}}\left(m\right)∥}_{{\mathbf{G}}^{-1}\left(m\right)}^2\mathrm{subject}\mathrm{to}\stackrel{⌣}{\mathbf{e}}\left(m\right)=\left[{\mathbf{1}}_M-\xi exp\left(-\frac{\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)}{2{\sigma }^2}\right)\right]\odot \mathbf{e}\left(m\right),$$

(13)

where $\stackrel{⌣}{\mathbf{e}}\left(m\right)=\mathbf{d}\left(m\right)-{\mathbf{X}}^T\left(m\right)\widehat{\mathbf{w}}(m+1)$, σ denotes the kernel width, and 1_M is a column vector whose elements are ones. $\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)$ denotes the Hadamard product between two estimated error vectors e(m). According to the Lagrange multiplier method (LMM) with multiple constraints, the cost function is presented as

$$J\left(\widehat{\mathbf{w}}(m+1)\right)={∥\widehat{\mathbf{w}}(m+1)-\widehat{\mathbf{w}}\left(m\right)∥}_{{\mathbf{G}}^{-1}\left(m\right)}^2+\mathit{\lambda }\left\{\stackrel{⌣}{\mathbf{e}}\left(m\right)-\left[{\mathbf{1}}_M-\xi exp\left(-\frac{\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)}{2{\sigma }^2}\right)\right]\odot \mathbf{e}\left(m\right)\right\},$$

(14)

where λ = [λ₁, λ₂, …, λ_M]. Then, let

$$\frac{\partial J\left(\widehat{\mathbf{w}}(m+1)\right)}{\partial \widehat{\mathbf{w}}(m+1)}=\mathbf{0}\mathrm{and}\frac{\partial J\left(\widehat{\mathbf{w}}(m+1)\right)}{\partial \mathit{\lambda }}=\mathbf{0}.$$

(15)

After performing algebraic operations, we get

$$\widehat{\mathbf{w}}(m+1)=\widehat{\mathbf{w}}\left(m\right)+\frac{1}{2}\mathbf{G}\left(m\right)\mathbf{X}\left(m\right){\mathit{\lambda }}^T,$$

(16)

and

$$\mathbf{d}\left(m\right)={\mathbf{X}}^T\left(m\right)\widehat{\mathbf{w}}(m+1)+\left[{\mathbf{1}}_M-\xi exp\left(-\frac{\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)}{2{\sigma }^2}\right)\right]\odot \mathbf{e}\left(m\right).$$

(17)

Solving Equations (16) and (17), the Lagrange multiplier vector is given by

$${\mathit{\lambda }}^T=2\xi {\left[{\mathbf{X}}^T\left(m\right)\mathbf{G}\left(m\right)\mathbf{X}\left(m\right)\right]}^{-1}exp\left(-\frac{\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)}{2{\sigma }^2}\right)\odot \mathbf{e}\left(m\right).$$

(18)

Substituting Equation (18) into Equation (16), the iteration of the PAPMCC is expressed as

$$\widehat{\mathbf{w}}(m+1)=\widehat{\mathbf{w}}\left(m\right)+\xi \mathbf{G}\left(m\right)\mathbf{X}\left(m\right){\left[{\mathbf{X}}^T\left(m\right)\mathbf{G}\left(m\right)\mathbf{X}\left(m\right)\right]}^{-1}exp\left(-\frac{\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)}{2{\sigma }^2}\right)\odot \mathbf{e}\left(m\right).$$

(19)

In practice, Equation (19) can be corrected to

$$\widehat{\mathbf{w}}(m+1)=\widehat{\mathbf{w}}\left(m\right)+\mu \mathbf{G}\left(m\right)\mathbf{X}\left(m\right){[{\mathbf{X}}^T\left(m\right)\mathbf{G}\left(m\right)\mathbf{X}\left(m\right)+{\mathbf{I}}_M{\delta }_{\mathrm{PAPMCC}}]}^{-1}exp\left(-\frac{\mathbf{e}\left(m\right)\odot \mathbf{e}\left(m\right)}{2{\sigma }^2}\right)\odot \mathbf{e}\left(m\right),$$

(20)

where $\mu =\xi$ acts as the step size, ${\delta }_{\mathrm{PAPMCC}}$ denotes the regularization factor, and $\mathbf{G}\left(m\right)$ is the weight assignment matrix that is defined in Equations (10)–(12).

The computation complexity of the devised PAPMCC algorithm is compared with the AP, ZA-AP, RZA-AP and PAP algorithms with respect to the total number of additions, multiplications, and divisions in each iteration. The comparison is presented in

Table 1. Computational complexity in each iteration.

Algorithm	Addition	Multiplication	Division
AP	$(2M^2+M)K$	$(2M^2+3M)K+M^2$	0
ZA-AP	$(2M^2+M+1)K$	$(2M^2+3M+1)K+M^2$	0
RZA-AP	$(2M^2+M+2)K$	$(2M^2+3M+2)K+M^2$	K
PAP	$2M{K}^2+(2M^2-M+1)K-1$	$2M{K}^2+(2M^2+3M+1)K+M^2$	K
PAPMCC	$2M{K}^2+(2M^2-M+1)K-1$	$2M{K}^2+(2M^2+3M+1)K+M^2+2M$	$K+M$

. It is clear to see that the computational complexity of the proposed PAPMCC algorithm is comparable to that of the PAP algorithm.

4. Experimental Results

Several experiments were constructed to give an analysis on the performance of the PAPMCC algorithm for implementing the sparse CE. Since the $\alpha$-stable distribution can well construct the non-Gaussian phenomenon, which is ubiquitous in practice, it was chosen to model the impulsive noise in the simulations. The $\alpha$-stable distribution function is defined as

$$f\left(t\right)=exp\left\{j\chi t-\gamma {\left|t\right|}^{\alpha }\left[1+j\beta \mathrm{sgn}\left(t\right)S(t,\alpha )\right]\right\},$$

(21)

where

$$S(t,\alpha )=\left\{\begin{array}{c}tan\frac{\alpha \pi }{2}\mathrm{if}\alpha \ne 1\\ \frac{2}{\pi }log\left|t\right|\mathrm{if}\alpha =1,\end{array}\right.$$

(22)

in which $\alpha \in (0,2]$ represents the characteristic index, which controls the behavior of the impulsive distribution. When parameter $\alpha$ is smaller, the impulsive intensity becomes larger. $\beta \in [-1,1]$ is the symmetric parameter, $\chi$ denotes positional parameter, and $\gamma >0$ represents the dispersion parameter. Furthermore, the $\alpha$-stable distribution is given by $V_{\alpha -\mathrm{stable}}(\alpha ,\beta ,\gamma ,\chi )$. Herein, $V_{\alpha -\mathrm{stable}}(1.5,0,0.2,0)$ is chosen to implement the impulsive noise. In all simulation experiments, $K=1024$, and $\sigma =1$ were selected, and the input signal power was 1. The network echo channel used for the experiments, which is classical sparse channel presented in Figure 2, whose active coefficients distributed in $[257,272]$, was considered to evaluate the proposed PAPMCC algorithm. The related parameters were set to be ${\delta }_{\mathrm{AP}}={\delta }_{\mathrm{ZA}-\mathrm{AP}}={\delta }_{\mathrm{RZA}-\mathrm{AP}}=0.01$ and ${\delta }_{\mathrm{PAP}}={\delta }_{\mathrm{PAPMCC}}=\frac{1}{K}{\delta }_{\mathrm{AP}}$ [48]. The performance for all used algorithms was evaluated by normalized misalignment (NM), which is written as $10{log}_{10}({∥\mathbf{w}-\widehat{\mathbf{w}}∥}_2^2/{∥\mathbf{w}∥}_2^2)$.

4.1. Performance of the PAPMCC Algorithm with Various Projection Orders M, Step-Sizes $\mu$ and Kernel Width $\sigma$

Firstly, the effects of the projection order M on the convergence for the PAPMCC algorithm was investigated. The colored noise, which was obtained from white Gaussian noise (WGN) filtering through an autoregressive with a pole at 0.8, was used as the input signal. Herein, $\mu =0.05$. The results given in Figure 3 point out that increasing the projection order M could speed up the convergence, while the steady-state misalignment was increased. Therefore, a trade-off between the convergence speed and steady-state misalignment should be taken into consideration.

Secondly, the effects of kernel width $\sigma$ on the convergence for the PAPMCC algorithm was analyzed and discussed. Herein, $M=4$ was selected. From Equation (20), the parameter $\sigma$ affects the estimation behaviors of the PAPMCC algorithm, while $\sigma$ is an important parameter for Gaussian kernel to suppress noise interference. Given the diversity and complexity of the target signal and noise, it is not easy to get the optimal solution of the kernel width $\sigma$ from the theoretical derivation. Therefore, the simulation experiments were used to determine the appropriate value $\sigma$. The results given in Figure 4 point out that the steady-state error of the PAPMCC algorithm increased with the increment of $\sigma$. The PAPMCC algorithm had high estimation error when $\sigma$ took a larger value since MCC behaved similar LMS when the value of $\sigma$ was very large.

Thirdly, the effects of $\mu$ on the convergence for the PAPMCC algorithm was investigated using colored noise as the input signal. From the above simulation results, $\sigma =1.0$ was selected, and other parameters were the same as the first experiments, and the results are presented in Figure 5. Parameter $\mu$ controls the convergence speed of the PAPMCC. With the increment of $\mu$, the normalized misalignment was decreased, while the convergence rate became fast. Consequently, the parameters $\mu$ and $\sigma$ are supposed to be reasonably selected in practical application.

4.2. Performance Comparisons of the Proposed PAPMCC Algorithm under Different Input Signals

According to the analysis presented above, we found that the devised PAPMCC algorithm had a lower steady-state MSE when $\sigma$ and $\mu$ were selected. Herein, the estimation behaviors of the PAPMCC was compared with the AP, RZA-AP,ZA-AP, and PAP algorithms. All algorithms were investigated by using WGN, colored noise, and speech signal as input signals, and the sampling frequency for the speech signal was 8 kHz. The used speech signal in this simulation is presented in Figure 6. The performance comparisons of the PAPMCC algorithm with various inputs for network echo channel are presented in Figure 7, Figure 8 and Figure 9, respectively. The PAPMCC algorithm achieved the lowest NM for the sake of comparison with the ZA-AP, AP and RZA-AP algorithms. The PAPMCC algorithm had lower steady-state error while its convergence speed was similar to that of the PAP algorithm. When the input signal was speech signal, the proposed PAPMCC was still better than the related algorithms by considering the convergence and estimation error.

4.3. SNR vs. Normalized Misalignment (NM) of the PAPMCC Algorithm

NM versus SNR was used to analyze the performance of the devised PAPMCC under colored input for estimating network echo channel. The performance results of the PAPMCC with various SNRs are presented in Figure 10, which shows that the estimation error decreased as the SNR increased from 0 to 20 dB. Clearly, the steady-state performance of the PAPMCC was significantly better than the related algorithms in low SNR environments.

4.4. Performance Comparisons of the Proposed PAPMCC Algorithm with the Conventional Robust AP Algorithms

Two conventional robust algorithms were taken into account for comparison to deal with impulsive noise, namely, APS and PAPS algorithms. Herein, the step sizes of the APS and PAPS algorithms were set to 0.005, while the step sizes for the PAP and improved PAP (IPAP) algorithm [49] were set to 0.5, and the bound of set-membership PAP (SM-PAP) algorithm [34] was set to $\sqrt{2{\sigma }_r^2}$ where ${\sigma }_r^2$ represents the power of the noise. The other parameters were consistent with the previous simulations. The results presented in Figure 11 indicate that the proposed PAPMCC algorithm could still achieve the lowest steady-state error.

5. Conclusions

In this paper, the proportionate affine projection maximum correntropy criterion (PAPMCC) has been put forward by the combination of the proportionate and affine projection schemes with the MCC to get a new cost function from the concept of the PAP. The proposed PAPMCC algorithm is carefully derived and investigated via the simulation from various experiments. The results indicate that the PAPMCC algorithm clearly improves the ability of the traditional PAP algorithm under impulsive noise environments. Moreover, compared with the AP, PAP, ZA-AP, RZA-AP, IPAP, APS, PAPS and SM-AP algorithms, the proposed PAPMCC algorithm achieves the lowest NM under three different input signals for estimating network echo channels.