The Residual ISI for Which the Convolutional Noise Probability Density Function Associated with the Blind Adaptive Deconvolution Problem Turns Approximately Gaussian

Abstract

In a blind adaptive deconvolution problem, the convolutional noise observed at the output of the deconvolution process, in addition to the required source signal, is—according to the literature—assumed to be a Gaussian process when the deconvolution process (the blind adaptive equalizer) is deep in its convergence state. Namely, when the convolutional noise sequence or, equivalently, the residual inter-symbol interference (ISI) is considered small. Up to now, no closed-form approximated expression is given for the residual ISI, where the Gaussian model can be used to describe the convolutional noise probability density function (pdf). In this paper, we use the Maximum Entropy density technique, Lagrange’s Integral method, and quasi-moment truncation technique to obtain an approximated closed-form equation for the residual ISI where the Gaussian model can be used to approximately describe the convolutional noise pdf. We will show, based on this approximated closed-form equation for the residual ISI, that the Gaussian model can be used to approximately describe the convolutional noise pdf just before the equalizer has converged, even at a residual ISI level where the “eye diagram” is still very closed, namely, where the residual ISI can not be considered as small.

1. Introduction

The convolutional noise brought on by a blind adaptive deconvolution or blind adaptive equalization system is the subject of this research. In a blind adaptive deconvolution (blind adaptive equalization) system, all that is available is the output sequence of an unidentified linear system (channel), and the goal is to recover the input sequence of that system without the aid of a training sequence [1,2,3,4,5,6,7]. Numerous fields, including seismology, underwater acoustics, image restoration, and digital communication, use blind adaptive deconvolution systems [7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39]. For a moment, let us think about the scenario of digital communication, where a source signal gets convolutedly distorted during transmission between its symbols and the channel impulse response. The recovery process is significantly hampered by this distortion, known as ISI, which produces detrimental distortions [39]. An adaptive blind equalizer is used to solve the ISI problem. The coefficients of the ideal blind adaptive equalizer are unknown because the channel coefficients are unknown. In a blind adaptive deconvolution system, the equalizer’s coefficients are only approximated values to the optimal ones, resulting in the addition of an error signal to the source signal at the deconvolution process’s output. This error signal is defined as the convolutional noise and is closely related to the residual ISI. If the system’s residual ISI level is relatively low after the blind adaptive equalization, it means that the convolutional noise is considered very small. Until recently, the Gaussian pdf was frequently used [1,2,6,34,40,41,42,43] to approximate the convolutional noise pdf throughout the iterative deconvolution process. However, according to [41], the convolutional noise pdf tends approximately to a Gaussian pdf only at the end of the iterative deconvolution process when the equalizer has converged to a relatively low residual ISI (where the convolutional noise is relatively low). The input sequence and the convolutional noise sequence are significantly associated in the early stages of the iterative deconvolution process because the ISI is often high, and the convolutional noise pdf is more uniform than Gaussian [41,44]. It should be noted that even though the Gaussian model was utilized for the convolutional noise sequence throughout the entire deconvolution procedure in [1,2,6,34,40,41,42,43], satisfying equalization performances were obtained. Recently, an attempt was made to approximate the convolutional noise pdf differently than with the Gaussian one in order to obtain improved equalization performance. In [4], the maximum entropy density approximation method [1,2,45,46] and Lagrange multipliers up to order four were used to approximate the convolutional noise pdf. In [5], the convolutional noise pdf was approximated with the Edgeworth Expansion series [47,48] up to order six. In [3], the Generalized Gaussian Density (GGD) [49,50] function and the Edgeworth Expansion [47,48] were applied to approximate the convolutional noise pdf. The GGD [49,50] is based on a shape parameter that modifies the pdf, which for shape parameters equal to one, two, and infinity, respectively, may have a Laplacian, or double exponential distribution, a Gaussian distribution, or a uniform distribution. Even though equalization performance was enhanced with these new approximation techniques for the convolutional noise pdf compared with the Gaussian case, a much higher equalization performance improvement was expected but not achieved. Thus, it makes us wonder if maybe the Gaussian model for the convolutional noise pdf is approximately correct even when the residual ISI is not so small. There is currently no closed-form approximated expression for the residual ISI where the Gaussian model can be used to describe approximately the convolutional noise pdf. It is well known that the equalizer can converge at a residual ISI level that might not be very low even high, depending on the applied step-size parameter used in the equalizer’s coefficients update mechanism. Furthermore, the equalization performance from the residual ISI point of view depends on the chosen equalization algorithm, equalizer’s tap length, input signal statistics, channel characteristics, and step size parameter. Thus, any closed-form approximated expression for the residual ISI where the convolutional noise pdf can be considered approximately Gaussian, must be a function of all the abovementioned parameters playing a role in the equalization performance from the residual ISI point of view. In this study, we address the noiseless and 16 Quadrature Amplitude Modulation (16QAM) situation. We use the Maximum Entropy density approximation technique [1,2,45,46] with Lagrange multipliers up to order six to approximate the pdf of the real part of the convolutional noise pdf. Then, we use Laplace’s integral method [51] and quasi-moment truncation technique [48] in order to obtain an approximated closed-form expression for the residual ISI for which the pdf of the real part of the convolutional noise can be approximately considered as Gaussian. This closed-form approximated expression is a function of the channel power, input sequence statistics, equalizer’s tap length and properties of the chosen equalizer. It is appropriate for the type of blind adaptive equalizers where the error that is given into the adaptive mechanism that updates the equalizer’s taps can be described as a polynomial function of the equalized output up to order three. It should be pointed out that Godard’s algorithm [52], for example, belongs to the mentioned type of blind equalizers. Based on this closed-form approximated expression for the residual ISI, we are able to show via simulation results that the Gaussian assumption for the convolutional noise pdf can be approximately made just before the equalizer has converged, even at a residual ISI level where the “eye diagram” is still very closed, namely, where the residual ISI can not be considered as very small. At that level of residual ISI, the fourth Lagrange multiplier in the approximated pdf of the real part of the convolutional noise is approximately zero, while the sixth Lagrange multiplier is very small and tends approximately to zero. Please note, since we deal with a two-independent quadrature carrier input case, all the even Lagrange multipliers in the convolutional noise pdf approximation are zero.

2. System Description

Figure 1 provides a description of the system under study. Please note that the described system is recalled from [53,54]. It should be noted here that the described system in Figure 1 is not unique but is a general description of a system using a blind adaptive equalizer to recover the input sequence from an unknown linear channel, as is the case in [52,55,56] and in other works.

In this paper, we make the following assumptions:

• The input sequence $x\left[n\right]$ is a 16QAM source, which can be expressed as $x\left[n\right]=x_1\left[n\right]+j{x}_2\left[n\right]$ where $x_1\left[n\right]$ and $x_2\left[n\right]$ are $x\left[n\right]$’s real and imaginary parts, respectively. 16QAM is a modulation that uses ± {1,3} levels for in-phase and quadrature components. $E\left[x\right[n\left]\right]=0$ and $E\left[\right(·\left)\right]$ denotes the expectation of $(·)$. The real and imaginary parts of $x\left[n\right]$ are independent.
• The unidentified channel $h\left[n\right]$ is a linear time-invariant filter that may not have a minimum phase and whose transfer function lacks “deep zeros,” or zeros that are sufficiently removed from the unit circle. The channel’s tap length is R.
• The filter $c\left[n\right]$ is a tap-delay line.
• The channel noise $w\left[n\right]$ is an additive Gaussian white noise.

The sequence $x\left[n\right]$ is sent through the channel $h\left[n\right]$ where the output sequence from the channel is corrupted with noise $w\left[n\right]$. The input sequence to the blind adaptive equalizer is denoted as $y\left[n\right]$ and expressed by:

$$y\left[n\right]=x\left[n\right]\ast h\left[n\right]+w\left[n\right]$$

(1)

where the convolution operation is indicated by the symbol ∗. $z\left[n\right]$ is the equalized output sequence and given by:

$$\begin{array}{c}z\left[n\right]=c\left[n\right]\ast y\left[n\right]=x\left[n\right]\ast s\left[n\right]+c\left[n\right]\ast w\left[n\right]=\\ \\ x\left[n\right]+p\left[n\right]+c\left[n\right]\ast w\left[n\right]\\ \mathit{where}\\ s\left[n\right]=h\left[n\right]\ast c\left[n\right];\hspace{1em}\hspace{1em}p\left[n\right]=x\left[n\right]\ast \left(s\right[n]-\delta [n\left]\right)\end{array}$$

(2)

where $p\left[n\right]$ is the convolutional noise arising for not having the optimal values for the equalizer’s coefficients. The real and imaginary components of $p\left[n\right]$ are denoted in the following as $p_1\left[n\right]$ and $p_2\left[n\right]$, respectively. Since we deal with a two independent quadrature carrier constellation input, $E\left[x_1^v\left[n\right]\right]=E\left[x_2^v\left[n\right]\right]$ and $E\left[p_1^v\left[n\right]\right]=E\left[p_2^v\left[n\right]\right]$ for $v=1,2,\dots ,V$. The ISI expression is used to evaluate the equalizer’s performance:

$$ISI=\frac{{\sum }_m|s_m{\left[n\right]|}^2-{\left|s_m\left[n\right]\right|}_{max}^2}{|s_m{\left[n\right]|}_{max}^2}$$

(3)

where $|·|$ is the value of $(·)$ in absolute terms and $|s_m{\left[n\right]|}_{max}$ is the component of $s\left[n\right]$ in (2) with the highest absolute value. For the noiseless case, $|s_m{\left[n\right]|}_{max}^2=1$ and the equalizer has entered its convergence state, we may write according to [1]:

$$ISI\simeq 10lo{g}_{10}\left(\frac{E\left[\right|p\left[n\right]{|}^2]}{E\left[\right|x\left[n\right]{|}^2]}\right)\mathrm{for}{\left|s_m\left[n\right]\right|}_{max}^2=1$$

(4)

Since we are dealing with the 16QAM constellation input situation of two independent quadrature carriers, we may express (4) as:

$$ISI\simeq 10lo{g}_{10}\left(\frac{E\left[p_1^2\left[n\right]\right]}{E\left[x_1^2\left[n\right]\right]}\right)\mathrm{for}{\left|s_m\left[n\right]\right|}_{max}^2=1$$

(5)

The equalizer’s update mechanism can be described by:

$$\underline{c}[n+1]=\underline{c}\left[n\right]-\mu \frac{\partial F\left[n\right]}{\partial z\left[n\right]}{\underline{y}}^{\ast }\left[n\right]$$

(6)

where the conjugate operation is ()*, the step-size parameter is $\mu$, the cost function is $F\left[n\right]$, $\frac{\partial F\left[n\right]}{\partial z\left[n\right]}$ is the cost function’s derivation from the equalized output sequence and $\underline{c}\left[n\right]$ is the equalizer vector where $\underline{y}\left[n\right]={[y\left[n\right]\dots y[n-N+1]]}^T$ is the input vector. The equalizer’s tap length is N, and the operator ()^T stands for the transpose of the function (). In this paper, the MMA algorithm [55,56] and Godard’s algorithm [52] are used. The equalizer’s coefficients are updated according to the MMA algorithm ([55,56]) by:

$$\begin{array}{c}\underline{c}[n+1]=\underline{c}\left[n\right]-{\mu }_{MMA}\left[Re\left[z\left[n\right]\right]\left[|Re\left[z\left[n\right]\right]{|}^2-\frac{E\left[|x_1{\left[n\right]|}^4\right]}{E\left[|x_1{\left[n\right]|}^2\right]}\right]+\right.\\ \\ \left.j\left[Im\left[z\left[n\right]\right]\right]\left[|Im\left[z\left[n\right]\right]{|}^2-\frac{E\left[|x_2{\left[n\right]|}^4\right]}{E\left[|x_2{\left[n\right]|}^2\right]}\right]\right]{\underline{y}}^{\ast }\left[n\right]\end{array}$$

(7)

where ${\mu }_{MMA}\ge 0$, $Re\left[·\right]$ and $Im\left[·\right]$ are the real and imaginary parts of $\left[·\right]$, respectively. The equalizer’s coefficients are updated according to Godard’s algorithm [52] by:

$$\underline{c}[n+1]=\underline{c}\left[n\right]-{\mu }_G\left({\left|z\left[n\right]\right|}^2-\frac{E\left[{\left|x\left[n\right]\right|}^4\right]}{E\left[{\left|x\left[n\right]\right|}^2\right]}\right)z\left[n\right]$$

(8)

where ${\mu }_G\ge 0$. It needs to be stated that according to [57], Godard’s algorithm is one of the most widely used blind equalization algorithms and has become the workhorse for blind equalization. According to [57], Godard’s algorithm is carrier phase independent. Therefore, carrier synchronization is not necessary prior to blind equalization. However, an arbitrary phase rotation is present in the constellation visible at the equalized output sequence [57]. As a result, a phase rotator is necessary at the equalizer’s convergence state in order to spin the constellation back into place [57]. The MMA method ([55,56]) avoids the necessity for a phase rotator, in accordance with [57], because it employs a separate error-calculation, i.e., for the real and imaginary parts of the received signal, separately. In this paper, we assume that $\frac{\partial F\left[n\right]}{\partial z\left[n\right]}$ can be expressed as a polynomial function of the equalized output namely as:

$$P\left[z\left[n\right]\right]=\frac{\partial F\left[n\right]}{\partial z\left[n\right]}$$

(9)

Thus, based on (9) and [58], the real part of the polynomial function $P\left[z\right[n\left]\right]$ of order up to three can be expressed by:

$$Re\left[P\left[z\right[n\left]\right]\right]=a_1{z}_1\left[n\right]+a_3{z}_1^3\left[n\right]+a_{12}{z}_1\left[n\right]z_2^2\left[n\right]$$

(10)

where the real and imaginary components of the equalized output $z\left[n\right]$ are denoted as $z_1\left[n\right]$ and $z_2\left[n\right]$, respectively. Please note that for the 16QAM constellation input case, $a_1=-\frac{E\left[{\left|x\left[n\right]\right|}^4\right]}{E\left[{\left|x\left[n\right]\right|}^2\right]}$, $a_3=1$ and $a_{12}=1$ for Godard’s algorithm [52] while for the MMA algorithm ([55,56]) we have that $a_1=-\frac{E\left[{\left|x_1\left[n\right]\right|}^4\right]}{E\left[{\left|x_1\left[n\right]\right|}^2\right]}$, $a_3=1$ and $a_{12}=0$.

3. The Residual ISI That Leads Approximately to a Gaussian pdf for the Convolutional Noise

In this section, we present a closed-form approximated expression for the residual ISI as a function of the system’s parameters (step-size parameter, input constellation statistics, Equalizer’s tap length, channel power and properties of the chosen equalizer via $a_1$, $a_3$ and $a_{12}$) for which the convolutional noise pdf can be approximately considered as a Gaussian pdf.

Theorem 1.

The residual ISI ($IS{I}_{res}$) for which the convolutional noise pdf associated with the blind adaptive equalization problem is approximately a Gaussian pdf can be expressed for the noiseless case by:

$$\begin{array}{c}IS{I}_{res}\simeq 10lo{g}_{10}{m}_p-10lo{g}_{10}{m}_2;m_2=E\left[x_1^2\left[n\right]\right];m_p=E\left[p_1{\left[n\right]}^2\right]\end{array}$$

(11)

where $m_p$ is the solution of the following equation:

$$\begin{array}{c}{A}_1{m}_p^3+A_2{m}_p^2+A_3{m}_p+A_4=0\\ \\ where\\ \\ A_1=105T\left(3B^2{a}_{12}^2\left(\frac{3}{T}-1\right)+B^2{a}_3^2\left(\frac{135}{T}-15\right)+6B^2{a}_3{a}_{12}\left(\frac{3}{T}-1\right)\right)-\\ \\ 30B^2\left(a_3^2\left(\frac{28350}{T}-945\right)-3a_{12}^2\left(\frac{3}{T}-1\right)\left(\frac{135}{T}-15\right)+2a_3{a}_{12}\left(\frac{1890}{T}-105\right)\right)\\ \\ A_2=12a_{12}\left(\frac{135}{T}-15\right)-30B^2\left(6m_2{a}_{12}^2\left(\frac{135}{T}-15\right)+15a_3^2{m}_2\left(\frac{1890}{T}-105\right)-\right.\\ 9m_2{a}_{12}^2{\left(\frac{3}{T}-1\right)}^2+2a_1{a}_{12}\left(\frac{135}{T}-15\right)+2a_1{a}_3\left(\frac{1890}{T}-105\right)+\\ \left.12a_3{m}_2{a}_{12}\left(\frac{135}{T}-15\right)+2a_3{m}_2{a}_{12}\left(\frac{1890}{T}-105\right)\right)+\\ 12a_3\left(\frac{1890}{T}-105\right)+105T\left(2B\left(a_{12}-3a_3\left(\frac{3}{T}-1\right)\right)-2B^2{a}_1{a}_{12}-\right.\\ 6B^2{m}_2{a}_{12}^2-12B^2{a}_3{m}_2{a}_{12}+6B^2{a}_1{a}_3\left(\frac{3}{T}-1\right)+\\ \left.45B^2{a}_3^2{m}_2\left(\frac{3}{T}-1\right)+3B^2{m}_2{a}_{12}^2\left(\frac{3}{T}-1\right)+6B^2{a}_3{m}_2{a}_{12}\left(\frac{3}{T}-1\right)\right)\\ \\ A_3=2B\left(6a_1\left(\frac{135}{T}-15\right)+18a_3{m}_2\left(\frac{135}{T}-15\right)+6m_2{a}_{12}\left(\frac{135}{T}-15\right)\right)-\\ 30B^2\left(a_1^2\left(\frac{135}{T}-15\right)+15a_3^2{m}_4\left(\frac{135}{T}-15\right)+m_4{a}_{12}^2\left(\frac{135}{T}-15\right)+\right.\\ 18m_2^2{a}_{12}^2\left(\frac{3}{T}-1\right)+6a_1{m}_2{a}_{12}\left(\frac{3}{T}-1\right)+6a_3{m}_4{a}_{12}\left(\frac{3}{T}-1\right)+\\ \left.12a_1{a}_3{m}_2\left(\frac{135}{T}-15\right)+2a_1{m}_2{a}_{12}\left(\frac{135}{T}-15\right)+12a_3{m}_2^2{a}_{12}\left(\frac{135}{T}-15\right)\right)-\\ 105T\left(B^2{a}_1^2-2B\left(a_1+3a_3{m}_2+m_2{a}_{12}\right)+\right.\\ 15B^2{a}_3^2{m}_4+B^2{m}_4{a}_{12}^2+6B^2{m}_2^2{a}_{12}^2+12B^2{a}_1{a}_3{m}_2+\\ \left.4B^2{a}_1{m}_2{a}_{12}+2B^2{a}_3{m}_4{a}_{12}+12B^2{a}_3{m}_2^2{a}_{12}\right)\\ \\ A_4=-105T\left(B^2{a}_1^2{m}_2+2m_4{B}^2{a}_1{a}_3+2B^2{a}_1{m}_2^2{a}_{12}+\right.\\ \left.m_6{B}^2{a}_3^2+2m_4{B}^2{a}_3{m}_2{a}_{12}+m_4{B}^2{m}_2{a}_{12}^2\right)-\\ 30B^2\left(3a_1^2{m}_2\left(\frac{3}{T}-1\right)+3a_3^2{m}_6\left(\frac{3}{T}-1\right)+6a_1{a}_3{m}_4\left(\frac{3}{T}-1\right)+\right.\\ \left.6a_1{m}_2^2{a}_{12}\left(\frac{3}{T}-1\right)+3m_2{m}_4{a}_{12}^2\left(\frac{3}{T}-1\right)+6a_3{m}_2{m}_4{a}_{12}\left(\frac{3}{T}-1\right)\right)\\ \\ where\\ \\ m_l=E\left[x_1^l\left[n\right]\right];l=2,4,6;B=2m_2\mu N\sum _{k=0}^{R-1}{\left|h_k\left[n\right]\right|}^{2}T\gg 100\end{array}$$

(12)

Proof of Theorem 1. 

Since we deal with the 16QAM constellation input (a two-independent quadrature carrier case), we consider in the following only the pdf of the real part of the convolutional noise. Please note that the pdf of the imaginary part of the convolutional noise is approximately equal to the pdf of the real part of the convolutional noise. The pdf of the real part of the convolutional noise at time indexes n and $n+1$, can be approximately expressed with the help of the Maximum Entropy density approximation technique [1,2,45,46] with Lagrange multipliers up to order six as:

$$\begin{array}{c}f\left(p_1\left[n\right]\right)\simeq exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]+{\lambda }_6{p}_1^6\left[n\right]\right)\\ \\ f\left(p_1\left[n+1\right]\right)\simeq exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n+1\right]+{\lambda }_4{p}_1^4\left[n+1\right]+{\lambda }_6{p}_1^6\left[n+1\right]\right)\end{array}$$

(13)

where ${\lambda }_2$, ${\lambda }_4$ and ${\lambda }_6$ are the Lagrange multipliers up to order six. Next, the difference between the pdf of the real part of the convolutional noise at time index $n+1$ with that of time index n is given by:

$$\begin{array}{c}\Delta f=f\left(p_1\left[n+1\right]\right)-f\left(p_1\left[n\right]\right)=\\ \\ exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n+1\right]+{\lambda }_4{p}_1^4\left[n+1\right]+{\lambda }_6{p}_1^6\left[n+1\right]\right)-exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]+{\lambda }_6{p}_1^6\left[n\right]\right)\\ \\ =exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]+{\lambda }_6{p}_1^6\left[n\right]\right)\left(exp\left({\lambda }_2\left(p_1^2\left[n+1\right]-p_1^2\left[n\right]\right)+\right.\right.\\ \\ \left.\left.{\lambda }_4(p_1^4\left[n+1\right]-p_1^4\left[n\right]\right)+{\lambda }_6(p_1^6\left[n+1\right]-p_1^6\left[n\right])-1\right)\end{array}$$

(14)

At the convergence state of the equalizer we may assume that:

$$\Delta f\simeq 0$$

(15)

Thus, based on (14) and (15) we may write:

$$E\left[exp\left({\lambda }_2\left(p_1^2\left[n+1\right]-p_1^2\left[n\right]\right)+{\lambda }_4\left(p_1^4\left[n+1\right]-p_1^4\left[n\right]\right)+{\lambda }_6(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right)-1\right]\simeq 0$$

(16)

By using Taylor’s expansion for the exponent [59] ($exp\left(Q\right)\simeq 1+Q$) we may write (16) as:

$$\begin{array}{c}E\left[exp\left({\lambda }_2\left(p_1^2\left[n+1\right]-p_1^2\left[n\right]\right)+{\lambda }_4\left(p_1^4\left[n+1\right]-p_1^4\left[n\right]\right)+{\lambda }_6(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right)-1\right]\simeq \\ \\ E\left[{\lambda }_2\left(p_1^2\left[n+1\right]-p_1^2\left[n\right]\right)+{\lambda }_4\left(p_1^4\left[n+1\right]-p_1^4\left[n\right]\right)+{\lambda }_6(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]\simeq 0\end{array}$$

(17)

Based on (17) we have for $E\left[(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]\ne 0$:

$$\begin{array}{c}{\lambda }_6=-{\lambda }_{2}G-{\lambda }_{4}F\\ \\ where\\ \\ G=\frac{E\left[p_1^2\left[n+1\right]-p_1^2\left[n\right]\right]}{E\left[(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]}F=\frac{E\left[p_1^4\left[n+1\right]-p_1^4\left[n\right]\right]}{E\left[(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]}\end{array}$$

(18)

By using (13) and (18) we have:

$$\begin{array}{c}f\left(p_1\left[n\right]\right)\simeq exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right]\right)\end{array}$$

(19)

In order to find closed-form approximated expressions for ${\lambda }_0$, ${\lambda }_2$ and ${\lambda }_4$ as a function of the convolutional noise statistics, we use:

$$\begin{array}{c}{\int }_{-\infty }^{\infty }f\left(p_1\left[n\right]\right)=1\\ \\ {\int }_{-\infty }^{\infty }{p}_1^2\left[n\right]f\left(p_1\left[n\right]\right)d{p}_1\left[n\right]=E\left[p_1^2\left[n\right]\right]\\ \\ {\int }_{-\infty }^{\infty }{p}_1^4\left[n\right]f\left(p_1\left[n\right]\right)d{p}_1\left[n\right]=E\left[p_1^4\left[n\right]\right]\end{array}$$

(20)

Now, based on (19) and (20) we can write:

$$\begin{array}{c}{\int }_{-\infty }^{\infty }exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right]\right)d{p}_1\left[n\right]\simeq 1\\ \\ {\int }_{-\infty }^{\infty }{p}_1^2\left[n\right]exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right]\right)d{p}_1\left[n\right]\simeq E\left[p_1^2\left[n\right]\right]\\ \\ {\int }_{-\infty }^{\infty }{p}_1^4\left[n\right]exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right]\right)d{p}_1\left[n\right]\simeq E\left[p_1^4\left[n\right]\right]\end{array}$$

(21)

The first integral in (21) can be written as:

$$\begin{array}{c}exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])exp\left({\lambda }_2\left(p_1^2\left[n\right]\right)\right)d{p}_1\left[n\right]=\\ \\ exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{g}_1\left(p_1\left[n\right]\right)exp(-\frac{\Psi \left(p_1\left[n\right]\right)}{\gamma })d{p}_1\left[n\right]\\ \\ \mathrm{where}\\ \\ g_1\left(p_1\left[n\right]\right)=exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])\\ \\ \Psi \left(p_1\left[n\right]\right)=p_1^2\left[n\right]\gamma =-\frac{1}{{\lambda }_2}\end{array}$$

(22)

According to the Laplace’s integral method [51], we can solve the integral in (22) by:

$$\begin{array}{c}{\int }_{-\infty }^{\infty }{g}_1\left(p_1\left[n\right]\right)exp(-\frac{\Psi \left(p_1\left[n\right]\right)}{\gamma })d{p}_1\left[n\right]\simeq \\ \\ exp\left(-\frac{\Psi \left(p_0\right)}{\gamma }\right)\sqrt{\frac{2\pi \gamma }{{\Psi }^{″}\left(p_0\right)}}\left(g_1\left(p_0\right)+\frac{g_1^{″}\left(p_0\right)}{2}\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}+\frac{g_1^{⁗}\left(p_0\right)}{8}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^2+\right.\\ \\ \left.\frac{g_1^{VI}\left(p_0\right)}{48}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^3+\frac{g_1^{VIII}\left(p_0\right)}{384}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^4+\frac{g_1^X\left(p_0\right)}{3840}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^5+O\left(\frac{{\gamma }^6}{{\left({\Psi }^{″}\left(p_0\right)\right)}^6}\right)\right)\end{array}$$

(23)

where ${\left(\right)}^{″}$, ${\left(\right)}^{⁗}$, ${\left(\right)}^{VI}$, ${\left(\right)}^{VIII}$ and ${\left(\right)}^X$ denote the second, fourth, sixth, eighth and tenth derivative of (), respectively. $O\left(v\right)$ is defined as $li{m}_{v\to 0}O\left(v\right)/v=r_{const}$ and $r_{const}$ is a constant. The function ${\Psi }^{″}\left(p_0\right)$ and $\left(p_0\right)$ are obtained via:

$$\begin{array}{c}{\Psi }^{\prime }\left(p_1\left[n\right]\right)=2p_1\left[n\right]⟹{\Psi }^{″}\left(p_1\left[n\right]\right)=2\\ \\ {\Psi }^{\prime }\left(p_1\left[n\right]\right)=2p_1\left[n\right]⟹{\Psi }^{\prime }\left(p_0\right)=2p_0=0⟹p_0=0\end{array}$$

(24)

while:

$$\begin{array}{c}{g}_1\left(p_0\right)=1;g_1^{″}\left(p_0\right)=0;g_1^{⁗}\left(p_0\right)=24{\lambda }_4;g_1^{VI}\left(p_0\right)=-720\left(F{\lambda }_4+G{\lambda }_2\right)\end{array}$$

(25)

Based on (23)–(25), we may write (22) as:

$$\begin{array}{c}exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])exp\left({\lambda }_2\left(p_1^2\left[n\right]\right)\right)d{p}_1\left[n\right]\simeq \\ \\ exp({\lambda }_0)\sqrt{\frac{-2\pi \frac{1}{{\lambda }_2}}{2}}\left(1+\frac{24{\lambda }_4}{8}{\left(\frac{-\frac{1}{{\lambda }_2}}{2}\right)}^2-720\frac{F{\lambda }_4+G{\lambda }_2}{48}{\left(\frac{-\frac{1}{{\lambda }_2}}{2}\right)}^3\right)\simeq \\ \\ exp\left({\lambda }_0\right)\sqrt{\frac{-\pi }{{\lambda }_2}}\left(1+\frac{3{\lambda }_4}{4{\lambda }_2^2}+\frac{15\left(F{\lambda }_4+G{\lambda }_2\right)}{8{\lambda }_2^3}\right)\end{array}$$

(26)

Based on (21) and (26) we have:

$$exp(-{\lambda }_0)\simeq \sqrt{\frac{-\pi }{{\lambda }_2}}\left(1+\frac{3{\lambda }_4}{4{\lambda }_2^2}+\frac{15\left(F{\lambda }_4+G{\lambda }_2\right)}{8{\lambda }_2^3}\right)$$

(27)

Next, we turn to calculate the second integral in (21), which can be expressed as:

$$\begin{array}{c}exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{p}_1^2\left[n\right]exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])exp\left({\lambda }_2\left(p_1^2\left[n\right]\right)\right)d{p}_1\left[n\right]=\\ \\ exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{g}_2\left(p_1\left[n\right]\right)exp(-\frac{\Psi \left(p_1\left[n\right]\right)}{\gamma })d{p}_1\left[n\right]\\ \\ \mathrm{where}\\ \\ g_2\left(p_1\left[n\right]\right)=p_1^2\left[n\right]exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])\end{array}$$

(28)

According to Laplace’s integral method [51], we can write (28) as:

$$\begin{array}{c}{\int }_{-\infty }^{\infty }{g}_2\left(p_1\left[n\right]\right)exp(-\frac{\Psi \left(p_1\left[n\right]\right)}{\gamma })d{p}_1\left[n\right]\simeq \\ \\ exp\left(-\frac{\Psi \left(p_0\right)}{\gamma }\right)\sqrt{\frac{2\pi \gamma }{{\Psi }^{″}\left(p_0\right)}}\left(g_2\left(p_0\right)+\frac{g_2^{″}\left(p_0\right)}{2}\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}+\frac{g_2^{⁗}\left(p_0\right)}{8}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^2+\right.\\ \\ \left.\frac{g_2^{VI}\left(p_0\right)}{48}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^3+\frac{g_2^{VIII}\left(p_0\right)}{384}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^4+O\left(\frac{{\gamma }^5}{{\left({\Psi }^{″}\left(p_0\right)\right)}^5}\right)\right)\end{array}$$

(29)

where:

$$\begin{array}{c}{g}_2\left(p_0\right)=0;g_2^{″}\left(p_0\right)=2;g_2^{⁗}\left(p_0\right)=0\\ \\ g_2^{VI}\left(p_0\right)=720{\lambda }_4;g_2^{VIII}\left(p_0\right)=-40320F{\lambda }_4-40320G{\lambda }_2\end{array}$$

(30)

Based on (21), (29) and (30) we may write (28) as:

$$\begin{array}{c}exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{p}_1^2\left[n\right]exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])exp\left({\lambda }_2\left(p_1^2\left[n\right]\right)\right)d{p}_1\left[n\right]\simeq \\ \\ exp\left({\lambda }_0\right)\sqrt{\frac{-\pi }{{\lambda }_2}}\left(-\frac{1}{2{\lambda }_2}-15{\lambda }_4\frac{1}{8{\lambda }_2^3}-\left(105F{\lambda }_4+105G{\lambda }_2\right)\frac{1}{16{\lambda }_2^4}\right)\simeq m_p\end{array}$$

(31)

Based on (27) and (31) we have:

$$\begin{array}{c}\sqrt{\frac{-\pi }{{\lambda }_2}}\left(-\frac{1}{2{\lambda }_2}-15{\lambda }_4\frac{1}{8{\lambda }_2^3}-\left(105F{\lambda }_4+105G{\lambda }_2\right)\frac{1}{16{\lambda }_2^4}\right)\simeq \\ \\ m_{p}exp(-{\lambda }_0)\simeq m_p\sqrt{\frac{-\pi }{{\lambda }_2}}\left(1+\frac{3{\lambda }_4}{4{\lambda }_2^2}+\frac{15\left(F{\lambda }_4+G{\lambda }_2\right)}{8{\lambda }_2^3}\right)\\ \\ \Downarrow \\ {\lambda }_4\simeq \frac{\left(-\frac{1}{2{\lambda }_2}-105G{\lambda }_2\frac{1}{16{\lambda }_2^4}\right)-m_p\left(1+\frac{15G{\lambda }_2}{8{\lambda }_2^3}\right)}{\frac{15}{8{\lambda }_2^3}+\frac{105F}{16{\lambda }_2^4}+m_p\left(\frac{3}{4{\lambda }_2^2}+\frac{15F}{8{\lambda }_2^3}\right)}\end{array}$$

(32)

Next, we turn to calculate the third integral in (21), which can be expressed as:

$$\begin{array}{c}exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{p}_1^4\left[n\right]exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])exp\left({\lambda }_2\left(p_1^2\left[n\right]\right)\right)d{p}_1\left[n\right]=\\ \\ exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{g}_3\left(p_1\left[n\right]\right)exp(-\frac{\Psi \left(p_1\left[n\right]\right)}{\gamma })d{p}_1\left[n\right]\\ \\ \mathrm{where}\\ \\ g_3\left(p_1\left[n\right]\right)=p_1^4\left[n\right]exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])\end{array}$$

(33)

According to Laplace’s integral method [51], we can write (33) as:

$$\begin{array}{c}{\int }_{-\infty }^{\infty }{g}_3\left(p_1\left[n\right]\right)exp(-\frac{\Psi \left(p_1\left[n\right]\right)}{\gamma })d{p}_1\left[n\right]\simeq \\ \\ exp\left(-\frac{\Psi \left(p_0\right)}{\gamma }\right)\sqrt{\frac{2\pi \gamma }{{\Psi }^{″}\left(p_0\right)}}\left(g_3\left(p_0\right)+\frac{g_3^{″}\left(p_0\right)}{2}\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}+\frac{g_3^{⁗}\left(p_0\right)}{8}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^2+\right.\\ \\ \left.\frac{g_3^{VI}\left(p_0\right)}{48}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^3+\frac{g_3^{VIII}\left(p_0\right)}{384}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^4+\frac{g_3^X\left(p_0\right)}{3840}{\left(\frac{\gamma }{{\Psi }^{″}\left(p_0\right)}\right)}^5+O\left(\frac{{\gamma }^6}{{\left({\Psi }^{″}\left(p_0\right)\right)}^6}\right)\right)\end{array}$$

(34)

where:

$$\begin{array}{c}{g}_3\left(p_0\right)=0;g_3^{″}\left(p_0\right)=0;g_3^{⁗}\left(p_0\right)=24\\ \\ g_3^{VI}\left(p_0\right)=0;g_3^{VIII}\left(p_0\right)=40320{\lambda }_4;g_3^X\left(p_0\right)=-3628800F{\lambda }_4-3628800G{\lambda }_2\end{array}$$

(35)

Based on (21), (34) and (35) we can write (33) as:

$$\begin{array}{c}exp\left({\lambda }_0\right){\int }_{-\infty }^{\infty }{p}_1^4\left[n\right]exp({\lambda }_4{p}_1^4\left[n\right]-{\lambda }_{2}G{p}_1^6\left[n\right]-{\lambda }_{4}F{p}_1^6\left[n\right])exp\left({\lambda }_2\left(p_1^2\left[n\right]\right)\right)d{p}_1\left[n\right]\simeq \\ \\ exp\left({\lambda }_0\right)\sqrt{\frac{-\pi }{{\lambda }_2}}\left(3\left(\frac{1}{4{\lambda }_2^2}\right)+105{\lambda }_4\left(\frac{1}{16{\lambda }_2^4}\right)-945\left(F{\lambda }_4+G{\lambda }_2\right)\left(\frac{1}{32{\lambda }_2^5}\right)\right)=E\left[p_1^4\left[n\right]\right]\end{array}$$

(36)

Based on (27) and (36) we may write:

$$\begin{array}{c}\sqrt{\frac{-\pi }{{\lambda }_2}}\left(3\left(\frac{1}{4{\lambda }_2^2}\right)+105{\lambda }_4\left(\frac{1}{16{\lambda }_2^4}\right)-945\left(F{\lambda }_4+G{\lambda }_2\right)\left(\frac{1}{32{\lambda }_2^5}\right)\right)\simeq \\ \\ \sqrt{\frac{-\pi }{{\lambda }_2}}\left(1+\frac{3{\lambda }_4}{4{\lambda }_2^2}+\frac{15\left(F{\lambda }_4+G{\lambda }_2\right)}{8{\lambda }_2^3}\right)E\left[p_1^4\left[n\right]\right]\\ \\ \Downarrow \\ \\ {\lambda }_4\simeq \frac{\left(1+\frac{15G}{8{\lambda }_2^2}\right)E\left[p_1^4\left[n\right]\right]-\frac{3}{4{\lambda }_2^2}+945G\left(\frac{1}{32{\lambda }_2^4}\right)}{\frac{105}{16{\lambda }_2^4}-945F\frac{1}{32{\lambda }_2^5}-\frac{3}{4{\lambda }_2^2}-\frac{15F}{8{\lambda }_2^3}}\end{array}$$

(37)

However, we already received ${\lambda }_4$ in (32). Thus, the expression for ${\lambda }_4$ in (37) and that obtained in (32) should be approximately the same. Thus, we may write:

$$\begin{array}{c}\frac{\left(-\frac{1}{2{\lambda }_2}-105G{\lambda }_2\frac{1}{16{\lambda }_2^4}\right)-m_p\left(1+\frac{15G{\lambda }_2}{8{\lambda }_2^3}\right)}{\frac{15}{8{\lambda }_2^3}+\frac{105F}{16{\lambda }_2^4}+m_p\left(\frac{3}{4{\lambda }_2^2}+\frac{15F}{8{\lambda }_2^3}\right)}\simeq \\ \\ \frac{\left(1+\frac{15G}{8{\lambda }_2^2}\right)E\left[p_1^4\left[n\right]\right]-\frac{3}{4{\lambda }_2^2}+945G\left(\frac{1}{32{\lambda }_2^4}\right)}{\frac{105}{16{\lambda }_2^4}-945F\frac{1}{32{\lambda }_2^5}-\frac{3}{4{\lambda }_2^2}-\frac{15F}{8{\lambda }_2^3}}\end{array}$$

(38)

Now, for $\frac{8{\lambda }_2^2}{105}\gg G$ we may write (38) as:

$$\begin{array}{c}\frac{-\frac{1}{2{\lambda }_2}-m_p}{\frac{15}{8{\lambda }_2^3}+\frac{105F}{16{\lambda }_2^4}+m_p\left(\frac{3}{4{\lambda }_2^2}+\frac{15F}{8{\lambda }_2^3}\right)}\simeq \frac{E\left[p_1^4\left[n\right]\right]-\frac{3}{4{\lambda }_2^2}+945G\left(\frac{1}{32{\lambda }_2^4}\right)}{\frac{105}{16{\lambda }_2^4}-945F\frac{1}{32{\lambda }_2^5}-\frac{3}{4{\lambda }_2^2}-\frac{15F}{8{\lambda }_2^3}}\end{array}$$

(39)

Next, let us write $G\simeq \frac{8{\lambda }_2^2}{105T}$ where T is very large positive value and use it in (39):

$$\begin{array}{c}\frac{-\frac{1}{2{\lambda }_2}-m_p}{\frac{15}{8{\lambda }_2^3}+\frac{105F}{16{\lambda }_2^4}+m_p\left(\frac{3}{4{\lambda }_2^2}+\frac{15F}{8{\lambda }_2^3}\right)}\simeq \frac{E\left[p_1^4\left[n\right]\right]-\frac{3}{4{\lambda }_2^2}+\frac{9}{4{\lambda }_2^{2}T}}{\frac{105}{16{\lambda }_2^4}-945F\frac{1}{32{\lambda }_2^5}-\frac{3}{4{\lambda }_2^2}-\frac{15F}{8{\lambda }_2^3}}\end{array}$$

(40)

A possible solution for the left side of (40) being equal to its right side is when:

$$\begin{array}{c}-\frac{1}{2{\lambda }_2}\simeq m_p;E\left[p_1^4\left[n\right]\right]-\frac{3}{4{\lambda }_2^2}+\frac{9}{4{\lambda }_2^{2}T}\simeq 0\\ \\ \Downarrow \\ \\ m_p\simeq -\frac{1}{2{\lambda }_2}\Rightarrow E\left[p_1^4\left[n\right]\right]\simeq 3m_p^2\left(1-\frac{3}{T}\right)\end{array}$$

(41)

Please note that when (41) holds it means that ${\lambda }_4\simeq 0$. In addition, for $T\to \infty$, $E\left[p_1^4\left[n\right]\right]\to 3m_p^2$, which holds in the Gaussian case. Next, by using (18), $G\simeq \frac{8{\lambda }_2^2}{105T}$ and (41) we may write:

$$\begin{array}{c}\frac{E\left[p^2\left[n+1\right]-p^2\left[n\right]\right]}{E\left[(p^6\left[n+1\right]-p^6\left[n\right])\right]}\simeq \frac{8{\lambda }_2^2}{105T}\simeq \frac{8{\left(-\frac{1}{2m_p}\right)}^2}{105T}\simeq \frac{2}{105m_p^{2}T}\\ \\ \Downarrow \\ \\ {\lambda }_6=-{\lambda }_2\frac{E\left[p_1^2\left[n+1\right]-p_1^2\left[n\right]\right]}{E\left[(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]}-{\lambda }_4\frac{E\left[p_1^4\left[n+1\right]-p_1^4\left[n\right]\right]}{E\left[(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]}\simeq \\ \\ -{\lambda }_2\frac{E\left[p_1^2\left[n+1\right]-p_1^2\left[n\right]\right]}{E\left[(p_1^6\left[n+1\right]-p_1^6\left[n\right])\right]}\simeq \frac{1}{105m_p^{3}T}\end{array}$$

(42)

By using (27), $G\simeq \frac{8{\lambda }_2^2}{105T}$, (41) and (42) we may write $f\left(p_1\left[n\right]\right)$ in (13) as:

$$\begin{array}{c}f\left(p_1\left[n\right]\right)\simeq exp\left({\lambda }_0+{\lambda }_2{p}_1^2\left[n\right]+{\lambda }_4{p}_1^4\left[n\right]+{\lambda }_6{p}_1^6\left[n\right]\right)\simeq \\ \\ \frac{1}{\sqrt{2\pi m_p}\left(1+\frac{1}{7T}\right)}exp\left(-\frac{1}{2m_p}{p}_1^2\left[n\right]+\frac{1}{105m_p^{3}T}{p}_1^6\left[n\right]\right)\end{array}$$

(43)

Please note that for $m_p\ne 0$ and $T\to \infty$, the convolutional noise pdf given in (43) tends to the Gaussian one. However, this does not tell us for which residual ISI this occurs. Thus, we turn to the expression of $G\simeq \frac{8{\lambda }_2^2}{105T}$ and ask ourselves what is the residual ISI or $m_p$ for which $G\simeq \frac{8{\lambda }_2^2}{105T}$ holds. In order to do this, we first have to obtain closed-form expressions for $E\left[p^2\left[n+1\right]-p^2\left[n\right]\right]$ and $E\left[p^6\left[n+1\right]-p^6\left[n\right]\right]$. According to [58], we have:

$$\begin{array}{c}\Delta {\tilde{g}}_i=\frac{\partial {\tilde{g}}_i}{\partial p_1\left[n\right]}\Delta p_1+\frac{1}{2}\frac{{\partial }^2{\tilde{g}}_i}{{\partial }^2{p}_1\left[n\right]}{\left(\Delta p_1\right)}^2+O{\left(\Delta p_1\right)}^3;i=1,2;\\ \\ \mathrm{where}\\ \\ \Delta {\tilde{g}}_i={\tilde{g}}_i[n+1]-{\tilde{g}}_i\left[n\right];\Delta p_1=-Re\left[\mu P\left[z\left[n\right]\right]\sum _{m=0}^{m=R-1}y\left[n-m\right]y^{\ast }\left[n-m\right]\right]\end{array}$$

(44)

where $P\left[z\right[n\left]\right]$ and $Re\left[P\right[z\left[n\right]\left]\right]$ are given in (9) and (10) respectively. Thus, based on (44) we have:

$$\begin{array}{c}{\tilde{g}}_1\left[n\right]=p_1^2\left[n\right]\Rightarrow E\left[p_1^2[n+1]-p_1^2\left[n\right]\right]\simeq 2E\left[p_1\left[n\right]\Delta p_1\right]+E\left[{\left(\Delta p_1\right)}^2\right]\\ \\ {\tilde{g}}_2\left[n\right]=p_1^6\left[n\right]\Rightarrow E\left[p_1^6[n+1]-p_1^6\left[n\right]\right]\simeq 6E\left[p_1^5\left[n\right]\Delta p_1\right]+15E\left[p_1^4\left[n\right]{\left(\Delta p_1\right)}^2\right]\end{array}$$

(45)

In the following, we assume that $E\left[{\left({\sum }_{m=0}^{m=R-1}y\left[n-m\right]y^{\ast }\left[n-m\right]\right)}^2\right]\simeq B^2$, as was done in [58]. Please note that in [58], the expression for $E\left[p_1^2[n+1]-p_1^2\left[n\right]\right]$ was already derived. However, in [58], the Gaussian case was assumed, while here, we do not use this assumption. Therefore, in our case, all the higher moments (higher than four) associated with the real part of the convolutional noise have to be obtained differently than in [58]. In addition, the obtained expression for $E\left[p_1^2[n+1]-p_1^2\left[n\right]\right]$ in [58] was set to zero to find the residual ISI applicable in the convergence state, while in our case it is not set to zero. In order to carry out the calculations of $E\left[p_1^2[n+1]-p_1^2\left[n\right]\right]$ and $E\left[p_1^6[n+1]-p_1^6\left[n\right]\right]$, we need the moments up to order ten of the real part of the convolutional noise. Since we can not use the Gaussian assumption but have on hand the fourth moment of the real part of the convolutional noise (41), thus we only need to find a technique that supplies us with all the higher order moments (higher than four) of the real part of the convolutional noise. The quasi moment truncation technique is related to the Hermite polynomials where the high-order central moments are approximated in terms of lower order central moments [48]. Thus, according to the quasi moment truncation technique [48] and with the help of (41) we have:

$$\begin{array}{c}E\left[p_1^6\right]\simeq 15m_{p}E\left[p_1^4\right]-30m_p^3=\left(15-\frac{135}{T}\right)m_p^3\\ \\ E\left[p_1^8\right]\simeq 28m_{p}E\left[p_1^6\right]-210m_p^{2}E\left[p_1^4\right]+315m_p^4=\left(105-\frac{1890}{T}\right)m_p^4\\ \\ E\left[p_1^{10}\right]\simeq 45m_{p}E\left[p_1^8\right]-630m_p^{2}E\left[p_1^6\right]+3150m_p^{3}E\left[p_1^4\right]-3780m_p^5=\left(945-\frac{28350}{T}\right)m_p^5\end{array}$$

(46)

Next, by using (41), (44)–(46), we may write:

$$\begin{array}{c}E\left[p_1^2[n+1]-p_1^2\left[n\right]\right]\simeq \left(-3B^2{a}_{12}^2\left(\frac{3}{T}-1\right)-B^2{a}_3^2\left(\frac{135}{T}-15\right)-\right.\\ \\ \left.6B^2{a}_3{a}_{12}\left(\frac{3}{T}-1\right)\right)m_p^3+\left(2B^2{a}_1{a}_{12}-2B\left(a_{12}-3a_3\left(\frac{3}{T}-1\right)\right)+\right.\\ \\ 6B^2{m}_2{a}_{12}^2+12B^2{a}_3{m}_2{a}_{12}-6B^2{a}_1{a}_3\left(\frac{3}{T}-1\right)-45B^2{a}_3^2{m}_2\left(\frac{3}{T}-1\right)-\\ \\ \left.3B^2{m}_2{a}_{12}^2\left(\frac{3}{T}-1\right)-6B^2{a}_3{m}_2{a}_{12}\left(\frac{3}{T}-1\right)\right)m_p^2+\\ \\ \left(B^2{a}_1^2-2B\left(a_1+3a_3{m}_2+m_2{a}_{12}\right)+15B^2{a}_3^2{m}_4+B^2{m}_4{a}_{12}^2+6B^2{m}_2^2{a}_{12}^2+\right.\\ \\ \left.12B^2{a}_1{a}_3{m}_2+4B^2{a}_1{m}_2{a}_{12}+2B^2{a}_3{m}_4{a}_{12}+12B^2{a}_3{m}_2^2{a}_{12}\right)m_p+\\ \\ \left(B^2{a}_1^2{m}_2+2m_4{B}^2{a}_1{a}_3+2B^2{a}_1{m}_2^2{a}_{12}+m_6{B}^2{a}_3^2+\right.\\ \\ \left.2m_4{B}^2{a}_3{m}_2{a}_{12}+m_4{B}^2{m}_2{a}_{12}^2\right)\end{array}$$

(47)

and:

$$\begin{array}{c}E\left[p_1^6[n+1]-p_1^6\left[n\right]\right]\simeq \left(-15B^2\left(a_3^2\left(\frac{28350}{T}-945\right)-3a_{12}^2\left(\frac{3}{T}-1\right)\left(\frac{135}{T}-15\right)+\right.\right.\\ \\ \left.\left.2a_3{a}_{12}\left(\frac{1890}{T}-105\right)\right)\right)m_p^5+\left(6a_{12}\left(\frac{135}{T}-15\right)-\right.\\ \\ 15B^2\left(6m_2{a}_{12}^2\left(\frac{135}{T}-15\right)+15a_3^2{m}_2\left(\frac{1890}{T}-105\right)-9m_2{a}_{12}^2{\left(\frac{3}{T}-1\right)}^2+\right.\\ \\ 2a_1{a}_{12}\left(\frac{135}{T}-15\right)+2a_1{a}_3\left(\frac{1890}{T}-105\right)+12a_3{m}_2{a}_{12}\left(\frac{135}{T}-15\right)+\\ \\ \left.\left.2a_3{m}_2{a}_{12}\left(\frac{1890}{T}-105\right)\right)+6a_3\left(\frac{1890}{T}-105\right)\right)m_p^4+\\ \\ \left(B\left(6a_1\left(\frac{135}{T}-15\right)+18a_3{m}_2\left(\frac{135}{T}-15\right)+6m_2{a}_{12}\left(\frac{135}{T}-15\right)\right)-\right.\\ \\ 15B^2\left(a_1^2\left(\frac{135}{T}-15\right)+15a_3^2{m}_4\left(\frac{135}{T}-15\right)+m_4{a}_{12}^2\left(\frac{135}{T}-15\right)+\right.\\ \\ 18m_2^2{a}_{12}^2\left(\frac{3}{T}-1\right)+6a_1{m}_2{a}_{12}\left(\frac{3}{T}-1\right)+6a_3{m}_4{a}_{12}\left(\frac{3}{T}-1\right)+12a_1{a}_3{m}_2\left(\frac{135}{T}-15\right)+\\ \\ \left.\left.2a_1{m}_2{a}_{12}\left(\frac{135}{T}-15\right)+12a_3{m}_2^2{a}_{12}\left(\frac{135}{T}-15\right)\right)\right)m_p^3+\\ \\ \left(-15B^2\left(3a_1^2{m}_2\left(\frac{3}{T}-1\right)+3a_3^2{m}_6\left(\frac{3}{T}-1\right)+6a_1{a}_3{m}_4\left(\frac{3}{T}-1\right)+\right.\right.\\ \\ \left.\left.6a_1{m}_2^2{a}_{12}\left(\frac{3}{T}-1\right)+3m_2{m}_4{a}_{12}^2\left(\frac{3}{T}-1\right)+6a_3{m}_2{m}_4{a}_{12}\left(\frac{3}{T}-1\right)\right)\right)m_p^2\end{array}$$

(48)

Based on (42) we may write:

$$2\left(E\left[(p^6\left[n+1\right]-p^6\left[n\right])\right]\right)-\left(E\left[p^2\left[n+1\right]-p^2\left[n\right]\right]\right)105m_p^{2}T\simeq 0$$

(49)

Next, by substituting (47) and (48) into (49) and dividing both sides of the obtained equation by $m_p^2$ for $m_p\ne 0$, we obtain (12). By substituting the solution for $m_p$ obtained in (12) into (5), together with $E\left[x_1^2\left[n\right]\right]$, (11) is obtained. □

4. Simulation

In the previous section we derived a closed-form approximated expression for the residual ISI (11) as a function of the system’s parameter (step-size parameter, input constellation statistics, equalizer’s tap length, channel power and properties of the chosen equalizer via $a_1$, $a_3$ and $a_{12}$) for which the pdf of the real part of the convolutional noise can be approximately considered as a Gaussian pdf. In this section we wish to calculate the residual ISI (11) and see if the obtained level for the residual ISI (11) is above the $-16$ [dB] where the “eye diagram” is still closed. Specifically, when decisions on the equalized output sequence cannot be made with confidence. In the following, we use the channel proposed by [7]: $h_n=\left\{0\mathrm{for}n<0;-0.4\mathrm{for}n=0;0.84·{0.4}^{n-1}\mathrm{for}n>0\right\}$.

Figure 2 describes the averaged ISI and equalized output constellation obtained for Godard’s algorithm [52], where decisions on the equalized output sequence can be made rather reliably. Figure 3 describes the comparison between the simulated ISI with Godard’s algorithm [52] and with those calculated via the expression for the residual ISI given in (11) for two different values for T and using the values for $a_1$, $a_{12}$ and $a_3$ associated with Godard’s algorithm [52]. According to Figure 3, the simulated ISI is above the $-16$ [dB] where decisions on the equalized output sequence cannot be made in a reliable manner as it can be clearly seen from Figure 4. According to Figure 3, the calculated residual ISI (11) for both cases ($T=500$ and $T=1000$) is above the obtained level for the simulated ISI. Thus, we have shown here via simulation that the Gaussian assumption for the convolutional noise pdf can be approximately made just before the equalizer has converged, even at a residual ISI level where no trustworthy judgements can be made. Next, we apply the MMA algorithm [55,56] for the blind adaptive equalization task. Figure 5 describes the comparison between the simulated ISI with the MMA algorithm [55,56] and with those calculated via the expression for the residual ISI given in (11) for two different values for T and using the values for $a_1$, $a_{12}$ and $a_3$ associated with the MMA algorithm [55,56]. According to Figure 5, the simulated ISI is above the $-16$ [dB] where decisions on the equalized output sequence cannot be made in a reliable manner as it can be clearly seen from Figure 6. According to Figure 5, the calculated residual ISI (11) for both cases ($T=500$ and $T=1000$) is above the obtained level for the simulated ISI. Thus, also here, we see via simulation that the Gaussian assumption for the convolutional noise pdf can be approximately made just before the equalizer has converged even at a residual ISI level where no reliable decisions can be carried out. Please note that we used $T=500$ and $T=1000$ since according to (12) the value for T should be $T\gg 100$.

5. Discussion

In this study, a closed-form approximated expression was established for the residual ISI (11) for which the real part of the convolutional noise’s pdf can be approximately regarded as Gaussian. In the previous section, we have shown via simulation that the Gaussian assumption for the real part of the convolutional noise’s pdf can be approximately made just before the equalizer has converged even at a residual ISI level where no reliable decisions can be made unlike it was believed in the literature [41]. Thus, this may be the reason for achieving satisfactory equalization results from an ISI and acquisition perspective from those blind adaptive equalizer’s ([1,2,6,34,40,41,42,43]) based on the Gaussian assumption throughout the entire deconvolution procedure for the convolutional noise pdf.

Let us go back for a moment to the obtained expression for the pdf associated with the real part of the convolutional noise given in (43). As it was already stated, for $m_p\ne 0$ and $T\to \infty$, the convolutional noise pdf given in (43) tends to the Gaussian one. Since $G\simeq \frac{8{\lambda }_2^2}{105T}$, it means that $G\simeq 0$ for $T\to \infty$. Now, based on (18), $G\simeq 0$ for $E\left[p_1^2\left[n+1\right]-p_1^2\left[n\right]\right]\simeq 0$. In [58], the expression for $E\left[p_1^2[n+1]-p_1^2\left[n\right]\right]$ was obtained for the Gaussian case and set to zero to find the residual ISI applicable in the convergence state for the noiseless case. Thus, for $T\to \infty$, the obtained expression for the residual ISI (11) for which the real part of the convolutional noise’s pdf can be approximately regarded as Gaussian is approximately the obtained expression for the residual ISI applicable in the convergence state given in [58]. This may be the reason why having very satisfying results in [58] for the 16QAM case, even for residual ISI levels above $-16$ [dB]. Although we considered in this paper only the 16QAM constellation input, the expression for the residual ISI given in (11) holds also for the real valued input case and for any other input constellation that belongs to the two-independent quadrature carrier case as the 64QAM and 256QAM inputs. Although we used only one channel for the simulation task, the expression for the residual ISI given in (11) holds also for any channel that complies with assumption two from the system description section. As was already pointed out, for $T\to \infty$, the obtained expression for the residual ISI (11) for which the real part of the convolutional noise’s pdf can be approximately regarded as Gaussian is approximately the obtained expression for the residual ISI applicable in the convergence state given in [58]. Eight different channels and three different input sources (16QAM, 64QAM and 4QAM) were considered in [58] for the simulation task. According to [58], very satisfying simulation results were obtained. Thus, in this paper, there was no need to take more channels and different input sources than the 16QAM for the simulation task.