Wavelet Density and Regression Estimators for Functional Stationary and Ergodic Data: Discrete Time

Abstract

The nonparametric estimation of density and regression function based on functional stationary processes using wavelet bases for Hilbert spaces of functions is investigated in this paper. The mean integrated square error over adapted decomposition spaces is given. To obtain the asymptotic properties of wavelet density and regression estimators, the Martingale method is used. These results are obtained under some mild conditions on the model; aside from ergodicity, no other assumptions are imposed on the data. This paper extends the scope of some previous results for wavelet density and regression estimators by relaxing the independence or the mixing condition to the ergodicity. Potential applications include the conditional distribution, curve discrimination, and time series prediction from a continuous set of past values.

1. Introduction

The statistical literature has recently become more interested in statistical issues concerning studying functional random variables or variables with values in an infinite-dimensional space. The availability of data measured on ever-finer temporal/spatial grids, such as in meteorology, medicine, satellite images, and many other research disciplines, is driving the growth of this research issue, statistically modeling these data as random functions revealed many complex theoretical and numerical research challenges. The reader may consult the monographs for a summary of the theoretical and practical aspects of functional data analysis. The work in Bosq [1] concerns linear models for random variables with values in a Hilbert space, Ramsay and Silverman [2] discussed scalar-on-function and function-on-function linear models, functional principal component analysis, and parametric discriminant analysis. The work in [3], on the other hand, concentrates on nonparametric methods, particularly kernel-type estimation for scalar-on-function nonlinear regression models. Such tools were extended to classification and discrimination analysis. Horváth and Kokoszka [4] discussed the application of several interesting statistical concepts to the functional data framework, including goodness-of-fit tests, portmanteau tests, and change point problems. The work in [5] focuses on analyzing variance for functional data, whereas that in [6] is more concerned with regression analysis for Gaussian processes. Recent studies and surveys on functional data modeling and analysis can be found in [7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22].

Motivated by diverse applications and their helpful role in statistical inference, the problem of estimating conditional models has been subjected to a wide range of statistical literature, employing many types of estimation approaches, the most common of which are the traditional kernel methods. Such methods, however, may have some limitations when estimating compactly supported or discontinuous curves at boundary points. Alternative wavelet methods are prominent due to their adaptability to discontinuities in the curve to be estimated. In practice, the wavelet procedure provides a simple estimation algorithm to implement and compute. For more information on wavelets theory, we refer to [23,24,25,26] and others. The work in [27] discusses wavelet approximation properties in detail, and surveys the use of wavelets in various curve estimation problems. Some wavelet theory applications are discussed in [28] by considering the estimation of the integrated squared derivative density function in the independent unidimensional case. The results in [28] were then extended by [29] to estimate the derivatives of a density for negatively and positively associated sequences, respectively. Rao [30] proposed wavelet estimators for the partial derivatives of a multivariate probability density function, where the rates of almost certain convergence for the independence case are obtained. We cite [31] on estimating partial derivatives of a multivariate probability density function in the presence of additive noise. At this point, we refer to [32]. In the i.i.d. framework, Ref. [33] investigated the density and regression estimation problems for functional data. The authors [33] developed a new adaptive procedure based on the term-by-term selection of wavelet coefficient estimators using wavelet bases for Hilbert spaces of functions. The primary goal of this paper is to extend the previous reference to stationary ergodic processes. To our knowledge, the consideration of the general dependence framework for wavelet analysis is unexplored, which motivates this study. Allow $\left\{{\mathbf{X}}_n,n\in \mathbb{Z}\right\}$ to be a stationary sequence. Consider the backward field ${\mathcal{A}}_n=\sigma \left({\mathbf{X}}_k:k\le n\right)$ and the forward field ${\mathcal{B}}_n=\sigma \left({\mathbf{X}}_k:k\ge n\right)$. The sequence is strongly mixing if

$$\underset{A\in {\mathcal{A}}_0,B\in {\mathcal{B}}_n}{sup}|\mathbb{P}(A\cap B)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)|=\alpha \left(n\right)\to 0\mathrm{as}n\to \infty .$$

The sequence is ergodic if

$$\underset{n\to \infty }{lim}\frac{1}{n}\sum _{k=0}^{n-1}\left|\mathbb{P}\left(A\cap {\tau }^{-k}B\right)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\right|=0,$$

where $\tau$ is the time-evolution or shift transformation. The naming of strong mixing in the above definition is more stringent than what is ordinarily referred to (when using the vocabulary of measure-preserving dynamical systems) as strong mixing, namely to that

$$\underset{n\to \infty }{lim}\mathbb{P}\left(A\cap {\tau }^{-n}B\right)=\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)$$

for any two measurable sets $A,B$, see, for instance, Ref. [34]. As a result, strong mixing implies ergodicity, whereas the converse is not always true (see, for example, Remark 2.6 on page 50 concerning Proposition 2.8 on page 51 in [35]). Some reasons for considering ergodic dependence structure in data rather than a mixing structure are discussed in [36,37,38,39,40,41,42,43,44], where details on the definition of ergodic property of processes are given, as well as illustrative examples of such processes. One of the arguments used in [45] to justify the ergodic setting is that for certain classes of processes, proving ergodic properties rather than the mixing condition can be much easier. As a result, the ergodicity hypothesis appears to be the best fit and offers a better framework for studying data series generated by noisy chaos. The work in [45] provided an example of an ergodic but non-mixing process in their discussion, which can be summarized as follows: Let $(T_i,{\lambda }_i):i\in \mathbb{Z}$ be a strictly stationary process such that $T_i\mid {\mathcal{T}}_{i-1}$ is a Poisson process with parameter $\lambda i$, where ${\mathcal{T}}_i$ is the $sigma$-field generated by $(T_i,{\lambda }_i,T_{i-1},\dots )$. Assume that

$${\lambda }_i=f({\lambda }_{i-1},T_{i-1}),$$

and $f:[0,\infty ]\times \mathbb{N}\to (0,\infty )$ is a given function. This process is not mixing in general (see Remark 3 of [46]). It is known that any sequence ${\left({\epsilon }_i\right)}_{i\in \mathbb{Z}}$ of i.i.d. random variables is ergodic. Hence, according to Proposition 2.10 in [35], it is easy to see that ${\left(Y_i\right)}_{i\in \mathbb{Z}}$ with

$$Y_i=\vartheta ((\dots ,{\epsilon }_{i-1},{\epsilon }_i),({\epsilon }_{i+1},{\epsilon }_{i+2},\dots )),$$

for some Borel-measurable function $\vartheta (·)$.

The primary goal of this paper is to provide the first complete theoretical justification for wavelet-based functional density and regression function estimation for stationary processes. To our knowledge, the mean integrated square error over adapted decomposition spaces using wavelet estimators in functional ergodic data frameworks has not yet been considered in the literature and thus remains a fundamentally unsolved open problem. By combining several Martingale theory techniques used in the mathematical development of the proofs, we hope to fill this gap in the literature. These tools are not the same as those used in regression estimation under strong mixing or in an independent setting. However, as we will see later, combining existing ideas and results is not enough to solve the problem. Dealing with wavelet estimators in an ergodic setting will involve detailed mathematical derivations.

The following is how the paper is structured. The multiresolution analysis is introduced in Section 2. The main results for density estimation are presented in Section 3. The main results for the regression estimation are presented in Section 4. Some potential applications are listed in Section 5. Section 6 contains some concluding remarks. Section 7 contains a collection of all proofs.

2. Multiresolution Analysis

We will now introduce some basic notation to define wavelet bases for Hilbert spaces of functions following [33,47] with some changes necessary for our setting. In our work, we consider nonlinear, thresholded, wavelet-based estimators. Firstly, we initiate our study by describing elements of the basic theory of wavelet methods and introducing nonlinear wavelet-based estimators; the interested reader may refer to [23,24], see also [48,49] and the references therein, although the wavelet bases on a separable Hilbert space $\mathbf{H}$ of real or complex-valued functions on a complete separable metric space was introduced later by [47], which we briefly recall here for the sake of reader’s convenience. Let $\mathbf{H}$ be a separable Hilbert space of real-valued functions defined on a complete separable metric space $\mathbf{S}$. Since the space $\mathbf{H}$ is separable, it has an orthonormal basis

$$\mathcal{E}=\left\{e_j:j\in \Delta \right\},$$

where $\Delta$ is a countable index set. The space $\mathbf{H}$ is equipped with an inner product $\langle ·,·\rangle$ and a norm $\Vert ·\Vert$.

Consider the sequence of subsets $\left\{{\mathcal{I}}_k;k\ge 0\right\}$ an increasing sequence of finite subsets of $\Delta$ such that

$$\bigcup _{k\ge 0}{\mathcal{I}}_k=\Delta .$$

The subset ${\mathcal{J}}_k$ denotes the orthogonal complement of ${\mathcal{I}}_k$ in ${\mathcal{I}}_{k+1}$, i.e.,

$${\mathcal{J}}_k={\mathcal{I}}_{k+1}{\mathcal{I}}_k.$$

Choose, for any $k\ge 0$, ${\zeta }_{k,\ell }\in \mathbf{S}$, $\ell \in {\mathcal{I}}_k$ and ${\eta }_{k,\ell }\in \mathbf{S}$, $\ell \in {\mathcal{J}}_k$, such that the following matrices

$$A_k={\left(e_j\left({\zeta }_{k,\ell }\right)\right)}_{(j,\ell )\in {\mathcal{I}}_k\times {\mathcal{I}}_k},B_k={\left(e_j\left({\eta }_{k,\ell }\right)\right)}_{(j,\ell )\in {\mathcal{J}}_k\times {\mathcal{J}}_k},$$

(1)

satisfy one of the two following conditions, for instance, see [33,47] and the references therein.

(A.1)

$A_k^{*}{A}_k=\mathrm{diag}{\left(a_{k,\ell }\right)}_{\ell \in {\mathcal{I}}_k}$ and $B_k^{*}{B}_k=\mathrm{diag}{\left(b_{k,\ell }\right)}_{{\ell }^{\prime }\in {\mathcal{J}}_k}$ where $a_{k,\ell }$ and $b_{k,\ell }$ for $\ell \in {\mathcal{I}}_k$ and ${\ell }^{\prime }\in {\mathcal{J}}_k$ are positive constants.

(A.2)

$A_k{A}_k^{*}=\mathrm{diag}{\left(c_{k,\ell }\right)}_{\ell \in {\mathcal{I}}_k}$ and $B_k{B}_k^{*}=\mathrm{diag}{\left(d_{k,\ell }\right)}_{{\ell }^{\prime }\in {\mathcal{J}}_k}$ where $c_{k,\ell }$ and $d_{k,\ell }$ for $\ell \in {\mathcal{I}}_k$ and ${\ell }^{\prime }\in {\mathcal{J}}_k$ are positive constants.

The condition (A.1) implies that

$$a_{k,\ell }=\sum _{j\in {\mathcal{I}}_k}|e_j\left({\zeta }_{k,\ell }\right){|}^2,\ell \in {\mathcal{I}}_k,\mathrm{and}{b}_{k,\ell }=\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2,\ell \in {\mathcal{I}}_k,$$

(2)

which means that all the columns of $A_k$ and $B_k$ are not the zero vector. As for (A.2), it gives

$$c_{k,\ell }=\sum _{\ell \in {\mathcal{I}}_k}|e_j\left({\zeta }_{k,\ell }\right){|}^2,j\in {\mathcal{I}}_k,\mathrm{and}{d}_{k,\ell }=\sum _{\ell \in {\mathcal{J}}_k}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2,j\in {\mathcal{I}}_k,$$

(3)

indicating that all the rows of $A_k$ and $B_k$ are not the zero vector. For any $\mathbf{x}\in \mathbf{S}$, we set

$$\left\{\begin{array}{c}{\displaystyle {\varphi }_k(·;{\zeta }_{k,\ell })=\sum _{j\in {\mathcal{I}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}\overline{e_j\left({\zeta }_{k,\ell }\right)}{e}_j(·),}\\ \\ {\displaystyle {\psi }_k(·;{\eta }_{k,\ell })=\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{h_{j,k,\ell }}}\overline{e_j\left({\eta }_{k,\ell }\right)}{e}_j(·),}\end{array}\right.$$

(4)

where

$$g_{j,k,\ell }=\left\{\begin{array}{c}{a}_{k,\ell }\mathrm{if}(\mathbf{A}.\mathbf{1}),\\ \\ c_{k,\ell }\mathrm{if}(\mathbf{A}.\mathbf{2}),\end{array}\right.h_{j,k,\ell }=\left\{\begin{array}{c}{b}_{k,\ell }\mathrm{if}(\mathbf{A}.\mathbf{1}),\\ \\ d_{k,\ell }\mathrm{if}(\mathbf{A}.\mathbf{2}),\end{array}\right.$$

(5)

The following collection form an orthonormal basis of $\mathbf{H}$ (see Theorem 2 of [47]):

$$\mathcal{B}=\left\{{\varphi }_0\left(x,{\zeta }_{0,\ell }\right),\ell \in {\mathcal{I}}_0;{\psi }_k\left(x,{\eta }_{k,\ell }\right),k\ge 0,\ell \in {\mathcal{J}}_k\right\}.$$

(6)

For more details, see [33,47,50]. Hence, we conclude that for any $f\in \mathbf{H}$, we have

$$f\left(\mathbf{x}\right)=\sum _{\ell \in {\mathcal{I}}_0}{\alpha }_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k\ge 0}\sum _{\ell \in {\mathcal{J}}_k}{\beta }_{k,\ell }{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(7)

where

$${\alpha }_{0,\ell }=\langle f,{\varphi }_0\left(·;{\zeta }_{0,\ell }\right)\rangle ,{\beta }_{k,\ell }=\langle f,{\psi }_k\left(·;{\eta }_{k;\ell }\right)\rangle .$$

(8)

In the following, we add two assumptions on the orthonormal basis $\mathcal{E}$:

(E.1) 

There exists a constant $C_1>0$ such that, for any integer $k\ge 0$, one has

(i)

$$\sum _{j\in {\mathcal{I}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\le C_1,$$

(ii)

$$\sum _{j\in {\mathcal{I}}_k}\frac{1}{h_{j,k,\ell }}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2\le C_1.$$

(E.2)

There exists a constant $C_2>0$ such that, for any integer $k\ge 0$, one has

$$\underset{\mathbf{x}\in \mathcal{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\le C_2\left|{\mathcal{J}}_k\right|.$$

Remark 1.

Clearly, one can see that assumption (E.1)is satisfied under assumption (A.1)when taking $C_1=1$, we may also refer to [47], Section 4, Example 2 and its applications for more details. [47,50] have presented three examples verifying the assumption (E.2)taking

$$\underset{\mathbf{x}\in \mathcal{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\le 1,$$

see also [50] Theorem 3.2. Moreover, Ref. [33] have used both assumptions in the case of i.i.d and functional data.

Besov Space

Over the years, many researchers have addressed, from a statistical point of view, the following question: given an estimation method and a prescribed estimation rate for a given loss function, what is the maximal space over which this rate is achieved, for instance, see [27,51] and the references therein. We are interested in the estimation methods based on wavelet bases’ thresholding procedures in a natural setting. It is well known that wavelet bases provide characterizations of smoothness spaces such as the Hölder spaces $C^s$, Sobolev spaces $W^s\left(L_p\right)$, and Besov spaces $B_q^s\left(L_p\right)$ for a range of indices s that depend both on the smoothness properties of $\psi$ and its dual function $\tilde{\psi }$, for instance, we refer to [51] for more detail and examples, at this point, we may refer to [52]. From a statistical point of view, the following definition is used in approximation theory for the study of nonlinear procedures such as thresholding and greedy algorithms, for instance we refer to [27,49,51,53].

Definition 1

(Besov space). Let $s>0$. We say that the function $f\in \mathbf{H}$, defined by (7), belongs to the Besov space ${\mathcal{B}}_{\infty }^s\left(\mathbf{H}\right)$ if and only if:

$$\underset{m\ge 0}{sup}|{\mathcal{J}}_m{|}^{2s}\sum _{k\ge m}\sum _{\ell \in {\mathcal{J}}_k}{\left|{\beta }_{k,\ell }\right|}^2<\infty .$$

(9)

Definition 2

(Weak Besov space). Let $r>0$. We say that the function $f\in \mathbf{H}$, defined by (7), belongs to the weak Besov space ${\mathcal{W}}^r\left(\mathbf{H}\right)$ if and only if:

$$\underset{\lambda \ge 0}{sup}{\lambda }^r\sum _{k\ge 0}\sum _{\ell \in {\mathcal{J}}_k}{\mathrm{𝟙}}_{\left\{|{\beta }_{k,\ell }|\ge \lambda \right\}}<\infty .$$

(10)

3. Problem Definition of the Density Estimation

Let ${\{{\mathbf{X}}_i,Y_i\}}_{i\ge 1}$ be a sequence strictly stationary ergodic pairs of random elements where $Y_i$ is a real or complex-valued variable and ${\mathbf{X}}_i$ takes values in a complete separable metric space of Hilbert space $\mathbf{S}$ associated with the corresponding Borel $\sigma$-algebra $\mathcal{B}$. Let ${\mathbb{P}}_{\mathbf{X}}$ be the probability measure induced by ${\mathbf{X}}_1$ on $(\mathbf{S},\mathcal{B})$. Suppose that there exists $\sigma$-finite measure $\nu$ on the measurable space $(\mathbf{S},\mathcal{B})$ such that ${\mathbb{P}}_{\mathbf{X}}$ is dominated by $\nu$. The Radon–Nikodym theorem ensures the existence of a non-negative measurable function $f(·)$ such that

$${\mathbb{P}}_{\mathbf{X}}\left(B\right)={\int }_{B}f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right),B\in \mathcal{B}.$$

(11)

In this context, we aim to estimate $f(·)$ based on n observed functional data ${\mathbf{X}}_1,\dots ,{\mathbf{X}}_n$– Examples of such random elements ${\mathbf{X}}_1$ are stochastic processes with continuous sample paths on a finite interval $[a,b]$ with $\mathbf{S}=C[a,b]$ associated with supremum norm and processes with square integrable sample paths on the real line when $\mathbf{S}=L_2\left(\mathbb{R}\right)$. We suppose that $f\in \mathbf{H}$, where $\mathbf{H}$ is a separable Hilbert space of real or complex-valued functions defined on $\mathbf{S}$ and square-integrable with respect to the $\sigma$-finite measure $\nu$. In this paper, we are particularly interested in the wavelet estimation procedures developed in the 1990s; see Meyer’s work for the functional data of a Hilbert space, more precisely, the nonlinear estimators. The majority of the approaches carried out in this model consist in introducing kernel estimators techniques to estimate the model’s functional part, refer to [54]. Let $f(·)$ be the common density function of the sample ${\mathbf{X}}_1,\dots ,{\mathbf{X}}_n$, which is assumed to be

(F.1) 

$\exists C_f>0$ a known constant such that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}f\left(\mathbf{x}\right)\le C_f.$$

(12)

3.1. Density Function Estimator

From now, we assume that the density function $f(·)\in \mathbf{H}$, a separable Hilbert space. Then, $f(·)$ fulfills the wavelet representation (7). Suppose that we observe a sequence ${\left\{({\mathbf{X}}_i,Y_i)\right\}}_{i=1}^n$ of copies of $(\mathbf{X},Y)$ that is assumed to be functional stationary and ergodic with $\mathbf{X}$ admitting the density function $f(·)$. We study density estimation through wavelet bases for Hilbert spaces of functions developed by [47]. We consider the estimates of the coefficients $\left\{{\alpha }_{k,\ell }\right\}$ and $\left\{{\beta }_{k,\ell }\right\}$ given, respectively, by (14) and (15). For any $j_0\le m$. Here, the resolution level $m=m\left(n\right)\to \infty$ at a rate specified below, since we assume that $\varphi (·)$ and ${\psi }_i(·)$ have a compact support so that the summations in (7) are finite for each fixed $\mathbf{x}$ (note that in this case the support of $\varphi (·)$ and ${\psi }_i(·)$ is a monotonically increasing function of their degree of differentiability [24]). We focus our attention on the nonlinear estimators (13) which will be studied in the mean integrated squared error over adapted decomposition spaces, in a similar way as in [33] in the setting of the i.i.d functional processes. The density wavelet hard thresholding estimator $\widehat{f}(·)$ is defined, for all $\mathbf{x}\in \mathbf{S}$, by

$${\widehat{f}}_n\left(\mathbf{x}\right)=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\alpha }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\beta }}_{k,\ell }{\mathrm{𝟙}}_{\left\{|{\widehat{\beta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(13)

where

$$\begin{array}{ccc}\hfill {\widehat{\alpha }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

(15)

Here, $\kappa$ is a large enough constant and $m_n$ is the integer satisfying

$$\frac{1}{2}\frac{n}{lnn}\le \left|{\mathcal{J}}_{m_n}\right|\le \frac{n}{lnn}.$$

(16)

3.2. Estimation Procedure Steps

Our estimation method is divided into three steps:

1.

Estimation of the wavelet coefficients ${\alpha }_{k,\ell }$ and ${\beta }_{k,\ell }$, see (8), by the estimators ${\widehat{\alpha }}_{k,\ell }$ and ${\widehat{\beta }}_{k,\ell }$ defined by Equations (14) and (15);

2.

Applying the hard thresholding to select the greatest ${\widehat{\beta }}_{k,\ell }$;

3.

Reconstructing the selected elements of the initial wavelet basis.

It is important to note that our choice is the universal threshold $\kappa {\left(\frac{lnn}{n}\right)}^{1/2}$ and the definition of $m_n$ is based on theoretical considerations. The considered estimator does not depend on the smoothness of $f(·)$; we may refer the reader to [50] for more details in the case of the linear wavelet estimator of $f(·)$. Furthermore, for more details on the case of $\mathbf{H}=\mathbb{L}\left(\right[a,b\left]\right)$ and more standard nonparametric models, see [27,55]. To state the results, we need some notation. Throughout the paper, we will denote by ${\mathcal{F}}_i$ the $\sigma -$field generated by $\{{\mathbf{X}}_j:0\le j\le i\}$ and ${\mathcal{G}}_i$ the $\sigma -$field generated by $\{({\mathbf{X}}_j,Y_j):{\mathbf{X}}_{i+1},0\le j\le i\}$. Let $B\in \mathcal{B},$ be an open set of the Borel $\sigma -$algebra $\mathcal{B}$. For any $i=1,\dots ,n$ define $f_{{\mathbf{X}}_i}^{{\mathcal{F}}_{i-1}}(·)$ as the conditional density of ${\mathbf{X}}_i$ given the $\sigma -$ field ${\mathcal{F}}_{i-1}$. Define

$$F_{{\mathbf{X}}_i}=\mathbb{P}\left({\mathbf{X}}_i\in B\right)={\mathbb{P}}_X\left(B\right),\mathrm{see} \left(11\right),$$

and

$$F_{{\mathbf{X}}_i}^{{\mathcal{F}}_{i-1}}=\mathbb{P}\left({\mathbf{X}}_i\in B|{\mathcal{F}}_{i-1}\right)$$

as the distribution function and the conditional distribution function, given the $\sigma -$field ${\mathcal{F}}_{i-1}$, respectively. The following assumptions will be needed throughout the paper.

(C.0) 

There is a non-negative measurable function $f^{{\mathcal{F}}_{i-1}}$ such that

$${\mathbb{P}}_{\mathbf{X}}^{{\mathcal{F}}_{i-1}}\left(B\right)={\int }_B{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right),B\in \mathcal{B}.$$

(17)

(C.1) 

For any $\mathbf{x}\in \mathbf{S}$

$$\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=1}^n{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)=f\left(\mathbf{x}\right),\mathrm{in} \mathrm{the}\mathrm{a}.\mathrm{s}.\mathrm{and}{L}^2\mathrm{sense}.$$

At this point, we may refer to [56] for further details.

Theorem 1.

Under the conditions (C.0), (C.1), (F.1), (E.1)and (E.2), and (16) for any $\theta \in (0,1)$,

$$f\in {\mathcal{B}}_{\infty }^{\theta /2}\left(H\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(H\right),$$

there exists a constant $C_1>0$ in such a way that

$$\mathbb{E}\left({∥\widehat{f}\left(\mathbf{x}\right)-f\left(\mathbf{x}\right)∥}^2\right)\le C_1{\left(\frac{lnn}{n}\right)}^{\theta },$$

(18)

for large enough n.

A direct consequence is the following upper bound result: for $s>0$, if

$$f\in {\mathcal{B}}_{\infty }^{s/(2s+1)}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2/(2s+1)}\left(\mathbf{H}\right),$$

then there exists a constant $C_2>0$ such that

$$\mathbb{E}\left(\parallel \widehat{f}{-f\parallel }^2\right)\le C_2{\left(\frac{lnn}{n}\right)}^{2s/(2s+1)}.$$

This rate of convergence corresponds to the near optimal one in the “standard” minimax setting (see, e.g., [27]). Moreover, applying [[49], Theorem 3.2], one can see that ${\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)\cap$${\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right)$ is the “maxiset” associated to $\widehat{f}(·)$ at the rate of convergence ${(lnn/n)}^{\theta }$, i.e.,

$$\underset{n\to \infty }{lim}{\left(\frac{n}{lnn}\right)}^{\theta }\mathbb{E}\left(\parallel \widehat{f}{-f\parallel }^2\right)<\infty \iff f\in {\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right).$$

4. Problem Definition of the Regression Estimation

For a measurable function $\rho :{\mathbb{R}}^q\to \mathbb{R}$, we define the regression function $\mathbf{m}(·,\rho )$ by

$$\rho \left(Y\right)=\mathbf{m}(\mathbf{X},\rho )+ϵ,$$

(19)

where $ϵ$ is a random variable independent of $\mathbf{X}$ with $\mathcal{N}(0,1)$. We suppose that $\mathbf{m}(·,\rho )\in \mathbf{H}$, where $\mathbf{H}$ is a separable Hilbert space of real or complex-valued functions defined on $\mathbf{S}$ and square integrable with respect to the $\sigma$-finite measure $\nu$. We shall suppose that there exist a known constant and $C_{\mathbf{m}}>0$ such that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}\mathbf{m}(\mathbf{x},\rho )\le C_{\mathbf{m}}.$$

(20)

In this context, we redefine the probability measure ${\mathbb{P}}_{\mathbf{X}}$ in (11) and suppose that $f(·)$ is a non-negative measurable known function.

(M.1) 

We shall suppose that there exist two known constant $C_{\mathbf{m}}>0$ such that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}\mathbf{m}(\mathbf{x};\rho )\le C_{\mathbf{m}}.$$

(M.2) 

We shall suppose that there exist two known constant $c_f>0$ such that

$$\underset{\mathbf{x}\in \mathbf{S}}{inf}f\left(\mathbf{x}\right)\ge c_f.$$

Regression Function Estimator

In this context, we aim to estimate $\mathbf{m}(·,\rho )$ based on n observed functional data $({\mathbf{X}}_1,Y_1),\dots ,({\mathbf{X}}_n,Y_n)$. The kernel estimator of the regression function for functional data was proposed by [57]

$${\widehat{\mathbf{m}}}_{n;h_n}(\mathbf{x},\rho ):=\frac{{\displaystyle \sum _{i=1}^n\rho \left(Y_i\right)K(d(\mathbf{x},{\mathbf{X}}_i)/h_n)}}{{\displaystyle \sum _{i=1}^{n}K(d(\mathbf{x},{\mathbf{X}}_i)/h_n)}}.$$

As combined with the work of [55], we define the wavelet hard thresholding estimator $\widehat{\mathbf{m}}(·,\rho )$, for all $\mathbf{x}\in \mathbf{S}$, by

$$\widehat{\mathbf{m}}(\mathbf{x},\rho )=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\theta }}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\widehat{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(21)

where

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \end{array}$$

(22)

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

(23)

where $\kappa$ is large enough constant and $m_n$ is the integer satisfying

$$\frac{1}{2}\frac{n}{{(lnn)}^2}\le \left|{\mathcal{J}}_{m_n}\right|\le \frac{n}{{(lnn)}^2}.$$

(24)

Theorem 2.

Under the conditions (E.1), (E.2) (M.1)–(M.2), (C.0)and (C.1), combined with the assumption (24), for any $\theta \in (0,1)$, $\mathbf{m}(·,\rho )\in {\mathcal{B}}_{\infty }^{\theta /2}\left(H\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(H\right)$, there exists a constant $C_3>0$ in such a way that

$$\mathbb{E}\left({∥\widehat{\mathbf{m}}(·,\rho )-\mathbf{m}(·;\rho )∥}^2\right)\le C{\left(\frac{lnn}{n}\right)}^{\theta },$$

(25)

for large enough n.

Suppose that $\mathbf{m}(·,\rho )$ and f satisfy (M.1) and, for any $\theta \in (0,1),\mathbf{m}(·,\rho )\in {\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right)$, where ${\mathcal{B}}_{\infty }^{\theta /2}\left(\mathbf{H}\right)$ is in (Definition 1) with $s=\theta /2$ and ${\mathcal{W}}^{2(1-\theta )}\left(\mathbf{H}\right)$ is in (Definition 2) with $r=2(1-\theta )$. Then, there exists a constant $C_3>0$ such that

$$\mathbb{E}\left({∥\widehat{\mathbf{m}}(·,\rho )-\mathbf{m}(·;\rho )∥}^2\right)\le C_3{\left(\frac{{(lnn)}^2}{n}\right)}^{\theta }$$

for n large enough. Again, note that, for $s>0$, if

$$\mathbf{m}(·,\rho )\in {\mathcal{B}}_{\infty }^{s/(2s+1)}\left(\mathbf{H}\right)\cap {\mathcal{W}}^{2/(2s+1)}\left(\mathbf{H}\right),$$

then there exists a constant $C_4>0$ such that

$$\mathbb{E}\left({∥\widehat{\mathbf{m}}(·,\rho )-\mathbf{m}(·;\rho )∥}^2\right)\le C_4{\left(\frac{{(lnn)}^2}{n}\right)}^{2s/(2s+1)}.$$

Up to an additional logarithmic term, this rate of convergence corresponds to the near-optimal one in the “standard” minimax setting (see, for example, [27]). Theorem 2 is the first to investigate an adaptive wavelet-based estimator for functional data in the context of nonparametric regression for ergodic processes.

Since the coefficients defined by (22) and (23) depend on the unknown function $f(·)$, one can use

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\rho \left(Y_i\right)}{\widehat{f}\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \end{array}$$

(26)

$$\begin{array}{ccc}\hfill {\tilde{\theta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\rho \left(Y_i\right)}{\widehat{f}\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

(27)

We define the wavelet hard thresholding estimator $\tilde{\mathbf{m}}(·,\rho )$, for all $\mathbf{x}\in \mathbf{S}$, by

$$\tilde{\mathbf{m}}(\mathbf{x},\rho )=\sum _{\ell \in {\mathcal{I}}_0}{\tilde{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\tilde{\theta }}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\tilde{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right).$$

(28)

Recall the following elementary observation

$$\frac{{\displaystyle 1}}{{\displaystyle \widehat{f}(·)}}=\frac{{\displaystyle 1}}{{\displaystyle f(·)}}+\frac{{\displaystyle (f(·)-\widehat{f}(·))}}{{\displaystyle f(·)\widehat{f}(·)}}.$$

From the last equation, we can infer the following:

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& {\widehat{\eta }}_{k,\ell }+\frac{1}{n}\sum _{i=1}^n\frac{(f\left({\mathbf{X}}_i\right)-\widehat{f}\left({\mathbf{X}}_i\right))}{f\left({\mathbf{X}}_i\right)\widehat{f}\left({\mathbf{X}}_i\right)}\rho \left(Y_i\right){\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \\ \hfill {\tilde{\theta }}_{k,\ell }& =& {\widehat{\theta }}_{k,\ell }+\frac{1}{n}\sum _{i=1}^n\frac{(f\left({\mathbf{X}}_i\right)-\widehat{f}\left({\mathbf{X}}_i\right))}{f\left({\mathbf{X}}_i\right)\widehat{f}\left({\mathbf{X}}_i\right)}\rho \left(Y_i\right){\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

By combining Theorem 1 with Theorem 2, we obtain the following corollary.

Corollary 1.

Under the conditions of Theorems 1 and 2, there exists a constant $C_5>0$ such that

$$\mathbb{E}\left({∥\tilde{\mathbf{m}}(\mathbf{x};\rho )-\mathbf{m}(\mathbf{x};\rho )∥}^2\right)\le C_5{\left(\frac{lnn}{n}\right)}^{\theta },$$

(29)

for large enough n.

Remark 2.

In our previous paper [40], we were concerned with the nonparametric estimation of the density and the regression function in a finite-dimensional setting using orthonormal wavelet bases. Our findings differ significantly from those presented in the present paper. In [40], we provided the strong uniform consistency properties with rates of these estimators, over compact subsets of ${\mathbb{R}}^d$, under a general ergodic condition on the underlying processes. We also establish the asymptotic normality of wavelet-based estimators. We used the Burkholder–Rosenthal inequality as the main ingredient in this paper, which is a more complicated tool than the exponential inequality used in the previous paper. More importantly, in the present paper, we look into the mean integrated square error over compact subsets, which is entirely different from the results of the previous paper.

5. Applications

5.1. The Conditional Distribution

Our result can be used to investigate the conditional distribution $F(y\mid x)$ for $y\in {\mathbb{R}}^d$. To be more precise, let $\rho \left(y\right)=\mathrm{𝟙}\{y\le t\}$. We define the wavelet hard thresholding estimator $\breve{\mathbf{m}}(·,\rho )$, for all $\mathbf{x}\in \mathbf{S}$, by

$$\widehat{F}(y\mid x)=\sum _{\ell \in {\mathcal{I}}_0}{\breve{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\breve{\theta }}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\breve{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(30)

where

$$\begin{array}{ccc}\hfill {\breve{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\mathrm{𝟙}\{Y_i\le t\}}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \end{array}$$

(31)

$$\begin{array}{ccc}\hfill {\breve{\theta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\mathrm{𝟙}\{Y_i\le t\}}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

(32)

A direct consequence of Theorem 2 is

$$\mathbb{E}\left({∥\widehat{F}(y\mid x)-F(y\mid x)∥}^2\right)\le C{\left(\frac{lnn}{n}\right)}^{\theta }.$$

(33)

5.2. The Curve Discrimination

We can state the curve discrimination problem in the following way. Let ${\left\{{\mathbf{X}}_i\right\}}_{i=1,\dots ,n}$ be a sample of curves, and each of them is known to belong to one among $\mathbf{G}$ groups $\iota =1,\dots ,\mathbf{G}$. Let us denote by $T_i$ the group of the curve ${\mathbf{X}}_i$. Assume that each pair of variables $\left({\mathbf{X}}_i,T_i\right)$ has the same distribution as the pair $(\mathbf{X},T)$. Given a new curve $\mathbf{x}$, the question is to know its class membership, and for that, we will estimate, for any $\iota =1,\dots ,\mathbf{G}$, the conditional probability:

$$p_{\iota }\left(x\right)=\mathbb{P}(T=\iota \mid \mathbf{X}=\mathbf{x}).$$

Following the idea proposed in [58,59] permitting the estimation of these probabilities by

$${\widehat{p}}_{\iota }\left(x\right)=\sum _{\ell \in {\mathcal{I}}_0}{\breve{\stackrel{˘}{\eta }}}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\breve{\stackrel{˘}{\theta }}}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\breve{\stackrel{˘}{\theta }}}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(34)

where

$$\begin{array}{ccc}\hfill {\breve{\stackrel{˘}{\eta }}}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\mathrm{𝟙}\left\{T_i{=}_{\iota }\right\}}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \end{array}$$

(35)

$$\begin{array}{ccc}\hfill {\breve{\stackrel{˘}{\theta }}}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\mathrm{𝟙}\left\{T_i{=}_{\iota }\right\}}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

(36)

A direct consequence of Theorem 2 is

$$\mathbb{E}\left({∥\widehat{F}(y\mid x)-F(y\mid x)∥}^2\right)\le C{\left(\frac{lnn}{n}\right)}^{\theta }.$$

(37)

As remarked by [58,59], for each $\iota$, we make use of the notation

$$Y=\left\{\begin{array}{cc}1\hfill & \mathrm{if}T=\iota \hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

then, we can write

$$p_{\iota }\left(x\right)=\mathbb{E}(Y\mid X=x).$$

An application of Theorem 2 is

$$\mathbb{E}\left({∥{\widehat{p}}_{\iota }\left(x\right)-p_{\iota }(x∥}^2\right)\le C{\left(\frac{lnn}{n}\right)}^{\theta }.$$

5.3. Time Series Prediction from Continuous Set of Past Values

A direct consequence of our results is the prediction of future values of some real time series, which we will follow from [59]. One of the purposes of the proposed functional method is to predict the future from a continuous set of past values of the process. Let ${\left\{Z\left(t\right)\right\}}_{t\in \mathbb{R}}$ denote a real-valued process and s denote a fixed non-negative real number. We are interested in the prediction’s problem of a future value $Z\left(\tau \right)$ given some past values $Z\left(t\right)$ for $\tau -T\le t<\tau$, at some time $\tau >0.$ Our goal can be seen as the estimation of the operator r:

$$Z(\tau +s)=r\left(Z\right(t)\mathrm{for}\tau -T\le t<\tau )+\epsilon ,$$

whenever it exists. Let us describe the model. Suppose that the process has been observed from $t=0$ until $t=t_{max}$ and without loss of generality, assume that

$$t_{max}=nT+s<\tau .$$

The methodology consists of splitting the observed process into n pieces of fixed length. Each piece of the process is denoted by

$${\mathbf{X}}_i=\{Z\left(t\right),(i-1)T\le t<iT\}.$$

Let us denote the response value $Y_i=Z(iT+s)$. This can be formulated by a regression problem in the following way:

$$Y_i=\mathbf{m}\left({\mathbf{X}}_i,Id\right)+{\epsilon }_i\mathrm{for}i=1,\dots ,n,$$

where $Id$ denotes the identity function. To fit the theoretical setting of the present paper, we assume that such a function $\mathbf{m}\left(·,Id\right)$ does not depend on i; this is the case for the stationary processes. Hence, at time $\tau$, we can use for predicting the value at time $\tau +s$ the following predictor, which is directly derived from (21):

$$\widehat{\mathbf{m}}(\mathbf{x},Id)=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\theta }}_{k,\ell }{\mathrm{𝟙}}_{{\displaystyle \left\{|{\widehat{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

(38)

where

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{Z(iT+s)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),\hfill \end{array}$$

(39)

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{Z(iT+s)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right),\hfill \end{array}$$

(40)

where $\mathbf{x}=\left\{Z\right(t)$, for $\tau -T\le t<\tau \}$. Our Theorem 2 gives mathematical support to this nonparametric functional predictor, and provides a different way of solving the prediction problem investigated in [3,59].

6. Concluding Remarks

In this work, we have investigated the nonparametric estimation of the density and the regression function based on the functional stationary processes, using wavelet bases for Hilbert spaces of functions. We have characterized the mean integrated square error over compact subsets. The asymptotic properties of these estimators are obtained employing the Martingale approach, which is completely different from the mixing and the independent setting. The assumption on the dependence of the process is ergodicity. To motivate the present paper, we have presented the conditional distribution, the curve discrimination and the time series prediction from a continuous set of past values. Extending the nonparametric functional ideas to the local stationary process is a somewhat underdeveloped field. It would be interesting to extend our work to the case of the functional local stationary process, which requires nontrivial mathematics; this would go well beyond the scope of the present paper.

7. Proofs

7.1. Proof of Theorem 1

In this paper, we need an upper bound inequality for partial sums of unbounded martingale differences that we use to derive the asymptotic results for the density and the regression functions estimates built upon functional strictly stationary and ergodic data. Here and in the sequel, we denote by “C” a positive constant that may be different from line to line. This inequality is given in the following lemmas. This lemma is stated following Notation 1 in [60].

Lemma 1

(Burkholder-Rosenthal inequality). Let ${\left(X_i\right)}_{i\ge 1}$ be a stationary Martingale adapted to the filtration ${\left({\mathcal{F}}_i\right)}_{i\ge 1}$, define ${\left(d_i\right)}_{i\ge 1}$ is the sequence of Martingale differences adapted to ${\left({\mathcal{F}}_i\right)}_{i\ge 1}$ and

$$S_n=\sum _{i=1}^n{d}_i.$$

Then, for any positive integer n,

$$\parallel \underset{1\le j\le n}{max}|S_j{\left|\right)\parallel }_p\ll n^{1/p}{\parallel d_1\parallel }_p+{∥\sum _{k=1}^n\mathbb{E}(d_k^2/{\mathcal{F}}_{k-1})∥}_{p/2}^{1/2},for anyp\ge 2;$$

(41)

where, as usual, the norm ${\parallel ·\parallel }_p={\left(\mathbb{E}\left[\right|·{|}^p]\right)}^{1/p}$.

Lemma 2

([61]). Let $\{Z_i,i\ge 1\}$ be a sequence of Martingale differences such that

$$|Z_i|\le B,a.s.,$$

then, for all $ϵ>0$ and all sufficiently large n, we have

$$\mathbb{P}\left\{\left|\sum _{i=1}^n{Z}_i\right|>ϵ\right\}\le 2exp\left\{-\frac{{ϵ}^2}{2n{B}^2}\right\}.$$

The following lemmas describe the asymptotic behavior of the estimators ${\widehat{\alpha }}_{k,\ell }$ and ${\widehat{\beta }}_{k,\ell }$.

Lemma 3.

For any $k\in \{0,\dots ,m_n\}$ and any $\ell \in {\mathcal{I}}_k$, under assumptions (C.0), (C.1), (F.1)and (E.1)(i), there exists a constant $C>0$ such that

$$\mathbb{E}\left({\left|{\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\right|}^2\right)\le C\left(\frac{lnn}{n}\right).$$

(42)

Lemma 4.

For any $k\in \{0,\dots ,m_n\}$ and any $\ell \in {\mathcal{J}}_k$, and under assumptions (C.0), (C.1), (F.1), (E.1)and (E.2), and condition (16), there exists a constant $C>0$ such that

$$\mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)=C{\left(\frac{lnn}{n}\right)}^2,a.s.$$

(43)

Lemma 5.

For any $k\in \{0,\dots ,m_n\}$ and any $\ell \in {\mathcal{J}}_k$, for $\kappa >0$ large enough and under assumptions (C.0), (C.1), (F.1), (E.1)and (E.2), and condition (16), there exists a constant $C>0$ such that

$$\mathbb{P}\left(\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)\le C{\left(\frac{lnn}{n}\right)}^2.$$

(44)

7.1.1. Proof of Theorem 1

Observe that the proof of Theorem 1 is a direct application of ([49], Theorem 3.1) with $c\left(n\right)={(lnn/n)}^{1/2}$, ${\sigma }_i=1$, $r=2$ and the Lemmas 3–5. We adapted and extended the method of demonstration of [33], Theorem 3.1 to the stationary ergodic process. □

7.1.2. Proof of Lemmas

Proof of Lemma 3

Consider the following decomposition

$$\begin{array}{ccc}\hfill {\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }& =& {\widehat{\alpha }}_{k,\ell }-{\tilde{\alpha }}_{k,\ell }+{\tilde{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\hfill \\ & =& A_{k,\ell ,1}+A_{k,\ell ,2},\hfill \end{array}$$

(45)

where

$${\tilde{\alpha }}_{k,\ell }=\frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].$$

Under the assumptions (C.0) and (C.2), we have

$$\begin{array}{ccc}\hfill {\tilde{\alpha }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n{\int }_{\mathcal{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(\frac{1}{n}\sum _{i=1}^n{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)+o\left(1\right)\hfill \\ & =& {\alpha }_{k,\ell }+o\left(1\right).\hfill \end{array}$$

We readily obtain

$${\tilde{\alpha }}_{k,\ell }={\alpha }_{k,\ell },\mathrm{as},n\to \infty ,$$

(46)

implying that

$$A_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(47)

Therefore, we infer that

$$\begin{array}{ccc}\hfill {\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }& =& A_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Let us now consider the term $A_{k,\ell ,1}$. We have

$$\begin{array}{ccc}\hfill A_{k,\ell ,1}& =& {\widehat{\alpha }}_{k,\ell }-{\tilde{\alpha }}_{k,\ell }\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\left({\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right).\hfill \end{array}$$

Notice that ${\left({\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right)}_{0\le k\le m_n}$ is a sequence of Martingale differences with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_i\right)}_{0\le k\le m_n}$. It is obvious, by Lemma 1 inequality, to see that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right].\hfill \end{array}$$

Applying the Burkholder–Rosenthal inequality (1), for $p=2$, we obtain

$$\begin{array}{l}\hfill {\displaystyle {\left(E\left[\left({\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right)\right]\right)}^{\frac{1}{2}}}\\ \le n^{1/2}\Vert {\mathsf{\Phi }}_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\Vert }_2+{\Vert \sum _{i=1}^{n}E\left[{\mathsf{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\Vert }_1^{1/2}\\ ={\mathsf{\Phi }}_{\left(1\right)}+{\mathsf{\Phi }}_{\left(2\right)}\end{array}$$

(48)

On one hand, using a very famous decomposition combined with the fact that ${\mathcal{F}}_0$ is the trivial $\sigma -$field, we obtain

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{n}}{\mathsf{\Phi }}_{\left(1\right)}^2& =& \Vert {\mathsf{\Phi }}_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\Vert }_2^2\hfill \\ \hfill & =& E\left[{\left|{\varphi }_k\left(\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right)-E\left[{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_0\right]\right|}^2\right]\hfill \\ \hfill & \le & E\left[\sum _{j=0}^2\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\mid }^j{\left(E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\mid \right]\right)}^{2-j}\right]\hfill \\ \hfill & =& \sum _{j=0}^2{C}_2^{j}E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\mid }^j\right].{\left(E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\mid \right]\right)}^{2-j}\hfill \\ & =& C_2^{2}E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\mid }^2\right]+C_2^1{\left(E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\mid \right]\right)}^2+C_2^{0}E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\mid }^2\right],\hfill \end{array}$$

(49)

By the Cauchy–Schwarz inequality, together with assumptions (E.1)(i), (E.2) and the condition (16), we obtain

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|& \le & \underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}|e_j\left({\zeta }_{k,\ell }\right)\left|\right|e_j\left(\mathbf{x}\right)|\hfill \\ \hfill & \le & {\left(\sum _{j\in {\mathcal{J}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\right)}^{1/2}{\left(\underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\right)}^{1/2}\hfill \\ \hfill & \le & C_1^{1/2}{C}_2^{1/2}\sqrt{|{\mathcal{J}}_k|}\hfill \\ \hfill & \le & C_3\sqrt{|{\mathcal{J}}_{m_n}|}\hfill \\ & \le & C_3\sqrt{\frac{n}{lnn}}.\hfill \end{array}$$

(50)

Observe that, under the assumptions (F.1) and (E.1)(i) and the fact that $\mathcal{E}$ is an orthonormal basis of H, we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|}^2\right]& =& {\int }_{\mathbf{S}}{\left|{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^{2}f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & \le & C_f{\int }_{\mathbf{S}}{\left|{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& C_f{\int }_{\mathbf{S}}{\left|\sum _{j\in {\mathcal{I}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}{e}_j\left({\zeta }_{k,\ell }\right)e_j\left(\mathbf{x}\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& C_f{\int }_{\mathbf{S}}\sum _{j\in {\mathcal{I}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ & \le & C_f{C}_1.\hfill \end{array}$$

(51)

where $C_1$ is a positive constant,

$$\begin{array}{ccc}\hfill E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\mid \right]& =& O\left(\sqrt{{\displaystyle \frac{n}{lnn}}}\right),\hfill \end{array}$$

(52)

$$\begin{array}{ccc}\hfill E\left[\mid {\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\mid }^2\right]& =& O\left(1\right)\hfill \end{array}$$

(53)

therefore,

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_{\left(1\right)}=O\left(n^{1/2}\right).\end{array}$$

(54)

On the other hand, we consider the second term of decomposition (48), observe that

$$\begin{array}{ccc}\hfill {\mathsf{\Phi }}_2& =& {\left(\mathbb{E}\left(\sum _{i=1}^n\mathbb{E}\left[{\mathsf{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)\right)}^{1/2}\hfill \\ & =& {\left(\sum _{i=1}^n\mathbb{E}\left(\mathbb{E}\left[{\mathsf{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)\right)}^{1/2}\hfill \\ & =& {\left(\sum _{i=1}^n\mathbb{E}\left[{\mathsf{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right]\right)}^{1/2}\hfill \end{array}$$

using the notable identity, we obtain

$$\begin{array}{c}\hfill {\displaystyle \mathbb{E}\left[{\mathsf{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right]}\\ & =& \mathbb{E}\left[{\left(\left|{\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[{\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right|\right)}^2\right]\hfill \\ & \le & \mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right){\mid }^2+2\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\mid \mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\mid |{\mathcal{F}}_{i-1}\right]\right.\hfill \\ & & +\left.\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right){\mid }^2|{\mathcal{F}}_{i-1}\right]\right]\hfill \\ & \le & 2\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right){\mid }^2\right]+2\mathbb{E}\left[\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right){\mid }^2|{\mathcal{F}}_{i-1}\right]\right]\hfill \\ & \le & 4\mathbb{E}\left[\mid {\varphi }_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right){\mid }^2\right]\hfill \end{array}$$

observe that, using (50) and (51), we obtain

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_2=O\left(n^{1/2}\right).\end{array}$$

(55)

therefore, we combine (54) and (55) to obtain

$$\begin{array}{c}\hfill {\left(E\left[{\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\right)}^{1/2}=O\left(n^{1/2}\right).\end{array}$$

Hence,

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \\ & =& \frac{1}{n^2}O\left(n\right)\hfill \\ & \le & C\left({\displaystyle \frac{lnn}{n}}\right)\hfill \end{array}$$

Therefore, there exists a constant $C=C_f{C}_1>0$, such that

$$\mathbb{E}\left({\left|{\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\right|}^2\right)\le \frac{4C}{n}\le 4C\left(\frac{lnn}{n}\right).$$

(56)

Hence, the proof is complete. □

Proof of Lemma 4

Consider the following decomposition

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& {\widehat{\beta }}_{k,\ell }-{\tilde{\beta }}_{k,\ell }+{\tilde{\beta }}_{k,\ell }-{\beta }_{k,\ell }\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

(57)

where

$${\tilde{\beta }}_{k,\ell }=\frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].$$

Observe that, under the assumptions (F.1) and (E.1)(i) and the fact that $\mathcal{E}$ is an orthonormal basis of H, and proceeding in a similar way as in (46), we show that

$${\tilde{\beta }}_{k,\ell }={\beta }_{k,\ell },\mathrm{as},n\to \infty .$$

(58)

This, in turn, implies that

$$B_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(59)

Therefore, we obtain

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Hence, we readily infer

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)& =& \frac{1}{n^4}\mathbb{E}\left({\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|}^4\right),\hfill \end{array}$$

(60)

where

$${\mathsf{\Psi }}_{i,k,\ell }={\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].$$

Notice that ${\left({\mathsf{\Psi }}_{i,k,\ell }\right)}_{0\le k\le n}$ is a sequence of Martingale differences with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_i\right)}_{0\le k\le n}$, applying the Burkholder–Rosenthal inequality for $p=4$ (see Lemma 1), we obtain

$$\begin{array}{ccc}\hfill {\left(\mathbb{E}\left({\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|}^4\right)\right)}^{1/4}& \le & {∥\underset{1\le j\le n}{max}\left|\sum _{i=1}^j{\mathsf{\Psi }}_{i,k,\ell }\right|∥}_4\hfill \\ \hfill & \ll & n^{1/4}{\parallel {\mathsf{\Psi }}_{1,k,\ell }\parallel }_4+{∥\sum _{i=1}^n\mathbb{E}\left({\mathsf{\Psi }}_{i,k,\ell }^2\right|{\mathcal{F}}_{i-1})∥}_{4/2}^{1/2}\hfill \\ & =& {\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}+{\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}.\hfill \end{array}$$

(61)

Consider the first term of Equation (61). We have

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& =& \parallel {\mathsf{\Psi }}_{1,k,\ell }{\parallel }_4^4\hfill \\ & =& \mathbb{E}\left({\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)|{\mathcal{F}}_0\right]\right|}^4\right)\hfill \\ & \le & \mathbb{E}\left[{\left(\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4\right].\hfill \end{array}$$

Using the classical identity

$${(a+b)}^n=\sum _{k=0}^n{C}_n^k{a}^k{b}^{n-k}$$

in connection with the Jensen inequality and taking $n=4$, we obtain

$$\begin{array}{c}{\displaystyle {\left(\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4}\hfill \\ =\sum _{k=0}^4{C}_4^k{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^k{\left(\mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^{4-k}\\ \le \sum _{k=0}^4{C}_4^k{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^k\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^{4-k}\right].\end{array}$$

This gives that

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& \le & \mathbb{E}\left[\sum _{k=0}^4{C}_4^k{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^k\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^{4-k}\right]\right]\hfill \\ & =& \sum _{k=0}^4{C}_4^k\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^k\right]\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^{4-k}\right].\hfill \end{array}$$

(62)

By proceeding in a similar way as in (51) and making use of the assumptions (F.1) and (E.1)(i), we infer that

$$\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^2\right]\le C,$$

(63)

where C is a positive constant. Moreover, by the Cauchy–Schwarz inequality together with assumptions (E.1)(ii), (E.2) and the condition (16), we obtain

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|& \le & \underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{h_{j,k,\ell }}}|e_j\left({\eta }_{k,\ell }\right)\left|\right|e_j\left(\mathbf{x}\right)|\hfill \\ \hfill & \le & {\left(\sum _{j\in {\mathcal{J}}_k}\frac{1}{h_{j,k,\ell }}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2\right)}^{1/2}{\left(\underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\right)}^{1/2}\hfill \\ \hfill & \le & C_1^{1/2}{C}_2^{1/2}\sqrt{|{\mathcal{J}}_k|}\hfill \\ \hfill & \le & C_4\sqrt{|{\mathcal{J}}_{m_n}|}\hfill \\ & \le & C_4\sqrt{\frac{n}{lnn}}.\hfill \end{array}$$

(64)

We then obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left[\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|\right]& =& O\left(\sqrt{\frac{n}{lnn}}\right),\hfill \end{array}$$

(65)

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^2\right]& =& O\left(1\right),\hfill \end{array}$$

(66)

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^3\right]& =& O\left(\sqrt{\frac{n}{lnn}}\right),\hfill \end{array}$$

(67)

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{x}}_1;{\eta }_{k,\ell }\right)\right|}^4\right]& =& O\left(\frac{n}{lnn}\right).\hfill \end{array}$$

(68)

Observe that the largest term is (68); now, using (68) in Equation (62), we deduce that

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& =& O\left(\frac{n}{lnn}\right).\hfill \end{array}$$

This implies that

$${\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}=O\left(\frac{n^{1/2}}{{\left(lnn\right)}^{1/4}}\right).$$

(69)

Let us now investigate the upper bound of ${\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}$ in (61). Observe that

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}& =& {∥\sum _{i=1}^n\mathbb{E}({\mathsf{\Psi }}_{i,k,\ell }^2/{\mathcal{F}}_{i-1})∥}_2^{1/2}\hfill \\ & =& {\left(\mathbb{E}\left[{\left(\sum _{i=1}^n\mathbb{E}\left[{\mathsf{\Psi }}_{i,k,\ell }^2|{\mathcal{F}}_{i-1}\right]\right)}^2\right]\right)}^{1/4},\hfill \end{array}$$

for all $i=1,\dots ,n$. Making use of the Jensen inequality with the fact that ${(a-b)}^2=a^2-2ab+b^2$, it follows that

$$\begin{array}{ccc}\hfill \sum _{i=1}^n\mathbb{E}\left[{\mathsf{\Psi }}_{i,k,\ell }^2|{\mathcal{F}}_{i-1}\right]& =& \sum _{i=1}^n\left(\mathbb{E}\left[{\left({\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)}^2|{\mathcal{F}}_{i-1}\right]\right)\hfill \\ & \le & 4\sum _{i=1}^n\mathbb{E}\left[{\left({\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)\right)}^2|{\mathcal{F}}_{i-1}\right].\hfill \end{array}$$

Observe, under the assumptions (F.1), (E.1)(i), (C.0) and (C.1) and (63):

$$\begin{array}{ccc}\hfill \sum _{i=1}^n\mathbb{E}\left[{\left({\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)\right)}^2|{\mathcal{F}}_{i-1}\right]& =& n{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(\frac{1}{n}\sum _{i=1}^n{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& n{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & \le & n\left(C_f+o\left(1\right)\right){\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ & \le & n{C}_f{C}_1.\hfill \end{array}$$

(70)

It follows that

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}& =& O\left(n^{1/2}\right).\hfill \end{array}$$

(71)

Combining (61), (69) and (71), we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|}^4\right)& =& O\left(\frac{n^2}{lnn}\right)+O\left(n^2\right).\hfill \end{array}$$

We conclude that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)& =& O\left(\frac{1}{n^{2}lnn}\right)+O\left(\frac{1}{n^2}\right).\hfill \end{array}$$

(72)

This implies that there exists a constant $C>0$, such that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|}^4\right)& \le & C{\left(\frac{lnn}{n}\right)}^2.\hfill \end{array}$$

(73)

The proof is achieved. □

Proof of Lemma 5

Consider the previous decomposition in Lemma 4, to write that

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& \left({\widehat{\beta }}_{k,\ell }-{\tilde{\beta }}_{k,\ell }\right)+\left({\tilde{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right)\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill B_{k,\ell ,1}& =& \frac{1}{n}\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }=\frac{1}{n}\sum _{i=1}^n\left({\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right),\hfill \\ \hfill B_{k,\ell ,2}& =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]-{\beta }_{k,\ell }.\hfill \end{array}$$

By using (59), we obtain the desired result for the term $B_{k,\ell ,2}$

$$\begin{array}{ccc}\hfill {\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right).\hfill \end{array}$$

Now, observe that

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\frac{1}{n}\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)& =& \mathbb{P}\left(\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{nlnn}\right).\hfill \end{array}$$

An application of Lemma 2 implies that

$$\begin{array}{ccc}\hfill \left|{\mathsf{\Psi }}_{i,k,\ell }\right|& =& \left|{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right|\hfill \\ \hfill & \le & 2\underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|\hfill \\ \hfill & \le & C_3\sqrt{\frac{n}{lnn}}\hfill \\ & \le & C_3\sqrt{n}.\hfill \end{array}$$

(74)

Let $B=C_3\sqrt{n}$. Then, for all ${ϵ}_n=\frac{\kappa }{2}\sqrt{nlnn}$ where n is sufficiently large, we have

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{nlnn}\right)& \le & 2exp\left\{-\frac{{ϵ}_n^2}{2n{B}^2}\right\}\hfill \\ \hfill & =& 2exp\left\{-\frac{{\left(\frac{\kappa }{2}\sqrt{nlnn}\right)}^2}{2n{\left(C_3\sqrt{n}\right)}^2}\right\}\hfill \\ \hfill & =& 2exp\left\{-\frac{{\kappa }^{2}lnn}{8C_3^{2}n}\right\}\hfill \\ \hfill & =& 2exp\left\{ln{n}^{-\frac{{\kappa }^2}{8C_3^{2}n}}\right\}\hfill \\ & =& 2n^{-w(\kappa ,n)},\hfill \end{array}$$

(75)

where

$$w(\kappa ,n)=\frac{{\kappa }^2}{8C_3^{2}n}.$$

By choosing $\kappa$ such that $w(n,\kappa )=2,$ we have

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\widehat{\beta }}_{k,\ell }-{\beta }_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)& \le & C\frac{1}{n^2}+o\left(1\right)\hfill \\ & \le & C{\left(\frac{lnn}{n}\right)}^2.\hfill \end{array}$$

(76)

The proof of (44) is achieved. □

7.1.3. Proof of Theorem 2

Recall that

$$\widehat{\mathbf{m}}(\mathbf{x},\rho )=\sum _{\ell \in {\mathcal{I}}_0}{\widehat{\eta }}_{0,\ell }{\varphi }_0\left(\mathbf{x};{\zeta }_{0,\ell }\right)+\sum _{k=0}^{m_n}\sum _{\ell \in {\mathcal{J}}_k}{\widehat{\theta }}_{k,\ell }{\mathrm{𝟙}}_{\left\{|{\widehat{\theta }}_{k,\ell }|\ge \kappa \sqrt{\frac{lnn}{n}}\right\}}{\psi }_k\left(\mathbf{x};{\eta }_{k;\ell }\right),$$

where

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right),{\widehat{\theta }}_{k,\ell }=\frac{1}{n}\sum _{i=1}^n\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right).\hfill \end{array}$$

Lemma 6.

For any $k\in \{0,\dots ,m_n\}$ and any $\ell \in {\mathcal{I}}_k$ and under the assumptions (E.1)(i), (M.1)- (M.2), (C.0)and (C.1), there exists a constant $C>0$ such that

$$\mathbb{E}\left({\left|{\widehat{\eta }}_{k,\ell }-{\eta }_{k,\ell }\right|}^2\right)\le C\left(\frac{lnn}{n}\right).$$

(77)

Lemma 7.

For any $k\in \{0,\dots ,m_n\}$ and any $\ell \in {\mathcal{J}}_k$, and under the assumptions (E.1), (E.2)(M.1)-(M.2), (C.0)and (C.1), combined with the condition (24), there exists a constant $C>0$ such that

$$\mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)=C{\left(\frac{lnn}{n}\right)}^2,a.s.$$

(78)

Lemma 8.

For any $k\in \{0,\dots ,m_n\}$ and any $\ell \in {\mathcal{J}}_k$, for $\kappa >0$ large enough, (E.1), (E.2) (M.1)–(M.2), (C.0)and (C.1), combined with the condition (24), there exists a constant $C>0$ such that

$$\mathbb{P}\left(\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)\le C{\left(\frac{lnn}{n}\right)}^2.$$

(79)

7.1.4. Proof of Theorem 2

Observe that the proof of Theorem 2 is a direct application of ([49], Theorem 3.1) with $c\left(n\right)={(lnn/n)}^{1/2}$, ${\sigma }_i=1$, $r=2$ and the following Lemmas 6–8. We extended the method of the proof in [33], Theorem 4.1. □

7.1.5. Proof of Lemmas

Proof of Lemma 6

Consider the following decomposition

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }-{\eta }_{k,\ell }& =& {\widehat{\eta }}_{k,\ell }-{\tilde{\eta }}_{k,\ell }+{\tilde{\eta }}_{k,\ell }-{\eta }_{k,\ell }\hfill \\ & =& A_{k,\ell ,1}+A_{k,\ell ,2},\hfill \end{array}$$

(80)

where

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\left(\mathbf{m}({\mathbf{X}}_i,\rho )+{ϵ}_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\mathbf{m}({\mathbf{X}}_i,\rho )}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & & +\frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{{ϵ}_i}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].\hfill \end{array}$$

From the independence between ${ϵ}_i$ and ${\mathbf{X}}_i$, we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{ϵ}_i|{\mathcal{G}}_{i-1}\right]& =& \mathbb{E}\left[{ϵ}_i|{\mathbf{X}}_i\right]\hfill \\ \hfill & =& \mathbb{E}\left[{ϵ}_i\right]\hfill \\ & =& 0.\hfill \end{array}$$

(81)

Observe that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[\frac{ϵ}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]& =& \mathbb{E}\left[\frac{\mathbb{E}\left[ϵ|{\mathcal{G}}_{i-1}\right]}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & =& \mathbb{E}\left[\frac{\mathbb{E}\left[ϵ|{\mathbf{X}}_i\right]}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & =& \mathbb{E}\left[\frac{\mathbb{E}\left[ϵ\right]}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & =& 0.\hfill \end{array}$$

This implies that

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\mathbf{m}({\mathbf{X}}_i,\rho )}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].\hfill \end{array}$$

Making use of the assumptions (M.1)–(M.2), (C.0) and (C.1), we have

$$\begin{array}{ccc}\hfill {\tilde{\eta }}_{k,\ell }& =& \frac{1}{n}\sum _{i=1}^n{\int }_{\mathcal{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(\frac{1}{n}\sum _{i=1}^n{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)\nu \left(d\mathbf{x}\right)\hfill \\ & =& {\int }_{\mathbf{S}}\frac{\mathbf{m}(\mathbf{x},\rho )}{f\left(\mathbf{x}\right)}{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)+o\left(1\right)\hfill \\ & =& {\eta }_{k,\ell }+o\left(1\right).\hfill \end{array}$$

We readily obtain that

$${\tilde{\eta }}_{k,\ell }={\eta }_{k,\ell },\mathrm{as},n\to \infty .$$

(82)

implying that

$$A_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(83)

Therefore,

$$\begin{array}{ccc}\hfill {\widehat{\eta }}_{k,\ell }-{\eta }_{k,\ell }& =& A_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Let us now turn our attention to the term $A_{k,\ell ,1}$ in (80), we have

$$\begin{array}{ccc}\hfill A_{k,\ell ,1}& =& {\widehat{\eta }}_{k,\ell }-{\tilde{\eta }}_{k,\ell }\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\left(\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right)\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right).\hfill \end{array}$$

Notice that ${\left({\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right)}_{0\le k\le m_n}$ is a sequence of Martingale differences with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_i\right)}_{0\le k\le m_n}$. It is obvious, proceeding as the proof of (56), that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \end{array}$$

where

$$\begin{array}{c}\hfill {\displaystyle {\left(\mathbb{E}\left[\left({\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right)\right]\right)}^{\frac{1}{2}}}\\ & \le & n^{1/2}\Vert {\mathsf{\Phi }}_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\Vert }_2+{\Vert \sum _{i=1}^n\mathbb{E}\left[{\mathsf{\Phi }}_k^2\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\Vert }_1^{1/2}\hfill \\ & =& {\mathsf{\Phi }}_{\left(1\right)}+{\mathsf{\Phi }}_{\left(2\right)}\hfill \end{array}$$

(84)

On the one hand, using a very famous decomposition combined with the fact that ${\mathcal{F}}_0$ is the trivial $\sigma -$field, we obtain

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{n}}{\mathsf{\Phi }}_{\left(1\right)}^2& =& \Vert {\mathsf{\Phi }}_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right){\Vert }_2^2\hfill \\ \hfill & =& \mathbb{E}\left[{\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\varphi }_k\left(\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right)-\mathbb{E}\left[\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)|{\mathcal{F}}_0\right]\right|}^2\right]\hfill \\ \hfill & \le & \mathbb{E}\left[\sum _{j=0}^2{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|}^j{\left(\mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|\right]\right)}^{2-j}\right]\hfill \\ \hfill & =& \sum _{j=0}^2{C}_2^j\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|}^j\right].{\left(\mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|\right]\right)}^{2-j}\hfill \\ & =& C_2^2\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|}^2\right]+C_2^1{\left(\mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|\right]\right)}^2\hfill \\ & & +C_2^0\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|}^2\right],\hfill \end{array}$$

It follows, from assumptions (M.1) and (M.2), that

$$|\rho \left(Y_i\right)|\le C_{\mathbf{m}}+\left|{ϵ}_i\right|,$$

(85)

combined with the independence between ${\mathbf{X}}_1$ and ${ϵ}_1$, $\mathbb{E}\left[{ϵ}_1^2\right]=1$. Observe that, under assumption (E.1)(i) and the fact that $\mathcal{E}$ is an orthonormal basis of H, we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{X}}_1;{\zeta }_{k,\ell }\right)\right|}^2\right]& \le & \frac{\left(C_{\mathbf{m}}^2+1\right)}{c_f}\mathbb{E}\left[\left|\frac{1}{f\left({\mathbf{X}}_1\right)}\right|{\left|{\varphi }_k\left({\mathbf{x}}_1;{\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \\ \hfill & \le & \frac{\left(C_{\mathbf{m}}^2+1\right)}{c_f}{\int }_{\mathbf{S}}\frac{1}{f\left(\mathbf{x}\right)}{\left|{\varphi }_k\left(\mathbf{x};{\zeta }_{k,\ell }\right)\right|}^{2}f\left(\mathbf{x}\right)\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& \frac{\left(C_{\mathbf{m}}^2+1\right)}{c_f}{\int }_{\mathbf{S}}{\left|\sum _{j\in {\mathcal{I}}_k}\frac{1}{\sqrt{g_{j,k,\ell }}}{e}_j\left({\zeta }_{k,\ell }\right)e_j\left(\mathbf{x}\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ \hfill & =& \frac{\left(C_{\mathbf{m}}^2+1\right)}{c_f}{\int }_{\mathbf{S}}\sum _{j\in {\mathcal{I}}_k}\frac{1}{g_{j,k,\ell }}{\left|e_j\left({\zeta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\hfill \\ & \le & \frac{\left(C_{\mathbf{m}}^2+1\right)}{c_f}{C}_1=O\left(1\right).\hfill \end{array}$$

(86)

Moreover, from Assumptions (M.1) and (M.2) and using (52), (85), combined with the independence between ${\mathbf{X}}_1$ and ${ϵ}_1$, $\mathbb{E}\left[{ϵ}_1\right]=0$. we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\varphi }_k\left({\mathbf{X}}_1;{\zeta }_{k,\ell }\right)\right|\right]& =& O\left(\sqrt{\frac{n}{lnn}}\right).\hfill \end{array}$$

(87)

Therefore,

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_{\left(1\right)}=O\left(n^{1/2}\right).\end{array}$$

(88)

On the other hand, we consider the second term of decomposition (84), and proceeding as in the proof of (55) and considering (86) and (87)

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_2=O\left(n^{1/2}\right).\end{array}$$

(89)

therefore, combining (88) and (89) to obtain

$$\begin{array}{c}\hfill {\left(E\left[{\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{X}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\right)}^{1/2}=O\left(n^{1/2}\right).\end{array}$$

Hence,

$$\begin{array}{ccc}\hfill \mathbb{E}\left[|A_{k,\ell ,1}{|}^2\right]& =& \frac{1}{n^2}\mathbb{E}\left[{\left|\sum _{i=1}^n{\mathsf{\Phi }}_k\left({\mathbf{x}}_i;{\zeta }_{k,\ell }\right)\right|}^2\right]\hfill \\ \hfill & =& \frac{1}{n^2}O\left(n\right)\hfill \\ & \le & C\left({\displaystyle \frac{lnn}{n}}\right)\hfill \end{array}$$

(90)

Therefore, combining (83) and (90), there exists a constant C such as

$$\mathbb{E}\left({\left|{\widehat{\alpha }}_{k,\ell }-{\alpha }_{k,\ell }\right|}^2\right)\le \frac{C}{n}\le C\left(\frac{lnn}{n}\right).$$

(91)

Hence, the proof is complete. □

Proof of Lemma 7

Consider the following decomposition

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& {\widehat{\theta }}_{k,\ell }-{\breve{\theta }}_{k,\ell }+{\breve{\theta }}_{k,\ell }-{\theta }_{k,\ell }\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

(92)

where

$${\breve{\theta }}_{k,\ell }=\frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].$$

Observe that, under the assumptions (M.1)–(M.2) and (C.1), the Equation (81) and the fact that $\mathcal{E}$ is an orthonormal basis of H, by proceeding as in (82), we show that

$${\breve{\theta }}_{k,\ell }={\theta }_{k,\ell },\mathrm{as},n\to \infty ,$$

(93)

implying that

$$B_{k,\ell ,2}=o\left(1\right),\mathrm{a}.\mathrm{s}.$$

(94)

Therefore,

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right),\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Hence, we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)& =& \frac{1}{n^4}\mathbb{E}\left({\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|}^4\right),\hfill \end{array}$$

(95)

where

$${\mathsf{\Psi }}_{i,k,\ell }=\frac{Y_i}{f\left({\mathbf{x}}_i\right)}{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{Y_i}{f\left({\mathbf{x}}_i\right)}{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right].$$

Notice that ${\left({\mathsf{\Psi }}_{i,k,\ell }\right)}_{0\le k\le n}$ is a sequence of Martingale differences with respect to the sequence of $\sigma -$fields ${\left({\mathcal{F}}_i\right)}_{0\le k\le n}$, applying the Burkholder–Rosenthal inequality for $p=4$ (see Lemma 1), we obtain

$$\begin{array}{ccc}\hfill {\left(\mathbb{E}\left({\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|}^4\right)\right)}^{1/4}& \le & {∥\underset{1\le j\le n}{max}\left|\sum _{i=1}^j{\mathsf{\Psi }}_{i,k,\ell }\right|∥}_4\hfill \\ \hfill & \ll & n^{1/4}{\parallel {\mathsf{\Psi }}_{1,k,\ell }\parallel }_4+{∥\sum _{i=1}^n\mathbb{E}\left({\mathsf{\Psi }}_{i,k,\ell }^2\right|{\mathcal{F}}_{i-1})∥}_{4/2}^{1/2}\hfill \\ & =& {\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}+{\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}.\hfill \end{array}$$

(96)

Consider the first term of Equation (96),

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& =& \parallel {\mathsf{\Psi }}_{1,k,\ell }{\parallel }_4^4\hfill \\ & =& \mathbb{E}\left({\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)|{\mathcal{F}}_0\right]\right|}^4\right)\hfill \\ & \le & \mathbb{E}\left[{\left(\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4\right].\hfill \end{array}$$

By combining the identity

$${(a+b)}^n=\sum _{k=0}^n{C}_n^k{a}^k{b}^{n-k}$$

and the Jensen inequality and taking $n=4$, we obtain

$$\begin{array}{c}{\displaystyle {\left(\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{x}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|+\mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^4}\hfill \\ =\sum _{k=0}^4{C}_4^k{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^k{\left(\mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|\right]\right)}^{4-k}\\ \le \sum _{k=0}^4{C}_4^k{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^k\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^{4-k}\right].\end{array}$$

Then, we have

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& \le & \mathbb{E}\left[\sum _{k=0}^4{C}_4^k{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^k\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^{4-k}\right]\right]\hfill \\ & =& \sum _{k=0}^4{C}_4^k\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^k\right]\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^{4-k}\right].\hfill \end{array}$$

(97)

By following the same reasoning as in (86) and under the same assumptions (M.1)-(M.2) and (E.1)(i), we have

$$\mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^2\right]\le C,$$

(98)

where C is a positive constant. Moreover, by the Cauchy–Schwarz inequality together with assumptions (E.1)(ii), (E.2) and condition (24), we obtain

$$\begin{array}{ccc}\hfill \underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|& \le & \underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}\frac{1}{\sqrt{h_{j,k,\ell }}}|e_j\left({\eta }_{k,\ell }\right)\left|\right|e_j\left(\mathbf{x}\right)|\hfill \\ \hfill & \le & {\left(\sum _{j\in {\mathcal{J}}_k}\frac{1}{h_{j,k,\ell }}{\left|e_j\left({\eta }_{k,\ell }\right)\right|}^2\right)}^{1/2}{\left(\underset{\mathbf{x}\in \mathbf{S}}{sup}\sum _{j\in {\mathcal{J}}_k}{\left|e_j\left(\mathbf{x}\right)\right|}^2\right)}^{1/2}\hfill \\ \hfill & \le & C_1^{1/2}{C}_2^{1/2}\sqrt{|{\mathcal{J}}_k|}\hfill \\ \hfill & \le & C_3\sqrt{|{\mathcal{J}}_{m_n}|}\hfill \\ & \le & C_3\sqrt{\frac{n}{{\left(lnn\right)}^2}}.\hfill \end{array}$$

(99)

Hence, we infer that

$$\begin{array}{ccc}\hfill \mathbb{E}\left[\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|\right]& =& O\left(\sqrt{\frac{n}{{\left(lnn\right)}^2}}\right),\hfill \end{array}$$

(100)

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^2\right]& =& O\left(1\right),\hfill \end{array}$$

(101)

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^3\right]& =& O\left(\sqrt{\frac{n}{{\left(lnn\right)}^2}}\right),\hfill \end{array}$$

(102)

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|\frac{\rho \left(Y_1\right)}{f\left({\mathbf{X}}_1\right)}{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^4\right]& =& O\left(\frac{n}{{\left(lnn\right)}^2}\right).\hfill \end{array}$$

(103)

Observe that the largest term is (103), now, using that same statement (103) in Equation (62), we deduce that

$$\begin{array}{ccc}\hfill \frac{1}{n}{\left({\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}\right)}^4& \le & C_4\mathbb{E}\left[{\left|{\psi }_k\left({\mathbf{X}}_1;{\eta }_{k,\ell }\right)\right|}^4\right]\hfill \\ & =& O\left(\frac{n}{{\left(lnn\right)}^2}\right).\hfill \end{array}$$

It follows that

$${\mathsf{\Psi }}_{k,\ell }^{\left(1\right)}=O{\left(\frac{n}{lnn}\right)}^{1/2}.$$

(104)

Let us now investigate the upper bound of ${\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}$ of (61). Observe that

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}& =& {∥\sum _{i=1}^n\mathbb{E}({\mathsf{\Psi }}_{i,k,\ell }^2/{\mathcal{F}}_{i-1})∥}_2^{1/2}\hfill \\ & =& {\left(\mathbb{E}\left[{\left(\sum _{i=1}^n\mathbb{E}\left[{\mathsf{\Psi }}_{i,k,\ell }^2|{\mathcal{F}}_{i-1}\right]\right)}^2\right]\right)}^{1/4},\hfill \end{array}$$

for all $i=1,\dots ,n$, using the Jensen inequality and the fact that

$${(a-b)}^2=a^2-2ab+b^2,$$

it follows

$$\begin{array}{c}{\displaystyle \sum _{i=1}^n\mathbb{E}\left[{\left|{\mathsf{\Psi }}_{i,k,\ell }\right|}^2|{\mathcal{F}}_{i-1}\right]}\hfill \\ =\sum _{i=1}^n\left(\mathbb{E}\left[{\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{x}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right|}^2|{\mathcal{F}}_{i-1}\right]\right)\\ \le 4\sum _{i=1}^n\mathbb{E}\left[{\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)\right|}^2|{\mathcal{F}}_{i-1}\right].\end{array}$$

From the independence between ${ϵ}_i$ and ${\mathbf{X}}_i$, we have

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{ϵ}^2|{\mathcal{G}}_{i-1}\right]& =& \mathbb{E}\left[{ϵ}^2|{\mathbf{X}}_i\right]\hfill \\ \hfill & =& \mathbb{E}\left[{ϵ}^2\right]=1.\hfill \end{array}$$

Under assumptions (M.1)–(M.2) and (E.1)(i) and (C.1), (63) and (105), we have

$$\begin{array}{c}{\displaystyle \sum _{i=1}^n\mathbb{E}\left[{\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)\right|}^2|{\mathcal{F}}_{i-1}\right]}\hfill \\ =\sum _{i=1}^n\mathbb{E}\left[\mathbb{E}\left[{\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)\right|}^2|{\mathcal{G}}_{i-1}\right]|{\mathcal{F}}_{i-1}\right]\\ \le \frac{n\left(C_m^2+1\right)}{c_f}{\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\left|\frac{\frac{1}{n}{\sum }_{i=1}^n{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)}{f\left(\mathbf{x}\right)}\right|\nu \left(d\mathbf{x}\right)\\ =\frac{n\left(C_m^2+1\right)}{c_f}(1+o\left(1\right)){\int }_{\mathbf{S}}{\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|}^2\nu \left(d\mathbf{x}\right)\\ \le nC,\end{array}$$

(105)

where

$$C=\frac{\left(C_m^2+1\right)}{c_f}(1+o\left(1\right)).$$

It follows

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_{k,\ell }^{\left(2\right)}& =& C{n}^{1/2}.\hfill \end{array}$$

(106)

Combining (61), (69) and (71), we obtain

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|}^4\right)& =& O\left({\left(\frac{n}{lnn}\right)}^2\right)+O\left(n^2\right),\hfill \end{array}$$

combining this with (95), we conclude

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)& =& O\left(\left(\frac{1}{n^4}\right){\left(\frac{n}{lnn}\right)}^2\right)+O\left(\left(\frac{1}{n^4}\right)n^2\right).\hfill \end{array}$$

(107)

Hence, there exists a constant $C>0$, such that

$$\begin{array}{ccc}\hfill \mathbb{E}\left({\left|{\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right|}^4\right)& \le & C{\left(\frac{1}{n}\right)}^2\le C{\left(\frac{lnn}{n}\right)}^2.\hfill \end{array}$$

(108)

The proof is achieved. □

Proof of Lemma 8

Considering the previous decomposition (92) in Lemma 7, we have

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& \left({\widehat{\theta }}_{k,\ell }-{\breve{\theta }}_{k,\ell }\right)+\left({\breve{\theta }}_{k,\ell }-{\theta }_{k,\ell }\right)\hfill \\ & =& B_{k,\ell ,1}+B_{k,\ell ,2},\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill B_{k,\ell ,1}& =& \frac{1}{n}\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\hfill \\ & =& \frac{1}{n}\sum _{i=1}^n\left(\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]\right),\hfill \\ \hfill B_{k,\ell ,2}& =& \frac{1}{n}\sum _{i=1}^n\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)|{\mathcal{F}}_{i-1}\right]-{\theta }_{k,\ell }.\hfill \end{array}$$

Statement (94) achieves the desired result for the term $B_{k,\ell ,2}$

$$\begin{array}{ccc}\hfill {\widehat{\theta }}_{k,\ell }-{\theta }_{k,\ell }& =& B_{k,\ell ,1}+o\left(1\right).\hfill \end{array}$$

We consider the next decomposition

$$\begin{array}{ccc}\hfill {\mathsf{\Psi }}_{i,k,\ell }& =& V_{i,k,\ell }+W_{i,k,\ell },\hfill \end{array}$$

(109)

where

$$V_{i,k,\ell }=\left(\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{x}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}|{\mathcal{F}}_{i-1}\right]\right),$$

$$W_{i,k,\ell }=\left(\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}^c-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}^c|{\mathcal{F}}_{i-1}\right]\right),$$

and

$${\mathrm{𝟙}}_{{\mathcal{A}}_i}=\left\{|{ϵ}_i|\ge c_{*}\sqrt{lnn}\right\},$$

and $c_{*}$ denotes a constant which will be chosen later. Now, observe that

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\widehat{\theta }}_{k,\ell }-{\breve{\theta }}_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)& \le & \mathbb{P}\left(\left|B_{k,\ell ,1}\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)+o\left(1\right)\hfill \\ \hfill & =& \mathbb{P}\left(\left|\sum _{i=1}^n{\mathsf{\Psi }}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{nlnn}\right)+o\left(1\right)\hfill \\ & =& I_1+I_2+o\left(1\right),\hfill \end{array}$$

(110)

where

$$\begin{array}{ccc}\hfill I_1& =& \mathbb{P}\left(\left|\sum _{i=1}^n{V}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{nlnn}\right),\hfill \\ \hfill I_2& =& \mathbb{P}\left(\left|\sum _{i=1}^n{W}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{nlnn}\right).\hfill \end{array}$$

First, we aim to bound the term $I_1$ of Equation (110). The Markov inequality and the Cauchy–Schwarz inequality yield

$$\begin{array}{ccc}\hfill I_1& \le & \frac{2}{\kappa \sqrt{nlnn}}\mathbb{E}\left(\left|\sum _{i=1}^n{V}_{i,k,\ell }\right|\right)\hfill \\ & \le & \frac{2}{\kappa \sqrt{nlnn}}\sum _{i=1}^n\mathbb{E}\left(\left|V_{i,k,\ell }\right|\right).\hfill \end{array}$$

(111)

Observe that

$$\begin{array}{c}\hfill {\displaystyle \mathbb{E}\left(\left|V_{i,k,\ell }\right|\right)}\\ \hfill & =& \mathbb{E}\left(\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}|{\mathcal{F}}_{i-1}\right]\right|\right)\hfill \end{array}$$

$$\begin{array}{ccc}& \le & \mathbb{E}\left(\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}\right|\right)+\mathbb{E}\left(\mathbb{E}\left[\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}\right||{\mathcal{F}}_{i-1}\right]\right)\hfill \\ \hfill & \le & 2\mathbb{E}\left(\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i}\right|\right)\hfill \end{array}$$

(112)

$$\begin{array}{ccc}& \le & 2{\left(\mathbb{E}\left({\left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right)\right|}^2\right)\right)}^{1/2}{\left(\mathbb{P}\left({\mathcal{A}}_i\right)\right)}^{1/2}.\hfill \end{array}$$

(113)

We use (98) combined with an elementary Gaussian inequality and take $c_{*}$ to have

$$\frac{c_{*}^2}{4}-1/2=2.$$

We obtain

$$\begin{array}{ccc}\hfill I_1& \le & \frac{2C}{\kappa }\sqrt{\frac{n}{lnn}}exp\left\{-\frac{c_{*}^{2}lnn}{4}\right\}\hfill \\ \hfill & \le & \frac{2C}{\kappa }\frac{n^{-\left(\frac{c_{*}^2}{4}-1/2\right)}}{\sqrt{lnn}}\hfill \\ & \le & C_{\kappa }\frac{1}{n^2},\hfill \end{array}$$

(114)

where $C_{\kappa }=\frac{2C}{\kappa }.$ We now intend to investigate an upper bound for $I_2$ of decomposition (110). We start by verifying the condition of Lemma 2. Suppose that assumptions (M.1) and (M.2) are satisfied combined with (99), we obtain

$$\begin{array}{ccc}\hfill |Y_i{\mathrm{𝟙}}_{{\mathcal{A}}_i^c}|& \le & C_m+c_{*}\sqrt{lnn}\hfill \\ & \le & C\sqrt{lnn},\hfill \end{array}$$

(115)

which implies

$$\begin{array}{ccc}\hfill \left|W_{i,k,\ell }\right|& \le & \left|\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i^c}-\mathbb{E}\left[\frac{\rho \left(Y_i\right)}{f\left({\mathbf{X}}_i\right)}{\psi }_k\left({\mathbf{X}}_i;{\eta }_{k,\ell }\right){\mathrm{𝟙}}_{{\mathcal{A}}_i^c}{\mathrm{𝟙}}_{{\mathcal{A}}_i^c}|{\mathcal{F}}_{i-1}\right]\right|\hfill \\ \hfill & \le & \frac{2C\sqrt{lnn}}{c_f}\underset{\mathbf{x}\in \mathbf{S}}{sup}\left|{\psi }_k\left(\mathbf{x};{\eta }_{k,\ell }\right)\right|\hfill \\ \hfill & \le & \frac{2C}{c_f}\sqrt{lnn}\sqrt{\frac{n}{{\left(lnn\right)}^2}}\hfill \\ \hfill & \le & C_3\sqrt{\frac{n}{lnn}}\hfill \\ & \le & C_3\sqrt{n},\hfill \end{array}$$

(116)

where $C_3=\frac{2C}{C_f}$, let $B=C_3\sqrt{n}$, then, for all ${ϵ}_n=\frac{\kappa }{2}\sqrt{nlnn}$ sufficiently large n, we have

$$\begin{array}{ccc}\hfill I_2=\mathbb{P}\left(\left|\sum _{i=1}^n{W}_{i,k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{nlnn}\right)& \le & 2exp\left\{-\frac{{ϵ}_n^2}{2n{B}^2}\right\}\hfill \\ \hfill & =& 2exp\left\{-\frac{{\left(\frac{\kappa }{2}\sqrt{nlnn}\right)}^2}{2n{\left(C_3\sqrt{n}\right)}^2}\right\}\hfill \\ \hfill & =& 2exp\left\{-\frac{{\kappa }^{2}lnn}{4C_3^{2}n}\right\}\hfill \\ \hfill & =& 2exp\left\{ln{n}^{-\frac{{\kappa }^2}{4C_3^{2}n}}\right\}\hfill \\ & =& 2n^{-w(\kappa ,n)},\hfill \end{array}$$

(117)

where

$$w(\kappa ,n)=\frac{{\kappa }^2}{4C_3^{2}n}.$$

Taking $\kappa$ such that $w(n,\kappa )=2,$ we have

$$\begin{array}{ccc}\hfill I_2& \le & C\frac{1}{n^2}.\hfill \end{array}$$

(118)

It follows from (110), (114) and (118) that

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\widehat{\theta }}_{k,\ell }-{\tilde{\theta }}_{k,\ell }\right|\ge \frac{\kappa }{2}\sqrt{\frac{lnn}{n}}\right)& \le & C\frac{1}{n^2}+o\left(1\right)\hfill \end{array}$$

(119)

$$\begin{array}{ccc}& \le & C{\left(\frac{lnn}{n}\right)}^2.\hfill \end{array}$$

(120)

The proof of (79) is achieved. □