Nonparametric Estimation of the Density Function of the Distribution of the Noise in CHARN Models

Abstract

This work is concerned with multivariate conditional heteroscedastic autoregressive nonlinear (CHARN) models with an unknown conditional mean function, conditional variance matrix function and density function of the distribution of noise. We study the kernel estimator of the latter function when the former are either parametric or nonparametric. The consistency, bias and asymptotic normality of the estimator are investigated. Confidence bound curves are given. A simulation experiment is performed to evaluate the performance of the results.

1. Introduction

The importance of multivariate time series is highlighted in a number of papers. They are widely studied in the econometric and statistical literature as can be seen, for instance, in [1,2,3,4,5,6]. Because of the interesting features of conditional heteroscedastic models in the analysis of financial time series, a lot of work has been devoted to the study of ARCH models [7,8,9] which, motivated by many reasons, have been generalized to CHARN models [10]. The CHARN model we deal with here is described as follows:

$$X_t=T\left(Y_t\right)+V\left(Y_t\right){\epsilon }_t,t\in \mathbb{Z},$$

(1)

where the sequence $\{X_t={(X_{t,1},\dots ,X_{t,k})}^{\top }\}$ is a stationary and ergodic k-dimensional process; the $Y_t={(X_{t-1}^{\top },X_{t-2}^{\top },\dots ,X_{t-p}^{\top })}^{\top }\in {\mathbb{R}}^{pk}$ are a $pk$-dimensional random vector; the ${\epsilon }_t={({\epsilon }_{t,1},\dots ,{\epsilon }_{t,k})}^{\top }$, $t\in \mathbb{Z}$ are i.i.d. k-dimensional standard white noise with density function f; and ${\epsilon }_t$ is uncorrelated with the $\sigma$-algebra spanned by the $Y_s$’s, $s<t$. The k-dimensional function $T\left(x\right)={(T_1\left(x\right),\dots ,T_k\left(x\right))}^{\prime }$ and the $k\times k$ positive definite matrix function $V\left(x\right)=(V_{ij}\left(x\right):1\le i,j\le k)$ are either of unknown or known forms but depend on unknown parameters. Note that the $T_j$’s and $V_{i,j}$’s, $1\le i,j\le k$ are real-valued functions defined on ${\mathbb{R}}^{pk}$.

It is well-known (see, for example, [6]) that an estimator of the noise distribution can be used for testing model assumptions. This is often used in time series analysis at the model validation step (see, for example, [11], Section 5.3, p. 164), where one checks whether the residuals from the model fit behave in a consistent manner with the model. In this sense, when the current model is fitted to observations, one should check if the residuals behave like observations from iid random vectors with a specified density $f=f_0$. There are many existing tests, for the iid assumption (see, for example, [11]). If $f_0$ is the Gaussian density, one can use the Shapiro–Wilk test for testing the Gaussianity of the residuals. For a more general density, a simple test based on confidence bound functions for f can be used if these bounds can be constructed. Then, if the curve of the given $f_0$ lies between the bounds, one would accept that $f=f_0$ and that the residuals are from $f_0$. Confidence bounds can be obtained if one constructs a consistent and asymptotic normal estimator of f. The present work focuses on the study of the kernel estimator of f based on the residuals. All along the paper, we assume that this function is positive, uniformly continuous and twice differentiable, with continuous and bounded derivatives.

If the ${\epsilon }_t$s are observed, the kernel estimator of f would be defined by

$$f_n\left(x\right)={\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^{n}K\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right),x\in {\mathbb{R}}^k,$$

(2)

where $a_n$ is the smoothing parameter and K a Parzen–Rosenblatt kernel (${\left|\right|x\left|\right|}^{k}K\left(x\right)⟶{0,\left|\right|x\left|\right|}^k\to \infty$). Since these are never observed, they have to be substituted for estimates as, for example, residual series from parametric or nonparametric fits. In this paper, we study the asymptotic behavior of such an estimator. In Section 2, we write a short review on the kernel estimation of the density function. In Section 3, we study some asymptotic properties of $f_n\left(x\right)$ when the functions T and V are parametric and the ${\epsilon }_t$s are substituted for the residuals from the fit. In Section 4, the same is done when the latter functions are nonparametric. We do not study the asymptotic normality in this case, as it is too technical and lengthy. The results are illustrated by a simulation experiment in Section 5, and Section 6 contains our conclusion. In order to have a self-contained paper, the external results used along the current work are gathered in Appendix A.

2. On the Kernel Density Estimation

2.1. A Short Review

During the last four decades, nonparametric functional estimation has undergone considerable developments. In particular, the estimation of the density of a probability law in ${\mathbb{R}}^k$, $k\ge 1,$ has been studied by very different methods, some of which include Refs. [12,13,14,15,16,17,18,19].

The first work in this field goes back to [20,21], which focused on the solution densities of a differential equation:

$$\frac{dg}{dx}=\frac{(x-a)g}{b_0+b_{1}x+b_2{x}^2},$$

where the constants $a,$ $b_0,$ $b_1$ and $b_2$ are expressed as functions of the first four moments of the distribution of the density g. For estimating g, the technique was to estimate the moments of this distribution.

Subsequently, many other methods have been proposed. Because of its ease, the kernel method received the most attention. This method was introduced by [22], from the idea of Fix and Hodges invoked in [23]. The idea consisted in estimating the density g at a point x by counting the number of observations located in the interval $[x-a,x+a]$ for $a>0$. Rosenblatt proposed a more general estimator defined by

$$g_n\left(x\right)=\frac{1}{n{a}_n}\sum _{i=1}^{n}K\left(\frac{x-U_i}{a_n}\right),x\in \mathbb{R},$$

(3)

where K is a probability density on $\mathbb{R}$, $\left(a_n\right)$ is a sequence of positive real numbers, and $U_1,\dots ,U_n$ is a sample from g. In the literature, $g_n$ is commonly referred to as the Parzen–Rosenblatt estimator [22,24]. Its extension to the multivariate case was studied by [25] who proposed as the estimator

$$g_n\left(x\right)=\frac{1}{n{a}_n^k}\sum _{i=1}^{n}K\left(\frac{x-U_i}{a_n}\right),x\in {\mathbb{R}}^k,k\ge 1,$$

(4)

where K is a probability density on ${\mathbb{R}}^k$ and the $U_i$s are k-dimensional samples.

The scaling factor $a_n^k$ appearing in the denominator in the right-hand side of (4) can be seen as the determinant of a $k\times k$ diagonal matrix with the $a_n$s on the main diagonal. Moreover, the argument of the function K can also be seen as the transformation of $x-U_i$ by the inverse of this matrix. From these observations [14], instead of considering a diagonal matrix, we rather considered a full matrix and defined the estimator

$${\tilde{g}}_n\left(x\right)=\frac{1}{n\left|\det \right(H_n\left)\right|}\sum _{i=1}^{n}K\left(H_n^{-1}(x-U_i)\right),x\in {\mathbb{R}}^k,$$

where $H_n$ is an invertible $k\times k$ matrix with determinant $\det \left(H_n\right)$.

Ref. [26] introduced a new class of kernel estimators of the density of a probability distribution on ${\mathbb{R}}^k$, $k\ge 1$. The originality of these estimators compared to the previous ones is that the usual bandwidth $a_n$ is a function of the observations and its moments of orders 1 and 2 coincide with the empirical moments of the law to be estimated. However, this study was done for the normal kernel only.

For more on the kernel density estimation, the reader can refer to the monographs of [13,16,18,19,27,28]. A more recent book is that of [29], which also gathers numerous recent references on the subject.

2.2. Some Properties of the Kernel Estimator

The method which seems convenient, simple to implement, robust and does not require a multiple choice of parameters is the kernel method, which has attracted the most interest since [22,24]. The properties of the resulting kernel estimator $g_n$ have been largely studied in the literature.

Several modes of convergence of the kernel estimator $g_n$ to g have been studied. Ref. [24] for $k=1$ and [25] for $k>1$, have established the convergence in probability. That is, as $n\to \infty$, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}|g_n\left(x\right)-g\left(x\right)|⟶0,$$

and the convergence to 0 of the mean square error $\left(MSE\right)$

$$MSE=\mathbb{E}\{{[g_n\left(x\right)-g\left(x\right)]}^2\}.$$

Ref. [30] obtained, for $k=1$, the convergence to 0 of the mean integrated squared error $\left(MISE\right)$

$$MISE={\int }_{{\mathbb{R}}^k}\mathbb{E}\{{[g_n\left(x\right)-g\left(x\right)]}^2\}dx.$$

Ref. [18] stated and proved various results on the properties of the kernel estimator $g_n$. Most of them are about its pointwise and uniform consistency, its asymptotic bias and variance, or its asymptotic normality. Those of these results used to establish our results are recalled in the last section.

2.3. Bandwidth Selection

Bandwidth selection is an important issue in the kernel estimation of density functions. It is crucial to the estimation accuracy. However, it is a difficult problem which has been the subject of a large amount of work (see, for example, [31] for a review). Unlike data-driven methods, such as least square or cross-validation selection (see, for example, [32]), the plug-in methods demonstrate their simplicity and adequacy in the symmetric density shape case. The following plug-in method is proposed by [14,33]. Resulted from the optimization of the $MISE$ above in certain cases, their bandwidth $a_n$ is taken to be

$$a_n={\left(\frac{\int K^2\left(t\right)dt}{{(\int t^{2}K\left(t\right)dt)}^2}\right)}^{1/5}\frac{S_n}{{(3/8\sqrt{\pi })}^{1/5}{n}^{1/5}},$$

where $S_n^2=n^{-1}{\sum }_{i=1}^n{(U_i-\overline{U})}^2$ is the sample variance. With the Epanechnikov kernel, this bandwidth is approximately defined as

$$a_n=1.0486\frac{S_n}{n^{1/5}}.$$

It is this bandwidth that is further used in the numerical study in Section 5 of this paper.

3. The Semi-Parametric Models Case

Here, we consider model (1) with the aim of estimating f by a kernel method in the case $T\left(x\right)=M(\theta ;x)$ and $V\left(x\right)=\mathrm{\Sigma }(\theta ;x)$ for some k-dimensional vector M and $k\times k$ matrix $\mathrm{\Sigma }$ functions of known forms, with the unknown parameter $\theta ={({\theta }_1,\dots ,{\theta }_q)}^{\prime }$ lying within a subset $\mathrm{\Theta }$ of ${\mathbb{R}}^q$. More precisely, we study some of the asymptotic properties of $f_n\left(x\right)$ defined by (2) when the ${\epsilon }_t$s are substituted for the residuals from the fit. This requires the estimation of the parameter vector $\theta$. In the following subsection, we briefly review some recent results on this estimation.

3.1. The Parameter Estimation

There is an increasing body of work on the parameter estimation of many classes of models contained within CHARN models (1). In this subsection, we give an overview of some of them.

The main estimators encountered in the time series literature are of least squares (LS) or of the likelihood type. Besides these, are some other less popular ones. Ref. [34] constructed a Godambe’s estimator, which they denoted by G, and proved that it is more efficient than the conditional least-squares estimator (CL) for autoregressive conditional heteroscedasticity (ARCH) models. In [35], the efficiency of the CL- and G-estimators for the CHARN model was studied. Ref. [35] deduced the condition according to which the estimator G is asymptotically optimal. The author proved the asymptotic normality of these two estimators and concluded that the G-estimator is less efficient than the CL-estimator.

The aim of [36] was to study the estimation function theory proposed by [37,38,39] for CHARN models. More precisely, Ref. [36] proposed a general estimator of the estimation function and derived its asymptotic distribution.

Ref. [40] introduced a method for the parameters estimation of ARCH models. They derived the optimal estimating function (EF) by combining linear and quadratic estimating functions. They proved that this estimator is more efficient than the quasi-maximum-likelihood estimator and, under the conditional normality assumption, the constructed EF-estimator is identical to that obtained by using the maximum-likelihood method in finite samples. The simulation results show that the finite-sample properties of the EF-estimators are attractive. The latter tends to display less bias and root mean squared error than the quasi-maximum likelihood estimator.

Ref. [41] studied parameter estimation in a class of CHARN models. He investigated the properties of the conditional least-squares estimators and those of the conditional likelihood estimators. He defined the kernel estimators of noise density and its derivatives, and showed that they are uniformly consistent.

Several papers discussed the problem of estimation and inference in the exponential autoregressive models (EXPAR) introduced by [42]. Among others, one can cite [43,44,45,46,47,48].

3.2. The Kernel Estimation of the Density Function of the Noise

Denote by $\left|\right|U\left|\right|$ an appropriate norm of a vector or a matrix U, and by ${\displaystyle \frac{\partial U}{\partial \theta }(\theta ;z)}$ the matrix of the partial derivatives with respect to $\theta$, of the entries of a vector or matrix function $U(\theta ;z)$. In the sequel, we restrict ourselves to identifiable models (1) for which the following hold:

(A1) 

For all $\theta \in \mathrm{\Theta }\subset {\mathbb{R}}^q$ and $z\in {\mathbb{R}}^{pk}$, $\mathrm{\Sigma }(\theta ;z)$ is invertible with inverse ${\mathrm{\Sigma }}^{-1}(\theta ;z)$.

(A2) 

The functions $\theta \mapsto M(\theta ;z)$, $\theta \mapsto \mathrm{\Sigma }(\theta ;z)$ and $\theta \mapsto {\mathrm{\Sigma }}^{-1}(\theta ;z)$ are each continuously differentiable with respect to $\theta \in \mathrm{int}\left(\mathrm{\Theta }\right)$. There exists a finite positive number r such that the closed ball $\overline{B}({\theta }_0,r)$ is contained within $\mathrm{int}\left(\mathrm{\Theta }\right)$, and a positive function $\gamma \left(z\right)$ with $\mathbb{E}\left[{\gamma }^{2+\beta }\left(Y_0\right)\right]<\infty$ such that

$$max\left\{\underset{\theta \in \overline{B}({\theta }_0,r)}{sup}\left|\left|\frac{\partial M}{\partial \theta }(\theta ;z)\right|\right|,\underset{\theta \in \overline{B}({\theta }_0,r)}{sup}\left|\left|\frac{\partial \mathrm{\Sigma }}{\partial \theta }(\theta ;z)\right|\right|,\underset{\theta \in \overline{B}({\theta }_0,r)}{sup}\left|\left|\frac{\partial {\mathrm{\Sigma }}^{-1}}{\partial \theta }(\theta ;z)\right|\right|\right\}\le \gamma \left(z\right).$$

(A3) 

The true parameter ${\theta }_0\in \mathrm{\Theta }$ has a consistent estimator ${\tilde{\theta }}_n={({\tilde{\theta }}_{n,1},\dots ,{\tilde{\theta }}_{n,q})}^{\top }$ satisfying

$$E[\left|\right|\sqrt{n}({\tilde{\theta }}_n-{\theta }_0){\left|\right|}^2]⟶\rho <\infty ,n\to \infty$$

and

$$\sqrt{n}({\tilde{\theta }}_n-{\theta }_0)⟹\mathcal{N}(0,\Delta ),$$

where “⟹” denotes the convergence in distribution, and $\Delta$ is a $q\times q$ positive definite matrix.

Remark 1.

Assumptions (A1)–(A2) hold at least for some usual stationary multivariate time series models, such as vector ARMA, vector EXPAR or vector ARCH models. Assumption (A3) holds for the estimators ${\tilde{\theta }}_n$ satisfying the so-called Bahadur representation, as the likelihood, pseudo-likelihood, least-squares, conditional likelihood or conditional least-square estimators.

From model (1), by (A1), define for any $t\in \mathbb{Z}$, and for any $\theta \in \mathrm{\Theta }$,

$${\epsilon }_t\left(\theta \right)={\mathrm{\Sigma }}^{-1}(\theta ;Y_t)\left[X_t-M(\theta ;Y_t)\right],\theta \in \mathrm{\Theta },and{\tilde{\epsilon }}_t={\epsilon }_t({\tilde{\theta }}_n).$$

(5)

Proposition 1.

Assume that assumptions (A1)–(A3) hold. Then, for any $t\in \mathbb{Z}$,

$$\widetilde{{\epsilon }_t}\stackrel{P}{⟶}{\epsilon }_t\left({\theta }_0\right),n⟶\infty .$$

Proof. 

For any $t\in \mathbb{Z}$, one can write

$$\begin{array}{ccc}\hfill {\tilde{\epsilon }}_t-{\epsilon }_t({\theta }_0)& =& {\epsilon }_t({\tilde{\theta }}_n)-{\epsilon }_t({\theta }_0)\hfill \\ & =& {\mathrm{\Sigma }}^{-1}({\tilde{\theta }}_n;Y_t)\left[X_t-M({\tilde{\theta }}_n;Y_t)\right]-{\mathrm{\Sigma }}^{-1}({\theta }_0;Y_t)\left[X_t-M({\theta }_0;Y_t)\right]\hfill \\ & =& {\mathrm{\Sigma }}^{-1}({\tilde{\theta }}_n;Y_t)\left[M({\theta }_0;Y_t)-M({\tilde{\theta }}_n;Y_t)\right]-\left[{\mathrm{\Sigma }}^{-1}({\theta }_0;Y_t)-{\mathrm{\Sigma }}^{-1}({\tilde{\theta }}_n;Y_t)\right]\left[X_t-M({\theta }_0;Y_t)\right]\hfill \\ & =& {\mathrm{\Sigma }}^{-1}({\tilde{\theta }}_n;Y_t)\left[M({\theta }_0;Y_t)-M({\tilde{\theta }}_n;Y_t)\right]-\left[{\mathrm{\Sigma }}^{-1}({\theta }_0;Y_t)-{\mathrm{\Sigma }}^{-1}({\tilde{\theta }}_n;Y_t)\right]\mathrm{\Sigma }({\theta }_0;Y_t){\epsilon }_t({\theta }_0).\hfill \end{array}$$

Applying a first-order Taylor expansion to the differences within the above couples of brackets, since ${\tilde{\theta }}_n$ is consistent to ${\theta }_0$, the desired result follows immediately by assumptions (A2) and (A3).

Let $\tilde{K}$ be an almost-everywhere-differentiable Parzen–Rosenblatt kernel with gradient function ${\displaystyle \frac{\partial \tilde{K}}{\partial x}\left(x\right)}$ at x. One considers the kernel estimator of f defined by

$${\tilde{f}}_n\left(x\right)={\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right),x\in {\mathbb{R}}^k,$$

where the kernel function $\tilde{K}$ and the bandwidth $a_n$ satisfy the following assumptions:

(A4) 

$\tilde{K}$ is positive, even, and has compact support and bounded variation.

(A5) 

${\displaystyle a_n⟶0,\sqrt{n}{a}_n^k⟶\infty ,n\to \infty .}$

(A6) 

For all $\ell \ge 0$ and $x,y\in {\mathbb{R}}^k,$ ${\displaystyle a_n^{-\ell }{y}^{\top }\frac{\partial \tilde{K}}{\partial x}\left(x{a}_n^{-1}\right)⟶0}$, as $n⟶\infty$, and $y⟶y_0$, and the sequence of functions ${\displaystyle a_n^{-\ell }\frac{\partial \tilde{K}}{\partial x}\left(x{a}_n^{-1}\right)}$ is bounded.

□

Remark 2.

It can be easily verified that assumptions (A4) and (A6) hold at least for Gaussian and Epanechnikov kernels. As a consequence of (A4), $\int {\left|\right|y\left|\right|}^2\tilde{K}\left(y\right)dy<\infty .$ Assumption (A5) implies that ${\displaystyle n{a}_n^k/log\left(n\right)⟶\infty }$ as $n\to \infty .$

3.2.1. The Consistency of ${\tilde{f}}_n$ to f

Proposition 2.

Assume that the assumptions (A1)–(A6) are satisfied. Then,

$$\underset{x\in {\mathbb{R}}^k}{sup}|{\tilde{f}}_n\left(x\right)-f\left(x\right)|=o_P\left(1\right).$$

Proof. 

For any $t\in \mathbb{Z}$, recall the form of the $k\times q$ matrix ${\displaystyle \frac{\partial {\epsilon }_t}{\partial \theta }}$:

$$\frac{\partial {\epsilon }_t}{\partial \theta }=\left(\begin{array}{c}\frac{{\partial }^{\top }{\epsilon }_{t,1}}{\partial \theta }\\ ⋮\\ \frac{{\partial }^{\top }{\epsilon }_{t,k}}{\partial \theta }\end{array}\right),$$

where for any, $t\in \mathbb{Z}$, $1\le j\le k$,

$$\frac{{\partial }^{\top }{\epsilon }_{t,j}}{\partial \theta }=\left(\frac{\partial {\epsilon }_{t,j}}{\partial {\theta }_1},\dots ,\frac{\partial {\epsilon }_{t,j}}{\partial {\theta }_q}\right).$$

For any $x\in {\mathbb{R}}^k$,

$$\begin{array}{ccc}\hfill \left|{\tilde{f}}_n\left(x\right)-f_n\left(x\right)\right|& =& \left|{\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\left[\tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)\right]\right|\hfill \\ & \le & {\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\left|\tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)\right|\hfill \\ & \le & {\displaystyle \frac{1}{n}}\sum _{t=1}^n\left|{\displaystyle {\displaystyle \frac{1}{a_n^{k+1}}}{({\tilde{\epsilon }}_t-{\epsilon }_t)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)}\right|,\hfill \end{array}$$

where ${\stackrel{.}{\epsilon }}_t$ lies between ${\tilde{\epsilon }}_t$ and ${\epsilon }_t.$

Since for $t\in \mathbb{Z}$, ${\tilde{\epsilon }}_t$ tends in probability to ${\epsilon }_t$ as n tends to infinity, in view of our assumptions, one has that, in probability,

$$\left|{\displaystyle {\displaystyle \frac{1}{a_n^{k+1}}}{({\tilde{\epsilon }}_t-{\epsilon }_t)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)}\right|⟶0\mathrm{as}n\to \infty .$$

By a stochastic version of the Cesàro theorem (see Appendix A), one has that, in probability, the right-hand side of the last inequality tends to 0 as n tends to infinity. Consequently, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}\left|{\tilde{f}}_n\left(x\right)-f_n\left(x\right)\right|⟶0,n\to \infty .$$

Now, by Theorem 1 in Appendix A.1, it follows that, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}\left|f_n\left(x\right)-f\left(x\right)\right|⟶0\mathrm{as}n\to \infty .$$

Whence, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}\left|{\tilde{f}}_n\left(x\right)-f\left(x\right)\right|⟶0\mathrm{as}n\to \infty .$$

□

3.2.2. The Bias Study

Proposition 3.

Assume that assumptions (A1)–(A6) hold. Then,

$$\forall x\in {\mathbb{R}}^k,\mathbb{E}[{\tilde{f}}_n\left(x\right)]-f\left(x\right)⟶0,n\to \infty .$$

Proof. 

For all $x=(x_1,\dots ,x_k)\in {\mathbb{R}}^k$, define the function

$${\chi }_k\left(x\right)=\sum _{1\le i,j\le k}\frac{{\partial }^{2}f}{\partial x_i\partial x_j}\left(x\right)\int t_i{t}_{j}K(t_1,\dots ,t_k)d{t}_1\dots d{t}_k.$$

(6)

Then for all $x=(x_1,\dots ,x_k)\in {\mathbb{R}}^k$,

$$\begin{array}{ccc}\hfill \mathbb{E}[{\tilde{f}}_n(x)]-f(x)& =& \mathbb{E}[{\tilde{f}}_n(x)-f_n(x)]+[\mathbb{E}(f_n(x))-f(x)]\hfill \\ & =& {\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\mathbb{E}[\tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)]\hfill \\ & +& {\displaystyle \frac{a_n^2}{2}}{\chi }_k\left(x\right)+o\left(a_n^2\right).\hfill \end{array}$$

For all $t\in \mathbb{Z}$ and $x\in {\mathbb{R}}^k$, define the functions

$$\begin{array}{ccc}\hfill d_t\left(x\right)& =& {\displaystyle \frac{1}{a_n^k}}\mathbb{E}\left[\tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)\right]\hfill \\ \hfill w_n\left(x\right)& =& {\displaystyle \frac{1}{n}}\sum _{t=1}^n{d}_t\left(x\right).\hfill \end{array}$$

With these, one can write, for all $x\in {\mathbb{R}}^k,$

$$\mathbb{E}[{\tilde{f}}_n\left(x\right)]-f\left(x\right)=w_n\left(x\right)+{\displaystyle \frac{a_n^2}{2}}{\chi }_k\left(x\right)+o\left(a_n^2\right).$$

Now, we show that for all $x\in {\mathbb{R}}^k$, in probability, $w_n\left(x\right)⟶0$ as n tends to ∞.

By a first-order Taylor expansion of the function $\tilde{K}$, one obtains for all $x\in {\mathbb{R}}^k$ and $t\in \mathbb{Z}$,

$${\displaystyle \tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)={\displaystyle \frac{1}{a_n}}{({\tilde{\epsilon }}_t-{\epsilon }_t)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-\stackrel{.}{{\epsilon }_t}}{a_n}}\right),}$$

where for all $t\in \mathbb{Z}$, $\stackrel{.}{{\epsilon }_t}$ lies between ${\epsilon }_t$ and ${\tilde{\epsilon }}_t.$

By another first-order Taylor expansion applied to ${\epsilon }_t={\epsilon }_t\left(\theta \right)$, one then obtains, for some ${\dot{\theta }}_n={({\dot{\theta }}_{1n},\dots ,{\dot{\theta }}_{qn})}^{\prime }$ lying between ${\tilde{\theta }}_n$ and $\theta$,

$$\begin{array}{ccc}\hfill \tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)& =& {\displaystyle \frac{1}{a_n}}{\left(\frac{\partial {\epsilon }_t}{\partial \theta }\left({\dot{\theta }}_n\right)({\tilde{\theta }}_n-\theta )\right)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-\stackrel{.}{{\epsilon }_t}}{a_n}}\right)\hfill \end{array}$$

It results that, for all $t\in \mathbb{Z}$ and $x\in {\mathbb{R}}^k$,

$$\begin{array}{ccc}\hfill d_t\left(x\right)& =& {\displaystyle \frac{1}{\sqrt{n}{a}_n^k}}\mathbb{E}\left[{\displaystyle {\displaystyle \frac{1}{a_n}}{\left(\frac{\partial {\epsilon }_t}{\partial \theta }\left({\dot{\theta }}_n\right)\sqrt{n}({\tilde{\theta }}_n-\theta )\right)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-\stackrel{.}{{\epsilon }_t}}{a_n}}\right)}\right].\hfill \end{array}$$

By the Cauchy–Schwarz inequality, one has for all $t\in \mathbb{Z}$ and $x\in {\mathbb{R}}^k$,

$$|d_t\left(x\right)|\le {\displaystyle \frac{1}{\sqrt{n}{a}_n^k}}{\mathbb{E}}^{1/2}\left\{{\left[{\displaystyle \frac{{\partial }^{\top }{\epsilon }_t}{\partial \theta }({\dot{\theta }}_n){\displaystyle \frac{1}{a_n}}\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-\stackrel{.}{{\epsilon }_t}}{a_n}}\right)}\right]}^2\right\}{\mathbb{E}}^{1/2}\left(||\sqrt{n}(\tilde{\theta }-\theta ){||}^2\right).$$

By our assumptions and the dominated convergence theorem, it is easy to see that for all $t\in \mathbb{Z},x\in {\mathbb{R}}^k$,

$$d_t\left(x\right)⟶0,n\to \infty .$$

Whence, by Cesàro’s theorem (see Appendix A), one has that for all $x\in {\mathbb{R}}^k$,

$$w_n\left(x\right)={\displaystyle \frac{1}{n}}\sum _{t=1}^n{d}_t\left(x\right)⟶0,n\to \infty .$$

Given that for all $x\in {\mathbb{R}}^k$, ${\displaystyle \frac{a_n^2}{2}}{\chi }_k\left(x\right)+o\left(a_n^2\right)$ tends to 0 as n tends to ∞, one can then conclude that for all $x\in {\mathbb{R}}^k$,

$$\mathbb{E}[{\tilde{f}}_n\left(x\right)]-f\left(x\right)=w_n\left(x\right)+{\displaystyle \frac{a_n^2}{2}}{\chi }_k\left(x\right)+o\left(a_n^2\right)⟶0,n\to \infty .$$

This shows that for all $x\in {\mathbb{R}}^k$, ${\tilde{f}}_n\left(x\right)$ is an asymptotically unbiased estimator of $f\left(x\right)$. □

3.2.3. Asymptotic Normality

Proposition 4.

Assume that (A1)–(A6) hold. Then

$$\forall x\in {\mathbb{R}}^k,\sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(x\right)-f\left(x\right)}{{(f\left(x\right)\int {\tilde{K}}^2\left(u\right)du)}^{1/2}}}\right)⟹\mathcal{N}(0,1).$$

Proof. 

One has for all $x\in {\mathbb{R}}^k$,

$$\sqrt{n{a}_n^k}[{\tilde{f}}_n\left(x\right)-f\left(x\right)]=\sqrt{n{a}_n^k}[{\tilde{f}}_n\left(x\right)-f_n\left(x\right)]+\sqrt{n{a}_n^k}[f_n\left(x\right)-f\left(x\right)].$$

We first study the first term in the right-hand side of the above equality.

For all $x=(x_1,\dots ,x_k)\in {\mathbb{R}}^k$, by applications of Taylor expansions to $\tilde{K}$ and ${\epsilon }_t$ respectively, one can write

$$\begin{array}{ccc}\hfill \sqrt{n{a}_n^k}[{\tilde{f}}_n\left(x\right)-f_n\left(x\right)]& =& {\displaystyle \frac{1}{\sqrt{n{a}_n^k}}}\sum _{t=1}^n\left[\tilde{K}\left({\displaystyle \frac{x-{\tilde{\epsilon }}_t}{a_n}}\right)-\tilde{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)\right]\hfill \\ & =& {\displaystyle \frac{1}{\sqrt{n{a}_n^k}}}\sum _{t=1}^n{\displaystyle {\displaystyle \frac{1}{a_n}}{({\epsilon }_t-{\tilde{\epsilon }}_t)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)}.\hfill \\ & =& {\displaystyle \frac{1}{\sqrt{n{a}_n^k}}}\sum _{t=1}^n{\displaystyle {\displaystyle \frac{1}{a_n}}{\left(\frac{\partial {\epsilon }_t}{\partial \theta }({\dot{\theta }}_n)(\tilde{\theta }-\theta )\right)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)}.\hfill \\ & =& {\displaystyle \frac{1}{n}}\sum _{t=1}^n{\displaystyle {\displaystyle \frac{1}{{a_n}^{1+\frac{k}{2}}}}{\left(\frac{\partial {\epsilon }_t}{\partial \theta }({\dot{\theta }}_n)\sqrt{n}(\tilde{\theta }-\theta )\right)}^{\top }\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)},\hfill \end{array}$$

where ${\stackrel{.}{\epsilon }}_t$ lies between ${\epsilon }_t$ and ${\tilde{\epsilon }}_t$, and ${\dot{\theta }}_n$ lies between ${\tilde{\theta }}_n$ and $\theta$.

Define for all $t\in \mathbb{Z}$ and all $x\in {\mathbb{R}}^k,$

$$l_t\left(x\right)={\displaystyle \frac{1}{a_n^{1+k}}}\left|{\displaystyle \frac{\partial {\epsilon }_t}{\partial \theta }({\dot{\theta }}_n)\frac{\partial \tilde{K}}{\partial x}\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)}\right|.$$

By our assumptions, for any $x\in {\mathbb{R}}^k$, $l_t\left(x\right)$ tends in probability to 0 as n tends to infinity. Now, it is easy to see that

$$\begin{array}{ccc}\hfill \left|\sqrt{n{a}_n^k}[{\tilde{f}}_n\left(x\right)-f_n\left(x\right)]\right|& \le & {\displaystyle \frac{1}{n}}\sum _{t=1}^n{l}_t\left(x\right)||\sqrt{n}({\tilde{\theta }}_n-\theta )||\sqrt{a_n}.\hfill \end{array}$$

By assumption (A3), $||\sqrt{n}({\tilde{\theta }}_n-\theta )||$ is tight. Thus, for all $t\in \mathbb{Z}$ and all $x\in {\mathbb{R}}^k,$ $l_t\left(x\right)||\sqrt{n}({\tilde{\theta }}_n-\theta )||$$\sqrt{a_n}$ tends in probability to 0 as n tends to infinity. By a stochastic version of the Cesàro theorem (see Appendix A.4), for all $x\in {\mathbb{R}}^k$, in probability,

$$\left|\sqrt{n{a}_n^k}[{\tilde{f}}_n\left(x\right)-f_n\left(x\right)]\right|⟶0,n\to \infty .$$

Since from Theorem A3 in Appendix A.3, one has that for all $x\in {\mathbb{R}}^k$,

$$\sqrt{n{a}_n^k}\left[f_n\left(x\right)-f\left(x\right)\right]⟹\mathcal{N}\left(0,f\left(x\right)\int {\tilde{K}}^2\left(u\right)du\right),$$

it results that for all $x\in {\mathbb{R}}^k,$ as n tends to infinity,

$$\sqrt{n{a}_n^k}[{\tilde{f}}_n\left(x\right)-f\left(x\right)]⟹\mathcal{N}\left(0,f\left(x\right)\int {\tilde{K}}^2\left(u\right)du\right).$$

Then one concludes that for all $x\in {\mathbb{R}}^k,$ as n tends to infinity,

$$\sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(x\right)-f\left(x\right)}{{(f\left(x\right)\int {\tilde{K}}^2\left(u\right)du)}^{1/2}}}\right)⟹\mathcal{N}(0,1).$$

□

Proposition 5.

Assume that assumptions (A1)–(A6) hold. Then,

$$\sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(t_i\right)-f\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right)⟹{\mathcal{N}}_{\ell }\equiv \mathcal{N}(\mathbf{0},I_{\ell }),$$

where $\mathcal{N}(\mathbf{0},I_{\ell })$ denotes the standard ℓ-dimensional Gaussian random vector, $\ell \in \mathbb{N}$, $t_i\in {\mathbb{R}}^k$, $1\le i\le k$.

Proof. 

For all $\ell \in \mathbb{N}$, one can write

$$\begin{array}{ccc}\hfill \sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(t_i\right)-f\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right)& =& \sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(t_i\right)-f_n\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right)\hfill \\ & +& \sqrt{n{a}_n^k}\left({\displaystyle \frac{f_n\left(t_i\right)-f\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right).\hfill \end{array}$$

From Theorem A3 in Appendix A.3, as $n\to \infty ,$ one has that

$$\sqrt{n{a}_n^k}\left({\displaystyle \frac{f_n\left(t_i\right)-f\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right)⟹{\mathcal{N}}_{\ell }.$$

It follows from the proof of the above Proposition 4 that for $t\in {\mathbb{R}}^k,$ fixed,

$$\sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(t\right)-f_n\left(t\right)}{{\left(f\left(t\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}}\right)⟶0, n\to \infty .$$

It results from the above that as $n\to \infty ,$ one has that

$$\sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(t_i\right)-f_n\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right)⟶(0,\dots ,0).$$

So, as $n\to \infty ,$ one has

$$\sqrt{n{a}_n^k}\left({\displaystyle \frac{{\tilde{f}}_n\left(t_i\right)-f\left(t_i\right)}{{\left(f\left(t_i\right)\int {\tilde{K}}^2\left(u\right)du\right)}^{\frac{1}{2}}}},1\le i\le \ell \right)⟹{\mathcal{N}}_{\ell }.$$

A direct consequence of the above proposition is given by the following result. □

Proposition 6.

Assume that assumptions (A1)–(A6) hold. Then, for all $x\in {\mathbb{R}}^k,$ a confidence band for $f\left(x\right)$ at risk $\alpha \in$$[0,1]$ is given by

$${CB}_{\alpha }\left(x\right)=\left[{\tilde{f}}_n\left(x\right)-{\displaystyle \frac{q_{1-\frac{\alpha }{2}}{({\tilde{f}}_n\left(x\right)\int {\tilde{K}}^2\left(u\right)du)}^{1/2}}{\sqrt{n{a}_n^k}}};{\tilde{f}}_n\left(x\right)+{\displaystyle \frac{q_{1-\frac{\alpha }{2}}{({\tilde{f}}_n\left(x\right)\int {\tilde{K}}^2\left(u\right)du)}^{1/2}}{\sqrt{n{a}_n^k}}}\right],$$

where $q_{1-\frac{\alpha }{2}}$ stands for the $(1-\frac{\alpha }{2})$-quantile of the standard Gaussian distribution.

4. The Nonparametric Models Case

Here, considering model (1), we estimate f by a kernel method in the case that the vector and matrix functions T and $V\left(x\right)$ are unknown with unknown forms. That is, they are nonparametric functions. Because the study is lengthy and too technical, unlike in the preceding section, we only study the consistency and the bias of $f_n\left(x\right)$ defined by (2) when the ${\epsilon }_t$s are substituted for the residuals from the fit. In the first subsection, we give a brief review of some recent results on nonparametric estimation of T and V. The consistency and the bias studies are performed in the second subsection.

4.1. The Conditional Mean and Variance Functions Estimation

In the literature, the conditional mean (autoregression function) and the conditional variance (volatility) are estimated in various nonparametric methods. The estimators of the autoregression function are generally constructed and studied in the same ways as those of the well-known regression function. The pointwise convergence of this function was studied by [49,50,51]. The uniform convergence due to [52] generalizes the earlier results of [53,54,55,56]. The speed of convergence was treated by [57,58]. The $L^p$ convergence was obtained by [59,60,61,62], Gyorfi (1981). The assessment of bias and asymptotic variance were obtained by [49] for $k=1$ and [63,64] for $k\ge 1$.

Ref. [65] considered model (1) in which both T and V are unknown functions of the past. Nonparametric estimators of these functions are constructed based on local polynomial fitting. They examined the rate of convergence of these estimators and established their asymptotic normality. Indeed, they generalized the result of [9] to a wider class of conditional mean and variance functions that can be seen as a limit of CHARN models, which themselves are generalizations of the ARCH structure.

Ref. [66] studied kernel methods based on weighted averages of response variables. The resulting estimators are very sensitive to large fluctuations in the data. To overcome this drawback, Ref. [67] used a robust estimation. This allowed the authors to extend the results of [66] to a large class of processes under stationarity and ergodicity. They proposed a family of robust nonparametric estimators of the regression or autoregression functions based on the kernel method. They subsequently established the uniform convergence of this family of estimators in the case that the observations are not bounded and belong to an increasing series of compacts. Ref. [68] considered model (1). They proposed an efficient and adaptive method for estimating the conditional variance. Their basic idea is to apply local linear regression to the squared residuals. They proved that, without knowing the regression function, they can estimate the conditional variance asymptotically as well as if the regression were given. Ref. [69] surveyed the semi-parametric and nonparametric methods in univariate and multivariate ARCH/GARCH models. They introduced some specific semi-parametric models and investigated the semi-parametric and nonparametric estimation techniques applied to the error density, the functional form of the volatility functions, the relationships between mean and variance, long memory processes, locally stationary processes, continuous time processes and multivariate models. Ref. [70] applied bootstrap methods to CHARN models and gave non-parametric estimators of the trend and volatility function. Ref. [71] studied the estimation of regression and volatility functions of nonlinear autoregressive models with ARCH errors. In [72], the nonparametric local exponential estimator was applied to estimate conditional volatility functions, ensuring its non-negativity. It is proved there that the obtained estimator is asymptotically fully adaptive to unknown conditional mean functions. Ref. [73] proposed a kernel weighted version of the standard realized integrated volatility estimator and gave sample properties. In addition, Ref. [74] constructed a spot volatility estimator for high-frequency financial data which contain market microstructure noise, proved the consistency of this estimator and derived its asymptotic distribution. The aim of [75] was to establish asymptotic results for estimators of functions and functionals linked to various models. The studied estimators are built on the basis of nonparametric methods. Ref. [76] considered a multidimensional nonparametric additive regression model with dependent observations. the authors used the marginal integration technique and wavelets methodology to develop a new adaptive estimator for a component of the additive regression function. Refs. [67,77,78] proposed a family of robust nonparametric estimators for regression or autoregression functions in a univariate or multivariate context.

4.2. The Kernel Estimation of the Density of the Noise

In this subsection, we assume the functions T and V unknown and nonparametric with unknown forms. We study the kernel estimator defined by (2) when the ${\epsilon }_t$s are substituted for the residuals from the fit. For the reasons mentioned in the introduction, we do not study the asymptotic normality of this estimator.

Let $\widehat{T}={({\widehat{T}}_1,\dots ,{\widehat{T}}_k)}^{\top }$, $\widehat{V}=({\widehat{V}}_{ij})$ and ${\widehat{V}}^{-1}=({\widehat{V}}_{ij}^{-1})$ be nonparametric estimators of $T={(T_1,\dots ,T_k)}^{\top }$, $V=(V_{ij})$ and $V^{-1}=(V_{ij}^{-1})$, respectively, satisfying, for any $t\in \mathbb{Z}$,

(A7) 

$\widehat{T}(Y_t)-T(Y_t)=o_P(1),\widehat{V}(Y_t)-V(Y_t)=o_P(1),{\widehat{V}}^{-1}(Y_t)-V^{-1}(Y_t)=o_P(1)$.

Remark 3.

(A7) is readily satisfied by estimators derived in some of the references cited above. $\widehat{T}$ can be obtained by kernel methods as in [66,67] or by local polynomial methods as in [6], or by adaptive methods as in [76]. $\widehat{V}$ can be obtained by local polynomial approaches as in [1,6,65] or by a local exponential method as in [72].

For any $t\in \mathbb{Z}$, define

$${\epsilon }_t={\epsilon }_t(T,V)=V^{-1}\left(Y_t\right)\left[X_t-T\left(Y_t\right)\right]and{\widehat{\epsilon }}_t={\epsilon }_t(\widehat{T},\widehat{V}).$$

It is easy to see from assumption (A7) that for any $t\in \mathbb{Z}$,

$${\widehat{\epsilon }}_t={\epsilon }_t+o_P\left(1\right).$$

In the sequel, the kernel considered is denoted by $\widehat{K}$ rather than $\tilde{K}$ as in the preceding section. We study the asymptotic behavior of the following kernel estimator:

$${\widehat{f}}_n\left(x\right)={\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\widehat{K}\left({\displaystyle \frac{x-{\widehat{\epsilon }}_t}{a_n}}\right),x\in {\mathbb{R}}^k.$$

Proposition 7.

Assume that (A4)–(A7) hold. Then, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}\left|{\widehat{f}}_n\left(x\right)-f\left(x\right)\right|⟶0,n\to \infty .$$

Proof. 

For all $x=(x_1,\dots ,x_k)\in {\mathbb{R}}^k,$ one can write

$$\begin{array}{ccc}\hfill \left|\widehat{f_n}\left(x\right)-f_n\left(x\right)\right|& =& \left|{\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\left[\widehat{K}\left({\displaystyle \frac{x-{\widehat{\epsilon }}_t}{a_n}}\right)-\widehat{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)\right]\right|\hfill \\ & =& |{\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n[\widehat{K}\left({\displaystyle \frac{x-{\widehat{V}}^{-1}\left(Y_t\right)\left[X_t-\widehat{T}\left(Y_t\right)\right]}{a_n}}\right)\hfill \\ & -& \widehat{K}\left({\displaystyle \frac{x-V^{-1}\left(Y_t\right)\left[X_t-T\left(Y_t\right)\right]}{a_n}}\right)\left]\right|\hfill \\ & \le & {\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n|\widehat{K}\left({\displaystyle \frac{x-{\widehat{V}}^{-1}\left(Y_t\right)\left[X_t-\widehat{T}\left(Y_t\right)\right]}{a_n}}\right)\hfill \\ & -& \widehat{K}\left({\displaystyle \frac{x-V^{-1}\left(Y_t\right)\left[X_t-T\left(Y_t\right)\right]}{a_n}}\right)|.\hfill \end{array}$$

Recalling that ${\frac{\partial K}{\partial x}}^{\top }\left(x\right)$ denotes the transpose of $\frac{\partial K}{\partial x}\left(x\right)$, by a Taylor expansion of $\widehat{K}$, one has

$$\begin{array}{ccc}& & \widehat{K}\left({\displaystyle \frac{x-{\widehat{V}}^{-1}\left(Y_t\right)\left[X_t-\widehat{T}\left(Y_t\right)\right]}{a_n}}\right)-\widehat{K}\left({\displaystyle \frac{x-V^{-1}\left(Y_t\right)\left[X_t-T\left(Y_t\right)\right]}{a_n}}\right)\hfill \\ & =& {\displaystyle \frac{1}{a_n}}{\frac{\partial \widehat{K}}{\partial x}}^{\top }\left(\frac{{\stackrel{.}{\epsilon }}_{t,x}}{a_n}\right)\left\{V^{-1}\left(Y_t\right)\left[X_t-T\left(Y_t\right)\right]-{\widehat{V}}^{-1}\left(Y_t\right)\left[X_t-\widehat{T}\left(Y_t\right)\right]\right\},\hfill \end{array}$$

where for any $t\in \mathbb{Z},x\in {\mathbb{R}}^k$, ${\stackrel{.}{\epsilon }}_{t,x}$ is a random vector lying between $x-{\epsilon }_t$ and $x-{\widehat{\epsilon }}_t$.

Now, observing that

$$\begin{array}{ccc}& & V^{-1}\left(Y_t\right)\left[X_t-T\left(Y_t\right)\right]-{\widehat{V}}^{-1}\left(Y_t\right)\left[X_t-\widehat{T}\left(Y_t\right)\right]\hfill \\ & =& \left[V^{-1}\left(Y_t\right)-{\widehat{V}}^{-1}\left(Y_t\right)\right]{\epsilon }_t(T,V)+{\widehat{V}}^{-1}\left(Y_t\right)\left[\widehat{T}\left(Y_t\right)-T\left(Y_t\right)\right]\hfill \end{array}$$

It results from this that

$$\begin{array}{ccc}\hfill \underset{\begin{array}{c}x\in {\mathbb{R}}^k\end{array}}{sup}\left|{\widehat{f}}_n\left(x\right)-f_n\left(x\right)\right|& \le & {\displaystyle \frac{1}{n}}\sum _{t=1}^n\underset{x\in {\mathbb{R}}^k}{sup}|{\displaystyle \frac{1}{a_n^{k+1}}}{\frac{\partial \widehat{K}}{\partial x}}^{\top }\left(\frac{{\stackrel{.}{\epsilon }}_{t,x}}{a_n}\right)\hfill \\ & & \left\{\left[V^{-1}\left(Y_t\right)-{\widehat{V}}^{-1}\left(Y_t\right)\right]{\epsilon }_t(T,V)+{\widehat{V}}^{-1}\left(Y_t\right)\left[\widehat{T}\left(Y_t\right)-T\left(Y_t\right)\right]\right\}|.\hfill \end{array}$$

By our assumptions, the terms in the summand in the right-hand side of the above inequality tends to 0 as n tends to infinity. Then, by a stochastic version of the Cesàro theorem (see Appendix A.4), in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}\left|{\widehat{f}}_n\left(x\right)-f_n\left(x\right)\right|⟶0,n\to \infty .$$

It is well known (see Theorem A1, Appendix A.1) that, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}|f_n\left(x\right)-f\left(x\right)|⟶0,n\to \infty .$$

Consequently, in probability,

$$\underset{x\in {\mathbb{R}}^k}{sup}\left|{\widehat{f}}_n\left(x\right)-f\left(x\right)\right|⟶0,n\to \infty .$$

□

Bias Study of ${\widehat{f}}_n$

Proposition 8.

Assume that (A4)–(A7) hold, and that $\widehat{K}$ is bounded by a positive number. Assume that $\int \left|\right|x\left|\right|f\left(x\right)dx<\infty$. Then, for all $x\in {\mathbb{R}}^k,$${\widehat{f}}_n\left(x\right)$ is an asymptotically unbiased estimator of $f\left(x\right)$.

Proof. 

For all $x\in {\mathbb{R}}^k,$

$$\mathbb{E}[{\widehat{f}}_n\left(x\right)]-f\left(x\right)=\mathbb{E}[\widehat{f}\left(x\right)-f_n\left(x\right)]+\left\{\mathbb{E}[f_n\left(x\right)]-f\left(x\right)\right\}.$$

By Theorem A2 in Appendix A.2, as $n\to \infty ,$

$$\mathbb{E}\left[f_n\left(x\right)\right]-f\left(x\right)⟶0.$$

Now, we show that for all $x\in {\mathbb{R}}^k,$ as $n\to \infty ,$

$$\mathbb{E}[{\widehat{f}}_n\left(x\right)-f_n\left(x\right)]⟶0.$$

Indeed, for $x\in {\mathbb{R}}^k,$

$$\begin{array}{ccc}\hfill \mathbb{E}[{\widehat{f}}_n\left(x\right)-f_n\left(x\right)]& =& {\displaystyle \frac{1}{n{a}_n^k}}\sum _{t=1}^n\mathbb{E}\left[\widehat{K}\left({\displaystyle \frac{x-{\widehat{\epsilon }}_t}{a_n}}\right)-\widehat{K}\left({\displaystyle \frac{x-{\epsilon }_t}{a_n}}\right)\right]\hfill \\ & =& {\displaystyle \frac{1}{n}}\sum _{t=1}^n\mathbb{E}\left[{\displaystyle \frac{1}{a_n^{k+1}}}{({\widehat{\epsilon }}_t-{\epsilon }_t)}^{\top }{\frac{\partial \widehat{K}}{\partial x}}^{\top }\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)\right]\hfill \\ & =& {\displaystyle \frac{1}{n}}\sum _{t=1}^n{L}_t\left(x\right),\hfill \end{array}$$

where, for any $t\in \mathbb{Z}$ and $x\in {\mathbb{R}}^k,$

$$\begin{array}{ccc}\hfill L_t\left(x\right)& =& \mathbb{E}\left[{\displaystyle \frac{1}{a_n^{k+1}}}{({\widehat{\epsilon }}_t-{\epsilon }_t)}^{\top }{\frac{\partial \widehat{K}}{\partial x}}^{\top }\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)\right]\hfill \\ & =& \mathbb{E}\{{\displaystyle \frac{1}{a_n^{k+1}}}{\frac{\partial \widehat{K}}{\partial x}}^{\top }\left({\displaystyle \frac{x-{\stackrel{.}{\epsilon }}_t}{a_n}}\right)\hfill \\ & & \left[V^{-1}\left(Y_t\right)-{\widehat{V}}^{-1}\left(Y_t\right)\right]{\epsilon }_t(T,V)+{\widehat{V}}^{-1}\left(Y_t\right)\left[\widehat{T}\left(Y_t\right)-T\left(Y_t\right)\right]\},\hfill \end{array}$$

where for all $t\in \mathbb{Z}$, $\stackrel{.}{{\epsilon }_t}$ lies between ${\widehat{\epsilon }}_t$ and ${\epsilon }_t$.

Since for all $t\in \mathbb{Z}$, $T\left(Y_t\right)-\widehat{T}\left(Y_t\right)$ and $V^{-1}\left(Y_t\right)-{\widehat{V}}^{-1}\left(Y_t\right)⟶0$ in probability as $n\to \infty$. The above terms in the pairs of brackets tend to 0 in probability as n tends to infinity. It follows from our assumptions that for all $t\in \mathbb{Z}$ and all $x\in {\mathbb{R}}^k,$

$$L_t\left(x\right)⟶0,n\to \infty .$$

Thus, by the Cesàro Theorem, for all $x\in {\mathbb{R}}^k,$

$$\mathbb{E}[{\widehat{f}}_n\left(x\right)-f_n\left(x\right)]={\displaystyle \frac{1}{n}}\sum _{t=1}^n{L}_t\left(x\right)⟶0,n\to \infty .$$

This concludes the proof of the result. □

5. Simulation Experiments

In this section, we conduct a simulation experiment to evaluate the finite sample properties of the estimators studied in the preceding sections. We restrict to the cases $k=1,2$. For $k=1$, we study the situation where the conditional mean and variance functions are parametric, and where they are nonparametric. For $k=2$, we only study the case that these functions are parametric.

Although we do not study the asymptotic normality of the kernel estimator in the case that the functions T and V are nonparametric, we construct heuristically asymptotic confidence-bound functions in the lines of those obtained in the case that T and V are parametric.

5.1. Unidimensional Case

Let $\theta ={({\theta }_0,{\theta }_1,{\theta }_2,{\theta }_3)}^{\top }$. For several values of $\theta$, we generate n observations from model (1) with

$$T\left(x\right)=\left({\theta }_0+{\theta }_1{e}^{-0.03x^2}\right)x\mathrm{and}V\left(x\right)=\sqrt{{\theta }_2+{\theta }_3{x}^2}$$

(7)

and f is the standard Gaussian density function. Once the data are generated, in order to apply our results, we first assume that functions T and V have the above forms with unknown $\theta$, estimated by a maximum likelihood method. Next, we assume that the forms of these functions are unknown and they are estimated by a kernel method.

For all the kernel methods used, the bandwidth is the same $a_n={\sigma }_n{n}^{-1/5}$, with ${\sigma }_n^2$ standing for the sample variance of the simulated observations. The kernel considered is the Gaussian kernel. For any $x\in \mathbb{R}$, the kernel estimator of $f\left(x\right)$ and its associated confidence bounds are computed on the basis of 1000 replications for a sample size of $n=200$ at each replication.

Figure 1 and Figure 2 respectively exhibit the results corresponding to the parametric and nonparametric cases. They both show the graph of the true density (black color), the upper-bound graph (red), the lower-bound graph (green) and that of the estimator $f_n$ (blue). Either of these graphs indicate that the Gaussian density of the noise is globally well estimated for the examples considered.

5.2. Bidimensional Case

Let $\theta ={({\theta }_1,{\theta }_2)}^{\top }$. For $k=2$, we generate n observations $X_1={(X_{1,1},X_{1,2})}^{\top },\dots ,$$X_n={(X_{n,1},X_{n,2})}^{\top }$ from model (1) with $T\left(x\right)={(T_1\left(x\right),T_2\left(x\right))}^{\top }$, where for any $x={(x_1,x_2)}^{\top }\in {\mathbb{R}}^2$, $T_1\left(x\right)={\theta }_1{e}^{0.03x_1^2}{x}_1$, $T_2\left(x\right)={\theta }_2{x}_2$, $V\left(x\right)=1$, and f is the standard bidimensional density function. We do this for ${\theta }_1=0.5$, ${\theta }_2=0.75$.

As mentioned above, we only treat the parametric case. That is, we now assume that the functions T and V have the above parametric form with unknown $\theta$ estimated by a maximum likelihood method. Then, the kernel estimator of f is computed with a bidimensional standard Gaussian kernel. Since this can be written as a product of two univariate standard Gaussian kernels, we consider, for each of these kernels, the smoothing parameters $a_{n,j}={\sigma }_{n,j}{n}^{-\frac{1}{5}}$, where ${\sigma }_{n,j}^2$ is the sample variance of the $X_{1,j},\dots ,X_{n,j}$, $j=1,2$.

Here, we take $n=800$, although smaller values give satisfactory results. For any $x\in {\mathbb{R}}^2$, $f_n\left(x\right)$ and its confidence bounds are computed on the basis of 1000 replications. Figure 3 shows, for the single model considered, the graph of the true density (blue color), the upper-bound graph (red), the lower-bound graph (green) and that of the estimator $f_n$ (blue). The four graphs indicate that the bidimensional Gaussian density of the noise is well estimated for the example considered.

6. Conclusions

We studied the kernel estimation of the density f of the noise in model (1) when the functions T and V are both parametric and when they are both nonparametric. In the former case, the consistency of the estimator as well as its pointwise asymptotic normality are established. This led to the construction of pointwise confidence-bound functions. It is interesting to note that from Proposition 5, Wald-type tests can be constructed for goodness-of-fit tests on f. In the case that T and V are nonparametric, only the the consistency of the estimator is studied. The main reason for this is that the study seemed to us too long and too technical.

The simulation experiment conducted shows that the estimator behaves quite well on the examples considered. It can be seen from the graphics associated with the results obtained for the parametric functions T and V that the curve of the true density function generally lies within the curves of the pointwise confidence-bound functions. Despite the fact that the pointwise asymptotic normality of ${\widehat{f}}_n$ was not established (T and V are nonparametric), in light of the case where they are parametric, we wrote down the expressions of the confidence bound functions for f. It seems from the simulation results (see Figure 2) that the curve of true density is usually between those of the confidence-bound curves.