Polynomial Regressions and Nonsense Inference

Abstract

Polynomial specifications are widely used, not only in applied economics, but also in epidemiology, physics, political analysis and psychology, just to mention a few examples. In many cases, the data employed to estimate such specifications are time series that may exhibit stochastic nonstationary behavior. We extend Phillips’ results (Phillips, P. Understanding spurious regressions in econometrics. J. Econom. 1986, 33, 311–340.) by proving that an inference drawn from polynomial specifications, under stochastic nonstationarity, is misleading unless the variables cointegrate. We use a generalized polynomial specification as a vehicle to study its asymptotic and finite-sample properties. Our results, therefore, lead to a call to be cautious whenever practitioners estimate polynomial regressions.

1. Introduction

There is some research on the effects of the nonstationarity of the variables on nonlinear relationships (spurious inference on linear regressions was uncovered by [1], and later explained by [2]). In [3], it is shown (both in finite samples and asymptotically) that six nonlinear tests (the Ramsey Regression Equation Specification Error Test (RESET), McLeod and Li test, Keenan test, Neural Network test, White’s information matrix, and the one proposed by [4]), when applied to independent random walks, tend to identify spurious (non-existing) nonlinear relationships (it is noteworthy that [5] studied the spurious regression phenomenon under stochastic nonstationarity when the logarithms of independent integrated of order one, I(1), variables are used; logarithmic transformations are commonly used in applied studies to deal with nonlinearity). The author of [6] extend these results by studying the behavior of two additional tests: the Brock-Dechert-Scheinkman (BDS) test and another one proposed by [7]; he finds that the former also yields results that do not make sense, whilst the latter proves to have good power properties even in small samples. The author of [8] studies the properties of the nonparametric Phillip’s unit root test applied to polynomials of integrated processes and concludes, broadly speaking, that the tests does not possess an asymptotic nuisance-parameter-free distribution, except under very specific conditions.

To the best of our knowledge, the “nonlinear relationship-spurious inference” literature (briefly sketched earlier) focuses on statistical tests rather than polynomial regressions. The latter are used to linearly relate the dependent variable to a k$th$ order polynomial on an independent variable, x. Such regressions therefore fit (through ordinary least squares, OLS) a nonlinear relationship between a polynomial on the independent variable and the conditional mean of y.

These specifications can be traced back to the nineteenth century, to impute series (see [9]). Despite its old age, polynomial regressions remain widely used in a large number of scientific fields, which include epidemiology/disease progression [10], geophysics [11], physics [12], political analysis [13], psychology [14], and, of course, statistics. Splines regression models (cubic splines, for example) can be used to smooth/impute series.

In empirical economics, polynomial specifications can be found in many subfields, such as, financial economics [15,16], labor economics [17,18], agricultural economics [19], macroeconomics (exchange rates, [20]) and environmental economics [21,22]. An evocative example can be found in the empirical research dealing with the Kuznets curve and the environmental Kuznets curve; the inverse U-shaped relationship between the variables is typically specified as the dependent variable regressed on the independent and its square (see [23,24]; it is noteworthy that Kuznets’ specifications usually employ even-order polynomials).

Even though polynomial regressions remain an important empirical tool, we could not find in the literature any attempt to study their properties when the variables behave as independent nonstationary processes. This might be so because the effect of nonstationarity is rather intuitive, and econometricians, at least those familiar with the spurious regression, could speculate that t-ratios diverge and the $R^2$ does not collapse. However, many researchers in diverse fields seem to be unaware of this possibility.

In this paper, we confirm that an inference drawn from a polynomial regression, when the variables are generated as independent integrated processes, is misleading (when the variables cointegrate, inference drawn from such a specification is no longer misleading). We provide evidence that generalizes Phillip’s results in two new directions: (i) we allow the exponent of the variables, both explanatory and dependent in a bivariate regression, to take any natural number; (ii) we allow for an arbitrary (natural number) order for the polynomial in x in a k-variate regression. The main objective of this work is to warn practitioners about the considerable risks of spurious inference when the powers of a nonstationary variable are used as regressors.

This paper is organized in a very simple manner. The next section presents the data-generating processes (DGPs) and the main results, divided in two theorems. A small Monte Carlo shows that the asymptotics are a sufficiently accurate representation of the finite sample behavior of the regressions.

2. Asymptotics of Polynomial Regressions

The variables, both dependent and independent, are generated as independent driftless unit roots:

$$z_t=z_{t-1}+u_{z,t}$$

(1)

for $z=x,y$. The innovations, $u_{y,t}$ and $u_{x,t}$, are independent of each other and obey the conditions stated by Phillips ([2], p. 313, Assumption 1). We use these variables to estimate the following specification:

$$y_t^m=\alpha +\beta x_t^k+u_t$$

(2)

where $m,k\in \mathbb{N}$. A word on notation; the symbol, $\stackrel{D}{\to }$, denotes weak convergence, and, for simplicity, $W_z\equiv W_z\left(r\right)$, for $z=x,y$, denotes a Wiener standard process. The stochastic integral, ${\int }_0^1$, is written as ∫.

Theorem 1.

Let ${\left\{y_t\right\}}_{t=1}^{\infty }$ and ${\left\{x_t\right\}}_{t=1}^{\infty }$ be independently generated by Equation (1). Estimate by OLS specification Equation (2). Then, as $T\to \infty$:

• $T^{-\frac{m}{2}}\widehat{\alpha }\stackrel{D}{\to }{\sigma }_y^m\left[\frac{\int w_y^m\int w_x^{2k}-\int w_x^k{w}_y^m\int w_x^k}{\int w_x^{2k}-{\left(\int w_x^k\right)}^2}\right]$
• $T^{-\frac{1}{2}(m-k)}\widehat{\beta }\stackrel{D}{\to }\frac{{\sigma }_y^m}{{\sigma }_x^k}\left[\frac{\int w_x^k{w}_y^m-\int w_x^k\int w_y^m}{\int w_x^{2k}-{\left(\int w_x^k\right)}^2}\right]\equiv \frac{{\sigma }_y^m}{{\sigma }_x^k}\tilde{\beta }$
• $T^{-\frac{1}{2}}{t}_{\widehat{\beta }}\stackrel{D}{\to }\frac{\int w_x^k{w}_y^m-\int w_x^k\int w_y^m}{{\left[\left(\int w_x^{2k}-{\left(\int w_x^k\right)}^2\right)\left(\int w_y^{2m}-{\left(\int w_y^m\right)}^2-{\left(\int w_x^k{w}_y^m-\int w_x^k\int w_y^m\right)}^2\right)\right]}^{\frac{1}{2}}}$
• $R^2\stackrel{D}{\to }{\tilde{\beta }}^2\left[\frac{\int w_x^{2k}-{\left(\int w_x^k\right)}^2}{\int w_y^{2m}-{\left(\int w_y^m\right)}^2}\right]$

Proof:

See Appendix A.

Note that all these results are an extension of [2]. It is noteworthy to mention that, for $k=m=1$, our results are exactly those of [2]. This implies that, no matter what power does the practitioner applies to the variables, the spurious regression phenomenon remains identical. That said, a more interesting specification should allow for a more complete polynomial of the independent variable, as in:

$$y_t={\beta }_0+{\beta }_1{x}_t+{\beta }_2{x}_t^2+\cdots +{\beta }_k{x}_t^k+u_t$$

(3)

where $k\in \mathbb{N}$. In this case, OLS estimates still generate a spurious regression:

Theorem 2.

Let ${\left\{y_t\right\}}_{t=1}^{\infty }$ and ${\left\{x_t\right\}}_{t=1}^{\infty }$ be independently generated by Equation (1). Estimate by OLS specification Equation (3). Then, as $T\to \infty$:

• $$\begin{array}{ccccc}\left[\begin{array}{c}{T}^{-\frac{1}{2}}{\widehat{\beta }}_0\\ \\ {\widehat{\beta }}_1\\ \\ T^{\frac{1}{2}}{\widehat{\beta }}_2\\ \\ ⋮\\ \\ T^{\frac{1}{2}(k-1)}{\widehat{\beta }}_k\end{array}\right]& \stackrel{D}{\to }& {\sigma }_y& {\left(\begin{array}{ccccc}1& 0& 0& \cdots & 0\\ \\ 0& {\sigma }_x& 0& \cdots & 0\\ \\ 0& 0& {\sigma }_x^2& \cdots & 0\\ \\ ⋮& ⋮& ⋮& \cdots & ⋮\\ \\ 0& 0& 0& \cdots & {\sigma }_x^k\end{array}\right)}^{-1}\times & \\ \\ & & & {\left(\begin{array}{ccccc}1& \int w_x& \int {\omega }_x^2& \cdots & \int w_x^k\\ \\ \int w_x& \int {\omega }_x^2& \int {\omega }_x^3& \cdots & \int {\omega }_x^{k+1}\\ \\ \int {\omega }_x^2& \int {\omega }_x^3& \cdots & \cdots & \int {\omega }_x^{k+2}\\ \\ ⋮& & & & ⋮\\ \\ \int w_x^k& \cdots & \cdots & \cdots & \int {\omega }_x^{2k}\\ \end{array}\right)}^{-1}& \left(\begin{array}{c}\int w_y\\ \\ \int {\omega }_x{\omega }_y\\ \\ \int {\omega }_x^2{\omega }_y\\ \\ ⋮\\ \\ \int {\omega }_x^k{\omega }_y\end{array}\right)\end{array}$$
• $S^2=Op\left(T\right)$, where $S^2=T^{-1}{\sum }_{t=1}^T{\left(y_t-{\sum }_{i=0}^k{\widehat{\beta }}_i{x}_t^i\right)}^2$
• $t_{{\widehat{\beta }}_i}=Op\left(T^{\frac{1}{2}}\right)$ for $i=0,1,2,\dots ,k$

Proof:

See Appendix B.

Table 1. Rejection rates of t-ratios.

T	Specification (2)							Specification (3)
		k		m				With k = 4
				1	2	3			${\beta }_1$	${\beta }_2$	${\beta }_3$	${\beta }_4$
100		1		0.77	0.71	0.71
		2		0.71	0.66	0.65			0.46	0.35	0.33	0.31
		3		0.72	0.66	0.66
250		1		0.85	0.82	0.82
		2		0.81	0.78	0.78			0.64	0.56	0.52	0.50
		3		0.82	0.78	0.78
500		1		0.89	0.87	0.87
		2		0.86	0.84	0.84			0.73	0.67	0.64	0.63
		3		0.88	0.84	0.84

Rejection rates of the t-ratio associated with: (i) for specification Equation (2), $\widehat{\beta }$; (ii) for specification Equation (3), all β’s. Data-generating process (DGP) parameters: $u_{z,t}\sim iid\mathcal{N}(0,1)$, for $z=x,y$. The code of this Monte Carlo experiment is available as supplementary material.

Table 1. Rejection rates of t-ratios.

T	Specification (2)							Specification (3)
		k		m				With k = 4
				1	2	3			${\beta }_1$	${\beta }_2$	${\beta }_3$	${\beta }_4$
100		1		0.77	0.71	0.71
		2		0.71	0.66	0.65			0.46	0.35	0.33	0.31
		3		0.72	0.66	0.66
250		1		0.85	0.82	0.82
		2		0.81	0.78	0.78			0.64	0.56	0.52	0.50
		3		0.82	0.78	0.78
500		1		0.89	0.87	0.87
		2		0.86	0.84	0.84			0.73	0.67	0.64	0.63
		3		0.88	0.84	0.84

Rejection rates of the t-ratio associated with: (i) for specification Equation (2), $\widehat{\beta }$; (ii) for specification Equation (3), all β’s. Data-generating process (DGP) parameters: $u_{z,t}\sim iid\mathcal{N}(0,1)$, for $z=x,y$. The code of this Monte Carlo experiment is available as supplementary material.

Note the linear pattern in the order of convergence of the parameters; whilst the constant term, ${\widehat{\beta }}_0$, diverges at rate $T^{\frac{1}{2}}$, ${\widehat{\beta }}_1$ neither diverges, nor collapses, ${\widehat{\beta }}_2$ collapses at rate $T^{-\frac{1}{2}}$, and so on. Nonetheless, all the t-ratios of the estimated parameters diverge at the usual rate $T^{\frac{1}{2}}$.

In both theorems, the convergence rate of the t-ratios associated with the estimates diverge. This implies that, for a sufficiently large sample, the null hypothesis that the parameters are equal to zero will eventually be rejected. Finite sample evidence suggests that this actually occurs in even rather small samples of 100–500 observations (Table 1).

3. Concluding Remarks

In this paper, we extended the results of what is known as spurious inference by studying the asymptotic and finite-sample behavior of the t-ratios in an OLS-estimated regression, where the dependent variable and/or the explanatory variable are nonlinearly transformed by means of a polynomial. When the variables are independent and stochastically nonstationary, the inference based on OLS estimates is misleading. Our results concern pure integrated of order one processes, but provide a natural guide to future research; near-integration, integrated of order two, and broken linear trend processes should be further studied. This result should be understood as a call to be cautious whenever practitioners estimate polynomial regressions.