Statistical Tests of Symbolic Dynamics ^†

Abstract

A novel general method for constructing nonparametric hypotheses tests based on the field of symbolic analysis is introduced in this paper. Several existing tests based on symbolic entropy that have been used for testing central hypotheses in several branches of science (particularly in economics and statistics) are particular cases of this general approach. This family of symbolic tests uses few assumptions, which increases the general applicability of any symbolic-based test. Additionally, as a theoretical application of this method, we construct and put forward four new statistics to test for the null hypothesis of spatiotemporal independence. There are very few tests in the specialized literature in this regard. The new tests were evaluated with the mean of several Monte Carlo experiments. The results highlight the outstanding performance of the proposed test.

1. Introduction

The construction and design of powerful statistical tests are crucial elements for both theoretical and applied scientists. The utility of a test generally depends on its degree of applicability, which is usually related to the assumptions contained in the design of the test, and the restrictions of the scientific field in which the test will be used. Nowadays, the utility of statistical tests also depends on efficiency: reducing the need for computational resources and speed, which are vital for real-time monitoring and control applications. Taking applicability and efficiency into account, in this paper we propose a new general, flexible statistical methodology to design and test central hypotheses, and we establish an asymptotic distribution theory for a wide range of tests by using the new proposed approach.

The new framework is based on symbolic analysis, which is a field of increasing interest for several scientific disciplines (see [1]) Symbolic analysis studies dynamical systems on the basis of the sequences of symbols which are obtained for a suitable (and generally selected by the user) partition of the state space. In other words, the idea behind the symbolic approach is to split the phase space into a finite number of regions, and then each region is labeled with a symbol. From this point of view, the symbolic approach is a coarse-grained description of dynamics. As coarse-grained methods, which are usually used to provide some description of the data generating process, symbolic analysis focuses on some essential features of the generating dynamics which are frequently of interest to the researcher, for example, (in)dependence, cycles and nonlinear structure. In general terms, it can be said that symbolic analysis allows for designing tests that only focus on the relevant information required for the problem at hand.

This approach is not new in science. In the particular case of time series analysis, the symbolic approach implies transforming raw time series into a sequence of symbols. Although seeminglycounter-intuitive, symbolic analysis is rooted in information theory and also in dynamics theory. For example, properties of symbols or codes are central to the theory of communication [2]. Not in vein, there is a well-established mathematical discipline, namely, symbolic dynamics, that studies the behavior of dynamical systems. The name of “symbolic dynamics” was firstly coined by [3], although the discipline started in 1898 with the pioneering work by Hadamard, who developed a symbolic description of sequences of geodesic flows. Interestingly, ref. [4] highlighted the power of the symbolic approach by showing that a complete description of the behavior of a dynamical system can be captured in terms of symbols. Notice that this property is crucial for the understanding of this paper as long as important characteristics of a random variable can also be captured by studying the symbols derived from it.

The symbolic approach has been useful in many areas of scientific research. In the experimentalist realm, relevant contributions have been made in several fields: astrophysics; biology and medicine; chemistry, mechanical systems and fluid flow; artificial intelligence, control and communication; and data mining, classification and rule discovery ([5,6,7], for an overview). In the non-experimentalist realm, symbolic analysis has been interestingly used. In economics and finance, data are transformed and analyzed in terms of particular symbols [8]. Two examples are recession indicators utilized to study and to determine the business cycle, and the indicators used to characterize the stock market bull and bear market periods. In geography, works like that of [9] show how qualitative variables (symbolic analysis) can be used to map descriptions. In spatial econometrics, economic spatial dependence has recently been studied by transforming data into symbols [10,11]. Other interesting applications are [12,13].

Despite all these interesting applications and the scientifically founded roots of the symbolic approach, there is no systematic body of statistical tools for conducting inference based on symbolic sequences. There are some notable exceptions: [14,15,16,17,18,19,20,21]. A common factor to all of these statistical approaches is that they are centered on ordinal patterns, which is one type of symbol. In this paper we present a novel, systematic and general framework for any potential symbol in order to test for wide range of potential null hypotheses that include, as particular cases, most of the previously indicated multidisciplinary situations, namely, ordinal patterns. We also provide a general asymptotic distribution theory for symbolic analysis. Particularly, this paper shows how, by means of symbols, it is possible to design nonparametric tests for a wide class of null hypotheses with special attention to limitations (restrictions) that typically appear in economics and finance. Therefore, this paper aims also to provide the theoretical basis for hypothesis testing by means of symbols.

An appealing advantage to symbolic analysis is that it requires very few assumptions about the data generating process in order to conduct statistical inference. This advantage is promising as the tools based on this method will share the model-free property, which avoids making unnecessary assumptions and provides more general results. Most of the econometric and statistical tests typically used in some of the mentioned disciplines cannot deal with potential nonlinear forms of dependence. By construction, nonlinear structures are not a limitation for symbolic analysis.

The capability of this approach is clearly illustrated by the scope of what we label “the symbolic main theorem” (SMT). Given a null hypothesis H${}_0$, for example, the null of serial independence, the SMT will give us four nonparametric asymptotic tests for that null, which are distribution free. The transformation of data into symbols is done by means of a symbolization map. Some of its properties are also studied in this paper. These symbolic-based tests have to deal with ordinary statistical problems that usually appear in economics and finance, such as data scarcity and suboptimal empirical power of the test. Given the flexibility of the symbols, we provide theoretical results and strategies to overcome such difficulties.

A clear example of the power of the new tool is illustrated by the spatio-temporal data modeling issues occupying a prominent role in spatial econometrics, geography and regional science, about which we can find a vast amount of literature ([3], and references therein). We constructed several symbolic-based tests by using the SMT. These tests also constitute an added-value of the paper, because there are currently very few available tests designed to deal with spatiotemporal dependence. The problem becomes more difficult if potential nonlinear dependence is considered. A notable exception is [9] who has treated nonlinearity in a spatial framework.

Finally, the results of this paper might be of interest to fields of research where information theory plays a relevant role. Particularly, nonparametric entropy measures and tests for serial dependence have drawn the attention of econometricians (see [22] and references therein). The clearest link between our results and information theory is through the concept of symbolic entropy. In the context of time series analysis, permutation entropy, which is a type of symbolic entropy, uses the probabilities of length-m ordinal patterns in the definition of Shannon entropy. An ordinal pattern is a particular type of symbolization map. Given the characteristics of this map, the SMT allows us to obtain an asymptotic distribution theory for a permutation entropy-based test. Providing the statistical foundation for permutation entropy is specially relevant because: (a) there are very few asymptotic distribution theories available for entropy, in general; and (b) permutation entropy is currently used in computer science due to its relation to “incompressibility”, and is also useful in the study of dynamical systems because of its connection to complexity.

From another point of view, some well-established nonparametric tests can be understood as particular types of symbolic analysis. For example, the nonparametric runs test for randomness by Wald-Wolfowitz (see [23]); joint-counting procedures for spatial association [24]; and in general, categorical data techniques [25] are simple examples that use the very general procedures of translating information into symbols. In this regard, symbolic analysis can be understood as a method related to this literature.

The paper is organized as follows: In Section 2, we provide the main notation and relevant concepts that will be used in the paper. Among them we highlight: symbolization maps, standard or non-standard maps and decomposable maps. Due to the generality of the method, we require the potential tests to be adaptable to different contexts that are to be able to deal with a wide range of null hypotheses. To this end we introduce the notion of perfect and non-perfect set on subindexes in Section 3. This allows us to give general theoretical results to tackle practical situations that might otherwise be intractable because of the problem and/or of the type of hypothesis. Therefore we distinguish between two main classes of theoretical situations that lead us to different statistical solutions. In Section 4 we show how to construct symbolic-based tests via likelihood ratio statistics and via asymptotic normality. Section 5 considers the theoretical case that the null hypothesis cannot be treated under perfect situations, and hence other results are applicable. Section 6 puts forward the main theorem of this paper. Under the general conditions of this theorem, we introduce four tests for serial independence, four tests for spatial independence and four new tests for spatiotemporal independence, in Section 7. These tests are based on different symbolization maps, according to those given in Section 2 and Section 3. Finally, in Section 8, we outline a Monte Carlo simulation experiment to show the capabilities of the spatiotemporal test for independence under linear and nonlinear settings. The paper ends with some conclusions.

2. Notation and Definitions

As indicated in the previous section, we give some definitions and introduce the basic notation that will be used throughout the rest of the paper.

Let ${\left\{X_i\right\}}_{i\in I}$ be a stationary real-valued process, where I is a set of indexes.

Let $\mathsf{\Gamma }=\{{\eta }_1,{\eta }_2,\cdots ,{\eta }_n\}$ be a set of $n>1$ elements that we label as symbols. Now assume that there exists a map

$$f:{\left\{X_i\right\}}_{i\in I^{\prime }}\to \mathsf{\Gamma }$$

for some subset of indexes $I^{\prime }\subseteq I$. We will say that $i\in I$ is of $\eta$-type if and only if $f\left(X_i\right)=\eta$. We will call the map f a symbolization map for ${\left\{X_i\right\}}_{i\in I}$.

Notice that it is possible to expand the definition of a symbolization map to the $k—$dimensional case by introducing the concept of decomposable maps: if $f_j:{\left\{X_{ij}\right\}}_{i\in I^{\prime }}\to {\mathsf{\Gamma }}_j$, $j=1,2,\cdots ,k$, are k symbolization maps, then the product

$$F=\prod _{j=1}^k{f}_j:\prod _{j=1}^k{\left\{X_{ij}\right\}}_{i\in I^{\prime }}\to \prod _{j=1}^k{\mathsf{\Gamma }}_j$$

is a symbolization map for the k-dimensional variable ${\left\{(X_{i1},X_{i2},\cdots ,X_{ik})\right\}}_{i\in I^{\prime }}$. We will call F a k-decomposable symbolization map.

Given a symbolization map $f:{\left\{X_i\right\}}_{i\in I^{\prime }}\to \mathsf{\Gamma }$ and a symbol $\eta \in \mathsf{\Gamma }$, we denote by $p_{\eta }$ the probability of occurrence of symbol $\eta$. Symbolization maps can be classified according to their behavior under the null hypothesis. If the symbolization map f is such that under a given null hypothesis (H${}_0$) all the symbols have the same probability to occur, we will say that f is a standard symbolization map. On the contrary, we will refer to f as a non-standard symbolization map.

The symbolic entropy of a process ${\left\{X_i\right\}}_{i\in I^{\prime }}$ is defined as the Shannon’s entropy of the n distinct symbols as follows:

$$h\left(\mathsf{\Gamma }\right)=-\sum _{\eta \in \mathsf{\Gamma }}{p}_{\eta }ln\left(p_{\eta }\right),$$

(1)

with the convention $0\times ln0=l\acute{ı}{m}_{x\to 0^{+}}xlnx=0.$

Symbolic entropy, $h\left(\mathsf{\Gamma }\right)$, can be understood as the information in terms of symbols $\eta \in \mathsf{\Gamma }$ of the process ${\left\{X_i\right\}}_{i\in I^{\prime }}$. Notice that $0\le h\left(\mathsf{\Gamma }\right)\le ln\left(n\right)$. Notice also that the lower bound is attained when only one symbol occurs, and the upper bound when all n possible symbols appear with the same probability.

Consider the following index $i\in I^{\prime }$ that we define an indicator random variable $Z_{\eta i}$ as follows:

$$Z_{\eta i}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}f\left(X_i\right)=\eta \hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

that is, we have that $Z_{\eta i}=1$ if and only if i is of $\eta$-type, and $Z_{\eta i}=0$ otherwise.

Then $Z_{\eta i}$ is a Bernoulli variable with probability of “success” $p_{\eta }$, where “success” means that i is of $\eta$-type. It is straightforward to see that

$$\sum _{\eta \in \mathsf{\Gamma }}{p}_{\eta }=1$$

Our interest is in knowing how many is are of $\eta$-type for all symbol $\eta \in \mathsf{\Gamma }$. In order to answer the question, we construct the following counting variable:

$$Y_{\eta }=\sum _{i\in I^{\prime }}{Z}_{\eta i}$$

The variable $Y_{\eta }$ can take the values $\{0,1,2,\cdots ,R\},$ where $|I^{\prime }|=R$.

To complete with notation, we will denote by

$$n_{\eta }=\left|\{i\in I^{\prime }|f\left(X_i\right)=\eta \}\right|,$$

the cardinality of the subset of symbolized indexes $I^{\prime }$ formed by all the elements of $\eta$-type.

Then, under the conditions above, one could easily compute the relative frequency of a symbol $\eta \in \mathsf{\Gamma }$ by:

$${\widehat{p}}_{\eta }:=\frac{\left|\{i\in I^{\prime }|i \mathrm{is} \mathrm{of} \eta -\mathrm{type}\}\right|}{R}$$

which is the maximum likelihood estimator of $p_{\eta }.$

3. On the Independence of Variables ${\left\{{\mathit{Z}}_{\mathit{\eta }\mathit{i}}\right\}}_{\mathit{i}\in {\mathit{I}}^{\prime }}$

If the subset of symbolized indexes $I^{\prime }$ is chosen such that the $Z_{\eta i}$ variables are independent for all $i\in I^{\prime }$, then $Y_{\eta }$ is a binomial random variable

$$Y_{\eta }\sim B(R,p_{\eta }).$$

Moreover, if the subset of indexes $I^{\prime }$ is such that the variables $Z_{{\eta }_s}$ and $Z_{{\sigma }_t}$ are independent for all symbol ${\eta }_s,{\sigma }_t\in \mathsf{\Gamma },$$s\ne t,$ where $s,t\in \{1,2,\cdots ,n\},$ then the joint probability density function of the n variables $(Y_{{\eta }_1},Y_{{\eta }_2},\cdots ,Y_{{\eta }_n})$ is:

$$P(Y_{{\eta }_1}=a_1,Y_{{\eta }_2}=a_2,\cdots ,Y_{{\eta }_n}=a_n)=\frac{(a_1+a_2+\cdots +a_n)!}{a_1!a_2!·\cdots ·a_n!}{p}_{{\eta }_1}^{a_1}{p}_{{\eta }_2}^{a_2}\cdots p_{{\eta }_n}^{a_n}$$

where $a_1+a_2+\cdots +a_n=R$. Consequently the joint distribution of the n variables $(Y_{{\eta }_1},Y_{{\eta }_2},\cdots ,Y_{{\eta }_n})$ is a multinomial distribution. In this case we will call the set of symbolized indexes $I^{\prime }$ a perfect subset.

It is possible to develop theoretical results that contemplate situations for which the researcher will benefit of constructing symbolization maps for which the set $I^{\prime }$ is not perfect. Given our previous definition of perfect set, the indexes subset $I^{\prime }$ can be nonperfect in the following cases:

• Case (a): $Z_{\eta i}$ and $Z_{\eta j}$ are not independent for all $i\ne j$ for some $\eta \in \mathsf{\Gamma }$, and hence $Y_{\eta }$ is not a binomial random variable.
• Case (b): $Z_{{\eta }_s}$ and $Z_{{\sigma }_t}$ are not independent for all $s\ne t$ for some ${\sigma }_t,{\eta }_s\in \mathsf{\Gamma }$, and hence $(Y_{{\eta }_1},Y_{{\eta }_2},\cdots ,Y_{{\eta }_n})$ is not a multinomial distribution.

Theoretical results for perfect and non-perfect subsets are the topics of the following two sections.

4. Constructing Tests with Perfect Subsets of $\mathit{I}$

In this section we establish a general framework for testing for a null hypothesis H${}_0$. This is done by focusing on the symbols’ distribution when the subset $I^{\prime }$ of symbolized symbols is perfect. Under this condition, we now show how to construct tests of hypothesis via likelihood ratio statistics and via asymptotic normality.

Procedure

In general, our procedure consists of proceeding systematically as follows:

• (step 1) Fix the null hypothesis H${}_0$ to be tested.
• (step 2) Define the set of symbols $\mathsf{\Gamma }$ and the symbolization map f.
• (step 3) Compute the distribution of the symbols under H${}_0$, namely, $p_{\eta }^{\left(0\right)}$ for all $\eta \in \mathsf{\Gamma }$.
• (step 4) Finally design and compute a desired test statistic.

Steps 1 to 2 will be developed later in the paper. To accomplish the aim with steps 3 and 4, we first show how to construct a likelihood ratio test.

Recall that when the set is perfect, $(Y_{{\eta }_1},\cdots ,Y_{{\eta }_n})$ follows a multinomial distribution and its likelihood function is:

$$L(p_{{\eta }_1},p_{{\eta }_2},\cdots ,p_{{\eta }_n})=\frac{R!}{n_{{\eta }_1}!n_{{\eta }_2}!·\cdots ·n_{{\eta }_n}!}{p}_{{\eta }_1}^{n_{{\eta }_1}}{p}_{{\eta }_2}^{n_{{\eta }_2}}\cdots p_{{\eta }_n}^{n_{{\eta }_n}}$$

and the likelihood ratio test statistic is

$$\lambda \left(Y\right)=\frac{\frac{R!}{n_{{\eta }_1}!n_{{\eta }_2}!·\cdots ·n_{{\eta }_n}!}{p}_{{\eta }_1}^{\left(0\right)n_{{\eta }_1}}{p}_{{\eta }_2}^{\left(0\right)n_{{\eta }_2}}\cdots p_{{\eta }_n}^{\left(0\right)n_{{\eta }_n}}}{\frac{R!}{n_{{\eta }_1}!n_{{\eta }_2}!·\cdots ·n_{{\eta }_n}!}{\widehat{p}}_{{\eta }_1}^{n_{{\eta }_1}}{\widehat{p}}_{{\eta }_2}^{n_{{\eta }_2}}\cdots {\widehat{p}}_{{\eta }_n}^{n_{{\eta }_n}}},$$

(2)

where ${\widehat{p}}_{{\eta }_i}$ is the maximum likelihood estimator of $p_{{\eta }_i}$ for all $i=1,2,\cdots ,n$. In this case, as shown in the Appendix A, maximum likelihood estimators are ${\widehat{p}}_{{\eta }_i}=\frac{n_{{\eta }_i}}{R}.$

Moreover, it is possible to show (see Appendix A.2) that

$$-2[RLn\left(R\right)+\sum _{i=1}^n{n}_{{\eta }_i}Ln\left(\frac{p_{{\eta }_i}}{n_{{\eta }_i}^0}\right)]\sim {\chi }_k^2,$$

where the k degrees of freedom will depend on the set of symbols.

It is also proved that, if standard symbolization maps are considered, then

$$2R[Ln\left(n\right)-h\left(\mathsf{\Gamma }\right)]\sim {\chi }_k^2,$$

(3)

which is an affine transformation of the symbolic entropy (1).

As we have emphasized in the Introduction that nonparametric results of entropy based measures are relevant for econometrics and for other fields of research. Given that the distribution of the affine transformation in (3) holds for standard symbolization maps under broad conditions, this result will be of general applicability. In particular, as we will show in other sections, permutation entropy, as introduced in [26], is a particular type of symbolic entropy that has drew the attention of several scholars for theoretical interests basically because stationary and ergodic processes coincide with Shannon entropy, and for an applied interest in nonlinear and complex systems or processes (see [1]). In this regard, (2) and (3) can be viewed as an initial step towards statistical inference of ordinal pattern distributions.

An alternative to likelihood ratio tests for symbolic maps can be considered by modifying step 4. Given that under a perfect subset $I^{\prime }$ the indicator variables $Z_{{\eta }_i}$ are independent, the random variable $\frac{Y_{\eta }-R{p}_{\eta }}{\sqrt{R{p}_{\eta }(1-p_{\eta })}}$ has a limiting normal distribution with zero mean and unit variance for all symbol $\eta \in \mathsf{\Gamma }$. Moreover

$$\left(\frac{Y_{{\eta }_1}-R{p}_{{\eta }_1}}{\sqrt{R{p}_{{\eta }_1}(1-p_{{\eta }_1})}},\frac{Y_{{\eta }_2}-R{p}_{{\eta }_2}}{\sqrt{R{p}_{{\eta }_2}(1-p_{{\eta }_2})}},\cdots ,\frac{Y_{{\eta }_n}-R{p}_{{\eta }_n}}{\sqrt{R{p}_{{\eta }_n}(1-p_{{\eta }_n})}}\right)$$

asymptotically distributes as a multivariate normal distribution M$N(0,I)$. In this case, the asymptotic distribution holds for standard or nonstandard symbolization maps at the cost of estimating $p_{{\eta }_i}$ that can be consistently estimated by ${\widehat{p}}_{{\eta }_i}$.

5. Constructing Tests with Non Perfect Subsets of $\mathit{I}$

In this section we establish the equivalent counterpart of the general framework (above presented) when the subset $I^{\prime }$ of symbolized symbols is non-perfect. Accordingly, we now show to what extent and under which situations the previous likelihood and asymptotically normal tests (elaborated under perfect situations) can be adapted to deal with them.

Non perfect sets of indexes I might be very useful for test design, especially for situations or scientific domains characterized by relatively scarce sample size as compared with the number of symbols. In macroeconomics, although not necessarily in finance, data scarcity can be the usual restriction. Ideally, one can be able to design perfect sets of indexes to carry out symbolic-based hypothesis testing. This section then tries to provide symbolic-based methods for constructing hypothesis tests for situations for which an ideal design is not possible because of the nature of the problem, because of the computational capabilities or because of any other potential reason.

5.1. Binomial Approximation

Let us consider that variables $Z_{\eta i}$ and $Z_{\eta j}$ are not independent for all $i\ne j$ for some $\eta \in \mathsf{\Gamma }$. In this case, $Y_{\eta }$ is not a binomial random variable. The interesting question is how far is $Y_{\eta }$ from $B(R,p_{\eta }).$ We are interested in studying under what assumptions the variable $Y_{\eta }$ can be approximated to a binomial random variable:

$$Y_{\eta }\approx B(R,p_{\eta }).$$

In fact, it is possible to compute a bound for this binomial approximation.

Denote by $\mathcal{L}\left(Y_{\eta }\right)$ the distribution of $Y_{\eta }$, and we are interested in the bound of the binomial approximation of the distribution of $Y_{\eta }$ measured in terms of total variation distance. The total variation distance $d_{TV}$ between two probability measures P and Q is defined by

$$d_{TV}(P,Q)=\underset{A}{sup}|P\left(A\right)-Q\left(A\right)|$$

where the supremum is taken over all measurable sets of the real line.

Following Theorem 1.1 in [27] and after a few calculations, a bound can be given as follows:

For each $i,j\in I^{\prime }$ let $Z_{\eta j},Z_{\eta i}$ and $J_{\eta ij}$ be defined in the same probability space where $J_{\eta ji}=\left(Z_{\eta j}\right|Z_{\eta i}=1)$. Let

$$W_i=\sum _{j\ne i}{Z}_{\eta j}{V}_i=\sum _{j\ne i}{J}_{\eta ji}$$

and

$$C_{R,p}=\frac{1-p^{R+1}-q^{R+1}}{\left(R+1\right)pq}.$$

$W_i$ counts the number of indexes that are of $\eta$-type and $p=p_{\eta }$ and $q=1-p_{\eta }$. On the other hand $V_i={\displaystyle \sum _{j\ne i}}\left(Z_{\eta j}\right|Z_{\eta i}=1)$ counts the number of indexes that are of $\eta$-type conditioned to location i is of $\eta$-type.

Then

$$d_{TV}(\mathcal{L}\left(Y_{\eta }\right),B(R,p))\le C_{R,p}p\sum _{i=1}^{R}E\left(\right|W_i-V_i\left|\right).$$

Therefore, in order to get a bound for the binomial approximation, we have to get bound

$$\sum _{i=1}^{R}E\left(\right|W_i-V_i\left|\right).$$

On the other hand, we have that $E\left(\right|W_i-V_i\left|\right)\le {\displaystyle \sum _{j\ne i}}E\left(\right|Z_{\eta j}-J_{\eta ji}\left|\right)$ and $pE\left(J_{\eta ji}\right)=E\left(Z_{\eta i}{Z}_{\eta j}\right)$. Now denote by $B_i^{\eta }$ a subset of indexes such that $Z_{\eta i}$ is independent of $\left\{Z_{\eta j}\right|j\notin B_i\}$. Therefore, we obtain that

$$\begin{array}{ccc}\hfill d_{TV}(\mathcal{L}\left(Y_{\eta }\right),B(R,p))& \le & C_{R,p}p\sum _{i=1}^{R}E\left(\right|W_i-V_i\left|\right)\hfill \\ & \le & C_{R,p}p\sum _{i=1}^R\sum _{j\in B_i\backslash \left\{i\right\}}E\left(\right|Z_{\eta j}-J_{\eta ji}\left|\right)\hfill \\ & \le & C_{R,p}p\sum _{i=1}^R\sum _{j\in B_i\backslash \left\{i\right\}}\left(p+E\left(J_{\eta ji}\right)\right)\hfill \\ & \le & C_{R,p}\left(\sum _{i=1}^R{p}^2\left(\right|B_i|-1)+\sum _{i=1}^R\sum _{j\in B_i\backslash \left\{i\right\}}E\left(Z_{\eta i}{Z}_{\eta j}\right)\right)\hfill \end{array}$$

Therefore, we have shown that the sum of dependent indicators can be approximated to a binomial random variable when the following two conditions are satisfied: (1) dependencies among the indicators are weak and (2) the probabilities of the indicators occurring under the null hypothesis are small. Notice that point (1) can be guaranteed by selecting the subset of symbolized indexes $I^{\prime }$ such that $|B_i|$ is small enough.

5.2. Normal Approximation

Additionally, when the indicator variables $Z_{\eta i}$ are not independent for all $i\in I^{\prime }$, the following central limit theorem for dependent indicators ensures the convergence to a normal distribution:

Theorem 1.

(Theorem 7.7.5 (Anderson, 1971)). Let $Z_1,Z_2,\cdots$ be a stationary stochastic process such that for every integer n and integers $t_1,t_2,\cdots ,t_n$ with $t_1<\cdots <t_n,$$Z_{t_1},\cdots ,Z_{t_n}$ are distributed independently of $Z_1,\cdots ,Z_{t_1-m-1}$ and $Z_{t_n+m+1},\cdots$. If $E\left(Z_t\right)=0$ and $E\left(Z_t^2\right)<\infty$, then

$$\frac{{\displaystyle \sum _{t=1}^T}{Z}_t}{\sqrt{T}}$$

has a limiting normal distribution with mean 0 and variance

$$E\left(Z_1^2\right)+2E\left(Z_1{Z}_2\right)+\cdots +2E\left(Z_1{Z}_{m+1}\right).$$

Theorem 1 states that, if the dependencies are weak (for instance, $|B_i|$ is small enough for all $i\in I^{\prime }$) we get that $\frac{Y_{\eta }-R{p}_{\eta }}{\sqrt{Var\left(Y_{\eta }\right)}}\stackrel{d}{\to }N(0,1)$ as $R\to \infty$. The variance of the variable $Y_{\eta }$ can be computed as follows:

$$\begin{array}{ccc}\hfill Var\left(Y_{\eta }\right)& =& E\left(Y_{\eta }^2\right)-E{\left(Y_{\eta }\right)}^2=\sum _{i=1}^{R}E\left(Z_{\eta i}^2\right)+\sum _{i=1}^R\sum _{j\ne i}E\left(Z_{\eta i}{Z}_{\eta j}\right)-R^2{p}_{\eta }^2\hfill \\ & =& R{p}_{\eta }-R^2{p}_{\eta }^2+\sum _{i=1}^R(R-|B_i\left|\right)p_{\eta }^2+\sum _{i=1}^R\sum _{j\in B_i\backslash \left\{i\right\}}E\left(Z_{\eta i}{Z}_{\eta j}\right)\hfill \\ & =& R{p}_{\eta }-R^2{p}_{\eta }^2+R^2{p}_{\eta }^2-p_{\eta }^2\sum _{i=1}^R\left|B_i\right|+\sum _{i=1}^R\sum _{j\in B_i\backslash \left\{i\right\}}E\left(Z_{\eta i}{Z}_{\eta j}\right)\hfill \\ & =& R{p}_{\eta }-p_{\eta }^2\sum _{i=1}^R\left|B_i\right|+\sum _{i=1}^R\sum _{j\in B_i\backslash \left\{i\right\}}E\left(Z_{\eta i}{Z}_{\eta j}\right)\hfill \end{array}$$

(4)

We now consider the case where $Z_{{\eta }_s}$ and $Z_{{\sigma }_t}$ are not independent for all $s\ne t$ for some ${\sigma }_t,{\eta }_s\in \mathsf{\Gamma }$, and hence $(Y_{{\eta }_1},Y_{{\eta }_2},\cdots ,Y_{{\eta }_n})$ is not a multinomial distribution (i.e., previous case (b)). To this end, denote by $B_i^{(\eta ,\sigma )}$ the subset of indexes satisfying that $Z_{\eta i}$ is independent of $\left\{Z_{\sigma j}\right|j\notin B_i^{(\eta ,\sigma )}\}$ and $X_{\eta }=\frac{Y_{\eta }-R{p}_{\eta }}{\sqrt{Var\left(Y_{\eta }\right)}}$. It is possible to show that $(X_{{\eta }_1},X_{{\eta }_2},\cdots ,X_{{\eta }_n})$ is a multivariate normal distribution. In fact, it is equivalent to proving that any linear combination of the $X_{\eta }$s is normal. In order to do so, notice that each variable $X_{\eta }$ is a sum of indicator variables, and therefore, any linear combination of these variables is again a sum of indicator variables. Consider an arbitrary linear combination as follows:

$$M={\alpha }_1{X}_{{\eta }_1}+{\alpha }_2{X}_{{\eta }_2}+\cdots +{\alpha }_n{X}_{{\eta }_n}$$

Notice that the variable $\alpha X_{\eta }$ can be written as $\alpha X_{\eta }={\displaystyle \sum _{i=1}^R}{I}_i^{\eta }$ where $I_i^{\eta }$ is an indicator variable for all $i\in I^{\prime }$. Therefore, it follows that the variable M is a sum of dependent indicator variables. Again, by Theorem 1 we get that M follows a normal distribution whenever the dependencies among the indicator variables are weak (for instance, when the cardinality of the set $B_i\cup B_i^{({\eta }_s,{\eta }_t)}$ is small enough for all $i\in I^{\prime }$ and all symbol $\eta ,{\eta }_s,{\eta }_t\in \mathsf{\Gamma }$). Then we get that $(X_{{\eta }_1},X_{{\eta }_2},\cdots ,X_{{\eta }_n})$ has a limiting multivariate normal distribution and we can compute the variance and covariance matrix for all ${\eta }_s,{\eta }_t\in \mathsf{\Gamma }$ as follows:

$$\begin{array}{ccc}\hfill Cov(Y_{{\eta }_s},Y_{{\eta }_t})& =& \sum _{i=1}^R\sum _{j=1}^{R}Cov(Z_{{\eta }_{s}i},Z_{{\eta }_{t}j})=\sum _{i=1}^R\sum _{j\in B_i^{({\eta }_s,{\eta }_t)}}Cov(Z_{{\eta }_{s}i},Z_{{\eta }_{t}j})\hfill \\ & =& \sum _{i=1}^R\sum _{j\in B_i^{({\eta }_s,{\eta }_t)}}\{E\left(Z_{{\eta }_{s}i}{Z}_{{\eta }_{t}j}\right)-E\left(Z_{{\eta }_{s}i}\right)E\left(Z_{{\eta }_{t}j}\right)\}\hfill \\ & =& p_{{\eta }_s}\sum _{i=1}^R\sum _{j\in B_i^{({\eta }_s,{\eta }_t)}}p(Z_{{\eta }_{t}j}=1|Z_{{\eta }_{s}i}=1)-p_{{\eta }_s}{p}_{{\eta }_t}\sum _{i=1}^R\left|B_i^{({\eta }_s,{\eta }_t)}\right|.\hfill \end{array}$$

(5)

6. The Symbolic Main Theorem

Previous partial results can be collected in the following main theorem.

Theorem 2.

Let ${\left\{X_i\right\}}_{i\in I^{\prime }}$ be a stationary real valued process. Let $\mathsf{\Gamma }=\{{\eta }_1,{\eta }_2,\cdots ,{\eta }_n\}$ be a finite set of symbols. Let $f:{\left\{X_i\right\}}_{i\in I^{\prime }}\to \mathsf{\Gamma }$ be a standard symbolization map for a null hypothesis H${}_0$. Then under the null H${}_0$ we have that:

• If the set of symbolized indexes $I^{\prime }$ is perfect:
  • Then $G\left(\mathsf{\Gamma }\right)=2R[ln(n)-h(\mathsf{\Gamma }\left)\right]$ is asymptotically ${\chi }_k^2$ distributed where k is the difference between the number of parameters to be estimated under the alternative hypothesis $H_1$ and the number of parameters to be estimated under the null H${}_0$.
  • Then $(\frac{Y_{{\eta }_1}-R{p}_{{\eta }_1}}{\sqrt{R{p}_{{\eta }_1}(1-p_{{\eta }_1})}},\frac{Y_{{\eta }_2}-R{p}_{{\eta }_2}}{\sqrt{R{p}_{{\eta }_2}(1-p_{{\eta }_2})}},\cdots ,\frac{Y_{{\eta }_n}-R{p}_{{\eta }_n}}{\sqrt{R{p}_{{\eta }_n}(1-p_{{\eta }_n})}})$ is a multivariate normal distribution MN$(0,I)$.
• The set of symbolized indexes $I^{\prime }$ is not perfect:
  • If the sets $B_i$ and $B_i^{({\eta }_s,{\eta }_t)}$ have good properties in the sense that for all symbol $\eta \in \mathsf{\Gamma }$ the variables $Y_{\eta }$ have a good approximation to a binomial distribution and $(Y_{{\eta }_1},Y_{{\eta }_2},\cdots ,Y_{{\eta }_n})$ has a good approximation to a multinomial distribution, then $G\left(\mathsf{\Gamma }\right)=2R[ln(n)-h(\mathsf{\Gamma }\left)\right]$ is asymptotically ${\chi }_k^2$ distributed where k is the difference between the number of parameters to be estimated under the alternative hypothesis $H_1$ and the number of parameters to be estimated under the null H${}_0$.
  • If $I^{\prime }$ is such that Theorem 1 holds, then $(\frac{Y_{{\eta }_1}-R{p}_{{\eta }_1}}{\sqrt{Var\left(Y_{{\eta }_1}\right)}},\frac{Y_{{\eta }_2}-R{p}_{{\eta }_2}}{\sqrt{Var\left(Y_{{\eta }_2}\right)}},\cdots ,\frac{Y_{{\eta }_n}-R{p}_{{\eta }_n}}{\sqrt{Var\left(Y_{{\eta }_n}\right)}})$ is a multivariate normal distribution MN$(0,\mathsf{\Sigma })$ where the variance and covariance matrix can be estimated using (4) and (5).

Notice that in the case of nonstandard symbolization maps, an analogous theorem to Theorem 2 will hold. More concretely, for point 1 of the SMT, a likelihood ratio test is also available, although not in the closed form as presented here (see for example [28]). Point 2 of the SMT hold independently of whether the symbolization map is standard.

The usefulness of the theorem will become evident in the next section, where it will be applied to specific null hypotheses. Naturally, it is possible to consider an alternatively bootstrap-based test for symbolization maps, instead of asymptotic ones, although we do not follow this way in this paper, it being a subfield for further research of interest since it partially might avoid taking care of nonperfect indexes sets.

7. Different Symbolizations for Different Nulls Related with Independence

According to the general symbolic theorem, in this section we show how it is possible to test interesting null hypotheses by using symbolic analysis. To concrete, we focus on testing for different nulls of independence as it is a well-known field of research and because recently published articles can be generally understood and extended under this new theoretical framework. Given that each null hypothesis (step 1) will require a particular symbolization map, in this section we present different symbolization procedures (step 2) to test for serial dependence, spatial dependence, and spatiotemporal dependence, respectively. Then we present the results of step 3 and step 4 depending on the statistic technique the researcher wants to use according to Theorem 2, i.e., either likelihood ratio statistics or/and asymptotically normal statistics. Given a null hypothesis, the behavior of the tests obtained from this approach will strongly depend on the expertise of the researcher in constructing the symbolization map. We emphasize that both power analysis of the class of tests, and power competition among alternative nonparametric tests were already given in previous work [10,16]; therefore, we are not going to replicate them here.

As we have indicated, the crucial component of the symbolic procedure is to choose a symbolic mapping which ensures that the distribution of the symbols can detect deviations from the null. The null hypotheses considered in this section are related to the important topic of “statistical independence”. This is a very well-studied topic in time series analysis and therefore there is a generous number of available tests. On the contrary, spatial independence is not so well-known and is non-trivial how to test for it. As we will show, it is needed to use another different symbolization map for detecting spatial patterns. Similar comments can be made for spatio-temporal independence. Needless to say, there are other hypotheses of interest in econometrics, and the researcher will have to design suitable symbolic maps for testing them. For example, in [29], the authors dealt with the opposite problem: how to test for a pure deterministic chaotic process. In these and other cases, the power of the tests will centrally depend on the ability of the research to design the symbolization map for the desired null hypothesis.

7.1. Serial Independence Tests

In the case of time series, refs. [15,16] used the following symbolization procedure to test for serial dependence: Let ${\left\{X_t\right\}}_{t\in I}$ be a real-valued time series (in this case the subindex t refers to time) for which we are interested in testing the null of serial independence (step 1). In order to complete step 2, we denote by ${\mathsf{\Gamma }}_1=S_m$ the symmetric group of order $m!$, that is, the group formed by all the permutations of length m (for a positive integer $m\ge 2)$. Let ${\pi }_i=(i_1,i_2,\cdots ,i_m)\in S_m$. The positive integer m is usually known as the embedding dimension.

An ordinal pattern for a symbol is defined as ${\pi }_i=(i_1,i_2,\cdots ,i_m)\in S_m$ at a given time $t\in I$. The time series can be embedded in an m-dimensional space:

$$X_m\left(t\right)=(X_{t+1},X_{t+2},\cdots ,X_{t+m})\mathrm{for}t\in I$$

It is said that t is of ${\pi }_i—$type if and only if ${\pi }_i=(i_1,i_2,\cdots ,i_m)$ is the unique symbol in the group $S_m$ satisfying the two following conditions:

$$\begin{array}{c}\left(a\right)X_{t+i_1}\le X_{t+i_2}\le \cdots \le X_{t+i_m},\mathrm{and}\hfill \\ \\ \left(b\right)i_{s-1}<i_{s}if{X}_{t+i_{s-1}}=X_{t+i_s}\hfill \end{array}$$

Notice that condition $\left(b\right)$ guaranties uniqueness of the symbol ${\pi }_i$. This is justified if the values of $X_t$ have a continuous distribution so that equal values are very uncommon, with a theoretical probability of occurrence of 0.

In this case, the symbolization map is defined as $f_1:{\left\{X_t\right\}}_{t\in I^{\prime }}\to S_m$ given by

$$f_1\left(X_t\right)=(i_1,i_2,\cdots ,i_m)$$

(6)

where $(i_1,i_2,\cdots ,i_m)\in S_m$ is such that t is of $(i_1,i_2,\cdots ,i_m)$-type. Now the design of the symbolization map (step 2) is completed.

Moreover, under the null of independence the distribution of the symbols is uniform and therefore the map $f_1$ is a standard symbolization map. Additionally, the set of symbolized indexes is $I^{\prime }=\{1,2,\cdots ,T-m+1\},$ which is not perfect.

Notice that in order to have a perfect set and therefore ensure the independence of the indicator variables $Z_{\pi t}$, it is enough to consider as a set of symbolized indexes

$$I^{\prime }=\{0,m-1,2(m-1),\cdots ,t(m-1),\cdots \}.$$

Accordingly, using this symbolization map, the next corollary straightforwardly follows from Theorem 2:

Corollary 1.

Let $f_1:{\left\{X_t\right\}}_{t\in I^{\prime }}\to {\mathsf{\Gamma }}_1$ be the symbolization map defined in (6) with $|I^{\prime }|=R$. Denote by $h\left({\mathsf{\Gamma }}_1\right)$ the permutation entropy defined in (1). If the time series ${\left\{X_t\right\}}_{t\in I}$ is independent, then

• If the set $I^{\prime }$ is perfect, then:

  1. 

  The affine transformation of the permutation entropy

  $$G\left(S_m\right)=2R[Ln(m!)-h\left(S_m\right)]$$

  is asymptotically ${\chi }_{m!-1}^2$ distributed.

  2. 

  Then $(\frac{Y_{{\pi }_1}-R{p}_{{\pi }_1}}{\sqrt{R{p}_{{\pi }_1}(1-p_{{\pi }_1})}},\frac{Y_{{\pi }_2}-R{p}_{{\pi }_2}}{\sqrt{R{p}_{{\pi }_2}(1-p_{{\pi }_2})}},\cdots ,\frac{Y_{{\pi }_{m!}}-R{p}_{{\pi }_{m!}}}{\sqrt{R{p}_{{\pi }_{m!}}(1-p_{{\pi }_{m!}})}})$ is a multivariate normal distribution NM$(0,I)$.

• If the set $I^{\prime }$ is not perfect:

  1. 

  Since the sets $B_i^{\pi }$’s has cardinality of at most $2m$, we can get a good approximation to the following result via [21]. The affine transformation of the permutation entropy

  $$G\left(S_m\right)=2R[Ln(m!)-h\left(S_m\right)]$$

  is asymptotically ${\chi }_{m!-1}^2$ distributed.

  2. 

  Then $(\frac{Y_{{\pi }_1}-R{p}_{{\pi }_1}}{\sqrt{Var\left(Y_{{\pi }_1}\right)}},\frac{Y_{{\pi }_2}-R{p}_{{\pi }_2}}{\sqrt{Var\left(Y_{{\pi }_2}\right)}},\cdots ,\frac{Y_{{\pi }_{m!}}-R{p}_{{\pi }_{m!}}}{\sqrt{Var\left(Y_{{\pi }_{m!}}\right)}})$ is a multivariate normal distribution MN$(0,\mathsf{\Sigma })$ where the variance and covariance matrix can be estimated using (4) and (5).

These results for permutation entropy are in relation to a relatively recent line of research based on order patterns for analyzing time series. Ordinal patterns can be, per se, used for descriptive purposes, like autocorrelation, with the added advantage that the require no assumptions such as Gaussianity or linearity. On the contrary, only mild stationary conditions can exist in the underlying process. The above corollary is a further step for the development of statistical inference for ordinal time series. Naturally, it is possible to obtain other kinds of statistical results by adding more assumptions to the generating process. In fact, notorious results can be found in [4]) if Gaussianity and ergodicity are assumed. In this regard, our asymptotic results for order patterns keep assumptions at a minimum. Additionally, by maintaining general applicability at minimum cost (in terms of assumptions) for serial independence tests, some bootstrap-based statistics for ordinal patterns have been put forward in [29].

An interesting property of the symbolization procedure presented in this section is that it can be also used for discrete distributions. To do so it necessary to consider a non-standard version of the map. Under such circumstances, the likelihood ratio (2) can be directly used once the behavior of $p_{{\eta }_i}$ is known under the null of serial independence.

7.2. Spatial Independence Tests

In the case of spatial processes, ref. [10] gave a symbolization procedure to test for spatial independence as follows: Let ${\left\{X_s\right\}}_{s\in S}$ be a real-valued spatial process, where S is a set of coordinates. Given a location $s_0$, we will denote by $({\rho }_i^0,{\theta }_i^0)$ the polar coordinates of location $s_i$ taking as origin $s_0$.

Let $m\in \mathbb{N}$ with $m\ge 2$. Consider now that the spatial process ${\left\{X_s\right\}}_{s\in S}$ is embedded in a different m-dimensional space as follows:

$$X_m\left(s_0\right)=(X_{s_0},X_{s_1},\cdots ,X_{s_{m-1}})\mathrm{for}{s}_0\in S$$

where $s_1,s_2,\cdots ,s_{m-1}$ are the $m-1$ nearest neighbors to $s_0$, which are ordered from lesser to higher Euclidean distance with respect to location $s_0$. Notice that in the case of two or more locations being equidistant to $s_0$, we will choose them in an anticlockwise manner. In formal terms, $s_1,s_2,\cdots ,s_{m-1}$ are the $m-1$ nearest neighbors to $s_0$ satisfying the following two conditions:

• (a) ${\rho }_1^0\le {\rho }_2^0\le \cdots \le {\rho }_{m-1}^0$;
• (b) If ${\rho }_i^0={\rho }_{i+1}^0$, then ${\theta }_i^0<{\theta }_{i+1}^0$.

Notice that conditions $\left(a\right)$ and $\left(b\right)$ ensure the uniqueness of $X_m\left(s\right)$ for all $s\in S$.

The proposed standard symbolization map f is defined as follows: denote by $Me$ the median of the spatial process ${\left\{X_s\right\}}_{s\in S}$ and let

$${\delta }_s=\left\{\begin{array}{c}0\mathrm{if}{X}_s\le Me\hfill \\ 1\mathrm{otherwise}\hfill \end{array}\right.$$

Now, define the indicator function

$${\mathcal{I}}_{s_1{s}_2}=\left\{\begin{array}{c}0\mathrm{if}{\delta }_{s_1}\ne {\delta }_{s_2}\hfill \\ 1\mathrm{otherwise}\hfill \end{array}\right.$$

(7)

Then, the standard symbolization map

$$f_2:{\left\{X_s\right\}}_{s\in I^{\prime }}\to {\mathsf{\Gamma }}_2$$

(8)

is defined as:

$$f_2\left(X_s\right)=({\mathcal{I}}_{s{s}_1},{\mathcal{I}}_{s{s}_2},\cdots ,{\mathcal{I}}_{s{s}_{m-1}}),$$

(9)

where ${\mathsf{\Gamma }}_2$ stands for the set of symbols defined by $f_2.$

Notice that under the null of spatial independence, the distribution of the symbols is uniform and therefore the map $f_2$ is a standard symbolization map.

Moreover, in this case $I^{\prime }=S$ is not a perfect symbolized set. To construct a perfect symbolized set $I^{\prime }$, one can proceed as follows. Take a location $s_0\in S$ at random. Let $N_s$ be the set of nearest neighbors to s. Now select the following element in $I^{\prime }$ by taking $s_1\in S$ such that $N_{s_1}\cap N_{s_0}=\varnothing$. Then construct recursively the set $I^{\prime }$ by taking $s_k\in S\backslash \{s_0,s_1,\cdots ,s_{k-1}\}$ satisfying $N_{s_i}\cap N_{s_j}=\varnothing$ for all $i\ne j$ with $i,j=1,2,\cdots ,k$.

As it is evident, the method is flexible enough to allow the researcher to select his own set and map of symbols for a given null. For example, if under the previous symbolization procedure, the power (or size) of the test is not satisfactory, one can always consider other possible symbolization procedures for the same null and for the same spatial process ${\left\{X_s\right\}}_{s\in S}$. Let ${\mathsf{\Gamma }}_3=\{1,2,\cdots ,k\}\times \{1,2,\cdots ,k\}$. Again, let $N_s$ be the set of nearest neighbors to s and let $n_s$ be its cardinality. Denote by $X_s^N=\frac{1}{n_s}{\displaystyle \sum _{s^{\prime }\in N_s}}{X}_{s^{\prime }}$. Denote by $q_i$ and $q_i^N$ the i-th quantile of the variables X and $X^N$ respectively, for $i\in \{1,2,\cdots ,k-1\}$. We will denote by $q_0=\underset{s\in S}{min}{X}_s$ (resp $q_0^N=\underset{s\in S}{min}{X}_s^N$) and $q_{k+1}=\underset{s\in S}{max}{X}_s$ (resp. $q_{k+1}^N=\underset{s\in S}{max}{X}_s^N$). Then we define the symbolization map

$$f_3\left(X_s\right)=(i,j)$$

(10)

if and only if $X_s\in [q_{i-1},q_i]$ and $X_s^N\in [q_{j-1}^N,q_j^N]$.

Again, under the null of independence the distribution of the symbols is uniform and therefore the map $f_3:{\left\{X_s\right\}}_{s\in I^{\prime }}\to \{1,2,\cdots ,k\}\times \{1,2,\cdots ,k\}$ is a standard symbolization map.

Again, the same set of recursively constructed symbolized indexes $S^{\prime }$ ensures the independence of the indicator variables $Z_{\eta s}$. Accordingly, using this symbolization map, the next corollary straightforwardly follows from Theorem 2:

Corollary 2.

Let $f_i:{\left\{X_s\right\}}_{s\in S^{\prime }}\to {\mathsf{\Gamma }}_i$, $i=2,3$ be the symbolization maps defined in (8) and (10) with $|S^{\prime }|=R$. Denote by $h\left({\mathsf{\Gamma }}_i\right)$ the symbolic entropy defined in (1). If the spatial process ${\left\{X_s\right\}}_{s\in S}$ is independent, it follows that:

• If the set $S^{\prime }$ is perfect then:

  1. 

  The affine transformation of the symbolic entropy

  $$SG\left({\mathsf{\Gamma }}_i\right)=2R[Ln\left(n_i\right)-h\left({\mathsf{\Gamma }}_i\right)]$$

  is asymptotically ${\chi }_{d_i}^2$ distributed where $d_2=2^{m-1}$, $d_3={(k-1)}^2+2$, $n_2=2^{m-1}$ and $n_3=k^2$.

  2. 

  Then $(\frac{Y_{{\sigma }_1}-R{p}_{{\sigma }_1}}{\sqrt{R{p}_{{\sigma }_1}(1-p_{{\sigma }_1})}},\frac{Y_{{\sigma }_2}-R{p}_{{\sigma }_2}}{\sqrt{R{p}_{{\sigma }_2}(1-p_{{\sigma }_2})}},\cdots ,\frac{Y_{{\sigma }_{ni}}-R{p}_{{\sigma }_{ni}}}{\sqrt{R{p}_{{\sigma }_{ni}}(1-p_{ni})}})$ is a multivariate normal distribution MN$(0,I)$.

• If the set $S^{\prime }$ is not perfect, then:

  1. 

  Since the sets $B_i^{\sigma }$ have small cardinality, we can get a good approximation to the following result of [17]. The affine transformation of the symbolic entropy

  $$SG\left({\mathsf{\Gamma }}_i\right)=2R[Ln\left(n_i\right)-h\left({\mathsf{\Gamma }}_i\right)]$$

  is asymptotically ${\chi }_{d_i}^2$ distributed where $d_2=2^{m-1}$, $d_3={(k-1)}^2+2$, $n_2=2^{m-1}$ and $n_3=k^2$.

  2. 

  Then $(\frac{Y_{{\sigma }_1}-R{p}_{{\sigma }_1}}{\sqrt{Var\left(Y_{{\sigma }_1}\right)}},\frac{Y_{{\sigma }_2}-R{p}_{{\sigma }_2}}{\sqrt{Var\left(Y_{{\sigma }_2}\right)}},\cdots ,\frac{Y_{{\sigma }_{ni}}-R{p}_{{\sigma }_{ni}}}{\sqrt{Var\left(Y_{{\sigma }_{ni}}\right)}})$ is a multivariate normal distribution MN$(0,\mathsf{\Sigma })$ where the variance and covariance matrix can be estimated using (4) and (5).

In Section 2 we indicate that there is a class of symbolization maps that are non-standard. Consider a situation in which a reduction in the number of possible symbols under study will benefit the behavior and properties of the test. In this, and other potential situations, non-standard maps might be useful. As an example, we now construct a non-standard symbolization map to test for independence in the spatial context. The following symbolization is an example of the most general procedure that we give in Appendix A.3.

Consider again the set ${\mathsf{\Gamma }}_2$ of symbols defined in (9) for a fixed embedding dimension m. Now we will denote by $\overline{a}$ the rest of the division of a over $m-1$.

Now define the following equivalence relation ∼:

$$({\mathcal{I}}_{s{s}_1},{\mathcal{I}}_{s{s}_2},\cdots ,{\mathcal{I}}_{s{s}_{m-1}})\sim ({\mathcal{I}}_{s^{\prime }{s}_1^{\prime }},{\mathcal{I}}_{s^{\prime }{s}_2^{\prime }},\cdots ,{\mathcal{I}}_{s^{\prime }{s}_{m-1}^{\prime }})$$

if and only if there exists an integer k such that ${\mathcal{I}}_{s^{\prime }{s}_i^{\prime }}={\mathcal{I}}_{s{s}_{\overline{i+k}}}$ for all $i\in \{1,2,\cdots ,m-1\}$.

Now we consider as a set of symbols ${\mathsf{\Gamma }}_4={\mathsf{\Gamma }}_2/\sim$ the set of classes in ${\mathsf{\Gamma }}_2$ modulo, the equivalence relation ∼.

Notice that, in general, in this case not all the symbols in ${\mathsf{\Gamma }}_4$ have the same probability of occurring, and therefore the symbolization map $f_4:{\left\{X_s\right\}}_{s\in S^{\prime }}\to \widetilde{{\mathsf{\Gamma }}_4}$ is non-standard.

7.3. Spatiotemporal Independence Tests

The issues related to spatiotemporal data modeling occupy a prominent role in current econometrics, where we can find recent literature devoted to this topic (see [9,30]). Spatiotemporal dependence introduces considerable difficulties with respect to modeling, computation and statistical theory. If independence can be taken for granted, and likewise the common assumption of cross-sectional independence, then computations and the application of inference rules simplifies significantly. It seems reasonable therefore to test first for spatiotemporal independence, and if the evidence for independence is strong, then proceed with the well-known methods. Unfortunately, tests for spatiotemporal independence are scarce. The aim of this section is twofold: to contribute to this rather scarce literature, and to highlight the usefulness of the novel general method presented in this paper. To this end we consider the relevant null of spatiotemporal dependence. Of particular interest for our tests is that dependence is not taken as a synonymous with correlation, and therefore nonlinearities are not restrictions for our test.

Consider the process ${\left\{X_{ts}\right\}}_{t\in I,s\in S}$. As in the previous cases, one can define several standard and non-standard symbolization maps. For simplicity, we adapt the previous symbolizations to the spatiotemporal case as follows:

For a fixed location $s_0\in S^{\prime }$ define $\left\{X_{t\left(s_0\right)}\right\}$ as the time series $\{X_{1s_0},X_{2s_0},\cdots ,X_{p\left(s_0\right)},\cdots \}$. Similarly for a fixed period $t_0\in I^{\prime }$ we define $\left\{X_{\left(t_0\right)s}\right\}$ as the spatial process

$$\{X_{t_0{s}_1},X_{t_0{s}_2},\cdots ,X_{t_0{s}_p},\cdots \}$$

Let $m_t,m_s\in \mathbb{N}$ with $m_t,m_s\ge 2$ be the time and space embedding dimensions respectively. Then under this setting we define the following decomposable symbolization maps $F_{1i}:{\left\{X_{ts}\right\}}_{t\in I^{\prime },s\in S^{\prime }}\to {\S }_m\times {\mathsf{\Gamma }}_i$ for $i=2,3$ and 4 defined by:

$$F_{1i}\left(X_{ts}\right)=(f_1\left(X_{t\left(s\right)}\right),f_i\left(X_{\left(t\right)s}\right))$$

(11)

where $f_1:\left\{X_{t\left(s\right)}\right\}\to S_m$ and $f_i:\left\{X_{\left(t\right)s}\right\}\to {\mathsf{\Gamma }}_i$ for $i=2,3,4$ are defined as above.

Notice that, when testing for spatiotemporal independence, when $i=2,3$ the symbolization map $F_{1i}$ is standard, while for $i=4$ is non-standard.

It is also possible to define an extension of the symbolization map $f_2$ in a spatiotemporal context. Indeed, consider the following map:

$$g:{\left\{X_{ts}\right\}}_{t\in I^{\prime },s\in S^{\prime }}\to \prod _{i=1}^{m_t}{\mathsf{\Gamma }}_2$$

(12)

defined by

$$g\left(X_{ts}\right)=(f\left(X_{\left(t\right)s}\right),f\left(X_{(t+1)s}\right),\cdots ,f\left(X_{(t+m_t-1)s}\right))$$

where $f\left(X_{(t+i)s}\right)=({\mathcal{I}}_{ts,(t+i)s_1},{\mathcal{I}}_{ts,(t+i)s_2},\cdots ,{\mathcal{I}}_{ts,(t+i)s_{m_t-1}}))$ for all $i=0,1,\cdots ,m_t-1$ and the indicator function ${\mathcal{I}}_{ts,(t+i)s_j}$ is defined as in (7).

Accordingly, using this symbolization map, the next corollary straightforwardly follows from Theorem 2:

Corollary 3.

Let $F_{1i}:{\left\{X_{ts}\right\}}_{t\in I^{\prime },s\in S^{\prime }}\to S_{m_t}\times {\mathsf{\Gamma }}_i$ and $g:{\left\{X_{ts}\right\}}_{t\in I^{\prime },s\in S^{\prime }}\to \prod _{i=1}^{m_t}{\mathsf{\Gamma }}_2$ be the standard symbolization maps defined in (11) with $i=2,3$ and in (12) respectively. Denote by $h(S_{m_t}\times {\mathsf{\Gamma }}_i)$ and $h\left({\displaystyle \prod _{i=1}^{m_t}}{\mathsf{\Gamma }}_2\right)$ the symbolic entropy defined in (1). If the spatiotemporal process ${\left\{X_{ts}\right\}}_{t\in I,s\in S}$ is independent, then:

• If indexes sets are perfect then:

  1. 

  The affine transformations of the symbolic entropy

  $$\begin{array}{ccc}\hfill STG(S_{m_t}\times {\mathsf{\Gamma }}_i)& =& 2RT[Ln\left(n_i\right)-h(S_{m_t}\times {\mathsf{\Gamma }}_i)]\hfill \end{array}$$

  (13)

  $$\begin{array}{ccc}\hfill STG\left(\prod _{i=1}^{m_t}{\mathsf{\Gamma }}_2\right)& =& 2RT[Ln\left(2^{m_t(m_s-1)}\right)-h\left(\prod _{i=1}^{m_t}{\mathsf{\Gamma }}_2\right)]\hfill \end{array}$$

  (14)

  are asymptotically ${\chi }_q^2$ distributed. In (13) for $i=2$$q=(m_t-1)!2^{m_s-1}$ and $n_2=m_t!2^{m_s-1}$, and for $i=3$$q=(m_t!-1)({(k-1)}^2+2)$ and $n_3=m_t!k^2$. In (14) $q=2^{m_t(m_s-1)}$.

  2. 

  Then $(\frac{Y_{{\sigma }_1}-R{p}_{{\sigma }_1}}{\sqrt{R{p}_{{\sigma }_1}(1-p_{{\sigma }_1})}},\frac{Y_{{\sigma }_2}-R{p}_{{\sigma }_2}}{\sqrt{R{p}_{{\sigma }_2}(1-p_{{\sigma }_2})}},\cdots ,\frac{Y_{{\sigma }_{2^{m-1}}}-R{p}_{\sigma n}}{\sqrt{R{p}_{{\sigma }_n}(1-p_{{\sigma }_n})}})$ is a multivariate normal distribution MN$(0,I)$ where n is the cardinality of the set of symbols.

• If indexes sets are not perfect then

  1. 

  If the sets $B_i$’s have small cardinality for all symbol η, then the affine transformations of the symbolic entropy

  $$\begin{array}{ccc}\hfill STG(S_m\times {\mathsf{\Gamma }}_i)& =& 2RT[Ln\left(n_i\right)-h(S_m\times {\mathsf{\Gamma }}_i)]\hfill \end{array}$$

  (15)

  $$\begin{array}{ccc}\hfill STG\left(\prod _{i=1}^m{\mathsf{\Gamma }}_2\right)& =& 2RT[Ln\left(2^{m_t(m-1)}\right)-h\left(\prod _{i=1}^{m_t}{\mathsf{\Gamma }}_2\right)]\hfill \end{array}$$

  (16)

  are asymptotically ${\chi }_q^2$ distributed. In (15) for $i=2$$q=(m_t-1)!2^{m_s-1}$ and $n_2=m_t!2^{m_s-1}$, and for $i=3$$q=(m_t!-1)({(k-1)}^2+2)$ and $n_3=m_t!k^2$. In (16) $q=2^{m_t(m_s-1)}$.

  2. 

  Then $(\frac{Y_{{\eta }_1}-R{p}_{{\eta }_1}}{\sqrt{Var\left(Y_{{\eta }_1}\right)}},\frac{Y_{{\eta }_2}-R{p}_{{\eta }_2}}{\sqrt{Var\left(Y_{{\eta }_2}\right)}},\cdots ,\frac{Y_{{\eta }_n}-R{p}_{{\eta }_n}}{\sqrt{Var\left(Y_{{\eta }_n}\right)}})$ is a multivariate normal distribution MN$(0,\mathsf{\Sigma })$ where the variance and covariance matrix can be estimated using (4) and (5) and n is the cardinality of the set of symbols.

8. Empirical Behavior of the Tests for Spatiotemporal Independence

In this section we evaluate the empirical behavior of the STG test with different configurations for the subset $I^{\prime }.$ The first aim of this section is to show the flexibility of Corollary 3 to cope with different scenarios. The second goal is to evaluate the empirical behavior of the new test. An the third intention of this simulation is to evaluate the incidence of the selection of $I^{\prime }$ on the empirical size of the test and on the power.

To those ends we designed a Monte Carlo experiment as follows: Firstly we consider the problem of testing for independence on regular lattices of several orders—R = 64 (8 × 8) and T = 150; R = 100 (10 × 10)—for which we consider two possible temporal scenarios, depending on data availability, T = 200 and T = 800. We also simulated richer regular lattices of order R = 400 (20 × 20), although on this occasion we only considered T = 200. The symbolization map follows from (12) with $m_s=4$ and $m_t=2$. The test under study was generated from Corollary 3 under Expression (13). Therefore, we used a perfect indexes subset. This subset was constructed recursively, as indicated in Section 7: $N_{s_i{t}_j}\cap N_{s_r{t}_k}=\varnothing$, where $N_{s_i{t}_j}$ is the set conformed with $s_i,$ the three nearest neighbors of $s_i$ in $t_j$ and the four spatial locations in the next time period. The power of the test is evaluated with the following DGPs:

$$\begin{array}{ccc}\hfill DGP1& :& y_t={(I-\gamma W)}^{-1}(\alpha y_{t-1}+\lambda +{\epsilon }_t)\hfill \\ \hfill DGP2& :& y_t=\sqrt[3]{{(I-\gamma W)}^{-1}(\alpha y_{t-1}+\lambda +{\epsilon }_t)}\hfill \end{array}$$

where ${\epsilon }_t\sim N(0,1)$, which was also used for evaluating the empirical size of the test. Parameters $\alpha ,\gamma$ intensified temporal and spatial dependencies, respectively, and $\lambda$ was fixed at five in all simulations. The weighting matrix W has been specified as a binary type using a contiguity criterion and rook-type movements.

Table 1. Size and power of STG test under a perfect subset of indexes.

			DGPs1			DGPs2
		Size	Power
			$\alpha =0.1$	$\alpha =2/80,$	$\alpha =1/80,$	$\alpha =0.1$	$\alpha =2/60,$	$\alpha =1/60,$
R	T		$\gamma =0.1$	$\gamma =2/50$	$\gamma =1/50$	$\gamma =0.1$	$\gamma =2/30$	$\gamma =1/30$
64	150	0.060	1.000	0.096	0.073	1.000	0.152	0.077
100	200	0.064	1.000	0.149	0.065	1.000	0.220	0.086
100	800	0.053	1.000	0.492	0.112	1.000	0.860	0.196
400	200	0.058	1.000	0.412	0.111	1.000	0.772	0.160

collects the empirical size and power of STG statistical test for 1000 repetitions. It is straightforward to observe that the size is controlled, and the test is powerful. For low intensity level of parameter ($\alpha =\gamma =0.1)$ the test is absolutely powerful. We have to set $\alpha =1/40,\gamma =1/25$ (or below) to lose power. This occurred despite the DGP under consideration.

Regular spatiotemporal configurations are interesting because (1) time series posit a natural order for observations, (2) lattice data provide the simplest extension of time series and (3) some scientific methods are compatible with this spatiotemporal configuration. However, irregular patterns are of frequent occurrence with spatial data. In geographical settings, data are liable to be recorded across heterogeneously-sized administrative regions, while economic distances do not correspond to regular spacing. Therefore, it is also useful to adapt the STG symbolic test to irregular spatiotemporal settings. In terms of our general methodology (see Corollary 3) this problem in tractable by considering the symbolization map $F_{1i},i=2$ where we control the dependence among the indicators by controlling on average the cardinality of the sets $B_i$. Particularly, we will select the set of indexes $I^{\prime }$ such that $\left|{\overline{B}}_i\right|\le (m_s+m_t-1)/2;$ i.e., the average of the cardinality of the sets $B_i$ is less than half of the number of spationtemporal neighbors.

Therefore, to complete the experiment (in the case of nonperfect lattices) we evaluate the STG-version for irregular lattices where coordinates of each spatial location are drawn from a N(0,1). We have considered the three nearest neighbors for irregular lattices. Afterwards, the resulting matrix was row-standardised in the usual way.

Table 2. Size and power of STG test under a non-perfect subset of indexes.

			DGPs1			DGPs2
		Size	Power
			$\alpha =0.1$	$\alpha =2/80,$	$\alpha =1/80,$	$\alpha =0.1$	$\alpha =2/60,$	$\alpha =1/60,$
R	T		$\gamma =0.1$	$\gamma =2/50$	$\gamma =1/50$	$\gamma =0.1$	$\gamma =2/30$	$\gamma =1/30$
64	150	0.077	1.000	0.094	0.070	1.000	0.082	0.082
100	200	0.070	1.000	0.211	0.098	1.000	0.286	0.096
100	800	0.077	1.000	0.794	0.201	1.000	0.410	0.274
400	200	0.060	1.000	0.758	0.196	1.000	0.467	0.278

collects the size and power for models constructed from DGP1 and DGP2. The introduction of irregular lattices has led us to introduce non-perfect indexes, and accordingly the size of the test slightly increased, although the levels seem acceptable, particularly for generous sample data. Power is as interesting as for the case of perfect indexes, and therefore the same comments applies (similar results are obtained in the case of using the multivariate normal approximation).

Comparison with Other Spatiotemporal Test for Independence

We now face our test with an unfavorable scenario characterized by small amount of available data on irregular lattices, also in linear and nonlinear setups. To this end we consider pairs of the following sample sizes: (36 × 10), (64 × 10), (100 × 10), (100 × 30) and (200 × 10). According to our theoretical discussion, given data scarcity and irregular spatial configuration, we use the non-perfect subset of indexes. Additionally, we consider the symbolization map based on equivalence relations ${\overline{F}}_{1,2}=(f_1,{\overline{f}}_2)$ as depicted in Appendix A.3 for nonstandard maps.

To complete the empirical study, we compare our test with another nonparametric spatiotemporal test [31] which is described in Appendix A.4 and we refer to it as STBP. Notice, however, that the STBP test requires one to correctly specify the weighting matrix, W; this is not a requirement for the symbolic test.

In terms of empirical size, both tests behave similarly well for linear processes (

Table 3. Size and power of STG and STBP tests under linearity.

		Size		Power
		STG	STBP	STG	STBP	STG	STBP	STG	STBP
R	T			($\alpha =5/80,\gamma =5/50)$		($\alpha =10/80,\gamma =10/50)$		($\alpha =15/80,\gamma =15/50)$
36	10	0.069	0.046	0.096	0.154	0.328	0.615	0.787	0.989
64	10	0.057	0.054	0.145	0.193	0.504	0.815	0.962	0.999
100	10	0.063	0.066	0.192	0.228	0.677	0.926	0.993	1.000
100	30	0.058	0.054	0.443	0.395	0.982	1.000	1.000	1.000
200	10	0.063	0.067	0.346	0.327	0.935	0.997	1.000	1.000

). On the contrary, for the nonlinear processes (

Table 4. Size and power of STG and STBP tests under nonlinearity.

		Size		Power
		STG	STBP	STG	STBP	STG	STBP	STG	STBP
R	T			($\alpha =1/10,\gamma =1/10)$		($\alpha =2/10,\gamma =2/10)$		($\alpha =3/10,\gamma =3/10)$
36	10	0.060	0.002	0.094	0.007	0.269	0.060	0.507	0.473
64	10	0.055	0.000	0.111	0.002	0.376	0.119	0.784	0.743
100	10	0.067	0.000	0.175	0.011	0.557	0.209	0.941	0.915
100	30	0.056	0.000	0.357	0.005	0.949	0.642	1.000	1.000
200	10	0.062	0.001	0.281	0.007	0.856	0.381	1.000	0.998

), the size of the STBP test is poor, while the symbolic-based test performs as expected. In terms of empirical power, the STBP test outperforms the STG test, especially for low intensity levels of dependence in the case of the linear process. However, under a nonlinear spatiotemporal configuration, the STG clearly presents a better balance between size and power and outperforms the STBP in all cases.

9. Conclusions

Central null hypotheses in experimental and non-experimental branches of science can be easily tested by means of symbolized information. This paper provides with the analytical tools to construct nonparametric hypothesis tests based on symbols. These tools are able to cope with different null hypotheses and with distinct scenarios in which some realistic limitations might be imposed to test designs.

A shared characteristic of all these symbolic test families is that few assumptions are needed to obtain asymptotic results. Therefore, general applicability of this method is guaranteed. In particular, in this paper we have shown that two well-known symbolic-based tests are particular cases of the main symbolic theorem (Theorem 2), which is stated in this paper for the first time. Furthermore, a set of new symbolic-based tests for spatiotemporal independence is put forward by using the main results of this paper collected under the main symbolic theorem (Theorem 2). Monte Carlo simulations provide evidence of the extraordinary power of the contrasted test. Currently, there are circumstances where robustness to speed, noise or computational cost are paramount, so fruitful applications of symbolic analysis are favored.

Further lines of research are worthy. We now indicate some of them on which we and other scholars are currently working: (i) One of the appealing properties of symbolic-based testing is that it requires few assumptions. In this paper we have assumed stationarity; however, it would be interesting to study whether it is possible to be less restrictive. (ii) In the context of time series analysis, most available techniques require the existence of second moments; however, by using certain symbolizations, it might be possible to waive this requirement. This will allow time series researchers to consider a wider variety of model classes. (iii) One of the main contributions of the paper is that it suggests that researchers can design a symbolization procedure (map) to test null hypotheses. It would be interesting to study what types of null hypotheses are more suitable to analysis using symbolic maps.