A Hypothesis Test for the Goodness-of-Fit of the Marginal Distribution of a Time Series with Application to Stablecoin Data ^†

Abstract

A bootstrap-based hypothesis test of the goodness-of-fit for the marginal distribution of a time series is presented. Two metrics, the empirical survival Jensen–Shannon divergence ($\mathcal{E}SJS$) and the Kolmogorov–Smirnov two-sample test statistic ($KS2$), are compared on four data sets—three stablecoin time series and a Bitcoin time series. We demonstrate that, after applying first-order differencing, all the data sets fit heavy-tailed $\alpha$-stable distributions with $1<\alpha <2$ at the 95% confidence level. Moreover, $\mathcal{E}SJS$ is more powerful than $KS2$ on these data sets, since the widths of the derived confidence intervals for $KS2$ are, proportionately, much larger than those of $\mathcal{E}SJS$.

1. Introduction

The empirical survival Jensen–Shannon divergence ($\mathcal{E}SJS$) has recently been proposed as a goodness-of-fit measure of a fitted parametric continuous distribution [1]. However, the important issue of hypothesis testing whether the output $\mathcal{E}SJS$ value is significant was left open.

To alleviate this problem, we propose a hypothesis test based on the parametric bootstrap [2,3], and evaluate the method on time series data [4,5]. As a proof of concept, we chose four cryptocurrency time series, three stablecoin [6] data sets, and, for reference, we employ a fourth, Bitcoin [7], data set. The stablecoins we chose maintain their “stability” by being pegged to the dollar, and thus one would expect their volatility to be low. Apart from the general interest in cryptocurrency time series, it has already been shown that Bitcoin data are heavy-tailed [8]; thus, demonstrating that stablecoins also exhibit heavy tails is interesting in its own right. One reason to experiment with heavy-tailed distributions, such as the $\alpha$-stable distribution [9] (or simply the stable distribution) employed herein, is that they pose additional problems compared to, say, the normal distribution (in the special case when $\alpha =2$) due to their variance being infinite (in the more general case when $\alpha <2$).

The rest of the paper is organised as follows: In Section 2, we introduce the $\mathcal{E}SJS$ and, for comparison purposes, also bring in the well-known Kolmogorov–Smirnov two-sample test statistic ($KS2$) [10] Section 6.3. In Section 3, we present a parametric bootstrap-based goodness-of-fit hypothesis test. Time series do not necessarily comprise independent and identically distributed (iid) random variables (as is assumed in, say, [11]), so utilising more general models, such as autoregressive models (as is assumed in, say, [12]), is more appropriate when generating time series bootstrap samples. Here, we assume an autoregressive process of order one [4,5], abbreviated to AR(1), with $\alpha$-stable innovations, as in [13,14]. In Section 4, we introduce the cryptocurrency time series we experiment with, and fit them to a stable distribution after applying first-order differencing to the raw data, to obtain stationary processes. In particular, we demonstrate that in this case $\alpha <2$, that is, they are not normally distributed. In Section 5, we apply the goodness-of-fit hypothesis test of Section 3 to the cryptocurrency time series described in Section 4 and discuss the results. Finally, in Section 6, we provide our concluding remarks. We note that all computations were carried out using the Matlab software package.

2. Empirical Survival Jensen–Shannon Divergence

To set the scene, we assume a time series, $\mathbf{x}=\{x_1,x_2,\dots ,x_n\}$, where $x_t$, for $t=1,2,\dots ,n$ is a value indexed by time, t, for example, modelling the movement of a stock price. More specifically, a time series of n values is a random sample generated by a stochastic process that forms a sequence of random variables $\mathbf{X}=X_1,X_2,\dots ,X_n$, where each value $x_i$ is a realisation of the random variable $X_i$. The stochastic process $\mathbf{X}$ may be a sequence of iids, but, more often than not, a time series exhibits temporal dependencies between its values, which is more realistic. We will also assume that the time series is stationary [4,5]. This makes sense in our context, since we are particularly interested in the marginal distribution of $\mathbf{x}$, which we suppose comes from an underlying parametric continuous distribution D.

The empirical survival function of a value z for the time series $\mathbf{x}$, denoted by $\widehat{S}\left(\mathbf{x}\right)\left[z\right]$, is given by

$$\widehat{S}\left(\mathbf{x}\right)\left[z\right]=\frac{1}{n}\sum _{i=1}^n{I}_{\{x_i>z\}},$$

(1)

where I is the indicator function. In the following, we will let $\widehat{P}\left(z\right)=\widehat{S}\left(\mathbf{x}\right)\left[z\right]$ stand for the empirical survival function $\widehat{S}\left(\mathbf{x}\right)\left[z\right]$, where the time series $\mathbf{x}$ is assumed to be understood from the context; we will generally be interested in the empirical survival function $\widehat{P}$, which we suppose arises from the survival function P of the parametric continuous distribution D, mentioned above.

The empirical survival Jensen–Shannon divergence ($\mathcal{E}SJS$) [1] between two empirical survival functions, ${\widehat{Q}}_1$ and ${\widehat{Q}}_2$, arising from the survival functions $Q_1$ and $Q_2$, is given by

$$\mathcal{E}SJS({\widehat{Q}}_1,{\widehat{Q}}_2)=\frac{1}{2}{\int }_0^{\infty }{\widehat{Q}}_1\left(z\right)log\left(\frac{{\widehat{Q}}_1\left(z\right)}{\widehat{M}(z}\right)+{\widehat{Q}}_2\left(z\right)log\left(\frac{{\widehat{Q}}_2\left(z\right)}{\widehat{M}\left(z\right)}\right)\mathrm{d}z,$$

(2)

where

$$\widehat{M}\left(z\right)=\frac{1}{2}\left({\widehat{Q}}_1\left(z\right)+{\widehat{Q}}_2\left(z\right)\right).$$

We note that the $\mathcal{E}SJS$ is bounded and can thus be normalised, so it is natural to assume its values are between 0 and 1; in particular, when ${\widehat{Q}}_1={\widehat{Q}}_2$ its value is zero. Moreover, its square root is a metric (cf. [1]).

For completeness, we provide the definition of the Kolmogorov–Smirnov two-sample test statistic ([10] Section 6.3) between ${\widehat{Q}}_1$ and ${\widehat{Q}}_2$ as above, which is given by

$$KS2({\widehat{Q}}_1,{\widehat{Q}}_2)=\underset{z}{max}|{\widehat{Q}}_1\left(z\right)-{\widehat{Q}}_2\left(z\right)|,$$

(3)

where $max$ is the maximum function, and $\left|v\right|$ is the absolute value of a number v. We note that $KS2$ is bounded between 0 and 1, and is also a metric.

Now, for a parametric continuous distribution D, we let $\varphi =\varphi (D,\widehat{P})$ be the parameters that are obtained from fitting D to the empirical survival function, $\widehat{P}$. The distribution D may, in principle, be any continuous distribution, although here we concentrate on the $\alpha$-stable distribution, since it allows for the modelling of heavy-tailed data, which poses additional problems to those of light-tailed data, due to the variance (and possibly the mean) being infinite. In particular, we have an interest in cryptocurrency data, which is likely to be heavy-tailed [8].

We now let $P_{\varphi }=S_{\varphi }(\mathbf{x})$ be the survival function of $\mathbf{x}$, for D with parameters $\varphi$. Thus, the empirical survival Jensen–Shannon divergence and the Kolmogorov–Smirnov two-sample test statistic, between $\widehat{P}$ and $P_{\varphi }$, are given by $\mathcal{E}SJS(\widehat{P},P_{\varphi })$ and $KS2(\widehat{P},P_{\varphi })$, respectively. These values provide us with two measures of goodness-of-fit for how well D, with parameters $\varphi$, is fitted to $\mathbf{x}$ (cf. [1]).

3. A Bootstrap-Based Goodness-of-Fit Hypothesis Test

Our hypothesis test makes use of the parametric bootstrap [2,3]; the pseudocode for the parametric bootstrap in our context is given in Algorithm 1. It takes as input a time series $\mathbf{x}$, the distribution D we hypothesise $\mathbf{x}$ comes from, and the number of bootstrap samples m; in the simulations we use the typical value of $m=1000$ samples [15]. The algorithm outputs two vectors, $BV\text{-}\mathcal{E}SJS$ and $BV\text{-}KS2$. The first contains m$\mathcal{E}SJS$ values, for $i=1,2,\dots ,m$, between the empirical survival function ${\widehat{P}}_i=\widehat{S}\left({\mathcal{B}}_i\right)$ for the ith bootstrap sample, ${\mathcal{B}}_i$, and the survival function $P_{\varphi }=S_{\varphi }\left(\mathbf{x}\right)$ of $\mathbf{x}$, for D with parameters $\varphi$. Correspondingly, the second contains m$KS2$ values, for $i=1,2,\dots ,m$, between ${\widehat{P}}_i$ and $P_{\varphi }$. The bootstrap samples are generated by an AR(1) process with $\alpha$-stable distribution innovations [14] (see also [13]), which is more realistic than assuming that the samples are generated from an iid process, as in [11].

Algorithm 1: Parametric-Boostrap($\mathbf{x},D,m$).

1.   begin  2.         Initialise $BV\text{-}\mathcal{E}SJS$ and $BV\text{-}KS2$ as the vector, $\langle 0,0,\cdots ,0\rangle$, of m zeros;  3.        Let n be the number of values in $\mathbf{x}$;  4.        Let $\varphi =\varphi (D,\widehat{P})$;  5.        Let $P_{\varphi }=S_{\varphi }\left(\mathbf{x}\right)$;  6.        for $i=1$ to m do  7.          Generate a bootstrap sample ${\mathcal{B}}_i=x_{i1}^{*},x_{i2}^{*},\dots ,x_{in}^{*}$,  8.            where ${\mathcal{B}}_i$ is generated from an AR(1) process with innovations                  derived from D with parameters $\varphi$;  9.          Let $\widehat{P_i}=\widehat{S}\left({\mathcal{B}}_i\right)$;  10.          Let $BV\text{-}\mathcal{E}SJS\left(i\right)=\mathcal{E}SJS({\widehat{P}}_i,P_{\varphi })$;  11.           Let $BV\text{-}KS2\left(i\right)=KS2({\widehat{P}}_i,P_{\varphi })$;  12.        end for  13.        return $BV\text{-}\mathcal{E}SJS$ and $BV\text{-}KS2$ sorted in ascending order.  14.   end

As we have assumed that the time series is stationary, the absolute value $\left|\rho \right|$ of the parameter $\rho$ of the AR(1) process generating $\mathbf{x}$ should be less than one. For the generation process, we use an estimate $\widehat{\rho }$ of $\rho$, and, as we will see in Section 4, $|\widehat{\rho }|<1$ is satisfied for the data sets we employ, as required. We also add a burn-in period of 100 steps to the AR(1) process generated, which we found to be sufficient for the data sets we used.

Given the bootstrap vectors, $BV\text{-}\mathcal{E}SJS$ and $BV\text{-}KS2$, and the output from Algorithm 1, we can form confidence intervals for $\mathcal{E}SJS(\widehat{P},P_{\varphi })$ and $KS2(\widehat{P},P_{\varphi })$, according to the bootstrap percentile method ([16] Section 3.1.2), which is the simplest way to construct a bootstrap confidence interval; see [16] for improvements on the percentile method. We assume that the significance level we are interested in for a hypothesis test is a percentage, and set the significance level to 5%, which is the value we will use in Section 5.

Subsequently, for a one-sided test, we would exclude the highest $5\%$ values from the parametric bootstrap vector, say $BV$, returned from Algorithm 1, and for a two-sided test we would exclude from $BV$ the lowest $2.5\%$ values and the highest $2.5\%$ values. For both $\mathcal{E}SJS$ and $KS2$ only a one-sided test makes sense, since both metrics are bounded below by zero. Therefore, the null hypothesis is that the distribution of $\widehat{P}$ is D, and so we reject the null hypothesis at the $5\%$ confidence level, if $\mathcal{E}SJS(\widehat{P},P_{\varphi })$ or, correspondingly, $KS2(\widehat{P},P_{\varphi })$ is greater than the upper bound of the constructed confidence interval, depending on which goodness-of-fit measure we are employing.

4. Cryptocurrencies and Heavy Tails

As a proof of concept, we analysed four time series data sets. These include the prices of three stablecoins [6]: Tether (https://tether.to, accessed on 1 June 2021), DAI (https://makerdao.com, accessed on 1 June 2021) and USDC (https://www.centre.io/usdc, accessed on 1 June 2021), which are all pegged to the dollar. In addition, for comparison purposes, we make use of a fourth time series data set, the price of the archetypal decentralised cryptocurrency, Bitcoin [7], the price of which has previously been hypothesised to follow the heavy-tailed stable distribution [8].

In

Table 1. Description of time series data used for experimentation; #Values is the number of values in the time series.

Currency	#Values	From	Until	Closing Rate
Tether	1264	06 January 2017	15 November 2020	daily
DAI	362	20 November 2019	15 November 2020	daily
USDC	772	28 September 2018	15 November 2020	daily
Bitcoin	8929	01 January 2020	07 January 2021	hourly

, we describe the details of the time series data we used for the empirical validation of the proposed goodness-of-fit method; the data were obtained from Coin Metrics (https://coinmetrics.io, accessed on 1 June 2021). For the stablecoins, 1 is subtracted from the daily closing rate, so that its value is positive if above 1, zero if exactly 1, and negative if below 1. For analysis purposes we applied first-order differencing [4,5] to all the time series, that is, we computed the difference between consecutive observations, which is useful for removing trends, transforming the price time series into a return series (in future work we will also consider analysing the raw data set without differencing; however, since our main aim is to introduce the hypothesis test, for brevity and clarity of exposition we will not consider this further analysis here). The time series, after differencing was applied to the raw data sets, are shown in Figure 1.

The $\alpha$-stable distribution (or simply the stable distribution) [9] has four parameters: (i) the characteristic exponent $\alpha \in (0,2]$; (ii) the skewness parameter $\beta \in [-1,1]$ (when $\beta =0$, the distribution is symmetric); (iii) the scale parameter $\gamma$; and (iv) the location parameter $\delta$. It is heavy-tailed unless $\alpha =2$, when the stable distribution reduces to the light-tailed normal distribution with $\beta =0$. When $\alpha <2$, the stable distribution is heavy-tailed, its variance as well as all its other higher moments are infinite; in the case of $\alpha \le 1$, its mean is also infinite. In the following we will refer to a distribution as stable when $\alpha <2$, and normal when $\alpha =2$.

In Figure 2, we show the histograms of the marginal distributions of the four cryptocurrencies overlaid with the curve of the maximum likelihood fit of the normal distribution to the data. It is visually evident that the normal distribution is not a good fit for these data sets. Kurtosis of a distribution, in this case the marginal distribution of a time series, indicates peakedness and tailedness of the data relative to the normal distribution [17] (for ease of comparison with the kurtosis of the normal distribution, which is 3, we will subtract 3 from the kurtosis, giving the excess kurtosis). In

Table 2. Excess kurtosis of the four cryptocurrencies.

Currency	Excess Kurtosis
Tether	86.0207
DAI	34.6573
USDC	10.1905
Bitcoin	59.7350

, we show the excess kurtosis of the four cryptocurrencies, which provides further evidence that none of them follow a normal distribution, and are in fact heavy-tailed.

Next, we fitted the stable distribution to the four data sets using the Matlab implementation provided by [18], which is based on the empirical characteristic function method [19]. The fitted parameters are shown in

Table 3. Parameters from fits of the stable distribution to the data of the four cryptocurrencies.

Fitted Parameters for Stable Distribution
Currency	$\alpha$	$\beta$	$\gamma$	$\delta$
Tether	1.0111	0.0019	0.0011	0.0001
DAI	1.1953	0.0821	0.0016	0.0003
USDC	1.2259	0.0125	0.0003	0.0000
Bitcoin	1.2261	0.0909	27.9685	7.3644

, noting that in all cases $1<\alpha <2$, implying that the means of the marginal distributions are finite but their variances are infinite.

5. Application of the Goodness-Of-Fit Hypothesis Test to Cryptocurrencies

We apply the bootstrap goodness-of-fit test presented in Section 3, based on the empirical survival Jensen–Shannon divergence ($\mathcal{E}SJS$) and Kolmogorov–Smirnov two-sample test statistic ($KS2$) metrics, to construct 95% confidence intervals for $\mathcal{E}SJS(\widehat{P},P_{\varphi })$ and $KS2(\widehat{P},P_{\varphi })$, where $\widehat{P}$ is the empirical survival function of the input time series and $P_{\varphi }$ is the survival function of time series $\mathbf{x}$, for D with parameters $\varphi$. When running Algorithm 1, we computed 1000 bootstrap samples, that is, we set $m=1000$. Moreover, it can be seen in

Table 4. Estimates $\widehat{\rho }$ of the parameter $\rho$ of the AR(1) process for the four cryptocurrencies, noting that, when $\left|\rho \right|<1$, the process is stationary.

Currency	$\widehat{\mathit{\rho }}$
Tether	−0.3604
DAI	−0.4045
USDC	−0.4948
Bitcoin	−0.0504

that, for all data sets, the estimate $\widehat{\rho }$ of the AR(1) parameter is less than one in absolute value, implying that the generated bootstrap time series, ${\mathcal{B}}_i$, are stationary as required.

In

Table 5. Parametric bootstrap results for the $\mathcal{E}SJS$ hypothesis test assuming the marginal distribution is stable; LB, UB, CI, Mean and STD stand for lower bound, upper bound, confidence interval, mean of samples and standard deviation of samples, respectively.

Parametric Bootstrap for $\mathcal{E}\mathit{SJS}$ Assuming a Stable Distribution
Currency	LB of CI	UB of CI	Width of CI	$\mathcal{E}SJS$	Mean	STD
Tether	0.0006	0.0232	0.0226	0.0090	0.0198	0.0741
DAI	0.0030	0.0345	0.0315	0.0156	0.0188	0.0096
USDC	0.0013	0.0247	0.0234	0.0119	0.0133	0.0063
Bitcoin	0.0004	0.0066	0.0062	0.0061	0.0036	0.0016

and

Table 6. Parametric bootstrap results for the $KS2$ hypothesis test assuming the marginal distribution is stable; LB, UB, CI, Mean and STD stand for lower bound, upper bound, confidence interval, mean of samples and standard deviation of samples, respectively.

Parametric Bootstrap for $\mathit{KS}2$ Assuming a Stable Distribution
Currency	LB of CI	UB of CI	Width of CI	$KS2$	Mean	STD
Tether	0.0014	0.0308	0.0294	0.0139	0.0289	0.0996
DAI	0.0029	0.0532	0.0503	0.0358	0.0299	0.0136
USDC	0.0035	0.0374	0.0339	0.0219	0.0210	0.0093
Bitcoin	0.0008	0.0103	0.0095	0.0088	0.0057	0.0025

, we show the results of the bootstrap hypothesis test when employing the $\mathcal{E}SJS$ and $KS2$ metrics, respectively. In particular, for all data sets, both metrics are within the 95% confidence interval, and thus with 95% confidence we cannot reject the null hypothesis that the marginal distribution of the input time series comes from an $\alpha$-stable distribution.

The bar chart in Figure 3 shows that for all four cryptocurrencies the width of the confidence interval for the $KS2$ goodness-of-fit measure is, proportionately, much larger than that of the $\mathcal{E}SJS$ goodness-of-fit measure. Statistical tests using measures resulting in smaller confidence intervals are normally considered to be more powerful as this implies, with high confidence, that a smaller sample size may be deployed [20].

Finally, to provide contrast to the stable distribution result, we now hypothesise that the marginal distribution of the time series is actually normal (i.e., $\alpha =2$). We see in

Table 7. Parametric bootstrap results for the $\mathcal{E}SJS$ and $KS2$ hypothesis tests assuming the marginal distribution of the time series for the four cryptocurrencies is normal; LB and UB stand for lower and upper bounds of the confidence intervals, respectively, and we abbreviate $\mathcal{E}SJS$ to $\mathcal{E}$ and $KS2$ to K.

Parametric Bootstrap Results Assuming a Normal Distribution
Currency	LB-$\mathcal{E}$	UB-$\mathcal{E}$	$\mathcal{E}SJS$	LB-K	UB-K	$KS2$
Tether	0.0001	0.0132	0.1440	0.0004	0.0182	0.2162
DAI	0.0003	0.0240	0.1160	0.0006	0.0330	0.1665
USDC	0.0002	0.0147	0.0830	0.0002	0.0227	0.1330
Bitcoin	0.0001	0.0067	0.1218	0.0000	0.0085	0.1708

that, for all four cryptocurrencies, we reject the null hypothesis that the marginal distribution is normal, as both the $\mathcal{E}SJS$ and $KS2$ are outside their respective 95% confidence intervals.

6. Concluding Remarks

We presented a proof of concept of the bootstrap-based goodness-of-fit test on four cryptocurrency time series, concentrating on the $\alpha$-stable distribution, which allows for the modelling of heavy-tailed data. Our results demonstrate that, when first-order differenced, the marginal distributions of all four time series are all $\alpha$-stable with $\alpha <2$. Moreover, for both $\mathcal{E}SJS$ and $KS2$, the confidence level of the bootstrap-based test is at the 95% level. Furthermore, $\mathcal{E}SJS$ is more powerful than $KS2$ on these data sets, since the widths of the derived confidence intervals for the $KS2$ measure are, proportionately, much larger than those for the $\mathcal{E}SJS$ measure.

We emphasise that the proposed goodness-of-fit test may be applied to any marginal distribution, not just to the heavy-tailed stable distributions. Thus, there is a need to further establish the validity of the proposed hypothesis test on more data sets and on a variety of distributions, which may or may not be heavy-tailed. In addition, it would be useful to look at the assumptions regarding the process underlying the generation of the time series, and to ascertain how this affects the hypothesis test.