Analyzing Crude Oil Prices under the Impact of COVID-19 by Using LSTARGARCHLSTM

Abstract

Under the influence of the COVID-19 pandemic and the concurrent oil conflict between Russia and Saudi Arabia, oil prices have exhibited unusual and sudden changes. For this reason, the volatilities of the West Texas Intermediate (WTI), Brent and Dubai crude daily oil price data between 29 May 2006 and 31 March 2020 are analysed. Firstly, the presence of chaotic and nonlinear behaviour in the oil prices during the pandemic and the concurrent conflict is investigated by using the Shanon Entropy and Lyapunov exponent tests. The tests show that the oil prices exhibit chaotic behavior. Additionally, the current paper proposes a new hybrid modelling technique derived from the LSTARGARCH (Logistic Smooth Transition Autoregressive Generalised Autoregressive Conditional Heteroskedasticity) model and LSTM (long-short term memory) method to analyse the volatility of oil prices. In the proposed LSTARGARCHLSTM method, GARCH modelling is applied to the crude oil prices in two regimes, where regime transitions are governed with an LSTAR-type smooth transition in both the conditional mean and the conditional variance. Separating the data into two regimes allows the efficient LSTM forecaster to adapt to and exploit the different statistical characteristics and ARCH and GARCH effects in each of the two regimes and yield better prediction performance over the case of its application to all the data. A comparison of our proposed method with the GARCH and LSTARGARCH methods for crude oil price data reveals that our proposed method achieves improved forecasting performance over the others in terms of RMSE (Root Mean Square Error) and MAE (Mean Absolute Error) in the face of the chaotic structure of oil prices.

1. Introduction

The petroleum market is currently going through one of the most volatile times in its history. The volatilities of crude oil prices are affected not only by macroeconomic and microeconomic variables but also by the speculative activities and non-economic variables such as geopolitical tensions, the Gulf war, and, nowadays, the coronavirus disease 2019 (COVID-19) and the conflict between Russia and Saudi Arabia. In the recent months, uncertainty and volatility in oil prices due to COVID-19 and the conflict between Russia and Saudi Arabia have impacted the investors’ decisions for portfolio allocation and manufacturers’ decisions for industrial production and economy. While the COVID-19 pandemic has led to worldwide recession, it has also resulted in a drop in demand for oil. The countries in the Middle East and North Africa (MENA) have been faced with both the COVID-19 pandemic as well as the steep decline in oil prices. That is, the petroleum market has endured the pressure of an increase in supply alongside a decrease in demand. In this respect, the differences in opinion between Russia and Saudi Arabia (on 9 April, Russia and Saudi Arabia resolved their opinion differences) and the disagreements among OPEC and countries on petroleum production reduction had an effect on the increase in supply.

Under the influence of COVID-19 and the conflict between Russia and Saudi Arabia, the petroleum market faced both a negative demand shock and a negative supply shock [1]. A negative supply shock arises from a drop in the workforce, which in the case of COVID-19 may be directly due to the workers acquiring the disease or may be indirect due to travel restrictions, lockdown conditions or tending to children staying at home. The supply is influenced by a drop in materials, capital and intermediate inputs due to the disturbances in the transportation network and admistration in MENA countries [2]. Negative demand shock exhibits global and local features [3]. Since the COVID-19 pandemic has caused recessions in all economies, the recessions, lockdown conditions, and disruptions to global value chains have resulted in a reduction in petroleum demand. It has shuttered factories; stopped people from travelling; increased the number of people staying at home; caused a drop in materials, capital and intermediate inputs due to the disturbances in the transportation network and admistration; and decreased production in many industrialised countries. That is, due to the restrictions in 187 countries and regions, petroleum demand has nearly stopped. The International Energy Agency (IEA) [4] has predicted that the petroleum demand will be 29 million barrels (mb/day) less in the month of April compared to in the same month a year ago. Moreover, this level is the lowest since 1995. For the second quarter of 2020, it is predicted that the demand will be 23.1 mb/day below the previous year’s demand.

Moreover, as mentioned above, together with the decline in petroleum demand, the petroleum supply, which maintained its pre-pandemic level, is responsible for the drop in petroleum prices. The petroleum storage capacities of countries have been filled for the first time in history [5]. On the other hand, to maintain market share, some oil producers stored their excess oil at sea by leasing tankers at high costs. As a consequence, the petroleum supply had to come down to the record low level of 12 mb/day in May. In future prices, West Texas Intermediate (WTI) entered the negative region and finished the day at the USD 37 per-barrel level.

Due to these unforeseen circumstances, the Middle East region is going through an unexpected turning point. Especially, the MENA countries are going through an economic recession triggered by the shock from COVID-19 [6], the breakdown in negotiations between the OPEC countries [5], the oil war between Russian and Saudi Arabia, and the uncertainties about China’s economic recovery. The economies of Saudi Arabia and Russia depend heavily on their oil revenues. Although the Russian economy is more diversified than Saudi Arabia’s economy, both economies face similar disruptions because oil revenues represent a very high share of their gross domestic products (GDPs). According to Kubursi [5], while Saudi Arabia needs a USD 80 per-barrel price to balance its budget, Russia needs a price of USD 60 to balance its government budget.

As the volatility of oil prices continues to increase, many countries and sectors will be affected. In this context, the volatility of oil prices points to both the regime models and the presence of a nonlinear and chaotic structure.

This paper aims to analyse the volatilities of WTI, Brent and Dubai crude oil prices between 29 May 2006 and 31 March 2020. Firstly, it explores the presence of chaotic and nonlinear behaviour in the oil prices by Shanon Entropy (SE) and Lyapunov exponent (L_e) tests, and secondly, it suggests the Logistic Smooth Transition Autoregressive Generalised Autoregressive Conditional Heteroskedasticity long-short term memory (LSTARGARCHLSTM) method. If the SE and (L_e) tests determine chaotic or nonlinear behavior, we propose a hybrid modelling technique developed by combining the LSTARGARCH [7] with LSTM. The LSTARGARCH model has STAR type nonlinearity in both the conditional mean and variance and allows the smooth transitions between the regimes to be governed by a logistic function. To evaluate the success or accuracy of our proposed method, we compare with GARCHLSTM and traditional methods; GARCH and LSTARGARCH. Finally, in this paper, since forecasting performance is accepted as a measure of the success of applied methods, we compare the forcasting performance of all the models.

This paper contributes to the related literature on theory, methodology and econometric applications by determining the differentiating characteristics of the volatility of the oil price in different regimes and by helping to specify the corresponding policies for these characteristics.

The paper is organised as follows: In Section 2, the related studies are reviewed. In Section 3, the proposed hybrid model and the methodology are described. The empirical results are presented in Section 4. The last part of the paper presents the concluding remarks and policy recommendations.

2. Related Work

Since the shocks stemming from COVID-19 have dramatically affected countries and many sectors, the impacts of the COVID-19 pandemic on the economy have been discussed by several papers. Baldwin [8] explored the connection between the inequality of wealth and health and the pandemic and argued that since for millions without higher education, financial and health difficulties abound, the number of hospitalisations and thereby the mortality rate due to COVID-19 is likely to be substantially higher for this group relative to that for the rest of the society. Gourinchas [9] pointed out that flattening the pandemic curve inevitably makes the macroeconomic recession curve steeper, which can be flattened by taking appropriate economic measures at a fiscal cost. Gopinath [10] discussed limiting the economic fallout from the coronavirus with targeted policies. Blanchard [11] and Alesina and Giavazzi [12] focused on ways to avoid another Euro crisis due to COVID-19. Gali [13] showed the importance of “helicopter money”, suggested by Gali [14] to avoid recession. Wyplosz [15] presented a solution to avoid a potential debt crisis during COVID-19. Cecchetti and Schoenholtz [16] accented the importance of Bank Runs and Panics. Cochrane [17] explored the impact of COVID-19 on monetary policy. Bénassy-Quéré [18] accented that the decline in oil prices will impact headline inflation and, as occurred in the past, could impact household and corporate inflation expectations. Arezki et al. [2,3] stressed that to limit the risk of financial instability, interest rates should be reduced and liquidity should be injected into the banking system and accented that, in places where inflation is low, liquidity injection and targeted cash transfers can be financed by “helicopter money” [14] that is banknotes printed by the central banks [13].

However, these studies do not consider the impacts of COVID-19 on the volatilities of oil prices, which has caused many issues. In our opinion, the volatility of oil prices points to both the regime structure and the presence of nonlinear and chaotic structure. Some previous work examined the volatility of the oil price and modelled it as a process with chaotic and nonlinear behavior. Barone-Adesi et al. [19] suggested a semiparametric method to examine the structure of oil prices. Adrangi et al. [20] determined the presence of low-dimensional chaotic structure in the oil prices. Lahmiri [21], Komijani et al. [22] and He [23] are the other studies that determine the presence of chaos in oil prices.

On the other hand, some previous studies emphasised the presence of outliers in the oil price. The outliers may impact the estimation and identification power of the generalised autoregressive conditional heteroskedasticity (GARCH) methods and can cause heteroscedasticity or conditional heteroscedasticity [24,25]. Moreover, they can affect the out-of-sample forecasting performance [24,26]. To address the issue of outliers, different methods have been proposed. Ané et al. [27] tested the performance of a method after noticing the outliers in a GARCH model by employing their proposed method. Charles and Darné [28] detected and corrected the outliers in the GARCH models. However, these methods do not have high forecasting performance.

Some papers combined STAR (Smooth Transition Autoregressive) and GARCH methods to reduce the problems mentioned above. The STAR technique by Teräsvirta [29], Luukkonen et al. [30] and Granger and Teräsvirta [31] suggest the nonlinear approach of the conditional mean based on smooth transition between stages of autoregressive processes by using the exponential and logistic squashing functions. Following these studies, González-Rivera [32], Hagerud [33], Dufrénot et al. [34] and Anderson et al. [35] suggested the STARGARCH (STGARCH) model, and Ané and Ureche-Rangau [36] suggested the Regime Switching Asymmetric Power GARCH (RS-APGARCH) model. Franses et al. [37] emphasised the importance of the STARGARCH methods. Lundbergh et al. [38] employed a STARGARCH method that permits non-linearity in both the conditional variance and the conditional mean. Lee and Degennaro [39], Chan and McAleer [40] showed the statistical properties of STARGARCH models. Bildirici and Ersin [41] suggested the STARSTGARCH model.

Some papers used various methods based on econometric models or intelligent algorithms such as Neural Network [42,43,44,45,46,47], genetic algorithms (GA) [48], support vector machines (SVM) [49] and support vector regression improved with a meta-heuristic algorithm [50,51]. The other studies in the literature adopted a decomposition-integration (EMD) method. Zhang et al. [52] and Yu et al. [53] used the EMD method to determine the main driving power behind the volatility of oil prices and showed that irregular events increase volatility.

Donaldson and Kamstra [42] suggested a class of Neural Network GARCH models. Gonzalez Miranda and Burgess [43], Hamid and Iqbal [44], and Bildirici and Ersin [45] combined the neural network method with the GARCH model to predict oil prices. Roh [47] combined the EWMA, GARCH and EGARCH models with a feedforward neural network model and determined that the EGARCH model with the feedforward neural network gives the best results. Kristjanpoller and Hernández [54] compared the predictability with a GARCH model against that with a hybrid neural network (HNN) and argued that, for forecasting volatility, HNN models are more suitable than the GARCH models.

A few papers analysed oil price volatility by combining regime models and neural network methods. Bildirici and Ersin [7] used the LSTARLSTGARCH family and LSTARLSTGARCHNN family to analyse oil prices. According to results of this work, the volatility, nonlinearity and asymmetric characteristics of oil prices were best captured by the LSTARLSTGARCHMLP family models. Bildirici and Ersin [55] suggested the LSTARLSTGARCHRBF models for forecasting the returns of oil prices. Their results indicated that the LSTARLSTGARCHRBF and LSTARLSTGARCHMLP models provide significant improvements over GARCH models.

On the other hand, some papers combined LSTM and GARCH models. Wex et al. [56] and Yu et al. [57] integrated financial news into the process of forecasting crude oil prices. Zhao et al. [58] used LSTM for oil price forecasting. Chen et al. [59] suggested the deep learning model to forecast oil prices. Gupta and Pandey [60] tested the volatility of oil prices by employing LSTM-based recurrent neural networks. For forecasting oil prices, Li et al. [61] suggested the EEMD-SBL-ADD method, which combines EEMD (ensemble empirical mode decomposition) and SBL (sparse Bayesian learning). Huang and Wu [62] used deep multiple kernel learning (DMKL) and showed that this method decreased the forecasting errors. Li et al. [63] suggested models based on deep learning to forecast oil prices where text and financial features were presented as complementary features for the deep learning models to obtain forecasts of oil prices with higher accuracy. Kim and Won [64] also proposed a hybrid model that combines the GARCH model with LSTM (long short-term memory) to forecast financial volatility. However, this work does not integrate deep learning with a regime switching GARCH method.

3. Methodology

In this paper, in accordance with Franses et al. [65], Terasvirta [66], and Chan and McAleer [67], STARGARCH type models are applied. The LSTARGARCH model supports STAR-type nonlinearity both in the conditional mean and in the conditional variance. In this model, the conditional volatility is based on the size and sign of the shocks on ${\epsilon }_{t-1}$. Since the relative impacts of positive and negative shocks with equal magnitude depend on their conditional volatility, negative and positive shocks with similar sizes produce different effects [41,55].

Consider the following STAR model with two regimes:

$$y_t=({\varphi }_1+{\displaystyle \sum }_{i=1}^r{\varphi }_{1i}{y}_{t-i}\left(1-F\left(s_{t-d};\gamma ,c\right)\right)+({\varphi }_2+{\displaystyle \sum }_{i=1}^r{\varphi }_{2i}{y}_{t-i}F\left(s_{t-d};\gamma ,c\right))+{\epsilon }_t$$

(1)

where

$$F\left(s_{t-d};\gamma ,c\right)=\frac{1}{1+e^{-\gamma \left(s_{t-d}-c\right)}}$$

(2)

It was defined with the logistic function. $F\left(s_{t-d};\gamma ,c\right)$ is the logistic transition function restricted to allow the transition to be a function of a single variable s and its respective distance to the threshold c.

$\gamma$ takes positive values. When $\gamma \to \infty$, it shows the TAR (threshold autoregressive) model. On the other hand when $\gamma \to 0$, the transition becomes smoother. If $\gamma =\infty$ and $s_{t-d}<c$, then $F=0$. However, if $s_{t-d}>c$, then $F=1$. The $F$ function moves very quickly from 0 to 1. When the transition function is too steep, the LSTAR model converts to a SETAR (self exciting threshold autoregressive) model with two regimes. When $\gamma \to 0$, $F$ converges to a constant, and if $\gamma =0$, $F$ is equal to 0.5 and the LSTAR model turns into a linear AR(p) model.

The LSTARGARCH model is a model that allows STAR-type nonlinearity in both the conditional mean and the conditional variance. When the error terms follow a smooth transition in the GARCH process, by allowing GARCH errors, is obtained following equation:

$${\sigma }_t^2=\left(w+{\displaystyle \sum }_{i=1}^p{\alpha }_i{\varsigma }_{t-i}^2+{\displaystyle \sum }_{i=1}^r{\beta }_i{\sigma }_{t-i}^2\right)$$

(3)

The model is called STARGARCH. Since the information matrix of the STARGARCH Log-likelihood (LL) function is block diagonal, the parameters in the conditional mean equations can be estimated separately from the parameters in the conditional variance equations, as in the case of ARMA-GARCH.

To achieve a smooth transition in the GARCH procedure for the error terms and to obtain the LSTARGARCH model, the following equation is used:

$$\begin{array}{c}{\sigma }_t^2=\left(w_1+{\displaystyle {\displaystyle \sum }_{i=1}^p}{\alpha }_{1i}{\varsigma }_{t-i}^2+{\displaystyle {\displaystyle \sum }_{i=1}^r}{\beta }_{1i}{\sigma }_{t-i}^2\right) \left(1-H\left({\varsigma }_{t-i};\epsilon ,n\right)\right)\\ +\left(w_2+{\displaystyle {\displaystyle \sum }_{i=1}^p}{\alpha }_{2i}{\varsigma }_{t-i}^2+{\displaystyle {\displaystyle \sum }_{i=1}^r}{\beta }_{2i}{\sigma }_{t-i}^2\right) H\left({\varsigma }_{t-1};\epsilon ,n\right)\end{array}$$

(4)

The transition function is:

$$H\left({\varsigma }_{t-i};\epsilon ,n\right)=\frac{1}{1+e^{-\epsilon \left({\varsigma }_{t-i}-n\right)}}$$

(5)

where n is the threshold coefficient and ${\epsilon }_t$ is the parameter defining the speed of the transition.

The Proposed Hybrid LSTARGARCHLSTM Model

LSTM is combined with GARCH methods (Figure 1) and LSTARGARCH (Figure 2) to gain assistance from the forecasting abilities of deep learning.

LSTM models have been proposed to exploit short- and long-term dependencies between the predicted time series data and the feature vectors extracted from it for performing the prediction. LSTM models were initially proposed in the seminal work of [68], which employed multiplicative gate units for maintaining a constant error flow through the constant error carousel unit to prevent the vanishing or blowing up of the error signals flowing backwards in time. A later variant of this fundamental architecture employed adaptive forget gates [69] to prevent the internal state from growing indefinitely by enabling the LSTM cell to learn to reset itself when its contents are outdated.

The equations governing the LSTM operation may be given as

$$m_t=f_t{{\displaystyle ⨀}}_{*}{m}_{t-1}+j_t{{\displaystyle ⨀}}_{*}{\tilde{m}}_t$$

(6)

$${\tilde{m}}_t=\mathit{tanh}\left(V_m{x}_t+Q_m{r}_{t-1}+a_m\right)$$

(7)

$$j_t=\sigma \left(V_j{x}_t+Q_j{r}_{t-1}+a_j\right)$$

(8)

$$f_t=\sigma \left(V_f{x}_t+Q_f{r}_{t-1}+a_f\right)$$

(9)

$$o_t=\sigma \left(V_o{x}_t+Q_o{r}_{t-1}+a_o\right)$$

(10)

$$r_t=o_t{{\displaystyle ⨀}}_{*}tanh\left(m_t\right)$$

(11)

where a is a bias vector, Q and V are weight matrices, the sigmoid function is denoted as σ(·), and ${{\displaystyle ⨀}}_{*}$ denotes element-wise multiplication.

In the above, LSTM has a memory cell that has three gates. $m_t$ is the cell state vector. The activation vectors of the three gates are found according to Equations (8)–(10). In these equations, the weighted sum of input vector $x_t$, the past output vector $r_{t-1}$ and the bias vector are passed through the sigmoid function to generate activation values in the range of 0 to 1. An activation value of one at each of these gates causes the information at the input to that gate to completely pass through, whereas a value of zero forces all the information at the input to be entirely filtered out. Hence, in Equation (6), the forget gate decides on the information in the most recent cell state ($m_{t-1}$) that passes to the current cell state. Similarly, in Equation (6), the input gate $j_t$ decides on the part of the new information to be stored in the current cell state ($m_t$). Equation (7) computes the new information ${\tilde{m}}_t$ at time t as a weighted sum of $x_t$, $r_{t-1}$, and the bias where the output of the tanh activation function ranges between −1 and 1. Finally, the normalised output value $r_t$ in Equation (11) is determined by filtering the tanh normalised (range −1 to 1) cell state $m_t$ by using the output gate $o_t$ activation determined in Equation (10). This process separates the necessary information from unnecessary information to yield the output vector $r_t$.

4. Data and Results

4.1. Data

We used Brent, Dubai and WTI crude oil price data in the current work. The volatilities of the Brent, Dubai and WTI crude oil prices are calculated by employing the daily closing prices between 29 May 2006 and 31 March 2020. lbop_t, ldop_t and lwop_t show the volatilities of the Brent, Dubai and WTI crude oil prices, respectively. They were calculated as lbop_t = $\ln ({\mathrm{brent}\mathrm{oil}\mathrm{price}}_{\mathrm{t}}/{\mathrm{brent}\mathrm{oil}\mathrm{price}}_{\mathrm{t}-1}$₎, ldop_t = $\ln ({\mathrm{Dubai}\mathrm{oil}\mathrm{price}}_{\mathrm{t}}/{\mathrm{Dubai}\mathrm{oil}\mathrm{price}}_{\mathrm{t}-1}$₎ and lwop_t = $\ln ({\mathrm{WTI}\mathrm{oil}\mathrm{price}}_{\mathrm{t}}/{\mathrm{WTI}\mathrm{oil}\mathrm{price}}_{\mathrm{t}-1}$).

In the process of the estimation of the GARCHLSTM and LSTARGARCHLSTM models, the sample is split between training, validation and out-of-sample elements in chronological order with respective proportions of 80%, 10% and 10%.

4.2. Results

The results were collected by the following five steps listed below:

• Firstly, some descriptive statistics were obtained. The Augmented Dickey-Fuller (ADF) unit root test [70,71] and Kapetanios, Shin, and Snell (KSS) unit root test [72] were applied. The ADF test is dependent upon a linear assumption that can cause the false results. Bigman et al. [73] showed that traditional unit root tests tends to produce “spurious regressions”. In this condition, for confirmation, we used the KSS test.
• Secondly, Tsay and Hsieh’s tests and the Brock–Dechert–Scheinkman (BDS) test were applied. These tests determined the presence of nonlinear structure, but they are not sufficient to determine the existence of chaotic behavior.
• Thirdly, SE and L_e tests were applied. L_e is a convenient means to decide on the presence of chaotic behavior.
• The LSTARGARCHLSTM method determines ARCH and GARCH effects. To evaluate the performance of our proposed method, we compared our proposed method with GARCHLSTM and traditional methods: GARCH and LSTARGARCH. For this purpose, GARCH and LSTARGARCH, and GARCHLSTM models were estimated and the most succesful model was determined.
• In the final step, the forecast accuracies of all of the models were determined.

4.2.1. Some Descriptive Statistics and Tsay and Hsieh’s Tests

In

Table 1. Descriptive statistics and unit root tests. lbop_t, ldop_t and lwop_t are the volatilities of the Brent, Dubai and WTI crude oil prices, respectively. ARCH shows the Engle’s [75] ARCH test statistic. White [76] shows White’s heteroscedasticity test statistic, RESET show the RESET statistic, and this test adds the second power of the fitted value as an additional regressor.

	Lbopt	Ldopt	Lwopt
Kurtosis	17.583	21.69	18.763
Skewness	−1.0417	−1.5635	−0.45
JB	346.64	282.65	208.5756
ARCH effect	17.89	27.67	19.25
White	14.36	13.73	10.88
RESET	13.58	10.72	2.18
Unit Root Tests
-	Level	Level	Level
ADF	−56.86	−15.069	−68.44
KSS	−54.25	−11.38	−53.58
Decision	I(0)	I(0)	I(0)

, several statistics of the data are shown. Since the data exhibit excess kurtosis, they cannot be modelled by a normal distribution. The normality assumption is rejected by the Jarque–Bera [74] (JB) normality test. The results show that nonnormality is observed and that normality is rejected.

On the other hand, in Table 1, the main problem appears to be excess kurtosis but not excess skewness.

The ADF and KSS unit root tests suggest the stationarity of the data at the level. The KSS test implies that the data are nonlinear and stationary.

4.2.2. BDS Test, Tsay Tests and Hsieh’s Coefficients Results

The BDS test (for detailed knowledge, see Brock et.al. [77]) was developed to test whether the data generating process of a series is deterministic (chaotic) or not [78]. The BDS test does not have a simple interpretation because the result of this test determines whether the stochastic process has chaos or nonlinearity [79].

The BDS test can be used to determine the presence of nonlinearity in the time series. Hsieh [80] calculate the following BDS test statistic: Let $C_m\left(e\right)$ be the fraction of the maximum norm of deviations of points in a time window of size m from points in a shifted time window of size m less than e. Under the null hypothesis of the time series $x_t$, and for fixed m and e, $C_m\left(e\right)\sim {\left[C_1\left(e\right)\right]}^m$, the BDS test statistic is the difference between $C_m\left(e\right)$ and ${\left[C_1\left(e\right)\right]}^m$ normalised by its standard deviation and asymptotically has the standard normal distribution. When the null hypothesis is rejected, nonlinear dynamics can be said to exist in the time series.The power of the test depends critically on the choice of e.

In

Table 2. Brock–Dechert–Scheinkman (BDS) test results.

Z Statistics
Dimension	Lbopt	Ldopt	Lwopt
2	34.40350	35.9270	18.999077
3	35.62271	38.6123	22.72316
4	38.02877	41.3846	24.84834
5	41.58379	44.5027	26.97560
6	46.53435	49.7965	28.99879

, the BDS test determined that the lbop_t, ldop_t and lwop_t variables indicate evidence of chaotic structure or nonlinear stochastic processes.

The Hsieh’s [81] coefficients are obtained in

Table 3. Tsay test and Hsieh’s coefficients. r_ij’s are Hsieh’s [81] third-order moment coefficients. Hsieh’s [81] third-order moment coefficients were obtained as $\left[\sum x_t{x}_{t-i}{x}_{t-j}/T\right]/{\left[\sum x_t^2/T\right]}^{1.5}$. Only coefficients for r(1, 1) and r(1, 2) were given.

Hsieh’s Coefficients						Tsay’s Nonlinearity Test Statistic
rij are Hsich’s Third-Order Moment Coefficients for Lags i and j						Tsay’s Nonlinearity Test Statistic
Lbopt		Ldopt		Lwopt		Lbopt	Ldopt	Lwopt
r(1)	r(2)	r(1)	r(2)	r(1)	r(2)	133.41	100.58	102.001
0.1	−0.42	−0.35	0.12	−0.124	0.45	-	-	-

. Some coefficients are high. Although the Hsieh’s coefficients are not aimed at testing models that are nonlinear in variance, the high coefficient values indicate some autoregressive conditional heteroscedastic (ARCH) effects and other kinds of nonlinearities (such as GARCH-in-Mean effects).

Similarly, as shown in Table 3, the Tsay nonlinearity test determined that the linear form is misspecified for the op (oil price) variables. The RESET and Tsay nonlinearity test statistics do suggest that the (linear) functional form is misspecified for the variables.

4.2.3. Lyapunov Exponent and Kolmogorov Entropy Tests

The chaotic level of the time series can be measured with L_e. L_e is used to measure the average divergence from or convergence to the initial point of a dynamical system. If the L_e has a large value, it indicates high sensivity to initial conditions. While a positive Lyapunov coefficient typically signifies chaotic structure, a negative coefficient typically shows convergence to initial conditions [21,82]. The positive value of L_e indicates the presence of chaotic structure in the oil prices.

Table 4. Lyapunov exponent and Shannon entropy results ¹.

Lyapunov Exponent Method			Shannon Entropy Method
Lbopt	Ldopt	Lwopt	Lbopt	Ldopt	Lwopt
0.9504	0.9071	0.8481	0.9617	0.983	0.9121

¹ Shannon entropy(SE) is defined as SE(x) = $-\sum _{i=1}^n{p}_i\log \left(p_i\right)$ where $p$_i is the probability mass of the ith discrete level such that $\sum _i{p}_i=1$ [83]. The R software DChaos package for calculating ${\mathrm{L}}_{\mathrm{e}}$ and R Software Entropy package for calculating SE are employed.

shows the results. Additionally, Adrangi and Chatrath [20], Bildirici and Sonustun [82], and Lahmiri [21] also determined the existence of chaotic structure for oil prices.

SE is applied to measure the randomness degree in the data. The SE is 0.9121 for lwop_t. Entropy may be understood as the degree of the distortion of the market information reflected in the price system. Positive values of the entropy accent that the oil price information can still be used to understand the oil market dynamics. For example, since the reciprocal of the entropy for Brent is $1/0.9617$, the corresponding time scale of an effective and rational forecast for that entropy value must be within ~11 days. He [23] found 36 days by using the Kolmogorov entropy, and Bildirici [84] found 68 days.

4.2.4. Results with the GARCH, LSTARGARCH, LSTARGARCHLSTM and GARCHLSTM Models

Firstly, the results with the basic GARCH model and LSTARGARCH model are presented in

Table 5. Baseline models. LogL: Log-likelihood, ARCH(p): pth order ARCH–LM test, AIC: Akaike information criterion, SIC: Schwarz information criterion, HQ: Hannan–Quinn information criterion.

	Lwopt			Lbopt			Ldopt
-	GARCH	LSTARGARCH		GARCH	LSTARGARCH		GARCH	LSTARGARCH
-		Regime 1	Regime 2		Regime 1	Regime 2		Regime 1	Regime 2
Cst(M)	0.0214 (2.13) (0.0) 1	00633 (3428) (00) 1	0116 (576) (00) 1	00215 (187) (00) 1	0015 (456) (00) 1	000520 (625) (00) 1	0.0754 (1.94) (0.0) 1	0.0004 (2.17) (0.0) 1	0.05377 (7.61) (0.0) 1
Cst(V)	0.255 (1.91) (0.0) 1	1109 (488) (00) 1	0287 (836) (00) 1	0312 (193) (00) 1	0338 (263) (00) 1	0089 (558) (00) 1	0.02905 (2.05) (0.0) 1	0.178 (1.93) (0.0) 1	0.205 (1.94) (0.0) 1
ARCH	0.19 (3.61) (0.0) 1	0128 (8689) (00) 1	0089 (1427) (00) 1	0189 (278) (00) 1	00287 (385) (00) 1	01052 (1246) (00) 1	0.19899 (2.67) (0.0) 1	0.201 (7.25) (0.0) 1	0.112 (2.27) (0.0) 1
GARCH	0.67 (4.78) (0.0) 1	0722 (686) (00) 1	0903 (516) (00) 1	0611 (1887) (00) 1	09401 (8136) (00) 1	08795 (1056) (00) 1	0.61922 (4.051) (0.0) 1	0.769 (2.105) (0.0) 1	0.872 (9.26) (0.0) 1
LogL	10536.28	271389		118421	316852		112345	388873
AIC:	10.353	−3913		77224	−33126		84167	−44956
SIC:	10.042	−389		77154	−32997		84869	−44836
HQ:	10.054	−385		77199	−33078		83940	−44612
ARCH (1–2):	0.035	0097		0056	0042		0083	0064
ARCH (1–5):	0.039	0095		0058	0041		0078	0062

¹ shows p–values.

. Statistical inference for assessing the empirical validity of the two regime switching process was carried out by using nonstandard Likelihood Ratio (LR) test [85]. However, these tests have some problems in the context of the LSTARGARCH method, since LSTARGARCH models support LSTAR-type nonlinearity in the conditional mean with GARCH-type heteroscedasticity in the conditional variance. We continue with LL test. There are two results: (1) the forms of the LL functions are obtained by the choice of the transition functions and (2) in the models, it can be possible to change the forms of the LL functions by transforming the parameters.

Table 5 gives the results determined by traditional methods. LSTARGARCH models were employed to analyse the non–linearity in the conditional variance and in the conditional mean. When the two groups of results with the GARCH and LSTARGARCH models in Table 5 are compared, several conclusions can be derived. In the GARCH model, the sum of the ARCH and GARCH coefficients is in the range 0.8–0.9. The the sum of the coefficients of ARCH and GARCH is <1 but close to 1 for the LSTARGARCH method.

The results of LSTARGARCH model show significant improvements over the GARCH model. The Log-likelihood values are considerably smaller for LSTARGARCH method, and the Schwarz information criterion (SIC) and Akaike information criterion (AIC) are significantly lower. As shown by the results in Table 5, when the variance is not constant, the LSTARGARCH model can provide improvements in modelling.

4.2.5. The Architecture of the GARCH–LSTM and LSTARGARCH–LSTM Models

We will give the architecture of models with LSTM in

Table 6. Generalised Autoregressive Conditional Heteroskedasticity long-short term memory (GARCHLSTM) and Logistic Smooth Transition Autoregressive (LSTAR) GARCHLSTM.

	Lbopt		Ldopt		Lwopt
	GARCHLSTM	LSTARGARCHLSTM	GARCHLSTM	LSTARGARCHLSTM	GARCHLSTM	LSTARGARCHLSTM
Training rho 1	0.89	0.92	0.88	0.93	0.91	0.95
Test rho	0.88	0.90	0.87	0.92	0.90	0.91
Training RMSE	0.24	0.04	0.21	0.08	0.33	0.03
Training MAE	0.23	0.03	0.20	0.07	0.31	0.03
Test RMSE	0.22	0.022	0.1	0.07	0.29	0.06
Test MAE	0.21	0.022	0.09	0.06	0.289	0.059

¹ Rho represents the training and test sample correlation coefficient, MAE and RMSE are the Mean Absolute Error and the Root Mean Squared Error, respectively.

. LSTM has been implemented in the Keras framework. The LSTM model is estimated with optimisation conducted simultaneously in the training and test samples. The optimisation is stopped at the epoch in which the test sample Mean Absolute Error (MAE) starts to increase while the training sample MAE continues to decrease (early stopping). The weight decay is also applied in the output and hidden layers to reduce overfitting. The LSTM network has been trained for 50 epochs by using approximately 80% of the data. Several models employing different numbers of layers of LSTM in a stack architecture have been evaluated by using the MAE measure on the validation error. The best model with the lowest MAE consisted of three LSTM layers (excluding the fully connected output layer). The selected methods are utilised for out-of-sample forecasting.

The determined models are used to forecast out-of-sample.

In our model, we use a stack of three LSTM layers followed by a fully connected layer at the output. A choice for the window size of 32 limits the support for the learning to the most recent 32 samples. The network is trained using an optimisation algorithm that adapts the parameter learning rate to both the first and second moments of the gradients to minimise the loss function. It can be shown as in Figure 1 and Figure 2.

4.2.6. The Results of the GARCHLSTM and LSTARGARCHLSTM Method

For comparative purposes, the results obtained by GARCHLSTM and our proposed methods are reported in

Table 7. GARCHLSTM and LSTARGARCHLSTM. LogL: Log-likelihood, ARCH(p): pth order ARCH–LM test.

	Lwopt			Lbopt			Ldopt
-	GARCH LSTM	LSTARGARCHLSTM		GARCH LSTM	LSTARGARCHLSTM		GARCH LSTM	LSTARGARCHLSTM
-	-	Regime 1	Regime 2	-	Regime 1	Regime 2	-	Regime 1	Regime 2
Cst(M)	00618 (256) (00) 1	0986 (212) (00) 1	0651 (474) (00) 1	025 (281) (00) 1	0156 (276) (00) 1	0554 (288) (00) 1	0173 (262) (00) 1	0263 (281) (00) 1	0361 (453) (00) 1
Cst(V)	0985 (265) (00) 1	0431 (226) (00) 1	0562 (382) (00) 1	0861 (288) (00) 1	0297 (263) (00) 1	0441 (376) (00) 1	0565 (288) (00) 1	0428 (287) (00) 1	0397 (432) (00) 1
ARCH	0207 (316) (00) 1	0102 (977) (00) 1	0023 (229) (00) 1	0127 (458) (00) 1	0111 (803) (00) 1	0095 (297) (00) 1	0198 (448) (00) 1	0118 (675) (00) 1	0037 (236) (00) 1
GARCH	0721 (571) (00) 1	0881 (558) (00) 1	0962 (356) (00) 1	0811 (631) (00) 1	0878 (558) (00) 1	0901 (356) (00) 1	0781 (756) (00) 1	0844 (287) (00) 1	0942 (522) (00) 1
LogL	2849.2	2038.21		2669.3	1984.18		2986.2	1989.75
AIC:	2.981	−1.413		2.661	−1.513		2.875	−1.897
SIC:	2.816	−1.391		2.514	−1.489		2.867	−1.791
HQ:	2.807	−1.388		2.507	−1.417		2.821	−1.745
ARCH (1–2):	0.123	0.089		0.107	0.076		0.109	0.081
ARCH (1–5):	0.124	0.090		0.114	0.071		0.112	0.082

¹ shows p–values.

. Our proposed method shows improvements over the LSTARGARCH model. The Log-likelihood value is decreased, and the SIC and AIC of our proposed method are lower than those of the LSTARGARCH model. Similarly, the results show improvements of the GARCHLSTM model in the GARCH models. The Log-likelihood values are decreased, and the SIC and AIC values are lower.

For the GARCHLSTM method and our proposed method, the stability conditions were realised. The Hannan–Quinn information criterion (HQ), SIC and AIC are are lower than for the GARCHLSTM. Moreover, after permitting the GARCH methods to follow LSTAR non–linearity, the dynamics are outstandingly different in terms of the estimated parameters.

5. Forecast Results

The forecast results are given under two subtitles: In–sample Forecast Results and Out of Sample Forecast Results.

5.1. In–Sample Forecast Results

In the in-sample forecast, the predictive capabilities of the models developed using observed data are formally evaluated to find out the effectiveness of the algorithms in reproducing data. The results show that our proposed method offers important advances in in-sample forecast accuracy (

Table 8. In-step forecast results.

	-	GARCH	GARCHLSTM	LSTARGARCH	LSTARGARCHLSTM
lbopt	RMSE	0.995	0.088	0.04	0.001
	MAE	0.84	0.072	0.027	0.0009
ldopt	RMSE	0.937	0.097	0.029	0.005
	MAE	0.79	0.079	0.014	0.0039
lwopt	RMSE	0.49	0.034	0.0291	0.006
	MAE	0.35	0.022	0.0216	0.0055

).

In Table 7, forecasting performance results are presented. When the models are appraised in terms of the RMSE criterion, our proposed method has the highest forecasting power followed by the LSTARGARCH method. In terms of the RMSE criterion, the our proposed method has the best generalisation capability followed by the LSTARGARCH method.

5.2. Out-of-Sample Forecast Results

The models are obtained to explore their 1, 10 and 20 day-ahead forecast accuracies, shown in

Table 9. Out of sample forecast results 1, 10 and 20 days ahead.

Lbopt
GARCH				LSTARGARCH
-	T + 1	T + 10	T + 20	T + 1	T + 10	T + 20
RMSE	0.5126	0.538	0.547	0.0107	0.0213	0.038
MAE	0.5028	0.536	0.51	0.0106	0.0209	0.0375
GARCHLSTM				LSTARGARCHLSTM
RMSE	0.052	0.0459	0.01438	0.0031	0.0038	0.0051
MAE	0.049	0.0448	0.01399	0.0029	0.0036	0.0049
ldopt
GARCH				LSTARGARCH
RMSE	0.5187	0.4896	0.626	0.01125	0.0308	0.0397
MAE	0.5098	0.4891	0.621	0.01117	0.0299	0.0394
GARCHLSTM				LSTARGARCHLSTM
RMSE	0.059	0.051	0.0495	0.005	0.006	0.0068
MAE	0.058	0.0501	0.0471	0.0038	0.0043	0.0052
lwopt
GARCH				LSTARGARCH
RMSE	0.4472	0.4526	0.5066	0.0131	0.0313	0.034
MAE	0.4463	0.4511	0.5012	0.0122	0.0310	0.032
GARCHLSTM				LSTARGARCHLSTM
RMSE	0.041	0.039	0.0385	0.0022	0.0024	0.0025
MAE	0.040	0.037	0.0381	0.0021	0.0022	0.0024

(denoted as T + 1, T + 10 and T + 20). Accordingly, the out-of-sample 1, 10 and 20 days ahead forecasts with our proposed methods exhibited the lowest RMSE and MAE followed by those from the LSTARGARCH, GARCHLSTM and GARCH models.

For example, for the WTI data, there is an order of magnitude improvement in the 10 and 20 days ahead forecast RMSE’s with the LSTARGARCH models when the models are augmented with the LSTM architecture. The RMSEs of our proposed methods for the WTI data in 1, 10 and 20 days are calculated as 0.0021, 0.0022 and 0.0025.

5.3. To Test for Forecast Accuracy

The Wilcoxon signed-rank (WS) and Diebold–Mariano (DM) tests were applied to confirm the equivalence of forecast accuracy [86]. The DM test is as follows:

$$DM=\frac{\overline{d}}{\sqrt{2\pi {\widehat{\mathfrak{f}}}_d}\left(0\right)/T}$$

(12)

where $\overline{d}=\frac{1}{T}\sum _{t=1}^T(g\left(u_{1,t}\right)-g\left(u_{2,t}\right))$ and ${\widehat{\mathfrak{f}}}_d\left(0\right)$ is a consistent estimate of ${\mathfrak{f}}_d\left(0\right)$.

The results of these tests are presented in

Table 10. Wilcoxon signed-rank and Diebold–Mariano results.

			WS TEST
			RMSEGARCH	RMSEGARCHLSTM	RMSELSTARGARCH	RMSELSTARGARCHLSTM
DM Test		RMSEGARCH	-	0.00	0.00	0.00
	lbopt	RMSEGARCHLSTM	0.00	-	0.00	0.00
		RMSELSTARGARCH	0.00	0.00	-	0.00
		RMSELSTARGARCHLSTM	0.00	0.00	0.00	-
		RMSEGARCH	-	0.00	0.00	0.00
	ldopt	RMSEGARCHLSTM	0.00	-	0.00	0.00
		RMSELSTARGARCH	0.00	0.00	-	0.00
		RMSELSTARGARCHLSTM	0.00	0.00	0.00	-
		RMSEGARCH	-	0.00	0.00	0.00
	lwopt	RMSEGARCHLSTM	0.00	-	0.00	0.00
		RMSELSTARGARCH	0.00	0.00	-	0.00
		RMSELSTARGARCHLSTM	0.00	0.00	0.00	-

. The p-values for the DM test are given below the diagonal, and the p-values of the WS test are given above the diagonal. The H₀ hypothesis of these tests assumes that the models have the same level of accuracy. For most cases, since the p-value is <0.05, the H₀ hypothesis is rejected. For both tests, the p-value is >0.05 only for the RMSE comparison of the GARCH and GARCHLSTM models. Hence, these two models are comparable in terms of RMSE performance.

6. Conclusions

This paper suggested a hybrid model for analysing oil price volatility between 29 May 2006 and 31 March 2020 on WTI, Brent and Dubai crude oil price datasets. Since under the influence of the COVID-19 pandemic and oil conflict between Russia and Saudi Arabia, the oil price has exhibited unusual changes, under the presence of chaotic structure of the oil price, analysis cannot be performed by traditional methods. The GARCH and LSTARGARCH methods and their extensions GARCHLSTM and LSTARGARCHLSTM based on deep learning were comparatively evaluated. The GARCH model may not achieve high forecasting performance under the presence of nonlinear or chaotic movement, since it considers the data period as a whole. Among these models, the LSTARGARCHLSTM model exhibited significantly better results in terms of out-of-sample accuracy than the others in the face of the chaotic structure of the oil price.

The LSTARGARCHLSTM model presents important improvements for time series estimation by dividing the data into regimes, and analysing each regime within itself with the effects of ARCH and GARCH. Specifically, the different GARCH and ARCH coefficients obtained in each of the negative and positive shocks show different ARCH and GARCH effects in each of the regimes and an obligation to set different policies in each of the regimes. According to our results, the impacts and the volatility of the oil shocks associated with COVID-19 must be evaluated by our proposed method.

According to the ARCH and GARCH coefficients of the LSTARGARCHLSTM methods, the oil price has ARCH and GARCH effects. That is, the oil supply and demand shocks associated with COVID-19 are likely to be short–lived, but their effects on many sectors and countries are persistent.

Oil demand and supply will get back to their pre-pandemic levels at a speed based on the length and depth of the disruption over time. Governments must develop policies to eliminate the adverse consequences of the supply and demand shocks caused by COVID-19. On the other hand, the COVID-19 pandemic will cause different results in oil-importing and -exporting countries. For oil-importing countries, the persistence of oil prices is important for determing policies for targeting inflation. Under the pressure on the prices of precious metals like gold and palladium, they can encounter problems related to policies of targeting inflation. The expectations have adverse effect and can produce unexpected results.

For oil-exporting countries, it is essential for Russia and OPEC countries to conduct policies to make the oil prices return to the levels before COVID-19, since oil revenues constitute a significant portion of the gross domestic products of these countries.

As accented by [2,3], in some oil-exporting and -importing countries, government coffers make the fight in containing the spread of COVID-19 and its economic and social impacts more difficult. Financial gaps in MENA countries as well as the high sovereign-risk premiums of many countries in the world present difficulties for additional foreign borrowing on private markets. For oil-importing and -exporting countries, helicopter money can be used as a recommended policy. However, fixed exchange rates in some countries may make it difficult for them to use helicopter money, since money printing is at odds with maintenance of the peg. As a result, in our opinion, if the oil prices do not return to the levels before the pandemic, oil-exporting countries will endure heavy losses.