Machine Learning and Algorithmic Pairs Trading in Futures Markets

Abstract

This study applies machine learning methods to develop a sustainable pairs trading market-neutral investment strategy across multiple futures markets. Cointegrated pairs with similar price trends are identified, and a hedge ratio is determined using an Error Correction Model (ECM) framework and support vector machine algorithm based upon the two-step Engle–Granger method. The study shows that normal backwardation and contango do not consistently characterize futures markets, and an algorithmic pairs trading strategy is effective, given the unique predominant price trends of each futures market. Across multiple futures markets, the pairs trading strategy results in larger risk-adjusted returns and lower exposure to market risk, relative to an appropriate benchmark. Backtesting is employed and results show that the pairs trading strategy may hedge against unexpected negative systemic events, specifically the COVID-19 pandemic, remaining profitable over the period examined.

1. Introduction

The classic rationale for futures prices is that the current futures contract price reflects the market consensus of the spot price at maturity. Since investors’ demand is conveyed in futures markets, the changes in futures prices and spot prices are strongly correlated [1,2,3,4]. Further, Tilton et al. [2] suggest that spot and futures prices exhibit a high correlation in strong contango, while they are less related during periods of backwardation. It is well documented that returns to futures investors are highly associated with market conditions [5,6,7]. In futures markets, there are two important market conditions: contango and backwardation, which are terms used to explain the structure of the forward curve. A futures market in a contango condition exhibits a futures contract forward price that is higher than the spot price. In contrast, a futures market in a backwardation condition exhibits a futures contract forward price below the spot price.

Various papers have studied investor’s ability to forecast futures prices [6,7,8], but the evidence is inconclusive. The absence of consistent results regarding forecasting ability may be due to a focus on certain futures markets over relatively short intervals, the impact of changes in market conditions, and the heterogeneous assumption that the rate of returns on futures contracts is normally distributed. In order to provide a more rigorous output, we hypothesize that hypothetical naïve speculators do not demonstrate persistent performance in futures markets using a nonparametric method, as suggested by Henriksson and Merton [9], and Chang [1]. Since the hypothetical speculators do not command any forecasting ability, a rejection of the hypothesis means speculators make positive gains from expected risk premiums. Conditional probabilities result in a rejection of the hypothesis, evidence of the effectiveness of speculator forecasting ability of a price trend. This suggests that investors can exploit a price trend over time in both contango and backwardation market conditions.

Previously, studies have identified that trend-following strategies (or momentum strategies) create significant abnormal returns in futures markets [10,11,12,13,14]. For example, Miffre and Rallis [12] show that trend-following strategies in commodity futures produce sizable returns after transaction costs. Clare et al. [15] identify that trend-following strategies offer appealing risk-adjusted returns. Hurst et al. [14] find that time-series momentum was achieved even during past crisis periods. Previous studies have had a common view on the performance of trend-following strategies—the outperformance of the strategy is not attributed to cross-sectional differences in expected returns across markets, but due to time-series dependence in realized returns [13].

However, there is very little research that has applied machine learning methods to futures investments, as Creamer [15] summarizes. Few studies employ Neural Network [16], Genetic Algorithm [17,18], Support Vector Machine [19], or Reinforcement Learning [20]. Most previous studies limited the application of these methods to index futures trading using technical indicators. Moreover, trend following strategies typically see reversals when futures contracts that previously signaled an upward trend unexpectedly reverse (e.g., COVID-19 market crash). To mitigate this risk, we use a pairs trading market neutral strategy, which involves long and short positions in two different futures contracts with similar time-series price trends. As Rad et al. [21] suggest, the presence of a cointegration relation between two securities indicates a long-term relationship between the two; helping investors accurately implement a pairs trading strategy. Thus, we choose cointegrated pairs that have a similar predominant price trend (e.g., contango and backwardation) and use Support Vector Machine (SVM) to determine a hedge ratio in an Error Correction Model (ECM) framework based upon the two-step Engle–Granger method [22].

From the empirical analysis of the U.S. futures markets, we find that normal backwardation and forecasting theory may not be mutually exclusive, although they can be contrasting. This study shows that normal backwardation and contango do not consistently characterize futures markets, but each futures market presents unique predominant price trends. This finding supports our approach of constructing pairs of futures contracts in the same futures markets as appropriate in developing algorithmic pairs trading strategies. We find that our pairs trading strategy makes notable risk-adjusted returns in various futures markets. Interestingly, our backtesting results show that our strategies can hedge against an unexpected decline in futures prices and importantly remain profitable through the COVID-19 pandemic.

The remainder of the paper is organized as follows. Section 2 presents the development of a measure of futures market conditions, details the model architecture, and a description of the methodology. Results and conclusions are presented in Section 3 and Section 4, respectively.

2. Methodology

2.1. Measurement for Backwardation and Contango

Normal backwardation includes the following assumptions regarding speculators: (i) they take net long positions, (ii) want positive profits, and (iii) do not have the skill to forecast prices. The implication of normal backwardation is that, on average, futures prices rise approaching maturity. Conversely, under the contango market condition, hedgers are net short, and the futures price lies above the expected future spot price; the price of the futures contract falls as the contract approaches maturity.

Assuming speculators are not equipped to forecast prices, the profits they generate can be categorized as a reward for bearing risk rather than for accurate forecasting. To examine this, we define the values and returns of long and short trading positions following Rockwell [23]. Let $V_m^L$ and $V_m^S$ be the total value of the trading group’s long and short positions, in a specific market m, aggregated over the selected period. Let $R_m$ be the rate of return on the long open position in that market. The net return of a trading group credited to basic forecasting skill is defined as in Equation (1).

$$R_m^B=\frac{R_m\left(V_m^L-V_m^S\right)}{V_m^L+V_m^S}$$

(1)

$R_m^A$ denotes the actual return for the group; the measure of the special forecasting skill is then obtained, defined as the residual.

$$R_m^F=R_m^A-R_m^B.$$

(2)

Hence, the aggregate return on any set of markets is defined as

$$R^B=\frac{{\sum }_m{R}_m\left(V_m^L-V_m^S\right)}{{\sum }_m{V}_m^L+V_m^S}$$

(3)

and its corresponding forecasting skill is computed by

$$R^F=R^A-R^B$$

(4)

When, on average, the group takes a net long position ($V_m^L-V_m^S>0$), and $R_m$ is positive, the $R_m^B$ measure will be positive. Conversely, the group is roughly net short ($V_m^L-V_m^S<0$) when $R_m$ is negative. From this relationship, we can define the basic skill as (i) the capacity to take a net long position in markets with rising prices, and (ii) the capacity to take a net short position in markets with falling prices.

The definition of the basic skill reflects the long-run skills of a trading group to generate profits in the futures market. Furthermore, forecasting skill reflects the facility of a trading group to alter its position over time in order to take advantage of short-run price trends. In markets where no strong evidence of backwardation or contango appears, forecasting skills will be crucial to make positive returns. For example, both contango and backwardation reflect the term structure of futures prices. Backwardation refers to prices that decline with maturity (downward sloping term structure), and contango is the opposite, with an upward slope. The future spot price is expected to remain similar to the current value so that in backwardation, low futures prices for distant maturities are expected to rise over time. This would provide an expected profit to the speculators who, in aggregate, took on the price risk from hedgers who were holding inventories in the spot market and selling futures to protect against a price drop. A similar argument when hedgers are net-long leads to a contango pattern and expected profits to speculators who are net short.

By analyzing the number of price increase periods over the total number periods (i.e., PIP/TP), we are able to understand price behaviors in futures markets. In terms of PIP/TP, we classify four patterns of futures prices over time: (i) normal backwardation (i.e., prices in the futures markets rise) if PIP/TP > 51%, (ii) contango (i.e., prices in the futures markets decline) if PIT/TP < 49%, (iii) weak backwardation if PIP/TP is between 50% and 51%, and (iv) weak contango if PIP/TP is between 49% and 50%.

2.2. Nonparametric Statistical Procedure

To evaluate forecasting ability, we use a nonparametric statistical procedure, as suggested by Henriksson and Merton [9] and Chang [1]. Originally, Henriksson and Merton [9] present this procedure to identify market-timing forecasting skills. As the test suggested by Henriksson and Merton [9] evaluates forecasting skills between any two assets, even though their test procedure is of market timing, Chang [1] altered their procedure to fit the test of forecasting ability. Following their procedure, we describe (i) the rational behavior of speculators in futures markets is to take a long position when futures contract prices are below prices projected for the point of liquidation; (ii) they will take a short position in futures contracts priced above those projected for the offset point. Let F(t) represent the futures contracts price at t, and R(t) represent the change in futures prices during the period t, given as

$$R\left(t\right)\equiv F\left(t\right)-F\left(t-1\right)$$

(5)

Thus, speculators forecast either R(t) > 0 or R(t) ≤ 0 before they liquidate or change their investment positions. γ(t) represents the speculator’s forecast variable. We consider two forecasting events. The first event is γ(t) = 1, indicating that the forecast at t–1 for period t is R(t) > 0. The second event is γ(t) = 0, indicating that the forecast at t − 1 is R(t) ≤ 0. Following Chang, we suggest conditional probabilities for γ(t) as in Equations (6) and (7).

$${\mathrm{P}}_1^{\mathrm{s}}\left(t\right)\equiv prob[\gamma \left(t\right)=0|R\left(t\right)\le 0]$$

(6)

$${\mathrm{P}}_2^{\mathrm{s}}\left(t\right)\equiv prob[\gamma \left(t\right)=1|R\left(t\right)>0]$$

(7)

$P_1^s\left(t\right)$ represents the conditional probability of a correct forecast at t − 1 given that R(t) ≤ 0. ${\mathrm{P}}_2^{\mathrm{s}}\left(t\right)$ represents the probability of an accurate forecast at t − 1 given that R(t) > 0.

When an investor incorrectly predicts the futures market, it implies there is zero value in forecasting. Our research hypothesis suggests that speculators do not have positive gains. ${\mathrm{H}}_0:P_1^S\left(t\right)+P_2^{\mathrm{S}}\left(t\right)=1$, $P_1^s\left(t\right)$ and $P_2^s\left(t\right)$ are unknown. Henriksson and Merton [9] and Chang [1] show this null hypothesis is defined and tested by the hypergeometric distribution.

Utilizing the hypergeometric distribution does not require the estimation of conditional probabilities. Distribution is independent for $P_1^s\left(t\right)$ and $P_2^s\left(t\right)$ and forecasts are known [1]. For large samples, the hypergeometric distribution is approximately equal to the normal distribution under the central limit theorem. We use parameters in the hypergeometric distribution for this approximation, as in Equations (8) and (9).

$$E\left(n_1\right)=\frac{n{N}_1}{N}$$

(8)

$${\sigma }^2\left(n_1\right)=\frac{n{N}_1\left(N-N1\right)\left(N-n\right)}{N^2\left(N-1\right)}$$

(9)

2.3. Cointegration and Pairs Trading

The pairs trading strategy is developed based on Geometric Brownian Motion (GMB), S(t), which is given by:

$$S\left(t\right)=S_0{e}^{X\left(t\right)}$$

(10)

where $X\left(t\right)=\mu t+\sigma B\left(t\right)$ is the Brownian Motion (BM) with drift and $S\left(0\right)=S_0>0$ is the inception value. Taking the logarithm provides:

$$\ln \left(S\left(t\right)\right)-\ln \left(S_0\right)=X\left(t\right)\hspace{1em}\hspace{1em}X\left(t\right)~N\left(\mu \mathsf{\Delta }t,\sigma \sqrt{\mathsf{\Delta }t}\right)$$

(11)

where $S\left(t\right)$ is log-normally distributed for each t, $\mathsf{\Delta }\mathrm{S}\left(\mathrm{t}\right)=\ln \left(S\left(t\right)\right)-\ln (S\left(0\right)$), and ${\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }S\left(t\right)\right)=\mu$. Consider $S_1\left(t\right)$ and $S_2\left(t\right)$, which are non-stationary I(1) variables, we can expect any form of a linear combination of $S_1\left(t\right)$ and $S_2\left(t\right)$ to be I(1) as well. However, we can conclude that $S_1\left(t\right)$ and $S_2\left(t\right)$ are cointegrated and have similar stochastic trends if the following linear form is a stationary I(0) process:

$$S_2\left(t\right)-\beta S_1\left(t\right)={\nu }_t$$

(12)

where ${\mathsf{\nu }}_{\mathrm{t}}$ is a stationary residual, or spread, and $\beta$ is the cointegration efficient. As Rad et al. [21] summarize, the Error Correction Model describes the cointegration relationship between two time series variable $S_1\left(t\right)$ and $S_2\left(t\right)$ as in the following forms:

$$S_2\left(t\right)-S_2\left(t-1\right)=\gamma \left[S_2\left(t-1\right)-\beta S_1\left(t-1\right)\right]+\epsilon \left(t\right)$$

(13)

where $\epsilon \left(t\right)$ is white noise, $\gamma$ represents a rate of correction, $S_2\left(t-1\right)-\beta S_1\left(t-1\right)$ is the cointegration term, and $\mathsf{\gamma }\left[S_2\left(t-1\right)-\beta S_1\left(t-1\right)\right]$ represents an error correction term associated with the mean-reversion property of cointegrated time series. A cointegrated time series demonstrates long-term equilibrium. The error correction term rectifies any short term deviations from equilibrium that may occur in some cases. Hence, the change of spread between time t and t − 1 is given by

$$\mathsf{\Delta }{\nu }_t={\nu }_t-{\nu }_{t-1}=\mathsf{\Delta }{S}_2\left(t\right)-\beta \Delta S_1\left(t\right)$$

(14)

Let denote ${\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }{S}_1\left(t\right)\right)={\mu }_1$ as the expected value of $\mathsf{\Delta }{S}_1\left(t\right)$, and ${\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }{S}_2\left(t\right)\right)={\mu }_2$ as the expected value of $\mathsf{\Delta }{S}_2\left(t\right)$. Then, the expected value of the spread, $\mathsf{\Delta }{\nu }_t$, is specified as

$${\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }{\nu }_t\right)={\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }{S}_2\left(t\right)\right)-{\mathrm{E}}_{\mathrm{t}}\left(\beta \Delta S_1\left(t\right)\right)={\mu }_2-\beta {\mu }_1$$

(15)

Setting ${\mu }_2-\beta {\mu }_1=0$ gives $\beta =\frac{{\mu }_2}{{\mu }_1}$. Replacing $\beta$ to $\frac{{\mu }_2}{{\mu }_1}$ yields the following form:

$${\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }{\nu }_t\right)={\mathrm{E}}_{\mathrm{t}}\left(\mathsf{\Delta }{S}_2\left(t\right)\right)-{\mathrm{E}}_{\mathrm{t}}\left(\beta \Delta S_1\left(t\right)\right)=0$$

(16)

The change of the spread for the time period, $\mathsf{\Delta }\mathrm{t}$, is associated with the profit of buying $S_2\left(t\right)$ and selling the quantity of β for that period.

A two-step Engle–Granger method is utilized to identify the presence of any cointegration between pairs. We first estimate the cointegration coefficient of β, and we estimate the ECM in the next step. In the first step, we use the SVM regression method, which is a machine learning tool suggested by Vapnik [24]. SVM regression is considered a nonparametric method in that it depends on kernel functions (e.g., polynomial kernel, gaussian kernel, radial basis kernel, sigmoid kernels, etc.). As Cao and Tay [25] summarize, given a set of data points $(x_1,y_1),(x_1,y_1),\dots ,(x_N,y_N)$, which are randomly and independently generated, where $x_i\in X\subseteq {\mathit{R}}^n$, $y_i\in Y\subseteq \mathit{R}$, and N is the total number of the training sample, SVM approximates the linear function using the form:

$$f\left(x\right)=x\prime \cdot \beta +b$$

(17)

where $x$ represents the high-dimensional feature spaces. We find $\beta$ that minimizes the following optimal function

$$\begin{array}{c}J\left(\beta \right)=\frac{1}{2}{\beta }^{\prime }\beta +C{\displaystyle \sum _{i=1}^N}\left({\xi }_i+{\xi }_i^{*}\right)\hfill \\ L_{\epsilon }\left(y_i,f\left(x_i\right)\right)=\{\begin{array}{c}\left|y_i-f\left(x_i\right)\right|-\epsilon \\ 0,\end{array},\hspace{1em}\begin{array}{c}\left|y_i-f\left(x_i\right)\right|\ge \epsilon \\ otherwise\end{array}\hfill \end{array}$$

(18)

subject to

$$\begin{array}{c}{y}_i-\left(x_i^{\prime }\beta +b\right)\le \epsilon +{\xi }_i\hfill \\ \left(x_i^{\prime }\beta +b\right)-y_i\le \epsilon +{\xi }_i^{*},\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}i=1,\dots ,N\hfill \\ {\xi }_i^{*}\ge 0\hfill \\ {\xi }_i\ge 0\hfill \end{array}$$

(19)

where $L_{\epsilon }$ represents the linear $\epsilon$ the insensitive loss function, $x_i\prime$ represents a vector of N input variables $x$ with observed response values $y_i$, ${\xi }_i$ and ${\xi }_i^{*}$ are slack variables for each point, C is constant, and $\epsilon$ is the tube size. By using Lagrange multipliers, we reach the following form:

$$\begin{array}{c}{\displaystyle \sum _{i=1}^N}\left(a_i-a_i^{*}\right)=0\hfill \\ 0\le a_i^{*}\le C,\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}i=1,\dots ,N\hfill \\ 0\le a_i\le C\hfill \end{array}$$

(20)

where $a_i$ and $a_i^{*}$ are non-negative Lagrange multipliers. The β parameter can be obtained using a linear combination of the variables, given by

$$\beta ={\sum }_{i=1}^N\left(a_i-a_i^{*}\right)\left(x_i^{\prime }x\right)+b$$

(21)

Employing the mean reverting properties and a hedge ratio of β, a systemic pairs trading strategy is designed. The strategy utilizes the cointegrated pairs deviations from long-term equilibrium to determine the long and short positions.

2.4. Systemic Trading Architecture

Our trading model, as depicted in Figure 1, consists of five modules, including a data step procedure, analytics procedures, hedging based on the machine learning method, pairs trading, and evaluation of trading performance.

In the first module, we let the trading system automatically gather futures daily data. In the second module, we select pairs among assets in the same market condition (e.g., contango or backwardation) using cointegration based on the error correction model (ECM) suggested by Engle and Granger [22]. In the third module, we calculate a hedge ratio from the SVM-regression after selecting a cointegrated pair from in-sample data. In the fourth module, we generate the spreads based on the hedge ratio using out-of-sample data. We determine long and short signals when the spread deviated from a range of upper and lower levels. In the last trading module, we evaluate our trading strategy using performance measures, including annualized return, annualized standard deviation, Sharpe ratio, and maximum drawdown.

Trading Strategy and Hedge Ratio from SVM

Extending on Rad et al. [21], we employ a three step approach in selecting the pairs among futures contracts using cointegration. We first classify futures contracts into seven types of contracts, including grain oil seeds, live stocks, soft, lumber, metal, energy, and financial futures. Secondly, we sort combinations of pairs within each type of futures contract based upon the sum of squared residuals using standardized futures prices. Last, we test cointegration for each selected pair and choose pairs that are statistically significant in the framework of the error correction model.

The Engle–Granger two-step approach [22] is used to test whether the selected pairs are cointegrated. The first step utilizes the SVM to estimate the cointegration regression. The second step is to estimate the error correction model (ECM). The SVM regression method in the first step provides the estimation of the cointegration coefficient $\beta$. Upon calculation of the spread, the long and short positions are simultaneously opened and closed. Specifically, if the spread at time t is greater than the 15-day rolling moving average plus the 15-day rolling standard deviation, we take a short position. In contrast, if the spread at t is less than the 15-day rolling moving average plus the 15-day rolling standard deviation, we take a long position. When the spread reverts toward the mean, we exit both positions as that is indicative of reverting to long-term equilibrium.

2.5. Research Data

Our data include futures settlement prices from January 1998 through December 2018 (the last trading day of each month) published in the Wall Street Journal (WSJ) Futures section and Investor’s Business Daily (IBD). The prices are used to create semi-monthly data points, from the first to the 15th day of the month and from the 16th to the end of the month. If the 15th day falls on a non-trading day, the next closest day which results in two equal, or as close to equal, semi-monthly data points is used. Subsequently, there are 504 data points for the futures contracts included in the study. The first data point included is on 15 January 1998 (settlement price on 15 January 1998 minus settlement price on 31 December 1997), and the last data point is obtained on 31 December 2018 (settlement price on 31 December 2018 minus settlement price on 15 December 2018). We then count the number of price increases and decreases in a semi-monthly period over the entire study period.

The price at the start and end point of the period is reported in each futures market, and the change in price over the period is calculated. Price changes are estimated for the most active trading months, an average annual percentage price change of the contract during the period is defined as:

$$\mathrm{Annual}\mathrm{Percentage}\mathrm{Price}\mathrm{Change}\left(\mathrm{APPC}\right)=\frac{\left(P_{end}-P_{start}\right)/P_{start}}{NumberofYears}\times 100$$

(22)

The fraction of semi-annual periods where prices increase relative to the total number of periods is calculated.

In order to examine the progression of the pricing mechanisms in futures markets, the sample is divided into two sub-periods 1998–2007 and 2008–2018. The resulting data points for the sub-periods are 240 for the first and 264 for the second. The purpose of this division is to determine if one pricing mechanism is dominant in each futures market and if the dominant mechanism is stable and sustained over the entire sample period.

3. Empirical Results and Discussions

Table 1. Futures Prices over 480 semi-months from Jan. 1998 to Dec. 2018.

Futures Markets	Price (01/98)	Price (12/18)	Diff. (t-Stat)	APPC	# of Price Increases (%)
AGRICULTURE
Grain and Oilseeds
Com, CBOT	283.25	350.75	67.5 (9.39) ***	+1.18	239 (49.8%)
Beans, CBOT	670.25	961.25	291.0 (5.61) ***	+2.17	251 (52.3%)
Meal, CBOT	194.4	316.8	122.4 (4.18) ***	+3.14	245 (51.0%)
S-Oil, CBOT	25.18	33.08	7.9 (7.37) ***	+1.56	234 (48.8%)
Livestock
Wheat, CBOT	351.0	427.0	76.0 (10.24) ***	+0.54	221 (46.0%)
F-Cattle, CME	78.02	142.67	64.7 (3.41) ***	+4.14	255 (53.1%)
L-Cattle, CME	67.97	122.52	54.6 (3.49) ***	+4.02	244 (50.8%)
Hogs, CME	55.75	75.65	19.9 (6.60) ***	+1.78	241 (50.2%)
Soft
Cocoa, CSCE	1574	1892	318 (10.90) ***	+1.01	246 (51.3%)
Coffee, CSCE	168.2	126.20	−42.0 (7.01) ***	−1.24	219 (45.6%)
Sugar, CSCE	11.17	15.16	4.0 (6.60) ***	+1.78	236 (49.2%)
Cotton, ICE	65.97	78.63	12.7 (11.42) ***	+0.95	225 (46.9%)
Frozen Orange Juice
Orange Juice, ICE	97.10	136.85	39.8 (5.89) ***	+2.04	246 (51.3%)
Lumber
Random Lumber, CME	274.5	441.9	167.4 (4.28) ***	+3.05	228 (47.5%)
METAL
Base
Copper, CMX	78.00	330.05	252.1 (5.61) ***	+16.15	247 (51.5%)
Precious
Gold, CMX	287.7	1309.3	1021.6 (5.42) ***	+17.75	237 (49.4%)
Platinum, NYM	388.6	938.3	549.7 (8.36) ***	+7.07	251 (52.3%)
Silver, CMX	513.6	1714.5	1200.9 (6.43) ***	+9.74	245 (51.0%)
ENERGY
Crude Oil, NYM	16.72	60.44	43.7 (6.11) ***	+13.07	265 (55.2%)
Heating Oil, NYM	0.4697	2.0136	1.5 (5.57) ***	+16.43	263 (54.8%)
Unleaded Gasoline 1, NYM	0.5356	1.9919	1.5 (6.01) ***	+13.59	276 (57.5%)
CURRENCY
Japanese Yen, CME	0.7774	0.8914	0.1 (14.64) ***	+0.73	229 (47.7%)
Canadian Dollar, CME	0.6979	0.7990	0.1 (14.81) ***	+0.72	255 (53.1%)
British Pound, CME	1.6246	1.3559	−0.3 (11.09) ***	−0.88	237 (49.4%)
Swiss Franc, CME	0.6712	1.0327	0.4 (4.71) ***	+2.69	232 (48.3%)
FINANCIAL
Eurodollar, CME	94.53	98.24	3.7 (51.96) ***	+0.19	250 (52.1%)
T-Bonds, CBOT	123.01	153.00	30.0 (9.20) ***	+1.22	251 (52.3%)
S & P 500, CME	955.1	2676.0	1720.9 (2.11) ***	+9.00	254 (52.9%)

¹ NY Harbor Gas Blend was used after November 2006. *** denote significance at the 1% level.

reports descriptive price changes across 28 futures markets over the period 1998 to 2018. The Annual Percentage Price Change (APPC) for most futures contracts, excluding the coffee and British Pound markets, shows a positive price increase, with a rising price trend in futures markets over 504 semi-monthly periods. The key reason for the negative APPC in coffee futures stems from Brazil, which represents 40% of global coffee production and has overproduced. Additionally, its currency (REAL) has substantially depreciated—approximately 60% since 2006. The negative APPC in GBP futures can be attributed to market participants’ concern regarding the currency market’s deeper decline due to Brexit uncertainty. The t-statistic for the difference between the starting and ending price across each futures market is shown in column (4) and indicates almost total support for rejecting the null hypothesis of zero equivalence. Most of the futures contracts experience price increases, excluding coffee and sterling, and the differences are statistically significant at the 1% level. In addition, based on PIP/TP, we observe that 11 futures markets are either contango or weak contango, while the remaining 17 markets are either normal backwardation or weak normal backwardation. APPC in column (5) shows that futures prices in metal and energy markets have sharply increased relative to other futures prices, with an APPC for energy and metal futures ranging from 9.74 to 16.43.

To investigate whether the impact of market mechanisms changes over time, we conduct a split sample analysis, dividing the futures price movement into two sub-periods.

Table 2. Futures Prices over 240 semi-months from Jan. 1998 to Dec. 2007.

Futures Markets	Price (01/98)	Price (12/07)	Diff. (t-stat)	APPC	# of Price Increases (%)
AGRICULTURE
Grain and Oilseeds
Com, CBOT	283.7	455.5	171.8 (4.30) ***	+6.05	119 (49.6%)
Beans, CBOT	670.25	1214.25	544.0 (3.46) ***	+8.11	122 (50.8%)
Meal, CBOT	194.4	336.7	142.3 (3.73) ***	+7.32	116 (48.3%)
S-Oil, CBOT	25.18	48.85	23.7 (3.13) ***	+9.40	115 (47.9%)
Wheat, CBOT	351.0	885.0	534.0 (2.31) **	+15.21	107 (44.6%)
Livestock
F-Cattle, CME	78.02	107.10	29.1 (6.37) ***	+3.72	124 (51.4%)
L-Cattle, CME	67.97	98.50	30.5 (5.45) ***	+4.49	119 (49.6%)
Hogs CME	55.75	63.75	8.0 (14.94) ***	+1.43	123 (51.3%)
Soft
Cocoa, CSCE	1574	2035	461.0 (7.83) ***	+2.92	120 (50.0%)
Coffee, CSCE	168.2	136.2	−32.0 (9.51) ***	−1.90	107 (44.6%)
Sugar, CSCE	11.17	10.82	−0.4 (6.83) ***	−0.31	116 (48.3%)
Cotton, ICE	65.97	68.01	2.0 (65.68) ***	+0.31	108 (45.0%)
Frozen Orange Juice
Orange Juice, ICE	87.10	144.80	57.7 (4.02) ***	+4.91	122 (50.8%)
Lumber
Random Length Lumber, CME	274.5	234.5	−40.0 (12.73) ***	−1.42	118 (49.2%)
METAL
Base
Copper, CMX	78.00	304.10	226.1 (5.85) ***	+28.98	126 (52.5%)
Precious
Gold, CMX	287.7	838.0	550.3 (7.09) ***	+19.12	112 (46.7%)
Platinum, NYM	388.6	1528.4	1139.8 (5.83) ***	+29.33	127 (52.9%)
Silver, CMX	581.5	1492.0	910.5 (7.89) ***	+15.66	120 (50.0%)
ENERGY
Crude Oil, NYM	16.72	95.78	79.1 (4.93) ***	+47.26	134 (55.8%)
Heating Oil, NYM	0.4697	2.6444	2.2. (4.96) ***	+46.29	133 (55.4%)
Unleaded Gasoline 1, NYM	0.5356	2.6530	2.1 (4.51) ***	+39.19	137 (57.1%)
CURRENCY
Japanese Yen, CME	0.7744	0.9013	0.1 (13.20) ***	+1.59	110 (45.8%)
Canadian Dollar, CME	0.6979	1.0099	0.3 (5.47) ***	+4.47	128 (53.3%)
British Pound, CME	1.6246	1.9785	0.4 (10.18) ***	+2.17	116 (48.3%)
Swiss Franc, CME	0.6712	0.8838	0.2 (7.31) ***	+3.17	109 (45.4%)
FINANCIAL
Eurodollar, CME	94.53	95.77	1.2 (54.47) ***	+0.13	120 (50.0%)
T-Bonds, CBOT	123.01	116.12	−6.9 (34.71) ***	−0.54	123 (51.3%)
S & P 500, CME	955.10	1077.2	122.1 (16.64) ***	+1.28	122 (50.8%)

¹ NY Harbor Gas Blend was used after November 2006. *** denote significance at the 1% level.

summarizes price changes in futures markets over 240 semi-monthly periods from 1998 to 2007. The prices for the futures contracts have increased in APPC, excluding coffee, lumber, and T-bonds futures. The last column of Table 2 shows that 21 futures markets exhibit price increases of over 50% of the total periods. Noticeably, energy and metal futures are backwardation oriented markets over the sub-sample period. The APPCs of metal and energy futures are approximately nine times greater than that of other futures (32.26 for metal and energy vs. 9.34 for others).

Table 3. Futures Prices over 264 semi-months from Jan. 2008 to Dec. 2018.

Futures Markets	Price (01/98)	Price (12/17)	Diff. (t-stat)	APPC	# of Price Increases (%)
AGRICULTURE
Grain and Oilseeds
Com, CBOT	514.75	350.75	−164.0 (5.28) ***	−3.18	120 (50.0%)
Beans, CBOT	1296.5	961.25	−355.3 (6.73) ***	−2.58	129 (53.8%)
Meal, CBOT	353.8	316.8	−37 (18.12) ***	−1.04	129 (53.8%)
S-Oil, CBOT	53.38	33.08	−20.3 (4.26) ***	−3.80	119 (49.6%)
Wheat, CBOT	851.5	423.0	−428.5 (2.97) ***	−4.38	114 (47.5%)
Livestock
F-Cattle, CME	106.75	142.67	35.9 (6.94) ***	+3.36	131 (54.6%)
L-Cattle, CME	93.40	122.52	29.1 (7.41) ***	+3.12	125 (52.1%)
Hogs, CME	61.10	75.65	14.6 (9.40) ***	+2.38	118 (49.2%)
Soft
Cocoa, CSCE	2172	1892	−280 (14.51) ***	−1.25	126 (52.5%)
Coffee, CSCE	137.60	126.20	−11.4 (23.14) ***	−0.83	112 (46.7%)
Sugar, CSCE	12.14	15.16	3.0 (9.04) ***	+2.49	120 (50.0%)
Cotton, ICE	73.03	78.63	5.6 (27.08) ***	+0.76	117 (48.8%)
Frozen Orange Juice
Orange Juice, ICE	137.60	136.85	−0.8 (1.15)	−0.05	124 (51.7%)
Lumber
Random Length Lumber, CME	250.6	441.9	191.3 (3.62) ***	+7.63	110 (45.8%)
METAL
Base
Copper, CMX	319.35	330.05	10.7 (210.24) ***	+0.33	121 (50.4%)
Precious
Gold, CMX	888.2	1309.3	421.1 (18.08) ***	+4.74	125 (52.1%)
Platinum, NYM	1567.1	938.3	−628.8(18.08) ***	−4.01	124 (51.7%)
Silver, CMX	1600.3	1714.5	114.2 (100.55) ***	+0.71	125 (52.1%)
ENERGY
Crude Oil, NYM	89.98	60.44	−29.5 (17.64) ***	−3.28	131 (54.6%)
Heating Oil, NYM	2.4684	2.0136	−0.5 (35.14) ***	−1.84	127 (52.9%)
Unleaded Gasoline 1, NYM	2.4600	1.9919	−0.5 (32.95) ***	−1.90	130 (54.2%)
CURRENCY
Japanese Yen, CME	0.9342	0.8914	−0.0 (1.66) *	−0.46	119 (49.6%)
Canadian Dollar, CME	0.9773	0.7990	−0.5 (34.14) ***	−1.82	127 (52.9%)
British Pound, CME	1.9594	1.3559	−0.5 (32.95) ***	−3.08	121 (50.4%)
Swiss Franc, CME	0.9107	1.0327	0.1 (1.72) *	+1.34	123 (51.3%)
FINANCIAL
Eurodollar, CME	96.91	98.24	1.3(146.73) ***	+0.14	130 (54.2%)
T-Bonds, CBOT	119.06	153.00	33.9 (8.02) ***	+2.84	128 (53.3%)
S & P 500, CME	1376.1	2676.0	1299.9 (3.12) ***	+9.45	132 (55.0%)

¹ Harbor Gas Blend was used after November 2006. ***, * denote significance at the 1% and 10% level, respectively.

presents price movement for 2008 through 2018. Similar to Table 2, the last column of Table 3 includes the number of semi-annual periods in 28 futures markets where there have been price increases. The fraction of the increased periods relative to the total number of periods is reported in parentheses. The majority of futures markets exhibit increases in more than 50% of total periods, indicating that an increasing pattern in futures prices is dominant in the period following the 2007 financial crisis.

Interestingly, in column (6), the grain and oilseeds and energy futures clearly display backwardation, although the energy market has experienced a price decline since the 2007 financial crisis. We observe that speculators could create trading profits by taking a simple long position. However, in column (4) and (5), all the differences for the markets are negative, and the t-statistics are statistically significant at a 1% level. Since this also indicates that naïve speculators could have gone short of creating trading profits, it is not conclusive that the rising trend in prices results in a gain to investors for bearing risk.

In summary, the tables show that the mechanisms exhibit a degree of consistency. For the majority of futures markets examined, the dominant mechanism remains constant. However, we also observe a change in the dominant mechanism of some markets over time (for example, corn, bean, meal, gold, platinum, crude oil, etc.).

3.1. Heterogeneous Price Movements

Results indicate heterogeneity exists across futures markets. We observe the coexistence of weak backwardation and weak contango mechanisms in futures markets. The frequency of each mechanism’s dominance for all futures markets, applying the fractional definitions presented in the previous section, are presented in

Table 4. Frequency of Mechanism Dominance.

	1998–2018		1998–2007		2008–2018
	No. Futures	%	No. Futures	%	No. Futures	%
Contango	7	25.00	10	35.71	4	14.29
Weak Contango	4	14.29	3	10.71	4	14.29
Weak Backwardation	2	7.14	6	21.43	3	10.71
Backwardation	15	53.57	9	32.14	17	60.71
Total	28	100	28	100	28	100

. Contango, weak contango, weak backwardation, and backwardation account for 25.00%, 14.29%, 7.14%, and 53.57% of the frequency of the dominant price mechanism. During the last two decades, approximately one-fourth of the contracts exhibit a sellers’ hedge, one-half of the contracts indicate buyers have an incentive to hedge, but 21 percent of contracts do not show strong future price directions. Additionally, during the first sub-sample period, 32 percent of contacts present a buyers’ hedge, and 35 percent of contracts display a sellers’ hedge. Since the financial crisis, the dominant market mechanism has shifted. During the second sub-sample period, the majority of contracts (60 percent) indicate investors who intend to hedge a price increase. Subsequently, each market is examined thoroughly, and the implications behind the functional mechanisms are discussed.

Contrary to a priori belief that sellers in the grain markets will demonstrate a propensity to hedge, only soybean futures show the farmer’s incentive to limit downside risk. Wheat contracts show the buyers’ (e.g., grain processors) incentive to hedge outweighs that of sellers. We find evidence of backwardation in the feeder cattle market. Alternatively, live cattle and lean hogs contracts show less consistency in price movement, indicating that the trader’s forecasting ability (i.e., weak contango or weak backwardation) may better determine their profits. In the remaining agriculture markets, evidence of both backwardation and contango is exhibited. In the livestock sector, live cattle in the second period and feeder cattle in both periods show ranchers’ incentives to hedge (52% and 53% respectively); conversely, there is no clear pattern in the lean hogs market. The results reveal the contango market condition, with price increase periods of less than 49%, in “soft” commodities such as coffee and cotton. The conclusion from these results is that over the last twenty years, selling futures contracts may have resulted in a higher probability for profits. Simultaneously, buyers (e.g., coffee brewers like Starbucks) have a strong incentive to hedge risk through buys in the futures market.

Energy markets primarily exhibit strong backwardation. For example, 56% of price increase periods are observed in crude oil. One explanation may be that oil-producing countries (e.g., OPEC members) are prepared to incur a premium to hedge risk. Among industrial commodities, only copper and silver show a mild backwardation in the first period (53% each). Even with a significant surge in price during the evaluation period, the gold futures market does not show backwardation over the whole period (49%). Rather it presents a mild form of contango in the first decade (47%). A short position in the gold futures market may have resulted in higher profitability over this period.

Additionally, we find that sizeable contango occurs in some financial futures markets. Most pronounced in the first period of our study are the Japanese Yen and Swiss Franc futures (46% and 45%, respectively). Logically, the financial futures markets should be the most efficient markets, partly due to higher trading volumes and open interests. However, the S&P 500 futures contracts present a substantial measure of backwardation (53% over the entire period and 55% in the second period). Why does the stock market show backwardation? The current spot index builds in the market’s belief that the expected stock return is the risk-free rate plus an equity risk premium, equaling approximately 5%. An investor who owns the index portfolio can hedge by shorting the index future. In order to prevent arbitrage, the futures price must be equal to the current spot price plus the riskless rate r minus the dividend yield d. This is less than the expected future spot price, which would be the current spot price plus r–d + 5%. That is, the future price is 5% lower than the expected spot price; thus the market is in backwardation, and short hedgers expect to pay 5% to speculators as the cost of insurance.

The backwardation indicates a high premium conferred from buyers to sellers in interest futures markets, Eurodollars and T-bonds. For example, the U.S. Treasury is strongly incentivized to pay an insurance premium to establish the sale price of government securities. One factor potentially driving this pattern is that over the observation period, the general interest rate levels have fallen. Decreasing interest rates and resulting in rising bonds prices, appear linked through normal backwardation.

3.2. Normal Backwardation and Contango: Price Patterns Over Time

Sub-periods are examined to determine the time-series consistency of functional mechanisms for each market, as shown in

Table 5. Cross Tabulated Patterns from Sub-period 1 to Sub-period 2.

Sub-Period (1998–2007)	Sub-Period (2008—2018)				Total
	Contango	Weak Contango	Weak Backwardation	Backwardation
Contango	3 (1.4)	3 (1.4)	1 (1.1)	3 (6.1)	10
Weak Contango	1 (0.4)	0 (0.4)	1 (0.3)	1 (1.8)	3
Weak Backwardation	0 (0.9)	0 (0.9)	0 (0.6)	6 (3.6)	6
Backwardation	0 (1.3)	1 (1.3)	1 (1.0)	7 (5.5)	9
Total	4	4	3	17	28
${\mathsf{\chi }}^2$ = 13.67, p-value = 0.000

Note: values in parentheses indicate expected frequencies from distribution.

. The results in Table 5 indicate that the mechanisms exhibit a tendency toward backwardation (60.7%), as observed in Table 4, most pronounced is the continuation from backwardation to backwardation (25.0%). Another distinct trend is that over the observation period, the contango mechanism seems to lose intensity (14.3%). We observe zero transitions from the backwardation to contango. The implication of the change in the pattern of futures contract prices is an increase in the sellers’ needs to hedge. Only coffee and wheat begin and remain contango in both decades. Some markets, such as gold, change from contango to backwardation. In these markets, buyer dominance, driven by large and small speculators, transforms into seller dominance, driven by large hedgers such as mining companies. In the currency market, Swiss Franc, British Pound, and Japanese Yen all convert from the contango to weak contango or weak backwardation mechanism. While there may be many explanations for these price pattern changes, conclusions should be interpreted with caution as multiple factors may drive the change.

Additionally, we run a nonparametric ${\mathsf{\chi }}^2$ test of independence in order to examine whether there is an association between market dominance over the first and the second sub-sample periods. The ${\mathsf{\chi }}^2$ of 13.67 rejects the null hypothesis that the past market mechanism is independent of the current market mechanism. This is consistent with our findings and supporting evidence that speculators generate positive expected risk premiums based on superior forecasting ability.

In summary, presented in Table 4 and Table 5, the prevailing trend across futures markets is backwardation, although there are cross-sectional differences. This implies that futures prices incorporate the insurance premiums of sellers seeking to shed risk. In addition, it indicates that futures prices reflect less of the buyer’s needs and more of the seller’s needs, as futures markets were originally intended.

3.3. Nonparametric Statistical Test

In addition to analyzing forecasting ability, nonparametric procedures are applied to futures contract price changes for each semi-monthly period. Following Chang (1985), “up-futures” are defined as positive price changes, and “down-futures” are defined as all negative price change intervals.

Table 6. Sum of Conditional Probabilities of a Correct Market Position of Speculation Given that R(t) ≤ 0 and R(t) > 0 for ${\mathrm{H}}_0:P_1^s\left(t\right)+P_2^s\left(t\right)=1$ and ${\mathrm{H}}_1:P_1^s\left(t\right)+P_2^s\left(t\right)>1$.

Futures Markets	Down Futures Markets ${\mathit{P}}_1^{\mathit{s}}\left(\mathit{t}\right)$	Up Futures Markets ${\mathit{P}}_2^{\mathit{s}}\left(\mathit{t}\right)$	All Futures Markets ${\mathit{P}}_1^{\mathit{s}}\left(\mathit{t}\right) + {\mathit{P}}_2^{\mathit{s}}\left(\mathit{t}\right)$
AGRICULTURE
Grain and Oilseeds
Corn, CBOT	0.472	0.619	1.091 **
Beans, CBOT	0.442	0.652	1.094 **
Meal, CBOT	0.315	0.892	1.207 ***
S-Oil, CBOT	0.439	0.607	1.046
Wheat, CBOT	0.461	0.587	1.048
Livestock
F-Cattle, CME	0.417	0.636	1.053
L-Cattle, CME	0.494	0.551	1.045
Hogs, CME	0.452	0.609	1.061 *
Soft
Cocoa, CSCE	0.461	0.779	1.240 ***
Coffee, CSCE	0.482	0.612	1.094 **
Sugar, CSCE	0.332	0.793	1.125 ***
Cotton, ICE	0.484	0.625	1.109 **
Frozen Orange Juice
Orange Juice, ICE	0.477	0.610	1.087 **
Lumber
Random Length Lumber, CME	0.429	0.656	1.085 ***
MINERAL
Base
Copper, CMX	0.331	0.721	1.092 **
Precious
Gold, CMX	0.287	0.896	1.183 ***
Platinum, NYM	0.411	0.764	1.175 ***
Silver, CMX	0.485	0.795	1.280 ***
ENERGY
Crude Oil, NYM	0.389	0.812	1.201 ***
Heating Oil, NYM	0.310	0.858	1.168 ***
Unleaded Gasoline 1, NYM	0.354	0.789	1.143 ***
CURRENCY
Japanese Yen CME	0.370	0.698	1.068
Canada Dollar CME	0.451	0.598	1.079 *
British Pound CME	0.418	0.662	1.090 **
Swiss Franc CME	0.473	0.582	1.035
FINANCIAL
Eurodollar CME	0.364	0.717	1.081 *
T-Bonds CBOT	0.369	0.861	1.230 ***
S & P 500 CME	0.273	0.886	1.159 ***

¹ NY Harbor Gas Blend was used after November 2006. ***, **, * denote significance at the 1%, 5%, and 10% level, respectively.

summarizes the null hypothesis ${\mathrm{H}}_0:P_1^S\left(t\right)+P_2^S\left(t\right)=1$ and the estimated conditional probabilities of speculators in 28 up and down futures contracts. The results show that traders may not be as consistent in their ability to forecast down-futures contracts as they are in forecasting up-futures. For example, the forecasting ability of the agricultural speculators in down-futures outperforms relative to that of other market traders. The average conditional probability in down-futures for agricultural, mineral, currency, and financial futures are 44.0%, 37.5%, 43.3%, 3.25%, respectively. Comparing conditional probabilities of down-futures in column 3 and those of up-futures in column 4, we find that, generally, speculators tend to have better forecasting ability in up-futures than in down-futures. Speculator positions are impactful across agricultural, mineral, currency, and financial futures markets. These findings are consistent with those of Chang [1] and Schwarz [26].

In column 5, the conditional probabilities for all markets are greater than zero. However, tests for S-Oil, Wheat, F-Cattle, L-Cattle, Japanese Yen, Swiss Franc, and Euro-dollar futures fail to reject the null hypothesis that speculators do not make positive gains at the 5% level of significance. Interestingly, tests of mineral futures result in a rejection of the null hypothesis providing evidence that speculators in metal markets are informed with superior forecasting ability.

3.4. Systemic Pairs Trading

To evaluate our pairs trading strategy, daily futures data are collected using Bloomberg over the sample period 2 January2015 to 9 July 2020. The dataset is divided into two sub-samples: in-sample data (2 January 2015–31 December 2017); out-of-sample data (2 January 2018–9 July 2020). The trading strategy is developed using in-sample-data, and backtesting is performed using the out-of-sample dataset. By using in-sample-data, we examine combinations of pairs in the same futures markets. Pairs that can be constructed across futures markets are not considered, as our previous findings suggest that each market has its own unique price trend. Regressions are run using various combinations of pairs constructed within the same markets. Next, the sum of squared residuals is calculated, and the pairs are ranked by the sum of squared residuals. Five pairs of futures remain after excluding pairs that are not statistically cointegrated: CME E-Mini S&P 500 and CME NASDAQ100 futures; COMEX Gold and COMEX Silver futures; COMEX Copper and NYMEX Platinum futures; CBOT Oats and CBOT Corn futures; CME Feeder cattle and CME Live cattle futures. The SVM regression analysis is performed with a tuning process using in-sample-data to estimate the best hedge ratios for the five selected pairs.

Table 7. Spread, $\mathsf{\Delta }{\nu }_t=\mathsf{\Delta }{S}_2\left(t\right)-\beta \mathsf{\Delta }{S}_1\left(t1\right)$, and Hedge ratios for the Selected Pairs from In-Sample-Data (2 January 2015–31 December 2017).

$\mathsf{\Delta }{\mathit{S}}_2\left(\mathit{t}\right)$	$\mathsf{\Delta }{\mathit{S}}_1\left(\mathit{t}1\right)$	$\mathbf{Hedge}\mathbf{Ratio}\left(\mathit{\beta }\right)$
CME E-Mini S&P 500	CME Nasdaq 100	0.72
COMEX Gold	COMEX Silver	0.24
CMEX Copper	NYMEX Platinum	0.02
CBOT Oats	CBOT Corn	0.04
CME Feeder Cattle	CME Live Cattle	0.49

summarizes the hedge ratios for the pairs, ranging from 0.02 (Oats and Corn) to 0.72 (S&P500 and NASDAQ 100). By utilizing the hedge ratios, we calculate spreads, and determine the size of hedging for backtesting. For example, we buy 1 dollar of E-mini S&P500 futures and sell 0.72 dollars of NASDAQ 100 when the spread between the two futures is below a specified lower bound. Conversely, when the spread is above a specified upper bound, 1/0.72 dollars of E-Mini S&P500 futures is shorted, and 1 dollar of NASDAQ 100 is purchased. The lower and upper bounds at time t are dynamically calculated based on the 15-day rolling average and standard deviation of the past spread.

Figure 2 illustrates cumulative returns for the five pairs traded over the out-of-sample period. In order to examine each pair’s performance, we compare our pairs trading approach and a buy-and-hold investment strategy. In each panel, a solid black line represents a time series of cumulative returns for pairs trading and a solid red line represents cumulative returns for a corresponding benchmark. In panel (a), the cumulative returns for the pair between E-mini S&P500 and NASDAQ 100 futures steadily grow over the out-of-sample period, while the cumulative returns for the S&P 500 index futures buy-and-hold strategy show wide fluctuation. Notably, when the number of deaths due to the COVID-19 pandemic steeply increased after 24 February 2020, the cumulative return for the benchmark plummeted, and the slope of the time series line reverted to negative. In contrast, the cumulative return for pairs trading still shows an upward trend. Pairs trading two index futures appears to hedge against unexpected systemic risk during the backtesting period. In panel (b), the cumulative returns for the gold and silver futures pair and for gold futures both show an upward trend, but the cumulative returns for pairs trading outperform the benchmark. In panel (c), pairs trading utilizing copper and platinum futures achieves an exceptional outcome. The time series line for pairs trading shows relatively little fluctuation and stays above the benchmark (i.e., a buy-and-hold copper trading) over the trading period. Additionally, the graph shows that pairs trading was not markedly affected by COVID-19, unlike the benchmark. The backtesting result for oats and corn futures in panel (d) shows the return from pairs trading is stable relative to the movement in the cumulative return of the oat futures benchmark. Similarly, in panel (e), pairs trading with feeder and live cattle futures is less risky than the feeder cattle benchmark. The slope of the cumulative return for the benchmark converted to negative with the onset of COVID-19, while the trend of the cumulative returns in the pairs trading was positive. This result also suggests that pairs trading can hedge against an adverse decline in futures prices stemming from a systemic event.

To examine whether the application of SVM to estimate cointegration coefficients can improve pairs trading performance in futures markets, we further estimate cointegration coefficients using the ordinary linear square (OLS) estimation method. Figure 3 exhibits the comparison of the cumulative returns of pairs trading between the SVM method and the OLS approach. Each panel provides evidence that SVM improves the pairs trading strategy. The pairs trading algorithm implemented by the SVM coefficient outperforms the OLS pairs trading strategy. Notably, the performance of SVM consistently outperforms prior to COVID-19 and through the pandemic compared to the performance of OLS. SVM can add considerable value in pairs trading with futures contacts.

In addition to the cumulative returns, performance measures are examined to evaluate the pairs trading system.

Table 8. Trading Performances for the Pairs over the Backtesting Period (2 January 2018 to 9 July 2020).

Pairs	Ann. Return (%)	Ann. Std. Dev (%)	Ann. SR	Max. Drawdown (%)
Panel A. E-mini S&P 500 Index and NASDAQ 100 Index Futures
Pair	4.23	5.23	0.81	3.51
E-mini S&P500	7.55	23.23	0.33	34.95
Panel B. Gold and Silver Futures
Pair	17.95	14.77	1.22	6.59
Gold	17.21	13.67	1.11	11.82
Panel C. Copper and Platinum
Pair	11.98	14.05	0.85	15.51
Copper	2.95	20.96	0.14	27.82
Panel D. Oats and Corn Futures
Pair	14.15	22.58	0.62	13.45
Oats	17.76	35.26	0.50	22.39
Panel E. Feeder Cattle and Live Cattle Futures
Pair	5.06	12.60	0.40	8.19
Feeder Cattle	−4.09	21.28	−0.19	32.63

reports the pairs trading performance for the five pairs. Panel A presents the performance of the pairs trading strategy for the E-mini S&P 500 and NASDAQ 100 index futures. The annualized return for the pairs trading is 4.23 percent, which is 3.32 percentage points less than that of the benchmark. However, the Sharpe Ratio (SR) of the pairs trading is higher, as the standard deviation of the benchmark is approximately four times greater than that of the pairs trading. The maximum drawdowns and the maximum observed loss from peak to trough during the trading period, are 3.51 percent for the pairs trading and 34.95 percent for the benchmark, respectively. This suggests that the pairs trading strategy is not considerably riskier than the E-mini S&P 500 index futures benchmark. In panel B, the gold and silver futures pair and gold futures benchmark are presented. They display similarities in annualized return and standard deviation. The pairs trading strategy has slightly higher SR than the benchmark (1.22 vs. 1.11) and has a lower maximum drawdown (6.59 vs. 11.82), which indicates that the pairs trading between gold and silver futures exhibits a relatively neutral performance to market risk events. Panel C reports that the pairs trading strategy for copper and platinum futures outperforms the benchmark. The annualized return for the pairs trading is approximately ten percentage points higher than that of the buy-and-hold copper futures benchmark. The SR of the pairs trading is around six times higher than that of the benchmark (0.85 vs. 0.14). The risk measurements (annualized standard deviation, and maximum drawdown) show the pairs trading strategy is less sensitive to market risk (Ann. Std. Deviation: 14.05 vs. 20.96; Max. Drawdown: 15.51 vs. 27.82). Panel D presents the performance of the oats and corn futures pair. The annualized returns are similar for the pairs trading and oats futures benchmark. The SR for the pairs trading is greater than the benchmark, indicating a larger risk-adjusted return from the pairs trading than the benchmark. The maximum drawdown also shows that the pairs trading is less exposed to downside risk than the oats futures benchmark (13.45 vs. 22.39). Similarly, the feeder and live cattle futures pair is examined in panel E, the annualized return for the pairs trading is 9.15 percentage points greater than that of the feeder cattle benchmark. The annualized standard deviation for the pairs trading is lower than the benchmark, with a higher SR for the pairs trading. During the trading period, the pairs trading strategy had a considerably lower maximum drawdown compared to the benchmark (8.19 vs. 32.63).

In short, the backtesting simulations for the selected pairs and benchmarks show the pairs trading strategy, devised from the SVM regression, demonstrates superior performance. Given the period examined includes the systemic impact of COVID-19 on the US futures markets, we examine how the pairs trading strategy can hedge against this systemic shock. The findings indicate that the pairs trading strategy results in higher risk-adjusted returns, with substantially lower exposure to downside risk.

4. Conclusions

This study develops an algorithmic pairs trading strategy developed through applied machine learning methods. The strategy is motivated by an examination of futures market price trends. We identify that normal backwardation and contango do not consistently characterize futures markets. Our results support certain a priori expectation for futures contract price behavior. For instance, the contango or normal backwardation condition in agriculture and metal markets indicates incentives for a buyer or seller hedging. However, a notable trend this study finds is that over the last two decades, the backwardation mechanism dominates the contango market condition. This implies that the prevailing influence on futures prices is the seller’s need for hedging, as was the original intention for futures markets.

Five combinations of statistically cointegrated pairs in the same futures markets are examined: CME E-Mini S&P 500 and CME NASDAQ100 futures; COMEX Gold and COMEX Silver futures; COMEX Copper and NYMEX Platinum futures; CBOT Oats and CBOT Corn futures; CME Feeder cattle and CME Live cattle futures. An SVM regression analysis is performed to estimate the hedge ratios, based upon the Engle–Granger two-step method (Engle and Granger, 1987). By comparing the pairs trading strategy to a buy-and-hold strategy in an appropriate benchmark, we find the pairs trading strategy consistently outperforms the benchmark and exhibits less fluctuation. Finally, the pairs trading strategy demonstrates an ability to withstand the negative impact of COVID-19, whereas the benchmark investments were all negatively impacted by the escalation of the pandemic.

Further examination of the performance of the pairs trading vs. benchmark strategies supports the conclusion that the pairs trading strategy results in higher risk-adjusted returns and lower downside risk. The Sharpe ratios for each pair are meaningfully larger and the maximum drawdowns are meaningfully smaller, than the respective benchmark values. Importantly, the utilization of a pairs trading strategy creates larger risk-adjusted returns and is characterized by a relatively neutral response to market risk events. Constructing pairs following the devised methodology of calculating spreads and hedge ratios based on the SVM machine learning algorithm can generate a more sustainable market neutral trading strategy.

However, our trading approach is limited to a theoretical price. The proposed method assumes that an asset can be bought and sold at the target price without any restriction. Expanding on our study, future research could make contributions to the literature by considering the actual order book dynamics and a scenario where stock trades might be queued.