On the Elicitability and Risk Model Comparison of Emerging Markets Equities

Abstract

The need for comparative backtesting in the Basel III framework presents the challenge for ranking of internal value-at-risk (VaR) and expected shortfall (ES) models. We use a joint loss function to score the elicitable joint VaR and ES models to select competing tail risk models for the top 9 emerging markets equities and the emerging markets composite index. We achieve this with the model confidence set (MCS) procedure. Our analysis span two sub-sample periods representing turbulent (Eurozone and Global Financial crises periods) and tranquil (post-Global Financial crisis period) market conditions. We find that many of the markets risk models are time-invariant and independent of market conditions. But for China and South Africa this is not true because their risk models are time-varying, market conditions-dependent, percentile-dependent and heterogeneous. Tail risk modelling may be difficult compared to other markets. The resemblance between China and South Africa can stem from the closeness between their equities composition. However, generally, there is evidence of more homogeneity than heterogeneity in risk models. This is indicated by a minimum of three models (out of six) per equity in most of the countries. This may ease the burden for risk managers to find the optimal set of models. Our study is important for internal risk modelling, regulatory oversight, reduce regulatory arbitrage and may bolster confidence in international investors with respect to emerging markets equities.

1. Introduction

Value-at-risk (VaR) and expected shortfall (ES) have been the two main regulatory capital requirement and portfolio risk measures for a long time. However, both of them suffer practical and coherent risk weaknesses (Burzoni et al. [1]; Cont et al. [2]; Fissler & Ziegel [3]; Fissler et al. [4]; Nolde & Ziegel [5]). Though, ES may have been espoused as a coherent measure of risk, it is sensitive to tails and can lead to greater periodic capital charges unlike VaR (Chang et al. [6]). Large ES values also tend to be more sensitive toward regulatory arbitrage and parameter specification (Kellner and Rösch [7]). But more importantly, it is the lack of elicitability of the ES that poses concerns (Fissler & Ziegel [3]; Nolde & Ziegel [5]). Elicitability can be defined as the property that allows competing risk measures to be comparable (Fissler & Ziegel [3]).

On the other hand, modeling tail risks of financial assets continue to be a daunting task for risk managers. There are a myriad of distributional innovations to choose from in the quest to address stylised facts of assets returns. The past decade has seen a proliferation of competing models (Bernardi & Catania [8]), not only for use by econometricians, but also for internal risk managers of financial firms. Hence, in recent time, the objective of any risk manager has not been to find a single best model but to find a relatively small set of ordered superior models, at a certain confidence level (Hansen et al. [9]).

Nonetheless, choosing a set of risk models should not only be an internal affair, but it should be in line with regulatory framework. The Basel III framework is expected to be fully phased by 1 January 2027 (Patton et al. [10], https://www.bis.org/press/p181004.htm, accessed on 31 August 2021). The Basel III framework requires VaR and ES models to be likened to their standardised approach known as comparative backtesting (i.e., where a bank’s internal risk model is a comparable standardised approach (Fissler, Ziegel, & Gneiting [4]. The approach involves ranking a set of competing models based on the forecast ability). While VaR models lend themselves to be ranked, ES lacks this property. However, at the higher level, the VaR and ES, hereafter referred to as (VaR, ES), is jointly elicitable. The Fissler & Ziegel [3] Loss (FZL) function is an appropriate associated score. This property of the (VaR, ES) offers the possibility to rank competing risk models based on a consistent scoring function. Elicitable risk measures, therefore, serve as the bridge between internal models of financial institutions and standardised regulatory approaches. This has opened a fledgling area in financial risk model selection in direct agreement with regulatory standards.

It is clear that risk modelling of EMs equities abound in the literature. Notable among them include Bao et al. [11], Del Brio et al. [12], Hwang & Satchell [13], Ji et al. [14], Miletic & Miletic [15], etc. Risk modelling of EMs equities has become important because the financial imprudence of both developed markets and EMEs can put the global financial system in despair. There are undeniable reasons for this assertion. The EMEs have become a large and important component of international finance since they connected with each other and with developed markets. For instance, the Organisation for Economic Corporation and Development Centre indicates that, economic and political power have moved towards EMEs over the last two decades (Kharas [16]). This indicated by an increasing number of EMEs becoming strong centres of growth as their share of global income have risen significantly (Kharas [16]). Further, given the explicit indication in the literature that EMs equities are high risk despite their high returns, a strong risk management process that aligns with regulatory framework is desirable. Some episodes of capital flight from EMEs can be attributed to this feature (see Aizenman et al. [17]; Enginar et al. [18]; Ghosh & Saggar [19]; Mishra et al. [20]). This risk modelling and management practice for EMs equities may elicit international investors, motivated not only by high returns, but also with the knowledge that risks are adequately quantified. It is for these reasons that our study is focused on EMEs to contribute to the literature on the best practice risk modelling and management on equities.

Nonetheless, many studies on model ranking and selection that are consistent with the current regulatory framework are largely missing. In this study, we seek to model and forecast tail risk of sampled emerging market equities using the (VaR, ES) model in light of the current regulatory framework. In doing so, we further employ the Hansen et al’s [9] model confidence set (MCS) procedure in ranking and selection of superior set models (SSMs) of the forecasted GAS models based on the FZL function. We use the 95% confidence level in selecting SSMs.

In the emerging market economies (EMEs) literature, several studies have used the Akaike, Consistent Akaike, Bayesian and Hannan-Quinn Information Criteria; AIC, BIC and HQIC, respectively, for risk model ranking and selection in the univariate case. Others also include the Log-likelihood (LogL), root mean squared error (RMSE), mean absolute error (MAE), etc. (Blazsek & Hernández [21]; Gong et al. [22]; Troster et al. [23]; Owusu Junior & Alagidede [24]). In light of recent arguments on combined tests, Khalaf et al. [25] and Kratz et al. [26] introduce multilevel and multinomial backtesting of ES based on series of VaR forecasts, respectively. Regression-based backtest approach has for both VaR and ES is also becoming popular in the literature (see Pradhan & Tiwari [27]; Barendse [28]; Dimitriadis & Bayer [29]; Couperier & Leymarie [30]; etc.).

However, these approaches have important shortcomings. First, they do not rank all models concurrently as a set. Model criteria values are compared after they have been estimated independent of each other. This approach plays down on the interdependence that may exist in the competing risk models (Han & Hausman [31]). Second, the criteria do not possess any consistent scoring function to allow for model ranking. Ranking based on only their magnitude may be inadequate.

Testing procedures for “best” fitting models include; Reality Check (White [32]), Stepwise Multiple Testing (Romano & Wolf [33]), Superior Predictive Ability (Hansen & Lunde [34]) and Conditional Predictive Ability (Giacomini & White [35]), are among the recent ones in the literature. These approaches lack a consistent scoring function such as the FZL function. Even though, the studies of Barendse [28], Dimitriadis & Bayer [29] and Couperier & Leymarie [30] employ a consistent scoring function, their regressions require the specification of covariates which may be arbitrary or complicate the models. Lastly, Taylor’s study [36] is also weak in the sense that it only applies the asymmetric Laplace distribution. We employ the model confidence set (MCS) technique of Hansen et al. [9] to construct a ‘superior’ set of competing (VaR, ES) models based on the FZL function of emerging markets (EMs) equities. To have a complete picture, we also analyse the Morgan Stanley Capital International (MSCI) emerging markets index (EM index). This is used in the absence of a portfolio consisting of the top 9 EMs equities. The MCS follows a sequence of tests to perform the dual task of creating a superior set model (SSM) and ranking the models therein. The null hypothesis of equal predictive ability (EPA) (Diebold & Mariano [37]) is used in selecting SSM.

Our study makes important contributions to the literature on EMs equities risk analysis. First, this is among the first studies to model tail risks in EMs equities with the joint (VaR, ES) model based on FZL function, hence, conforming to the current regulatory framework (i.e., Basel III). The number of available risk models has never been greater and hence intensifies the dilemma of risk managers to ascertain the “best” model while adhering to regulatory requirements. Moreover, it remains a concern for both internal risk managers and regulators to quantify tail risks to safeguard financial catastrophes that may result from inaccurate estimates and forecasts. We offer some novel insights to perform this task on EMs equities in order to reduce regulatory arbitrage.

Second, we surmise that risk modelling and selection that are consistent with regulatory standards may bolster confidence in the vogue international investor concerning EMs equities. One can imagine the prospects of capital flows into EMEs under the circumstance. Third, given that EMEs which are prone to episodes of turbulent market dynamics, this study is particularly useful as we perform analysis across both turbulent and tranquil market conditions. These are Eurozone crisis and global financial crisis (EC-GFC) and post-crisis (PC). The EC-GFC period spans between 5 January 2007 and 7 June 2013 and the PC period spans between 10 June 2012 and 18 January 2019. The selected sub-samples are supported in the literature (see, Dimitrakopoulos et al. [38]; Mollah & Mobarek [39]). This provides a time-varying assessment of EMs risk dynamics and mitigates the problem of model misspecification. Further, we use different distributional innovations to fit the risk models to sidestep single model misspecification as well as improve forecasting performance (Bernardi & Catania [8]).

Four, our study provides an opportunity to classify EMs equities portfolios as well diversified or not. It can be inferred from the MCS procedure that equities which exhibit a large SSM size are homogeneous and those with small SSM sizes are heterogeneous. We find that SSMs with at least one half of the initial set of models to be homogeneous. Similarly, we take SSMs with less than one half of the initial set of models to be heterogeneous. Empirically, homogeneity in the SSM is suggestive of well diversified portfolios. This knowledge is critical to inform international investors in asset selection and risk management (see Bernardi & Catania [8]).

Our empirical results show that, for many of the equities, most of the six initial models selected make the SSM. The least number of members in the SSM recorded is three. However, the EM index has SSM of one across sub-periods and percentiles and hence the most heterogeneous. We also find that model ranks differ for many markets in the different sub-periods. This suggests the need to be mindful of market dynamics when modelling tail risk. Although six may not be a large number, large SSM is important for portfolio selection under different market conditions.

2. Theoretical Models and Empirical Methodology

This study partly reproduces the methods description of Owusu Junior & Alagidede [24]) since the section uses parts of the methods employed in that paper. The selected techniques appeal to the elicitability of the joint (VaR, ES) model (Gneiting [40]; Weber [41]). They permit model selection, estimation, forecast comparison and forecast ranking, etc. (Fissler et al. [4]).

2.1. Univariate GAS Model Specification

The relationship between the FZL function and the generalised autoregressive score (GAS) models (Creal et al. [42]; Harvey [43]) is exploited as they both use loss function as an impetus of time-variation in the parameters (Ardia et al. [44]).

Let $N\times 1$ vector $y_t$ be the dependent variable, $f_t$ the time-varying parameter vector, $x_t$ a vector of covariates, and $\theta$ a vector of static parameters; the filtration $\left\{f_t,F_t\right\}\in {ℱ}_t$ is given as

$${ℱ}_t=\left\{Y^{t-1},F^{t-1}, X^t\right\}, for t=1, \dots , n$$

(1)

where $Y^t=\left\{y_1,\dots ,y_t\right\}, F^t=\left\{f_0,f_1,\dots ,f_t\right\},$ and $X^t=\left\{x_1,\dots ,x_t\right\}$. Further, $y_t$ is generated by observation density and parameter vector $\theta$ driven by

$$y_t~p\left(y_t|f_t, {ℱ}_t;\theta \right)$$

(2)

as the score of the conditional distribution. $f_t$ is updated by an autoregressive mechanism in (3):

$$f_{t+1}=\omega +{\displaystyle \sum }_{i=1}^p{A}_i{s}_{t-i+1}+{\displaystyle \sum }_{j=1}^q{B}_j{f}_{t-j+1}, i=1, \dots , p, j=1, \dots , q,$$

(3)

where $A_i$ and $B_j$ are matrices of requisite dimensions, $\omega$ is a vector of constants and $s_t=s_t\left(y_t,f_t, {ℱ}_t;\theta \right)$ is the function of past data. The unknown parameters ($p+q+1)$ are all functions of $\theta$ ($\omega =\omega \left(\theta \right), A_i=A_i\left(\theta \right), B_i=B_i\left(\theta \right)$) evaluated at $r_t=ln{P}_t-ln{P}_{t-1},$ log-return in conventional notations). $f_t$ is updated to $f_{t+1}$ in (3) via (4) with a realised $y_t$ where $S(.)$ is a matrix.

$$s_t=S_t{\nabla }_t, {\nabla }_t=\frac{\partial lnp(y_t|f_t, {ℱ}_t;\theta )}{\partial f_t}, S_t=S\left(t,f_t, {ℱ}_t;\theta \right)$$

(4)

The Equations (2)–(4) define the GAS with orders $p$ and $q$. $GAS\left(p,q\right)$ gives dependence of the driving mechanism in (3) on the scaled score vector in (4). We use $GAS\left(1,1\right)$ in this study for all estimations (see Creal et al. [42]; Ardia et al. [44]). It may be worthwhile to compare the GAS models with generalised autoregressive conditional heteroscedasticity (GARCH) models of various dimensions, however, the outputs from our estimations in this study are overly voluminous (the tables presented are compressed) for brevity reasons. Further, we do not want deviate from the focus of the study by over-populating results. The GARCH models can be explored for further research purposes.

2.2. The FZL Function

For a forecast $x\in {ℝ}^k$ issued in the occurrence of the event $y\in {ℝ}^d$, the forecasting model is penalised by the real value $S\left(x,y\right)$, and $k, d$ are dimensions. Thus $S(.)$ is a squared error inversely relating forecast values and their original counterparts. Fissler and Ziegel [3] vouched for a strictly $ℱ$-consistent for functional $T\left(F\right)$ of a random variable $X$. This should be the unique minimiser of the expected loss ${\mathbb{E}}_F\left[S\left(x,y\right)\right]$ for every $F\in ℱ$ if the probability distribution of $ℱ$ is in the domain of $T$. Given a non-strictly consistent scoring function $S:\to {ℝ}^2\to ℝ$, for any random variable $X$ with finite mean, and for $\alpha \in \left(0,1\right)$, ES can be defined as

$$E{S}_{\alpha }\left(X\right)=\frac{1}{\alpha }{{\displaystyle \int }}_0^{\alpha }Va{R}_{\beta }\left(X\right)d\beta ,$$

(5)

where $Va{R}_{\alpha }\left(X\right)=\inf \left\{x\in ℝ:\mathbb{P}\left(X\le x\right)\right\}$. Unlike ES, VaR at $\alpha \in \left(0,1\right)$ has a strictly consistent scoring function of the form

$$S_V\left(v,x\right)=\left(\mathbb{I}\left\{x\le v\right\}-\alpha \right)\left(G\left(v\right)-G\left(x\right)\right),$$

(6)

and thus, elicitable with a unique $\alpha$-quantile, where $G$ is strictly increasing. Nonetheless, $\left(Va{R}_{\alpha }, E{S}_{\alpha }\right)$ is conjointly elicitable as shown in (7)

$$\left(Va{R}_{\alpha }\left(X\right), E{S}_{\alpha }\left(X\right)\right)=argmi{n}_{v,e\in {ℝ}^2}\mathbb{E}\left[S_{V,E}\left(v,\mathrm{e},X\right)\right],$$

(7)

with a scoring function, specifically the FZL function given in (8)

$$S_{V,E}\left(v,e,x\right)=\left(\mathbb{I}\left\{x\le v\right\}-\alpha \right)\left(G_1\left(v\right)-G_1\left(x\right)\right)+\frac{1}{\alpha }{G}_2\left(e\right)\mathbb{I}\left\{x\le v\right\}\left(v-x\right)+G_2\left(e\right)\left(e-v\right)-{\mathbb{G}}_2\left(e\right),$$

(8)

where $V, E, v$ and $e$ denote $Va{R}_{\alpha }, E{S}_{\alpha }$ and their corresponding estimates.

It is obvious from (7) that $G_1$ and $G_2$ are strictly increasing continuously differentiable functions such that $\mathbb{E}\left[G_1\left(X\right)\right]$ exists, $li{m}_{x\to -\infty }{G}_2\left(x\right)=0$, and ${\mathbb{G}}_2^{\prime }=G_2$ (Fissler et al. [4]). The second part of $S_{V,E}$ shows the dependence of $E{S}_{\alpha }$ on $Va{R}_{\alpha }$ to be elicitable. The FZL used in the GAS framework is the parameterisation of Patton et al. [10] by specifying the difference for two forecasts $(v_{1t}, e_{1t})$ and $(v_{2t}, e_{2t})$ as $L_{FZ}\left(X_t, v_{1t}, e_{1t};\alpha , G_1,G_2\right)-L_{FZ}\left(X_t, v_{2t}, e_{2t};\alpha , G_1,G_2\right)$. The resulting loss function FZ0 is given as

$$L_{FZ0}\left(X, v,e;\alpha \right)=\frac{1}{\alpha e}\mathbb{I}\left\{X\le v\right\}\left(v-X\right)+\frac{v}{e}+\log \left(-e\right)-1.$$

(9)

For (9) to be valid the strictly negative and homogeneous loss differences assumptions must hold for VaR and ES (see, Fissler et al. [4]; Fissler & Ziegel [3]; Patton et al. [10]).

2.3. The MCS Procedure

Because of elicitability of the (VaR, ES) model, the FZL function permits the MCS procedure to construct a SSM and ranks them based on a loss function that satisfies generic weak stationarity conditions (Bernardi & Catania [8]). We employ the Mariano and Preve [45] (MDM) as well as West [46] to examine the equal predictive ability (EPA) hypothesis. The MDM is the multivariate version of Diebold & Mariano [37] which is only for the bivariate case. Together with the MCS procedure, these fit well with the multiple competing models in this models in this study.

In formal terms let $r_t,{\widehat{r}}_{i,t}$ denote the log-return at time $t$ and the output of model $i$ at time $t$, respectively. Then the loss function

$${ℒ}_{i,t}=ℒ\left(r_t,{\widehat{r}}_{i,t}\right),$$

(10)

can be defined as the difference between ${\widehat{r}}_{i,t}$ and $r_t$. Following González-Rivera et al. [47] and Bernardi et al. [48], the FZL loss function (under MCS) can be defined as

$$\ell \left(r_t,FZ{L}_t^{\tau }\right)=\left(\tau -d_t^{\tau }\right)\left(r_t-FZ{L}_t^{\tau }\right),$$

(11)

where $FZ{L}_t^{\tau }$ signifies the $\tau$-level predicted FZL at time $t$, in the filtration ${ℱ}_{t-1}$, and $d_t^{\tau }=\mathbb{I}(r_t<FZ{L}_t^{\tau })$ is the $\tau$-level FZL loss function. The MCS algorithm begins with an all-encompassing initial set of alternative models of $m-$dimension, $M^0$ at a confidence level $\alpha$. It then builds $m^{*}\le M^0$ set models called the superior set model (SSM). The SSM, ${\widehat{M}}_{1-\alpha }^{*}$ comprises all models with superior predictive ability as per the selected loss function.

For model selection, let

$$d_{ij,t}={\ell }_{i,t}-{\ell }_{j,t}, i,j=1,\dots ,m, t=1,\dots ,n$$

(12)

denote the loss differential between models $i$ and $j$ at time $t$. Also let

$$d_{i.,t}={\left(m-1\right)}^{-1}{\displaystyle \sum }_{j\in M}{d}_{ij,t} i=1,\dots ,m,$$

(13)

denote the sample loss of model $i$ relative of another model $j$ at time $t$ and $M$ is the number of models. The hypothesis for EPA can be formulated as

$$\begin{array}{c}{H}_{o,M}=c_{i,j}=0,\forall i,j=1,\dots ,m,\hfill \\ H_{A,M}=c_{i,j}\ne 0,for somei,j=1,\dots ,n,\hfill \end{array}$$

(14)

where $c_{i,j}$ is the estimated loss of model $i$ relative of other set of models $j$. The required test statistic is

$$t_{i,j}=\frac{{\overline{d}}_{ij}}{\sqrt{\widehat{var}\left({\overline{d}}_{ij}\right)}}, for i,j\in M,$$

(15)

where ${\overline{d}}_{ij}=m^{-1}{{\displaystyle \sum }}_{t=1}^m{d}_{ij,t}$ denotes the relative sample loss between the $ith$ and $jth$ models, and $\widehat{var}\left({\overline{d}}_{ij}\right)$ is the bootstrapped estimate of $var({\overline{d}}_{ij})$. We use the MCS package in R (Bernardi & Catania [49]) to perform a block-bootstrap with 5000 re-samples. Significant parameters are obtained by fitting an autoregressive ($AR\left(p\right)$) process on all $d_{i,j}$ terms with a maximum block length $p$. Lastly, according to (Hansen et al. [9]), $H_{o,M}$ naturally fits into the test statistic:

$$T_{R,M}=\underset{i,j\in M}{\max }\left|t_{ij}\right|$$

(16)

For a complete description of the MCS procedure one may refer to (Bernardi & Catania [8,49]; Hansen & Lunde [34]; West [46]; White [32]; etc.).

With support from the literature that financial returns exhibit heavy-tails as well as volatility clusters (Cajueiro & Tabak [50]; McNeil & Frey [51]; McNeil et al. [52]), we select distributions that can capture these features. We use six different asymmetric distributions of fit (VaR, ES) in the GAS model. These are skewed-Gaussian (SNORM), student-t (STD), skewed-student-t (SSTD) (Fernández & Steel [53]); asymmetric student-t with two tail decay parameters (AST), asymmetric student-t with one tail decay parameter (AST1) (Zhu & Galbraith [54,55]); and asymmetric Laplace distribution (ALD) (Kotz et al. [56]). These distribution have been proven to work in the baseline GAS framework (Benardi & Catania [49]).

3. Data and Preliminary Analysis

We use the daily log-returns $r_t=ln{P}_t-ln{P}_{t-1}$, where $P_t$ and $P_{t-1}$ are index prices (in US dollars) at time $t$and $t-1$, respectively. Our data are the top nine (9) emerging markets (EMs) indices according to Morgan Stanley Capital International (MSCI) classification from 5 January 2007 to February 18, 2019. The top 9 EMs index country constituents are China (33%), South Korea (13.02%), Taiwan (11.35%), India (9.16%), Brazil (7.23%), South Africa (5.89%), Russia (3.77%), Mexico (2.65%) and Thailand (2.34%). All others take up the remaining 11.59%. The price data were gleaned from the Bloomberg Terminal. Our analysis span across Eurozone crisis and global financial crisis (EC-GFC) and post-crisis (PC). The EC-GFC period spans between 5 January 2007 and 7 June 2013 and the PC period span between 10 June 2012 and 18 January 2019, representing stressed and tranquil market periods, respectively. Our selection of sub-sample periods are informed by the fact that both the EZC and GFC affected EMEs alike (Cheung et al. [57]; Crotty [58]; Martin [59]; Mollah et al. [60]; Nguyen & Pontell [61]).

We follow a typical out-of-sample exercise (i.e., estimates are made using in-sample period of length $M$, predictions on the conditional distribution in the out-of-sample period $H,$ and model comparison made according to their out-of-sample performance). Hence $h$-step ahead prediction of the return distribution at time $M+h$ along corresponding (VaR, ES) model level are generated in a recursive manner in line with Marcellino et al. [62] until the end of the series $T$. We use $h=1$ for one-step ahead daily forecasting.

The estimation and forecasting windows are chosen as follows:

EC-GFC gets M = 1117 (5 January 2007–18 April 2011), H = 559 (19 April 2011–7 June 2013) and PC has M = 740 (21 November 2014–14 July 2017), H = 368 (22 July 2017–19 February 2019), where M denotes in-sample (IS) and H denotes out-of-sample (OOS) forecast lengths. The OOS is chosen such that it is a minimum of one trading year. The loss functions are also estimated at the 97.5 and 99 percentiles, a standard requirement by the Bank for International Settlements (BIS [63]).

Descriptive Statistics

In Figure 1 and Figure 2 we show both the price and log-return plots of our two sub-samples. We see that the fluctuations in the plots are typical of high frequency financial data of daily periodicity. They exhibit varied levels of volatility across sub-sample periods. Log-returns also exhibit volatility clusters which conform to the price fluctuations as expected. The summary statistics in

Table 1. Summary statistic of top 9 EMs equities.

	China	S. Korea	Taiwan	India	Brazil	S. Africa	Russia	Mexico	Thailand
EC-GFC periods
In-sample: 5 January 2007 to 18 April 2011
Mean	0.0003	0.0003	0.0001	0.0003	0.0005	0.0002	−0.0002	0.0001	0.0006
Variance	0.0005	0.0006	0.0003	0.0005	0.0008	0.0005	0.0009	0.0004	0.0004
Skewness	0.03	−0.13	–0.20	0.19	−0.34	−0.26	−0.40	0.02	−0.59
Excess kurtosis	5.02	17.11	2.40	6.89	7.39	4.31	13.79	6.46	6.17
Normtest.W*	0.94	0.86	0.96	0.94	0.90	0.95	0.85	0.91	0.94
Observations	1117	1117	1117	1117	1117	1117	1117	1117	1117
Out-of-sample: 19 April 2011 to 7 June 2013
Mean	−0.0003	−0.0002	−0.0001	−0.0004	−0.0008	−0.0003	−0.0007	0.000	0.0003
Variance	0.0002	0.0003	0.0002	0.0002	0.0003	0.0003	0.0003	0.0002	0.0002
Skewness	−0.077	−0.248	−0.167	0.030	−0.401	−0.082	−0.477	−0.533	0.034
Excess kurtosis	2.896	2.457	1.931	1.334	2.815	1.562	2.642	3.795	2.959
Normtest.W*	0.959	0.965	0.970	0.985	0.971	0.982	0.963	0.961	0.967
Observations	559	559	559	559	559	559	559	559	559
Post-crises period
In-sample: 10 June 2013 to 21 July 2017
Mean	0.0002	0.0001	0.0002	0.0002	−0.0003	0.0001	−0.0002	−0.0002	−0.0001
Variance	0.0002	0.0001	0.0001	0.0001	0.0004	0.0003	0.0004	0.0002	0.0002
Skewness	−0.17	−0.16	−0.15	−0.51	0.18	−0.25	−0.03	−0.60	−0.07
Excess kurtosis	3.07	1.45	2.31	4.22	1.82	2.94	7.26	4.97	3.87
Normtest.W*	0.96	0.98	0.97	0.95	0.98	0.97	0.93	0.96	0.95
Observations	990	990	990	990	990	990	990	990	990
Out-of-sample: 22 July 2017 to 18 February 2019
Mean	0.0003	0.0001	0.0001	0.0001	0.0004	−0.0002	0.0001	−0.0003	0.0004
Variance	0.0001	0.0001	0.0001	0.0001	0.0003	0.0003	0.0002	0.0002	0.0001
Skewness	−0.12	−0.36	−0.96	−0.37	−1.26	−0.22	−1.82	−0.57	−0.09
Excess kurtosis	0.57	1.71	6.98	0.87	12.68	1.21	17.03	3.08	2.23
Normtest.W*	0.99	0.98	0.93	0.99	0.92	0.99	0.91	0.97	0.96
Observations	497	497	497	497	497	497	497	497	497

Note: Normtest.W* indicate that normality is rejected at all levels of significance. S. Korea and S. Africa denote South Korea and South Africa, respectively.

and

Table 2. Summary statistic of EM index.

Mean	Variance	Skewness	Excess Kurtosis	Normtest.W*	Observations
EC-GFC periods
In-sample: 5 January 2007 to 18 April 2011
0.0002	0.0003	−0.4159	6.3206	0.92	1117
Out-of-sample: 19 April 2011 to 7 June 2013
−0.0003	0.0001	−0.3898	3.4102	0.96	559
Post-crises period
In-sample: 10 June 2013 to 21 July 2017
−0.0001	0.0001	−0.2912	1.9301	0.98	990
Out-of-sample: 22 July 2017 to 18 February 2019
0.0002	0.0001	−0.5911	1.0884	0.98	497

Note: Normtest.W* indicate that normality is rejected at all levels of significance.

show skewness and excess kurtosis values. This validates non-normality and leptokurtic behaviour in the equities. The mixture of positive (negative) skewness values indicate the possibility of positive (negative) returns are more than negative (positive) returns for the different equities between the IS and OOS periods. Leptokurtoses also suggest the presence of extreme returns across the board but of different magnitudes. These do not only suggest non-normality in the equities, but they also indicate time-varying dynamics. Further, the Shapiro–Wilk test of normality corroborate these as it rejects the Gaussian assumption for all significance levels. These support the need to use time-varying and asymmetric distributional approaches in modelling the tail risks of equity returns.

In

Table 3. SSM of univariate GAS (VaR, ES) model forecasts per market.

Model	RankR,M	ti	p-ValueR,M	Loss	Model	RankR,M	ti	p-ValueR,M	Loss
Eurozone and Global Financial Crises (EC-GFC) Periods:19 April 2011 to 7 June 2013
	Brazil 1%				Brazil 2.5%
SNORM	2	−1.98	1.00	−3.05	SNORM	3	−1.43	1.00	−3.16
STD	3	−1.50	1.00	−2.93	STD	1	−2.56	1.00	−3.16
SSTD	1	−2.23	1.00	−2.97	SSTD	2	−2.49	1.00	−3.13
AST	6	1.81	0.11	−2.51	AST	6	1.95	0.08	−2.89
AST1	5	1.81	0.11	−2.51	AST1	5	1.95	0.08	−2.89
ALD	4	−1.12	1.00	−2.96	ALD	4	−1.15	1.00	−3.13
p-value	0.105				p-value	0.084
	Mexico 1%				Mexico 2.5%
SNORM	2	−1.32	1.00	−3.17	SNORM	4	−0.77	1.00	−3.32
STD	4	0.13	1.00	−3.05	STD	2	−1.17	1.00	−3.33
SSTD	1	−2.96	1.00	−3.13	SSTD	1	−3.47	1.00	−3.32
AST	6	1.43	0.29	−2.93	AST	6	1.68	0.22	−3.21
AST1	5	1.43	0.29	−2.93	AST1	5	1.68	0.22	−3.21
ALD	3	−0.93	1.00	−3.15	ALD	3	−0.87	1.00	−3.32
p-value	0.292				p-value	0.223
	Russia 1%				Russia 2.5%
SNORM	2	−1.11	1.00	−2.69	SNORM	2	−0.21	1.00	−2.88
STD	3	1.08	0.44	−2.60	STD	3	0.54	0.83	−2.86
SSTD	4	2.06	0.06	−2.54	SSTD	4	1.89	0.10	−2.82
ALD	1	−1.50	1.00	−2.76	ALD	1	−1.78	1.00	−2.93
p-value	0.059				p-value	0.095
	South Africa 1%				South Africa 2.5%
SNORM	1	−0.96	1.00	−2.92	SNORM	3	1.54	0.18	−3.03
STD	3	0.56	0.86	−2.86	STD	2	−0.16	1.00	−3.07
SSTD	4	1.19	0.43	−2.85	ALD	1	−1.10	1.00	−3.10
ALD	2	−0.66	1.00	−2.93
p-value	0.425				p-value	0.180
	China 1%				China 2.5%
SNORM	1	−0.52	1.00	−3.06	SNORM	3	1.03	0.44	−3.21
STD	3	−0.27	1.00	−3.04	STD	2	−0.30	1.00	−3.24
SSTD	4	1.41	0.30	−2.98	ALD	1	−0.57	1.00	−3.25
ALD	2	−0.34	1.00	−3.05
p-value	0.300				p-value	0.443
	India 1%				India 2.5%
SNORM	4	0.23	0.97	−2.90	SNORM	4	0.86	0.62	−3.08
STD	1	−5.01	1.00	−3.17	STD	1	−5.55	1.00	−3.33
SSTD	2	−3.92	1.00	−3.08	SSTD	2	−4.11	1.00	−3.22
AST	6	2.17	0.06	−2.66	AST	6	2.62	0.02	−2.95
AST1	5	2.17	0.06	−2.66	AST1	5	2.62	0.02	−2.95
ALD	3	−1.31	1.00	−3.06	ALD	3	−2.66	1.00	−3.28
p-value	0.061				p-value	0.016
	South Korea 1%				South Korea 2.5%
SNORM	4	−0.86	1.00	−2.87	SNORM	4	−0.98	1.00	−3.05
STD	1	−2.95	1.00	−2.99	STD	1	−3.53	1.00	−3.14
SSTD	2	−1.73	1.00	−2.88	SSTD	3	−1.53	1.00	−3.04
AST	6	1.98	0.09	−2.52	AST	6	2.43	0.03	−2.81
AST1	5	1.98	0.09	−2.52	AST1	5	2.43	0.03	−2.81
ALD	3	−1.26	1.00	−2.95	ALD	2	−1.58	1.00	−3.09
p-value	0.087				p-value	0.031
	Taiwan 1%				Taiwan 2.5%
SNORM	3	−1.36	1.00	−3.12	SNORM	3	−1.26	1.00	−3.28
STD	1	−3.52	1.00	−3.08	STD	1	−4.68	1.00	−3.31
SSTD	4	−1.31	1.00	−2.89	SSTD	4	−1.21	1.00	−3.16
AST	6	2.03	0.06	−2.25	AST	6	2.50	0.02	−2.78
AST1	5	2.03	0.06	−2.25	AST1	5	2.50	0.02	−2.78
ALD	2	−2.02	1.00	−3.09	ALD	2	−3.08	1.00	−3.32
p-value	0.064				p-value	0.022
	Thailand 1%				Thailand 2.5%
SNORM	4	0.74	0.80	−2.96	SNORM	6	1.32	0.37	−3.17
STD	1	−4.27	1.00	−3.20	STD	1	−5.55	1.00	−3.41
SSTD	2	−2.49	1.00	−3.13	SSTD	2	−2.87	1.00	−3.34
AST	6	1.08	0.58	−2.93	AST	5	1.02	0.55	−3.20
AST1	5	1.08	0.58	−2.93	AST1	4	1.02	0.55	−3.20
ALD	3	−0.36	1.00	−3.07	ALD	3	−0.28	1.00	−3.28
p-value	0.584				p-value	0.367
		Post−crises (PC) period:22 July 2017 to 18 February 2019
	Brazil 1%				Brazil 2.5%
SNORM	2	−1.62	1.00	−2.59	SNORM	4	−0.85	1.00	−2.95
STD	4	−0.91	1.00	−2.57	STD	1	−3.30	1.00	−3.02
SSTD	1	−1.81	1.00	−2.60	SSTD	3	−1.09	1.00	−2.96
AST	6	2.17	0.06	−2.32	AST	6	2.05	0.06	−2.75
AST1	5	2.17	0.06	−2.32	AST1	5	2.05	0.06	−2.75
ALD	3	−0.93	1.00	−2.64	ALD	2	−1.14	1.00	−2.98
p-value	0.058				p-value	0.062
	Mexico 1%				Mexico 2.5%
SNORM	4	0.84	0.72	−2.91	SNORM	4	0.84	0.74	−3.26
STD	1	−3.26	1.00	−3.18	STD	1	−3.74	1.00	−3.41
SSTD	3	0.01	1.00	−3.02	SSTD	3	−0.71	1.00	−3.34
AST	6	1.11	0.54	−2.92	AST	6	1.45	0.35	−3.23
AST1	5	1.11	0.54	−2.92	AST1	5	1.45	0.35	−3.23
ALD	2	−1.09	1.00	−3.17	ALD	2	−1.62	1.00	−3.40
p-value	0.536				p-value	0.349
	Russia 1%				Russia 2.5%
SNORM	4	−0.32	1.00	−2.75	SNORM	4	0.26	0.97	−3.08
STD	2	−2.06	1.00	−2.82	STD	1	−3.94	1.00	−3.22
SSTD	1	−2.99	1.00	−2.82	SSTD	2	−3.24	1.00	−3.20
AST	6	1.20	0.40	−2.48	AST	6	1.03	0.54	−3.01
AST1	5	1.20	0.40	−2.48	AST1	5	1.03	0.54	−3.01
ALD	3	−0.56	1.00	−2.79	ALD	3	−0.61	1.00	−3.16
p-value	0.402				p-value	0.536
	South Africa 1%				South Africa 2.5%
SNORM	3	−0.77	1.00	−2.72	SNORM	3	0.01	1.00	−2.97
STD	1	−3.82	1.00	−2.81	STD	1	−3.48	1.00	−3.06
SSTD	4	−0.31	1.00	−2.63	SSTD	4	2.19	0.06	−2.87
AST	6	2.16	0.06	−2.37	ALD	2	−0.46	1.00	−2.99
AST1	5	2.16	0.06	−2.37
ALD	2	−1.49	1.00	−2.78
p-value	0.063				p-value	0.059
	China 1%				China 2.5%
SNORM	2	−0.91	1.00	−3.40	SNORM	4	−1.58	1.00	−3.50
STD	3	−0.15	1.00	−3.38	STD	1	−4.80	1.00	−3.55
SSTD	1	−1.12	1.00	−3.41	SSTD	3	−1.58	1.00	−3.50
ALD	4	1.38	0.26	−3.31	AST	6	2.78	0.01	−3.25
					AST1	5	2.78	0.01	−3.25
					ALD	2	−1.69	1.00	−3.48
p-value	0.264				p-value	0.010
	India 1%				India 2.5%
SNORM	2	−1.24	1.00	−3.53	SNORM	4	−0.72	1.00	−3.67
STD	1	−3.21	1.00	−3.59	STD	1	−3.11	1.00	−3.75
SSTD	3	−1.21	1.00	−3.52	SSTD	3	−0.80	1.00	−3.67
AST	5	1.92	0.09	−3.18	AST	6	1.95	0.09	−3.47
AST1	6	1.92	0.09	−3.18	AST1	5	1.95	0.09	−3.47
ALD	4	−1.10	1.00	−3.51	ALD	2	−1.95	1.00	−3.71
p-value	0.08				p-value	0.094
	South Korea 1%				South Korea 2.5%
SNORM	3	−0.40	1.00	−3.18	SNORM	3	−0.78	1.00	−3.42
STD	4	1.92	0.09	−3.02	STD	4	2.08	0.06	−3.30
SSTD	1	−1.77	1.00	−3.21	SSTD	2	−1.22	1.00	−3.42
ALD	2	−1.18	1.00	−3.26	ALD	1	−1.97	1.00	−3.48
p-value	0.093				p-value	0.063
	Taiwan 1%				Taiwan 2.5%
SNORM	4	−0.94	1.00	−3.04	SNORM	4	−1.42	1.00	−3.44
STD	1	−2.10	1.00	−3.13	STD	1	−2.61	1.00	−3.48
SSTD	3	−1.21	1.00	−3.06	SSTD	3	−1.49	1.00	−3.45
AST	5	1.73	0.14	−2.56	AST	6	2.07	0.06	−3.08
AST1	6	1.73	0.14	−2.56	AST1	5	2.07	0.06	−3.08
ALD	2	−2.06	1.00	−3.15	ALD	2	−1.97	1.00	−3.48
p-value	0.140				p-value	0.065
	Thailand 1%				Thailand 2.5%
SNORM	4	−0.24	1.00	−3.55	SNORM	4	1.05	0.47	−3.67
STD	1	−4.42	1.00	−3.75	STD	1	−5.34	1.00	−3.95
SSTD	2	−2.80	1.00	−3.64	SSTD	2	−2.61	1.00	−3.83
AST	6	1.52	0.19	−3.26	AST	6	1.44	0.24	−3.63
AST1	5	1.52	0.19	−3.26	AST1	5	1.44	0.24	−3.63
ALD	3	−0.86	1.00	−3.60	ALD	3	−1.06	1.00	−3.81
p-value	0.188				p-value	0.245
Eurozone and Global Financial Crises (EC-GFC) Periods:19 April 2011 to 7 June 2013
	EM index 1%				EM index 2.5%
ALD	1	−6.03	1.00	−0.02	ALD	1	−5.84	1.00	−0.02
p-value	0.000				p-value	0.000
Post−crises (PC) period:22 July 2017 to 18 February 2019
	EM index 1%				EM index 2.5%
ALD	1	−6.00	1.00	−0.024	ALD	1	−5.92	1.00	−0.02
p-value	0.000				p-value	0.000

Note: (VaR, ES) models are of the GAS(1,1) specification. SNORM, STD, SSTD, ALD, AST and AST1 denote Skewed Gaussian, Student t, Skewed Student t, asymmetric Laplace, asymmetric student-t with two tail decay parameters and asymmetric student-t with one tail decay parameter innovations in the models, respectively. Eliminated models are not listed since they can easily be inferred from the SSM listed. t_i denotes the test statistic derived from sample loss of the $ith$ model relative to the average across models in $M^{*}$ (Hansen et al. [9]). Values in bold print are overall p-value for the respective SSM (selected at the 95% confidence level).

we present the SSM from MCS algorithm for both 1% and 2.5% (VaR, ES) forecasts. Individual p-values can be interpreted as the level of belongingness in the SSM. Similarly, overall p-values indicate that the SSM contains true models with a probability no less than $1-\alpha$ ($\alpha =0.01$ for both 1% and 2.5% (VaR, ES) forecasts). All the p-values support the belongingness of the models in the SSM at all levels of significance. However, it is interesting to note that the magnitude of p-values do not correspond with the size of SSMs. It implies that the number of true models (in SSM) does not have to be large for the respective equity risk to be sufficiently modelled.

Though the SSM shows ranking of the models, we focus on the number of models it contains. This provides insight into the extent of heterogeneity and/or homogeneity among the risk models for a specific equity. If SSM contains a larger portion of $M^0$ then the competing models are statistically significant with respect to the forecast capability of (VaR, ES) for the particular equity, and vice versa (Bernardi & Catania [8]). It is worth noting that the models in the SSM are those characterised by very strong non-linear dynamics for their conditional volatility process.

Our SSMs reveal that there seems not to be any general distinct differences in the model heterogeneity/homogeneity during turbulent market (EC-GFC periods) and tranquil times (PC period) conditions for both 97.5 and 99 percentiles. The (VaR, ES) model in the SSM range between six (6) and three (3) for both EC-GFC and PC periods. We note that 6 is the initial set $M^0$. Thus, for those equities, all the initial models, SSM which suggest a great deal of model homogeneity. Equities with all six models in the SSM during the turbulent market (EC-GFC) periods are Brazil, Mexico, India, Korea, Taiwan and Thailand. However, in the tranquil (PC) period, they are Brazil, Mexico, Russia, India, Taiwan and Thailand. These drive home the point that, except for Russia and South Korea, tail risk modelling in the EMs equities may be indifferent to market dynamics and thus time-invariant. They exhibit homogeneity (with respect to the size of SSMs) between the two market periods. We find that Russia has SSM of size four (4) in during EC-GFC but 6 in PC period. South Korea exhibits the exact opposite of Russia (SSM of size 6 in EC-GFC and size 4 in PC period).

China and South Africa also present different patterns. First, in EC-GFC periods the two markets have smaller SSMs of size 4 (99% level) and 3 (97.5% level). Compared to $M^0$, these countries show model heterogeneity. In the PC, South Africa has 6 models (at the 99% level) and 3 models (at the 97.5% level), whereas China has 4 models (at the 99% level) and 6 models (at the 97.5% level). These indicate that in this period, while South Africa demonstrate model homogeneity at the 99th percentile, but for China it is at the 97.5th percentile. In essence, during the PC period, China and South Africa portray percentile-dependent tail risk model dynamics. But we also can confirm time-dependent tail risk modelling in the two markets because of the difference in SSMs between EC-GFC and PC periods.

The implications for tail risk modelling in China and South Africa may not be trivial. We can infer that it is more difficult to model tail risks in these equities as compared to the other seven markets. This is because, not only should specific market dynamics be considered, but risk managers may be wary of specific confidence levels. The pattern between China and South Africa may be explained by the composition of their respective equity indices. For instance, Tencent Holdings Limited, a Chinese Information Technology company, is first of the top ten (10) constituents of the MSCI EMs index. The South African giant, Naspers Limited (5th of the top 10 MSCI EMs index constituents) owns 31.1% of Tencent Holdings Limited (Morgan [64]). It is, therefore, no surprise that the two markets exhibit similar tail risk features.

For the EM index, we find only the SSM contains the ALD for both 1% and 2.5% levels and sub-periods. This confirms the superiority of the ALD as suggested by Taylor [36]. It also shows that EM index exhibits the most heterogeneous set and thus not a diversified portfolio. The reason for the glaring difference between the SSMs for EM index and the individual equities may be attributed to the size of the data. It sends a clear message that the risk dynamics of the index is different from the constituent equities and portfolio decisions should take this into account.

We surmise from the results that the ease/difficulty with which to model tail risks does not depend on the actual conditions of the financial markets for an equity except for China and South Africa. Bernardi & Catania [8] made similar remarks in using the MCS algorithm to compare VaR models of four major global stock indices from Asia, North America and Europe. It follows that risk managers would prefer a SSM with a size closer to $M^0$ to those that are smaller. The latter shows models are statistically equivalent in their (VaR, ES) forecasting abilities and thus reduce the task of finding the singular best model. Moreover, given that SSMs are ranked, risk managers are at the same time, at ease of choosing the best ranked model for their equities.

Furthermore, that many of the SSMs are independent of market conditions is instructive for tail risk modelling for those equities. That is to say each EM equity may have to be subjected to the rigour of tail risk modelling, irrespective of the market condition. This brings to the fore an important difference and/or similarity amongst emerging markets. We evidence that despite the fact that EMEs are bracketed into one market class, the tail risks in their equities could not be more different. As a consequence, we suggest that portfolio diversifications benefits involving EMs equities are promising given the right combinations.

Empirically, the homogeneity in the SSM is suggestive of well diversified portfolios for the respective equity, given that different distributional assumptions are applied to the returns. This applies to all the equities with the exception of China and South Africa. Bernardi & Catania [8] opine that, the fact that diversified portfolios are characterised by inversely related risks and returns properties, so does diversification mitigate against negative and positive tail events that affect conditional distribution and kurtosis of equity returns.

It is worthwhile to note that the specific distributional innovations in each SSMs bear some important implications. Different distributional innovations are employed to capture the complex and unpredictable data-generating process in equities. These are reflected in the tails of distributions and are largely dependent on market dynamics which are also time-dependent. Hence, fitting a single distribution to the data is at a high risk of model misspecification as suggested in Bernardi & Catania [8]. It is clear from Table 3 that is it only the EM index where a single distributions fits that data across the different market periods, yet at different loss levels. It is almost impossible to hold down the exact reason why the model fits better with different distributions between the two periods and at what levels of predictive ability. The important signal is that, like many equities, the EM equities exhibit varying price and return dynamics which can be captured by using many distributional of diverse tail properties.

4. Conclusions and Recommendations

In this paper we constructed different sets of models for (VaR, ES) based on the FZL function to examine the forecast performance of the top 9 EMs equity returns. Our study period was between 5 January 2007 and 19 February 2019, further split into EC-GFC and PC periods. The superior set model (SSM) for each equity was constructed using the MCS procedure of Hansen et al. [9]. Among other things, it is appropriate to quantify, rank and select competing risks models. In this study, we fit different asymmetric distributional innovations in the GAS framework to the (VaR, ES) to forecast risks in the EMs equities. We combine the elicitability the joint (VaR, ES) scored by the FZL function, its richness in drawing from both VaR and ES and the ability of MCS to rank models to select the superior set of tail risk models.

Not only does this work contribute to growing need to correctly model and forecast tail risk for internal risk management purposes, but it also fits well with the Basel III framework for comparative backtesting. We also use both the 99 and 97.5 percentiles in the risk forecasting as suggested in the Basel III framework. This is a new study for EMs equities as far as risk analysis is concerned. The practice of aligning internal models with standardised approaches improves regulatory oversight and hence may bolter confidence in international investors about EMs equities.

Our results show that for many of the markets, risk models are time-invariant (with respect to the number of models in the SSM, but not with the rank of the models) between turbulent and tranquil market periods. The SSMs are of size 6 which is equal to the original set of models. This implies homogeneity in risk modelling and hence reduces the burden of risk managers. We opine that, this hints at usefulness of the MCS procedure in choosing from a large number of models with ease. This further implies that the MCS procedure offers more flexibility in modelling, forecasting and comparing competing tail models. However, for China and South Africa, the risk models are more time-varying, percentile-dependent and more heterogeneous than homogeneous. We infer that the pattern in China and South African may be attributed to their respective equity index compositions. We find that the largest equity in South Africa Naspers Limited own 31.1% stake in Tencent Holdings Limited which is the largest Chinese equity and the biggest constituents of the MSCI EMs index (Morgan [64]).

Empirically, the homogeneity in the SSM is suggestive of well diversified portfolios for the respective equity, given that different distributional assumptions are applied to the returns. This applies to all the equities with the exception of Russia (in EC-GFC periods), South Korea (in PC period) and China and South Africa (in both periods). Bernardi & Catania [8] opine that, the fact that diversified portfolios are characterised by inversely related risks and returns properties, so does diversification mitigate against negative and positive tail events that affect conditional distribution and kurtosis of equity returns.

For further research, GAS volatility models can be compared with GARCH with asymmetric/leverage effects. Specifically, the robust bootstrap forecast densities for GARCH returns and volatilities (Trucíos et al. [65]) can be applied on the data. In addition, it would be interesting to see the dynamics of the data under the semi-nonparametric backtesting techniques applied with VaR, ES and Median Shortfall (see Jiménez et al. [66]). Finally, while this study captures two crises periods, it could further be expanded to reflect the effects of the ongoing COVID-19 pandemic of EMs equities and risks.