Meta-Learning Approaches for Recovery Rate Prediction

Abstract

While previous academic research highlights the potential of machine learning and big data for predicting corporate bond recovery rates, the operations management challenge is to identify the relevant predictive variables and the appropriate model. In this paper, we use meta-learning to combine the predictions from 20 candidates of linear, nonlinear and rule-based algorithms, and we exploit a data set of predictors including security-specific factors, macro-financial indicators and measures of economic uncertainty. We find that the most promising approach consists of model combinations trained on security-specific characteristics and a limited number of well-identified, theoretically sound recovery rate determinants, including uncertainty measures. Our research provides useful indications for practitioners and regulators targeting more reliable risk measures in designing micro- and macro-prudential policies.

1. Introduction

Credit risk is the main concern for financial institutions. It is governed by three factors: the exposure at default (EAD), the default likelihood (that is, the probability of default, PD) and the loss given default (LGD). The latter can also be expressed as $(1-RR)$, where the recovery rate $RR$ represents the percentage of EAD that can be recovered in the event of default of the reference entity. For a long time, financial institutions focused on the modeling of PDs. Ratings, for example, essentially quantify the risk of a default event, the loss severity receiving little attention. However, EAD and $RR$ are key drivers of credit risk. For instance, they act as scaling constants when computing banks’ regulatory capital requirements (BCBS 2017; Loterman et al. 2012; Zhang and Thomas 2012) in Basel II (BCBS 2006) and Basel III frameworks (BCBS 2011) that banks can estimate using internal models in the advanced (A-IRB) approach. But $RR$ also influences the value of non-performing loans (Bellotti et al. 2021), corporate bonds and credit derivatives (Andersen and Sidenius 2004; Berd 2005; Gambetti et al. 2018; Pykthin 2003), as well as the price of mainstream OTC derivatives’ products such as equity options and interest rates swaps due to counterparty risk (Gregory 2012). This explains why recovery rates modeling and forecasting receive more and more attention in current times.

Academic research highlights how nonlinear techniques (Loterman et al. 2012; Nazemi and Fabozzi 2018; Nazemi et al. 2018; Qi and Zhao 2011; Tobback et al. 2014; Yao et al. 2015) and ensemble methods (Bastos 2014; Bellotti et al. 2021; Hartmann-Wendels et al. 2014; Nazemi et al. 2017) should be preferred to the traditional parametric regressions used in earlier studies on recovery rate determinants. This finding is not confined only to RR forecasting, but it is also in credit scoring (Liu et al. 2022; Wang et al. 2022). However, the possibility of combining predictions from a large number of different algorithms remains widely unexplored. This is surprising given that it is now well recognized that combining models might be preferable (Atiya 2020) to selecting a single model. In this paper, we confirm this conclusion by showing that meta-learning techniques exhibit better forecasting performance and lower model risk than methods in previous research when dealing with $RR$.1

Meta-learning can be defined as the act of a) selecting and b) optimally combining the outputs of multiple learning machines (first-level learners), according to some combination scheme that is learned from the data (meta-learner or second-level learner) (Santos et al. 2017). Meta-learning is very flexible: the models to be combined can be represented by expert judgment, individual learners (e.g., lasso, MARS) or ensemble learners (e.g., random forests, boosted trees), whose predictions themselves are already combinations of the outputs of different models. Furthermore, the models to combine can also be trained with different predictors. This contrasts with homogeneous ensemble methods in which, despite the model combinations that are involved, they generally combine weak learners2 of the same type, e.g., regression trees are combined into a random forest.

The main advantage of meta-learning is, hence, the ability to exploit a wide spectrum of functional forms. Its promising applicability for recovery rate prediction was first reported in Nazemi et al. (2017) using a fuzzy decision fusion approach. However, several other alternatives are worth exploring. For example, Roccazzella et al. (2021) propose a robust and sparse combination method that mitigates estimation uncertainty and implicitly features forecast selection in a single step. This approach has the additional advantage of relying on a closed-form solution for the estimation of the optimal combination strategy.

The quality of the forecasts depends also on the predictive variables we consider. Prior studies on defaulted bonds highlight the importance of security-specific characteristics (Altman and Kishore 1996; Bris et al. 2006; Jankowitsch et al. 2014; Schuermann 2004), economic conditions and the credit cycle (Acharya et al. 2007; Altman et al. 2005; Betz et al. 2018, 2020; Bruche and González-Aguado 2010).3 Models based on big data and variable selection techniques have been proposed by Nazemi et al. (2017, 2018) and Nazemi and Fabozzi (2018). Gambetti et al. (2019) show that economic uncertainty is the most important systematic determinant of recovery rate distributions. However, the latter study is based on an ex post analysis. Given that uncertainty shapes the economic outlook (ECB 2009, 2016; Gieseck and Largent 2016; Kose and Terrones 2012) and its proxies are particularly capable of anticipating economic downturns (Ludvigson et al. 2019), it remains to be explored whether uncertainty proxies can also improve predictive performance.

In this paper, we extend the literature on recovery rate modeling studying the forecasting performance of linear, nonlinear, rule-based and meta-learning algorithms across different specifications of the predictors set.

We start our analysis by studying whether a larger set of macro-financial indicators, uncertainty measures and idiosyncratic features in bond recovery rate forecasting models offers a significant advantage compared to a more parsimonious but economically grounded framework. Specifically, we extend the set of macroeconomic predictors used in Nazemi et al. (2017) and Nazemi et al. (2018) with 55 pricing factors and industry portfolio returns. We also enlarge the spectrum of uncertainty proxies considered by Gambetti et al. (2019) with 11 additional measures of economic uncertainty derived from text-analysis techniques. This offers two advantages. First, as Jurado et al. (2015) point out, text-based indexes can show significant independent variations from other popular uncertainty proxies, suggesting that much of the variation in these proxies is not driven by uncertainty itself. Second, textual data is more granular, allowing for the identification of the categories of uncertainty, e.g. government spending uncertainty or monetary policy uncertainty, that better predict recovery rates. In contrast to previous studies (Nazemi and Fabozzi 2018; Nazemi et al. 2017, 2018), we find that more parsimonious models that rely on well-documented recovery rate determinants outperform those making use of the entire set of predictors and those relying on data-driven variable selection. A limited number of economically grounded predictors also makes the model easier to implement and manage. Among those, uncertainty measures are particularly relevant for recovery rate prediction. Moreover, it makes its results more transparent, hence, more prone to regulatory validation. This is consistent with the regulatory requirements of IRB approaches (BCBS 2006, 2017).

Second, we provide the largest benchmark study of machine learning methods in the context of bond recovery rate prediction. We consider a total of 20 predictive algorithms, and we obtain similar conclusions to those of Bellotti et al. (2021) in the case of recovery rates for nonperforming loans: random forests, quantile random forests, boosted trees and cubist display the most promising performance and seem to be the best class of models. Bagged MARS and model-averaged neural networks also seem to be competitive.

Third, we empirically show that meta-learning can be used to improve recovery rate predictions compared to traditional machine learning methods while considerably reducing model risk. This evidence is preserved both when looking at the predictive performance within a chosen predictor set and when jointly considering predictions across all specifications of the predictor set. The proposed methods outperform competitive combining methods such as the equally weighted forecast and the hill-climbing algorithm of Caruana et al. (2004), which showed promising results in the field of credit risk scoring (for more details, see Lessmann et al. 2015). Furthermore, despite the main concern of this paper, which is recovery rate predictions, the idea of combining heterogeneous models trained using diverse predictors sets is not specific to credit risk modeling, and it can be used to hedge model uncertainty across the whole field of management science.

The remainder of the paper is structured as follows. Section 2 contains an overview of the machine learning algorithms involved in our meta-learning approach and explains the latter. Section 3 provides a description of the data. Section 4 describes the main results of our benchmark study. Section 5 discusses the practical implications implied by our results. Section 6 concludes the paper.

2. Methodology

An overview of our predictive strategy can be found in Figure 1. After specifying the bond recovery rate predictors, first-level learners are trained to minimize the mean square error (MSE). Subsequently, the fitted residuals of the first-level learners (meta-data) become the input of second-level learners (meta-learners), which combine the original models with the goal of making the aggregate forecast error variance as small as possible. Finally, we evaluate the predictive performance of the various classes of linear, nonlinear, rule-based and meta-learning methods.

We briefly review the predictive algorithms and combination strategies that will serve as first- and second-level learners, respectively.

2.1. First-Level Learners

To undertake an unbiased benchmark study, the spectrum of competing models should be as rich as possible. This requirement has rarely been respected in the context of recovery rate modeling, except for the large-scale benchmark studies of Loterman et al. (2012) and Bellotti et al. (2021).

For comparison purposes, we used a similar set of techniques as Bellotti et al. (2021), who employ 20 algorithms belonging to three different classes: linear, nonlinear and rule-based.

We provide the list in

Table 1. List of first-level learners and corresponding R algorithms. The three blocks of table identify linear (top), nonlinear (middle) and rule-based (bottom) models. Note that we consider two versions of ridge and lasso: (i) the standard one involving the penalty term associated with the best-in-sample performance and (ii) that based on the one-standard-error rule of Hastie et al. (2009).

Description	Acronym	R Algorithm	Reference
Linear regression	lm	lm	R Core Team (2017)
Backward step-wise selection	lm_bs	leaps	Lumley (2017)
Ridge regression	ridge	glmnet	Friedman et al. (2019)
Lasso regression	Lasso	glmnet	″
Elastic net regression	elnet	glmnet	″
MARS	mars	earth	Milborrow (2018)
Bagged MARS	bmars	earth	″
Model-averaged neural networks	avnnet	nnet	Ripley and Venables (2016)
Support vector regression	svr	ksvr	Karatzoglou et al. (2004)
Relevance vector regression	rvm	rvm	″
Gaussian processes	gauss	gausspr	″
Regression trees	cart	rpart	Therneau et al. (2017)
Conditional inference trees	cit	ctree	Hothorn et al. (2006)
Boosted tree	bst	bst	Wang (2018)
Stochastic gradient boosting	gbm	gbm	Greenwell et al. (2018)
Random forests	rf	randomForest	Liaw and Wiener (2002)
Quantile random forests	qrf	quantregForest	Meinshausen (2017)
Cubist	cubist	cubist	Kuhn and Quinlan (2018)

together with the corresponding R implementations. For more details on nonlinear and rule-based methods, we refer to Appendix A.4

2.1.1. Linear Models

We consider seven linear models with and without penalization. Following Bellotti et al. (2021), they can be framed as the following minimization problem:

$$\underset{{\mathit{\beta }}_0,\mathit{\beta }\in {\mathbb{R}}^p}{arg\; min}{\parallel \mathit{y}-\mathit{X}\mathit{\beta }-{\mathit{\beta }}_0\parallel }_2^2+\lambda \left({(1-\alpha )\parallel \mathit{\beta }\parallel }_2^2+\alpha {\parallel \mathit{\beta }\parallel }_1\right)$$

(1)

where $\mathit{y}$ denotes the vector of the observations in the sample, $\mathit{X}$ is the N-by-p matrix of predictors and $\mathit{\beta }$ stands for the vector of regression coefficients. Different specifications of the penalty term $\lambda \ge 0$ and the mixing factor $0\le \alpha \le 1$ yield the standard linear regression model (with and without backward selection), ridge, lasso and elastic net. Models of these types can only reproduce linear relationships such as those reproduced in Figure 2.

2.1.2. Nonlinear Models

We dealt with six types of nonlinear models. Among the kernel methods, we considered support vector regression (SVR), relevance vector machines (RVM) regression and Gaussian processes. We further considered multivariate adaptive regression splines (MARS) and two nonlinear ensembles: model-averaged neural networks and bagged MARS. We refer the reader to Bellotti et al. (2021) for an extended treatment of each of them. These models are naturally suited to capture nonlinear relationships, but they can be prone to overfit. An example of nonlinear fit is included in Figure 2.

2.1.3. Rule-Based Models

According to Gambetti et al. (2019), rule-based methods are able to identify clusters of data with similar properties and to reproduce step-like relationships (with or without slopes depending on the particular algorithm). We included seven types of rule-based methods. Individual models include regression trees and conditional inference trees, while ensembles are represented by cubist, random forests, quantile random forests and boosted trees with and without stochastic gradient boosting. An example of rule-based model fit is included in Figure 2.

2.2. Second-Level Learners

Starting from m predictions ${\widehat{\mathit{y}}}_1,\dots ,{\widehat{\mathit{y}}}_m$ of the random variable y returned by m first-level learners, two strategies can be adopted. The standard procedure consists of selecting the best forecast, say $\widehat{\mathit{y}}:={\widehat{\mathit{y}}}_{i^{\star }}$, where model $i^{\star }$ is selected according to some criteria. Alternatively, these predictions can be aggregated to form a single prediction $\widehat{\mathit{y}}:=\varphi ({\widehat{\mathit{y}}}_1,\dots ,{\widehat{\mathit{y}}}_m)$ using some function $\varphi$. Model combination offers diversification gains that makes it attractive when we cannot identify ex ante the best single model.5 In addition, in the rare cases where the best model can be identified, meta-learning techniques could still take full advantage of the available information when the first-level learners rely on various data sources or cover a wide spectrum of modeling assumptions.

Meta-learning learns the combination strategy $\varphi$ directly from the data with the explicit goal of minimizing a loss function. Precisely, let $\mathit{y}$ be an n-dimensional column vector containing n observations of the target variable and $\widehat{\mathit{Y}}$ be an n-by-m matrix of m unbiased candidate forecasts of $\mathit{y}$. The optimal combination strategy consists of estimating the function $\varphi (\widehat{\mathit{Y}})$ that solves

$${\varphi }^{*}:=\underset{\varphi \in \mathsf{\Phi }}{arg\; min}\parallel \mathit{y}-\varphi (\widehat{\mathit{Y}}){\parallel }_2^2,$$

(2)

where $\mathsf{\Phi }:={\mathbb{R}}^n\times {\mathbb{R}}^m\to {\mathbb{R}}^n$ is some conformable functional space.

Nevertheless, its success depends on how accurately the combination strategy can be determined. The use of in-sample data both to train the individual models and to consecutively combine them on the basis of their respective fitted residuals can significantly amplify the initial estimation error and consequently produce poor out-of-sample predictions.

Meta-learning relies on validation techniques to assess how well the combination weights will generalize to the out-of-sample predictive exercise. We opt to combine by minimizing the variance of the aggregate prediction error and include robust combination methods to hedge against potential instability.6

2.2.1. Linear Meta-Learners

Restricting ourselves to the class of linear combinations with weight summing to one, i.e., for $\mathsf{\Phi }$ being the set of functions from ${\mathbb{R}}^m\times {\mathbb{R}}^n\to \mathbb{R}$ of the form $\varphi (x_1,\dots ,x_m):={\sum }_{i=1}^m{\mathit{w}}_i{x}_i$ satisfying ${\sum }_{i=1}^m{\mathit{w}}_i=1$ leads to a combined prediction of the form $\widehat{\mathit{y}}={\sum }_{i=1}^m{\mathit{w}}_i{\widehat{\mathit{y}}}_i$. In this case, the optimization problem becomes

$${\mathit{w}}^{*}:=\underset{\mathit{w}\in \mathcal{W}}{arg\; min}\parallel \mathit{y}-\widehat{\mathit{Y}}\mathit{w}{\parallel }_2^2\mathrm{where}\mathcal{W}:=\{\mathit{w}\in {\mathbb{R}}^m|{\mathbf{1}}^T\mathit{w}=1\}.$$

(3)

Constraining the weights to sum to 1 keeps the aggregate prediction unbiased provided that the candidate forecasts are also unbiased (which is the case in this paper). By additionally constraining the weights to be nonnegative, this approach corresponds to Breiman’s stacked regressions (Breiman 1996), where linear combinations of different predictors are considered to further improve prediction accuracy. Granger and Ramanathan (1984) show that this approach is equivalent to selecting the weights ${\mathit{w}}^{*}$ that minimize the variance of the meta-learner’s prediction error.

Nevertheless, this combination strategy can lack robustness when the covariance matrix of prediction errors is poorly estimated. This often occurs because of sample size limitations, considerable background noise or when first-level forecasts are highly collinear (Claeskens et al. 2016). We can tackle the potential instability of ${\mathit{w}}^{*}$ by using the linear shrinkage estimator of the covariance matrix of prediction errors, which we denote by ${\mathsf{\Sigma }}_{\lambda }$. The latter consists of combining the sample covariance matrix (which is easy to compute, asymptotically unbiased but prone to estimation errors) with an estimator that is misspecified and biased but more robust to estimation errors (Ledoit and Wolf 2004). This approach, initially derived in a portfolio optimization context, was recently transposed in Roccazzella et al. (2021) to the forecast combination problem, showing that constrained optimization with shrinkage (COS) of $\mathsf{\Sigma }$ can provide a single-step, fast and robust optimal forecast combination strategy. Here we adapt the COS to act as a linear meta-learner, which combines the first-level learners only on the basis of in-sample information. This leads to the following optimization problem:

$${\mathit{w}}_{\lambda }^{*}:=\underset{\mathit{w}\in {\mathcal{W}}^{+}}{arg\; min}{\mathit{w}}^T{\mathsf{\Sigma }}_{\lambda }\mathit{w}\mathrm{where}{\mathsf{\Sigma }}_{\lambda }:=\left(1-\lambda \right)\mathsf{\Sigma }+\lambda \overline{\mathsf{\Sigma }},$$

(4)

where $\lambda \in [0,1]$ is the shrinkage intensity, $\mathsf{\Sigma }$ is the sample covariance matrix of prediction errors and $\overline{\mathsf{\Sigma }}$ is a predetermined reference covariance matrix. We estimate the optimal shrinkage intensity by minimizing the expected Frobenius norm of the difference between ${\mathsf{\Sigma }}_{\lambda }$ and the population covariance matrix of prediction errors $\mathit{S}$ (Ledoit and Wolf 2004).7

$${\lambda }^{*}:=\underset{\lambda \in [0,1]}{arg\; min}\mathbb{E}{∥{\mathsf{\Sigma }}_{\lambda }-\mathit{S}∥}_2^2.$$

(5)

We consider two shrinkage directions for $\overline{\mathsf{\Sigma }}$.

Definition 1 (COS-E—Constrained Optimization with Shrinkage towards Equal weights).

The target covariance matrix ${\overline{\mathsf{\Sigma }}}_{COS-E}$ corresponds to the case where first-level learners are assumed to have identical prediction error variance ${\overline{\sigma }}^2$ and identical pairwise correlation coefficients $\overline{\rho }$:

$${\overline{\mathsf{\Sigma }}}_{COS-E}:={\overline{\sigma }}^2\left[\begin{array}{cccc}1& \overline{\rho }& \cdots & \overline{\rho }\\ \overline{\rho }& 1& \ddots & ⋮\\ ⋮& \ddots & \ddots & \overline{\rho }\\ \overline{\rho }& \cdots & \overline{\rho }& 1\end{array}\right].$$

(6)

In practice, we set ${\overline{\sigma }}^2=\frac{1}{m}{\sum }_{i=1}^m{\sigma }_i^2$, where ${\sigma }_i^2$ is the sample prediction error variance of model i (the average error prediction variance in the set of first-level learners) and by $\overline{\rho }=\frac{2}{m(m-1)}{\sum }_{i=1}^{m-1}{\sum }_{j=i+1}^m{\rho }_{i,j}$ (the average correlation coefficient of first learners’ in-sample prediction error).

Definition 2 (COS-IL—Constrained Optimization with Shrinkage towards Inverse Loss-based weights).

The target covariance matrix ${\overline{\mathsf{\Sigma }}}_{COS-IL}$ corresponds to the case where first-level learners have an identical pairwise correlation coefficient $\overline{\rho }$, but their respective prediction error variance is estimated using sample data:

$${\overline{\mathsf{\Sigma }}}_{COS-IL}:=\left[\begin{array}{cccc}{\sigma }_1^2& \overline{\rho }{\sigma }_1{\sigma }_2& \cdots & \overline{\rho }{\sigma }_1{\sigma }_m\\ \overline{\rho }{\sigma }_2{\sigma }_1& {\sigma }_2^2& \ddots & ⋮\\ ⋮& \ddots & \ddots & ⋮\\ ⋮& \ddots & {\sigma }_{m-1}^2& \overline{\rho }{\sigma }_{m-1}{\sigma }_m\\ \overline{\rho }{\sigma }_m{\sigma }_1& \cdots & \overline{\rho }{\sigma }_m{\sigma }_{m-1}& {\sigma }_m^2\end{array}\right].$$

(7)

As benchmarks, we also include the equally weighted forecast, i.e., $\varphi (x_1,\dots ,x_m):={\sum }_{i=1}^m\frac{1}{m}{x}_i$ and the hill-climbing algorithm of Caruana et al. (2004). In this latter case, forward step-wise selection is used to identify the equally weighted combination that minimizes the root mean square error (RMSE) in a validation fold sampled from the training data.

2.2.2. Nonlinear Meta-Learners

Thus far, we have described linear weighting schemes. Nevertheless, meta-learning is more general. For example, in the field of image classification, meta-learning techniques typically involve nonlinear methods such as deep learning or recurrent models (Santoro et al. 2016), metric learning (Koch et al. 2015) and learning optimizers (Ravi and Larochelle 2017). Despite being more flexible than linear methods, nonlinear meta-learners are also more complex and prone to overfit, especially in a relatively small data set.8 For these reasons, we opt for ensembles of shallow (one hidden layer) feed-forward neural networks (NNs), which can still approximate any measurable function at any desired degree of accuracy provided that sufficiently many hidden units are available (Hornik et al. 1989). In this case, we consider $\mathsf{\Phi }$ to be the set of functions from ${\mathbb{R}}^m\times {\mathbb{R}}^n\to {\mathbb{R}}^k$ of the form $\varphi (x_1,\dots ,x_m):={\sum }_{i=1}^k\frac{{\mathit{w}}_i}{1+exp\{-{\sum }_{j=1}^m{\alpha }_{ij}{x}_j\}}$, where k is the number of neurons in the hidden layer.

As for the linear combination schemes, we train the NNs to minimize the in-sample MSE using, as input, the first-level predictions. However, in this case, the output of the meta-learning is not a linear function of the inputs, and the combining weights are not constrained to sum to one. Therefore, studying the marginal contribution of each first-level prediction onto the aggregate prediction is not straightforward. In our empirical exercise, we consider two feed-forward neural networks with 1 (NN-1) and 2 (NN-2) hidden layers and 8 and 16-8 neurons, respectively.

3. Data

3.1. Recovery Rates and Security-Specific Characteristics

The main references on bond recovery rate modeling are generally based on the Moody’s Default & Recovery Database (Moody’s DRD) or the Standard & Poor’s Capital IQ Database. We employ Moody’s DRD in this paper.

Following the standard market convention (Mora 2015; Schuermann 2004), the recovery rate of each bond is computed as the bond price measured 30 days after the default date, declared by the rating agency, and divided by the face value. To make our analyses more comparable with earlier studies such as Nazemi et al. (2017), Nazemi et al. (2018) and Nazemi and Fabozzi (2018), we apply a similar filtering strategy. We selected dollar-denominated bonds issued by U.S. companies and with at least USD 5 million in face value. To replicate the same economic conditions, we also filter for default issues in the period 2002–2012. We only retain the observations associated with known values for the following security-specific characteristics: debt seniority, issuer’s industrial sector, default type, coupon level, maturity, presence of additional guarantees different from the issuer’s asset and default date. We are thus left with 768 observations. Notice that machine learning methods provide useful insights even with a sample of this size. In particular, the considered predictive models mitigate the risk of overfitting, can identify the most relevant predictive variables and explore the existence of potential nonlinearities in the data (nonlinear and rule-based methods).

Figure 3 includes a histogram of our recovery rate sample. We provide the corresponding summary statistics in

Table 2. Summary statistics of our recovery rate sample.

	N	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.	St. Dev.
Recovery rate	768	0.01%	10.00%	20.00%	30.98%	51.41%	118.00%	27.58%

.

Summary statistics of recovery rates conditional on the seniority of the defaulted bond, issuer’s industrial sector and default type are provided in

Table 3. Summary statistics of recovery rates according to the seniority of the defaulted bond.

Debt Seniority	N	Median	Mean	St. Dev.	Skewness
Senior Secured	85	63.00%	60.92%	32.27%	$-0.29$
Senior Unsecured	533	19.00%	28.03%	23.90%	$0.98$
Senior Subordinated	129	19.13%	25.77%	26.87%	$1.33$
Subordinated	21	9.13%	16.71%	23.37%	$2.20$

,

Table 4. Summary statistics of recovery rates according to the industrial sector of the bond issuer.

Industrial Sector	N	Median	Mean	St. Dev.	Skewness
Banking	18	18.00%	23.47%	24.44%	$0.46$
Capital Industries	189	29.00%	36.44%	28.55%	$0.72$
Consumer Industries	88	30.25%	39.86%	28.68%	$0.49$
Energy & Environment	45	40.00%	44.10%	25.76%	$0.64$
FIRE	166	10.00%	11.95%	10.32%	$3.88$
Media & Publishing	90	43.62%	40.72%	31.45%	$0.001$
Retail & Distribution	32	36.25%	34.92%	25.89%	$0.58$
Technology	72	15.00%	19.89%	16.99%	$2.45$
Transportation	57	22.25%	31.32%	23.81%	$1.16$
Utilities	11	92.50%	91.89%	6.54%	$-0.32$

and

Table 5. Summary statistics of recovery rates according to the default type.

Default Type	N	Median	Mean	St. Dev.	Skewness
Chapter 11	371	10.50%	25.68%	25.97%	$1.53$
Missed interest payment	281	28.50%	37.55%	26.76%	$0.53$
Missed principal and interest payments	14	58.12%	56.14%	18.85%	$0.05$
Missed principal payment	8	23.04%	29.76%	29.12%	$0.99$
Others	6	11.00%	15.59%	17.47%	$1.50$
Prepackaged Chapter 11	71	12.00%	31.95%	33.01%	$0.71$
Receivership	7	0.50%	4.57%	9.70%	$2.01$
Suspension of payments	10	18.50%	30.05%	26.75%	$2.01$

, respectively. They are consistent with previous findings on recovery rate determinants. For instance, bonds with higher seniority are associated with higher recovery rates on average, and those of senior secured bonds are the most dispersed (Altman and Kalotay 2014; Altman and Kishore 1996). Recovery rates are also higher, on average, when the issuer is operating in an industrial sector featuring higher asset tangibility and in the utility sector in particular (Acharya et al. 2007; Altman and Kishore 1996; Schuermann 2004). Similarly, milder default procedures lead to higher recovery rates (Altman and Karlin 2009; Bris et al. 2006; Davydenko and Franks 2008; Franks and Torous 1994). Defaults on security cash flows are expected to recover more than company reorganizations or liquidations. Controlled reorganizations (prepackaged Chapter 11) also display higher recovery than uncontrolled reorganizations and asset liquidations (receivership and procedures included in the “others” category).

Table 6. Average recovery rate by coupon level.

Coupon	$[0\%$–$2.5\%)$	$[2.5\%$–$5\%)$	$[5\%$–$7.5\%)$	$[7.5\%$–$10\%)$	≥$10\%$
Average RR	14.82%	24.22%	23.01%	33.11%	36.95%

,

Table 7. Average recovery rate by maturity level.

Maturity (Years)	$[0$–$5\mathbf{y})$	$[5\mathbf{y}$–$10\mathbf{y})$	$[10\mathbf{y}$–$15\mathbf{y})$	$[15\mathbf{y}$–$20\mathbf{y})$	≥$20\mathbf{y}$
Average RR	43.30%	37.88%	31.92%	19.39%	18.90%

and

Table 8. Average recovery rate for bonds with and without backing.

Backing	Yes	No
Average RR	40.02%	29.33%

show that our sample of recovery rates is also consistent with prior findings about the role of coupons, maturity and backing guarantees. Recovery rates are higher in the presence of backing guarantees, increase with coupons and decrease with maturity (Jankowitsch et al. 2014). All these security-specific characteristics have been used as control variables in recent articles on bond recovery rate determinants (François 2019; Gambetti et al. 2019) and are used as predictors in our study.

3.2. Systematic Factors

We extract industry default rates from Moody’s DRD and the remaining systematic factors from the databases managed by the Federal Reserve Bank of St. Louis: FRED and FRED-MD (McCracken and Ng 2015). The latter include a large set of time series referring to output and income, labor market, housing, consumption, orders and inventories, money and credit, interest and exchange rates, prices and the stock market.9

We then extend this data set with a large number of economic uncertainty measures from three different classes: survey-based, news-based and volatility-based.10 All measures are retrieved from the websites of the authors (Baker et al. 2016; Jurado et al. 2015; Ludvigson et al. 2019). We consider all five measures employed in the study by Gambetti et al. (2019) plus 11 additional measures of categorical economic policy uncertainty.

Table 9. List of uncertainty measures considered in this study.

Name	Type	Methodology	References
Inflation uncertainty	Survey-based	Dispersion of forecasts from the Federal Reserve Bank of Philadelphia’s Survey of Professional Forecasters.	Zarnowitz and Lambros (1987) Bachmann et al. (2013) Baker et al. (2016)
Federal/State/Local expenditures uncertainty
Economic policy uncertainty	News-based	Normalized volume of newspaper articles published in a given month containing expressions referring to specific types of economic uncertainty.	Baker et al. (2016) Alexopoulos and Cohen (2015)
Monetary policy uncertainty
Fiscal policy (taxes or spending) uncertainty
Tax uncertainty
Government spending uncertainty
Healthcare uncertainty
National security uncertainty
Entitlement programs uncertainty
Regulation uncertainty
Financial regulation uncertainty
Trade policy uncertainty
Sovereign debt, currency crises uncertainty
VIX	Volatility-based (Stock market)	Stock market implied volatility index from the Chicago Board Options Exchange.	Bloom (2009) Bekaert et al. (2013)
Financial uncertainty	Volatility-based (Forecast error)	Conditional volatility of the purely unforecastable prediction error of financial time series.	Jurado et al. (2015) Ludvigson et al. (2019)

gives an overview. Factors relating to the market price of risk and industry portfolio returns are instead retrieved from the Fama-French database. Predictors are measured one month before the default date.

We now proceed to present the results (Section 4) and to discuss their relevance with respect to practical implementation in the industry (Section 5).

4. Results

We compare the performance of the considered algorithms across nine specifications of the predictor set. All model specifications feature security-specific characteristics but differ in the systematic variables that are included. We use two approaches to determine the latter.

(a)

Statistical approach: the algorithms can access either the full data set of systematic variables or a set created with variable selection techniques. For variable selection, we consider a model based on lasso-selected systematic variables and one based on lasso with stability selection (Meinshausen and Bühlmann 2010). While the latter has been used in Nazemi and Fabozzi (2018) to check the reliability of their lasso-selected macroeconomic variables, those retained by lasso with stability selection have never been used to feed predictive algorithms.11

(b)

Economic approach: we create models by relying exclusively on well-identified factors based on the results of Gambetti et al. (2019) and prior studies on recovery rate determinants.

Table 10. Description of the different model specifications.

Specification ID	Systematic Variables	Reference
1	Full data set	-
2	Lasso-selected macroeconomic variables	Nazemi and Fabozzi (2018)
3	Lasso-selected variables with stability control	-
4	Industry default rate, commercial and industrial loans delinquency rates, industrial production, market index returns, PMI	${\mathcal{M}}_{3+}$ in Gambetti et al. (2019)
5	As in 4, with financial uncertainty substituted with industry default rates	${\mathcal{M}}_{4+}$ in   ″
6	As in 4, with VIX substituted with industry default rates	${\mathcal{M}}_{5+}$ in   ″
7	As in 4, plus financial uncertainty, news-based economic policy uncertainty, inflation uncertainty and federal/state/local expenditures uncertainty	${\mathcal{M}}_{4++}$ in   ″
8	As in 4, plus all uncertainty measures of Table 9	-
9	No systematic variables	${\mathcal{M}}_2$ in   ″ Nazemi and Fabozzi (2018) Nazemi et al. (2018)

includes a summary of the model specifications.

We compare the considered predictive strategies by analyzing the root mean square forecast error (RMFE), and we also assess the significance of a performance difference via a model confidence set test (hereafter MCS, Hansen et al. 2011).12

4.1. Predictive Models vs. Historical Averages

The performance of our predictive algorithms across model specifications is summarized in Figure 4; the detailed out-of-sample RMSE are reported in

Table 11. Out-of-sample root mean square error (RMSE) metrics across model specifications. We indicate in bold the RMSE of the models that join the superior set of models with 10% confidence level.

	1	2	3	4	5	6	7	8	9	Mean
Linear Models	0.291	0.204	0.211	0.211	0.211	0.210	0.212	0.213	0.236	0.222
Lin. regression	0.774	0.206	0.205	0.206	0.206	0.206	0.206	0.209	0.233	0.272
Lin. reg. back. sel.	0.221	0.205	0.205	0.207	0.207	0.206	0.209	0.214	0.235	0.212
Lasso 1	0.203	0.203	0.206	0.207	0.207	0.207	0.208	0.209	0.233	0.209
Lasso 2	0.218	0.203	0.221	0.221	0.220	0.218	0.220	0.221	0.241	0.220
Ridge 1	0.202	0.203	0.208	0.207	0.207	0.207	0.208	0.209	0.233	0.209
Ridge 2	0.222	0.211	0.226	0.220	0.221	0.220	0.222	0.223	0.242	0.223
Elastic net	0.201	0.202	0.206	0.207	0.208	0.207	0.209	0.209	0.233	0.209
Nonlinear Models	0.211	0.203	0.207	0.206	0.211	0.208	0.207	0.213	0.237	0.211
MARS	0.207	0.199	0.214	0.197	0.202	0.198	0.194	0.216	0.237	0.207
Gaussian processes	0.211	0.204	0.205	0.207	0.207	0.208	0.207	0.211	0.234	0.210
RVM	0.236	0.212	0.200	0.207	0.223	0.214	0.216	0.212	0.230	0.217
SVM	0.191	0.197	0.212	0.212	0.212	0.212	0.211	0.213	0.245	0.212
Rule-based Models	0.232	0.225	0.233	0.216	0.217	0.213	0.221	0.216	0.240	0.224
Regression tree	0.230	0.223	0.244	0.218	0.215	0.215	0.222	0.211	0.256	0.226
Conditional inference tree	0.234	0.226	0.222	0.214	0.219	0.211	0.220	0.221	0.225	0.221
Nonlinear Ensembles	0.195	0.197	0.197	0.196	0.200	0.200	0.206	0.206	0.235	0.203
Neural networks	0.202	0.201	0.202	0.199	0.204	0.207	0.209	0.208	0.238	0.208
Bagged MARS	0.189	0.194	0.191	0.193	0.195	0.193	0.203	0.204	0.233	0.199
Rule-based Ensembles	0.189	0.191	0.186	0.186	0.187	0.185	0.183	0.185	0.229	0.191
Cubist	0.184	0.182	0.184	0.187	0.194	0.186	0.188	0.183	0.229	0.191
Boosted trees s.g.b.	0.194	0.198	0.195	0.189	0.187	0.188	0.189	0.187	0.238	0.196
Boosted trees	0.182	0.191	0.174	0.177	0.185	0.179	0.177	0.182	0.223	0.185
Quantile random forests	0.193	0.195	0.192	0.192	0.185	0.187	0.180	0.187	0.236	0.194
Random forests	0.190	0.187	0.187	0.185	0.185	0.187	0.182	0.184	0.220	0.190
Linear Meta-Learning	0.187	0.191	0.180	0.185	0.179	0.184	0.179	0.179	0.228	0.188
Opt	0.188	0.193	0.181	0.183	0.180	0.182	0.177	0.181	0.227	0.188
Opt+	0.187	0.191	0.180	0.186	0.179	0.184	0.180	0.178	0.228	0.188
COS-E	0.187	0.191	0.180	0.186	0.179	0.184	0.180	0.178	0.228	0.188
COS-IL	0.187	0.191	0.180	0.186	0.179	0.184	0.180	0.178	0.228	0.188
Equally weighted for.	0.186	0.187	0.188	0.187	0.188	0.188	0.187	0.188	0.224	0.192
Hill climbing	0.193	0.195	0.180	0.189	0.178	0.188	0.189	0.187	0.225	0.192
NonLinear Meta-Learning	0.224	0.200	0.194	0.194	0.192	0.191	0.195	0.213	0.247	0.205
NN - 1	0.241	0.198	0.193	0.187	0.187	0.180	0.189	0.234	0.241	0.205
NN - 2	0.208	0.203	0.195	0.201	0.196	0.201	0.201	0.192	0.253	0.206

. Each line corresponds to a different algorithm. The solid horizontal line (in red) corresponds to the model where we use the in-sample mean recovery rate as a prediction; it can be assimilated to the regulatory standard approach and foundation IRB approach (BCBS 2006, 2011), which are largely based on historical LGD values. Figure 4 highlights that forecasting recovery rates using an algorithm always leads to more precise estimates than using the average of previously observed values. The only exception is the linear regression, which, as expected, suffers from the curse of dimensionality when trained on the full set of predictors (specification 1).

4.2. Models Based on Systematic Variables

Table 11 and Figure 4 also make clear that, in line with the literature, models exploiting systematic variables (specifications 1 to 8) always yield better forecasting performance than a model based on security-specific factors alone (specification 9).

For the selection of macroeconomic variables with data-driven methods, we observe from Table 11 that lasso selection seems to bring more benefit to linear models than to nonlinear or rule-based models. In particular, it appears that ensemble methods (neural networks, bagged MARS, boosted trees, random forests, quantile random forests, cubist) are deprived of useful predictive information when they are trained on lasso-selected variables (specification 2). The performance of these latter algorithms improves if we instead implement the lasso procedure with stability selection (specification 3). We obtain particularly competitive RMSE values in this latter case; the best performance of first-level learners reaches an RMSE of $0.174$ with boosted trees. From an aggregate perspective, it is clear that models relying on lasso selection (with or without stability control) reduce the embedded model risk compared to using the complete set of predictors.

We find that lasso with stability selection retains financial uncertainty, uncertainty related to government spending and uncertainty related to sovereign debt and currency crises (

Table 12. List of predictor probabilities from lasso with stability selection.

Variable	Selection Probability
Financial Uncertainty	$0.95$
Consumer Price Index for All Urban Consumers: Apparel	$0.88$
New One-Family Houses Sold: United States	$0.86$
Industrial Production: Fuels	$0.82$
Number Unemployed for 5–14 Weeks	$0.77$
Continued Claims (Insured Unemployment)	$0.73$
ISM Manufacturing: Supplier Deliveries Index	$0.72$
Securities in Bank Credit, All Commercial Banks	$0.71$
Industry Returns: Agricultural	$0.70$
Money Zero Maturity: Money Stock	$0.69$
Total Consumer Loans and Leases Owned and Securitized by Finance Companies	$0.66$
Industrial Production: Residential Utilities	$0.62$
Employment Cost Index: Benefits: Private Industry Workers	$0.61$
Reserves of Depository Institutions, Nonborrowed	$0.58$
Number Unemployed for Less than 5 Weeks	$0.53$
Total Borrowings of Depository Institutions from the Federal Reserve	$0.53$
Gross Saving	$0.53$
Economic Policy Uncertainty: Government Spending	$0.50$
M1 Money Stock	$0.46$
Light Weight Vehicle Sales: Autos and Light Trucks	$0.44$
Industry Portfolio Returns: Other	$0.43$
Economic Policy Uncertainty: Sovereign Debt Currency Crises	$0.40$
Fama-French Factor: Momentum	$0.38$
All Employees, Government	$0.36$
Civilian Employment Level	$0.35$
Change in Private Inventories	$0.35$
Industry Portfolio Returns: Drugs	$0.34$
Nonperforming Commercial Loans	$0.34$
Industry Portfolio Returns: Smoke	$0.33$
CBOE NASDAQ 100 Volatility Index	$0.33$
Consumer Sentiment Index	$0.30$
Economic Policy Uncertainty: Trade policy	$0.29$
Consumer Price Index for All Urban Consumers: Medical Care	$0.29$
Industrial Production: Nondurable Consumer Goods	$0.28$
National Income	$0.28$
Number Unemployed for 15–26 Weeks	$0.27$
Civilian Labor Force Level	$0.26$
University of Michigan: Inflation Expectation	$0.26$
Corporate Profits after Tax with IVA and CCAdj: Net Dividends	$0.26$
Industrial Production: Materials	$0.25$
Excess Reserves of Depository Institutions	$0.25$

). Financial uncertainty is associated with the highest probability of being selected ($95\%$). The second and third factors, an index of consumption expenditures and a housing market indicator, are associated with probabilities of $88\%$ and $86\%$, respectively.13 Given these results, we confirm the importance of uncertainty measures for forecasting bond recovery rates.

Moreover, if we consider the features of data-driven selection methods against those of economically motivated models, we should prioritize the use of the latter. Notwithstanding that both methods exhibit comparable predictive performance, the economically motivated models offer several advantages. By being based on a low-dimensional set of well-identified economic factors, they are easier to implement and monitor and yield more interpretable results. In contrast, data-driven selection methods can fail to correctly select the best subset of predictors when the latter feature high correlation and require the specification of additional hyperparameters. This increases the complexity and the computational burden, with no guarantee of perfectly selecting the full set of significant predictors.

4.3. First-Level Learners

We now discuss the findings regarding the out-of-sample performance of our algorithms across model structures. In Figure 4, we highlight (in black) the performance of ensemble methods: model-averaged NNs, bagged MARS, boosted trees (with and without stochastic gradient boosting), cubist, random forests and quantile random forests.

In this respect, our results point in the same direction as what Bellotti et al. (2021) discover for nonperforming loans. Rule-based ensemble methods are always associated with the most promising performance compared to both individual learners (linear or nonlinear) and the two other ensembles (bagged MARS and model-averaged NNs). Nonetheless, even if they are less competitive than rule-based ensembles, bagged MARS and model-averaged neural networks outperform individual learners in almost all model specifications.

This can be explained as follows. First, the prediction errors of ensemble methods generally have a lower variance than those of individual learners. This effect is actually a consequence of the aggregation of several base learners; it is particularly visible when comparing the performance of MARS against that of its bagged version. Second, rule-based ensembles are better suited to capture subgroups of data with similar properties and to build a separate model for each group. Recovery rates are particularly prone to behave in this manner, as explained in Yao et al. (2015), Nazemi et al. (2018) and Bellotti et al. (2021). Third, rule-based methods are better suited to reproduce predictor–response relationships that are defined on a closed interval, as is generally the case for recovery rates.

It appears that the only individual learner that can compete with rule-based ensembles is the SVR. However, this method yields good performance only when trained using the full set of predictors or when systematic predictors are selected using lasso without stability selection (which corresponds to the framework adopted in Nazemi and Fabozzi (2018)).

4.4. Meta-Learning: Within and across Predictor Sets

Let us now analyze the performance of linear meta-learning algorithms within each specification of the predictors set (highlighted in Figure 4 and Table 11). We notice a sharp drop in the average RMSE metrics for almost all specifications compared to the other models. In fact, the architecture of these algorithms allows the creation of more flexible functional forms thanks to the combination of various first-level learners. The different strengths of first-level learners are merged together to yield more accurate forecasts. The performance of linear meta-learning algorithms can be compared to those of the best rule-based ensembles. Indeed, Table 11 shows that OPT+, COS-E and COS-IL always join the superior set of models with at least 10% probability outperforming the equally weighted forecast and the hill-climbing algorithm, which, nevertheless, remain competitive especially in specification 3, 4 and 5.

The variation of the aggregate errors within linear meta-learners is much lower than within the class of individual or ensemble models, and the same holds true for the maximum average loss. Therefore, model risk is sensibly reduced when relying on meta-learners compared to on both individual learners and ensembles. This feature can be observed from Figure 4 and holds across all model specifications. We also find that nonlinear meta-learners (NNs) should generally be avoided. They usually display larger RMSE values than those of linear meta-learners across all specifications of the predictor set. Moreover, their partial advantage over ensemble methods and first-level learners in some specifications is neutralized by considerably higher errors in others (i.e., specifications 1, 8 and 9). Hence, in terms of model risk, ensembles and linear meta-learners should be preferred.14

Nevertheless, in practice, we do not know ex ante which among the nine specifications of the predictors set will offer the best performance. While we have previously employed meta-learning to learn the predictive setup within each specification, we now use the same tools to combine all models across all specifications. In other words, we now jointly consider 180 base learners (20 models times 9 predictors specifications).

Table 13. Ranking of top 20 predictive frameworks according to out-of-sample root mean square error (RMSE) metrics across all specifications of the predictors set. The symbols *, ** and *** mark $1\%$, $5\%$ and $10\%$ significance levels of the model confidence test with L2 loss, respectively. For the sake of comparison, we also display the results corresponding to the equally weighted forecast and hill-climbing algorithm, despite ranking, respectively, 24th and 37th.

Model	Specification	RMSE	MAE	${\mathit{R}}^2$	MCS
Boosted trees	7	0.1735	0.1076	0.6298	***
Opt+	.	0.1752	0.1041	0.6175	***
COS-E	.	0.1752	0.1041	0.6175	***
COS-IL	.	0.1752	0.1041	0.6175	***
Boosted trees	3	0.1765	0.1123	0.6142	***
Boosted trees	6	0.1771	0.1140	0.6115	***
Boosted trees	4	0.1789	0.1118	0.6048	***
Quantile random forests	3	0.1796	0.0963	0.6080	***
Boosted trees	2	0.1819	0.1117	0.5920	***
Random forests	3	0.1820	0.1140	0.5927	***
Boosted trees	9	0.1823	0.1140	0.5886
Cubist	8	0.1824	0.1053	0.5878
Cubist	2	0.1831	0.1053	0.5858
Cubist	7	0.1836	0.1076	0.5841
Cubist	9	0.1836	0.1029	0.5843
Random forests	2	0.1842	0.1136	0.5818
Opt	.	0.1843	0.1078	0.5852
Boosted trees	5	0.1845	0.1205	0.5781
Random forests	6	0.1849	0.1164	0.5869
Quantile random forests	5	0.1853	0.0964	0.5887
Equally weighted for.	.	0.1862	0.1289	0.5962
Hill climbing	.	0.1890	0.1213	0.5622

displays the RMSE, mean absolute error (MAE) and $R^2$ of the best 20 predictive strategies in this exercise. Boosted trees (specification 2, 4, 6, 7), quantile random forest (specification 3) and random forest (specification 3) are the only base learners that join the superior set of models. However, their performance clearly depends on the specification of the predictors set. Among meta-learners, while Opt suffers from the increased dimensionality of the problem and performs poorly, meta-learners with nonnegative combining weights (Opt+, COS-E and COS-IL) occupy three of the top four places and join the superior set of models, hence mitigating the uncertainty related to the choice of both the predictors set and the modeling framework, i.e., individual vs. ensembles, linear vs. nonlinear vs. rule-based methods. The results do not significantly differ when analyzing $R^2$ instead of RMSE; this is not surprising given the close relationship between the two metrics. The ranking remains coherent with previous results, especially among the top-performing methods: linear meta-learning techniques are still at the top, beaten only by boosted trees in specification 7, i.e., when inflation uncertainty and federal/state/local expenditures uncertainty are considered together with industry default rate, commercial and industrial loans delinquency rates, industrial production, market index returns, PMI and individual characteristics. Nevertheless, this difference in performance is not statistically significant: COS-E, COS-IL and Opt+ join the superior set of models with 10% confidence level. Conversely, equally weighted forecast and the hill-climbing algorithm perform poorly and are both excluded from the superior set of models. The main difference that emerges when considering the MAE is that quantile random forests (QRF) outperform their competitors, on average. This should not come as a surprise: QRF are trained to minimize the MAE to mitigate the impact of outliers on estimation error, while the loss functions of other predictive strategies are instead related to the RMSE. Overall, COS-E, COS-IL and Opt+ still provide very competitive MAE, while, at the same time, mitigating the uncertainty related to the choice of both the predictors set and the modeling framework.

5. Discussion and Practical Considerations

The results of the previous subsections convey important practical considerations that we now summarize.

First, financial institutions should accelerate the implementation of LGD internal models instead of maintaining the use of historical averages. We show in Section 4.1 that the predictive model does not need to be complicated to be effective. All model specifications outperform the historical average approach, which, however, is the method underlying the standard and foundation IRB frameworks. Unfortunately, recent regulatory guidelines (BCBS 2017) are now pointing in the opposite direction, favoring fixed recovery rate approaches. Our results suggest that this is perhaps not the best route to follow, as LGD internal models can produce more reliable risk figures.

Second, in Section 4.2, we confirm the necessity of including economic factors that can capture systematic fluctuations in recovery rates. This result is in line with the current regulatory prescriptions. Models relying on well-identified economic determinants provide remarkable results, while big data and variable selection procedures do not necessarily translate into better performance because results strongly depend on the considered predictive algorithms. We found that financial uncertainty and text-based economic uncertainty measures are relevant predictors of the recovery rate ex ante. This extends the finding of Gambetti et al. (2019), where financial uncertainty and the Economic Policy Uncertainty Index were found to explain ex post recovery rates. Similarly, we also confirm the finding of Nazemi and Fabozzi (2018): inflation measures, inflation expectation, industrial production, corporate profits after tax, indicators of housing market and stock market volatility are not only systemic determinants of recovery rates but also important predictors. This is also confirmed when using lasso with stability selection (Table 12): the probability of including financial uncertainty and text-based economic policy uncertainty measures is particularly high. Specifically, financial uncertainty, economic policy uncertainty/government spending, economic policy uncertainty/sovereign debt currency crises and economic policy uncertainty/trade policy have 95%, 50%, 40% and 29% probability of being selected, respectively. Including such predictors in more parsimonious and theoretically justified models makes the predictive setting easier to implement, interpret and backtest, making it more prone to be validated by regulators.

Third, in a context where increasing amounts of data become publicly available, the common practice of using standard linear regression to forecast recovery rates should be abandoned. In Section 4.1, we confirm that this is incompatible with the possibility of exploiting the information contained in a large data set of candidate predictors and embeds high model risk. This result is in line with the recent findings of Dong et al. (2020) in a similar financial context.

Fourth, we show in Section 4.3 that ensemble methods and meta-learning techniques should be fostered in the industry. The choice of the A-IRB model should be directed toward ensemble methods with respect to individual learners, as they always yield better performance. However, an incorrect choice among ensembles may still lead to worse results than an incorrect choice among meta-learning techniques. Hence, meta-learning considerably reduces model risk compared to ensembles; it should be considered by the regulator for a new set of A-IRB guidelines. This is particularly true for linear meta-learning techniques that also have the advantage of providing interpretable insights about the contribution of each individual model. As they show, the benefits of diversification can already be appreciated by combining a limited number of high-performing algorithms.

6. Conclusions

In this paper, we explore for the first time the applicability and the potential of meta-learning in order to forecast bond recovery rates. Meta-learning consists of combining the predictions arising from several first-level algorithms into a single aggregated forecast. More specifically, our purpose is to test the performance of a wide set of machine learning methods for predicting recovery rates on defaulted corporate bonds using a data set of predictors of unprecedented size and see whether combining them might display superior performance. We consider as first-level algorithms both individual and ensemble methods belonging to the classes of linear, nonlinear and rule-based models.

We contribute to the literature on bond recovery rate prediction in three ways. Our first contribution deals with the set of predictors to use. We find that including a limited number of well-identified recovery rate determinants and economic uncertainty yields better predictive performance than using the full set of macroeconomic predictors, even when combined with variable selection techniques such as lasso. This finding prevails in all considered algorithms and contrasts with the current big data trend. The use of a limited number of economically grounded predictors offers the additional advantages of making the model easier to implement, to backtest and therefore, when applicable, to be validated by regulatory instances because of mitigating the risk of overfitting and data mining. Interestingly, we confirm the central importance of uncertainty measures that are always retained by the considered variable selection methods.

The second and third contributions relate to the predictive method to use. Random forests, quantile random forests, boosted trees and cubist display the most promising results, but their performances are unstable across the considered specifications of the predictors set: it seems difficult to identify ex ante the right pair of model and predictors set to use. By contrast, we empirically show that meta-learning can beat this challenge. Indeed, meta-learning can improve recovery rate predictions compared to traditional individual learning machines while, at the same time, considerably reduce model risk. This evidence is preserved both when looking at the predictive performance within a chosen predictor set and when jointly considering predictions across all specifications of the predictor set.

In all specifications, the historical average approach performs significantly worse. Yet, this is the method underlying the standardized approach and foundation IRB framework used to compute regulatory capital reserves, which is the primary figure used worldwide to monitor the credit worthiness of banks. Therefore, our findings suggest that regulators and policy makers promoting the use of more reliable risk figures should foster the implementation of LGD internal models that use meta-learning techniques, while the use of traditional linear regressions or historical averages should be challenged. Eventually, our findings will be of practical interest to design new tools for internal economic capital calculations as well as for stress testing purposes.