Modeling and Forecasting the Probability of Crypto-Exchange Closures: A Forecast Combination Approach
Abstract
:1. Introduction
- Ronin Network: $625 million (March 2022);
- Poly Network: $611 million (August 2021);
- FTX: $600 million (November 2022).
- The 24 h trading volume;
- Exchange lifetime;
- Cybersecurity measures;
- Number of supported cryptocurrencies;
- Presence of a public developer team.
- A forecast combination approach yields superior statistical metrics and enhances forecast accuracy compared to individual algorithms;
- The probability of crypto-exchanges remaining operational is significantly influenced by their lifetime, daily trading volume, and cybersecurity scores.
2. Literature Review
3. Materials and Methods
3.1. Machine Learning Techniques
3.1.1. Overview of MethodologicalApproach
3.1.2. Probabilistic and Linear Classifiers (Credit Scoring Models)
3.1.3. Ensemble Methods
3.2. Forecast Combination Approach
- Generalized Linear Pool (GLP):
- Linear Pool: ;
- Harmonic Pool: ;
- Logarithmic Pool: ;
- Normal Pool: , where is the cumulative distribution function (CDF) of the standard N(0,1) normal distribution.
- Beta-Transformed Linear Pool (BLP):
- Beta-Mixture Combination (BMC):
3.3. Evaluation Metrics for Binary Classification
3.3.1. The Confusion Matrix and Associated Metrics
- Sensitivity (Recall or True Positive Rate): Reflects the ability to identify positive cases.
- Precision: Measures the proportion of true positive predictions among all the positive predictions.
- F1-Score: Harmonic mean of precision and recall, balancing the false positives and false negatives.
3.3.2. The H-Measure
- is derived from the severity ratio , which specifies the relative costs of misclassification for the two classes ();
- is the prior probability of class i, i.e., its true share in the whole sample;
- is a redundant scaling factor excluded from minimization;
- is the cumulative distribution function (CDF) of the scores for class i.
3.3.3. The Model Confidence Set (MCS) Procedure
4. Results
4.1. Data
- (a)
- Binary variables:
- decentralized: whether the exchange is decentralized;
- wire_transfer: availability of fund deposits via bank transfer;
- credit_card: availability of payment via credit or debit card;
- public_team: presence of a publicly available senior leadership team profile;
- pen_test: evidence of penetration tests assessing security resilience;
- proof_of_funds: disclosure of reserve holdings by the exchange;
- bug_bounty: existence of a bug bounty program incentivizing ethical hackers to identify vulnerabilities;
- hacked: history of a security breach at the exchange.
- (b)
- Quantitative variables:
- 9.
- lifetime: time in months from the exchange’s foundation to its closure, or to May 2024 if still active;
- 10.
- coins_traded: number of cryptocurrencies available for trading;
- 11.
- pairs_traded: number of trading pairs offered by the exchange;
- 12.
- cer_score: cybersecurity score assigned by the CER platform;
- 13.
- mozilla_score: website security score provided by Mozilla Observatory;
- 14.
- volume_mln: daily trading volume (in million USD).
4.2. Empirical Analysis: Machine Learning Models
4.3. Empirical Analysis: Forecast Combination Approach
- Performance of Combination Methods:
- Both the BLP and BMC methods (with two and three components) achieved the highest performance metrics, with an AUC of 0.924, F1-score of 0.767, Brier Score of 0.099, and H-measure of 0.647. These represent an improvement in the H-measure by 4.1% compared to random forest (H-measure = 0.621) and a reduction in the Brier Score by 2.8% (from 0.102 to 0.099). The inclusion of the BLP and BMC models in the model confidence set (MCS) at a 95% significance level, coupled with the exclusion of all other models, confirms that their improvements in forecasting performance are statistically significant.
- Simpler combination methods, such as the linear pool, also improved performance relative to random forest, achieving an H-measure of 0.632 (an increase of 1.8%) and a Brier Score of 0.100 (a reduction of 2.0%). However, these improvements are less pronounced compared to the BLP and BMC methods.
- Bias–Variance Tradeoff:
- The harmonic and logarithmic pools exhibited slightly lower performances than the random forest baseline, with H-measures of 0.612 and 0.631, respectively. Additionally, the harmonic pool had the highest Brier Score of 0.105, indicating a poorer calibration of probabilities. This suggests that overly simplistic or rigid pooling strategies may fail to capitalize on the diversity of forecasts effectively.
- In contrast, the BLP and BMC methods demonstrated a better balance between bias and variance, achieving the lowest Brier Score of 0.099 and the highest H-measure of 0.647, indicating robust and well-calibrated forecasts. This supports the hypothesis that more flexible combination techniques can effectively harness the strengths of individual models without introducing excessive variance.
- Validation of the Forecast Combination Hypothesis: The superior performance of the BLP and BMC methods provides strong evidence in support of our first hypothesis—combining forecasts enhances accuracy compared to relying on a single model. The BLP and BMC methods not only achieved the highest AUC and H-measure values but also consistently outperformed simpler pooling methods in terms of calibration and overall predictive ability. These results are particularly valuable in contexts like ours, where high-stakes decisions require robust and well-calibrated predictions.
5. Discussion and Conclusions
- The application of ensemble methods, particularly the beta-transformed linear pool (BLP) and beta-mixture combination (BMC), resulted in a significant improvement in forecast quality. These methods increased the robust H-measure by over 4% and reduced the Brier Score by 2.8% compared to the already highly accurate Random Forest classifier. This demonstrates the value of combining forecasts to achieve superior predictive performance.
- The analysis of feature importance revealed that the lifetime of a crypto-exchange and its daily trading volume account for over 30% of feature importance. When security-related features such as CER and Mozilla security scores are included, this proportion exceeds 50%. These findings strongly support the hypothesis that operational longevity, trading activity, and robust security measures are critical factors in determining the survival of cryptocurrency exchanges.
5.1. Limitations of the Study
- Sample Size: The dataset includes 228 exchanges, which, while sufficient for the initial analysis, limits the generalizability of the findings. A larger sample size would enable the use of more sophisticated validation techniques, such as a train-validate-test split, and provide more robust estimates of model performance.
- Data Quality and Availability: The manually collected dataset relies on multiple external sources, which may introduce biases or inconsistencies. Furthermore, historical data for closed exchanges often depend on archived websites, which could lack accuracy or completeness.
- Model Complexity: While ensemble methods like BLP and BMC showed significant improvements, the study avoided overly complex models to mitigate the risk of overfitting given the small sample size. This decision may have excluded some advanced techniques that could perform better with larger datasets.
- Dynamic Factors: The crypto market evolves rapidly, with new factors such as regulatory changes, technological innovations, and macroeconomic conditions influencing exchange closures. Our static dataset does not fully capture these dynamic effects, potentially limiting the predictive power of the models in changing environments.
5.2. Future Research Directions
- Expanding the Dataset: Incorporating additional exchanges and updating the dataset with more recent closures and newly established platforms would provide a more comprehensive view of the market. A larger sample size would also enable the application of deep learning techniques and more complex ensemble methods.
- Dynamic Modeling: Future studies could investigate time-dependent models to capture the evolving nature of the cryptocurrency market. Approaches such as dynamic survival models or recurrent neural networks could provide insights into how risks change over time.
- Alternative Feature Engineering: While this study focused on operational and security features, future work could explore additional predictors, such as user sentiment analysis from social media, blockchain activity data, or regulatory announcements.
- Explainability and Interpretability: As machine learning models become increasingly complex, incorporating methods to enhance model interpretability (e.g., SHAP or LIME, see Lundberg and Lee (2017) and Ribeiro et al. (2016)) could make the results more actionable for stakeholders.
- Scenario Analysis and Stress Testing: Developing models that can evaluate the impact of extreme events, such as major hacks or regulatory crackdowns, would provide valuable insights for risk management in the crypto sector.
5.3. Concluding Remarks
- For Investors: By identifying the key factors that influence exchange survival—such as operational longevity, trading volume, and security features—this research provides a data-driven framework to assess the risks associated with specific exchanges. Investors can use these insights to make informed decisions about where to allocate their funds, mitigating potential losses from exchange closures.
- For Exchange Operators: The results highlight the importance of robust security measures and sustained trading activity in maintaining operational longevity. Exchange operators can leverage these findings to prioritize cybersecurity investments and strategies to increase trading volume, thereby improving their chances of long-term success.
- For Regulators: The study offers a foundation for developing regulatory frameworks aimed at enhancing market stability. By focusing on the key risk factors identified in this research, regulators can create guidelines that promote transparency, security, and sustainability within the cryptocurrency market.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
3xbit | 6x | Aax | ABCC |
Abucoins | AlphaX | AlterDice | Altilly |
Altsbit | AscendEx | Azbit | B2BX |
Backpack | bancor | BHEX (HBTC) | bibox |
Biconomy | BigOne | BiKi | Bilaxy |
binance | BingX | Bit2Me | Bitazza |
Bitbank | BitBNS | Bitcastle | Bitci TR |
Bitcointry | BitDelta | Bitexen | Bitfinex |
bitFlyer | BitForex | Bitfront (Bitbox) | Bitget |
BitGrail | Bithumb | BITKER | Bitkub |
Bitlish | Bitlo | BitMart | BitMesh |
BitMex | BitoPro | Bitrue | bitso |
BitStamp | Bitsten | BitStorage | Bittrex |
Bitunix | Bitvavo | BitVenus | BKEX |
Bleutrade | Blockchain.com (The PIT) | Blofin | BTCbear |
BTCEX | BtcTurk | BTSE | Bullish |
Bybit | BYDFi | C-CEX | C-Patex |
Catex | Chainrift | ChaoEX | Chilebit.net |
CITEX | Cobinhood | Coinbase | CoinBene |
Coinchangex | Coincheck | CoinCorner (Coinfloor) | CoinDeal |
Coineal | CoinEgg | CoinEx | CoinFalcon |
Coinhub | CoinJar | CoinLim | Coinlist |
Coinmetro | Coinnest | Coinone | Coinrate |
Coins.ph | Coinsbit | Coinstore | Coinsuper |
CoinTiger | CoinTR Pro | CoinW | CPDAX |
CredoEx | Cryptal | Crypto Dao | Crypto.com |
CryptoBridge DEX | Cryptology | CryTrEx | Currency.com |
Dcoin | Deepcoin | Deribit | Dex-Trade |
DigiFinex | Emirex | Exmo | Fairdesk |
Fastex | FatBTC | Fcoin | Fisco |
FMFW.io | Foxbit | FTX | Gate.io |
GDAC | Gemini | GMO Japan | GokuMarket |
GoPax | Hashkey | HB.top | HBUS |
HitBTC | Hoo.com | Hotbit | Hotcoin |
HTX (Huobi) | iCE3 | ICOCryptex | Icrypex |
Independent Reserve | Indodax | Instant Bitex | IQFinex |
itBit | Kanga | KickEx | KoinBX |
Koinpark | Korbit | Kraken | KuCoin |
Kuna | LakeBTC | LATOKEN | Lbank |
LCX | LEOxChange | Liquid | Livecoin |
LocalTrade | Lukki | Luno (BitX) | Max Maicoin |
Mercado Bitcoin | Mercatox | MEXC | Narkasa |
Neraex | Nicehash | NLexch | Nominex |
Nonkyc.io | OceanEx | Okcoin | OKX (OKEx) |
One Trading (Bitpanda) | OPNX | OrangeX | OTCBTC |
P2B | Paribu | Phemex | Pionex |
PointPay | Poloniex | ProBit | Purcow |
QMall | Shortex | Sistemkoin | Slex |
Sparkdex | SpectroCoin (Bankera) | STEX | StormGain |
Tapbit | TheRockTrading | Thodex (Koineks) | Tidex |
Tokenize | TokensNet | TokoCrypto | Tokpie |
Toobit | TopBTC | Trade Satoshi | Tux Exchange |
Txbit | Unichange | Upbit | VALR |
Vbitex | Vebitcoin | VirWox | WazirX |
Websea | WEEX | WhiteBIT | WOO X |
Worldcore | XeggeX | XT.com | YoBit |
Zaif | ZebPay | ZG.top | zondacrypto (BitBay) |
1 | Definition by the world-leading source of financial content (Investopedia https://www.investopedia.com/terms/c/cryptocurrency.asp, accessed on 1 December 2024). |
2 | Data from Coingecko (https://www.coingecko.com, accessed on 1 December 2024). |
3 | Investopedia research (https://www.investopedia.com/news/largest-cryptocurrency-hacks-so-far-year/, accessed on 2 December 2023). |
References
- Bracke, P., Datta, A., Jung, C., & Sen, S. (2019). Machine learning explainability in finance: An application to default risk analysis (Tech. Rep.; Working paper n. 816). Bank of England. [Google Scholar]
- Brier, G. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3. [Google Scholar] [CrossRef]
- Casarin, R., Mantoan, G., & Ravazzolo, F. (2016). Bayesian calibration of generalized pools of predictive distributions. Econometrics, 4(1), 17. [Google Scholar] [CrossRef]
- Fantazzini, D., & Calabrese, R. (2021). Crypto exchanges and credit risk: Modeling and forecasting the probability of closure. Journal of Risk and Financial Management, 14(11), 516. [Google Scholar] [CrossRef]
- Fonseca, P. G., & Lopes, H. D. (2017). Calibration of machine learning classifiers for probability of default modelling. arXiv, arXiv:1710.08901. [Google Scholar]
- Gneiting, T., & Ranjan, R. (2013). Combining predictive distributions. Electronic Journal of Statistics, 7, 1747–1782. [Google Scholar] [CrossRef]
- Hand, D. J. (2009). Measuring classifier performance: A coherent alternative to the area under the ROC curve. Machine Learning, 77(1), 103–123. [Google Scholar] [CrossRef]
- Hand, D. J., & Anagnostopoulos, C. (2014). A better Beta for the H measure of classification performance. Pattern Recognition Letters, 40, 41–46. [Google Scholar] [CrossRef]
- Hanley, J., & McNeil, B. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36. [Google Scholar] [CrossRef] [PubMed]
- Hansen, P., Lunde, A., & Nason, J. (2011). The model confidence set. Econometrica, 79(2), 453–497. [Google Scholar] [CrossRef]
- James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112). Springer. [Google Scholar]
- Lahiri, K., Peng, H., & Zhao, Y. (2015). Testing the value of probability forecasts for calibrated combining. International Journal of Forecasting, 31(1), 113–129. [Google Scholar] [CrossRef] [PubMed]
- Lahiri, K., & Yang, L. (2013). Forecasting binary outcomes. In Handbook of economic forecasting (Vol. 2, pp. 1025–1106). Elsevier. [Google Scholar]
- Lee, C. F., & Yu, H.-C. (2021). Application of discriminant analysis, factor analysis, logistic regression, and KMV-Merton model in credit risk analysis. In Handbook of financial econometrics, mathematics, statistics, and machine learning (pp. 4313–4348). World Scientific. [Google Scholar]
- Lee, S. A., & Milunovich, G. (2023). Digital exchange attributes and the risk of closure. Blockchain: Research and Applications, 4(2), 100131. [Google Scholar] [CrossRef]
- Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Proceedings of the 31st international conference on neural information processing systems (pp. 4768–4777). Curran Associates Inc. [Google Scholar]
- Metz, C. (1978). Basic principles of ROC analysis. Seminars in Nuclear Medicine, 8, 283–298. [Google Scholar] [CrossRef] [PubMed]
- Metz, C., & Kronman, H. (1980). Statistical significance tests for binormal ROC curves. Journal of Mathematical Psychology, 22(3), 218–243. [Google Scholar] [CrossRef]
- Milunovich, G., & Lee, S. A. (2022). Cryptocurrency exchanges: Predicting which markets will remain active. Journal of Forecasting, 41(5), 945–955. [Google Scholar] [CrossRef]
- Moore, T., Christin, N., & Szurdi, J. (2018). Revisiting the risks of bitcoin currency exchange closure. ACM Transactions on Internet Technology, 18(4), 1–18. [Google Scholar]
- Nabipour, M., Nayyeri, P., Jabani, H., Shahab, S., & Mosavi, A. (2020). Predicting stock market trends using machine learning and deep learning algorithms via continuous and binary data; a comparative analysis. IEEE Access, 8, 150199–150212. [Google Scholar] [CrossRef]
- Nakamoto, S. (2008). A peer to peer electronic cash system (Tech. Rep.). Available online: https://bitcoin.org/bitcoin.pdf (accessed on 1 December 2024).
- Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1135–1144). Association for Computing Machinery. [Google Scholar]
- Sammut, C., & Webb, G. (2011). Encyclopedia of machine learning. Springer. [Google Scholar]
- Schueffel, P., & Groeneweg, N. (2019). Evaluating crypto exchanges in the absence of governmental frameworks—A multiple criteria scoring model. Available online: https://ssrn.com/abstract=3432798 (accessed on 1 December 2024).
- Wattanachit, N., Ray, E. L., McAndrew, T. C., & Reich, N. G. (2023). Comparison of combination methods to create calibrated ensemble forecasts for seasonal influenza in the US. Statistics in Medicine, 42(26), 4696–4712. [Google Scholar] [CrossRef] [PubMed]
Mean | Std | Min | 25% | 50% | 75% | Max | |
---|---|---|---|---|---|---|---|
closed | 0.33 | 0.47 | 0 | 0 | 0 | 1 | 1 |
decentralized | 0.04 | 0.184 | 0 | 0 | 0 | 0 | 1 |
wire_transfer | 0.68 | 0.468 | 0 | 0 | 1 | 1 | 1 |
credit_card | 0.53 | 0.5 | 0 | 0 | 1 | 1 | 1 |
lifetime | 67.82 | 35.612 | 5 | 39 | 67 | 84.75 | 154 |
coins_traded | 170.81 | 292.36 | 1 | 19.75 | 62.5 | 200.25 | 2424 |
pairs_traded | 254.58 | 448.1 | 1 | 28.5 | 98.5 | 262.25 | 3452 |
public_team | 0.71 | 0.45 | 0 | 0 | 1 | 1 | 1 |
cer_score | 4.92 | 2.46 | 0.76 | 2.7 | 4.29 | 7.31 | 10 |
pen_test | 0.30 | 0.46 | 0 | 0 | 0 | 1 | 1 |
proof_of_funds | 0.49 | 0.5 | 0 | 0 | 0 | 1 | 1 |
bug_bounty | 0.41 | 0.493 | 0 | 0 | 0 | 1 | 1 |
mozilla_score | 43.25 | 27.51 | 0 | 25. | 47.5 | 70 | 110 |
hacked | 0.28 | 0.45 | 0 | 0 | 0 | 1 | 1 |
volume_mln | 361.21 | 1248.9 | 0 | 1.575 | 31 | 256.25 | 17 |
AUC | F1-Score | Brier Score | H | MCS | |
---|---|---|---|---|---|
Naive Bayes | 0.841 | 0.748 | 0.162 | 0.523 | No |
Logistic Regression | 0.878 | 0.775 | 0.124 | 0.553 | No |
SVC | 0.857 | 0.715 | 0.132 | 0.527 | No |
CatBoost | 0.914 | 0.769 | 0.103 | 0.614 | Yes |
Random Forest | 0.921 | 0.696 | 0.102 | 0.621 | Yes |
AUC | F1-Score | Brier Score | H | MCS | |
---|---|---|---|---|---|
Random Forest | 0.921 | 0.775 | 0.102 | 0.621 | No |
Linear Pool | 0.922 | 0.772 | 0.100 | 0.632 | No |
Harmonic Pool | 0.901 | 0.757 | 0.105 | 0.612 | No |
Logarithmic Pool | 0.919 | 0.755 | 0.100 | 0.631 | No |
N(0,1) Pool | 0.921 | 0.772 | 0.100 | 0.631 | No |
BLP | 0.924 | 0.767 | 0.099 | 0.647 | Yes |
BMC(2) | 0.924 | 0.767 | 0.099 | 0.647 | Yes |
BMC(3) | 0.924 | 0.767 | 0.099 | 0.647 | Yes |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Magomedov, S.; Fantazzini, D. Modeling and Forecasting the Probability of Crypto-Exchange Closures: A Forecast Combination Approach. J. Risk Financial Manag. 2025, 18, 48. https://doi.org/10.3390/jrfm18020048
Magomedov S, Fantazzini D. Modeling and Forecasting the Probability of Crypto-Exchange Closures: A Forecast Combination Approach. Journal of Risk and Financial Management. 2025; 18(2):48. https://doi.org/10.3390/jrfm18020048
Chicago/Turabian StyleMagomedov, Said, and Dean Fantazzini. 2025. "Modeling and Forecasting the Probability of Crypto-Exchange Closures: A Forecast Combination Approach" Journal of Risk and Financial Management 18, no. 2: 48. https://doi.org/10.3390/jrfm18020048
APA StyleMagomedov, S., & Fantazzini, D. (2025). Modeling and Forecasting the Probability of Crypto-Exchange Closures: A Forecast Combination Approach. Journal of Risk and Financial Management, 18(2), 48. https://doi.org/10.3390/jrfm18020048