All-Cause Mortality Prediction in Subjects with Diabetes Mellitus Using a Machine Learning Model and Shapley Values

Abstract

Background/Objectives: Diabetes mellitus (DM) is a prevalent disease with an increased risk of complications. Identifying risk factors for mortality in these patients is crucial, as early recognition can facilitate prompt therapeutic intervention. Machine learning (ML) models have proved to be valuable tools in different scenarios of healthcare decision making. We aimed to develop and test an ML model to predict all-cause mortality in a large cohort of subjects with DM. Methods: We included 1969 consecutive patients with DM type 1 (T1DM, n = 255) and type 2 (T2DM, n = 1714). eXtreme Gradient Boosting (XGBoost) was used for the prediction of all-cause mortality in this cohort and the Shapley additive explanation (SHAP) was used to assess the importance of each feature of the classifier. The missing values were imputed using the Missforest methodology. Results: The all-cause mortality rate was 21% during 5.5 ± 1.1 years of follow-up. The ML model achieved 90% sensitivity and 87% specificity with an AUC of 0.88 and an accuracy of 88% for predicting all-cause mortality. The SHAP analysis identified a lower glomerular filtration rate (eGFR), duration of insulin therapy, and a lower level of hemoglobin as the first three factors that contribute to the higher mortality rate. Conclusions: ML models can become valuable tools in clinical practice due to their unique ability to simultaneously assess the cumulative influence of multiple parameters and discover high-order interactions. The application of such models in clinical practice could improve the early identification of subjects at risk for complications and mortality and prompt early therapeutical interventions.

1. Introduction

According to the International Diabetes Federation (IDF), as of 2021, over 500 million people worldwide are living with diabetes mellitus (DM). Furthermore, this number is projected to increase significantly, with estimates suggesting that, by 2045, nearly 700 million individuals could be affected by diabetes [1]. This anticipated rise is concerning, and it justifies the urgency of taking proactive measures to address this global health challenge.

Type 2 diabetes mellitus (T2DM) is the most prevalent form of diabetes, accounting for nearly 90% of all cases, while type 1 diabetes mellitus (T1DM) is comparatively rare [2]. Individuals with diabetes mellitus (DM) experience a significantly higher mortality rate, largely attributable to long-term complications associated with the disease. These include cardiovascular disease, kidney disease, and neuropathy, which collectively contribute to the increased burden of morbidity and mortality in this population [3,4,5,6].

In recent years, computational modeling and machine learning (ML) algorithms have emerged as powerful tools in healthcare, demonstrating significant potential for improving the predictive modeling of disease outcomes [7,8]. These techniques can be used to analyze large and complex datasets, favoring the development of more accurate and individualized predictive models.

However, the foremost important challenge associated with ML models, particularly in the healthcare setting, is the difficulty of interpreting and translating the ‘black box’ results in clinical practice, thus limiting their applicability. To address this limitation, several approaches to interpreting model predictions have been proposed, and the Shapley additive explanation (SHAP) has proven particularly advantageous by providing a framework to explain model outputs [9]. The advantage of using the SHAP plots lies in their ability to allocate a rank of contribution to each feature. This ranking provides clinicians with an interpretable view of the rationale behind the model’s predictions, reducing the gap between complex ML outputs and practical decisions.

Thereby, in this study, we integrated commonly available clinical and biological DM characteristics by using an ML algorithm and tested the accuracy of predicting all-cause mortality in a large cohort of subjects with DM by using Shapley values to clarify the importance of each feature.

2. Materials and Methods

2.1. Data Collection

We retrospectively selected all consecutive patients aged above 18 years old with a documented diagnosis of diabetes mellitus who were admitted to the Emergency County Hospital of Craiova, Romania, a reference hospital for the region, between January 2016 and December 2018. The dataset was obtained from the digital database of the hospital. All patients admitted to the hospital are required to sign a general informed consent form upon admission, which includes their agreement to the use of medical data for scientific purposes.

The study was approved by the Ethics Committee of Emergency County Hospital of Craiova (no. 74 from 7 September 2020) and conducted in accordance with local legislation and institutional requirements.

2.2. Choice of Parameters

We selected a subset of 14 demographic/clinical parameters, 17 biological tests, and 5 parameters relevant to DM characterization. All parameters are listed in

Table 1. Clinical, biological, and diabetes mellitus characteristics included in the study.

Demographic and Clinical	Biological Data	DM Related
Gender (male/female) Age (years) Height (cm) Current weight (kg) Maximum weight (kg) Current BMI (kg/m2) Maximum BMI (kg/m2) Abdominal circumference (cm) Hip circumference (cm) Systolic blood pressure (mmHg) Diastolic blood pressure (mmHg) Heart rate (bpm) Arterial hypertension (yes/no) Atrial fibrillation (yes/no)	Uric acid (mg/dL) ALT (U/L) AST (U/L) GGT (U/L) Serum amylase (U/L) Creatinine (mg/dL) eGFR (mL/min/1.73 m2) Urea (mg/dL) Alkaline phosphatase (U/L) Total cholesterol (mg/dL) HDL cholesterol (mg/dL) LDL cholesterol (mg/dL) Triglycerides (mg/dL) Hemoglobin (g/dL) Platelet count (number/µL) Glycated hemoglobin (%) Maximum glycemia (mg/dL)	Type of DM (type 1/2) Duration of DM (years) Duration of Insulin treatment (years) DM Medication: Biguanides Sulfonylureas Alpha-glucosidase inhibitors SGLT2 inhibitors Dipeptidyl peptidase-4 inhibitors GLP-1 receptor agonists Number of classes of DM medication

ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; DM, diabetes mellitus; eGFR, estimated glomerular filtration rate calculated using the Modification of Diet in Renal Disease (MDRD) equation; GGT, gamma-glutamyl transferase; GLP1, glucagon-like peptide 1; SGLT2, sodium–glucose cotransporter 2.

. We specifically selected for the study parameters that are commonly obtained during admission in our hospital.

Hypertension history was assessed from the medical records. The diagnosis of atrial fibrillation (AFib) was made either if the patient had a documented history of AFib or if it was present on the resting electrocardiogram obtained during hospital admission. Arterial hypertension was defined as a history of blood pressure above 140/90 mmHg, and/or current hypertension treatment [10].

2.3. Machine Learning Model

For the analysis, we used eXtreme Gradient Boosting (XGBoost), a modified version of the gradient boosting classifier [11]. XGBoost is an ensemble classifier that combines many weak predictive models, usually decision trees, to create a more powerful and efficient model. This approach is iterative: at each iteration, a new weak learner is added to the ensemble, and the weights of the previous learners are adjusted and fixed. This iterative process continues until the model’s performance converges, resulting in a robust predictive model. The input for the model consisted of 36 measurements. Datasets were split according to the 80/20 rule, i.e., 80% of the dataset was used to train the model, and 20% was used to test it.

2.4. Imputation and Imbalance Treatment

The dataset was obtained from a clinical setting and thus showed missing measurements. Additionally, the dataset was inspected, and outliers (values that were non-physiological for the given parameters) due to typing or unit of measurement errors were excluded from the analysis. The missing values were imputed using the Missforest methodology [12]. The percentage of imputation for each measurement is listed in

Table 2. Characteristics of the study population, according to the type of diabetes mellitus.

Parameter	Unit	All (n = 1969)	T1DM (n = 255)	T2DM (n = 1714)	p Value *
Anthropometric data
Age	years	59 ± 13	37 ± 13	62 ± 10	<0.01
Current weight	kg	81 ± 20	68 ± 14	83 ± 20	<0.01
Maximum weight	kg	91 ± 21	78 ± 18	93 ± 21	<0.01
Current BMI	kg/m2	29 ± 7	24 ± 4	30 ± 7	<0.01
Maximum BMI	kg/m2	33 ± 7	27 ± 5	34 ± 6	<0.01
Abdominal circumference	cm	102 ± 16	85 ± 12	105 ± 15	<0.01
Hip circumference	cm	105 ± 13	96 ± 9	106 ± 12	<0.01
DM characteristics
Duration of DM	years	10.0 ± 8.5	13 ± 11	10 ± 8	<0.01
Duration of insulin treatment	years	4.4 ± 7.1	13 ± 11	4 ± 5	<0.01
HbA1c	%	10.1 ± 2.6	10.1 ± 3	10.0 ± 2.5	<0.01
Maximum glycemia	mg/dL	309 ± 98	341 ± 98	305 ± 98	<0.01
SBP	mmHg	135 ± 19	124 ± 18	137 ± 19	<0.01
DBP	mmHg	80 ± 12	77 ± 11	80 ± 12	<0.01
Heart rate	bpm	83 ± 14	87 ± 14	83 ± 13	ns
Laboratory data
Hemoglobin	g/dL	13.5 ± 1.9	13.6 ± 2.0	13.5 ± 1.8	<0.01
Platelet count	number/µL	221.000 ± 87.000	234.800 ± 78.000	219.200 ± 88.000	<0.01
Creatinine	mg/dL	1.0 ± 2.0	1.0 ± 1.8	1.0 ± 2.0	<0.01
Urea	mg/dL	44 ± 25	35 ± 19	45 ± 26	<0.01
eGFR	mL/min/1.73 m2	82 ± 25	100 ± 24	79 ± 25	<0.01
Uric acid	mg/dL	5.1 ± 1.8	4.3 ± 1.7	7.8 ± 1.8	<0.01
Alkaline phosphatase	U/L	80 ± 44	86 ± 53	79 ± 42	<0.01
GGT	U/L	80 ± 181	56 ± 165	84 ± 183	<0.01
AST	U/L	31 ± 27	27 ± 27	31 ± 27	<0.01
ALT	U/L	36 ± 33	28 ± 27	37 ± 34	<0.01
Total cholesterol	mg/dL	190 ± 61	190 ± 56	190 ± 62	ns
HDL cholesterol	mg/dL	41 ± 14	50 ± 18	40 ± 13	<0.01
LDL cholesterol	mg/dL	111 ± 45	113 ± 42	111 ± 46	<0.01
Triglycerides	mg/dL	201 ± 280	144 ± 140	210 ± 300	<0.01

Values represent mean ± standard deviation; * p value represents the statistical significance of a 2-sided t-test, T1DM parameters vs. T2DM parameters. ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; DM, diabetes mellitus; DBP, diastolic blood pressure; eGFR, estimated glomerular filtration rate calculated using the Modification of Diet in Renal Disease (MDRD) equation; GGT gamma-glutamyl transferase; HbA1c, glycated hemoglobin; HDL, high-density level; LDL, low-density level; ns, not significant; SBP, systolic blood pressure.

. As the imbalance of classes can result in a bias of the model toward the larger class, we used a combination of SMOTE and Tomek’s links named the SMOTE–Tomek approach [13]. In this approach, some artificial samples were synthesized using the synthetic minority oversampling technique (SMOTE) algorithm and some pairs of observations from two classes were removed according to the Tomek link rules.

2.5. Feature Importance in the Model

The Shapley additive explanation was used to assess the importance of each feature of the classifier. The unique advantage of the summary plot is its combination of feature importance and feature effect, allowing for the interpretation of the underlying relationships between features and outcomes. A positive value indicates that the feature promotes the likelihood of a positive outcome, whereas a negative one indicates that the feature decreases the likelihood of a positive outcome. The distribution of the Shapley values per feature is shown by overlapping points that are jittered in the y-axis direction. Blue dots indicate values below the cut-off of each measurement, whereas red dots indicate values above the specific cut-off.

2.6. Outcome

The status of alive/deceased was obtained either from the records of our hospital or, when not available, from the National Registry of Persons. The status of each subject (dead/alive) was updated in December 2022.

2.7. Statistical Analysis

The statistical analysis was performed using SPSS (ver. 20.0) software (IBM, Chicago, IL, USA). The categorical variables are reported as the mean ± standard deviation (STDEV). A Student’s t-test was used to compare categorical data. For all tests, p  <  0.05 was deemed statistically significant.

3. Results

3.1. Study Population

A total of 1969 subjects were included in this study (males, n = 936). The average age was 59 ± 13 years. Most subjects had T2DM (n = 1714, 87%), 692 of whom (40%) had insulin in their treatment protocol. Subjects with T1DM were younger (p < 0.001) and had a longer history of DM (p < 0.001). The characteristics of the two groups are listed in Table 2.

3.2. Outcome

The mean follow-up period was 5.5 ± 1.1 years. The overall mortality rate in the population was 21%. Subjects with T1DM showed a lower mortality rate when compared to subjects with T2DM (8% vs. 23%, p < 0.01). Overall, subjects with poor survival were older, with a longer duration of DM and more impaired kidney function (

Table 3. Characteristics of the entire study population and separated according to outcome.

Parameter	Unit	All (n = 1969)	Alive (n = 1560)	Deceased (n = 409)	p Value *
Anthropometric data
Age	years	59 ± 14	57 ± 14	67 ± 11	<0.001
Current weight	kg	81 ± 20	81 ± 19	79 ± 21	ns
Max weight	kg	91 ± 21	91 ± 21	90 ± 22	ns
Current BMI	kg/m2	29 ± 7	29 ± 7	29 ± 7	ns
Maximum BMI	kg/m2	33 ± 7	33 ± 7	33 ± 7	ns
Abdominal circumference	cm	102 ± 16	102 ± 16	104 ± 17	ns
Hip circumference	cm	105 ± 13	105 ± 12	105 ± 14	ns
DM characteristics
Duration of DM	years	10.0 ± 8.5	9.5 ± 8.3	11.4 ± 9.1	<0.001
Duration of insulin treatment	years	4.4 ± 7.1	4.3 ± 7.1	4.3 ± 7.4	ns
HbA1c	%	10.1 ± 2.6	10.0 ± 2.5	10.2 ± 2.9	ns
Maximum glycemia	mg/dL	309 ± 98	302 ± 95	333 ± 105	<0.001
SBP	mmHg	135 ± 19	135 ± 19	136 ± 21	ns
DBP	mmHg	80 ± 12	80 ± 11	78 ± 13	<0.001
Heart rate	bpm	83 ± 14	84 ± 13	83 ± 15	ns
Laboratory data
Hemoglobin	g/dL	13.5 ± 1.9	13.7 ± 1.7	12.7 ± 2.2	<0.001
Platelet count	number/µL	221.000 ± 87.000	222.000 ± 75.000	218.000 ± 121.000	ns
Creatinine	mg/dL	1.0 ± 2.0	0.9 ± 0.4	1.4 ± 4.2	<0.001
Urea	mg/dL	44 ± 25	40 ± 21	56 ± 35	<0.001
eGFR	mL/min/1.73 m2	82 ± 25	85.5 ± 24.2	68.4 ± 27.6	<0.001
Uric acid	mg/dL	5.1 ± 1.8	5.0 ± 1.9	5.5 ± 2.0	<0.001
Alkaline phosphatase	U/L	80 ± 44	78 ± 39	87 ± 57	<0.001
GGT	U/L	80 ± 181	72 ± 150	111 ± 268	<0.001
AST	U/L	31 ± 27	29 ± 23	36 ± 38	<0.001
ALT	U/L	36 ± 33	36 ± 30	38 ± 44	ns
Total cholesterol	mg/dL	190 ± 61	192 ± 60	180 ± 66	<0.001
HDL cholesterol	mg/dL	41 ± 14	41 ± 13	38 ± 15	<0.001
LDL cholesterol	mg/dL	111 ± 45	113 ± 45	103 ± 44	<0.001
Triglycerides	mg/dL	201 ± 280	202 ± 290	199 ± 273	ns

Values represent mean ± standard deviation; * p value represents the statistical significance of a 2-sided t-test, alive vs. deceased. ALT, alanine aminotransferase; AST, aspartate aminotransferase; BMI, body mass index; DM, diabetes mellitus; DBP, diastolic blood pressure; eGFR, estimated glomerular filtration rate calculated using the Modification of Diet in Renal Disease (MDRD) equation; GGT, gamma-glutamyl transferase; HbA1c, glycated hemoglobin; HDL, high-density level; LDL, low-density level; ns, not significant; SBP, systolic blood pressure.

).

3.3. Imputation Rate

The imputation rate ranged from 0% to 18.1%. The imputation percentage for each parameter used in the analysis is listed in Supplemental Table S1.

3.4. Machine Learning Model and Feature Importance

The ML model demonstrated good results in predicting all-cause mortality in our study cohort, with an accuracy of 89% and an area under the curve of 88%. The sensitivity was 90% and the specificity was 87%, while the F1-score was 88%.

The feature importance for the first 20 parameters in order of position is shown in Figure 1. We can observe that eGFR had the highest ranking, followed by mean years of therapy with insulin and hemoglobin value, GGT, and urea.

Regarding the direction of contribution, a lower eGFR, lower hemoglobin, higher GGT, and lower urea values were associated with a higher mortality rate. The interpretation of the duration of insulin treatment is challenging as it seems that both short and prolonged insulin therapy has an impact on mortality.

4. Discussion

Diabetes mellitus is a major public health challenge of the twenty-first century. It affects a large and growing number of people, impacting their quality of life and inducing serious multi-organ complications, which significantly increase the risk of mortality. Integrating machine learning and artificial intelligence models in the care of patients with DM could prove beneficial on an individual and societal level.

For this study, we used a machine learning (ML) model to integrate clinical, anthropometric, and biological parameters that are widely available and commonly obtained during hospital admissions or routine evaluations of individuals with diabetes mellitus to predict all-cause mortality. Moreover, we employed Shapley values to illustrate the contribution of each variable to the model, indicating both the magnitude and the direction (positive or negative) of their impact on the prediction of all-cause mortality. A single larger-scale study investigated the combination of ML models and SHAP to predict one-year mortality in subjects with DM, showing promising results. However, the model included mostly anthropometric and administrative parameters, and the performance was below that reported in our study, in the range of 0.78–0.799 in terms of AUC [14].

4.1. Study Population

Most of the subjects included in this study had T2DM, while only a small proportion had T1DM. Subjects with T1DM were significantly younger, with a lower BMI, and a longer history of diabetes and insulin intake.

A large percentage of patients had arterial hypertension (72%). Our results are similar to those reported in the literature [15,16] and reflect the common coexistence of these two pathologies.

Due to improvements in medical care and treatment, overall mortality in diabetes subjects has shown a significant decline in recent decades. There has been up to a 70% reduction in the all-cause death rate in subjects with T2DM and roughly a 30% reduction in all-cause mortality in subjects with T1DM [17]. In our population, those with T2DM had a significantly higher mortality rate, results that are not supported by larger scale studies [18,19], which conversely, reported higher mortality rates in subjects with T1DM. The differences may be due to the imbalanced study population, with subjects with T1DM being represented by a small fraction in our cohort. Nonetheless, the overall mortality rate was comparable to that reported in previous studies [20,21].

4.2. Machine Learning Model and Feature Importance

The ML model demonstrated good accuracy in predicting mortality in subjects with diabetes mellitus.

A low eGFR value was associated with the highest mortality risk in our model. The relationship between diabetes and kidney disease onset and progression is well established, with diabetes being the most determinant cause of kidney transplants in developed countries [22]. Moreover, all-cause mortality is also higher in diabetic subjects with impaired kidney function [23,24], highlighting the crucial need to identify subjects with reduced renal function in order to provide early target treatment and prevent progression to end-stage renal disease and cardiovascular complications [25].

Secondly, the years of treatment with insulin ranked as the second most important feature in the model. Insulin therapy [26] and escalating doses [27] were associated with a significant increase in all-cause and cardiac death in patients with diabetes compared to those receiving oral hypoglycemic medications alone.

Interestingly, the SHAP values did not show a clear direction. A shorter period of insulin therapy was associated both negatively and positively with mortality. On one hand, patients in whom insulin was recently introduced might showed a lower mortality rate due to better glycemic control, resulting in a lower rate of complications, or due to a shorter evolution of the disease with fewer complications. Conversely, shorter periods of insulin therapy could also imply that insulin was introduced recently, possibly due to a more severe disease with poorer glycemic control or diabetes-related complications and thus a higher death risk.

Next, the SHAP analysis indicated that a lower level of hemoglobin was found to directly increase all-cause mortality. In both diabetic and non-diabetic patients, the presence of anemia has been shown to increase significantly all-cause mortality risk [28,29], supporting the results of our analysis. Moreover, we demonstrated that a lower platelet count is also associated with mortality. Previous research suggested that platelets might play a significant role in the pathogenesis of micro- and macrovascular complications of DM [30]. Nonetheless, the risk for mortality has been consistently associated with higher mean volume and platelet count and a pro-coagulative state, while the data regarding the impact of thrombocytopenia on DM are scarce. Larger studies are warranted to understand the interplay between platelet count and diabetes complications.

A high level of gamma-glutamyl transferase was associated with a higher risk for mortality in our cohort. GGT has been identified as a marker of oxidative stress, and the association between increased plasma levels of GGT and the risk of developing DM [31,32], as well as the risk for mortality in already diagnosed patients, has been described [33].

Interestingly, in our analysis, treatment with biguanides showed a clear positive association with mortality. The most commonly used representative of the biguanide class is metformin, which is usually the first-line oral anti-hyperglycemic agent for treating patients with non-insulin-dependent diabetes mellitus. In previous large-scale studies [34,35], biguanides were found to have a negative effect on diabetic patients, particularly those with associated ischemic heart disease. Since our data were limited regarding the ischemic status of the patients, we could not verify the association between the treatment with biguanides, underlying ischemic heart disease, and the risk of mortality.

Although controversies are present in the literature [36,37], in our study, treatment with sulfonylureas showed a clear protective role for all-cause mortality.

Obesity is closely linked with T2DM etiology. The ‘obesity paradox’ observed in diabetic patients describes a particular association between BMI and mortality, with higher mortality in patients with a low BMI and lower mortality seen in overweight and obese individuals [38]. Subjects with a lower BMI (<25 kg/m²) were more likely to require treatment with insulin compared to diabetic patients with a higher BMI, and treatment with insulin was associated with a higher risk for complications and mortality [39]. Among the suggested hypotheses are certain underlying genetic abnormalities of subjects who have DM and a low-to-normal BMI or the late recognition of DM in thinner subjects, resulting in later treatment initiation.

Last, glycated hemoglobin represents the most commonly used screening tool of diabetes control in clinical practice, and the association between a high value of HbA1c and mortality in diabetic patients remains debatable as results from studies are contradictory [40,41]. In our study, a higher HbA1c was associated with higher all-cause mortality.

5. Study Limitations

Our study has several limitations. The major limitation is that the data were collected from a single center, which is a large regional emergency hospital treating more severely ill patients with poorer DM control and longer evolution of disease who require admission for treatment adjustments or DM-associated complications. This may influence the percentage of mortality, which is expected to be higher in this population. Moreover, our results were not tested on an external database to validate their reproducibility. Secondly, we did not have any data regarding the underlying cause of death in order to perform a subanalysis. Third, this study was retrospective; thus, we did not have any quality control over the data existing in the medical records. Lastly, the study involved an imbalanced cohort with a smaller representation of patients with type T1DM, and a separate subgroup analysis is warranted to better delineate the specific risk factors and predictors for T1DM and T2DM populations.

These limitations should be considered when interpreting our study findings, underlying the need for sustained efforts to enhance the robustness and applicability of the models.

6. Conclusions

We developed and tested a machine learning model that identified individuals with diabetes mellitus who are at high risk for all-cause mortality and the SHAP plot provided an overview of the impact of each parameter in the model. Key findings included the strong predictive value of low eGFR, highlighting the critical role of impaired renal function in mortality among diabetic patients. Duration of insulin therapy emerged as the second most important predictor, with its dual-directional impact suggesting complex underlying dynamics between disease progression, glycemic control, and mortality risk.