1. Introduction
Diabetic foot sepsis (DFS) is among the leading causes of hospital admission in individuals with diabetes mellitus (DM) [
1,
2]. The two most feared complications of DFS are major amputation and death [
1,
2,
3,
4,
5,
6]. An amputation of the lower extremity is considered major if it is proximal to the ankle joint [
2,
3,
4]. Around 20% of patients admitted with DFS end with a major amputation, and the in-hospital mortality is around 3–7% [
2,
4]. However, the mortality due to DFS is higher in patients with co-morbidities like hypertension, coronary artery disease and chronic kidney disease [
1,
7,
8]. A state of chronic low-grade systemic inflammation from the ongoing release of pro-inflammatory cytokines is characteristic of DM [
9]. Individuals with DM are likely to be overweight or obese, hypertensive and have coronary artery disease, all of which increase the risk of post-operative complications like surgical site infection (SSI) and acute kidney injury (AKI) [
10].
Individuals with DM are at high risk of severe COVID-19, and among the features of severe COVID-19 disease is the over-exuberant systemic inflammatory response, the so-called “cytokine-storm”, which adds to a state of chronic low-grade inflammation that is prevalent in individuals with DM [
4,
11,
12]. The risk of severe COVID-19, and its associated mortality, is higher in individuals older than 60 years of age who are obese or overweight with or without co-morbid conditions like CKD and DM [
9,
13,
14,
15,
16].
The rise of COVID-19 cases led to the implementation of various lockdown stages, which limited movement in some countries, including South Africa [
17]. Access to healthcare establishments for emergency surgical services was limited [
17]. Routine pre-operative COVID-19 PCR testing became mandatory for patients who required surgical intervention at some of the hospitals. Some hospitals assigned one theatre for surgery on all patients who tested positive for COVID-19 and required emergency surgery regardless of the acuity or severity of the illness. Patients who required debridement or amputation of DFS were often not prioritised because DFS was regarded as not immediately life-threatening. However, treatment of DFS is time-sensitive, and delay in the initiation of treatment often leads to the spread of the infection and an increased possibility of major amputation or death.
The severity and, thus, likelihood of major amputation and mortality in patients with DFS, is sometimes difficult to predict even when classification systems are used [
18,
19,
20]. Recent studies have shown the beneficial role of artificial intelligence (AI) in diagnosis, classification, treatment planning, and prediction of outcomes in patients with DFU and DFS [
21,
22,
23]. Machine learning (ML) is the most basic form of AI. Machine learning algorithms may be supervised or unsupervised. The commonly used ML algorithms include Random Forest, Support Vector Machine and K-Nearest Neighbour for classification of categorical variables, whereas Linear Regression, Decision Tree Regression and Support Vector Regression are for continuous variables [
24]. This study investigated the rate of major amputation and mortality in patients with DFS during the COVID-19 pandemic. Subsequently, the ability of ML algorithms to predict major amputation and death was studied.
2. Materials and Methods
This was a retrospective review of records of patients who were admitted and treated for DFS at a regional hospital in the Ekurhuleni District of Gauteng Province of South Africa from 1st March 2020 to 30th October 2021. Participants were identified using hospital admission records, and data were retrieved from in-hospital, theatre and laboratory findings. Extracted data were entered in an Excel spreadsheet. Reviewed records were limited to the period from admission until death or discharge. Records of consecutive patients who were 18 years or older and were admitted and treated for DFS were included, but patients who required re-admission were excluded.
Data retrieved included demography, site and severity of DFS, co-morbidities, HIV and COVID-19 status, laboratory results, type of treatment, level of amputation and outcome. Blood test results retrieved included haemoglobin level, white cell count (WCC), platelet count, C-reactive protein (CRP), potassium, urea, creatinine and glycosylated haemoglobin (HbA1c) levels. Post-operative complications and types of definitive treatment, including the level of amputations, were also captured. Demography and clinical findings were obtained from patients’ admission records and theatre notes and blood test results from laboratory records. The main outcome measures were major amputation and mortality. The main exposure variable was COVID-19 status and potential confounders were age, gender, HIV status and the level of haemoglobin and HbA1c. The effect modifiers included occurrence of post-operative complications including pneumonia, surgical site infection and acute kidney injury.
The reported in-hospital mortality of patients admitted and treated for DFS ranges from 3 to 7%. For this study, the predicted mortality rate for patients with DFS and who were COVID-19 PCR negative was 10% compared to an estimate of 20% for positive participants. The estimated overall sample size for a two-sample proportions with alpha set at <0.05, power at 80% and odd’s ratio of mortality of 2.25 in positive participants was 437. It was estimated that 10% of the records would be incomplete. The overall sample size was, therefore, 486. With 1:1 allocation, the net sample size for each arm was 243 participants. Approximately 120–150 with DFS are admitted annually to the hospital. The study was planned to review records of patients with DFS seen over a 4-year period from April 2020–March 2024. However, the number of COVID-19 cases reduced significantly from the middle of 2022.
Data Analysis
The STATA© Statistics and Data Science 17.0 Standard Edition statistical package was used for the statistical analysis. Actual numbers and percentages were used to summarise categorical data, which included gender, HIV and COVID-19 statuses, types of amputation, post-operative complications and the overall outcome. The association between the overall outcome and each of the categorical variables was tested using the two-sample proportions Pearson’s chi-squared test or Fisher’s exact test, with the latter used when the count for one of the variables was less than 5%. We summarised the continuous data using either the mean with standard deviation or median and interquartile range (IQR) if the data were parametric or non-parametric, respectively. Normally, the distribution of continuous data was tested using the Shapiro–Wilk test.
Comparisons of two group means or medians for age and blood levels of potassium, haemoglobin and HbA1c of two groups of participants used the two-sided independent t-test as they were normally distributed. We used the Kruskal–Wallis test to compare medians of WCC, CRP, platelet count, urea and creatinine of the two groups of patients. The Statistical significance was set at a
p-value below 0.05. A multivariate logistic regression was carried out to evaluate factors that had a compounding effect on mortality due to DFS in patients who had COVID-19 and adopted the purposeful selection method to determine the variables to include in multivariate logistic regression [
18]. The cut-off used for inclusion in the logistic regression was a
p-value below 0.26 following univariate analysis and less than 0.1 following multivariate logistic regression. We determined the odds of mortality with each unit change in the variables and reported it together with a 95% confidence interval. Subsequently, eight supervised ML algorithms were used to determine factors that had a correlation with mortality.
The eight supervised ML used included Logistic Regression (LG), Naïve Bayes (NB), Decision Trees (DT), Random Forest (RF), Artificial Neural Network (ANN), Support Vector Machine (SVM), AdaBoost and CN2 Rule Induction. Additional analysis was conducted following the setting of maximum depth of each decision tree of the RF a 4 through to 11, making it RF4, RF5, RF6, RF7, RF8, RF9, RF10 and RF11. Sigmoid curves were constructed to compare the performance of the RF models. Factors that were significantly associated with mortality were matched and correlated using the ML algorithms. The performance of the ML algorithms was compared using the area under the curve (AUC) of receiver operating characteristic curves. Finally, the Kaplan–Meier survival curve to com-pare the time to in-hospital mortality between COVID-19 negative and positive participants.
3. Results
There were 114 records for review; 48 (42.1%) were of female patients. The mean (SD) age of the entire group was 55 (14) years. Leg sepsis was the most prevalent diagnosis; 79 (69.3%) and right laterization 70 (61%). There were 54 (47.4%) who tested positive for COVID-19 and 41 (36%) were HIV positive. The majority—42 (36.8%) had BKA surgery, a major amputation was conducted in 79 (69.3%), and 43 (37.7%) of the patients admitted with DFS died. The median hospital stay was 11 (6–16) days. Pneumonia and SSI developed in 63 (55.3%) and 64 (56.1%), respectively. Of the 41 patients who tested positive for HIV, 56.1% (23/41) were males. Fifty-four (47.4%:54/114) patients tested positive for COVID-19, 53.7% (29/54) of whom were males. The mean (SD) age of patients who tested positive for COVID-19 was 56 (12.3) years compared to 55.5 (15.7) years of the COVID-19 negative group. Concomitant HIV and COVID-19 positive status was recorded in 21.9% (25/114) of the cases. Blood test results showed median (IQR) haemoglobin of 10.3 (9.1–12.1) g/dl, HbA1c 11.2 (9.2–12.6) and potassium 4.6 (1.2–5.4), WCC was 15 (11.9–20.1), CRP 168 (86–238), platelets 401 (260–559), urea 7.8 (4.6–14.2) and creatinine 92 µmml/L (63–192) (
Table 1).
One-hundred and twelve (98.2%: 112/114) patients had surgical intervention, comprising amputations in 86.8% (97/112) and debridement in 11.4% (13/112). Thirty-seven (32.5%: 37/114) of patients ended up with an above knee intervention while 36.8% (42/114) had below knee amputation, for a major amputation rate of 69.2% (79/114). Sixty-three (55.3%: 63/114) and 26.3% (30/114) of the patients developed pneumonia and acute kidney, respectively, during admission (
Table 2).
The median (IQR) length of stay of all the patients was 11 (7–17) days. Forty-three (37.7%:43/114) of the patients died, 62.8% (27/43) of whom were males. However, the influence of gender on mortality was not statistically significant (
p-value = 0.410). The patients who died were significantly older, at a mean (SD) age of 60.3 (15.5) years, compared to the mean age of 53 (12.6) years for those who were discharged (
p-value = 0.007). Other parameters that were statistically, and, thus, significantly different in patients who died, compared to those who were discharged, included HIV positive status of 37.2%, versus 19.7% COVID-19 infection rate of 83.7%, versus 25.4% and higher serum levels of potassium (
Table 3).
Fifteen (34.9%) of the patients who died had AKA compared to 31% of those who got discharged. The difference in treatment options between patients who died and those who were discharged was not statistically significant (
p-value = 0.178). Similarly, the difference in the rate of major amputation of 79.1% and 63.4% in patients who died and those who died was not statistically significant (
p = 0.78). However, differences in the rate of occurrence of SSI (
p-value < 0.001), AKI (
p-value < 0.001), pneumonia (
p-value < 0.001) and UTI (
p-value < 0.001) between patients who died and were discharged were statistically significant (
Table 4).
Although the lower level of platelet count (
p-value = 0.400) and the raised serum potassium (
p-value = 0.108), urea (
p-value = 0.591) and creatinine (
p-value = 0.653) were significantly associated with mortality in the univariate analysis, their influence diminished following a multivariate analysis, unlike older aged and concurrent COVID-19 and HIV infection(s) (
Table 5).
Among those who died, the mean/median was: 60 (15) years for age, 9.8 (8–11.3) for haemoglobin level, 315 (201–559) for platelets, 5.1 (4.4–5.9) for potassium, 13 (5.6–21.2) for urea, 176 (66–302) for creatinine; and the mean/median length of their hospital stay was 7 (5–16) days. Of those who died, 36 (84%) had positive COVID-19 results, and 27 (63%) were HIV positive. In relation to the information gain of our model, the following variables were the top five predictors for the outcome of death or discharge: pneumonia, COVID-19, SSI, AKI and creatinine with information gain of 0.333, 0.248, 0.195, 0.166 and 0.15, respectively.
Based on the data obtained, several models were evaluated to predict an outcome of death or discharged in patients with DFS. The models were assessed using various metrics, including Area Under the Receiver Operating Characteristic (AUC under the ROC curve), Classification Accuracy (CA), F1 Score, Precision, Recall and Matthews Correlation Coefficient (MCC). The RF model demonstrated exceptional performance with an AUC of 0.965, a CA of 0.895, an F1 score of 0.893, and a high MCC of 0.775. These metrics suggest that the RF model has a strong ability to predict outcomes accurately, with balanced precision and recall.
Gradient Boosting also performed well, achieving an AUC of 0.947, a CA of 0.886, and an F1 score of 0.885. However, it was slightly less effective than the RF model, as indicated by its lower MCC of 0.755. Similarly, the Naïve Bayes had a commendable performance with an AUC of 0.931 and an F1 score of 0.868. However, it still lagged the RF model. Both the RF and Naïve Bayes fared much better compared to Logistic Regression, SVM and Neural Network. Logistic Regression, SVM, and Neural Network had significantly lower AUC values, like 0.478 of Logistic Regression, indicating a poor predictive capability for the given dataset (
Table 6).
The ensemble technique, referred to as “Stack”, outperformed all individual models, achieving the highest AUC of 0.966, CA of 0.904, and an F1 score of 0.903. Its MCC of 0.793 also indicates strong performance and generalizability. The RF model was tested across multiple folds from RF4 to RF11 and showed consistent results throughout. The AUC remained around 0.970, with a CA of 0.895 and an F1 score of 0.893. The consistency of these results across different folds confirms the robustness of the RF model (
Table 7).
Among all the models tested, the RF Stack ensemble technique performed the best, with the highest overall metrics across the board. However, when focusing on individual models, RF5 provided the most reliable performance, particularly in terms of AUC and MCC. These findings indicate that ensemble methods (Stack) or RF can be highly effective in predicting clinical outcomes in DFU patients, offering reliable and robust predictions. The final models with a favourable F1 score of 0.919 was RF5 with an AUC of 0.967, CA of 0.895, PREC of 0.883, recall of 0.958 and MCC of 0.775.
Figure 1 compares the sigmoid curves of RF5 max, RF6 max and RF5 max.
Pneumonia and surgical site infection were stronger predictors of mortality (
Figure 2).
In comparison to the age of different biomarkers for different outcomes, the scatterplots show variations. A WCC < 30 × 10
9/L was in older patients who died, and CRP in older patients had a wide scatter range, while platelet counts were lowest in the majority of patients who died. The scatter plots indicating the correlation of age and HbA1c range show that more scatter is observed in ages over 50 years. For scatter plots showing the comparison of correlation age versus WCC, CRP, platelet count, and HbA1c, is shown in
Figure 3a–d.
Most patients died or were discharged within 20 days; the majority of those who died did not exceed 15 days across all age ranges. There was a strong positive correlation for urea and creatinine r = 0.893, CRP and WCC r = 0.681, and urea and potassium r = 0.602 (
Figure 4a,b).
The median length of hospital stay of patients was statistically significantly shorter than that of those who were discharged (
p = 0.018). The length of hospital stay of the patients was however not influenced by gender (
p = 0.2800) or COVID-19 status (
p = 0.492). Majority of the deaths in patients who tested positive for COVID-19 occurred within the first 10 days following admission to the hospital (
Figure 5).
4. Discussion
Diabetic foot sepsis is responsible for the majority of hospital admissions of individuals with DM. Measures of outcome following admission of a patient include, the number of debridement, the need and level of amputation, length of hospital stay and the 30-day mortality [
5,
6]. This study set out to investigate the influence of the COVID-19 pandemic on the rate of major amputation and in-hospital mortality in patients who had DFS. Among the main findings, there was a high rate of COVID-19 infections in patients admitted with DFS, a major amputation rate of 70% and an overall mortality rate of 38%. Additionally, most of the patients who died were above the age of 60 years, and 84% of those who died were patients who tested positive for COVID-19.
Complications of DM, including DFS, were more common in men as the current study demonstrated [
25]. Furthermore, more men with DFS tested positive for COVID-19, which is in line with previous findings [
3,
26]. Among the plausible explanations for the high rate of COVID-19 infection and mortality in men is a higher density of angiotensin-converting enzyme receptors 2 (ACE2) in their hearts, lungs, kidneys, gastrointestinal tract and blood vessels, compared to women [
27,
28]. Furthermore, men often delay seeking treatment, which might have also contributed to higher mortality [
6].
The mean age of the patients in the current study was 59 years, which is like findings from studies conducted in other countries [
3,
4,
6]. The mortality rate in our study was highest in patients who were older than 60 years, which was not surprising as the elderly are likely to have co-morbidities like coronary artery disease, hypertension and chronic kidney disease that increase the likelihood of post-operative death [
14,
26]. Some of the patients who concurrently were hypertensive might have been on ACE inhibitors. The use of ACE inhibitors can cause the up-regulation of ACE2 receptors, which increases the risk of COVID-19 infection, and the development of severe disease [
11]. The likelihood of severe COVID-19 and its complications, including death, is higher in individuals above the age of 60 [
11,
25].
Patients with DFS may be known or previously unknown with DM [
8]. Regardless of their background history, HbA1c in individuals with DM complicated by DFS is usually above 7.5%, as was the case for most patients in the current study [
7]. Serum levels of potassium, urea and creatinine in the patients who died were markedly higher than in those who survived, which was not surprising as a combination of severe sepsis due to DFS and the cytokine storm of COVID-19 increases the risk of acute kidney injury [
9,
14,
15]. The likelihood of acute injury is especially higher in individuals who are already at an increased risk of renal dysfunction, like patients with DM [
9,
14,
15]. The low platelet count levels observed in patients who died are consistent with the expectations in severe sepsis and the cytokine storm of COVID-19.
The key priorities during the management of a patient with DFS include fluid resuscitation, correction of electrolyte derangements, glycaemic control and early initiation of treatment with a broad-spectrum antibiotic [
20]. Acute kidney injury is among the common complications of DFS, as demonstrated in the study [
14]. Acute care of a patient who has DFS is, however, labour-intensive and requires hourly monitoring, which might not have been feasible during the COVID-19 pandemic [
27]. Debridement and/or amputation of DFS is conducted in theatre following fluid resuscitation, and when a patient’s hydration status, electrolyte derangements and acid-base status have been corrected, as well as when their level of blood glucose is below 15 mmol/L [
4,
8]. The need for amputation depends on the severity of the infection, and patients with spreading necrotizing infection or wet gangrene require a guillotine amputation [
4].
Of concern to the current study was the 70% rate of major amputation, which was much higher than the 12.9% reported by Aulivola et al. [
26]. Although high, the rate of major amputation in our study mirrors the findings by Cheddie and colleagues in a study carried out in KwaZulu-Natal in South Africa [
3]. What was more concerning was the 47% of major amputations that were above the knee. Interestingly, neither concurrent HIV nor COVID-19 infection had an influence on the rate of major amputation, which is contrary to the findings by Chaudhary et al., 2021 [
4] and Zayed et al., 2022 [
16].
Patients with DM are immunocompromised and prone to local and systemic post-operative complications. The current study only focused on the final amputation and did not document the overall number of procedures conducted in each patient. The most common local complication following amputation for DFS is SSI, which often necessitates “salami” amputations [
7]. The other complications that are common in patients admitted for management of DFS with or without COVID-19 are pneumonia and acute kidney injury, which was also the case in the present study [
3,
8]. The overall mortality in the study was 38%, which is four times higher than the 8.6% reported before the COVID-19 pandemic [
20]. The majority—84% of the deaths in patients with DFS was among patients who tested positive for COVID-19, which conforms with the high rate of severe COVID-19 and mortality in individuals with DM [
11]. Most deaths in the current study were in individuals above the age of 60, which is consistent with previous findings [
6]. Additionally, most of the mortalities occurred in the participants who had a major amputation, which aligns with findings from previous studies [
7,
28,
29,
30].
Recent studies have shown the beneficial role of artificial intelligence (AI) in the diagnosis, classification, treatment planning, and prediction of outcome in patients with DFU and DFS [
21,
22,
23,
24]. Machine learning (ML) is the most basic form of AI. Machine learning algorithms may be supervised or unsupervised. The commonly used ML algorithms include Random Forest, Support Vector Machine and K-Nearest Neighbour for the classification of categorical variables, whereas Linear Regression, Decision Tree Regression and Support Vector Regression are for continuous variables [
22,
24]. The RF model demonstrated exceptional performance with an AUC of 0.965, and a strong ability to predict outcomes accurately, with balanced precision and recall.
Although there are a limited number of studies on the role of ML-aided screening, diagnosis and decision-making in the management of DFS, there is a lot written on the role of AI in DM [
31,
32]. The use of AI for the screening of DM-associated peripheral neuropathy has grown exponentially [
31]. Peripheral neuropathy is the most common complication in patients with DM and is the major risk factor for DFU [
31]. Most patients with DFS had DFU, which did not heal. Once a neuropathic or neuro-ischaemic DFU has developed, the focus should be on expediting healing and preventing the development of DFS. Patients with DFS are likely to end up with a major amputation and subsequently die within 5 years [
33].
Patients with DFS require timeous and aggressive treatment to prevent amputation or death. Knowing which patients most at risk of death are is important for tailoring of the aggressiveness of the treatment. In a study by Stefanopoulos et al., using ML-algorithm ages above 40, gangrene, septic shock, low haemoglobin levels, and anaemia were among the factors that contributed significantly to the need for major amputation during admission [
34]. Similarly, following the application of ML predictive models, Xie et al. (2021) found that patients with DFS had elevated WCC and serum creatinine during admission; among other factors, this increased the likelihood of them ending up with a major amputation [
35]. Our study found that pneumonia, COVID-19, SSI, AKI and creatinine are strongly associated with mortality in patients with DFS. A study by Radunovic et al. (2023) combined categorical and continuous variables and found that among others, the age of a patient, haemoglobin levels, urea, creatinine, glomerular filtration rate and length of hospital stay were significantly associated with mortality from DFS [
31]. However, the aim of the above study was to determine the survival rate of patients with DFU over a 5-year and 10-year period.
This study was retrospective; it is likely that we missed some records. The sample size is relatively small, and, thus, the sample was not divided into training, testing and validation set. A small sample size might have led to over-fitting. We did not investigate the influence of potential compounding factors, such as the expertise involved in treatment decisions and waiting time for theatre, and the study was based at one facility. Therefore, the findings may not be generalizable.