Next Article in Journal
CAM-Xenograft Model Provides Preclinical Evidence for the Applicability of [68Ga]Ga-Pentixafor in CRC Imaging
Previous Article in Journal
The Influence of the Ketogenic Diet on the Immune Tolerant Microenvironment in Glioblastoma
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 14
Issue 22
10.3390/cancers14225551
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

External Validation of a Risk Model for Severe Complications following Pancreatoduodenectomy Based on Three Preoperative Variables

by
Zahraa M. Alhulaili
1,
Rick G. Pleijhuis
2,
Maarten W. Nijkamp
1 and
Joost M. Klaase
1,*
1
Department of Hepato-Pancreato-Biliary Surgery and Liver Transplantation, University Medical Center Groningen, University of Groningen, P.O. Box 30001, 9700 RB Groningen, The Netherlands
2
Department of Internal Medicine, University Medical Center Groningen, University of Groningen, P.O. Box 30001, 9700 RB Groningen, The Netherlands
*
Author to whom correspondence should be addressed.
Cancers 2022, 14(22), 5551; https://doi.org/10.3390/cancers14225551
Submission received: 2 October 2022 / Revised: 4 November 2022 / Accepted: 6 November 2022 / Published: 11 November 2022
Browse Figures
Versions Notes

Abstract

:

Simple Summary

Up to 30% of patients develop severe complications following pancreatoduodenectomy (PD). With respect to risk stratification and shared decision making, prediction models to predict complications are crucial. In 2015, a risk model for severe complications was developed by Schroder et al. based on three preoperative variables: BMI, ASA classification and mean Hounsfield Units of the pancreatic body on the preoperative abdominal CT scan. However, external validation of this model has not yet been performed. It is important to validate prediction models externally before implementing them in clinical practice to confirm their accuracy and generalizability when applied to a different patient population. Our aim was to externally validate this risk prediction model using an independent cohort of patients.

Abstract

Background: Pancreatoduodenectomy (PD) is the only cure for periampullary and pancreatic cancer. It has morbidity rates of 40–60%, with severe complications in 30%. Prediction models to predict complications are crucial. A risk model for severe complications was developed by Schroder et al. based on BMI, ASA classification and Hounsfield Units of the pancreatic body on the preoperative CT scan. These variables were independent predictors for severe complications upon internal validation. Our aim was to externally validate this model using an independent cohort of patients. Methods: A retrospective analysis was performed on 318 patients who underwent PD at our institution from 2013 to 2021. The outcome of interest was severe complications Clavien–Dindo ≥ IIIa. Model calibration, discrimination and performance were assessed. Results: A total of 308 patients were included. Patients with incomplete data were excluded. A total of 89 (28.9%) patients had severe complications. The externally validated model achieved: C-index = 0.67 (95% CI: 0.60–0.73), regression coefficient = 0.37, intercept = 0.13, Brier score = 0.25. Conclusions: The performance ability, discriminative power, and calibration of this model were acceptable. Our risk calculator can help surgeons identify high-risk patients for post-operative complications to improve shared decision-making and tailor perioperative management.

1. Introduction

Pancreatoduodenectomy (PD) is the operative management for patients with pancreatic head cancer and malignant or premalignant periampullary pathologies [1,2]. The mortality rates following this surgery have decreased over time to less than 5% [2,3,4,5]. This reduction in mortality rates is due to the improvements in perioperative management and the advances in surgical skills and procedures in combination with centralization [2,6]. The indications for this operation have been expanded to incorporate benign pathologies which may reflect on the increase of survival rates as well [5]. Despite the reduction in mortality rates, the overall complication rates remain relatively high, ranging from 40–60%, with up to 30% with severe complications [1,4,5,7,8].
Many severe complications can occur following PD, the most frequently mentioned in the literature are post-operative pancreatic fistula (POPF), delayed gastric emptying (DGE) and post-pancreatectomy hemorrhage (PPH). All three complications are classified according to the International Study Group of Pancreatic Surgery (ISGPS) classification system [9]. DGE is the most common complication following PD with an incidence rate of 20–50% [3,10]. POPF is a serious and life-threatening complication with incidence rates between 5 and 40% [3]. POPF is graded according to the ISGPS definition in which grade A, also known as biochemical leakage, is not clinically relevant [11]. Grades B and C result in negative consequences for the patient [11]. PPH and deep surgical site infection are complications that can be related to POPF with incident rates from 3% to 16% and from 12 to 51%, respectively [12,13]. Less frequently occurring is bile duct leakage [5].
To classify the severity of complications after gastrointestinal surgical procedures, the Clavien–Dindo classification system is commonly used [14]. This classification consists of five grades in which grades three and four are further subdivided into A and B subgrades, and grade five indicates mortality [3,14]. Each grade depends on the required intervention to resolve the complications, the higher the grade is, the more complex the intervention needed [3,14]. In the present study, severe complications were defined as grade IIIa or higher. This means that the complications at least demand surgical, radiological, or endoscopic interventions under local anesthesia [3,14]. There are multiple pre-operative risk factors that may increase the occurrence of complications, such as advanced age, high body mass index (BMI), male gender, high American Society of Anesthesiology (ASA) classification, and comorbidities such as diabetes and cardiovascular disease [15,16,17]. The most important intra-operative risk factors are a soft pancreatic texture, a small pancreatic duct less than 3 mm, intra-operative blood loss, pancreatic steatosis and/or the absence of pancreatic fibrosis [17,18,19].
Several risk calculators for pancreatic surgery have been developed over the past decades. Most researchers focused with their risk calculators on survival or POPF [20,21,22]. Nevertheless, there are in general very limited risk calculators for severe complications [1]. For the individual patient the risk of severe complications (Clavien–Dindo classification IIIa or more) outweighs the risk of POPF only. The risk calculators, especially when they are based on pre-operatively known risk factors, can be used during the informed consent process [2]. It is also important for the surgeon to make a well-considered decision regarding an individual’s surgical risks and evaluate whether the benefits for such an operation outweighs the risks [2]. Several precautions can be taken, such as, additional preoperative assessment, choosing the most suitable reconstruction technique, careful post-operative monitoring, and better preparation for other possible complications [6,8,23]. Another important argument to estimate the post-operative risk is to reduce the adverse consequences of the complications. It is known that complications can lead to multiple organ failure or mortality, prolong hospital stay and/or required readmission and increased costs [24,25,26]. A complicated course after pancreatoduodenectomy may lead to impaired long-term survival [27].
Recently, a risk model for severe complications after PD was developed by Schroder et al. based on three preoperative variables: ASA classification, BMI, and mean Hounsfield Units (HU) of pancreatic body on the preoperative CT scan [28]. The authors reported that high BMI, ASA III classification and low mean Hounsfield Units of the pancreatic body on the preoperative abdominal CT scan were independent predictors for severe complications upon internal validation. However, no external model validation has yet been performed to confirm the robustness and generalizability of this model. In the current study, we sought to externally validate the above-mentioned risk model using an independent and larger cohort of patients treated at our institution.

2. Materials and Methods

2.1. Study Design and Study Population

The present study is a retrospective cohort study. Patients who underwent pancreatic surgery were registered in the mandatory national registration of pancreatic surgery, the Dutch Pancreatic Cancer Audit (DPCA) database. This electronic database was used in our study. The medical records of 318 consecutive patients who underwent pancreatoduodenectomy (PD) or pylorus-preserving pancreaticoduodenectomy (PPPD) at the University Medical Center Groningen in the Netherlands between the periods January 2013 and December 2021 were reviewed.

2.2. Data Collection and Endpoint Definition

The following data were collected for each patient: age, gender, Body Mass Index (BMI), American Society of Anesthesiologist (ASA) classification, date of operation, type of operation, date of the pre-operative abdominal CT scan, CT scan phase, slice thickness, and Hounsfield Units (HU) of the pancreatic body. The BMI and ASA classification were identified from the pre-operative anesthesiology report. HU of the pancreatic body were measured on pre-operative contrast enhanced portal venous CT scans using a region of interest (ROI) greater than 1 cm without involvement of the pancreatic duct or any of the adjacent blood vessels. Slice thicknesses ranged between 0.75 and 5.00 mm.
Complications were scored using the Clavien–Dindo classification system. In our study, the outcome of interest was the occurrence of severe complications defined as Clavien–Dindo ≥ IIIa within 30 days after surgery. Both the researcher and the data analyzer were blinded for the outcome of interest. The Clavien–Dindo classifications were made available to the researcher when all measurements were completed. The sample was then tested by the online calculator and three risk groups were obtained, namely low, intermediate, and high (Figure 1). The same applied to data validation. An anonymized dataset was used by the data analyzer to validate the study.

2.3. Statistical Analysis

Statistical analysis of the study population was conducted using SPSS Statistics (IBM Corp., version 28, Armonk, NY, USA). Continuous variables were reported as mean ± standard deviation (SD) when the variables were normally distributed or as median and interquartile ranges (IQR) when the variables were non-normally distributed. Categorical variables were expressed as percentages and frequencies. A p value was calculated using a two-tailed t-test and a p < 0.05 was considered statically significant. To validate the prediction model, the collected anonymized data of the validation cohort was entered into the Evidencio platform (Evidencio BV, version 3.12, Haaksbergen, The Netherlands). This platform facilitates external validation through an online semi-automated validation tool, calculating multiple model performance parameters simultaneously [29].

2.4. Evaluated Parameters of Model Performance

Calibration was assessed both visually (calibration plot) and through calculation of the slope and intercept, evaluating the level of agreement between predicted probabilities versus observed outcomes. Model discrimination was assessed by calculating the area under the receiver-operating characteristic curve (C-statistic with 95% confidence interval). A classification plot was generated to plot sensitivity versus 1-specificity. Overall model performance was evaluated using the Brier score as a composite measure of both model discrimination and calibration. The scaled Brier score was calculated to take the baseline prevalence of severe post-operative complications into account.

2.5. Ethics

This study falls within the scope of the non–WMO system. The Dutch Medical Research with Human Subjects Law is not applicable to this study. All other necessary steps have already been completed. There was no need for approval from the Medical Ethical Evaluation Committee (METc) at our hospital.

3. Results

3.1. Baseline Characteristics of the Patients

A total of 308 patients who underwent PD or PPPD at our institution and for whom data was complete were included. Ten patients were excluded due to the presence of inappropriate or incomplete CT scans. There were no other exclusion criteria applied, such as vascular resection, borderline tumor and neoadjuvant therapy. The median age of the study cohort was 67 years (range 59–73), consisting of slightly more males (52.9%, n = 163) than females. PPPD was performed in 245 patients (79.5%), whereas PD in 63 patients (20.5%). The majority of the patients had ASA II classification (64.3%, n = 198) and the mean BMI was 26.6 ± 11.8. The median range of HU of the pancreatic body as measured on a pre-operative CT scan was 85.3 (range 70.7–96.4) with lower ranges in those patients who had severe complications being 79.6 (range 62.9–89.9). The most common pathology was adenocarcinoma (57.1%, n = 176), followed by intestinal type adenoma and neuroendocrine neoplasm (11.7%, n = 36 and 7.8%, n = 24, respectively). Severe complications defined as Clavien–Dindo IIIa or higher occurred in 89 (28.9%) patients. Baseline characteristics of this validation cohort are reported in Table 1.
The most common complication in our cohort was chylous leakage followed by DGE, 31.5% and 26%, respectively. Chylous leakage was mainly grade A (19.5%, n = 60), whereas DGE was mainly grades A or B (10.4%, n = 32 and 10.7%, n = 33, respectively). The superficial surgical site infection rate was 18.2% (n = 56). There were 44 patients who developed clinically relevant POPF, grade B (12.7%, n = 39) or grade C (1.6%, n = 5). The incidence of PPH grade B was more than grades A and C (3.6%, n = 11, 2.6%, n = 8 and 0.6%, n = 2, respectively). Pneumonia and bile leakage were rare in our cohort, only 10 patients had either of these complications. Mortality rate was 1.6 (n = 5). Observed post-operative complications are summarized in Table 2 and Figure 2.
There were similarities and differences in some variables observed in the validation cohort in comparison to the development cohort. Both cohorts were comparable in terms of age (65 ± 10.6 and 66 ± 9.3, respectively), more males (52.9%, n = 163 and 61.8%, n = 68, respectively), HU of the pancreatic body (85.3 (70.7–96.4) and 79.4 (61.1–92.9), respectively), more ASA II (64.3%, n = 198 and 90.9%, n = 100), and BMI (26.6 ± 11.8 and 25 ± 3.7, respectively). However, PPPD was performed less in the development cohort, namely 64.5% compared to 79.5% in the validation cohort. Severe complications occurred more frequently in the development cohort compared to the validation cohort, 33% and 28.9%, respectively.

3.2. Model Discrimination

Our model demonstrated a C-statistic of 0.67 (95% confidence interval (CI): 0.60–0.73). The C-index can range between 0.5 and 1.0, in which values of 0.5 to 0.6, 0.6 to 0.7, 0.7 to 0.8, and 0.8 to 0.9 indicate poor, fair, moderate, and good model discrimination, respectively, while a value of 1.0 implies perfect discrimination. In our study C-statistic of 0.67 implies fair discrimination. A classification plot was generated to display the sensitivity and 1-specificity of the model (Figure 3).

3.3. Model Calibration

A calibration plot was generated in which the x-axis represents the predicted and the y-axis the observed severe complication rates (Figure 4). This model had a slope of 0.37 and an intercept of 0.13 representing an acceptable model.

3.4. Overall Model Performance

The validated model achieved a Brier score of 0.25, with a score of 0 suggesting good accuracy and a score of 1 suggesting inaccuracy of the model. The calculated scaled Brier score was <0.001. This indicates moderate accuracy of the model.

4. Discussion

The aim of the present study was to externally validate the risk model for severe complications after PD developed by Schroder et al. consisting of three variables: BMI, ASA classification and mean HU of the pancreatic body on the pre-operative abdominal CT scan. Our validation cohort consisted of 308 patients who underwent PD or PPPD at our institution. Baseline characteristics were comparable between the development and validation cohorts. Severe complications occurred more often in the development cohort, namely 33%. In our study the severe complication rate was 28.9%, which is lower than other studies [1,23]. Assessment of the model’s performance parameters resulted in a C-index of 0.67, a regression slope of 0.37, an intercept of 0.13, a Brier score of 0.25, and a scaled Brier score of <0.001. Based on these parameters, we considered it a fair risk model for prediction of severe complications.
The risk of developing pancreatic cancer is higher amongst elderly people [30]. As the population is aging, more people are developing pancreatic or periampullary cancer which can only be cured with surgery. A complex procedure such as PD with a severe complication risk rate up to 30% is more dangerous at advanced age due to the presence of comorbidities, such as cardiovascular diseases and diabetes mellitus [31]. A pre-operative risk calculator for severe complications may help to identify high-risk patients, thereby guiding clinical decision-making. Prehabilitation might also be effective in patients with comorbidities [32]. Frail patients can undergo a prehabilitation program to increase their general fitness and anerobic threshold while awaiting the surgery. This could decrease the prevalence of post-operative complications [33]. Moreover, the surgical technique can be adjusted, for example, an isolated roux loop for the pancreatic anastomosis can be used separating the pancreatojejunostomy and hepatojejunostomy [33].
The majority of the published risk models use one or more intra-operative risk factors, for example, pancreatic texture and intra-operative blood loss. Our risk calculator is unique in this respect as it only requires pre-operative variables. ASA classification and BMI can be obtained from the anesthesiology report which is made during the pre-operative assessment. The HU of the pancreatic body can be easily measured on the pre-operative CT scan. All measurements can be performed by the surgeons themselves and it does not require assistance from a radiologist nor special applications. All measurements can be conducted on the desktop. When all variables are available, they can be entered into the online calculator. Based on their predicted individual risk, patients are than stratified into one of three categorical risk groups: low, intermediate, or high.
Compared with other risk prediction models, this model was developed to estimate the risk of severe complications following PD. Most existing models evaluate only the risk of POPF and do not involve other complications. Although POPF is a serious complication, focusing on the POPF risk alone would lead to a significant underestimation of complication rates. In addition, an adequate explanation of the risk model and how it can be used by surgeons is not always specified. Furthermore, there were few risk models that used the pre-operative CT scan to predict post-operative complications without developing a risk calculator [34,35]. Notably, some researchers studied the role of the pre-operative CT scan to predict the risk of POPF alone [36,37,38,39,40]. Table 3 provides a summary of the risk calculators that were established to estimate the risk of severe complications after PD.
Amongst the risk calculators that focus on all types of severe complications is the Braga calculator [4]. It has been externally validated in the Joliat G et al. series and it showed a good accuracy with a C-index of 0.99, Hosmer–Lemeshow test p = 0.82 [23]. Despite good accuracy, this study has some limitations, as it involved only patients who had PD as a resection technique and not PPPD. The results may have been biased because the Clavien–Dindo classification was published in 2004 and these two studies included patients before this period. Moreover, this calculator uses many intra-operative variables which makes it unsuitable for use during the pre-operative phase. The pancreatoduodenectomy risk tool published by Abbas A et al. is available online [24]. It showed a C-index for severe complications of 0.63; however, no model calibration measure was reported, and this study also has several limitations. The model requires many variables in which pancreatic texture was considered as a pre-operative variable and evaluated based on the type of pathology. Likewise, no clear classification was provided in terms of severe complications and their definitions. The risk model for complications established by Chen L et al. also involves pancreatic texture as an intra-operative variable, and it is graded subjectively during the operation [1]. Clavien–Dindo of II or higher was considered significant, whereas most studies reported in the literature use alternative cut-offs (IIIa, IIIb, or higher) [2,4,8,23,28]. The PREPARE calculator was developed by Uzunoglu F et al. and validated in Rodriguez-Lopez M et al. series [41,42]. It demonstrated a moderate accuracy with an AUC of 0.71 (95% CI: 0.56–0.87) and Hosmer–Lemeshow, p = 0.86, yet the validation cohort incorporated a small number of patients (n = 50), there were patients who underwent distal pancreatectomy, and it was not mentioned how many patients had this type of resection. This might have had an impact on the outcomes because this type of resection is associated with a lower morbidity rate [41,42]. The risk calculator developed by Wiltberger G et al. used preoperative variables and there was no pancreas specific variable [16]. It has not been validated yet as well [16]. The risk model of Aoki S et al. involved 20 pre-operative variables and one intra-operative variable (vascular resection) [2]. Severe complications were defined as Clavien–Dindo ≥ IV and the AUC was 0.71 (95% CI: 0.69–0.73). Again, no calibration measure was reported, and no external validation has been performed yet.
There were many studies established using the American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) database. For example, a procedure-specific ACS-NSQIP was developed by McMillan M et al. and comparable to the non-procedure-specific ACS-NSQIP calculator for general surgery [43]. It differs in using fistula Risk score (FRS), hospital volume and surgical skills as additional predictors for complication risk. The C-statistic was 0.65, Brier score 0.19, and Spiegelhalter’s z-test p = 0.189. Nevertheless, FRS calculation requires intra-operative blood loss and pancreatic texture which makes it inappropriate for pre-operative usage. Another pancreas specific calculator is the ACS-NSQIP pancreatoduodenectomy risk calculator which uses 10 preoperative variables [7]. The authors reported a C-statistic of 0.61 upon internal validation of the model, while no external validation was performed. In comparison, we found a C-statistic of 0.67 for our model when assessed in an independent patient cohort. Furthermore, the ACS-NSQIP uses many variables and involved approximately 34% distal pancreatectomies which may reduce the reliability of the outcomes. The study by Greenblatt D et al. used the same database [44]. It requires many variables in which no pancreas specific variable was used, and severe complications were not defined according to the Clavien–Dindo definition.

Strengths and Limitations

We have successfully validated the novel model to predict severe complications after PD. Model discrimination and calibration were considered fair. An important strength of our study was the simultaneous evaluation of multiple model performance measurements, providing a comprehensive indication of overall model performance [45]. Another strength is that our model is unique and easy to use. The necessary instructions for the calculation are available in the online calculator. Moreover, in this study the researchers were blinded while collecting and analyzing the data to prevent bias and to acquire trustworthy validation outcomes.
To our knowledge, this is the first risk calculator that incorporates pre-operative CT scans in estimating the risk of severe complications. Until present, researchers have used the pre-operative CT scan in developing calculators for POPF alone and not all severe complications together [37,40]. HU of the pancreatic body on a portal venous CT scan is a measure of the fatty content of the pancreas. This means that if the HU is low, there will be more fat present. It is known that a fatty pancreas increases the risk of complications [28]. However, until today this can only be evaluated on CT or MRI scans pre-operatively or subjectively during the operation. It is remarkable that, until now, imaging scans, which are always available before surgery, are not frequently used to measure the pancreatic texture. Furthermore, our calculator includes BMI, which is a morbidity measure, ASA classification which is an indicator for the patient’s condition to tolerate surgery, and a pancreas specific variable (HU of the pancreatic body).
The current study has several limitations. First, we performed a retrospective validation based on a single center cohort of patients. This may result in a selection bias because each hospital has specific protocols when deciding which patients can undergo the surgery and while selecting the reconstruction technique. Second, this score is developed and validated to predict all possible complications based on severity lacking the ability to predict a specific complication. Therefore, it is difficult to estimate beforehand the type of complications that may occur. Third, the development study did not quantify the type of complications that occurred post-operatively, making it difficult to compare the two cohorts in this respect.
Our risk calculator is validated on patients who underwent an open PD or PPPD. We recommend further validation of our model on patients who underwent robotic-assisted PD, as we believe that this procedure will be performed more in the future. Finally, we would like to encourage future researchers to validate the existing prediction models instead of developing new ones. External validation is considered an essential step prior to actual application of prediction models in the clinical practice. This applies to our model as well. We invite other researchers to validate our model in different group of patients.

5. Conclusions

Our risk model was superior to most of the established risk models predicting severe complications after PD. Assessment of the different model parameters of discrimination, calibration and performance indicated an acceptable overall model for predicting severe complications. Our model can be used by surgeons to predict the risk of severe complications and take the necessary measures in perioperative care of their patients.

Author Contributions

Z.M.A. study design, data collection, data analysis, writing and reviewing the manuscript. R.G.P. Data validation and analysis, writing and reviewing the manuscript. M.W.N. writing and reviewing the manuscript. J.M.K. study design, writing and reviewing the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research has not received any funding to be conducted.

Institutional Review Board Statement

There was no need for ethical approval from the local committee of our hospital according to the institutional requirements.

Informed Consent Statement

Patient consent was waived due to local ethics committee requirements at our hospital.

Data Availability Statement

Data can be provided anonymously when requested.

Acknowledgments

The authors would like to thank Allard Wijma for his assistance with data retrieval.

Conflicts of Interest

R.G.P. reports a minority stake in Evidencio B.V., an online platform offering free services regarding the creation, validation, and implementation of clinical prediction models. Evidencio was not involved in the development of any of the prediction models mentioned nor is expected to experience financial gain by the publication of this manuscript.

References

  1. Chen, L.; Peng, L.; Wang, C.; Li, S.C.; Zhang, M. New score for prediction of morbidity in patients undergoing open pancreaticoduodenectomy. J. Int. Med. Res. 2021, 49, 3000605211001984. [Google Scholar] [CrossRef] [PubMed]
  2. Aoki, S.; Miyata, H.; Konno, H.; Gotoh, M.; Motoi, F.; Kumamaru, H.; Wakabayashi, G.; Kakeji, Y.; Mori, M.; Seto, Y.; et al. Risk factors of serious postoperative complications after pancreaticoduodenectomy and risk calculators for predicting postoperative complications: A nationwide study of 17,564 patients in Japan. J. Hepatobiliary Pancreat. Sci. 2017, 24, 243–251. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. DeOliveira, M.L.; Winter, J.M.; Schafer, M.; Cunningham, S.C.; Cameron, J.L.; Yeo, C.J.; Clavien, P.A. Assessment of complications after pancreatic surgery: A novel grading system applied to 633 patients undergoing pancreaticoduodenectomy. Ann. Surg. 2006, 244, 931–937; Discussion 937–939. [Google Scholar] [CrossRef] [PubMed]
  4. Braga, M.; Capretti, G.; Pecorelli, N.; Balzano, G.; Doglioni, C.; Ariotti, R.; Di Carlo, V. A prognostic score to predict major complications after pancreaticoduodenectomy. Ann. Surg. 2011, 254, 702–707; Discussion 707–708. [Google Scholar] [CrossRef] [PubMed]
  5. Gervais, D.A.; Fernandez-del Castillo, C.; O’Neill, M.J.; Hahn, P.F.; Mueller, P.R. Complications after pancreatoduodenectomy: Imaging and imaging-guided interventional procedures. Radiographics 2001, 21, 673–690. [Google Scholar] [CrossRef]
  6. Simon, R. Complications after Pancreaticoduodenectomy. Surg. Clin. N. Am. 2021, 101, 865–874. [Google Scholar] [CrossRef]
  7. Parikh, P.; Shiloach, M.; Cohen, M.E.; Bilimoria, K.Y.; Ko, C.Y.; Hall, B.L.; Pitt, H.A. Pancreatectomy risk calculator: An ACS-NSQIP resource. HPB 2010, 12, 488–497. [Google Scholar] [CrossRef] [Green Version]
  8. Kirks, R.C.; Cochran, A.; Barnes, T.E.; Murphy, K.; Baker, E.H.; Martinie, J.B.; Iannitti, D.A.; Vrochides, D. Developing and validating a center-specific preoperative prediction calculator for risk of pancreaticoduodenectomy. Am. J. Surg. 2018, 216, 498–505. [Google Scholar] [CrossRef]
  9. Wente, M.N.; Bassi, C.; Dervenis, C.; Fingerhut, A.; Gouma, D.J.; Izbicki, J.R.; Neoptolemos, J.P.; Padbury, R.T.; Sarr, M.G.; Traverso, L.W.; et al. Delayed gastric emptying (DGE) after pancreatic surgery: A suggested definition by the International Study Group of Pancreatic Surgery (ISGPS). Surgery 2007, 142, 761–768. [Google Scholar]
  10. Penumadu, P.; Barreto, S.G.; Goel, M.; Shrikhande, S.V. Pancreatoduodenectomy—Preventing complications. Indian J. Surg. Oncol. 2015, 6, 6–15. [Google Scholar] [CrossRef] [Green Version]
  11. Bassi, C.; Marchegiani, G.; Dervenis, C.; Sarr, M.; Hilal, M.A.; Adham, M.; Allen, P.; Andersson, R.; Asbun, H.J.; Besselink, M.G.; et al. The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 Years After. Surgery 2017, 161, 584–591. [Google Scholar] [CrossRef] [PubMed]
  12. Floortje van Oosten, A.; Smits, F.J.; van den Heuvel, D.A.F.; van Santvoort, H.C.; Molenaar, I.Q. Diagnosis and management of postpancreatectomy hemorrhage: A systematic review and meta-analysis. HPB 2019, 21, 953–961. [Google Scholar] [CrossRef] [PubMed]
  13. Suragul, W.; Rungsakulkij, N.; Vassanasiri, W.; Tangtawee, P.; Muangkaew, P.; Mingphruedhi, S.; Aeesoa, S. Predictors of surgical site infection after pancreaticoduodenectomy. BMC Gastroenterol. 2020, 20, 201. [Google Scholar] [CrossRef] [PubMed]
  14. Morand, G.B.; Anderegg, N.; Kleinjung, T.; Bohlender, J.E.; Veraguth, D.; Broglie, M.A.; Holzmann, D.; Huber, A.M.; Röösli, C.; Soyka, M.B. Assessment of Surgical Complications With Respect to the Surgical Indication: Proposal for a Novel Index. Front. Surg. 2021, 8, 638057. [Google Scholar] [CrossRef] [PubMed]
  15. Sandini, M.; Bernasconi, D.P.; Ippolito, D.; Nespoli, L.; Baini, M.; Barbaro, S.; Fior, D.; Gianotti, L. Preoperative Computed Tomography to Predict and Stratify the Risk of Severe Pancreatic Fistula after Pancreatoduodenectomy. Medicine 2015, 94, e1152. [Google Scholar] [CrossRef]
  16. Wiltberger, G.; Muhl, B.; Benzing, C.; Atanasov, G.; Hau, H.M.; Horn, M.; Krenzien, F.; Bartels, M. Preoperative risk stratification for major complications following pancreaticoduodenectomy: Identification of high-risk patients. Int. J. Surg. 2016, 31, 33–39. [Google Scholar] [CrossRef]
  17. Roberts, K.J.; Hodson, J.; Mehrzad, H.; Marudanayagam, R.; Sutcliffe, R.P.; Muiesan, P.; Isaac, J.; Bramhall, S.R.; Mirza, D.F. A preoperative predictive score of pancreatic fistula following pancreatoduodenectomy. HPB 2014, 16, 620–628. [Google Scholar] [CrossRef] [Green Version]
  18. Guilbaud, T.; Birnbaum, D.J.; Lemoine, C.; Chirica, M.; Risse, O.; Berdah, S.; Girard, E.; Moutardier, V. C-Reactive Protein on Postoperative Day 1 Is a Reliable Predictor of Pancreas-Specific Complications After Pancreaticoduodenectomy. J. Gastrointest. Surg. 2018, 22, 818–830. [Google Scholar] [CrossRef]
  19. Mathur, A.; Pitt, H.A.; Marine, M.; Saxena, R.; Schmidt, C.M.; Howard, T.J.; Nakeeb, A.; Zyromski, N.J.; Lillemoe, K.D. Fatty pancreas: A factor in postoperative pancreatic fistula. Ann. Surg. 2007, 246, 1058–1064. [Google Scholar] [CrossRef]
  20. Callery, M.P.; Pratt, W.B.; Kent, T.S.; Chaikof, E.L.; Vollmer, C.M. A prospectively validated clinical risk score accurately predicts pancreatic fistula after pancreatoduodenectomy. J. Am. Coll. Surg. 2013, 216, 1–14. [Google Scholar] [CrossRef]
  21. Latenstein, A.E.J.; the Dutch Pancreatic Cancer Group; Van Roessel, S.; Msc, L.G.M.V.D.G.; Bonsing, B.A.; Dejong, C.H.C.; Koerkamp, B.G.; De Hingh, I.H.J.T.; Homs, M.Y.V.; Klaase, J.M.; et al. Conditional Survival After Resection for Pancreatic Cancer: A Population-Based Study and Prediction Model. Ann. Surg. Oncol. 2020, 27, 2516–2524. [Google Scholar] [CrossRef] [PubMed]
  22. van Roessel, S.; Strijker, M.; Steyerberg, E.W.; Groen, J.V.; Mieog, J.S.; Groot, V.P.; He, J.; De Pastena, M.; Marchegiani, G.; Bassi, C.; et al. International validation and update of the Amsterdam model for prediction of survival after pancreatoduodenectomy for pancreatic cancer. Eur. J. Surg. Oncol. 2020, 46, 796–803. [Google Scholar] [CrossRef] [PubMed]
  23. Joliat, G.R.; Petermann, D.; Demartines, N.; Schäfer, M. Prediction of Complications after Pancreaticoduodenectomy: Validation of a Postoperative Complication Score. Pancreas 2015, 44, 1323–1328. [Google Scholar] [CrossRef] [PubMed]
  24. Al Abbas, A.I.; Borrebach, J.D.; Pitt, H.A.; Bellon, J.; Hogg, M.E.; Zeh, H.J.; Zureikat, A.H. Development of a Novel Pancreatoduodenectomy-Specific Risk Calculator: An Analysis of 10,000 Patients. J. Gastrointest. Surg. 2021, 25, 1503–1511. [Google Scholar] [CrossRef]
  25. Smits, F.J.; Verweij, M.E.; Daamen, L.A.; van Werkhoven, C.H.; Goense, L.; Besselink, M.G.; Bonsing, B.A.; Busch, O.R.; Van Dam, R.M.; Van Eijck, C.H.; et al. Impact of Complications after Pancreatoduodenectomy on Mortality, Organ Failure, Hospital Stay, and Readmission: Analysis of a Nationwide Audit. Ann. Surg. 2022, 275, e222–e228. [Google Scholar] [CrossRef]
  26. Linnemann, R.J.; Kooijman, B.J.; van der Hilst, C.S.; Sprakel, J.; Buis, C.I.; Kruijff, S.; Klaase, J.M. The Costs of Complications and Unplanned Readmissions after Pancreatoduodenectomy for Pancreatic and Periampullary Tumors: Results from a Single Academic Center. Cancers 2021, 13, 6271. [Google Scholar] [CrossRef]
  27. Bonaroti, J.W.; Zenati, M.S.; Al-Abbas, A.I.; Rieser, C.J.; Zureikat, A.H.; Hogg, M.E.; Zeh, H.J.; Boone, B.A. Impact of postoperative pancreatic fistula on long-term oncologic outcomes after pancreatic resection. HPB 2021, 23, 1269–1276. [Google Scholar] [CrossRef]
  28. Schröder, F.F.; de Graaff, F.; Bouman, D.E.; Brusse-Keizer, M.; Slump, K.H.; Klaase, J.M. The Preoperative CT-Scan Can Help to Predict Postoperative Complications after Pancreatoduodenectomy. BioMed Res. Int. 2015, 2015, 824525. [Google Scholar] [CrossRef] [Green Version]
  29. van Steenbeek, C.D.; van Maaren, M.C.; Siesling, S.; Witteveen, A.; Verbeek, X.A.A.M.; Koffijberg, H. Facilitating validation of prediction models: A comparison of manual and semi-automated validation using registry-based data of breast cancer patients in the Netherlands. BMC Med. Res. Methodol. 2019, 19, 117. [Google Scholar] [CrossRef]
  30. Higuera, O.; Ghanem, I.; Nasimi, R.; Prieto, I.; Koren, L.; Feliu, J. Management of pancreatic cancer in the elderly. World J. Gastroenterol. 2016, 22, 764–775. [Google Scholar] [CrossRef]
  31. Sperti, C.; Moletta, L.; Pozza, G. Pancreatic resection in very elderly patients: A critical analysis of existing evidence. World J. Gastrointest. Oncol. 2017, 9, 30–36. [Google Scholar] [CrossRef] [PubMed]
  32. Ngo-Huang, A.; Parker, N.H.; Bruera, E.; Lee, R.E.; Simpson, R.; O’Connor, D.P.; Petzel, M.Q.B.; Fontillas, R.C.; Schadler, K.; Xiao, L.; et al. Home-Based Exercise Prehabilitation During Preoperative Treatment for Pancreatic Cancer Is Associated with Improvement in Physical Function and Quality of Life. Integr. Cancer Ther. 2019, 18, 1534735419894061. [Google Scholar] [CrossRef] [PubMed]
  33. van Wijk, L.; Bongers, B.C.; Berkel, A.E.; Buis, C.I.; Reudink, M.; Liem, M.S.; Slooter, G.D.; van Meeteren, N.L.; Klaase, J.M. Improved preoperative aerobic fitness following a home-based bimodal prehabilitation programme in high-risk patients scheduled for liver or pancreatic resection. Br. J. Surg. 2022, 109, 1036–1039. [Google Scholar] [CrossRef] [PubMed]
  34. Hikita, K.; Chiba, N.; Nakagawa, M.; Koganezawa, I.; Yokozuka, K.; Kobayashi, T.; Sano, T.; Tomita, K.; Tsutsui, R.; Hirano, H.; et al. Efficacy of Peak Hounsfield Units of the Visceral Fat Area in Predicting Postoperative Complications after Pancreaticoduodenectomy. Dig. Surg. 2020, 37, 331–339. [Google Scholar] [CrossRef] [PubMed]
  35. McAuliffe, J.C.; Parks, K.; Kumar, P.; McNeal, S.F.; Morgan, D.E.; Christein, J.D. Computed tomography attenuation and patient characteristics as predictors of complications after pancreaticoduodenectomy. HPB 2013, 15, 709–715. [Google Scholar] [CrossRef] [Green Version]
  36. Savin, M.; Mihai, F.; Gheorghe, L.; Ursulescu, C.L.; Negru, D.; Trofin, A.; Zabara, M.; Nutu, V.; Cadar, R.; Blaj, M.; et al. Proposal of a Preoperative CT-Based Score to Predict the Risk of Clinically Relevant Pancreatic Fistula after Cephalic Pancreatoduodenectomy. Medicina 2021, 57, 650. [Google Scholar] [CrossRef]
  37. Emekli, E.; Gündoğdu, E. Computed tomography evaluation of early post-operative complications of the Whipple procedure. Pol. J. Radiol. 2020, 85, e104–e109. [Google Scholar] [CrossRef]
  38. Roh, Y.H.; Kang, B.K.; Song, S.Y.; Lee, C.M.; Jung, Y.K.; Kim, M. Preoperative CT anthropometric measurements and pancreatic pathology increase risk for postoperative pancreatic fistula in patients following pancreaticoduodenectomy. PLoS ONE 2020, 15, e0243515. [Google Scholar]
  39. Kolbinger, F.R.; Lambrecht, J.; Leger, S.; Ittermann, T.; Speidel, S.; Weitz, J.; Hoffmann, R.-T.; Distler, M.; Kühn, J.-P. The image-based preoperative fistula risk score (preFRS) predicts postoperative pancreatic fistula in patients undergoing pancreatic head resection. Sci. Rep. 2022, 12, 4064. [Google Scholar] [CrossRef]
  40. Nahm, C.B.; Lui, I.; Naidoo, C.S.; Roseverne, L.; Alzaabi, S.; Maher, R.; Mann, G.; Blome, S.; Gill, A.J.; Samra, J.S.; et al. Density and enhancement of the pancreatic tail on computer tomography predicts acinar score and pancreatic fistula after pancreatoduodenectomy. HPB 2019, 21, 604–611. [Google Scholar] [CrossRef]
  41. Uzunoglu, F.G.; Reeh, M.; Vettorazzi, E.; Ruschke, T.; Hannah, P.; Nentwich, M.F.; Vashist, Y.K.; Bogoevski, D.; König, A.; Janot, M.; et al. Preoperative Pancreatic Resection (PREPARE) score: A prospective multicenter-based morbidity risk score. Ann. Surg. 2014, 260, 857–863; Discussion 863–864. [Google Scholar] [CrossRef] [PubMed]
  42. Rodriguez-Lopez, M.; Tejero-Pintor, F.J.; Perez-Saborido, B.; Barrera-Rebollo, A.; Bailon-Cuadrado, M.; Pacheco-Sanchez, D. Severe morbidity after pancreatectomy is accurately predicted by preoperative pancreatic resection score (PREPARE): A prospective validation analysis from a medium-volume center. Hepatobiliary Pancreat. Dis. Int. 2018, 17, 559–565. [Google Scholar] [CrossRef] [PubMed]
  43. McMillan, M.; Allegrini, V.; Asbun, H.J.; Ball, C.G.; Bassi, C.; Beane, J.; Behrman, S.W.; Berger, A.C.; Bloomston, M.; Callery, M.P.; et al. Incorporation of Procedure-specific Risk Into the ACS-NSQIP Surgical Risk Calculator Improves the Prediction of Morbidity and Mortality After Pancreatoduodenectomy. Ann. Surg. 2017, 265, 978–986. [Google Scholar] [CrossRef] [PubMed]
  44. Greenblatt, D.Y.; Kelly, K.J.; Rajamanickam, V.; Wan, Y.; Hanson, T.; Rettammel, R.; Winslow, E.R.; Cho, C.S.; Weber, S.M. Preoperative factors predict perioperative morbidity and mortality after pancreaticoduodenectomy. Ann. Surg. Oncol. 2011, 18, 2126–2135. [Google Scholar] [CrossRef]
  45. Steyerberg, E.W.; Vergouwe, Y. Towards better clinical prediction models: Seven steps for development and an ABCD for validation. Eur. Heart J. 2014, 35, 1925–1931. [Google Scholar] [CrossRef] [PubMed]
Cancers 14 05551 g001 550
Figure 1. Results obtained from the online test of our cohort (n = 308) with the calculator. It represents the percentage of patients within each category and whether the risk acquired by the calculator was low, intermediate, or high.
Figure 1. Results obtained from the online test of our cohort (n = 308) with the calculator. It represents the percentage of patients within each category and whether the risk acquired by the calculator was low, intermediate, or high.
Cancers 14 05551 g001
Cancers 14 05551 g002 550
Figure 2. Overall morbidity and their rates in the validation cohort (n = 308) patients.
Figure 2. Overall morbidity and their rates in the validation cohort (n = 308) patients.
Cancers 14 05551 g002
Cancers 14 05551 g003 550
Figure 3. Classification plot representing the relationship between sensitivity and 1-specificity.
Figure 3. Classification plot representing the relationship between sensitivity and 1-specificity.
Cancers 14 05551 g003
Cancers 14 05551 g004 550
Figure 4. Graphical representation of model calibration, plotting the predicted probability (x-axis) with corresponding 95% CI against the actually observed occurrence of severe post-operative complications (defined as Clavien–Dindo IIIa or higher) in the validation cohort. The dashed line represents perfect calibration. Points under the line display overestimation, and above the line underestimation of the risk of severe post-operative complications.
Figure 4. Graphical representation of model calibration, plotting the predicted probability (x-axis) with corresponding 95% CI against the actually observed occurrence of severe post-operative complications (defined as Clavien–Dindo IIIa or higher) in the validation cohort. The dashed line represents perfect calibration. Points under the line display overestimation, and above the line underestimation of the risk of severe post-operative complications.
Cancers 14 05551 g004
Table
Table 1. Baseline characteristics of the study cohort.
Table 1. Baseline characteristics of the study cohort.
All
(308)		No or Mild Complications
Clavien–Dindo (I, II)
(219)		Severe Complications
Clavien–Dindo ≥ IIIa (89)		p Value
Age (years) 0.358
Median (IQR) 67 (59–73)67 (58–73)67 (63–72)
Mean ± SD65 ± 10.665 ± 11.566 ± 8.2
Gender, n (%) 0.348
Male163 (52.9%)118 (53.9%)45 (50.6%)
Female145 (47.1%)101 (46.1%)44 (49.4%)
BMI (kg/m2) Median (IQR)25.4 (22.4–28.7)25.1 (22.3–27.9)26.4 (23–30)0.638
Mean ± SD26.6 ± 11.826.5 ± 13.727 ± 5.1
(25 n (%))138 (44.8%)108(49.3%)30 (33.7%)
(25 n (%))170 (55.2%)111(50.7%)59 (66.3%)
ASA score, n (%) 0.330
I31 (10.1%)21 (9.6%)10 (11.2%)
II198 (64.3%)152 (69.4%)46 (51.7%)
III79 (25.6%)46 (21%)33 (37.1%)
HU of the pancreas 0.635
body, Median (IQR)85.3 (70.7–96.4)87.6 (72.7–98.7)79.6 (62.9–89.9)
Mean ± SD84.6 ± 17.987.2 ± 17.678.2 ± 17.3
Pathology 0.556
Adenocarcinoma176 (57.1%)134 (61.2%)42 (47.2%)
IPMN16 (5.2%)12 (5.5%)4 (4.5%)
Neuroendocrine neoplasm24 (7.8%)18 (8.2%)6 (6.7%)
Adenoma intestinal type36 (11.7%)20 (9.1%)16 (18%)
Mucinous cystic neoplasm6 (1.9%)4 (1.8%)2 (2.2%)
Serous cystadenoma2 (0.6%)1 (0.5%)1 (1.1%)
Chronic pancreatitis5 (1.6%)5 (2.3%)0
Chronic cholangitis9 (2.9%)7 (3.2%)2 (2.2%)
Other/Unknown34 (11%)18 (8.3%)16 (17.9%)
Operation type n (%) 0.825
PD63 (20.5%)48 (21.9%)15 (16.9%)
PPPD245 (79.5%)171 (78.1%)74 (83.1%)
Hounsfield Units (HU), Body Mass Index (BMI), American Society of Anesthesiologist (ASA), Pancreatoduodenectomy (PD), Pylorus-preserving pancreatoduodenectomy (PPPD), Standard deviation (SD), Interquartile range (IQR), Intraductal Papillary Mucinous Neoplasm (IPMN).
Table
Table 2. Postoperative complications in detail.
Table 2. Postoperative complications in detail.
Complicationn%
POPF
No24579.5
Grade A196.2
Grade B3912.7
Grade C51.6
DGE
No22874
Grade A3210.4
Grade B3310.7
Grade C154.9
PPH
No28793.2
Grade A82.6
Grade B113.6
Grade C20.6
Bile leakage
No30398.4
Grade A00
Grade B51.6
Grade C00
Chylous leakage
No21168.5
Grade A6019.5
Grade B3712
Grade C00
Surgical site infection
No25281.8
Yes5618.2
Pneumonia
No30398.4
Yes51.6
Mortality
No30398.4
Yes51.6
Post-Operative Pancreatic Fistula (POPF), Delayed Gastric Emptying (DGE), Post-Pancreatectomy Hemorrhage (PPH).
Table
Table 3. A summary of the established risk calculators focusing on postoperative severe complications following pancreatoduodenectomy.
Table 3. A summary of the established risk calculators focusing on postoperative severe complications following pancreatoduodenectomy.
Study IDVariablesTypes of VariablesSevere Complication DefinitionLimitations of the Calculator
#Braga et al. [4]ASA classification*
pancreatic texture, interoperative blood loss, pancreatic duct diameter.		Pre-operative intraoperativeClavien–Dindo ≥ IIIaMany intraoperative variables.
The results may have been biased because the Clavien–Dindo was published in 2004 and it included patients between 2002 and 2010.
√Joliat et al. [23]ASA classification* pancreatic texture, interoperative blood loss, pancreatic duct diameter.Pre-operative intraoperativeClavien–Dindo ≥ IIIaMany intraoperative variables.
It only included PD and not PPPD.The results may have been biased because the Clavien–Dindo was published in 2004 and it included patients between 2000 and 2012.
#Abbas et al. [24]age, gender, BMI*, ASA classification* hypertension, gland texture, duct size, DM* adenocarcinoma.Pre-operativeNo clear classification has been providedDifficult to use.
Requires many variables.
Pancreatic texture was considered as a preoperative variable.
No model calibration measure was reported.
#Chen et al. [1]BMI*, pancreatic texture, WBCs*, sodium concentration respiratory diseasesPre-operative
Intra-operative		Clavien–Dindo ≥ IIPancreatic texture is an intraoperative variable, and it is graded subjectively during the operation.
#Uzunoglu et al. [41]ASA classification* albumin, hemoglobin, heart rate, blood pressure, origin of tumor, elective surgery, type of operation.Pre-operativeClavien–Dindo ≥ IIIValidated in a small cohort (n = 50).
√Rodriguez-Lopez et al. [42]ASA classification* albumin, hemoglobin, heart rate, blood pressure, origin of tumor, elective surgery, type of operation.Pre-operativeClavien–Dindo ≥ IIISmall number of patients (n = 50).
There were patients who underwent distal pancreatectomy, and it was not mention how many patients had this type of resection.
#Wiltberger et al. [16]BMI*, ASA classification*the presence of vascular or pulmonary comorbidities.Pre-operativeClavien–Dindo ≥ IIIbNo external validation.
No pancreas specific variable.
#Aoki et al. [2]20 preoperative variables, one intraoperative variable (vascular resection)Pre-operative
Intra-operative		Clavien–Dindo ≥ IVNo external validation.
Requires an intraoperative variable.
Not available online.
No model calibration measure was reported.
#McMillan et al. [43]The same variables as the ACS-NSQIP* calculator. It used FRS*, hospital volume and surgical skills as additional predictors.Pre-operative
Intra-operative		Accordion grade ≥ IIIFRS calculation requires intraoperative blood loss and pancreatic texture.
Requires many variables.
Severe complications were not defined according to the Clavien–Dindo definition.
#Parikh et al. [7]Age, gender, BMI*, history of sepsis, functional status, ASA classification* CHD*, dyspnea, bleeding disorder, type of resectionPre-operativeDefined depending on the complication. Examples are provided in the article.Many patients underwent distal pancreatectomy 34%.
Requires many variables.
Severe complications were not defined according to the Clavien–Dindo definition.
#Greenblatt et al. [44]Age, gender, BMI, functional status, COPD*, steroid use, bleeding disorder, WBCs*, creatinine, albuminPre-operativeDefined depending on the complication. Examples are provided in the article.Requires many variables.
Severe complications were not defined according to the Clavien–Dindo definition.
No pancreas specific variable.
ASA* American Society of Anesthesiology, BMI* Body Mass Index, DM* Diabetes Mellitus, WBCs* White Blood Cells, FRS* Fistula Risk Score, ACS-NSQIP* American College of Surgeons-National Surgical Quality Improvement Program, CHD* Coronary Heart Disease, COPD* Chronic Obstructive Pulmonary Disease. # Development study √ External validation study.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Alhulaili, Z.M.; Pleijhuis, R.G.; Nijkamp, M.W.; Klaase, J.M. External Validation of a Risk Model for Severe Complications following Pancreatoduodenectomy Based on Three Preoperative Variables. Cancers 2022, 14, 5551. https://doi.org/10.3390/cancers14225551

AMA Style

Alhulaili ZM, Pleijhuis RG, Nijkamp MW, Klaase JM. External Validation of a Risk Model for Severe Complications following Pancreatoduodenectomy Based on Three Preoperative Variables. Cancers. 2022; 14(22):5551. https://doi.org/10.3390/cancers14225551

Chicago/Turabian Style

Alhulaili, Zahraa M., Rick G. Pleijhuis, Maarten W. Nijkamp, and Joost M. Klaase. 2022. "External Validation of a Risk Model for Severe Complications following Pancreatoduodenectomy Based on Three Preoperative Variables" Cancers 14, no. 22: 5551. https://doi.org/10.3390/cancers14225551

APA Style

Alhulaili, Z. M., Pleijhuis, R. G., Nijkamp, M. W., & Klaase, J. M. (2022). External Validation of a Risk Model for Severe Complications following Pancreatoduodenectomy Based on Three Preoperative Variables. Cancers, 14(22), 5551. https://doi.org/10.3390/cancers14225551

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop