Next Article in Journal
Rectum Dose Constraints for Carbon Ion Therapy: Relative Biological Effectiveness Model Dependence in Relation to Clinical Outcomes
Next Article in Special Issue
Negative Impact of Wound Complications on Oncologic Outcome of Soft Tissue Sarcomas of the Chest Wall
Previous Article in Journal
Integrin α6β4 in Colorectal Cancer: Expression, Regulation, Functional Alterations and Use as a Biomarker
Previous Article in Special Issue
Tumor Suppressor Function of miR-127-3p and miR-376a-3p in Osteosarcoma Cells
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Individualizing Follow-Up Strategies in High-Grade Soft Tissue Sarcoma with Flexible Parametric Competing Risk Regression Models

by
Maria Anna Smolle
1,†,
Michiel van de Sande
2,†,
Dario Callegaro
3,
Jay Wunder
4,
Andrew Hayes
5,
Lukas Leitner
1,
Marko Bergovec
1,
Per-Ulf Tunn
6,
Veroniek van Praag
2,
Marta Fiocco
7,8,9,
Joannis Panotopoulos
10,
Madeleine Willegger
10,
Reinhard Windhager
10,
Sander P. D. Dijkstra
2,
Winan J. van Houdt
11,
Jakob M. Riedl
12,
Michael Stotz
12,
Armin Gerger
12,
Martin Pichler
12,
Herbert Stöger
12,
Bernadette Liegl-Atzwanger
13,
Josef Smolle
14,
Dimosthenis Andreou
15,
Andreas Leithner
1,
Alessandro Gronchi
3,
Rick L. Haas
16,17,*,‡ and
Joanna Szkandera
12,*,‡
add Show full author list remove Hide full author list
1
Department of Orthopaedics and Trauma, Medical University of Graz, 8036 Graz, Austria
2
Department of Orthopaedic Surgery, Leiden University Medical Centre, 2333 ZA Leiden, The Netherlands
3
Department of Surgery, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy
4
University Musculoskeletal Oncology Unit, Mount Sinai Hospital, University of Toronto, Toronto, ON M5G IX5, Canada
5
Department of Surgery, Royal Marsden Hospital NHS Foundation Trust, London SW3 6JJ, UK
6
Sarcoma Centre, HELIOS-Klinikum Berlin-Buch, 13125 Berlin, Germany
7
Institute of Mathematics, Leiden University Medical Centre, 2333 ZA Leiden, The Netherlands
8
Medical Statistics, Department of Biomedical Data Science, Leiden University Medical Centre, 2333 ZA Leiden, The Netherlands
9
Princess Máxima Center for Pediatric Oncology, Trial and Data Center, 3584 CT Utrecht, The Netherlands
10
Department of Orthopaedics and Traumatology, Medical University of Vienna, 1090 Vienna, Austria
11
Department of Surgery, The Netherlands Cancer Institute, 1066 CX Amsterdam, The Netherlands
12
Division of Clinical Oncology, Department of Medicine, Medical University of Graz, 8036 Graz, Austria
13
Institute of Pathology, Medical University of Graz, 8010 Graz, Austria
14
Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, 8036 Graz, Austria
15
Sarcoma Centre, HELIOS Klinikum Bad Saarow, 15526 Bad Saarow, Germany
16
Department of Radiotherapy, Leiden University Medical Centre, 2333 ZA Leiden, The Netherlands
17
Department of Radiotherapy, The Netherlands Cancer Institute Amsterdam, 1066 CX Amsterdam, The Netherlands
*
Authors to whom correspondence should be addressed.
These authors share the first authorship.
These authors share the last authorship.
Cancers 2020, 12(1), 47; https://doi.org/10.3390/cancers12010047
Submission received: 1 December 2019 / Revised: 15 December 2019 / Accepted: 19 December 2019 / Published: 21 December 2019
(This article belongs to the Special Issue Bone and Soft Tissue Tumors)

Abstract

:
Currently, patients with extremity soft tissue sarcoma (eSTS) who have undergone curative resection are followed up by a heuristic approach, not covering individual patient risks. The aim of this study was to develop two flexible parametric competing risk regression models (FPCRRMs) for local recurrence (LR) and distant metastasis (DM), aiming at providing guidance on how to individually follow-up patients. Three thousand sixteen patients (1931 test, 1085 validation cohort) with high-grade eSTS were included in this retrospective, multicenter study. Histology (9 categories), grading (time-varying covariate), gender, age, tumor size, margins, (neo)adjuvant radiotherapy (RTX), and neoadjuvant chemotherapy (CTX) were used in the FPCRRMs and performance tested with Harrell-C-index. Median follow-up was 50 months (interquartile range: 23.3–95 months). Two hundred forty-two (12.5%) and 603 (31.2%) of test cohort patients developed LR and DM. Factors significantly associated with LR were gender, size, histology, neo- and adjuvant RTX, and margins. Parameters associated with DM were margins, grading, gender, size, histology, and neoadjuvant RTX. C-statistics was computed for internal (C-index for LR: 0.705, for DM: 0.723) and external cohort (C-index for LR: 0.683, for DM: 0.772). Depending on clinical, pathological, and patient-related parameters, LR- and DM-risks vary. With the present model, implemented in the updated Personalised Sarcoma Care (PERSARC)-app, more individualized prediction of LR/DM-risks is made possible.

Graphical Abstract

1. Introduction

Patients with high-grade extremity soft tissue sarcoma (eSTS) are at risk of developing local recurrences (LR) and even more so of developing distant metastases (DM) after having undergone surgical resection of the primary tumor [1,2,3,4,5]. These rates differ substantially per size, grade, and subtype [6]. Close follow-up regimens are currently used in order to detect LR and DM at stages where they are still potentially treatable by re-resection or metastasectomy, respectively [7]. There is no clear consensus when, by what means, and how often to perform follow-up in eSTS patients, with many centers and guidelines having introduced a heuristic approach: for the first 3 years after surgery, patients would be checked three or four times a year, then biannually for the following two years and annually thereafter [8,9].
Imposing all eSTS patients on these strict follow-up regimens has raised public, scientific, and health economic concerns over the last years. Numerous factors interact with the risk of developing LR or DM, such as histological STS-subtypes, surgical margins, tumor size, grade, administration of neo(adjuvant) radiotherapy (RTX) or chemotherapy (CTX), and patient-derived factors [1,2,3,4,10,11,12]. Consequently, the current approach of “one-size-fits-all” may not account for the unequal risk of recurrence in the heterogeneous eSTS population, involving an excessive number of surveillance imaging, possibly leading to unnecessary delivery of imaging-induced radiation exposure, and the inherent burden for radiology departments, as well as inappropriately refraining from it, a high number of outpatient visits and financial costs and emotional stress for each individual patient [13]. However, an evaluation of prognostic factors for LR and DM taking into consideration the time-varying rate for the occurrence of events in a multicenter cohort, including important patient-(i.e., age, gender), tumor-(e.g., size, grade, histological subtype), and treatment-related features (e.g., margins, (neo)adjuvant CTX/RTX), is currently missing.
Therefore, the aim of the present study was to estimate and validate two models predicting risks of LR and DM over the first 5 years of follow-up by applying flexible parametric competing risk regression modeling in a large, multicenter cohort of patients with primary localized high-grade eSTS. The results have been implemented into the Personalised Sarcoma Care (PERSARC)-app [14] for Individualized Sarcoma Care and follow-up.

2. Results

Patients had undergone surgery with curative intent between January 1994 and October 2014 for the test cohort and between January 2000 and December 2013 for the validation cohort, respectively. There was a slight male predominance (n = 1038; 53.8%) and the median patient age was 59 years (interquartile range (IQR): 44.7–70 years). With 55.8%, 17.9%, and 13.9%, most tumors in the test cohort were located in the thigh (n = 1078), upper arm (n = 346), and lower leg (n = 268), while the lower arm (n = 142), the foot or toes (n = 65), and the hand or fingers (n = 32) were affected in 7.3%, 3.4%, and 1.7%, respectively. Further clinicopathological features for both the test and validation cohort are listed in Table 1.
Five- and 10-year overall survival (OS) was 73.6% (95%CI: 71.3–75.7) and 62.7% (95%CI: 59.8–65.5) in the test cohort. In the validation cohort, 5- and 10-year OS were 64.9% (95%CI: 61.8–67.8) and 52.9% (95%CI: 48.9–56.8), respectively. Gender, tumor size, histological subtype (except for angiosarcoma/vascular sarcoma (p = 0.127) and dedifferentiated/pleomorphic liposarcoma (p = 0.254), margins, neoadjuvant and adjuvant RTX, as well as adjuvant CTX (all p < 0.05) had a significant influence on risk of LR in the stepwise backward selection of the Fine and Gray model. Grading as a time-dependent effect was kept in the model (p = 0.108), while age (p = 0.082) and neoadjuvant CTX (p = 0.214) were excluded. Consequently, gender, grading, tumor size, neoadjuvant and adjuvant RTX, histological subtype, and adjuvant CTX were included in the flexible parametric competing risk regression model (Table 2).
The subdistribution hazard and cumulative incidence functions for LR using ten clinical examples are shown in Figure 1A,B (definition of these examples found in Table S1, together with estimated conditional risks of LR). As an example, a male patient with a G2 myxofibrosarcoma sized 10 cm, with contaminated resection margins (R1/2), no neoadjuvant or adjuvant RTX, and no adjuvant CTX, has a significantly increased risk of developing LR, especially within the first 15 months of follow-up (=clinical example IX). On the other hand, a male patient with a 6 cm large, G3 synovial sarcoma, resected with clear margins (R0), without adjuvant CTX or (neo-)adjuvant RTX, has a moderate LR risk during the first 15 months, and an estimated low risk thereafter (=clinical example VIII).
In the stepwise backward selection of the Fine and Gray model for distant metastasis (DM) histological subtype (except for myxofibrosarcoma (p = 0.641), angiosarcoma/vascular sarcoma (p = 0.067) and dedifferentiated/pleomorphic liposarcoma (p = 0.592), grading, tumor size, gender, margins, and neoadjuvant RTX (all p < 0.05) were significantly associated with development of metastases. Age (p = 0.852), adjuvant RTX (p = 0.116), neoadjuvant CTX (p = 0.095), and adjuvant CTX (p = 0.536) were excluded via stepwise backward selection. Thus, histological subtype, grading, tumor size, margins, gender, and neoadjuvant RTX were included in the flexible parametric competing risk regression model (Table 3).
In Figure 2A,B, subdistribution hazard and cumulative incidence functions for DM (using the same ten clinical examples as in Figure 1A,B) are shown. Once again referring to clinical examples IX (male, myxofibrosarcoma, G2, 10 cm, R1/2-margins, no neoadjuvant RTX) and VIII (male, synovial sarcoma, G3, 6cm, R0-margins, no neoadjuvant RTX), risk of DM is lower in clinical example IX in comparison to clinical example VI, while LR-risks are just the opposite, highlighting the importance of an individualized follow-up strategy.
The conditional risks of these ten clinical examples changing over time estimated based on the models presented above are provided in Table S1 for LR and Table S2 for DM. Conditional risks for all possible combinations of prognostic factors may be estimated and have been implemented in the updated version of the PERSARC app.
The Harrell C index for LR was equal to 0.705 and 0.683 for the internal and external cohort, respectively. For DM, Harrell C statistics was equal to 0.723 for the internal cohort and 0.772 for the external cohort. Calibration plots for LR (Figure 3A) using test and validation cohort showed that the LR model tended to underestimate the actual patient risk, especially in the validation cohort. On the other hand, calibration plots for DM with test and validation cohort (Figure 3B) showed very good model calibration.

3. Discussion

In the present retrospective multicenter cohort study, flexible parametric competing risk regression modeling was applied in order to estimate individual three-to-six-month risks for local recurrence and distant metastasis during the first 5 years of follow-up in patients undergoing curative resection for high-grade extremity soft tissue sarcoma. It offers an evidence-based opportunity to individually schedule follow-up visits instead of adhering to calendar-based guidelines [8,9]. The number of radiological investigations for assessing disease status, especially after R0 resections and taking histological subtype into account, could be significantly restricted, reducing patient- and healthcare burden. The advantage of using flexible parametric competing risk regression models to estimate LR- and DM-risks in eSTS-patients is based on the fact that these rates strongly vary upon time (i.e., they do not constantly increase or decrease). To overcome this issue, flexible parametric competing risk regression models represent the baseline distribution function as a restricted cubic spline function of log time instead of a linear function of log time [15]. Moreover, it allows smooth estimation of both the cause-specific hazard rates and cumulative incidence functions. Both models performed well at internal and external calibration, with c-indexes comparable to previously published studies [14,16].
One of the limitations of the present study is its retrospective design, resulting in possible selection biases regarding diagnosis, treatment, and follow-up of patients included, due to slightly differing policies at the respective centers. By incorporating these factors in the statistical models, we aimed at reducing this bias. Moreover, during the study period, several histological STS-subtypes were reclassified (i.e., malignant fibrous histiocytoma to undifferentiated pleomorphic sarcoma). At some, but not all, participating centers, all histological diagnoses had been reevaluated by pathologists and, if applicable, reclassified according to the current classification systems. In order to limit the impact of this limitation, we only included patients treated in tertiary reference sarcoma centers with experienced and dedicated sarcoma pathologists.
Another limitation of the present study is that the models were developed based on patient cohorts from experienced, tertiary tumor centers. This implies that generalizability of the predicted risks to patients not treated at such centers has to be questioned. Moreover, considering that we did only include patients with high-grade (G2/3), primary eSTS who had undergone surgery with curative intent, the risks estimated are not applicable to patients with low-grade disease or metastases at initial presentation. Furthermore, estimated risks of the current models should be applied with caution after patients have already developed an event (i.e., LR or DM) during follow-up. Due to the retrospective design of the study, not all variables could be ascertained in every patient, thus potentially reducing the statistical power. However, it can be assumed that in this large patient collective, missing data may have little to no bias to the conclusions made, wherefore cases with missing information on clinical and/or pathological variables were not excluded [17].
As outlined in the introduction, current follow-up strategies follow a heuristic approach, with 3- to 4-months intervals for the first three years, followed by biannual check-ups until the end of the 5th year and annual appointments thereafter [8,9].
In clinical practice, it is not only of interest to estimate a patient’s cumulative risk after a specific period of time but also to know about the conditional risks from one follow-up appointment to the next, in case no event had occurred. We addressed this question by calculating conditional risks for LR and DM depending on different, clinically relevant, examples. Notably, the present model allows risk prediction at any constellation of variables, which are at the moment included in the updated PERSARC app. This app allows the patient’s individualized risk of LR and DM to be estimated by entering relevant prognostic parameters, such as histological subtype, tumor size, and resection margins. With the estimated event-risk in time, physicians and patients may decide together when the next follow-up examination should be scheduled. In light of the heterogeneity of eSTS with part significantly differing outcomes, estimated event risks would facilitate planning of an individualized follow-up protocol for each patient.
Arbitrary thresholds of 4% for LR and 2% for DM were chosen in the present study to be of clinical “relevance”, considering that LR is usually detected during clinical examination or even noticed by patients themselves, while DM (most commonly to the lungs) require visualization by chest x-ray or thoraci computed tomography (CT) scan [18,19]. However, thresholds should be changed on patients’ preference and clinical significance.
Previously published studies have well-investigated risks of LR, DM, and overall survival (OS) in large, retrospective cohorts of patients with eSTS [14,16,20,21]. The nomogram for OS by Kattan et al. [21] in 2002 and the two more recent nomograms for DM and OS by Callegaro et al. [16] published in 2016 added significant value to predict individual patient risks. Both studies used Cox-regression models as the basis for their nomograms. In comparison to Cox-regression models, flexible parametric competing risk regression models have a major advantage; while the Cox-regression models only estimate the relative effects (i.e., hazard rates), flexible parametric competing risk regression models estimate the baseline hazard using restricted cubic splines [22]. The cumulative incidence functions of LR and DM predicted from flexible parametric competing risk regression models demonstrate the clear variance in event rates. By applying a flexible parametric competing risk regression model, we aimed at incorporating non-constant hazards, time-varying covariates, and death as the competing event in order to obtain a robust, comprehensible, and accurate prediction of individual patient risks. Moreover, with the clinical examples provided, the risk peaks during the first year of follow-up is clearly visible. Although appointments may be safely skipped in some patients due to very low risks of LR and/or DM, others would benefit from closer follow-up intervals.
Potentially due to the application of the present statistical model, interesting results emerged: Female gender was independently associated with a significantly lower risk of LR and DM. An association between gender and overall survival (OS) has been observed by Maretty-Nielsen et al. and Wu et al. [2,23]. However, an association between gender and LR-free as well as DM-free survival has not been described thus far [24]. Moreover, tumor grading, which is a well-known prognostic factor of LR, was not significantly associated with an altered risk in the current flexible parametric competing risk regression model. This may be explained by the fact that patients with usually fast-growing, highly-aggressive G3 tumors will present with LR at early time points, while in those with relatively slower-growing G2 tumors, LR is most probably detected at a later date. This hypothesis is corroborated by the fact that grading did not meet the proportional hazards assumption, wherefore it was treated as a time-varying covariate. On the other hand, another recently published multicenter study for grade III eSTS did not incorporate grade II in the multivariate model for OS [14]. The current model has broader applicability as it also incorporates patients with grade II eSTS. Additionally, margins as classified in the current study only divide “clear” from “contaminated” margins, not taking into consideration that histological subtypes with infiltrative growth pattern as undifferentiated pleiomorphic sarcoma (UPS) and myxofibrosarcoma potentially require broader margins to markedly reduce LR-risk [25]. Thus, unsurprisingly, also in the current flexible parametric competing risk regression model for LR, these histological subtypes showed significantly higher LR-risks in comparison to other histologies.

4. Materials and Methods

In this retrospective multicenter study, 1931 consecutive patients with primary nonmetastatic high-grade (G2/3) eSTS managed with surgery at a curative intent were included in the test cohort, with patient information deriving from prospectively maintained STS databases at 5 participating tertiary sarcoma referral centers. Patients with missing information on oncological follow-up (i.e., development of LR/DM) had to be excluded (n = 42). Extremity STS were defined as tumors from the shoulder to the fingers (=upper limb) and from the pelvic girdle, excluding intrapelvic STS, to the foot (=lower limb). The validation cohort consisted of 1085 patients with identical inclusion criteria as for the test cohort from two independent tertiary sarcoma referral centers. As described above, patient monitoring after surgery usually followed a standardized approach with clinical examination, radiography using chest X-ray (CXR) or chest CT-scan (chest-CT) for control of DM and sonography or magnetic resonance imaging (MRI) for control of LR.
Demographic variables (patient age at diagnosis, gender), tumor-related parameters (tumor size, depth, location, grading, histological subtype), treatment (histological margins, (neo)adjuvant CTX/RTX), and outcome variables (date of LR or DM, date of death/last follow-up) were reported. Histological resection margins were divided into “clear” margins (=R0) and “contaminated” margins (=R1/2), as classification and definition of margin status have changed over time [26,27,28]. Histological subtypes were classified into 9 categories, with myxoid liposarcoma as the reference, compliant with previous studies and the current World Health Organisation (WHO) classification (Table 1) [6,16]. The FNCLCC grading system (Fédération Française des Centres de Lutte Contre le Cancer/French Federation of Centres for the Fight against Cancer) was used to categorize tumors into either intermediate (=G2) or high-grade (G3) [29]. (Neo-)adjuvant RTX and CTX had been administered in case a high risk of LR or DM had been anticipated by the multidisciplinary tumor board, according to locally preferred guidelines, LR was defined as a radiologically and/or histologically confirmed tumor recurrence. DM must have been confirmed by radiography (sonography, MRI, CXR, chest-CT) and/or histologically. In the case of pulmonary nodules without subsequent surgical exploration, an increase in size of the suspected metastasis must have been present. Patient, tumor, and treatment-related factors were ascertained using medical records, histological reports, and prospectively maintained databases at the respective centers.
Ethics approval was obtained in each participating center. The study was performed according to the Declaration of Helsinki and approved by the Ethics Committee of the Medical University of Graz, Austria (IRB-approval-number: 31-046 ex 18/19; date of approval: 24 May 2019).

Statistical Analysis

We focused on the first five years of follow-up to predict the conditional risk at the usually scheduled follow-up times (every 3 months from 1st to 3rd year; every 6 months in 4th and 5th year), i.e., the risk of experiencing an event at X + Y, given that the patient has not developed an event before X months. The variables age and tumor size were centered at their mean value in order to allow prediction at the average in case variables were not specified upon calculation. We used the Royston and Parmar approach to fit a flexible parametric competing risk regression model in order to estimate the risk of LR and DM, with death as the competing event [30]. In this model, the baseline distribution is modeled as a restricted cubic spline function of log time [15,22]. Splines constitute flexible mathematical functions defined by piecewise polynomials together with distinct constraints, ensuring that the overall curve is smooth [22]. A feature of restricted cubic splines as used in the present model is that the fitted function is forced to be linear before the first and after the last knot [31]. As automatic stepwise backward selection of variables is currently not available for the flexible parametric competing risk regression model, variable selection for the LR and DM models was based on a stepwise backward procedure using a multivariable Fine and Gray model [32]. Variables with a p-value < 0.05 were excluded from the model, except for histology, where all subtypes were kept in the analyses. The LR and DM models were fit on the log cumulative subdistribution hazard scale, directly modeling the cause-specific cumulative incidence function. Grading was incorporated in the model as a time-dependent effect for LR and DM, as it did not meet the proportional hazards assumption. The number of knots of the flexible parametric competing risk regression model for LR and DM was chosen based on the lowest AIC (=Akaike Information Criterion) after fitting several models with knots from 0 to 5. For the flexible parametric competing risk regression model estimating the risk of LR, two knots turned out as most accurate, while for the model predicting the risk of DM, four knots were used (with no internal knots for grading as a time-dependent covariate). Cumulative incidence functions were estimated based on the defined models. Conditional risks at the 3–6-months intervals were calculated based on the cumulative incidence functions of the flexible parametric competing risk regression model. Threshold was set to 4% for LR, considering that they are often palpable and diagnosed during the clinical examination or by patients themselves [18]. On the other hand, a 2% threshold for DM was chosen, as DM (and most commonly lung metastases) can only safely be diagnosed by chest X-ray or CT-scans of the thorax [19]. Model discrimination was tested using the Harrell C index, estimating the probability of concordance between observed and predicted outcomes. A value of 0.5 indicates no predictive discrimination, while a value of 1.0 indicates a perfect separation of patients with different outcomes [33]. Furthermore, calibration plots were compiled to assess model calibration in the test and validation cohort.

5. Conclusions

In conclusion, the present study provides a model to individually predict patient’s LR and DM risks during follow-up, applying a flexible parametric competing risk regression approach. These models are at the moment being included in the updated version of the PERSARC app for Individualized Sarcoma Care and follow-up. Although a risk-threshold of 4% for LR and 2% for DM was chosen in the present study, the “optimal” threshold upon which an individual patient should undergo imaging with MRI, chest-CT, or CXR, is still subjected to experts’ opinion and should be further discussed with patients concerned.

Supplementary Materials

The following are available online at https://www.mdpi.com/2072-6694/12/1/47/s1, Table S1: Conditional Risks for Local Recurrence–Threshold 4%; Table S2: Conditional Risks for Distant Metastasis–Threshold 2%.

Author Contributions

Conceptualization, M.A.S., A.L., R.L.H., and J.S. (Joanna Szkandera); data curation, M.A.S., D.C., J.W., A.H., L.L., M.B., P.-U.T., V.v.P., M.F., J.P., M.W., R.W., S.P.D.D., W.J.v.H., J.M.R., M.S., A.G. (Armin Gerger), M.P., H.S., B.L.-A., D.A., and A.G. (Alessandro Gronchi); formal analysis, J.S. (Josef Smolle); investigation, D.C., J.W., A.H., L.L., M.B., P.-U.T., V.v.P., J.P., M.W., R.W., S.D., W.J.v.H., J.M.R., M.S., A.G. (Alessandro Gronchi), M.P., H.S., B.L.-A., and A.G. (Armin Gerger); methodology, M.v.d.S., M.F., S.D., J.S. (Josef Smolle), R.L.H., and J.S. (Joanna Szkandera); project administration, M.v.d.S., A.L., R.L.H., and J.S. (Joanna Szkandera); supervision, M.v.d.S., D.A., and R.L.H.; validation, V.v.P., M.F., D.A. and J.S. (Joanna Szkandera); visualization, M.A.S. and J.S. (Josef Smolle); writing—Original draft, M.A.S., M.v.d.S., A.L., R.L.H., and J.S. (Joanna Szkandera); writing—Review and editing, D.C., J.W., A.H., L.L., M.B., P.-U.T., V.v.P., J.P., M.W., R.W., S.P.D.D., W.J.v.H., J.M.R., M.S., A.G. (Armin Gerger), M.P., H.S., B.L.-A., D.A., and A.G. (Alessandro Gronchi). All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Dutch Cancer Society (DCS)—KWF Kankerbestrijding [UL2015-8028]. The funding source had no role in the design of this study; execution, analyses, and interpretation of the data; report writing; or decision to submit the article for publication.

Conflicts of Interest

Author van de Sande reports grants from Daiichi Sankyo, outside the submitted work. The remaining authors (Maria A Smolle, Dario Callegaro, Jay Wunder, Andrew J. Hayes, Lukas Leitner, Marko Bergovec, Per-Ulf Tunn, Veroniek van Praag, Marta Fiocco, Joannis Panotopoulos, Madeleine Willegger, Reinhard Windhager, Sander Djikstra, Winan J van Houdt, Jakob M Riedl, Michael Stotz, Armin Gerger, Martin Pichler, Herbert Stöger, Bernadette Liegl-Atzwanger, Josef Smolle, Dimosthenis Andreou, Andreas Leithner, Alessandro Gronchi, Rick L. Haas, and Joanna Szkandera) have no conflicts of interest to declare.

References

  1. Italiano, A.; Le Cesne, A.; Mendiboure, J.; Blay, J.Y.; Piperno-Neumann, S.; Chevreau, C.; Delcambre, C.; Penel, N.; Terrier, P.; Ranchere-Vince, D.; et al. Prognostic factors and impact of adjuvant treatments on local and metastatic relapse of soft-tissue sarcoma patients in the competing risks setting. Cancer 2014, 120, 3361–3369. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Maretty-Nielsen, K.; Aggerholm-Pedersen, N.; Safwat, A.; Jorgensen, P.H.; Hansen, B.H.; Baerentzen, S.; Pedersen, A.B.; Keller, J. Prognostic factors for local recurrence and mortality in adult soft tissue sarcoma of the extremities and trunk wall: A cohort study of 922 consecutive patients. Acta Orthop. 2014, 85, 323–332. [Google Scholar] [CrossRef] [PubMed]
  3. Novais, E.N.; Demiralp, B.; Alderete, J.; Larson, M.C.; Rose, P.S.; Sim, F.H. Do surgical margin and local recurrence influence survival in soft tissue sarcomas? Clin. Orthop. Relat. Res. 2010, 468, 3003–3011. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Willeumier, J.; Fiocco, M.; Nout, R.; Dijkstra, S.; Aston, W.; Pollock, R.; Hartgrink, H.; Bovee, J.; van de Sande, M. High-grade soft tissue sarcomas of the extremities: Surgical margins influence only local recurrence not overall survival. Int. Orthop. 2015, 39, 935–941. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Singer, S.; Demetri, G.D.; Baldini, E.H.; Fletcher, C.D. Management of soft-tissue sarcomas: An overview and update. Lancet Oncol. 2000, 1, 75–85. [Google Scholar] [CrossRef]
  6. Fletcher, C.D.; Bridge, J.A.; Hodgendoorn, P.C.W.; Mertens, F. WHO Classification of Tumours of Soft Tissue and Bone; IARC: Lyon, France, 2013. [Google Scholar]
  7. Smolle, M.A.; van Praag, V.M.; Posch, F.; Bergovec, M.; Leitner, L.; Friesenbichler, J.; Heregger, R.; Riedl, J.M.; Pichler, M.; Gerger, A.; et al. Surgery for metachronous metastasis of soft tissue sarcoma—A magnitude of benefit analysis using propensity score methods. Eur. J. Surg. Oncol. 2018, 45, 242–248. [Google Scholar] [CrossRef]
  8. Casali, P.G.; Abecassis, N.; Bauer, S.; Biagini, R.; Bielack, S.; Bonvalot, S.; Boukovinas, I.; Bovee, J.; Brodowicz, T.; Broto, J.M.; et al. Soft tissue and visceral sarcomas: ESMO-EURACAN Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann. Oncol. 2018, 29, iv51–iv67. [Google Scholar] [CrossRef]
  9. von Mehren, M.; Randall, R.L.; Benjamin, R.S.; Boles, S.; Bui, M.M.; Ganjoo, K.N.; George, S.; Gonzalez, R.J.; Heslin, M.J.; Kane, J.M., 3rd; et al. Soft Tissue Sarcoma, Version 2.2018, NCCN Clinical Practice Guidelines in Oncology. J. Natl. Compr. Cancer Netw. 2018, 16, 536–563. [Google Scholar] [CrossRef]
  10. Gingrich, A.A.; Bateni, S.B.; Monjazeb, A.M.; Darrow, M.A.; Thorpe, S.W.; Kirane, A.R.; Bold, R.J.; Canter, R.J. Neoadjuvant Radiotherapy is Associated with R0 Resection and Improved Survival for Patients with Extremity Soft Tissue Sarcoma Undergoing Surgery: A National Cancer Database Analysis. Ann. Surg. Oncol. 2017, 24, 3252–3263. [Google Scholar] [CrossRef]
  11. Gronchi, A.; Ferrari, S.; Quagliuolo, V.; Broto, J.M.; Pousa, A.L.; Grignani, G.; Basso, U.; Blay, J.Y.; Tendero, O.; Beveridge, R.D.; et al. Histotype-tailored neoadjuvant chemotherapy versus standard chemotherapy in patients with high-risk soft-tissue sarcomas (ISG-STS 1001): An international, open-label, randomised, controlled, phase 3, multicentre trial. Lancet Oncol. 2017, 18, 812–822. [Google Scholar] [CrossRef]
  12. Posch, F.; Partl, R.; Doller, C.; Riedl, J.M.; Smolle, M.; Leitner, L.; Bergovec, M.; Liegl-Atzwanger, B.; Stotz, M.; Bezan, A.; et al. Benefit of Adjuvant Radiotherapy for Local Control, Distant Metastasis, and Survival Outcomes in Patients with Localized Soft Tissue Sarcoma: Comparative Effectiveness Analysis of an Observational Cohort Study. Ann. Surg. Oncol. 2018, 25, 776–783. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Royce, T.J.; Punglia, R.S.; Chen, A.B.; Patel, S.A.; Thornton, K.A.; Raut, C.P.; Baldini, E.H. Cost-Effectiveness of Surveillance for Distant Recurrence in Extremity Soft Tissue Sarcoma. Ann. Surg. Oncol. 2017, 24, 3264–3270. [Google Scholar] [CrossRef] [PubMed]
  14. van Praag, V.M.; Rueten-Budde, A.J.; Jeys, L.M.; Laitinen, M.K.; Pollock, R.; Aston, W.; van der Hage, J.A.; Dijkstra, P.D.S.; Ferguson, P.C.; Griffin, A.M.; et al. A prediction model for treatment decisions in high-grade extremity soft-tissue sarcomas: Personalised sarcoma care (PERSARC). Eur. J. Cancer 2017, 83, 313–323. [Google Scholar] [CrossRef] [PubMed]
  15. Royston, P.; Lambert, P.C. Flexible Parametric Survival Analysis Using Stata: Beyond the Cox Model; Stata Press: College Statoin, TX, USA, 2011. [Google Scholar]
  16. Callegaro, D.; Miceli, R.; Bonvalot, S.; Ferguson, P.; Strauss, D.C.; Levy, A.; Griffin, A.; Hayes, A.J.; Stacchiotti, S.; Pechoux, C.L.; et al. Development and external validation of two nomograms to predict overall survival and occurrence of distant metastases in adults after surgical resection of localised soft-tissue sarcomas of the extremities: A retrospective analysis. Lancet Oncol. 2016, 17, 671–680. [Google Scholar] [CrossRef]
  17. Kang, H. The prevention and handling of the missing data. Korean J. Anesthesiol. 2013, 64, 402–406. [Google Scholar] [CrossRef] [PubMed]
  18. Puri, A.; Ranganathan, P.; Gulia, A.; Crasto, S.; Hawaldar, R.; Badwe, R.A. Does a less intensive surveillance protocol affect the survival of patients after treatment of a sarcoma of the limb? updated results of the randomized TOSS study. Bone Jt. J. 2018, 100-B, 262–268. [Google Scholar] [CrossRef]
  19. Hovgaard, T.B.; Nymark, T.; Skov, O.; Petersen, M.M. Follow-up after initial surgical treatment of soft tissue sarcomas in the extremities and trunk wall. Acta Oncol. 2017, 56, 1004–1012. [Google Scholar] [CrossRef] [Green Version]
  20. Rueten-Budde, A.J.; van Praag, V.M.; PERSARC studygroup; van de Sande, M.A.J.; Fiocco, M. Dynamic prediction of overall survival for patients with high-grade extremity soft tissue sarcoma. Surg. Oncol. 2018, 27, 695–701. [Google Scholar] [CrossRef]
  21. Kattan, M.W.; Leung, D.H.; Brennan, M.F. Postoperative nomogram for 12-year sarcoma-specific death. J. Clin. Oncol. 2002, 20, 791–796. [Google Scholar] [CrossRef]
  22. Lambert, P.C.; Royston, P. Further development of flexible parametric models for survival analysis. Stata J. 2009, 9, 265–290. [Google Scholar] [CrossRef] [Green Version]
  23. Wu, J.; Qian, S.; Jin, L. Prognostic factors of patients with extremity myxoid liposarcomas after surgery. J. Orthop. Surg. Res. 2019, 14, 90. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Trovik, C.S.; Bauer, H.C.; Alvegard, T.A.; Anderson, H.; Blomqvist, C.; Berlin, O.; Gustafson, P.; Saeter, G.; Walloe, A. Surgical margins, local recurrence and metastasis in soft tissue sarcomas: 559 surgically-treated patients from the Scandinavian Sarcoma Group Register. Eur. J. Cancer 2000, 36, 710–716. [Google Scholar] [CrossRef]
  25. Fujiwara, T.; Stevenson, J.; Parry, M.; Tsuda, Y.; Tsoi, K.; Jeys, L. What is an adequate margin for infiltrative soft-tissue sarcomas? Eur. J. Surg. Oncol. 2019. [Google Scholar] [CrossRef]
  26. Kainhofer, V.; Smolle, M.A.; Szkandera, J.; Liegl-Atzwanger, B.; Maurer-Ertl, W.; Gerger, A.; Riedl, J.; Leithner, A. The width of resection margins influences local recurrence in soft tissue sarcoma patients. Eur. J. Surg. Oncol. 2016, 42, 899–906. [Google Scholar] [CrossRef] [PubMed]
  27. Tunn, P.U.; Kettelhack, C.; Durr, H.R. Standardized approach to the treatment of adult soft tissue sarcoma of the extremities. Recent Results Cancer Res. 2009, 179, 211–228. [Google Scholar]
  28. Wittekind, C.; Compton, C.C.; Greene, F.L.; Sobin, L.H. TNM residual tumor classification revisited. Cancer 2002, 94, 2511–2516. [Google Scholar] [CrossRef]
  29. Trojani, M.; Contesso, G.; Coindre, J.M.; Rouesse, J.; Bui, N.B.; de Mascarel, A.; Goussot, J.F.; David, M.; Bonichon, F.; Lagarde, C. Soft-tissue sarcomas of adults; study of pathological prognostic variables and definition of a histopathological grading system. Int. J. Cancer 1984, 33, 37–42. [Google Scholar] [CrossRef]
  30. Mozumder, S.I.; Rutherford, M.J.; Lambert, P.C. stpm2cr: A flexible parametric competing risks model using a direct likelihood approach for the cause-specific cumulative incidence function. Stata J. 2017, 17, 462–489. [Google Scholar] [CrossRef] [Green Version]
  31. Durrleman, S.; Simon, R. Flexible regression models with cubic splines. Stat. Med. 1989, 8, 551–561. [Google Scholar] [CrossRef]
  32. Fine, J.P.; Gray, R.J. A proportional hazards model for the subdistribution of a competing risk. J. Am. Stat. Assoc. 1999, 94, 496–509. [Google Scholar] [CrossRef]
  33. Harrell, F.E., Jr.; Lee, K.L.; Mark, D.B. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 1996, 15, 361–387. [Google Scholar] [CrossRef]
Figure 1. Subdistribution hazard function (A) and cumulative incidence function (B) of the flexible parametric competing risk regression model for local recurrence using ten clinical examples (constellation of parameters shown in Tables S1 and S2).
Figure 1. Subdistribution hazard function (A) and cumulative incidence function (B) of the flexible parametric competing risk regression model for local recurrence using ten clinical examples (constellation of parameters shown in Tables S1 and S2).
Cancers 12 00047 g001
Figure 2. Subdistribution hazard function (A) and cumulative incidence function (B) of the flexible parametric competing risk regression model for distant metastasis using the same ten clinical examples as in Figure 1A,B (constellation of parameters shown in Tables S1 and S2).
Figure 2. Subdistribution hazard function (A) and cumulative incidence function (B) of the flexible parametric competing risk regression model for distant metastasis using the same ten clinical examples as in Figure 1A,B (constellation of parameters shown in Tables S1 and S2).
Cancers 12 00047 g002
Figure 3. Calibration plots for the flexible parametric competing risk regression model regarding local recurrence (A) and distant metastasis (B) for the test (top) and validation cohort (bottom).
Figure 3. Calibration plots for the flexible parametric competing risk regression model regarding local recurrence (A) and distant metastasis (B) for the test (top) and validation cohort (bottom).
Cancers 12 00047 g003
Table 1. Patient characteristics and clinical, pathological, and treatment-related parameters.
Table 1. Patient characteristics and clinical, pathological, and treatment-related parameters.
VariablesTest Cohort (n = 1931)Validation Cohort (n = 1085)
N (%)Missing (%)N (%)Missing (%)p-Value *
Age (continuous; years; median + IQR)59 (44.7–70)45 (2.3)61 (47–74)0 (0.0)<0.0001
GenderMale1038 (53.8)0 (0.0)615 (56.7)0 (0.0)0.121
Female893 (46.2)470 (43.3)
Tumor LocationUpper Extremity520 (26.9)1 (0.05)312 (28.8)0 (0.0)0.285
Lower Extremity1410 (73.1)773 (71.2)
DepthEpifascial518 (26.9)2 (0.1)291 (26.8)0 (0.0)0.984
Subfascial1411 (73.1)794 (73.2)
Tumor Size (continuous; cm; median + IQR)7 (4–11)30 (1.6)7.5 (5–12)5 (0.5)0.026
HistologyMyxoid liposarcoma222 (11.6)16 (0.8)111 (10.2)0 (0.0)<0.0001
MPNST83 (4.3)43 (4.0)
Myxofibrosarcoma451(23.6)104 (9.6)
Synovial Sarcoma174 (9.1)79 (7.3)
UPS325 (17.0)375 (34.6)
Angiosarcoma/Vascular Sarcoma22 (1.1)20 (1.8)
Dedifferentiated/Pleomorphic Liposarcoma141 (7.4)85 (7.8)
Leiomyosarcoma221 (11.5)118 (10.9)
Others276 (14.4)150 (13.8)
GradingG2719 (37.8)30 (1.6)382 (35.8)19 (1.8)0.282
G31182 (62.2)684 (64.2)
MarginsR01494 (77.4)0 (0.0)768 (70.8)0 (0.0)<0.0001
R1/2437 (22.6)317 (29.2)
CTXNo1408 (73.0)1 (0. 05)1039 (95.8)0 (0.0)<0.0001
Neoadjuvant262 (13.6)40 (3.7)
Adjuvant209 (10.8)6 (0.6)
Neoadjuvant + Adjuvant51 (2.6)0 (0.0)
RTXNo619 (32.9)50 (2.6)335 (30.9)0 (0.0)<0.0001
Neoadjuvant303 (16.1)460 (42.4)
Adjuvant956 (50.8)275 (25.4)
Neoadjuvant + Adjuvant3 (0.2)15 (1.4)
Follow-up (continuous; months; median + IQR)50 (23.3–95)11 (0.6)56 (21–91)0 (0.0)0.254
* p-values calculated with Wilcoxon rank sum test for continuous variables, with chi2-test for binary and categorical variables. p-values in bold are considered statistically significant. Abbreviations: CTX = chemotherapy. IQR = interquartile range. MPNST = Malignant Peripheral Nerve Sheat Tumor. RTX = radiotherapy. UPS = Undifferentiated pleiomorphic sarcoma.
Table 2. Estimated coefficients along with their 95% confidence intervals for local recurrence.
Table 2. Estimated coefficients along with their 95% confidence intervals for local recurrence.
VariablesCoefficient95%-CIp-value
LowerUpper
Local Recurrence
GenderMale10.011
Female0.6980.5290.921
GradingG210.199
G30.8160.5981.113
Tumor size1.0261.0041.0490.019
MarginsR01<0.001
R1/R22.7612.0213.774
HistologyMyxoid Liposarcoma1
MPNST4.2271.8379.7290.001
Myxofibrosarcoma4.1562.0568.400<0.001
Synovial Sarcoma3.1161.4297.0140.005
UPS3.3731.6207.0250.001
Angiosarcoma/Vascular Sarcoma3.3160.98112.3410.074
Dedifferentiated/Pleomorphic Liposarcoma1.7270.7194.1430.221
Leiomyosarcoma2.7791.2945.9660.009
Others2.3851.1235.0650.024
Neoadjuvant RTXNo1<0.001
Yes0.2980.1780.494
Adjuvant RTXNo10.001
Yes0.6030.4470.814
Adjuvant CTXNo10.008
Yes1.7111.1542.538
Restricted cubic spline 12.1041.8512.392<0.001
Restricted cubic spline 21.3321.2301.442<0.001
Restricted cubic spline 30.9800.9371.0260.391
Restricted cubic spline for time-dependent effect of grading 0.9440.8131.0960.449
Constant0.0480.0240.097<0.001
Death
GenderMale10.005
Female0.7360.5950.910
GradingG21<0.001
G32.2151.6552.964
Tumor size1.0651.0481.081<0.001
MarginsR010.296
R1/R21.1530.8831.057
HistologyMyxoid Liposarcoma1
MPNST1.2050.6642.1870.540
Myxofibrosarcoma1.2080.7951.8360.375
Synovial Sarcoma1.4610.8882.4040.136
UPS1.1500.7531.7580.517
Angiosarcoma/Vascular Sarcoma4.7292.3359.580<0.001
Dedifferentiated/Pleomorphic Liposarcoma1.4200.8632.3380.167
Leiomyosarcoma2.1541.4023.309<0.001
Others1.5160.9752.3560.065
Neoadjuvant RTXNo10.007
Yes1.5431.1272.111
Adjuvant RTXNo10.296
Yes1.1450.8881.476
Adjuvant CTXNo10.022
Yes0.6790.4880.946
Restricted cubic spline 14.2203.3935.250<0.001
Restricted cubic spline 21.4871.3291.663<0.001
Restricted cubic spline 30.9650.9211.0110.139
Restricted cubic spline for time-dependent effect of grading 0.7140.5800.8890.002
Constant0.0500.0320.078<0.001
p-values in bold are considered statistically significant. CI = Confidence interval. RTX = radiotherapy. CTX = chemotherapy. UPS = undifferentiated pleomorphic sarcoma. MPNST = Malignant Peripheral Nerve Sheat Tumour.
Table 3. Estimated coefficients along with their 95% confidence intervals for distant metastasis.
Table 3. Estimated coefficients along with their 95% confidence intervals for distant metastasis.
Coefficient95%-CIp-Value
LowerUpper
Distant Metastasis
GenderMale1<0.001
Female0.7200.6050.857
GradingG21<0.001
G31.7371.4122.136
Tumor size1.0691.0561.083<0.001
MarginsR010.006
R1/R21.3471.0871.669
HistologyMyxoid Liposarcoma1
MPNST1.8251.1582.8750.009
Myxofibrosarcoma1.0640.7501.5080.729
Synovial Sarcoma1.9861.3433.9760.001
UPS1.4451.0332.0220.032
Angiosarcoma/Vascular Sarcoma2.0161.0223.7970.043
Dedifferentiated/Pleomorphic Liposarcoma1.2090.7861.8610.387
Leiomyosarcoma2.6891.9003.797<0.001
Other1.8351.2932.6040.001
Neoadjuvant RTXNo10.005
Yes1.3511.0971.663
Restricted cubic spline 12.9282.5913.308<0.001
Restricted cubic spline 21.4581.3741.547<0.001
Restricted cubic spline 30.9650.9261.0060.096
Restricted cubic spline 41.0401.0201.062<0.001
Restricted cubic spline 50.9950.9821.0080.427
Restricted cubic spline for time-dependent effect of grading0.7230.6400.817<0.001
Constant0.1080.0780.149<0.001
Death
GenderMale10.864
Female0.9680.6661.407
GradingG210.018
G31.8731.1163.145
Tumor size1.0230.9971.0500.087
MarginsR010.198
R1/R21.3780.8462.244
HistologyMyxoid Liposarcoma1
MPNST2.5060.8447.4420.098
Myxofibrosarcoma3.1361.3257.4180.009
Synovial Sarcoma0.6000.1502.4160.472
UPS1.7810.7144.4430.216
Angiosarcoma/Vascular Sarcoma11.165 *3.507 *35.542 *<0.001 *
Dedifferentiated/Pleomorphic Liposarcoma3.3311.2598.8120.015
Leiomyosarcoma1.7980.6754.7820.241
Other2.4080.9634.7820.060
Neoadjuvant RTXNo10.048
Yes0.5410.2950.993
Restricted cubic spline 13.6042.4945.211<0.001
Restricted cubic spline 21.2701.0601.5230.010
Restricted cubic spline 30.9520.8631.0490.316
Restricted cubic spline 40.9530.9081.0010.057
Restricted cubic spline 50.9740.5691.1990.097
Restricted cubic spline for time-dependent effect of grading0.8260.5691.1990.314
Constant0.0100.0040.025<0.001
p-values in bold are considered statistically significant; CI = Confidence interval. RTX = radiotherapy. CTX = chemotherapy. UPS = undifferentiated pleomorphic sarcoma. * too few events.

Share and Cite

MDPI and ACS Style

Smolle, M.A.; van de Sande, M.; Callegaro, D.; Wunder, J.; Hayes, A.; Leitner, L.; Bergovec, M.; Tunn, P.-U.; van Praag, V.; Fiocco, M.; et al. Individualizing Follow-Up Strategies in High-Grade Soft Tissue Sarcoma with Flexible Parametric Competing Risk Regression Models. Cancers 2020, 12, 47. https://doi.org/10.3390/cancers12010047

AMA Style

Smolle MA, van de Sande M, Callegaro D, Wunder J, Hayes A, Leitner L, Bergovec M, Tunn P-U, van Praag V, Fiocco M, et al. Individualizing Follow-Up Strategies in High-Grade Soft Tissue Sarcoma with Flexible Parametric Competing Risk Regression Models. Cancers. 2020; 12(1):47. https://doi.org/10.3390/cancers12010047

Chicago/Turabian Style

Smolle, Maria Anna, Michiel van de Sande, Dario Callegaro, Jay Wunder, Andrew Hayes, Lukas Leitner, Marko Bergovec, Per-Ulf Tunn, Veroniek van Praag, Marta Fiocco, and et al. 2020. "Individualizing Follow-Up Strategies in High-Grade Soft Tissue Sarcoma with Flexible Parametric Competing Risk Regression Models" Cancers 12, no. 1: 47. https://doi.org/10.3390/cancers12010047

APA Style

Smolle, M. A., van de Sande, M., Callegaro, D., Wunder, J., Hayes, A., Leitner, L., Bergovec, M., Tunn, P. -U., van Praag, V., Fiocco, M., Panotopoulos, J., Willegger, M., Windhager, R., Dijkstra, S. P. D., van Houdt, W. J., Riedl, J. M., Stotz, M., Gerger, A., Pichler, M., ... Szkandera, J. (2020). Individualizing Follow-Up Strategies in High-Grade Soft Tissue Sarcoma with Flexible Parametric Competing Risk Regression Models. Cancers, 12(1), 47. https://doi.org/10.3390/cancers12010047

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop