Next Article in Journal
Exercise as a Promising Agent against Cancer: Evaluating Its Anti-Cancer Molecular Mechanisms
Next Article in Special Issue
AI-Based Detection of Oral Squamous Cell Carcinoma with Raman Histology
Previous Article in Journal
The Association between Proton Pump Inhibitors and the Effectiveness of CDK Inhibitors in HR+/HER- Advanced Breast Cancer Patients: A Systematic Review and Meta-Analysis
Previous Article in Special Issue
Intraoperative Imaging in Hepatopancreatobiliary Surgery
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 15
Issue 21
10.3390/cancers15215134
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Radiomics Signature Based on Support Vector Machines for the Prediction of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer

by
Chao Li
1,†,
Haiyan Chen
1,†,
Bicheng Zhang
1,
Yimin Fang
2,
Wenzheng Sun
1,
Dang Wu
1,
Zhuo Su
1,
Li Shen
1,* and
Qichun Wei
1,*
1
Department of Radiation Oncology (Key Laboratory of Cancer Prevention and Intervention, China National Ministry of Education, Key Laboratory of Molecular Biology in Medical Sciences), The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou 310009, China
2
Department of Colorectal Surgery and Oncology (Key Laboratory of Cancer Prevention and Intervention, China National Ministry of Education, Key Laboratory of Molecular Biology in Medical Sciences), The Second Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou 310009, China
*
Authors to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Cancers 2023, 15(21), 5134; https://doi.org/10.3390/cancers15215134
Submission received: 18 September 2023 / Revised: 15 October 2023 / Accepted: 19 October 2023 / Published: 25 October 2023
(This article belongs to the Special Issue Recent Advances in Oncology Imaging)
Browse Figures
Versions Notes

Abstract

:

Simple Summary

This study developed CT-based radiomics signatures using the least absolute shrinkage and selection operator (LASSO), random forest (RF) and support vector machine (SVM) algorithms to predict the pathological complete response (pCR) in locally advanced rectal cancer (LARC) patients who underwent neoadjuvant chemoradiotherapy. Among these methods, the SVM-based radiomics score (Radscore) exhibited superior performance compared to the others, with area under the receiver operating characteristic curves (AUCs) of 0.880 and 0.830 in the training and validation cohorts, respectively. By integrating the SVM-based Radscore with clinical indicators, a nomogram was created for predicting pCR, achieving AUCs of 0.910 and 0.866 in the training and validation cohorts, respectively. The study highlighted the promising performance of the SVM-based Radscore and the value of the radiomics nomogram for predicting pCR in LARC patients. Additionally, the identification of an optimal radiomics signature can significantly improve the accuracy of pCR prediction.

Abstract

The objective of this study was to evaluate the discriminative capabilities of radiomics signatures derived from three distinct machine learning algorithms and to identify a robust radiomics signature capable of predicting pathological complete response (pCR) after neoadjuvant chemoradiotherapy in patients diagnosed with locally advanced rectal cancer (LARC). In a retrospective study, 211 LARC patients were consecutively enrolled and divided into a training cohort (n = 148) and a validation cohort (n = 63). From pretreatment contrast-enhanced planning CT images, a total of 851 radiomics features were extracted. Feature selection and radiomics score (Radscore) construction were performed using three different machine learning methods: least absolute shrinkage and selection operator (LASSO), random forest (RF) and support vector machine (SVM). The SVM-derived Radscore demonstrated a strong correlation with the pCR status, yielding area under the receiver operating characteristic curves (AUCs) of 0.880 and 0.830 in the training and validation cohorts, respectively, outperforming the RF and LASSO methods. Based on this, a nomogram was developed by combining the SVM-based Radscore with clinical indicators to predict pCR after neoadjuvant chemoradiotherapy. The nomogram exhibited superior predictive power, achieving AUCs of 0.910 and 0.866 in the training and validation cohorts, respectively. Calibration curves and decision curve analyses confirmed its appropriateness. The SVM-based Radscore demonstrated promising performance in predicting pCR for LARC patients. The machine learning-driven nomogram, which integrates the Radscore and clinical indicators, represents a valuable tool for predicting pCR in LARC patients.

1. Introduction

Locally advanced rectal cancer (LARC) is a prevalent malignancy that is encountered globally. The standard treatment for LARC involves total mesorectal excision (TME) surgery following neoadjuvant chemoradiotherapy (nCRT), resulting in pathologic complete response (pCR) in approximately 15% to 27% of patients [1,2,3]. Controversially, the “watch and wait” strategy has emerged as a preferred option due to reduced surgical risks and improved postoperative quality of life in patients achieving a complete response [4,5]. Currently, pCR can only be determined through the histopathologic examination of surgically resected specimens. Therefore, the critical challenge lies in developing a non-invasive and accurate method to identify complete response after chemoradiotherapy.
Medical imaging techniques such as positron emission tomography (PET), endorectal ultrasound (EUS) and magnetic resonance imaging (MRI) have been utilized as non-invasive methods to assess the response to CRT and evaluate pCR [6,7,8,9,10,11]. PET/CT can effectively detect abnormal metabolic activity and metastases, and the features extracted from PET/CT imaging have shown potential in predicting treatment response. EUS allows the direct visualization of the rectal wall and adjacent structures, making it valuable in predicting response to CRT. Similarly, MRI holds promise in predicting responses to CRT due to its significant role in cancer diagnosis and staging. Different imaging modalities have been employed to assist in predicting pCR, yielding promising results. However, accessing these additional imaging modalities such as PET, EUS and MRI often requires more time and effort compared to CT imaging. Moreover, CT imaging plays a crucial role in radiotherapy planning for LARC patients. It provides detailed anatomical information, allowing for the accurate delineation of the gross tumor volume (GTV) in the rectal region. This delineation is essential for precise targeting during radiotherapy delivery. Additionally, CT imaging facilitates the extraction of radiomics features, which are quantitative metrics derived from medical images. These features capture tumor heterogeneity and pathophysiological characteristics and can be used for predictive modeling and treatment planning.
Recent studies in radiomics have highlighted the remarkable potential of CT-based radiomics analyses in predicting the response to CRT in LARC, as evidenced by numerous references [12,13,14,15,16,17,18,19,20]. The majority of these studies primarily focused on predicting the pCR status [12,13,14,15,16,17,18]. Despite these valuable contributions, the current body of research still faces several limitations that need to be addressed. One of the prominent challenges is related to the quality and characteristics of the datasets utilized. This includes the lack of independent validation datasets [13,15], inadequately small sample sizes [12,15] and the presence of imbalanced test datasets [17]. Despite achieving exceptional performance on the training datasets, these models often exhibit significant underperformance when tested on independent validation datasets [16]. Consequently, concerns arise regarding the robustness and generalizability of the developed models. Another critical gap in the existing literature is the insufficient integration of clinical indicators to enhance response prediction [12,13,14,15]. Incorporating relevant clinical factors in combination with radiomic features can provide a more comprehensive understanding of the predictive models, yielding more accurate and reliable results. Furthermore, it is paramount to explore a robust classifier to ensure the development of reliable predictive models [18]. Different machine learning algorithms should be evaluated to identify the most suitable and effective approach for predicting pCR in LARC based on radiomics scores.
The radiomics score (Radscore), which is calculated by assigning weights to selected features based on their respective coefficients, has been employed in prior research to predict the response to CRT in patients with LARC [16,18,21,22]. However, to the best of our knowledge, no study has comprehensively evaluated its feasibility and effectiveness in combination with diverse machine learning techniques. This research gap emphasizes the need for further investigations to leverage the potential synergies between Radscores and machine learning techniques (LASSO, RF and SVM), aiming to advance the accuracy and applicability of predictive models for pCR prediction in LARC. Therefore, the purpose of our study was to develop and validate a robust radiomics nomogram that incorporates the Radscore derived from planning CT scans, along with clinical indicators, for the preoperative prediction of pCR in patients with LARC. Our work aims to non-invasively evaluate patient outcomes and assist LARC patients in potentially avoiding surgery through the use of a radiomics signature obtained from planning CT scans.

2. Materials and Methods

2.1. Patients

This retrospective study was approved by our institutional review board. We consecutively enrolled patients diagnosed with LARC who underwent CRT and subsequent surgery at the Second Affiliated Hospital, Zhejiang University School of Medicine, from October 2015 to June 2021. The patient recruitment process, along with the inclusion and exclusion criteria, are summarized in Figure S1.
The inclusion criteria for this study were as follows: (1) LARC patients diagnosed with histologically confirmed rectal adenocarcinoma; (2) patients who underwent long-course neoadjuvant CRT followed by TME surgical resection; (3) patients with available pretreatment contrast-enhanced planning CT images acquired using the same CT scanner. On the other hand, the exclusion criteria were: (1) patients whose surgery was either canceled or delayed for more than 8 months after completing CRT; (2) patients diagnosed with non-adenocarcinoma LARC; (3) patients with inadequate clinicopathological data or CT images of poor quality.
A total of 211 consecutive patients were enrolled in this study, and they were divided into two datasets in a ratio of 7:3 using computer-generated random numbers. The training cohort consisted of 148 patients, while the validation cohort included 63 patients. The allocation was conducted in a manner that maintained a balanced distribution of pCR rates between the two cohorts. Baseline clinical characteristics, such as age, gender, body mass index (BMI), T stage, N stage, distance from anal verge, tumor volume, tumor diameter, tumor length, prescription dose, levels of carcinoembryonic antigen (CEA) and carbohydrate antigen 19-9 (CA19-9), were extracted from the patients’ pretreatment medical records (Table S1).

2.2. Standard of Reference

All patients included in this study underwent intensity-modulated radiation therapy (IMRT) with standard contouring. The total radiation dose administered was either 50 Gy (delivered in daily fractions of 2.0 Gy) or 57.5 Gy (delivered in daily fractions of 2.3 Gy), administered 5 days a week over a duration of 5 weeks. Concurrently, chemotherapy was administered alongside radiation therapy using capecitabine at a dose of 825 mg/m2 orally twice daily on radiation therapy days. Then, patients receive capecitabine alone or CAPEOX chemotherapy for two cycles before surgery. TME surgery was performed within 12 weeks of the completion of neoadjuvant CRT.
The surgical resection specimens were meticulously evaluated by specialized gastrointestinal pathologists who were blinded to the patients’ clinical and CT findings. Pathological tumor staging was conducted in accordance with the American Joint Committee on Cancer (AJCC) TNM system. The response to CRT was assessed using the tumor response grading (TRG) system. Based on their response, patients were then categorized into two groups: pCR and non-pCR. pCR was defined as the absence of viable tumor cells in both the primary tumor and the associated lymph nodes. The study workflow is depicted in Figure 1.

2.3. Feature Extraction

All patients in the study underwent a simulation CT scan utilizing the LightSpeed RT system (GE Healthcare, Chicago, IL, USA), a 16-channel multi-detector row CT scanner specifically designed for radiotherapy treatment planning. These scans involved the administration of contrast-enhanced CT images following the intravenous injection of 80–100 mL of iodinated contrast material (Medrad, Bayer, Leverkusen, Germany) at a controlled rate of 2.0–3.0 mL/s. For each patient, images were acquired during the portal venous phase with a fixed delay time of 50 s. The acquired CT images were subsequently reconstructed to provide a pixel matrix of 512 × 512 and a slice thickness of 2.5 mm, ensuring precise visualization and analysis. To mitigate any potential variability arising from parameters related to voxel size, radiomics data were extracted from images that had been resampled to isometric voxels measuring 1 × 1 × 1 mm3. This standardized voxel size allowed consistent and comparable radiomics analysis across the dataset.
The primary tumors were independently contoured manually by two experienced abdominal radiation oncologists at our institution. Staging MR T2-weighted sequences were utilized as a supplementary tool to assist in target definition. To assess the reliability and consistency of the extracted features, intra- and interclass correlation coefficients (ICCs) were employed. Features with ICC values exceeding 0.75 were deemed suitable for further analysis in this study. Subsequently, the CT images and GTV structures were retrieved from the Eclipse Treatment Planning System (TPS), developed by Varian Medical System, and then transferred to an external workstation running auto-segmentation software named AccuContour 3.2 (Manteia, Milwaukee, WI, USA). This software allowed the automatic calculation of radiomics features based on the provided CT images and GTV structures.
A total of 851 imaging features were extracted from the planning CTs for each patient. These features were classified into several categories, which included 18 first-order features, 14 shape features, 24 grey-level co-occurrence matrix (GLCM) features, 16 grey-level run-length matrix (GLRLM) features, 16 gray-level size zone matrix (GLSZM) features, 14 gray-level dependence matrix (GLDM) features, and 5 neighbor gray-level difference matrix (NGTDM) features. Additionally, these categories comprised 744 high-dimensional feature types, encompassing both the first-order features and texture features calculated from the images using a wavelet filter with a three-step depth of decomposition. This comprehensive analysis involved considering various combinations of wavelet decompositions, specifically HHH, HHL, HLH, HLL, LHH, LHL, LLH and LLL (Table S4). Following feature extraction, all radiomics features were normalized using a z-score transformation method. This involved transforming the data into standardized intensity ranges for each imaging modality across all subjects, resulting in a mean of 0 and a standard deviation of 1.

2.4. Feature Selection and Radscore Construction

To construct a robust radiomics signature and select the optimal radiomics features from the initial pool of 851 radiomics features, we employed three feature selection steps. Initially, we conducted Pearson correlation analysis with an r threshold of 0.8 to identify and eliminate redundant features. Subsequently, features were selected based on univariate logistic regression between the pCR and non-pCR groups in the training cohort, using a threshold of 0.1 to avoid confounding effects during multivariate analysis.
In the next step, three commonly used machine learning methods were employed to select key features from the previously identified set: the least absolute shrinkage and selection operator (LASSO), random forest (RF) and support vector machine (SVM). These algorithms were utilized to identify the most relevant features for the task at hand. Radscores, which represent the radiomics scores, were calculated for each patient by linearly combining the selected features and weighting them by their respective coefficients. These Radscores were then used to predict the probability of pCR.
To obtain the LASSO radiomics score, we summed the selected radiomics features weighted by their non-zero coefficients. Features with minimal impact on the target variable were assigned a weight of zero and excluded from further analysis. In order to address potential overfitting issues, we employed the Fast Unified Random Forests for Survival, Regression, and Classification (RF-SRC) feature selection approach. This method ranks the variables based on minimal depth and determines the optimal number of features with the lowest out-of-bag (OOB) error. The RF radiomics score was obtained by summing the selected radiomics features and weighting them according to their importance derived from the RF-SRC method [23,24,25,26]. Additionally, the SVM-based Recursive Feature Elimination (SVM-RFE) algorithm was utilized to determine the optimal subset of features. The SVM-RFE algorithm iteratively removes features with the lowest weights and selects the most relevant features for the classification task. The SVM radiomics score was then calculated by summing the selected radiomics features weighted by their corresponding non-zero coefficients obtained from the linear SVM model [27,28,29,30].
By employing these advanced feature selection techniques and leveraging the importance of variables and coefficients, we were able to generate informative radiomics scores that effectively captured the discriminatory power of the selected radiomic features. These scores contribute to the development of robust classifiers and predictive models in radiomics research, enabling improved accuracy and reliability in various applications.
In the third step, we identified the radiomics classifier with the highest area under the curve (AUC) and accuracy as the optimal Radscore. All three classifiers were trained using 10-fold cross-validation on the training cohort to determine the optimal parameter configuration for each classifier. Subsequently, they were tested on the validation cohort. The performance of each classifier was evaluated using receiver operating characteristic (ROC) curves, and the AUC was used as the metric to assess their performance. Accuracy, sensitivity, specificity, negative-predictive value (NPV), and positive-predictive value (PPV) in both the training and validation sets were calculated based on the Youden index.

2.5. Nomogram Integrating Radscore and Clinical Indicators

The Radscore and all the aforementioned clinical candidate predictors were subjected to univariable and multivariable logistic regression analyses in the training cohort. In order to enhance the interpretability and facilitate statistical analysis, we employed a transformation method for continuous variables. Specifically, the continuous clinical indicators were transformed into categorical variables using ROC values. This allowed us to determine the optimal cut-off point for each continuous clinical feature. This categorization of variables aided in simplifying the statistical analysis process while ensuring its robustness and academic integrity.
Subsequently, a radiomics nomogram was developed based on the results of the multivariate logistic analysis. This nomogram was designed to estimate the individual probability of achieving a pCR within the training cohort. To assess the discriminatory ability of the radiomics nomogram, several performance metrics were calculated, including AUC, accuracy, sensitivity, specificity, PPV, and NPV. These metrics provided an evaluation of the nomogram’s effectiveness in distinguishing between different response outcomes and predicting the likelihood of achieving a pCR. To assess the calibration of the radiomics nomogram, a calibration curve was generated by plotting the actual probability of pCR against the predicted probability of pCR from the nomogram. Furthermore, the goodness-of-fit of the radiomics nomogram was evaluated using the Hosmer–Lemeshow test.
In order to evaluate the clinical utility of the radiomics nomogram, a decision curve analysis (DCA) was conducted. Net benefits were calculated at various threshold probabilities to determine the potential value of the nomogram in clinical decision making.
To validate the predictive performance of the radiomics nomogram, the multivariable logistic regression model derived from the training cohort was applied to all patients in the validation cohort. This allowed us to assess the generalizability and robustness of the nomogram in an independent dataset.

2.6. Statistical Analysis

All statistical analyses were performed using RStudio version 2022.12.0+353. The following packages in R software were utilized: “caret”, “glmnet”, “rms”, “pROC”, “rmda”, “MASS”, “fmsb”, “ggplot2”, “randomForestSRC”, “ggRandomForests”, “ResourceSelection”, “msvmRFE” and “e1071”. To compare the AUC values of the radiomics signature, a deLong test was employed. Chi-square tests were used to compare the differences in categorical variables (gender, age, BMI, tumor volume, tumor diameter, tumor length, T stage, N stage, CEA and CA19-9 levels, distance from anal verge, prescription dose), and two-sample t-test was used to compare the differences in continuous variable (Radscore). All statistical tests were two-sided, and p-values below 0.05 were considered statistically significant.

3. Results

3.1. Clinical Characteristics

In this study, a total of 211 consecutive patients were included, and 50 of them achieved pCR, representing a rate of 23.7%. The distribution of the pCR rate was similar in the training and validation cohorts (p = 0.390). Additionally, other pretreatment clinical characteristics, such as gender, age, BMI, tumor volume, tumor diameter, tumor length, T stage, CEA and CA19-9 levels, distance from anal verge, and prescription dose, were balanced between the cohorts, with the exception of N stage (p = 0.025). Furthermore, there were no significant differences in these clinical characteristics between the pCR and non-pCR groups, except for tumor volume within the training cohort (p = 0.005). For a detailed summary of the clinical characteristics, please refer to Table 1.

3.2. Feature Selection and Radscore Construction

A total of 851 imaging features were extracted from the pretreatment planning CT images. To construct the Radscore using the aforementioned dataset, we initially performed Pearson correlation analysis and univariate logistic regression in the training cohort. This process resulted in the selection of 44 features as potential predictors. Subsequently, the LASSO, RF and SVM algorithms were trained using these selected features, and their differentiation abilities were evaluated in the validation cohort.
The LASSO was used to identify the optimized subset of features and calculate the Radscore for each patient. Fourteen features with non-zero coefficients were identified using minimum criteria (Figure 2A,B). For the RF algorithm, 16 features were chosen based on the minimal depth criterion in the training cohort (Figure 2C,D). The SVM algorithm selected the top 18 features with the highest accuracy for pCR prediction (Figure 2E,F). The coefficients and the calculation formula for the Radscores are provided in Table S3. Consequently, the Radscores were calculated for each patient in both the training and validation cohorts (Figure S2A–C).
The Radscore derived from the SVM method demonstrated AUC values of 0.880 (95% confidence interval [CI], 0.823–0.946) and 0.830 (95% CI, 0.722–0.928) in the training and validation cohorts, respectively. Similarly, the Radscore derived from the LASSO and RF methods exhibited AUCs of 0.841 (95% CI, 0.758–0.924) and 0.818 (95% CI, 0.734–0.902) in the training cohort, which were further validated in the validation cohort with AUCs of 0.806 (95% CI, 0.696–0.916) and 0.791 (95% CI, 0.670–0.912). These findings suggest that the Radscore derived from the SVM model demonstrated better predictive efficacy compared to the other two methods (Figure 3A,B).
The accuracy, sensitivity, specificity, NPV and PPV were high for the SVM in both the training and validation cohorts, while specificity and PPV were relatively lower in the training cohort. The radar chart depicted the performance metrics of accuracy, sensitivity, specificity, NPV, and PPV for the three classifiers in both the training and validation cohorts (Figure 3C,D). For more detailed information on the performance of the Radscore, please refer to Table S2.
The Radscore derived from the SVM was significantly higher in the pCR group compared to the non-pCR group in both the training cohort (−0.2601 ± 1.1362 vs. −1.6742 ± 0.8722, p < 0.0001) and the validation cohort (−0.3133 ± 1.0479 vs. −1.6379 ± 0.9098, p < 0.0001) (Table 1, Figure 4).

3.3. Nomogram Integrating Radscore and Clinical Indicators

The Radscore based on a SVM algorithm, along with clinical indicators such as pretreatment T stage, were identified as independent predictors for predicting the pCR status. These results were obtained through univariable and multivariate logistic regression analyses, as shown in Table 2. Subsequently, we constructed a radiomics nomogram incorporating the independent predictor, as illustrated in Figure 5.
The AUC for the probability of achieving pCR derived from the radiomics nomogram was 0.910 (95% CI, 0.815–0.976) in the training cohort and 0.866 (95% CI, 0.762–0.970) in the validation cohort. For comparison, AUC values of 0.770 (95% CI, 0.629–0.911) in the training cohort and 0.725 (95% CI, 0.642–0.808) in the validation cohort were obtained from the clinical indicators alone. These findings indicate that the radiomics nomogram exhibited superior predictive efficacy compared to both the Radscore alone and the clinical indicators alone, as shown in Figure 6A,B. Furthermore, the Hosmer–Lemeshow test yielded non-significant p-values of 0.281 and 0.585 in the training and validation cohorts, respectively, suggesting a good fit of the nomogram to the observed data. Detailed information regarding the performance of the three models can be found in Table 3.
The predictive performance of the LASSO-based Radscore, incorporating clinical indicators, was compared to that of the RF-based Radscore in conjunction with clinical indicators. These models were subsequently assessed against an SVM-based nomogram. In the training cohort, the LASSO-based Radscore achieved an AUC of 0.889 (95% CI: 0.824–0.954), while in the validation cohort, it yielded an AUC of 0.825 (95% CI: 0.722–0.928). Similarly, the RF-based Radscore, when utilized with clinical indicators, exhibited an AUC of 0.835 (95% CI: 0.748–0.922) in the training cohort and an AUC of 0.815 (95% CI: 0.709–0.922) in the validation cohort. For a more comprehensive assessment of the performance of these three models, please consult Figure S3.
The calibration curve of the radiomics nomogram displayed favorable agreement between the probabilities predicted by the nomogram and the actual probabilities of pCR observed in the training cohort. This favorable calibration and strong performance of the radiomics nomogram were confirmed in the validation cohort, as demonstrated in Figure 6C,D. The calibration plot for the nomogram showed closer alignment with the diagonal line compared to the other models, indicating a higher predictive accuracy.
Lastly, the decision curve analysis (DCA) revealed that the radiomics nomogram, incorporating both radiomics scores and clinical indicators, provided the highest clinical net benefit in both the training and validation cohorts. This suggests a strong performance of the nomogram in terms of its clinical applicability, as depicted in Figure 6E,F.

4. Discussion

In our study, we employed a radiomics approach to extract 851 quantitative imaging features from pretreatment contrast-enhanced planning CT scans. Subsequently, we developed and validated a radiomics nomogram that incorporates the planning CT-based Radscore and the selected clinical indicator (pretreatment T stage). The proposed nomogram could be used for the individualized prediction of the pCR before neoadjuvant CRT for the patients with LARC. This was consistent with the findings of Cui et al. [21] and Chen et al. [31].
To develop the Radscore, we employed the SVM methodology to select the 18 suitable radiomics predictors from the initial 851 candidate radiomics features. The Radscore demonstrated favorable prognostic performance for pCR in the training cohort, with AUC values of 0.880, which were subsequently validated in the validation cohort with AUC values of 0.830. Numerous studies showed a correlation between radiomics score and treatment response. Our study confirmed this association, as the radiomics scores derived from planning CT images were significantly higher in the pCR group, with all corresponding AUC values being greater than or nearly 80%. While some studies favored SVM classifiers over the LASSO or RF methods [32,33,34,35] in the development of radiomics models, other studies reported the superior performance of the LASSO or RF classifiers [27,36,37,38,39]. These divergent findings suggest that the choice of machine learning model influenced the predictive performance. In our study, the SVM method outperformed the LASSO or RF classifiers in the prediction of pCR to neoadjuvant chemoradiotherapy in LARC. This may be because the SVM model performed well in high-dimensional space [40,41]. Additionally, SVM classifiers could handle small datasets effectively [42], as they require only a small number of support vectors to define decision boundaries. Furthermore, SVM demonstrates robustness against noise [43], making it a reliable method for non-invasively characterizing intratumoral heterogeneity.
Limited literature exists on the use of CT-based radiomics to predict pCR to neoadjuvant CRT in LARC patients (refer to Table S5). Yuan et al. and Lutsyk et al. reported achieving an AUC of 0.872 and an accuracy of 0.839, respectively, for predicting pCR in LARC patients receiving neoadjuvant CRT using the RF method based on non-contrast-enhanced CT images [12,13]. Conversely, the model reported by Hamerla et al. [44] using the RF method showed no predictive power for treatment response. In our study, we utilized contrast-enhanced CT images for radiomic analysis, which should offer superior performance compared to non-contrast-enhanced images. Contrast enhancement enables the visualization of more intricate details regarding the heterogeneous internal architecture of malignant tumors. Previous results also support the use of radiomics based on contrast-enhanced CT images in predicting treatment response in LARC [14,15,16,17,18]. Bonomo et al. and Bibault et al. investigated the capability of radiomics features derived from contrast-enhanced CT for predicting pCR; their findings revealed AUC values of 0.63 and a predictive accuracy of 0.80 [14,15]. These values are significantly lower than the AUC values obtained in our study, which utilized a greater number of high-dimensional features, providing more detailed information on intratumoral heterogeneity. It is important to note that the absence of independent validation or small sample size in previous studies may have hindered the clinical applicability of their findings.
A recently established predictive model which incorporates clinical indicators and the LASSO-based Radscore has shown improved prognostic performance with an AUC of 0.822 [16]. Similarly, another predictive model utilizing the RF method, integrating clinical indicators, dose-volume histogram (DVH), and radiomics texture, achieved an AUC of 0.828 [17]. These findings align with Mao et al.’s study [18], where the integration of clinical indicators and Radscore via a nomogram successfully improved the prediction of pCR in patients with LARC using the LASSO method (AUC: 0.872). Consistent with these results, our study demonstrates that the integration of the SVM-based Radscore with clinical indicators (AUC: 0.866) yields superior performance when contrasted with both the LASSO method combined with clinical indicators (AUC: 0.825) and the RF method coupled with clinical indicators (AUC: 0.815) (see Figure S3). Our findings suggest that the SVM-based nomogram, characterized by a higher AUC and superior calibration, exhibits enhanced predictive efficacy compared to either the Radscore or clinical indicators in isolation. Consequently, our study introduces a radiomics nomogram as a readily applicable and valuable tool for the non-invasive analysis and characterization of rectal cancer.
However, the successful implementation of radiomics in clinical practice faces challenges, including the accurate segmentation and extraction of stable and comparable quantitative image features. Compared to MRI or PET/CT, planning CT images, readily available in radiation therapy departments, provide higher consistency and professionalism, as all tumor regions of interest (ROIs) were manually contoured by experienced radiation oncologists. Thus, the crucial factor lies in identifying a reliable feature selection method that maximizes the accuracy of the radiomics signature. To address this, our study applied various machine learning classifiers, including LASSO, RF and SVM, to the extracted features for key feature selection and generating a radiomics score in both the training and validation cohorts.
Despite these findings, our study exhibits several limitations. Firstly, the sample size of patients with pCR was small, with just 12 patients achieving a pCR in the validation cohort. This limitation is further compounded by the inherent challenge of an imbalanced dataset. Although we diligently addressed this imbalance through the application of diverse machine learning methods and conducted a comprehensive performance evaluation, it is evident that the model’s robustness may be affected. Secondly, our radiomics model only utilized CT images, warranting the further investigation of various modalities such as MRI, molecular biomarkers and gene expression. Moreover, our study solely utilized pre-CRT CT images rather than combining pre- and post-CRT images. Nevertheless, current studies in this field suffer from overall low quality and exhibit heterogeneity, limiting the robustness and replicability of the conclusions [45]. Lastly, as a retrospective study conducted in a single center, it is essential to validate our proposed model in a large-scale, independent, multicenter cohort to assess reproducibility and robustness.

5. Conclusions

A Radscore derived from SVM and based on pretreatment contrast-enhanced planning CT scans was developed for predicting pCR in patients with LARC. The Radscore exhibited promising performance in accurately predicting pCR. Moreover, the proposed nomogram, which incorporates the SVM-derived Radscore and clinical indicators, holds considerable value as a non-invasive tool for evaluating treatment outcomes in LARC patients. This approach has the potential to enhance clinical decision making and contribute to personalized patient management strategies.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers15215134/s1. Supplementary materials are available regarding the patient recruitment pathway, radiomic features, and Radscore construction, providing further details beyond the initial descriptions. Figure S1: Patient recruitment pathway of LARC who received neoadjuvant CRT; Figure S2: Radiomic feature coefficients of three radiomics classifiers; Figure S3: Predictive performance evaluation of three methods: LASSO, RF and SVM, when combined with clinical indicators; Table S1: Pretreatment clinical characteristics of patients with LARC; Table S2: Performance evaluation of three radiomics classifiers; Table S3: Radiomics score calculation formulas of three radiomics classifiers; Table S4: Summary of radiomic features; Table S5: Summary of differences between our study and previous radiomics studies predicting pCR status in rectal cancer based on CT imaging.

Author Contributions

Conception and design: C.L., H.C., L.S. and Q.W.; Data collection and assembly: C.L., H.C., B.Z., Y.F., W.S. and Z.S.; Methodology development: C.L., H.C., D.W. and L.S.; Data analysis and interpretation: C.L., H.C., B.Z., D.W. and Z.S.; Manuscript drafting: C.L. and H.C.; Manuscript revision: W.S., L.S. and Q.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Natural Science Foundation of China (No. 82073142, DW, PW, CL; No. 82103498, HC; No. 62103366, WS); Beijing Xisike Clinical Oncology Research Foundation (Y-zai2022/qn-0213, CL).

Institutional Review Board Statement

This study received approval from the Institutional Review Board of the Second Affiliated Hospital of Zhejiang University School of Medicine (Ethical approval code: 2023 0496).

Informed Consent Statement

Informed consent was obtained from all study participants.

Data Availability Statement

The datasets utilized in the present study can be obtained from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Van Gijn, W.; Marijnen, C.A.; Nagtegaal, I.D.; Kranenbarg, E.M.-K.; Putter, H.; Wiggers, T.; Rutten, H.J.; Påhlman, L.; Glimelius, B.; van de Velde, C.J.; et al. Preoperative radiotherapy combined with total mesorectal excision for resectable rectal cancer: 12-year follow-up of the multicentre, randomised controlled TME trial. Lancet Oncol. 2011, 12, 575–582. [Google Scholar] [CrossRef] [PubMed]
  2. Kapiteijn, E.; Marijnen, C.A.; Nagtegaal, I.D.; Putter, H.; Steup, W.H.; Wiggers, T.; Rutten, H.J.; Pahlman, L.; Glimelius, B.; Van Krieken, J.H.; et al. Preoperative radiotherapy combined with total mesorectal excision for resectable rectal cancer. N. Engl. J. Med. 2001, 345, 638–646. [Google Scholar] [CrossRef] [PubMed]
  3. Maas, M.; Nelemans, P.J.; Valentini, V.; Das, P.; Rödel, C.; Kuo, L.-J.; Calvo, F.A.; García-Aguilar, J.; Glynne-Jones, R.; Haustermans, K.; et al. Long-term outcome in patients with a pathological complete response after chemoradiation for rectal cancer: A pooled analysis of individual patient data. Lancet Oncol. 2010, 11, 835–844. [Google Scholar] [CrossRef]
  4. Renehan, A.G.; Malcomson, L.; Emsley, R.; Gollins, S.; Maw, A.; Myint, A.S.; Rooney, P.S.; Susnerwala, S.; Blower, A.; Saunders, M.P.; et al. Watch-and-wait approach versus surgical resection after chemoradiotherapy for patients with rectal cancer (the OnCoRe project): A propensity-score matched cohort analysis. Lancet Oncol. 2016, 17, 174–183. [Google Scholar] [CrossRef] [PubMed]
  5. Maas, M.; Beets-Tan, R.G.; Lambregts, D.M.; Lammering, G.; Nelemans, P.J.; Engelen, S.M.; van Dam, R.M.; Jansen, R.L.; Sosef, M.; Leijtens, J.W.; et al. Wait-and-see policy for clinical complete responders after chemoradiation for rectal cancer. J. Clin. Oncol. 2011, 29, 4633–4640. [Google Scholar] [CrossRef]
  6. Horvat, N.; Veeraraghavan, H.; Khan, M.; Blazic, I.; Zheng, J.; Capanu, M.; Sala, E.; Garcia-Aguilar, J.; Gollub, M.J.; Petkovska, I. MR imaging of rectal cancer: Radiomics analysis to assess treatment response after neoadjuvant therapy. Radiology 2018, 287, 833–843. [Google Scholar] [CrossRef]
  7. Van Stiphout, R.G.; Valentini, V.; Buijsen, J.; Lammering, G.; Meldolesi, E.; van Soest, J.; Leccisotti, L.; Giordano, A.; Gambacorta, M.A.; Dekker, A.; et al. Nomogram predicting response after chemoradiotherapy in rectal cancer using sequential PETCT imaging: A multicentric prospective study with external validation. Radiother. Oncol. 2014, 113, 215–222. [Google Scholar] [CrossRef]
  8. Li, N.; Dou, L.; Zhang, Y.; Jin, J.; Wang, G.; Xiao, Q.; Li, Y.; Wang, X.; Ren, H.; Fang, H.; et al. Use of sequential endorectal US to predict the tumor response of preoperative chemoradiotherapy in rectal cancer. Gastrointest. Endosc. 2017, 85, 669–674. [Google Scholar] [CrossRef]
  9. Joye, I.; Deroose, C.M.; Vandecaveye, V.; Haustermans, K. The role of diffusion-weighted MRI and (18)F-FDG PET/CT in the prediction of pathologic complete response after radiochemotherapy for rectal cancer: A systematic review. Radiother. Oncol. 2014, 113, 158–165. [Google Scholar] [CrossRef]
  10. Shin, J.; Seo, N.; Baek, S.-E.; Son, N.-H.; Lim, J.S.; Kim, N.K.; Koom, W.S.; Kim, S. MRI radiomics model predicts pathologic complete response of rectal cancer following chemoradiotherapy. Radiology 2022, 303, 351–358. [Google Scholar] [CrossRef]
  11. Giannini, V.; Mazzetti, S.; Bertotto, I.; Chiarenza, C.; Cauda, S.; Delmastro, E.; Bracco, C.; Di Dia, A.; Leone, F.; Medico, E.; et al. Predicting locally advanced rectal cancer response to neoadjuvant therapy with 18F-FDG PET and MRI radiomics features. Eur. J. Nucl. Med. Mol. Imaging 2019, 46, 878–888. [Google Scholar] [CrossRef] [PubMed]
  12. Yuan, Z.; Frazer, M.; Zhang, G.G.; Latifi, K.; Moros, E.G.; Feygelman, V.; Felder, S.; Sanchez, J.; Dessureault, S.; Imanirad, I.; et al. CT-based radiomic features to predict pathological response in rectal cancer: A retrospective cohort study. J. Med. Imaging Radiat. Oncol. 2020, 64, 444–449. [Google Scholar] [CrossRef] [PubMed]
  13. Lutsyk, M.; Gourevich, K.; Keidar, Z. Complete pathologic response prediction by radiomics wavelets features of unenhanced CT simulation images in locally advanced rectal cancer patients after neoadjuvant chemoradiation. Isr. Med. Assoc. J. 2021, 23, 805–810. [Google Scholar] [CrossRef] [PubMed]
  14. Bonomo, P.; Fernandez, J.S.; Thorwarth, D.; Casati, M.; Livi, L.; Zips, D.; Gani, C. Simulation CT-based radiomics for prediction of response after neoadjuvant chemo-radiotherapy in patients with locally advanced rectal cancer. Radiat. Oncol. 2022, 17, 84. [Google Scholar] [CrossRef]
  15. Bibault, J.E.; Giraud, P.; Housset, M.; Durdux, C.; Taieb, J.; Berger, A.; Coriat, R.; Chaussade, S.; Dousset, B.; Nordlinger, B. Deep learning and radiomics predict complete response after neo-adjuvant chemoradiation for locally advanced rectal cancer. Sci. Rep. 2018, 8, 12611. [Google Scholar] [CrossRef]
  16. Zhuang, Z.; Liu, Z.; Li, J.; Wang, X.; Xie, P.; Xiong, F.; Hu, J.; Meng, X.; Huang, M.; Deng, Y.; et al. Radiomic signature of the FOWARC trial predicts pathological response to neoadjuvant treatment in rectal cancer. J. Transl. Med. 2021, 19, 256. [Google Scholar] [CrossRef] [PubMed]
  17. Wang, D.; Lee, S.H.; Geng, H.; Zhong, H.; Plastaras, J.; Wojcieszynski, A.; Caruana, R.; Xiao, Y. Interpretable machine learning for predicting pathologic complete response in patients treated with chemoradiation therapy for rectal adenocarcinoma. Front. Artif. Intell. 2022, 5, 1059033. [Google Scholar] [CrossRef]
  18. Mao, Y.; Pei, Q.; Fu, Y.; Liu, H.; Chen, C.; Li, H.; Gong, G.; Yin, H.; Pang, P.; Lin, H.; et al. Pre-treatment computed tomography radiomics for predicting the response to neoadjuvant chemoradiation in locally advanced rectal cancer: A retrospective study. Front. Oncol. 2022, 12, 850774. [Google Scholar] [CrossRef]
  19. Zhang, Z.; Yi, X.; Pei, Q.; Fu, Y.; Li, B.; Liu, H.; Han, Z.; Chen, C.; Pang, P.; Lin, H.; et al. CT radiomics identifying non-responders to neoadjuvant chemoradiotherapy among patients with locally advanced rectal cancer. Cancer Med. 2023, 12, 2463–2473. [Google Scholar] [CrossRef]
  20. Vandendorpe, B.; Durot, C.; Lebellec, L.; Le Deley, M.-C.; Sylla, D.; Bimbai, A.-M.; Amroun, K.; Ramiandrisoa, F.; Cordoba, A.; Mirabel, X.; et al. Prognostic value of the texture analysis parameters of the initial computed tomographic scan for response to neoadjuvant chemoradiation therapy in patients with locally advanced rectal cancer. Radiother. Oncol. 2019, 135, 153–160. [Google Scholar] [CrossRef]
  21. Cui, Y.; Yang, X.; Shi, Z.; Yang, Z.; Du, X.; Zhao, Z.; Cheng, X. Radiomics analysis of multiparametric MRI for prediction of pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Eur. Radiol. 2019, 29, 1211–1220. [Google Scholar] [CrossRef] [PubMed]
  22. Zhang, S.; Tang, B.; Yu, M.; He, L.; Zheng, P.; Yan, C.; Li, J.; Peng, Q. Development and validation of a radiomics model based on lymph-node regression grading after neoadjuvant chemoradiotherapy in locally advanced rectal cancer. Int. J. Radiat. Oncol. Biol. Phys. 2023, 117, 821–833. [Google Scholar] [CrossRef] [PubMed]
  23. Chen, H.-Z.; Wang, X.-R.; Zhao, F.-M.; Chen, X.-J.; Li, X.-S.; Ning, G.; Guo, Y.-K. The development and validation of a CT-based radiomics nomogram to preoperatively predict lymph node metastasis in high-grade serous ovarian cancer. Front. Oncol. 2021, 11, 711648. [Google Scholar] [CrossRef] [PubMed]
  24. Yang, X.; Dong, X.; Wang, J.; Li, W.; Gu, Z.; Gao, D.; Zhong, N.; Guan, Y. Computed Tomography-Based Radiomics Signature: A Potential Indicator of Epidermal Growth Factor Receptor Mutation in Pulmonary Adenocarcinoma Appearing as a Subsolid Nodule. Oncologist 2019, 24, e1156–e1164. [Google Scholar] [CrossRef] [PubMed]
  25. Ye, C.; Ren, S.; Sadula, A.; Guo, X.; Yuan, M.; Meng, M.; Li, G.; Zhang, X.; Yuan, C. The expression characteristics of transmembrane protein genes in pancreatic ductal adenocarcinoma through comprehensive analysis of bulk and single-cell RNA sequence. Front. Oncol. 2023, 13, 1047377. [Google Scholar] [CrossRef]
  26. Ishwaran, H.; Lu, M. Standard errors and confidence intervals for variable importance in random forest regression, classification, and survival. Stat. Med. 2019, 38, 558–582. [Google Scholar] [CrossRef]
  27. Chen, H.-Z.; Wang, X.-R.; Zhao, F.-M.; Chen, X.-J.; Li, X.-S.; Ning, G.; Guo, Y.-K. A CT-based radiomics nomogram for predicting early recurrence in patients with high-grade serous ovarian cancer. Eur. J. Radiol. 2021, 145, 110018. [Google Scholar] [CrossRef]
  28. Liu, Z.; Li, M.; Zuo, C.; Yang, Z.; Yang, X.; Ren, S.; Peng, Y.; Sun, G.; Shen, J.; Cheng, C.; et al. Radiomics model of dual-time 2-[18F]FDG PET/CT imaging to distinguish between pancreatic ductal adenocarcinoma and autoimmune pancreatitis. Eur. Radiol. 2021, 31, 6983–6991. [Google Scholar] [CrossRef]
  29. Gu, D.; Xie, Y.; Wei, J.; Li, W.; Ye, Z.; Zhu, Z.; Tian, J.; Li, X. MRI-Based Radiomics Signature: A Potential Biomarker for Identifying Glypican 3-Positive Hepatocellular Carcinoma. J. Magn. Reson. Imaging 2020, 52, 1679–1687. [Google Scholar] [CrossRef]
  30. Duan, K.-B.; Rajapakse, J.; Wang, H.; Azuaje, F. Multiple SVM-RFE for gene selection in cancer classification with expression data. IEEE Trans. Nanobiosci. 2005, 4, 228–234. [Google Scholar] [CrossRef]
  31. Chen, B.-Y.; Xie, H.; Li, Y.; Jiang, X.-H.; Xiong, L.; Tang, X.-F.; Lin, X.-F.; Li, L.; Cai, P.-Q. MRI-based radiomics features to predict treatment response to neoadjuvant chemotherapy in locally advanced rectal cancer: A single center, prospective study. Front. Oncol. 2022, 12, 801743. [Google Scholar] [CrossRef] [PubMed]
  32. Sun, S.; Ren, L.; Miao, Z.; Hua, L.; Wang, D.; Deng, J.; Chen, J.; Liu, N.; Gong, Y. Application of MRI-Based Radiomics in Preoperative Prediction of NF2 Alteration in Intracranial Meningiomas. Front. Oncol. 2022, 12, 879528. [Google Scholar] [CrossRef]
  33. Luo, Y.; Chen, H.; Cui, M. Radiomics and Hybrid Models Based on Machine Learning to Predict Levodopa-Induced Dyskinesia of Parkinson’s Disease in the First 6 Years of Levodopa Treatment. Diagnostics 2023, 13, 2511. [Google Scholar] [CrossRef] [PubMed]
  34. Xiang, Y.; Li, S.; Wang, H.; Song, M.; Hu, K.; Wang, F.; Wang, Z.; Niu, Z.; Liu, J.; Cai, Y.; et al. MRI-based radiomics to predict neoadjuvant chemoradiotherapy outcomes in locally advanced rectal cancer: A multicenter study. Clin. Transl. Radiat. Oncol. 2022, 38, 175–182. [Google Scholar] [CrossRef]
  35. Deng, Z.; Dong, W.; Xiong, S.; Jin, D.; Zhou, H.; Zhang, L.; Xie, L.; Deng, Y.; Xu, R.; Fan, B. Machine learning models combining computed tomography semantic features and selected clinical variables for accurate prediction of the pathological grade of bladder cancer. Front. Oncol. 2023, 13, 1166245. [Google Scholar] [CrossRef]
  36. Meng, X.; Xia, W.; Xie, P.; Zhang, R.; Li, W.; Wang, M.; Xiong, F.; Liu, Y.; Fan, X.; Xie, Y.; et al. Preoperative radiomic signature based on multiparametric magnetic resonance imaging for noninvasive evaluation of biological characteristics in rectal cancer. Eur. Radiol. 2019, 29, 3200–3209. [Google Scholar] [CrossRef] [PubMed]
  37. Gao, X.; Cui, J.; Wang, L.; Wang, Q.; Ma, T.; Yang, J.; Ye, Z. The value of machine learning based radiomics model in preoperative detection of perineural invasion in gastric cancer: A two-center study. Front. Oncol. 2023, 13, 1205163. [Google Scholar] [CrossRef] [PubMed]
  38. Wang, Q.; Chen, Y.; Qin, S.; Liu, X.; Liu, K.; Xin, P.; Zhao, W.; Yuan, H.; Lang, N. Prognostic Value and Quantitative CT Analysis in RANKL Expression of Spinal GCTB in the Denosumab Era: A Machine Learning Approach. Cancers 2022, 14, 5201. [Google Scholar] [CrossRef]
  39. Shi, Z.-X.; Li, C.-F.; Zhao, L.-F.; Sun, Z.-Q.; Cui, L.-M.; Xin, Y.-J.; Wang, D.-Q.; Kang, T.-R.; Jiang, H.-J. Computed tomography radiomic features and clinical factors predicting the response to first transarterial chemoembolization in intermediate-stage hepatocellular carcinoma. Hepatobiliary Pancreat. Dis. Int. 2023, in press. [Google Scholar] [CrossRef]
  40. Cortes, C.; Vapnik, V. Support vector networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
  41. Schölkopf, B.; Platt, J.C.; Shawe-Taylor, J.; Smola, A.J.; Williamson, R.C. Estimating the support of a high-dimensional distribution. Neural Comput. 2001, 13, 1443–1471. [Google Scholar] [CrossRef] [PubMed]
  42. Cervantes, J.; Garcia-Lamont, F.; Rodríguez-Mazahua, L.; Lopez, A. A comprehensive survey on support vector machine classification: Applications, challenges and trends. Neurocomputing 2020, 408, 189–215. [Google Scholar] [CrossRef]
  43. Ranzato, F.; Zanella, M. Robustness verification of support vector machines. In Static Analysis, Proceedings of the 26th International Symposium, SAS 2019, Porto, Portugal, 8–11 October 2019; Springer: Berlin/Heidelberg, Germany, 2019; p. 11822. [Google Scholar]
  44. Hamerla, G.; Meyer, H.J.; Hambsch, P.; Wolf, U.; Kuhnt, T.; Hoffmann, K.T.; Surov, A. Radiomics model based on non-contrast CT shows no predictive power for complete pathological response in locally advanced rectal cancer. Cancers 2019, 11, 1680. [Google Scholar] [CrossRef] [PubMed]
  45. Nardone, V.; Reginelli, A.; Grassi, R.; Boldrini, L.; Vacca, G.; D’ippolito, E.; Annunziata, S.; Farchione, A.; Belfiore, M.P.; Desideri, I.; et al. Delta radiomics: A systematic review. Radiol. Med. 2021, 126, 1571–1583. [Google Scholar] [CrossRef] [PubMed]
Cancers 15 05134 g001
Figure 1. Radiomics workflow adopted in our study, encompassing the following steps: (I) Manual segmentation of tumors was performed on the planning CT images, delineating the gross tumor volume (GTV) with a red circle. (II) A total of 850 radiomics features were extracted from the defined region of interest. (III) Radiomics feature selection techniques were employed to identify the most informative features. (IV) A radiomics nomogram was developed by integrating the radiomics score, calculated from the selected radiomics features, and relevant clinical indicators.
Figure 1. Radiomics workflow adopted in our study, encompassing the following steps: (I) Manual segmentation of tumors was performed on the planning CT images, delineating the gross tumor volume (GTV) with a red circle. (II) A total of 850 radiomics features were extracted from the defined region of interest. (III) Radiomics feature selection techniques were employed to identify the most informative features. (IV) A radiomics nomogram was developed by integrating the radiomics score, calculated from the selected radiomics features, and relevant clinical indicators.
Cancers 15 05134 g001
Cancers 15 05134 g002aCancers 15 05134 g002bCancers 15 05134 g002c
Figure 2. Development process of the three algorithms utilized in our study. (A) LASSO feature selection was performed using a minimum criteria approach. The optimal tuning parameter λ was determined to be 0.014, and 14 non-zero coefficients were selected. (B) The LASSO coefficient profiles of the radiomics features were examined. (C) RF feature selection was conducted based on the minimal depth approach, resulting in the selection of 16 features with depth higher than 3.222. (D) The optimal tuning parameters (x point) for the RF were determined as mtry = 18 and nodesize = 25 using the out-of-bag error. (E) SVM feature selection was employed to identify the most informative features using the feature rank approach. The optimal tuning parameters, gamma and cost, were determined to be 1 and 10, respectively. (F) A total of 18 features were selected based on evaluation of the cross-validation error. LASSO: least absolute shrinkage and selection operator; RF: random forest; SVM: support vector machines.
Figure 2. Development process of the three algorithms utilized in our study. (A) LASSO feature selection was performed using a minimum criteria approach. The optimal tuning parameter λ was determined to be 0.014, and 14 non-zero coefficients were selected. (B) The LASSO coefficient profiles of the radiomics features were examined. (C) RF feature selection was conducted based on the minimal depth approach, resulting in the selection of 16 features with depth higher than 3.222. (D) The optimal tuning parameters (x point) for the RF were determined as mtry = 18 and nodesize = 25 using the out-of-bag error. (E) SVM feature selection was employed to identify the most informative features using the feature rank approach. The optimal tuning parameters, gamma and cost, were determined to be 1 and 10, respectively. (F) A total of 18 features were selected based on evaluation of the cross-validation error. LASSO: least absolute shrinkage and selection operator; RF: random forest; SVM: support vector machines.
Cancers 15 05134 g002aCancers 15 05134 g002bCancers 15 05134 g002c
Cancers 15 05134 g003aCancers 15 05134 g003b
Figure 3. Predictive performance evaluation of the three methods, namely LASSO, RF and SVM, in both the training and validation cohorts. (A) ROC curves were plotted to visualize the performance in the training cohort. (B) Similarly, ROC curves were generated to assess the performance in the validation cohort. (C) The accuracy, sensitivity, specificity, NPV and PPV of the LASSO, RF and SVM were evaluated in the training cohort. (D) Likewise, the accuracy, sensitivity, specificity, NPV and PPV of the three classifiers were assessed in the validation cohort. LASSO: least absolute shrinkage and selection operator; RF: random forest; SVM: support vector machines; ROC: receiver operating characteristic; NPV: negative-predictive value; PPV: positive-predictive value.
Figure 3. Predictive performance evaluation of the three methods, namely LASSO, RF and SVM, in both the training and validation cohorts. (A) ROC curves were plotted to visualize the performance in the training cohort. (B) Similarly, ROC curves were generated to assess the performance in the validation cohort. (C) The accuracy, sensitivity, specificity, NPV and PPV of the LASSO, RF and SVM were evaluated in the training cohort. (D) Likewise, the accuracy, sensitivity, specificity, NPV and PPV of the three classifiers were assessed in the validation cohort. LASSO: least absolute shrinkage and selection operator; RF: random forest; SVM: support vector machines; ROC: receiver operating characteristic; NPV: negative-predictive value; PPV: positive-predictive value.
Cancers 15 05134 g003aCancers 15 05134 g003b
Cancers 15 05134 g004
Figure 4. Performance evaluation of the radiomics score (Radscore) calculated by the SVM, assessing its predictive abilities in both the training and validation cohorts. (A) The Radscore for each patient in the training cohort is illustrated, demonstrating the distribution and variation in the Radscore values. (B) Similarly, the Radscore for each patient is visualized in the validation cohort. SVM: support vector machines.
Figure 4. Performance evaluation of the radiomics score (Radscore) calculated by the SVM, assessing its predictive abilities in both the training and validation cohorts. (A) The Radscore for each patient in the training cohort is illustrated, demonstrating the distribution and variation in the Radscore values. (B) Similarly, the Radscore for each patient is visualized in the validation cohort. SVM: support vector machines.
Cancers 15 05134 g004
Cancers 15 05134 g005
Figure 5. Nomogram that integrates the radiomics score with the clinical indicators in our study.
Figure 5. Nomogram that integrates the radiomics score with the clinical indicators in our study.
Cancers 15 05134 g005
Cancers 15 05134 g006aCancers 15 05134 g006bCancers 15 05134 g006c
Figure 6. Performance evaluation of the clinical, radiomics, and nomogram models in both the training and validation cohorts. (A) ROC curves were plotted to assess and compare the discriminatory power of the clinical, radiomics and nomogram models in the training cohort. (B) Similarly, ROC curves were generated to evaluate the discriminatory abilities of these models in the validation cohort. (C) Calibration curves were constructed to examine the agreement between the predicted and observed outcomes of the clinical, radiomics and nomogram models in the training cohort. (D) Likewise, calibration curves were analyzed to assess the agreement between predicted and observed outcomes in the validation cohort for all three models. (E) DCA was performed to evaluate the clinical utility and net benefit of the clinical, radiomics and nomogram models in the training cohort. (F) Additionally, DCA was conducted to assess the clinical utility and net benefit of these models in the validation cohort. ROC: receiver operator characteristic; DCA: decision curve analysis.
Figure 6. Performance evaluation of the clinical, radiomics, and nomogram models in both the training and validation cohorts. (A) ROC curves were plotted to assess and compare the discriminatory power of the clinical, radiomics and nomogram models in the training cohort. (B) Similarly, ROC curves were generated to evaluate the discriminatory abilities of these models in the validation cohort. (C) Calibration curves were constructed to examine the agreement between the predicted and observed outcomes of the clinical, radiomics and nomogram models in the training cohort. (D) Likewise, calibration curves were analyzed to assess the agreement between predicted and observed outcomes in the validation cohort for all three models. (E) DCA was performed to evaluate the clinical utility and net benefit of the clinical, radiomics and nomogram models in the training cohort. (F) Additionally, DCA was conducted to assess the clinical utility and net benefit of these models in the validation cohort. ROC: receiver operator characteristic; DCA: decision curve analysis.
Cancers 15 05134 g006aCancers 15 05134 g006bCancers 15 05134 g006c
Table 1. Pretreatment characteristics of patients with pCR in the training and validation cohorts.
Table 1. Pretreatment characteristics of patients with pCR in the training and validation cohorts.
Training CohortValidation Cohort
CharacteristicsNon-pCRpCRp ValueNon-pCRpCRp Value
Gender 0.819 0.960
Male74 (67.3%)27 (71.1%) 37 (72.5%)8 (66.7%)
Female36 (22.7%)11 (28.9%) 14 (27.5%)4 (33.3%)
Age 0.341 0.052
>6049 (44.5%)21 (55.3%) 27 (52.9%)2 (16.7%)
≤6061 (55.5%)17 (44.7%) 24 (47.1%)10 (83.3%)
BMI 0.056 0.687
>21.766 (60.0%)30 (78.9%) 37 (72.5%)10 (83.3%)
≤21.744 (40.0%)8 (21.1%) 14 (27.5%)2 (16.7%)
Tumor volume110380.005 *51120.914
>51.7 cm363 (57.3%)11 (28.9%) 29 (56.9%)6 (50.0%)
≤51.7 cm347 (42.7%)27 (71.1%) 22 (43.1%)6 (50.0%)
Tumor diameter 0.061 0.237
>4.7 cm53 (48.2%)11 (28.9%) 25 (49.0%)3 (25.0%)
≤4.7 cm57 (51.8%)27 (71.1%) 26 (51.0%)9 (75.0%)
Tumor length 0.093 0.934
>4.7 cm79 (71.8%)21 (55.3%) 35 (31.4%)9 (75.0%)
≤4.7 cm31 (28.2%)17 (44.7%) 16 (68.6%)3 (25.0%)
Distance from anal verge 0.356 0.898
>4.0 cm86 (78.2%)33 (86.8%) 39 (76.5%)10 (83.3%)
≤4.0 cm24 (21.8%)5 (13.2%) 12 (23.5%)2 (16.7%)
T stage 0.951 0.225
24 (3.4%)1 (3.2%) 2 (3.9%)0 (0.0%)
359 (54.0%)21 (61.9%) 29 (56.9%)10 (83.3%)
447 (42.6%)16 (34.9%) 20 (39.2%)2 (16.7%)
N stage 0.650 0.627
09 (8.2%)3 (7.9%) 8 (25.7%)1 (8.3%)
133 (30.0%)12 (31.6%) 12 (23.5%)5 (41.7%)
237 (33.6%)16 (42.1%) 26 (51.0%)5 (41.7%)
331 (28.2%)7 (18.4%) 5 (9.8%)1 (8.3%)
CEA 1 0.237
>5 ng/mL49 (44.5%)17 (44.7%) 25 (49.0%)3 (25.0%)
≤5 ng/mL61 (55.5%)21 (55.3%) 26 (51.0%)9 (75.0%)
CA19-9 0.453 1
>37 U/mL13 (11.8%)7 (7.9%) 6 (11.8%)1 (8.3%)
≤37 U/mL97 (88.2%)31 (92.1%) 45 (88.2%)11 (91.7%)
Prescription dose 1 0.171
57.5 Gy80 (72.7%)27 (71.1%) 29 (56.9%)10 (83.3%)
50 Gy30 (27.3%)11 (28.9%) 22 (43.1%)2 (16.7%)
Radscore (mean ± SD)−1.6742 ± 0.8722−0.2601 ± 1.13620.000 *−1.6379 ± 0.9098−0.3133 ± 1.04790.000 *
pCR: pathological complete response; BMI: body mass index; CEA: carcinoembryonic antigen; CA19-9: carbohydrate antigen 19-9. * p < 0.05.
Table 2. Logistic regression analysis of risk factors for pCR.
Table 2. Logistic regression analysis of risk factors for pCR.
Univariate AnalysisMultivariate Analysis
FeaturesβOR (95% CI)p ValueβOR (95% CI)p Value
Gender0.1771.194 (0.543–2.757)0.666
Age0.4301.538 (0.734–3.261)0.255
BMI0.9162.500 (1.090–6.307)0.039 *
Tumor volume−1.1910.304 (0.133–0.659)0.003 *
Tumor diameter−0.8250.438 (0.191–0.949)0.042 *
Tumor length−0.7240.485 (0.226–1.045)0.063
Distance from anal verge0.6111.842 (0.694–5.825)0.251
T stage−1.0130.363 (0.137–0.595)0.035 *−1.1670.311 (0.104–0.841)0.027 *
N stage−0.1280.880 (0.588–1.317)0.533
CEA0.0081.008 (0.476–2.114)0.984
CA19-90.5221.685 (0.588–4.505)0.308
Prescription dose−0.0110.989 (0.889–1.107)0.842
Radscore1.4514.266 (2.658–7.470)0.000 *1.6515.212 (2.993–10.161)0.000 *
pCR: pathological complete response; β: regression coefficient; OR: odds ratio; CI: confidence interval; BMI: body mass index; CEA: carcinoembryonic antigen; CA19-9: carbohydrate antigen 19-9. * p < 0.05.
Table 3. Performance evaluation of three predictive models.
Table 3. Performance evaluation of three predictive models.
ClinicalRadiomicsNomogram
MetricsTraining (95% CI)Validation (95% CI)Training (95% CI)Validation (95% CI)Training (95%CI)Validation (95% CI)
AUC0.770 (0.629–0.911)0.725 (0.642–0.808)0.880 (0.823–0.946)0.830 (0.722–0.928)0.910 (0.815–0.976)0.866 (0.762–0.970)
Accuracy0.750 (0.676–0.764)0.683 (0.540–0.857)0.851 (0.703–0.912)0.810 (0.667–0.873)0.885 (0.797–0.946)0.841 (0.651–0.937)
Sensitivity0.605 (0.553–0.842)0.750 (0.500–1)0.711 (0.632–1)0.917 (0.750–1)0.816 (0.684–0.947)0.917 (0.667–1)
Specificity0.800 (0.554–0.918)0.667 (0.471–0.902)0.900 (0.627–0.964)0.784 (0.588–0.882)0.909 (0.763–0.991)0.824 (0.588–0.961)
PPV0.511 (0.343–0.786)0.346 (0.250–0.611)0.711 (0.462–0.879)0.500 (0.344–0.611)0.756 (0.581–0.968)0.550 (0.333–0.833)
NPV0.854 (0.815–0.926)0.919 (0.857–0.975)0.900 (0.880–0.989)0.976 (0.921–1)0.935 (0.897–0.980)0.977 (0.907–1)
MCC0.385 (0.094–0.742)0.332 (0.023–0.798)0.611 (0.228–0.934)0.577 (0.266–0.766)0.708 (0.409–0.947)0.624 (0.201–0.908)
F1 score0.554 (0.423–0.813)0.474 (0.333–0.759)0.711 (0.534–0.936)0.647 (0.472–0.759)0.785 (0.628–0.957)0.688 (0.444–0.909)
AUC: area under the receiver operator characteristic curves; NPV: negative-predictive value, PPV: positive-predictive value; CI: confidence interval; MCC: Matthew’s correlation coefficient.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Li, C.; Chen, H.; Zhang, B.; Fang, Y.; Sun, W.; Wu, D.; Su, Z.; Shen, L.; Wei, Q. Radiomics Signature Based on Support Vector Machines for the Prediction of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer. Cancers 2023, 15, 5134. https://doi.org/10.3390/cancers15215134

AMA Style

Li C, Chen H, Zhang B, Fang Y, Sun W, Wu D, Su Z, Shen L, Wei Q. Radiomics Signature Based on Support Vector Machines for the Prediction of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer. Cancers. 2023; 15(21):5134. https://doi.org/10.3390/cancers15215134

Chicago/Turabian Style

Li, Chao, Haiyan Chen, Bicheng Zhang, Yimin Fang, Wenzheng Sun, Dang Wu, Zhuo Su, Li Shen, and Qichun Wei. 2023. "Radiomics Signature Based on Support Vector Machines for the Prediction of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer" Cancers 15, no. 21: 5134. https://doi.org/10.3390/cancers15215134

APA Style

Li, C., Chen, H., Zhang, B., Fang, Y., Sun, W., Wu, D., Su, Z., Shen, L., & Wei, Q. (2023). Radiomics Signature Based on Support Vector Machines for the Prediction of Pathological Complete Response to Neoadjuvant Chemoradiotherapy in Locally Advanced Rectal Cancer. Cancers, 15(21), 5134. https://doi.org/10.3390/cancers15215134

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop