Next Article in Journal
Can Delta Radiomics Improve the Prediction of Best Overall Response, Progression-Free Survival, and Overall Survival of Melanoma Patients Treated with Immune Checkpoint Inhibitors?
Next Article in Special Issue
Comparative Analysis of Repeatability in CT Radiomics and Dosiomics Features under Image Perturbation: A Study in Cervical Cancer Patients
Previous Article in Journal
Development of a Multidisciplinary Care Pathway for Fracture Prevention in Men with Prostate Cancer at Initiation of Androgen Deprivation Therapy
Previous Article in Special Issue
18F-Fluorodeoxyglucose Positron Emission Tomography-Based Risk Score Model for Prediction of Five-Year Survival Outcome after Curative Resection of Non-Small-Cell Lung Cancer
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cancers
Volume 16
Issue 15
10.3390/cancers16152668
cancers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review

by
Valeria Trojani
1,*,
Maria Chiara Bassi
2,
Laura Verzellesi
1 and
Marco Bertolini
1
1
Medical Physics, Azienda USL-IRCCS, 42123 Reggio Emilia, Italy
2
Medical Library, Azienda USL-IRCCS, 42123 Reggio Emilia, Italy
*
Author to whom correspondence should be addressed.
Cancers 2024, 16(15), 2668; https://doi.org/10.3390/cancers16152668
Submission received: 21 June 2024 / Revised: 16 July 2024 / Accepted: 24 July 2024 / Published: 26 July 2024
(This article belongs to the Special Issue Radiomics and Imaging in Cancer Analysis)
Browse Figures
Review Reports Versions Notes

Abstract

:

Simple Summary

This review investigates how preprocessing parameters are related to the reproducibility and reliability of radiomic features derived from multimodality imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), cone-beam CT (CBCT), and positron emission tomography (PET)/CT. Radiomics, which involves extracting quantitative features from medical images, shows great potential as a source of non-invasive clinical biomarkers but is hindered by variability in imaging parameters, especially during acquisition, reconstruction, and preprocessing. Standardizing and reporting preprocessing procedures is essential for consistent extraction of radiomic features, given their significant role in determining the robustness and reproducibility of these features.

Abstract

Background: Lately, radiomic studies featuring the development of a signature to use in prediction models in diagnosis or prognosis outcomes have been increasingly published. While the results are shown to be promising, these studies still have many pitfalls and limitations. One of the main issues of these studies is that radiomic features depend on how the images are preprocessed before their computation. Since, in widely known and used software for radiomic features calculation, it is possible to set these preprocessing parameters before the calculation of the radiomic feature, there are ongoing studies assessing the stability and repeatability of radiomic features to find the most suitable preprocessing parameters for every used imaging modality. Materials and Methods: We performed a comprehensive literature search using four electronic databases: PubMed, Cochrane Library, Embase, and Scopus. Mesh terms and free text were modeled in search strategies for databases. The inclusion criteria were studies where preprocessing parameters’ influence on feature values and model predictions was addressed. Records lacking information on image acquisition parameters were excluded, and any eligible studies with full-text versions were included in the review process, while conference proceedings and monographs were disregarded. We used the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) tool to investigate the risk of bias. We synthesized our data in a table divided by the imaging modalities subgroups. Results: After applying the inclusion and exclusion criteria, we selected 43 works. This review examines the impact of preprocessing parameters on the reproducibility and reliability of radiomic features extracted from multimodality imaging (CT, MRI, CBCT, and PET/CT). Standardized preprocessing is crucial for consistent radiomic feature extraction. Key preprocessing steps include voxel resampling, normalization, and discretization, which influence feature robustness and reproducibility. In total, 44% of the included works studied the effects of an isotropic voxel resampling, and most studies opted to employ a discretization strategy. From 2021, several studies started selecting the best set of preprocessing parameters based on models’ best performance. As for comparison metrics, ICC was the most used in MRI studies in 58% of the screened works. Conclusions: From our work, we highlighted the need to harmonize the use of preprocessing parameters and their values, especially in light of future studies of prospective studies, which are still lacking in the current literature.

1. Introduction

In recent years, there has been a notable increase in research examining the potential of quantitative imaging features to reveal additional information from medical images beyond what is perceptible to the human eye [1,2,3]. Handcrafted radiomics systematically extracts quantitative imaging features from medical images to interpret biological information [4,5]. The term “radiomics” yields over 11,100 studies on PubMed. This approach takes manually delineated regions of interest, such as tumors, on medical images and extracts a high number of quantitative features with pre-determined mathematical formulas [6]. One of the biggest advantages of radiomics is that those values can be extracted from medical images which can be normally acquired from the clinical pathway of the patient. In radiomics studies, the hypothesis is that radiomic features (RFs) can act as clinical biomarkers, individually or in combination, even with clinical parameters [4].
Numerous investigations have explored and documented RFs’ ability to predict clinical outcomes, including both a prognostic aim (overall survival and treatment response) as well as diagnostic aim (e.g., differential diagnosis) [7,8,9,10,11,12,13,14,15,16,17,18].
As the field of radiomics has advanced, there has been increasing attention on its limitations [5,19]. The primary identified limitation is the sensitivity of RFs to variations in image acquisition and reconstruction parameters [20,21,22,23,24,25]. To be effective clinical biomarkers, RFs must demonstrate reproducibility across different imaging parameters for generalization [25]. However, numerous studies have documented differences in imaging acquisition and reconstruction parameters [25,26,27,28,29,30,31,32,33,34,35,36,37]. Moreover, the impact of variation in a single acquisition or reconstruction parameter on RF reproducibility depends on the individual feature [31,38]. Two works also highlight that reconstruction kernels notably affect RF reproducibility [21,37]. There are several works exploring various methods to address RF reproducibility across differently acquired scans, as well as how to standardize them.
Image preprocessing occurs between the image segmentation and feature extraction stages. There are several studies in the literature indicating that the repeatability of extracted radiomic features is significantly influenced by the image preprocessing settings employed [39,40,41,42,43,44]. Usually, the acquisition and reconstructed voxels are not isotropic, especially if no voxel re-segmentation is applied (e.g., CT scans are usually not isotropic at acquisition).
Normalization or range re-segmentation is used to remove voxels in the segmented volume of interest (VOI) that fall outside a specific gray-level range and is commonly used to ensure data consistency, especially for imaging modalities with calibrated units. In fact, normalization is typically necessary for CT and PET. However, this method does not apply to MRI data, which employ arbitrary intensity units instead. In this case, intensity outlier filtering is applied by calculating the mean (µ) and the standard deviation (σ) of gray levels within the VOI and excluding gray levels outside the range of µ ± n σ, where 3 is usually selected as the value of n [44,45,46,47]. Other relative normalizations have been proposed to determine MRI radiomic feature robustness [48,49]. While the results in the radiomic field are shown to be promising, these studies still have many pitfalls and limitations. One of the main issues in radiomic studies is that features depend on how the images are preprocessed before their computation. Preprocessing medical images with different parameters gives different feature values associated with the same image set, thus meaning that the associated results in terms of predictive model performances are also influenced. This work focused on gathering information about voxel resampling, normalization, and discretization preprocessing parameters from our included studies. We also investigated the study aim, anatomic district, and comparison metrics to better understand the up-to-date overview of radiomic studies. This review aims to provide an up-to-date and comprehensive overview of the most commonly used preprocessing techniques.

2. Materials and Methods

This systematic review followed the recommendations of the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols) 2020 [50]. The inclusion criteria for the records in the review process were the presence of specific preprocessing parameters evaluated in CBCT, CT, MRI, and PET/CT radiomic studies.
We performed a comprehensive literature search using four electronic databases: PubMed, Cochrane Library, Embase, and Scopus. Since radiomic studies are relatively recent, we conducted the literature search from January 2008 to December 2023. Mesh terms and free text were modeled in search strategies for databases using the one designed for PubMed: ((((“Positron-Emission Tomography” [Mesh]) OR “Tomography, X-ray Computed” [Mesh]) OR “Magnetic Resonance Imaging” [Mesh] OR PET OR Positron emission tomography OR computed tomography OR CT OR MRI OR magnetic resonance) AND (radiomics)) AND (preprocessing OR pre-processing). The inclusion criteria were studies with CT, PET/CT, or MRI as imaging modalities, where preprocessing parameters’ influence on feature values and model predictions was addressed. The study was exclusively confined to English-language research without publication status restrictions. The authors did not contact other institutions or researchers to uncover additional studies.
Two reviewers were involved in the study: a senior reviewer with 8 years of experience in the radiomic field of literature and a junior reviewer with 4 years of experience. These two reviewers independently screened records by reviewing their titles and abstracts and extracting general study characteristics (such as study title, first author’s name, publication year, journal, abstract, corresponding reviewer, and research keywords) for records meeting the inclusion criteria, using a customized data extraction form. In this first step, the authors independently used the ASReview tool (version 1.5) [51] to speed up the article selection. The final screening was manual; we used ASReview as a support tool. ASReview assigned a priority to the screened literature works, based on a subset of 4 relevant articles and 4 non-relevant articles (which were chosen accordingly by the two reviewers). After the tool scored the studies, we checked them in descending assigned priority order. Beyond a threshold priority, we confirmed that the works below that level were not relevant for the topic anymore and thus were excluded. The authors compared their decisions, resolving disagreements until a consensus was reached. Records lacking information on image acquisition parameters were excluded, and any eligible studies with full-text versions were included in the review process, while conference proceedings and monographs were disregarded. Additional articles not meeting the inclusion criteria upon full-text reading were identified and excluded at this stage. Reviewers independently selected full-text articles, resolving any uncertainties by reaching a consensus. Out of all the 53 full-text articles screened for inclusion, discussion to reach a consensus was required for 17 works (32%).
From each selected full-text article, the following parameters were identified and extracted:
  • Acquisition modality (CBCT, CT, MRI, PET/CT);
  • Number of patients or phantoms;
  • Name of disease/s (if appropriate);
  • Equipment vendor and model;
  • Presence of acquisition parameters;
  • Total number of features;
  • Type of features subsampled in FO (first order), SM (shape metric), and TA textural features;
  • Type of software used in the radiomic feature extraction;
  • Image filtering used (Y/N; if Y, the type was reported);
  • Voxel resampling;
  • Normalization process;
  • Discretization technique;
  • Retrospective study (Y/N or NA);
  • Statistical analysis: intraclass correlation coefficient (ICC), concordance correlation coefficient (CCC), area under the receiver operation curve (AUC), mean, average percentage difference, relative difference, Spearman correlation, Kolmogorov–Smirnov test, double-sample test, and two-way ANOVA;
  • Type of study (reproducibility/repeatability/both or best performance);
  • Main findings.
No contact with the authors of the records for complementary information was necessary. We synthesized our data in a table divided by the imaging modality subgroups.
Quality scoring for study selection was not utilized. The reviewers reviewed articles in duplicate, resolving disagreements by achieving an agreement. Data from the studies were standardized to mitigate potential biases. Data extracted from studies that did not meet the criteria for standardization were excluded.
This review aimed to describe the most used preprocessing pipeline in the radiomic workflow of the CBCT, CT, MRI, and PET/CT studies schematically shown in Figure 1. It also sought to find possible standard radiomic preprocessing setups and describe a potential evolution over time, aiming to uncover any increased awareness of the technique’s standardization.
Concerning the normalization strategies, we counted the number of papers that chose absolute, relative, or combination strategies. When this information was not applied, we set “None”. A normalization strategy enhances the robustness and reliability of the radiomic analysis. The absolute normalization strategy involves standardizing the intensity values of imaging by transforming them to a common scale based on absolute reference points. Conversely, relative normalization concerns adjusting the intensity values of images relative to a certain reference within each individual image or dataset; this technique often includes scaling the pixel values based on the mean (µ), median, or a specific percentile of the intensity values in the image. This approach accounts for differences in overall intensity levels between images by normalizing the intensities relative to their own distribution.
In terms of discretization strategies, we identified the number of studies that utilized a bin number (BN), a bin width (BW), or both the bin number and width (BN + BW) approach. When this information was not applied or not specified, we set “None”. The discretization strategy involves adjusting values to standardize the feature extraction process across different images or imaging modalities. The BN discretization ensures that the radiomic features are comparable and consistent by transforming the intensity values into a fixed number of discrete bins. The BW method assures that each bin represents a fixed range of intensity values. The BN + BW strategy is a dual approach that guarantees the simultaneous adjustment of the number of bins and the width of each bin used to discretize image intensity values.
These concepts were summarized in Figure 2.
We used the QUADAS-2 tool to investigate the risk of bias [52]. The tool evaluates four key domains: patient selection, index test, reference standard, and flow and timing. Each domain is evaluated for the risk of bias and applicability. We used this tool only for the first two domains because reference standards and flow and timing are not defined in this review’s field. Regarding the patient selection’s risk of bias, we checked the method used (lower marks were given in the case of inappropriate exclusions or not consecutive/random selection), while the applicability was evaluated if the included patients matched the typical clinical scenario.
We considered the index test the metric used in the result comparison. In this context, the risk of bias consisted of consistently evaluating the calculation/use of the aforementioned metric, and the applicability referred to the methodology of its use in practice.

3. Results

3.1. Literature Search

The computer-assisted search across PubMed, Cochrane Library, Embase, and Scopus yielded 546 records (as shown in Table 1). Following the elimination of duplicate entries, 459 unique records remained. Subsequently, a screening process based on the coherence of the title and abstract reduced the number of articles to 72 (refer to Figure 3).
Reviews and conference proceedings were excluded from further consideration. Consequently, 53 full-text articles underwent evaluation by the reviewers to determine their eligibility. Ten articles were excluded from the database for the following reasons: six lacked preprocessing results, one focused on a modality outside the scope of this review, two utilized only a digital phantom, and one was unfound. A comprehensive table with all the included entries can be found in the Supplementary Materials.

3.2. Data Collection and Elaboration

The selected papers were grouped by considering the acquisition modality: one about CBCT, twelve about CT, twenty-six about MRI, and five about PET/CT.
CT studies were the most represented from 2013 to 2023. MRI works showed an increase in publications over the years, starting from one in 2017 to nine in 2023. PET/CT studies were published in 2019 (two) and 2021 (three).
In 2017, CT phantom studies were published to study the stability and reproducibility of radiomic features using phantoms made of different materials compared to those traditionally used in quality control (QC) [39,53,54].
The number of patients has increased, although not monotonically, from 256 in 2013 to 3000 in 2023. The modality that saw the greatest increase was MRI, demonstrating the growing interest of the scientific community in this type of imaging.

3.2.1. Acquisition Parameter Presence and Voxel Resampling

  • Table 2 shows the studies reporting the acquisition parameters and the voxel resampling information. Only in PET and CBCT did we find that most papers used isotropic voxel resampling, while in CT and MRI, only 41.7% and 41.3% used it, respectively.
  • For CT and CBCT, the most studied voxel resampling interpolation was 1 × 1 × 1 mm3. Furthermore, a significant number of studies used 1 × 1 × 1 mm3 in MRI, but more voxel sizes were investigated in the range of 0.9–4.8 mm3. This result is expected because in MRI the image characteristics are strongly influenced by the acquisition protocols.
  • For PET/CT, voxel resampling dimensions were in the range of 1–4 mm3. This imaging modality, which employs a small resampling dimension, might introduce biases due to its intrinsic resolution (around non-isotropic 3–5 mm3).

3.2.2. Normalization Strategies

Table 3 reports the normalization strategies adopted by the authors, considering the different modalities. In CT and PET/CT, most works did not use a normalization strategy, while in MRI, relative normalization was the most used, as expected. In CBCT, a relative strategy was used because the image gray levels are not in Hounsfield but relative units.

3.2.3. Discretization Strategies

Most studies opted for a discretization strategy: 75% in CT, 100% in CBCT, 92.3% in MRI, and 100% in PET/CT. The combination of BN + BW was the most represented in this latter modality (Table 4).
In CT, the most investigated BNs were 32, 64, and 128, while BW ranged between 5 and 50 Hounsfield units (HU). In CBCT, the only work selected used BNs equal to 64, 128, and 256.
In MRI, the BN = 32 was the most used bin number, and eleven works studied it (42.3%), followed by BN = 64 and 128 (38.5% of the papers).
In PET/CT, the most studied BNs were 32 and 64, while BW typically ranged from 0.01 to 0.5 SUV.

3.2.4. Study Aims

Table 5 shows the count of study aims for each imaging modality.
Most works employing CT and MRI aimed to study reproducibility and its association with repeatability. It is worth noting that since 2021, best performance has also been well investigated. Moreover, the CBCT work studied the aim of repeatability and reproducibility. PET/CT works consisted of three works about repeatability published in 2019 and 2021 and two works about best performance issued in 2021. Best performance means that the authors studied the influence of the preprocessing parameters based on the figure of merit of their chosen model, i.e., highlighting which preprocessing parameters yielded the best results in terms of performance of the model (AUC, accuracy, etc.); in fact, the highlighted studies in Section 3.2.5 use AUC as a comparison metric. The works under “best performance” aim to identify the set of preprocessing parameters that give the best performance with the predictive model they used.

3.2.5. Anatomic District

Table 6 reports the anatomic districts categorized in the main four body regions. The CT studies focused on the trunk’s anatomical regions (chest, abdomen, and pelvis), while MRI studies were polarized between the brain and pelvis. In addition, CBCT and PET/CT investigations were centered on the pelvis or chest.

3.2.6. Comparison Metrics

Table 7 illustrates the most common metrics used in the result comparison. In absolute terms, the most used metric was ICC, which is used predominantly in MRI, with 15 out of 26 works. Some works used more than one metric, so the number of papers reported in Table 7 is more than the number of works included.

3.3. Risk-of-Bias Analysis

The risk-of-bias assessment revealed that about 35% of the studies had a low risk of bias across the patient selection domain, while the remainder did not specify or were not clearly specified. In addition, 60% had a low risk of bias across the index test. The patient selection and index test applicability was higher than 72% (Figure 4).

4. Discussion

The volume of published studies on predictive modeling using radiomic features has been proliferating. However, no global consensus exists on which features are consistently repeatable and reproducible. This lack of agreement could potentially hinder future discussions on clinical applicability and the feasibility of prospective multi-institutional external validation trials.
Our work is a comprehensive image of the current literature, which highlights that there are indeed a range of preprocessing values that can be safely employed in radiomic studies without compromising the reliability of the obtained results.
We are aware that there are several papers published on US radiomics, since it is becoming a widely used imaging modality in medicine to investigate several diseases. Lately, the literature on US-based radiomics has been increasing [94,95,96]. In our study, we wanted to investigate 3D imaging, which is most used in oncologic and neurologic treatment, so we left US imaging out of our literature search.
This review’s main objective was to identify preprocessing strategies for radiomic features in the four primary 3D imaging modalities: CBCT, CT, MRI, and PET. It also aimed to identify common preprocessing parameter settings to evaluate the current state of the art in this field and raise awareness, thereby promoting more stable and reproducible results. In fact, the preprocessing technique has the greatest impact on feature reproducibility [58].
In 2017, CT phantom studies were published to study the stability and reproducibility of radiomic features using phantoms made of materials different from those traditionally used in QC [39,53,54]. Moreover, Palani et al. [63] published a phantom study in 2023, highlighting the scientific interest in this topic.
This review showed how the choice of 1 × 1 × 1 mm3 could be a feasible starting point for CT and MRI examinations if the original voxel size parameters are not too different from these values. On the other hand, for CT, the dimension related to slice thickness (the third dimension) is notably larger compared to the other two, which in turn depend only on the FOV and reconstruction matrix size (typically 512 × 512). Also, for this reason, Larue et al. [53] and Shafiq-ul Hassan [39] studied 1 × 1 × 3 and 1 × 1 × 2 mm3, respectively. In MRI, however, the discussion is broader. From a radiomic analysis perspective, a volumetric acquisition might be the best choice to introduce minimal bias in voxel resampling. However, this choice is not necessarily optimal for all anatomical regions/lesions, and in any case, it does not apply to retrospective studies. For PET/CT, it seems clear that an isotropic voxel of approximately 3 × 3 × 3 or 4 × 4 × 4 mm3 is a widely accepted choice and a good compromise for obtaining comparable results. The challenges in isotropic reconstruction are similar to those previously described for CT, even though they relate to different physical processes.
A limitation of this review is that it did not delve into the issue of the resampling function used (e.g., linear, bicubic, nearest-neighbor interpolation, etc.) because the studies were too varied to obtain a meaningful result. Larue et al. [53] demonstrated that linear interpolation resulted in the narrowest range of feature values for nearly half of the features, and cubic interpolation did so for a smaller portion. In contrast, nearest-neighbor interpolation produced the broadest range for most CT study features.
Other limitations of this study lay in the impossibility extracting from the screened literature works a definite set of recommended values to use for the preprocessing step of radiomic works. Furthermore, the inclusion of a third reviewer to resolve the disputes and the lack of consensus would have been beneficial for our work.
Different techniques have been considered for normalization strategies. Starting from CT, Larue et al. [53] studied the influence of BW from 5 to 50 HU at 5 HU intervals. Fave et al. [61] preprocessed the CT images with 8-bit depth resampling using a 16 HU bin width. Kolossváry et al. [60] compared two equally sized bins and equally probable bins where each bin contains a proportion of the data, finding that all GLRLM features were significantly affected by binning type; in addition, BNs significantly affected the values for all GLCMs and GLRLMs. Gray-level resampling affects only second- and higher-order radiomics features, while voxel size variation can influence first-, second-, and higher-order features [39]. The optimal discretization methods for feature inter- and intra-sample reproducibility depend on the imaging modality. The choice of discretization significantly affects intensity distributions, feature values, and reproducibility [45]. Defining only bin number or width is insufficient, as these two quantities are related to the maximum number of gray levels used. The IBSI group mentions preprocessing recommendations in their work [45]. In modalities that use calibrated imaging intensity units (e.g., CT and PET/CT), when a re-segmentation range is defined, either fixed bin size or fixed bin width can be defined. If no re-segmentation range is used in calibrated imaging modalities, then fixed bin number is recommended. When a non-calibrated imaging modality (e.g., MRI) is employed, they recommend using the fixed bin number method without intensity re-segmentation. The issue when using BN is that it introduces a normalization effect, directly comparing feature values across patients, whereas BW preserves the relationship between PET units and their physical meaning. Xu et al. [91] indicate that the BN discretization scheme achieves relatively high stability compared to BW, in line with IBSI recommendations [45], contrasting with Pfaehler et al. [28].
Most CT and MRI studies have focused on reproducibility and its relationship with repeatability. Notably, since 2021, there has been significant investigation into achieving optimal performance. Additionally, CBCT studies have examined both repeatability and reproducibility. PET/CT research includes three studies on repeatability published in 2019 and 2021, and two studies on optimal performance published in 2021. The selection of the best performance as the criterion for evaluating the quality of a model (including the choice of preprocessing parameters) should be limited, as this choice could introduce biases.
About the anatomical part investigated, CT studies focused on the torso’s anatomical regions (chest, abdomen, and pelvis), while MRI studies examined the brain and pelvis regions. Additionally, CBCT and PET/CT investigations were centered on the pelvis or chest. The distribution of the anatomical regions per imaging modality aligns with what is expected in a therapeutic pathway (e.g., using MRI for further investigation of the brain or prostate/cervix due to its capability to distinguish between soft tissues compared to CT).
Concerning comparison metrics, the most commonly used metric was ICC, predominantly applied in MRI studies, with 15 out of 26 works [65,66,67,68,71,73,74,79,82,83,84,85,86,88,89]. ICC is extensively studied in radiomics research due to its role in quantitative assessment, reliability, wide applicability, and standard practice. It serves as a quantitative measure to gauge agreement or consistency between measurements or observations, crucial for evaluating the reproducibility of radiomic features. ICC evaluates measurement reliability by accounting for systematic and random variations, making it effective for assessing feature stability across different scans or observers. Its versatility extends to various data types, including continuous and categorical variables, enabling the evaluation of different radiomic features across diverse imaging modalities and patient groups. Overall, ICC is favored in radiomics for its comprehensive and standardized approach to assessing feature reproducibility and stability, thereby supporting the development and validation of radiomics as a quantitative imaging biomarker. CCC is another essential statistical metric used in radiomics research to evaluate measurement agreement and reliability. It quantifies the agreement between two sets of continuous measurements, assessing precision (how closely measurements cluster around the line of perfect agreement) and accuracy (how close measurements are to the line of identity), offering a comprehensive measure of agreement that is easy to interpret. Like ICC, CCC accounts for systematic bias (shift) and random variation (spread) between measurements. This makes it suitable for evaluating the reproducibility and stability of radiomic features across different scans or observers. AUC quantifies the overall discriminatory ability of a diagnostic test or biomarker. In radiomics, this metric assesses the ability of radiomic features or models to distinguish between different clinical conditions or outcomes based on imaging data. Higher AUC values indicate better discriminatory performance of the radiomic feature or model.
From our investigated studies, it was not possible to identify a recommended set of preprocessing parameters or a subset of stable features due to the variability in the experimental setups. However, due to their definitions, higher-order class features (e.g., GLRM and GLCM-based) are more affected by preprocessing parameters as well as acquisition and reconstruction parameters, so they should be used with caution when dealing with retrospective multicentric studies. We can conclude that, when building a predictive model, first-order class features should be preferred for inclusion should they show a correlation with the chosen outcome, especially if a preprocessing strategy is not employed.
Regarding the risk of bias analysis, it was not possible to assess it for the majority of the studies for patient selection. Since most of the radiomic studies are retrospective, it is not possible to understand how the patient selection process might affect the results. Many works showed low concern for risk of bias in the index test, due to the widely accepted choice of well-known statistical indexes, already broadly used in the literature to test repeatability even in other scientific fields. Regarding applicability, most studies raised low concern for both patient selection and the index test.
From our analysis, we also found out that a significant number of our included studies pursued best model performance in order to find the optimal preprocessing parameters [55,56,59,70,72,75,76,77,80,82,90,91]. While this is still a feasible approach given the early stage of the literature and research in this field, it would be advised to strive toward a standardization that is based on the feature stability rather than individual model performances as these are influenced by other variables (such as training/validation datasets).
The problem of generalizability remains open, but using a preprocessing strategy improves the robustness of the analysis, especially when multiple vendors and machine models are included.
We think that an effort toward the standardization of preprocessing parameters should be conducted by the scientific community. For example, the IBSI group has taken a step in that direction. From this work, we can see that there are several sets of preprocessing values that are employable; with such initiatives, a consensus on this topic can be reached.

5. Conclusions

To date, it does not seem possible to establish a standardized recipe for the unequivocal selection of preprocessing parameter combinations for radiomic analysis. However, it is noteworthy that certain combinations of voxel resampling, normalization, and digitalization are more commonly used than others.
For reproducibility, a study must report the sequence of preprocessing parameters used so that the methodology employed is evident and transparent.
Our work is a comprehensive image of the current literature which highlights that there are indeed a range of preprocessing values that can be safely employed in radiomic studies without compromising the reliability of the obtained results.
From our work, we highlighted the need to harmonize the use of preprocessing parameters and their values, especially considering future studies of prospective studies, which are still lacking in the current literature.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cancers16152668/s1, Table S1: Summary table of results.

Author Contributions

Conceptualization, M.B. and V.T.; methodology, M.B. and V.T.; formal analysis, M.B. and V.T.; investigation, M.C.B.; resources, M.C.B.; data curation, M.B., V.T. and L.V.; writing—original draft preparation, V.T.; writing—review and editing, M.B., V.T. and L.V.; visualization, M.B.; supervision, M.B.; project administration, V.T. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding. This study was partially supported by Italian Ministry of Health—Ricerca Corrente Annual Program 2025.

Data Availability Statement

The data were obtained from a computer-assisted search of PubMed, Cochrane Library, Embase, and Scopus.

Acknowledgments

We thank Giovanni Costi for helping us in the analysis.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Gillies, R.J.; Kinahan, P.E.; Hricak, H. Radiomics: Images Are More than Pictures, They Are Data. Radiology 2016, 278, 563–577. [Google Scholar] [CrossRef]
  2. Walsh, S.; De Jong, E.E.C.; Van Timmeren, J.E.; Ibrahim, A.; Compter, I.; Peerlings, J.; Sanduleanu, S.; Refaee, T.; Keek, S.; Larue, R.T.H.M.; et al. Decision Support Systems in Oncology. JCO Clin. Cancer Inform. 2019, 3, 1–9. [Google Scholar] [CrossRef]
  3. Shur, J.; Blackledge, M.; D’Arcy, J.; Collins, D.J.; Bali, M.; O’Leach, M.; Koh, D.-M. MRI Texture Feature Repeatability and Image Acquisition Factor Robustness, a Phantom Study and in Silico Study. Eur. Radiol. Exp. 2021, 5, 2. [Google Scholar] [CrossRef]
  4. Lambin, P.; Rios-Velazquez, E.; Leijenaar, R.; Carvalho, S.; Van Stiphout, R.G.P.M.; Granton, P.; Zegers, C.M.L.; Gillies, R.; Boellard, R.; Dekker, A.; et al. Radiomics: Extracting More Information from Medical Images Using Advanced Feature Analysis. Eur. J. Cancer 2012, 48, 441–446. [Google Scholar] [CrossRef]
  5. Yip, S.S.F.; Aerts, H.J.W.L. Applications and Limitations of Radiomics. Phys. Med. Biol. 2016, 61, R150–R166. [Google Scholar] [CrossRef]
  6. Hosny, A.; Aerts, H.J.; Mak, R.H. Handcrafted versus Deep Learning Radiomics for Prediction of Cancer Therapy Response. Lancet Digit. Health 2019, 1, e106–e107. [Google Scholar] [CrossRef]
  7. Aerts, H.J.W.L.; Velazquez, E.R.; Leijenaar, R.T.H.; Parmar, C.; Grossmann, P.; Carvalho, S.; Bussink, J.; Monshouwer, R.; Haibe-Kains, B.; Rietveld, D.; et al. Decoding Tumour Phenotype by Noninvasive Imaging Using a Quantitative Radiomics Approach. Nat. Commun. 2014, 5, 4006. [Google Scholar] [CrossRef]
  8. Bae, S.; Choi, Y.S.; Ahn, S.S.; Chang, J.H.; Kang, S.-G.; Kim, E.H.; Kim, S.H.; Lee, S.-K. Radiomic MRI Phenotyping of Glioblastoma: Improving Survival Prediction. Radiology 2018, 289, 797–806. [Google Scholar] [CrossRef]
  9. Oikonomou, A.; Khalvati, F.; Tyrrell, P.N.; Haider, M.A.; Tarique, U.; Jimenez-Juan, L.; Tjong, M.C.; Poon, I.; Eilaghi, A.; Ehrlich, L.; et al. Radiomics Analysis at PET/CT Contributes to Prognosis of Recurrence and Survival in Lung Cancer Treated with Stereotactic Body Radiotherapy. Sci. Rep. 2018, 8, 4003. [Google Scholar] [CrossRef]
  10. Wu, W.; Parmar, C.; Grossmann, P.; Quackenbush, J.; Lambin, P.; Bussink, J.; Mak, R.; Aerts, H.J.W.L. Exploratory Study to Identify Radiomics Classifiers for Lung Cancer Histology. Front. Oncol. 2016, 6, 71. [Google Scholar] [CrossRef]
  11. Blüthgen, C.; Patella, M.; Euler, A.; Baessler, B.; Martini, K.; Von Spiczak, J.; Schneiter, D.; Opitz, I.; Frauenfelder, T. Computed Tomography Radiomics for the Prediction of Thymic Epithelial Tumor Histology, TNM Stage and Myasthenia Gravis. PLoS ONE 2021, 16, e0261401. [Google Scholar] [CrossRef]
  12. E, L.; Lu, L.; Li, L.; Yang, H.; Schwartz, L.H.; Zhao, B. Radiomics for Classification of Lung Cancer Histological Subtypes Based on Nonenhanced Computed Tomography. Acad. Radiol. 2019, 26, 1245–1252. [Google Scholar] [CrossRef] [PubMed]
  13. Stefan, P.-A.; Puscas, M.E.; Csuak, C.; Lebovici, A.; Petresc, B.; Lupean, R.; Mihu, C.M. The Utility of Texture-Based Classification of Different Types of Ascites on Magnetic Resonance. J. BUON 2020, 25, 1237–1244. [Google Scholar]
  14. Csutak, C.; Ștefan, P.-A.; Lupean, R.-A.; Lenghel, L.M.; Mihu, C.M.; Lebovici, A. Computed Tomography in the Diagnosis of Intraperitoneal Effusions: The Role of Texture Analysis. Bosn. J. Basic Med. Sci. 2021, 21, 488–494. [Google Scholar] [CrossRef]
  15. Horvat, N.; Veeraraghavan, H.; Khan, M.; Blazic, I.; Zheng, J.; Capanu, M.; Sala, E.; Garcia-Aguilar, J.; Gollub, M.J.; Petkovska, I. MR Imaging of Rectal Cancer: Radiomics Analysis to Assess Treatment Response after Neoadjuvant Therapy. Radiology 2018, 287, 833–843. [Google Scholar] [CrossRef]
  16. Tharmalingam, H.; Tsang, Y.M.; Alonzi, R.; Beasley, W.; Taylor, N.J.; McWilliam, A.; Padhani, A.; Choudhury, A.; Hoskin, P.J. Changes in Magnetic Resonance Imaging Radiomic Features in Response to Androgen Deprivation Therapy in Patients with Intermediate- and High-Risk Prostate Cancer. Clin. Oncol. (R. Coll. Radiol.) 2022, 34, e246–e253. [Google Scholar] [CrossRef]
  17. Ștefan, P.-A.; Lupean, R.-A.; Mihu, C.M.; Lebovici, A.; Oancea, M.D.; Hîțu, L.; Duma, D.; Csutak, C. Ultrasonography in the Diagnosis of Adnexal Lesions: The Role of Texture Analysis. Diagnostics 2021, 11, 812. [Google Scholar] [CrossRef]
  18. Ștefan, R.-A.; Ștefan, P.-A.; Mihu, C.M.; Csutak, C.; Melincovici, C.S.; Crivii, C.B.; Maluțan, A.M.; Hîțu, L.; Lebovici, A. Ultrasonography in the Differentiation of Endometriomas from Hemorrhagic Ovarian Cysts: The Role of Texture Analysis. JPM 2021, 11, 611. [Google Scholar] [CrossRef] [PubMed]
  19. Lohmann, P.; Bousabarah, K.; Hoevels, M.; Treuer, H. Radiomics in Radiation Oncology-Basics, Methods, and Limitations. Strahlenther. Onkol. 2020, 196, 848–855. [Google Scholar] [CrossRef] [PubMed]
  20. Ibrahim, A.; Primakov, S.; Beuque, M.; Woodruff, H.C.; Halilaj, I.; Wu, G.; Refaee, T.; Granzier, R.; Widaatalla, Y.; Hustinx, R.; et al. Radiomics for Precision Medicine: Current Challenges, Future Prospects, and the Proposal of a New Framework. Methods 2021, 188, 20–29. [Google Scholar] [CrossRef]
  21. Mali, S.A.; Ibrahim, A.; Woodruff, H.C.; Andrearczyk, V.; Müller, H.; Primakov, S.; Salahuddin, Z.; Chatterjee, A.; Lambin, P. Making Radiomics More Reproducible across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods. JPM 2021, 11, 842. [Google Scholar] [CrossRef]
  22. Midya, A.; Chakraborty, J.; Gönen, M.; Do, R.K.G.; Simpson, A.L. Influence of CT Acquisition and Reconstruction Parameters on Radiomic Feature Reproducibility. J. Med. Imaging 2018, 5, 011020. [Google Scholar] [CrossRef]
  23. Reiazi, R.; Abbas, E.; Famiyeh, P.; Rezaie, A.; Kwan, J.Y.Y.; Patel, T.; Bratman, S.V.; Tadic, T.; Liu, F.-F.; Haibe-Kains, B. The Impact of the Variation of Imaging Parameters on the Robustness of Computed Tomography Radiomic Features: A Review. Comput. Biol. Med. 2021, 133, 104400. [Google Scholar] [CrossRef]
  24. Espinasse, M.; Pitre-Champagnat, S.; Charmettant, B.; Bidault, F.; Volk, A.; Balleyguier, C.; Lassau, N.; Caramella, C. CT Texture Analysis Challenges: Influence of Acquisition and Reconstruction Parameters: A Comprehensive Review. Diagnostics 2020, 10, 258. [Google Scholar] [CrossRef]
  25. Zhao, B. Understanding Sources of Variation to Improve the Reproducibility of Radiomics. Front. Oncol. 2021, 11, 633176. [Google Scholar] [CrossRef]
  26. Shiri, I.; Abdollahi, H.; Shaysteh, S.; Rabi Mahdavi, S. Test-Retest Reproducibility and Robustness Analysis of Recurrent Glioblastoma MRI Radiomics Texture Features. Iran. J. Radiol. 2017, 5, e48035. [Google Scholar] [CrossRef]
  27. Peerlings, J.; Woodruff, H.C.; Winfield, J.M.; Ibrahim, A.; Van Beers, B.E.; Heerschap, A.; Jackson, A.; Wildberger, J.E.; Mottaghy, F.M.; DeSouza, N.M.; et al. Stability of Radiomics Features in Apparent Diffusion Coefficient Maps from a Multi-Centre Test-Retest Trial. Sci. Rep. 2019, 9, 4800. [Google Scholar] [CrossRef]
  28. Pfaehler, E.; Beukinga, R.J.; Jong, J.R.; Slart, R.H.J.A.; Slump, C.H.; Dierckx, R.A.J.O.; Boellaard, R. Repeatability of 18F-FDG PET Radiomic Features: A Phantom Study to Explore Sensitivity to Image Reconstruction Settings, Noise, and Delineation Method. Med. Phys. 2019, 46, 665–678. [Google Scholar] [CrossRef]
  29. Prayer, F.; Hofmanninger, J.; Weber, M.; Kifjak, D.; Willenpart, A.; Pan, J.; Röhrich, S.; Langs, G.; Prosch, H. Variability of Computed Tomography Radiomics Features of Fibrosing Interstitial Lung Disease: A Test-Retest Study. Methods 2021, 188, 98–104. [Google Scholar] [CrossRef] [PubMed]
  30. Zhao, B.; Tan, Y.; Tsai, W.-Y.; Qi, J.; Xie, C.; Lu, L.; Schwartz, L.H. Reproducibility of Radiomics for Deciphering Tumor Phenotype with Imaging. Sci. Rep. 2016, 6, 23428. [Google Scholar] [CrossRef] [PubMed]
  31. Ibrahim, A.; Refaee, T.; Primakov, S.; Barufaldi, B.; Acciavatti, R.J.; Granzier, R.W.Y.; Hustinx, R.; Mottaghy, F.M.; Woodruff, H.C.; Wildberger, J.E.; et al. The Effects of In-Plane Spatial Resolution on CT-Based Radiomic Features’ Stability with and without ComBat Harmonization. Cancers 2021, 13, 1848. [Google Scholar] [CrossRef] [PubMed]
  32. Zhovannik, I.; Bussink, J.; Traverso, A.; Shi, Z.; Kalendralis, P.; Wee, L.; Dekker, A.; Fijten, R.; Monshouwer, R. Learning from Scanners: Bias Reduction and Feature Correction in Radiomics. Clin. Transl. Radiat. Oncol. 2019, 19, 33–38. [Google Scholar] [CrossRef]
  33. Ibrahim, A.; Refaee, T.; Primakov, S.; Barufaldi, B.; Acciavatti, R.J.; Granzier, R.W.Y.; Hustinx, R.; Mottaghy, F.M.; Woodruff, H.C.; Wildberger, J.E.; et al. Reply to Orlhac, F.; Buvat, I. Comment on “Ibrahim et al. The Effects of In-Plane Spatial Resolution on CT-Based Radiomic Features’ Stability with and without ComBat Harmonization. Cancers 2021, 13, 1848”. Cancers 2021, 13, 3080. [Google Scholar] [CrossRef]
  34. Ibrahim, A.; Lu, L.; Yang, H.; Akin, O.; Schwartz, L.H.; Zhao, B. The Impact of Image Acquisition Parameters and ComBat Harmonization on the Predictive Performance of Radiomics: A Renal Cell Carcinoma Model. Appl. Sci. 2022, 12, 9824. [Google Scholar] [CrossRef] [PubMed]
  35. Park, J.E.; Park, S.Y.; Kim, H.J.; Kim, H.S. Reproducibility and Generalizability in Radiomics Modeling: Possible Strategies in Radiologic and Statistical Perspectives. Korean J. Radiol. 2019, 20, 1124–1137. [Google Scholar] [CrossRef]
  36. Meyer, M.; Ronald, J.; Vernuccio, F.; Nelson, R.C.; Ramirez-Giraldo, J.C.; Solomon, J.; Patel, B.N.; Samei, E.; Marin, D. Reproducibility of CT Radiomic Features within the Same Patient: Influence of Radiation Dose and CT Reconstruction Settings. Radiology 2019, 293, 583–591. [Google Scholar] [CrossRef]
  37. Lu, L.; Ehmke, R.C.; Schwartz, L.H.; Zhao, B. Assessing Agreement between Radiomic Features Computed for Multiple CT Imaging Settings. PLoS ONE 2016, 11, e0166550. [Google Scholar] [CrossRef]
  38. Ibrahim, A.; Refaee, T.; Leijenaar, R.T.H.; Primakov, S.; Hustinx, R.; Mottaghy, F.M.; Woodruff, H.C.; Maidment, A.D.A.; Lambin, P. The Application of a Workflow Integrating the Variable Reproducibility and Harmonizability of Radiomic Features on a Phantom Dataset. PLoS ONE 2021, 16, e0251147. [Google Scholar] [CrossRef]
  39. Shafiq-ul-Hassan, M.; Zhang, G.G.; Latifi, K.; Ullah, G.; Hunt, D.C.; Balagurunathan, Y.; Abdalah, M.A.; Schabath, M.B.; Goldgof, D.G.; Mackin, D.; et al. Intrinsic Dependencies of CT Radiomic Features on Voxel Size and Number of Gray Levels. Med. Phys. 2017, 44, 1050–1062. [Google Scholar] [CrossRef]
  40. Wichtmann, B.D.; Harder, F.N.; Weiss, K.; Schönberg, S.O.; Attenberger, U.I.; Alkadhi, H.; Pinto Dos Santos, D.; Baeßler, B. Influence of Image Processing on Radiomic Features From Magnetic Resonance Imaging. Investig. Radiol. 2023, 58, 199–208. [Google Scholar] [CrossRef] [PubMed]
  41. Altazi, B.A.; Zhang, G.G.; Fernandez, D.C.; Montejo, M.E.; Hunt, D.; Werner, J.; Biagioli, M.C.; Moros, E.G. Reproducibility of F18-FDG PET Radiomic Features for Different Cervical Tumor Segmentation Methods, Gray-level Discretization, and Reconstruction Algorithms. J. Appl. Clin. Med. Phys. 2017, 18, 32–48. [Google Scholar] [CrossRef]
  42. Bailly, C.; Bodet-Milin, C.; Couespel, S.; Necib, H.; Kraeber-Bodéré, F.; Ansquer, C.; Carlier, T. Revisiting the Robustness of PET-Based Textural Features in the Context of Multi-Centric Trials. PLoS ONE 2016, 11, e0159984. [Google Scholar] [CrossRef]
  43. Leijenaar, R.T.H.; Nalbantov, G.; Carvalho, S.; Van Elmpt, W.J.C.; Troost, E.G.C.; Boellaard, R.; Aerts, H.J.W.L.; Gillies, R.J.; Lambin, P. The Effect of SUV Discretization in Quantitative FDG-PET Radiomics: The Need for Standardized Methodology in Tumor Texture Analysis. Sci. Rep. 2015, 5, 11075. [Google Scholar] [CrossRef]
  44. Van Timmeren, J.E.; Cester, D.; Tanadini-Lang, S.; Alkadhi, H.; Baessler, B. Radiomics in Medical Imaging—“How-to” Guide and Critical Reflection. Insights Imaging 2020, 11, 91. [Google Scholar] [CrossRef]
  45. Zwanenburg, A.; Vallières, M.; Abdalah, M.A.; Aerts, H.J.W.L.; Andrearczyk, V.; Apte, A.; Ashrafinia, S.; Bakas, S.; Beukinga, R.J.; Boellaard, R.; et al. The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-Based Phenotyping. Radiology 2020, 295, 328–338. [Google Scholar] [CrossRef]
  46. Collewet, G.; Strzelecki, M.; Mariette, F. Influence of MRI Acquisition Protocols and Image Intensity Normalization Methods on Texture Classification. Magn. Reson. Imaging 2004, 22, 81–91. [Google Scholar] [CrossRef]
  47. Vallières, M.; Freeman, C.R.; Skamene, S.R.; El Naqa, I. A Radiomics Model from Joint FDG-PET and MRI Texture Features for the Prediction of Lung Metastases in Soft-Tissue Sarcomas of the Extremities. Phys. Med. Biol. 2015, 60, 5471–5496. [Google Scholar] [CrossRef] [PubMed]
  48. Giannini, V.; Panic, J.; Regge, D.; Balestra, G.; Rosati, S. Could Normalization Improve Robustness of Abdominal MRI Radiomic Features? Biomed. Phys. Eng. Express 2023, 9, 055002. [Google Scholar] [CrossRef] [PubMed]
  49. Kremer, L.E.; Perri, N.; Sorber, E.; Chapman, A.; Armato, S.G. Normalization of MRI Signal Intensity in Polycystic Kidney Disease and the Effect on Radiomic Features. In Proceedings of the Medical Imaging 2022: Computer-Aided Diagnosis; Iftekharuddin, K.M., Drukker, K., Mazurowski, M.A., Lu, H., Muramatsu, C., Samala, R.K., Eds.; SPIE: San Diego, CA, USA, 2022; p. 132. [Google Scholar]
  50. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  51. Van De Schoot, R.; De Bruin, J.; Schram, R.; Zahedi, P.; De Boer, J.; Weijdema, F.; Kramer, B.; Huijts, M.; Hoogerwerf, M.; Ferdinands, G.; et al. An Open Source Machine Learning Framework for Efficient and Transparent Systematic Reviews. Nat. Mach. Intell. 2021, 3, 125–133. [Google Scholar] [CrossRef]
  52. Whiting, P.F. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann. Intern. Med. 2011, 155, 529. [Google Scholar] [CrossRef] [PubMed]
  53. Larue, R.T.H.M.; Van Timmeren, J.E.; De Jong, E.E.C.; Feliciani, G.; Leijenaar, R.T.H.; Schreurs, W.M.J.; Sosef, M.N.; Raat, F.H.P.J.; Van Der Zande, F.H.R.; Das, M.; et al. Influence of Gray Level Discretization on Radiomic Feature Stability for Different CT Scanners, Tube Currents and Slice Thicknesses: A Comprehensive Phantom Study. Acta Oncol. 2017, 56, 1544–1553. [Google Scholar] [CrossRef] [PubMed]
  54. Mackin, D.; Fave, X.; Zhang, L.; Yang, J.; Jones, A.K.; Ng, C.S.; Court, L. Harmonizing the Pixel Size in Retrospective Computed Tomography Radiomics Studies. PLoS ONE 2017, 12, e0178524. [Google Scholar] [CrossRef] [PubMed]
  55. Stüber, A.T.; Coors, S.; Schachtner, B.; Weber, T.; Rügamer, D.; Bender, A.; Mittermeier, A.; Öcal, O.; Seidensticker, M.; Ricke, J.; et al. A Comprehensive Machine Learning Benchmark Study for Radiomics-Based Survival Analysis of CT Imaging Data in Patients With Hepatic Metastases of CRC. Investig. Radiol. 2023, 58, 874–881. [Google Scholar] [CrossRef] [PubMed]
  56. Demircioğlu, A. The Effect of Preprocessing Filters on Predictive Performance in Radiomics. Eur. Radiol. Exp. 2022, 6, 40. [Google Scholar] [CrossRef] [PubMed]
  57. Linsalata, S.; Borgheresi, R.; Marfisi, D.; Barca, P.; Sainato, A.; Paiar, F.; Neri, E.; Traino, A.C.; Giannelli, M. Radiomics of Patients with Locally Advanced Rectal Cancer: Effect of Preprocessing on Features Estimation from Computed Tomography Imaging. BioMed Res. Int. 2022, 2022, 2003286. [Google Scholar] [CrossRef] [PubMed]
  58. Au, R.C.; Tan, W.C.; Bourbeau, J.; Hogg, J.C.; Kirby, M. Impact of Image Pre-Processing Methods on Computed Tomography Radiomics Features in Chronic Obstructive Pulmonary Disease. Phys. Med. Biol. 2021, 66, 245015. [Google Scholar] [CrossRef]
  59. Nazari, M.; Shiri, I.; Zaidi, H. Radiomics-Based Machine Learning Model to Predict Risk of Death within 5-Years in Clear Cell Renal Cell Carcinoma Patients. Comput. Biol. Med. 2021, 129, 104135. [Google Scholar] [CrossRef] [PubMed]
  60. Kolossváry, M.; Szilveszter, B.; Karády, J.; Drobni, Z.D.; Merkely, B.; Maurovich-Horvat, P. Effect of Image Reconstruction Algorithms on Volumetric and Radiomic Parameters of Coronary Plaques. J. Cardiovasc. Comput. Tomogr. 2019, 13, 325–330. [Google Scholar] [CrossRef]
  61. Fave, X.; Zhang, L.; Yang, J.; Mackin, D.; Balter, P.; Gomez, D.; Followill, D.; Jones, A.K.; Stingo, F.; Court, L.E. Impact of Image Preprocessing on the Volume Dependence and Prognostic Potential of Radiomics Features in Non-Small Cell Lung Cancer. Transl. Cancer Res. 2016, 5, 349–363. [Google Scholar] [CrossRef]
  62. Hunter, L.A.; Krafft, S.; Stingo, F.; Choi, H.; Martel, M.K.; Kry, S.F.; Court, L.E. High Quality Machine-Robust Image Features: Identification in Nonsmall Cell Lung Cancer Computed Tomography Images: Robust Quantitative Image Features. Med. Phys. 2013, 40, 121916. [Google Scholar] [CrossRef] [PubMed]
  63. Palani, D.; Ganesh, K.M.; Karunagaran, L.; Govindaraj, K.; Shanmugam, S. Statistical Analysis on Impact of Image Preprocessing of CT Texture Patterns and Its CT Radiomic Feature Stability: A Phantom Study. Asian Pac. J. Cancer Prev. 2023, 24, 2061–2072. [Google Scholar] [CrossRef] [PubMed]
  64. Delgadillo, R.; Spieler, B.O.; Ford, J.C.; Kwon, D.; Yang, F.; Studenski, M.; Padgett, K.R.; Abramowitz, M.C.; Dal Pra, A.; Stoyanova, R.; et al. Repeatability of CBCT Radiomic Features and Their Correlation with CT Radiomic Features for Prostate Cancer. Med. Phys. 2021, 48, 2386–2399. [Google Scholar] [CrossRef]
  65. Li, Y.; Ammari, S.; Balleyguier, C.; Lassau, N.; Chouzenoux, E. Impact of Preprocessing and Harmonization Methods on the Removal of Scanner Effects in Brain MRI Radiomic Features. Cancers 2021, 13, 3000. [Google Scholar] [CrossRef]
  66. McHugh, D.J.; Porta, N.; Little, R.A.; Cheung, S.; Watson, Y.; Parker, G.J.M.; Jayson, G.C.; O’Connor, J.P.B. Image Contrast, Image Pre-Processing, and T1 Mapping Affect MRI Radiomic Feature Repeatability in Patients with Colorectal Cancer Liver Metastases. Cancers 2021, 13, 240. [Google Scholar] [CrossRef] [PubMed]
  67. Carré, A.; Klausner, G.; Edjlali, M.; Lerousseau, M.; Briend-Diop, J.; Sun, R.; Ammari, S.; Reuzé, S.; Alvarez Andres, E.; Estienne, T.; et al. Standardization of Brain MR Images across Machines and Protocols: Bridging the Gap for MRI-Based Radiomics. Sci. Rep. 2020, 10, 12340. [Google Scholar] [CrossRef]
  68. Traverso, A.; Kazmierski, M.; Welch, M.L.; Weiss, J.; Fiset, S.; Foltz, W.D.; Gladwish, A.; Dekker, A.; Jaffray, D.; Wee, L.; et al. Sensitivity of Radiomic Features to Inter-Observer Variability and Image Pre-Processing in Apparent Diffusion Coefficient (ADC) Maps of Cervix Cancer Patients. Radiother. Oncol. 2020, 143, 88–94. [Google Scholar] [CrossRef]
  69. Um, H.; Tixier, F.; Bermudez, D.; Deasy, J.O.; Young, R.J.; Veeraraghavan, H. Impact of Image Preprocessing on the Scanner Dependence of Multi-Parametric MRI Radiomic Features and Covariate Shift in Multi-Institutional Glioblastoma Datasets. Phys. Med. Biol. 2019, 64, 165011. [Google Scholar] [CrossRef] [PubMed]
  70. Bleker, J.; Roest, C.; Yakar, D.; Huisman, H.; Kwee, T.C. The Effect of Image Resampling on the Performance of Radiomics-Based Artificial Intelligence in Multicenter Prostate MRI. Magn. Reson. Imaging 2024, 59, 1800–1806. [Google Scholar] [CrossRef]
  71. Bologna, M.; Tenconi, C.; Corino, V.D.A.; Annunziata, G.; Orlandi, E.; Calareso, G.; Pignoli, E.; Valdagni, R.; Mainardi, L.T.; Rancati, T. Repeatability and Reproducibility of MRI-Radiomic Features: A Phantom Experiment on a 1.5 T Scanner. Med. Phys. 2023, 50, 750–762. [Google Scholar] [CrossRef]
  72. Marzi, C.; Marfisi, D.; Barucci, A.; Del Meglio, J.; Lilli, A.; Vignali, C.; Mascalchi, M.; Casolo, G.; Diciotti, S.; Traino, A.C.; et al. Collinearity and Dimensionality Reduction in Radiomics: Effect of Preprocessing Parameters in Hypertrophic Cardiomyopathy Magnetic Resonance T1 and T2 Mapping. Bioengineering 2023, 10, 80. [Google Scholar] [CrossRef]
  73. Marfisi, D.; Tessa, C.; Marzi, C.; Del Meglio, J.; Linsalata, S.; Borgheresi, R.; Lilli, A.; Lazzarini, R.; Salvatori, L.; Vignali, C.; et al. Image Resampling and Discretization Effect on the Estimate of Myocardial Radiomic Features from T1 and T2 Mapping in Hypertrophic Cardiomyopathy. Sci. Rep. 2022, 12, 10186. [Google Scholar] [CrossRef] [PubMed]
  74. Koçak, B.; Yüzkan, S.; Mutlu, S.; Karagülle, M.; Kala, A.; Kadıoğlu, M.; Solak, S.; Sunman, Ş.; Temiz, Z.H.; Ganiyusufoğlu, A.K. Influence of Image Preprocessing on the Segmentation-Based Reproducibility of Radiomic Features: In Vivo Experiments on Discretization and Resampling Parameters. Diagn. Interv. Radiol. 2024, 30, 152–162. [Google Scholar] [CrossRef] [PubMed]
  75. Brynolfsson, P.; Nilsson, D.; Torheim, T.; Asklund, T.; Karlsson, C.T.; Trygg, J.; Nyholm, T.; Garpebring, A. Haralick Texture Features from Apparent Diffusion Coefficient (ADC) MRI Images Depend on Imaging and Pre-Processing Parameters. Sci. Rep. 2017, 7, 4041. [Google Scholar] [CrossRef] [PubMed]
  76. Abunahel, B.M.; Pontre, B.; Petrov, M.S. Effect of Gray Value Discretization and Image Filtration on Texture Features of the Pancreas Derived from Magnetic Resonance Imaging at 3T. J. Imaging 2022, 8, 220. [Google Scholar] [CrossRef]
  77. Toffoli, T.; Saut, O.; Etchegaray, C.; Jambon, E.; Le Bras, Y.; Grenier, N.; Marcelin, C. Differentiation of Small Clear Renal Cell Carcinoma and Oncocytoma through Magnetic Resonance Imaging-Based Radiomics Analysis: Toward the End of Percutaneous Biopsy. JPM 2023, 13, 1444. [Google Scholar] [CrossRef]
  78. Abunahel, B.M.; Pontre, B.; Ko, J.; Petrov, M.S. Towards Developing a Robust Radiomics Signature in Diffuse Diseases of the Pancreas: Accuracy and Stability of Features Derived from T1-Weighted Magnetic Resonance Imaging. J. Med. Imaging Radiat. Sci. 2022, 53, 420–428. [Google Scholar] [CrossRef]
  79. Dewi, D.E.O.; Sunoqrot, M.R.S.; Nketiah, G.A.; Sandsmark, E.; Giskeødegård, G.F.; Langørgen, S.; Bertilsson, H.; Elschot, M.; Bathen, T.F. The Impact of Pre-Processing and Disease Characteristics on Reproducibility of T2-Weighted MRI Radiomics Features. Magn. Reson. Mater. Phys. 2023, 36, 945–956. [Google Scholar] [CrossRef]
  80. Foltyn-Dumitru, M.; Schell, M.; Rastogi, A.; Sahm, F.; Kessler, T.; Wick, W.; Bendszus, M.; Brugnara, G.; Vollmuth, P. Impact of Signal Intensity Normalization of MRI on the Generalizability of Radiomic-Based Prediction of Molecular Glioma Subtypes. Eur. Radiol. 2023, 34, 2782–2790. [Google Scholar] [CrossRef]
  81. Salome, P.; Sforazzini, F.; Grugnara, G.; Kudak, A.; Dostal, M.; Herold-Mende, C.; Heiland, S.; Debus, J.; Abdollahi, A.; Knoll, M. MR Intensity Normalization Methods Impact Sequence Specific Radiomics Prognostic Model Performance in Primary and Recurrent High-Grade Glioma. Cancers 2023, 15, 965. [Google Scholar] [CrossRef]
  82. Ubaldi, L.; Saponaro, S.; Giuliano, A.; Talamonti, C.; Retico, A. Deriving Quantitative Information from Multiparametric MRI via Radiomics: Evaluation of the Robustness and Predictive Value of Radiomic Features in the Discrimination of Low-Grade versus High-Grade Gliomas with Machine Learning. Phys. Medica 2023, 107, 102538. [Google Scholar] [CrossRef]
  83. Granzier, R.W.Y.; Ibrahim, A.; Primakov, S.; Keek, S.A.; Halilaj, I.; Zwanenburg, A.; Engelen, S.M.E.; Lobbes, M.B.I.; Lambin, P.; Woodruff, H.C.; et al. Test–Retest Data for the Assessment of Breast MRI Radiomic Feature Repeatability. Magn. Reson. Imaging 2022, 56, 592–604. [Google Scholar] [CrossRef] [PubMed]
  84. Hoebel, K.V.; Patel, J.B.; Beers, A.L.; Chang, K.; Singh, P.; Brown, J.M.; Pinho, M.C.; Batchelor, T.T.; Gerstner, E.R.; Rosen, B.R.; et al. Radiomics Repeatability Pitfalls in a Scan-Rescan MRI Study of Glioblastoma. Radiol. Artif. Intell. 2021, 3, e190199. [Google Scholar] [CrossRef] [PubMed]
  85. Scalco, E.; Belfatto, A.; Mastropietro, A.; Rancati, T.; Avuzzi, B.; Messina, A.; Valdagni, R.; Rizzo, G. T2w-MRI Signal Normalization Affects Radiomics Features Reproducibility. Med. Phys. 2020, 47, 1680–1691. [Google Scholar] [CrossRef]
  86. Shiri, I.; Hajianfar, G.; Sohrabi, A.; Abdollahi, H.; Shayesteh, S.P.; Geramifar, P.; Zaidi, H.; Oveisi, M.; Rahmim, A. Repeatability of Radiomic Features in Magnetic Resonance Imaging of Glioblastoma: Test–Retest and Image Registration Analyses. Med. Phys. 2020, 47, 4265–4280. [Google Scholar] [CrossRef]
  87. Simpson, G.; Ford, J.C.; Llorente, R.; Portelance, L.; Yang, F.; Mellon, E.A.; Dogan, N. Impact of Quantization Algorithm and Number of Gray Level Intensities on Variability and Repeatability of Low Field Strength Magnetic Resonance Image-Based Radiomics Texture Features. Phys. Medica 2020, 80, 209–220. [Google Scholar] [CrossRef]
  88. Schwier, M.; Van Griethuysen, J.; Vangel, M.G.; Pieper, S.; Peled, S.; Tempany, C.; Aerts, H.J.W.L.; Kikinis, R.; Fennessy, F.M.; Fedorov, A. Repeatability of Multiparametric Prostate MRI Radiomics Features. Sci. Rep. 2019, 9, 9441. [Google Scholar] [CrossRef]
  89. Traverso, A.; Kazmierski, M.; Shi, Z.; Kalendralis, P.; Welch, M.; Nissen, H.D.; Jaffray, D.; Dekker, A.; Wee, L. Stability of Radiomic Features of Apparent Diffusion Coefficient (ADC) Maps for Locally Advanced Rectal Cancer in Response to Image Pre-Processing. Phys. Medica 2019, 61, 44–51. [Google Scholar] [CrossRef]
  90. Ferreira, M.; Lovinfosse, P.; Hermesse, J.; Decuypere, M.; Rousseau, C.; Lucia, F.; Schick, U.; Reinhold, C.; Robin, P.; Hatt, M.; et al. [18F]FDG PET Radiomics to Predict Disease-Free Survival in Cervical Cancer: A Multi-Scanner/Center Study with External Validation. Eur. J. Nucl. Med. Mol. Imaging 2021, 48, 3432–3443. [Google Scholar] [CrossRef]
  91. Xu, H.; Lv, W.; Zhang, H.; Ma, J.; Zhao, P.; Lu, L. Evaluation and Optimization of Radiomics Features Stability to Respiratory Motion in 18F-FDG 3D PET Imaging. Med. Phys. 2021, 48, 5165–5178. [Google Scholar] [CrossRef]
  92. Crandall, J.P.; Fraum, T.J.; Lee, M.; Jiang, L.; Grigsby, P.; Wahl, R.L. Repeatability of 18F-FDG PET Radiomic Features in Cervical Cancer. J. Nucl. Med. 2021, 62, 707–715. [Google Scholar] [CrossRef] [PubMed]
  93. Ger, R.B.; Meier, J.G.; Pahlka, R.B.; Gay, S.; Mumme, R.; Fuller, C.D.; Li, H.; Howell, R.M.; Layman, R.R.; Stafford, R.J.; et al. Effects of Alterations in Positron Emission Tomography Imaging Parameters on Radiomics Features. PLoS ONE 2019, 14, e0221877. [Google Scholar] [CrossRef] [PubMed]
  94. Duron, L.; Savatovsky, J.; Fournier, L.; Lecler, A. Can We Use Radiomics in Ultrasound Imaging? Impact of Preprocessing on Feature Repeatability. Diagn. Interv. Imaging 2021, 102, 659–667. [Google Scholar] [CrossRef] [PubMed]
  95. Li, M.-D.; Cheng, M.-Q.; Chen, L.-D.; Hu, H.-T.; Zhang, J.-C.; Ruan, S.-M.; Huang, H.; Kuang, M.; Lu, M.-D.; Li, W.; et al. Reproducibility of Radiomics Features from Ultrasound Images: Influence of Image Acquisition and Processing. Eur. Radiol. 2022, 32, 5843–5851. [Google Scholar] [CrossRef]
  96. Jia, Y.; Yang, J.; Zhu, Y.; Nie, F.; Wu, H.; Duan, Y.; Chen, K. Ultrasound-Based Radiomics: Current Status, Challenges and Future Opportunities. Med. Ultrason. 2022, 24, 451. [Google Scholar] [CrossRef]
Cancers 16 02668 g001
Figure 1. Schematic radiomic workflow. Regarding voxel resampling, we counted the number of works that used single isotropic, multiple isotropic, and non-isotropic voxel interpolation. When the original voxel size was used, we put “N.A.”, and when the information was not indicated, “None”.
Figure 1. Schematic radiomic workflow. Regarding voxel resampling, we counted the number of works that used single isotropic, multiple isotropic, and non-isotropic voxel interpolation. When the original voxel size was used, we put “N.A.”, and when the information was not indicated, “None”.
Cancers 16 02668 g001
Cancers 16 02668 g002
Figure 2. Preprocessing scheme used in the evaluation of the items. In addition, we investigated the study aims, the anatomical districts considered, and the metric used within the works. Concerning the study aims, we counted how many studies used best performance, repeatability, or both strategies. The anatomical districts were categorized into “abdomen”, “brain”, “thorax”, and “pelvis” for simplicity’s sake. When no anatomical part was specified, e.g., for phantom studies, we set “N.A.”. Moreover, the metric indexes were grouped into ICC, CCC, AUC, or “Other” when different statistical metrics were used.
Figure 2. Preprocessing scheme used in the evaluation of the items. In addition, we investigated the study aims, the anatomical districts considered, and the metric used within the works. Concerning the study aims, we counted how many studies used best performance, repeatability, or both strategies. The anatomical districts were categorized into “abdomen”, “brain”, “thorax”, and “pelvis” for simplicity’s sake. When no anatomical part was specified, e.g., for phantom studies, we set “N.A.”. Moreover, the metric indexes were grouped into ICC, CCC, AUC, or “Other” when different statistical metrics were used.
Cancers 16 02668 g002
Cancers 16 02668 g003
Figure 3. PRISMA-P flow chart: article selection process.
Figure 3. PRISMA-P flow chart: article selection process.
Cancers 16 02668 g003
Cancers 16 02668 g004
Figure 4. Risk of bias (left) and applicability concerns (right).
Figure 4. Risk of bias (left) and applicability concerns (right).
Cancers 16 02668 g004
Table 1. The distribution of found records across databases.
Table 1. The distribution of found records across databases.
Database (n° of Record)n° of Total Records
(with Duplicates)		n° of Total Records
(without Duplicates)
MedlineEmbaseCochraneScopus
2082861141546459
Table 2. The number of articles included and stratified by voxel resampling type (N.A. indicates this information is missing).
Table 2. The number of articles included and stratified by voxel resampling type (N.A. indicates this information is missing).
ModalityAcquisition Parameter ReportingVoxel Resampling
IsotropicMultiple IsotropicNon-IsotropicN.A.None
CT10 (83.3%)2 (16.7%)3 (25.0%)2 (16.7%)5 (41.7%) 0 (0.0%)
Ref. [55,56][54,57,58][39,53][59,60,61,62,63]
CBCT1 (100%)1 (100%)0 (0%)0 (0%)0 (0%)0 (0%)
Ref. [64]
MRI21 (80.8%)9 (34.6%)2 (7.7%)3 (11.5%)5 (19.2%)7 (26.9%)
Ref. [56,65,66,67,68,69,70,71,72][73,74][75,76,77][78,79,80,81,82][83,84,85,86,87,88,89]
PET/CT5 (100%)2 (40%)2 (40%)0 (0%)0 (0%)1 (20%)
Ref. [90,91][28,92] [93]
Table 3. Normalization strategies grouped by modality. Combination stands for absolute and relative strategies used for comparison.
Table 3. Normalization strategies grouped by modality. Combination stands for absolute and relative strategies used for comparison.
ModalityAbsoluteRelativeCombinationNone
CT0 (0%)1 (8.3%)1 (8.3%)10 (83.3%)
Ref. [56][58][39,53,54,55,57,59,60,61,62,63]
CBCT0 (0%)1 (100%)0 (0%)0 (0%)
Ref. [64]
MRI0 (0%)14 (53.8%)4 (15.4%)8 (30.8%)
Ref. [56,68,71,74,76,77,79,80,81,82,83,84,87,88][65,66,67,85][69,70,72,73,75,78,86,89]
PET/CT0 (0%)1 (20%)1 (20%)3 (60%)
Ref. [90][92][28,91,93]
Table 4. Discretization strategies grouped by modality.
Table 4. Discretization strategies grouped by modality.
BNBWBN + BWNone
CT2 (16.7%)6 (50%)1 (8.3%)3 (25%)
Ref.[39,59][53,55,56,57,58,61][60][54,62,63]
CBCT1 (100%)0 (0%)0 (0%)0 (0%)
Ref.[64]
MRI10 (38.5%)9 (36.6%)5 (19.2%)2 (7.7%)
Ref.[65,69,71,75,81,82,84,85,86,87][56,66,72,73,76,77,83,88,89][67,68,74,78,79][70,80]
PET/CT1 (20%)0 (0%)4 (80%)0 (0%)
Ref.[92] [28,90,91,93]
Table 5. The study aims per modality.
Table 5. The study aims per modality.
Best PerformanceRepeatabilityReproducibilityRepeatability + Reproducibility
CT3 (25%)2 (16.7%)5 (41.6%)2 (16.7%)
Ref.[55,56,59][54,62][39,57,58,60,61][53,63]
CBCT0 (0%)0 (0%)0 (0%)1 (100%)
Ref. [64]
MRI8 (30.8%)7 (26.9%)6 (23.1%)5 (19.2%)
Ref.[56,70,72,75,77,78,80,82][66,73,76,84,86,87,88][65,67,68,74,79,81][69,71,83,85,89]
PET/CT2 (40%)3 (60%)0 (0%)0 (0%)
Ref.[90,91][28,92,93]
Table 6. Investigated “macro” anatomic districts divided per modality.
Table 6. Investigated “macro” anatomic districts divided per modality.
AbdomenBrainThoraxPelvisN.A.
CT1 (8.4%)03 (25%)4 (33.3%)4 (33.3%)
Ref.[55] [54,61,62][57,58,59,60][39,53,56,63]
CBCT000.0%1 (100%)
Ref. [64]
MRI2 (7.7%)10 (38.5%)2 (7.7%)9 (34.6%)4 (15.3%)
Ref.[76,78][65,67,69,74,75,80,81,82,84,86][72,73][66,68,70,75,77,79,85,88,89][56,71,83,87]
PET/CT003 (60%)2 (40%)0
Ref. [28,91,93][90,92]
Table 7. Comparison metrics per imaging modality.
Table 7. Comparison metrics per imaging modality.
ICCCCCAUCOther
CT3324
Ref.[57,58,60][53,54,62][56,59][39,55,61,63]
CBCT0100
Ref. [64]
MRI15759
Ref.[65,66,67,68,71,73,74,79,82,83,84,85,86,88,89][67,76,83,85,86,87,89][56,70,77,80,82][68,69,71,72,75,78,81,84,85]
PET/CT2111
Ref.[28,93][92][90][91]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Trojani, V.; Bassi, M.C.; Verzellesi, L.; Bertolini, M. Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review. Cancers 2024, 16, 2668. https://doi.org/10.3390/cancers16152668

AMA Style

Trojani V, Bassi MC, Verzellesi L, Bertolini M. Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review. Cancers. 2024; 16(15):2668. https://doi.org/10.3390/cancers16152668

Chicago/Turabian Style

Trojani, Valeria, Maria Chiara Bassi, Laura Verzellesi, and Marco Bertolini. 2024. "Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review" Cancers 16, no. 15: 2668. https://doi.org/10.3390/cancers16152668

APA Style

Trojani, V., Bassi, M. C., Verzellesi, L., & Bertolini, M. (2024). Impact of Preprocessing Parameters in Medical Imaging-Based Radiomic Studies: A Systematic Review. Cancers, 16(15), 2668. https://doi.org/10.3390/cancers16152668

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop