Next Article in Journal
Reproducibility and Repeatability of Coronary Computed Tomography Angiography (CCTA) Image Segmentation in Detecting Atherosclerosis: A Radiomics Study
Next Article in Special Issue
Robot-Assisted Radical Cystectomy: A Single-Center Experience and a Narrative Review of Recent Evidence
Previous Article in Journal
Imperative Role of Machine Learning Algorithm for Detection of Parkinson’s Disease: Review, Challenges and Recommendations
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

The Role of Fluorescence In Situ Hybridization in the Surveillance of Non-Muscle Invasive Bladder Cancer: An Updated Systematic Review and Meta-Analysis

Department of Urology, Institute of Urology, West China Hospital, Sichuan University, No. 37, Guoxue Road, Chengdu 610041, China
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Diagnostics 2022, 12(8), 2005; https://doi.org/10.3390/diagnostics12082005
Submission received: 27 July 2022 / Revised: 11 August 2022 / Accepted: 16 August 2022 / Published: 19 August 2022
(This article belongs to the Special Issue State of the Art of Bladder Cancer: From Diagnosis to Therapy)

Abstract

:
Background: Fluorescence in situ hybridization (FISH) has become a popular biomarker for subsequent monitoring the recurrence of non-muscle invasive bladder cancer (NMIBC), several studies have investigated the ability of FISH to detect recurrence in the surveillance of NMIBC. However, the results were inconsistent. Methods: We conducted a systematic literature search extensively on authenticated databases including PubMed/Medline, Embase, Web of Science, Ovid, and Cochrane Library. Meta-analysis was performed to find out the sensitivity and specificity of FISH in predicting recurrence of NMIBC. Results: 15 studies were ultimately included in this meta-analysis, a total of 2941 FISH evaluations from 2385 NMIBC patients were available. The pooled sensitivity of FISH was 68% (95% CI: 0.58–0.76), and the pooled specificity was 64% (95% CI: 0.53–0.74). Subgroup analyses were performed in 7 studies without Bacillus Calmette–Guerin (BCG) treatment, the pooled sensitivity was 82% (95% CI: 0.68–0.90), and the pooled specificity was 63% (95% CI: 0.37–0.82). And in 9 studies using “UroVysion standard” to define positive FISH results showed a pooled sensitivity of 60% (95% CI: 0.50–0.70) and specificity of 70% (95% CI: 0.61–0.78). Conclusions: The findings of this study indicate that FISH has a satisfactory sensitivity (68%) and specificity (64%) and could be a potential biomarker in the surveillance of NMIBC. Moreover, BCG treatment and different FISH methods may have an impact on the sensitivity and specificity, these factors should be taken into account when making clinical strategy.

1. Introduction

Bladder cancer (BC) is the 11th most commonly diagnosed cancer worldwide with a significant risk of cancer morbidity and mortality. Approximately 75% of patients with BC present with a disease confined to the mucosa (stage Ta and carcinoma in situ) or submucosa (stage T1) which are defined as non-muscle invasive bladder cancer (NMIBC) [1,2]. Patients diagnosed with NMIBC need lifelong follow-up after transurethral resection of bladder tumor (TUR) based on the risk profile for the reason that NMIBC is at high risk of recurrence and progression [3]. The follow-up regimens consist of cystoscopy, urine cytology, and upper urinary tract imaging [1]. However, cystoscopy is invasive, relatively expensive and has the risk of morbidity concomitantly. Urine cytology is noninvasive and useful but its sensitivity for the detection of low-grade tumors is low. Computed tomography (CT) urography has poor sensitivity as well. Therefore, a noninvasive urinary biomarker with high sensitivity and specificity is needed to reduce the frequency of invasive testing in order to detect recurrent and progression in NMIBC patients and help predict response to therapies, which can improve quality of life and decrease costs [4,5]. Urine fluorescence in situ hybridization (FISH) is a technique to detect genetic alterations of chromosome 3, 7, 17 and the deletion of locus 9p21, which are most commonly associated with BC. FISH will not be influenced by the inflammatory response of the bladder to Bacillus Calmette–Guerin (BCG) compared to cystoscopy and urine cytology. Thus, it’s considered to be a potential biomarker for the surveillance of NMIBC patients undergoing intravesical instillation therapies [6,7,8]. Several studies have evaluated the ability of FISH to detect recurrence and predict progression. However, the results were inconsistent. Therefore, we conducted a systematic review and meta-analysis to synthesize available studies focused on FISH in surveillance of patients with NMIBC to assess its prognostic value for predicting recurrent disease.

2. Materials and Methods

2.1. Protocol for the Study

We conducted the meta-analysis referred to the protocol which had been registered in the International Prospective Register of Systematic Reviews database, and the protocol identification number was CRD42019121035 [9].

2.2. Evidence Acquisition and Selection Criteria

We conducted a systematic literature search extensively based on PubMed/Medline, Embase, Web of Science, Ovid and Cochrane Library. The searching strategy was used in the databases as cited: (“FISH” or “fluorescence in situ hybridization”) and (“bladder cancer” or “bladder carcinoma” or “urothelial carcinoma” or “urothelial cancer”) and (“recurrence” or “progression” or “surveillance” or “monitor”), Medical subject Headings (MeSH) words and free words were both used in the literature search to improve recall. The aim of the literature search was to identify studies focused on FISH in the surveillance of NMIBC published in English up to January 2021. After removing duplicates, a first screening was carried out based on the title and abstract. Studies qualified for further assessment must meet the following criteria: (1) randomized controlled or observational trials; (2) reported primary outcomes or available data to acquire sensitivity or specificity of FISH, as well as the control group; (3) containing at least 15 NMIBC patients. The exclusion criteria were as follows: (1) no full text available; (2) categorized as case, reports, editorials, reviews, letters, and meta; (3) containing less than 15 NMIBC patients.
Initially, we screened all included studies on titles, abstracts, irrelevant studies were excluded. The remaining studies were assessed by reading full-text articles exhaustively. Two independent reviewers participated in the screening process. When there were conflicts, the two reviewers discussed together until an agreement is reached, if failed to reach a consensus, a third senior reviewer was consulted to settle the claim.

2.3. Data Extraction and Quality Evaluation

Two independent reviewers conducted the data extraction process. The number of true positives, true negatives, false positives and false negatives were extracted from the included studies, as well as the characteristics of the patients such as author name, publish year, country, study design, age of the patients, the method and definition of positivity of FISH assay, follow-up time, definition of recurrence, postoperative adjuvant therapy, and parameters of correlated outcomes. Disputes were resolved by discussion or consultation with a third reviewer. The details were shown in Table 1. Standard quality evaluation of the included studies was performed based on the Newcastle–Ottawa scale and quality assessment of diagnostic accuracy studies-2 [10]. This study was conducted in line with the list of standards for reporting of diagnostic accuracy 2015 tools [11] and PRISAM 2009 checklist [12].

2.4. Statistical Analysis

The pooled sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic likelihood ratios (DLRs) along with the 95% confidence intervals (CIs) were calculated using the extracted data. The sensitivity and specificity estimates were shown by forest plots using STATA 14.2 (StataCorp, College Station, TX, USA). The hierarchical summary receiver operating characteristic (HSROC) model and the bivariate model for meta analyses diagnostic accuracy test were carried out by metandi to summarize test performance command in STATA 14.2 (StataCorp) [13,14,15].These methods respected the binomial structure of diagnostic accuracy data and summarized paired measures synchronously such as sensitivity, specificity and positive or negative likelihood ratios. Furthermore, it was known that on account of different or implicit thresholds, heterogeneity was widespread across included studies. The bivariate/HSROC meta-analysis, a random effects approach, allowed pooling results [16]. Review Manager 5.3 (The Cochrane Collaboration, London, UK) was used to estimate the quality of the included studies.

3. Results

3.1. Study Selection and Identification

After comprehensive searching and rigorous evaluation, 35 studies were identified appropriate for initial assessment after screening titles and abstracts. Among the 35 relevant studies, 16 studies were excluded following the inclusion and exclusion criteria, and 19 studies were necessary for full-text articles assessed for eligibility. Then, 3 studies unable to extract data and 1 study fewer than 15 NMIBC patients were excluded. Finally, 15 studies with high reliability, adequate sample size, and comprehensible design with accessible data and full texts were included for ultimate meta-analysis (Figure 1).

3.2. Study and Patient Characteristics

These studies were published from 2005 to 2019, the number of NMIBC patients participated in these studies ranged from the highest 664 to the lowest 34 (Table 1). 9 of the 15 studies used the “UroVysion standard” to define positive cases according to the specifications of the kit [17], 5 studies used author defined standard for the discrimination of positive cases, that we called the “Author’s standard”. 1 study did not illustrate the positive definition. In the included studies, FISH results were collected at different times. Some studies performed FISH tests before and after BCG instillation, while some of them collected FISH results at 3 time points, including t0: before BCG; t1: at the end of BCG induction at 6 weeks; t2: 3 months after initial TUR.

3.3. Quality of Included Studies

We used Newcastle–Ottawa scale and QUADAS-2 tool to assess quality of the included studies. All included studies were scored above six (Table 1) based on the Newcastle–Ottawa scale and QUADAS-2 quality assessment result indicated low risk of bias (Figure 2). Hence, these studies were supposed to have a high rating of quality and very low risk of bias.

3.4. Overall Analysis

A total of 2941 FISH evaluations from 2385 NMIBC patients were available, 1078 (36.7%) of which developed recurrence events during the follow-up, including FISH positive in 700 (54.6%) and FISH negative in 378 (22.8%). The pooled sensitivity (Figure 3) of FISH in surveillance of NMIBC recurrence in the included studies was 68% (95% CI: 0.58–0.76), and the pooled specificity was 64% (95% CI: 0.53–0.74). The positive likelihood ratio (PLR) was 1.90 (95% CI: 1.44–2.51), the negative likelihood ratio (NLR) was 0.50 (95% CI: 0.38–0.65) (Figure 4). I2 test was performed to evaluate heterogeneity to select the appropriate calculation model. The heterogeneity was significant, I2 was 84.27% for sensitivity and 92.80% for specificity, respectively. Therefore, a random effect model was used. The summary receiver operating characteristic (SROC) plot could assess threshold effect, and heterogeneity of data in ROC space between sensitivity and specificity. The area under the curve (AUC) showed summary of test performance and the AUC in this study was 0.71 (95% CI: 0.36–0.92; Figure 5). The SROC curve was in the upper left corner, which demonstrated the best combination of sensitivity and specificity for the diagnostic test, so the diagnosis had a significant discriminatory potency.

3.5. Subgroup Analysis

Subgroup analysis was performed in 9 studies which used “UroVysion standard” to define positive FISH results and 7 studies whose patients did not receive BCG treatment after TUR. In “UroVysion standard” studies, the pooled sensitivity was 60% (95% CI: 0.50–0.70; Figure 6A), and the pooled specificity was 70% (95% CI: 0.61–0.78; Figure 6A). The AUC was 0.70 (95% CI: 0.66–0.74; Figure 6B). Among studies whose patients did not receive BCG treatment, the pooled sensitivity was 82% (95% CI: 0.68–0.90; Figure 6C), and the pooled specificity was 63% (95% CI: 0.37–0.82; Figure 6C). The AUC was 0.82 (95% CI: 0.78–0.85; Figure 6D).

4. Discussion

NMIBC patients have a high risk of recurrence and progression after transurethral resection, even though receiving adjuvant instillation therapy, and regular postoperative follow-up is essential to detect early recurrence in a timely manner [33].The combination of cystoscopy and urine cytology is considered the gold standard in the initial diagnosis of BC [34], and it is recommended for the surveillance of NMIBC [1]. Cystoscopy remains the primary modality for NMIBC postoperative surveillance. However, cystoscopy is invasive, painful and can make patients feel uncomfortable and dysphoric [18]. Long-term and frequent cystoscopies not only contribute to the physical pain and psychological burden of the patients, but also increase the risk of associated complications, particularly in old patients who are frail and have more underlying medical conditions. Furthermore, unnecessary cystoscopies without oncological benefits will increase the cost of the patients and waste the public healthcare resources [35,36]. Urine cytology is the most common used non-invasive urinary biomarker in the diagnosis and surveillance of NMIBC. Xie et al. conducted a meta-analysis to evaluate the diagnostic value of urine cytology in detecting BC, and the pooled sensitivity and specificity of urine cytology were 37% (95% CI, 0.35–0.39) and 95% (95% CI, 0.94–0.95), respectively [37]. Other studies reported the urine cytology with a sensitivity of 7.7–40.6% and a specificity of 88.3–98.0% [38,39,40,41]. Although cytology has a high specificity, the application is limited in terms of the low sensitivity [42]. It is essential to search for new non-invasive diagnostic biomarkers of both high sensitivity and specificity beyond cytology in predicting early recurrence of NMIBC [43].
FISH is non-invasive and convenient, previous studies have shown that FISH is a more accurate biomarker for detecting and predicting recurrence in NMIBC patients with a higher sensitivity than cytology. However, the specificity of cytology is superior when compared with FISH [44,45]. This is consistent with our results, the pooled sensitivity of FISH (68%) is much higher than that of cytology (7.7–40.6%) reported previously, although the pooled specificity (64%) of FISH is considerably high, there is still a gap from that of cytology (88.3–98.0%). Positive FISH results may help predict recurrence of NMIBC during the surveillance process, while the cystoscopy is negative but cytology is dubious [19,20,46].
Postoperative surveillance on the early recurrence of NMIBC is critical and relevant to the success of treatment, for the reason that early detection makes early intervention possible and avoids progression to MIBC to a certain extent. Once disease progressed to MIBC, patients usually need to receive radical cystectomy (RC) as a standard treatment, however, RC is associated with high risk of postoperative complications and reoperation [47]. Furthermore, the application of RC in elderly patients is controversial, particularly for those who are over 80 years old. Although extraperitoneal RC and ureterostomy have been indicated to have a limited mortality and mobility, the high risk of associated complications is still worrisome, especially in old patients with high level of frailty [48,49].
In addition to urine biomarkers, a number of clinical variables can also help predict the risk of recurrence and progression of NMIBC. Cicione et al. found that detrusor muscle thickness (DWT) measured by ultrasound was associated with NMIBC recurrence and progression, and patients with a greater DWT (>2.5 mm) had a higher risk of postoperative recurrence (OR 4.9, 95% CI: 2.5–9.5, p = 0.001) and progression (OR 2.21, 95% CI: 1.71–4.73, p = 0.001) [50]. FISH combined with DWT may be a non-invasive and available novel method with potentially clinically value to improve the accuracy of predicting recurrence and progression of NMIBC, which also can lengthen the surveillance intervals of cystoscopies and contribute to less pain and cost of the patients. Further studies are needed to explore and confirm the above assumption.
Liem et al. [51] conducted a meta-analysis in 2020 to assess the value of fluorescence in situ hybridization to predict early recurrence in patients with NMIBC at intermediate and high risk treated with BCG. 408 patients from 4 studies were included in the final analysis, recurrence hazard ratio (HR) was 1.20 (95% CI 0.81–1.79) before BCG, 2.23 (95% CI 1.31–3.62) at 6 weeks, 3.70 (95% CI 2.34–5.83) at 3 months and 23.44 (95% CI 5.26–104.49) at 6 months when FISH test is positive. Their result indicated that a positive FISH test result after BCG was associated with higher risk of recurrence, the preferred timing of FISH was 3 months after TUR. Liem’s study focused on the intermediate and high risk NMIBC patients treated with BCG, and reported HR in different time points, however, some studies including different stages of NMIBC patients, patients without BCG treatment after TUR and studies without time-related endings whose HR analysis was unavailable were not obtained in their analysis. As a result, the sample size was limited as well as the credibility and generalizability of the results. We conducted a systematic literature search extensively on authenticated databases and clinical trials published in English up to January 2022 focused on the utility of FISH in surveillance of patients with NMIBC were eligible. Our systematic review and meta-analysis included a total of 2941 FISH evaluations from 2385 NMIBC patients in initial 15 studies and reported the sensitivity and specificity as primary outcomes. Based on literature search, this meta-analysis is the largest and the latest study evaluating FISH focused on the utility of FISH in surveillance of patients with NMIBC to assess its prognostic value for predicting recurrence, which indicated that FISH has a satisfactory sensitivity and specificity in predicting recurrence in NMIBC patients.
Since BCG instillation is recommended for patients with intermediate- and high-risk tumors, the efficacy of FISH for the surveillance of NMIBC after BCG instillation is of great interest for practical purposes. Our results showed that the overall sensitivity of FISH for detection NMIBC recurrence was 68%, and the specificity was 64%, among studies whose patients did not receive BCG treatment, the pooled sensitivity was 82% and specificity was 63%, which indicated that BCG instillation may interfere with the FISH result. Possible explanations given for the lower accuracy for BCG instillation patients noted in the current study might be attributed to: (1) Follow-up time: longer follow-up time resulting in higher recurrence rate [21,52]. However, the follow-up time between BCG instillation studies and non-BCG instillation studies in our research has no statistical difference (95% CI: −14.19–8.00; p = 0.54). (2) BCG instillation therapy may have a screening effect, which can induce tumor mutations, and cause the difference between primary and recurrent urothelial carcinoma (UC). For recurrent UC, it may be necessary to detect the genes and chromosomes that may be mutated to find a more accurate urine marker [53]. Further investigation concerning different targeted genes or chromosomes may be needed to adjust the accuracy of FISH test for the surveillance of NMIBC patients undergoing BCG instillation.
Table 1. Characteristics of the included studies and patients.
Table 1. Characteristics of the included studies and patients.
AuthorYearCountryStudy DesignPatients (No)Mean Age (Years)FISH MethodTime to FISH (no)Follow-Up Time (mo)NOS Score
Bollmann et al. [22]2005GermanyRetrospective3469.0UroVysion FISHNA29.07
Mian et al. [23]2006ItalyProspective7571.7UroVysion FISHNA39.38
Mengual et al. [24]2007SpanishProspective6570.0UroVysion FISHBefore the first BCG instillationNA7
Mengual et al. [24]2007SpanishProspective6570.0UroVysion FISHAfter the last BCG instillationNA7
Yoder et al. [19]2007USAProspective25072.0UroVysion FISHAt the time of the cytologic diagnosis23.08
Gofrit et al. [25]2008USAProspective6462.0UroVysion FISHAt the conclusion of cystoscopy13.58
Kipp et al. [26]2009USARetrospective30370.0UroVysion probe setNA15.08
Savic et al.-1 [27]2009SwitzerlandProspective6871.0Multitarget UroVysionBefore the first BCG instillation (n = 50) + before TUR (n = 18)19.58
Savic et al.-2 [27]2009SwitzerlandProspective6871.0Multitarget UroVysionAfter the sixth BCG instillation.19.58
Maffezzini et al. [28]2010ItalyProspective12669.2UroVysion FISHNA14.08
Galvan et al. [18]2011SpainProspective22272.9Vysis FISH Pretreatment Reagent KitBefore cystoscopy8.07
Huang et al. [29]2014ChinaProspective41NABladder Cancer KitsBefore cystoscopy29.58
Seideman et al. [20]2015USARetrospective66470.0NANA26.08
Todenhofer et al. [30]2015GermanyProspective11470.0UroVysion FISHBefore cystoscopy24.07
Kamat et al. [21]2016USAProspective9566.0UroVysion kitAt the first BCG instillation + 6 weeks + 3 months + 6 months24.08
Liem et al. [31]2017The NetherlandsProspective11470.7UroVysion FISHBefore the first BCG instillation23.08
Liem et al. [31]2017The NetherlandsProspective10670.7UroVysion FISHAt 6 weeks following TUR23.08
Liem et al. [31]2017The NetherlandsProspective6670.7UroVysion FISHAt 3 months following TUR23.08
Lotan et al. [32]2019USAProspective15072.0UroVysion FISHBefore the first BCG instillation9.07
Lotan et al. [32]2019USAProspective13372.0UroVysion FISHBefore the sixth BCG instillation9.07
Lotan et al. [32]2019USAProspective11872.0UroVysion FISHAt 3 months after BCG initiation9.07
FISH: Fluorescence in situ hybridization; TUR: transurethral resection of bladder tumor; BCG: Bacillus Calmette–Guerin; NA: not available; NOS: Newcastle–Ottawa.
It is worth mentioning that specificity of a few studies which did not use the UroVysion FISH method to test the specimen or follow the “UroVysion standard” to define a positive FISH result is 60%, lower than the pooled overall sensitivity of 68%. And the specificity is 70%, higher than the pooled overall specificity of 64%. Different FISH test methods seem to influence the sensitivity and specificity in some ways, so it is recommended that further researches focused on FISH perform the test according to a unified approach such as the UroVysion FISH method to avoid the bias and improve the accuracy of the results. Subgroup analyses revealed that BCG treatment and FISH methods may have an impact on FISH result.
Although the included studies have high quality, we acknowledge several limitations in this study. First, the definition of positive FISH result was not standardized with wide differences between studies which introduce heterogeneous results, but we performed subgroup analysis based on FISH methods, the results indicated that FISH test methods seem to influence the sensitivity and specificity, the more compliant FISH criteria could have led to overestimating positive FISH tests. Second, although we included 2941 FISH evaluations from 15 studies, the number of papers qualifying for the analysis is still small, therefore recall bias may have occurred. Third, HR analysis was not available since some studies didn’t have time-related endings. Our results should be interpreted with caution in consideration of the risk of bias across studies and statistical heterogeneity.

5. Conclusions

In this meta-analysis, our results reveal that FISH has a satisfactory sensitivity of 68% and specificity of 64% in predicting recurrence in patients with NMIBC, our findings suggest that FISH is a potential biomarker in the prognosis surveillance of NMIBC. Patients with positive FISH test still may need more intensive follow-up, even undertake precautionary therapies, such as additional cycles of maintenance instillation or conversion to radical cystectomy. However, BCG treatment and different FISH methods may have an impact on the sensitivity and specificity, these factors should be taken into account when making clinical strategy.

Author Contributions

Conceptualization, W.Z. and T.L.; Methodology, Z.C.; Software, W.Z.; Validation, W.Z., T.L. and Y.B.; Formal Analysis, W.Z.; Investigation, D.C.; Resources, P.Z.; Data Curation, W.Z.; Writing—Original Draft Preparation, W.Z.; Writing—Review & Editing, W.Z.; Visualization, L.Y.; Supervision, Q.W.; Project Administration, Q.W.; Funding Acquisition, T.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Sichuan Provincial Science and Technology Department (2020YFS0046).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author Qiang Wei. The data are not publicly available due to them containing information that could compromise research participant privacy. We confirm that the authors are accountable for all aspects of the work (if applied, including full data access, the integrity of the data, and the accuracy of the data analysis) in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Acknowledgments

We thank the “Tianhai Lin” for relative protocol and statistics analysis.

Conflicts of Interest

All authors declare that no competing interest associated with the manuscript needs to disclose.

References

  1. Babjuk, M.; Burger, M.; Comperat, E.M.; Gontero, P.; Mostafid, A.H.; Palou, J.; van Rhijn, B.W.G.; Roupret, M.; Shariat, S.F.; Sylvester, R.; et al. European Association of Urology Guidelines on Non-muscle-invasive Bladder Cancer (TaT1 and Carcinoma In Situ)—2019 Update. Eur. Urol. 2019, 76, 639–657. [Google Scholar] [CrossRef] [PubMed]
  2. Burger, M.; Catto, J.W.; Dalbagni, G.; Grossman, H.B.; Herr, H.; Karakiewicz, P.; Kassouf, W.; Kiemeney, L.A.; La Vecchia, C.; Shariat, S.; et al. Epidemiology and risk factors of urothelial bladder cancer. Eur. Urol. 2013, 63, 234–241. [Google Scholar] [CrossRef] [PubMed]
  3. Felsenstein, K.M.; Theodorescu, D. Precision medicine for urothelial bladder cancer: Update on tumour genomics and immunotherapy. Nat. Rev. Urol. 2018, 15, 92–111. [Google Scholar] [CrossRef] [PubMed]
  4. Kamat, A.M.; Hegarty, P.K.; Gee, J.R.; Clark, P.E.; Svatek, R.S.; Hegarty, N.; Shariat, S.F.; Xylinas, E.; Schmitz-Drager, B.J.; Lotan, Y.; et al. ICUD-EAU International Consultation on Bladder Cancer 2012: Screening, diagnosis, and molecular markers. Eur. Urol. 2013, 63, 4–15. [Google Scholar] [CrossRef]
  5. Tilki, D.; Burger, M.; Dalbagni, G.; Grossman, H.B.; Hakenberg, O.W.; Palou, J.; Reich, O.; Roupret, M.; Shariat, S.F.; Zlotta, A.R. Urine markers for detection and surveillance of non-muscle-invasive bladder cancer. Eur. Urol. 2011, 60, 484–492. [Google Scholar] [CrossRef]
  6. Halling, K.C.; King, W.; Sokolova, I.A.; Meyer, R.G.; Burkhardt, H.M.; Halling, A.C.; Cheville, J.C.; Sebo, T.J.; Ramakumar, S.; Stewart, C.S.; et al. A comparison of cytology and fluorescence in situ hybridization for the detection of urothelial carcinoma. J. Urol. 2000, 164, 1768–1775. [Google Scholar] [CrossRef]
  7. Pycha, A.; Mian, C.; Hofbauer, J.; Haitel, A.; Wiener, H.; Marberger, M. Does topical instillation therapy influence chromosomal aberrations in superficial bladder cancer? J. Urol. 1998, 159, 265–269. [Google Scholar] [CrossRef]
  8. Lin, T.; Liu, Z.; Liu, L.; Yang, L.; Han, P.; Zhang, P.; Wei, Q. Prospective evaluation of fluorescence in situ hybridization for diagnosing urothelial carcinoma. Oncol. Lett. 2017, 13, 3928–3934. [Google Scholar] [CrossRef]
  9. Lin, T.; Jin, H.; Gong, L.; Yu, R.; Sun, S.; Yang, L.; Zhang, P.; Han, P.; Cheng, J.; Liu, L.; et al. Surveillance of non-muscle invasive bladder cancer using fluorescence in situ hybridization: Protocol for a systematic review and meta-analysis. Medicine 2019, 98, e14573. [Google Scholar] [CrossRef]
  10. Stang, A. Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur. J. Epidemiol. 2010, 25, 603–605. [Google Scholar] [CrossRef]
  11. Bossuyt, P.M.; Reitsma, J.B.; Bruns, D.E.; Gatsonis, C.A.; Glasziou, P.P.; Irwig, L.; Lijmer, J.G.; Moher, D.; Rennie, D.; de Vet, H.C.; et al. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015, 351, h5527. [Google Scholar] [CrossRef] [PubMed]
  12. Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gøtzsche, P.C.; Ioannidis, J.P.; Clarke, M.; Devereaux, P.J.; Kleijnen, J.; Moher, D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Ann. Intern. Med. 2009, 151, W65–W94. [Google Scholar] [CrossRef] [PubMed]
  13. Rutter, C.M.; Gatsonis, C.A. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat. Med. 2001, 20, 2865–2884. [Google Scholar] [CrossRef] [PubMed]
  14. Reitsma, J.B.; Glas, A.S.; Rutjes, A.W.; Scholten, R.J.; Bossuyt, P.M.; Zwinderman, A.H. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J. Clin. Epidemiol. 2005, 58, 982–990. [Google Scholar] [CrossRef]
  15. Harbord, R.; Whiting, P. Metandi: Meta-analysis of Diagnostic Accuracy Using Hierarchical Logistic Regression. Stata J. 2009, 9, 211–229. [Google Scholar] [CrossRef]
  16. Jin, H.; Lin, T.; Hao, J.; Qiu, S.; Xu, H.; Yu, R.; Sun, S.; Zhang, P.; Liu, Z.; Yang, L.; et al. A comprehensive comparison of fluorescence in situ hybridization and cytology for the detection of upper urinary tract urothelial carcinoma: A systematic review and meta-analysis. Medicine 2018, 97, e13859. [Google Scholar] [CrossRef]
  17. Halling, K.C. Vysis UroVysion for the detection of urothelial carcinoma. Expert Rev. Mol. Diagn. 2003, 3, 507–519. [Google Scholar] [CrossRef]
  18. Galván, A.B.; Salido, M.; Espinet, B.; Placer, J.; Pijuan, L.; Juanpere, N.; Lloreta, J.; Solé, F.; Gelabert-Mas, A. A multicolor fluorescence in situ hybridization assay: A monitoring tool in the surveillance of patients with a history of non-muscle-invasive urothelial cell carcinoma: A prospective study. Cancer Cytopathol. 2011, 119, 395–403. [Google Scholar] [CrossRef]
  19. Yoder, B.J.; Skacel, M.; Hedgepeth, R.; Babineau, D.; Ulchaker, J.C.; Liou, L.S.; Brainard, J.A.; Biscotti, C.V.; Jones, J.S.; Tubbs, R.R. Reflex UroVysion testing of bladder cancer surveillance patients with equivocal or negative urine cytology: A prospective study with focus on the natural history of anticipatory positive findings. Am. J. Clin. Pathol. 2007, 127, 295–301. [Google Scholar] [CrossRef]
  20. Seideman, C.; Canter, D.; Kim, P.; Cordon, B.; Weizer, A.; Oliva, I.; Rao, J.; Inman, B.A.; Posch, M.; Herr, H.; et al. Multicenter evaluation of the role of UroVysion FISH assay in surveillance of patients with bladder cancer: Does FISH positivity anticipate recurrence? World J. Urol. 2015, 33, 1309–1313. [Google Scholar] [CrossRef]
  21. Kamat, A.M.; Willis, D.L.; Dickstein, R.J.; Anderson, R.; Nogueras-González, G.; Katz, R.L.; Wu, X.; Barton Grossman, H.; Dinney, C.P. Novel fluorescence in situ hybridization-based definition of bacille Calmette-Guérin (BCG) failure for use in enhancing recruitment into clinical trials of intravesical therapies. BJU Int. 2016, 117, 754–760. [Google Scholar] [CrossRef] [PubMed]
  22. Bollmann, M.; Heller, H.; Bankfalvi, A.; Griefingholt, H.; Bollmann, R. Quantitative molecular urinary cytology by fluorescence in situ hybridization: A tool for tailoring surveillance of patients with superficial bladder cancer? BJU Int. 2005, 95, 1219–1225. [Google Scholar] [CrossRef] [PubMed]
  23. Mian, C.; Lodde, M.; Comploj, E.; Lusuardi, L.; Palermo, S.; Mian, M.; Maier, K.; Pycha, A. Multiprobe fluorescence in situ hybridisation: Prognostic perspectives in superficial bladder cancer. J. Clin. Pathol 2006, 59, 984–987. [Google Scholar] [CrossRef] [PubMed]
  24. Mengual, L.; Marin-Aguilera, M.; Ribal, M.J.; Burset, M.; Villavicencio, H.; Oliver, A.; Alcaraz, A. Clinical utility of fluorescent in situ hybridization for the surveillance of bladder cancer patients treated with bacillus Calmette-Guerin therapy. Eur. Urol. 2007, 52, 752–759. [Google Scholar] [CrossRef] [PubMed]
  25. Gofrit, O.N.; Zorn, K.C.; Silvestre, J.; Shalhav, A.L.; Zagaja, G.P.; Msezane, L.P.; Steinberg, G.D. The predictive value of multi-targeted fluorescent in-situ hybridization in patients with history of bladder cancer. Urol. Oncol. 2008, 26, 246–249. [Google Scholar] [CrossRef]
  26. Kipp, B.R.; Tanasescu, M.; Else, T.A.; Bryant, S.C.; Jeffrey Karnes, R.; Sebo, T.J.; Halling, K.C. Quantitative fluorescence in situ hybridization and its ability to predict bladder cancer recurrence and progression to muscle-invasive bladder cancer. J. Mol. Diagn. 2009, 11, 148–154. [Google Scholar] [CrossRef]
  27. Savic, S.; Zlobec, I.; Thalmann, G.N.; Engeler, D.; Schmauss, M.; Lehmann, K.; Mattarelli, G.; Eichenberger, T.; Dalquen, P.; Spieler, P.; et al. The prognostic value of cytology and fluorescence in situ hybridization in the follow-up of nonmuscle-invasive bladder cancer after intravesical Bacillus Calmette-Guerin therapy. Int. J. Cancer 2009, 124, 2899–2904. [Google Scholar] [CrossRef]
  28. Maffezzini, M.; Campodonico, F.; Capponi, G.; Canepa, G.; Casazza, S.; Bandelloni, R.; Tamagno, S.; Puntoni, M. Prognostic significance of fluorescent in situ hybridisation in the follow-up of non-muscle-invasive bladder cancer. Anticancer Res. 2010, 30, 4761–4765. [Google Scholar]
  29. Huang, J.W.; Mu, J.G.; Li, Y.W.; Gan, X.G.; Song, L.J.; Gu, B.J.; Fu, Q.; Xu, Y.M.; An, R.H. The utility of fluorescence in situ hybridization for diagnosis and surveillance of bladder urothelial carcinoma. Urol. J. 2014, 11, 1974–1979. [Google Scholar]
  30. Todenhofer, T.; Hennenlotter, J.; Guttenberg, P.; Mohrhardt, S.; Kuehs, U.; Esser, M.; Aufderklamm, S.; Bier, S.; Harland, N.; Rausch, S.; et al. Prognostic relevance of positive urine markers in patients with negative cystoscopy during surveillance of bladder cancer. BMC Cancer 2015, 15, 155. [Google Scholar] [CrossRef]
  31. Liem, E.; Baard, J.; Cauberg, E.C.C.; Bus, M.T.J.; de Bruin, D.M.; Laguna Pes, M.P.; de la Rosette, J.; de Reijke, T.M. Fluorescence in situ hybridization as prognostic predictor of tumor recurrence during treatment with Bacillus Calmette-Guerin therapy for intermediate- and high-risk non-muscle-invasive bladder cancer. Med. Oncol. 2017, 34, 172. [Google Scholar] [CrossRef] [PubMed]
  32. Lotan, Y.; Inman, B.A.; Davis, L.G.; Kassouf, W.; Messing, E.; Daneshmand, S.; Canter, D.; Marble, H.T.; Joseph, A.M.; Jewell, S.; et al. Evaluation of the Fluorescence In Situ Hybridization Test to Predict Recurrence and/or Progression of Disease after bacillus Calmette-Guerin for Primary High Grade Nonmuscle Invasive Bladder Cancer: Results from a Prospective Multicenter Trial. J. Urol. 2019, 202, 920–926. [Google Scholar] [CrossRef] [PubMed]
  33. Witjes, J.A. Follow-up in non-muscle invasive bladder cancer: Facts and future. World J. Urol. 2021, 39, 4047–4053. [Google Scholar] [CrossRef] [PubMed]
  34. Karaoglu, I.; van der Heijden, A.G.; Witjes, J.A. The role of urine markers, white light cystoscopy and fluorescence cystoscopy in recurrence, progression and follow-up of non-muscle invasive bladder cancer. World J. Urol. 2014, 32, 651–659. [Google Scholar] [CrossRef]
  35. Bhat, A.; Kwon, D.; Soodana-Prakash, N.; Mouzannar, A.; Punnen, S.; Gonzalgo, M.L.; Parekh, D.J.; Ritch, C.R. Surveillance Intensity in Intermediate Risk, Nonmuscle Invasive Bladder Cancer: Revisiting the Optimal Timing and Frequency of Cystoscopy. J. Urol. 2021, 206, 22–28. [Google Scholar] [CrossRef] [PubMed]
  36. Mossanen, M.; Wang, Y.; Szymaniak, J.; Tan, W.S.; Huynh, M.J.; Preston, M.A.; Trinh, Q.D.; Sonpavde, G.; Kibel, A.S.; Chang, S.L. Evaluating the cost of surveillance for non-muscle-invasive bladder cancer: An analysis based on risk categories. World J. Urol. 2019, 37, 2059–2065. [Google Scholar] [CrossRef]
  37. Xie, Q.; Huang, Z.; Zhu, Z.; Zheng, X.; Liu, J.; Zhang, M.; Zhang, Y. Diagnostic Value of Urine Cytology in Bladder Cancer. A Meta-Analysis. Anal. Quant. Cytopathol. Histopathol. 2016, 38, 38–44. [Google Scholar]
  38. Gontero, P.; Montanari, E.; Roupret, M.; Longo, F.; Stockley, J.; Kennedy, A.; Rodriguez, O.; McCracken, S.R.C.; Dudderidge, T.; Sieverink, C.; et al. Comparison of the performances of the ADXBLADDER test and urinary cytology in the follow-up of non-muscle-invasive bladder cancer: A blinded prospective multicentric study. BJU Int. 2021, 127, 198–204. [Google Scholar] [CrossRef]
  39. Cancel-Tassin, G.; Roupret, M.; Pinar, U.; Gaffory, C.; Vanie, F.; Ondet, V.; Compérat, E.; Cussenot, O. Assessment of Xpert Bladder Cancer Monitor test performance for the detection of recurrence during non-muscle invasive bladder cancer follow-up. World J. Urol. 2021, 39, 3329–3335. [Google Scholar] [CrossRef]
  40. Cowan, B.; Klein, E.; Jansz, K.; Westenfelder, K.; Bradford, T.; Peterson, C.; Scherr, D.; Karsh, L.I.; Egerdie, B.; Witjes, A.; et al. Longitudinal follow-up and performance validation of an mRNA-based urine test (Xpert® Bladder Cancer Monitor) for surveillance in patients with non-muscle-invasive bladder cancer. BJU Int. 2021, 128, 713–721. [Google Scholar] [CrossRef]
  41. Moonen, P.M.; Merkx, G.F.; Peelen, P.; Karthaus, H.F.; Smeets, D.F.; Witjes, J.A. UroVysion compared with cytology and quantitative cytology in the surveillance of non-muscle-invasive bladder cancer. Eur. Urol. 2007, 51, 1275–1280; discussion 1280. [Google Scholar] [CrossRef] [PubMed]
  42. Woldu, S.L.; Bagrodia, A.; Lotan, Y. Guideline of guidelines: Non-muscle-invasive bladder cancer. BJU Int. 2017, 119, 371–380. [Google Scholar] [CrossRef] [PubMed]
  43. Lokeshwar, V.B.; Habuchi, T.; Grossman, H.B.; Murphy, W.M.; Hautmann, S.H.; Hemstreet, G.P., 3rd; Bono, A.V.; Getzenberg, R.H.; Goebell, P.; Schmitz-Dräger, B.J.; et al. Bladder tumor markers beyond cytology: International Consensus Panel on bladder tumor markers. Urology 2005, 66 (Suppl. S1), 35–63. [Google Scholar] [CrossRef] [PubMed]
  44. Faiena, I.; Rosser, C.J.; Chamie, K.; Furuya, H. Diagnostic biomarkers in non-muscle invasive bladder cancer. World J. Urol. 2019, 37, 2009–2016. [Google Scholar] [CrossRef]
  45. Zhou, L.; Yang, K.; Li, X.; Ding, Y.; Mu, D.; Li, H.; Yan, Y.; Li, J.; Wang, D.; Li, W.; et al. Application of fluorescence in situ hybridization in the detection of bladder transitional-cell carcinoma: A multi-center clinical study based on Chinese population. Asian J. Urol. 2019, 6, 114–121. [Google Scholar] [CrossRef]
  46. Sarosdy, M.F.; Schellhammer, P.; Bokinsky, G.; Kahn, P.; Chao, R.; Yore, L.; Zadra, J.; Burzon, D.; Osher, G.; Bridge, J.A.; et al. Clinical evaluation of a multi-target fluorescent in situ hybridization assay for detection of bladder cancer. J. Urol 2002, 168, 1950–1954. [Google Scholar] [CrossRef]
  47. Novara, G.; Catto, J.W.; Wilson, T.; Annerstedt, M.; Chan, K.; Murphy, D.G.; Motttrie, A.; Peabody, J.O.; Skinner, E.C.; Wiklund, P.N.; et al. Systematic review and cumulative analysis of perioperative outcomes and complications after robot-assisted radical cystectomy. Eur. Urol. 2015, 67, 376–401. [Google Scholar] [CrossRef]
  48. De Nunzio, C.; Cicione, A.; Leonardo, F.; Rondoni, M.; Franco, G.; Cantiani, A.; Tubaro, A.; Leonardo, C. Extraperitoneal radical cystectomy and ureterocutaneostomy in octogenarians. Int. Urol. Nephrol. 2011, 43, 663–667. [Google Scholar] [CrossRef]
  49. De Nunzio, C.; Cicione, A.; Izquierdo, L.; Lombardo, R.; Tema, G.; Lotrecchiano, G.; Minervini, A.; Simone, G.; Cindolo, L.; D’Orta, C.; et al. Multicenter Analysis of Postoperative Complications in Octogenarians After Radical Cystectomy and Ureterocutaneostomy: The Role of the Frailty Index. Clin. Genitourin. Cancer 2019, 17, 402–407. [Google Scholar] [CrossRef]
  50. Cicione, A.; Manno, S.; Ucciero, G.; Cantiello, F.; Damiano, R.; Lima, E.; Posti, A.; Balloni, F.; De Nunzio, C. A larger detrusor wall thickness increases the risk of non muscle invasive bladder cancer recurrence and progression: Result from a multicenter observational study. Ital. J. Urol. Nephrol. 2018, 70, 310–318. [Google Scholar] [CrossRef]
  51. Liem, E.; Oddens, J.R.; Vernooij, R.W.M.; Li, R.; Kamat, A.; Dinney, C.P.; Mengual, L.; Alcaraz, A.; Izquierdo, L.; Savic, S.; et al. The Role of Fluorescence In Situ Hybridization for Predicting Recurrence after Adjuvant bacillus Calmette-Guerin in Patients with Intermediate and High Risk Nonmuscle Invasive Bladder Cancer: A Systematic Review and Meta-Analysis of Individual Patient Data. J. Urol 2020, 203, 283–291. [Google Scholar] [CrossRef] [PubMed]
  52. Bao, Y.; Tu, X.; Chang, T.; Qiu, S.; Yang, L.; Geng, J.; Quan, L.; Wei, Q. The role of fluorescence in situ hybridization to predict patient response to intravesical Bacillus Calmette-Guérin therapy for bladder cancer: A diagnostic meta-analysis and systematic review. Medicine 2018, 97, e12227. [Google Scholar] [CrossRef] [PubMed]
  53. Dunn, G.P.; Bruce, A.T.; Ikeda, H.; Old, L.J.; Schreiber, R.D. Cancer immunoediting: From immunosurveillance to tumor escape. Nat. Immunol. 2002, 3, 991–998. [Google Scholar] [CrossRef] [PubMed]
Figure 1. PRISMA 2009 flow diagram of the research.
Figure 1. PRISMA 2009 flow diagram of the research.
Diagnostics 12 02005 g001
Figure 2. Overall quality assessment on the risk of bias and applicability concerns of the included studies using QUADAS-2 tool. Proportion of studies with high, unclear, and low risk of bias, and proportion of studies with high, unclear, and low concerns regarding applicability [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].
Figure 2. Overall quality assessment on the risk of bias and applicability concerns of the included studies using QUADAS-2 tool. Proportion of studies with high, unclear, and low risk of bias, and proportion of studies with high, unclear, and low concerns regarding applicability [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].
Diagnostics 12 02005 g002
Figure 3. Paired forest plots of pooled sensitivity and specificity of FISH in surveillance of NMIBC recurrence. The point estimates of sensitivity and specificity from each study are indicate as squares and 95% confidence interval are shown with horizontal bars. (t2: Lotan et al.–2, Liem et al.–2, Mengual et al.–2; t1: Lotan et al.–1, Liem et al.–1, Savic et al.–2; t0: Lotan et al.–0, Liem et al.–0, Savic et al.–1, Mengual et al.–1) [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].
Figure 3. Paired forest plots of pooled sensitivity and specificity of FISH in surveillance of NMIBC recurrence. The point estimates of sensitivity and specificity from each study are indicate as squares and 95% confidence interval are shown with horizontal bars. (t2: Lotan et al.–2, Liem et al.–2, Mengual et al.–2; t1: Lotan et al.–1, Liem et al.–1, Savic et al.–2; t0: Lotan et al.–0, Liem et al.–0, Savic et al.–1, Mengual et al.–1) [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].
Diagnostics 12 02005 g003
Figure 4. Positive likelihood ratio (PLR) and the negative likelihood ratio (NLR) of FISH in surveillance of NMIBC recurrence. (t2: Lotan et al.-2, Liem et al.-2, Mengual et al.-2; t1: Lotan et al.-1, Liem et al.-1, Savic et al.-2; t0: Lotan et al.-0, Liem et al.-0, Savic et al.-1, Mengual et al.-1) [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].
Figure 4. Positive likelihood ratio (PLR) and the negative likelihood ratio (NLR) of FISH in surveillance of NMIBC recurrence. (t2: Lotan et al.-2, Liem et al.-2, Mengual et al.-2; t1: Lotan et al.-1, Liem et al.-1, Savic et al.-2; t0: Lotan et al.-0, Liem et al.-0, Savic et al.-1, Mengual et al.-1) [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].
Diagnostics 12 02005 g004
Figure 5. Summary receiver operating characteristic curve for the included studies of the research.
Figure 5. Summary receiver operating characteristic curve for the included studies of the research.
Diagnostics 12 02005 g005
Figure 6. (A) Paired forest plots of pooled sensitivity and specificity of the “UroVysion standard” studies. (B) Summary receiver operating characteristic (SROC) plot of the “UroVysion standard” studies. (C) Paired forest plots of pooled sensitivity and specificity of the studies without BCG treatment. (D) Summary receiver operating characteristic (SROC) plot of the studies without BCG treatment [18,19,21,22,23,25,26,27,28,29,30,31,32].
Figure 6. (A) Paired forest plots of pooled sensitivity and specificity of the “UroVysion standard” studies. (B) Summary receiver operating characteristic (SROC) plot of the “UroVysion standard” studies. (C) Paired forest plots of pooled sensitivity and specificity of the studies without BCG treatment. (D) Summary receiver operating characteristic (SROC) plot of the studies without BCG treatment [18,19,21,22,23,25,26,27,28,29,30,31,32].
Diagnostics 12 02005 g006
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Zheng, W.; Lin, T.; Chen, Z.; Cao, D.; Bao, Y.; Zhang, P.; Yang, L.; Wei, Q. The Role of Fluorescence In Situ Hybridization in the Surveillance of Non-Muscle Invasive Bladder Cancer: An Updated Systematic Review and Meta-Analysis. Diagnostics 2022, 12, 2005. https://doi.org/10.3390/diagnostics12082005

AMA Style

Zheng W, Lin T, Chen Z, Cao D, Bao Y, Zhang P, Yang L, Wei Q. The Role of Fluorescence In Situ Hybridization in the Surveillance of Non-Muscle Invasive Bladder Cancer: An Updated Systematic Review and Meta-Analysis. Diagnostics. 2022; 12(8):2005. https://doi.org/10.3390/diagnostics12082005

Chicago/Turabian Style

Zheng, Weitao, Tianhai Lin, Zeyu Chen, Dehong Cao, Yige Bao, Peng Zhang, Lu Yang, and Qiang Wei. 2022. "The Role of Fluorescence In Situ Hybridization in the Surveillance of Non-Muscle Invasive Bladder Cancer: An Updated Systematic Review and Meta-Analysis" Diagnostics 12, no. 8: 2005. https://doi.org/10.3390/diagnostics12082005

APA Style

Zheng, W., Lin, T., Chen, Z., Cao, D., Bao, Y., Zhang, P., Yang, L., & Wei, Q. (2022). The Role of Fluorescence In Situ Hybridization in the Surveillance of Non-Muscle Invasive Bladder Cancer: An Updated Systematic Review and Meta-Analysis. Diagnostics, 12(8), 2005. https://doi.org/10.3390/diagnostics12082005

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop