Next Article in Journal
Cardiac Magnetic Resonance in Hypertensive Heart Disease: Time for a New Chapter
Next Article in Special Issue
Vision-Transformer-Based Transfer Learning for Mammogram Classification
Previous Article in Journal
Measuring Impairments of Mentalization with the 15-Item Mentalization Questionnaire (MZQ) and Introducing the MZQ-6 Short Scale: Reliability, Validity and Norm Values Based on a Representative Sample of the German Population
Previous Article in Special Issue
The Frequency and Causes of Not-Detected Breast Malignancy in Dynamic Contrast-Enhanced MRI
 
 
Correction published on 12 May 2023, see Diagnostics 2023, 13(10), 1713.
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessment of Enhancement Kinetics Improves the Specificity of Abbreviated Breast MRI: Performance in an Enriched Cohort

Department of Radiology and Center for Imaging Science, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea
*
Author to whom correspondence should be addressed.
Diagnostics 2023, 13(1), 136; https://doi.org/10.3390/diagnostics13010136
Submission received: 19 November 2022 / Revised: 28 December 2022 / Accepted: 29 December 2022 / Published: 30 December 2022 / Corrected: 12 May 2023
(This article belongs to the Special Issue Breast Cancer Imaging: Successes and Challenges)

Abstract

:
Objective: To investigate the added value of kinetic information for breast lesion evaluation on abbreviated breast MRI (AB-MRI). Methods: This retrospective study analyzed 207 breast lesions with Breast Imaging Reporting and Data System categories 3, 4, or 5 on AB-MRI in 198 consecutive patients who had breast MRI for screening after breast cancer surgery between January 2017 and December 2019. All lesions were pathologically confirmed or stable on follow-up images for 2 years or more. Kinetic information of the lesions regarding the degree and rate of enhancement on the first post-contrast-enhanced image and the enhancement curve type from two post-contrast-enhanced images were analyzed on a commercially available computer-assisted diagnosis system. The diagnostic performances of AB-MRI with morphological analysis alone and with the addition of kinetic information were compared using the McNemar test. Results: Of 207 lesions, 59 (28.5%) were malignant and 148 (71.5%) were benign. The addition of an enhancement degree of ≥90% to the morphological analysis significantly increased the specificity of AB-MRI (29.7% vs. 52.7%, p < 0.001) without significantly reducing the sensitivity (94.9% vs. 89.8%, p = 0.083) compared to morphological analysis alone. Unnecessary biopsy could have been avoided in 34 benign lesions, although three malignant lesions could have been missed. For detecting invasive cancer, adding an enhancement degree ≥107% to the morphological analysis significantly increased the specificity (26.5% vs. 57.6%, p < 0.001) without significantly decreasing the sensitivity (94.6% vs. 86.5%, p = 0.083). Conclusion: Adding the degree of enhancement on the first post-contrast-enhanced image to the morphological analysis resulted in higher AB-MRI specificity without compromising its sensitivity.

Graphical Abstract

1. Introduction

Breast MRI is the most sensitive imaging modality for detecting breast cancer [1,2,3]. However, high costs and long scan and interpretation times are major drawbacks for the wide availability of breast MRI as a screening method [4]. To increase the availability of MRI, Fischer et al. [5] initially presented a new concept of first-pass MRI, which consists of two post-contrast sequences with a total examination time of 4–5 min in the case of normal findings. Subsequently, Kuhl et al. [6] proposed the concept of abbreviated MRI (AB-MRI), which consists of one pre- and one post-contrast acquisition and their derived images (first post-contrast-subtracted and maximum-intensity projection [MIP]). Following these initial studies, multiple studies have reported that the diagnostic accuracy of AB-MRI is equivalent to that of the conventional full diagnostic protocol of MRI (FDP-MRI), with shorter acquisition and reading times, thereby making it more cost-effective as a screening tool [7,8,9,10,11,12,13,14,15,16].
Although morphological analysis is the first step of breast MRI interpretation, kinetic information helps differentiating benign from malignant lesions [17,18,19]. Enhancement kinetics analysis of FDP-MRI is performed at two different phases: early phase enhancement describing the steepness of the first part of the kinetic curve, during the first 1-2 min after contrast injection and delayed phase enhancement describing the time–signal intensity curve, at 3 min or more after contrast injection [19]. Consequently, not acquiring delayed contrast-enhanced images on AB-MRI does not mean that kinetic information is not provided. AB-MRI obtains the most important early kinetic information, which is a key component of AB-MRI. As cancers enhance faster and stronger than normal tissue or benign lesions, early post-contrast imaging is best for detecting invasive breast cancer (IDC) and high-grade ductal carcinoma in situ (DCIS) [6,10,20]. Similarly, as the contrast between cancer and background parenchymal enhancement (BPE) is the highest during the early post-contrast phase, morphological characteristics could also be obtained more easily during this early phase [21,22,23]. Although identifying significant enhancements (i.e., enhancement beyond physiologic BPE) in the MIP images is the first step in interpreting AB-MRI based on the original concept, little has been reported regarding the added value of this early kinetic information to morphological analysis alone for differentiating benign from malignant breast lesions on AB-MRI.
In 2018, Comstock and Kuhl [24] suggested a standardized method for AB-MRI interpretation, which recommended whether a follow-up or biopsy should be performed based on the morphological features of the lesions. Therefore, the purpose of this study was to investigate whether adding kinetic information to morphological analysis improves the diagnostic performance of AB-MRI by using the interpretation guideline, and to suggest the best way to use the abbreviated protocol of MRI for breast cancer screening.

2. Materials and Methods

2.1. Patients

This retrospective study was approved by the Institutional Review Board of the Ethics Committee, which waived the requirement for informed consent. Our institution has used breast AB-MRI to screen women at all risk for breast cancer since September 2015, but most of them had a personal history of breast cancer. Postoperative screening MRI has not been routinely performed for all patients with a personal history of breast cancer. Some had dense breast, were diagnosed with breast cancer at <50 years of age, had high-risk factors (breast cancer gene (BRCA) mutation, family history of breast cancer), or had a history of interstitial mammoplasty, and underwent postoperative screening MRI. This was additionally performed at the request of patients or clinicians. From January 2017 to December 2019, a total of 2397 asymptomatic women with a personal history of breast cancer underwent 3164 consecutive AB-MRI scans to screen for breast cancer after surgery. Among them, 2912 AB-MRI of 2188 patients with Breast Imaging Reporting and Data System (BI-RADS) categories 1 or 2 were excluded. As we included only one AB-MRI per patient, which was the first AB-MRI with BI-RADS category 3 or higher during the period; an additional 43 follow-up AB-MRI were excluded. Finally, 209 AB-MRI of 209 patients with BI-RADS categories 3, 4, or 5 were considered eligible for our study. Of 209 patients, 14 had two lesions with BI-RADS category 3 or higher per AB-MRI. Among 223 lesions of 209 patients, 16 (nine extramammary lymph nodes, four MR negative lesions with final assessment based on their integration with other modalities, one typical diffuse treatment-related change, one with no available computer-assisted diagnosis (CAD) image, and one inadequate protocol) lesions were excluded. Finally, 207 lesions from 198 patients were included (Figure 1). All lesions were confirmed as benign or malignant by biopsy or were considered benign when the lesion showed benign findings and stability on follow-up imaging studies for at least 2 years. Of the 84 lesions that were confirmed by biopsy, 80 lesions showed corresponding lesions on US and had US-guided biopsy, one lesion showed suspicious calcifications in corresponding area and underwent stereotactic biopsy, and three lesions had MR-guided biopsy. Of the 198 patients, nine had two lesions with BI-RADS category 3 or higher; two had two benign lesions, four with one benign and one malignant lesion, and three had two malignant lesions. The results of other imaging studies, including ultrasound and mammography, were not considered. Of the 198 AB-MRI, 136 were the first screening round, while 62 were the subsequent screening round. The median interval between cancer surgery and the screening AB-MRI was 21.1 months (interquartile range, 13.8–31.9 months). The pathology and electronic medical records were reviewed.

2.2. Breast MRI Acquisition

All AB-MRI scans were performed using a 3.0- or 1.5-T Achieva scanner (Philips Medical Systems) with a dedicated bilateral phased-array breast coil. The protocol included a T2-weighted sequence, one pre-contrast sequence, and two post-contrast-enhanced sequences. The contrast-enhanced images were obtained after a 0.1-mmol/kg bolus injection of gadobutrol (Gadovist; Bayer Healthcare, Berlin, Germany), followed by a 20-mL saline flush. Scanning for post-contrast imaging began 30 s after the contrast injection, with a temporal resolution of 60 s. The first post-contrast sequence was obtained from 30 s to 90 s after the beginning of the scan, and the second post-contrast sequence was sequentially obtained from 90 s to 150 s. After image acquisition, the subtraction and bilateral axial and sagittal MIP images were reformatted. The parameters for a 3.0-T scanner were: repetition time (TR)/echo time (TE) (ms), 4.6/2.3; field of view, 32 × 32 cm; matrix size, 512 × 512; flip angle, 24°; 1.5 mm sections with no gap; those for a 1.5-T scanner were: TR/TE (ms), 6.5/2.5; field of view, 32 × 32 cm; matrix size, 376 × 374; flip angle, 12°; 1.5 mm sections with no gap. The total scan time was 10 min or less (Supplementary Table S1).

2.3. Morphological Analysis

The findings of the lesions on AB-MRIs were retrospectively reviewed by two breast radiologists (E.Y.K. and H.K., with 19 and 2 years of breast MRI experience, respectively). Discordant cases were discussed to reach a consensus. Morphological analysis was performed according to the suggested interpretation guidelines [24] and diagnosed as positive when biopsy was recommended and negative when follow-up was recommended or benign findings were suggested (Supplementary Figures S1 and S2). The lesions were divided into mass and non-mass enhancement. The margins (circumscribed vs. irregular/spiculated), T2 signal intensities (high vs. not high), internal enhancement patterns (homogeneous vs. heterogeneous), and presence of rim enhancement were analyzed for the masses, whereas the distributions (linear/segmental vs. focal/regional/multiple regions/diffuse) and internal enhancement patterns (homogeneous vs. heterogeneous/clumped/clustered ring) were analyzed for non-mass enhancement. The size of the enhancing lesion was defined as the largest diameter.

2.4. Kinetic Analysis

Kinetic information was retrospectively analyzed on a dedicated workstation using a CAD system (CadstreamTM v6.0, Merge Healthcare, Inc., Hartland, WI, USA). If pixel value increased above a user-defined minimum enhancement threshold on first post-contrast images compared to pre-contrast images, the pixel was automatically identified by color overlays on each MRI slice. We defined minimum enhancement threshold as 50% increase in enhancement. Kinetic analysis of the enhancing lesions detected by CAD was automatically performed by clicking on the lesion, without the need to directly draw regions of interest. The enhancement degree was defined as peak enhancement percentage on the first post-contrast images. The enhancement rate, defined as the signal change between the pre- and first post-contrast images, was categorized as slow (<50% increase), medium (50–100%), or rapid (>100%). The enhancement curve, determined by the signal change between the first and second post-contrast images, was categorized as persistent (>10% increase), plateau (≤10% increase or ≤10% decrease), or washout (>10% decrease). When two or more types of curve were mixed within the same lesion, the most suspicious curve type was recorded. For lesions that could not be detected by CAD, we manually identified a single voxel within each lesion in the area of the highest enhancement on the first post-contrast images. The kinetic information was automatically generated from this voxel.

2.5. Statistical Analysis

Continuous variables, including age, tumor size, and enhancement degree, were analyzed using Student t- and Wilcoxon rank sum tests. Categorical variables including family history of breast cancer, BRCA mutation, MRI features, enhancement rate, enhancement curve type, BI-RADS category, biopsy recommend by interpretation guideline, MRI magnetic field strength, and screening round were compared using Fisher exact and x2 tests. The diagnostic performances of morphological analysis following the interpretation guideline, kinetic information including enhancement degree, enhancement rate, enhancement curve type, and tumor size were calculated by measuring the area under the receiver operating characteristic (ROC) curve (AUC), which were compared using the DeLong test. After identifying parameters with significantly higher AUC compared to morphologic analysis alone, we defined their optimal cutoff values as the points that increased the specificity of the guideline without significantly sacrificing its sensitivity to continuous variables and applied the Youden index for categorical variables. The diagnostic performances of morphological analysis alone and morphology with kinetic information were compared using the McNemar test. Two-sided p < 0.05 was considered statistically significant. All statistical analyses were performed using SAS software (version 9.4; SAS Institute, Cary, NC, USA).

3. Results

3.1. Patient and Lesion Characteristics

The patient and lesion characteristics are summarized in Table 1. Of the 207 lesions, 59 (28.5%) were malignant (35 IDCs, 22 DCIS, and 2 metaplastic carcinomas) and 148 (71.5%) were benign (113 stable for ≥2 years on imaging follow-up; 10 fat necrosis; 8 fibroadenomas; 4 stromal fibrosis; 4 fibrocystic changes; 2 intraductal papillomas; 2 benign breast tissues; and 1 each of sclerosing adenosis, usual ductal hyperplasia, adenomyoepithelioma, cholesterol granuloma, and fibromatosis confirmed by biopsies). Both groups had more women without family history of breast cancer (88.1–91.2%) than women with family history of breast cancer (8.8–11.9%), and more women without BRCA mutation (81.4–87.8%) than with BRCA mutation (12.2–18.6%). The malignant lesions were significantly larger (p < 0.001). Mass was a more common lesion type in both benign and malignant lesions; however, non-mass enhancement was more common in malignant lesions than in benign lesions (30.5% vs. 17.6%, p = 0.040). In the morphological analysis, malignant masses more frequently showed non-circumscribed margins (p < 0.001), heterogeneous enhancement (p = 0.012), and rim enhancement (p = 0.014). Furthermore, non-mass lesions showed no significant differences between benign and malignant lesions in morphological analysis. In the kinetic analysis, malignant lesions showed a significantly higher degree of enhancement (p < 0.001) and more frequently showed rapid enhancement rates (p < 0.001) than the benign lesions. None of the malignant lesions showed slow enhancement rate. Regarding the enhancement curve type, malignant lesions were more likely to show washout type than benign lesions, whereas the latter were more likely to feature persistent lesions compared to malignant lesions (p < 0.001). Benign lesions were more likely to be assessed as BI-RADS category 3 than malignant lesions (77.0% vs. 15.3%, p < 0.001), while none of the benign lesions were assessed as BI-RADS category 4C and 5. According to the interpretation guideline, 70.3% of benign lesions were recommended to undergo biopsy while 94.9% malignant lesions were recommended for biopsy (p < 0.001). MRI magnetic field strength of 3.0-T (68.9–74.6%) were more frequent than 1.5-T (25.4–31.1%) for both groups, and most AB-MRI were the first screening round (66.2–71.2%).

3.2. Receiver Operating Characteristics Curve Analysis of Parameters for Differentiating Benign and Malignant Breast Lesions

The AUC values of the morphological analysis following AB-MRI interpretation guidelines, kinetic parameters including enhancement degree, enhancement rate, enhancement curve type, and size are shown in Table 2. For the detection of all malignancies including in situ carcinoma, the enhancement degree, enhancement curve type, and size showed significantly better AUC values compared to morphological analysis alone (0.72–0.74 vs. 0.62; p < 0.05 for all parameters). The ROC curves for each parameter are shown in Figure 2. For the detection of invasive cancers, enhancement degree and size showed significantly better AUC values compared to morphological analysis alone (0.72 for each vs. 0.61; p < 0.05 for both parameters). The ROC curves for each parameter are shown in Figure 3.

3.3. Optimal Cutoff Values of the Parameters

As shown in Table 2, the optimal cutoff values of enhancement degree that showed the highest specificity without significantly decreasing sensitivity were 90% for the detection of all malignancy and 107% for the detection of invasive cancer. The cutoff value of the enhancement curve type for the detection of all malignancies was defined as a plateau by the Youden index, which was the point that maximized the sum of sensitivity and specificity. Although a tumor size of 0.5 cm was the limit that significantly improved thw specificity of the guidelines for the detection of all malignancies, we were unable to set the cutoff value because the sensitivity was significantly decreased. The tumor size of 0.6 cm was selected as the cutoff for the detection of invasive cancers.

3.4. Diagnostic Performance of the Combined Morphology and Kinetic Information Compared to Morphological Analysis Alone

The results of a comparison of the diagnostic performances are shown in Table 3. AB-MRI with morphological analysis alone, following the interpretation guidelines, showed 94.9% sensitivity and 29.7% specificity. When only lesions with enhancement degree ≥90% among the biopsy-recommended lesions according to the interpretation guidelines undergo biopsy, the specificity of AB-MRI significantly increassed (52.7%, p < 0.001) without significantly reducing sensitivity (89.8%, p = 0.080). Thus, 34 benign lesions for which biopsy was recommended by morphological analysis alone would be classified as negative after considering the degree of enhancement (Figure 4). However, three malignant lesions, one DCIS and two IDCs, could have been missed (Figure 5). A slow enhancement rate was observed only in benign lesions; however, as the AUC value of enhancement rate was not significantly higher than the guideline, the addition of the enhancement rate to the morphological analysis was not analyzed. When the enhancement curve type (plateau or washout) was added to the morphological analysis, the specificity significantly increased to 79.1% (p < 0.001), but the sensitivity also significantly decreased to 66.1% (p < 0.001). Adding both enhancement curve type (plateau or washout) and enhancement degree (≥90%) also significantly increased in specificity to 80.4% (p < 0.001), but significantly decreased in sensitivity to 64.4% (p < 0.001).
For the detection of invasive cancer, AB-MRI following the interpretation guidelines showed 94.6% sensitivity and 26.5% specificity. When an enhancement degree of ≥107% was added to the morphological analysis, the specificity significantly increased (57.6%, p < 0.001), and the sensitivity was not significantly reduced (86.5%, p = 0.083). Adding size ≥0.6 cm to the morphological analysis also significantly increased the specificity (38.8%, p < 0.001) without a significant decrease in sensitivity (86.5%, p = 0.083). Adding both enhancement degree ≥107% and size ≥0.6 cm to the morphologic criteria significantly increased the specificity (63.5%, p < 0.001), but also significantly reduced the sensitivity (78.4%, p = 0.014).
In the subgroup analysis, according to the magnetic field strengths (1.5-T vs. 3.0-T), adding an enhancement degree of ≥90% to detect all malignancy or ≥107% for invasive cancer to the morphological analysis significantly increased the specificity in both 1.5-T and 3.0-T (p < 0.001). The sensitivity decreased only at 1.5-T, but not significantly (p = 0.083). Adding size ≥0.6 cm to the morphologic criteria significantly increased the specificity (37.8%, p < 0.001) at 3.0-T, without a significant decrease in sensitivity (85.2%, p = 0.157).
Finally, the only parameter that significantly improved specificity without significantly compromising sensitivity was the degree of enhancement (≥90% for all malignancy; ≥107% for invasive cancer) at both magnetic field strengths.

4. Discussion

Our study demonstrated that the use of the quantitative enhancement degree on first post-contrast images significantly improved the specificity of AB-MRI in differentiating benign from malignant breast lesions. We also proposed cutoff values of 90% for detecting all malignancy and 107% for invasive cancer, which significantly increased the specificity without sacrificing the sensitivity of AB-MRI. These results were consistent for both 1.5-T and 3.0-T MRI.
Kinetic information is a key component of the concept of AB-MRI, as identifying the significant enhancement on the MIP images is the first step in AB-MRI interpretation [6]. However, no evidence is available regarding the explicit use of kinetics in conjunction with the pure morphological assessment of AB-MRI. In practice, many factors may affect the final determination of the BI-RADS category, including radiologists’ experience, available previous imaging findings, patient history, and the subjective interpretation of kinetic information by visually assessing the dynamic images due to CAD system unavailability. Therefore, it is difficult to evaluate the pure added value of kinetic information compaed to the morphological analysis alone in the designed study. We used the suggested interpretation guideline of AB-MRI for objective morphological analysis alone, and no additional information was considered, although the guidelines actually recommend considering kinetic information if available. This study design might explain the low specificity of our study results compared with that of other reports on the performance of AB-MRI. In our study, the sensitivity and specificity of AB-MRI with morphological analysis alone were 94.9% and 29.7% for the detection of all malignancy and 94.6% and 26.5% for the detection of invasive cancer, respectively. Although the sensitivities were similar, the specificities were far lower than previous reports, with pooled values ranging from 81.8 to 100% and from 75.4 to 97.2%, respectively [21].
We set an optimal cutoff value that did not sacrifice sensitivity while significantly increasing specificity. Therefore, we focused on downgrading lesions that did not require subsequent biopsy. Improving the specificity has significant clinical implications, as the biopsy of many benign lesions places a high emotional burden on women and adds substantial additional costs, reducing its feasibility as a screening tool [25,26,27]. The use of quantitative kinetic information derived from AB-MRI may reduce the time and cost burdens by reducing false-positive and recall rates, as unnecessary biopsy could have been avoided in 34 benign lesions in our results if lesions with an enhancement degree <90% were considered benign. However, based on these criteria, three malignant lesions could have been missed, two of which were invasive. One IDC was a tumor that recurred at the previous operation site; the other was an increased mass compared to the previous MRI. Therefore, in addition to considering the kinetic information, the consideration of situations such as a newly developed mass compared to previous imaging studies is needed to maintain sensitivity.
In our study, we derived the enhancement curve type from two post-contrast images obtained 60 s and 120 s after contrast injection. Although few studies have reported AB-MRI with two post-contrast sequences, they showed that the diagnostic performance of the shortened protocol was similar to that for FDP-MRI, with a reduced scan time [28]. Our study results showed that kinetic curve type derived from first and second post-contrast images differed significantly between benign and malignant lesions, with malignant lesions showing more frequent washout type and benign lesions showing more persistent enhancement, suggesting that early kinetic information from only two post-contrast images at 60 s and 120 s was sufficient to assess the enhancement curve type of the lesion. However, there was no added value as this significantly decreased sensitivity at the cost of a significantly increased specificity. Since our results concluded that first post-contrast images are sufficient for differential diagnosis, we support the use of the currently more widely used AB-MRI protocol, which obtains only one post-contrast image.
Many studies have evaluated the kinetics of MRI, focusing on sequences with a temporal resolution of <10–20 s, such as ultrafast imaging [29,30,31]. However, these methods require additional scanning with a high-end MRI equipment. The main advantage of our results is that they can be quickly incorporated into current clinical practice by using commercially available CAD, without obtaining additional sequences.
We obtained consistent results with both 1.5-T and 3.0-T; thus, our results could be applied widely, regardless of the type of MRI magnetic field strengths. However, the increase in the specificity of combined diagnostic criteria of morphology and enhancement degree was higher at 1.5-T than at 3.0-T. As 3.0-T offers a higher signal-to-noise ratio, better image quality, and diagnostic confidence than 1.5-T, using CAD-assisted enhancement degree information might be more helpful for differential diagnosis at 1.5-T [32,33,34].
This study has several limitations. First, this was a retrospective study in an enriched cohort of patients who underwent breast cancer surgery and included only lesions with BI-RADS category 3, 4, and 5. As we did not include MRI with negative results, the absolute values of diagnostic performance might not be representative and not reflect the realities of clinical practice. However, we intended to minimize the interobserver variability of morphological analysis by using the interpretation guidelines and focused on the added value of kinetic information. Second, because we manually identified a single voxel in the area of highest enhancement within the lesions that could not be detected by CAD, t hismight not have represented the characteristics of the entire lesion, as malignant masses are more heterogeneous than benign masses [35]. However, the BI-RADS guidelines suggest analyzing the lesion’s most suspicious portion to assess kinetic information. Thus, we considered the area of highest enhancement on the first post-contrast-enhanced image to be the most suspicious portion of a lesion. Third, the sizes of benign and malignant lesions differed significantly, with malignant lesions being larger in diameter. This might be because some cases of multiple stippled enhancing nodules that could be considered as BPE were categorized as BI-RADS category 3 and included in our study. However, even if the lesion is small in size, it is less likely to affect the enhancement degree because we analyzed the quantitative voxel value of the enhancement degree of the most suspicious portion of the lesion.
In conclusion, we suggest a method to best utilize the kinetic information provided by the abbreviated protocol, as follows. The combination of the enhancement degree on the first post-contrast-enhanced image (≥90% for all malignancy and ≥107% for invasive cancer) with morphological analysis can improve the specificity of AB-MRI in differentiating malignant and benign breast lesions without compromising sensitivity. We suggest that a single post-contrast acquisition is sufficient, as additional post-contrast imaging to observe kinetic curve type did not further improve the differential diagnosis over the simple assessment of the degree of enhancement.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics13010136/s1, Table S1: Abbreviated protocol of screening breast MRI. Figure S1: Interpretation guideline of mass on AB-MRI. Figure S2: Interpretation guideline of non-mass enhancement on AB-MRI.

Author Contributions

Conceptualization: H.K. and E.Y.K.; data curation: H.K., E.Y.K., M.K.K., J.S.C., E.S.K. and B.-K.H.; formal analysis: H.K., E.Y.K., J.S.C., E.S.K. and B.-K.H.; investigation: H.K., E.Y.K. and J.S.C.; Methodology: H.K., E.Y.K. and E.S.K.; project administration: H.K. and E.Y.K.; resources: H.K., E.Y.K., J.S.C., E.S.K. and B.-K.H.; software: H.K. and E.Y.K.; supervision: E.Y.K.; validation: H.K., E.Y.K., J.S.C., E.S.K. and B.-K.H.; Visualization: H.K. and E.Y.K.; writing: original draft: H.K. and E.Y.K.; writing: review and editing: H.K., E.Y.K. and K.E.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was approved by the Institutional Review Board in Samsung Medical Center (IRB No. 2020-03-191-001).

Informed Consent Statement

Patient consent was waived due to retrospective study design.

Data Availability Statement

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bakker, M.F.; de Lange, S.V.; Pijnappel, R.M.; Mann, R.M.; Peeters, P.H.M.; Monninkhof, E.M.; Emaus, M.J.; Loo, C.E.; Bisschops, R.H.C.; Lobbes, M.B.I.; et al. Supplemental MRI Screening for Women with Extremely Dense Breast Tissue. N. Engl. J. Med. 2019, 381, 2091–2102. [Google Scholar] [CrossRef] [PubMed]
  2. Cho, N.; Han, W.; Han, B.K.; Bae, M.S.; Ko, E.S.; Nam, S.J.; Chae, E.Y.; Lee, J.W.; Kim, S.H.; Kang, B.J.; et al. Breast Cancer Screening with Mammography Plus Ultrasonography or Magnetic Resonance Imaging in Women 50 Years or Younger at Diagnosis and Treated with Breast Conservation Therapy. JAMA Oncol. 2017, 3, 1495–1502. [Google Scholar] [CrossRef]
  3. Kriege, M.; Brekelmans, C.T.; Boetes, C.; Besnard, P.E.; Zonderland, H.M.; Obdeijn, I.M.; Manoliu, R.A.; Kok, T.; Peterse, H.; Tilanus-Linthorst, M.M.; et al. Efficacy of MRI and mammography for breast-cancer screening in women with a familial or genetic predisposition. N. Engl. J. Med. 2004, 351, 427–437. [Google Scholar] [CrossRef] [PubMed]
  4. Mann, R.M.; Cho, N.; Moy, L. Breast MRI: State of the Art. Radiology 2019, 292, 520–536. [Google Scholar] [CrossRef] [PubMed]
  5. Fischer, U.; Korthauer, A.; Baum, F.; Luftner-Nagel, S.; Heyden, D.; Marten-Engelke, K. Short first-pass MRI of the breast. Acta Radiol. 2012, 53, 267–269. [Google Scholar] [CrossRef]
  6. Kuhl, C.K.; Schrading, S.; Strobel, K.; Schild, H.H.; Hilgers, R.D.; Bieling, H.B. Abbreviated breast magnetic resonance imaging (MRI): First postcontrast subtracted images and maximum-intensity projection-a novel approach to breast cancer screening with MRI. J. Clin. Oncol. 2014, 32, 2304–2310. [Google Scholar] [CrossRef]
  7. Park, K.W.; Han, S.B.; Han, B.K.; Ko, E.S.; Choi, J.S.; Rhee, S.J.; Ko, E.Y. MRI surveillance for women with a personal history of breast cancer: Comparison between abbreviated and full diagnostic protocol. Br. J. Radiol. 2020, 93, 20190733. [Google Scholar] [CrossRef]
  8. Kwon, M.R.; Ko, E.Y.; Han, B.K.; Ko, E.S.; Choi, J.S.; Park, K.W. Diagnostic performance of abbreviated breast MRI for screening of women with previously treated breast cancer. Medicine 2020, 99, e19676. [Google Scholar] [CrossRef]
  9. Moschetta, M.; Telegrafo, M.; Rella, L.; Stabile Ianora, A.A.; Angelelli, G. Abbreviated Combined MR Protocol: A New Faster Strategy for Characterizing Breast Lesions. Clin. Breast Cancer 2016, 16, 207–211. [Google Scholar] [CrossRef]
  10. Heacock, L.; Melsaether, A.N.; Heller, S.L.; Gao, Y.; Pysarenko, K.M.; Babb, J.S.; Kim, S.G.; Moy, L. Evaluation of a known breast cancer using an abbreviated breast MRI protocol: Correlation of imaging characteristics and pathology with lesion detection and conspicuity. Eur. J. Radiol. 2016, 85, 815–823. [Google Scholar] [CrossRef]
  11. Harvey, S.C.; Di Carlo, P.A.; Lee, B.; Obadina, E.; Sippo, D.; Mullen, L. An Abbreviated Protocol for High-Risk Screening Breast MRI Saves Time and Resources. J. Am. Coll. Radiol. 2016, 13, R74–R80. [Google Scholar] [CrossRef]
  12. Mango, V.L.; Morris, E.A.; David Dershaw, D.; Abramson, A.; Fry, C.; Moskowitz, C.S.; Hughes, M.; Kaplan, J.; Jochelson, M.S. Abbreviated protocol for breast MRI: Are multiple sequences needed for cancer detection? Eur. J. Radiol. 2015, 84, 65–70. [Google Scholar] [CrossRef]
  13. Grimm, L.J.; Soo, M.S.; Yoon, S.; Kim, C.; Ghate, S.V.; Johnson, K.S. Abbreviated screening protocol for breast MRI: A feasibility study. Acad. Radiol. 2015, 22, 1157–1162. [Google Scholar] [CrossRef]
  14. Kim, E.S.; Cho, N.; Kim, S.Y.; Kwon, B.R.; Yi, A.; Ha, S.M.; Lee, S.H.; Chang, J.M.; Moon, W.K. Comparison of Abbreviated MRI and Full Diagnostic MRI in Distinguishing between Benign and Malignant Lesions Detected by Breast MRI: A Multireader Study. Korean J. Radiol. 2021, 22, 297–307. [Google Scholar] [CrossRef]
  15. Baxter, G.C.; Selamoglu, A.; Mackay, J.W.; Bond, S.; Gray, E.; Gilbert, F.J. A meta-analysis comparing the diagnostic performance of abbreviated MRI and a full diagnostic protocol in breast cancer. Clin. Radiol. 2021, 76, 154.e23–154.e32. [Google Scholar] [CrossRef]
  16. Kim, S.Y.; Cho, N.; Hong, H.; Lee, Y.; Yoen, H.; Kim, Y.S.; Park, A.R.; Ha, S.M.; Lee, S.H.; Chang, J.M.; et al. Abbreviated Screening MRI for Women with a History of Breast Cancer: Comparison with Full-Protocol Breast MRI. Radiology 2022, 305, 36–45. [Google Scholar] [CrossRef]
  17. Szabó, B.K.; Aspelin, P.; Wiberg, M.K.; Boné, B. Dynamic MR imaging of the breast. Analysis of kinetic and morphologic diagnostic criteria. Acta Radiol. 2003, 44, 379–386. [Google Scholar] [CrossRef]
  18. Kuhl, C.K.; Mielcareck, P.; Klaschik, S.; Leutner, C.; Wardelmann, E.; Gieseke, J.; Schild, H.H. Dynamic breast MR imaging: Are signal intensity time course data useful for differential diagnosis of enhancing lesions? Radiology 1999, 211, 101–110. [Google Scholar] [CrossRef]
  19. Kuhl, C. The current status of breast MR imaging. Part I. Choice of technique, image interpretation, diagnostic accuracy, and transfer to clinical practice. Radiology 2007, 244, 356–378. [Google Scholar] [CrossRef]
  20. Heacock, L.; Lewin, A.A.; Gao, Y.; Babb, J.S.; Heller, S.L.; Melsaether, A.N.; Bagadiya, N.; Kim, S.G.; Moy, L. Feasibility analysis of early temporal kinetics as a surrogate marker for breast tumor type, grade, and aggressiveness. J. Magn. Reson. Imaging 2018, 47, 1692–1700. [Google Scholar] [CrossRef]
  21. Gao, Y.; Heller, S.L. Abbreviated and Ultrafast Breast MRI in Clinical Practice. Radiographics 2020, 40, 1507–1527. [Google Scholar] [CrossRef] [PubMed]
  22. Kim, S.Y.; Cho, N.; Choi, Y.; Shin, S.U.; Kim, E.S.; Lee, S.H.; Chang, J.M.; Moon, W.K. Ultrafast Dynamic Contrast-Enhanced Breast MRI: Lesion Conspicuity and Size Assessment according to Background Parenchymal Enhancement. Korean J. Radiol. 2020, 21, 561–571. [Google Scholar] [CrossRef] [PubMed]
  23. Uematsu, T.; Kasami, M.; Watanabe, J. Background enhancement of mammary glandular tissue on breast dynamic MRI: Imaging features and effect on assessment of breast cancer extent. Breast Cancer 2012, 19, 259–265. [Google Scholar] [CrossRef] [PubMed]
  24. Comstock, C.E.; Kuhl, C. Abbreviated MRI of the Breast: A Practical Guide; Georg Thieme Verlag KG: Stuttgart, Germany, 2018. [Google Scholar]
  25. Keyzer-Dekker, C.M.; De Vries, J.; van Esch, L.; Ernst, M.F.; Nieuwenhuijzen, G.A.; Roukema, J.A.; van der Steeg, A.F. Anxiety after an abnormal screening mammogram is a serious problem. Breast 2012, 21, 83–88. [Google Scholar] [CrossRef] [PubMed]
  26. van der Steeg, A.F.; Keyzer-Dekker, C.M.; De Vries, J.; Roukema, J.A. Effect of abnormal screening mammogram on quality of life. Br. J. Surg. 2011, 98, 537–542. [Google Scholar] [CrossRef]
  27. Tosteson, A.N.; Fryback, D.G.; Hammond, C.S.; Hanna, L.G.; Grove, M.R.; Brown, M.; Wang, Q.; Lindfors, K.; Pisano, E.D. Consequences of false-positive screening mammograms. JAMA Intern. Med. 2014, 174, 954–961. [Google Scholar] [CrossRef]
  28. Choudhery, S.; Chou, S.S.; Chang, K.; Kalpathy-Cramer, J.; Lehman, C.D. Kinetic Analysis of Lesions Identified on a Rapid Abridged Multiphase (RAMP) Breast MRI Protocol. Acad. Radiol. 2020, 27, 672–681. [Google Scholar] [CrossRef]
  29. Milon, A.; Vande Perre, S.; Poujol, J.; Trop, I.; Kermarrec, E.; Bekhouche, A.; Thomassin-Naggara, I. Abbreviated breast MRI combining FAST protocol and high temporal resolution (HTR) dynamic contrast enhanced (DCE) sequence. Eur. J. Radiol. 2019, 117, 199–208. [Google Scholar] [CrossRef]
  30. Goto, M.; Sakai, K.; Yokota, H.; Kiba, M.; Yoshida, M.; Imai, H.; Weiland, E.; Yokota, I.; Yamada, K. Diagnostic performance of initial enhancement analysis using ultra-fast dynamic contrast-enhanced MRI for breast lesions. Eur. Radiol. 2019, 29, 1164–1174. [Google Scholar] [CrossRef]
  31. Abe, H.; Mori, N.; Tsuchiya, K.; Schacht, D.V.; Pineda, F.D.; Jiang, Y.; Karczmar, G.S. Kinetic Analysis of Benign and Malignant Breast Lesions With Ultrafast Dynamic Contrast-Enhanced MRI: Comparison With Standard Kinetic Assessment. AJR Am. J. Roentgenol. 2016, 207, 1159–1166. [Google Scholar] [CrossRef]
  32. Schmitz, A.C.; Peters, N.H.; Veldhuis, W.B.; Gallardo, A.M.; van Diest, P.J.; Stapper, G.; van Hillegersberg, R.; Mali, W.P.; van den Bosch, M.A. Contrast-enhanced 3.0-T breast MRI for characterization of breast lesions: Increased specificity by using vascular maps. Eur. Radiol. 2008, 18, 355–364. [Google Scholar] [CrossRef]
  33. Kuhl, C.K. Breast MR imaging at 3T. Magn. Reson. Imaging Clin. N. Am. 2007, 15, 315–320. [Google Scholar] [CrossRef]
  34. Kuhl, C.K.; Jost, P.; Morakkabati, N.; Zivanovic, O.; Schild, H.H.; Gieseke, J. Contrast-enhanced MR imaging of the breast at 3.0 and 1.5 T in the same patients: Initial experience. Radiology 2006, 239, 666–676. [Google Scholar] [CrossRef]
  35. Shimauchi, A.; Abe, H.; Schacht, D.V.; Yulei, J.; Pineda, F.D.; Jansen, S.A.; Ganesh, R.; Newstead, G.M. Evaluation of Kinetic Entropy of Breast Masses Initially Found on MRI using Whole-lesion Curve Distribution Data: Comparison with the Standard Kinetic Analysis. Eur. Radiol. 2015, 25, 2470–2478. [Google Scholar] [CrossRef]
Figure 1. Flowchart of the study population and exclusion criteria. AB-MRI = abbreviated breast MRI, BI-RADS = Breast Imaging Reporting and Data System, CAD = computer-assisted diagnosis.
Figure 1. Flowchart of the study population and exclusion criteria. AB-MRI = abbreviated breast MRI, BI-RADS = Breast Imaging Reporting and Data System, CAD = computer-assisted diagnosis.
Diagnostics 13 00136 g001
Figure 2. ROC curves of the parameters for differentiating malignant from benign breast lesions. ROC = receiver operating characteristic.
Figure 2. ROC curves of the parameters for differentiating malignant from benign breast lesions. ROC = receiver operating characteristic.
Diagnostics 13 00136 g002
Figure 3. ROC curves of the parameters for detecting invasive cancer. ROC = receiver operating characteristic.
Figure 3. ROC curves of the parameters for detecting invasive cancer. ROC = receiver operating characteristic.
Diagnostics 13 00136 g003
Figure 4. Screening abbreviated breast MRI (AB-MRI) images of a 48-year-old woman with a history of right total mastectomy and left breast-conserving surgery due to bilateral breast cancer. Two years after surgery, pre-contrast and two post-contrast-enhanced axial images (a) show a 0.9 cm irregular enhancing mass (arrows) in the left breast at the 1 o’clock direction. The T2 signal intensity is not high (b). This lesion was assessed as BI-RADS 4B category in the original report. According to the interpretation guideline, it is classified as a suspicious lesion for which biopsy is recommended. However, kinetic analysis shows a low enhancement degree of 22% on the first post-contrast-enhanced image (c). This lesion was detected by second-look ultrasound (US) and confirmed as sclerosing adenosis by US-guided core biopsy. In conclusion, considering kinetic information could have avoided an unnecessary biopsy of this false-positive lesion.
Figure 4. Screening abbreviated breast MRI (AB-MRI) images of a 48-year-old woman with a history of right total mastectomy and left breast-conserving surgery due to bilateral breast cancer. Two years after surgery, pre-contrast and two post-contrast-enhanced axial images (a) show a 0.9 cm irregular enhancing mass (arrows) in the left breast at the 1 o’clock direction. The T2 signal intensity is not high (b). This lesion was assessed as BI-RADS 4B category in the original report. According to the interpretation guideline, it is classified as a suspicious lesion for which biopsy is recommended. However, kinetic analysis shows a low enhancement degree of 22% on the first post-contrast-enhanced image (c). This lesion was detected by second-look ultrasound (US) and confirmed as sclerosing adenosis by US-guided core biopsy. In conclusion, considering kinetic information could have avoided an unnecessary biopsy of this false-positive lesion.
Diagnostics 13 00136 g004
Figure 5. Screening abbreviated breast MRI (AB-MRI) images of a 67-year-old woman with a history of right breast-conserving surgery due to breast cancer. Four years after surgery, pre-contrast and two post-contrast-enhanced axial images (a) show a 1.1 cm irregular enhancing mass (arrows) in the right breast at the operation site. The T2 signal intensity is not high (b). This lesion was assessed as BI-RADS 4C category in the original report. According to the interpretation guideline, it is classified as a suspicious lesion for which biopsy is recommended. However, kinetic analysis shows a low enhancement degree of 68% on the first post-contrast-enhanced image (c). This lesion was detected by second-look ultrasound (US) and confirmed as invasive ductal carcinoma by US-guided core biopsy and subsequent surgery. This is a false-negative case when the enhancement degree on the first post-contrast-enhanced image is combined with morphological analysis.
Figure 5. Screening abbreviated breast MRI (AB-MRI) images of a 67-year-old woman with a history of right breast-conserving surgery due to breast cancer. Four years after surgery, pre-contrast and two post-contrast-enhanced axial images (a) show a 1.1 cm irregular enhancing mass (arrows) in the right breast at the operation site. The T2 signal intensity is not high (b). This lesion was assessed as BI-RADS 4C category in the original report. According to the interpretation guideline, it is classified as a suspicious lesion for which biopsy is recommended. However, kinetic analysis shows a low enhancement degree of 68% on the first post-contrast-enhanced image (c). This lesion was detected by second-look ultrasound (US) and confirmed as invasive ductal carcinoma by US-guided core biopsy and subsequent surgery. This is a false-negative case when the enhancement degree on the first post-contrast-enhanced image is combined with morphological analysis.
Diagnostics 13 00136 g005
Table 1. Patient and lesion characteristics of benign and malignant lesions.
Table 1. Patient and lesion characteristics of benign and malignant lesions.
CharacteristicsBenign (n = 148)Malignant (n = 59)p Value
Age (years) *49.1 ± 9.149.7 ± 7.90.561
Family history of breast cancer 0.498
  No135 (91.2)52 (88.1)
  Yes13 (8.8)7 (11.9)
BRCA1 or BRCA2 mutation 0.225
  Negative130 (87.8)48 (81.4)
  Positive18 (12.2)11 (18.6)
Tumor size (cm) †0.7 (0.3-10)1.3 (0.3-6.2)<0.001
Lesion type 0.040
  Mass122 (82.4)41 (69.5)
  NME26 (17.6)18 (30.5)
Mass margin <0.001
  Circumscribed64 (52.5)5 (12.2)
  Not circumscribed58 (47.5)36 (87.8)
Mass internal enhancement 0.012
  Homogeneous55 (45.1)9 (22.0)
  Heterogeneous67 (54.9)32 (78.0)
Mass rim enhancement 0.014
  Yes10 (8.2)7 (17.1)
  No112 (91.8)34 (82.9)
NME distribution 0.400
  Linear/segmental14 (53.8)12 (66.7)
  Focal/regional/multiple regions/diffuse12 (46.2)6 (33.3)
NME internal enhancement 0.790
  Homogeneous1 (3.8)1 (5.6)
  Heterogeneous/clumped/clustered ring25 (96.2)17 (94.4)
  Enhancement degree (%) *141.3 ± 97.8238.0 ± 128.5<0.001
Enhancement rate <0.001
  Slow24 (16.2)0 (0)
  Intermediate35 (23.7)5 (8.5)
  Rapid89 (60.1)54 (91.5)
Enhancement curve type <0.001
  Persistent100 (67.6)18 (30.5)
  Plateau31 (21.0)14 (23.7)
  Washout17 (11.5)27 (45.8)
BI-RADS category <0.001
  3 (Probably benign)114 (77.0)9 (15.3)
  4A (Low suspicion for malignancy)27 (18.2)13 (22.0)
  4B (Moderate suspicion for malignancy)7 (4.7)13 (22.0)
  4C (High suspicion for malignancy)0 (0)12 (20.3)
  5 (Highly suggestive of malignancy)0 (0)12 (20.3)
Biopsy recommend by guideline <0.001
  No44 (29.7)3 (5.1)
  Yes104 (70.3)56 (94.9)
MRI magnetic field strength 0.420
  1.5-T46 (31.1)15 (25.4)
  3.0-T102 (68.9)44 (74.6)
Screening round 0.490
  First98 (66.2)42 (71.2)
  Second or more50 (33.8)17 (28.8)
Unless otherwise specified, data are number of lesions with percentage in parentheses. * Number is mean ± standard deviation. † Number is median with ranges in parentheses. BI-RADS = Breast Imaging Reporting and Data System; BRCA = BReast CAncer gene; MRI = magnetic resonance imaging; NME = non-mass enhancement.
Table 2. Receiver operating characteristics curve analysis of parameters for differentiating benign and malignant breast lesions.
Table 2. Receiver operating characteristics curve analysis of parameters for differentiating benign and malignant breast lesions.
ParameterAUC95% CIp ValueCutoff
Morphological analysis alone0.620.58–0.67
Enhancement degree (%)
  All malignancy 0.740.67–0.810.00990%
  Invasive cancer 0.720.64–0.810.029107%
Enhancement rate
  All malignancy0.660.61–0.720.265
  Invasive cancer0.650.59–0.700.304
Enhancement curve type
  All malignancy0.720.65–0.800.036Plateau
  Invasive cancer0.710.62–0.800.066
Tumor size (cm)
  All malignancy0.730.65–0.810.0120.5 cm
  Invasive cancer 0.720.63–0.820.0180.6 cm
AUC = area under curve; CI = confidence interval.
Table 3. Diagnostic performance of parameters for differentiating benign and malignant breast lesions.
Table 3. Diagnostic performance of parameters for differentiating benign and malignant breast lesions.
ParameterSensitivity (%)p ValueSpecificity (%)p Value
For detection of all malignancy
  Morphological analysis alone94.9 29.7
  Morphological analysis + Enhancement degree ≥ 90%
    All89.80.08052.7<0.001
    1.5-T73.30.08376.1<0.001
    3.0-T95.5N/A42.2<0.001
  Morphological analysis + Enhancement curve type ≥ plateau66.1<0.00179.1<0.001
  Morphological analysis + Enhancement degree ≥ 90%
  + Enhancement curve type ≥ plateau
64.4<0.00180.4<0.001
For detection of invasive cancer
  Morphological analysis alone94.6 26.5
  Morphological analysis + Enhancement degree ≥ 107%
    All86.50.08357.6<0.001
    1.5-T70.00.08380.4<0.001
    3.0-T92.6N/A47.9<0.001
  Morphological analysis + Size ≥ 0.6 cm
    All86.50.08338.8<0.001
    1.5-T90.00.37141.20.083
    3.0-T85.20.15737.8<0.001
  Morphological analysis + Enhancement degree ≥ 107%
  + Size ≥ 0.6 cm
78.40.01463.5<0.001
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kim, H.; Ko, E.Y.; Kim, K.E.; Kim, M.K.; Choi, J.S.; Ko, E.S.; Han, B.-K. Assessment of Enhancement Kinetics Improves the Specificity of Abbreviated Breast MRI: Performance in an Enriched Cohort. Diagnostics 2023, 13, 136. https://doi.org/10.3390/diagnostics13010136

AMA Style

Kim H, Ko EY, Kim KE, Kim MK, Choi JS, Ko ES, Han B-K. Assessment of Enhancement Kinetics Improves the Specificity of Abbreviated Breast MRI: Performance in an Enriched Cohort. Diagnostics. 2023; 13(1):136. https://doi.org/10.3390/diagnostics13010136

Chicago/Turabian Style

Kim, Haejung, Eun Young Ko, Ka Eun Kim, Myoung Kyoung Kim, Ji Soo Choi, Eun Sook Ko, and Boo-Kyung Han. 2023. "Assessment of Enhancement Kinetics Improves the Specificity of Abbreviated Breast MRI: Performance in an Enriched Cohort" Diagnostics 13, no. 1: 136. https://doi.org/10.3390/diagnostics13010136

APA Style

Kim, H., Ko, E. Y., Kim, K. E., Kim, M. K., Choi, J. S., Ko, E. S., & Han, B. -K. (2023). Assessment of Enhancement Kinetics Improves the Specificity of Abbreviated Breast MRI: Performance in an Enriched Cohort. Diagnostics, 13(1), 136. https://doi.org/10.3390/diagnostics13010136

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop