Next Article in Journal
A Study of Biomarkers Associated with Metabolic Dysfunction-Associated Fatty Liver Disease in Patients with Type 2 Diabetes
Next Article in Special Issue
An Automatic Method for Assessing Spiking of Tibial Tubercles Associated with Knee Osteoarthritis
Previous Article in Journal
Comparison of the Ultrasound Visibility of Tissue Markers in Metastatic Lymph Nodes after Neoadjuvant Chemotherapy in Patients with Breast Cancer
Previous Article in Special Issue
Detecting Endotracheal Tube and Carina on Portable Supine Chest Radiographs Using One-Stage Detector with a Coarse-to-Fine Attention
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Differential Diagnosis of Alzheimer Disease vs. Mild Cognitive Impairment Based on Left Temporal Lateral Lobe Hypomethabolism on 18F-FDG PET/CT and Automated Classifiers

1
Unit of Nuclear Medicine, Department of Medicine, Surgery and Pharmacy, Università degli Studi di Sassari, 07100 Sassari, Italy
2
Department of Engineering, Università degli Studi di Perugia, 06125 Perugia, Italy
3
Cognitive Disorders and Dementia Unit, USL Umbria 1, 06127 Perugia, Italy
4
Department of Nuclear Medicine, Santa Maria Goretti Hospital, 04100 Latina, Italy
5
Section of Nuclear Medicine and Health Physics, Department of Medicine and Surgery, Università degli Studi di Perugia, 06132 Perugia, Italy
*
Author to whom correspondence should be addressed.
Diagnostics 2022, 12(10), 2425; https://doi.org/10.3390/diagnostics12102425
Submission received: 17 August 2022 / Revised: 22 September 2022 / Accepted: 4 October 2022 / Published: 7 October 2022
(This article belongs to the Special Issue Artificial Neural Networks in Medical Diagnosis)

Abstract

:
Purpose: We evaluate the ability of Artificial Intelligence with automatic classification methods applied to semi-quantitative data from brain 18F-FDG PET/CT to improve the differential diagnosis between Alzheimer Disease (AD) and Mild Cognitive Impairment (MCI). Procedures: We retrospectively analyzed a total of 150 consecutive patients who underwent diagnostic evaluation for suspected AD (n = 67) or MCI (n = 83). All patients received brain 18F-FDG PET/CT according to the international guidelines, and images were analyzed both Qualitatively (QL) and Quantitatively (QN), the latter by a fully automated post-processing software that produced a z score metabolic map of 25 anatomically different cortical regions. A subset of n = 122 cases with a confirmed diagnosis of AD (n = 53) or MDI (n = 69) by 18–24-month clinical follow-up was finally included in the study. Univariate analysis and three automated classification models (classification tree –ClT-, ridge classifier –RC- and linear Support Vector Machine –lSVM-) were considered to estimate the ability of the z scores to discriminate between AD and MCI cases in. Results: The univariate analysis returned 14 areas where the z scores were significantly different between AD and MCI groups, and the classification accuracy ranged between 74.59% and 76.23%, with ClT and RC providing the best results. The best classification strategy consisted of one single split with a cut-off value of ≈ −2.0 on the z score from temporal lateral left area: cases below this threshold were classified as AD and those above the threshold as MCI. Conclusions: Our findings confirm the usefulness of brain 18F-FDG PET/CT QL and QN analyses in differentiating AD from MCI. Moreover, the combined use of automated classifications models can improve the diagnostic process since its use allows identification of a specific hypometabolic area involved in AD cases in respect to MCI. This data improves the traditional 18F-FDG PET/CT image interpretation and the diagnostic assessment of cognitive disorders.

1. Introduction

Alzheimer disease (AD), the most frequent form of neurodegenerative dementia [1,2], has an increasing incidence due to the progressive aging of the population [3]. The disease evolution of AD is a progressive continuum starting from the subclinical phase of Mild Cognitive Impairment (MCI), which is characterized by the absence of objective evidence of damage to functional autonomy [4,5]. This condition is considered high risk for AD [6]: approximately 10% of MCI cases per year progress to AD or other forms of dementia; however, a fraction of MCI patients will not develop clinical dementia, even after 10 years [5,7,8].
It is, therefore, crucial to identify those MCI cases that are more likely to progress to AD or other forms of dementia. This would allow direct patients toward adequate clinical trials or prevention strategies since no disease-modifying therapy is currently available [1,9].
There are different tools to evaluate the risk of conversion to AD; among them, the assessment of cerebral metabolism by 18F-fluoro-deoxyglucose-PET/CT (18F-FDG PET/CT) either alone or in conjunction with other procedures is one of the most accurate [10,11,12,13,14,15]. Moreover, this procedure can be strengthened by semi-quantitative evaluation [16,17], which provides reproducible, standardized parameters.
In recent years, Artificial Intelligence (AI) methods—including machine learning, deep learning and radiomics—have been successfully applied to neurological diseases, particularly to contribute to the diagnosis of Parkinson’s disease and dementia [18,19,20]. Various authors have advocated the use of automatic classification for discriminating between AD and MCI [21] based on MR images [22,23,24] or 18F-FDG PET/CT images [21,25,26], also combined with different types of biomarkers.
The aim of our study was to evaluate further the ability of AI applied to semi-quantitative data from 18F-FDG PET/CT of the brain to improve the diagnosis of cognitive disorders and, in particular, AD and MCI.

2. Materials and Methods

2.1. Study Population

We retrospectively investigated 150 consecutive patients evaluated for cognitive impairment (64 males, 86 females; age = 70.59 ± 9.14 [40–85] year) who underwent 18F-FDG brain PET for differential diagnosis of MCI and dementia between November 2017 and January 2021. Of the 150 cases, 67 had suspected AD (29 males and 38 females) and 83 suspected MCI (35 males and 48 females).
All patients were evaluated for neurological familiar diseases and neurological- and general-related diseases (hypertension and diabetes mellitus). Laboratory analyses excluded secondary cognitive disorders, and patients with other ascertained neurological diseases were excluded.
Before performing 18F-FDG brain PET, all patients underwent neurological examination, neuropsychological tests (Mini Mental State Examination -MMSE-) and Magnetic Resonance Imaging (MRI), the latter in order to evaluate the morphological brain assessment, especially to detect the presence of atrophy and gliosis as potential signs of white matter chronic cerebral vasculopathy. A final subset of n = 122 cases with confirmed AD (n = 53) or MDI (n = 69) was eventually retained for the study. The standard of reference for the diagnosis was a clinical follow-up of 18–24 months. Table 1 reports demographic, clinical, MMSE and MRI data of the study population; Figure 1 shows the STARD diagram for patient selection. Figure 2 summarizes the whole workflow of the study.

2.2. 18F-FDG Brain PET/CT

2.2.1. Acquisition Protocol

Before the procedure, written informed consent was obtained from all patients, and their data were treated in accordance with the local privacy rules and regulations. In the informed consent, the patients signed to accept that their data could be used for scientific purposes. The present study was in accordance with the Helsinki Doctrine on Human Experimentation.
18F-FDG brain PET/CT was performed according to international guidelines [27]. Patients were advised to fast for at least 4 to 6 h to ensure that 18F-FDG uptake would not be influenced by increased serum glucose levels. The latter were checked before injection, and the radiopharmaceutical administration was allowed only if/when the serum glucose values were <160 mg/dl.
The patients were invited to rest comfortably in a quiet, dimly-lit room for at least 15 min before 18F-FDG administration and for at least 20 min during the subsequent phase of tracer uptake. They were also instructed not to speak, read, listen to music/sounds or perform any other similar activities during the procedure.
A dose of 3.7 MBq/kg of 18F-FDG was then injected intravenously using a previously positioned cannula. Images were acquired 45 min after administration for 15 min at a single bed position using a PET/CT GE Healthcare Discovery 710 tomograph. CT parameters were 120 kVs, 45 effective mAs and one rotation. Slice thickness was 3 mm, and the reconstruction interval was 1.5 mm.

2.2.2. Image Reconstruction and Processing

The iterative reconstruction technique was applied for image reconstruction in axial, coronal and sagittal planes. Three nuclear medicine specialists (SN, AS and PB), all with at least > 20 yr experience in PET neuroimaging, independently interpreted the images. The three specialists were informed about the potentially pathological conditions of the patients but blind to the neurological evaluations.
18F-FDG PET/CT images were analyzed both Qualitatively (QL) and Quantitatively (QN). For QL analysis, images were differentiated in normal and pathological metabolism, considering as normal homogenous and symmetric tracer uptake in cortical areas of both hemispheres and as pathological cortical areas with reduced or asymmetric tracer uptake.
QN analysis was performed on a fully-automated post-processing software (Cortex ID SUITE, GE Healthcare, Chicago, IL, United States). All scans, spatially realigned and normalized, were sampled at 16,000 predefined cortical locations and projected on a three-dimensional image. The data were further normalized to the pons and compared with a normal, age-matched segmented database. Finally, a three-dimensional stereotactic surface projection and a Z-score metabolic map were produced [28,29]. In particular, the software computed the radiotracer uptake at 25 predefined regions of interest (ROI), compared the values with those of normal subjects and returned the deviation in terms of Z-score. The ROIs corresponded to the following anatomical cortical areas: prefrontal lateral left (L) and right (R), prefrontal medial L and R, sensorimotor L and R, anterior cingulate L and R, posterior cingulate L and R, precuneus L and R, parietal superior L and R, parietal inferior L and R, occipital lateral L and R, primary visual L and R, temporal lateral L and R, temporal mesial L and R, and whole cerebellum. Z-scores <= −2.0 were considered significant, and for each patient, the maximum negative values achieved in each ROI were also evaluated [19,20].
For univariate analysis and automated classification models, 122 of the 150 initial cases with the diagnosis confirmed by 18–24 months of clinical follow-up (53 AD and 69 MCI) were considered. The remaining 28 patients were excluded from the analysis since the initial clinical suspected diagnosis was not confirmed.
We provide the complete anonymous dataset as Supplementary Material (dataset.xls).

2.3. Statistical Analysis

Univariate analysis was performed to determine, for each area, whether there were statistically significant differences in the z scores between the AD and MCI groups. The analysis was based on Welch’s test at a significance level α = 0.05; Bonferroni correction was also applied to counteract the effects of multiple tests. Pairwise correlation between the significant features was assessed via Pearson’s correlation coefficient.

2.4. Classification

The automated classification was carried out to estimate the ability of the z scores from the areas with significant differences to discriminate between AD and MCI cases. Three classification models were considered for this task: classification tree (ClT), ridge classifier (RC) and linear Support Vector Machine (lSVM). For each classifier, optimal hyper-parameter values (see Table 2) were determined by grid-search and four-fold cross-validation over 50 random splits; the final accuracy was estimated via leave-one-out cross-validation.

2.5. Execution, Data and Code Availability

Data analysis and visualization were based on Python 3.8.4 and functions from the matplotlib, numpy, Pandas, scikit-learn and seaborn packages. The experiments were carried out on an ASUS ProArt Laptop PC with Intel CoreTM i7-9750H @ 2.60 GHz CPU, 36 Gb RAM and Windows 10 Pro 64-bit. The total execution time was ≈2 min.

3. Results

Univariate analysis (Table 3) returned 14 areas where the z scores were significantly different between the AD and MCI groups. These were: prefrontal lateral (L and R), prefrontal medial (L and R), posterior cingulate (L and R), precuneus (L and R), parietal inferior (L and R), occipital lateral L temporal lateral (L and R) and temporal mesial L. Figure 3 reports the box-plots, strip-plots and p-values for each area.
The correlation analysis (Figure 4) identified 19 pairs of areas with very strong positive correlation (Pearson’s r ≥ 0.8) [30], 44 with moderately strong positive correlation (0.6 ≤ r < 0.8) and 28 with fair positive correlation (0.3 ≤ r < 0.6). There were no pairs of areas with poor (0.0 ≤ r < 0.3) or negative correlation. The pairs with the strongest correlation (r > 0.85) were: posterior cingulate L and R, parietal inferior R and temporal lateral R, parietal inferior L and temporal lateral L, precuneus L and parietal inferior L, prefrontal lateral L and R, parietal inferior L and R, precuneus R and parietal inferior R, prefrontal lateral L and parietal inferior L, precuneus L and R, and prefrontal medial L and R.
As can be seen from Table 4, the classification accuracy ranged between 74.59% (91/122) and 76.23% (93/122), with ClT and RC providing the best results. As for the classification tree, it is worth noting that the best classification strategy consisted of one single split with a cut-off value of ≈ −2.0 on the z score from temporal lateral left area: cases below this threshold were classified as AD and those above the threshold as MCI (Figure 5).

4. Discussion

In the Amyloid-PET era, 18F-FDG PET/CT still plays a significant role in the diagnosis of Alzheimer Disease (AD) and Mild Cognitive Impairment (MCI) [31,32,33]. In 2011, the National Institute on Aging and Alzheimer’s Association (NIA-AA) proposed separate diagnostic recommendations for the preclinical mild cognitive impairment and dementia stages of Alzheimer’s disease based on different biomarkers able to discriminate, in vivo, the different pathological entities, including beta-amyloid deposition, pathologic tau and neurodegeneration. This classification generated an [A/T/N] system where “A” represents the biomarkers of Ab plaques (including cortical amyloid PET ligand binding or low CSF Ab42), “T” the biomarkers of fibrillary tau (including elevated CSF phosphorylated tau (P-tau) and cortical tau PET ligand binding) and “N” the labeled biomarkers of neurodegeneration or neuronal injury as, CSF T-tau, 18FDG-PET/CT hypometabolism and atrophy on MRI [34]. This interesting system represented a basis for a biological definition of AD [6] and paved the way for subsequent studies [35,36,37].
Iaccarino [38] showed that in a group of 518 MCI subjects and 269 healthy controls from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu), a positive amyloid PET scan was not associated with clinical progression in the majority (≥60%) of subjects, although it represented a significant risk factor, while a negative 18F-FDG PET/CT scan at baseline strongly predicted clinical stability with high negative predictive values (>0.80) for both groups of subjects. The authors concluded that 18F-FDG PET/CT brain metabolism or other neurodegeneration measures should be coupled with amyloid-PET to identify clinically stable individuals in order to exclude them from clinical trials.
Ottoy and co-workers [14] showed that patients with MCI progressed to AD at an annual rate of 31%, and this could be best predicted by combining neuropsychological testing with MRI-based Hippocampal Volume and 18F-FDG PET/CT (specificity = 96%, sensitivity = 92%).
Chatelat et al. [39] reported that 18F-FDG PET/CT could predict the clinical outcome in patients with MCI who already have an amyloid-PET scan. In their work, a normal 18F-FDG-PET scan was associated with long-term clinical stability–even in amyloid-positive cases; by contrast, a pathological 18F-FDG-PET scan was indicative of an increased risk of progressive cognitive decline even in amyloid-negative cases.
Tondo et al. [40] observed 142 subjects with amnestic MCI for 4–19 years and determined that hypometabolism patterns on baseline 18F-FDG PET/CT could predict long-term outcomes in terms of stability or progression to AD. Specifically, they reported that limbic-predominant hypometabolism pattern was associated with clinical stability, thus making progression to AD very unlikely.
Arbizu et al. [32] investigated the additional value of 18F-FDG PET/CT beyond clinical neuropsychological examination to support the diagnosis of prodromal Alzheimer’s Disease (AD), frontotemporal lobar degeneration (FTLD) and prodromal dementia with Lewy bodies (DLB) in mild cognitive impairment (MCI) subjects. A panel of seven experts (four from the European Association of Nuclear Medicine and 3 from the European Academy) provided recommendations about the incremental value of 18F-FDG PET/CT to evaluate the etiology of MCI (AD, FTLD or DLB). The study identified 55 relevant papers, which were obtained through a population, intervention, comparison and outcome (PICO) search string. The meta-analysis indicated that 18F-FDG PET/CT patterns enabled the correct identification of MCI etiology due to AD with accuracy between 58% and 100% and area-under-curve between 0.66 and 0.97; however, no specific data were found in regard to MCI due to FTLD or DLB.
However, the clinical use of 18F-FDG PET/CT reached the consensus recommendations in MCI subjects for the high negative predictive value and the existence of different disease-specific patterns of hypometabolism. Worthy of mention is that 123I-ioflupane and 123I-MIBG can be useful in the case of prodromal DLB [41].
In this scenario, our findings confirm the ability of 18F-FDG PET/CT to differentiate AD from MCI. Specifically, the univariate analysis (Table 1, Figure 3) indicates that AD and MCI cases had significantly different readings in many strategic areas, in particular, precuneus, posterior cingulate and parietal and temporal regions.
We also demonstrated the ability of automated classifiers (ClT, lSVM and RC) to discriminate between AD and MCI based on the z scores of the significant areas. Our classification accuracy ranged between 74.59% (91/122) and 76.23% (93/122), with ClT and RC providing the best results. As for the classification tree, it is worth noting that the best classification strategy consisted of one single split with a cut-off value of ≈ −2.0 on the z score from temporal lateral left area: cases below this threshold were classified as AD and those above the threshold as MCI (Figure 5). This suggests that radiopharmaceutical uptake in the temporal lateral left area is the “marker” region able to discriminate AD and MCI in our series of the patient. This is consistent with the hierarchical pathologic progression of neurofibrillary tangles that spread from the middle temporal to the lateral temporal areas. Indeed, tau pathology initially affects transentorhinal, followed by entorhinal, after fusiform and lingual and later reach lateral temporal association areas [42,43,44].
Finally, we wish to underline the usefulness of semi-quantitative analysis to assist visual reading, as it has been widely evidenced by international literature [15,37]. We used Cortex ID suite, a fully automated post-processing software for quantifying 18F-FDG PET/CT and beta-amyloid brain scans that used three-dimensional stereotactic surface projections (3D-SSP) for statistical image analysis [45,46]. Future works may focus on the combination of semi-quantitative analysis with direct extraction of traditional and/or deep learning imaging features from the brain scans, as discussed in [47,48,49].
In conclusion, our study indicates that the combination of semi-quantitative analysis and automatic classification can improve the diagnostic process through the identification of the metabolically impaired areas specific to the different disorders. The development of computer-aided diagnosis (CAD) systems is also receiving attention not only as a means to support the diagnostic process by the calculation of cut-off values but also to assess correlations between clinical data and pathologies. This improves traditional image interpretation and diagnostic assessment in many neurodegenerative diseases [49,50,51]. The role played by artificial intelligence techniques, i.e., Machine Learning, Radiomics and Deep Learning is pivotal to building diagnostic models for personalized care. This is of particular importance in neurodegenerative diseases such as AD and MCI, as they fall in a sort of “grey area” where a clear diagnosis is often difficult. Our paper confirms the clinical value of 18F-FDG brain PET/CT as an essential diagnostic first step to contribute to the differential diagnosis of dementia disorders also in the amyloid PET and biological markers (i.e., amyloid and tau protein) era since, as it is well known, accurate and early diagnosis of Alzheimer disease in respect of Mild Cognitive Impairment is crucial for improving the condition of patients. The combined use of semi-quantitative analysis of 18F-FDG PET and automatic classification seems to be a supportive tool for clinical diagnosis in order to consider effective preventive early measures to delay the appearance of the full-blown disease or to suggest further investigations such as amyloid PET or more advanced treatments and therapeutic approaches [52,53].

Limitations and Future Work

A number of limitations apply to this paper, among which are the retrospective nature of the study and the relatively contained sample size. Furthermore, it is to be noted that our method relies on semi/quantitative data, the calculation of which is delegated to an external software package (Cortex ID). Extraction of custom imaging features directly from the PET/CT scans via hand-crafted methods and/or Deep Learning (as discussed in [54,55]) is an interesting subject for future studies.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12102425/s1.

Author Contributions

Conceptualization, S.N. and B.P.; methodology, F.B. and M.L.F.; software, S.C.; validation, A.S. and S.N.; formal analysis, S.N. F.B. and B.P.; investigation, F.B. and M.L.F.; data curation, M.R. A.L. and A.M.; writing—original draft preparation, S.N., B.P., S.A. and L.F.; writing—review and editing, S.N. F.B. and B.P.; visualization, A.S.; supervision, B.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This retrospective study was conducted in accordance with the Declaration of Helsinki. Ethical review and approval were not required for this study due to the retrospective nature and the analysis of data in anonymous form.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The supporting data can be found in the Supplementary Material.

Conflicts of Interest

The authors declare that they have no conflict of interest.

References

  1. Scheltens, P.; Blennow, K.; Breteler, M.M.; de Strooper, B.; Frisoni, G.B.; Salloway, S.; van der Flier, W.M. Alzheimer’s disease. Lancet 2016, 388, 505–517. [Google Scholar] [CrossRef]
  2. Goodman, R.A.; Lochner, K.A.; Thambisetty, M.; Wingo, T.S.; Posner, S.F.; Ling, S.M. Prevalence of dementia subtypes in United States Medicare fee-for-service beneficiaries, 2011–2013. Alzheimers Dement. 2017, 13, 28–37. [Google Scholar] [CrossRef] [Green Version]
  3. Hebert, L.E.; Scherr, P.A.; Bienias, J.L.; Bennett, D.A.; Evans, D.A. Alzheimer disease in the US population: Prevalence estimates using the 2000 census. Arch. Neurol. 2003, 60, 1119–1122. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Petersen, R.C.; Smith, G.E.; Waring, S.C.; Ivnik, R.J.; Tangalos, E.G.; Kokmen, E. Mild cognitive impairment: Clinical characterization and outcome. Arch. Neurol. 1999, 56, 303–308. [Google Scholar] [CrossRef] [PubMed]
  5. Petersen, R.C.; Roberts, R.O.; Knopman, D.S.; Boeve, B.F.; Geda, Y.E.; Ivnik, R.J.; Smith, G.E.; Jack, C.R., Jr. Mild cognitive impairment: Ten years later. Arch. Neurol. 2009, 66, 1447–1455. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Jack, C.R., Jr.; Bennett, D.A.; Blennow, K.; Carrillo, M.C.; Dunn, B.; Haeberlein, S.B.; Holtzman, D.M.; Jagust, W.; Jessen, F.; Karlawish, J.; et al. NIA-AA research framework: Toward a biological definition of Alzheimer’s disease. Alzheimers Dement. 2018, 14, 535–562. [Google Scholar] [CrossRef] [PubMed]
  7. Mitchell, A.J.; Shiri-Feshki, M. Rate of progression of mild cognitive impairment to dementia—Meta-analysis of 41 robust inception cohort studies. Acta Psychiatr. Scand. 2009, 119, 252–265. [Google Scholar] [CrossRef]
  8. Schneider, J.A.; Arvanitakis, Z.; Leurgans, S.E.; Bennett, D.A. The neuropathology of probable Alzheimer disease and mild cognitive impairment. Ann. Neurol. 2009, 66, 200–208. [Google Scholar] [CrossRef] [Green Version]
  9. Modrego, P.J. Predictors of conversion to dementia of probable Alzheimer type in patients with mild cognitive impairment. Curr. Alzheimer Res. 2006, 3, 161–170. [Google Scholar] [CrossRef]
  10. Shaffer, J.L.; Petrella, J.R.; Sheldon, F.C.; Choudhury, K.R.; Calhoun, V.D.; Coleman, R.E.; Doraiswamy, P.M. Alzheimer’s Disease Neuroimaging Initiative. Predicting cognitive decline in subjects at risk for Alzheimer disease by using combined cerebrospinal fluid, MR imaging, and PET biomarkers. Radiology 2013, 266, 583–591. [Google Scholar] [CrossRef] [PubMed]
  11. Eckerström, C.; Olsson, E.; Bjerke, M.; Malmgren, H.; Edman, A.; Wallin, A.; Nordlund, A. A combination of neuropsychological, neuroimaging, and cerebrospinal fluid markers predicts conversion from mild cognitive impairment to dementia. J. Alzheimers Dis. 2013, 36, 421–431. [Google Scholar] [CrossRef] [PubMed]
  12. Nesteruk, M.; Nesteruk, T.; Styczyńska, M.; Mandecka, M.; Barczak, A.; Barcikowska, M. Combined use of biochemical and volumetric biomarkers to assess the risk of conversion of mild cognitive impairment to Alzheimer’s disease. Folia Neuropathol. 2016, 4, 369–374. [Google Scholar] [CrossRef] [PubMed]
  13. Caminiti, S.P.; Ballarini, T.; Sala, A.; Cerami, C.; Presotto, L.; Santangelo, R.; Fallanca, F.; Vanoli, E.G.; Gianolli, L.; Iannaccone, S.; et al. FDG-PET and CSF biomarker accuracy in prediction of conversion to different dementias in a large multicentre MCI cohort. Neuroimage Clin. 2018, 18, 167–177. [Google Scholar] [CrossRef] [PubMed]
  14. Ottoy, J.; Niemantsverdriet, E.; Verhaeghe, J.; de Roeck, E.; Struyfs, H.; Somers, C.; Wyffels, L.; Ceyssens, S.; van Mossevelde, S.; van den Bossche, T.; et al. Association of short-term cognitive decline and MCI-to-AD dementia conversion with CSF, MRI, amyloid- and 18F-FDG-PET imaging. Neuroimage Clin. 2019, 22, 101771. [Google Scholar] [CrossRef]
  15. Nuvoli, S.; Tanda, G.; Stazza, M.L.; Palumbo, B.; Frantellizzi, V.; De Vincentis, G.; Spanu, A.; Madeddu, G. 123I-Ioflupane SPECT and 18F-FDG PET combined use in the characterization of movement and cognitive associated disorders in neurodegenerative diseases. Curr. Alzheimer Res. 2021, 18, 196–207. [Google Scholar] [CrossRef]
  16. Nuvoli, S.; Tanda, G.; Stazza, M.L.; Madeddu, G.; Spanu, A. Qualitative and quantitative analyses of brain 18Fluoro-Deoxy-Glucose positron emission tomography in primary progressive aphasia. Dement. Geriatr. Cogn. Disord. 2019, 48, 250–260. [Google Scholar] [CrossRef]
  17. Massa, F.; Chincarini, A.; Bauckneht, M.; Raffa, S.; Peira, E.; Arnaldi, D.; Pardini, M.; Pagani, M.; Orso, B.; Donegani, M.I.; et al. Added value of semiquantitative analysis of brain FDG-PET for the differentiation between MCI-Lewy bodies and MCI due to Alzheimer’s disease. Eur. J. Nucl. Med. Mol. Imaging 2022, 49, 1263–1274. [Google Scholar] [CrossRef]
  18. Palumbo, B.; Fravolini, M.L.; Nuvoli, S.; Spanu, A.; Paulus, K.S.; Schillaci, O.; Madeddu, G. Comparison of two neural network classifiers in the differential diagnosis of essential tremor and Parkinson’s disease by (123)I-FP-CIT brain SPECT. Eur. J. Nucl. Med. Mol. Imaging 2010, 37, 2146–2153. [Google Scholar] [CrossRef]
  19. Nuvoli, S.; Spanu, A.; Fravolini, M.L.; Bianconi, F.; Cascianelli, S.; Madeddu, G.; Palumbo, B. [123I]Metaiodobenzylguanidine (MIBG) cardiac scintigraphy and automated classification techniques in Parkinsonian disorders. Mol. Imaging Biol. 2020, 22, 703–710. [Google Scholar] [CrossRef]
  20. Li, Y.; Jiang, J.; Lu, J.; Jiang, J.; Zhang, H.; Zuo, C. Radiomics: A novel feature extraction method for brain neuron degeneration disease using 18F-FDG PET imaging and its implementation for Alzheimer’s disease and mild cognitive impairment. Ther. Adv. Neurol. Disord. 2019, 12, 1756286419838682. [Google Scholar] [CrossRef]
  21. Gray, K.R.; Aljabar, P.; Heckemann, R.A.; Hammers, A.; Rueckert, D. Random forest-based similarity measures for multi-modal classification of Alzheimer’s disease. Neuroimage 2013, 65, 167–175. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Bicacro, E.; Silveira, M.; Marques, J.S. Alternative feature extraction methods in 3D brain image-based diagnosis of Alzheimer’s disease. In Proceedings of the 19th IEEE International Conference on Image Processing (ICIP), Orlando, FL, USA, 30 September–3 October 2012; pp. 1237–1240. [Google Scholar]
  23. Morgado, P.; Silveira, M.; Marques, J.S. Diagnosis of Alzheimer’s disease using 3D local binary patterns. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2013, 1, 2–12. [Google Scholar] [CrossRef]
  24. Padilla, P.; Lopez, M.; Gorriz, J.; Ramirez, J.; Salas-Gonzalez, D.; Alvarez, I. NMF-SVM based CAD tool applied to functional brain images for the diagnosis of Alzheimer’s disease. IEEE Trans. Med. Imaging 2012, 31, 207–221. [Google Scholar] [CrossRef] [PubMed]
  25. Silveira, M.; Marques, J. Boosting Alzheimer’s disease diagnosis using PET images. In Proceedings of the 20th International Conference on Pattern Recognition (ICPR), Istanbul, Turkey, 23–26 August 2010; pp. 2556–2559. [Google Scholar]
  26. McEvoy, L.; Fennema-Notestine, C.; Roddey, J.; Hagler, D.; Holland, D.; Karow, D.; Pung, C.; Brewer, J.; Dale, A. Alzheimer disease: Quantitative structural neuroimaging for detection and prediction of clinical and structural changes in mild cognitive impairment. Radiology 2009, 251, 195–205. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Guedj, E.; Varrone, A.; Boellaard, R.; Albert, N.L.; Barthel, H.; van Berckel, B.; Brendel, M.; Cecchin, D.; Ekmekcioglu, O.; Garibotto, V.; et al. EANM procedure guidelines for brain PET imaging using [18F]FDG, version. Eur. J. Nucl. Med. Mol. Imaging 2022, 49, 632–651. [Google Scholar] [CrossRef]
  28. Clergue-Duval, V.; Questel, F.; Azuar, J.; Paquet, C.; Cognat, E.; Amami, J.; Queneau, M.; Dereux, A.; Barré, T.; Bellivier, F.; et al. Brain 18FDG-PET pattern in patients with alcohol-related cognitive impairment. Eur. J. Nucl. Med. Mol. Imaging 2019, 47, 281–291. [Google Scholar] [CrossRef] [PubMed]
  29. van Essen, D.C.; Drury, H.A.; Dickson, J.; Harwell, J.; Hanlon, D.; Anderson, C. An integrated software suite for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 2001, 8, 443–459. [Google Scholar] [CrossRef] [Green Version]
  30. Chan, Y.H. Biostatistics 104: Correlational analysis. Singap. Med. J. 2003, 44, 614–619. [Google Scholar]
  31. Minoshima, S.; Mosci, K.; Cross, D.; Thientunyakit, T. Brain [F-18]FDG PET for Clinical Dementia Workup: Differential Diagnosis of Alzheimer’s Disease and Other Types of Dementing Disorders. Semin. Nucl. Med. 2021, 5, 230–240. [Google Scholar] [CrossRef] [PubMed]
  32. Arbizu, J.; Festari, C.; Altomare, D.; Walker, Z.; Bouwman, F.; Rivolta, J.; Orini, S.; Barthel, H.; Agosta, F.; Drzezga, A.; et al. EANM-EAN task force for the prescription of FDG-PET for dementing neurodegenerative disorders. Clinical utility of FDG-PET for the clinical diagnosis in MCI. Eur. J. Nucl. Med. Mo. Imaging 2018, 45, 1497–1508. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Teng, L.; Li, Y.; Zhao, Y.; Hu, T.; Zhang, Z.; Yao, Z.; Hu, B. Alzheimer’ s Disease Neuroimaging Initiative (ADNI). Predicting MCI progression with FDG-PET and cognitive scores: A longitudinal study. BMC Neurol. 2020, 21, 148. [Google Scholar]
  34. McKhann, G.M.; Knopman, D.S.; Chertkow, H.; Hyman, B.T.; Jack, C.R., Jr.; Kawa, C.H.; Klunk, W.E.; Koroshetz, W.J.; Manly, J.J.; Mayeux, R.; et al. The diagnosis of dementia due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011, 7, 263–269. [Google Scholar] [CrossRef] [PubMed]
  35. Ou, Y.N.; Xu, W.; Li, J.Q.; Guo, Y.; Cui, M.; Chen, K.L.; Huang, Y.Y.; Dong, Q.; Tan, L.; Yu, J.T. Alzheimer’s Disease Neuroimaging Initiative. FDG-PET as an independent biomarker for Alzheimer’s biological diagnosis: A longitudinal study. Alzheimers Res. Ther. 2019, 11, 57. [Google Scholar] [CrossRef] [Green Version]
  36. Tiepolt, S.; Patt, M.; Aghakhanyan, G.; Meyer, P.M.; Hesse, S.; Barthel, H.; Sabri, O. Current radiotracers to image neurodegenerative diseases. EJNMMI Radiopharm. Chem. 2019, 4, 17. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Nobili, F.; Arbizu, J.; Bouwman, F.; Drzezga, A.; Agosta, F.; Nestor, P.; Walker, Z.; Boccardi, M. EANM-EAN task force for the prescription of FDG-PET for dementing neurodegenerative disorders. European Association of Nuclear Medicine and European Academy of Neurology recommendations for the use of brain 18 F-fluorodeoxyglucose positron emission tomography in neurodegenerative cognitive impairment and dementia: Delphi consensus. Eur. J. Neurol. 2018, 25, 1201–1217. [Google Scholar]
  38. Iaccarino, L.; Sala, A.; Perani, D. Alzheimer’s Disease neuroimaging initiative. Predicting long-term clinical stability in amyloid-positive subjects by FDG-PET. Ann. Clin. Transl. Neurol. 2019, 6, 1113–1120. [Google Scholar] [CrossRef] [Green Version]
  39. Chételat, G.; Arbizu, J.; Barthel, H.; Garibotto, V.; Law, I.; Morbelli, S.; van de Giessen, E.; Agosta, F.; Barkhof, F.; Brooks, D.J.; et al. Amyloid-PET and (18)F-FDG-PET in the diagnostic investigation of Alzheimer’s disease and other dementias. Lancet Neurol. 2020, 19, 951–962. [Google Scholar] [CrossRef]
  40. Tondo, G.; Carli, G.; Santangelo, R.; Mattoli, M.V.; Presotto, L.; Filippi, M.; Magnani, G.; Iannaccone, S.; Cerami, C.; Perani, D. Alzheimer’s Disease Neuroimaging Initiative. Biomarker-based stability in limbic-predominant amnestic mild cognitive impairment. Eur. J. Neurol. 2021, 28, 1123–1133. [Google Scholar] [CrossRef]
  41. Nuvoli, S.; Palumbo, B.; Malaspina, S.; Madeddu, G.; Spanu, A. 123I-ioflupane SPET and 123I-MIBG in the diagnosis of Parkinson’s disease and parkinsonian disorders and in the differential diagnosis between Alzheimer’s and Lewy’s bodies dementias. Hell. J. Nucl. Med. 2018, 21, 60–68. [Google Scholar] [PubMed]
  42. Braak, H.; Braak, E. Neuropathological staging of Alzheimer-related changes. Acta Neuropathol. 1991, 82, 239–259. [Google Scholar] [CrossRef]
  43. Braak, H.; Alafuzoff, I.; Arzberger, T.; Kretzschmar, H.; Del Tredici, K. Staging of Alzheimer disease-associated neurofibrillary pathology using paraffin sections and immunocytochemistry. Acta Neuropathol. 2006, 112, 389–404. [Google Scholar] [CrossRef] [Green Version]
  44. Gertz, H.J.; Xuereb, J.H.; Huppert, F.A.; Brayne, C.; McGee, M.A.; Paykel, E.S.; Harrington, C.; Mukaetova-Ladinska, E.; Arendt, T.; Wischik, C.M. Examination of the validity of the hierarchical model of neuropathological staging in normal aging and Alzheimer’s disease. Acta Neuropathol. 1998, 95, 154–158. [Google Scholar] [CrossRef] [PubMed]
  45. Wagatsuma, K.; Sakata, M.; Ishibashi, K.; Hirayama, A.; Kawakami, H.; Miwa, K.; Suzuki, Y.; Ishii, K. Direct comparison of brain [18F]FDG images acquired by SiPM-based and PMT-based PET/CT: Phantom and clinical studies. EJNMMI Phys. 2020, 7, 70. [Google Scholar] [CrossRef] [PubMed]
  46. Lindström, E.; Oddstig, J.; Danfors, T.; Jögi, J.; Hansson, O.; Lubberink, M. Image reconstruction methods affect software-aided assessment of pathologies of [18F]flutemetamol and [18F]FDG brain-PET examinations in patients with neurodegenerative diseases. Neuroimage Clin. 2020, 28, 102386. [Google Scholar] [CrossRef] [PubMed]
  47. Zhou, P.; Zeng, R.; Yu, L.; Feng, Y.; Chen, C.; Li, F.; Liu, Y.; Huang, Y.; Huang, Z. Deep-learning radiomics for discrimination Conversion of Alzheimer’s Disease in patients with mild cognitive impairment: A study based on 18F-FDG PET imaging. Front. Aging Neurosci. 2021, 13, 764872. [Google Scholar] [CrossRef] [PubMed]
  48. Cui, W.; Yan, C.; Yan, Z.; Peng, Y.; Leng, Y.; Liu, C.; Chen, S.; Jiang, X.; Zheng, J.; Yang, X. BMNet: A new region-based metric learning method for early Alzheimer’s Disease identification with FDG-PET images. Front. Neurosci. 2022, 16, 831533. [Google Scholar] [CrossRef] [PubMed]
  49. Alongi, P.; Laudicella, R.; Panasiti, F.; Stefano, A.; Comelli, A.; Giaccone, P.; Arnone, A.; Minutoli, F.; Quartuccio, N.; Cupidi, C.; et al. Radiomics analysis of brain [18F]FDG PET/CT to predict Alzheimer’s Disease in patients with Amyloid PET Positivity: A preliminary report on the application of SPM cortical segmentation, pyradiomics and machine-learning analysis. Diagnostics 2022, 8, 933. [Google Scholar] [CrossRef] [PubMed]
  50. Cascianelli, S.; Scialpi, M.; Amici, S.; Forini, N.; Minestrini, M.; Fravolini, M.L.; Sinzinger, H.; Schillaci, O.; Palumbo, B. Role of Artificial Intelligence Techniques (Automatic Classifiers) in molecular imaging modalities in neurodegenerative diseases. Curr. Alzheimer Res. 2017, 14, 198–207. [Google Scholar] [CrossRef] [PubMed]
  51. Palumbo, B.; Bianconi, F.; Nuvoli, S.; Spanu, A.; Fravolini, L.M. Artificial intelligence techniques support nuclear medicine modalities to improve the diagnosis of Parkinson’s disease and Parkinsonian syndromes. Clin. Transl. Imaging 2021, 9, 19–35. [Google Scholar] [CrossRef]
  52. Huang, Z.; Sun, M.; Guo, C. Automatic diagnosis of Alzheimer’s Disease and mild cognitive impairment Based on CNN+SVM networks with end-to-end training. Comput. Intell. Neurosci. 2021, 13, 9121770. [Google Scholar] [CrossRef] [PubMed]
  53. Doroszkiewicz, J.; Mroczko, B. new possibilities in the therapeutic approach to Alzheimer’s Disease. Int. J. Mol. Sci. 2022, 23, 8902. [Google Scholar] [CrossRef]
  54. Odusami, M.; Maskeliūnas, R.; Damaševičius, R. An intelligent system for early recognition of Alzheimer’s Disease using neuroimaging. Sensors 2022, 22, 740. [Google Scholar] [CrossRef] [PubMed]
  55. Razzak, I.; Naz, S.; Ashraf, A.; Khalifa, F.; Bouadjenek, M.R.; Mumtaz, S. Mutliresolutional ensemble PartialNet for Alzheimer detection using magnetic resonance imaging data. Int. J. Intell. Syst. 2022, 37, 6613–6630. [Google Scholar] [CrossRef]
Figure 1. STARD diagram for the study population.
Figure 1. STARD diagram for the study population.
Diagnostics 12 02425 g001
Figure 2. Overall procedure workflow.
Figure 2. Overall procedure workflow.
Diagnostics 12 02425 g002
Figure 3. Box-plots and strip-plots of the Z-score for each area with significant difference between AD and MCI.
Figure 3. Box-plots and strip-plots of the Z-score for each area with significant difference between AD and MCI.
Diagnostics 12 02425 g003
Figure 4. Correlation heat-map. Values report Pearson’s correlation coefficient (r) between the z scores from each pair of areas.
Figure 4. Correlation heat-map. Values report Pearson’s correlation coefficient (r) between the z scores from each pair of areas.
Diagnostics 12 02425 g004
Figure 5. Prototype classification tree. Key to abbreviations: AD = Alzheimer’s disease, MCI = Mild Cognitive Impairment.
Figure 5. Prototype classification tree. Key to abbreviations: AD = Alzheimer’s disease, MCI = Mild Cognitive Impairment.
Diagnostics 12 02425 g005
Table 1. Demographic, clinical, Mini Mental State Examination (MMSE) and Magnetic Resonance Imaging (MRI) data of Alzheimer Disease (AD) and Mild Cognitive Impairment (MCI) patients.
Table 1. Demographic, clinical, Mini Mental State Examination (MMSE) and Magnetic Resonance Imaging (MRI) data of Alzheimer Disease (AD) and Mild Cognitive Impairment (MCI) patients.
AD
67 cases
MCI
83 cases
AgeRange
55–83 year
Mean ±standard deviation
69.5 ± 8.64
Range
40–85 year
Mean ±standard deviation
71.4 ± 9.37
Sex29 male38 female35 male48 female
Family history for dementiaPositive 25/67Negative
42/67
Positive
38/83
Negative
45/83
Correct Mini Mental State Examination (MMSE)Range
9.9/30–26/30
Mean ±standard deviation
22 ± 4.8
Range
25.3/30–30/30
Mean ±standard deviation
25.3 ± 3.04
MRISlight to severe atrophy: 15/67 casesSlight to severe atrophy: 17/83 cases
Focal/diffuse gliosis: 15/67 casesFocal/diffuse gliosis: 16/83 cases
Diffuse cerebrovascular lesions and atrophy: 19/67 casesDiffuse cerebrovascular lesions and atrophy: 20/83 cases
No significant alteration: 18/67 casesNo significant alteration: 30/83 cases
Table 2. Hyper-parameter tuning. Key to symbols: SC = splitting criterion; MD = maximum depth of the classification tree; C = regularization parameter; α = regularization strength.
Table 2. Hyper-parameter tuning. Key to symbols: SC = splitting criterion; MD = maximum depth of the classification tree; C = regularization parameter; α = regularization strength.
Classification ModelGrid Search DomainOptimal Hyper-Parameters
Classification treeSC = {“entropy”, “gini”}
MD = {1, 2, 4, 6, 8}
SC = “gini”
MD = 1
Linear SVMC = {0.01, 0.1, 1.0, 10.0}C = 1.0
Ridge classifierα = {0.01, 0.1, 1.0, 10.0}α = 1.0
Table 3. Results of the univariate analysis. The values report the corresponding z score (mean ± std) for each area. Key to abbreviations: AD = Alzheimer disease, MCI = Mild Cognitive Impairment.
Table 3. Results of the univariate analysis. The values report the corresponding z score (mean ± std) for each area. Key to abbreviations: AD = Alzheimer disease, MCI = Mild Cognitive Impairment.
AreaDiagnosisp-ValueSignificant
ADMCI
Prefrontal Lateral R−2.21 ± 1.23−1.33 ± 1.54<0.001Yes
Prefrontal Lateral L−2.24 ± 1.18−1.24 ± 1.38<0.001Yes
Prefrontal Medial R−1.73 ± 1.12−0.94 ± 1.30<0.001Yes
Prefrontal Medial L−1.64 ± 0.96−0.92 ± 1.31<0.001Yes
Sensorimotor R−0.93 ± 1.52−0.53 ± 1.510.157No
Sensorimotor L−0.89 ± 1.37−0.49 ± 1.480.129No
Anterior Cingulate R−1.30 ± 0.87−0.74 ± 1.170.003No
Anterior Cingulate L−1.27 ± 0.82−0.72 ± 1.220.003No
Posterior Cingulate R−2.29 ± 1.08−1.31 ± 1.41<0.001Yes
Posterior Cingulate L−2.28 ± 1.10−1.21 ± 1.42<0.001Yes
Precuneus R−2.48 ± 1.56−1.44 ± 1.55<0.001Yes
Precuneus L−2.27 ± 1.43−1.35 ± 1.47<0.001Yes
Parietal Superior R−2.15 ± 1.35−1.38 ± 1.540.004No
Parietal Superior L−1.76 ± 1.37−1.11 ± 1.600.018No
Parietal Inferior R−2.84 ± 1.32−1.56 ± 1.56<0.001Yes
Parietal Inferior L−2.76 ± 1.30−1.46 ± 1.51<0.001Yes
Occipital Lateral R−1.47 ± 1.50−0.65 ± 1.480.004No
Occipital Lateral L−1.70 ± 1.39−0.75 ± 1.54<0.001Yes
Primary Visual R−0.69 ± 1.27−0.41 ± 1.270.225No
Primary Visual L−0.68 ± 1.19−0.32 ± 1.360.128No
Temporal Lateral R−2.73 ± 1.22−1.41 ± 1.55<0.001Yes
Temporal Lateral L−2.74 ± 1.10−1.31 ± 1.46<0.001Yes
Temporal Mesial R−2.13 ± 1.43−1.36 ± 1.890.012No
Temporal Mesial L−2.56 ± 1.56−1.46 ± 1.90<0.001Yes
Cerebellum Whole−0.49 ± 1.30−0.37 ± 1.260.613No
Table 4. Classification performance. Accuracy was estimated via leave-one-out cross-validation.
Table 4. Classification performance. Accuracy was estimated via leave-one-out cross-validation.
Classification ModelAccuracy
Classification tree76.23% (93/122)
Linear SVM76.23% (93/122)
Ridge classifier74.59% (91/122)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Nuvoli, S.; Bianconi, F.; Rondini, M.; Lazzarato, A.; Marongiu, A.; Fravolini, M.L.; Cascianelli, S.; Amici, S.; Filippi, L.; Spanu, A.; et al. Differential Diagnosis of Alzheimer Disease vs. Mild Cognitive Impairment Based on Left Temporal Lateral Lobe Hypomethabolism on 18F-FDG PET/CT and Automated Classifiers. Diagnostics 2022, 12, 2425. https://doi.org/10.3390/diagnostics12102425

AMA Style

Nuvoli S, Bianconi F, Rondini M, Lazzarato A, Marongiu A, Fravolini ML, Cascianelli S, Amici S, Filippi L, Spanu A, et al. Differential Diagnosis of Alzheimer Disease vs. Mild Cognitive Impairment Based on Left Temporal Lateral Lobe Hypomethabolism on 18F-FDG PET/CT and Automated Classifiers. Diagnostics. 2022; 12(10):2425. https://doi.org/10.3390/diagnostics12102425

Chicago/Turabian Style

Nuvoli, Susanna, Francesco Bianconi, Maria Rondini, Achille Lazzarato, Andrea Marongiu, Mario Luca Fravolini, Silvia Cascianelli, Serena Amici, Luca Filippi, Angela Spanu, and et al. 2022. "Differential Diagnosis of Alzheimer Disease vs. Mild Cognitive Impairment Based on Left Temporal Lateral Lobe Hypomethabolism on 18F-FDG PET/CT and Automated Classifiers" Diagnostics 12, no. 10: 2425. https://doi.org/10.3390/diagnostics12102425

APA Style

Nuvoli, S., Bianconi, F., Rondini, M., Lazzarato, A., Marongiu, A., Fravolini, M. L., Cascianelli, S., Amici, S., Filippi, L., Spanu, A., & Palumbo, B. (2022). Differential Diagnosis of Alzheimer Disease vs. Mild Cognitive Impairment Based on Left Temporal Lateral Lobe Hypomethabolism on 18F-FDG PET/CT and Automated Classifiers. Diagnostics, 12(10), 2425. https://doi.org/10.3390/diagnostics12102425

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop