Next Article in Journal
Surgery in Advanced Ovary Cancer: Primary versus Interval Cytoreduction
Previous Article in Journal
Detection of Incomplete Irreversible Electroporation (IRE) and Microwave Ablation (MWA) of Hepatocellular Carcinoma (HCC) Using Iodine Quantification in Dual Energy Computed Tomography (DECT)
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 12
Issue 4
10.3390/diagnostics12040987
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Detection of Chronic Blast-Related Mild Traumatic Brain Injury with Diffusion Tensor Imaging and Support Vector Machines

by
Deborah L. Harrington
1,2,†,
Po-Ya Hsu
3,†,
Rebecca J. Theilmann
1,
Annemarie Angeles-Quinto
1,2,
Ashley Robb-Swan
1,2,
Sharon Nichols
4,
Tao Song
1,
Lu Le
5,
Carl Rimmele
5,
Scott Matthews
5,
Kate A. Yurgil
6,7,
Angela Drake
8,
Zhengwei Ji
1,
Jian Guo
9,
Chung-Kuan Cheng
3,
Roland R. Lee
1,2,
Dewleen G. Baker
7,10 and
Mingxiong Huang
1,2,*
1
Department of Radiology, University of California at San Diego, San Diego, CA 92121, USA
2
Research, Radiology, and Psychiatry Services, VA San Diego Healthcare System, San Diego, CA 92161, USA
3
Department of Computer Science and Engineering, University of California, San Diego, CA 92093, USA
4
Department of Neurosciences, University of California, San Diego, CA 92093, USA
5
ASPIRE Center, VASDHS Residential Rehabilitation Treatment Program, San Diego, CA 92110, USA
6
Department of Psychological Sciences, Loyola University, New Orleans, LA 70118, USA
7
VA Center of Excellence for Stress and Mental Health, San Diego, CA 92161, USA
8
Department of Psychiatry and Behavioral Medicine, University of California, Davis, CA 95817, USA
9
School of Computer Science, Nanjing University of Posts & Telecommunications, Nanjing 210023, China
10
Department of Psychiatry, University of California, San Diego, CA 92093, USA
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Diagnostics 2022, 12(4), 987; https://doi.org/10.3390/diagnostics12040987
Submission received: 15 February 2022 / Revised: 7 April 2022 / Accepted: 8 April 2022 / Published: 14 April 2022
(This article belongs to the Topic Medical Image Analysis)
Browse Figures
Review Reports Versions Notes

Abstract

:
Blast-related mild traumatic brain injury (bmTBI) often leads to long-term sequalae, but diagnostic approaches are lacking due to insufficient knowledge about the predominant pathophysiology. This study aimed to build a diagnostic model for future verification by applying machine-learning based support vector machine (SVM) modeling to diffusion tensor imaging (DTI) datasets to elucidate white-matter features that distinguish bmTBI from healthy controls (HC). Twenty subacute/chronic bmTBI and 19 HC combat-deployed personnel underwent DTI. Clinically relevant features for modeling were selected using tract-based analyses that identified group differences throughout white-matter tracts in five DTI metrics to elucidate the pathogenesis of injury. These features were then analyzed using SVM modeling with cross validation. Tract-based analyses revealed abnormally decreased radial diffusivity (RD), increased fractional anisotropy (FA) and axial/radial diffusivity ratio (AD/RD) in the bmTBI group, mostly in anterior tracts (29 features). SVM models showed that FA of the anterior/superior corona radiata and AD/RD of the corpus callosum and anterior limbs of the internal capsule (5 features) best distinguished bmTBI from HCs with 89% accuracy. This is the first application of SVM to identify prominent features of bmTBI solely based on DTI metrics in well-defined tracts, which if successfully validated could promote targeted treatment interventions.

1. Introduction

Traumatic brain injury (TBI) can lead to long-term physical, cognitive, and behavioral dysfunction [1,2,3]. Improvised explosive devices are a common cause of TBI in military personnel [4,5,6]. Among TBIs in the United States military, 82% are mild TBI (mTBI), with blast exposure being the leading cause [7]. The pathophysiology of blast-related mTBI (bmTBI) is not thoroughly understood, and controversy remains over its long-term consequences [8,9]. Currently, there are no optimal rehabilitation interventions, in part, because the mechanism (s) of injury are not fully understood [10]. Mild TBIs typically go undetected by standard diagnostic approaches, even in individuals with persistent post-concussive symptoms (PCS) [11,12,13,14]. This deficiency underscores the need for more sensitive techniques that characterize the predominant pathophysiology, which is essential for accurate diagnosis and targeted interventions.
In recent years, diffusion tensor imaging (DTI) has been used to investigate mTBI [15,16,17,18], as the pathophysiology disrupts microstructural white matter connections [19,20,21], which are not visible using conventional anatomical MRI (i.e., T1-weighted, T2-weighted, T2*-weighted, and fluid-attenuated inversion recovery (FLAIR) sequences) [22,23]. DTI measures in vivo water diffusion in tissues that are more restricted perpendicular as opposed to parallel to the fiber directions within axon bundles, thus permitting the local microstructural characteristics of tissue to be inferred [24,25]. Common metrics derived from anisotropic diffusivity are fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD), and radial diffusivity (RD) [15,26,27]. Although some research reports significant differences in patterns of diffusion metrics between healthy control (HC) and bmTBI cohorts, findings across studies have been inconsistent [15,24,28,29], partly owing to differences in the post-injury timeframe, which have included acute (<1 month post-injury) [30,31], subacute/chronic (1 month to 1 year) [32,33], and remote (>1 year) injuries [34,35,36]. Studies of military bmTBI, also differ in terms of whether control participants are civilians or military and whether they have a history of blast exposure [37]. Also, many studies include bmTBI participants with comorbidities that can influence DTI findings, including depression and post-traumatic stress disorder (PTSD) [34,36,37,38]. DTI analytic approaches also differ amongst studies [34,39,40], including the use of metric(s) that can elucidate the pathogenesis of white-matter injury, which requires analyses that evaluate not only the degree of anisotropy, but also the components (AD and RD eigenvalues) of FA and MD [41,42]. Collectively, these factors can influence DTI outcomes.
Despite the ability of DTI to detect white-matter abnormalities in mTBI at the group level, its low sensitivity (18–41%) limits its use for individual diagnosis. Most studies of mTBI have employed a single DTI metric (e.g., FA) in specific white-matter regions, which can be reliable for comparing two groups, but not for diagnostic categorization [15,24,28]. For example, FA sensitivities for identifying individuals with mTBI were 41% for the anterior corona radiata, 29% for the uncinate fasciculus, 21% for the inferior longitudinal fasciculus, 18% for the cingulum bundle, and 29% for the genu of corpus callosum [43]. Another study found that only 26% of mTBI patients could be accurately identified based on FA in the cingulum, uncinate fasciculus, and anterior limb of the internal capsule [44]. The lack of diagnostic sensitivity impels the need for leveraging more powerful approaches. Specifically, analytic approaches that evaluate multiple DTI metrics from white-matter tracts throughout the brain are be needed to understand optimal combinations of metrics from different brain regions that better distinguish mTBI patients from controls [15,45,46]. In this regard, machine learning (ML) approaches may pave the way for developing personalized applications of neuroimaging data [25]. One such approach is support vector machines (SVM), which can handle classification challenges associated with multivariate datasets where the number of dimensions can exceed the number of cases [47], which is common in medicine where obtaining large datasets for some clinical populations can be challenging.
The present study leveraged SVM modeling to uncover optimal combinations of DTI metrics from multiple tracts that best distinguished white-matter abnormalities in subacute/chronic bmTBI participants from healthy controls (HC). As our study aimed to build a diagnostic model for future verification, feature selection and model evaluation were performed using the same sample, as in many studies of mTBI [29,39,48,49,50]. Although the use of nonindependent samples can introduce bias into model evaluation, approaches to limiting bias included the selection of the best features to guard against overfitting of the data [51]. Rather than focusing on a single metric, clinically relevant features for modeling were selected using voxelwise tract-based analyses that identified group differences throughout white-matter tracts in five DTI metrics, which together elucidate the pathogenesis of white-matter injuries. Subsequently, SVM models with cross-validation were applied to the metrics of these thresholded clusters to identify sets of consensus features that best classified each participant as either bmTBI or HC. We hypothesized that the SVM method would enhance sensitivity beyond that of any individual metric from the tract-based analysis, thereby distinguishing prominent features of mild blast-related injuries that could potentially provide a promising diagnostic model to guide future validation studies.

2. Methods

2.1. Participants and Assessments

Participants were 19 HC without a history of TBI and 20 bmTBI with persistent post-concussive symptoms (PCS). All participants were male combat-deployed active-duty service members or veterans from the Operation Enduring Freedom/Operation Iraqi Freedom era. The study was approved by institutional review boards of the VA San Diego Healthcare System, San Diego, CA, USA and Naval Health Research Center, San Diego, CA, USA. Participants gave written informed consent for all procedures. Characteristics of the two groups are described in Table 1. The number of days post-injury ranged between 130 and 758. Fifteen bmTBI participants were tested 4 months to 1-year post-injury (subacute/early chronic) and 5 participants were tested 1 to 2 years post-injury (remote) 28. MRI scans were reviewed by a board-certified neuroradiologist (RRL) and were negative in all bmTBI participants (see Supplementary Figure S1 for examples). Two bmTBI participants had a PTSD diagnosis. There were no group differences in age or years of education.
The diagnosis of bmTBI was based on standard Veterans Affairs/Department of Defense diagnostic criteria [52]: (1) loss of consciousness <30 min or transient confusion, disorientation, or impaired consciousness immediately after the trauma; (2) post-traumatic amnesia <24 h; and (3) an initial Glasgow Coma Scale [53] between 13–15. Using a modified Head Injury Symptom Checklist (HISC) [54], ongoing PCS were assessed for 21 categories (Table 2). Participants with persistent symptoms in at least three of the categories were recruited into the study. Exclusion criteria for study participation included: (1) prior head injury; (2) history of other neurological, developmental or psychiatric disorders (e.g., brain tumor, stroke, epilepsy, bipolar disorder, major depressive disorder prior to injury, or self-reported diagnosis of learning disability); (3) substance or alcohol abuse (DSM-V criteria) within the six months prior to the study, based on a clinical interview; (4) currently taking sedative neuroleptics and hypnotic medications; and (5) suicidal thoughts or wishes, confirmed in follow-up risk assessment.
Subtests from the Delis-Kaplan Executive Function System (D-KEFS) [55] and the Wechsler Adult Intelligence Scale Version 3 (WAIS-III) [56] respectively assessed executive function and processing speed (Table 1), two cognitive domains that are vulnerable in mTBI [57]. The Number-Letter Sequencing and Category Switching subtests test to measure cognitive flexibility and set shifting. The Letter Fluency and Category Fluency subtests assess phonemic and semantic retrieval. Processing speed was assessed by the Symbol Search and Digit Symbol Coding subtests. Tests were performed in a single session, within one week of the MRI session. All scores were age-corrected scaled scores from normative data.

2.2. Imaging Protocols

Imaging was conducted at the UCSD Radiology Imaging Laboratory using a General Electric (Chicago, IL, USA) 1.5 T HD Excite Twin Speed MRI system with an eight-channel head coil. T1-weighted high-resolution anatomic images were collected (Spoiled Gradient Recalled, TR = 10.6 milliseconds (ms), TE = 43.8 ms, FOV = 24 cm, slice thickness = 1.2 millimeter (mm), NEX = 1, Flip Angle = 10°). Two-dimension T2-weighted images were AC/PC aligned and acquired with a fractional fast spin echo sequence (FRFSE-XL, TR = 6462, TE = 86 ms, number of slices = 33 with no gap, slice thickness = 4 mm, FOV = 25.6 cm, matrix = 320 × 256). The imaging protocol also included axial T2*-weighted images (TR = 500 ms, TE = 30 ms, number of slices = 33, slice thickness/space = 4 mm/4.4 mm, matrix = 320 × 192) and axial FLAIR images (TR = 8800 ms, TI = 2200 ms, TE = 80 ms, FOV = 25 cm, number of slices = 25 with no gap, slice thickness = 4 mm, matrix = 320 × 224). DTI images were acquired in the axial plane using a single-shot EPI sequence with diffusion encoding along 51 directions, b-value = 1000 s/mm2, five non-diffusion weighted image (b0), 56 slices, thickness = 2.5 mm, TR = 15.1 s, TE = 80.4 ms, matrix = 96 × 96 mm (automatically re-gridded onto a 128 × 128 matrix), FOV = 24 cm, and voxel size 1.875 × 1.875 × 2.5 mm3. The same number of slices with no gap were used for all participants. Study participants underwent two DTI acquisitions to increase the signal-to-noise ratio and ensure at least one artifact-free scan. The acquisition time for each DTI scan was 13 min.

2.3. DTI Analyses

DTI analyses were conducted using the Functional Magnetic Resonance Imaging of the Brain Software Library (FSL) version 4.1.5 (http://www.fmrib.ox.ac.uk/fsl, accessed on 10 February 2019) (Supplementary Materials 1.0 for details). For each participant, images from each DTI acquisition were concatenated into a single dataset and corrected for eddy currents and motion by registering all brain volumes to the first non-diffusion weighted image using a rigid body transformation (no rotation). By avoiding rotations in our motion correction (i.e., we only considered motion in the X, Y, and Z direction), issues with changing the encoding direction prior to the analysis were avoided. Each DTI scan was visually inspected to ensure the absence of susceptibility artifacts and significant rotation in the brain volume. If significant motion was present, the entire brain volume was removed from the dataset (Supplementary Materials 2.0 for details). Data were fit to the diffusion tensor model for each voxel using FMRIB’s Diffusion Toolbox (FDT) [58]. The tensor was then diagonalized to three eigenvalues which are used to calculate FA, RD, MD, and AD. We also analyzed the AD RD ratio to evaluate the shape (length over width) of the fiber [59], since FA alone is insufficient to confirm specific pathologic mechanisms of mTBI [42] such as inflammation or axonal cytotoxic edema in mTBI [50,60,61,62].
Voxel-wise statistical analyses were carried out using track-based statistical analyses (TBSS) [63]. First, a study-specific FA template image was produced by aligning all participants’ FA data to Montreal Neurological Institute (MNI) 152 standard space by non-linear registration. From the FA template, a white matter skeleton was created with an FA threshold > 0.20 to represent the core of white-matter tracts common to all participants. Group differences in FA within the skeleton were tested by applying the FSL general linear model (GLM) tool. Permutation testing (5000) was then conducted to establish voxel-by-voxel p-values to quantify group differences in the DTI parameter (FA). In each permutation, group differences were evaluated by nonparametric two-sample t-tests on MNI152 standard space. Threshold Free Cluster Enhancement [64] was used to identify clusters of FA that significantly differed between the two groups (familywise error rate, FWE, corrected p < 0.05) after permutation statistical testing). Using the same registration matrix for FA (standard TBSS procedure), AD, RD, MD, and AD/RD were registered to the FA skeleton in MNI152 standard space and then voxel-wise tests for group differences in these metrics were conducted.

2.4. SVM Analyses with Cross Validation

DTI features for the SVM analyses were selected using the following procedures (Supplementary Figures S2 and S3). First, clusters that showed significant group differences in the most representative DTI metric were identified as voxels located within a white-matter tract defined by the International Consortium of Brain Mapping (ICBM) DTI-81 atlas [65]. We selected FA as the representative metric since the largest number of voxels were found for this metric in tests for group differences. Next, we included the remaining DTI metrics that showed significant group differences. The features for the SVM analyses for MD, AD, RD, and AD/RD were those lying within the representative FA cluster mask. The mean DTI metric (i.e., averaged across voxels within the cluster mask) was computed for each cluster that showed significant group differences, which served as the inputs into the SVM models.
Stepwise SVM-based models were implemented to distinguish bmTBI from HCs based on linear combinations of multiple diffusion metrics from clusters in multiple white-matter regions as identified above (2.4). Supplementary Figures S2 and S3 show a flowchart and visualization of the procedure to construct the classification model. The complete procedure includes one algorithm and three routines (see Supplementary Materials 3.0). The core concept of the procedure is to try all possible combinations of the DTI metric set and select the optimal combination (s). For each subset of metrics, features for classification are selected by removing the least relevant feature recursively.
Cross-validation was used to evaluate the performance of a built model with a small dataset [66]. Cross-validation is a robust technique that provides enhanced confidence of ML classification results through the validation of testing datasets with the model built on training datasets [49,51]. Thus, combining SVM together with cross-validation can potentially generate a robust model of features that best separate bmTBI participants from HC. In this study, we randomly leave five subjects out (k = 5) in each validation (10,000) because normally, the segmentation of validation data in small datasets is around 10−15% [67]. The process randomly selects five samples out of the combined bmTBI and HC cohorts and calculates the averaged feature weights, correctness, sensitivity, and specificity across validations (Supplementary Materials 3.0).

2.5. Correlations between SVM Model Features and Behavioral Variables

The Statistical Package for the Social Sciences (SPSS version 28) was used to perform Pearson correlations, which tested for relationships between the DTI features in the optimal SVM model and neuropsychological variables that showed significant group differences (false discovery rate adjusted). These correlation analyses were conducted for the combined bmTBI and HC groups. For the bmTBI group only, the same DTI features were also correlated with total number of PCS and days post-injury.

3. Results

3.1. Neuropsychological Test Performance

Mean neuropsychological test performances fell within the average range for both groups (Table 1). Scores on Category Switching (total correct responses) were significantly lower in the bmTBI than the HC group (p < 0.05). A subthreshold trend for group differences was observed for Letter Fluency (p = 0.055).

3.2. TBSS Results

Among the five metrics, we found significant group differences in FA, RD, and AD/RD (Table 3), but not MD or AD. FA (Figure 1a) and AD/RD (Figure 1c) were significantly higher in the bmTBI group than the HC group in the genu and body of the corpus callosum (gCC/bCC), the anterior corona radiata (ACR), the anterior and posterior limbs of the internal capsule (ALIC/PLIC), and the superior corona radiata (SCR) (p < 0.05, FWE corrected). RD was reduced in the bmTBI group in all aforementioned regions except the left PLIC (Figure 1b).
For all white-matter features listed in Table 3, power analyses were performed using G*Power [68]. All features showed power > 0.80 at two-tailed α = 0.05 level, with the exceptions of the left ACR RD feature (power = 0.71) and the left PLIC RD feature (power = 0.37).

3.3. SVM with Cross-Validation Results

Figure 2 (top) displays the tracts from which aberrant DTI features in bmTBI from the TBSS analyses (Table 3) were extracted for the SVM analyses. We chose FA as the representative metric since FA had the largest number of voxels showing significant group differences in TBBS analyses. We then selected the voxels lying within significantly increased FA and the ICBM-DTI-81 white-matter atlas as the mask for generating FA, RD, and AD/RD features for stepwise SVM modeling. Altogether, 29 features were selected for SVM modeling (Table 3).
Figure 2 (bottom) illustrates how SVM separates the two groups using the two dominant DTI features (i.e., largest weightings) to draw the separation. Supplementary Figure S1b demonstrates the averaged error in each step after recursively removing the least contributing feature. Experiments to build the SVM model were conducted with both the inclusion and exclusion of the AD/RD metric. Without AD/RD, in which the four standard DTI metrics were considered, the FA-only model was the best SVM model, showing 75% accuracy, 82% sensitivity, and 79% specificity. Despite group differences in RD, the RD-only model achieved 72% accuracy, 77% sensitivity, and 64% specificity. Furthermore, when both FA and RD were input features in an SVM model, the FA-only model was output by the SVM algorithm. With both FA and AD/RD as input features, performance of the SVM model was boosted, showing 89% accuracy, 90% sensitivity, and 88% specificity.
Table 4 lists the five DTI features and associated weightings that were selected by our best stepwise SVM model. The top three features based on their weightings were dominated by AD/RD for left ALIC, gCC, and right ALIC. The fourth and the fifth features were FA of the right ACR and the left SCR. In the SVM model, AD/RD for the left ALIC took on the heaviest weighting.

3.4. Correlations between the SVM Features and Behavioral Variables

In the entire sample, weighted combinations of DTI features from the best SVM model did not significantly correlate with neuropsychological test scores, nor did individual features from this model. SVM features were also not correlated with total number of PCS and days post-injury in the bmTBI group.

4. Discussion

We identified an optimal linear combination of DTI features that distinguished bmTBI patients from HC with 89% overall accuracy, 90% sensitivity, and 88% specificity in cross-validation tests. The most discerning DTI features of bmTBI consisted of abnormally increased FA in right ACR and left SCR together with elevated AD/RD in the gCC and ALIC bilaterally. Despite high classification accuracy, SVM features failed to correlate with neuropsychological test scores or clinical variables, possibly due to relatively intact executive functioning of the bmTBI group on most tests [69], although statistical tests may also have been underpowered given the relatively small sample sizes.
To our knowledge, this is the first study to use ML to distinguish bmTBI and HC solely based on DTI features. Importantly, feature selection was constrained by outcomes from tests of group differences in five DTI metrics to select features that were clinically relevant for diagnostic modeling and to avoid overfitting of the data [51]. Vergara’s group also used an SVM approach to characterize white-matter features in noncombat mTBI patients [48]. Both FA and resting-state functional network connectivity (rsFNC) served as features in their models. They reported that functional connectivity within the default mode network provided the best classification accuracy (84%) followed by FA (75.5%) as indexed by a Z score that represented FA abnormalities throughout spatially heterogeneous white-matter relative to the control group. Their findings are similar to ours in terms of both increased FA in mTBI and our 75% correct classification for FA alone, despite measuring FA from clusters within skeleton tracts. Yet modeling of both rsFNC and FA failed to boost classification accuracy (74.5%) in their study, whereas our classification accuracy (89%) was improved by combining the AD/RD ratio and FA features. The performance of our SVM model indicates that multivariate combinations of different diffusion metrics in well-defined tracts are potentially potent markers of bmTBI. Our model also identified the most discerning abnormal anatomical tracts that together generated the best SVM model, which was not investigated by Vergara’s group.

4.1. Abnormal Diffusivity in bmTBI

Increased FA coupled with decreased RD are probably due to the deformation of the microstructural white-matter regions caused by blast, which can stretch and twist the axons [70,71]. This distortion is likely to tighten white-matter fiber bundles and disrupt neurofilaments and microtubules that elongate down the axons. Secondary to the compacted tightened axons and disrupted neuronal substances, the axons swell and reduce extracellular space [72], thereby restricting water diffusion perpendicular to axons [72]. Such restriction limits the movement of water molecules, leading to abnormally decreased RD, increased AD/RD, and by inference, increased FA [50]. Our results align with studies of acute mTBI in adolescents [73,74], acute/subacute mTBI [50] and bmTBI with comorbid PTSD [30], and acute/chronic sports-related concussions [75]. Rats and swine with blast-induced TBI also show increased FA [70,76]. Moreover, rats with concussive injury show increased FA and decreased RD, but normal AD relative to rats with sham injury [42].
The post-injury timeframe in our bmTBI cohort was somewhat constrained (four months to two years) to reduce heterogeneity on a variable that influences neuroimaging outcomes. For example, abnormally increased and decreased FA have been found in bmTBI during acute post-injury periods of less than four months [28,30,31] and after one to 12 months post injury [30,33]. Yet for people with remote post-injury times longer than two years, FA is typically decreased in bmTBI [34,36,40,77,78,79]. Collectively, FA appears to decrease for post-injury periods longer than two years, whereas for shorter post-injury durations, discrepant findings could be explained by different fiber recovery stages in subacute injuries [71] and the long term misalignments of fibers and axonal degeneration in early-chronic mTBI [80]. Future research is needed to systematically track longitudinal changes in white-matter diffusivity including DTI components (AD, RD, AD/RD) that can elucidate the recovery from pathological mechanisms of blast injuries.

4.2. Optimal Features of bmTBI

Of the 29 DTI features selected for SVM modeling, five features in tracts that best distinguish bmTBI from HCs involved the ALIC, ACR, SCR, and gCC, which aligns with the preponderance of abnormal diffusivity in anterior regions of in TBI [81,82,83,84]. Indeed, frontal areas are the most vulnerable to blast exposures owing to the design of military helmets [81,83]. Corpus callosum injury is also common because non-impact rotational acceleration from the lateral direction can produce uniform traumatic callosal injury [85,86,87]. The damage seen in corona radiata is also consistent with a rotational shearing mechanism [43].
Unlike many studies, we included AD/RD as a scalar metric, as recommended by others [59] who found that AD/RD in more cylinder-like tracts best differentiated older and younger adults. The addition of AD/RD to the SVM model boosted classification accuracy of bmTBI from 75% to almost 90%. As water diffusivity in corpus callosum and corticospinal tracts are better approximated as cylindrical diffusion, AD/RD is an important metric that should be used in DTI research.
Our DTI feature selection method is biologically reasonable and computationally efficient. The choice of features was based on well-accepted anatomical templates, and our SVM approach focused on regional group differences in white-matter tracts, rather than whole-brain white matter, because injuries in bmTBI are typically focal [88]. By using the average metric of voxels within a tract as a feature to reduce the dimensionality of the data and minimize overfitting of the SVM model [89], this also reduced computational costs, which otherwise become large and difficult to handle [90].

4.3. Limitations

Limitations include the small bmTBI and control cohorts, which could bias the selection of abnormal clusters for the SVM analyses. Specifically, we applied feature selection to the whole dataset in a supervised mode, which can bias the SVM approach. Ideally, feature selection is applied using the SVM training dataset, and then SVM performance is evaluated using the testing dataset. The consistency of the extracted DTI features across the cross-validation folds can then be evaluated. This approach was not used in the present study owing to our small sample size, for which instability of the selected features may exist if not using the whole data set. An alternative approach is to test the robustness of the selected features using new samples, which is what we recommend for future studies. In this regard, larger samples may uncover regions of interest not identified in our study. At the same time, our results may be more specific to mild blast injuries owing to a more homogeneous bmTBI cohort than past studies, with only two patients having PTSD and none with depression or other psychiatric conditions. Likewise, age and years education did not differ between our bmTBI and controls, who were also Veterans/military personnel with a history of blast exposure, which adjusts for potential effects of occupational exposure to catastrophic experiences on neurotrauma. Second, owing the high comorbidity of PTSD in veterans with blast injuries, two participants with comorbid bmTBI-PTSD were included in our study, as considerable overlap would be expected in the white-matter tracts that are affected by a blast force, regardless of PTSD. Indeed, a bmTBI study of people with and without comorbid PTSD reported distributed white-matter pathology in tracts that overlapped with our study, yet pathology was not related to a diagnosis of PTSD or depressive symptoms [77]. Moreover, in univariate tract-based analyses we failed to find group differences in FA of the cingulum, which is common in PTSD [34,91,92,93]. Altogether, it is unlikely that our results are confounded by the inclusion of two comorbid participants. Still, relationships between bmTBI and PTSD are not well understood [94], and future studies are needed that compare bmTBI, PTSD, and comorbid cohorts to fully understand similarities and differences in the pathogenesis of white-matter disturbances amongst these conditions. Third, similar to many studies of mTBI or bmTBI [29,39,48,49] nonindependent samples were used to define DTI abnormalities and estimate classification accuracy, which can overestimate diagnostic utility. Our focus on selecting clinically relevant features for modeling may have minimized this problem, together with cross-validation of SVM models wherein consensus features defined diagnostic accuracy [49,51]. Nonetheless, this is the first application of SVM using DTI metrics, thus replication cohorts will be needed to validate the sensitivity and specificity of the optimal SVM model generated in our study. More generally, replication is always important as bmTBI cohorts often differ in the extent to which they present with comorbidities (e.g., PTSD, depression) or other features (e.g., prior head injury, positive MRI) that can affect DTI results.
Heterogeneous injuries across individuals from diverse mechanical forces may also play a role in discrepant findings. For this reason, some studies employ techniques that quantify normal and abnormal FA throughout spatially heterogeneous white matter [48]. In contrast to our approach, however, classification accuracy is modest (≈75%) possibly due to using FA as the sole metric and/or absence of information about the spatial location of injuries. Our results indicated that the combination of multiple DTI metrics in well-defined tracts is essential for improving diagnostic accuracy. Moreover, the ROIs uncovered in our study rested in frontal tracks, which are the most vulnerable to blast exposures [81,83,85,86,87], and therefore may be the most potent markers of bmTBI, irrespective of abnormalities in other white-matter tract injuries. Future studies comparing tract-based and spatially heterogeneous approaches, which employ multiple DTI metrics, would elucidate the relative value of these approaches in handling diagnostic challenges associated with heterogeneous injuries across individuals.

5. Conclusions

DTI biomarkers derived from modeling in the present study SVM differentiated bmTBI and HCs with high sensitivity and specificity. These preliminary findings indicate that the most potent discriminating features rested in anterior tracks for which AD/RD and FA were abnormally increased. The high classification accuracy of these features points to their potential for diagnosis, which we plan to validate in larger, independent cohorts. If our findings are successfully validated, this knowledge could promote treatment interventions such as transcranial magnetic or electrical stimulation. For example, we found that transcranial electrical stimulation treatments in people with chronic mTBI from blast and non-blast injuries markedly reduced or eliminated persistent post concussive symptoms and reduced abnormal slow-waves in approximately the same brain areas that showed abnormal slow-wave generation prior to treatment [95]. Transcranial magnetic stimulation also reduces post-concussive depression, headaches, and may improve cognition [96,97,98,99]. Both stimulation approaches, however, are not typically guided by knowledge about specific regions of brain damage. Rather, stimulation is usually administered at the same anatomical site in all participants, which may limit treatment effectiveness for some patients. Knowledge about the most prominent loci of abnormal white matter disturbances may therefore guide the development of stimulation targets that are optimal for individual patients. Lastly, longitudinal changes in regional diffusion metrics should also be tracked to elucidate mechanisms of recovery, which could also have important implications for clinical interventions.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12040987/s1. References [66,67] are also cited in the Supplementary Materials.

Author Contributions

D.L.H. aided in the execution of the analyses, interpreted the results, and wrote the research report. P.-Y.H. executed the DTI and SVM analyses. R.J.T. supervised the DTI analyses and contributed to the interpretation of the results. A.A.-Q., A.R.-S. and S.N. collected the DTI and neuropsychological data. C.-K.C. oversaw the SVM analyses and revised the manuscript. T.S., Z.J. and J.G. contributed to the DTI acquisition and interpretation. L.L., C.R., S.M. and A.D. played important roles in subject recruitment, symptom assessment, and data interpretation. R.R.L. reviewed MRI scans. D.G.B. and K.A.Y. supervised the clinical assessment and contributed to the interpretation of the results and revised the manuscript. M.H. conceived, designed, aided in the execution of the research, interpreted the results, and wrote revisions of the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by Merit Review Grants from the U.S. Department of Veterans Affairs (P.I.: M.X.H., I01-CX002035-01, NURC-007-19S, I01-CX000499, MHBA-010-14F, I01-RX001988, B1988-I, NURC-022-10F, NEUC-044-06S; P.I.: D.L.H., I01-CX000146), by Naval Medical Research Center’s Advanced Medical Development program (Naval Medical Logistics Command Contract #N62645-11-C-4037, for MRS-II to D.G.B. and M.X.H.), and by Congressionally Directed Medical Research Programs/Department of Defense (P.I.: D.G.B., W81XWH-16-1-0015).

Institutional Review Board Statement

The study protocol (VA IRB # 060812) was approved by institutional review boards of the VA San Diego Healthcare System and Naval Health Research Center at San Diego, CA, USA.

Informed Consent Statement

All participants gave written informed consent prior to study procedures. The informed consent followed the ethical guidelines of the Declarations of Helsinki (sixth revision, 2008).

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions imposed by U.S. Department of Veterans Affairs and U.S. Department of Defense.

Acknowledgments

The authors would like to thank Qian Shen for her technical support.

Conflicts of Interest

The authors declare that they have no competing financial interests or personal relationships that could appear to influence the work reported in this paper.

References

  1. Langlois, J.A.; Rutland-Brown, W.; Wald, M.M. The epidemiology and impact of traumatic brain injury: A brief overview. J. Head Trauma Rehabil. 2006, 21, 375–378. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Roozenbeek, B.; Maas, A.I.; Menon, D.K. Changing patterns in the epidemiology of traumatic brain injury. Nat. Rev. Neurol. 2013, 9, 231–236. [Google Scholar] [CrossRef] [PubMed]
  3. Rubiano, A.M.; Carney, N.; Chesnut, R.; Puyana, J.C. Global neurotrauma research challenges and opportunities. Nature 2015, 527, S193–S197. [Google Scholar] [CrossRef] [PubMed]
  4. Clark, A.L.; Bangen, K.J.; Sorg, S.F.; Schiehser, D.M.; Evangelista, N.D.; McKenna, B.; Liu, T.T.; Delano-Wood, L. Dynamic association between perfusion and white matter integrity across time since injury in Veterans with history of TBI. Neuroimage Clin. 2017, 14, 308–315. [Google Scholar] [CrossRef]
  5. Hoge, C.W.; McGurk, D.; Thomas, J.L.; Cox, A.L.; Engel, C.C.; Castro, C.A. Mild traumatic brain injury in US soldiers returning from Iraq. N. Engl. J. Med. 2008, 358, 453–463. [Google Scholar] [CrossRef]
  6. Porter, K.E.; Stein, M.B.; Martis, B.; Avallone, K.M.; McSweeney, L.B.; Smith, E.R.; Simon, N.M.; Gargan, S.; Liberzon, I.; Hoge, C.W.; et al. Postconcussive symptoms (PCS) following combat-related traumatic brain injury (TBI) in Veterans with posttraumatic stress disorder (PTSD): Influence of TBI, PTSD, and depression on symptoms measured by the Neurobehavioral Symptom Inventory (NSI). J. Psychiatr. Res. 2018, 102, 8–13. [Google Scholar] [CrossRef]
  7. Turner, P.R. TBI in military and veterans: Incidence, symptoms, and resources. In Proceedings of the North Star Holistic Health Conference, Penrith, UK, 22–23 April 2015. [Google Scholar]
  8. Bigler, E.D.; Bazarian, J.J. Diffusion tensor imaging: A biomarker for mild traumatic brain injury? Neurology 2010, 74, 626–627. [Google Scholar] [CrossRef] [Green Version]
  9. DePalma, R.G.; Hoffman, S.W. Combat blast related traumatic brain injury (TBI): Decade of recognition promise of progress. Behav. Brain Res. 2018, 340, 102–105. [Google Scholar] [CrossRef]
  10. Gardner, R.C.; Yaffe, K. Epidemiology of mild traumatic brain injury and neurodegenerative disease. Mol. Cell. Neurosci. 2015, 66, 75–80. [Google Scholar] [CrossRef] [Green Version]
  11. Bigler, E.D. Structural neuroimaging in sport-related concussion. Int. J. Psychophysiol. 2018, 132, 105–123. [Google Scholar] [CrossRef]
  12. Bigler, E.D.; Orrison, W.W. Neuroimaging in sports-related brain injury. In Traumatic Brain Injury in Sports; Taylor & Francis: Abingdon, UK, 2020; pp. 71–93. [Google Scholar]
  13. Johnston, K.M.; Ptito, A.; Chankowsky, J.; Chen, J.K. New frontiers in diagnostic imaging in concussive head injury. Clin. J. Sport Med. Off. J. Can. Acad. Sport Med. 2001, 11, 166–175. [Google Scholar] [CrossRef] [PubMed]
  14. Kirkwood, M.W.; Yeates, K.O.; Wilson, P.E. Pediatric sport-related concussion: A review of the clinical management of an oft-neglected population. Pediatrics 2006, 117, 1359–1371. [Google Scholar] [CrossRef] [PubMed]
  15. Douglas, D.B.; Iv, M.; Douglas, P.K.; Anderson, A.; Vos, S.B.; Bammer, R.; Zeineh, M.; Wintermark, M. Diffusion Tensor Imaging of TBI: Potentials and Challenges. Top. Magn. Reson. Imaging 2015, 24, 241–251. [Google Scholar] [CrossRef]
  16. Niogi, S.N.; Mukherjee, P. Diffusion tensor imaging of mild traumatic brain injury. J. Head Trauma Rehabil. 2010, 25, 241–255. [Google Scholar] [CrossRef] [PubMed]
  17. Shenton, M.E.; Hamoda, H.M.; Schneiderman, J.S.; Bouix, S.; Pasternak, O.; Rathi, Y.; Vu, M.-A.; Purohit, M.P.; Helmer, K.; Koerte, I.; et al. A review of magnetic resonance imaging and diffusion tensor imaging findings in mild traumatic brain injury. Brain Imaging Behav. 2012, 6, 137–192. [Google Scholar] [CrossRef]
  18. Wallace, E.J.; Mathias, J.L.; Ward, L. Diffusion tensor imaging changes following mild, moderate and severe adult traumatic brain injury: A meta-analysis. Brain Imaging Behav. 2018, 12, 1607–1621. [Google Scholar] [CrossRef]
  19. Geschwind, N. Disconnexion syndromes in animals and man. Sel. Pap. Lang. Brain 1974, 105–236. [Google Scholar]
  20. Peerless, S.J.; Rewcastle, N.B. Shear injuries of the brain. Can. Med. Assoc. J. 1967, 96, 577. [Google Scholar] [PubMed]
  21. Strich, S.J. Diffuse degeneration of the cerebral white matter in severe dementia following head injury. J. Neurol. Neurosurg. Psychiatry 1956, 19, 163. [Google Scholar] [CrossRef] [Green Version]
  22. Christopher Fox, W.; Park, M.S.; Belverud, S.; Klugh, A.; Rivet, D.; Tomlin, J.M. Contemporary imaging of mild TBI: The journey toward diffusion tensor imaging to assess neuronal damage. Neurol. Res. 2013, 35, 223–232. [Google Scholar] [CrossRef]
  23. Xiong, K.; Zhu, Y.; Zhang, W. Diffusion tensor imaging and magnetic resonance spectroscopy in traumatic brain injury: A review of recent literature. Brain Imaging Behav. 2014, 8, 487–496. [Google Scholar] [CrossRef] [PubMed]
  24. Hayes, J.P.; Bigler, E.D.; Verfaellie, M. Traumatic brain injury as a disorder of brain connectivity. J. Int. Neuropsychol. Soc. 2016, 22, 120–137. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Sharp, D.J.; Scott, G.; Leech, R. Network dysfunction after traumatic brain injury. Nat. Rev. Neurol. 2014, 10, 156–166. [Google Scholar] [CrossRef]
  26. Jellison, B.J.; Field, A.S.; Medow, J.; Lazar, M.; Salamat, M.S.; Alexander, A.L. Diffusion tensor imaging of cerebral white matter: A pictorial review of physics, fiber tract anatomy, and tumor imaging patterns. Am. J. Neuroradiol. 2004, 25, 356–369. [Google Scholar] [PubMed]
  27. Marrale, M.; Collura, G.; Brai, M.; Toschi, N.; Midiri, F.; La Tona, G.; Casto, A.L.; Gagliardo, C. Physics, techniques and review of neuroradiological applications of diffusion kurtosis imaging (DKI). Clin. Neuroradiol. 2016, 26, 391–403. [Google Scholar] [CrossRef] [PubMed]
  28. Asken, B.M.; DeKosky, S.T.; Clugston, J.R.; Jaffee, M.S.; Bauer, R.M. Diffusion tensor imaging (DTI) findings in adult civilian, military, and sport-related mild traumatic brain injury (mTBI): A systematic critical review. Brain Imaging Behav. 2018, 12, 585–612. [Google Scholar] [CrossRef] [PubMed]
  29. Davenport, N.D.; Lim, K.O.; Armstrong, M.T.; Sponheim, S.R. Diffuse and spatially variable white matter disruptions are associated with blast-related mild traumatic brain injury. NeuroImage 2012, 59, 2017–2024. [Google Scholar] [CrossRef]
  30. Li, L.; Sun, G.; Liu, K.; Li, M.; Li, B.; Qian, S.-W.; Yu, L.-L. White matter changes in posttraumatic stress disorder following mild traumatic brain injury: A prospective longitudinal diffusion tensor imaging study. Chin. Med. J. 2016, 129, 1091. [Google Scholar] [CrossRef]
  31. Mac Donald, C.L.; Johnson, A.M.; Cooper, D.; Nelson, E.C.; Werner, N.J.; Shimony, J.S.; Snyder, A.Z.; Raichle, M.E.; Witherow, J.R.; Fang, R.; et al. Detection of blast-related traumatic brain injury in U.S. military personnel. N. Engl. J. Med. 2011, 364, 2091–2100. [Google Scholar] [CrossRef] [Green Version]
  32. Costanzo, M.E.; Chou, Y.-Y.; Leaman, S.; Pham, D.L.; Keyser, D.; Nathan, D.E.; Coughlin, M.; Rapp, P.; Roy, M.J. Connecting combat-related mild traumatic brain injury with posttraumatic stress disorder symptoms through brain imaging. Neurosci. Lett. 2014, 577, 11–15. [Google Scholar] [CrossRef]
  33. Warden, D.L.; French, L.M.; Shupenko, L.; Fargus, J.; Riedy, G.; Erickson, M.E.; Jaffee, M.S.; Moore, D.F. Case report of a soldier with primary blast brain injury. Neuroimage 2009, 47, T152–T153. [Google Scholar] [CrossRef] [PubMed]
  34. Davenport, N.D.; Lim, K.O.; Sponheim, S.R. White matter abnormalities associated with military PTSD in the context of blast TBI. Hum. Brain Mapp. 2015, 36, 1053–1064. [Google Scholar] [CrossRef] [PubMed]
  35. Holtkamp, M.D.; Grimes, J.; Ling, G. Concussion in the military: An evidence-base review of mTBI in US military personnel focused on posttraumatic headache. Curr. Pain Headache Rep. 2016, 20, 37. [Google Scholar] [CrossRef] [PubMed]
  36. Isaac, L.; Main, K.L.; Soman, S.; Gotlib, I.H.; Furst, A.J.; Kinoshita, L.M.; Fairchild, J.K.; Yesavage, J.A.; Ashford, J.W.; Bayley, P.J. The impact of depression on Veterans with PTSD and traumatic brain injury: A diffusion tensor imaging study. Biol. Psychol. 2015, 105, 20–28. [Google Scholar] [CrossRef]
  37. Hayes, J.P.; Miller, D.R.; Lafleche, G.; Salat, D.H.; Verfaellie, M. The nature of white matter abnormalities in blast-related mild traumatic brain injury. Neuroimage Clin. 2015, 8, 148–156. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Matthews, S.C.; Strigo, I.A.; Simmons, A.N.; O’Connell, R.M.; Reinhardt, L.E.; Moseley, S.A. A multimodal imaging study in US veterans of Operations Iraqi and Enduring Freedom with and without major depression after blast-related concussion. Neuroimage 2011, 54, S69–S75. [Google Scholar] [CrossRef]
  39. Jorge, R.E.; Acion, L.; White, T.; Tordesillas-Gutierrez, D.; Pierson, R.; Crespo-Facorro, B.; Magnotta, V.A. White matter abnormalities in veterans with mild traumatic brain injury. Am. J. Psychiatry 2012, 169, 1284–1291. [Google Scholar] [CrossRef] [Green Version]
  40. Miller, D.R.; Hayes, J.P.; Lafleche, G.; Salat, D.H.; Verfaellie, M. White matter abnormalities are associated with chronic postconcussion symptoms in blast-related mild traumatic brain injury. Hum. Brain Mapp. 2016, 37, 220–229. [Google Scholar] [CrossRef] [Green Version]
  41. Budde, M.D.; Kim, J.H.; Liang, H.-F.; Schmidt, R.E.; Russell, J.H.; Cross, A.H.; Song, S.-K. Toward accurate diagnosis of white matter pathology using diffusion tensor imaging. Magn. Reson. Med. 2007, 57, 688–695. [Google Scholar] [CrossRef]
  42. Hoogenboom, W.S.; Rubin, T.G.; Ye, K.; Cui, M.-H.; Branch, K.C.; Liu, J.; Branch, C.A.; Lipton, M.L. Diffusion Tensor Imaging of the Evolving Response to Mild Traumatic Brain Injury in Rats. J. Exp. Neurosci. 2019, 13, 1179069519858627. [Google Scholar] [CrossRef]
  43. Niogi, S.N.; Mukherjee, P.; Ghajar, J.; Johnson, C.; Kolster, R.A.; Sarkar, R.; Lee, H.; Meeker, M.; Zimmerman, R.D.; Manley, G.T. Extent of microstructural white matter injury in postconcussive syndrome correlates with impaired cognitive reaction time: A 3T diffusion tensor imaging study of mild traumatic brain injury. Am. J. Neuroradiol. 2008, 29, 967–973. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Kou, Z.; Wu, Z.; Tong, K.A.; Holshouser, B.; Benson, R.R.; Hu, J.; Haacke, E.M. The role of advanced MR imaging findings as biomarkers of traumatic brain injury. J. Head Trauma Rehabil. 2010, 25, 267–282. [Google Scholar] [CrossRef] [PubMed]
  45. Jeter, C.B.; Hergenroeder, G.W.; Hylin, M.J.; Redell, J.B.; Moore, A.N.; Dash, P.K. Biomarkers for the diagnosis and prognosis of mild traumatic brain injury/concussion. J. Neurotrauma 2013, 30, 657–670. [Google Scholar] [CrossRef] [PubMed]
  46. Zetterberg, H.; Smith, D.H.; Blennow, K. Biomarkers of mild traumatic brain injury in cerebrospinal fluid and blood. Nat. Rev. Neurol. 2013, 9, 201–210. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Burges, C.J. A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 1998, 2, 121–167. [Google Scholar] [CrossRef]
  48. Vergara, V.M.; Mayer, A.R.; Damaraju, E.; Kiehl, K.A.; Calhoun, V. Detection of Mild Traumatic Brain Injury by Machine Learning Classification Using Resting State Functional Network Connectivity and Fractional Anisotropy. J. Neurotrauma 2017, 34, 1045–1053. [Google Scholar] [CrossRef] [Green Version]
  49. Watts, R.; Thomas, A.; Filippi, C.G.; Nickerson, J.P.; Freeman, K. Potholes and molehills: Bias in the diagnostic performance of diffusion-tensor imaging in concussion. Radiology 2014, 272, 217–223. [Google Scholar] [CrossRef] [Green Version]
  50. Ling, J.M.; Pena, A.; Yeo, R.A.; Merideth, F.L.; Klimaj, S.; Gasparovic, C.; Mayer, A.R. Biomarkers of increased diffusion anisotropy in semi-acute mild traumatic brain injury: A longitudinal perspective. Brain 2012, 135, 1281–1292. [Google Scholar] [CrossRef]
  51. Ball, T.M.; Squeglia, L.M.; Tapert, S.F.; Paulus, M.P. Double Dipping in Machine Learning: Problems and Solutions. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 2020, 5, 261–263. [Google Scholar] [CrossRef]
  52. Statements, Q. VA/DoD clinical practice guideline for management of concussion/mild traumatic brain injury. J. Rehabil. Res. Dev. 2009, 46, 1–60. [Google Scholar]
  53. Teasdale, G.; Jennett, B. Assessment of coma and impaired consciousness: A practical scale. Lancet 1974, 304, 81–84. [Google Scholar] [CrossRef]
  54. Alvin, M., Jr.; Sureyya, D.; Allen, R.W.; James, L.G. Psychosocial functioning at 1 month after head injury. Neurosurgery 1984, 14, 393–399. [Google Scholar] [CrossRef]
  55. Delis, D.C.; Kaplan, E.; Kramer, J.H. Delis-Kaplan Executive Function System; The Psychological Corporation: San Antonio, TX, USA, 2001. [Google Scholar]
  56. Wechsler, D. WAIS-III Wechsler Adult Intelligence Scale; The Psychological Corporation: San Antonio, TX, USA, 1997; Volume 3. [Google Scholar]
  57. Robb Swan, A.; Nichols, S.; Drake, A.; Angeles, A.; Diwakar, M.; Song, T.; Lee, R.R.; Huang, M.-X. Magnetoencephalography Slow-Wave Detection in Patients with Mild Traumatic Brain Injury and Ongoing Symptoms Correlated with Long-Term Neuropsychological Outcome. J. Neurotrauma 2015, 32, 1510–1521. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  58. Behrens, T.E.; Woolrich, M.W.; Jenkinson, M.; Johansen-Berg, H.; Nunes, R.G.; Clare, S.; Matthews, P.M.; Brady, J.M.; Smith, S.M. Characterization and propagation of uncertainty in diffusion-weighted MR imaging. Magn. Reson. Med. 2003, 50, 1077–1088. [Google Scholar] [CrossRef] [PubMed]
  59. Hasan, K.M.; Narayana, P.A. Retrospective measurement of the diffusion tensor eigenvalues from diffusion anisotropy and mean diffusivity in DTI. Magn. Reson. Med. 2006, 56, 130–137. [Google Scholar] [CrossRef]
  60. Qin, Y.; Li, G.-L.; Xu, X.-H.; Sun, Z.-Y.; Gu, J.-W.; Gao, F.-B. Brain structure alterations and cognitive impairment following repetitive mild head impact: An in vivo MRI and behavioral study in rat. Behav. Brain Res. 2018, 340, 41–48. [Google Scholar] [CrossRef]
  61. Raikes, A.C.; Bajaj, S.; Dailey, N.S.; Smith, R.S.; Alkozei, A.; Satterfield, B.C.; Killgore, W.D.S. Diffusion Tensor Imaging (DTI) Correlates of Self-Reported Sleep Quality and Depression Following Mild Traumatic Brain Injury. Front. Neurol. 2018, 9, 468. [Google Scholar] [CrossRef]
  62. Khong, E.; Odenwald, N.; Hashim, E.; Cusimano, M.D. Diffusion Tensor Imaging Findings in Post-Concussion Syndrome Patients after Mild Traumatic Brain Injury: A Systematic Review. Front. Neurol. 2016, 7, 156. [Google Scholar] [CrossRef] [Green Version]
  63. Smith, S.M.; Jenkinson, M.; Johansen-Berg, H.; Rueckert, D.; Nichols, T.E.; Mackay, C.E.; Watkins, K.E.; Ciccarelli, O.; Cader, M.Z.; Matthews, P.M. Tract-based spatial statistics: Voxelwise analysis of multi-subject diffusion data. Neuroimage 2006, 31, 1487–1505. [Google Scholar] [CrossRef]
  64. Smith, S.M.; Nichols, T.E. Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference. Neuroimage 2009, 44, 83–98. [Google Scholar] [CrossRef] [Green Version]
  65. Hua, K.; Zhang, J.; Wakana, S.; Jiang, H.; Li, X.; Reich, D.S.; Calabresi, P.A.; Pekar, J.J.; van Zijl, P.C.; Mori, S. Tract probability maps in stereotaxic spaces: Analyses of white matter anatomy and tract-specific quantification. Neuroimage 2008, 39, 336–347. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  66. Shen, D.; Wu, G.; Suk, H.-I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 2017, 19, 221–248. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  67. Breiman, L. Statistical modeling: The two cultures. Qual. Control. Appl. Stat. 2003, 48, 81–82. [Google Scholar]
  68. Faul, F.; Erdfelder, E.; Buchner, A.; Lang, A.-G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods 2009, 41, 1149–1160. [Google Scholar] [CrossRef] [Green Version]
  69. Miller, D.R.; Hayes, J.P.; Lafleche, G.; Salat, D.H.; Verfaellie, M. White matter abnormalities are associated with overall cognitive status in blast-related mTBI. Brain Imaging Behav. 2017, 11, 1129–1138. [Google Scholar] [CrossRef]
  70. Browne, K.D.; Chen, X.-H.; Meaney, D.F.; Smith, D.H. Mild traumatic brain injury and diffuse axonal injury in swine. J. Neurotrauma 2011, 28, 1747–1755. [Google Scholar] [CrossRef] [Green Version]
  71. Povlishock, J.T.; Katz, D.I. Update of neuropathology and neurological recovery after traumatic brain injury. J. Head Trauma Rehabil. 2005, 20, 76–94. [Google Scholar] [CrossRef] [Green Version]
  72. Rosenblum, W.I. Cytotoxic edema: Monitoring its magnitude and contribution to brain swelling. J. Neuropathol. Exp. Neurol. 2007, 66, 771–778. [Google Scholar] [CrossRef] [Green Version]
  73. Chu, Z.; Wilde, E.A.; Hunter, J.V.; McCauley, S.R.; Bigler, E.D.; Troyanskaya, M.; Yallampalli, R.; Chia, J.M.; Levin, H.S. Voxel-based analysis of diffusion tensor imaging in mild traumatic brain injury in adolescents. Am. J. Neuroradiol. 2010, 31, 340–346. [Google Scholar] [CrossRef] [Green Version]
  74. Wilde, E.A.; Hunter, J.V.; Bigler, E.D. Neuroimaging in neurorehabilitation. NeuroRehabilitation 2012, 31, 223–226. [Google Scholar] [CrossRef]
  75. Henry, L.C.; Tremblay, J.; Tremblay, S.; Lee, A.; Brun, C.; Lepore, N.; Theoret, H.; Ellemberg, D.; Lassonde, M. Acute and chronic changes in diffusivity measures after sports concussion. J. Neurotrauma 2011, 28, 2049–2059. [Google Scholar] [CrossRef] [PubMed]
  76. Rubovitch, V.; Ten-Bosch, M.; Zohar, O.; Harrison, C.R.; Tempel-Brami, C.; Stein, E.; Hoffer, B.J.; Balaban, C.D.; Schreiber, S.; Chiu, W.-T. A mouse model of blast-induced mild traumatic brain injury. Exp. Neurol. 2011, 232, 280–289. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  77. Morey, R.A.; Haswell, C.C.; Selgrade, E.S.; Massoglia, D.; Liu, C.; Weiner, J.; Marx, C.E.; MIRECC Work Group; Cernak, I.; McCarthy, G. Effects of chronic mild traumatic brain injury on white matter integrity in Iraq and Afghanistan war veterans. Hum. Brain Mapp. 2013, 34, 2986–2999. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  78. Petrie, E.C.; Cross, D.J.; Yarnykh, V.L.; Richards, T.; Martin, N.M.; Pagulayan, K.; Hoff, D.; Hart, K.; Mayer, C.; Tarabochia, M.; et al. Neuroimaging, behavioral, and psychological sequelae of repetitive combined blast/impact mild traumatic brain injury in Iraq and Afghanistan war veterans. J. Neurotrauma 2014, 31, 425–436. [Google Scholar] [CrossRef] [Green Version]
  79. Taber, K.H.; Hurley, R.A.; Haswell, C.C.; Rowland, J.A.; Hurt, S.D.; Lamar, C.D.; Morey, R.A. White matter compromise in veterans exposed to primary blast forces. J. Head Trauma Rehabil. 2015, 30, E15. [Google Scholar] [CrossRef]
  80. Johnson, V.E.; Stewart, W.; Smith, D.H. Axonal pathology in traumatic brain injury. Exp. Neurol. 2013, 246, 35–43. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  81. Council, N.R. Linking Helmet Protection to Brain Injury. In Review of Department of Defense Test Protocols for Combat Helmets; National Academies Press (US): Washington, DC, USA, 2014. [Google Scholar]
  82. Hulkower, M.B.; Poliak, D.B.; Rosenbaum, S.B.; Zimmerman, M.E.; Lipton, M.L. A decade of DTI in traumatic brain injury: 10 years and 100 articles later. Am. J. Neuroradiol. 2013, 34, 2064–2074. [Google Scholar] [CrossRef] [Green Version]
  83. Taylor, P.A.; Ford, C.C. Simulation of blast-induced early-time intracranial wave physics leading to traumatic brain injury. J. Biomech. Eng. 2009, 131, 061007. [Google Scholar] [CrossRef]
  84. Zappala, G.; de Schotten, M.T.; Eslinger, P.J. Traumatic brain injury and the frontal lobes: What can we gain with diffusion tensor imaging? Cortex 2012, 48, 156–165. [Google Scholar] [CrossRef]
  85. Aoki, Y.; Inokuchi, R.; Gunshin, M.; Yahagi, N.; Suwa, H. Diffusion tensor imaging studies of mild traumatic brain injury: A meta-analysis. J. Neurol. Neurosurg. Psychiatry 2012, 83, 870–876. [Google Scholar] [CrossRef] [Green Version]
  86. Gennarelli, T.A.; Thibault, L.E.; Adams, J.H.; Graham, D.I.; Thompson, C.J.; Marcincin, R.P. Diffuse axonal injury and traumatic coma in the primate. Ann. Neurol 1982, 12, 564–574. [Google Scholar] [CrossRef] [PubMed]
  87. Gentry, L.R.; Thompson, B.; Godersky, J.C. Trauma to the corpus callosum: MR features. Am. J. Neuroradiol. 1988, 9, 1129–1138. [Google Scholar] [PubMed]
  88. Arfanakis, K.; Haughton, V.M.; Carew, J.D.; Rogers, B.P.; Dempsey, R.J.; Meyerand, M.E. Diffusion tensor MR imaging in diffuse axonal injury. Am. J. Neuroradiol. 2002, 23, 794–802. [Google Scholar] [PubMed]
  89. Friedman, J.; Hastie, T.; Tibshirani, R. The Elements of Statistical Learning; Springer: New York, NY, USA, 2001. [Google Scholar]
  90. Liu, H.; Motoda, H. Computational Methods of Feature Selection; CRC Press: Boca Raton, FL, USA, 2007. [Google Scholar]
  91. Fani, N.; King, T.Z.; Jovanovic, T.; Glover, E.M.; Bradley, B.; Choi, K.; Ely, T.; Gutman, D.A.; Ressler, K.J. White Matter Integrity in Highly Traumatized Adults With and Without Post-Traumatic Stress Disorder. Neuropsychopharmacology 2012, 37, 2740–2746. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  92. O’Doherty, D.C.M.; Ryder, W.; Paquola, C.; Tickell, A.; Chan, C.; Hermens, D.F.; Bennett, M.R.; Lagopoulos, J. White matter integrity alterations in post-traumatic stress disorder. Hum. Brain Mapp. 2018, 39, 1327–1338. [Google Scholar] [CrossRef] [PubMed]
  93. Weis, C.N.; Belleau, E.L.; Pedersen, W.S.; Miskovich, T.A.; Larson, C.L. Structural Connectivity of the Posterior Cingulum Is Related to Reexperiencing Symptoms in Posttraumatic Stress Disorder. Chronic Stress Thousand Oaks Calif 2018, 2, 2470547018807134. [Google Scholar] [CrossRef]
  94. Kaplan, G.B.; Leite-Morris, K.A.; Wang, L.; Rumbika, K.K.; Heinrichs, S.C.; Zeng, X.; Wu, L.; Arena, D.T.; Teng, Y.D. Pathophysiological Bases of Comorbidity: Traumatic Brain Injury and Post-Traumatic Stress Disorder. J. Neurotrauma 2018, 35, 210–225. [Google Scholar] [CrossRef]
  95. Huang, M.-X.; Swan, A.R.; Quinto, A.A.; Matthews, S.; Harrington, D.L.; Nichols, S.; Bruder, B.J.; Snook, C.C.; Huang, C.W.; Baker, D.G.; et al. A pilot treatment study for mild traumatic brain injury: Neuroimaging changes detected by MEG after low-intensity pulse-based transcranial electrical stimulation. Brain Inj. 2017, 31, 1951–1963. [Google Scholar] [CrossRef]
  96. Lee, S.A.; Kim, M.-K. Effect of Low Frequency Repetitive Transcranial Magnetic Stimulation on Depression and Cognition of Patients with Traumatic Brain Injury: A Randomized Controlled Trial. Med. Sci. Monit. Int. Med. J. Exp. Clin. Res. 2018, 24, 8789–8794. [Google Scholar] [CrossRef]
  97. Leung, A.; Shukla, S.; Fallah, A.; Song, D.; Lin, L.; Golshan, S.; Tsai, A.; Jak, A.; Polston, G.; Lee, R. Repetitive Transcranial Magnetic Stimulation in Managing Mild Traumatic Brain Injury-Related Headaches. Neuromodul. J. Int. Neuromodul. Soc. 2016, 19, 133–141. [Google Scholar] [CrossRef]
  98. Mollica, A.; Safavifar, F.; Fralick, M.; Giacobbe, P.; Lipsman, N.; Burke, M.J. Transcranial Magnetic Stimulation for the Treatment of Concussion: A Systematic Review. Neuromodul. J. Int. Neuromodul. Soc. 2021, 24, 803–812. [Google Scholar] [CrossRef] [PubMed]
  99. Vaninetti, M.; Lim, M.; Khalaf, A.; Metzger-Smith, V.; Flowers, M.; Kunnel, A.; Yang, E.; Song, D.; Lin, L.; Tsai, A.; et al. fMRI findings in MTBI patients with headaches following rTMS. Sci. Rep. 2021, 11, 9573. [Google Scholar] [CrossRef] [PubMed]
Diagnostics 12 00987 g001 550
Figure 1. Significant group differences in FA, RD, and AD/RD in white-matter fiber tracts. In all figures, green skeletons show the averaged skeletonized FA of the HC and bmTBI participants on both sagittal (top) and axial (bottom) views. Red regions and arrows/tract labels designate the locations of the voxels that showed statistically significant group differences in a diffusion metric (FEW corrected p value < 0.05). (A) Tracts showing significantly higher FA in the bmTBI group than the healthy control group. (B) Tracts showing significantly decreased RD in the bmTBI group in comparison to the HC group. (C) Tracts showing significantly higher AD/RD in the bmTBI group than in the HC group. ACR = anterior corona radiata; bCC = body of corpus callosum; gCC = genu of corpus callosum; LIC = anterior limb of internal capsule; SCR = superior corona radiata.
Figure 1. Significant group differences in FA, RD, and AD/RD in white-matter fiber tracts. In all figures, green skeletons show the averaged skeletonized FA of the HC and bmTBI participants on both sagittal (top) and axial (bottom) views. Red regions and arrows/tract labels designate the locations of the voxels that showed statistically significant group differences in a diffusion metric (FEW corrected p value < 0.05). (A) Tracts showing significantly higher FA in the bmTBI group than the healthy control group. (B) Tracts showing significantly decreased RD in the bmTBI group in comparison to the HC group. (C) Tracts showing significantly higher AD/RD in the bmTBI group than in the HC group. ACR = anterior corona radiata; bCC = body of corpus callosum; gCC = genu of corpus callosum; LIC = anterior limb of internal capsule; SCR = superior corona radiata.
Diagnostics 12 00987 g001
Diagnostics 12 00987 g002 550
Figure 2. DTI features and inputs into the supervised vector machine (SVM) model. The top figure displays the locations of candidate features that were used to build the stepwise SVM model. Green skeletons are the averaged skeletonized FA of the control and bmTBI groups. Red regions show the mapping of tract locations for the features. The leftmost column (sagittal views) shows the two features generated from the corpus callosum, namely the bCC (top) and gCC (bottom). The remaining columns (axial views) display the anatomical locations of the ACR, ALIC, PLIC, and SCR features for the right (top row) and left (bottom row) hemispheres. The graph below visualizes how a plane separates controls and bmTBI participants in SVM with the x-axis being normalized AD/RD of left ALIC and the y-axis being normalized AD/RD of gCC. ACR = anterior corona radiata; ALIC = anterior limb of internal capsule; bCC = body of corpus callosum; gCC = genu of corpus callosum; PLIC = posterior limb of internal capsule; SCR = superior corona radiata.
Figure 2. DTI features and inputs into the supervised vector machine (SVM) model. The top figure displays the locations of candidate features that were used to build the stepwise SVM model. Green skeletons are the averaged skeletonized FA of the control and bmTBI groups. Red regions show the mapping of tract locations for the features. The leftmost column (sagittal views) shows the two features generated from the corpus callosum, namely the bCC (top) and gCC (bottom). The remaining columns (axial views) display the anatomical locations of the ACR, ALIC, PLIC, and SCR features for the right (top row) and left (bottom row) hemispheres. The graph below visualizes how a plane separates controls and bmTBI participants in SVM with the x-axis being normalized AD/RD of left ALIC and the y-axis being normalized AD/RD of gCC. ACR = anterior corona radiata; ALIC = anterior limb of internal capsule; bCC = body of corpus callosum; gCC = genu of corpus callosum; PLIC = posterior limb of internal capsule; SCR = superior corona radiata.
Diagnostics 12 00987 g002
Table
Table 1. Demographic characteristics and neuropsychological test performances in the control and bmTBI groups.
Table 1. Demographic characteristics and neuropsychological test performances in the control and bmTBI groups.
Control GroupbmTBI Group
MeanSDMeanSDp-ValueCohen’s d
Age28.003.50027.406.2270.64530.1234
Years of education12.580.76913.052.0590.2635−0.3324
Months post-injury 10.386.271
D-KEFS
Number-Letter Sequencing11.371.38310.551.6690.10490.5374
Letter Fluency10.793.3268.7503.0930.0547 ^0.6356
Category Fluency11.583.09710.402.9630.23210.3894
Category Switching (total
correct)		11.792.9749.8003.0020.0447 *0.6660
Category Switching (total switching accuracy)12.052.36810.752.1000.09380.5819
WAIS-III
Symbol Search10.793.79410.552.4170.81450.0773
Digit Symbol Coding10.162.7548.8002.6080.12220.5073
Group differences were tested using independent t-tests. Neuropsychological measures are expressed as scaled scores. SD = standard deviation; D-KEFS = Delis-Kaplan Executive Function System; WAIS-III = Wechsler Adult Intelligence Scale Version 3; * p < 0.05; ^ subthreshold trend for group differences (0.05 ≤ p < 0.1).
Table
Table 2. Percentage symptoms endorsed on the HISC ± in the bmTBI and control groups.
Table 2. Percentage symptoms endorsed on the HISC ± in the bmTBI and control groups.
SymptomsbmTBI (%)Control (%)SymptomsbmTBI (%)Control (%)
Headaches90.05.26Lack of spontaneity0.000.00
Dizziness70.05.26Affective lability10.05.26
Fatigue50.010.5Depression20.05.26
Memory difficulty85.015.8Concentration10.010.5
Irritability60.015.8Bothered by noise0.000.00
Anxiety55.00.00Bothered by light0.005.26
Sleep problems60.05.26Coordination/balance15.010.5
Hearing difficulties60.010.5Motor difficulty10.00.00
Visual difficulties10.00.00Speech difficulty0.005.26
Personality changes25.05.26Numbness/tingling20.00.00
Apathy5.000.00
± HISC = modified Head Injury Symptom Checklist (Alvin Jr et al., 1984).
Table
Table 3. Group differences in DTI metrics of white-matter features.
Table 3. Group differences in DTI metrics of white-matter features.
TableGroupFAp ValueRD ±p ValueAD/RDp ValueVoxels
lACRHC0.44 ± 0.024<0.0010.54 ± 0.0270.00982.04 ± 0.11<0.001
bmTBI0.47 ± 0.0180.52 ± 0.0212.15 ± 0.08675
rACRHC0.45 ± 0.029<0.0010.56 ± 0.033<0.0012.11 ± 0.13<0.001
bmTBI0.48 ± 0.0240.53 ± 0.0252.24 ± 0.131073
lALICHC0.54 ± 0.015<0.0010.47 ± 0.018<0.0012.63 ± 0.10<0.001580
bmTBI0.56 ± 0.0130.45 ± 0.0142.83 ± 0.10
rALICHC0.57 ± 0.020<0.0010.45 ± 0.0230.00262.80 ± 0.14<0.001364
bmTBI0.60 ± 0.0190.43 ± 0.0183.03 ± 0.16
lPLICHC0.62 ± 0.017<0.0010.41 ± 0.0160.06083.18 ± 0.14<0.001
bmTBI0.64 ± 0.0190.40 ± 0.0213.40 ± 0.231107
rPLICHC0.63 ± 0.0170.00150.40 ± 0.0170.00563.29 ± 0.17<0.001
bmTBI0.65 ± 0.0200.38 ± 0.0203.54 ± 0.261472
lSCRHC0.45 ± 0.027<0.0010.53 ± 0.0230.00582.10 ± 0.130.0088
bmTBI0.48 ± 0.0270.50 ± 0.0222.23 ± 0.15260
rSCRHC0.45 ± 0.0220.00260.53 ± 0.0220.02152.15 ± 0.100.0067
bmTBI0.48 ± 0.0260.51 ± 0.0262.27 ± 0.14247
gCCHC0.64 ± 0.0320.00140.46 ± 0.0430.0013.45 ± 0.32<0.001
bmTBI0.68 ± 0.0240.42 ± 0.0293.94 ± 0.35222
bCCHC0.66 ± 0.028<0.0010.43 ± 0.042<0.0014.08 ± 0.390.0031
bmTBI0.69 ± 0.0200.39 ± 0.0284.41 ± 0.32170
Group differences in diffusion metrics were tested using independent t-tests. l = left; r = right; ACR = anterior corona radiata; ALIC = anterior limb of internal capsule; bCC = body of corpus callosum; gCC = genu of corpus callosum; PLIC = poster limb of internal capsule; SCR = superior corona radiata; ± RD is expressed in the unit of mm2/s. FA and AD/RD are unitless.
Table
Table 4. Features and performances of the best SVM model.
Table 4. Features and performances of the best SVM model.
ModelAccuracySensitivitySpecificityKernelFeatures: Weightings (Mean ± Standard Deviation)
FA and AD/RD89%90%88%LinearNormalized
AD/RD lALIC: (1.43 ± 0.16)
AD/RD gCC: (0.84 ± 0.14)
AD/RD rALIC: (0.77 ± 0.14)
FA rACR: (−1.08 ± 0.15)
FA lSCR: (0.69 ± 0.12)
l = left hemisphere; r = right hemisphere; ACR = anterior corona radiata; ALIC = anterior limb of internal capsule; gCC = genu of corpus callosum; SCR = superior corona radiata.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Harrington, D.L.; Hsu, P.-Y.; Theilmann, R.J.; Angeles-Quinto, A.; Robb-Swan, A.; Nichols, S.; Song, T.; Le, L.; Rimmele, C.; Matthews, S.; et al. Detection of Chronic Blast-Related Mild Traumatic Brain Injury with Diffusion Tensor Imaging and Support Vector Machines. Diagnostics 2022, 12, 987. https://doi.org/10.3390/diagnostics12040987

AMA Style

Harrington DL, Hsu P-Y, Theilmann RJ, Angeles-Quinto A, Robb-Swan A, Nichols S, Song T, Le L, Rimmele C, Matthews S, et al. Detection of Chronic Blast-Related Mild Traumatic Brain Injury with Diffusion Tensor Imaging and Support Vector Machines. Diagnostics. 2022; 12(4):987. https://doi.org/10.3390/diagnostics12040987

Chicago/Turabian Style

Harrington, Deborah L., Po-Ya Hsu, Rebecca J. Theilmann, Annemarie Angeles-Quinto, Ashley Robb-Swan, Sharon Nichols, Tao Song, Lu Le, Carl Rimmele, Scott Matthews, and et al. 2022. "Detection of Chronic Blast-Related Mild Traumatic Brain Injury with Diffusion Tensor Imaging and Support Vector Machines" Diagnostics 12, no. 4: 987. https://doi.org/10.3390/diagnostics12040987

APA Style

Harrington, D. L., Hsu, P. -Y., Theilmann, R. J., Angeles-Quinto, A., Robb-Swan, A., Nichols, S., Song, T., Le, L., Rimmele, C., Matthews, S., Yurgil, K. A., Drake, A., Ji, Z., Guo, J., Cheng, C. -K., Lee, R. R., Baker, D. G., & Huang, M. (2022). Detection of Chronic Blast-Related Mild Traumatic Brain Injury with Diffusion Tensor Imaging and Support Vector Machines. Diagnostics, 12(4), 987. https://doi.org/10.3390/diagnostics12040987

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop