Next Article in Journal
Structural Analysis of Botulinum Neurotoxins Type B and E by Cryo-EM
Previous Article in Journal
Acute Liver Failure after Ingestion of Fried Rice Balls: A Case Series of Bacillus cereus Food Poisonings
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Toxins
Volume 14
Issue 1
10.3390/toxins14010013
toxins-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Predictors of Clinically Important Improvements in Motor Function and Daily Use of Affected Arm after a Botulinum Toxin A Injection in Patients with Chronic Stroke

by
Jen-Wen Hung
1,2,*,
Wen-Chi Wu
1,
Yi-Ju Chen
1,
Ya-Ping Pong
1,2 and
Ku-Chou Chang
2,3
1
Department of Rehabilitation, Chang Gung Memorial Hospital-Kaohsiung Medical Center, Kaohsiung 83301, Taiwan
2
School of Medicine, College of Medicine, Chang Gung University, Taoyuan 33302, Taiwan
3
Division of Cerebrovascular Diseases, Department of Neurology, Chang Gung Memorial Hospital, Kaohsiung 83301, Taiwan
*
Author to whom correspondence should be addressed.
Toxins 2022, 14(1), 13; https://doi.org/10.3390/toxins14010013
Submission received: 26 October 2021 / Revised: 16 December 2021 / Accepted: 20 December 2021 / Published: 23 December 2021
Versions Notes

Abstract

:
Identifying patients who can gain minimal clinically important difference (MCID) in active motor function in the affected upper extremity (UE) after a botulinum toxin A (BoNT-A) injection for post-stroke spasticity is important. Eighty-eight participants received a BoNT-A injection in the affected UE. Two outcome measures, Fugl–Meyer Assessment Upper Extremity (FMA-UE) and Motor Activity Log (MAL), were assessed at pre-injection and after 24 rehabilitation sessions. We defined favorable response as an FMA-UE change score ≥5 or MAL change score ≥0.5.Statistical analysis revealed that the time since stroke less than 36 months (odds ratio (OR) = 4.902 (1.219–13.732); p = 0.023) was a significant predictor of gaining MCID in the FMA-UE. Medical Research Council scale -proximal UE (OR = 1.930 (1.004–3.710); p = 0.049) and post-injection duration (OR = 1.039 (1.006–1.074); p =0.021) were two significant predictors of MAL amount of use. The time since stroke less than 36 months (OR = 3.759 (1.149–12.292); p = 0.028), naivety to BoNT-A (OR = 3.322 (1.091–10.118); p = 0.035), and education years (OR = 1.282 (1.050–1.565); p = 0.015) were significant predictors of MAL quality of movement. The findings of our study can help optimize BoNT-A treatment planning.
Keywords:
stroke; upper limb spasticity; motor function; botulinum toxin; predictors
Key Contribution: These results can help optimize BoNT-A injection.

1. Introduction

Spasticity after stroke is common. Approximately 43% and 38% of patients are found to have spasticity at 6 months [1] and 12 months [2] post-stroke, respectively. Spasticity significantly reduces the affected upper extremity (UE) motor function, which is required for daily life activities [3]. Increased daily arm use is important for patients with chronic stroke in order to maximize gains in their quality of life [4]. Therefore, treating spasticity to improve active function is an important issue in stroke UE rehabilitation training.
Botulinum toxin A (BoNT-A) is commonly used for treating focal spasticity [5]. Although the efficacy of BoNT-A in spasticity reduction is well established, the impact of active functional outcomes is controversial [6,7,8].The International Consensus Statement declared that motor function improved in some patients after BoNT-A injections; however, more studies are needed to elucidate the effects [9].
The controversial results [6,7,8] may be due to the fact that that not all patients with spasticity could benefit from active motor function after BoNT-A treatment. Many interacting factors may affect treatment outcomes. Thus, predictive models should be developed to identify responders and non-responders to interventions [10]. Despite the extensive use of BoNT-A for spasticity treatment, the patient cohort that might benefit the most in active function after BoNT-A treatment remains unclear. Detecting predictors may help to adapt treatment plans and more correctly stratify patients for a better outcome from BoNT-A treatment. In this study, we selected two active functional outcome measures, the Fugl–Meyer Assessment upper extremity (FMA-UE) [11] and Motor Activity Log (MAL) [12], in reference to the International Classification of Functioning, Disability, and Health (ICF) framework published by the World Health Organization [13].The FMA-UE represents the body function domain, while MAL represents activity and participation domains.
Labeling the responders to each intervention is an important issue. Efficacy is defined as the improvement in some determined outcome measures in response to a specific treatment. Statistically significant changes are usually used to prove efficacy. However, for each outcome measure, it is important to know if the change is meaningful, rather than statistically significant. A statistically significant difference does not necessarily convert to clinically meaningful improvements [14]. In some studies with large sample sizes, small effects could be statistically significant, but these effects may have no clinical relevance. The idea of minimal clinically important difference (MCID) was proposed by Jaeschke et al. in 1989 [15].MCID defines a threshold that is considered to be an important change; this measure prevents the problem of mere statistical significance and provides valuable information for clinical practice.
This study aimed to explore the predictors of clinically important changes in active motor function and daily use of the affected UE after a BoNT-A injection for post-stroke UE spasticity.

2. Results

Eighty-eight participants with chronic stroke that received a BoNT-A injection in the affected UE and 24 sessions of rehabilitation training within 4 months post injection were evaluated. A total of 62 men and 26 women, with a mean age of 49.32 ± 10.95 years, were enrolled.Fifty-one patients (57.95%) had left hemiplegia, and 53patients (60.2%) had cerebral infarction. The time since stroke onset to BoNT-A injection was 33.23 ± 22.44 months. Forty-eight (54.55%) patients were naïve to BoNT-A injection. The baseline demographic and clinical characteristics of the participants are shown in Table 1.
The mean total injection dose was 327U (range 100–400 U). The injected muscles and doses were the pectoralis major (62.86 ± 16.84 U in 14 (15.91%) patients); the biceps brachii (53.33 ± 25.17 U in three (3.41%) patients); the brachialis (63.69 ± 24.64 U in 65 (73.86%) patients); the brachioradialis (46.00 ± 23.75 U in 50 (56.82%) patients); the triceps brachii (43.33 ± 16.14 U in 12 (13.64%) patients); the pronator teres (43.72 ± 13.90 U in 74 (84.09%) patients; the pronator quaratus (32.13 ± 10.43 U in 40 (45.45%) patients); the flexor carpi radialis (48.01 ± 16.21 U in 68 (77.27%) patients); the flexor carpi ulnaris (29.52 ± 10.86 U in 52 (59.09%) patients); the flexor digitorum profundus (34.46 ± 17.12 U in 28 (31.82%) patients); the flexor digitorum superficialis (85.48 ± 33.46 U in 84 (95.45%) patients); the lumbricallis (19.13 ± 7.64 U in 23 (26.14%) patients); the pollicis adductor(10.96 ± 3.47 U in 26 (29.55%) patients); and the flexor pollicis longus (27.29 ± 7.17 U in 72 (81.82%) patients).
No serious adverse events were observed. Three (3.41%) patients reported muscular weakness after injection: grasping, forearm pronation, and elbow flexion, respectively. All patients had mild symptoms for less than 1 month, and no intervention was indicated. No participants withdrew due to AEs.The outcome measures were assessed at 76.18 ± 17.18 days post injection. Based on the information collected for the 88 patients, all outcomes showed statistically significant improvements after the intervention (p < 0.01) (Table 2). These results indicated that patients had better motor function as well as greater and better use of the affected UE after a BoNT-A injection and rehabilitation training.
Twenty-five (28.41%), 45 (51.14%), and 37 (42.45%) participants reached the MCID for FMA-UE, MAL amount of use (AOU), and MAL quality of movement (QOM), respectively. The results of χ2 and independent-sample t tests revealed three predictors: the time since stroke less than 36 months, Wolf Motor Function Test (WMFT) quality score, and Medical Research Council scale (MRC)-distal UE score for the multivariate logistic regression analysis of the FMA model. Five predictors: naïve to BoNT-A, education years, post-injection duration, MRC proximal UE, and proprioception scores were selected for the MAL AOU model. Six predictors such as the time since stroke less than 36 months, education years, naïve to BoNT-A, post-injection duration, FMA-UE proximal, and proprioception scores were selected for the MAL QOM model (Table 3).
Table 4 summarizes the results of multivariate logistic regression analyses. The time since stroke less than 36 months (odds ratio (OR) = 4.902 (1.219–13.732); p = 0.023) was a significant predictor of gaining MCID in the FMA-UE. MRC-proximal UE (OR = 1.930 (1.004–3.710); p = 0.049) and post-injection duration (OR = 1.039 (1.006–1.074); p = 0.021) were the two significant predictors of MAL AOU. The time since stroke less than 36 months (OR = 3.759 (1.149–12.292); p = 0.028), naivety to BoNT-A (OR = 3.322 (1.091–10.118); p = 0.035), and education years (OR = 1.282 (1.050–1.565); p = 0.015) were significant predictors of MAL QOM.
All variance inflation factor (VIF) values of the predictors were <10 (range: 1.02–1.34), indicating the presence of weak multicollinearity. The p values of all Hosmer–Lemeshow tests were >0.05, which represented the consistency of the model’s predictions with the expectations of the model itself.

3. Discussion

To the best of our knowledge, this is the first study to examine clinically significant improvements in motor function and daily use of the affected UE after BoNT-A injections. Patients with the time since stroke less than 36 months had a greater chance of achieving clinically significant improvements in FMA-UE. Patients with a longer post-injection duration and/or greater proximal UE muscle strength could use their affected UEs more frequently in activity of daily living (ADL). Patients with a time since stroke less than 36 months, naïve to BoNT-A injection, and/or higher education level had a greater possibility of achieving MCID in MAL QOM.
Our results support the importance of the time since stroke in active function gained after a BoNT-A treatment. An investigation revealed that BoNT-A could be more helpful in subacute patients than in chronic patients for decreasing spasticity, contracture, and improving function [16]. Furthermore, an international consensus statement recommended early BoNT-A injection for active functional improvement [9]. By recruiting patients with chronic stroke, we demonstrated that injecting BoNT-A in the affected UE earlier than 3 years post-stroke may be more beneficial for active motor function gains.
The post-injection duration was relevant to the improvement in the frequency of UE use in ADL. As all patients received the same dosage of training, the post-injection duration represented the time patients could self-practice at home. We believe that the amount of practice in ADL is an important factor for the increase in the frequency of use of the affected UE in daily life.
One of the probable reasons for the lack of active effect from a BoNT-A injection is the greater role of weakness than spasticity on functional disability. Thus, patients with better underlying strength may benefit more from BoNT-A injections [7]. We found that participants with greater muscle strength of proximal UE had a higher probability of achieving MCID in the MAL AOU. Reaching is an essential element of many daily living activities. The reaching performance of the affected UE after stroke is reported to markedly depend on UE muscle power [17]. The act of reaching out requires proximal UE muscle power to take the weight of the UE or stabilize it in space. Patients with greater muscle strength of the proximal UE at baseline could have better reaching ability after BoNT-A injection and would use the affected UE more in ADL than those with less muscle strength of proximal UE.
Using the UE to perform ADL requires fine (e.g., grasping) and coarse (e.g., reaching) motor abilities. Several studies have indicated that manual dexterous function significantly affects patient’s ADL performance [18,19]. However, our study did not reveal such results. In our study, neither the MRC-distal UE nor the FMA-UE distal score could predict affected UE use in ADL, as shown in MAL. One possible reason for this result is the recruitment of moderate to severe cases and the generally poor manual dexterity in our patients.
We found that patients with higher education had a higher chance of gaining MCID of MAL QOM. No previous studies have reported that years of education could be a predictor of UE functional improvement after interventions. We suspected that the higher education level may be related to higher learning ability to improve the quality of affected UE use in ADL. Further studies should be conducted to clarify the influence of the years of education.
Spasticity in the UE might interfere with the performance of UE movements. Several studies have revealed that less spasticity predicts greater improvements in UE rehabilitation training [20,21,22]. In our study, we did not find the MAS of UE at baseline as a negative predictor of active function gain. Such findings may be due to all patients receiving BoNT-A treatment and the decrease in the spasticity effect.
The selection bias for recruiting patients that are naïve or non-naïve to BoNT-A treatment is concerning, as non-naïve patients maybe more tolerable to BoNT-A treatment, which might also be more effective in this cohort [23]. However, we found that patients who were naïve to BoNT-A had a higher chance of gaining the MCID of MAL QOM. One explanation is that the non-naïve to BoNT-A patients had better MAL QOM than naïve patients (1.07 ± 0.85 vs. 0.75 ± 0.54), and a ceiling effect occurred for non-naïve patients. Despite the accumulated effects after repeated injections [24,25,26], patients naïve to BoNT-A may have more potential to improve their QOM than non-naïve patients for one BoNT-A injection. After the first BoNT-A injection, training the functional use of the affected UE in ADL is strongly suggested.
BoNT-A injection programs, such as concentration, dosage, number of injected muscles, injectors, and co-interventions, may influence the outcome of BoNT-A treatment. We standardized some BoNT-A injection procedures, such as the use of same concentration (50 U BoNT-A per 1 mL), performance of echo guidance for injection localization, and injection administered by only two physiatrists, who had similar injection experience and principle. We found that the total injection dosage was not a predictor of active function gain. There were also no differences between the two injectors in any of the outcome measures (data not shown). However, it is difficult to standardize other factors. The muscles selected for the BoNT-A injection and dose per muscle were individualized according to patterns and severity of spasticity. We could not identify the effects of the number of injected muscles or the dosage of each injected muscle. In the future, a large study with subgroups of standardized BoNT-A injection programs according to patients’ conditions is needed to assess the effects of the diversity of BoNT-A injection programs and to build a standardized program for BoNT-A treatment for post-stroke UE spasticity.
Safety is an important issue in BoNT-A treatment. The Adult Spasticity International Registry (ASPIRE) study is a large real-world, 2-year follow-up study on onabotulinumtoxin A utilization for spasticity. In the UE results, 15 treatment-related adverse events (AEs) were found in 14 patients (2.9%). The most common AE was muscular weakness (1.4%) [23]. In our study, similar findings were observed in three patients (3.41%) who reported muscular weakness after injection: one for grasping, one for forearm pronation, and one for elbow flexion. All patients had mild symptoms lasting less than 1 month. Intervention was not indicated and no patient withdrew from the study due to AE. No other AEs were reported. The patient who had post-injection grasping weakness achieved MCID of FMA and MAL AOU and QOM after the intervention. The patient who had post-injection forearm pronation weakness achieved MCID of MAL AOU and QOM after the intervention. The patient who had post-injection elbow flexor weakness achieved MCID of MAL AOU after the intervention. Post-injection weakness might not be an absolute negative factor for active function gains. The number of AEs in our study was small; therefore, we did not perform further analyses. A large study is needed to assess the impact of BoNT-A-related AEs on functional gains.

Study Limitations

Our results may provide clinicians with simple and valuable tools to identify patients who could have clinically important gains in motor function and UE use after BoNT-A injection. However, in the interpretation of our results, caution should be exercised owing to the limitations of this study. First, this was a retrospective, secondary data analysis study that had potential confounding factors. Second, we had a small sample size. Considering the limited case number, only candidate predictors with a significance level of p < 0.05 identified by χ2 and independent-sample t-tests were entered into the multivariate logistic regression models. Some potential factors may be excluded under such strict filters. In addition, we included only the potential predictors of baseline descriptive characteristics and clinical assessments. Other potential factors that might affect treatment outcomes, such as psychosocial function, occupation, contextual variables, or kinematic parameters, need further investigation to improve the model. Third, the homogeneity of patients in some variables, such as cognition function and proprioception deficits, may have resulted in the non-significance of these variables in predicting important changes in outcome measures. Fourth, because we only investigated the predictors of functional gain after one injection, no information on the predictors of repeated injection outcomes is presented. Fifth, our endpoints were only up to 4 months after a single injection; however, long-term follow-up might be needed to assess the maintenance effect. Sixth, combining a rehabilitation program after BoNT-A injection was suggested as an approach to optimize spasticity treatment outcomes [5]. There is no agreement on the most effective approach [27]. All patients received 24 sessions of rehabilitation training; however, the training programs varied. We did not consider the effects of the different adjuvant therapies as each training program only included a small number of participants.

4. Conclusions

The time since stroke less than 36 months, longer post-injection duration, greater muscle strength of the proximal UE, more education years, and naïve to BoNT-A treatment are valuable in predicting favorable outcomes in motor function and the use of the affected UE following stroke spasticity treatment with BoNT-A. Both predictors and the outcome measures are clinically relevant and can be easily conducted, which make our results valuable for clinical applications. The findings of our study can help optimize BoNT-A treatment planning. Further studies with a more comprehensive set of factors, a larger number of patients, repeated injections, and long-term follow-up are recommended for improving the predictive models.

5. Materials and Methods

5.1. Patients and Experimental Setup

This was a retrospective, secondary data analysis. For this study, we originally enrolled 88 participants from our previous studies that aimed to determine the effects of BoNT-A injection combined with rehabilitation training for post-stroke spasticity. The requirement for informed consent was waived owing to the retrospective design of the study.
The inclusion criteria were as follows: (1) clinical and imaging diagnosis of a first or recurrent unilateral stroke of ≥6 months; (2) UE spasticity (at least one UE muscle with a MAS ≥ 1+; (3) initial motor part of the FMA-UE score ranging from 13 to 56, indicating moderate to severe movement impairment; (4) Mini Mental State Exam score > 20, indicating no serious cognitive impairment; and (5) age ≥ 18 years. The exclusion criteria were as follows: (1) bilateral hemispheric or cerebellar lesions; (2) severe aphasia; (3) significant visual field deficits; (4) treatment with BoNT-A ≤ 4 months before recruitment; or (5) history of orthopedic or other neurologic diseases or medical conditions that would prevent adherence to the rehabilitation protocol [28].

5.2. Procedure

After a baseline assessment, participants received a BoNT-A injection for UE spasticity by one of two senior rehabilitation physicians. Botox brand BoNT-A purified neurotoxin complex (Allergan, an AbbVie Company, Irvine, CA, USA) was used. We added 2 mL normal saline into one vial of BoNT-A (100 U) to achieve a BoNT-A concentration of 50 U per 1 mL and a maximum dose of 400 units [29,30]. We used echo guidance to localize the targeted muscle. Muscles selected for BoNT-A injection and dose per muscle were individualized according to patterns and severity of spasticity. Within 1 week after the injection, all participants received 24 rehabilitation training sessions by well-trained occupational therapists. There were 2–4 training sessions per week for 6–12 weeks. Each training session included 45–60 min of robot-assisted training, mirror therapy, or traditional occupational therapy, followed by 30 min of functional training; the raters were blinded to participants’ treatment allocation. The following assessments were extracted before BoNT-A injection (T0) and after 24 sessions of rehabilitation therapy (T1). BoNT-A related AEs during the intervention period were also recorded.

5.3. Outcome Measures

5.3.1. FMA-UE

FMA-UE evaluates UE motor impairments of stroke patients. The score ranges from 0 to 66, with higher scores indicating fewer motor deficits. FMA-UE has good reliability, validity, and responsiveness in patients with stroke [11,31,32]. We defined the MCID of the FMA-UE as a change score of ≥5 points based on previous reports [33,34].

5.3.2. MAL

MAL is a semi-structured interview questionnaire in which patients rate AOU and QOM while using their affected UE to accomplish 30 daily activities. The score ranges from 0 (never using the affected UE for the activity) to 5 (ability to use the affected UE for that task as good as before stroke), with higher scores indicating better performance. MAL has good reliability and concurrent validity [12]. Changes in MAL scores could reflect improvements in both participation and functional independence [35,36].The values of MCID were estimated to be approximately 10% of the range of the scale for chronic patients [37].Thus, we defined the MCID in MAL AOU and MAL QOM after BoNT-A injection as an improvement of ≥0.5 [12,38].

5.4. Potential Predictors

We selected potential predictors according to previous studies of upper limb recovery in stroke patients [30,39]. The potential predictors included the demographic (age, sex, education, and cognition level), clinical records (time since stroke, side of lesion, type of stroke, lesion location, naïve to BoNT-A or not, post injection duration, and injection dose), and clinical assessment scores (MAS, MRC, joint proprioception score, FMA-UE, WMFT, MAL).

5.4.1. MAS

We used MAS, which has shown good reliability and validity, to assess the severity of spasticity [40,41]. For statistical analysis, we record a 1+ score as 1.5. In addition, we estimated mean MAS scores of wrist flexors/extensors and finger flexors/extensors in each participant as the MAS of the distal UE and the mean of MAS scores of shoulder flexors/extensors, abductors/adductors, internal/external rotators, elbow flexor/extensors, and forearm pronator/supinator as the MAS of the proximal UE for further analysis.

5.4.2. MRC

We used the MRC scale to measure limb muscle power. The MRC scale is a reliable measurement that ranges from 0 (no contraction) to 5 (normal power) [42,43]. We estimated the mean MRC score in the shoulder flexors/abductors, elbow flexors/extensors as MRC proximal, mean MRC score in the wrist flexors/extensors, and finger flexors/extensors as MRC distal.

5.4.3. Joint Proprioception Sensation

The Revised Nottingham Sensory Assessment (rNSA) [44] is a reliable scale for assessing sensory function in stroke patients. Proprioception subscales of the rNSA were extracted. The affected UE was supported and moved by the assessor in various directions at one joint at a time. The patient was asked to mirror the change in movement with the unaffected UE. A scoring of 0 indicated no appreciation of movement taking place; 1, the patient could appreciate and mirror the direction of the test movement taking place each time but is inaccurate in the new position; and 2, accurately mirrors the test movement to within 10” of the new test position. We estimated the mean score of the affected shoulder, elbow, wrist, and fingers for further analysis.

5.4.4. WMFT

The WMFT is a quantitative measure of UE motor ability by using timed and functional tasks [45]. The WMFT includes 17 tasks. Performances were timed and rated using a 6-point ordinal scale. WMFT has good interrater reliability and criterion validity for patients with hemiparesis [46].

5.4.5. Time since Stroke Onset to BoNT-A Injection

Due to the fact that, in our study, the mean time since stroke onset to BoNT-A injection was 33 months, we divided the patients to two groups as with the time since stroke onset to BoNT-A injection less than 36 months vs. not less than 36 months for further analysis.

5.5. Data Analysis

Paired t-tests were used to compare FMA and MAL data between pre-intervention and post-intervention assessments. A patient with a changed score that reached the MCID for the FMA (≥5) or the MAL (≥0.5) was coded as 1 (responder), and a patient who did not reach the MCID was coded as 0 (non-responder).
We used χ2 and independent sample t-tests to compare the participants’ baseline characteristics between responders and non-responders to identify the potential predictors of achieving MCID in FMA-UE and MAL scores.
Considering the limited case number, the candidate predictors with a significance level of p < 0.05, identified by χ2 and independent-sample t-tests, were entered into the multivariate logistic regression models with an enter procedure. Predictive equations and odds ratios of the significant predictors were generated from the analyses, with a significance level of p < 0.05. In order to ensure the quality of the models, VIF and Hosmer–Lemeshow goodness-of-fit tests were used. A VIF value of <10 indicates the absence of multicollinearity. A Hosmer–Lemeshow test p-value > 0.05 indicated the consistency of the model’s predictions with the expectations of the model itself [47].
All tests were executed using the SPSS software version 25 (International Business Machines Corp., Armonk, NY, USA) at a significance level of α = 0.05.

Author Contributions

Conceptualization, J.-W.H.; formal analysis, J.-W.H., W.-C.W. and Y.-J.C.; funding acquisition, J.-W.H. and K.-C.C.; investigation, J.-W.H., Y.-J.C., Y.-P.P. and W.-C.W.; methodology, J.-W.H. and Y.-J.C.; project administration, J.-W.H.; resources, J.-W.H. and K.-C.C.; supervision, J.-W.H. and K.-C.C.; validation, J.-W.H.; writing—original draft, J.-W.H.; writing—review and editing, J.-W.H. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by a research grant from Chang Gung Memorial Hospital CMRPG8H1261.

Institutional Review Board Statement

This was a retrospective, secondary data analysis. The data were extracted from our previous three studies. All the studies were conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Chang Gung Medical Foundation, (protocol code 20159732A3 and date of approval 15 February 2016; protocol code 201601931A3 and date of approval 25 January 2017; protocol code 201702227A3 and date of approval 22 February 2018).

Informed Consent Statement

In this study, the requirement for informed consent was waived due to the retrospective design of the study. Informed consent was obtained from all subjects involved in the previous original three studies.

Data Availability Statement

The data presented in this study are available upon request from the corresponding author. The data are not publicly available due to ethical issues.

Acknowledgments

We appreciate the help of statistical analysis from Biostatistics Center, Kaohsiung Chang Gung Memorial Hospital. We also thank the study participants; the research assistants who worked on this study; and people who supported recruitment efforts.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Urban, P.P.; Wolf, T.; Uebele, M.; Marx, J.r.J.; Vogt, T.; Stoeter, P.; Bauermann, T.; Weibrich, C.; Vucurevic, G.D.; Schneider, A. Occurence and clinical predictors of spasticity after ischemic stroke. Stroke 2010, 41, 2016–2020. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Watkins, C.; Leathley, M.; Gregson, J.; Moore, A.; Smith, T.; Sharma, A. Prevalence of spasticity post stroke. Clin. Rehabil. 2002, 16, 515–522. [Google Scholar] [CrossRef] [PubMed]
  3. Sommerfeld, D.K.; Eek, E.U.-B.; Svensson, A.-K.; Holmqvist, L.W.; Von Arbin, M.H. Spasticity after stroke: Its occurrence and association with motor impairments and activity limitations. Stroke 2004, 35, 134–139. [Google Scholar] [CrossRef] [Green Version]
  4. Kelly, K.M.; Borstad, A.L.; Kline, D.; Gauthier, L.V. Improved quality of life following constraint-induced movement therapy is associated with gains in arm use, but not motor improvement. Top. Stroke Rehabil. 2018, 25, 467–474. [Google Scholar] [CrossRef] [PubMed]
  5. Esquenazi, A.; Novak, I.; Sheean, G.; Singer, B.; Ward, A. International consensus statement for the use of botulinum toxin treatment in adults and children with neurological impairments–introduction. Eur. J. Neurol. 2010, 17, 1–8. [Google Scholar] [CrossRef]
  6. Cardoso, E.; Rodrigues, B.; Lucena, R.; Oliveira, I.R.; Pedreira, G.; Melo, A. Botulinum toxin type A for the treatment of the upper limb spasticity after stroke: A meta-analysis. Arq. Neuro-Psiquiatr. 2005, 63, 30–33. [Google Scholar] [CrossRef] [Green Version]
  7. Levy, J.; Molteni, F.; Cannaviello, G.; Lansaman, T.; Roche, N.; Bensmail, D. Does botulinum toxin treatment improve upper limb active function? Ann. Phys. Rehabil. Med. 2019, 62, 234–240. [Google Scholar] [CrossRef]
  8. Bethoux, F. Spasticity Management After Stroke. Phys. Med. Rehabil. Clin. N. Am. 2015, 26, 625–639. [Google Scholar] [CrossRef]
  9. Sheean, G.; Lannin, N.; Turner-Stokes, L.; Rawicki, B.; Snow, B. Botulinum toxin assessment, intervention and after-care for upper limb hypertonicity in adults: International consensus statement. Eur. J. Neurol. 2010, 17, 74–93. [Google Scholar] [CrossRef]
  10. Winstein, C.J.; Stein, J.; Arena, R.; Bates, B.; Cherney, L.R.; Cramer, S.C.; Deruyter, F.; Eng, J.J.; Fisher, B.; Harvey, R.L. Guidelines for adult stroke rehabilitation and recovery: A guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2016, 47, e98–e169. [Google Scholar] [CrossRef]
  11. Fugl-Meyer, A.R.; Jääskö, L.; Leyman, I.; Olsson, S.; Steglind, S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand. J. Rehabil. Med. 1975, 7, 13–31. [Google Scholar]
  12. Van der Lee, J.; Beckerman, H.; Knol, D.; De Vet, H.; Bouter, L. Clinimetric properties of the motor activity log for the assessment of arm use in hemiparetic patients. Stroke 2004, 35, 1410–1414. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Doucet, B.M.; Gutman, S.A. Quantifying function: The rest of the measurement story. Am. J. Occup. Ther. 2013, 67, 7–9. [Google Scholar] [CrossRef] [Green Version]
  14. Sedaghat, A.R. Understanding the minimal clinically important difference (MCID) of patient-reported outcome measures. Otolaryngol.–Head Neck Surg. 2019, 161, 551–560. [Google Scholar] [CrossRef] [PubMed]
  15. Jaeschke, R.; Singer, J.; Guyatt, G.H. Measurement of health status: Ascertaining the minimal clinically important difference. Control. Clin. Trials 1989, 10, 407–415. [Google Scholar] [CrossRef]
  16. Lim, Y.-H.; Choi, E.-H.; Lim, J.Y. Comparison of effects of botulinum toxin injection between subacute and chronic stroke patients: A pilot study. Medicine 2016, 95, e2851. [Google Scholar] [CrossRef]
  17. Wagner, J.M.; Lang, C.E.; Sahrmann, S.A.; Edwards, D.F.; Dromerick, A.W. Sensorimotor impairments and reaching performance in subjects with poststroke hemiparesis during the first few months of recovery. Phys. Ther. 2007, 87, 751–765. [Google Scholar] [CrossRef]
  18. Lin, K.C.; Chuang, L.L.; Wu, C.Y.; Hsieh, Y.W.; Chang, W.Y. Responsiveness and validity of three dexterous function measures in stroke rehabilitation. J. Rehabil. Res. Dev. 2010, 47, 563–571. [Google Scholar] [CrossRef]
  19. Yancosek, K.E.; Howell, D. A narrative review of dexterity assessments. J. Hand Ther. 2009, 22, 258–269. [Google Scholar] [CrossRef]
  20. Huang, P.-C.; Hsieh, Y.-W.; Wang, C.-M.; Wu, C.-Y.; Huang, S.-C.; Lin, K.-C. Predictors of motor, daily function, and quality-of-life improvements after upper-extremity robot-assisted rehabilitation in stroke. Am. J. Occup. Ther. 2014, 68, 325–333. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Leem, M.J.; Kim, G.S.; Kim, K.H.; Im Yi, T.; Im Moon, H. Predictors of functional and motor outcomes following upper limb robot-assisted therapy after stroke. Int. J. Rehabil. Res. 2019, 42, 223–228. [Google Scholar] [CrossRef] [PubMed]
  22. Baldan, F.; Turolla, A.; Rimini, D.; Pregnolato, G.; Maistrello, L.; Agostini, M.; Jakob, I. Robot-assisted rehabilitation of hand function after stroke: Development of prediction models for reference to therapy. J. Electromyogr. Kinesiol. 2021, 57, 102534. [Google Scholar] [CrossRef]
  23. Francisco, G.E.; Jost, W.H.; Bavikatte, G.; Bandari, D.S.; Tang, S.F.T.; Munin, M.C.; Largent, J.; Adams, A.M.; Zuzek, A.; Esquenazi, A. Individualized OnabotulinumtoxinA Treatment for Upper Limb Spasticity Resulted in High Clinician- and Patient-Reported Satisfaction: Long-Term Observational Results from the ASPIRE Study. PM R 2020, 12, 1120–1133. [Google Scholar]
  24. Gracies, J.M.; O’Dell, M.; Vecchio, M.; Hedera, P.; Kocer, S.; Rudzinska-Bar, M.; Rubin, B.; Timerbaeva, S.L.; Lusakowska, A.; Boyer, F.C. Effects of repeated abobotulinumtoxinA injections in upper limb spasticity. Muscle Nerve 2018, 57, 245–254. [Google Scholar] [CrossRef]
  25. Turner-Stokes, L.; Jacinto, J.; Fheodoroff, K.; Brashear, A.; Maisonobe, P.; Lysandropoulos, A.; Ashford, S. Longitudinal goal attainment with integrated upper limb spasticity management including repeat injections of botulinum yoxin A: Findings from the prospective, observational Upper Limb International Spasticity (ULIS-III) cohort study. J. Rehabil. Med. 2021, 53, jrm00157. [Google Scholar] [CrossRef] [PubMed]
  26. Ro, T.; Ota, T.; Saito, T.; Oikawa, O. Spasticity and Range of Motion Over Time in Stroke Patients Who Received Multiple-Dose Botulinum Toxin Therapy. J. Stroke Cereb. Dis. 2020, 29, 104481. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Mills, P.B.; Finlayson, H.; Sudol, M.; O’Connor, R. Systematic review of adjunct therapies to improve outcomes following botulinum toxin injection for treatment of limb spasticity. Clin. Rehabil. 2016, 30, 537–548. [Google Scholar] [CrossRef]
  28. Hung, J.-W.; Chen, Y.-W.; Chen, Y.-J.; Pong, Y.-P.; Wu, W.-C.; Chang, K.-C.; Wu, C.-Y. The Effects of Distributed vs. Condensed Schedule for Robot-Assisted Training with Botulinum Toxin A Injection for Spastic Upper Limbs in Chronic Post-Stroke Subjects. Toxins 2021, 13, 539. [Google Scholar] [CrossRef]
  29. Abo, M.; Shigematsu, T.; Hara, H.; Matsuda, Y.; Nimura, A.; Yamashita, Y.; Takahashi, K. Efficacy and Safety of OnabotulinumtoxinA 400 Units in Patients with Post-Stroke Upper Limb Spasticity: Final Report of a Randomized, Double-Blind, Placebo-Controlled Trial with an Open-Label Extension Phase. Toxins 2020, 12, 127. [Google Scholar] [CrossRef] [Green Version]
  30. Wu, J.; Zhang, J.; Bai, Z.; Chen, S.; Cai, S. Predictive factors of upper limb motor recovery for stroke survivors admitted to a rehabilitation programme. Eur. J. Phys. Rehabil. Med. 2020, 56, 706–712. [Google Scholar]
  31. Hsieh, Y.-W.; Wu, C.-Y.; Lin, K.-C.; Chang, Y.-F.; Chen, C.-L.; Liu, J.-S. Responsiveness and validity of three outcome measures of motor function after stroke rehabilitation. Stroke 2009, 40, 1386–1391. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  32. Platz, T.; Pinkowski, C.; van Wijck, F.; Kim, I.-H.; Di Bella, P.; Johnson, G. Reliability and validity of arm function assessment with standardized guidelines for the Fugl-Meyer Test, Action Research Arm Test and Box and Block Test: A multicentre study. Clin. Rehabil. 2005, 19, 404–411. [Google Scholar] [CrossRef] [PubMed]
  33. Page, S.J.; Fulk, G.D.; Boyne, P. Clinically important differences for the upper-extremity Fugl-Meyer Scale in people with minimal to moderate impairment due to chronic stroke. Phys. Ther. 2012, 92, 791–798. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Lo, A.C.; Guarino, P.D.; Richards, L.G.; Haselkorn, J.K.; Wittenberg, G.F.; Federman, D.G.; Ringer, R.J.; Wagner, T.H.; Krebs, H.I.; Volpe, B.T. Robot-assisted therapy for long-term upper-limb impairment after stroke. N. Engl. J. Med. 2010, 362, 1772–1783. [Google Scholar] [CrossRef] [Green Version]
  35. Uswatte, G.; Taub, E.; Morris, D.; Light, K.; Thompson, P. The Motor Activity Log-28: Assessing daily use of the hemiparetic arm after stroke. Neurology 2006, 67, 1189–1194. [Google Scholar] [CrossRef] [PubMed]
  36. Taub, E.; Miller, N.E.; Novack, T.A.; Cook, E.W.; Fleming, W.C.; Nepomuceno, C.S.; Connell, J.S.; Crago, J. Technique to improve chronic motor deficit after stroke. Arch. Phys. Med. Rehabil. 1993, 74, 347–354. [Google Scholar]
  37. Lang, C.E.; Edwards, D.F.; Birkenmeier, R.L.; Dromerick, A.W. Estimating minimal clinically important differences of upper-extremity measures early after stroke. Arch. Phys. Med. Rehabil. 2008, 89, 1693–1700. [Google Scholar] [CrossRef] [Green Version]
  38. Van der Lee, J.H.; Wagenaar, R.C.; Lankhorst, G.J.; Vogelaar, T.W.; Devillé, W.L.; Bouter, L.M. Forced use of the upper extremity in chronic stroke patients: Results from a single-blind randomized clinical trial. Stroke 1999, 30, 2369–2375. [Google Scholar] [CrossRef]
  39. Coupar, F.; Pollock, A.; Rowe, P.; Weir, C.; Langhorne, P. Predictors of upper limb recovery after stroke: A systematic review and meta-analysis. Clin. Rehabil. 2012, 26, 291–313. [Google Scholar] [CrossRef]
  40. Bohannon, R.W.; Smith, M.B. Interrater reliability of a modified Ashworth scale of muscle spasticity. Phys. Ther. 1987, 67, 206–207. [Google Scholar] [CrossRef]
  41. Pandyan, A.D.; Johnson, G.R.; Price, C.I.; Curless, R.H.; Barnes, M.P.; Rodgers, H. A review of the properties and limitations of the Ashworth and modified Ashworth Scales as measures of spasticity. Clin. Rehabil. 1999, 13, 373–383. [Google Scholar] [CrossRef]
  42. Gregson, J.M.; Leathley, M.J.; Moore, A.P.; Smith, T.L.; Sharma, A.K.; Watkins, C.L. Reliability of measurements of muscle tone and muscle power in stroke patients. Age Ageing 2000, 29, 223–228. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Paternostro-Sluga, T.; Grim-Stieger, M.; Posch, M.; Schuhfried, O.; Vacariu, G.; Mittermaier, C.; Bittner, C.; Fialka-Moser, V. Reliability and validity of the Medical Research Council (MRC) scale and a modified scale for testing muscle strength in patients with radial palsy. J. Rehabil. Med. 2008, 40, 665–671. [Google Scholar] [CrossRef] [Green Version]
  44. Lincoln, N.; Jackson, J.; Adams, S. Reliability and revision of the Nottingham Sensory Assessment for stroke patients. Physiotherapy 1998, 84, 358–365. [Google Scholar] [CrossRef]
  45. Uswatte, G.; Taub, E. Constraint-induced movement therapy: New approaches to outcome measurement in rehabilitation. Cogn. Neurorehabilit. 1999, 215–229. [Google Scholar]
  46. Morris, D.M.; Uswatte, G.; Crago, J.E.; Cook, E.W., III; Taub, E. The reliability of the wolf motor function test for assessing upper extremity function after stroke. Arch. Phys. Med. Rehabil. 2001, 82, 750–755. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Chen, H.-L.; Lin, K.-C.; Liing, R.-J.; Wu, C.-Y.; Chen, C.-L. Kinematic measures of Arm-trunk movements during unilateral and bilateral reaching predict clinically important change in perceived arm use in daily activities after intensive stroke rehabilitation. J. Neuroeng. Rehabil. 2015, 12, 84. [Google Scholar] [CrossRef] [Green Version]
Table
Table 1. Demographics for the 88 participants.
Table 1. Demographics for the 88 participants.
CharacteristicsValue
Age (years)49.32 ± 10.95
Sex (Male/Female)62 (70.5)/26 (29.5)
Education years12.09 ± 3.51
Side of Hemiplegia (Rt/Lt)37 (42.0)/51 (58.0)
Nature (Hemorrhage/Infarction)35 (39.8)/53 (60.2)
Lesion (Cortical/Subcortical/Brainstem)29 (33.0)/58 (65.9)/1 (/1.1)
Naïve to BoNT-A (Yes/No)48 (54.55)/40 (45.45)
Total injection dose (U)326.70 ± 95.66
NOTE. Values are mean ± SD or N (%). BoNT-A, Botulinum toxin A.
Table
Table 2. Comparison of pre and post-intervention assessments within the 88 participants.
Table 2. Comparison of pre and post-intervention assessments within the 88 participants.
OutcomeMean ± SDp
FMA_UE
Baseline30.11 ± 8.25<0.01
Post-treatment33.20 ± 8.30
MAL AOU
Baseline1.27 ± 0.72<0.01
Post-treatment1.82 ± 0.84
MAL QOM
Baseline0.90 ± 0.70<0.01
Post-treatment1.37 ± 0.81
NOTE. Values are mean ± SD. FMA_UE, upper limb subtest of the Fugl–Meyer assessment; MAL AOU, Motor Activity Log Amount of use score; MAL QOM, Motor Activity Log Quality of movement score.
Table
Table 3. Univariate analysis comparing patients with or without clinically important improvements after intervention in the FMA-UE and MAL.
Table 3. Univariate analysis comparing patients with or without clinically important improvements after intervention in the FMA-UE and MAL.
Candidate PredictorFMA-UEMAL AOUMAL QOM
Change ≥ 5Change < 5pChange ≥ 0.5Change < 0.5pChange ≥ 0.5Change < 0.5p
n = 25n = 63 n = 45n = 43 n = 37n = 51
General Information
Age (years)46.19 ± 13.6750.56 ± 9.370.15948.59 ± 11.4750.09 ± 10.310.52747.70 ± 11.3650.50 ± 10.480.249
Sex (male/female)17/845/180.75129/1633/100.20625/1237/140.613
Education years11.52 ± 3.9012.32 ± 3.310.38112.82 ± 3.4111.33 ± 3.440.046 †13.43 ± 2.9511.12 ± 3.550.001 †
Time since stroke (<36 months/≧36 months)21/434/290.014 †31/1424/190.20529/826/250.009 †
Lesion Side (left/right)11/1426/370.81521/2416/270.36920/1717/340.052
Nature (hemorrhage/infarction)8/1727/360.34818/2717/260.96416/2119/320.571
Naïve to BoNT-A (Yes/No)14/1134/290.86330/1518/250.019 †28/920/310.001 †
Post-injection days75.88 ± 16.9776.30 ± 17.120.91881.87 ± 17.6770.41 ± 13.530.001 †82.24 ± 19.0771.78 ± 13.910.007 †
Injection dose (U)298.00 ± 88.14338.10 ± 96.150.071310.11 ± 84.82344.07 ± 103.010.100313.38 ± 82.69336.37 ± 102.990.255
Clinical Assessment at Baseline
MMSE27.52 ± 2.3526.92 ± 2.500.30326.98 ± 2.6027.21 ± 2.330.66527.41 ± 2.6426.86 ± 2.330.326
FMA-UE
Proximal26.76 ± 6.1026.10 ± 6.420.65727.49 ± 6.1925.02 ± 6.250.07028.43 ± 5.5924.73 ± 6.400.005 †
Distal4.04 ± 2.543.84 ± 3.130.7624.20 ± 2.933.58 ± 3.000.3363.95 ± 2.583.86 ± 3.240.895
Proprioception score8.36 ± 3.877.62 ± 3.380.4158.82 ± 3.266.79 ± 3.530.007 †8.97 ± 3.147.00 ± 3.590.008†
MAS
Proximal UE7.66 ± 4.308.91 ± 3.310.2068.51 ± 4.058.60 ± 3.220.9068.84 ± 3.608.35 ± 3.700.544
Distal UE3.18 ± 1.833.87 ± 1.740.1153.63 ± 1.813.71 ± 1.770.8433.89 ± 1.803.51 ± 1.760.323
MRC
Proximal UE13.72 ± 3.4112.90 ± 3.170.31713.89 ± 3.3312.35 ± 2.990.026 †13.86 ± 3.1412.61 ± 3.250.075
Distal UE9.32 ± 3.407.38 ± 2.920.018 †8.09 ± 2.987.77 ± 3.380.6427.68 ± 2.668.12 ± 3.510.508
WMFT
Time (mean)8.77 ± 4.1910.87 ± 8.720.1369.52 ± 4.9411.06 ± 9.840.3669.54 ± 4.4010.80 ± 9.460.409
Quality (mean)2.51 ± 0.402.25 ± 0.470.016 †2.39 ± 0.462.25 ± 0.470.1612.42 ± 0.472.26 ± 0.460.118
MAL
AOU (mean)1.29 ± 0.621.26 ± 0.750.8461.28 ± 0.651.27 ± 0.780.9351.26 ± 0.611.28 ± 0.790.859
QOM (mean)0.88 ± 0.630.91 ± 0.730.8380.88 ± 0.570.92 ± 0.820.7800.85 ± 0.510.93 ± 0.810.576
FMA-UE, upper limb subtest of the Fugl–Meyer assessment; MAL, Motor Activity Log; AOU, amount of use; QOM, quality of movement; BoNT-A, Botulinum toxin A; MMSE, Mini-Mental Status Examination; MAS, Modified Ashworth Scale; UE, upper extremity; MRC, Medical Research Council scale; WMFT, Wolf Motor Function Test; † indicates that the predictors were selected into the multivariate logistic regression analyses (p < 0.05).
Table
Table 4. Multivariate logistic regression analysis for the predictors of clinically important changes in FMA-UE and MAL.
Table 4. Multivariate logistic regression analysis for the predictors of clinically important changes in FMA-UE and MAL.
PredictorFMA-UEMAL AOUMAL QOM
βpOR (95% CI)βpOR (95% CI)βpOR (95% CI)
Constant−4.5940.002 −7.225<0.001 −9.921<0.001
Time since stroke less than 36 months1.4090.023 †4.092 (1.219–13.732) 1.6120.0012†5.013 (1.420–17.699)
Education year 0.0990.1991.104 (0.949–1.284)0.2480.015†1.282 (1.050–1.565)
Naïve to BoNT-A 0.6050.2291.831 (0.683–4.910)1.2010.035†3.322 (1.091–10.118)
Post-injection duration 0.0390.021†1.039 (1.006–1.074)0.0260.1311.026 (0.992–1.061)
MRC proximal UE 0.6570.049†1.930 (1.004–3.710)
MRC distal UE0.5670.1351.762 (0.839–3.704)
WMFT quality0.6330.3331.883 (0.523–6.786)
FMA-UE proximal 0.0910.0541.096 (0.999–1.202)
Proprioception 0.0870.2281.091 (0.947–1.257)0.0780.3681.081 (0.913–1.280)
FMA-UE, upper limb subtest of the Fugl–Meyer assessment; MAL, Motor Activity Log; AOU, amount of use; QOM, quality of movement; BoNT-A, Botulinum toxin A; MRC, Medical Research Council scale; UE, upper extremity; WMFT, Wolf Motor Function Test; † indicates p < 0.05.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hung, J.-W.; Wu, W.-C.; Chen, Y.-J.; Pong, Y.-P.; Chang, K.-C. Predictors of Clinically Important Improvements in Motor Function and Daily Use of Affected Arm after a Botulinum Toxin A Injection in Patients with Chronic Stroke. Toxins 2022, 14, 13. https://doi.org/10.3390/toxins14010013

AMA Style

Hung J-W, Wu W-C, Chen Y-J, Pong Y-P, Chang K-C. Predictors of Clinically Important Improvements in Motor Function and Daily Use of Affected Arm after a Botulinum Toxin A Injection in Patients with Chronic Stroke. Toxins. 2022; 14(1):13. https://doi.org/10.3390/toxins14010013

Chicago/Turabian Style

Hung, Jen-Wen, Wen-Chi Wu, Yi-Ju Chen, Ya-Ping Pong, and Ku-Chou Chang. 2022. "Predictors of Clinically Important Improvements in Motor Function and Daily Use of Affected Arm after a Botulinum Toxin A Injection in Patients with Chronic Stroke" Toxins 14, no. 1: 13. https://doi.org/10.3390/toxins14010013

APA Style

Hung, J. -W., Wu, W. -C., Chen, Y. -J., Pong, Y. -P., & Chang, K. -C. (2022). Predictors of Clinically Important Improvements in Motor Function and Daily Use of Affected Arm after a Botulinum Toxin A Injection in Patients with Chronic Stroke. Toxins, 14(1), 13. https://doi.org/10.3390/toxins14010013

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop