Next Article in Journal
Potential Biochemical Pesticide—Synthesis of Neofuranocoumarin and Inhibition the Proliferation of Spodoptera frugiperda Cells through Activating the Mitochondrial Pathway
Next Article in Special Issue
Characterization of Serotype CD Mosaic Botulinum Neurotoxin in Comparison with Serotype C and A
Previous Article in Journal
The Short-Term Impact of Botulinum Neurotoxin-A on Muscle Morphology and Gait in Children with Spastic Cerebral Palsy
Previous Article in Special Issue
Does Ultrasound Guidance Improve the Effectiveness of Neurotoxin Injections in Patients with Cervical Dystonia? (A Prospective, Partially-Blinded, Clinical Study)
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Reasons and Determinants of BoNT-A Treatment Discontinuation in Patients Living with Spasticity: A 10-Year Retrospective Analysis

1
Spasticity and Movement Disorder Unit, Physical Medicine and Rehabilitation, Policlinico Riuniti, University of Foggia, Viale Pinto 1, 71122 Foggia, Italy
2
Villa Beretta Rehabilitation Center, Valduce Hospital, Via Nazario Sauro 17, 23845 Costa Masnaga, Italy
3
Physical Medicine and Rehabilitation, Department of Health Sciences, Università del Piemonte Orientale, viale Piazza d’armi 1, 28100 Novara, Italy
*
Author to whom correspondence should be addressed.
Toxins 2022, 14(10), 675; https://doi.org/10.3390/toxins14100675
Submission received: 21 August 2022 / Revised: 25 September 2022 / Accepted: 26 September 2022 / Published: 29 September 2022
(This article belongs to the Special Issue Botulinum Neurotoxins: From Toxin to Medicine)

Abstract

:
Background: The present study aimed to evaluate the reasons and determinants of BoNT-A discontinuation in patients with stroke, multiple sclerosis, spinal cord injury, and traumatic brain injury. Methods: It is a retrospective study of 56 discontinuer patients treated with botulinum toxin between January 2011 and December 2021. Discontinuation rates and their predictors were estimated using Kaplan–Meier, Log rank test, and Cox’s regression method of analyses. Results: The mean age was 56.54 years, 53.57% were affected by post-stroke spasticity, 17.86% by spinal cord injury, 12.5% and 16.07% by traumatic brain injury and multiple sclerosis, respectively. The median discontinuation time was 5 months. The main reason for discontinuation were logistic problems (37%) and orthopedic surgeries or intrathecal baclofen (27%). Discontinuers were more likely to have severe spasticity (R = 1.785), have no pain (HR = 1.320), no access to rehabilitation services (HR = 1.402), and have cognitive impairment (HR = 1.403). Conclusions: The main reasons for discontinuation are related to logistic issues (due to distance or the absence of an adequate caregiver) and surgical interventions for spasticity, including intrathecal baclofen. It is crucial to identify possible predictors of discontinuation to improve the effectiveness of a multidisciplinary management. The study confirms the crucial role of rehabilitation and caregivers in achieving better long-term outcomes.
Key Contribution: Identification of the reasons for BoNT-A discontinuation would provide insights to enable the development of measures to support long-term treatment goals.

1. Introduction

Spasticity is a well-known motor dysfunction arising from upper motor neuron lesions due to stroke, spinal cord injury, multiple sclerosis, and traumatic brain injury. Estimates of the incidence and prevalence of spasticity vary due to the lack of a strict definition and clinical measurement of spasticity [1]. The rate of post-stroke spasticity has been reported to be 4 to 27% during the first six weeks after onset, 19% at three months, 21.7 to 42.6% at four and six months, and 17 to 38% at 12 months. It is estimated to occur in around 80% of persons with multiple sclerosis [2] and 65–78% in those with spinal cord injuries [3]. Spasticity in people with traumatic brain injuries (TBI) depends on the severity of the injury andthe prevalence is up to 40% with severe brainstem involvement [4].
Long-term consequences of spasticity include pain, distorted joint position, posture and hygiene difficulties, and in the long term, permanent joint contractures and deformities [5,6,7,8].
Despite the ease of diagnosis, effective spasticity management is often challenging for clinicians. The goals of spasticity management may include increasing mobility and range of motion, attaining better hygiene, improving body image and functional level, and facilitating splint wear [9,10,11,12,13].
Botulinum toxin type A (BoNT-A) is a well-established treatment for treating focal spasticity in the upper motor neuron syndrome [14]. Over the past 30 years, accumulated evidence has established the effectiveness of BoNT-A [11,15,16,17]. BoNT-A plays also a role in the management of other conditions, such as chronic migraine, focal dystonia, hemifacial spasm, myofascial pain, and cerebral palsy [18,19,20,21]. Despite the considerable and well-known beneficial effects, many patients suffering from spasticity discontinue treatment. Consequently, there is a lack of knowledge about BoNT-A discontinuation and its determinants in daily clinical practice.
In a previous study, Latino and coll. retrospectively collected data on patients with multiple sclerosis treated with BoNT-A from 2002 to 2014 and analyzed the reasons for treatment discontinuation [22]. No other studies to date, have been conducted so far to investigate the long-term effectiveness and persistence of treatment with BoNT-A in patients suffering from spasticity in stroke, traumatic brain injury (TBI), spinal cord injury (SCI), and multiple sclerosis (MS). Important aspects of discontinuation of BoNT-A include both the reasons for discontinuation and treatment characteristics that are associated with a higher incidence of discontinuation. The results may be useful to differentiate patient-driven discontinuation from caregiver difficulty since spasticity is a complex health condition in which the patient is only partially involved.
The present study aimed to determine the rate of treatment discontinuation over 10 years in Italian patients living with spasticity deriving from stroke, MS, SCI, and TBI.
The primary outcome was the main reason for discontinuation; secondly, we focused our attention on the durability of the treatment, defined as the time in months from the beginning of the first BoNT-A regimen until its discontinuation for any cause and the potential determinants of discontinuation.

2. Results

2.1. Patients Sociodemographic Characteristics:

In the past decade, 75 patients have discontinued BoNT-A treatment. Excluding the deceased patients, 62 patients met the inclusion criteria, while six were not contactable. Finally, a total of 56 discontinuers (Females: n = 26, Males: n = 30) satisfied the inclusion criteria and were included in this analysis. Table 1 summarizes the sociodemographic characteristics. The mean age is 56.54 years with a standard deviation of 18.73 (range 18–88 years). Most of the patients were over 60 (n = 30), and there were only five patients under 30 (8.93%).

2.2. Patients’ Clinical Characteristics

The commonest etiology was stroke [30, (53.57%)], followed by spinal cord injury [10, (17.86%)], multiple sclerosis, and traumatic brain injury accounted respectively 16.07% and 12.5% (Table 1).
Table 1. Sociodemographic characteristics of discontinuers.
Table 1. Sociodemographic characteristics of discontinuers.
VariablesDiscontinuers
No of patientsn = 56
Age (Mean)56.54 (18.73)
Sex n (%)-
Male30 (53.57%)
Female26 (46.43%)
VariableFrequency %
Age group distribution5 (8.93%)
≤308 (14.29%)
31–406 (10.71%)
41–506 (10.71%)
51–6013 (23.21%)
>7018 (32.14%)
Disease -
Stroke30 (53.57%)
Multiple Sclerosis9 (16.07%)
SCI10 (17.86%)
TBI7 (12.5%)
Neurological picture-
Monoparesis0
Hemiparesis 32 (57.14%)
Paraparesis24 (42.86%)
Tetraparesis9 (25%)
Severity of UL spasticity -
Mild15 (26.79%)
Moderate24 (42.86%)
Severe17 (30.36%)
Severity of LL spasticity -
Mild12 (21.43%)
Moderate14 (25%)
Severe30 (53.57%)
Functional status-
No disability9 (16.07%)
Moderate disability18 (32.14%)
Severe disability29 (51.79%)
Comorbidities-
None3 (5.36%)
Moderate16 (28.57%)
Severe37 (66.07%)
Duration of spasticity-
<3 years21 (37.5%)
3–5 years5 (8.93%)
5–10years7 (12.5%)
>10 years23 (41.07%)
Pain-
Yes12 (21.43%)
No44 (78.57%)
UL upper limb; LL lower limb. Data are reported as frequency (%).
Hemiparesis, paraparesis, and tetraparesis accounted for 32 (57.14%), 24 (42.86%), and 9 (25%), respectively. The severity of the overall spasticity was interpreted based on clinical experience as mild, moderate, or severe related to the last injection visit. Functional status was measured using the Barthel Index, with 100 points indicating no disability, 60–95 points indicating moderate disability and 0–55 points indicating severe disability [23]. Comorbidities were measured using the Charlson Comorbidity Index, with 0 point indicating none, 1–2 points indicating moderate comorbidities and ≥3 points indicating severe comorbidities [24]. The majority of patients, presented spasticity from less than 3 years (37.5%) and over 10 years (41.07%).

2.3. Reasons for Discontinuing BoNT-A Injection

Several key themes of factors influencing BoNT-A discontinuation were identified: unmet efficacy and adverse events; orthopedic functional surgery or intratechal baclofen; pain during injection; clinical worsening and logistical reasons. For logistical issues, we mean contingent situations that concern the inability of the patient (or caregiver) to reach the center where to receive BoNT-A treatment. The main reasons given for treatment discontinuation are shown in Figure 1. Overall, for all discontinuers, logistic reasons (including distance problems and difficulty in being accompanied) were the most common reason reported for treatment discontinuation (21, [37.5%]), especially in stroke patients (14, [46.67%]). Orthopedic surgeries and intrathecal baclofen therapy (ITB) were frequently a reason for discontinuation for spinal cord injury (7, [70%]) and traumatic brain injury (5, [71.43%]). Progression of pathology and worsening of the clinical condition was also the most common reason for discontinuation of BoNT-A in post-stroke patients (26.67%). Importantly, only 3.3% of respondents considered adverse events (AEs) as a reason for discontinuation (Table 2).

2.4. Factors Associated with Discontinuation of BoNT-A injection

Table 3 shows the comparison of the estimation of the median time to discontinue care among the patients. At the bivariate analysis the severity of spasticity (p = 0.027), presence of comorbidities (p < 0.001), cognitive impairment (p = 0.034) presence of pain (p < 0.001) and access to rehabilitation (p < 0.001), were all associated with the median time to discontinuation of BoNT-A injections. Those who presented with severe spasticity and comorbidities had cognitive impairment with no pain were more likely to discontinue BoNT-A treatment earlier than others (Figure 2 and Figure 3).

2.5. Predictors of Discontinuation of BoNT-A Injection

Table 4 showed the results of the Cox-proportional hazard model on variables associated with discontinuation of BoNT-A treatment. The multivariate analysis with the Cox-proportional hazard model showed a statistically significant association with loss to follow-up at p < 0.1 from Kaplan–Meier method and Log-rank test. Patients severe spasticity (95% CI = 1.047-1.639), no pain (95% CI = 1.396-2.302), with no rehabilitation regimen (95% CI = 1.120-1.760), with cognitive impairment (95% CI = 1.110-1.670) are significantly more likely to discontinue follow-up early.

3. Discussion

The current definition of spasticity is intended to include muscle hypertonia and other positive signs of upper motor neuron lesions, such as spasms, clonus, hyper-reflexia, and muscle coactivation [25]. To the best of our knowledge, this is the first study analyzing the reasons for BoNT-A treatment discontinuation in patients with stroke, MS, TBI, and SCI patients in the last decade. We also assessed whether BoNT-A treatment discontinuation was associated with severity of spasticity, patient characteristics, and access to rehabilitation service.
Treatment discontinuation was defined as interruption of the initial treatment injection. Prior research indicates that discontinuance rates vary by clinical presentationin multiple sclerosis, suggesting that severalreasons may underlie patients’ premature discontinuation of treatments involved [26].
Discontinuance reasons are likely multifaceted (adverse events, contraindications, no effect), and the characteristics of patients who discontinue BoNT-A treatment are diverse.
Our results suggest that the main reason for discontinuation in stroke patients is related to access reasons with a crucial role of caregivers.Older age and concomitant cognitive impairments may be the main determinants.Greater disability has been shown to be associated with worse related quality of life and greater caregiver dependence [27]. Indeed, in an ASPIRE study, caregivers of patients treated with OnabotulinumtoxinA for spasticity reported less burden with a global improvement in emotional and general health, increased time spent with family and friends, more energy, and a better quality of life [28]. Although caregivers’ burden is frequently overlooked by clinicians, it has been demonstrated that caregivers are an appropriate and independent target for more focused therapeutic strategies in multiple sclerosis [28,29].
Importantly, we noticed that reduced or absent efficacy was reported in only 14% of subjects, half of them were affected by multiple sclerosis. This result matches with a previously published study, in which 37% of multiple sclerosis patients reported a loss of efficacy [18]. It is possible that prevalence figures in MS may incorrectly include motor dysfunction unresponsive to BoNT-A such as ataxia, apraxia, and fatigue [30].
The crucial balance between stiffness and paresis in MS needs attention and the risk of limiting functional autonomy should be considered.
Similarly, 70% of patients affected by SCI and TBI reportedsurgical management of spasticity. As known, BoNT-A can be inefficacious to handle generalized spasticity; on the other hand, it has been verified that ITB treatment decreases severe generalized spasticity in patients who have predominantly lower limb spasticity. Most of the patients affected by SCI and TBI included in our study presented severe spasticity. Although the indications for BoNT-A in SCI are reduced by the generalized nature of the spasticity, there may be an indication in the incomplete forms with focal spasticity.especially in AISC and D spinal cord injuries [31]. ITB is indicated in patients with severe spasticity and spasms but more recently it has been shown to be effective in improving mobility in carefully selected individuals; prompting ITB to be considered first to improve or maintain walking. [32,33]. In the same way, ITB in ambulatory patients with multiple sclerosis was shown to reduce spasticity, spasm frequency, and self-reported pain up to 1 year post-pump insertion without negative impact on walking and transfers [34,35].
We observed a minor proportion of stroke patients discontinuing BoNT-A treatment for surgical management of spasticity (about 10%), although neuro-orthopedic surgery of upper limbs, and especially lower limbs, is well established [36,37]. The main reason for this is that in most cases, neuro-orthopedic surgery, which includes tendon lengthening, tendon releases, tendon transfers, and neurotomies [38], is performed at the level of the lower limb; the upper limb is still treated with BoNT-A. Overall it has to be said that BoNT-A and neuro-orthopedic surgery can coexist. Data from a systematic review show level 1 evidence that when BoNT-A is administered preoperatively, it is effective for reducing pain, spasticity, and the use of analgesics in pediatric patients with cerebral palsy. This is supported by the findings of a retrospective case series in which patients who were treated with BoNT-A injections 23–31 days before surgery had reduced post-operative pain compared with those who were not injected [39]. Level 5 evidence from case reports confirms the potential for the use of BoNT-A in the peri-operative period [40].
The percentage of adverse events is very low, approximately 2%. In our case, only one patient reported mild fatigue. Although randomized controlled trials could contribute data on the safety of BoNT-A, they actually do not report serious or long-term adverse events of botulinum toxin. Most serious adverse events of BoNT-A come from spontaneous reporting systems and case reports in particular higher weight-adjusted doses can increase the possibility of a systemic spread causing distant-generalized weakness [41,42].
Our analysis shows that most patients (53.57%) presented severe spasticity in lower limbs and severe disability (51.79%). This finding could be explained by the major involvement of lower limbs in patients with SCI, TBI, and multiple sclerosis.
The mean duration of spasticity before the first cycle of injections was 14.52 months (median 5). Our results are in line with a recent study published in 2020, in which adherence to BoNT-A therapy revealed that most treatment dropouts occurred within the first 8 years [43]. In particular, authors found that long-term therapy adherence was better in blepharospasm, hemifacial spasm, and cervical dystonia patients than in spasticity patients.
We found that the predictors of discontinuation among patients treated with BoNT-A include severe spasticity and absence of pain. The lack of efficacy results in a loss of interest in continuing BoNT-A treatment. Our data show that the ineffectiveness of the first BoNT-A treatment affects discontinuation rates. We strongly believe it is therefore important to discuss with patientsand caregivers the concrete objectives that can also be achieved with subsequent injection cycles, increasing total dose, or introducing adjuvant therapies.
On the contrary, pain represents an encouraging factor of therapeutic continuity. Previously published papers showed significant efficacyof BoNT-A administration in patients suffering from neuropathic pain due to postherpetic neuralgia [44,45], SCI [46,47], peripheral nerve lesion [48], diabetic neuropathy post-traumatic/postoperative neuropathies [49,50,51], carpal tunnel syndrome [52] and stroke [53,54]. BoNT-A might improve neuropathic pain through various mechanisms such as inhibiting the secretion of pain mediators (substance P, glutamate, and calcitonin gene-related protein (CGRP)) into peripheral nerve terminals, dorsal root ganglia, and spinal cord neurons. Additionally, BoNT-A might have an anti-inflammatory effect on acute injury and chronic inflammation by reducing the release of peripheral neuro-transmitters and inflammatory mediators such as CGRP, substance P, and COX-2 [55,56]. In the present analysis, patients with cognitive impairment were more likely to be lost following BoNT-A injection. As the cognitive impairment progresses, more support is needed to keep therapeutic adherence. In our cohort, another important predictor of interruption was the absence of rehabilitation (p < 0.005). Latino et al. observed a similar trend in patients with MS [22]. This finding is consistent with existing literature showing that the maximum benefit of BoNT-A treatment in the management of spasticity is the association with multi-modal rehabilitation [57,58,59]. The underlying mechanism is that physiotherapy in combination with BoNT-A injections can improve the overall response to toxins, likewise by enhancing its diffusion and leading to more effective uptake by the target muscles [60]. Our results reveal that it is crucial to identify possible predictors of discontinuation to improve the effectiveness of a multidisciplinary management. Several contingency factors may influence BoNT-A treatment, especially those linked to the difficulty of accompanying patients. Long-term adherence prevents contractures or joint deformities and limits systemic side effects deriving from oral anti-spasticity medications (e.g., baclofen, tizanidine, or diazepam).When used in conjunction with other rehabilitative interventions improves spasticity outcomes, reduces the burden of care, avoids additional healthcare expenditure, and improvesthe overall quality of life [61].Therefore, the clinical benefits of BoNT-A treatment outweigh the apparent high costs of this intervention, showing it to be a cost-effective treatment.
The findings of this study have to be interpreted in light of its limitations. The present study is subject to the limitations of a retrospective database analysis. Although this contributes to the high external validity of the data, it reduces the internal validity of the dataand is susceptible to multiple sources of bias for comparing variables.We did not collect information on the single formulation of BoNT-A injected in patients included in the present study. Furthermore, it is possible that rates of adherence were affected by other reasons that have been forgotten after a long time at the time of the interview.

4. Conclusions

Since spasticity is one of the aspects of the global concept of disability and it is rarely isolated, monitoring the rate and the reasons for patient adherence may be beneficial to the optimal management of spasticity. Early identification of factors predicting treatment discontinuation would allow timely adjustments for better management of the treatment plan. The main reasons for discontinuation were logistic issues and alternative management of spasticity, especially in TBI and SCI patients. The BoNT-A treatment is safe and no serious adverse events were reported.
Patients with severe spasticity, no pain, with no rehabilitation regimen are significantly more likely to discontinue follow-up early. The data from this analysis highlight patient perspectives on reasons for treatment discontinuation and the factors associated. Comprehensive care, which also includes the cognitive component, and the use of adjuvant therapies is the key message beneficial to spasticity management.

5. Materials and Methods

5.1. Patients’ Selection

This was a retrospective cohort study of patients being treated with BoNT-A and discontinued for many reasons. The patient population consisted of all patients admitted to the “Physical and Rehabilitation Medicine department” of the University of Foggia, Foggia, Italy with a diagnosis of spasticity after stroke, MS, SCI, and TBI.
We identified patients aged ≥18 yearsoldwho had received at least one botulinum toxin injection. We included patients with all forms of spasticity, focal or not. Hospital case files with follow-up records of patients being treated with BoNT-A attended to between January 2011 and December 2021 were retrieved from our clinic medical records by two investigators.
The reasons for patient discontinuation were investigated involving one-to-one interviews. Each interview was voice recorded and notes taken: data were collectively analyzed, to identify a set of most common themes of discontinuation. Using a structured interview with patients, or where not possible, with the caregiver, we obtained information on potential predisposing, enabling, and reinforcing factors for discontinuation. The functional status was then measured with a telephonic Barthel Index assessment [62].
Exclusion criteria were: insufficient information in the medical records to complete the database, other movement disorder pathologies (cerebral palsy, dystonia, blepharospasm, and hemifacial spasms), and death. For patients who discontinued therapy, the duration of treatment was measured as the number of months from initiation of the first BoNT-A injection to discontinuation of therapy. The study conforms to all STROBE guidelines.

5.2. Statistical Analysis

Reasons for discontinuation were analyzed using frequency distribution. Factors potentially associated with discontinuation of BoNT-A injection were compared using descriptive statistics. The data were carefully entered and analyzed using SPSS version 16.0. Regular checks were done to detect and correct errors.The dependent for the outcome variable was defined as the time from the first BoNT-A injection to the discontinuation of following treatments. Discontinuation of care is defined as missing three consecutive follow-up appointments. The independent variables explored were sociodemographic characteristics and clinical factors such as age, pathology, severity of spasticity, duration of disease, functional status, pain, cognitive impairment, comorbidities, and concomitant rehabilitation treatment. Frequency, percentage, mean, median, range, and standard deviation were used to summarize the sociodemographic, clinical, and treatment variables of the patients and presented using tables and graphs. Discontinuation curves were plotted using the Kaplan–Meier method and curves were compared using Log-rank test. Log-rank test was used to test for an association between the dependent variable (loss following injection) and independent variables.
The variables were considered to show significant association when the p value was less than 0.05. The discontinuation of patients of spasticity was compared using Log-rank test according to clinical features. The clinical features were dichotomized for ease of comparison between the clinical subgroups. Multivariate analysis was carried out using Cox- proportional hazard models to determine the predictors of discontinuation of follow-up. This was done using covariates that showed statistically significant association with follow-up discontinuation at p < 0.1 on bivariate analysis. In the analysis of discontinuation using Kaplan–Meier, the time of origin was taken as the time of the first botulinum toxin injection. The endpoint of the patient was follow-up discontinuation. The median discontinuation times were obtained from the Kaplan–Meier discontinuation curve.

Author Contributions

Conceptualization, N.C. and A.S.; Formal analysis, N.C., S.S. and S.F.; Investigation, N.C.; Project administration, A.B., P.M. and A.S.; Resources, M.B.; Supervision, L.S. and A.B.; Visualization, P.M.; Writing—original draft, N.C.; Writing—review & editing, N.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the amended Declaration of Helsinki.The Ethics Review and Privacy Board of Policlinico Riuniti University Hospital of Foggia approved the study protocol (ID protocol number:000019081, date: 21 September 2022).

Informed Consent Statement

Patient consent was waived because no treatment was performed.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy issue.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Li, S.; Francisco, G.E. The Use of Botulinum Toxin for Treatment of Spasticity. Handb. Exp. Pharmacol. 2021, 263, 127–146. [Google Scholar] [PubMed]
  2. Patejdl, R.; Zettl, U.K. Spasticity in Multiple Sclerosis: Contribution of Inflammation, Autoimmune Mediated Neuronal Damage and Therapeutic Interventions. Autoimmun. Rev. 2017, 16, 925–936. [Google Scholar] [CrossRef] [PubMed]
  3. Holtz, K.A.; Lipson, R.; Noonan, V.K.; Kwon, B.K.; Mills, P.B. Prevalence and Effect of Problematic Spasticity After Traumatic Spinal Cord Injury. Arch. Phys. Med. Rehabil. 2017, 98, 1132–1138. [Google Scholar] [CrossRef] [PubMed]
  4. Wedekind, C.; Lippert-Grüner, M. Long-Term Outcome in Severe Traumatic Brain Injury Is Significantly Influenced by Brainstem Involvement. Brain Inj. 2005, 19, 681–684. [Google Scholar] [CrossRef]
  5. Ayoub, S.; Smith, J.G.; Cary, I.; Dalton, C.; Pinto, A.; Ward, C.; Saverino, A. The Positive and the Negative Impacts of Spasticity in Patients with Long-Term Neurological Conditions: An Observational Study. Disabil. Rehabil. 2021, 43, 3357–3364. [Google Scholar] [CrossRef]
  6. King, B.W.; Ruta, D.J.; Irwin, T.A. Spastic Foot and Ankle Deformities: Evaluation and Treatment. Foot Ankle Clin. 2014, 19, 97–111. [Google Scholar] [CrossRef]
  7. Allieu, Y. [Surgical Management of the Adult Spastic Hand]. Chir. Main 2011, 30, 159–175. [Google Scholar] [CrossRef]
  8. Picelli, A.; Santamato, A.; Chemello, E.; Cinone, N.; Cisari, C.; Gandolfi, M.; Ranieri, M.; Smania, N.; Baricich, A. Adjuvant Treatments Associated with Botulinum Toxin Injection for Managing Spasticity: An Overview of the Literature. Ann. Phys. Rehabil. Med. 2019, 62, 291–296. [Google Scholar] [CrossRef]
  9. Turner-Stokes, L.; Jacinto, J.; Fheodoroff, K.; Maisonobe, P.; Senturk, O.; Ashford, S. Relief of Spasticity-Related Pain with Botulinum Neurotoxin-A (Bont-A) in Real Life Practice. Post-Hoc Analysis from a Large International Cohort Series. Ann. Phys. Rehabil. Med. 2018, 61, e67–e68. [Google Scholar] [CrossRef]
  10. Andringa, A.; van de Port, I.; van Wegen, E.; Ket, J.; Meskers, C.; Kwakkel, G. Effectiveness of Botulinum Toxin Treatment for Upper Limb Spasticity Poststroke Over Different ICF Domains: A Systematic Review and Meta-Analysis. Arch. Phys. Med. Rehabil. 2019, 100, 1703–1725. [Google Scholar] [CrossRef]
  11. López de Munain, L.; Valls-Solé, J.; Garcia Pascual, I.; Maisonobe, P.; on behalf of the VALGAS investigators group. Botulinum Toxin Type A Improves Function According to Goal Attainment in Adults with Poststroke Lower Limb Spasticity in Real Life Practice. Eur. J. Neurol. 2019, 82, 1–8. [Google Scholar] [CrossRef] [PubMed]
  12. Santamato, A.; Micello, M.F.; Panza, F.; Fortunato, F.; Picelli, A.; Smania, N.; Logroscino, G.; Fiore, P.; Ranieri, M. Adhesive Taping vs. Daily Manual Muscle Stretching and Splinting after Botulinum Toxin Type A Injection for Wrist and Fingers Spastic Overactivity in Stroke Patients: A Randomized Controlled Trial. Clin. Rehabil. 2015, 29, 50–58. [Google Scholar] [CrossRef] [PubMed]
  13. Ashford, S.; Fheodoroff, K.; Jacinto, J.; Turner-Stokes, L. Common Goal Areas in the Treatment of Upper Limb Spasticity: A Multicentre Analysis. Clin. Rehabil. 2016, 30, 617–622. [Google Scholar] [CrossRef] [PubMed]
  14. Esquenazi, A.; Albanese, A.; Chancellor, M.B.; Elovic, E.; Segal, K.R.; Simpson, D.M.; Smith, C.P.; Ward, A.B. Evidence-Based Review and Assessment of Botulinum Neurotoxin for the Treatment of Adult Spasticity in the Upper Motor Neuron Syndrome. Toxicon 2013, 67, 115–128. [Google Scholar] [CrossRef] [PubMed]
  15. Santamato, A.; Cinone, N.; Panza, F.; Letizia, S.; Santoro, L.; Lozupone, M.; Daniele, A.; Picelli, A.; Baricich, A.; Intiso, D.; et al. Botulinum Toxin Type A for the Treatment of Lower Limb Spasticity after Stroke. Drugs 2019, 79, 143–160. [Google Scholar] [CrossRef]
  16. Santamato, A.; Ranieri, M.; Solfrizzi, V.; Lozupone, M.; Vecchio, M.; Daniele, A.; Greco, A.; Seripa, D.; Logroscino, G.; Panza, F. High Doses of IncobotulinumtoxinA for the Treatment of Post-Stroke Spasticity: Are They Safe and Effective? Expert Opin. Drug Metab. Toxicol. 2016, 12, 843–846. [Google Scholar] [CrossRef]
  17. Santamato, A.; Ranieri, M.; Panza, F.; Solfrizzi, V.; Frisardi, V.; Lapenna, L.M.; Moretti, B.; Fiore, P. Botulinum Toxin Type A in the Treatment of Painful Adductor Muscle Contracture after Total Hip Arthroplasty. Orthopedics 2009, 32, 10. [Google Scholar]
  18. Dodick, D.; Blumenfeld, A.; Silberstein, S.D. Botulinum neurotoxin for the treatment of migraine and other primary headache disorders. Clin. Dermatol. 2004, 22, 76–81. [Google Scholar] [CrossRef]
  19. Pihut, M.; Ferendiuk, E.; Szewczyk, M.; Kasprzyk, K.; Wieckiewicz, M. The efficiency of botulinum toxin type A for the treatment of masseter muscle pain in patients with temporomandibular joint dysfunction and tension-type headache. J. Headache Pain 2016, 17, 29. [Google Scholar]
  20. Ramos-Herrada, R.M.; Arriola-Guillén, L.E.; Atoche-Socola, K.J.; Bellini-Pereira, S.A.; Castillo, A.A. Effects of botulinum toxin in patients with myofascial pain related to temporomandibular joint disorders: A systematic review. Dent. Med. Probl. 2022, 59, 271–280. [Google Scholar] [CrossRef]
  21. Novak, I.; Morgan, C.; Fahey, M.; Finch-Edmondson, M.; Galea, C.; Hines, A.; Langdon, K.; Namara, M.M.; Paton, M.C.; Popat, H.; et al. State of the Evidence Traffic Lights 2019: Systematic Review of Interventions for Preventing and Treating Children with Cerebral Palsy. Curr. Neurol. Neurosci. Rep. 2020, 20, 3. [Google Scholar] [CrossRef] [PubMed]
  22. Latino, P.; Castelli, L.; Prosperini, L.; Marchetti, M.R.; Pozzilli, C.; Giovannelli, M. Determinants of Botulinum Toxin Discontinuation in Multiple Sclerosis: A Retrospective Study. Neurol. Sci. 2017, 38, 1841–1848. [Google Scholar] [CrossRef] [PubMed]
  23. Balu, S. Differences in Psychometric Properties, Cut-off Scores, and Outcomes between the Barthel Index and Modified Rankin Scale in Pharmacotherapy-Based Stroke Trials: Systematic Literature Review. Curr. Med. Res. Opin. 2009, 25, 1329–1341. [Google Scholar] [CrossRef] [PubMed]
  24. Charlson, M.E.; Carrozzino, D.; Guidi, J.; Patierno, C. Charlson Comorbidity Index: A Critical Review of Clinimetric Properties. Psychother. Psychosom. 2022, 91, 8–35. [Google Scholar] [CrossRef] [PubMed]
  25. Creamer, M.; Cloud, G.; Kossmehl, P.; Yochelson, M.; Francisco, G.E.; Ward, A.B.; Wissel, J.; Zampolini, M.; Abouihia, A.; Berthuy, N.; et al. Intrathecal Baclofen Therapy versus Conventional Medical Management for Severe Poststroke Spasticity: Results from a Multicentre, Randomised, Controlled, Open-Label Trial (SISTERS). J. Neurol. Neurosurg. Psychiatry 2018, 89, 642–650. [Google Scholar] [CrossRef]
  26. Lieber, R.L.; Steinman, S.; Barash, I.A.; Chambers, H. Structural and Functional Changes in Spastic Skeletal Muscle. Muscle Nerve 2004, 29, 615–627. [Google Scholar] [CrossRef]
  27. Doan, Q.V.; Brashear, A.; Gillard, P.J.; Varon, S.F.; Vandenburgh, A.M.; Turkel, C.C.; Elovic, E.P. Relationship Between Disability and Health-Related Quality of Life and Caregiver Burden in Patients With Upper Limb Poststroke Spasticity. PM&R 2012, 4, 4–10. [Google Scholar]
  28. Wittenberg, G.F.; Ngo, K.; Largent, J.; Zuzek, A.; Francisco, G.; Jost, W.H.; Ellenbogen, A.L.; Esquenazi, A. OnabotulinumtoxinA Treatment in Adults with Spasticity Reduces Caregiver Burden: Results from the ASPIRE Study. Arch. Phys. Med. Rehabil. 2019, 100, e23. [Google Scholar] [CrossRef]
  29. Pozzilli, C.; Palmisano, L.; Mainero, C.; Tomassini, V.; Marinelli, F.; Ristori, G.; Gasperini, C.; Fabiani, M.; Battaglia, M.A. Relationship between Emotional Distress in Caregivers and Health Status in Persons with Multiple Sclerosis. Mult. Scler. (Houndmills Basingstoke Engl.) 2004, 10, 442–446. [Google Scholar] [CrossRef]
  30. Adelman, R.D.; Tmanova, L.L.; Delgado, D.; Dion, S.; Lachs, M.S. Caregiver Burden: A Clinical Review. JAMA 2014, 311, 1052–1060. [Google Scholar] [CrossRef]
  31. Dressler, D.; Bhidayasiri, R.; Bohlega, S.; Chahidi, A.; Mo Chung, T.; Ebke, M.; Jorge Jacinto, L.; Kaji, R.; Koçer, S.; Kanovsky, P.; et al. Botulinum Toxin Therapy for Treatment of Spasticity in Multiple Sclerosis: Review and Recommendations of the IAB-Interdisciplinary Working Group for Movement Disorders Task Force. J. Neurol. 2017, 264, 112–120. [Google Scholar] [CrossRef]
  32. Ben Smaïl, D.; Denys, P.; Bussel, B. Toxine Botulique et Paraplégie. Ann. Réadapt. Méd. Phys. 2003, 46, 296–298. [Google Scholar] [CrossRef]
  33. Sadiq, S.A.; Wang, G.C. Long-Term Intrathecal Baclofen Therapy in Ambulatory Patients with Spasticity. J. Neurol. 2006, 253, 563–569. [Google Scholar] [CrossRef]
  34. Sammaraiee, Y.; Stevenson, V.L.; Keenan, E.; Buchanan, K.; Lee, H.; Padilla, H.; Farrell, R.A. Evaluation of the Impact of Intrathecal Baclofen on the Walking Ability of People with Multiple Sclerosis Related Spasticity. Mult. Scler. Relat. Disord. 2020, 46, 102503. [Google Scholar] [CrossRef]
  35. Lee, B.S.; Jones, J.; Lang, M.; Achey, R.; Dai, L.; Lobel, D.A.; Nagel, S.J.; Machado, A.G.; Bethoux, F. Early Outcomes after Intrathecal Baclofen Therapy in Ambulatory Patients with Multiple Sclerosis. J. Neurosurg. 2018, 129, 1056–1062. [Google Scholar] [CrossRef]
  36. Boffeli, T.J.; Collier, R.C.; Neubauer, E.F.; Malay, D.S. Surgical Outcomes After Minimally Invasive Release of Stroke-Related Equinovarus Contracture of the Foot and Ankle. J. Foot Ankle Surg. 2019, 58, 1108–1117. [Google Scholar] [CrossRef]
  37. Tranchida, G.V.; Van Heest, A. Preferred Options and Evidence for Upper Limb Surgery for Spasticity in Cerebral Palsy, Stroke, and Brain Injury. J. Hand Surg. Eur. Vol. 2020, 45, 34–42. [Google Scholar] [CrossRef]
  38. Genêt, F.; Denormandie, P.; Keenan, M.A. Orthopaedic Surgery for Patients with Central Nervous System Lesions: Concepts and Techniques. Ann. Phys. Rehabil. Med. 2019, 62, 225–233. [Google Scholar] [CrossRef]
  39. Dohin, B.; Garin, C.; Vanhems, P.; Kohler, R. Intérêt de la toxine botulique pour les suites opératoires en chirurgie des membres chez l’enfant infirme moteur cérébral. Rev. Chir. Orthopéd. Réparatrice L’appareil Mot. 2007, 93, 674–681. [Google Scholar] [CrossRef]
  40. Saeidiborojeni, S.; Mills, P.B.; Reebye, R.; Finlayson, H. Peri-operative Botulinum Neurotoxin injection to improve outcomes of surgeries on spastic limbs: A systematic review. Toxicon 2020, 188, 48–54. [Google Scholar] [CrossRef]
  41. Bakheit, A.M.O. The Possible Adverse Effects of Intramuscular Botulinum Toxin Injections and Their Management. Curr. Drug Saf. 2006, 1, 271–279. [Google Scholar] [CrossRef] [PubMed]
  42. Yiannakopoulou, E. Serious and Long-Term Adverse Events Associated with the Therapeutic and Cosmetic Use of Botulinum Toxin. Pharmacology 2015, 95, 65–69. [Google Scholar] [CrossRef] [PubMed]
  43. Lee, J.-I.; Jansen, A.; Samadzadeh, S.; Kahlen, U.; Moll, M.; Ringelstein, M.; Soncin, G.; Bigalke, H.; Aktas, O.; Moldovan, A.; et al. Long-Term Adherence and Response to Botulinum Toxin in Different Indications. Ann. Clin. Transl. Neurol. 2021, 8, 15–28. [Google Scholar] [CrossRef] [PubMed]
  44. Apalla, Z.; Sotiriou, E.; Lallas, A.; Lazaridou, E.; Ioannides, D. Botulinum Toxin A in Postherpetic Neuralgia: A Parallel, Randomized, Double-Blind, Single-Dose, Placebo-Controlled Trial. Clin. J. Pain. 2013, 29, 857–864. [Google Scholar] [CrossRef] [PubMed]
  45. Xiao, L.; Mackey, S.; Hui, H.; Xong, D.; Zhang, Q.; Zhang, D. Subcutaneous Injection of Botulinum Toxin a Is Beneficial in Postherpetic Neuralgia. Pain Med. 2010, 11, 1827–1833. [Google Scholar] [CrossRef] [PubMed]
  46. Chun, A.; Levy, I.; Yang, A.; Delgado, A.; Tsai, C.-Y.; Leung, E.; Taylor, K.; Kolakowsky-Hayner, S.; Huang, V.; Escalon, M.; et al. Treatment of At-Level Spinal Cord Injury Pain with Botulinum Toxin A. Spinal Cord. Ser. Cases 2019, 5, 77. [Google Scholar] [CrossRef]
  47. Han, Z.-A.; Song, D.H.; Oh, H.-M.; Chung, M.E. Botulinum Toxin Type A for Neuropathic Pain in Patients with Spinal Cord Injury. Ann. Neurol. 2016, 79, 569–578. [Google Scholar] [CrossRef]
  48. Attal, N.; de Andrade, D.C.; Adam, F.; Ranoux, D.; Teixeira, M.J.; Galhardoni, R.; Raicher, I.; Üçeyler, N.; Sommer, C.; Bouhassira, D. Safety and Efficacy of Repeated Injections of Botulinum Toxin A in Peripheral Neuropathic Pain (BOTNEP): A Randomised, Double-Blind, Placebo-Controlled Trial. Lancet Neurol. 2016, 15, 555–565. [Google Scholar] [CrossRef]
  49. Salehi, H.; Moussaei, M.; Kamiab, Z.; Vakilian, A. The Effects of Botulinum Toxin Type A Injection on Pain Symptoms, Quality of Life, and Sleep Quality of Patients with Diabetic Neuropathy: A Randomized Double-Blind Clinical Trial. Iran J. Neur. 2019, 18, 99–107. [Google Scholar] [CrossRef]
  50. Yuan, R.Y.; Sheu, J.J.; Yu, J.M.; Chen, W.T.; Tseng, I.J.; Chang, H.H.; Hu, C.J. Botulinum Toxin for Diabetic Neuropathic Pain: A Randomized Double-Blind Crossover Trial. Neurology 2009, 72, 1473–1478. [Google Scholar] [CrossRef]
  51. Taheri, M.; Sedaghat, M.; Solhpour, A.; Rostami, P.; Safarpour Lima, B. The Effect of Intradermal Botulinum Toxin a Injections on Painful Diabetic Polyneuropathy. Diabetes Metab. Syndr. 2020, 14, 1823–1828. [Google Scholar] [CrossRef] [PubMed]
  52. Breuer, B.; Sperber, K.; Wallenstein, S.; Kiprovski, K.; Calapa, A.; Snow, B.; Pappagallo, M. Clinically Significant Placebo Analgesic Response in a Pilot Trial of Botulinum B in Patients with Hand Pain and Carpal Tunnel Syndrome. Pain Med. 2006, 7, 16–24. [Google Scholar] [CrossRef] [PubMed]
  53. Cinone, N.; Letizia, S.; Santoro, L.; Gravina, M.; Amoruso, L.; Molteni, F.; Ranieri, M.; Santamato, A. Intra-Articular Injection of Botulinum Toxin Type A for Shoulder Pain in Glenohumeral Osteoarthritis: A Case Series Summary and Review of the Literature. J. Pain Res. 2018, 11, 1239–1245. [Google Scholar] [CrossRef] [Green Version]
  54. Trompetto, C.; Marinelli, L.; Mori, L.; Puce, L.; Avanti, C.; Saretti, E.; Biasotti, G.; Amella, R.; Cotellessa, F.; Restivo, D.A.; et al. Effectiveness of Botulinum Toxin on Pain in Stroke Patients Suffering from Upper Limb Spastic Dystonia. Toxins 2022, 14, 39. [Google Scholar] [CrossRef]
  55. Park, J.; Park, H.J. Botulinum Toxin for the Treatment of Neuropathic Pain. Toxins 2017, 9, 260. [Google Scholar] [CrossRef]
  56. Rivera Día, R.C.; Arcila Lotero, M.A.; Avellaneda Suarez, M.V.; Echeverri Saldarriaga, S.; Gómez Martínez, M. Toxina Botulínica Para Tratamiento Del Dolor Crónico. Revisión de La Evidencia. Rev. Colomb. Anestesiol. 2014, 42, 205–213. [Google Scholar] [CrossRef]
  57. Cinone, N.; Letizia, S.; Santoro, L.; Facciorusso, S.; Armiento, R.; Picelli, A.; Ranieri, M.; Santamato, A. Combined Effects of Isokinetic Training and Botulinum Toxin Type A on Spastic Equinus Foot in Patients with Chronic Stroke: A Pilot, Single-blind, Randomized Controlled Trial. Toxins 2019, 11, 210. [Google Scholar] [CrossRef]
  58. Hara, T.; Momosaki, R.; Niimi, M.; Yamada, N.; Hara, H.; Abo, M. Botulinum Toxin Therapy Combined with Rehabilitation for Stroke: A Systematic Review of Effect on Motor Function. Toxins 2019, 11, 707. [Google Scholar] [CrossRef]
  59. Carda, S.; Invernizzi, M.; Baricich, A.; Cisari, C. Casting, Taping or Stretching after Botulinum Toxin Type A for Spastic Equinus Foot: A Single-Blind Randomized Trial on Adult Stroke Patients. Clin. Rehabil. 2011, 25, 1119–1127. [Google Scholar] [CrossRef]
  60. Giovannelli, M.; Borriello, G.; Castri, P.; Prosperini, L.; Pozzilli, C. Early Physiotherapy after Injection of Botulinum Toxin Increases the Beneficial Effects on Spasticity in Patients with Multiple Sclerosis. Clin. Rehabil. 2007, 21, 331–337. [Google Scholar] [CrossRef]
  61. Esquenazi, A. Improvements in healthcare and cost benefits associated with botulinum toxin treatment of spasticity and muscle overactivity. Eur. J. Neurol. 2006, 13 (Suppl. S4), 27–34. [Google Scholar] [CrossRef] [PubMed]
  62. Prasad, K.; Kumar, A.; Misra, S.; Yadav, A.K.; Johri, S.; Sarkar, R.S.; Gorthi, S.P.; Hassan, K.M.; Prabhakar, S.; Misra, U.K.; et al. For InveST study group Reliability and validity of telephonic Barthel Index: An experience from multi-centric randomized control study. Acta Neurol. Belg. 2018, 118, 53–59. [Google Scholar] [CrossRef]
Figure 1. The main reason for discontinuation for patients n = 56.
Figure 1. The main reason for discontinuation for patients n = 56.
Toxins 14 00675 g001
Figure 2. The overall survival of patients who discontinued BoNT-A treatment, spasticity duration. A > 10 years, B < 10 years.
Figure 2. The overall survival of patients who discontinued BoNT-A treatment, spasticity duration. A > 10 years, B < 10 years.
Toxins 14 00675 g002
Figure 3. The overall survival of patients who discontinued BoNT-A treatment, comorbidities. A: comorbidities, B no comorbidities.
Figure 3. The overall survival of patients who discontinued BoNT-A treatment, comorbidities. A: comorbidities, B no comorbidities.
Toxins 14 00675 g003
Table 2. Reasons for BoNT-A discontinuation, % frequency. MS: Multiple sclerosis. SCI: spinal cord injury. TBI: Traumatic brain injury.
Table 2. Reasons for BoNT-A discontinuation, % frequency. MS: Multiple sclerosis. SCI: spinal cord injury. TBI: Traumatic brain injury.
Reason for BoNT-A
Discontinuation
Stroke (n = 30)MS (n = 9)SCI (n = 10)TBI (n = 7)
Absent/poor efficacy2 (6.67%)5 (55.56%)1 (10%)0
Adverse Events1 (3.33%)-00
Logistic reasons14 (46.67%)3 (33.33%)2 (20%)2 (28.57%)
Orthopedic functional surgery or ITB3 (10%)-7 (70%)5 (71.43%)
Pain during injections2 (6.67%)-00
Clinical worsening8 (26.67%)1 (11.11%)00
ITB: Intratechal baclofen therapy.
Table 3. Log-rank test estimation of BoNT-A discontinuation by clinical features. MDT= median discontinuation time; p < 0.05.
Table 3. Log-rank test estimation of BoNT-A discontinuation by clinical features. MDT= median discontinuation time; p < 0.05.
VariableMDT MonthsLog Rank
Chi-Square χ²
p Value
Age
<40 years40.310.571
>40 years6
Severity of spasticity
Mild (MAS < 2)156.850.027
Severe (MAS 3-4)5
Duration of spasticity
<10 years5.50.180.662
>10 years5
Comorbidities
Yes3.5510.83<0.001
No16
Pain
Yes15.515.75<0.001
No2
Rehabilitation
Yes 15.515.75<0.001
No2
Cognitive Impairment
Yes4.512.240.034
No14
Table 4. Cox regression analysis of hazard ratios of predictors of discontinuation of BoNT-A injection. p value < 0.05.
Table 4. Cox regression analysis of hazard ratios of predictors of discontinuation of BoNT-A injection. p value < 0.05.
Variable HR 95%CIp-Value
Severity of spasticity
Mild (MAS (2)1.7851.396–2.302<0.001
Severe (MAS 3-4)1.00
Pain
Yes1.3201.047–1.6390.025
No1.00
Rehabilitation
Yes 1.001.120–1.7600.05
No1.402
Comorbidities
Yes 1.000.961–1.6270.085
No1.256
Cognitive Impairment
Yes1.4031.110–1.6700.034
No1.00
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Cinone, N.; Santoro, L.; Spina, S.; Facciorusso, S.; Battaglia, M.; Baricich, A.; Marcogiuseppe, P.; Santamato, A. Reasons and Determinants of BoNT-A Treatment Discontinuation in Patients Living with Spasticity: A 10-Year Retrospective Analysis. Toxins 2022, 14, 675. https://doi.org/10.3390/toxins14100675

AMA Style

Cinone N, Santoro L, Spina S, Facciorusso S, Battaglia M, Baricich A, Marcogiuseppe P, Santamato A. Reasons and Determinants of BoNT-A Treatment Discontinuation in Patients Living with Spasticity: A 10-Year Retrospective Analysis. Toxins. 2022; 14(10):675. https://doi.org/10.3390/toxins14100675

Chicago/Turabian Style

Cinone, Nicoletta, Luigi Santoro, Stefania Spina, Salvatore Facciorusso, Marco Battaglia, Alessio Baricich, Pasqua Marcogiuseppe, and Andrea Santamato. 2022. "Reasons and Determinants of BoNT-A Treatment Discontinuation in Patients Living with Spasticity: A 10-Year Retrospective Analysis" Toxins 14, no. 10: 675. https://doi.org/10.3390/toxins14100675

APA Style

Cinone, N., Santoro, L., Spina, S., Facciorusso, S., Battaglia, M., Baricich, A., Marcogiuseppe, P., & Santamato, A. (2022). Reasons and Determinants of BoNT-A Treatment Discontinuation in Patients Living with Spasticity: A 10-Year Retrospective Analysis. Toxins, 14(10), 675. https://doi.org/10.3390/toxins14100675

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop