Next Article in Journal
Towards Improved Use of Vaccination in the Control of Infectious Bronchitis and Newcastle Disease in Poultry: Understanding the Immunological Mechanisms
Next Article in Special Issue
The Next Generation of Influenza Vaccines: Towards a Universal Solution
Previous Article in Journal
Confidence and Receptivity for COVID-19 Vaccines: A Rapid Systematic Review
Previous Article in Special Issue
Influenza Vaccination and Non-Pharmaceutical Measure Effectiveness for Preventing Influenza Outbreaks in Schools: A Surveillance-Based Evaluation in Beijing
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Vaccines
Volume 9
Issue 1
10.3390/vaccines9010019
vaccines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Influenza Vaccination and Myo-Pericarditis in Patients Receiving Immune Checkpoint Inhibitors: Investigating the Likelihood of Interaction through the Vaccine Adverse Event Reporting System and VigiBase

by
Milo Gatti
1,
Emanuel Raschi
1,
Ugo Moretti
2,
Andrea Ardizzoni
3,
Elisabetta Poluzzi
1 and
Igor Diemberger
4,*
1
Department of Medical and Surgical Sciences, Alma Mater Studiorum—University of Bologna, 40138 Bologna, Italy
2
Department of Public Health and Community Medicine, University of Verona, 37134 Verona, Italy
3
Medical Oncology Unit, Department of Experimental, Diagnostic and Specialty Medicine, Policlinico S. Orsola-Malpighi, Alma Mater Studiorum—University of Bologna, 40138 Bologna, Italy
4
Cardiology Unit, Department of Experimental, Diagnostic and Specialty Medicine, Alma Mater Studiorum—University of Bologna, 40138 Bologna, Italy
*
Author to whom correspondence should be addressed.
Vaccines 2021, 9(1), 19; https://doi.org/10.3390/vaccines9010019
Submission received: 4 December 2020 / Revised: 23 December 2020 / Accepted: 28 December 2020 / Published: 4 January 2021
(This article belongs to the Special Issue Research in Immune Responses to Vaccines Antigen during Influenza A Infection)
Versions Notes

Abstract

:
Background: Evidence on whether the influenza vaccine could exacerbate immune-related adverse events, including myopericarditis (MP), in patients treated with immune checkpoint inhibitors (ICIs), is still conflicting. We explored this issue through a global real-world approach. Methods: We queried the Vaccine Adverse Event Reporting System (VAERS) and VigiBase to retrieve cases of MP in which the influenza vaccine and ICIs were recorded as suspect and were concomitantly reported. For the included cases, causality assessment and Drug Interaction Probability Scale (DIPS) algorithms were applied. Results: There were 191 and 399 reports of MP with the influenza vaccine that were retrieved (VAERS and VigiBase, respectively). No case of MP reporting the concomitant use of ICIs and the influenza vaccine was found in VAERS, while three cases of myocarditis were retrieved in VigiBase. All of the cases were unclassifiable for a causality assessment because of the lack of data concerning latency. According to the DIPS, one report was categorized as possible and two as doubtful. Conclusion: The paucity of cases coupled with the doubtful causality assessment make the potential interaction between influenza vaccines and ICIs in cancer patients negligible from clinical and epidemiological standpoints. These findings support the cardiovascular safety of the influenza vaccination, which remains strongly recommended in cancer patients, especially in the current COVID-19 era.

1. Introduction

The safety of vaccines and medications is a current global safety issue. The annual vaccination against the influenza virus is the primary means of preventing influenza and its complications in high-risk subjects, including adult cancer patients, as described by observational studies, suggesting lower mortality and infection-related outcomes with influenza vaccination [1]. However, in the recent past, potential pharmacokinetic interactions have been proposed, considering that vaccines can influence the drug metabolism via inflammatory cytokines [2]. Therefore, potential interactions between influenza vaccines and drugs used for chronic diseases (e.g., immunosuppressive agents) cannot be excluded [3]. Moreover, cases of myo-pericarditis (MP) after immunization, although rarely, are reported [4,5].
Myocarditis and pericarditis represent serious and life-threatening inflammatory diseases involving myocardium and pericardium, potentially associated with the use of several drugs and vaccines [6,7,8]. In particular, cases of MP were described with the smallpox vaccine in early 2000 and, more recently, very rarely are reported after influenza immunization [4,5]. Immune checkpoint inhibitors (ICIs), such as anti-PD1, anti-PDL1, and anti-CTLA4 monoclonal antibodies, are approved as first-line agents in the management of melanoma and non-small cell lung cancer. They may cause a variegate spectrum of cardiovascular events, including MP, with a higher mortality compared with other immune-related adverse events (irAEs) [9,10,11,12,13]. In a summary of the product characteristics of the different ICIs, pericarditis and myocarditis are reported as uncommon (≥1/1000 to <1/100) and rare (≥1/10,000 to <1/1000, respectively) AEs. Notably, the detection of myocarditis with ICIs requires the permanent discontinuation of treatment, no matter the severity.
Consequently, the question arises as to whether pharmacokinetic and pharmacodynamic interactions occur in patients receiving the influenza vaccination (recommended both by oncologists and cardiologists) and ICIs, possibly causing or exacerbating irAEs such MP. Although no cases of myocarditis have been reported, evidence on whether the influenza vaccine exacerbates irAEs is still conflicting and poorly investigated [14]. Only a retrospective study investigated the clinical features of myocarditis with ICIs in patients receiving the influenza vaccine; reduced myocardial injury and a lower risk of major adverse cardiac events among recipients of the influenza vaccine was found compared with not vaccinated patients [15].
In the recent past, analysis of the spontaneous reporting systems (SRSs) has attracted considerable interest among clinicians for the accurate and timely characterization of drug- and vaccine-related risks occurring in the real world, where comorbidities and polypharmacotherapy exist. By offering a global epidemiological perspective, these pharmacovigilance studies have been pursued to test the hypothesis of potential associations, including refusing the likelihood of interactions [3,16,17], especially for rare, unexpected, and delayed AEs, such as MP.
In this study, we investigated the likelihood of interaction between the influenza vaccination and ICIs by analyzing spontaneous reports of MP collected from the Vaccine Adverse Event Reporting System (VAERS; P.O. Box 1100; Rockville, MD 20849-1100) and the World Health Organization’s (WHO) global Individual Case Safety Report database (VigiBase®; Uppsala Monitoring Centre Box, 1051 SE-751 40, Uppsala, Sweden).

2. Materials and Methods

2.1. Study Conception and Design

The study was conceived as an observational retrospective analysis of spontaneous reports of MP collected from the VAERS and the VigiBase® to (a) characterize relevant clinical features; (b) highlight the concomitant use of agents known to cause MP, including ICIs; and (c) assess the causality and probability of interaction between the influenza vaccine and ICIs.
VAERS is a national system to monitor the safety of US-licensed vaccines [18], whereas VigiBase® collects worldwide reports on vaccines and drugs, thus making these archives act as complementary approaches [19].
VAERS transmits its vaccine adverse event reports to the VigiBase, in order to contribute to the global pharmacovigilance effort along with other countries that employ passive vaccine safety monitoring systems [18], thus possible duplicates between the two databases may exist.

2.2. Data Source

Established in 1990, the VAERS is co-managed by the Centers for Disease Control and Prevention and the US Food and Drug Administration (FDA); it collects and analyzes reports of AEs following immunization (AEFI) for vaccines licensed in the US, receiving approximately 28,000 reports of AEFI annually.
AEFI may be any unfavorable or unintended sign, abnormal laboratory finding, symptom, or disease that occurs following or during the administration of a vaccine. AEs are temporally associated events that may either be caused by a vaccine or be coincidental to it, i.e., not necessarily related to the vaccination [20]. VAERS may be used to detect unexpected and rare patterns of AEFI, unlikely to arise in pivotal trials because of the limited number of patients involved [21]. Health-care professionals, vaccine manufacturers, and consumers (patients, parents, and caregivers) can submit reports of AEs to VAERS.
VigiBase® is one of the largest and most comprehensive pharmacovigilance databases, maintained by the Uppsala Monitoring Centre in Sweden, and containing over 20 million of Individual Case Safety Reports from 110 countries over the five continents. VigiBase® collects reports of AEs for authorized drugs, vaccines, and food supplements, submitted from healthcare professionals, pharmaceutical companies, and patients.
For each vaccine/drug, the characterization of the vaccine/drug role indicated by the primary reporter (i.e., the original source of the information) includes the following three categories: suspect, concomitant, and interacting. All spontaneous reports should have at least one suspect vaccine/drug, namely involved, presumably, in the occurrence of AEFI. If the reporter indicates a suspected interaction, all interacting vaccines/drugs are considered to be suspect vaccines/drugs.
In both databases, AEs are codified through the Medical Dictionary for Regulatory Activities (MedDRA) terminology, and described and organized in terms of Preferred Terms (PTs).

2.3. Data Extraction and Analysis

A multi-step approach was followed for data extraction in each database:
(1)
Reports for individuals receiving any type of vaccine against the influenza virus categorized as suspect AEFI submitted to VAERS (from July 1990 to September 2020) and VigiBase® (from inception to October 2020) were selected.
(2)
Cases of myocarditis or pericarditis were extracted through specific PTs and lowest level terms, in line with previous studies on the influenza vaccination [4] and the potential immune-related basis of cardiotoxicity documented for ICIs by recent pharmacovigilance analyses [9,10]: myocarditis, pericarditis, immune-mediated myocarditis, and myopericarditis.
(3)
Reports recording AEs of interest were finally assessed for co-reported drugs/vaccines of interest.
In VigiBase, retained reports (refer to point 2) with ICIs, namely PD-1 inhibitors (nivolumab, pembrolizumab, and cemiplimab), PD-L1 inhibitors (atezolizumab, avelumab, and durvalumab), and CTLA-4 inhibitors (ipilimumab and tremelimumab), were evaluated by extracting the following data: demographic features (age, gender, weight, year, and reporter country), reporter qualification, comorbidities, concomitant medications (including dose and route of administration when available), latency, degree of seriousness, outcome, and management of the AE. The same approach was applied to identify cases of interest reported for vaccines against the influenza virus, both as suspect alone or concomitant to ICI exposure.
In VAERS, for each report of interest with the influenza vaccines, the following data were extracted: demographic features (age, gender, year, and reporter country), type of vaccine against the influenza virus, medical history, concomitant drugs or administered vaccines, laboratory data, AEs codified as PTs, latency, AE degree of seriousness, outcome, and narratives, when available.
According to WHO criteria, a serious AE is any untoward medical occurrence that, no matter the dose, results as fatal, causing a life-threatening event, requiring hospitalization of the patient, causing serious/permanent disability, causing congenital abnormalities, or other clinically relevant conditions [22].
Co-reported medications and comorbidities known to cause myocarditis or pericarditis (i.e., potential confounders) were further characterized according to the lists proposed by Adler et al. [6], Caforio et al. [7], and Butany et al. [8].

2.4. Causality Assessment and Evaluation of Drug–Vaccine Interaction

The probability of interaction when the influenza vaccine and ICIs are co-reported was evaluated by applying the established WHO criteria for causality assessment [20], as well as the Drug Interaction Probability Scale (DIPS) [23].
The following items were accounted for in the AEFI causality assessment: temporal relationship, alternate explanations, proof of association, prior evidence, population-based evidence, and biological plausibility [20]. Cases without adequate information were classified as “unclassifiable”; cases with adequate information were categorized as: (1) “consistent with a causal relationship”, when the available evidence supported a causal relationship between the vaccine and the AEFI in the individual, but it did not rule out the possibility that the AEFI may have been caused by a factor other than the vaccine; (2) “inconsistent with a causal association”, when the available evidence did not support a causal relationship between vaccine administration and the reported AEFI in the individual; and (3) “indeterminate”, when the temporal relationship was consistent but the available evidence insufficient to support or rule out a causal relationship in the individual.
The DIPS is a 10-item tool specifically developed to assess the likelihood of drug–drug interactions [23]. ICIs were considered as the objective drugs (i.e., the one affected by the presence of another vaccine/drug), while the influenza vaccine was considered as the precipitant agent (i.e., the one causing a change on the object drug). To avoid overemphasis on the role of the vaccine, the answer to the second question was always “unknown” (knowledge on the mechanism of the interaction is hypothesized and literature data are very scarce/uncertain); in addition, questions 5, 6, and 10 were not assessable/applicable (dechallenge, rechallenge, and dose adjustments are unfeasible for vaccines considering the peculiarities of their administration), as previously performed [3]. The final summary score could reach 10 points. Higher total scores correspond to a higher likelihood of drug–vaccine interaction (i.e., >8 = highly probable; 5–8 = probable; 2–4 = possible; <2 = doubtful).

3. Results

3.1. Demographic and Clinical Data

Over the observed period, out of a total of 712,776 AEFI, 191 (0.03%) reports of MP mentioning the influenza vaccine as suspect were collected within the VAERS.
The clinical and demographic characteristics are provided in Table 1. The reports included 124 men (64.9%) and 62 women (in five cases gender was not reported), aged between 1 and 88 years (mean age 44.5 years). Pericarditis were reported in 117 cases (61.3%), followed by myocarditis (81 cases; 42.4%) and MP (7 cases; 3.7%). No cases of immune-mediated myocarditis were retrieved.
The median onset was seven days; in 34.0% of reports, the AEFI of interest occurred in the first 3 days after the administration of the influenza vaccine. Comorbidities potentially implicated in the occurrence of the AEFI of interest were retrieved in six cases (3.1%), with ulcerative colitis and systemic lupus erythematosus being the most represented. Other vaccines were concomitantly administered in 59 cases (30.9%), accounting for a mean number of 3.1 vaccines per patient.
In 141 cases (73.8%), the report was classified as serious, and 16 fatal cases (8.4%) were found. Recovery occurred in 31.4% of cases.
In the VigiBase®, 246,864 reports mentioning the influenza vaccine as a suspect agent were found, and myocarditis/pericarditis were reported in 399 cases (0.16%; Table 1).
The cases showed a mean age of 46.7 years, with a male preponderance (65.9%). Reports were largely submitted from US (45.9%) and Europe (45.1%). Pericarditis was reported in 225 cases (56.4%), followed by myocarditis (193 cases; 48.4%) and myopericarditis (29; 7.3%). No cases of immune-mediated myocarditis were found.
Median onset was five days, and in 29.8% of reports, AEs of interest occurred in the first three days after the administration of the influenza vaccine. Comorbidities potentially implicated in the occurrence of myocarditis/pericarditis were retrieved in two patients (0.5%) affected by ulcerative colitis. Other vaccines were concomitantly administered in 87 cases (21.8%), accounting for a mean number of 2.5 vaccines per patient.
In 303 cases (76.0%), the report was classified as serious, and 28 fatal cases (7.0%) were found. Recovering occurred in 41.1% of cases.

3.2. Co-Reported Medications and Detection of Cases with ICI Administration

Overall, concomitant medications were found in 49 (25.7%) and 57 (14.3%) cases with the influenza vaccine in VAERS and VigiBase®, respectively (Table 1). In 14 (7.3%) and 13 (3.3%) cases, five or more concomitant drugs were reported in VAERS and VigiBase®, respectively.
Concomitant medications potentially implicated in the occurrence of myocarditis/pericarditis were retrieved in 21 (11.0%) and 24 (6.0%) cases in VAERS and VigiBase®, respectively, with hydrochlorothiazide being the most frequent in both databases (in six and five cases, respectively).
Only six cases (one classified as serious, where Guillain-Barré syndrome was recorded) with the concomitant use of ICIs and the influenza vaccine were found in VAERS (Table S1), but no cases of myocarditis/pericarditis. In VigiBase®, two cases of myocarditis reporting the concomitant administration of ICIs and the influenza vaccine both mentioned as suspect agents were retrieved. A third case of myocarditis in which the influenza vaccine was classified as a concomitant agent was detected (Table 2).
The three cases were classified as serious (causing prolonged hospitalization), included two men and one woman, aged 67–77 years, and affected by lung or renal carcinoma. Nivolumab, pembrolizumab, and ipilimumab/nivolumab combination therapy were reported as the ICI regimen. In two cases ICI was withdrawn, while recovering occurred in only one case.

3.3. Causality Assessment and Evaluation of Drug–Vaccine Interaction

These three cases showed an unclassifiable causality assessment due to the lack of data concerning the latency and time window of increased risk (Table 2).
By applying the adapted DIPS algorithm, one report was categorized as possible (due to the lack of underlying diseases or co-reported agents known to cause or precipitate myocarditis, and to the detection of troponin increase) and two as doubtful (only the absence of underlying diseases or co-reported agents known to cause or precipitate myocarditis were recognized; Table S2).

4. Discussion

To the best of our knowledge, this is the first real-world study investigating the potential interactions between influenza vaccines and ICIs resulting in myocarditis or pericarditis in cancer patients, by assessing spontaneous reports submitted to the VAERS and VigiBase®. This global post-marketing safety study stems from recent conflicting real-world evidence surrounding the possible exacerbation of irAEs with influenza vaccines in patients treated with ICIs [14,15,24,25,26,27,28,29], thus joining in the wider debate on the bidirectional relationship between immunotherapy and the influenza vaccination, potentially affecting the clinical and humoral efficacy of the vaccine, performance of ICIs, and safety [14].
Four major findings emerged from our analysis: (a) myocarditis and pericarditis represented a very rare AEFI (with a non-negligible proportion of death, approximately 8–9%), based on the low reporting frequency retrieved in both SRSs (<0.1–0.2% of overall AEs), also considering the influenza vaccination coverage rates (estimated at about 25% per year in Europe [30,31]) and the relevant million doses distributed worldwide; (b) the reporting of myocarditis in patients concomitantly receiving influenza vaccines and ICIs is limited to only three out of a total of 1465 cases of myocarditis/pericarditis found with influenza vaccines or ICIs (considered as suspect agents) in the two databases; (c) these cases were unclassifiable for causality assessment with a doubtful probability of drug–vaccine interaction (according to the adapted DIPS algorithm), thus suggesting that the vaccine is not directly involved in the occurrence of myocarditis (and in one of the three cases, influenza vaccine was reported only as concomitant and not as suspect or interacting agent); and (d) no other cardiovascular AEs were found in patients concomitantly receiving influenza vaccination and ICIs, as well as no other irAEs were reported, except for a single case of Guillain-Barré syndrome, a condition already associated with both ICIs and the influenza vaccine [32,33,34].
Theoretically, a possible pharmacodynamic interaction between influenza vaccines and ICIs leading to the exacerbation of irAEs (including MP), could be supposed. The blockade of a PD-1/PD-L1 pathway together with the vaccination (particularly in conjunction with a strong vaccine adjuvant) could enhance one or more of the mechanisms associated with irAEs onset (infiltration of central memory T cells into the tissues, cross-presentation of shared antigens, and exacerbation of previously subclinical auto-immune syndromes) [24].
Our specific focus on the myo-pericardium stems from the peculiar clinical features of myocarditis in terms of severity and mortality compared with other irAEs, usually requiring immunotherapy discontinuation [9,10,11,12,13], as reported in two out of our three cases. Although, in all three cases, no underlying diseases or other agents known to cause or precipitate myocarditis were recorded, the lack of data concerning latency and solid literature studies allows for minimizing, if not completely excluding, the clinically relevant contribution of a potential drug–vaccine interaction in our cases.
Furthermore, the only study investigating the association between influenza vaccine and the development of myocarditis among patients on ICIs found that the administration of the vaccine was not associated with an increased risk of subsequent myocarditis [15]. Additionally, myocarditis cases in which the influenza vaccine was administered showed lower troponin levels at presentation and a lower risk of major adverse cardiac events at follow-up compared with non-vaccinate patients developing myocarditis with ICIs [15], thus suggesting a protective role for the influenza vaccine in this setting.
ICIs caused a paradigm shift in cancer treatment, and are currently used as first-line agents in the management of non-small cell lung cancer, melanoma, and renal carcinoma [35]. Considering their evolving role and expected increasing uptake, the assessment of cardiovascular safety is of paramount importance. Likewise, the safety of influenza vaccines represents an important issue, given the high morbidity and mortality rates caused by influenza in cancer patients [36]. Notably, the development of influenza infection may also, albeit rarely, be associated itself with an increased risk of myocarditis and major adverse cardiovascular events [37,38]. Therefore, the suggested protective role of the influenza vaccine on cardiovascular outcome [15] cannot be overlooked.
Collectively, our findings provide a reassuring message in terms of cardiovascular safety for cancer patients treated with ICIs and requiring the influenza vaccination. Of note, less than 20% of patients affected by malignancies received the influenza vaccine, with a gradual decline in the last decade [39]. We support and promote the achievement of an optimal vaccination coverage rate in cancer patients for several reasons, including the following: (a) direct association with a lower mortality and infection-related outcomes in immunosuppressed adults [1]; (b) a better overall survival recently reported in patients treated with ICIs receiving influenza vaccination [40]; and (c) the current COVID-19 pandemic, to reduce the strain on the healthcare system while protecting vulnerable subjects from the dramatic impact of a possible co-infection [41]. Cancer patients receiving immunotherapy are at high risk of severe events as a result of COVID-19 systemic involvement, including pneumonitis and myocarditis [42], and a recent systematic review found no significant increase in the risk of infection or in the illness severity or lethality of COVID-19 in subjects receiving the influenza vaccine, with some studies reporting a significantly inverse association [43]. Therefore, the implementation of measures aimed at raising influenza vaccination coverage in frail patients is strongly recommended.
We acknowledge the limitations of our study, mainly inherent to the nature of SRSs data. VAERS and VigiBase® are subject to reporting bias, including under- and over-reporting of adverse events, although there are not clues for major distortions, considering that serious events such as MP are less prone to under-reporting, and no specific warnings were posted by regulatory agencies, thus minimizing the existence and impact of a stimulated reporting [3,18]. We recognize the potential occurrence of subclinical myocarditis, which may be under-diagnosed/under-detected, and is thus likely to be under-reported. Furthermore, the quality and completeness of the reports collected in both databases are variable, and many records lack valid medical diagnoses, thus making the assessment of causality challenging. Additionally, DIPS was not specifically developed to assess drug–vaccine interactions, although we implemented an adapted version in order to the better focus on the possible precipitating role of the influenza vaccine.
Notwithstanding these limitations, pharmacovigilance assessment represents an invaluable opportunity to monitor vaccine safety and identify novel rare signals, potentially arising from drug–vaccine interactions, both from a local and international perspective. Furthermore, our findings are consistent between the two databases, thus supporting the lack of evidence of a clinically relevant drug–vaccine interaction. The identification of myocarditis with ICIs, in line with previous findings [9,10,11], further corroborates the ability of our post-marketing approach to identify actual true-positive associations.
In conclusion, the paucity of cases coupled with a lack of certainty in terms of causality assessment and doubtful probability make the risk of myocarditis and pericarditis by interaction between influenza vaccines and ICIs in cancer patients negligible from both clinical and epidemiological standpoints. Our findings support the cardiovascular safety of influenza vaccines in subjects treated with immunotherapy, thereby emphasizing the importance of a flu vaccination in this population, especially in the current COVID-19 era.

Supplementary Materials

The following are available online at https://www.mdpi.com/2076-393X/9/1/19/s1. Table S1: Case-by-case assessment of reports recorded in VAERS in which influenza vaccine and immune checkpoint inhibitors (ICIs) were concomitantly used. Table S2: Application of the adapted version of Drug Interaction Probability Scale in cases of myocarditis/pericarditis in which immune checkpoint inhibitors (ICIs) and influenza vaccine were concomitantly administered.

Author Contributions

M.G.: methodology, investigation, formal analysis, and writing—original draft. E.R.: conceptualization, methodology, formal analysis, and writing—reviewing and editing. U.M.: writing—reviewing and editing. A.A.: writing—reviewing and editing. E.P.: conceptualization, methodology, formal analysis, and writing—reviewing and editing. I.D.: conceptualization and writing—reviewing and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Institutional Review Board Statement

Under current legislation, institutional review board approval is not required when performing analysis of the publicly available VAERS and VigiBase databases, because it contains anonymized data that cannot allow patients’ identification.

Informed Consent Statement

Patient consent was waived due to VAERS and VigiBase databases contain anonymized data that cannot allow patients’ identification.

Data Availability Statement

Data supporting the findings of this study were derived from the following resource available in the public domains: https://wonder.cdc.gov/vaers.html and http://www.vigiaccess.org/.

Acknowledgments

We would like to thank Annette Rudolph for data extraction from VigiBase®. Authors at the University of Bologna are supported by institutional research funds (Ricerca Fondamentale Orientata). No sources of funding were received for the preparation of this article.

Conflicts of Interest

M.G., E.R., E.P., U.M., and I.D. have no conflicts of interest relevant to the content of the present work. A.A. reports grants and personal fees from BMS, as well as grants from MSD, Roche, Astra Zeneca, Eli-Lilly, Takeda, and Bayer, outside of the submitted work.

References

  1. Bitterman, R.; Eliakim-Raz, N.; Vinograd, I.; Zalmanovici Trestioreanu, A.; Leibovici, L.; Paul, M. Influenza vaccines in immunosuppressed adults with cancer. Cochrane Database Syst. Rev. 2018, 1, CD008983. [Google Scholar] [CrossRef] [PubMed]
  2. Pellegrino, P.; Perrotta, C.; Clementi, E.; Radice, S. Vaccine-Drug Interactions: Cytokines, Cytochromes, and Molecular Mechanisms. Drug Saf. 2015, 38, 781–787. [Google Scholar] [CrossRef] [PubMed]
  3. Carnovale, C.; Raschi, E.; Leonardi, L.; Moretti, U.; De Ponti, F.; Gentili, M.; Pozzi, M.; Clementi, E.; Poluzzi, E.; Radice, S. No signal of interactions between influenza vaccines and drugs used for chronic diseases: A case-by-case analysis of the vaccine adverse event reporting system and vigibase. Expert Rev. Vaccines 2018, 17, 363–381. [Google Scholar] [CrossRef] [PubMed]
  4. Mei, R.; Raschi, E.; Forcesi, E.; Diemberger, I.; De Ponti, F.; Poluzzi, E. Myocarditis and pericarditis after immunization: Gaining insights through the Vaccine Adverse Event Reporting System. Int. J. Cardiol. 2018, 273, 183–186. [Google Scholar] [CrossRef] [PubMed]
  5. Mei, R.; Raschi, E.; Poluzzi, E.; Diemberger, I.; De Ponti, F. Recurrence of pericarditis after influenza vaccination: A case report and review of the literature. BMC Pharmacol. Toxicol. 2018, 19, 20. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Adler, Y.; Charron, P.; Imazio, M.; Badano, L.; Barón-Esquivias, G.; Bogaert, J.; Brucato, A.; Gueret, P.; Klingel, K.; Lionis, C.; et al. 2015 ESC Guidelines for the diagnosis and management of pericardial diseases: The Task Force for the Diagnosis and Management of Pericardial Diseases of the European Society of Cardiology (ESC)Endorsed by: The European Association for Cardio-Thoracic Surgery (EACTS). Eur. Heart J. 2015, 7, 2921–2964. [Google Scholar]
  7. Caforio, A.L.; Pankuweit, S.; Arbustini, E.; Basso, C.; Gimeno-Blanes, J.; Felix, S.B.; Fu, M.; Heliö, T.; Heymans, S.; Jahns, R.; et al. Current state of knowledge on aetiology, diagnosis, management, and therapy of myocarditis: A position statement of the European Society of Cardiology Working Group on Myocardial and Pericardial Diseases. Eur. Heart J. 2013, 34, 2636–2648. [Google Scholar] [CrossRef]
  8. Butany, J.; Ahn, E.; Luk, A. Drug-related cardiac pathology. J. Clin. Pathol. 2009, 62, 1074–1084. [Google Scholar] [CrossRef] [Green Version]
  9. Salem, J.-E.; Manouchehri, A.; Moey, M.; Lebrun-Vignes, B.; Bastarache, L.; Pariente, A.; Gobert, A.; Spano, J.-P.; Balko, J.M.; Bonaca, M.P.; et al. Cardiovascular toxicities associated with immune checkpoint inhibitors: An observational, retrospective, pharmacovigilance study. Lancet Oncol. 2018, 19, 1579–1589. [Google Scholar] [CrossRef]
  10. Moslehi, J.J.; Salem, J.E.; Sosman, J.A.; Lebrun-Vignes, B.; Johnson, D.B. Increased reporting of fatal immune checkpoint inhibitor-associated myocarditis. Lancet 2018, 10, 933. [Google Scholar] [CrossRef] [Green Version]
  11. Raschi, E.; Diemberger, I.; Poluzzi, E.; De Ponti, F. Reporting of immune checkpoint inhibitor-associated myocarditis. Lancet 2018, 392, 383. [Google Scholar] [CrossRef]
  12. Martins, F.; Sofiya, L.; Sykiotis, G.P.; Lamine, F.; Maillard, M.; Fraga, M.; Shabafrouz, K.; Ribi, C.; Cairoli, A.; Guex-Crosier, Y.; et al. Adverse effects of immune-checkpoint inhibitors: Epidemiology, management and surveillance. Nat. Rev. Clin. Oncol. 2019, 16, 563–580. [Google Scholar] [CrossRef] [PubMed]
  13. Postow, M.A.; Hellmann, M.D. Adverse Events Associated with Immune Checkpoint Blockade. N. Engl. J. Med. 2018, 22, 1165. [Google Scholar]
  14. Bersanelli, M.; Buti, S.; De Giorgi, U.; Di Maio, M.; Giannarelli, D.; Pignata, S.; Banna, G.L. State of the art about influenza vaccination for advanced cancer patients receiving immune checkpoint inhibitors: When common sense is not enough. Crit. Rev. Oncol. Hematol. 2019, 139, 87–90. [Google Scholar] [CrossRef] [PubMed]
  15. Awadalla, M.; Golden, D.L.A.; Mahmood, S.S.; Alvi, R.M.; Mercaldo, N.D.; Hassan, M.Z.O.; Banerji, D.; Rokicki, A.; Mulligan, C.; Murphy, S.P.T.; et al. Influenza vaccination and myocarditis among patients receiving immune checkpoint inhibitors. J. Immunother. Cancer 2019, 22, 53. [Google Scholar] [CrossRef] [PubMed]
  16. Raschi, E.; Gatti, M.; Gelsomino, F.; Ardizzoni, A.; Poluzzi, E.; De Ponti, F. Lessons to be Learnt from Real-World Studies on Immune-Related Adverse Events with Checkpoint Inhibitors: A Clinical Perspective from Pharmacovigilance. Target Oncol. 2020, 15, 449–466. [Google Scholar] [CrossRef] [PubMed]
  17. Antonazzo, I.C.; Poluzzi, E.; Forcesi, E.; Salvo, F.; Pariente, A.; Marchesini, G.; De Ponti, F.; Raschi, E. Myopathy with DPP-4 inhibitors and statins in the real world: Investigating the likelihood of drug-drug interactions through the FDA adverse event reporting system. Acta Diabetol. 2020, 57, 71–80. [Google Scholar] [CrossRef]
  18. Shimabukuro, T.T.; Nguyen, M.; Martin, D.; DeStefano, F. Safety monitoring in the Vaccine Adverse Event Reporting System (VAERS). Vaccine 2015, 33, 4398–4405. [Google Scholar] [CrossRef] [Green Version]
  19. Gatti, M.; Ippoliti, I.; Poluzzi, E.; Antonazzo, I.C.; Moro, P.A.; Moretti, U.; Menniti-Ippolito, F.; Mazzanti, G.; De Ponti, F.; Raschi, E. Assessment of adverse reactions to α-lipoic acid containing dietary supplements through spontaneous reporting systems. Clin. Nutr. 2020. [Google Scholar] [CrossRef]
  20. Causality Assessment of an Adverse Event Following Immunization (AEFI). User Manual for the Revised WHO Classification. Available online: https://www.who.int/vaccine_safety/publications/CausalityAssessmentAEFI_EN.pdf?ua=1 (accessed on 4 November 2020).
  21. Singleton, J.A.; Lloyd, J.C.; Mootrey, G.T.; Salive, M.E.; Chen, R.T. An overview of the vaccine adverse event reporting system (VAERS) as a surveillance system. VAERS Working Group. Vaccine 1999, 17, 2908–2917. [Google Scholar] [CrossRef]
  22. Edwards, J.R.; Aronson, J.K. Adverse drug reactions: Definitions, diagnosis, and management. Lancet 2000, 356, 1255e9. [Google Scholar] [CrossRef]
  23. Horn, J.R.; Hansten, P.D.; Chan, L.N. Proposal for a new tool to evaluate drug interaction cases. Ann. Pharmacother. 2007, 41, 674–680. [Google Scholar] [CrossRef] [PubMed]
  24. Läubli, H.; Balmelli, C.; Kaufmann, L.; Stanczak, M.; Syedbasha, M.; Vogt, D.; Hertig, A.; Müller, B.; Gautschi, O.; Stenner-Liewen, F.; et al. Influenza vaccination of cancer patients during PD-1 blockade induces serological protection but may raise the risk for immune-related adverse events. J. Immunother. Cancer 2018, 6, 40. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Chung, C. To do or not to do: A concise update of current clinical controversies in immune checkpoint blockade. J. Oncol. Pharm. Pr. 2019, 25, 663–673. [Google Scholar] [CrossRef] [PubMed]
  26. Chong, C.R.; Park, V.J.; Cohen, B.; Postow, M.A.; Wolchok, J.D.; Kamboj, M. Safety of Inactivated Influenza Vaccine in Cancer Patients Receiving Immune Checkpoint Inhibitors. Clin. Infect. Dis. 2020, 70, 193–199. [Google Scholar] [CrossRef] [PubMed]
  27. Failing, J.J.; Ho, T.P.; Yadav, S.; Majithia, N.; Riaz, I.B.; Shin, J.Y.; Schenk, E.L.; Xie, H. Safety of Influenza Vaccine in Patients with Cancer Receiving Pembrolizumab. JCO Oncol. Pract. 2020, 16, e573–e580. [Google Scholar] [CrossRef]
  28. Wijn, D.H.; Groeneveld, G.H.; Vollaard, A.M.; Muller, M.; Wallinga, J.; Gelderblom, H.; Smit, E.F. Influenza vaccination in patients with lung cancer receiving anti-programmed death receptor 1 immunotherapy does not induce immune-related adverse events. Eur. J. Cancer 2018, 104, 182–187. [Google Scholar] [CrossRef]
  29. Gwynn, M.E.; DeRemer, D.L.; Saunders, K.M.; Parikh, J.; Bollag, R.J.; Clemmons, A.B. Immune-mediated adverse events following influenza vaccine in cancer patients receiving immune checkpoint inhibitors. J. Oncol. Pharm. Pract. 2020, 26, 647–654. [Google Scholar] [CrossRef]
  30. Blank, P.R.; Schwenkglenks, M.; Szucs, T.D. Influenza vaccination coverage rates in five European countries during season 2006/07 and trends over six consecutive seasons. BMC Public Health. 2008, 8, 272. [Google Scholar] [CrossRef] [Green Version]
  31. Müller, D.; Szucs, T.D. Influenza vaccination coverage rates in 5 European countries: A population-based cross-sectional analysis of the seasons 02/03, 03/04 and 04/05. Infection 2007, 35, 308–319. [Google Scholar] [CrossRef] [Green Version]
  32. Martín Arias, L.H.; Sanz, R.; Sáinz, M.; Treceño, C.; Carvajal, A. Guillain-Barre syndrome and influenza vaccines: A meta-analysis. Vaccine 2015, 17, 3773–3778. [Google Scholar] [CrossRef]
  33. Supakornnumporn, S.; Katirji, B. Guillain-Barre Syndrome Triggered by Immune Checkpoint Inhibitors: A Case Report and Literature Review. J. Clin. Neuromuscul. Dis. 2017, 19, 80–83. [Google Scholar] [CrossRef] [PubMed]
  34. Yuen, C.; Kamson, D.; Soliven, B.; Kramer, C.; Goldenberg, F.; Rezania, K. Severe Relapse of Vaccine-Induced Guillain-Barre Syndrome After Treatment with Nivolumab. J. Clin. Neuromuscul. Dis. 2019, 20, 194–199. [Google Scholar] [CrossRef] [PubMed]
  35. Emens, L.A.; Ascierto, P.A.; Darcy, P.K.; Demaria, S.; Eggermont, A.M.M.; Redmond, W.L.; Seliger, B.; Marincola, F.M. Cancer immunotherapy: Opportunities and challenges in the rapidly evolving clinical landscape. Eur. J. Cancer 2017, 81, 116–129. [Google Scholar] [CrossRef] [PubMed]
  36. Cooksley, C.D.; Avritscher, E.B.; Bekele, B.N.; Rolston, K.V.; Geraci, J.M.; Elting, L.S. Epidemiology and outcomes of serious influenza-related infections in the cancer population. Cancer 2005, 104, 618–628. [Google Scholar] [CrossRef]
  37. Udell, J.A.; Zawi, R.; Bhatt, D.L.; Keshtkar-Jahromi, M.; Gaughran, F.; Phrommintikul, A.; Ciszewski, A.; Vakili, H.; Hoffman, E.B.; Farkouh, M.E.; et al. Association between influenza vaccination and cardiovascular outcomes in high-risk patients: A meta-analysis. JAMA 2013, 310, 1711–1720. [Google Scholar] [CrossRef]
  38. Mamas, M.A.; Fraser, D.; Neyses, L. Cardiovascular manifestations associated with influenza virus infection. Int. J. Cardiol. 2008, 130, 304–309. [Google Scholar] [CrossRef]
  39. Poeppl, W.; Lagler, H.; Raderer, M.; Sperr, W.R.; Zielinski, C.; Herkner, H.; Burgmann, H. Influenza vaccination perception and coverage among patients with malignant disease. Vaccine 2015, 30, 1682–1687. [Google Scholar] [CrossRef]
  40. Bersanelli, M.; Buti, S.; Banna, G.L.; De Giorgi, U.; Cortellini, A.; Rebuzzi, S.E.; Tiseo, M.; Fornarini, G.; Mazzoni, F.; Panni, S.; et al. Impact of influenza syndrome and flu vaccine on survival of cancer patients during immunotherapy in the INVIDIa study. Immunotherapy 2020, 12, 151–159. [Google Scholar] [CrossRef]
  41. Capone, A. Simultaneous circulation of COVID-19 and flu in Italy: Potential combined effects on the risk of death? Int. J. Infect. Dis. 2020, 99, 393–396. [Google Scholar] [CrossRef]
  42. Bersanelli, M.; Zielli, T.; Perrone, F.; Casartelli, C.; Pratticò, F.; Rapacchi, E.; Camisa, R.; Tognetto, M.; Clemente, A.; Giannarelli, D.; et al. Clinical impact of COVID-19 in a single-center cohort of a prospective study in cancer patients receiving immunotherapy. Immunotherapy 2020, 12, 1139–1148. [Google Scholar] [CrossRef] [PubMed]
  43. Del Riccio, M.; Lorini, C.; Bonaccorsi, G.; Paget, J.; Caini, S. The Association between Influenza Vaccination and the Risk of SARS-CoV-2 Infection, Severe Illness, and Death: A Systematic Review of the Literature. Int. J. Environ. Res. Public Health 2020, 17, 7870. [Google Scholar] [CrossRef] [PubMed]
Table
Table 1. Demographic and clinical feature of cases of myocarditis/pericarditis reported with influenza vaccine as suspect in Vaccine Adverse Event Reporting System (VAERS) and the World Health Organization’s (WHO) global Individual Case Safety Report database (VigiBase®).
Table 1. Demographic and clinical feature of cases of myocarditis/pericarditis reported with influenza vaccine as suspect in Vaccine Adverse Event Reporting System (VAERS) and the World Health Organization’s (WHO) global Individual Case Safety Report database (VigiBase®).
Demographic FeaturesVAERSVigiBase
Overall number of cases191399
Proportion of cases (based on overall number of reports with the flu vaccine)0.03% (712,776)0.16% (246,864)
Age (mean)44.5 ± 22.946.7 ± 22.3
Sex
Female62 (32.5%)131 (32.8%)
Male124 (64.9%)263 (65.9%)
Not specified5 (2.6%)5 (1.3%)
Reporter country
US109 (57.1%)183 (45.9%)
Non-US60 (31.4%)-
Europe-180 (45.1%)
Asia-10 (2.5%)
Oceania-19 (4.8%)
America (except US)-7 (1.7%)
Not specified22 (11.5%)-
Reported symptoms (preferred terms) *
Myocarditis81 (42.4%)193 (48.4%)
Pericarditis117 (61.3%)225 (56.4%)
Myopericarditis7 (3.7%)29 (7.3%)
Immune-mediated myocarditis0 (0.0%)0 (0.0%)
Co-medications
Overall number of cases49 (25.7%)57 (14.3%)
≥5 concomitant drugs14 (7.3%)13 (3.3%)
1–4 concomitant drugs35 (18.4%)44 (11.0%)
No concomitant drugs142 (74.3%)342 (85.7%)
Co-medications potentially implicated in the occurrence of myocarditis/pericarditis **
Number of cases21 (11.0%) §24 (6.0%) #
Immune checkpoint inhibitors (ICIs)None2
Hydrochlorothiazide65
Indomethacin22
Glipizide2-
Clonazepam2-
Alprazolam2-
Furosemide11
Bendroflumethiazide12
Mesalazine12
Colchicine11
Doxycycline12
Cotrimoxazole11
Lorazepam1-
Amoxicillin11
Isosorbide dinitrate12
Tetracycline-2
Spironolactone-1
Cefuroxime-1
Heparin-1
Bromazepam-1
Comorbidities potentially implicated in occurrence of myocarditis/pericarditis **
Number of cases6 (3.1%)2 (0.5%)
Ulcerative colitis22
Systemic lupus erythematosus2-
Juvenile rheumatoid arthritis1-
Insulin-dependent diabetes mellitus1-
Onset (days; median)7 (1.5–13)5 (1–12)
≤3 days65 (34.0%)119 (29.8%)
4–7 days19 (10.0%)50 (12.5%)
8–14 days35 (18.4%)61 (15.3%)
≥15 days36 (18.8%)56 (14.0%)
Not specified36 (18.8%)113 (28.4%)
Seriousness
Serious141 (73.8%)303 (76.0%)
Non-serious50 (26.2%)48 (12.0%)
Not specified-48 (12.0%)
Seriousness criteria *
Congenital anomaly0 (0.0%)0 (0.0%)
Death16 (8.4%)28 (7.0%)
Hospitalization125 (65.5%)222 (55.6%)
Life-threatening30 (15.7%)46 (11.5%)
Permanent disability12 (6.3%)12 (3.0%)
Other outcomes30 (15.7%)60 (15.0%)
Median time of hospitalization (days)3 (2–4)NA
Recovering
Recovered60 (31.4%)164 (41.1%)
Not recovered64 (33.5%)61 (15.3%)
Not specified67 (35.1%)174 (43.6%)
Concomitant other vaccines
Overall number of cases59 (30.9%)87 (21.8%)
Mean number of vaccines per patient3.1 ± 1.42.5 ± 1.7
* One case may exhibit more than one seriousness criteria. ** According to 2015 the European Society of Cardiology (ESC) guidelines for the diagnosis and management of pericardial diseases [6], Caforio et al. [7] and Butany et al. [8]. § In one case, the concomitant use of glipizide and indomethacin, and colchicine and indomethacin. # In one case, concomitant use of amoxicillin and tetracycline, hydrochlorothiazide and cotrimoxazole, and bromazepam and heparin. Flu vaccine types (VAERS): FLU 3 (trivalent injected): 69 patients (36.1%); FLUX SEASONAL (influenza virus vaccine, no brand name): 62 (32.5%); FLU 4 (quadrivalent injected): 20 (10.5%); FLUN 3 (trivalent intranasal spray): 19 (9.9%); FLUN 4 (quadrivalent intranasal spray): 7 (3.7%); FLU H1N1 (monovalent injected): 5 (2.6%); FLUA 3 (trivalent adjuvant injected): 3 (1.6%); FLUN H1N1 (monovalent intranasal spray): 2 (1.1%); FLUX H1N1 (monovalent unknown manufacturer): 2 (1.1%); FLUC 4 (quadrivalent cell-culture-derived injected): 2 (1.1%); FLUC 3 (trivalent cell-culture-derived injected): 1 (0.5%). One patient received both FLUX SEASONAL and FLUX (H1N1).
Table
Table 2. Case-by-case assessment of reports concerning myocarditis or pericarditis in which the influenza vaccine and immune checkpoint inhibitors (ICIs) were concomitantly used.
Table 2. Case-by-case assessment of reports concerning myocarditis or pericarditis in which the influenza vaccine and immune checkpoint inhibitors (ICIs) were concomitantly used.
Case IDDrugs/RoleDoseYearAge/SexReporter CountryReporter QualificationReactionsSeriousnessOutcomeConcomitant MedicationsComorbiditiesManagementCausality AssessmentAdapted DIPS Score
#1Nivolumab (suspect)
Influenza vaccine(suspect)		3 mg/kg every 2 weeks IV-201870/FJapanPhysicianMyocarditisSerious
(life-threatening; prolonged hospitalization; other outcomes)		RecoveredUrsodeoxycholic acid 200 mg/day
Bezafibrate
200 mg/day
Calcitriol
0.5 mcg/day		NSCLC
Liver disorder
Hyperlipidemia
Osteoporosis		Nivolumab withdrawnUnclassifiable (data on latency and time window of increased risk are lacking) Synergistic effect between influenza vaccine and ICIs cannot be excluded1
Doubtful
#2Pembrolizumab
(suspect)
Influenza vaccine (suspect)		NA
IV
0.5 mL
IM		201867/MUSPharmacistMyocarditis,
stress, cardiomyopathy,
weight decreased,
headache,
cardiac failure, congestive,
cerebral infarction,
confusional state, and
dyspnoea		Serious
(life-threatening; prolonged hospitalization)		NANALung neoplasm malignantNAUnclassifiable
(data on latency and time window of increased risk are lacking)
Synergistic effect between influenza vaccine and ICIs cannot be excluded		1
Doubtful
#3Ipilimumab
(suspect)
Nivolumab
(suspect)
Influenza vaccine (concomitant)		80 mg every 3 weeks IV
240 mg every 3 weeks IV-		201977/MCanadaPhysicianMyocarditis,
pulmonary hypertension,
dyspnoea chest discomfort asthenia, troponin increased c-reactive protein, increased diastolic dysfunction, oedema, peripheral urticaria, and pruritus		Serious
(prolonged hospitalization; other outcomes)		NANAMetastatic renal cell carcinomaIpilimumab and nivolumab withdrawnUnclassifiable
(data on latency and time window of increased risk are lacking)
Synergistic effect between influenza vaccine and ICIs cannot be excluded		2
Possible
US—United States of America; NSCLC—non-small cell lung cancer; NA—not available; IV—intravenous; IM—intramuscular; DIPS—drug-interaction probability scale.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Gatti, M.; Raschi, E.; Moretti, U.; Ardizzoni, A.; Poluzzi, E.; Diemberger, I. Influenza Vaccination and Myo-Pericarditis in Patients Receiving Immune Checkpoint Inhibitors: Investigating the Likelihood of Interaction through the Vaccine Adverse Event Reporting System and VigiBase. Vaccines 2021, 9, 19. https://doi.org/10.3390/vaccines9010019

AMA Style

Gatti M, Raschi E, Moretti U, Ardizzoni A, Poluzzi E, Diemberger I. Influenza Vaccination and Myo-Pericarditis in Patients Receiving Immune Checkpoint Inhibitors: Investigating the Likelihood of Interaction through the Vaccine Adverse Event Reporting System and VigiBase. Vaccines. 2021; 9(1):19. https://doi.org/10.3390/vaccines9010019

Chicago/Turabian Style

Gatti, Milo, Emanuel Raschi, Ugo Moretti, Andrea Ardizzoni, Elisabetta Poluzzi, and Igor Diemberger. 2021. "Influenza Vaccination and Myo-Pericarditis in Patients Receiving Immune Checkpoint Inhibitors: Investigating the Likelihood of Interaction through the Vaccine Adverse Event Reporting System and VigiBase" Vaccines 9, no. 1: 19. https://doi.org/10.3390/vaccines9010019

APA Style

Gatti, M., Raschi, E., Moretti, U., Ardizzoni, A., Poluzzi, E., & Diemberger, I. (2021). Influenza Vaccination and Myo-Pericarditis in Patients Receiving Immune Checkpoint Inhibitors: Investigating the Likelihood of Interaction through the Vaccine Adverse Event Reporting System and VigiBase. Vaccines, 9(1), 19. https://doi.org/10.3390/vaccines9010019

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop