Next Article in Journal
Investigation of Neurological Complications after COVID-19 Vaccination: Report of the Clinical Scenarios and Review of the Literature
Next Article in Special Issue
First Year of Special Issue “New Insights in Vaccination and Public Health”: Opinions and Considerations
Previous Article in Journal
Assessment of Acceptability and Determinants of Uptake and Schedule Completion of Human Papillomavirus (HPV) Vaccine by 25 to 45 Years Old Women in Slovenia
Previous Article in Special Issue
Prevalence and Predictors of Seasonal Influenza Vaccine Uptake in Saudi Arabia Post COVID-19: A Web-Based Online Cross-Sectional Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

The Use of Adaptive Sampling to Reach Disadvantaged Populations for Immunization Programs and Assessments: A Systematic Review

Global Immunization Division, Centers for Disease Control and Prevention (CDC), Atlanta, GA 30329, USA
*
Author to whom correspondence should be addressed.
Vaccines 2023, 11(2), 424; https://doi.org/10.3390/vaccines11020424
Submission received: 21 November 2022 / Revised: 25 January 2023 / Accepted: 2 February 2023 / Published: 13 February 2023
(This article belongs to the Special Issue New Insight in Vaccination and Public Health)

Abstract

:
Vaccines prevent 4–5 million deaths every year, but inequities in vaccine coverage persist among key disadvantaged subpopulations. Under-immunized subpopulations (e.g., migrants, slum residents) may be consistently missed with conventional methods for estimating immunization coverage and assessing vaccination barriers. Adaptive sampling, such as respondent-driven sampling, may offer useful strategies for identifying and collecting data from these subpopulations that are often “hidden” or hard-to-reach. However, use of these adaptive sampling approaches in the field of global immunization has not been systematically documented. We searched PubMed, Scopus, and Embase databases to identify eligible studies published through November 2020 that used an adaptive sampling method to collect immunization-related data. From the eligible studies, we extracted relevant data on their objectives, setting and target population, and sampling methods. We categorized sampling methods and assessed their frequencies. Twenty-three studies met the inclusion criteria out of the 3069 articles screened for eligibility. Peer-driven sampling was the most frequently used adaptive sampling method (57%), followed by geospatial sampling (30%), venue-based sampling (17%), ethnographic mapping (9%), and compact segment sampling (9%). Sixty-one percent of studies were conducted in upper-middle-income or high-income countries. Data on immunization uptake were collected in 65% of studies, and data on knowledge and attitudes about immunizations were collected in 57% of studies. We found limited use of adaptive sampling methods in measuring immunization coverage and understanding determinants of vaccination uptake. The current under-utilization of adaptive sampling approaches leaves much room for improvement in how immunization programs calibrate their strategies to reach “hidden” subpopulations.

1. Introduction

Mortality and morbidity attributable to vaccine-preventable diseases (VPDs) have substantially declined over the past decades due to improvements in vaccination coverage globally, particularly in low- and middle-income countries (LMICs) [1,2]. However, 14 million children missed out on life-saving vaccines in 2019 alone [3]—pointing to inequities in vaccine coverage across and within countries. Some sub-populations may be consistently missed from immunization services due to structural inequities such as barriers to health care access (e.g., refugees), transient movement (e.g., nomads) or socioeconomic reasons (e.g., residents in slum settlements). Moreover, these sub-populations are often marginalized and stigmatized in the larger society, which then perpetuates their mistrust in authorities. Among refugee and immigrant communities, for example, mistrust of the healthcare system due to fears of deportation and perceived differential treatment negatively impact health-seeking behaviors [4]. The historical mistrust compounded over time with other geopolitical dynamics may lead to inequitable access and use of services by these sub-populations. In the context of immunization, “hidden” sub-populations and other hard-to-reach groups may not only be at high risk of being under-immunized, but also most likely to be missed from conventional methods for estimating immunization coverage that rely on official records (e.g., census) or household surveys as the basis for sampling frames [5,6].
Missing hidden populations may lead to overestimation of survey-based estimates of population-level immunization coverage, which may partly explain the continued occurrence of sizable outbreaks of VPDs in countries that purportedly have high childhood immunization coverage based on results from coverage surveys [7]. In Nigeria, targeted outreach activities aimed at nomadic and migratory populations in 2012 found that 52% of enumerated settlements in these populations had been historically excluded from microplans used to implement past polio supplementary immunization services such as mass immunization campaigns [8]. Isolated subpopulations that are unvaccinated can sustain transmission of VPDs even when surrounded by high population-level immunity in the general population [9]. Strategies to effectively reach zero-dose children (i.e., those who have never been vaccinated) and those with incomplete, delayed, or invalid vaccination are a key priority for the “last mile” of polio eradication and measles elimination efforts [9,10]. Disease eradication and elimination efforts are consequently threatened when certain subpopulations are consistently missed from vaccination services and left susceptible to VPDs and adverse health outcomes.
In 2018, the World Health Organization (WHO) updated their guidance on sampling strategies for assessing vaccination coverage in an effort to improve the rigor of conventional sampling methods [11]. Among other changes, the updated guidance suggests using probability sampling methods using census data and recommends using lists of enumeration areas to establish sampling frames. Although this guidance will undoubtedly improve the precision of survey results in comparison to non-probability sampling, the updated methods do not address potential selection bias due to the systematic underrepresentation of disadvantaged populations. Adaptive sampling methods, defined here as methods that modify or expand on conventional survey methods, may offer an alternative strategy for more effectively reaching population sub-groups. Adaptive sampling methods such as peer-driven sampling, for example, start with initial participants who are then tasked with recruiting their peers in a chain-referral method. Adaptive sampling methods have been used to estimate the size of hidden populations, understand their uptake of services, and inform strategies for delivering services to hard-to-reach [12] and “hidden” populations for HIV and sexually transmitted infection (STI) prevention [13,14]. Key subpopulations (e.g., sex workers, people who inject drugs, men who have sex with men) are at disproportionately greater risk for acquiring and transmitting HIV and other STIs than the general population [15]. Many individuals at high risk for HIV infection are socially marginalized due to their engagement in behaviors that are deemed illegal or illicit, and creating a sampling frame that sufficiently represents these groups using conventional approaches is infeasible or cost-prohibitive [13,14]. Despite the fact that these populations each represent a small proportion of the overall population size of any one particular country, they are important to target for population-based surveys [14,16,17].
Similarly to HIV and STIs, key subpopulations are at disproportionately greater risk of being under-immunized. Beyond implications for accurately estimating vaccination coverage, disadvantaged populations may also be missed in assessments examining the behavioral and social drivers of under-vaccination [18]. The duality of systemically missing sub-populations in immunization coverage surveys and socio-behavioral assessments may pose a serious threat to vaccine equity. Ensuring that the right to health is distributed equitably requires that “vaccines must be delivered to areas that are isolated geographically, culturally, socially or otherwise and to marginalized populations such as displaced people and migrants and those affected by conflict, political instability and natural disasters” (Immunization Agenda 2030) [19]. Successfully delivering immunizations to these groups, however, first requires a commitment to ensuring they are represented in assessments of immunization coverage and barriers to vaccine uptake. While the benefits of adaptive strategies have been documented in some public health fields, there is a lack of understanding regarding the potential utility of adaptive sampling approaches to reach hard-to-reach and hard-to-vaccinate subpopulations for immunization programs and assessments [12].
The SARS-CoV-2 pandemic is estimated to have put an additional 80 million children at increased risk of vaccine-preventable diseases due to disruptions in routine immunization services and delays in seeking services [3]. Despite the disruptions and delays, the pandemic has also illustrated the power of safe and effective immunizations in saving lives and protecting the human right to health [19]. Adaptive sampling approaches may be an important and necessary tool in the unprecedented global vaccination campaign to reach all individuals with COVID-19 vaccines. Here, we aim to review how adaptive sampling approaches have been used to estimate immunization coverage and/or assess the underlying behavioral and social drivers of vaccination uptake.

2. Materials and Methods

2.1. Search Strategy

We searched Scopus, PsychInfo, Medline, Embase, Cochrane Library, and CINAHL to identify relevant studies regardless of published language and published through November 2020. Search terms included keywords related to immunization (e.g., “vaccin*”, “immuni*”) and keywords associated with adaptive sampling. Adaptive sampling keywords consisted of terms used to describe common adaptive sampling methods (e.g., “respondent-driven”, “geospatial”) and terms related to study populations typically targeted by adaptive sampling (e.g., “transient”, “hard-to-reach”) (see Appendix A for a full list of search terms). The search terms were identified and refined based on a review of key references and discussion among co-authors, with assistance from a reference librarian at the US Centers for Disease Control and Prevention (US CDC). We uploaded search results to covidence (www.covidence.org), which we used to manage the subsequent steps of the review.

2.2. Study Selection

For both stages of study selection (abstract review, full text review), two independent reviewers (A.K and A.I) screened each article for eligibility. All discrepancies between reviewers were resolved by discussion and consensus.
Studies were included if they met the following inclusion criteria: (i) described research on any vaccines (defined as substances used to stimulate the production of antibodies and provide immunity against one or several diseases) or immunization programs; (ii) described the use of adaptive sampling approaches to identify individuals for service delivery or participation in primary data collection; adaptive sampling was defined as substantive modifications to or expansions of conventional survey methods (with probability or non-probability sampling) to reach population sub-groups that would otherwise have a high likelihood of being missed; (iii) conducted among humans; (iv) published in English, French, or Spanish.
Studies were excluded if they met any of the following criteria: (i) abstract or full text not available; (ii) duplicative studies; (iii) described immunologic surveys without assessing immunization coverage; (iv) not published in a peer-reviewed journal (e.g., dissertations, conference proceedings). Studies that used adaptive methods (e.g., geospatial mapping) solely to improve immunization systems without conducting any sampling or primary data collection from respondents [20,21] were excluded.

2.3. Data Extraction

We extracted data from the full text of all included studies using a data extraction form developed by the authors. Data extraction for all included studies was conducted independently by two reviewers (A.K and A.I), and any discrepancies in data extracted were resolved by discussion and consensus.
We extracted data on population type, eligibility criteria for the target population, data collection type (i.e., quantitative, qualitative, or both), vaccine-preventable disease of interest, sampling methods, study objectives, and whether immunization services were delivered as part of the study. We categorized studies that collected data on vaccines generally without specifying a vaccine-preventable disease as collecting data related to all immunizations generally. We categorized sampling methods into one of the following groups: peer-driven sampling (e.g., respondent-driven sampling, snowball sampling, etc.), geospatial sampling (e.g., using satellite imagery to identify housing structures and/or populations), compact segment sampling [22], ethnographic mapping [6], and venue-based sampling (e.g., time–location sampling). Categories for sampling methods were developed iteratively based on the data extracted during the literature review.
Studies that utilized peer-driven sampling were further categorized as being partially or fully compliant with respondent-driven sampling approaches. Studies were categorized as partially compliant with respondent-driven sampling approaches if they mentioned the term “respondent-driven sampling” and at least one of the following methodological attributes: (i) use of coupons or referral management systems; (ii) use of seeds; and/or (iii) use of specialized analytic software for respondent-driven sampling (e.g., Respondent Driven Sampling Analyst (RDSA)). Respondent-driven sampling is a specialized form of peer-driven sampling that uses a formal mechanism for tracking referrals and adjusting for network size in data analysis to calculate analytic weights that qualify the method as a probability sampling method (as opposed to non-probability sampling). Studies that utilized peer-driven sampling were, therefore, only categorized as fully compliant with respondent-driven sampling approaches if all of the above methodological attributes were included in the manuscript.
We extracted the study objectives listed in each manuscript and utilized inductive thematic analysis [23] to allow the extracted data to guide our selection of major themes. We then categorized study objectives based on six major themes: measuring coverage; measuring attitudes, behaviors and vaccine uptake barriers; delivering vaccines; enhancing microplanning; health systems improvement; and comparing sampling methods. In addition, we generated three separate binary categorical variables (0 = no; 1 = yes) to indicate whether a study collected and presented data on (i) knowledge and attitudes about immunizations (knowledge on availability of services, perceived need for immunizations, etc.); (ii) barriers to vaccination uptake or reasons for under-vaccination; and/or (iii) vaccination uptake. For this initial systematic review, we noted only whether the data were present or absent in the manuscript and did not extract specific data presented in each manuscript. Studies were categorized as having collected data on immunization uptake if they collected data on immunization status based on self-report, immunization cards, or serological testing. We created a categorical variable to classify studies as having collected immunization-related data from adults about themselves, from mothers/caregivers about their children, or both. For studies using quantitative data collection methods, we created additional categorical variables indicative of whether the study examined predictors of immunization knowledge/attitudes and/or uptake.

2.4. Data Analysis

We performed descriptive analyses to examine the variability in study characteristics, focusing on the following areas: type of adaptive sampling method; reason for using adaptive sampling; type of study population; antigen or vaccine-preventable disease of focus; type of data collection method (quantitative or qualitative); type of data presented. We categorized countries by income level using the 2021 World Bank classifications [24]. Statistical analyses were conducted in Stata 16 (College Station, TX, USA).

3. Results

After the exclusion of duplicates, our search identified 3069 studies for abstract screening, of which 132 met the criteria for full-text review. We identified 23 studies [25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47] that satisfied the inclusion criteria after full-text review (Figure 1).
The 23 studies were conducted in 19 countries from 1987 to 2017 (Table 1).
Studies included a mix of qualitative (30%; N = 7) and quantitative (70%; N = 16) data collection methods and no mixed-methods studies were identified. We identified more studies in upper-middle-income or high-income countries (61%; N = 14) compared to lower-middle-income or low-income countries (39%; N = 9) [15].
A majority of studies identified in this review (65%) did not focus on key disadvantaged subpopulations that may be more likely to be missed in conventional sampling approaches, such as sex workers, migratory populations, refugees, persons living in high-density urban slums, or undocumented immigrants. Adaptive sampling methods were used to collect data from a variety of target populations including the general population (26%), mothers and caregivers of children (39%), healthcare providers (17%), sex workers (13%), and other groups (35%) such as refugees, migrant populations, immigrants/ethnic minorities, internally displaced persons (IDPs), prisoners, and men who have sex with men. Immunization-related data were most often collected from adults about themselves (35%; N = 8), from mothers and caregivers about their children (35%; N = 8), or both (13%; N = 3). Among all antigens, studies most frequently collected immunization-related data on hepatitis B specifically (26%) and all immunizations generally (26%). Stratified by income, studies conducted in lower-middle-income or low-income countries most frequently collected data on cholera (33%) or measles (33%), while studies in upper-middle-income or high-income countries most frequently collected data on hepatitis B (43%).

3.1. Sampling Methods

Peer-driven sampling was the most frequently used adaptive sampling method (N = 13; 57%), followed by geospatial sampling (30%), venue-based sampling (17%), ethnographic mapping (9%), and compact-segment sampling (9%). Five studies utilized multiple sampling methods [26,29,44,46,47]. Four studies compared geospatial sampling or compact-segment sampling methods to conventional approaches on metrics such as population size (i.e., number of households identified), immunization coverage estimates, cost, time, amount of advanced preparation and expertise needed, design effect, rate of homogeneity, and sociodemographic characteristics of identified populations [36,42,45,46]. The advantages and disadvantages of adaptive strategies varied. For example, Milligan et al. and Gong et al. found no meaningful differences in vaccination coverage between compact-segment sampling methods and/or GIS sampling and conventional EPI sampling methods [45,46]. In contrast, Barau et al. identified over 3000 settlements that were previously not included in microplans for polio vaccination campaigns in Nigeria when using satellite imagery to identify households compared to hand-drawn maps [36].
Only three studies used adaptive sampling methods to deliver immunization services. Puga et al. and Magalhaes et al. used respondent-driven sampling methods to deliver hepatitis B vaccines to female sex workers in Brazil by recruiting an initial sample of sex workers from prostitution environments (e.g., night clubs, brothels) and incentivizing them to recruit their peers [31,39]. Barau et al. utilized geospatial sampling to provide polio vaccines to children in Nigeria by using high-resolution satellite imagery to identify settlements for vaccination teams to visit [36]. Two studies utilized adaptive sampling as part of outbreak prevention and response activities. Specifically, geospatial sampling was used to identify households in response to cholera outbreaks in South Sudan [33] and Zambia [43].
Among 13 of 23 studies that used peer-driven sampling [25,28,30,31,32,34,35,37,39,40,41,44,47], only 2 studies were partially compliant with accepted respondent-driven sampling methods [31,35], and one study was fully compliant with respondent-driven sampling methods [39]. All three studies that were partially or fully compliant with respondent-driven sampling methods were focused on hepatitis B and collected data or delivered services to female sex workers [31,39] or other adults aged 16 and above [35]. Studies that used peer-driven sampling but were not compliant with respondent-driven sampling methods primarily utilized snowball sampling methods in which study participants referred their peers, but no formal mechanism of tracking referrals or adjusting for network size in data analysis was used.
Geospatial methods included random spatial sampling to select data collection starting points [33,38,42,43,47] for the use of satellite imagery to identify potential residential structures and conduct grid-based sampling [46] and the use of geographic information systems (GIS) to generate maps for data enumerators and use GPS tracking to follow enumerators [36]. Among studies that used venue-based sampling (4 of 23), two studies sampled only based on location [27,44], and two studies utilized sampling based on both time and location [26,29]. Coreil et al. and Gullion et al. utilized venue-based sampling to collect data on routine childhood immunizations from mothers/caregivers identified at Mother’s clubs, gatherings of roadside sellers and neighbors, at rally posts delivering child health services in Haiti [27] and other venues including but not limited to schools, bookstores, chiropractors, health food stores, holistic health centers, and yoga classes in the United States [44]. Baars et al. utilized time–location sampling at bars and brothels to collect data and deliver hepatitis B vaccination services to female sex workers and men who have sex with men [26,29].

3.2. Types of Data Collected

Data on immunization uptake were collected in 65% of studies (N = 15) [26,29,31,32,33,34,35,38,39,40,42,43,45,46,47], while data on knowledge and attitudes about immunizations were collected in 57% of studies (N = 13) [25,26,27,28,29,30,31,32,34,38,40,43,44]. Among studies that collected data on immunization uptake (N = 15), 27% aimed to assess coverage following antigen-specific supplementary immunization campaigns (SIAs) for cholera or meningitis [33,38,42,43]. Fifty-two percent of studies specifically collected data on reasons for non-vaccination or barriers to vaccination (N = 12) [25,26,27,29,32,33,34,38,40,43,44,47]. Among studies that collected quantitative data (16 of 23), six studies analyzed predictors of immunization-related knowledge and attitudes (38%) and nine studies analyzed predictors of immunization uptake (56%).

4. Discussion

We identified 23 studies in the published literature that utilized adaptive sampling strategies to collect quantitative and qualitative data for immunization coverage assessments, microplanning for vaccine delivery, VPD outbreak response, and behavioral assessments of barriers to vaccine uptake. Adaptive sampling strategies were more frequently used in high-income settings, and few studies collected data from populations known to be hard-to-reach. Among the 13 studies that utilized peer-driven approaches, only one study incorporated methodological attributes that provided a probability sample. This review shows that the current use of adaptive sampling methods in the immunization literature is limited, and opportunities exist to improve immunization programs by re-orienting the use of adaptive sampling approaches to find and learn about hidden populations that are disproportionately affected by VPDs.
The use of adaptive sampling in other health sectors can act as a roadmap for the potential utility of these approaches in supplementing conventional immunization surveys. Population-based approaches alone were not sufficient to control the HIV/AIDS epidemic, and over time, the need for interventions focused on marginalized high-risk groups became apparent. The WHO Biobehavioral survey guidelines for populations at risk for HIV aims to address these challenges by providing a menu of sampling strategies including conventional and adaptive approaches along with decision-making criteria to select sampling methods [48]. Factors considered in the selection of sampling methods include whether there are existing sampling frames, the size of the target population, whether the population is socially networked, and whether the population is accessible at physical locations [48]. Similarly, for immunizations, efforts for improving coverage have historically utilized a population-based approach in defined geographic areas rather than focusing on subpopulations at high risk of under-immunization. Despite global progress in expanding immunization coverage, gains in coverage have stalled short of targets in many countries due to under-vaccinated subpopulations [12,49,50]. Immunization Agenda 2030 outlines a shift in strategy to prioritize global health equity and reach zero-dose children, which will likely necessitate adaptive sampling methods highlighted in this review.
Adaptive approaches warrant consideration for supplementing conventional approaches, particularly in settings where key differences in uptake, knowledge, attitudes, and behaviors may exist between those that would be included and excluded by conventional methods as well as in settings in which in-depth behavioral insights are needed to address barriers to vaccination [51]. While for HIV/STIs, adaptive strategies are used to sample adults in high-risk groups, our findings illustrate the feasibility of using adaptive strategies for sampling mothers and caregivers in order to reach immunization-eligible children. Our findings also demonstrate that the added value of adaptive approaches relative to conventional sampling approaches is context-specific. The WHO Behavioral and Social Drivers of Vaccination (BeSD) guidebook includes some guidance on developing a sampling plan and the advantages and disadvantages of adaptive sampling strategies such as snowball sampling when trying to understand reasons for low vaccine uptake [52]. Additional studies are needed to help researchers and immunization program mangers assess whether adaptive sampling strategies are needed in their context. Even among the 13 studies identified in this review that utilized peer-driven sampling, there was methodological heterogeneity, and few studies were compliant with accepted respondent-driven sampling methods. Standardized guidance on how to use adaptive methodologies to achieve various goals of immunization programs (e.g., coverage surveys) will be critical for achieving the global health equity agenda.
The COVID-19 pandemic underscores how increasing global coverage of SARS-CoV-2 vaccines is not only an ethical obligation but also critical for preventing the emergence and spread of harmful SARS-CoV-2 variants in the global landscape [53]. Adaptive sampling strategies are among the many tools that can be used to ensure equitable immunization coverage for vaccines against SARS-CoV-2 as well as routine immunizations. The feasibility of adaptive sampling approaches for reaching a variety of disadvantaged populations has been documented in many settings [36,46,54,55,56]. While adaptive strategies are cost-effective in some settings, cost-effectiveness can vary by specific sampling strategy, target population, and setting. Additional research is needed to evaluate the utility and feasibility of these approaches in strengthening routine immunization systems as well as VPD outbreak response activities.
This review has several limitations. We did not conduct any risk of bias assessments and did not exclude studies on the basis of whether adaptive sampling methods were used appropriately. We likely underestimated the number of qualitative studies related to immunizations that have utilized adaptive sampling methodologies such as snowball sampling, given that these methods are more common in qualitative research and may, therefore, have been excluded from the Methods sections of published articles.

5. Conclusions

Our systematic review revealed that there is a paucity of studies in the literature that have used adaptive sampling approaches to collect data related to immunizations. Although the studies we identified used adaptive sampling strategies for various purposes, many of them were limited to high-income settings or did not have a specific focus on disadvantaged populations. Taken together, our findings highlight the overarching need to more proactively explore the potential effectiveness and practicability of applying adaptive sampling strategies in ensuring equitable immunization coverage. The current under-utilization of adaptive sampling approaches leaves much room for improvement in how immunization programs calibrate their strategies to reach disadvantaged populations that may be recurrently missed in immunization assessments and service delivery.

Author Contributions

Conceptualization, D.P.; methodology, D.P., A.K. and A.I.; formal analysis, A.K. and A.I.; investigation, A.K. and A.I.; data curation, A.K. and A.I.; writing—original draft preparation, A.K. and A.I.; writing—review and editing, A.K., A.I., D.D., M.F.J., A.S.W. and D.P.; supervision, D.P. and D.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors would like to thank Joanna Taliano, Shibani Kulkarni, and Dale Rhoda for their contributions to the conceptualization and implementation of this systematic review. A.I. is currently a staff member at the World Health Organization.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A

Literature search terms:
  Five journal databases (PubMed, EMBASE, Medline, PsycInfo, CINAHL) were searched based on the following search terms:
  Sampling stud* OR (Adaptive ADJ2 sampling) OR (modif* ADJ2 sampling) OR (Snowball ADJ2 sampling) OR (Respondent driven ADJ2 sampling) OR (Indigenous field worker ADJ2 sampling) OR (Targeted ADJ2 sampling) OR (Time location ADJ2 sampling) OR (Capture recapture ADJ2 sampling) OR Rapid assessment response OR (Spatial ADJ2 sampling) OR (Geospatial ADJ2 sampling) OR Satellite imagery OR (compact segment ADJ2 sampling) OR grid sampling OR sampling grid OR (second stage ADJ2 sampling) OR (two cluster ADJ2 sampling)
AND
  Vaccin* OR immuni* OR virus* OR disease*
AND
  subpopulation* OR sub-population* OR vulnerable* OR homeless* OR marginalized OR under-represented OR hard-to-reach OR hidden population* OR at-risk OR high risk OR displaced OR migrant* OR migrating OR nomad* OR transient* OR settlement* OR refugee* OR inaccessible OR infant* OR children OR preschool* OR pre-school* OR target population* OR rural population* OR rural area*

References

  1. Rodrigues, C.M.C.; Plotkin, S.A. Impact of Vaccines; Health, Economic and Social Perspectives. Front. Microbiol. 2020, 11, 1526. [Google Scholar] [CrossRef] [PubMed]
  2. Li, X.; Mukandavire, C.; Cucunubá, Z.M.; Londono, S.E.; Abbas, K.; Clapham, H.E.; Jit, M.; Johnson, H.L.; Papadopoulos, T.; Vynnycky, E.; et al. Estimating the health impact of vaccination against ten pathogens in 98 low-income and middle-income countries from 2000 to 2030: A modelling study. Lancet 2021, 397, 398–408. [Google Scholar] [CrossRef]
  3. Immunization Coverage: Are We Losing Ground? [Internet]. UNICEF DATA. 2020. Available online: https://data.unicef.org/resources/immunization-coverage-are-we-losing-ground/ (accessed on 4 January 2021).
  4. Hacker, K.; Anies, M.; Folb, B.L.; Zallman, L. Barriers to health care for undocumented immigrants: A literature review. Risk Manag Health Policy 2015, 8, 175–183. [Google Scholar] [CrossRef] [PubMed]
  5. Cutts, F.T.; Claquin, P.; Danovaro-Holliday, M.C.; Rhoda, D.A. Monitoring vaccination coverage: Defining the role of surveys. Vaccine 2016, 34, 4103–4109. [Google Scholar] [CrossRef] [PubMed]
  6. Watters, J.K.; Biernacki, P. Targeted Sampling: Options for the Study of Hidden Populations. Soc. Probl. 1989, 36, 416–430. [Google Scholar] [CrossRef]
  7. Zucker, J.R.; Rosen, J.B.; Iwamoto, M.; Arciuolo, R.J.; Langdon-Embry, M.; Vora, N.M.; Rakeman, J.L.; Isaac, B.M.; Jean, A.; Asfaw, M.; et al. Consequences of Undervaccination—Measles Outbreak, New York City, 2018–2019. N. Engl. J. Med. 2020, 382, 1009–1017. [Google Scholar] [CrossRef]
  8. Gidado, S.O.; Ohuabunwo, C.; Nguku, P.M.; Ogbuanu, I.U.; Waziri, N.E.; Biya, O.; Wiesen, E.S.; Mba-Jonas, A.; Vertefeuille, J.; Oyemakinde, A.; et al. Outreach to underserved communities in northern Nigeria, 2012–2013. J. Infect. Dis. 2014, 210 (Suppl. 1), S118–S124. [Google Scholar] [CrossRef]
  9. Thompson, K.M.; Duintjer Tebbens, R.J. Lessons From the Polio Endgame: Overcoming the Failure to Vaccinate and the Role of Subpopulations in Maintaining Transmission. J. Infect. Dis 2017, 216, S176–S182. [Google Scholar] [CrossRef]
  10. Chard, A.N.; Gacic-Dobo, M.; Diallo, M.S.; Sodha, S.V.; Wallace, A.S. Routine Vaccination Coverage—Worldwide, 2019. MMWR Morb. Mortal. Wkly. Rep. 2020, 69, 1706–1710. [Google Scholar] [CrossRef]
  11. Vaccination Coverage Cluster Surveys: Reference Manual; Report No.: (WHO/IVB/18.09); World Health Organization: Geneva, Switzerland, 2018.
  12. Ozawa, S.; Yemeke, T.T.; Evans, D.R.; Pallas, S.E.; Wallace, A.S.; Lee, B.Y. Defining hard-to-reach populations for vaccination. Vaccine 2019, 37, 5525–5534. [Google Scholar] [CrossRef]
  13. Thompson, S.K.; Collins, L.M. Adaptive sampling in research on risk-related behaviors. Drug Alcohol Depend. 2002, 68 (Suppl. 1), S57–S67. [Google Scholar] [CrossRef]
  14. Magnani, R.; Sabin, K.; Saidel, T.; Heckathorn, D. Review of sampling hard-to-reach and hidden populations for HIV surveillance. AIDS 2005, 19, S67. [Google Scholar] [CrossRef] [PubMed]
  15. UNAIDS. Seizing the Moment: Tackling Entrenched Inequalities to End Epidemics. Global AIDS Update; UNAIDS: Geneva, Switzerland, 2020. [Google Scholar]
  16. Abdul-Quader, A.S.; Baughman, A.L.; Hladik, W. Estimating the size of key populations: Current status and future possibilities. Curr. Opin. HIV AIDS 2014, 9, 107–114. [Google Scholar] [CrossRef] [PubMed]
  17. Mills, S.; Saidel, T.; Magnani, R.; Brown, T. Surveillance and modelling of HIV, STI, and risk behaviours in concentrated HIV epidemics. Sex. Transm. Infect. 2004, 80, ii57–ii62. [Google Scholar] [CrossRef] [PubMed]
  18. World Health Organization. Weekly Epidemiological Record. Report No.: No 20, 2022, 97. May 2022; pp. 209–224. Available online: https://apps.who.int/iris/bitstream/handle/10665/354458/WER9720-eng-fre.pdf (accessed on 2 July 2022).
  19. World Health Organization. Immunization Agenda 2030: A Global Strategy to Leave No One Behind. April 2020. Available online: https://cdn.who.int/media/docs/default-source/immunization/strategy/ia2030/ia2030-draft-4-wha_b8850379-1fce-4847-bfd1-5d2c9d9e32f8.pdf?sfvrsn=5389656e_66&download=true (accessed on 14 December 2022).
  20. Higgins, J.; Adamu, U.; Adewara, K.; Aladeshawe, A.; Aregay, A.; Barau, I.; Berens, A.; Bolu, O.; Dutton, N.; Iduma, N.; et al. Finding inhabited settlements and tracking vaccination progress: The application of satellite imagery analysis to guide the immunization response to confirmation of previously-undetected, ongoing endemic wild poliovirus transmission in Borno State, Nigeria. Int. J. Health Geogr. 2019, 18, 11. [Google Scholar] [CrossRef]
  21. Rosencrans, L.C.; Sume, G.E.; Kouontchou, J.-C.; Voorman, A.; Anokwa, Y.; Fezeu, M.; Seaman, V.Y. Mapping for Health in Cameroon: Polio Legacy and Beyond. J. Infect. Dis. 2017, 216, S337–S342. [Google Scholar] [CrossRef] [PubMed]
  22. Turner, A.G.; Magnani, R.J.; Shuaib, M. A Not Quite as Quick but Much Cleaner Alternative to the Expanded Programme on Immunization (EPI) Cluster Survey Design. Int J. Epidemiol 1996, 25, 198–203. [Google Scholar] [CrossRef]
  23. Nowell, L.S.; Norris, J.M.; White, D.E.; Moules, N.J. Thematic Analysis: Striving to Meet the Trustworthiness Criteria. Int. J. Qual. Methods 2017, 16, 1609406917733847. [Google Scholar] [CrossRef]
  24. The World Bank. World Bank Country and Lending Groups. 2021. Available online: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups (accessed on 10 November 2022).
  25. Ben Natan, M.; El Kravchenko, B.; Sakashidlo, K.; Mor, S. What drives pregnant women’s decisions to accept the pertussis vaccine? Appl. Nurs. Res. 2017, 38, 60–63. [Google Scholar] [CrossRef]
  26. Baars, J.E.; Boon, B.J.; Garretsen, H.F.; van de Mheen, D. Vaccination uptake and awareness of a free hepatitis B vaccination program among female commercial sex workers. Womens Health Issues 2009, 19, 61–69. [Google Scholar] [CrossRef]
  27. Coreil, J.; Augustin, A.; Holt, E.; Halsey, N.A. Use of ethnographic research for instrument development in a case-control study of immunization use in Haiti. Int. J. Epidemiol. 1989, 18, S33–S37. [Google Scholar] [CrossRef] [PubMed]
  28. Russell, G.; Sutton, J.; Reid, G.J.; Beynon, C.; Cohen, I.; Huffman, D. Universal influenza immunization. Were Ontario family physicians prepared? Can. Fam. Physician 2003, 49, 1315–1321. [Google Scholar] [PubMed]
  29. Baars, J.E.; Boon, B.J.; Garretsen, H.F.; van de Mheen, D. The reach of a hepatitis B vaccination programme among men who have sex with men. Eur. J. Public Health 2011, 21, 333–337. [Google Scholar] [CrossRef]
  30. Pearce, C.; Leask, J.; Ritchie, J. Tapping midwives’ views about the neonatal hepatitis B vaccine: How welcome is a move towards a health promoting orientation? Health Promot. J. Aust. 2008, 19, 161–163. [Google Scholar] [CrossRef]
  31. Brito Magalhães, R.d.L.; Moura Carvalho, V.; Ibiapina Brito, G.M.; Braz de Oliveira, L.; Gimeniz Galvão, M.T.; Gir, E. Risk practices and immunization against hepatitis B among female sex workers. Rev. Rene 2016, 17, 636–642. [Google Scholar] [CrossRef]
  32. Jama, A.; Ali, M.; Lindstrand, A.; Butler, R.; Kulane, A. Perspectives on the Measles, Mumps and Rubella Vaccination among Somali Mothers in Stockholm. Int. J. Environ. Res. Public Health 2018, 15, 2428. [Google Scholar] [CrossRef]
  33. Parker, L.A.; Rumunu, J.; Jamet, C.; Kenyi, Y.; Lino, R.L.; Wamala, J.F.; Mpairwe, A.M.; Muller, V.; Llosa, A.E.; Uzzeni, F.; et al. Neighborhood-targeted and case-triggered use of a single dose of oral cholera vaccine in an urban setting: Feasibility and vaccine coverage. PLoS Negl. Trop. Dis. 2017, 11, e0005652. [Google Scholar] [CrossRef]
  34. Shand, L.; Burney, S.; Fletcher, J. Knowledge of cervical cancer, pap testing and the human papillomavirus among young Australian women. Health Promot. J. Aust. 2010, 21, 202–207. [Google Scholar] [CrossRef] [PubMed]
  35. Yang, Y.; Zhao, X.; Qiu, P.; Ma, X.; Chou, C.P. Integrating spatial technology into studying the generational differences of migrants’ health protection status in urban China. Int. J. Equity Health 2015, 14, 27. [Google Scholar] [CrossRef]
  36. Barau, I.; Zubairu, M.; Mwanza, M.N.; Seaman, V.Y. Improving polio vaccination coverage in Nigeria through the use of geographic information system technology. J. Infect. Dis. 2014, 210 (Suppl. 1), S102–S110. [Google Scholar] [CrossRef] [Green Version]
  37. Koehlmoos, T.P.; Uddin, J.; Sarma, H. Impact of measles eradication activities on routine immunization services and health systems in Bangladesh. J. Infect. Dis. 2011, 204 (Suppl. 1), S90–S97. [Google Scholar] [CrossRef]
  38. Massing, L.A.; Aboubakar, S.; Blake, A.; Page, A.L.; Cohuet, S.; Ngandwe, A.; Mukomena Sompwe, E.; Ramazani, R.; Allheimen, M.; Levaillant, P.; et al. Highly targeted cholera vaccination campaigns in urban setting are feasible: The experience in Kalemie, Democratic Republic of Congo. PLoS Negl. Trop. Dis. 2018, 12, e0006369. [Google Scholar] [CrossRef] [PubMed]
  39. Puga, M.A.M.; Bandeira, L.M.; Weis, S.; Fernandes, F.R.P.; Castro, L.S.; Tanaka, T.S.O.; Rezende, G.R.; Teles, S.A.; Castro, V.O.L.; Murat, P.G.; et al. High-risk behaviors for hepatitis B and C infections among female sex workers. Rev. Da Soc. Bras. De Med. Trop. 2018, 51, 198–202. [Google Scholar] [CrossRef]
  40. McDonald, P.; Limaye, R.J.; Omer, S.B.; Buttenheim, A.M.; Mohanty, S.; Klein, N.P.; Salmon, D.A. Exploring California’s new law eliminating personal belief exemptions to childhood vaccines and vaccine decision-making among homeschooling mothers in California. Vaccine 2019, 37, 742–750. [Google Scholar] [CrossRef] [PubMed]
  41. Peel, R.; Ren, S.; Hure, A.; Levi, C.R.; Sturm, J.; Durrheim, D.; McEvoy, M.; Newby, D.A.; Hansbro, P.M.; Attia, J.R.; et al. Evaluating recruitment strategies for AUSPICE, a large Australian community-based randomised controlled trial. Med. J. Aust. 2019, 210, 409–415. [Google Scholar] [CrossRef] [PubMed]
  42. Grais, R.F.; Rose, A.M.; Guthmann, J.P. Don’t spin the pen: Two alternative methods for second-stage sampling in urban cluster surveys. Emerg. Themes Epidemiol. 2007, 4, 8. [Google Scholar] [CrossRef]
  43. Ferreras, E.; Matapo, B.; Chizema-Kawesha, E.; Chewe, O.; Mzyece, H.; Blake, A.; Moonde, L.; Zulu, G.; Poncin, M.; Sinyange, N.; et al. Delayed second dose of oral cholera vaccine administered before high-risk period for cholera transmission: Cholera control strategy in Lusaka, 2016. PLoS ONE 2019, 14, e0219040. [Google Scholar] [CrossRef] [PubMed]
  44. Gullion, J.S.; Henry, L.; Gullion, G. Deciding to opt out of childhood vaccination mandates. Public Health Nurs. 2008, 25, 401–408. [Google Scholar] [CrossRef] [PubMed]
  45. Milligan, P.; Njie, A.; Bennett, S. Comparison of two cluster sampling methods for health surveys in developing countries. Int. J. Epidemiol. 2004, 33, 469–476. [Google Scholar] [CrossRef]
  46. Gong, W.; Taighoon Shah, M.; Firdous, S.; Jarrett, B.A.; Moulton, L.H.; Moss, W.J.; Hayford, K.; O’Brien, K.L.; Chandir, S. Comparison of three rapid household survey sampling methods for vaccination coverage assessment in a peri-urban setting in Pakistan. Int. J. Epidemiol. 2019, 48, 583–595. [Google Scholar] [CrossRef]
  47. Roberton, T.; Weiss, W.; Jordan Health Access Study, T.; Lebanon Health Access Study, T.; Doocy, S. Challenges in Estimating Vaccine Coverage in Refugee and Displaced Populations: Results From Household Surveys in Jordan and Lebanon. Vaccines 2017, 5, 22. [Google Scholar] [CrossRef]
  48. WHO, CDC, UNAIDS, FHI 360. Biobehavioral Survey Guidelines for Populations at Risk for HIV; World Health Organization: Geneva, Switzerland, 2017.
  49. Sodha, S.V.; Dietz, V. Strengthening routine immunization systems to improve global vaccination coverage. Br. Med. Bull. 2015, 113, 5–14. [Google Scholar] [CrossRef]
  50. Oyo-Ita, A.; Wiysonge, C.S.; Oringanje, C.; Nwachukwu, C.E.; Oduwole, O.; Meremikwu, M.M. Interventions for Improving Coverage of Childhood Immunisation in Low- and Middle-Income Countries. Cochrane Effective Practice and Organisation of Care Group, Editor. Cochrane Database of Systematic Reviews [Internet]. 10 July 2016. Available online: https://doi.wiley.com/10.1002/14651858.CD008145.pub3 (accessed on 7 January 2023).
  51. Cutts, F.T. Strategies to improve immunization services in urban Africa. Bull World Health Organ 1991, 69, 407–414. [Google Scholar] [PubMed]
  52. Behavioural and Social Drivers of Vaccination: Tools and Practical Guidance for Achieving High Uptake; World Health Organization: Geneva, Switzerland, 2022.
  53. Editorial the Guardian View on Global Vaccine Inequality: Unwise as Well as Unethical. The Guardian [Internet]. 25 August 2021. Available online: https://www.theguardian.com/commentisfree/2021/aug/25/the-guardian-view-on-global-vaccine-inequality-unwise-as-well-as-unethical (accessed on 8 October 2021).
  54. Badowski, G.; Somera, L.P.; Simsiman, B.; Lee, H.-R.; Cassel, K.; Yamanaka, A.; Ren, J. The efficacy of respondent-driven sampling for the health assessment of minority populations. Cancer Epidemiol. 2017, 50, 214–220. [Google Scholar] [CrossRef] [PubMed]
  55. Garcini, L.M.; Galvan, T.; Peña, J.M.; Chen, N.; Klonoff, E. Effectiveness of Respondent-Driven Sampling for Conducting Health Studies Among Undocumented Immigrants at a Time of Heightened Immigration Enforcement. J. Immigr. Minor. Health 2022, 24, 102–110. [Google Scholar] [CrossRef] [PubMed]
  56. Jauffret-Roustide, M.; Le Strat, Y. Design, Sampling, Organising the Field for a TLS Survey and New Developments: Coquelicot, a Survey of Drug Users. Methodol. Innov. Online 2010, 5, 26–37. [Google Scholar] [CrossRef]
Figure 1. Flow diagram for study screening and selection.
Figure 1. Flow diagram for study screening and selection.
Vaccines 11 00424 g001
Table 1. Characteristics of included studies and major themes of adaptive sampling use.
Table 1. Characteristics of included studies and major themes of adaptive sampling use.
StudyYearCountrySampling TypeTarget PopulationAge EligibilityVaccines AStudy Objectives
Milligan, P. et al. [45] 2000GMBCompact segmentMothers and caregivers of childrenMothers and caregivers of children aged 12–23 months Routine immunizations; yellow feverMeasure coverage;
Compare sampling methods
Baars, J. E. et al. [29] 2004NLDEthnographic mapping;
Venue-based
Men who have sex with menNot specifiedHepatitis BMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Baars, J. E. et al. [26] 2004NLDEthnographic mapping;
Venue-based
Sex workersNot specifiedHepatitis BMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Grais, R. F. et al. [42] 2006NERGeospatialGeneral populationNot specifiedMeningitisMeasure coverage;
Compare sampling methods
Massing, L. A. et al. [38] 2014DRCGeospatialGeneral population≥1 yearOral choleraMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Ferreras, E. et al. [43] 2016ZMBGeospatialGeneral population>1 yearOral choleraMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Parker, L. A. et al. [33]2015SSDGeospatialGeneral population;
internally displaced persons;
slum residents;
healthcare providers;
prisoners/inmates
>1 yearOral choleraMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Barau, I. et al. [36] 2012NGAGeospatialMothers and caregivers of childrenNot specifiedOral polioEnhance microplanning;
Deliver vaccines;
Compare sampling methods
Gong, W. et al. [46]2016PAKGeospatial;
Compact segment
Mothers and caregivers of childrenMothers and caregivers of children aged 12–23 months Routine immunizationsMeasure coverage;
Compare sampling methods
Shand, L. et al. [34]2008AUSPeer-drivenGeneral population18–26 yearsHuman papillomavirusMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Peel, R. et al. [41]2016AUSPeer-drivenGeneral population55–60 yearsPneumococcal polysaccharide Compare sampling methods
Pearce, C. et al. [30]2003AUSPeer-drivenHealthcare providersNot specifiedHepatitis B; All immunizations generallyMeasure attitudes, behaviors and vaccine uptake barriers
Russell, G. et al. [28] 2001CANPeer-drivenHealthcare providersNot specifiedInfluenzaMeasure attitudes, behaviors and vaccine uptake barriers
Koehlmoos, T. P. et al. [37]UnknownBGDPeer-drivenHealthcare providers; government staff; health systems staff that do not directly provide care Not specifiedAll immunizations generally; Routine immunizations Health systems improvement
Jama, A. et al. [32]2013SWEPeer-drivenMothers and caregivers of children; immigrants/ethnic minorityMothers and caregivers of children aged 18 months to 5 yearsRoutine immunizations Measure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Ben Natan, M. et al. [25]2016ISRPeer-drivenMothers and caregivers of childrenNot specifiedPertussisMeasure attitudes, behaviors and vaccine uptake barriers
McDonald, P. et al. [40]2017USAPeer-drivenMothers and caregivers of childrenMothers and caregivers of children in kindergarten (~5 years) through 12th grade (~17–18 years)All immunizations generallyMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Puga, M. A. M. et al. [39]2009BRAPeer-driven and fully compliant with RDSSex workersNot specifiedHepatitis BMeasure coverage;
Deliver vaccines;
Measure attitudes, behaviors and vaccine uptake barriers
Yang, Y. et al. [35]2008CHNPeer-driven and partially compliant with RDSMigrantsNot specifiedHepatitis BMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Magalhaes, R.L.B et al. [31]2014BRAPeer-driven and partially compliant with RDSSex workers≥18 yearsHepatitis BMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers;
Deliver services
Roberton, T. et al. [47]2014JOR; LBNPeer-driven;
Geospatial
Mothers and caregivers of children; RefugeesMothers and caregivers of children aged 12–23 months All immunizations generally; Routine immunizationsMeasure coverage;
Measure attitudes, behaviors and vaccine uptake barriers
Gullion, J. S. et al. [44]UnknownUSAPeer-driven;
Venue-based
Mothers and caregivers of childrenNot specifiedAll immunizations generallyMeasure attitudes, behaviors and vaccine uptake barriers
Coreil, J. et al. [27] 1987HTIVenue-basedMothers and caregivers of childrenMothers and caregivers of children aged 12–23 months All immunizations generallyMeasure attitudes, behaviors and vaccine uptake barriers
A Routine immunizations include childhood vaccines to protect against hepatitis, polio (oral polio vaccine), diphtheria, pertussis, tetanus, Haemophilus influenzae type B, measles, mumps, rubella, and tuberculosis (Bacillus Calmette-Guerin).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Koyuncu, A.; Ishizumi, A.; Daniels, D.; Jalloh, M.F.; Wallace, A.S.; Prybylski, D. The Use of Adaptive Sampling to Reach Disadvantaged Populations for Immunization Programs and Assessments: A Systematic Review. Vaccines 2023, 11, 424. https://doi.org/10.3390/vaccines11020424

AMA Style

Koyuncu A, Ishizumi A, Daniels D, Jalloh MF, Wallace AS, Prybylski D. The Use of Adaptive Sampling to Reach Disadvantaged Populations for Immunization Programs and Assessments: A Systematic Review. Vaccines. 2023; 11(2):424. https://doi.org/10.3390/vaccines11020424

Chicago/Turabian Style

Koyuncu, Aybüke, Atsuyoshi Ishizumi, Danni Daniels, Mohamed F. Jalloh, Aaron S. Wallace, and Dimitri Prybylski. 2023. "The Use of Adaptive Sampling to Reach Disadvantaged Populations for Immunization Programs and Assessments: A Systematic Review" Vaccines 11, no. 2: 424. https://doi.org/10.3390/vaccines11020424

APA Style

Koyuncu, A., Ishizumi, A., Daniels, D., Jalloh, M. F., Wallace, A. S., & Prybylski, D. (2023). The Use of Adaptive Sampling to Reach Disadvantaged Populations for Immunization Programs and Assessments: A Systematic Review. Vaccines, 11(2), 424. https://doi.org/10.3390/vaccines11020424

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop