Conducting Population Health Research during the COVID-19 Pandemic: Impacts and Recommendations

Abstract

The COVID-19 pandemic has resulted in many changes, including restrictions on indoor gatherings and visitation to residential aged care facilities, hospitals and certain communities. Coupled with potential restrictions imposed by health services and academic institutions, these changes may significantly impact the conduct of population health research. However, the continuance of population health research is beneficial for the provision of health services and sometimes imperative. This paper discusses the impact of COVID-19 restrictions on the conduct of population health research. This discussion unveils important ethical considerations, as well as potential impacts on recruitment methods, face-to-face data collection, data quality and validity. In addition, this paper explores potential recruitment and data collection methods that could replace face-to-face methods. The discussion is accompanied by reflections on the challenges experienced by the authors in their own research at an oral health service during the COVID-19 pandemic and alternative methods that were utilised in place of face-to-face methods. This paper concludes that, although COVID-19 presents challenges to the conduct of population health research, there is a range of alternative methods to face-to-face recruitment and data collection. These alternative methods should be considered in light of project aims to ensure data quality is not compromised.

1. Introduction

Since the announcement of COVID-19 as a pandemic in early March, countries around the world have been instating measures to prevent its spread. Among developed countries, these measures have centered around restricting the movement and gathering of people [1,2]. In Australia, specific measures have included closures of state borders, staying at home where possible, restriction on the size of outdoor and indoor assemblies, minimum space requirements for each person in enclosed areas, and the requirement to keep a 1.5-metre distance from others [3,4,5]. Furthermore, measures have been taken to protect groups considered at higher risk of contracting COVID-19, such as older people, those with chronic diseases, hospitalised individuals and Aboriginal and Torres Strait Islander peoples. These measures have included restrictions of visitors to residential aged care facilities and hospitals [6,7] and restriction of movement into remote Aboriginal and Torres Strait Islander communities [8].

It is, therefore, understandable that such constraints have, and will continue to have, a significant impact on many aspects of human undertakings and social interactions, including conducting health research over this period of change. In Australia, national guidelines have been released regarding the safe conduct of clinical trials during COVID-19, which inform contingency planning and alternative models to mitigate risks, particularly if the trial involves participants who are symptomatic for COVID-19 [9]. However, there remains a lack of such guidance for other types of research studies, including those which utilise population health research methods. Population health research, also known as public health research, has been defined as research that investigates and analyses factors that affect the health of populations or population groups or that tests and evaluates interventions to improve population health [10]. This research can employ quantitative, qualitative and mixed methods methodologies. Population health research is essential to inform health policies and programs, to ensure minimisation of gaps and inequities in healthcare provision and reduce strain on health services through preventative measures, an important consideration during COVID-19 [10]. Thus, it is essential that, where ethical and feasible, measures are taken to ensure population health research continues, particularly during this period when a number of health issues related to the pandemic and its control measures are emerging.

Over recent decades, as technology advanced and new techniques emerged, the use of alternatives to face-to-face research, such as e-research methods and secondary data sources in population health research, have been heavily critiqued. These methods have been shown to provide both benefits, such as reduced time and costs for large samples, and challenges, such as low response rates [11,12]. However, given the global situation, there is increasing pressure to reduce the need for population health researchers to conduct face-to-face research with participants [13,14]. To ensure ethical conduct of research and adherence with guidelines from governing institutions, researchers may no longer have the option to recruit participants, deliver interventions or gather data in person, and may actually be required to outline contingency plans if this becomes the case [15,16]. Thus, it is essential for researchers to consider alternative methods to undertake population health research that do not rely on face-to-face contact. The aims of this article are, therefore, to:

(i)

Explore the potential impact of COVID-19 on the conduct of population health research, and

(ii)

Discuss potential alternative recruitment and data collection strategies

This discussion will be supported by existing literature and supplemented with reflections from an ongoing population health research project, the conduct of which has been impacted by the COVID-19 pandemic.

2. Review of the Literature

2.1. Impact of COVID-19 on the Conduct of Population Health Research

Due to ethical considerations, policy changes and new guidelines that have arisen from the COVID-19 pandemic, the conduct of population health research has been impacted in various ways [13,14,15,16]. This section will discuss the effects of these changes in light of the current literature and reflect upon the authors’ own experience of COVID-19 during their research.

2.1.1. Research and Ethics

During times of public health emergency, it is imperative for health services to prioritise responses that address issues relating to the emergency [17]. However, concurrent research that explores the general health of the population during such times, or produces evidence that supports the response, should be given equal importance. Redirecting personnel and mobilizing equipment, facilities or other resources for outbreak response in pandemics, such as COVID-19, must be balanced with the ongoing need to facilitate research, both regarding the pandemic as well as research that is unrelated to the pandemic but could benefit the health of the general population [18]. In line with this, when assessing the ethical nature of their projects, researchers need to take measures to consider the value of their projects in view of the global context of COVID-19.

As a result, healthcare bodies and ethics committees are adopting different approaches when assessing new research projects, with some organisations choosing to postpone all face-to-face non-trial research activities and defer approval for commencement of new studies which are not related to COVID-19 [19]. Researchers will need to act in accordance with the policies and recommendations of their governing bodies during this time, and thus they may not be able to commence or continue their population health research projects.

2.1.2. Recruitment of Participants

Moreover, it has been identified that COVID-19 may be impacting vulnerable populations in a much more severe way than other population groups, and thus individuals from vulnerable population groups may experience more risk from participation in population health research [20]. However, the exclusion of such participants on the basis of being from a vulnerable population group is unethical, as any exclusion criteria must be based upon robust and current evidence [17]. Thus, the risks and benefits of involving individuals from vulnerable population groups in population health research must be carefully considered during this time, and it is vital to ensure that the benefits of participation outweigh the risks [21].

Although there is a range of recruitment methods that can be ethically employed in health research, there are several methods that could be affected by restrictions related to the COVID-19 pandemic. In the circumstance where the researcher is recruiting patients in a clinical setting that they are not a member of, changes in visitation policies may restrict the entry of the researcher to meet potential participants face-to-face [7,22]. Therefore, any recruitment methods dependent on face-to-face contact to inform potential participants and obtain consent may need to be reconsidered.

Furthermore, if the recruitment method involves clinical team members identifying eligible participants and explaining the study or being study participants themselves, there are additional factors to be considered [22]. Even prior to the pandemic, clinicians’ workload and time availability already posed significant challenges to research participation [23]. This is now even more of a challenge, considering that the changes in context, practice and policy resulting from the COVID-19 pandemic have increased the workloads of clinical staff around the world. Along with the stress and burnout staff are currently experiencing, their capacity to participate in the research project may be further limited [24]. Thus, it is likely that researchers may be unable to rely upon recruitment methods or participant samples that heavily involve the input of clinicians.

In both clinical and public settings, convenience recruitment methods, such as placement of research advertisement materials in areas where people congregate, for example, clinic waiting rooms, universities or community centers, may also be affected by COVID-19 restrictions. This is due to the fact that physical distancing and closures of public spaces and community facilities could greatly reduce the number of people frequenting such areas [25]. In addition, many health services are providing temporary telehealth services for patients, circumventing the need to attend clinics in person at all [9]. In addition, many universities have taken measures to operate partially or completely online [13]. As a result, many convenience recruitment methods may not be feasible for the foreseeable future.

2.1.3. Data Collection

The safety of research participants and research staff is paramount, and due to the communicable nature of COVID-19, many institutions are mandating the suspension of face-to-face data collection methods altogether [14]. Unfortunately, many population health research methods traditionally rely on face to face contact. An example of one of these methods used in both qualitative and quantitative research is observation, which is often conducted face-to-face in the natural environment [26]. However, not only may a researcher have limited ability to access this natural environment, but furthermore, in light of physical distance measures, it may not be permissible or ethical for a researcher to observe a participant in close proximity. Data collection that requires researchers and participants to be in close contact, such as with biophysical measures, will also pose difficulties for similar reasons [27]. Such limitations to data collection methods may impact the types of outcomes researchers can explore.

Similar to guidance stipulated for clinical trials, in cases where face-to-face data collection may proceed, it may be within the researchers’ duty of care to monitor participants for COVID symptoms and/or provide COVID testing [9]. This may be particularly crucial in instances where participants and research teams will be working with healthcare services or where group data collection methods, such as focus groups, will be used. This may require modification of study methods and allocation of additional resources to monitor symptoms and conduct COVID tests, and thus may have budget and staffing implications.

In addition, COVID-19 has had a significant socioeconomic impact resulting in job losses, financial instability, and family stressors [28] which may disproportionately increase respondent burden, particularly among vulnerable populations. Respondent burden is defined as how difficult, time-consuming or emotionally stressful an individual perceives participation in a research project to be, and may result in non-responses, or lower data quality [29]. This must also be considered in choosing data collection methods.

2.1.4. Data Quality

There are some data collection methods that, although easily conducted remotely, may obtain better data quality when conducted face-to-face with a researcher. For example, questionnaire data are often more complete and of higher quality when participants are assisted or supervised by a researcher [30].

There must also be considerations regarding how the current global context may impact data validity. Workforces around the world may be required to change their workplace practices [31]. These changes have been particularly extreme for those working in healthcare settings, including additional guidelines regarding hygiene and cleaning, use of personal protective equipment, procedures used during face-to-face consultations, postponement of non-emergency care and transition of some services to telehealth [32]. Given these changes to how healthcare is being delivered and received, it is important to consider comparability of data before and after such changes occurred and that such changes may impact the internal validity of longitudinal studies [33]. Furthermore, given that some practice changes, such as postponement of non-emergency consultations, are intended as temporary measures, researchers may need to consider the generalizability and/or relevance of any data collected while these measures are in place. This phenomenon, referred to as the interaction of history and treatment, is a major threat to the external validity of study findings [34].

Overall, there is the potential for COVID-19 to significantly impact the conduct of population health research projects. In some recent scoping reviews on population oral health strategies involving the broader workforce, only five out of 75 primary research studies followed qualitative or mixed methods, which all involved direct contact with participants [35,36,37,38]. A total of 33 out of the 34 quantitative intervention studies required direct contact with participants for delivery of the intervention and/or data collection [35,36,37,38]. A further 23 of the remaining 36 quantitative observational studies also required face-to-face contact for data collection, meaning a total of 81% of primary research studies in these scoping reviews would have been impacted by COVID-19 restrictions [35,36,37,38]. For many quantitative studies, face-to-face data collection methods may be the only appropriate way to collect data, particularly where participants are considered ‘hard-to-reach’, that is, from socially disadvantaged or underprivileged groups [39]. This could include those who are homeless and transient, those who have a chronic mental illness, those with low literacy levels, and indigenous peoples [39]. Furthermore, investigators may choose these methods to minimise nonresponse, or for sensitive topics [30]. This highlights that depending on the research area, there is the potential for the majority of studies to be impacted.

2.2. Alternative Recruitment and Data Collection Strategies

Despite the impacts identified above, particularly on recruitment of participants and data collection, there are many alternatives to face-to-face data collection that could be of merit during COVID-19. This section will discuss the benefits and limitations of these alternatives and draw upon reflection from the authors’ research to describe how such alternatives could be implemented.

2.2.1. Recruitment Strategies

Although face-to-face and certain convenience recruitment methods may not be feasible during a pandemic, there is a range of other recruitment methods that may be used. These include distribution of advertisement material via emails, letters, phone calls and the use of social media platforms. Evidence suggests that second to face-to-face recruitment, telephone recruitment may result in the highest response rates from participants and thus should be considered as an alternative during COVID-19 [40]. Limitations are that such methods can be costly and time-consuming, with one study reporting that telephone recruitment resulted in 1680 h spent on the phone and a total cost of $79 USD per participant [40]. Mailing advertising materials was shown to have a similar response rate [40] while incurring lower costs, with one study citing costs of printing and postage to be around $52 USD per participant [41].

With the advancement of online technology, research has highlighted the merit of online recruitment methods in health research. Although emailing of advertising materials may result in lower response rates than telephone or postal mail methods, it enables the identification and contact of a higher number of eligible participants [42]. Furthermore, with the growing popularity of various online and social media platforms, which are now being accessed by millions of people, their value as a recruitment tool is increasingly recognised [42]. Research indicates recruitment via social media remains valid in the COVID-19 context and has been able to reach a large number of participants [43]. One of the most commonly used platforms for research recruitment is Facebook, with its advantages of being the largest social media platform and having the ability for targeted advertisements to be shown to people with specific demographic characteristics, thereby, increasing chances of identifying eligible individuals [42]. This may also be considerably more cost-effective than other methods, with costs per participant reported to be from $0.60 to $20 USD [42]. Similarly, Instagram, which is owned by Facebook, can create similar targeted advertisements, which can also be linked to Facebook. However, Instagram users are generally a younger demographic, thus desired participant demographics should be taken into consideration when using these social media platforms for recruitment [44]. Twitter is also a commonly used platform, where users can pay for posts to be promoted, that is, displayed in feeds of users who do not follow the posting account, however, it is not able to target advertisements based on specific demographic characteristics [44]. Studies have also reported advertisements on search engines, such as Google, to be an effective recruitment strategy, and may have similar costs when compared to advertisements on social media platforms [45]. Despite the known merits of online recruitment methods, it has also been identified that samples from social media strategies can be biased, reducing diversity in age, socioeconomic status, location and ethnic background, and often result in lower retention rates [40,41,42,45]. Thus, in designing an online recruitment strategy, researchers should consider the potential limitations of these methods and have strategies in place to ensure a representative sample [43].

2.2.2. Data Collection Strategies

As discussed previously, face-to-face quantitative data collection methods, such as observations, biophysical measures and face-to-face questionnaires, may no longer be feasible or ethical during this period. However, there are several alternative data collection options. An example of this is the retrospective review of medical records, particularly electronic medical records, preventing unnecessary visits to healthcare facilities, with the added advantage of increased accessibility to a larger volume of patient data [46]. Medical records can be an effective and valid source of data for outcomes, such as length of stay, discharge destination and diagnosis, and thus should be considered as alternative data sources during the COVID-19 pandemic [46,47]. However, there are some variations seen between medical records and other forms of data collection, and this should be considered, for example, when interpreting results [48,49]. Another potential alternative to observations or physical measures is self-report. Studies have shown this to be valid for measures, such as weight, height and physical activity, and therefore, could serve as valid proxy measures for physical measurements [50,51]. However, some inaccuracies are seen in self-report of less socially accepted behaviors, such as cigarette smoking status [52]. This phenomenon of self-report bias is well recognised in the literature, however, when these limitations are acknowledged and accounted for, self-report data can still provide significant contributions to the existing body of evidence [53]. Compared to using medical records and self-report data in place of observations and physical measures, adapting face-to-face questionnaires for mail, telephone or online modes seems much more straightforward. Despite the idea that face-to-face questionnaires provide higher data quality, evidence continues to emerge that telephone, mail and online questionnaires yield sufficiently similar data quality and thus may be viable alternatives during COVID-19 [54,55].

The use of alternative methods when face-to-face options are not possible is probably more controversial when it comes to qualitative methodologies. For example, the use of telephone interviews for qualitative research has been widely criticised, with suspected losses of contextual and nonverbal data due to the absence of visual cues, as well as a lack of rapport and probing, making interpretation more challenging [56]. However, there is a lack of evidence to support these claims, which currently suggests that quality of data from telephone interviews is similar to that from face-to-face interviews, thus telephone interviews could be a simple alternative to use during COVID-19 [56,57]. Additionally, as various online platforms become more accessible with advancements in technology, the popularity of conducting interviews via online means is increasing [58]. Email has been used as a platform for qualitative interviews for years, although it presents some challenges regarding trustworthiness of data, specifically credibility, and also requires asynchronous interview techniques to be considered [58,59]. However, it has also been recognised as a viable and convenient method that may be particularly useful during COVID-19 [58,59]. Moreover, synchronous online qualitative interviews have become more feasible in recent years due to widespread access to electronic devices and internet connections, improvements in internet speeds and improvements in instant messaging and video conferencing software, such as Skype [60,61,62]. These platforms overcome the challenges presented with asynchronous interviews, and although instant messaging does not allow for visual cues, video conferencing does [61,62]. These methods are particularly valuable during times of limited flexibility or access to research participants, such as COVID-19, and have the potential to produce data comparable to face-to-face interviews, as long as any technical difficulties are considered and addressed [60,61,62]. Online methods may also be viable alternatives for the conduct of focus groups during COVID-19. For example, online discussion boards have been effectively used as an alternative to face-to-face focus groups, despite having similar limitations to other asynchronous online qualitative methods [63]. Likewise, the use of video conferencing software, such as Skype, may have similar effectiveness for focus groups as when used with interviews [60,63,64].

3. Reflection

3.1. Impact of COVID-19 on the Conduct of the Authors’ Project

The authors’ own research project aims to develop and evaluate a guideline implementation strategy to facilitate the translation of children’s healthy weight guidelines for public oral health staff into practice. All primary research associated with this project is centred around two phases: (i) Developing the guideline implementation strategy using co-design focus groups and input from an expert panel, and (ii) evaluation of the guideline implementation strategy using a pretest-posttest design and a validated questionnaire. Although ethical approval had already been obtained for the conduct of the project, this was done prior to the announcement of the pandemic, and the authors anticipated modification of study methods, and therefore, ethics amendments may be required. Data collection was scheduled to commence just as the pandemic was announced.

As the first phase was based around face-to-face focus groups with both public dental practitioners and parents of children in the community, the authors immediately had concerns regarding the safety and logistics of such gatherings in an enclosed space. Moreover, the implementation of children’s healthy weight guidelines for public oral health staff was dependent on routine care being provided at the public oral health services. However, as a response to the COVID-19 pandemic, the participating public oral health services were only providing emergency care at the scheduled time of data collection, and many staff were being redeployed to COVID-19 testing clinics. As a result, the internal and external validity of any data collected during this time of altered practice would be greatly compromised if data collection were to proceed. Finally, as the authors intended to use convenience methods to recruit parents by distributing flyers in oral health service waiting rooms, it was anticipated that these recruitment methods would need modification. Therefore, it was determined that the development and evaluation of the guideline implementation strategy should be postponed until routine dental practice resumed. Once this time came, it was anticipated that the recruitment and data collection strategies would need to be modified to ensure the most effective methods were chosen, and risks to participants were minimised.

3.2. Alternative Recruitment and Data Collection Strategies

As stated in the previous section, primarily due to internal validity concerns, the development and evaluation of the guideline implementation strategy for the authors’ research project were postponed until the dental services could resume practice as normal. Even with the inability to undertake the development and evaluation component as planned, there were still research activities that could be undertaken during this time. This included the development and evaluation of the questionnaire that would be used to assess the guideline implementation strategy. The authors initially planned to take a five-step validation process for the questionnaire: (i) Item generation using a review of the literature, (ii) face validation using a face-to-face reference group, (iii) construct validation using an online questionnaire with an expert panel, (iv) pilot testing and (v) factorial validation using a large online sample. Once the COVID-19 restrictions were instated, the authors scrutinised this process and realised that very few face-to-face steps were included in the process, and the face-to-face steps that were included could be easily modified to instead utilise alternative data collection strategies with minimal impact on data quality.

The main step requiring modification was step 2: Face validation using a face-to-face reference group. The first consideration was the recruitment method, which although it was already intended to be purposive recruitment via email, the original participation incentive was provision of refreshments during the time of the reference group. As this reference group would no longer be assembling face-to-face, the authors decided to re-allocate the funds for the light refreshments to supermarket gift vouchers, and this change was approved by the governing ethics committee. This more direct method to reimburse time and effort of participation was also deemed necessary due to the potential of increased respondent burden that participants may be feeling during this time.

In addition, the data collection method also required modification. The reference group became a virtual, online reference group conducted via email. With the difficulties that the enforced lockdown presented for participants, including the need for child minding, homeschooling and redeployment of health staff, it was agreed that this method would increase flexibility, allowing a two-week period where participants could provide feedback at their convenience. As per the initial design, this reference group did not require synchronous discussion, rather, each participant was to provide written feedback on their own hard copy of the questionnaire. Thus, the asynchronous format of providing the same briefing and feedback via email was seen to have minimal impact on the quality of data obtained. Using these methods, the authors were able to successfully obtain complete feedback on the questionnaire from all four members of the reference group.

Shortly after the questionnaire validation was completed, practice at the dental health services resumed as normal, therefore, the development and evaluation of the guideline implementation strategy could proceed. However, the authors were increasingly aware that the risks of COVID-19 with face to face data collection may not change in the near future. As a result, it was concluded that alternative data collection strategies would be implemented for the remainder of the project to eliminate the need for face to face contact. For the focus groups, it was decided that internet videoconferencing would be the best way to go, as visual brainstorming was an essential component of the focus group. Despite concerns about accessibility of videoconferencing software for study participants, throughout recruitment, it was found that participants preferred participating via videoconferencing, with platforms, such as Zoom, becoming more familiar to them over lockdown periods. A total of four videoconference focus groups were conducted for the project, and the authors were able to still utilise conventional focus group techniques to collect rich data from all participants. While the evaluation phase of this study is yet to commence, it was concluded that administering the evaluation questionnaires via an online survey platform would be an appropriate alternative to hard copy questionnaires. To ensure response rates are as high as possible in this phase, study champions will be nominated at each study site to help promote and coordinate this step. It is hypothesised that the authors’ careful consideration of the possible alternatives to data collection and recruitment contributed to the success of these substitute methods, which have traditionally been considered less desirable.

4. Conclusions

COVID-19 presents a range of challenges that can impact the conduct and quality of population health research. Regardless, there is a range of alternative strategies that can be employed to facilitate the safe and effective conduct of population health research during uncertain times. While all research methods have their own strengths and limitations, researchers should carefully consider the methods most appropriate for their study’s specific needs without compromising the quality of their data.