Next Article in Journal
Preference of Older Adults for Flexibility in Service and Providers in Community-Based Social Care: A Discrete Choice Experiment
Next Article in Special Issue
Association between Benign Paroxysmal Positional Vertigo and Previous Proton Pump Inhibitor Use: A Nested Case–Control Study Using a National Health Screening Cohort
Previous Article in Journal
A Brief Online Implicit Bias Intervention for School Mental Health Clinicians
Previous Article in Special Issue
Effect of Family Practice Contract Services on the Perceived Quality of Primary Care among Patients with Multimorbidity: A Cross-Sectional Study in Guangdong, China
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 2
10.3390/ijerph19020685
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Usefulness of Cochrane Reviews in Clinical Guideline Development—A Survey of 585 Recommendations

by
Christoffer Bruun Korfitsen
1,*,
Marie-Louise Kirkegaard Mikkelsen
1,
Anja Ussing
1,
Karen Christina Walker
1,2,
Jeanett Friis Rohde
1,2,
Henning Keinke Andersen
1,
Simon Tarp
1 and
Mina Nicole Händel
1,2
1
The Danish Health Authority, Islands Brygge 67, 2300 Copenhagen, Denmark
2
The Parker Institute, Bispebjerg and Frederiksberg Hospital, 2000 Frederiksberg, Denmark
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(2), 685; https://doi.org/10.3390/ijerph19020685
Submission received: 24 November 2021 / Revised: 4 January 2022 / Accepted: 6 January 2022 / Published: 7 January 2022
(This article belongs to the Special Issue Second Edition: Healthcare Services and Epidemiology for Chronic Disease Management and Prevention)
Browse Figures
Review Reports Versions Notes

Abstract

:
The Danish Health Authority develops clinical practice guidelines to support clinical decision-making based on the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system and prioritizes using Cochrane reviews. The objective of this study was to explore the usefulness of Cochrane reviews as a source of evidence in the development of clinical recommendations. Evidence-based recommendations in guidelines published by the Danish Health Authority between 2014 and 2021 were reviewed. For each recommendation, it was noted if and how Cochrane reviews were utilized. In total, 374 evidence-based recommendations and 211 expert consensus recommendations were published between 2014 and 2021. Of the 374 evidence-based recommendations, 106 included evidence from Cochrane reviews. In 28 recommendations, all critical and important outcomes included evidence from Cochrane reviews. In 36 recommendations, a minimum of all critical outcomes included evidence from Cochrane reviews, but not all important outcomes. In 33 recommendations, some but not all critical outcomes included evidence from Cochrane reviews. Finally, in nine recommendations, some of the important outcomes included evidence from Cochrane reviews. In almost one-third of the evidence-based recommendations, Cochrane reviews were used to inform clinical recommendations. This evaluation should inform future evaluations of Cochrane review uptake in clinical practice guidelines concerning outcomes important for clinical decision-making.

1. Introduction

Clinical practice guidelines (hereafter referred to as guidelines) are needed to assist clinicians and policymakers in making informed decisions on healthcare and public health policy based on current best evidence [1,2,3,4]. High-quality guidelines can support medical decision making and improve care by identifying practices that maximize benefit and minimize harm [5]. High-quality guidelines are developed with methodological rigor by multidisciplinary panels. Guidelines consider several factors such as patients’ preferences and values, available resources and costs, and cultural heterogeneity, in addition to the evidence [6].
Conducting or using existing high-quality systematic reviews to synthesize current best evidence is essential in guideline development [4]. Cochrane reviews are regarded as high-quality summaries presenting unbiased information useful for developing guidelines and, thus, impact clinical decision-making [7,8].
Cochrane reviews are highly warranted in guideline development and can save guideline developers time (e.g., using existing searches/data). By following high methodological standards [9,10,11,12], the Cochrane Review Groups have published more than 8704 systematic reviews across all health-related topics [13], and Cochrane reviews have informed both national and international guidelines [14,15,16].
However, utilizing Cochrane reviews in guideline development is challenging if there are large discrepancies between the scope of the review and the guideline [17]. Discrepancies are often found in the population of interest, intervention, comparison and/or outcomes of interest. For the findings to influence policy and practice, the outcomes need to be especially relevant and important to key stakeholders, including patients and the public, health care professionals and others making decisions about health care [18,19,20,21]. Examples of critical outcomes not reported as standard in clinical trials and Cochrane reviews have been reported [22,23]. Standardization of critical and important outcomes is largely available in Core Outcome Set (COS), an agreed standardized set of outcomes that should be measured and reported, as a minimum, in all clinical trials in specific health areas [18,24]. However, the uptake of COS in both Cochrane reviews [23,25] and clinical trials is are not yet consistent [26,27].
To the best of our knowledge, no guideline development group has investigated how Cochrane reviews have informed specific critical and important outcomes of interest in guidelines. Investigation of the extent to which Cochrane reviews are used in guidelines has previously been done by citation searches [15,16]. However, this method can affect the validity of the findings since Cochrane reviews can inform both the background of the guideline and specifically inform the systematic searches for relevant evidence [28]. Thus, the objective of this study was to explore the usefulness of Cochrane reviews as a source of evidence in the development of clinical recommendations.
The methodology used in the Danish Health Authority guideline development has previously been described [29,30,31,32,33,34,35,36,37,38,39,40]. In short, the methodology is based on the recommendations for conducting systematic reviews and meta-analyses from the Cochrane Handbook for systematic reviews [8]. Since 2013, the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) methodology has been used [41]. GRADE has become the most recognized and transparent method available to date to develop clinical guidelines. The clinical questions are operationalized according to the population, intervention, comparator, outcomes format (PICO), and pre-specified explicitly in a protocol approved by the management of the Danish Health Authority. Outcomes are judged as critical or important to patients, and their timing and effect measures are defined a priori [19,41]. According to GRADE, a critical outcome is defined as being patient-important and critical for decision making. The overall quality of the evidence depends on the certainty of the evidence of the critical outcome(s). An important outcome is defined as important to patients but not critical for decision making [19]. Outcomes are preferably chosen from COS [24,42,43,44,45,46] if available, or the guideline panel will agree on critical and important outcomes based on outcomes used in clinical practice and clinical trials. Outcomes are always prioritized and adapted to national contextual factors [19]. The Danish national recommendations are continuously monitored; at least every three years, an updated guideline search and search for Cochrane reviews will inform a need to update the recommendation. The methodology for conducting the systematic reviews within the guidelines process is the same used for the entire study period.

2. Materials and Methods

Guidelines published by the Danish Health Authority between 2014 and 2021 were identified and reviewed [47]. The total number of recommendations included in the Danish guidelines were mapped and categorized into evidence-based recommendations or expert consensus recommendations. The full guidelines are available in Danish with all supporting material on the Danish Health Authority website [29] and the MAGICapp website [48].
Data from the recommendations were extracted to a pilot-tested Microsoft Excel datasheet (see Supplementary Table S1). The pilot-testing included data checking by one reviewer (C.B.K.) of the first five guidelines with extracted data (starting from 2014). Any discrepancies resulted in continuous adjustments to the datasheet. One out of four reviewers (C.B.K., M.L.K.M., A.U., J.F.R.) followed a series of steps to extract information about how recommendations were based on evidence derived from Cochrane reviews. First, we identified the Cochrane review citations by screening the reference list of the guidelines. Secondly, we noted whether the use of the Cochrane review in the guideline was applied either as background information or included as a source of evidence. Finally, we extracted information from the results section and assorted the outcomes into the following four non-overlapping categories: (I) all outcomes (both critical and important outcomes) informed by a Cochrane review, (II) all critical outcomes informed, but not all important outcomes (III) at least one critical outcome informed, or (IV) no critical but one or more important outcomes informed by a Cochrane review. Each recommendation was only categorized once.
The distributions and proportions of recommendations informed by Cochrane reviews across topics were summarized with median and interquartile range (IQR). We analyzed whether the proportion of evidence-based recommendations in the Danish guidelines varied over time. Moreover, the proportions of recommendations with evidence from Cochrane reviews across medical topics (using the Cochrane Topic taxonomy [49]) were also analyzed. The analyses were performed in Microsoft Excel (Microsoft Cooperation, Redmond, United States, 2019).

3. Results

Between 2014 and 2021, 56 guidelines were published by the Danish Health Authority, comprising 374 (64%) evidence-based recommendations and 211 (36%) expert consensus recommendations (Table 1). The 374 recommendations included 107 citations of Cochrane reviews, where 106 (28%) were included as evidence (Table 1). Cochrane reviews informed all outcomes in 28 (7%) evidence-based recommendations (Table 1). For 36 (10%) of the evidence-based recommendations, all critical outcomes were informed by a Cochrane review, but not all of the important outcomes were. In 33 (9%) of the evidence-based recommendations, at least one critical outcome was informed by Cochrane reviews. Finally, nine (2%) evidence-based recommendations had at least one important outcome informed by evidence from a Cochrane review.
When a Cochrane review was available and used as a source of evidence, the Cochrane review informed all critical outcomes and important outcomes in 26% of all recommendations informed by Cochrane reviews (Table 2). A total of 34% informed all critical but not all important outcomes, 31% informed at least one critical outcome and 8% informed at least one important but no critical outcome (Table 2).
From 2014 to 2021, the proportion of evidence-based recommendations in the Danish guidelines varied from 31% in 2014 to 81% in 2021 (Figure 1). The proportion of recommendations with evidence informed by Cochrane reviews varied from 24% in 2014 to 20% in 2021 (range 17–42%) (Figure 1). Recommendations with evidence from Cochrane reviews varied between 0 and 80% across medical topics, with a median of 29% (IQR: 14–38%) (Figure 2). Across topics, Pregnancy and Childbirth (75%), Pain and Anesthesia (63%) and Dentistry and Oral Health (40%) had the highest use of evidence from Cochrane reviews. In contrast, the topics Eyes and Vision (11%), Endocrine and Metabolic (5%) and Skin Disorders (0%) had the lowest.

4. Discussion

Since 2014, recommendations published by the Danish Health Authority have increasingly been based on evidence rather than consensus on best practice. However, less than one-third of the evidence-based recommendations published by the Danish Health Authority used evidence from Cochrane reviews to inform outcomes in the clinical question of interest. The use varied across medical topics. In 17% of these recommendations, evidence from Cochrane reviews was used for all critical outcomes. However, in 93% of the evidence-based recommendations, the Cochrane reviews did not inform all outcomes. However, when a Cochrane review was used as evidence in a recommendation, in 60% of these recommendations it informed all critical outcomes (when merging the two categories: “all critical and important outcomes covered” and “all critical outcomes covered/informed”). Additionally, in 91%, at least one critical outcome was informed (when merging all categories except “at least one important but no critical outcome”).
There are several possible explanations for the sparse use of Cochrane reviews in the evidence-based recommendations reviewed. One explanation could be the lack of updated Cochrane reviews [15]. The Cochrane collaboration aims to impact global health by publishing updated systematic reviews. Thus, the Danish National Health Authority expects Cochrane reviews to not only be performed to the highest standards but also include all patient-relevant outcomes. However, Cochrane’s relevance and value will depend on its ability to respond innovatively and promptly to decision-makers’ needs. This is confirmed in experiences reported from guideline development groups from the United Kingdom National Institute for Health Research [15,17].
Another explanation for the findings in our study might be related to the aim of the Danish guidelines, such as specific eligibility criteria related to population and/or intervention [15]. Another aspect could be that nationally developed guidelines often adapt clinical questions to the context of the healthcare system in that country. For example, the Danish Health Authority ensures that clinical questions in the guidelines cover the following criteria: considerable health- or resource-burden, variation in clinical practice, professional disagreement or doubt about best practice, new precarious technology, or considerable possible change in treatment/screening indication. These criteria ensure that clinical questions are relevant for clinical practice [50]. Therefore, the recommendations deal with selected well-defined aspects of diagnostics, treatment, care and rehabilitation for specific patient groups, where the need for clarifying the evidence has shown to be relevant. It is emphasized that the topics address important issues, i.e., either controversial, unresolved, or intrusive, and not trivial or already well-described issues.
A major threat to the validity of clinical trials and systematic reviews is missing outcome data or inconsistent reporting of outcome data in clinical research [51,52,53,54]. In one-fifth of 283 Cochrane reviews, more than 50% of the patient data for the primary outcome was missing [55]. Cochrane reviews and guidelines often include many outcomes about the harms and benefits of an intervention or management strategy [19,56,57]. When choosing to include a Cochrane Review as a source of evidence, it is enough if the review has just one relevant outcome of interest. Cochrane states that established COS should be used where available, and patient-reported outcomes should be included where possible [7,56]. However, evidence suggests that these guidelines are not consistently followed [23,25,58].
This study showed a tendency to include fewer Cochrane reviews in evidence-based recommendations between 2014 and 2021. This was surprising since guideline developers following the GRADE method are likely more prone to include Cochrane reviews [16]. This may be due to the changed criteria or priority of clinical questions for the guideline development at the Danish Health Authority since the GRADE method was incorporated in 2013 [50]. Here, the clinical questions selected were the ones that are difficult to answer in standard clinical trial designs, thus hypothetically leading to less available evidence to answer the question and possibly limiting the transferability to a Cochrane review.
A limitation to this study was that only one author extracted information per guideline, and thus there was no validation of the extracted information. Additionally, the authors did not extract information about the uptake of COS in the Danish guidelines, and this information could have informed the discussion of the findings. It is worth noting that there has not been a standardized use nor citation of COS across guidelines in the development of the Danish guidelines in this period.
This descriptive study focused on outcomes informed by evidence from Cochrane reviews. More investigation of the use of systematic reviews (not limited to Cochrane reviews) and specific use of COS could have further clarified the findings and informed future evaluations. Unfortunately, information on the use of COS was only reported in a few Danish guidelines, primarily the latest published.
The descriptive nature of this study calls for further investigation of how Cochrane reviews are used to inform outcomes of interest in guidelines in other countries using GRADE. Additionally, few studies have been undertaken to assess the uptake of COS in trials and reviews, and to our knowledge, no studies have assessed the uptake of COS in guidelines [26]. This could help facilitate international collaborations.
Rather than simply presenting the status of applying Cochrane reviews (the objective of this study), the next step might be investigating the effect of using Cochrane reviews on methodology process parameters, such as time and resources. Such evaluation could be relevant when investigating adaption of Cochrane reviews (using both evidence-base, risk of bias assessments and analyses). Adapting Cochrane reviews or other high-quality systematic reviews is highly required since this, besides saving guideline developers resources, can also help limit research waste [59,60,61]. However, a barrier to adapting reviews is the lack of transparency on data extraction, i.e., clarity on which study data were applied in meta-analyses when multiple data are available for the same outcome. This often leads guideline developers to not favor review adaption [17]. Standardized methods for data extraction [62] and prioritizing specific outcome measures is warranted, e.g., when multiple patient-reported outcome measures within a domain are available [63]. Future studies should investigate the uptake of COS in trials, reviews and guidelines, and the use of adaptions in guideline development.

5. Conclusions

Almost one-third of the Danish Health Authority evidence-based recommendations used Cochrane reviews to inform clinical recommendations. For nearly two-thirds of these recommendations, Cochrane reviews were used for all critical outcomes. This emphasizes the demand for using core outcome sets in clinical trials, Cochrane reviews and guidelines. This evaluation should inform future evaluations of Cochrane review uptake in clinical practice guidelines concerning outcomes critical and important for clinical decision-making.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/ijerph19020685/s1, Supplementary Table S1: Datasheet.

Author Contributions

Methodology, C.B.K., M.-L.K.M., A.U., K.C.W., J.F.R., H.K.A., S.T. and M.N.H.; formal analysis, C.B.K. and M.-L.K.M.; investigation, C.B.K., M.-L.K.M., A.U. and J.F.R.; data curation, C.B.K. and M.-L.K.M.; writing—original draft preparation, C.B.K., M.-L.K.M., A.U., K.C.W., J.F.R., H.K.A., S.T. and M.N.H.; writing—review and editing, C.B.K., M.-L.K.M., A.U., K.C.W., J.F.R., H.K.A., S.T. and M.N.H.; visualization, C.B.K., M.-L.K.M., A.U., K.C.W., J.F.R., H.K.A., S.T. and M.N.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding. The Parker Institute is supported by a core grant by the OAK foundation (OCAY-18-774-OFIL).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available in Supplementary Table S1.

Acknowledgments

The authors would like to sincerely thank all the associated guideline panels, reference groups, search specialists, methods consultants, hearing organizations and patient representatives for their involvement in the years of guidelines development.

Conflicts of Interest

All authors hold positions within The Danish Health Authority. After submitting the present paper, C.B.K. has acquired a position as research assistant with Cochrane Denmark. H.K.A. previously (until 2018) held a position as Managing Editor for the Cochrane Colorectal Group. The authors declare no other conflicts of interest.

Abbreviations

COSCore Outcome Set
GRADEGrading of Recommendations Assessment, Development, and Evaluation
IQRInterquartile range
PICOPopulation, Intervention, Comparison and Outcome

References

  1. Schünemann, H.J.; Wiercioch, W.; Etxeandia-Ikobaltzeta, I.; Falavigna, M.; Santesso, N.; Mustafa, R.; Ventresca, M.; Brignardello-Petersen, R.; Laisaar, K.-T.; Kowalski, S.; et al. Guidelines 2.0: Systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ Can. Med. Assoc. J. 2014, 186, E123–E142. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. World Health Organization. WHO Handbook for Guideline Development; World Health Organization: Geneva, Switzerland, 2014; Available online: https://apps.who.int/iris/handle/10665/145714 (accessed on 28 December 2020).
  3. Oxman, A.D.; Schünemann, H.J.; Fretheim, A. Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res. Policy Syst. 2006, 4, 20. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Zhang, Y.; Akl, E.A.; Schünemann, H.J. Using systematic reviews in guideline development: The GRADE approach. Res. Synth. Methods 2019, 10, 312–329. [Google Scholar] [CrossRef] [PubMed]
  5. Laine, C.; Taichman, D.B.; Mulrow, C. Trustworthy Clinical Guidelines. Ann. Intern. Med. 2011, 154, 774–775. [Google Scholar] [CrossRef] [Green Version]
  6. Alonso-Coello, P.; Oxman, A.D.; Moberg, J.; Brignardello-Petersen, R.; A Akl, E.; Davoli, M.; Treweek, S.; A Mustafa, R.; O Vandvik, P.; Meerpohl, J.; et al. GRADE Evidence to Decision (EtD) frameworks: A systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ 2016, 353, i2089. [Google Scholar] [CrossRef] [Green Version]
  7. MECIR Manual. Available online: https://community.cochrane.org/mecir-manual (accessed on 19 February 2020).
  8. Cochrane Handbook for Systematic Reviews of Interventions. Available online: https://training.cochrane.org/handbook/current (accessed on 23 December 2020).
  9. Jadad, A.R.; Cook, D.J.; Jones, A.; Klassen, T.P.; Tugwell, P.; Moher, M.; Moher, D. Methodology and reports of systematic reviews and meta-analyses: A comparison of Cochrane reviews with articles published in paper-based journals. JAMA 1998, 280, 278–280. [Google Scholar] [CrossRef] [Green Version]
  10. Jadad, A.R.; Moher, M.; Browman, G.P.; Booker, L.; Sigouin, C.; Fuentes, M.; Stevens, R. Systematic reviews and meta-analyses on treatment of asthma: Critical evaluation. BMJ 2000, 320, 537–540. [Google Scholar] [CrossRef] [Green Version]
  11. Goldkuhle, M.; Narayan, V.M.; Weigl, A.; Dahm, P.; Skoetz, N. A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer. BMJ Open 2018, 8, e020869. [Google Scholar] [CrossRef] [Green Version]
  12. Jørgensen, A.W.; Hilden, J.; Gøtzsche, P.C. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: Systematic review. BMJ 2006, 333, 782. [Google Scholar] [CrossRef] [Green Version]
  13. Search | Cochrane Library. Available online: https://www.cochranelibrary.com/cdsr/reviews (accessed on 15 October 2021).
  14. Use of Cochrane Reviews to Inform WHO Guidelines. Available online: https://www.cochrane.org/news/use-cochrane-reviews-inform-who-guidelines (accessed on 11 January 2021).
  15. Alderson, P.; Tan, T. The use of Cochrane Reviews in NICE clinical guidelines. Cochrane Database Syst. Rev. 2011, 8, 1465–1858. [Google Scholar] [CrossRef]
  16. Garegnani, L.I.; Rosón, P.; Liquitay, C.E.; Meza, N.; Arancibia, M.; Madrid, E.; Franco, J.V.A. Use of Cochrane reviews in nationally-developed clinical practice guidelines in Latin America. Medwave 2020, 20, e8027. [Google Scholar] [CrossRef]
  17. Bunn, F.; Trivedi, D.; Alderson, P.; Hamilton, L.; Martin, A.; Iliffe, S. The impact of Cochrane Systematic Reviews: A mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research. Syst. Rev. 2014, 3, 125. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Williamson, P.R.; Altman, D.G.; Bagley, H.; Barnes, K.L.; Blazeby, J.; Brookes, S.T.; Clarke, M.; Gargon, E.; Gorst, S.; Harman, N.; et al. The COMET Handbook: Version 1.0. Trials 2017, 18, 280. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Guyatt, G.H.; Oxman, A.D.; Kunz, R.; Atkins, D.; Brozek, J.; Vist, G.; Alderson, P.; Glasziou, P.; Falck-Ytter, Y.; Schünemann, H.J. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J. Clin. Epidemiol. 2011, 64, 395–400. [Google Scholar] [CrossRef] [PubMed]
  20. Woolf, S.; Schünemann, H.J.; Eccles, M.P.; Grimshaw, J.M.; Shekelle, P. Developing clinical practice guidelines: Types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implement. Sci. 2012, 7, 61. [Google Scholar] [CrossRef] [Green Version]
  21. Williamson, P.; Altman, D.; Blazeby, J.; Clarke, M.; Gargon, E. Driving up the Quality and Relevance of Research Through the Use of Agreed Core Outcomes. J. Health Serv. Res. Policy 2012, 17, 1–2. [Google Scholar] [CrossRef] [Green Version]
  22. Ioannidis, J.P.A.; Horbar, J.D.; Ovelman, C.M.; Brosseau, Y.; Thorlund, K.; E Buus-Frank, M.; Mills, E.J.; Soll, R.F. Completeness of main outcomes across randomized trials in entire discipline: Survey of chronic lung disease outcomes in preterm infants. BMJ 2015, 350, h72. [Google Scholar] [CrossRef] [Green Version]
  23. Williamson, P.R.; de Ávila Oliveira, R.; Clarke, M.; Gorst, S.L.; Hughes, K.; Kirkham, J.J.; Li, T.; Saldanha, I.J.; Schmitt, J. Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: A review. BMJ Open 2020, 10, e036562. [Google Scholar] [CrossRef]
  24. Gargon, E.; Gorst, S.L.; Matvienko-Sikar, K.; Williamson, P.R. Choosing important health outcomes for comparative effectiveness research: 6th annual update to a systematic review of core outcome sets for research. PLoS ONE 2021, 16, e0244878. [Google Scholar] [CrossRef] [PubMed]
  25. Verbeek, J.; Ijaz, S.; Mischke, C. Inclusion criteria for outcomes of studies not clearly reported in Cochrane systematic reviews. J. Clin. Epidemiol. 2017, 87, 98–106. [Google Scholar] [CrossRef] [Green Version]
  26. Hughes, K.L.; Clarke, M.; Williamson, P.R. A systematic review finds core outcome set uptake varies widely across different areas of health. J. Clin. Epidemiol. 2021, 129, 114–123. [Google Scholar] [CrossRef] [PubMed]
  27. Matvienko-Sikar, K.; Avery, K.; Blazeby, J.M.; Devane, D.; Dodd, S.; Egan, A.M.; Gorst, S.L.; Hughes, K.; Jacobsen, P.; Kirkham, J.J.; et al. Use of core outcome sets was low in clinical trials published in major medical journals. J. Clin. Epidemiol. 2022, 142, 19–28. [Google Scholar] [CrossRef]
  28. Barnes, K.L.; Kirkham, J.J.; Clarke, M.; Williamson, P.R. Citation analysis did not provide a reliable assessment of core outcome set uptake. J. Clin. Epidemiol. 2017, 86, 153–159. [Google Scholar] [CrossRef] [Green Version]
  29. Development of CPG. Available online: https://www.sst.dk/da/opgaver/patientforloeb-og-kvalitet/nationale-kliniske-retningslinjer-nkr/saadan-udarbejdes-nkr (accessed on 28 December 2020).
  30. Liaghat, B.; Ussing, A.; Petersen, B.H.; Andersen, H.K.; Barfod, K.W.; Jensen, M.B.; Hoegh, M.; Tarp, S.; Juul-Kristensen, B.; Brorson, S. Supervised training compared with no training or self-training in patients with subacromial pain syndrome: A systematic review and meta-analysis. Arch. Phys. Med. Rehabil. 2021, 102, 2428–2441.e10. [Google Scholar] [CrossRef]
  31. Kjaer, P.; Kongsted, A.; Hartvigsen, J.; Isenberg-Jørgensen, A.; Schiøttz-Christensen, B.; Søborg, B.; Krog, C.; Møller, C.M.; Halling, C.M.B.; Lauridsen, H.; et al. National clinical guidelines for non-surgical treatment of patients with recent onset neck pain or cervical radiculopathy. Eur. Spine J. 2017, 26, 2242–2257. [Google Scholar] [CrossRef]
  32. Keller, A.; Rimestad, M.L.; Rohde, J.F.; Petersen, B.H.; Korfitsen, C.B.; Tarp, S.; Lauritsen, M.B.; Händel, M.N. The Effect of a Combined Gluten- and Casein-Free Diet on Children and Adolescents with Autism Spectrum Disorders: A Systematic Review and Meta-Analysis. Nutrients 2021, 13, 470. [Google Scholar] [CrossRef]
  33. Elers, J.; Hornum Bing, M.; Birkefoss, K.; Rohde, J.F.; Ussing, A.; Glavind, K. TVT or TVT-O?—A systematic review and meta-analysis comparing efficacy, complications and re-operations. Eur. J. Obstet. Gynecol. Reprod. Biol. 2021, 258, 146–151. [Google Scholar] [CrossRef]
  34. Ussing, A.; Mikkelsen, M.-L.K.; Villumsen, B.R.; Wejlgaard, J.; Bistrup, P.E.; Birkefoss, K.; Bandholm, T. Supervised exercise therapy compared with no exercise therapy to reverse debilitating effects of androgen deprivation therapy in patients with prostate cancer: A systematic review and meta-analysis. Prostate Cancer Prostatic Dis. 2021, 1–16. [Google Scholar] [CrossRef] [PubMed]
  35. Grønlund, C.; Devantier, L.; Callesen, H.E.; Hougaard, D.D.; Händel, M.N.; Schmidt, J.H.; Guldfred, F.L.-A.; Djurhuus, B.D. Vertiginous Episodes in Menière Disease following Transmyringeal Ventilation Tube Insertion: A Systematic Review on the Current State of Evidence. Int. Arch. Otorhinolaryngol. 2021, 25, e463–e470. [Google Scholar] [CrossRef] [PubMed]
  36. Händel, M.N.; Rohde, J.F.; Rimestad, M.L.; Bandak, E.; Birkefoss, K.; Tendal, B.; Lemcke, S.; Callesen, H.E. Efficacy and Safety of Polyunsaturated Fatty Acids Supplementation in the Treatment of Attention Deficit Hyperactivity Disorder (ADHD) in Children and Adolescents: A Systematic Review and Meta-Analysis of Clinical Trials. Nutrients 2021, 13, 1226. [Google Scholar] [CrossRef]
  37. DeVantier, L.; Hougaard, D.; Händel, M.N.; Guldfred, F.L.-A.; Schmidt, J.H.; Djurhuus, B.; Callesen, H.E. Using betahistine in the treatment of patients with Menière’s disease: A meta-analysis with the current randomized-controlled evidence. Acta Otolaryngol. 2020, 140, 845–853. [Google Scholar] [CrossRef]
  38. Devantier, L.; Djurhuus, B.; Hougaard, D.; Händel, M.N.; Guldfred, F.L.-A.; Schmidt, J.H.; Edemann-Callesen, H. Intratympanic Steroid for Menière’s Disease: A Systematic Review. Otol. Neurotol. 2019, 40, 806–812. [Google Scholar] [CrossRef]
  39. DeVantier, L.; Schmidt, J.H.; Djurhuus, B.; Hougaard, D.D.; Händel, M.N.; Guldfred, F.L.-A.; Edemann-Callesen, H. Current state of evidence for endolymphatic sac surgery in Menière’s disease: A systematic review. Acta Otolaryngol. (Stockh) 2019, 139, 953–958. [Google Scholar] [CrossRef] [PubMed]
  40. DeVantier, L.; Guldfred, F.L.-A.; Djurhuus, B.D.; Händel, M.N.; Schmidt, J.H.; Hougaard, D.D.; Edemann-Callesen, H. Positive pressure device treatment for Menière’s disease: An overview of the current evidence and a meta-analysis. Eur. Arch. Otorhinolaryngol. 2019, 276, 1263–1273. [Google Scholar] [CrossRef] [PubMed]
  41. Guyatt, G.; Oxman, A.D.; Akl, E.A.; Kunz, R.; Vist, G.; Brozek, J.; Norris, S.; Falck-Ytter, Y.; Glasziou, P.; DeBeer, H.; et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J. Clin. Epidemiol. 2011, 64, 383–394. [Google Scholar] [CrossRef] [PubMed]
  42. Gargon, E.; Gorst, S.L.; Williamson, P.R. Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research. PLoS ONE 2019, 14, e0225980. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Gargon, E.; Gurung, B.; Medley, N.; Altman, D.G.; Blazeby, J.M.; Clarke, M.; Williamson, P.R. Choosing Important Health Outcomes for Comparative Effectiveness Research: A Systematic Review. PLoS ONE 2014, 9, e99111. [Google Scholar] [CrossRef] [Green Version]
  44. Gorst, S.L.; Gargon, E.; Clarke, M.; Blazeby, J.M.; Altman, D.G.; Williamson, P.R. Choosing Important Health Outcomes for Comparative Effectiveness Research: An Updated Review and User Survey. PLoS ONE 2016, 11, e0146444. [Google Scholar] [CrossRef]
  45. Gorst, S.L.; Gargon, E.; Clarke, M.; Smith, V.; Williamson, P.R. Choosing Important Health Outcomes for Comparative Effectiveness Research: An Updated Review and Identification of Gaps. PLoS ONE 2016, 11, e0168403. [Google Scholar] [CrossRef] [Green Version]
  46. Davis, K.; Gorst, S.L.; Harman, N.; Smith, V.; Gargon, E.; Altman, U.G.; Blazeby, J.M.; Clarke, M.; Tunis, S.; Williamson, P.R. Choosing important health outcomes for comparative effectiveness research: An updated systematic review and involvement of low and middle income countries. PLoS ONE 2018, 13, e0190695. [Google Scholar] [CrossRef] [Green Version]
  47. CPG by Topic. Available online: https://www.sst.dk/da/opgaver/patientforloeb-og-kvalitet/nationale-kliniske-retningslinjer-nkr/nkr-udgivelser-efter-emne (accessed on 28 December 2020).
  48. MAGICapp-Making GRADE the Irresistible Choice-Guidelines and Evidence Summaries. Available online: https://app.magicapp.org/#/guidelines (accessed on 20 May 2021).
  49. Browse by Topic|Cochrane Library. Available online: https://www.cochranelibrary.com/browse-by-topic (accessed on 18 December 2020).
  50. Eccles, M.P.; Grimshaw, J.M.; Shekelle, P.; Schünemann, H.J.; Woolf, S. Developing clinical practice guidelines: Target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implement. Sci. 2012, 7, 60. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Chan, A.W.; Hróbjartsson, A.; Haahr, M.T.; Gøtzsche, P.C.; Altman, D.G. Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials: Comparison of Protocols to Published Articles. JAMA 2004, 291, 2457–2465. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Wayant, C.; Scheckel, C.; Hicks, C.; Nissen, T.; LeDuc, L.; Som, M.; Vassar, B.M. Evidence of selective reporting bias in hematology journals: A systematic review. PLoS ONE 2017, 12, e0178379. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Dwan, K.; Altman, D.G.; Clarke, M.; Gamble, C.; Higgins, J.P.T.; Sterne, J.A.C.; Williamson, P.R.; Kirkham, J. Evidence for the Selective Reporting of Analyses and Discrepancies in Clinical Trials: A Systematic Review of Cohort Studies of Clinical Trials. PLOS Med. 2014, 11, e1001666. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  54. Raghav, K.P.S.; Mahajan, S.; Yao, J.C.; Hobbs, B.P.; Berry, D.A.; Pentz, R.D.; Tam, A.L.; Hong, W.K.; Ellis, L.M.; Abbruzzese, J.L.; et al. From Protocols to Publications: A Study in Selective Reporting of Outcomes in Randomized Trials in Oncology. J. Clin. Oncol. 2015, 33, 3583–3590. [Google Scholar] [CrossRef] [PubMed]
  55. Kirkham, J.J.; Gargon, E.; Clarke, M.; Williamson, P.R. Can a core outcome set improve the quality of systematic reviews?—A survey of the Co-ordinating Editors of Cochrane review groups. Trials 2013, 14, 21. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  56. Selecting Outcomes to Be Addressed for Studies Included in the Review (C14-C18). Available online: https://community.cochrane.org/mecir-manual/standards-conduct-new-cochrane-intervention-reviews-c1-c75/developing-protocol-review-c1-c23/selecting-outcomes-be-addressed-studies-included-review-c14-c18 (accessed on 5 April 2021).
  57. Chapter 19: Adverse Effects. Available online: https://training.cochrane.org/handbook/current/chapter-19 (accessed on 4 November 2021).
  58. Tsujimoto, Y.; Tsutsumi, Y.; Kataoka, Y.; Banno, M.; Furukawa, T.A. Around ten percent of most recent Cochrane reviews included outcomes in their literature search strategy and were associated with potentially exaggerated results: A research-on-research study. J. Clin. Epidemiol. 2021, 141, 74–81. [Google Scholar] [CrossRef]
  59. Glasziou, P.; Chalmers, I. Research waste is still a scandal—an essay by Paul Glasziou and Iain Chalmers. BMJ 2018, 363, k4645. [Google Scholar] [CrossRef]
  60. Chalmers, I.; Glasziou, P. Avoidable waste in the production and reporting of research evidence. Lancet 2009, 374, 86–89. [Google Scholar] [CrossRef]
  61. Lund, H.; Juhl, C.; Christensen, R. Systematic reviews and research waste. Lancet 2016, 387, 123–124. [Google Scholar] [CrossRef] [Green Version]
  62. Elamin, M.B.; Flynn, D.N.; Bassler, D.; Briel, M.; Alonso-Coello, P.; Karanicolas, P.J.; Guyatt, G.H.; Malaga, G.; Furukawa, T.; Kunz, R.; et al. Choice of data extraction tools for systematic reviews depends on resources and review complexity. J. Clin. Epidemiol. 2009, 62, 506–510. [Google Scholar] [CrossRef] [PubMed]
  63. Juhl, C.; Lund, H.; Roos, E.M.; Zhang, W.; Christensen, R. A hierarchy of patient-reported outcomes for meta-analysis of knee osteoarthritis trials: Empirical evidence from a survey of high impact journals. Arthritis 2012, 2012, 136245. [Google Scholar] [CrossRef] [PubMed]
Ijerph 19 00685 g001 550
Figure 1. Evidence-based recommendations with outcomes informed by evidence from Cochrane reviews in Danish National Clinical Guideline development from 2014–2021.
Figure 1. Evidence-based recommendations with outcomes informed by evidence from Cochrane reviews in Danish National Clinical Guideline development from 2014–2021.
Ijerph 19 00685 g001
Ijerph 19 00685 g002 550
Figure 2. Evidence-based recommendations informed by evidence from Cochrane reviews by topic in Danish National Clinical Guideline development from 2014 to 2021.
Figure 2. Evidence-based recommendations informed by evidence from Cochrane reviews by topic in Danish National Clinical Guideline development from 2014 to 2021.
Ijerph 19 00685 g002
Table
Table 1. Recommendations in Danish National Clinical Guideline development between 2014 and 2021.
Table 1. Recommendations in Danish National Clinical Guideline development between 2014 and 2021.
Total
Recommendations		Total
Evidence-Based Recommendations
n585374
Total guidelines56
Total recommendations585
Evidence-based recommendations37464%
Best practice 1 recommendations21136%0%
Recommendations informed by Cochrane review10618%28%
Categories of outcomes
(I) All critical and important outcomes covered 2285%7%
(II) All critical outcomes covered/informed 2366%10%
(III) At least one critical outcome covered/informed 2336%9%
(IV) At least one important but no critical outcome 292%2%
1 Recommendation by expert consensus only. 2 Non-overlapping categories.
Table
Table 2. Proportions of categories of outcomes informed by Cochrane reviews in Danish National Clinical Guideline development.
Table 2. Proportions of categories of outcomes informed by Cochrane reviews in Danish National Clinical Guideline development.
Recommendations Informed by Cochrane Review
Categories of Outcomesn106
(I) All critical and important outcomes covered 12826%
(II) All critical outcomes covered/informed 13634%
(III) At least one critical outcome covered/informed 13331%
(IV) At least one important but no critical outcome 198%
1 Non-overlapping categories.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Korfitsen, C.B.; Mikkelsen, M.-L.K.; Ussing, A.; Walker, K.C.; Rohde, J.F.; Andersen, H.K.; Tarp, S.; Händel, M.N. Usefulness of Cochrane Reviews in Clinical Guideline Development—A Survey of 585 Recommendations. Int. J. Environ. Res. Public Health 2022, 19, 685. https://doi.org/10.3390/ijerph19020685

AMA Style

Korfitsen CB, Mikkelsen M-LK, Ussing A, Walker KC, Rohde JF, Andersen HK, Tarp S, Händel MN. Usefulness of Cochrane Reviews in Clinical Guideline Development—A Survey of 585 Recommendations. International Journal of Environmental Research and Public Health. 2022; 19(2):685. https://doi.org/10.3390/ijerph19020685

Chicago/Turabian Style

Korfitsen, Christoffer Bruun, Marie-Louise Kirkegaard Mikkelsen, Anja Ussing, Karen Christina Walker, Jeanett Friis Rohde, Henning Keinke Andersen, Simon Tarp, and Mina Nicole Händel. 2022. "Usefulness of Cochrane Reviews in Clinical Guideline Development—A Survey of 585 Recommendations" International Journal of Environmental Research and Public Health 19, no. 2: 685. https://doi.org/10.3390/ijerph19020685

APA Style

Korfitsen, C. B., Mikkelsen, M. -L. K., Ussing, A., Walker, K. C., Rohde, J. F., Andersen, H. K., Tarp, S., & Händel, M. N. (2022). Usefulness of Cochrane Reviews in Clinical Guideline Development—A Survey of 585 Recommendations. International Journal of Environmental Research and Public Health, 19(2), 685. https://doi.org/10.3390/ijerph19020685

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop