The Emergency Medicine Facing the Challenge of Open Science

Abstract

(1) Background: The availability of research datasets can strengthen and facilitate research processes. This is specifically relevant in the emergency medicine field due to the importance of providing immediate care in critical situations as the very current Coronavirus (COVID-19) Pandemic is showing to the scientific community. This work aims to show which Emergency Medicine journals indexed in Journal Citation Reports (JCR) currently meet data sharing criteria. (2) Methods: This study analyzes the editorial policies regarding the data deposit of the journals in the emergency medicine category of the JCR and evaluates the Supplementary material of the articles published in these journals that have been deposited in the PubMed Central repository. (3) Results: It has been observed that 19 out of the 24 journals contained in the emergency medicine category of Journal Citation Reports are also located in PubMed Central (PMC), yielding a total of 5983 articles. Out of these, only 9.4% of the articles contain supplemental material. Although second quartile journals of JCR emergency medicine category have quantitatively more articles in PMC, the main journals involved in the deposit of supplemental material belong to the first quartile, of which the most used format in the articles is pdf, followed by text documents. (4) Conclusion: This study reveals that data sharing remains an incipient practice in the emergency medicine field, as there are still barriers between researchers to participate in data sharing. Therefore, it is necessary to promote dynamics to improve this practice both qualitatively (the quality and format of datasets) and quantitatively (the quantity of datasets in absolute terms) in research.

Dataset

Sixto-Costoya, A.; Aleixandre-Benavent, R.; Lucas-Dominguez, L.; Vidal-infer, A. Use and sharing of raw data in the Journal Citation Reports’ Emergency Medicine Category: Metrics and Journals including Supplementary material classification sorted by quartile of the JCR emergency medicine category. (Version 1) [Data set]. Zenodo. 2018. Available online: Available online: http://doi.org/10.5281/zenodo.2384871 (accessed on 20 February 2020).

1. Introduction

Due to the severity and urgency of the events faced by the emergency departments, they must provide immediate care in critical situations of illness and injury related to communicable and non-communicable diseases [1]. Both the need for rapid responses and the advantages that saving time can have for patients, make any research innovation highly useful at the same time that can contribute to solve the challenges related to global health [2]. As described in previous studies [3,4,5], open data implies that the data, including the data underlying the scientific publications, remain freely available online and can be used and shared. Data sharing is one of the existing practices in science used to save time and resources while ensuring the validation and reproduction of research [3]. This practice is currently promoted by the Open Science movement, which advocates that data should be as open as possible as closed as necessary [4,5]. Data sharing and open data are based on the commitment of researchers, funders, journals, and institutions to contribute to sharing research data with the rest of the scientific community, creating a mutual benefit for all parties [6]. In the scientific context, it is especially important for researchers the retrieval of raw research datasets to operate with, incluiding “statistics, results of experiments, measurements, observations resulting from fieldwork, survey results, interview recordings and images. The focus is on research data that is available in digital form” [7]. Specifically, when the raw data are referred to the medical context, it can be, for example, mathematical data, classifications and codes for diseases, pictures from imaging procedures, sensor data from biosignals or vital parameter measurements, biomaterial data from laboratory testing, administrative patient information, as well as audio-visual data, models, and visualizations [8].

In health sciences research, the advantages of sharing data are sound, being that discarded data can be used by other groups to produce new results, and the cycle of scientific production might be favored consequently [9,10]. Evidence of the usefulness of data sharing applied to emergency medicine (EM) includes the 2014 Ebola epidemic, where it was found that limited access to information related to the outbreak could have adversely affected the break of the crisis. However, the lesson was learned, and in 2016, the Zika virus problem was approached much more quickly, partly due to an increased use of data sharing, which meant a much more coordinated global action. Furthermore, a very recent example has been the rapid release to the public of the genome sequence of the new Coronavirus SARS-CoV-2 responsible for COVID-19 [11,12]. Related to this last crisis, which is being considered as a pandemic by the World Health Organization (WHO) while these lines are being written, there are already publications that consider the research data sharing as the basis of a specific public health action, including all the types of data related to health research from clinical trial to observational studies, operational research, genetic sequences, monitoring of disease control programs, survey results, etc. [13]. On the other hand, in the EM department context, models including data sharing with the police to prevent injuries have shown great results [14]. In this regard, the efforts for reusing data in the EM field are revealed as essential to achieve goal 3 "Ensure healthy lives and promote well-being for all at all ages" included among the Sustainable Development Goals by 2030 established by the United Nations [2].

Until now, there is not enough scientific literature to evaluate whether the research in the EM field involves raw data sharing in the same way that it has already been shown in other areas of the health sciences such as dentistry, stem cell research and addictions [15,16,17]. This work aims to evaluate the editorial policies related to data sharing of the EM category journals, as well as to analyze the quantity and quality of the data being shared.

2. Results

First, the 24 journals of the emergency medicine category of Web of Science (WoS) were organized in quartiles according to the Journal Citation Reports (JCR) ranking. After that, information about the journal title, publisher, impact factor, and openness criteria were analyzed. The information about openness criteria was classified as 1) the journal’s access modality (Open Acess (OA) journals free for authors and readers, OA journals with publication fees for authors, traditional journals with publication fees for readers, and hybrid journals, both traditional and OA with publication fees for authors); 2) storage policies in institutional or thematic repositories; 3) reuse policies after the publication of the article; 4) publication policies in official and/or author’s websites; and 5) statement of Supplementary material (SM). “Section 1”, “Section 2”, “Section 3” and “Section 4” refer to the general availability of the content of the article, while "5" is related to the raw data as SM. Regarding the journal’s access modality, Figure 1 shows that hybrid access was the most usual modality in journals from Q1, Q2 and Q3. Q4 journals did not show an evident preference.

Regarding the second, third and fourth openness criteria of each journal analyzed in this study (storage in thematic or institutional repositories, reusen e and publication on the website), it was revealed that “Accepted with conditions” was the most frequent option among Q1, Q2 and Q3 journals. Finally, the fifth criteria (statement of Supplementary Material (SM)) was apparently different. In this case, the “Accepted” option was the most frequent in Q1, Q2 and Q3. In Q4 journals, the acceptance or not of SM was mostly “Not specified” (

Table 1. OA characteristics of the Emergency Medicine journals according to their JCR quartile.

	Storage in Thematic or Institutional Repositories				Reuse				Publication on Website				Statement of SM
Q	A	NA	NS	AC	A	NA	NS	AC	A	NA	NS	AC	A	NA	NS	AC
Q1	1	1	1	3	1	2	0	3	1	2	0	3	5	0	1	0
Q2	1	0	0	5	1	0	0	5	1	0	0	5	5	0	1	0
Q3	0	0	0	6	0	0	0	6	0	0	0	6	4	0	2	0
Q4	2	1	1	2	1	2	1	2	2	1	1	2	2	0	4	0

Q: quartile, A: accepted, NA: not accepted, NS: not specified, AC: accepted with conditions.

.).

In regards to the articles retrieved in PubMed (PM) and PubMed Central (PMC), 20 out of the 24 journals contained in the EM category of JCR were indexed in PM (86,056 articles). 19 journals were also located in PMC, yielding a total of 5983 articles (7% of total in PM). Q2 journals had quantitatively more articles in PMC. Only 564 of the PMC articles from 13 journals contained SM (9.4% of the total in PMC) (

Table 2. Percentage of Emergency Medicine journal articles in PMC in relation to PubMed and the percentage of SM contained in PMC emergency medicine articles.

Quartile	PM Articles	PMC Articles	%PMC Articles (1)	Articles with SM	%Articles with SM (2)
Q1	27,646	1518	5.5%	253	16.7%
Q2	30,772	4206	13.7%	294	7.0%
Q3	21,643	253	1.3%	17	6.7%
Q4	5995	6	0.1%	0	3.0%
TOTAL	86,056	5983	7.0%	564	9.4%

PM: PubMed/Medline. PMC: PubMed Central. SM: Supplementary material. (1) Calculated from the total PM articles, (2) Calculated from the total PMC articles.

).

Based on the analysis of the SM content, pdf files (495, 49%) were the most used followed by text documents (doc/docx) (385, 38%) in comparison with spreadsheets or tabular data formats (xls/xlsx and csv) that were scarcer (Figure 2). The 94% of the spreadsheets files belonged to articles from Q1 and Q2 journals.

3. Discussion

The 2030 Agenda for Sustainable Development has proposed a plan of action for people, planet and prosperity composed by 17 development goals, which includes as goal 3 envisaging a world free of disease [2]. Currently, the rise of chronic non-communicable diseases, such as diabetes or high blood pressure, annual outbreaks of epidemics, and, ultimately, the growing aging population, have led to a rising demand for health care services, including EM services [18]. These services are considered to be one of the most important in health systems, as they often have to provide appropriate treatment at the site of a health emergency. Furthermore, the World Health Organization promotes the exchange of emergency medicine data as an important component in the public health approach to prevent injuries as a way to support the management of these situations not only related to the research context, but also to the daily practice in hospitals and other health centers [19,20].

The publication of SM alongside the article is a useful way to share data and provide extra information for researchers to replicate the study or use the data for secondary analysis [21]. Our study reveals that 16 of the 24 journals (67%) included in the EM category of JCR accepted inclusion of SM. These data coincide with those of other studies in the areas of dentistry [15] and substance abuse [17]. The study of the variable "reuse" shows that SM is accepted without conditions in only three journals (12.5%). In addition, there is a rise of hybrid journals, those that offer to authors an OA option with a fee per publication, apart from the traditional payment method for readers. The analysis of these criteria shows that there is not a direct relation between the OA criteria and the acceptance of SM by the journals. Open access to scientific literature and open data including the data underlying publications are included in the Open Science movement. Interestingly the journals of this study generally accept SM regardless of whether they are open access, hybrid or traditional. Based on the analysis of SM for articles in PMC, the most frequent formats were pdf and text files followed to a lesser extent by documents commonly considered as raw data files such as spreadsheets and tabular data formats, images and videos [22]. Although Q2 journals of JCR category had quantitatively more articles in PMC, the main journals involved in the deposit of supplemental material belong to Q1 (16.7%), which may indicate a positive correlation between being a top JCR journal and having an open data policy.

4. Methods

This study was conducted in four phases. First, the websites of the 24 journals included in the “Emergency Medicine” category of the 2016 Journal Citation Reports (JCR) were analyzed. Journals were classified in quartiles according to the JCR ranking. This information was collected in May 2018. According to our previous studies [15,16,17], the following data were retrieved from each journal: a) journal title; b) publisher; c) journal impact factor and quartile (unit used to measure the position of journals of a particular category, ordered from highest to lowest impact factor); and d) information about openness criteria, which was consulted in each journal’s website. The information about openness criteria was specifically 1) the journal’s access modality; 2) storage policies in institutional or thematic repositories; 3) reuse policies after the publication of the article; 4) publication policies in official and/or author’s websites; and 5) statement of Supplementary material (SM). “Section 1”, “Section 2”, “Section 3” and “Section 4” refer to the general availability of the content of the article, while “5” is related to the raw data as SM. The information found in each section was classified as: “accepted (A)”; “not accepted (NA)”; “accepted with conditions (AC)” when, in the case of hybrid journals, it was only accepted when the option OA was chosen; and “not specified (NE)”, when no reference was found.

Regarding to the fifth section (statement of Supplementary material), this study is only focused on analyzing the raw data shared as Supplementary material (type of material that can be reused and shared). Following other studies [23,24], this modality is one of the three different ways currently existing to share data in the scientific context: i) adding the raw data as Supplementary material to the publication, ii) being available upon request to the authors and iii) uploading the raw data to a data repository.

The second phase consisted of a search for the articles belonging to the JCR Emergency Medicine journals in PubMed/Medline (PM) to assess the number of records (journal articles, clinical trials, reviews…) included in this database.

A third phase was conducted in PubMed Central (PMC), the digital repository of the US National Institutes of Health. In this repository, the type of SM contained in the articles of each journal was analyzed through a search equation developed for this purpose (i.e., (“title of the journal” [journal]) AND (<Supplementary-material> OR supplemental information)). 13 JCR journals out of 24 were found in PMC with SM.

The fourth phase consisted of a qualitative analysis of the format of each file attached as SM. The types of files were identified as: pdf, doc/docx, jpg/tif/png, xls/xlsx/csv, mov/mp4/wmv/mpg/avi, ppt/pptx, zip.

5. Conclusions

Open Access and data sharing represent the existing practices used to make science available, increase reproducibility, and save time and resources. The collection of data sets related to the provision of emergency health care and their subsequent integration, analysis and interpretation can help to understand the complex mechanisms involved in emergency medical care and the functioning of these departments contributing to accelerate the pace of progress made in fighting critical situations of illness and injury derived from accidents, communicable and non-communicable diseases as well as outbreaks and epidemics

There is a need to develop technologies that enable the integration of dispersed emergency health care data from a variety of sources such as emergency departments’ information systems, electronic medical records, and data accompanying articles as supplemental material or deposited in repositories. Aggregation of these data can detect the relationships between diseases and the risk factors that produce them. However, there is a need for appropriate data selection that follow properly the FAIR (Findable, Accessible, Interoperable, Reusable) principles [25] and the development of tools to ensure the consistency and validity of data from a variety of sources.