Accuracy of the qSOFA Score and RED Sign in Predicting Critical Care Requirements in Patients with Suspected Infection in the Emergency Department: A Retrospective Observational Study
by
, ,
Jong Eun Park
1,†, 
Sung Yeon Hwang
1,†,
Ik Joon Jo
1,
Min Seob Sim
1,
Won Chul Cha
1,
Hee Yoon
1,
Tae Rim Kim
1,
Gun Tak Lee
1,
Hye Seung Kim
2,
InSuk Sohn
2 and
Tae Gun Shin
1,* 1
Department of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Korea
2
Statistics and Data Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul 06351, Korea
*
Author to whom correspondence should be addressed.
†
Jong Eun Park and Sung Yeon Hwang contributed equally to this work.
Medicina 2020, 56(1), 42; https://doi.org/10.3390/medicina56010042
Submission received: 26 November 2019
/
Revised: 12 January 2020
/
Accepted: 16 January 2020
/
Published: 19 January 2020
(This article belongs to the Section Emergency Medicine)
Abstract
:Background and objectives: We aimed to compare the accuracy of positive quick sequential organ failure assessment (qSOFA) scores and the RED sign in predicting critical care requirements (CCRs) in patients with suspected infection who presented to the emergency department (ED). Materials and Methods: In this retrospective observational study, we examined adult patients with suspected infection in the ED from June 2018 to September 2018. A positive qSOFA (qSOFA+) was defined as the presence of ≥2 of the following criteria: altered mental status (AMS), systolic blood pressure (SBP) < 100 mmHg, and respiratory rate (RR) ≥ 22 breaths/min. A positive RED sign (RED sign+) was defined as the presence of at least one of the RED sign criteria: AMS, skin mottling, SBP < 90 mmHg, heart rate >130 beats/min, or RR > 30 breaths/min. A qSOFA/RED+ was defined as the presence of qSOFA+ or RED+. We applied these tools twice using the initial values upon ED arrival and all values within 2 h after ED arrival. The accuracy of qSOFA+, RED+, and qSOFA/RED+ in predicting CCR was assessed. Results: Data from 5353 patients with suspected infection were analyzed. The area under the receiver operating characteristic curve (AUC) of RED+ (0.67, 95% confidence interval [CI]: 0.65–0.70) and that of qSOFA/RED+ (0.68, 95% CI: 0.66–0.70, p < 0.01) were higher than the AUC of qSOFA+ (0.59, 95% CI: 0.57–0.60) in predicting CCR on ED arrival. The qSOFA/RED+ within 2 h showed the highest accuracy (AUC 0.72, 95% CI: 0.70–0.75, p < 0.001). Conclusions: The accuracy of the RED sign in predicting CCR in patients with suspected infection who presented at ED was better than that of qSOFA. The combined use of the RED sign and qSOFA (positive qSOFA or RED sign) showed the highest accuracy.
1. Introduction
Sepsis is a major health problem affecting millions of people each year and accounts for high mortality among hospitalized patients worldwide [1]. A recent meta-analysis reported a global estimate of 31.5 million cases of sepsis with potentially 5.3 million deaths annually [2]. The overall hospital mortality rate in patients with sepsis ranges between 20% and 40%, reaching as high as 50% in patients with septic shock [3,4,5]. Efforts to improve the survival rate of patients with sepsis and septic shock are still needed [6,7]. Identifying patients at a high risk of sepsis and septic shock would be the first step to initiate proper therapeutic strategies [8].
The quick sequential organ failure assessment (qSOFA) is a simple scoring system using three physiological parameters (respiratory rate (RR) ≥ 22 breaths/min, altered mental status (AMS), and systolic blood pressure (SBP) ≤100 mmHg (1 point each; score range of 0–3 points)). It was originally proposed as a screening tool to identify patients with suspected infection outside the intensive care unit (ICU) who are at a high risk for poor outcomes, including hospital mortality, in accordance with the new sepsis-3 definition [8].
However, the predictive accuracy of qSOFA might be limited according to recent studies, particularly in the initial evaluation of high-risk patients in the emergency department (ED) [9,10]. In the original qSOFA study, ED populations were not analyzed separately from the larger study population, and the poor discriminative ability of qSOFA has raised concerns about its role for ED patients requiring early recognition and timely intervention [11,12]. Additionally, an extreme variation in a single physiological parameter (RED sign) is not considered to be positive in the qSOFA system. For example, hypotension, severe tachypnea or tachycardia, altered mentation, and skin mottling are signs of critically ill patients with sepsis, and patients with these conditions can be easily examined at their bedside and should therefore not be overlooked. Finally, qSOFA focused on the in-hospital mortality and ICU length of stay as outcomes. For initial evaluation in ED, however, screening patients requiring early critical interventions such as vasopressor use, mechanical ventilator use (MV), renal replacement therapy (RRT), or ICU admission may be more appropriate than predicting in-hospital mortality or ICU stay [13].
This study aimed to compare the accuracy of positive qSOFA and the RED sign in predicting critical care requirements (CCRs) in patients with suspected infection who presented to the ED. We also aimed to evaluate whether the predictive accuracy of qSOFA improved when combined with the RED sign.
2. Materials and Methods
2.1. Study Design and Setting
This was a single-center, retrospective, observational study in patients with suspected infection who presented to the ED at Samsung Medical Center (a 1960-bed, university-affiliated, tertiary care referral hospital located in a metropolitan city with an annual census over 70,000). The study period was from June 2018 through September 2018. This study was approved on 12/14/2018 by the institutional review board of Samsung Medical Center (IRB number, 2018-12-053); informed consent was waived because this study was retrospective and observational in nature, and patients’ data were anonymized. This study followed the Standards for Reporting of Diagnostic Accuracy Studies guidelines [14].
2.2. Study Population
Patients aged 18 years or older and who had suspected infection upon arrival to the ED were included in the study. Patients who were younger than 18 years, had limitations on invasive care (e.g., patients who had previously signed a do-not-resuscitate order), were transferred to another hospital, or had inadequate data on electronic medical records (EMR) were excluded.
2.3. Definition and Data Collection
Suspected infection was defined as cases where body fluid culture and antibiotic therapy were performed. The qSOFA score consists of the following three clinical variables: SBP ≤ 100 mmHg, RR ≥ 22 breaths/min, and AMS. The presence of two or more of these criteria was defined as a positive qSOFA score (qSOFA+). The RED sign consists of the following five clinical variables: SBP < 90 mmHg, HR > 130 beats/min, RR > 30 breaths/min, presence of skin mottling on the abdomen or knee, and AMS. The presence of at least one of the RED sign criteria was defined as a positive RED sign (RED+). A positive qSOFA or RED sign (qSOFA/RED+) was defined as the presence of qSOFA+ or RED+. AMS was defined as a Glasgow Coma Scale score of <15 or an alert, voice, pain, and unresponsive scale rating other than “alert” that was documented as a new-onset change from baseline mental status. Any deterioration from baseline mental status in patients who were not fully alert was also regarded as AMS.
Eligible cases were electronically identified based on the definition of suspected infection. The following data were extracted from the hospital database: age, sex, comorbidities, vital signs, mental status, presence of skin mottling, suspected infection focus, initial laboratory tests, survival data, and therapeutic interventions including vasopressor use, MV use, RRT, and ICU admission.
Data regarding AMS and skin mottling were extracted from the initial ED physician’s EMR using a predefined template, and then, initial values were used to quantify these variables. We determined the qSOFA score and the RED sign twice for each patient. The qSOFA or RED sign was first evaluated on ED arrival according to the first vital signs and assessment data. The qSOFA or RED sign within 2 h was considered positive if the criteria were met based on the initial or follow-up values within 2 h after ED arrival. We used the simultaneous values instead of the worst ones for the qSOFA score.
2.4. Outcome Measures
The primary outcome was CCRs, which is a composite of vasopressor use, MV use, RRT, and ICU admission. The secondary outcomes were in-hospital and 48-hour mortality after ED arrival.
2.5. Statistical Analysis
Standard descriptive statistics were analyzed for all variables. The results are expressed as medians with interquartile ranges (IQR) for continuous variables and as the number of patients with percentages for categorical data. Continuous variables were analyzed using Wilcoxon rank-sum tests, while categorical variables were analyzed using chi-square tests. The prognostic performance of qSOFA+, RED+, and qSOFA/RED+ for outcomes was assessed as sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, area under the receiver operating curve (AUC), and their corresponding 95% confidence interval (CI). For a comparison of the AUC of two diagnostic modalities, we used a nonparametric approach for dependent receiver operating characteristic curves [15]. Prognostic parameters such as positive and negative predictive values were also calculated using the Wilson’s method and compared using McNemar’s test and Bennett’s test. A p-value < 0.05 was considered significant and was corrected using Bonferroni’s method for multiple testing. STATA (version 13.0; StataCorp., College Station, TX, USA) was used for statistical analysis.
3. Results
3.1. Baseline Characteristics
A total of 5353 patients with suspected infection who presented to the ED were included (Figure 1), of whom 495 (9.3%) received critical care interventions as the primary outcome. The baseline characteristics of all patients and a comparison between patients who received critical care and those who did not require critical care are shown in Table 1. The median age was 58 years (IQR 47–71), and 51.3% (n = 2745) were men. The most common cause of infection was intra-abdominal infection (36.8%), followed by respiratory infection (25.5%). In-hospital mortality was 3.3% (n = 179), while the 48-h mortality was 0.7% (n = 35).
The proportions of patients with qSOFA+, RED+, and qSOFA/RED+ were 4.5%, 12.1%, and 13.4%, respectively, upon ED arrival, and 6.8%, 14.8%, and 16.7%, respectively, within 2 h. All three indexes were more frequently observed in the critical care group than in the non-critical care group (At ED arrival: qSOFA+ 20.0% vs. 3.0%, RED+ 43.4% vs. 8.9%, qSOFA/RED + 45.9% vs. 10.1%; within 2 h after ED arrival: qSOFA+ 31.7% vs. 4.2%, RED+ 53.7% vs. 10.8%, qSOFA/RED+ 57.4% vs. 12.5%, respectively).
3.2. Outcomes
The primary and secondary outcomes are presented in Table 2.
The qSOFA/RED+ showed significantly increased sensitivity (46%, 95% CI: 42–50% upon ED arrival; 57%, 95% CI: 53–62% within 2 h) for predicting CCRs compared with qSOFA+ or RED+. The AUC of RED+ upon ED arrival was greater than that of qSOFA+ upon ED arrival (0.673, 95% CI: 0.650–0.695 vs. 0.585, 95% CI: 0.567–0.603, p < 0.001 corrected by Bonferroni’s method) for predicting CCRs (Table 3, Figure 2). The AUC of qSOFA/RED+ upon ED arrival was also higher than that of RED+, but it was not statistically significant (p = 0.154 corrected by Bonferroni’s method). The AUC of RED+ within 2 h for predicting CCRs was also significantly higher than that of qSOFA+ within 2 h (0.715, 95% CI: 0.692–0.737 vs. 0.637, 95% CI: 0.617–0.658, p < 0.001 corrected by Bonferroni’s method) (Table 3, Figure 3). The AUC of qSOFA/RED+ within 2 h was higher than that of RED+ within 2 h, but the statistical significance was marginal (0.724, 95% CI: 0.702–0.747 vs. 0.715, 95% CI: 0.692–0.737, p = 0.056 corrected by Bonferroni’s method). Receiver operating characteristic curves for predicting 48-h and in-hospital mortality are elsewhere (see Supplementary Materials, Figures S1 and S2).
4. Discussion
In this study, we compared the accuracy of positive qSOFA and RED sign in predicting the need for critical care in patients with suspected infection who presented to the ED. RED sign showed better prognostic accuracy for predicting CCRs than qSOFA. It also showed better performance in predicting in-hospital mortality and 48-h mortality than qSOFA. The combination of qSOFA and RED sign was the most accurate model for predicting outcomes.
The original qSOFA study was conducted among patients from outside the ICU, and it revealed that the rate of patients with severe infection was relatively low and that ED populations were not analyzed separately [8]. Several studies investigated the role of qSOFA as a screening tool in the ED. Moskowitz et al. [13] compared qSOFA and systemic inflammatory response syndrome criteria in predicting the need for critical care intervention (vasopressor use, assisted ventilation, continuous insulin infusion, ≥4000 mL intravenous fluid within 12 h of ICU admission, placement of invasive catheters, or RRT) in patients with suspected infection who presented to the ED. They showed that the sensitivity of qSOFA+ was only 13% in predicting the need for critical care intervention. Among patients with a qSOFA of 1, 23.5% received a critical care intervention. Williams et al. [12] demonstrated that qSOFA+ had high specificity but poor sensitivity (96.1%; 95% CI, 95.7–96.6% and 29.9%; 95% CI, 27.9–31.8%, respectively), with an AUC of 0.73 (95% CI, 0.72–0.74) for organ dysfunction. Henning et al. [16] performed secondary analysis of three prospectively collected observational cohorts of ED patients with infection. They showed that qSOFA+ had a sensitivity of 52% (95% CI 46–57%) and specificity of 86% (95% CI 85–87%) in predicting mortality. One study showed relatively high sensitivity (70%, 95% CI 59–80) in predicting in-hospital mortality; however, the worst value during the ED stay was used to calculate the qSOFA score in the study [17]. Our study was consistent with previous studies and suggests that qSOFA alone has limitations for evaluating patients with suspected infection in the ED. Therefore, qSOFA should be interpreted with caution [18,19].
Various studies have been conducted to improve the performance of qSOFA. Shetty et al. [20] performed a retrospective cohort study of suspected or proven sepsis patients presenting to ED. They added lactate level (≥2 mmol/L) to the qSOFA score of ≥2 and showed improved performance compared to qSOFA of ≥2 alone for identifying patients at a risk for in-hospital mortality or prolonged ICU stay (AUC 0.74 [95% CI 0.73–0.74] vs. 0.68 [95% CI 0.68–0.69]; sensitivity 65.5% [95% CI 62.6–68.4] vs. 47.6% [95% CI 44.6–50.6]). We used an extreme physiologic parameter or physical finding, which is not considered as positive in the qSOFA scoring system. The RED sign showed better performance than qSOFA in predicting CCRs in our study and can be easily detected at the bedside. RED signs should be a trigger for rapid intervention in patients with suspected or confirmed infection. Considering the limitations of qSOFA and our results, we suggest that a reasonable and practical approach may be to screen for a RED sign at the initial evaluation and check the qSOFA and RED signs serially if there is no abnormal finding.
There are several strengths to our study. First, our study focused on the composite outcome of CCRs, which is an important issue for ED physicians in starting early aggressive management for sepsis. Second, we extracted objective data from EMRs of a large patient population. Thus, we could include the majority of adult patients who had suspected infection regardless of disease severity, and very few patients were excluded from the study. Therefore, our study could show rigorous statistics and significant findings. Third, we investigated qSOFA and RED sign using the initial and follow-up data from the early period. We did not use the worst value during the entire ED stay because its clinical application at the bedside could be limited.
This study also had some limitations. First, this was a single-center study conducted in the ED of a tertiary academic institution. The incidence rates of outcomes may be different from those of other settings. Hence, the generalizability of these results to other settings is limited, and external validation is required. Second, this study had inherent limitations as a retrospective observational study. Most of the data were electronically extracted from the hospital database. Notably, unlike vital signs that were frequently reassessed, data including mental status and skin mottling might be relatively lacking. However, adding initial assessment of these variables and recording it on the EMR was mandatory in our institution. In the original qSOFA study, over 75% of patients outside of the ICU in the derivation cohort lacked GCS data. Third, the study period was brief. However, our institution, as a tertiary referral hospital located in an urban area, had a large volume of patients.
5. Conclusions
Our study revealed that the accuracy of RED sign in predicting CCRs and mortality in patients with suspected infection who presented at ED was better than that of qSOFA. The combined use of RED sign and qSOFA (positive qSOFA or RED sign) showed the highest diagnostic accuracy, and it can be used as a more accurate screening tool for identifying high-risk patients who will need critical care intervention compared to the qSOFA or RED sign alone.
Supplementary Materials
The following are available at online at https://www.mdpi.com/1648-9144/56/1/42/s1. Figure S1: Receiver operating characteristic curve for predicting in-hospital mortality. Figure S2: Receiver operating characteristic curve for predicting 48-h mortality.
Author Contributions
Conceptualization, T.G.S. and J.E.P.; Methodology, T.G.S., S.Y.H., and J.E.P.; Formal Analysis, T.G.S., S.Y.H., H.S.K., I.S., and J.E.P.; Investigation, T.G.S., S.Y.H., J.E.P., I.J.J., and M.S.S.; Resources, I.J.J., W.C.C., M.S.S., H.Y., T.R.K., and G.T.L.; Software, H.S.K., I.S., Data Curation, T.R.K., G.T.L., I.J.J., W.C.C., M.S.S., H.Y., and T.G.S.; Writing—Original Draft Preparation, T.G.S., S.Y.H., and J.E.P.; Writing—Review & Editing, all authors; Project Administration, T.G.S. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Walkey, A.J.; Lagu, T.; Lindenauer, P.K. Trends in sepsis and infection sources in the United States. A population-based study. Ann. Am. Thorac. Soc. 2015, 12, 216–220. [Google Scholar] [CrossRef] [PubMed]
- Fleischmann, C.; Scherag, A.; Adhikari, N.K.; Hartog, C.S.; Tsaganos, T.; Schlattmann, P.; Angus, D.C.; Reinhart, K. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am. J. Respir. Crit. Care Med. 2016, 193, 259–272. [Google Scholar] [CrossRef] [PubMed]
- Rhee, C.; Dantes, R.; Epstein, L.; Murphy, D.J.; Seymour, C.W.; Iwashyna, T.J.; Kadri, S.S.; Angus, D.C.; Danner, R.L.; Fiore, A.E.; et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs. Claims Data, 2009–2014. JAMA 2017, 318, 1241–1249. [Google Scholar] [CrossRef] [PubMed]
- Marik, P.E.; Khangoora, V.; Rivera, R.; Hooper, M.H.; Catravas, J. Hydrocortisone, Vitamin C, and Thiamine for the Treatment of Severe Sepsis and Septic Shock: A Retrospective Before-After Study. Chest 2017, 151, 1229–1238. [Google Scholar] [CrossRef] [PubMed]
- Venkatesh, B.; Finfer, S.; Cohen, J.; Rajbhandari, D.; Arabi, Y.; Bellomo, R.; Billot, L.; Correa, M.; Glass, P.; Harward, M.; et al. Adjunctive Glucocorticoid Therapy in Patients with Septic Shock. N. Engl. J. Med. 2018, 378, 797–808. [Google Scholar] [CrossRef] [PubMed]
- Seymour, C.W.; Gesten, F.; Prescott, H.C.; Friedrich, M.E.; Iwashyna, T.J.; Phillips, G.S.; Lemeshow, S.; Osborn, T.; Terry, K.M.; Levy, M.M. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N. Engl. J. Med. 2017, 376, 2235–2244. [Google Scholar] [CrossRef] [PubMed]
- Rhodes, A.; Evans, L.E.; Alhazzani, W.; Levy, M.M.; Antonelli, M.; Ferrer, R.; Kumar, A.; Sevransky, J.E.; Sprung, C.L.; Nunnally, M.E.; et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med. 2017, 43, 304–377. [Google Scholar] [CrossRef] [PubMed]
- Seymour, C.W.; Liu, V.X.; Iwashyna, T.J.; Brunkhorst, F.M.; Rea, T.D.; Scherag, A.; Rubenfeld, G.; Kahn, J.M.; Shankar-Hari, M.; Singer, M.; et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016, 315, 762–774. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wang, J.Y.; Chen, Y.X.; Guo, S.B.; Mei, X.; Yang, P. Predictive performance of quick Sepsis-related Organ Failure Assessment for mortality and ICU admission in patients with infection at the ED. Am. J. Emerg. Med. 2016, 34, 1788–1793. [Google Scholar] [CrossRef] [PubMed]
- Kim, K.S.; Suh, G.J.; Kim, K.; Kwon, W.Y.; Shin, J.; Jo, Y.H.; Lee, J.H.; Lee, H. Quick Sepsis-related Organ Failure Assessment score is not sensitive enough to predict 28-day mortality in emergency department patients with sepsis: A retrospective review. Clin. Exp. Emerg. Med. 2019, 6, 77–83. [Google Scholar] [CrossRef] [PubMed]
- Hwang, S.Y.; Jo, I.J.; Lee, S.U.; Lee, T.R.; Yoon, H.; Cha, W.C.; Sim, M.S.; Shin, T.G. Low Accuracy of Positive qSOFA Criteria for Predicting 28-Day Mortality in Critically Ill Septic Patients During the Early Period After Emergency Department Presentation. Ann. Emerg. Med. 2018, 71, 1–9. [Google Scholar] [CrossRef] [PubMed]
- Williams, J.M.; Greenslade, J.H.; McKenzie, J.V.; Chu, K.; Brown, A.F.T.; Lipman, J. Systemic Inflammatory Response Syndrome, Quick Sequential Organ Function Assessment, and Organ Dysfunction: Insights From a Prospective Database of ED Patients With Infection. Chest 2017, 151, 586–596. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Moskowitz, A.; Patel, P.V.; Grossestreuer, A.V.; Chase, M.; Shapiro, N.I.; Berg, K.; Cocchi, M.N.; Holmberg, M.J.; Donnino, M.W. Quick Sequential Organ Failure Assessment and Systemic Inflammatory Response Syndrome Criteria as Predictors of Critical Care Intervention Among Patients With Suspected Infection. Crit. Care Med. 2017, 45, 1813–1819. [Google Scholar] [CrossRef] [PubMed]
- Cohen, J.F.; Korevaar, D.A.; Altman, D.G.; Bruns, D.E.; Gatsonis, C.A.; Hooft, L.; Irwig, L.; Levine, D.; Reitsma, J.B.; de Vet, H.C.; et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: Explanation and elaboration. BMJ Open 2016, 6, e012799. [Google Scholar] [CrossRef] [PubMed]
- DeLong, E.R.; DeLong, D.M.; Clarke-Pearson, D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988, 44, 837–845. [Google Scholar] [CrossRef] [PubMed]
- Henning, D.J.; Puskarich, M.A.; Self, W.H.; Howell, M.D.; Donnino, M.W.; Yealy, D.M.; Jones, A.E.; Shapiro, N.I. An Emergency Department Validation of the SEP-3 Sepsis and Septic Shock Definitions and Comparison with 1992 Consensus Definitions. Ann. Emerg. Med. 2017, 70, 544–552. [Google Scholar] [CrossRef] [PubMed]
- Freund, Y.; Lemachatti, N.; Krastinova, E.; Van Laer, M.; Claessens, Y.E.; Avondo, A.; Occelli, C.; Feral-Pierssens, A.L.; Truchot, J.; Ortega, M.; et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. JAMA 2017, 317, 301–308. [Google Scholar] [CrossRef] [PubMed]
- Jiang, J.; Yang, J.; Mei, J.; Jin, Y.; Lu, Y. Head-to-head comparison of qSOFA and SIRS criteria in predicting the mortality of infected patients in the emergency department: A meta-analysis. Scand. J. Trauma Resusc. Emerg. Med. 2018, 26, 56. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Churpek, M.M.; Snyder, A.; Han, X.; Sokol, S.; Pettit, N.; Howell, M.D.; Edelson, D.P. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. Am. J. Respir. Crit. Care Med. 2017, 195, 906–911. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Shetty, A.; MacDonald, S.P.; Williams, J.M.; van Bockxmeer, J.; de Groot, B.; Esteve Cuevas, L.M.; Ansems, A.; Green, M.; Thompson, K.; Lander, H.; et al. Lactate ≥2 mmol/L plus qSOFA improves utility over qSOFA alone in emergency department patients presenting with suspected sepsis. Emerg. Med. Australas. 2017, 29, 626–634. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Flowchart of the study process. ED, emergency department; qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+.
Figure 1.
Flowchart of the study process. ED, emergency department; qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+.
Figure 2.
Receiver operating characteristic curve for predicting critical care requirements on ED arrival. (A) qSOFA+, (B) RED+, and (C) qSOFA/RED+. qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+; AUC, area under the receiver operating characteristic; ED, emergency department.
Figure 2.
Receiver operating characteristic curve for predicting critical care requirements on ED arrival. (A) qSOFA+, (B) RED+, and (C) qSOFA/RED+. qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+; AUC, area under the receiver operating characteristic; ED, emergency department.
Figure 3.
Receiver operating characteristic curve for predicting critical care requirements within 2 h after ED arrival. (A) qSOFA+, (B) RED+, and (C) qSOFA/RED+. qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+; AUC, area under the receiver operating characteristic; ED, emergency department.
Figure 3.
Receiver operating characteristic curve for predicting critical care requirements within 2 h after ED arrival. (A) qSOFA+, (B) RED+, and (C) qSOFA/RED+. qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+; AUC, area under the receiver operating characteristic; ED, emergency department.
Table 1.
Baseline characteristics of patients.
| Total (n = 5353) | Critical Care (n = 495) | No-Critical Care (n = 4858) | p-Value | |
|---|---|---|---|---|
| Age (year) | 58 (47–71) | 64 (55–75) | 60 (46–71) | <0.001 |
| Male sex | 2745 (51.3) | 287 (58.0) | 2458 (50.6) | 0.002 |
| Infection focus | ||||
| Lung | 1365 (25.5) | 163 (32.9) | 1202 (24.7) | <0.001 |
| Abdomen | 1969 (36.8) | 179 (36.2) | 1790 (36.9) | 0.763 |
| Urinary tract | 743 (13.9) | 65 (13.1) | 678 (13.9) | 0.613 |
| Bone/soft tissue | 480 (9) | 19 (4) | 461 (9.5) | <0.001 |
| Neutropenic fever | 438 (8.1) | 34 (6.9) | 404 (8.3) | 0.263 |
| Others | 261 (4.9) | 39 (7.8) | 222 (4.5) | 0.001 |
| Unknown | 179 (3.3) | 9 (1.8) | 170 (3.5) | 0.048 |
| Laboratory test | ||||
| Lactate (mmol/L) | 1.5 (1.1–2.1) | 2.3 (1.6–4) | 1.4 (1.1–1.9) | <0.001 |
| CRP (mg/dL) | 4.1 (1.1–9.8) | 7.9 (2.4–18.3) | 3.8 (1–9.1) | <0.001 |
| Vital sign, upon ED arrival | ||||
| SBP | 120 (104–137) | 110 (89–133) | 120 (105–137) | <0.001 |
| HR | 97 (83–110) | 106 (88–123) | 96 (83–109) | <0.001 |
| RR | 18 (18–20) | 20 (18–24) | 18 (18–20) | <0.001 |
| Vital sign, 2 h after ED arrival | ||||
| SBP | 115 (103–131) | 104 (91–125) | 116 (104–132) | <0.001 |
| HR | 90 (78–102) | 99 (85–116) | 89 (78–101) | <0.001 |
| RR | 18 (16–20) | 20 (18–24) | 18 (16–20) | <0.001 |
| qSOFA+ upon ED arrival | 243 (4.5) | 99 (20.0) | 144 (3.0) | <0.001 |
| qSOFA+ within 2 h after ED arrival | 363 (6.8) | 157 (31.7) | 206 (4.2) | <0.001 |
| *RED+ upon ED arrival | 649 (12.1) | 215 (43.4) | 434 (8.9) | <0.001 |
| SBP < 90 mmHg | 207 (2.4) | 119 (26.8) | 87 (4.0) | <0.001 |
| HR > 130 beats/min | 298 (3.2) | 69 (15.9) | 116 (5.3) | <0.001 |
| RR > 30 breaths/min | 90 (1.0) | 18 (4.2) | 21 (1.0) | <0.001 |
| Skin mottling | 10 (0.2) | 7 (1.4) | 3 (0.1) | <0.001 |
| Altered mental status | 80 (1.5) | 36 (7.3) | 44 (0.9) | <0.001 |
| RED+ within 2 h after ED arrival | 790 (14.8) | 266 (53.7) | 524 (10.8) | <0.001 |
| SBP < 90 mmHg | 197 (2.1) | 102 (23) | 44 (2.0) | <0.001 |
| HR > 130 beats/min | 83 (0.9) | 50 (11.3) | 33 (1.5) | <0.001 |
| RR > 30 breaths/min | 64 (0.7) | 30 (6.8) | 34 (1.6) | <0.001 |
| qSOFA/RED+ upon ED arrival | 718 (13.4) | 227 (45.9) | 491 (10.1) | <0.001 |
| qSOFA/RED+ within 2 h after ED arrival | 893 (16.7) | 284 (57.4) | 609 (12.5) | <0.001 |
| 48-h mortality | 35 (0.65) | 24 (4.9) | 11 (0.2) | <0.001 |
| In-hospital mortality | 177 (3.3) | 83 (16.8) | 94 (1.9) | <0.001 |
The data are presented as mean ± standard deviations, median (interquartile ranges), or numbers (%). CRP, C-reactive protein; ED, emergency department; SBP, systolic blood pressure; HR, heart rate; RR, respiratory rate; qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, positive qSOFA or RED sign. * The presence of at least one of the RED sign criteria (SBP < 90 mmHg, HR > 130 beats/min, RR > 30 breaths/min, presence of skin mottling and altered mental status) was defined as positive RED sign.
Table 2.
Primary and secondary outcomes.
| Outcomes | Total | On ED Arrival | Within 2 h after ED Arrival | ||||
|---|---|---|---|---|---|---|---|
| qSOFA+ (n = 243) | * RED+ (n = 649) | qSOFA/RED+ (n = 718) | qSOFA+ (n = 363) | RED+ (n = 790) | qSOFA/RED+ (n = 893) | ||
| Critical care requirements † | 495 (9.3) | 99 (40.7) | 215 (33.1) | 227 (31.6) | 157 (43.3) | 266 (33.7) | 284 (31.8) |
| Vasopressor use | 426 (8) | 93 (38.3) | 206 (31.7) | 216 (30.1) | 149 (41.0) | 253 (32.0) | 268 (30.0) |
| MV use ‡ | 144 (2.7) | 34 (14.0) | 64 (9.9) | 67 (9.3) | 46 (12.7) | 77 (9.7) | 82 (9.2) |
| RRT § | 27 (0.5) | 10 (4.1) | 15 (2.3) | 15 (2.1) | 12 (3.3) | 19 (2.4) | 20 (2.2) |
| ICU admission | 258 (4.8) | 55 (22.6) | 106 (16.3) | 113 (15.7) | 86 (23.7) | 133 (16.8) | 143 (16.0) |
| 48-h mortality | 35 (0.65) | 14 (5.7) | 19 (2.9) | 20 (2.8) | 20 (5.5) | 24 (3.0) | 24 (2.7) |
| In-hospital mortality | 177 (3.3) | 37 (15.2) | 68 (10.5) | 78 (10.9) | 53 (14.6) | 83 (10.5) | 93 (10.4) |
* The RED sign consists of the following five clinical variables: systolic blood pressure < 90 mmHg, heart rate > 130 beats/min, respiratory rate > 30 breaths/min, presence of skin mottling on the abdomen or knee, and altered mental status. The presence of at least one of the RED sign criteria was defined as a positive RED sign. † A composite of vasopressor use, MV use, RRT, or ICU admission. ‡ Use of MV within 24 h after ED arrival. § RRT within 24 h after ED arrival. ED, emergency department; qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+; ICU, intensive care unit; MV, mechanical ventilator; RRT, renal replacement therapy.
Table 3.
Prognostic performance of scoring systems for predicting critical care requirements and mortality in patients with suspected infection.
Table 3.
Prognostic performance of scoring systems for predicting critical care requirements and mortality in patients with suspected infection.
| Sensitivity, % (95% CI) | Specificity, % (95% CI) | PPV, % (95% CI) | NPV, % (95% CI) | Accuracy, % (95% CI) | AUC (95% CI) | |||
|---|---|---|---|---|---|---|---|---|
| Requirement of Critical Care | ||||||||
| On ED arrival | qSOFA+ | 20 (17–40) | 97 (97–97) | 41 (35–47) | 92 (91–93) | 90 (89–91) | 0.585 (0.567–0.603) | |
| RED+ | 43 (39–48) * | 91 (90–92) * | 33 (30–37) * | 94 (93–95) * | 86 (86–88) * | 0.673 (0.650–0.695) * | ||
| qSOFA/RED+ | 46 (42–50) † | 90 (89–91) † | 32 (28–35) † | 94 (94–95) † | 86 (84–87) † | 0.679 (0.656–0.701) | ||
| Within 2 h after ED arrival | qSOFA+ | 32 (27–36) | 96 (95–96) | 43 (38–48) | 93 (93–94) | 90 (89–91) | 0.637 (0.617–0.658) | |
| RED+ | 54 (50–58) * | 89 (88–90) * | 34 (30–37) * | 95 (94–96) * | 98 (85–87) * | 0.715 (0.692–0.737) * | ||
| qSOFA/RED+ | 57 (53–62) † | 87 (86–88) † | 32 (29–35) † | 95 (95–96) † | 85 (84–86) † | 0.724 (0.702–0.747) | ||
| In–hospital Mortality | ||||||||
| On ED arrival | qSOFA+ | 21 (16–27) | 96 (95–96) | 15 (11–20) | 97 (97–98) | 94 (93–94) | 0.585 (0.555–0.615) | |
| RED+ | 38 (32–46) * | 89 (88–90) * | 10 (8–13) * | 98 (97–98) * | 97 (86–88) * | 0.636 (0.600–0.672) * | ||
| qSOFA/RED+ | 44 (37–51) † | 88 (87–89) † | 11 (9–13) | 98 (97–98) † | 86 (85–87) † | 0.659 (0.622–0.696) † | ||
| Within 2 h after ED arrival | qSOFA+ | 30 (24–37) | 94 (93–95) | 15 (11–19) | 98 (97–98) | 92 (91–93) | 0.620 (0.596–0.654) | |
| RED+ | 47 (40–54) * | 86 (85–87) * | 11 (9–13) * | 10 (10–10) * | 85 (84–86) * | 0.666 (0.629–0.703) * | ||
| qSOFA/RED+ | 52 (45–60) † | 85 (84–86) † | 10 (9–13) | 98 (98–98) † | 83 (82–84) † | 0.685 (0.648–0.723) | ||
| 48-h Mortality | ||||||||
| On ED arrival | qSOFA+ | 40 (25–56) | 96 (95–96) | 6 (3–9) | 100 (99–100) | 95 (95–96) | 0.679 (0.596–0.761) | |
| RED+ | 54 (38–70) | 88 (87–89) * | 3 (2–5) * | 100 (99–100) | 88 (87–88) * | 0.712 (0.628–0.796) | ||
| qSOFA/RED+ | 57 (41–72) | 87 (86–88) † | 3 (2–4) | 100 (99–100) | 87 (86–88) † | 0.720 (0.637–0.803) | ||
| Within 2 h after ED arrival | qSOFA+ | 57 (41–72) | 94 (93–94) | 6 (4–8) | 10 (10–10) | 93 (93–94) | 0.754 (0.670–0.837) | |
| RED+ | 69 (52–81) | 86 (84–86) * | 3 (2–4) * | 10 (10–10) * | 85 (84–86) * | 0.771 (0.693–0.849) † | ||
| qSOFA/RED+ | 69 (52–81) | 84 (83–85) † | 3 (2–4) † | 10 (10–10) † | 84 (83–85) † | 0.761 (0.683–0.839) | ||
* A significant difference was observed between the results of RED+ and those of qSOFA+, p < 0.05 corrected by Bonferroni’s method. † A significant difference was observed between the results of RED+ and those of qSOFA/RED+, p < 0.05 corrected by Bonferroni’s method. CI, confidence intervals; PPV, positive predictive value; NPV, negative predictive value; AUC, area under the receiver operating characteristic curve; ED, emergency department; qSOFA+, positive quick sequential organ failure assessment; RED+, positive RED sign; qSOFA/RED+, qSOFA+ or RED+.
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Park, J.E.; Hwang, S.Y.; Jo, I.J.; Sim, M.S.; Cha, W.C.; Yoon, H.; Kim, T.R.; Lee, G.T.; Kim, H.S.; Sohn, I.; et al. Accuracy of the qSOFA Score and RED Sign in Predicting Critical Care Requirements in Patients with Suspected Infection in the Emergency Department: A Retrospective Observational Study. Medicina 2020, 56, 42. https://doi.org/10.3390/medicina56010042
AMA Style
Park JE, Hwang SY, Jo IJ, Sim MS, Cha WC, Yoon H, Kim TR, Lee GT, Kim HS, Sohn I, et al. Accuracy of the qSOFA Score and RED Sign in Predicting Critical Care Requirements in Patients with Suspected Infection in the Emergency Department: A Retrospective Observational Study. Medicina. 2020; 56(1):42. https://doi.org/10.3390/medicina56010042
Chicago/Turabian StylePark, Jong Eun, Sung Yeon Hwang, Ik Joon Jo, Min Seob Sim, Won Chul Cha, Hee Yoon, Tae Rim Kim, Gun Tak Lee, Hye Seung Kim, InSuk Sohn, and et al. 2020. "Accuracy of the qSOFA Score and RED Sign in Predicting Critical Care Requirements in Patients with Suspected Infection in the Emergency Department: A Retrospective Observational Study" Medicina 56, no. 1: 42. https://doi.org/10.3390/medicina56010042
APA StylePark, J. E., Hwang, S. Y., Jo, I. J., Sim, M. S., Cha, W. C., Yoon, H., Kim, T. R., Lee, G. T., Kim, H. S., Sohn, I., & Shin, T. G. (2020). Accuracy of the qSOFA Score and RED Sign in Predicting Critical Care Requirements in Patients with Suspected Infection in the Emergency Department: A Retrospective Observational Study. Medicina, 56(1), 42. https://doi.org/10.3390/medicina56010042
