Next Article in Journal
Correlation between Apnea Severity and Sagittal Cephalometric Features in a Population of Patients with Polysomnographically Diagnosed Obstructive Sleep Apnea
Previous Article in Journal
Hemodynamic Performance of Two Current-Generation Transcatheter Heart Valve Prostheses in Severely Calcified Aortic Valve Stenosis
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Soluble RAGE as a Prognostic Marker of Worsening in Patients Admitted to the ICU for COVID-19 Pneumonia: A Prospective Cohort Study

1
Department of Anesthesiology and Critical Care, Rouen University Hospital, UNIROUEN, INSERM U1096, Normandie Université, F-76000 Rouen, France
2
Rouen University Hospital, INSERM CIC-CRB 1404, F-76000 Rouen, France
3
Department of General Biochemistry, Rouen University Hospital, F-76000 Rouen, France
4
Department of Biostatistics, Rouen University Hospital, F-76000 Rouen, France
5
Department of Pharmacology, Rouen University Hospital, UNIROUEN, INSERM U1096, Normandie Université, F-76000 Rouen, France
6
Medical Intensive Care Unit, Rouen University Hospital, UNIROUEN, INSERM U1096, Normandie Université, F-76000 Rouen, France
7
UNIROUEN, INSERM U1096, Normandie Université, F-76000 Rouen, France
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2022, 11(15), 4571; https://doi.org/10.3390/jcm11154571
Submission received: 3 July 2022 / Revised: 20 July 2022 / Accepted: 2 August 2022 / Published: 5 August 2022

Abstract

:
Background: The different waves of SARS-CoV-2 infection have strained hospital resources and, notably, intensive care units (ICUs). Identifying patients at risk of developing a critical condition is essential to correctly refer patients to the appropriate structure and to spare limited resources. The soluble form of RAGE (sRAGE), the endoplasmic stress response and its surrogates, GRP78 and VEGF-A, may be interesting markers. Methods: This was a prospective monocenter cohort study of adult patients admitted to the ICU for severe COVID-19 pneumonia. The plasma levels of sRAGE, GRP78 and VEGF-A were measured within the first 24 h. Patients were classified as critical if they further needed vasopressor therapy, renal replacement therapy, or invasive mechanical ventilation, or died during their ICU stay, and were otherwise classified as not critical. Results: A total of 98 patients were included and 39 developed a critical condition. Critical patients presented higher sRAGE (626 [450–1043] vs. 227 [137–404] pg/mL, p < 0.0001), interleukin-6 (43 [15–112] vs. 11 [5–20] pg/mL, p < 0.0001), troponin T (17 [9–39] vs. 10 [6–18] pg/mL, p = 0.003) and NT-pro-BNP (321 [118–446] vs. 169 [63–366] pg/mL, p = 0.009) plasma levels. No difference was observed for VEGF-A and GRP78. The variables independently associated with worsening in the ICU were sRAGE (1.03 [1.01–1.05] per 10 pg/mL) and age (1.7 [1.2–2.4] per 5 years). An sRAGE value of 449.5 pg/mL predicted worsening with a sensitivity of 77% and a specificity of 80%. Conclusion: sRAGE may allow the identification of patients at risk of developing a critical form of COVID-19 pneumonia, and thus may be useful to correctly refer patients to the appropriate structure of care.

1. Introduction

Since the first identification of SARS-CoV-2 infection in humans, the subsequent COVID-19 pandemic has been responsible for at least half a billion cases worldwide, with an estimated mortality of at least 6 million people (data from the 3 July 2022).
Many efforts have been focused on understanding the mechanisms underlying virus entry into the host system and its pathogenesis, but also to understand the pathophysiological changes. Dysregulated inflammation may participate in the pathological systemic changes observed during COVID-19, triggering endothelium dysfunction, coagulation disorders, and eventually leading to shock [1,2,3]. High levels of cytokines, such as interleukin-6 (IL-6), are associated with the severity of the pathology and poor prognosis, and likely explain the beneficial effects of dexamethasone in the most seriously ill patients [1,4]. Nevertheless, the use of IL-6 or CRP as prognostic markers seems inconsistent during COVID-19. In a recent study, Picod et al. demonstrated an association between IL-6 and mortality or organ support, but the marker’s performance in predicting mortality appeared limited, questioning its usefulness in selecting the most at-risk patients [5]. Several hypotheses may explain this lack of performance, including the pharmacokinetics of this interleukin, with an early peak of synthesis, whereas COVID-19 presents a rather slow evolution, with several days between the first symptoms and the admission to the ICU. Another explanation may be the possible phenotypic variations between individuals concerning the production of the natural antagonist, the soluble receptor of IL-6 [6]. A strategy to overcome the limitations of the classical inflammatory biomarkers may be to focus on indirect biomarkers of inflammation, i.e., pathways activated by primary inflammation and thus more likely to last over time. A better understanding of the relationship between such biomarkers and clinical outcomes might help identify patients at the highest risk of clinical deterioration.
The receptor for advanced glycation-end products (RAGE) is implicated in several pathways of inflammatory processes, notably because of its ability to directly or indirectly interact with different types of ligands [7]. Thus, RAGE acts as a major component of acute inflammation, mediated by both pathogen-associated molecular patterns (PAMPs), such as lipopolysaccharides, and damage-associated molecular patterns (DAMPs), such as the high-mobility group box 1 protein (HMGB1) [8,9]. RAGE is preferentially expressed in the type 1 alveolar epithelial cells of the lungs, and at lower levels in the skin. It may also be expressed in various types of tissue during aging, and in cases of chronic disease, such as diabetes mellitus, obesity, atherosclerosis, and some nephropathies [7,10], some of which are known to be associated with worse outcomes during SARS-CoV-2 infection.
The endoplasmic reticulum is responsible for protein regulation, owing to specific chaperones such as the 78 kDa glucose-regulated protein (GRP78), which participates in the correct folding of proteins [11]. Inflammatory settings can alter the correct endoplasmic folding, a situation known as endoplasmic reticulum stress (ERS). ERS has been described to be associated with organ failure in patients suffering from a systemic inflammation response during septic shock or cardiac surgery with cardiopulmonary bypass [12,13]. SARS-CoV-2 infection and replication within the host cell may compete with the normal folding process, and may be responsible for both GRP78 overexpression and an increased unfolded protein response [14,15,16,17]. However, there are currently no data describing the GRP78 level in COVID-19 ICU patients.
The vascular endothelial growth factor type A is a key actor implicated in the homeostasis of the microcirculation, by regulating capillary permeability, vascular tone and cellular interactions [18,19]. Pro-inflammatory cytokines induced a rise in VEGF production, notably through the indirect effect of JAK/STAT3 and MAPK pathways, with elevated plasma levels observed in septic patients [20,21].
Because inflammation is associated with higher mortality in COVID-19 disease, the identification of surrogate biomarkers may be of interest to identify patients at risk of worsening, notably during acute epidemic waves when critical care resources may be strained by a massive inflow of critically ill patients. In a large cohort, we hypothesize that plasma levels of soluble RAGE (sRAGE), GRP78, and VEGF-A, three markers of pathways activated by primary inflammation, may be associated with worse outcomes for COVID-19 ICU patients, and thus may be used as reliable prognostic biomarkers.

2. Materials and Methods

2.1. Study Design

We conducted a prospective monocenter cohort study in three intensive care units of a university hospital. This cohort was approved by an ethics committee on the 7 April 2020 (Comité de Protection des Personnes Est-III, approval number 2020-A00885-34). According to the French law, verbal approval was required from the patient or their relatives. The study was conducted in accordance with the principles of the Declaration of Helsinki related to human research. The elaboration of the manuscript was in accordance with the STROBE statement.
This cohort was described in a previous publication [3]. Briefly, patients were eligible for inclusion if they presented a “severe” condition related to a documented infection of SARS-CoV-2 (determined by polymerase chain reaction assay), defined as the need for ICU admission because of acute respiratory failure. Non-inclusion criteria were pregnancy, a documented bacterial co-infection, known limitations in life support because of patient choice or important comorbidities, and an expected death within 24 h.
Clinical data were collected at admission and during the ICU stay, including conventional characteristics, specific treatments for COVID-19 disease (anticoagulation, corticosteroids, and others), Simplified Acute Physiology Score II (SAPS-II) and Sequential Organ Failure Assessment (SOFA) scores, use of non-invasive ventilation (NIV) or high-flow nasal cannula (HFNC) therapies and the subsequent ROX index (defined as ROX = SpO 2 / FiO 2 R e s p i r a t o r y   R a t e ) and PaO2/FiO2 ratio. Conventional biological characteristics included blood count, urea, creatinine, C-reactive protein (CRP), lactatemia, high-sensitivity troponin T, and N-terminal pro-brain natriuretic peptide (NT-pro-BNP). Patients were followed for up to 28 days or until death.

2.2. Endpoints

The primary endpoint was the clinical worsening from a “severe” towards a “critical” condition within 28 days after ICU admission, according to NMA-COVID Initiative Network definitions [22]. According to the NIH COVID-19 treatment guidelines [23], a patient was defined as “critical” if they presented at least one of the following outcomes: (1) need for invasive mechanical ventilation, (2) shock (i.e., vasopressors needed), (3) renal replacement therapy, and (4) death.
The secondary endpoints were the plasma levels of biomarkers: highly sensitive troponin, NT-pro-BNP, interleukin-6 (IL-6), sRAGE, VEGF-A and GRP78.

2.3. Blood Samples

Whole blood was collected on ethylenediaminetetraacetic acid (EDTA) tubes within the first 24 h after ICU admission, using a venous or arterial catheter, depending on their availability. Tubes were immediately centrifugated for 15 min (2000 G, 4 °C) and plasma was frozen at −20 °C for a maximum of 2 weeks before being frozen at −80 °C until final assays.

2.4. Assays

The present study is an ancillary study of the COVID-THELIUM cohort. Assays were performed using an Enzyme-Linked Immunosorbent Assay for sRAGE, VEGF-A (Invitrogen, ThermoFisher Scientific, Waltham, MA, USA) and GRP78 (Enzo Life Sciences, Villeurbanne, France), and using an Electro-Chemiluminescence Immunoassay for IL-6 (Roche Diagnostics, Meylan, France).

2.5. Data and Statistical Analysis

The primary objective of the study was to evaluate the association between the plasma levels of the different biomarkers at ICU admission and the occurrence of a worsening towards a “critical” condition during the 28 first days, as previously defined above. Severe patients who did not fulfil the “critical” criteria were defined as “not critical”.
Because of the non-normal distribution of most of the data, as observed using a D’Agostino test, the results are expressed as medians with interquartile ranges (IQR) for quantitative data, and as absolute numbers and percentages (n, %) for qualitative data. In cases of missing data, no value was imputed.
First, we compared “critical” and “not critical” patients using a Mann–Whitney two-tailed test for unpaired values. A χ2 test (or Fisher’s exact test) was used for categorical data. Statistical significance was assessed using an alpha level of 0.05.
Then, we assessed the association between worsening towards a critical condition and the plasma levels of biomarkers by considering clinical characteristics of interest at ICU admission that are known to be early predictors, including age, body mass index (BMI), Simplified Acute Physiology Score II (SAPS-II), and treatment with a corticosteroid or anticoagulant at admission. The predictor variables, with a p value of <0.2 in this univariate analysis, were included in the model, using a logistic regression model with a stepwise selection process. p values were computed using Wald’s method. The discrimination and the calibration of the model were internally validated by means of the Hosmer–Lemeshow goodness of fit test and the area under the operating characteristic curve (AUC) to assess model discrimination.
Thereafter, the predictive values and corresponding 95% confidence intervals of the significant factors identified in the multivariable analysis were assessed by the area under the receiver operating characteristic (ROC) curve.
Optimal cut-off values, defined as the best combination of sensitivity and specificity, were determined using the Youden index.
All statistical tests were two-sided, and the 0.05 probability level was used to establish statistical significance. The statistical analyses were performed by means of the statistical software SAS (version 9.4; SAS Institute, Cary, NC, USA).

3. Results

A total of 100 patients were included between May and October 2020. Among them, two were excluded because of a wrong diagnosis of COVID-19 pneumonia, resulting in a total of 98 analyzed patients. Finally, 59 patients remained stable, whereas 39 patients progressed to a critical condition. Among these 39 patients, 29 required vasopressors, 37 required invasive mechanical ventilation, three required renal replacement therapy, and 13 died. The clinical and biological characteristics of interest at ICU admission are presented in Table 1. Briefly, at admission, patients worsening towards a critical condition were older, and presented higher SAPS-2 and SOFA severity scores, and lower ROX indexes and PaO2/FiO2 ratios. The median duration of invasive mechanical ventilation was 15 [8–21] days. Critical patients presented worse kidney function. No difference was observed concerning the specific treatment for COVID-19 disease at ICU admission.
Concerning biomarkers, critical patients presented higher plasma levels of troponin (16.5 [9–39] vs. 10 [6–18] pg/mL, p = 0.003), NT-pro-BNP (320.5 [118–1446] vs. 169 [63–366] pg/mL, p = 0.009), sRAGE (626 [449.5–1043] vs. 227 [137–404] pg/mL, p < 0.0001) and IL-6 (43 [15–112] vs. 11 [5–20] pg/mL, p < 0.0001). No difference was observed for VEGF-A (197 [75–395] vs. 169 [64.5-286] pg/mL, p = 0.6) and GRP78 (1596 [1190–2014] vs. 1569 [1187–1988] pg/mL, p = 0.97). The results are presented in Figure 1.
The univariable analyses showed an association between the occurrence of a critical condition and age (p = 0.0002), SAPS-II score (p < 0.0001), plasma levels of creatinine (p = 0.007), troponin T (p = 0.006) and sRAGE (p < 0.0001) (Table 2). Then, a multivariable analysis was performed including these variables and NT-pro-BNP and IL-6 plasma levels (p < 0.2). Only sRAGE plasma levels and age were significantly and independently associated with the occurrence of a critical condition within the first 28 days of ICU stay, with an odds-ratio of 1.03 [1.01–1.05] per 10 pg/mL of sRAGE and an odds ratio of 1.7 [1.2–2.4] per five years of age. Evaluation of the multivariate prediction model showed an AUC range from 0.86 to 0.98 and a p value for the Hosmer–Lemeshow (HL) goodness of fit of 0.359. This indicates good discrimination (i.e., separating “critical” patients from “not critical” patients) and calibration (the model fits the data well).
A ROC curve for sRAGE was elaborated for the prediction of worsening condition and showed an AUC of 0.82 [0.74–0.91] (Figure 2). A cut-off value of 449.5 pg/mL predicted worsening with a specificity of 80 and a sensitivity of 77% (Youden index), resulting in a positive predictive value of 71% and a negative predictive value of 84% in this cohort. A more sensitive cut-off value of 202 pg/mL predicted the worsening with a sensitivity of 97% and a specificity of 42%, resulting in a positive predictive value of 53% and a highly negative predictive value of 96% in this cohort.

4. Discussion

In this cohort study, we demonstrated that the soluble form of RAGE was independently associated with a worsening of the vital status of patients admitted to the ICU for severe COVID-19 pneumonia, in contrast to the other explored biomarkers. RAGE is highly expressed in alveolar type 1 cells and is a key player in pro-inflammatory pathways, as demonstrated by the increased alveolar level in experimental models of acute lung injury [24,25,26]. Soluble forms of RAGE are produced both from the proteolytic cleavage of the membrane receptor and from endogenous production by alternative splicing [27]. Elevated plasma levels were observed during pulmonary inflammation [28]. In a meta-analysis including 746 patients from eight studies, sRAGE was independently associated with 90-day mortality during acute respiratory distress syndrome (ARDS), highlighting the prognostic relevance of this marker [29]. Pathophysiological changes in RAGE pathways during COVID-19 disease make this marker even more interesting in this specific setting. Indeed, beyond the acute pulmonary damages related to the virus penetration in pneumocytes, a direct crosstalk may exist between RAGE and SARS-CoV-2 infection. The spike protein interacts with angiotensin-converting enzyme 2 (ACE2) as a receptor for viral penetration. A side effect of this interaction is the downregulation of ACE2 and the subsequent accumulation of angiotensin II within cells. This induces inflammatory activation through angiotensin receptor type 1, with the production of inflammatory markers including HMGB1. This latter factor may amplify inflammation by the activation of RAGE pathways [7]. This specific mechanism of action may explain the associated comorbidities described as worsening the outcomes of COVID-19 patients, such as hypertension, obesity, diabetes, or aging, all situations where the RAGE axis is often disturbed, notably by the higher production of AGE, HMGB1 or S100 ligands [30]. In a recent cohort study of hospitalized non-ICU COVID-19 patients, higher sRAGE plasma levels were associated with higher mortality [31]. In another cohort study, Lim et al. described sRAGE plasma levels in 164 COVID-19 patients of varying severity (including 32 patients of severe status, based on a definition similar to the one we used) [32]. RAGE plasma levels were correlated with the severity of the disease and predicted the need for invasive mechanical ventilation with good accuracy. In our study, we focused only on critically ill patients with respiratory failure, with 94% of patients treated with HFNC or NIV at admission. Soluble RAGE appeared to be a good prognostic marker of worsening towards multi-organ failure or death, and thus may be useful to identify patients at risk of complications, and who may be eligible for reinforced monitoring in the ICU, rather than intermediate care. In comparison, IL-6 plasma levels within 24 h of ICU admission presented an AUC of 0.62 in a cohort of critically ill COVID-19 patients, whereas sRAGE in our cohort presented a higher AUC of 0.82, suggesting a better prognosis performance. These potentially novel findings confirm, in a larger population size, that plasma sRAGE may be a promising biomarker in COVID-19 pneumonia. Although the measurement of plasma sRAGE is currently reserved for research purposes, our data, in combination with previous studies, argue for the development of the routine laboratory analysis of sRAGE in order to identify “at-risk” patients and to refer them as soon as possible to the most appropriate unit.
In our study, no association was observed regarding GRP78 and ICU outcome. GRP78 is an essential chaperone protein involved in the unfolded protein response (UPR) and its expression is regulated by activating transcription factor 4, which is activated in stress situations where there is a risk of protein misfolding, such as oxidative stress, anoxia/hypoxia, acute inflammation and amino acid deprivation [16]. UPR consists of a decrease in protein synthesis and an enhanced capacity of endoplasmic reticulum protein folding, in order to maintain cellular homeostasis. Imbalance in this process may lead to cellular apoptosis [33]. Some studies have explored the level of expression of GRP78 during COVID-19 disease. In an autopsy study, a higher expression of GRP78 was observed in the pneumocytes and alveolar macrophages of a COVID-19 patient with pneumonia, compared with control patients [15]. In a small cohort study, plasma levels of GRP78 were shown to be significantly higher in COVID-19 patients in comparison with control patients. Nevertheless, no difference was observed between COVID-19 patients with respiratory failure compared with asymptomatic patients, suggesting, similarly to our results, that GRP78 is elevated in COVID-19 patients but is not associated with the severity of disease [34]. Several hypotheses may be raised to explain these results. First, most severely ill COVID-19 patients mainly present isolated respiratory failure, which may explain the low sensitivity in the plasma of GRP78 as a prognostic biomarker, even though UPR signaling may be of great magnitude in the lungs. Indeed, very few patients presented vascular or renal failure at admission in our study. Secondly, ischemia/reperfusion and reactive oxygen species (ROS) are important determinants of ER stress and UPR activation. Contrary to cardiac surgery or sepsis, where these phenomena are frequent and generalized to the whole organism, ROS production during COVID-19 disease seems to be mainly localized to the lungs [35,36,37,38]. Thus, in our study, GRP78 does not appear as a biomarker of severity in a COVID-19 ICU population.
Another biomarker we evaluated was VEGF-A, with no difference observed according to the prognosis of patients. VEGF-A is a growth factor of importance in the regulation of endothelial function, and of microcirculation in general, notably because of its action through the type 1 and 2 VEGF receptors. The activation of these receptors induces many phosphorylation processes, and notably the destabilization of intercellular junctions, leading to vascular permeability [18]. Many studies have demonstrated increased plasma levels of VEGF-A during sepsis, a pathology where microcirculation is highly disturbed [21,39]. Microcirculation is also challenged during the COVID-19 disease course, as demonstrated both by elevated plasma levels of angiopoietin 2 and by the in vivo demonstration of a reduction in small-vessel density [1,40]. Nevertheless, and despite plasma levels similar to those observed during sepsis, VEGF-A was not able to discriminate patients with worse outcomes in our study. This absence of discrimination of VEGF-A, according to the prognosis of COVID-19 patients, has already been described in small cohort studies, contrary to other endothelial biomarkers, such as angiopoietin-2 or the Intercellular Adhesion Molecule type 1 (ICAM-1) [1,41,42]. Thus, we confirmed the lack of interest of VEGF-A in a larger population.
As evoked for GRP78, the inability of GRP78 and VEGF-A to discriminate patients with a poor prognosis in our study may be related to the relatively few cases with systemic insults. Thus, the measurement of these biomarkers at admission, when the patients only presented a lung affliction, may have been too early in the course of the disease. Because RAGE is essentially expressed in the lung, and because of the existence of a soluble form, this could explain why this marker may be efficient to discriminate patients as early as admission.
We also demonstrated in this study that patients who evolved towards a critical condition presented higher plasma levels of troponin and NT-proBNP at ICU admission. Although significant, these levels were only slightly higher than those observed in healthy patients and were not independently associated with a worsening condition. Moreover, we only observed one patient in the whole cohort with confirmed cardiogenic shock, but we could not rule out cases of sub-clinical myocarditis. Indeed, higher levels of troponin have been described in up to 25% of hospitalized COVID-19 patients, but with only 7% presenting true myocardial suffering. Moreover, patients with higher troponin levels were more likely to require ICU admission [43]. Taken together, troponin appears to be associated with the severity of SARS-CoV-2 infection, but not with patient prognosis.
We observed a significant and independent association with worsening condition in elderly patients, as previously published [44]. Indeed, elderly patients frequently present comorbidities that put them at risk of severe presentations of COVID-19, including hypertension, diabetes, obesity, pulmonary or renal chronic diseases and dementia [45]. Other clinical or biological markers may be of interest in identifying patients at risk of a worsening condition. Among them, the alteration of hemostatic processes may be of interest, predominantly because of the higher incidence of thrombotic events during COVID-19 disease. We previously described the alteration in the coagulation process in our cohort, with a particular role identified for immature platelets and thrombin generation assay, whereas d-dimers presented discouraging results [3,46]. On the other hand, clinical parameters such as the ROX index, which quantifies the severity of respiratory failure under HFNC, may predict the respiratory worsening of patients [47]. Thus, it seems to us wise to consider the triage of patients eligible for admission to the ICU according to a set of clinical and biological parameters, among which age, severity of respiratory failure and plasma levels of sRAGE may play a promising role.
Our study presents several limitations. First, despite a relatively important number of patients, we cannot rule out that the cohort is underpowered to demonstrate independent associations between certain biomarkers (in particular, troponin and NT-pro-BNP) and outcomes. Secondly, and as discussed above, the measurements were performed very early in the course of the ICU stay. While it is of interest to identify patients “at-risk” of worsening as early as possible, GRP78 and VEGF-A levels may have been different between the two groups later in the ICU stay, and especially when patients developed shock or kidney failure. Thus, even if these biomarkers cannot be used as early biomarkers of prognosis, it is very plausible that their pathological pathways are disturbed later in course of the disease.

5. Conclusions

Our study suggested that sRAGE monitoring may be useful to identify COVID-19 patients at risk of worsening towards a critical and life-threatening condition. The routine measurement of this marker may help practitioners in the triage process, and particularly when hospital resources are strained during epidemic waves.

Author Contributions

Conceptualization, E.B., F.T. and T.C.; methodology, C.T.; formal analysis, E.B. and C.T.; investigation, P.L., S.R. and V.B.; writing—original draft preparation, E.B.; writing—review and editing, Z.D., J.B. and T.C.; supervision, E.B.; project administration, E.B.; funding acquisition, E.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Comité de Protection des Persones EST-III (number of approval 2020-A00885-34 on the 8 April 2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Because of the non-interventional nature of the study, the ethical board waived the need for written consent, according to the French law.

Data Availability Statement

Data are available upon reasonable request from the corresponding author.

Acknowledgments

We thank Ali Bouncer and the Direction de la Recherche Clinique et de l’Innovation for their support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Abou-Arab, O.; Bennis, Y.; Gauthier, P.; Boudot, C.; Bourdenet, G.; Gubler, B.; Beyls, C.; Dupont, H.; Kamel, S.; Mahjoub, Y. Association between inflammation, angiopoietins, and disease severity in critically ill COVID-19 patients: A prospective study. Br. J. Anaesth. 2021, 126, e127–e130. [Google Scholar] [CrossRef] [PubMed]
  2. Nesseler, N.; Fadel, G.; Mansour, A.; Para, M.; Falcoz, P.-E.; Mongardon, N.; Porto, A.; Bertier, A.; Levy, B.; Cadoz, C.; et al. Extracorporeal membrane oxygenation for respiratory failure related to COVID-19: A nationwide cohort study. Anesthesiology 2022, 136, 732–748. [Google Scholar] [CrossRef] [PubMed]
  3. Billoir, P.; Leprêtre, P.; Thill, C.; Bellien, J.; Duchez, V.L.C.; Selim, J.; Tamion, F.; Clavier, T.; Besnier, E. Routine and advanced laboratory tests for hemostasis disorders in COVID-19 patients: A prospective cohort study. J. Clin. Med. 2022, 11, 1383. [Google Scholar] [CrossRef] [PubMed]
  4. RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with COVID-19. N. Engl. J. Med. 2021, 384, 693–704. [Google Scholar] [CrossRef] [PubMed]
  5. Picod, A.; Morisson, L.; de Roquetaillade, C.; Sadoune, M.; Mebazaa, A.; Gayat, E.; Davison, B.A.; Cotter, G.; Chousterman, B.G. Systemic inflammation evaluated by interleukin-6 or c-reactive protein in critically ill patients: Results from the frog-icu study. Front. Immunol. 2022, 13, 868348. [Google Scholar] [CrossRef]
  6. Jones, S.A.; Horiuchi, S.; Topley, N.; Yamamoto, N.; Fuller, G.M. The Soluble interleukin 6 receptor: Mechanisms of production and implications in disease. FASEB J. 2001, 15, 43–58. [Google Scholar] [CrossRef] [Green Version]
  7. Chiappalupi, S.; Salvadori, L.; Donato, R.; Riuzzi, F.; Sorci, G. Hyperactivated RAGE in comorbidities as a risk factor for severe COVID-19—The role of RAGE-Ras crosstalk. Biomolecules 2021, 11, 876. [Google Scholar] [CrossRef]
  8. Wang, L.; Wu, J.; Guo, X.; Huang, X.; Huang, Q. RAGE plays a role in LPS-induced NF-ΚB activation and endothelial hyperpermeability. Sensors 2017, 17, 722. [Google Scholar] [CrossRef] [Green Version]
  9. Kokkola, R.; Andersson, A.; Mullins, G.; Ostberg, T.; Treutiger, C.-J.; Arnold, B.; Nawroth, P.; Andersson, U.; Harris, R.A.; Harris, H.E. RAGE is the major receptor for the proinflammatory activity of Hmgb1 in rodent macrophages. Scand. J. Immunol. 2005, 61, 1–9. [Google Scholar] [CrossRef]
  10. Kierdorf, K.; Fritz, G. RAGE regulation and signaling in inflammation and beyond. J. Leukoc. Biol. 2013, 94, 55–68. [Google Scholar] [CrossRef]
  11. Khan, M.M.; Yang, W.-L.; Wang, P. Endoplasmic reticulum stress in sepsis. Shock 2015, 44, 294–304. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Clavier, T.; Demailly, Z.; Semaille, X.; Thill, C.; Selim, J.; Veber, B.; Doguet, F.; Richard, V.; Besnier, E.; Tamion, F. A weak response to endoplasmic reticulum stress is associated with postoperative organ failure in patients undergoing cardiac surgery with cardiopulmonary bypass. Front. Med. 2020, 7, 613518. [Google Scholar] [CrossRef] [PubMed]
  13. Clavier, T.; Grangé, S.; Pressat-Laffouilhere, T.; Besnier, E.; Renet, S.; Fraineau, S.; Thiebaut, P.-A.; Richard, V.; Veber, B.; Tamion, F. Gene expression of protein tyrosine phosphatase 1B and endoplasmic reticulum stress during septic shock. Front. Med. 2019, 6, 240. [Google Scholar] [CrossRef] [PubMed]
  14. Bartolini, D.; Stabile, A.M.; Vacca, C.; Pistilli, A.; Rende, M.; Gioiello, A.; Cruciani, G.; Galli, F. Endoplasmic reticulum stress and NF-kb activation in SARS-CoV-2 infected cells and their response to antiviral therapy. IUBMB Life 2022, 74, 93–100. [Google Scholar] [CrossRef] [PubMed]
  15. Puzyrenko, A.; Jacobs, E.R.; Sun, Y.; Felix, J.C.; Sheinin, Y.; Ge, L.; Lai, S.; Dai, Q.; Gantner, B.N.; Nanchal, R.; et al. Pneumocytes are distinguished by highly elevated expression of the ER stress biomarker GRP78, a co-receptor for SARS-CoV-2, in COVID-19 autopsies. Cell Stress Chaperones 2021, 26, 859–868. [Google Scholar] [CrossRef]
  16. Shahriari-Felordi, M.; Alikhani, H.K.; Hashemian, S.-M.R.; Hassan, M.; Vosough, M. Mini review ATF4 and GRP78 as novel molecular targets in er-stress modulation for critical COVID-19 patients. Mol. Biol. Rep. 2022, 49, 1545–1549. [Google Scholar] [CrossRef]
  17. Fung, T.S.; Liu, D.X. Coronavirus infection, ER stress, apoptosis and innate immunity. Front. Microbiol. 2014, 5, 296. [Google Scholar] [CrossRef] [Green Version]
  18. Bates, D.O. Vascular endothelial growth factors and vascular permeability. Cardiovasc. Res. 2010, 87, 262–271. [Google Scholar] [CrossRef] [Green Version]
  19. Ourradi, K.; Blythe, T.; Jarrett, C.; Barratt, S.L.; Welsh, G.I.; Millar, A.B. VEGF isoforms have differential effects on permeability of human pulmonary microvascular endothelial cells. Respir. Res. 2017, 18, 1–12. [Google Scholar] [CrossRef] [Green Version]
  20. Kaur, S.; Bansal, Y.; Kumar, R.; Bansal, G. A panoramic review of IL-6: Structure, pathophysiological roles and inhibitors. Bioorg. Med. Chem. 2020, 28, 115327. [Google Scholar] [CrossRef]
  21. Shapiro, N.I.; Yano, K.; Okada, H.; Fischer, C.; Howell, M.; Spokes, K.C.; Ngo, L.; Angus, D.C.; Aird, W.C. A Prospective, Observational study of soluble FLT-1 and vascular endothelial growth factor in sepsis. Shock 2008, 29, 452–457. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Guérin, P.J.; McLean, A.R.D.; Rashan, S.; Lawal, A.; Watson, J.A.; Strub-Wourgaft, N.; White, N.J. Definitions matter: Heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis. PLoS Glob. Public Health 2021, 2, e0000561. [Google Scholar] [CrossRef]
  23. National Institutes of Health. Clinical Spectrum of SARS-CoV-2 Infection-COVID-19 Treatment Guidelines; National Institutes of Health: Bethesda, MD, USA, 2021; pp. 32–34.
  24. Shirasawa, M.; Fujiwara, N.; Hirabayashi, S.; Ohno, H.; Iida, J.; Makita, K.; Hata, Y. Receptor for advanced glycation end-products is a marker of type I lung alveolar cells. Genes Cells 2004, 9, 165–174. [Google Scholar] [CrossRef] [PubMed]
  25. Bierhaus, A.; Humpert, P.M.; Morcos, M.; Wendt, T.; Chavakis, T.; Arnold, B.; Stern, D.M.; Nawroth, P.P. Understanding RAGE, the receptor for advanced glycation end products. J. Mol. Med. 2005, 83, 876–886. [Google Scholar] [CrossRef]
  26. Uchida, T.; Shirasawa, M.; Ware, L.B.; Kojima, K.; Hata, Y.; Makita, K.; Mednick, G.; Matthay, Z.A.; Matthay, M.A. Receptor for Advanced Glycation End-Products Is a Marker of Type I Cell Injury in Acute Lung Injury. Am. J. Respir. Crit. Care Med. 2006, 173, 1008–1015. [Google Scholar] [CrossRef] [Green Version]
  27. Riuzzi, F.; Sorci, G.; Sagheddu, R.; Chiappalupi, S.; Salvadori, L.; Donato, R. RAGE in the Pathophysiology of skeletal muscle. J. Cachexia Sarcopenia Muscle 2018, 9, 1213–1234. [Google Scholar] [CrossRef] [Green Version]
  28. Calfee, C.S.; Ware, L.B.; Eisner, M.D.; Parsons, P.E.; Thompson, B.T.; Wickersham, N.; Matthay, M.A. Network plasma receptor for advanced glycation end products and clinical outcomes in acute lung injury. Thorax 2008, 63, 1083–1089. [Google Scholar] [CrossRef] [Green Version]
  29. Jabaudon, M.; Blondonnet, R.; Pereira, B.; Cartin-Ceba, R.; Lichtenstern, C.; Mauri, T.; Determann, R.M.; Drabek, T.; Hubmayr, R.D.; Gajic, O.; et al. Plasma sRAGE is independently associated with increased mortality in ARDS: A meta-analysis of individual patient data. Intensiv. Care Med. 2018, 44, 1388–1399. [Google Scholar] [CrossRef] [Green Version]
  30. Sanyaolu, A.; Okorie, C.; Marinkovic, A.; Patidar, R.; Younis, K.; Desai, P.; Hosein, Z.; Padda, I.; Mangat, J.; Altaf, M. Comorbidity and its Impact on Patients with COVID-19. SN Compr. Clin. Med. 2020, 2, 1069–1076. [Google Scholar] [CrossRef]
  31. Wick, K.D.; Siegel, L.; Neaton, J.D.; Oldmixon, C.; Lundgren, J.; Dewar, R.L.; Lane, H.C.; Thompson, B.T.; Matthay, M.A. RAGE has potential pathogenetic and prognostic value in nonintubated hospitalized patients with COVID-19. JCI Insight 2022, 7, e157499. [Google Scholar] [CrossRef]
  32. Lim, A.; Radujkovic, A.; Weigand, M.A.; Merle, U. Soluble receptor for advanced glycation end products (sRAGE) as a biomarker of COVID-19 disease severity and indicator of the need for mechanical ventilation, ARDS and mortality. Ann. Intensiv. Care 2021, 11, 1–13. [Google Scholar] [CrossRef] [PubMed]
  33. Ron, D.; Walter, P. Signal integration in the endoplasmic reticulum unfolded protein response. Nat. Rev. Mol. Cell Biol. 2007, 8, 519–529. [Google Scholar] [CrossRef] [PubMed]
  34. Sabirli, R.; Koseler, A.; Goren, T.; Turkcuer, I.; Kurt, O. High GRP78 levels in Covid-19 infection: A case-control study. Life Sci. 2020, 265, 118781. [Google Scholar] [CrossRef]
  35. Vaage, J.; Valen, G. Pathophysiology and Mediators of Ischemia-Reperfusion Injury with Special Reference to Cardiac Surgery: A Review. Scand. J. Thorac. Cardiovasc. Surg. 1993, 27, 1–18. [Google Scholar] [CrossRef] [PubMed]
  36. Levi, M.; Choi, G.; Schoots, I.; Schultz, M.; Van Der Poll, T. Beyond sepsis: Activated protein C and ischemia–reperfusion injury. Crit. Care Med. 2004, 32, S309–S312. [Google Scholar] [CrossRef]
  37. LaForge, M.; Elbim, C.; Frère, C.; Hémadi, M.; Massaad, C.; Nuss, P.; Benoliel, J.-J.; Becker, C. Tissue damage from neutrophil-induced oxidative stress in COVID-19. Nat. Rev. Immunol. 2020, 20, 515–516. [Google Scholar] [CrossRef]
  38. Vardakas, P.; Skaperda, Z.; Tekos, F.; Kouretas, D. ROS and COVID. Antioxidants 2022, 11, 339. [Google Scholar] [CrossRef]
  39. Hou, P.C.; Filbin, M.R.; Wang, H.; Ngo, L.; Huang, D.T.; Aird, W.C.; Yealy, D.M.; Angus, D.C.; Kellum, J.A.; Shapiro, N.I. Endothelial permeability and hemostasis in septic shock: Results from the ProCESS trial. Chest 2017, 152, 22–31. [Google Scholar] [CrossRef]
  40. Rovas, A.; Osiaevi, I.; Buscher, K.; Sackarnd, J.; Tepasse, P.-R.; Fobker, M.; Kühn, J.; Braune, S.; Göbel, U.; Thölking, G.; et al. Microvascular dysfunction in COVID-19: The MYSTIC study. Angiogenesis 2020, 24, 145–157. [Google Scholar] [CrossRef]
  41. Pine, A.B.; Meizlish, M.L.; Goshua, G.; Chang, C.H.; Zhang, H.; Bishai, J.; Bahel, P.; Patel, A.; Gbyli, R.; Kwan, J.M.; et al. Circulating markers of angiogenesis and endotheliopathy in COVID-19. Pulm. Circ. 2020, 10, 2045894020966547. [Google Scholar] [CrossRef]
  42. Vassiliou, A.; Keskinidou, C.; Jahaj, E.; Gallos, P.; Dimopoulou, I.; Kotanidou, A.; Orfanos, S. ICU Admission Levels of Endothelial Biomarkers as Predictors of Mortality in Critically Ill COVID-19 Patients. Cells 2021, 10, 186. [Google Scholar] [CrossRef] [PubMed]
  43. Sheth, A.; Modi, M.; Dawson, D.; Dominic, P. Prognostic value of cardiac biomarkers in COVID-19 infection. Sci. Rep. 2021, 11, 1–9. [Google Scholar] [CrossRef] [PubMed]
  44. Marin, B.G.; Aghagoli, G.; Lavine, K.; Yang, L.; Siff, E.J.; Chiang, S.S.; Salazar-Mather, T.P.; Dumenco, L.; Savaria, M.C.; Aung, S.N.; et al. Predictors of COVID-19 severity: A literature review. Rev. Med. Virol. 2021, 31, 1–10. [Google Scholar] [CrossRef] [PubMed]
  45. Alves, V.; Casemiro, F.; Araujo, B.; Lima, M.; Oliveira, R.; Fernandes, F.; Gomes, A.; Gregori, D. Factors Associated with Mortality among Elderly People in the COVID-19 Pandemic (SARS-CoV-2): A Systematic Review and Meta-Analysis. Int. J. Environ. Res. Public Health 2021, 18, 8008. [Google Scholar] [CrossRef]
  46. Billoir, P.; Alexandre, K.; Duflot, T.; Roger, M.; Miranda, S.; Goria, O.; Joly, L.M.; Demeyere, M.; Feugray, G.; Brunel, V.; et al. Investigation of Coagulation Biomarkers to Assess Clinical Deterioration in SARS-CoV-2 Infection. Front. Med. 2021, 8, 670694. [Google Scholar] [CrossRef]
  47. Prakash, J.; Bhattacharya, P.K.; Yadav, A.K.; Kumar, A.; Tudu, L.C.; Prasad, K. ROX index as a good predictor of high flow nasal cannula failure in COVID-19 patients with acute hypoxemic respiratory failure: A systematic review and meta-analysis. J. Crit. Care 2021, 66, 102–108. [Google Scholar] [CrossRef]
Figure 1. Plasma levels of the different biomarkers at ICU admission with regard to worsening towards a critical condition (critical group) or not (not critical group). ** p < 0.01, **** p < 0.0001.
Figure 1. Plasma levels of the different biomarkers at ICU admission with regard to worsening towards a critical condition (critical group) or not (not critical group). ** p < 0.01, **** p < 0.0001.
Jcm 11 04571 g001
Figure 2. Receiver operating characteristic (ROC) curve for the prediction of the worsening of patients towards a critical condition during their ICU stay, according to the plasma level of sRAGE at ICU admission. The cut-off value of 449.5 pg/mL predicted worsening with a sensitivity of 77% and a specificity of 80%, as identified using the Youden index.
Figure 2. Receiver operating characteristic (ROC) curve for the prediction of the worsening of patients towards a critical condition during their ICU stay, according to the plasma level of sRAGE at ICU admission. The cut-off value of 449.5 pg/mL predicted worsening with a sensitivity of 77% and a specificity of 80%, as identified using the Youden index.
Jcm 11 04571 g002
Table 1. Clinical and biological characteristics at admission to the ICU.
Table 1. Clinical and biological characteristics at admission to the ICU.
ParametersAll
(n = 98)
Not Critical
(n = 59)
Critical
(n = 39)
p-Value
Age (years)67 [58–73]61 [53–69]72 [64–75]<0.0001
Male (n, %)68 (69.4)40 (67.8)28 (71.8)0.6742
Body mass index (kg/m2)29.2 [25.3–33.9]29.9 [26.1–35.2]29 [25.1–32.4]0.38
Obesity (BMI ≥ 30 kg/m2) (n, %) (n = 58/39)46 (47.4)29 (50)17 (40.2)0.54
Underlying comorbidity (n, %)
Chronic pulmonary disease9 (9.2)5 (8.5)4 (10.3)1
Asthma15 (15.3)12 (20.3)3 (7.7)0.09
Diabetes42 (42.9)24 (40.7)18 (46.2)0.6
Hypertension60 (61.2)35 (59.3)25 (64.1)0.63
Peripheral arterial disease3 (3.1)1 (1.7)2 (5.1)0.56
Coronaropathy9 (9.2)4 (6.8)5 (12.8)0.48
Smoking4 (4.1)2 (3.4)2 (5.1)/
Active neoplasia9 (9.2)7 (11.9)2 (5.1)0.31
COVID-19 related treatment at admission (n, %)
Corticosteroid90 (91.8)56 (94.9)34 (87.2)0.26
Remdesivir17 (17.4)13 (22)4 (10.3)0.13
Lopinavir/ritonavir7 (7.1)4 (6.8)3 (7.7)1
Tocilizumab1 (1.0)1 (1.7)0 (0)/
Anticoagulation therapy (n, %)90 (91.8)55 (93.2)35 (89.7)0.71
Prophylactic intensity8 (8.2)4 (6.7)4 (10.5)
Intermediate intensity61 (62.2)42 (70)19 (50.0)
Therapeutic intensity22 (22.4)10 (16.7)12 (31.6)
ICU transfer since the onset of symptoms (days)9 [6–10]9 [7–10]7 [4–10]0.06
SAPS II score32.5 [25–40]27 [22–35]40 [34–52]<0.0001
SOFA score3 [1–4]2 [1–3]4 [2–6]<0.0001
Non-invasive respiratory support (HFNC or NIV) (n, %)92 (93.9)59 (98.3)33 (86.8)0.8
PaO2/FiO2 ratio (n =)135 [95–165]144 [117–168]115.5 [78–159]0.026
ROX index (n = 71)6.5 [5.4–8.8]7.5 [6.0–10.3]5.4 [4.4–6.2]<0.0001
Biological parameters
Creatinine (µmol/L)75 [55–101]65 [52–81]94 [76–156]0.003
Urea (mmol/L)7 [4.9–9.6]5.7 [4.5–8]8.9 [5.5–17.2]0.0007
Hemoglobin (g/dL)12.6 [11.4–13.7]12.6 [11.6–13.7]12.6 [11.2–13.6]0.59
Platelets (G/L)228.5 [160–287]236 [179–303]197 [138–253]0.02
Leukocytes (G/L)7.6 [6.0–11.3]7.5 [5.8–10.2]8.4 [5.9–12.1]0.5
C-reactive protein (mg/L) (n = 59/37)112 [72.5–187.5]122 [76–186]102 [63–189]0.82
Lactatemia (mmol/L) (n = 57/39)1.3 [1.0–1.7]1.3 [0.9–1.6]1.4 [1–1.7]0.11
Glutamic–pyruvic transaminase (U/L)45 [34–68]44 [34–60]50 [34–72]0.2
Glutamic–oxaloacetic transaminase (U/L)38 [24–65]43 [29–68]31 [23–58]0.1
BMI: body mass index; ICU: intensive care unit; SAPS: Simplified Acute Physiology Score; SOFA: Sequential Organ Failure Assessment; NT-pro-BNP: N-terminal pro-brain natriuretic peptide.
Table 2. Associations between clinical and biological variables at ICU admission and worsening towards critical condition during ICU stay.
Table 2. Associations between clinical and biological variables at ICU admission and worsening towards critical condition during ICU stay.
VariableUnivariable AnalysisMultivariable AnalysisMultivariable Analysis with a Stepwise Selection Process
OR [95% CI]OR [95% CI]OR [95% CI]pOR [95% CI]p
Age (per 5 years)1.63 [1.26–2.10]0.00021.6 [1.04–2.5]0.031.7 [1.2–2.4]0.04
Male gender1.21 [0.50–2.93]0.67
BMI (per kg.m−2)0.97 [0.91–1.04]0.40
SAPS-II score1.10 [1.05–1.15]<0.00011.06 [0.99–1.13]0.10
No corticosteroid2.75 [0.62–12.2]0.19
No anticoagulation1.57 [0.37–6.69]0.54
Creatinine (per µmol/L)1.02 [1.00–1.03]0.0070.99 [0.98–1.01]0.4
Troponin (per pg/mL)1.05 [1.01–1.08]0.0061.02 [0.98–1.07]0.25
NT-pro-BNP (per 10 pg/mL)1.002 [0.99–1.005]0.121.00 [0.99–1.004]0.85
sRAGE (per 10 pg/mL)1.03 [1.02–1.05]<0.00011.04 [1.01–1.06]0.0051.03 [1.01–1.05]0.001
IL-6 (per 5 pg/mL)1.02 [0.99–1.05]0.11.02 [0.99–1.05]0.13
VEGF (per 5 pg/mL)0.99 [0.99–1.004]0.81
GRP78 (per 10 ng/mL)0.99 [0.99–1.01]0.830.99 [0.99–1.01]0.87
Bold variables with p values < 0.2 are included in logistic regression model analysis. BNP: brain natriuretic peptide; GRP-78: glucose-related protein 78; IL-6: interleukin-6; SAPS: Simplified Acute Physiology Score; sRAGE: soluble receptor for advanced glycation end product; VEGF: vascular endothelial growth factor. Analyses were performed using a logistic regression model to explain the occurrence of worsening towards a critical condition during the first 28 days of ICU stay (invasive mechanical ventilation, renal replacement therapy, vasopressor therapy and/or death).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Besnier, E.; Brunel, V.; Thill, C.; Leprêtre, P.; Bellien, J.; Demailly, Z.; Renet, S.; Tamion, F.; Clavier, T. Soluble RAGE as a Prognostic Marker of Worsening in Patients Admitted to the ICU for COVID-19 Pneumonia: A Prospective Cohort Study. J. Clin. Med. 2022, 11, 4571. https://doi.org/10.3390/jcm11154571

AMA Style

Besnier E, Brunel V, Thill C, Leprêtre P, Bellien J, Demailly Z, Renet S, Tamion F, Clavier T. Soluble RAGE as a Prognostic Marker of Worsening in Patients Admitted to the ICU for COVID-19 Pneumonia: A Prospective Cohort Study. Journal of Clinical Medicine. 2022; 11(15):4571. https://doi.org/10.3390/jcm11154571

Chicago/Turabian Style

Besnier, Emmanuel, Valéry Brunel, Caroline Thill, Perrine Leprêtre, Jérémy Bellien, Zoe Demailly, Sylvanie Renet, Fabienne Tamion, and Thomas Clavier. 2022. "Soluble RAGE as a Prognostic Marker of Worsening in Patients Admitted to the ICU for COVID-19 Pneumonia: A Prospective Cohort Study" Journal of Clinical Medicine 11, no. 15: 4571. https://doi.org/10.3390/jcm11154571

APA Style

Besnier, E., Brunel, V., Thill, C., Leprêtre, P., Bellien, J., Demailly, Z., Renet, S., Tamion, F., & Clavier, T. (2022). Soluble RAGE as a Prognostic Marker of Worsening in Patients Admitted to the ICU for COVID-19 Pneumonia: A Prospective Cohort Study. Journal of Clinical Medicine, 11(15), 4571. https://doi.org/10.3390/jcm11154571

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop