Next Article in Journal
Glycemic Control after Initiation of Anti-VEGF Treatment for Diabetic Macular Edema
Next Article in Special Issue
A Systematic Review of Treatments of Post-Concussion Symptoms
Previous Article in Journal
Epilepsy in Pediatric Patients—Evaluation of Brain Structures’ Volume Using VolBrain Software
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Reference Values for the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) from General Population Samples in the United Kingdom, Italy, and The Netherlands

by
Marina Zeldovich
1,*,
Fabian Bockhop
1,
Amra Covic
1,
Katrin Cunitz
1,
Suzanne Polinder
2,
Juanita A. Haagsma
2,
Nicole von Steinbuechel
1 and
on behalf of the CENTER-TBI Participants and Investigators
1
Institute of Medical Psychology and Medical Sociology, University Medical Center Göttingen, Waldweg 37A, 37073 Göttingen, Germany
2
Department of Public Health, Erasmus MC, University Medical Center Rotterdam, 3000 CA Rotterdam, The Netherlands
*
Author to whom correspondence should be addressed.
The full list of the CENTER-TBI participants and investigators is provided in the Supplementary Materials.
J. Clin. Med. 2022, 11(16), 4658; https://doi.org/10.3390/jcm11164658
Submission received: 20 June 2022 / Revised: 3 August 2022 / Accepted: 7 August 2022 / Published: 9 August 2022
(This article belongs to the Special Issue Traumatic Brain Injury (TBI): Recent Trends and Future Perspectives)

Abstract

:
After traumatic brain injury (TBI), individuals may experience short- or long-term health burdens, often referred to as post-concussion symptoms (PCS). The Rivermead Post-Concussion Symptoms Questionnaire (RPQ) is one of the commonly used instruments to assess self-reported PCS. To date, no reference values for RPQ have been provided, although they are crucial for clinical practice when evaluating a patient’s health status relative to a comparable healthy population. Therefore, the aim of this study is to provide reference values for the United Kingdom, the Netherlands, and Italy. A total of 11,759 individuals (50.3% women) from representative general population samples participated in an online survey (4646 individuals from the UK, 3564 from the Netherlands, and 3549 from Italy). The factorial structure of the RPQ was examined using confirmatory factor analysis (CFA), and results from the general population samples were compared with those from respective TBI samples recruited within the international CENTER-TBI study using multigroup CFA. Reference values were stratified by sex, health status, age, and education using percentiles. The three-factorial model outperformed the one-factorial structure. The general population samples were largely comparable to the corresponding TBI samples, except for items such as dizziness, vision, and sensory sensitivity, which can be considered more TBI-specific. Because of the significant differences between the general population samples, we provided reference values for the total score and for the somatic, emotional, and cognitive scales for each country separately. The reference values provided can now be used in clinical practice and research. Future studies should obtain stratified reference values for other countries and languages to improve accuracy in the diagnosis and treatment of symptom burden after TBI.

1. Introduction

Traumatic brain injury (TBI) is defined as a change in brain functioning caused by an external force [1]. Those affected often suffer from life-long limitations [2] and experience a range of physical, emotional, and cognitive disabilities. The effects of TBI limit functioning [1] of the affected individuals, and often lead to substantial burden for caregivers, family members [3], and health care systems [2]. Even after mild traumatic brain injury (scores of 13–15 on the Glasgow Coma Scale; GCS [4]), which accounts for the majority of all TBI cases [2], individuals do not fully recover to their premorbid level of functioning and report being bothered by a range of health problems [5].
After sustaining a TBI, individuals may experience short-term or long-lasting physical (e.g., headaches), cognitive (e.g., difficulties concentrating), and emotional/behavioral (e.g., fatigue) [6] symptoms. These symptoms are often collectively referred to as post-concussion symptoms (PCS). PCS typically emerge after mild to moderate TBI [7], but also individuals after a severe TBI frequently report comparable short- or long-lasting deficits termed post-concussion-like (PC-like) symptoms [8,9].
Clinicians and researchers often rely on self-reporting of the concerned individuals using patient-reported outcome measures (PROMs) to screen and assess PCS. Among the most frequently used PROMs for self-reported PCS is the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) [7]. The RPQ was suggested in the Common Data Elements (CDE) recommendations for assessing symptom burden following TBI in adults [10,11]. The questionnaire requires rating of the presence of 16 symptoms (headaches, dizziness, nausea and/or vomiting, noise sensitivity, sleep disturbance, fatigue, irritability, depression, frustration, forgetfulness and poor memory, poor concentration, slow thinking, blurred vision, light sensitivity, double vision, and restlessness) during the past 24 h before the assessment compared to the health condition before TBI. This questionnaire has been translated into a wide range of languages [12] and is broadly applied in PCS assessment [13]. However, despite its wide application, no reference values have yet been established for the RPQ.
Voormolen et al. (2019) [14] found a high prevalence of PC-like symptoms in the general population samples from the United Kingdom, The Netherlands, and Italy. Overall, 45.1% rated at least three symptoms at least as mild with fatigue showing the highest prevalence (49.9%) followed by sleep disturbance (42.4%). In addition, 17.5% reported at least three symptoms as being a moderate problem. In extension to this finding, providing reliable information on the clinical relevance of PCS reported by individuals after TBI is crucial. Patients’ current health status after TBI can be evaluated most accurately based on the comparison of health values of a representative general population sample with similar characteristics (e.g., same gender, age, or initial health status).
The comparison of RPQ scores of individuals’ after TBI with reference values obtained from the general population is of interest especially for clinicians, but also for researchers in the field of TBI. Since values may vary from country to country, the identification of problems in a single individual (i.e., individual health status) benefits from the comparison with country-specific values collected from the reference population (i.e., healthy general population sample).
Before providing reference values, the applicability of the RPQ in general population samples should be investigated by applying psychometric testing of the RPQ in the general population and testing for the equivalent assessment of the construct (i.e., PCS) in general and TBI populations. If results suggest that RPQ scores from both populations would be considered comparable, reference values can be established.
The aims of the present study are:
  • To investigate the applicability of the RPQ in general population samples from the United Kingdom (English sample), the Netherlands (Dutch sample), and Italy (Italian sample);
  • To provide reference values for these three countries in order to increase reliability and validity of outcome assessments, thereby strengthening the psychometric properties of the RPQ.
The comparison of an individual’s health status after TBI to values of a healthy reference population can provide guidance for both clinical decision makers and researchers.

2. Materials and Methods

2.1. General Population Samples

2.1.1. Data Collection

Data collection was carried out through an online survey from 29 June to 31 July 2017. Participant recruitment was handled by a private market research agency (https://www.dynata.com/, accessed on 20 June 2022), employed a large custom online panel and included survey dissemination, data processing, and hosting. The study sample was conceived to be representative with regard to the distribution of age, gender, and educational status in the United Kingdom, the Netherlands, and Italy, respectively. For more details, see Figure 1—left part “General population samples”.
To enhance representativeness by sampling participants from diverse social backgrounds, multiple recruitment sources were taken advantage of (e.g., proprietary loyalty partnerships, open recruitment to traditional online panels, and integrated partnerships with online communities, publishers, as well as social networks).
Study invitations did not include specific details on project aims to avoid self-selection bias, instead inviting suitable individuals to “take a survey”. Complete participation was reimbursed by the market research agency by handing out cash money, survey points, prizes, or sweepstakes. Participants who completed the survey in less than five minutes were classified as “speeders” and were deleted from the dataset. Respondents were required to answer every questionnaire item, since the electronic data collection system did not allow for missing responses. The recruitment process continued until the required quotas were met.

2.1.2. Informed Consent

The recruiting agency obtained informed consent from all individuals who agreed to participate in the online survey. The process is described in the privacy agreement, which can be found at https://www.dynata.com/privacy-policy/ (accessed on 12 January 2022). Participants were informed on the survey welcome page as to the goal of the survey, which was to better understand the impact of TBI on patients’ lives, that the survey would take approximately 20 min to complete, as well as that all responses would be confidential and anonymous. The data were anonymized with each participant assigned a number in the order they completed the survey.

2.1.3. Ethical Approval

The general population study was part of the Collaborative European NeuroTrauma Effectiveness Research (CENTER-TBI; clinicaltrials.gov NCT02210221) project. Ethical approval was obtained from the Leids Universitair Centrum—Commissie Medische Ethiek (approval P14.222/NV/nv).

2.2. TBI Samples

2.2.1. Data Collection

Data collection for the TBI samples took place as part of the CENTER-TBI study between December 2014 and December 2019. A total of 4509 individuals after TBI from 63 centers in 18 European countries and Israel were enrolled in the core study. Inclusion criteria for the CENTER-TBI core study were the clinical diagnosis of TBI, indication for computed tomography (CT), admission within 24 h of injury, and informed consent for study participation. To avoid bias in outcome assessment, patients with severe preexisting neurologic disorders (e.g., epilepsy, stroke) were excluded from the study. Patients were either evaluated in the emergency room (ER) and then discharged or admitted to either the hospital ward or an intensive care unit (ICU). Further study details can be found elsewhere [15]. Data were retrieved from the CENTER-TBI database via the Neurobot tool (core data set 2.1, November 2019).
The following analyses comprised individuals belonging to the English, Dutch, and Italian language samples aged 16 years or older who filled out the RPQ six months after TBI (N = 1088). For more information, see Figure 1—right part “TBI samples”. Language samples composition has been described in more detail elsewhere [16].

2.2.2. Informed Consent

Informed consent was obtained according to the respective local and national requirements for all patients recruited in the core dataset of CENTER-TBI and documented in the e-CRF [17].

2.2.3. Ethical Approval

The CENTER-TBI study (EC grant 602150) has been conducted in accordance with all relevant laws of the EU if directly applicable or of direct effect and all relevant laws of the country where the recruiting sites were located, including but not limited to, the relevant privacy and data protection laws and regulations (the “Privacy Law”), the relevant laws and regulations on the use of human materials, and all relevant guidance relating to clinical studies from time to time in force including, but not limited to, the ICH Harmonized Tripartite Guideline for Good Clinical Practice (CPMP/ICH/135/95) (“ICH GCP”) and the World Medical Association Declaration of Helsinki entitled “Ethical Principles for Medical Research Involving Human Subjects”. Informed Consent was obtained for all patients recruited in the Core Dataset of CENTER-TBI and documented in the e-CRF. Ethical approval was obtained for each recruiting site. The list of sites, Ethical Committees, approval numbers, and approval dates can be found on the project’s website (https://www.center-tbi.eu/project/ethical-approval, accessed on 20 June 2022).

2.3. Sample Characteristics

All study participants provided information on their age, sex (TBI sample) or gender (general population samples), and level of education. Individuals from the general population samples additionally indicated whether they suffered from one or more chronic health conditions (i.e., asthma, heart disease, stroke, diabetes, back problems, osteoarthritis, rheumatism, cancer, memory problems due to a neurological condition such as dementia, memory problems due to aging, depression, or other problems). Information on chronic health conditions was merged for further analyses to test for differences regarding RPQ symptoms in individuals without chronic health conditions and participants suffering from at least one health problem. TBI severity has been assessed using the Glasgow Coma Scale (GCS) [4] with values 13–15 indicating mild, 9–12 indicating moderate, and less or equal to 8 indicating severe TBI.

2.4. The Rivermead Post-Concussion Symptoms Questionnaire

The Rivermead Post-Concussion Symptoms Questionnaire (RPQ) [7] is a self-assessment instrument comprising a list of 16 PCS. Respondents are asked to rate impairment associated with these symptoms during the past 24 h compared to their condition before TBI on a five-point Likert scale (from 0 “not experienced at all” to 4 “a severe problem”). Based on the originally proposed unidimensional factor structure, ratings are summarized into a total score which ranges from 0 to 64 with higher values indicating greater symptom severity.
The original English version of the RPQ alongside Dutch and Italian translations were used in this study. The latter versions were translated and linguistically validated in preparation for the CENTER-TBI study [12]. The RPQ was adapted for application in the general population by deleting any reference to TBI in the questionnaire introduction. Consequently, items such as “Compared with before the accident, do you now (i.e., over the last 24 h) suffer from…” were reworded to “Do you now (i.e., over the last 24 h) suffer from…”.

2.5. Statistical Analyses

2.5.1. Descriptive Statistics and Psychometric Properties of the RPQ in General Population Samples

Investigation of the comparability of psychometric characteristics of the RPQ in general population samples with the results obtained from the TBI samples followed the approach described in the study by von Steinbuechel et al. (2021) [16]. Specifically, item characteristics per language version are provided including sample sizes, mean (M), standard deviation (SD), skewness (SK), and kurtosis (KU). SK and KU values from −2 to +2 were considered acceptable [18]. Response behavior was analyzed using absolute and relative frequencies in item categories.
Reliability analyses included Cronbach’s alpha (values from 0.70 to 0.95 indicating good to excellent internal consistency [19]), split-half reliability (odd vs. even items) with Spearman-Brown correction, and Cronbach’s alpha after omission of respective item (should not exceed the overall Cronbach’s alpha).
Item-total correlations were calculated to test the discriminatory quality of the items and correlation coefficients were taken as indicators of how well items discriminate between individuals with low as opposed to high symptom severity. Lower correlation coefficients indicate weaker discriminatory power and the cut-off was determined at a medium effect size (r ≥ 0.30) [20].

2.5.2. Factorial Structure of the RPQ in General Population Samples

The factorial structure of the RPQ has been repeatedly discussed during the last decades. The initially proposed one-factor solution [7] has been more or less abandoned in favor of multi-factor solutions, of which several different models have been proposed [21,22,23,24,25,26,27]. However, there is still no consensus on the most suitable factorial structure.
One of the suggested RPQ structures is the three-factor solution reported in Smith-Seemiller and colleagues (2003) [21] covering somatic (nine items), emotional (four items), and cognitive (three items) scales. Evidence points to satisfactory fit of this model both in a cross-sectional investigation of six RPQ translations [28] as well as in a longitudinal study involving a large TBI sample [29]. Considering the satisfactory results in the TBI samples, we applied this factor solution to analyze factorial structure of the RPQ in general population groups in addition to the original single-factor model. Investigations utilized confirmatory factor analyses (CFA) with robust weighted least squares estimators (WLSMV) [30] for ordered categorical data in each language sample.
The goodness-of-fit of estimated models was evaluated with the help of multiple indices (cut-off criteria shown in brackets): χ 2 and degrees of freedom (df), as well as the ratio χ 2 / d f (≤2) [31], the comparative fit index (CFI; ≥0.95) [32,33], the Tucker–Lewis index (TLI; ≥0.95) [33,34], the root mean square error of approximation (RMSEA; excellent fit at 0.05, mediocre fit at 0.10) [35,36] including 90% confidence interval (CI90%), and the standardized root mean square residual (SRMR; <0.08) [33]. All fit indices were considered simultaneously to evaluate the model fit since the indices have not yet been validated for ordinal data [37].

2.5.3. Comparability of the RPQ Scores in General Population and TBI Samples

The comparability of the RPQ scores in general population and TBI samples was investigated for each language sample using a multi-group CFA approach (i.e., general population sample vs. TBI sample in each language), also referred to as measurement invariance (MI) testing. For this purpose, we followed the framework originally proposed by Wu and Estabrook (2016) [38] and updated by Svetina, Rutkowski, and Rutkowski (2020) [39]. First, we fitted the baseline model (1). Second, this model was then restrained by requiring measurement invariance of thresholds (2) and intercepts as well as thresholds (3) across the groups. Finally, differences in model fit were evaluated using chi-square difference test and changes in the comparative fit index (ΔCFI) and RMSEA (ΔRMSEA). Should the chi-square difference test not suggest significant differences, as well as ΔCFI < 0.01 [40], and ΔRMSEA ≤ 0.01 [41], models were considered equivalent. In that case, the simpler model with fewer restraints would be retained. The assumption of MI was considered reasonable when the baseline model was chosen over the restrained models. The RPQ scores in general population and TBI samples can be thus considered comparable allowing for calculation of the reference values derived from general population samples.

2.5.4. Reference Values

To identify factors associated with PC-like symptoms in general population samples from the United Kingdom, Italy, and the Netherlands, linear regression models were estimated for the RPQ total score, and emotional, somatic, and cognitive scales, respectively. Country, sex, age, education, and health status as well as all possible second-order interactions between the factors (e.g., country × sex, country × age, country × health status etc.) served as independent variables. For RPQ scores, responses rated as 1 (no more of a problem than before) were recoded to 0 as proposed for the original scoring of the RPQ [7]. Significant factors were used to stratify reference values.
Reference values were computed based on percentiles which represent the value that a certain percentage of observations falls below. This information can be used to determine whether an individual’s RPQ value after TBI is below, equal to, or above the reference population value. For patient-level interpretation, the following percentiles are provided: 2.5%, 5%, 16%, 30%, 40%, 50%, 60%, 70%, 85%, 95%, and 97.5%. RPQ values that exceed the reference average by one standard deviation or more are considered clinically relevant [42]. This corresponds to the 85%-quantile in data that present a normal distribution. Interpretation examples are provided in the results section.
All analyses were carried out with R version 4.0.2. [43] and packages table1 [44] for descriptive analyses, psych [45] for psychometric properties, and lavaan [46] for the CFA and MI testing. The significance level was set at 5%.

3. Results

3.1. Sample Characteristics

Overall, 11,759 individuals (50.3% females) aged on average 44.6 ± 15.3 years (median: 44.0; range 18.0–75.0) from general population samples completed the questionnaire, including 4646 individuals from the United Kingdom, 3564 from the Netherlands, and 3549 from Italy. Most participants held a middle school degree (over 40% across all language samples). Approximately half of all participants (50.9%) reported to suffer from at least one chronic health condition. The total average RPQ score in all countries was 12.0 (SD = 13, median = 8), indicating clinically relevant symptom burden according to the mean but not the median when applying a cut-off of 12 [23].
The TBI sample consisted of 1088 individuals after TBI (35.6% females) aged on average 51.3 ± 19.2 years (median: 54.0; range: 16.0–95.0) from the Dutch (N = 597), English (N = 223), and Italian (N = 268) language samples. Most participants completed secondary or high school (52.1%) and sustained a mild TBI (74.3%). Average RPQ total score across the language groups was 14.7 (SD = 12.6, Median = 12), indicating clinically relevant impairment [23]. Individuals after mild TBI reported lower symptom severity on average (M = 14.1, SD = 12.6, Median = 12) compared with those who had sustained moderate or severe TBI (M = 16.2, SD = 12.4, Median = 13).
Comparison of the sample characteristics between the general population samples with respective TBI language samples revealed significant differences in distribution of sex and education (p < 0.001). Participants from the TBI samples were significantly older compared with those from the respective general population (p < 0.001; approx. five years on average). For more details, see Table 1 (left part—“General population samples”, right part—“TBI samples”).
Distribution of chronic health complaints reported by participants from general population samples is provided in the Table 2. Across all countries, the most commonly reported health complaints were depression (18.7%), other (13%), asthma (10.2%), and back pain (9.7%). The “other” category included an open text field for participants to complete. Multiple mentions of other health complaints included COPD, mental disorders other than depression, epilepsy, fibromyalgia, kidney disease, and others.

3.2. Item Characteristics and Psychometric Properties of the RPQ in the General Population Samples

Dizziness (22%), noise sensitivity (19.6%), light sensitivity (17.5%), nausea (14.3%), and blurred (18%) and double vision (9.4%) showed the lowest percentage of endorsement. In contrast, fatigue (41.4%), headaches (35.8%), sleeping problems (35.3%), and being irritable (34.3%) were more common among general population samples. For more detail, see Appendix ATable A1.
On average, the distribution of RPQ items was less skewed across the general population samples (SK: M = 1.10, SD = 0.56, KU: M = 0.46, SD = 1.79) compared to the TBI samples (SK: M = 1.22, SD = 0.73; KU: M = 0.99, SD = 2.78) [16]. See Appendix ATable A2 for more detail.
The RPQ showed excellent reliability in the general population samples. Cronbach’s alpha values were above 0.90 across the general population samples (Dutch: 0.94; English: 0.95; Italian: 0.92) and comparable with the TBI samples (Dutch: 0.93; English: 0.92; Italian: 0.91) [16]. The values of the Cronbach’s alpha if an item was omitted were smaller than the Cronbach’s alpha in each language sample. The item-total correlations were above 0.30 across all samples. For more detail, see Appendix ATable A3.

3.3. Factorial Structure of the RPQ in the General Population Samples

Comparable to the results obtained from the respective TBI samples [16], the original one-factor structure could not be replicated for any of the general population samples. In contrast, the three-factor solution comprising somatic, emotional, and cognitive scales showed widely satisfactory results in all language samples (see Appendix ATable A4). Consequently, the following comparisons of general population samples with TBI samples were based on this factorial solution.

3.4. Measurement Invariance

The MI analyses between general population samples and TBI samples revealed non-significant differences between the baseline models (1) and the thresholds models (2) in the English language sample and significant differences in Dutch and Italian samples. There were no differences between threshold models (2) and intercepts and threshold models (3) across all language samples (see Appendix ATable A5, upper part). Significant differences between the baseline (1) and threshold models (2) in the Dutch and Italian samples indicated differences in the probability of choosing response categories between the general population samples and the TBI samples. The ΔCFI and ΔRMSEA values did not exceed the cut-off values, and the relative differences between the samples in most of the items were negligible (i.e., they did not exceed 5%). However, some items exceed the 5%-cut-off across all language samples (i.e., light sensitivity, double vision, and restlessness in all language samples and dizziness, noise sensitivity, and fatigue in Dutch samples). For further details, see Appendix AFigure A1, Figure A2 and Figure A3.
In addition, the comparison between the general population samples (i.e., Dutch vs. English vs. Italian) showed significant differences across constrained models (see Appendix ATable A5, lower part).
Given that most of the differences in symptoms between the language samples did not exceed the permissible cut-off value, that most of the symptoms characterized by the significant differences were specifically relevant to TBI and rarely occur in the healthy general population [47,48], and that thresholds and intercepts and threshold models did not show significant differences in all samples, it seemed appropriate to establish reference values using the data obtained from the general population samples. Furthermore, the violation of the MI assumption between general population samples at all levels suggested that providing separate reference values for each language sample is reasonable.

3.5. Regression Analyses

The results of the regression analyses indicated significant effects of all independent variables on both the RPQ total score and the scales, suggesting the provision of separate reference scores for countries stratified by sociodemographic and health factors. However, considering the interactions between factors, education was found to have a significant relationship with RPQ scores only in combination with country. Therefore, reference values for each country were reported stratified by sex, age, and health status information. For more details on the results of the regression analyses, see Appendix ATable A6.

3.6. Reference Values

Reference values for the RPQ total score obtained from the English, Italian, and Dutch general samples are presented in Table 3, Table 4 and Table 5. Table A7, Table A8 and Table A9 in Appendix B provide reference values for the scales. Below we provide an example for application of the reference values.
A 45-year-old healthy woman from the UK had sustained a TBI. Her RPQ total score was 25. Table 2 shows that 95% of healthy individuals in her respective age, gender, and health status group report the same or lower symptom severity. In other words: Only 5% of the reference population suffer from more severe symptoms. Therefore, the reported RPQ symptoms are rated as clinically relevant.
If this woman additionally reports at least one chronic health condition (e.g., asthma prior TBI), approximately 60% of the reference population from her age, gender, and health status group report less severe symptoms and 40% of the general population with similar characteristics report more severe PCS than she does. In this case, the woman’s individual score is within the normal range.

4. Discussion

The present study aimed to supply clinicians and researchers with reference values for the RPQ obtained from general population samples from the United Kingdom, Italy, and the Netherlands. These samples were designed to be representative for the general population in the respective countries and thus provide a reliable basis for reference values.
The results showed that the factorial structure of the RPQ based on the general population data reflects the problems reported in the TBI population [21,22,23,24,25,26,27]. This underlines problems with replicability of the original one-factor solution. Therefore, in addition to the reference values based on the original RPQ total score, we derived reference values for the three scales covering somatic, emotional, and cognitive symptoms which showed a good fit in previous studies [28,29]. As there is no consensus yet on the underlying factorial structure of the RPQ, we found this additional information helpful to clinicians in assessing each patient’s PCS.
Differences between the language samples indicate the need for language- and country-specific reference values. These findings can be supported by previous research, although relatively little has been published on PC-like symptoms in general population samples or direct comparisons between healthy individuals from different countries. Voormolen et al. [14] documented the highest reported number of PC-like symptoms in the English general population sample (47.8%) closely followed by the Italian sample (46.2%), when using a cut-off of 2 implying rating single RPQ symptoms as at least mild. Applying a cut-off of 3 (i.e., reporting at least a moderate problem), the participants from the United Kingdom again exhibited most symptoms (20.9%), followed by individuals from the Dutch sample (16.3%). In other English-speaking countries, participants from general population samples report varying levels of PC-like symptoms. A Canadian study [49] found that 35.9% to 71.8% of healthy community volunteers experienced one or more of the 13 symptoms—comparable to those assessed with the RPQ–of the British Columbia Post-Concussion Symptom Inventory (BC-PSI) [50] at least once or twice in the past two weeks. Overall, between 24.1% (noise sensitivity) and 57.3% (fatigue and feeling nervous) of symptoms were rated as at least mild by participants in an Australian student and community sample using the BC-PSI [51].
Given the variability between the general populations of different countries in the prevalence of PC-like symptoms, we would encourage the use of language/country-specific reference values rather than relying on the commonly proposed cut-off values (e.g., a cut-off of 12 proposed by Potter et al. [23]).
Apart from country-specific findings, we generally found a relatively high prevalence of PC-like symptoms in the general population samples included in the present study. This is largely consistent with previous research findings. In recent decades, the rate of self-reported symptoms in healthy individuals has been considered high to very high [14,47,48,49,51,52], indicating the need to account for this when diagnosing PCS in individuals after TBI. In addition, as reflected by the distribution of reference values, these symptoms are more prevalent in the general population subsamples with chronic health complaints. Some studies have shown that the occurrence of PCS is less TBI-specific [53,54] and more related to the premorbid health status of the affected individuals than to TBI [55,56]. Furthermore, individuals suffering from chronic health conditions may experience cognitive, emotional, and somatic symptoms comparable to PCS (e.g., chronic pains [21], depression [51], stroke and multiple sclerosis [57], etc.). However, this does not mean that symptoms that appear to be due to conditions other than TBI should be ignored or left untreated. Rather, they indicate that patients may have multiple needs that may be interacting and hindering the healing process. When possible, further differential diagnosis should be made to provide the best possible treatment for the affected individual.
Considering this information, we strongly recommend assessing the premorbid health status of TBI patients or individuals with suspected TBI to determine the most appropriate reference population for use of the RPQ. Only by comparing individuals after TBI with a reasonably similar reference population, results can be reliably classified, and appropriate measures and targeted therapeutic interventions derived. Since the selection of reference values is crucial for the assessment of the individual symptomatology as well as the derived treatment and its success, in the clinical context, an accurate collection of medical history is necessary to select the appropriate reference population. With the reference values presented here, clinicians have now the opportunity to consider the most important factors (i.e., gender, age, education, and health status) when assessing patient outcomes after TBI with the RPQ.

Strengths and Limitations

This study has important strengths as well as some limitations. Due to the large sample size and the representativeness of the quotas for gender, age, and education level in the language samples, we were able to provide reliable reference values for further application in clinical practice and research for the United Kingdom, Italy, and the Netherlands. In addition, we considered health status in establishing reference values, which allows for a more detailed evaluation of PCS or PC-like symptoms in single individuals after TBI.
However, some issues related to the nature of data collection should be considered. First, participants were recruited exclusively through Internet platforms via online surveys which are often associated with (self-)selection and response bias [58]. Furthermore, we do not have information on the number of individuals contacted who declined to participate in the study as well as no characteristics of those who dropped out during the survey. Second, we lacked information on a possible TBI experience in the general population samples. Therefore, despite all attempts made during data collection, some sample selection bias may occur (e.g., only individuals with access to the Internet were able to complete the panel). These points as well as the legitimacy of data use, have been discussed in more detail elsewhere [59]. Finally, because of the relatively small number of cases within specific groups of chronic conditions, further stratification of reference values was not possible. Therefore, distinguishing between minor and severe chronic health conditions is not possible when applying reference values presented here. Since the effects of chronic health conditions may influence symptom burden in different ways, future studies should focus on differences in PC-like symptoms in specific diseases.
Furthermore, the relatively small and unequally distributed across the countries sample size of the TBI language samples used in the MI analyses could have had an impact on the results since there was a low variability in responses and extreme answer categories of some items were rarely endorsed. Finally, characteristics between general population and TBI samples differed significantly which could have impacted the results.

5. Conclusions

This study provides RPQ reference values for three European countries that are ready for use in clinical practice and research. Future studies should target reference values for other countries and languages to improve accuracy in the diagnosis and treatment of PCS after TBI.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jcm11164658/s1.

Author Contributions

Conceptualization, M.Z. and N.v.S.; methodology, M.Z.; software, M.Z. and F.B.; formal analysis, M.Z. and F.B.; investigation, J.A.H.; data curation, J.A.H. and M.Z.; writing—original draft preparation, M.Z.; writing—review and editing, M.Z., F.B., A.C., K.C., S.P., J.A.H., and N.v.S.; visualization, M.Z.; supervision, N.v.S.; funding acquisition, N.v.S. and S.P. All authors have read and agreed to the published version of the manuscript.

Funding

CENTER-TBI was supported by the European Union 7th Framework programme (EC grant 602150). Additional funding was obtained from the Hannelore Kohl Stiftung (Germany), from OneMind (USA), and from Integra LifeSciences Corporation (USA).

Institutional Review Board Statement

The CENTER-TBI study (EC grant 602150) has been conducted in accordance with all relevant laws of the European Union (EU) if directly applicable or of direct effect and all relevant laws of the country where the recruiting sites were located. Informed consent by the patients and/or the legal representative/next of kin was obtained, accordingly to the local legislations, for all patients recruited in the Core Dataset of CENTER-TBI and documented in the electronic case report form (e-CRF). For the full list of sites, ethical committees, and ethical approval details, see the official CENTER-TBI website (https://www.center-tbi.eu/project/ethical-approval, accessed on 4 November 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

All relevant data are available upon request from CENTER-TBI, and the authors are not legally allowed to share it publicly. The authors confirm that they received no special access privileges to the data. CENTER-TBI is committed to data sharing and in particular to responsible further use of the data. Hereto, we have a data sharing statement in place: https://www.center-tbi.eu/data/sharing. The CENTER-TBI Management Committee, in collaboration with the General Assembly, established the Data Sharing policy, and Publication and Authorship Guidelines to assure correct and appropriate use of the data as the dataset is hugely complex and requires help of experts from the Data Curation Team or Bio- Statistical Team for correct use. This means that we encourage researchers to contact the CENTER-TBI team for any research plans and the Data Curation Team for any help in appropriate use of the data, including sharing of scripts. Requests for data access can be submitted online: https://www.center-tbi.eu/data (accessed on 12 January 2022). The complete Manual for data access is also available online: https://www.center-tbi.eu/files/SOP-Manual-DAPR-2402020.pdf (accessed on 12 January 2022).

Acknowledgments

We gratefully thank all CENTER-TBI participants and investigators. We are immensely grateful to our participants for helping us in our efforts to improve care and outcome for TBI.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

Table A1. Distribution of the item responses of the RPQ in the general population samples.
Table A1. Distribution of the item responses of the RPQ in the general population samples.
ItemResponse
Category
EnglishItalianDutchTotal
(N = 4646)(N = 3549)(N = 3564)(N = 11,759)
Headaches02094 (45.1%)1399 (39.4%)1963 (55.1%)5456 (46.4%)
1764 (16.4%)625 (17.6%)381 (10.7%)1770 (15.1%)
21137 (24.5%)1033 (29.1%)736 (20.7%)2906 (24.7%)
3499 (10.7%)442 (12.5%)360 (10.1%)1301 (11.1%)
4152 (3.3%)50 (1.4%)124 (3.5%)326 (2.8%)
Feeling of Dizziness02661 (57.3%)2200 (62.0%)2374 (66.6%)7235 (61.5%)
1741 (15.9%)553 (15.6%)445 (12.5%)1739 (14.8%)
2792 (17.0%)540 (15.2%)487 (13.7%)1819 (15.5%)
3356 (7.7%)211 (5.9%)203 (5.7%)770 (6.5%)
496 (2.1%)45 (1.3%)55 (1.5%)196 (1.7%)
Nausea and/or Vomiting03145 (67.7%)2174 (61.3%)2715 (76.2%)8034 (68.3%)
1766 (16.5%)746 (21.0%)397 (11.1%)1909 (16.2%)
2481 (10.4%)470 (13.2%)278 (7.8%)1229 (10.5%)
3176 (3.8%)137 (3.9%)131 (3.7%)444 (3.8%)
478 (1.7%)22 (0.6%)43 (1.2%)143 (1.2%)
Noise
Sensitivity, easily upset by loud noise
03012 (64.8%)2114 (59.6%)2450 (68.7%)7576 (64.4%)
1611 (13.2%)587 (16.5%)387 (10.9%)1585 (13.5%)
2569 (12.2%)536 (15.1%)398 (11.2%)1503 (12.8%)
3308 (6.6%)247 (7.0%)243 (6.8%)798 (6.8%)
4146 (3.1%)65 (1.8%)86 (2.4%)297 (2.5%)
Sleep Disturbance01830 (39.4%)1582 (44.6%)1682 (47.2%)5094 (43.3%)
1634 (13.6%)585 (16.5%)455 (12.8%)1674 (14.2%)
21050 (22.6%)828 (23.3%)753 (21.1%)2631 (22.4%)
3686 (14.8%)412 (11.6%)422 (11.8%)1520 (12.9%)
4446 (9.6%)142 (4.0%)252 (7.1%)840 (7.1%)
Fatigue, tiring more easily01529 (32.9%)1176 (33.1%)1392 (39.1%)4097 (34.8%)
1674 (14.5%)665 (18.7%)450 (12.6%)1789 (15.2%)
21132 (24.4%)953 (26.9%)818 (23.0%)2903 (24.7%)
3792 (17.0%)594 (16.7%)581 (16.3%)1967 (16.7%)
4519 (11.2%)161 (4.5%)323 (9.1%)1003 (8.5%)
Being Irritable, easily angered01902 (40.9%)1222 (34.4%)1797 (50.4%)4921 (41.8%)
1822 (17.7%)764 (21.5%)617 (17.3%)2203 (18.7%)
21004 (21.6%)913 (25.7%)654 (18.4%)2571 (21.9%)
3593 (12.8%)504 (14.2%)364 (10.2%)1461 (12.4%)
4325 (7.0%)146 (4.1%)132 (3.7%)603 (5.1%)
Feeling Depressed or Tearful02050 (44.1%)1593 (44.9%)2161 (60.6%)5804 (49.4%)
1703 (15.1%)677 (19.1%)519 (14.6%)1899 (16.1%)
2846 (18.2%)706 (19.9%)460 (12.9%)2012 (17.1%)
3620 (13.3%)397 (11.2%)312 (8.8%)1329 (11.3%)
4427 (9.2%)176 (5.0%)112 (3.1%)715 (6.1%)
Feeling Frustrated or Impatient01825 (39.3%)1483 (41.8%)1838 (51.6%)5146 (43.8%)
1817 (17.6%)721 (20.3%)587 (16.5%)2125 (18.1%)
21018 (21.9%)791 (22.3%)640 (18.0%)2449 (20.8%)
3606 (13.0%)423 (11.9%)375 (10.5%)1404 (11.9%)
4380 (8.2%)131 (3.7%)124 (3.5%)635 (5.4%)
Forgetfulness, poor memory02302 (49.5%)1810 (51.0%)2074 (58.2%)6186 (52.6%)
1893 (19.2%)742 (20.9%)518 (14.5%)2153 (18.3%)
2813 (17.5%)688 (19.4%)636 (17.8%)2137 (18.2%)
3432 (9.3%)249 (7.0%)245 (6.9%)926 (7.9%)
4206 (4.4%)60 (1.7%)91 (2.6%)357 (3.0%)
Poor Concentration02248 (48.4%)1656 (46.7%)2004 (56.2%)5908 (50.2%)
1888 (19.1%)823 (23.2%)561 (15.7%)2272 (19.3%)
2866 (18.6%)761 (21.4%)605 (17.0%)2232 (19.0%)
3425 (9.1%)248 (7.0%)286 (8.0%)959 (8.2%)
4219 (4.7%)61 (1.7%)108 (3.0%)388 (3.3%)
Taking Longer to Think02348 (50.5%)2042 (57.5%)2040 (57.2%)6430 (54.7%)
1864 (18.6%)711 (20.0%)537 (15.1%)2112 (18.0%)
2831 (17.9%)571 (16.1%)607 (17.0%)2009 (17.1%)
3414 (8.9%)177 (5.0%)293 (8.2%)884 (7.5%)
4189 (4.1%)48 (1.4%)87 (2.4%)324 (2.8%)
Blurred Vision02948 (63.5%)2149 (60.6%)2444 (68.6%)7541 (64.1%)
1759 (16.3%)670 (18.9%)466 (13.1%)1895 (16.1%)
2587 (12.6%)518 (14.6%)418 (11.7%)1523 (13.0%)
3249 (5.4%)175 (4.9%)166 (4.7%)590 (5.0%)
4103 (2.2%)37 (1.0%)70 (2.0%)210 (1.8%)
Light Sensitivity, easily upset by bright light03110 (66.9%)2088 (58.8%)2484 (69.7%)7682 (65.3%)
1664 (14.3%)632 (17.8%)476 (13.4%)1772 (15.1%)
2525 (11.3%)556 (15.7%)356 (10.0%)1437 (12.2%)
3227 (4.9%)215 (6.1%)176 (4.9%)618 (5.3%)
4120 (2.6%)58 (1.6%)72 (2.0%)250 (2.1%)
Double Vision03559 (76.6%)2688 (75.7%)2806 (78.7%)9053 (77.0%)
1592 (12.7%)472 (13.3%)394 (11.1%)1458 (12.4%)
2310 (6.7%)268 (7.6%)218 (6.1%)796 (6.8%)
3121 (2.6%)96 (2.7%)88 (2.5%)305 (2.6%)
464 (1.4%)25 (0.7%)58 (1.6%)147 (1.3%)
Restlessness02337 (50.3%)1782 (50.2%)2077 (58.3%)6196 (52.7%)
1714 (15.4%)739 (20.8%)493 (13.8%)1946 (16.5%)
2917 (19.7%)743 (20.9%)628 (17.6%)2288 (19.5%)
3439 (9.4%)223 (6.3%)263 (7.4%)925 (7.9%)
4239 (5.1%)62 (1.7%)103 (2.9%)404 (3.4%)
Note: N: absolute frequencies, %: relative frequencies, 0: not experienced at all, 1: no more of a problem (than before), 2: a mild problem, 3: a moderate problem, 4: a severe problem.
Table A2. Item characteristics.
Table A2. Item characteristics.
SampleItemNMSDSKKU
DutchHeadaches35640.961.210.88−0.50
Feeling of Dizziness35640.631.021.461.09
Nausea and/or Vomiting35640.430.882.153.98
Noise Sensitivity, easily upset by loud noise35640.631.071.561.31
Sleep Disturbance35641.191.330.68−0.82
Fatigue, tiring more easily35641.441.380.39−1.17
Being Irritable, easily angered35640.991.200.89−0.38
Feeling Depressed or Tearful35640.791.151.250.37
Feeling Frustrated or Impatient35640.981.200.90−0.39
Forgetfulness, poor memory35640.811.111.130.19
Poor Concentration35640.861.141.090.07
Taking Longer to Think35640.841.121.080.03
Blurred Vision35640.581.001.661.89
Light Sensitivity, easily upset by bright light35640.560.991.762.22
Double Vision35640.370.842.526.11
Restlessness35640.831.131.120.14
EnglishHeadaches46461.111.190.65−0.73
Feeling of Dizziness46460.811.101.100.10
Nausea and/or Vomiting46460.550.941.762.45
Noise Sensitivity, easily upset by loud noise46460.701.111.451.02
Sleep Disturbance46461.421.380.44−1.11
Fatigue, tiring more easily46461.591.380.27−1.20
Being Irritable, easily angered46461.271.300.60−0.85
Feeling Depressed or Tearful46461.281.380.63−0.96
Feeling Frustrated or Impatient46461.331.330.55−0.92
Forgetfulness, poor memory46461.001.200.94−0.23
Poor Concentration46461.031.210.90−0.29
Taking Longer to Think46460.971.190.96−0.19
Blurred Vision46460.671.031.481.30
Light Sensitivity, easily upset by bright light46460.621.031.641.81
Double Vision46460.390.832.365.33
Restlessness46461.041.240.87−0.42
ItalianHeadaches35491.191.130.38−1.07
Feeling of Dizziness35490.691.011.280.61
Nausea and/or Vomiting35490.620.901.361.05
Noise Sensitivity, easily upset by loud noise35490.751.061.230.45
Sleep Disturbance35491.141.220.65−0.75
Fatigue, tiring more easily35491.411.230.31−1.05
Being Irritable, easily angered35491.321.200.43−0.89
Feeling Depressed or Tearful35491.121.240.75−0.59
Feeling Frustrated or Impatient35491.151.190.64−0.71
Forgetfulness, poor memory35490.871.060.95−0.09
Poor Concentration35490.941.050.83−0.25
Taking Longer to Think35490.730.991.210.60
Blurred Vision35490.670.971.300.80
Light Sensitivity, easily upset by bright light35490.741.031.220.52
Double Vision35490.390.802.184.35
Restlessness35490.891.050.91−0.12
Total
(characteristics across language samples)
Min3549.000.370.800.27−1.20
Max4646.001.591.382.526.11
M3919.670.901.121.100.46
SD524.030.320.160.561.79
Note: N: absolute frequencies, M: mean, SD: standard deviation, SK: skewness, KU: kurtosis, Min: minimum, Max: maximum. Values in bold indicate acceptable SK and KU (−2 to +2).
Table A3. Psychometric properties of the RPQ in the general population samples.
Table A3. Psychometric properties of the RPQ in the general population samples.
SampleItemCronbach’s AlphaAlpha If Item OmittedSplit-Half ReliabilityItem-Total Correlation
DutchHeadaches0.940.940.960.55
Feeling of Dizziness0.940.62
Nausea and/or Vomiting0.940.56
Noise Sensitivity, easily upset by loud noise0.940.64
Sleep Disturbance0.940.65
Fatigue, tiring more easily0.930.71
Being Irritable, easily angered0.930.73
Feeling Depressed or Tearful0.930.73
Feeling Frustrated or Impatient0.930.74
Forgetfulness, poor memory0.930.74
Poor Concentration0.930.78
Taking Longer To Think0.930.76
Blurred Vision0.930.65
Light Sensitivity, easily upset by bright light0.930.66
Double Vision0.940.59
Restlessness0.930.73
EnglishHeadaches0.950.940.960.57
Feeling of Dizziness0.940.67
Nausea and/or Vomiting0.940.60
Noise Sensitivity, easily upset by loud noise0.940.68
Sleep Disturbance0.940.68
Fatigue, tiring more easily0.940.73
Being Irritable, easily angered0.940.76
Feeling Depressed or Tearful0.940.75
Feeling Frustrated or Impatient0.940.77
Forgetfulness, poor memory0.940.76
Poor Concentration0.940.80
Taking Longer to Think0.940.79
Blurred Vision0.940.65
Light Sensitivity, easily upset by bright light0.940.67
Double Vision0.940.59
Restlessness0.940.75
ItalianHeadaches0.920.920.950.49
Feeling of Dizziness0.920.56
Nausea and/or Vomiting0.920.55
Noise Sensitivity, easily upset by loud noise0.920.60
Sleep Disturbance0.920.63
Fatigue, tiring more easily0.920.70
Being Irritable, easily angered0.920.68
Feeling Depressed or Tearful0.920.68
Feeling Frustrated or Impatient0.920.72
Forgetfulness, poor memory0.920.67
Poor Concentration0.920.72
Taking Longer to Think0.920.70
Blurred Vision0.920.60
Light Sensitivity, easily upset by bright light0.920.60
Double Vision0.920.54
Restlessness0.920.68
Total
(characteristics across language samples)
Min0.920.920.950.49
Max0.950.940.960.80
M0.930.930.950.67
SD0.010.010.010.08
Note: Min: minimum, Max: maximum, M: mean, SD: standard deviation. Values in bold are within acceptable ranges.
Table A4. Examination of the factorial structure of the RPQ in the general population samples.
Table A4. Examination of the factorial structure of the RPQ in the general population samples.
Model Fit
Indices
Original One-Factor ModelThree-Factor model
Smith-Seemiller et al. (2003) [21]
DutchEnglishItalianDutchEnglishItalian
χ27042.4595050.5064054.5063382.9472056.3361846.032
df104104104101101101
p<0.001<0.001<0.001<0.001<0.001<0.001
χ2/df684939332018
CFI0.9880.9790.9890.9940.9920.995
TLI0.9870.9760.9870.9930.9900.994
RMSEA0.1200.1160.1030.0840.0740.070
CI90%[0.117, 0.122][0.113, 0.119][0.101, 0.106][0.081, 0.086][0.071, 0.077][0.067, 0.072]
SRMR0.0750.0800.0690.0610.0580.053
Note. χ2: chi square, df: degrees of freedom, χ2/df: ratio (cut-off: ≤ 2), p: p-value, CFI: Comparative Fit Index (cut-off: >0.95), TLI: Tucker–Lewis Index (cut-off: >0.95), RMSEA: root mean square error of approximation (cut-off: 0.05–0.10) with 90% confidence interval (CI), SRMR: standardized root mean square residual (cut-off: <0.08). Values in bold are within acceptable range. The grey-shaded model (i.e., the three-factors model) provides a better model fit.
Table A5. Results of the measurement invariance (MI) analyses.
Table A5. Results of the measurement invariance (MI) analyses.
SamplesConstrainsχ2dfpCFITLIRMSEACI90%Δχ2ΔdfΔpΔCFIΔRMSEA
Dutch
(General population sample vs. TBI sample)
baseline3604.965202<0.0010.9710.9660.090[0.088, 0.093]-----
thresholds3840.568233<0.0010.9690.9680.086[0.084, 0.089]155.0431<0.0010.0020.004
intercepts and thresholds3665.442246<0.0010.9710.9720.082[0.080, 0.084]15.154130.298−0.0020.004
English
(General population sample vs. TBI sample)
baseline5020.305202<0.0010.9740.9690.099[0.097, 0.101]-----
thresholds5093.583233<0.0010.9730.9730.093[0.090, 0.095]26.53310.6950.0010.006
intercepts and thresholds4713.479246<0.0010.9760.9760.086[0.084, 0.089]14.13130.365−0.0030.007
Italian
(General population sample vs. TBI sample)
baseline3413.452202<0.0010.9610.9540.091[0.089, 0.094]-----
thresholds3521.641233<0.0010.9600.9590.086[0.084, 0.089]77.7331<0.0010.0010.005
intercepts and thresholds3246.144246<0.0010.9640.9640.080[0.078, 0.082]13.82130.387−0.0040.006
Dutch vs. English vs. Italian
(General population samples)
baseline12,665.04303<0.0010.9630.9560.102[0.101, 0.104]-----
thresholds13,347.41365<0.0010.9610.9620.095[0.094, 0.097]145.6162<0.0010.0020.007
intercepts and thresholds12,725.06391<0.0010.9630.9660.090[0.088, 0.091]94.7926<0.001−0.0020.005
Note. χ2: chi square, df: degree of freedom, χ2/df: ratio (cut-off: ≤2), p: p-value, CFI: Comparative Fit Index (cut-off: >0.95), TLI: Tucker–Lewis Index (cut-off: >0.95), RMSEA: root mean square error of approximation (cut-off: <0.08) with 90% confidence interval (CI), Δχ2: change in chi square values between compared models, Δdf: change in degrees of freedom between compared models, ΔCFI: change in CFI between compared models (cut-off: <0.01), ΔRMSEA: change in RMSEA between compared models (cut-off: ≤0.01). Values in bold indicate at least satisfactory/mediocre model fit according to the respective cut-offs and/or are within acceptable range.
Figure A1. Relative difference between thresholds models (inverse of the back-transformed thresholds) of the Dutch general population and TBI samples. Items marked with red color indicate difference above acceptable cut-off of 5%.
Figure A1. Relative difference between thresholds models (inverse of the back-transformed thresholds) of the Dutch general population and TBI samples. Items marked with red color indicate difference above acceptable cut-off of 5%.
Jcm 11 04658 g0a1
Figure A2. Relative difference between thresholds models (inverse of the back-transformed thresholds) of the English general population and TBI samples. Items marked with red color indicate difference above acceptable cut-off of 5%.
Figure A2. Relative difference between thresholds models (inverse of the back-transformed thresholds) of the English general population and TBI samples. Items marked with red color indicate difference above acceptable cut-off of 5%.
Jcm 11 04658 g0a2
Figure A3. Relative difference between thresholds models (inverse of the back-transformed thresholds) of the Italian general population and TBI samples. Items marked with red color indicate difference above acceptable cut-off of 5%.
Figure A3. Relative difference between thresholds models (inverse of the back-transformed thresholds) of the Italian general population and TBI samples. Items marked with red color indicate difference above acceptable cut-off of 5%.
Jcm 11 04658 g0a3
Table A6. Results of regression analyses.
Table A6. Results of regression analyses.
RPQ Total ScoreSomaticEmotionalCognitive
VariableReference GroupEstimateS.E.tpEstimateS.E.tpEstimateS.E.tpEstimateS.E.tp
(Intercept)-8.760.6713.000.0004.340.3612.12<0.0012.850.2312.19<0.0011.560.188.94<0.001
ItalianEnglish−1.580.72−2.200.028−0.490.38−1.290.197−0.720.25−2.890.004−0.370.19−1.970.048
Dutch−0.340.74−0.460.6470.130.390.330.739−0.650.26−2.540.0110.180.190.950.342
41–64 years18–40 years−2.800.67−4.18<0.001−1.480.36−4.17<0.001−0.740.23−3.200.001−0.570.17−3.280.001
65–75 years−5.410.94−5.74<0.001−2.510.50−4.99<0.001−1.870.33−5.71<0.001−1.040.24−4.23<0.001
FemaleMale2.500.633.94<0.0011.250.343.71<0.0010.870.223.93<0.0010.390.162.340.019
MiddleLow−1.950.67−2.900.004−0.970.36−2.710.007−0.510.23−2.200.028−0.460.17−2.660.008
High−2.370.74−3.220.001−1.130.39−2.870.004−0.820.26−3.190.001−0.430.19−2.240.025
At least one health complaintNo health complaints13.170.6420.47<0.0016.100.3417.81<0.0014.300.2219.22<0.0012.780.1716.63<0.001
Italian * 41–64 yearsEnglish * 18–40 years2.020.583.480.0011.050.313.410.0010.560.202.770.0060.410.152.710.007
Dutch * 41–64 years−0.510.56−0.910.362−0.580.30−1.940.0530.120.190.640.522−0.060.15−0.400.687
Italian * 65–75 yearsEnglish * 18–40 years3.760.874.31<0.0011.570.463.390.0011.050.303.450.0011.140.235.05<0.001
Dutch * 65–75 years0.690.870.790.428−0.010.47−0.030.9750.490.301.610.1080.220.230.970.332
Italian * FemaleEnglish * Male1.130.532.110.0350.500.281.760.0780.580.193.130.0020.040.140.310.754
Dutch * Female−0.900.52−1.720.085−0.560.28−2.020.043−0.150.18−0.820.413−0.190.14−1.390.164
Italian * MiddleEnglish * Low0.640.621.030.3050.070.330.210.8330.410.221.920.0550.150.160.950.341
Dutch * Middle0.270.650.420.6770.200.340.590.5550.150.220.680.498−0.090.17−0.510.612
Italian * High2.570.843.060.0021.260.452.830.0050.880.293.040.0020.420.221.930.054
Dutch * High−0.190.72−0.260.7940.080.380.220.8260.120.250.470.638−0.390.19−2.090.037
Italian * At least one health complaintEnglish * No health complaints−4.150.54−7.74<0.001−1.620.29−5.69<0.001−1.440.19−7.76<0.001−1.080.14−7.76<0.001
Dutch * At least one health complaint−3.130.53−5.95<0.001−0.960.28−3.440.001−1.500.18−8.24<0.001−0.660.14−4.86<0.001
41–64 years * Female18–40 years * Male−0.070.47−0.150.8780.340.251.360.173−0.380.16−2.320.020−0.030.12−0.280.784
65–75 years * Female−0.590.71−0.830.4080.020.380.050.959−0.390.25−1.600.110−0.210.18−1.160.247
41–64 * Middle18–40 years * Low0.600.571.050.2920.360.301.200.2300.110.200.540.5880.130.150.880.381
65–75 * Middle0.330.840.390.6970.310.440.700.4820.030.290.090.929−0.010.22−0.060.950
41–64 * High0.640.670.960.3380.390.361.100.2710.060.230.270.7900.190.171.080.279
65–75 * High0.571.010.570.5720.120.540.220.8230.500.351.410.159−0.040.26−0.160.870
41–64*At least one health complaint18–40 * No health complaints−0.660.47−1.410.157−0.290.25−1.160.246−0.370.16−2.300.0210.000.120.010.993
65–75 * At least one health complaint−4.390.71−6.17<0.001−1.970.38−5.20<0.001−1.620.25−6.57<0.001−0.800.18−4.32<0.001
Female * MiddleMale * Low−0.340.51−0.670.504−0.140.27−0.500.614−0.240.18−1.330.1830.030.130.240.808
Female * High−0.520.62−0.840.399−0.150.33−0.470.638−0.200.21−0.920.356−0.170.16−1.050.294
Female * At least one health complaintMale * No health complaints0.520.431.210.2250.340.231.490.1360.180.151.230.2200.000.11−0.010.989
Middle * At least one health complaintLow * No health complaints0.530.521.020.3080.220.280.800.4220.280.181.530.1270.030.140.240.809
High * At least one health complaint−0.800.62−1.290.196−0.350.33−1.050.294−0.180.22−0.850.395−0.270.16−1.700.090
Note: *: interaction between the variables; Estimate: regression coefficient; S.E.: standard error; t: t-value; p: p-value; values in bold are significant at 5%.

Appendix B

Table A7. Reference values for the RPQ scale scores obtained from the English general population sample stratified by sex, health status, and age.
Table A7. Reference values for the RPQ scale scores obtained from the English general population sample stratified by sex, health status, and age.
Gender × Health Status x Age Low Symptom Severity −1 SD Md +1 SD High Symptom Severity
ScaleGenderHealth statusAgeN2.5%5%16%30%40%50%60%70%85%95%97.5%
SomaticMaleHealthy18–405660000022481618
41–644600000002241014
65–751180000000251212
At least one chronic health condition18–404460024681013182428
41–64542000456810162427
65–7515600002457111518
FemaleHealthy18–4047400002346101719
41–644220000023481317
65–751190000022461113
At least one chronic health condition18–405640036891114192628
41–646020036791114192527
65–7517700034679131821
Total464600022469142125
EmotionalMaleHealthy18–405660000002371012
41–6446000000002489
65–7511800000000489
At least one chronic health condition18–40446000257810121516
41–6454200024579121516
65–751560000023481214
FemaleHealthy18–404740000002481112
41–64422000000026910
65–7511900000000279
At least one chronic health condition18–40564002468911131616
41–64602000246810121516
65–751770000024691112
Total464600000247101415
CognitiveMaleHealthy18–4056600000000479
41–6446000000000266
65–7511800000000266
At least one chronic health condition18–404460000235691112
41–645420000024681112
65–7515600000022699
FemaleHealthy18–4047400000000479
41–6442200000000268
65–7511900000000144
At least one chronic health condition18–405640000246691212
41–646020000246691212
65–7517700000002689
Total4646000000247911
Note: 50% percentiles represent 50% of the distribution corresponding to the median (Md); SD: standard deviation; values from −1 standard deviation (16%) to +1 standard deviation (85%) are within the normal range (i.e., not clinically relevant symptom severity); values below 16% indicate low symptoms severity (i.e., absence of RPQ symptoms) and values above 85% indicate high symptom severity (i.e., presence of clinically relevant RPQ symptoms).
Table A8. Reference values for the RPQ scale scores obtained from the Italian general population sample stratified by sex, health status, and age.
Table A8. Reference values for the RPQ scale scores obtained from the Italian general population sample stratified by sex, health status, and age.
Gender × Health Status × Age Low Symptom Severity −1 SD Md +1 SD High Symptom Severity
ScaleGenderHealth statusAgeN2.5%5%16%30%40%50%60%70%85%95%97.5%
SomaticMaleHealthy18–404670000022471418
41–644540000022481418
65–75106000002247910
At least one chronic health condition18–40227002456912161923
41–64391000246710142124
65–7512500022468101719
FemaleHealthy18–4041900023456101618
41–6440300022457111618
65–75910000002471314
At least one chronic health condition18–403020026791013182325
41–644350025791113182326
65–75129002446810152225
Total354900022468131922
EmotionalMaleHealthy18–40467000000226910
41–64454000000026911
65–7510600000000367
At least one chronic health condition18–4022700024567101315
41–6439100002457101214
65–751250000022261113
FemaleHealthy18–404190000024681111
41–644030000002481112
65–7591000000006910
At least one chronic health condition18–4030200036789121416
41–6443500024679111315
65–751290000245791213
Total35490000024691213
CognitiveMaleHealthy18–4046700000000267
41–6445400000000267
65–7510600000000245
At least one chronic health condition18–4022700000235799
41–64391000000246910
65–7512500000024699
FemaleHealthy18–4041900000002479
41–6440300000000679
65–759100000000468
At least one chronic health condition18–40302000002457912
41–6443500000236799
65–7512900000024699
Total354900000002689
Note. 50% percentiles represent 50% of the distribution corresponding to the median (Md); SD: standard deviation; values from −1 standard deviation (16%) to +1 standard deviation (85%) are within the normal range (i.e., not clinically relevant symptom severity); values below 16% indicate low symptoms severity (i.e., absence of RPQ symptoms) and values above 85% indicate high symptom severity (i.e., presence of clinically relevant RPQ symptoms).
Table A9. Reference values for the RPQ scale scores obtained from the Dutch general population sample stratified by sex, health status, and age.
Table A9. Reference values for the RPQ scale scores obtained from the Dutch general population sample stratified by sex, health status, and age.
Gender × Health Status × Age Low Symptom Severity −1 SD Md +1 SD High Symptom Severity
ScaleGenderHealth statusAgeN2.5%5%16%30%40%50%60%70%85%95%97.5%
SomaticMaleHealthy18–404440000022471619
41–644120000002261215
65–759400000002247
At least one chronic health condition18–402760003581014182328
41–6444700024579152326
65–7510900000246101923
FemaleHealthy18–4036300002246101416
41–642980000022471315
65–7566000002225810
At least one chronic health condition18–403720036791113182327
41–64536002456810152023
65–7514700024679122024
Total356400002458131923
EmotionalMaleHealthy18–40444000000024910
41–64412000000004810
65–759400000000025
At least one chronic health condition18–4027600002457101315
41–6444700000246101214
65–75109000000025911
FemaleHealthy18–403630000002361011
41–64298000000024910
65–756600000000267
At least one chronic health condition18–4037200024578111416
41–6453600000246101214
65–751470000002461011
Total35640000002481213
CognitiveMaleHealthy18–4044400000000468
41–6441200000000266
65–759400000000036
At least one chronic health condition18–402760000024571012
41–644470000003671011
65–7510900000002678
FemaleHealthy18–4036300000002478
41–6429800000000267
65–756600000000266
At least one chronic health condition18–403720000034681112
41–64536000000457910
65–7514700000022699
Total3564000000026910
Note: 50% percentiles represent 50% of the distribution corresponding to the median (Md); SD: standard deviation; values from −1 standard deviation (16%) to +1 standard deviation (85%) are within the normal range (i.e., not clinically relevant symptom severity); values below 16% indicate low symptoms severity (i.e., absence of RPQ symptoms) and values above 85% indicate high symptom severity (i.e., presence of clinically relevant RPQ symptoms).

References

  1. Menon, D.K.; Schwab, K.; Wright, D.W.; Maas, A.I. Position Statement: Definition of Traumatic Brain Injury. Arch. Phys. Med. Rehabil. 2010, 91, 1637–1640. [Google Scholar] [CrossRef] [PubMed]
  2. Maas, A.I.R.; Menon, D.K.; Adelson, P.D.; Andelic, N.; Bell, M.J.; Belli, A.; Bragge, P.; Brazinova, A.; Büki, A.; Chesnut, R.M.; et al. Traumatic Brain Injury: Integrated Approaches to Improve Prevention, Clinical Care, and Research. Lancet Neurol. 2017, 16, 987–1048. [Google Scholar] [CrossRef]
  3. Powell, J.M.; Wise, E.K.; Brockway, J.A.; Fraser, R.; Temkin, N.; Bell, K.R. Characteristics and Concerns of Caregivers of Adults with Traumatic Brain Injury. J. Head Trauma Rehabil. 2017, 32, E33–E41. [Google Scholar] [CrossRef]
  4. Teasdale, G.; Jennett, B. Assessment of Coma and Impaired Consciousness. A Practical Scale. Lancet 1974, 2, 81–84. [Google Scholar] [CrossRef]
  5. Nelson, L.D.; Temkin, N.R.; Dikmen, S.; Barber, J.; Giacino, J.T.; Yuh, E.; Levin, H.S.; McCrea, M.A.; Stein, M.B.; Mukherjee, P.; et al. Recovery After Mild Traumatic Brain Injury in Patients Presenting to US Level I Trauma Centers: A Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) Study. JAMA Neurol. 2019, 76, 1049. [Google Scholar] [CrossRef] [PubMed]
  6. World Health Organization ICD-10: International Statistical Classification of Diseases and Related Health Problems, 10th Revision, 2nd Ed. 2004. Available online: https://apps.who.int/iris/handle/10665/42980 (accessed on 12 January 2022).
  7. King, N.S.; Crawford, S.; Wenden, F.J.; Moss, N.E.; Wade, D.T. The Rivermead Post Concussion Symptoms Questionnaire: A Measure of Symptoms Commonly Experienced after Head Injury and Its Reliability. J. Neurol. 1995, 242, 587–592. [Google Scholar] [CrossRef]
  8. Sigurdardottir, S.; Andelic, N.; Roe, C.; Jerstad, T.; Schanke, A.-K. Post-Concussion Symptoms after Traumatic Brain Injury at 3 and 12 Months Post-Injury: A Prospective Study. Brain Inj. 2009, 23, 489–497. [Google Scholar] [CrossRef]
  9. Zeldovich, M.; Wu, Y.-J.; Gorbunova, A.; Mikolic, A.; Polinder, S.; Plass, A.; Covic, A.; Asendorf, T.; Andelic, N.; Voormolen, D.; et al. Influence of Sociodemographic, Premorbid, and Injury-Related Factors on Post-Concussion Symptoms after Traumatic Brain Injury. J. Clin. Med. 2020, 9, 1931. [Google Scholar] [CrossRef]
  10. Wilde, E.A.; Whiteneck, G.G.; Bogner, J.; Bushnik, T.; Cifu, D.X.; Dikmen, S.; French, L.; Giacino, J.T.; Hart, T.; Malec, J.F.; et al. Recommendations for the Use of Common Outcome Measures in Traumatic Brain Injury Research. Arch. Phys. Med. Rehabil. 2010, 91, 1650–1660.e17. [Google Scholar] [CrossRef]
  11. NINDS Project Overview. Available online: https://www.commondataelements.ninds.nih.gov/Traumatic%20Brain%20Injury (accessed on 17 May 2022).
  12. von Steinbuechel, N.; Rauen, K.; Krenz, U.; Wu, Y.-J.; Covic, A.; Plass, A.; Cunitz, K.; Mueller, I.; Bockhop, F.; Polinder, S.; et al. Translation and Linguistic Validation of Outcome Instruments for Traumatic Brain Injury Research and Clinical Practice: A Step-by-Step Approach within the Observational CENTER-TBI Study. J. Clin. Med. 2021, 10, 2863. [Google Scholar] [CrossRef]
  13. Polinder, S.; Cnossen, M.C.; Real, R.G.L.; Covic, A.; Gorbunova, A.; Voormolen, D.C.; Master, C.L.; Haagsma, J.A.; Diaz-Arrastia, R.; von Steinbuechel, N. A Multidimensional Approach to Post-Concussion Symptoms in Mild Traumatic Brain Injury. Front. Neurol. 2018, 9, 1113. [Google Scholar] [CrossRef] [PubMed]
  14. Voormolen, D.C.; Cnossen, M.C.; Polinder, S.; Gravesteijn, B.Y.; Von Steinbuechel, N.; Real, R.G.L.; Haagsma, J.A. Prevalence of Post-Concussion-like Symptoms in the General Population in Italy, The Netherlands and the United Kingdom. Brain Inj. 2019, 33, 1078–1086. [Google Scholar] [CrossRef] [PubMed]
  15. Steyerberg, E.W.; Wiegers, E.; Sewalt, C.; Buki, A.; Citerio, G.; De Keyser, V.; Ercole, A.; Kunzmann, K.; Lanyon, L.; Lecky, F.; et al. Case-Mix, Care Pathways, and Outcomes in Patients with Traumatic Brain Injury in CENTER-TBI: A European Prospective, Multicentre, Longitudinal, Cohort Study. Lancet Neurol. 2019, 18, 923–934. [Google Scholar] [CrossRef]
  16. von Steinbuechel, N.; Rauen, K.; Bockhop, F.; Covic, A.; Krenz, U.; Plass, A.; Cunitz, K.; Polinder, S.; Wilson, L.; Steyerberg, E.; et al. Psychometric Characteristics of the Patient-Reported Outcome Measures Applied in the CENTER-TBI Study. J. Clin. Med. 2021, 10, 2396. [Google Scholar] [CrossRef] [PubMed]
  17. Maas, A.I.R.; Menon, D.K.; Steyerberg, E.W.; Citerio, G.; Lecky, F.; Manley, G.T.; Hill, S.; Legrand, V.; Sorgner, A. Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) A Prospective Longitudinal Observational Study. Neurosurgery 2015, 76, 67–80. [Google Scholar] [CrossRef]
  18. Bulmer, M.G. Principles of Statistics; Dover Publications: New York, NY, USA, 1979; ISBN 978-0-486-63760-0. [Google Scholar]
  19. Terwee, C.B.; Bot, S.D.M.; de Boer, M.R.; van der Windt, D.A.W.M.; Knol, D.L.; Dekker, J.; Bouter, L.M.; de Vet, H.C.W. Quality Criteria Were Proposed for Measurement Properties of Health Status Questionnaires. J. Clin. Epidemiol. 2007, 60, 34–42. [Google Scholar] [CrossRef]
  20. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; L. Erlbaum Associates: Hillsdale, NJ, USA, 1988; ISBN 978-0-8058-0283-2. [Google Scholar]
  21. Smith-Seemiller, L.; Fow, N.R.; Kant, R.; Franzen, M.D. Presence of Post-Concussion Syndrome Symptoms in Patients with Chronic Pain vs Mild Traumatic Brain Injury. Brain Inj. 2003, 17, 199–206. [Google Scholar] [CrossRef]
  22. Eyres, S.; Carey, A.; Gilworth, G.; Neumann, V.; Tennant, A. Construct Validity and Reliability of the Rivermead Post-Concussion Symptoms Questionnaire. Clin. Rehabil. 2005, 19, 878–887. [Google Scholar] [CrossRef]
  23. Potter, S.; Leigh, E.; Wade, D.; Fleminger, S. The Rivermead Post Concussion Symptoms Questionnaire: A Confirmatory Factor Analysis. J. Neurol. 2006, 253, 1603–1614. [Google Scholar] [CrossRef]
  24. Herrmann, N.; Rapoport, M.J.; Rajaram, R.D.; Chan, F.; Kiss, A.; Ma, A.K.; Feinstein, A.; Mc Cullagh, S.; Lanctôt, K.L. Factor Analysis of the Rivermead Post-Concussion Symptoms Questionnaire in Mild-to-Moderate Traumatic Brain Injury Patients. J. Neuropsychiatry Clin. Neurosci. 2009, 21, 181–188. [Google Scholar] [CrossRef]
  25. Lannsjö, M.; Borg, J.; Björklund, G.; af Geijerstam, L.; Lundgren-Nilsson, Å. Internal Construct Validity of the Rivermead Post-Concussion Symptoms Questionnaire. J. Rehabil. Med. 2011, 43, 997–1002. [Google Scholar] [CrossRef] [PubMed]
  26. Thomas, M.; Skilbeck, C.; Cannan, P.; Slatyer, M. The Structure of the Rivermead Post-Concussion Symptoms Questionnaire in Australian Adults with Traumatic Brain Injury. Brain Impair. 2018, 19, 166–182. [Google Scholar] [CrossRef]
  27. Agtarap, S.; Kramer, M.D.; Campbell-Sills, L.; Yuh, E.; Mukherjee, P.; Manley, G.T.; McCrea, M.A.; Dikmen, S.; Giacino, J.T.; Stein, M.B.; et al. Invariance of the Bifactor Structure of Mild Traumatic Brain Injury (MTBI) Symptoms on the Rivermead Postconcussion Symptoms Questionnaire Across Time, Demographic Characteristics, and Clinical Groups: A TRACK-TBI Study. Assessment 2021, 28, 1656–1670. [Google Scholar] [CrossRef] [PubMed]
  28. Zeldovich, M.; Bockhop, F.; Plass, A.M.; Covic, A.; Mueller, I.; Polinder, S.; Mikolic, A.; van der Vlegel, M.; Steinbuechel, N.; CENTER-TBI Participants and Investigators. Comparability of the Six Rivermead Post-Concussion Symptoms Questionnaire Translations: Results from the CENTER-TBI Study.
  29. Rivera, D.; Greving, S.; Arango-Lasprilla, J.C.; von Steinbuechel, N.; Zeldovich, M.; CENTER-TBI Participants and Investigators. Comparability of (Post-Concussion) Symptoms across Time in Individuals after Traumatic Brain Injury: Results from the CENTER-TBI Study. J. Clin. Med. 2022, 11, 4090. [Google Scholar] [CrossRef] [PubMed]
  30. Brown, T.A. Confirmatory Factor Analysis for Applied Research, 2nd ed.; Methodology in the Social Sciences; The Guilford Press: New York, NY, USA; London, UK, 2015; ISBN 978-1-4625-1779-4. [Google Scholar]
  31. Cole, D.A. Utility of Confirmatory Factor Analysis in Test Validation Research. J. Consult. Clin. Psychol. 1987, 55, 584–594. [Google Scholar] [CrossRef]
  32. Bentler, P.M. Comparative Fit Indexes in Structural Models. Psychol. Bull. 1990, 107, 238–246. [Google Scholar] [CrossRef]
  33. Hu, L.; Bentler, P.M. Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  34. Bentler, P.M.; Bonett, D.G. Significance Tests and Goodness of Fit in the Analysis of Covariance Structures. Psychol. Bull. 1980, 88, 588–606. [Google Scholar] [CrossRef]
  35. Steiger, J.H. Statistically Based Tests for the Number of Common Factors. In Proceedings of the the Annual Meeting of the Psychometric Society, Iowa City, IA, USA; 1980. [Google Scholar]
  36. Finch, W.H.; French, B.F. A Simulation Investigation of the Performance of Invariance Assessment Using Equivalence Testing Procedures. Struct. Equ. Model. Multidiscip. J. 2018, 25, 673–686. [Google Scholar] [CrossRef]
  37. Xia, Y.; Yang, Y. RMSEA, CFI, and TLI in Structural Equation Modeling with Ordered Categorical Data: The Story They Tell Depends on the Estimation Methods. Behav. Res. Methods 2019, 51, 409–428. [Google Scholar] [CrossRef]
  38. Wu, H.; Estabrook, R. Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes. Psychometrika 2016, 81, 1014–1045. [Google Scholar] [CrossRef] [PubMed]
  39. Svetina, D.; Rutkowski, L.; Rutkowski, D. Multiple-Group Invariance with Categorical Outcomes Using Updated Guidelines: An Illustration Using Mplus and the Lavaan/Semtools Packages. Struct. Equ. Model. Multidiscip. J. 2020, 27, 111–130. [Google Scholar] [CrossRef]
  40. Hirschfeld, G.; von Brachel, R. Improving Multiple-Group Confirmatory Factor Analysis in R—A Tutorial in Measurement Invariance with Continuous and Ordinal Indicators. Pract. Assess. Res. Eval. 2014, 19, 7. [Google Scholar] [CrossRef]
  41. Cheung, G.W.; Rensvold, R.B. Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Struct. Equ. Model. 2002, 9, 233–255. [Google Scholar] [CrossRef]
  42. Ware, J.E.; Kosinski, M.; Bjorner, J.B.; Turner-Bowker, D.M.; Gandek, B.; Maruish, M.E. User’s Manual for the SF-36v2 Health Survey, 2nd ed.; QualityMetric Incorporated: Lincoln, RI, USA, 2007. [Google Scholar]
  43. R Core Team R: A Language and Environment for Statistical Computing 2021. Vienna, Austria. Available online: https://www.R-project.org/ (accessed on 12 January 2022).
  44. Rich, B. Table1: Tables of Descriptive Statistics in HTML. 2021. Available online: https://CRAN.R-project.org/package=table1 (accessed on 12 January 2022).
  45. Revelle, W. Psych: Procedures for Personality and Psychological Research. 2019. Available online: https://CRAN.R-project.org/package=psych (accessed on 12 January 2022).
  46. Rosseel, Y. Lavaan: An R Package for Structural Equation Modeling. J. Stat. Softw. 2012, 48, 1–36. [Google Scholar] [CrossRef]
  47. Chan, R.C. Base Rate of Post-Concussion Symptoms among Normal People and Its Neuropsychological Correlates. Clin. Rehabil. 2001, 15, 266–273. [Google Scholar] [CrossRef]
  48. Zakzanis, K.K.; Yeung, E. Base Rates of Post-Concussive Symptoms in a Nonconcussed Multicultural Sample. Arch. Clin. Neuropsychol. 2011, 26, 461–465. [Google Scholar] [CrossRef]
  49. Iverson, G.L.; Lange, R.T. Examination of “Postconcussion-Like” Symptoms in a Healthy Sample. Appl. Neuropsychol. 2003, 10, 137–144. [Google Scholar] [CrossRef]
  50. Iverson, G.L.; Zasler, N.D.; Lange, R.T. Post-Concussive Disorders. In Brain Injury Medicine: Principles and Practice; Demos Medical Publishing: New York, NY, USA, 2007; ISBN 978-1-888799-93-4. [Google Scholar]
  51. Garden, N.; Sullivan, K.A. An Examination of the Base Rates of Post-Concussion Symptoms: The Influence of Demographics and Depression. Appl. Neuropsychol. 2010, 17, 1–7. [Google Scholar] [CrossRef]
  52. Wang, Y.; Chan, R.; Deng, Y. Examination of Postconcussion-like Symptoms in Healthy University Students: Relationships to Subjective and Objective Neuropsychological Function Performance. Arch. Clin. Neuropsychol. 2006, 21, 339–347. [Google Scholar] [CrossRef]
  53. Dean, P.J.A.; O’Neill, D.; Sterr, A. Post-Concussion Syndrome: Prevalence after Mild Traumatic Brain Injury in Comparison with a Sample without Head Injury. Brain Inj. 2012, 26, 14–26. [Google Scholar] [CrossRef] [PubMed]
  54. van der Vlegel, M.; Polinder, S.; Toet, H.; Panneman, M.J.M.; Haagsma, J.A. Prevalence of Post-Concussion-Like Symptoms in the General Injury Population and the Association with Health-Related Quality of Life, Health Care Use, and Return to Work. J. Clin. Med. 2021, 10, 806. [Google Scholar] [CrossRef] [PubMed]
  55. McLean, S.A.; Kirsch, N.L.; Tan-Schriner, C.U.; Sen, A.; Frederiksen, S.; Harris, R.E.; Maixner, W.; Maio, R.F. Health Status, Not Head Injury, Predicts Concussion Symptoms after Minor Injury. Am. J. Emerg. Med. 2009, 27, 182–190. [Google Scholar] [CrossRef]
  56. Snell, D.L.; Martin, R.; Macleod, A.D.; Surgenor, L.J.; Siegert, R.J.; Jean, E.J.; Hay-Smith, C.; Melzer, T.; Hooper, G.J.; Anderson, T. Untangling Chronic Pain and Post-Concussion Symptoms: The Significance of Depression. Brain Inj. 2018, 32, 583–592. [Google Scholar] [CrossRef] [PubMed]
  57. Ulrichsen, K.M.; Kaufmann, T.; Dørum, E.S.; Kolskår, K.K.; Richard, G.; Alnæs, D.; Arneberg, T.J.; Westlye, L.T.; Nordvik, J.E. Clinical Utility of Mindfulness Training in the Treatment of Fatigue After Stroke, Traumatic Brain Injury and Multiple Sclerosis: A Systematic Literature Review and Meta-Analysis. Front. Psychol. 2016, 7, 912. [Google Scholar] [CrossRef] [PubMed]
  58. Wright, K.B. Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services. J. Comput.-Mediat. Commun. 2006, 10. [Google Scholar] [CrossRef]
  59. Gorbunova, A.; Zeldovich, M.; Voormolen, D.C.; Krenz, U.; Polinder, S.; Haagsma, J.A.; Hagmayer, Y.; Covic, A.; Real, R.G.L.; Asendorf, T.; et al. Reference Values of the QOLIBRI from General Population Samples in the United Kingdom and The Netherlands. J. Clin. Med. 2020, 9, 2100. [Google Scholar] [CrossRef]
Figure 1. Composition of the study samples.
Figure 1. Composition of the study samples.
Jcm 11 04658 g001
Table 1. Sample characteristics.
Table 1. Sample characteristics.
General Population SamplesTBI SampleComparison 5
GroupEnglishItalianDutchTotalEnglishDutchItalianTotalp
Variable(General Population Samples/
TBI Samples)
(N = 4646)(N = 3549)(N = 3564)(N = 11759)(N = 223)(N = 597)(N = 268)(N = 1088)
AgeMean (SD)44.1 (15.6)45.0 (14.8)44.8 (15.3)44.6 (15.3)48.3 (17.1)52.9 (19.1)50.2 (20.6)51.3 (19.2)All group comparisons < 0.001
Median
[Min, Max]
44.0
[18.0, 75.0]
45.0
[18.0, 75.0]
45.0
[18.0, 75.0]
44.0
[18.0, 75.0]
51.0
[16.0, 85.0]
57.0
[16.0, 95.0]
53.0
[16.0, 93.0]
54.0
[16.0, 95.0]
Age in groups 118–24/16–24601 (12.9%)346 (9.7%)429 (12.0%)1376 (11.7%)29 (13.0%)74 (12.4%)46 (17.2%)149 (13.7%)-
25–34880 (18.9%)608 (17.1%)598 (16.8%)2086 (17.7%)25 (11.2%)57 (9.5%)30 (11.2%)112 (10.3%)
35–44918 (19.8%)806 (22.7%)732 (20.5%)2456 (20.9%)28 (12.6%)56 (9.4%)29 (10.8%)113 (10.4%)
45–54912 (19.6%)726 (20.5%)743 (20.8%)2381 (20.2%)60 (26.9%)84 (14.1%)39 (14.6%)183 (16.8%)
55–64765 (16.5%)612 (17.2%)646 (18.1%)2023 (17.2%)44 (19.7%)142 (23.8%)48 (17.9%)234 (21.5%)
65–75/65+570 (12.3%)451 (12.7%)416 (11.7%)1437 (12.2%)37 (16.6%)184 (30.8%)76 (28.4%)297 (27.3%)
Gender/sex 2Male2288 (49.2%)1770 (49.9%)1782 (50.0%)5840 (49.7%)74 (33.2%)229 (38.4%)84 (31.3%)387 (35.6%)All group comparisons < 0.001
Female2358 (50.8%)1779 (50.1%)1782 (50.0%)5919 (50.3%)149 (66.8%)368 (61.6%)184 (68.7%)701 (64.4%)
Education 3Low/
primary school/none/unknown
1066 (22.9%)1200 (33.8%)1064 (29.9%)3330 (28.3%)2 (0.9%)40 (6.7%)64 (23.9%)106 (9.7%)All group comparisons < 0.001
Middle/
secondary school/high school
1986 (42.7%)1968 (55.5%)1601 (44.9%)5555 (47.2%)140 (62.8%)363 (60.8%)64 (23.9%)567 (52.1%)
High/
post-high school
1594 (34.3%)381 (10.7%)899 (25.2%)2874 (24.4%)61 (27.4%)125 (20.9%)101 (37.7%)287 (26.4%)
Missing----20 (9.0%)69 (11.6%)39 (14.6%)128 (11.8%)
Chronic health complaints/
TBI severity 4
no chronic health complaints/
mild TBI
2159 (46.5%)1940 (54.7%)1677 (47.1%)5776 (49.1%)160 (71.7%)474 (79.4%)174 (64.9%)808 (74.3%)
at least one chronic health complaint/
moderate/severe TBI
2487 (53.5%)1609 (45.3%)1887 (52.9%)5983 (50.9%)62 (27.8%)108 (18.1%)94 (35.1%)264 (24.3%)-
Missing----1 (0.4%)15 (2.5%)0 (0%)16 (1.5%)
RPQ total scoreMean (SD)13.2 (14.0)11.6 (11.9)10.7 (12.4)12.0 (13.0)15.1 (12.7)14.8 (13.0)14.1 (11.7)14.7 (12.6)-
Median
[Min, Max]
8.00 [0, 64.0]8.00 [0, 64.0]6.00 [0, 64.0]8.00 [0, 64.0]13.0 [0, 56.0]12.0 [0, 64.0]12.0 [0, 56.0]12.0 [0, 64.0]-
Somatic scaleMean (SD)6.52 (7.14)6.03 (6.37)5.71 (6.64)6.13 (6.77)7.02 (6.41)7.15 (6.72)6.75 (6.03)7.02 (6.49)-
Median
[Min, Max]
4.00 [0, 36.0]4.00 [0, 36.0]4.00 [0, 36.0]4.00 [0, 36.0]6.00 [0, 30.0]5.00 [0, 36.0]6.00 [0, 29.0]5.50 [0, 36.0]-
Emotional scaleMean (SD)4.27 (4.79)3.66 (4.20)2.97 (4.08)3.69 (4.44)4.12 (4.00)3.76 (4.14)3.78 (3.78)3.84 (4.03)-
Median
[Min, Max]
2.00 [0, 16.0]2.00 [0, 16.0]0 [0, 16.0]2.00 [0, 16.0]3.00 [0, 16.0]3.00 [0, 16.0]3.00 [0, 16.0]3.00 [0, 16.0]-
Cognitive ScaleMean (SD)2.43 (3.47)1.90 (2.90)2.05 (3.14)2.15 (3.22)3.92 (3.65)3.85 (3.51)3.62 (3.24)3.81 (3.47)-
Median
[Min, Max]
0 [0, 12.0]0 [0, 12.0]0 [0, 12.0]0 [0, 12.0]3.00 [0, 12.0]3.00 [0, 12.0]3.00 [0, 12.0]3.00 [0, 12.0]-
1 Age in groups slightly differs between general population samples (i.e., 18–24 and 65–75) and TBI samples (i.e., 16–24, 65+). 2 Gender is provided for the general population samples, information on sex is collected for the TBI samples. 3 Educational attainment was categorized as low (lower school), middle (comprehensive school), and high (college and university) for the general population samples and primary school, secondary/high school and post-high school for the TBI samples. 4 Chronic health complaints are provided for the general population samples, information on TBI severity (GCS; mild ≥ 13, moderate/severe ≤ 12) is reported for the TBI samples. 5 General population and TBI samples were compared in terms of age in years (t-test), sex, and education (Chi-squared-test); significance level was set at 5%. Note. N: absolute frequencies; %: relative frequencies; SD: standard deviation; Min: minimum; Max: maximum.
Table 2. Distribution of chronic health complaints reported by participants from general population samples.
Table 2. Distribution of chronic health complaints reported by participants from general population samples.
EnglishItalianDutchTotal
N (%)N (%)N (%)N (%)
Asthma602 (13.0%)258 (7.3%)336 (9.4%)1196 (10.2%)
Heart Disease109 (2.3%)68 (1.9%)102 (2.9%)279 (2.4%)
Stroke74 (1.6%)40 (1.1%)81 (2.3%)195 (1.7%)
Diabetes390 (8.4%)247 (7.0%)274 (7.7%)911 (7.7%)
Back complaints567 (12.2%)224 (6.3%)355 (10.0%)1146 (9.7%)
Arthrosis141 (3.0%)345 (9.7%)346 (9.7%)832 (7.1%)
Rheumatism192 (4.1%)235 (6.6%)218 (6.1%)645 (5.5%)
Cancer128 (2.8%)66 (1.9%)140 (3.9%)334 (2.8%)
Dementia82 (1.8%)62 (1.7%)94 (2.6%)238 (2.0%)
Aging problems205 (4.4%)149 (4.2%)82 (2.3%)436 (3.7%)
Depression1254 (27.0%)522 (14.7%)423 (11.9%)2199 (18.7%)
Other 1493 (10.6%)354 (10.0%)687 (19.3%)1534 (13.0%)
Total4646 (100%)3549 (100%)3564 (100%)11,759 (100%)
1 Choosing category “other” implied adding a comment in an open text field. Note. N = absolute frequencies, relative frequencies.
Table 3. Reference values for the RPQ total score obtained from the English general population sample stratified by sex, health status, and age.
Table 3. Reference values for the RPQ total score obtained from the English general population sample stratified by sex, health status, and age.
Gender × Health Status × Age Low Symptoms Severity −1 SD Md +1 SD High Symptoms Severity
GenderHealth statusAgeN2.5%5%16%30%40%50%60%70%85%95%97.5%
MaleHealthy18–4056600000258163033
41–6446000000024102128
65–751180000002492124
At least one chronic health condition18–40446004915192428364650
41–64542002610151924344853
65–751560002461113243140
FemaleHealthy18–40474000036812213134
41–6442200000348162530
65–7511900000234101824
At least one chronic health condition18–405640071317212529374952
41–646020061015202429384852
65–7517700247101216253639
Total46460002581319304148
Note: 50% percentiles represent 50% of the distribution corresponding to the median (Md); SD: standard deviation; values from −1 standard deviation (16%) to +1 standard deviation (85%) are within the normal range (i.e., not clinically relevant symptom intensity); values below 16% indicate low symptoms intensity (i.e., absence of RPQ symptoms) and values above 85% indicate high symptom intensity (i.e., presence of clinically relevant RPQ symptoms).
Table 4. Reference values for the RPQ total score obtained from the Italian general population sample stratified by sex, health status, and age.
Table 4. Reference values for the RPQ total score obtained from the Italian general population sample stratified by sex, health status, and age.
Gender × Health Status × Age Low Symptoms Severity −1 SD Md +1 SD High Symptoms Severity
GenderHealth statusAgeN2.5%5%16%30%40%50%60%70%85%95%97.5%
MaleHealthy18–4046700002347142632
41–6445400000247142632
65–7510600000246101719
At least one chronic health condition18–40227002711151823313839
41–6439100248111520293944
65–751250002581014193136
FemaleHealthy18–404190003471115213032
41–64403000246812203133
65–759100000267122431
At least one chronic health condition18–403020051115172227334044
41–644350061014172024334247
65–75129002710121618273739
Total35490002581216253540
Note: 50% percentiles represent 50% of the distribution corresponding to the median (Md); SD: standard deviation; values from −1 standard deviation (16%) to +1 standard deviation (85%) are within the normal range (i.e., not clinically relevant symptom intensity); values below 16% indicate low symptoms intensity (i.e., absence of RPQ symptoms) and values above 85% indicate high symptom intensity (i.e., presence of clinically relevant RPQ symptoms).
Table 5. Reference values for the RPQ total score obtained from the Dutch general population sample stratified by sex, health status, age, and education level.
Table 5. Reference values for the RPQ total score obtained from the Dutch general population sample stratified by sex, health status, age, and education level.
Gender × Health Status × Age Low Symptoms Severity −1 SD Md +1 SD High Symptoms Severity
GenderHealth statusAgeN2.5%5%16%30%40%50%60%70%85%95%97.5%
MaleHealthy18–4044400000246142732
41–6441200000024112128
65–7594000000024810
At least one chronic health condition18–4027600068141924334449
41–6444700047101520294247
65–75109000024610182835
FemaleHealthy18–40363000024610182832
41–6429800002248132329
65–756600000234101819
At least one chronic health condition18–403720051114182126334551
41–6453600268121621293744
65–751470004681115223235
Total3564000246914253543
Note: 50% percentiles represent 50% of the distribution corresponding to the median (Md); SD: standard deviation; values from −1 standard deviation (16%) to +1 standard deviation (85%) are within the normal range (i.e., not clinically relevant symptom intensity); values below 16% indicate low symptoms intensity (i.e., absence of RPQ symptoms) and values above 85% indicate high symptom intensity (i.e., presence of clinically relevant RPQ symptoms).
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Zeldovich, M.; Bockhop, F.; Covic, A.; Cunitz, K.; Polinder, S.; Haagsma, J.A.; von Steinbuechel, N.; on behalf of the CENTER-TBI Participants and Investigators. Reference Values for the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) from General Population Samples in the United Kingdom, Italy, and The Netherlands. J. Clin. Med. 2022, 11, 4658. https://doi.org/10.3390/jcm11164658

AMA Style

Zeldovich M, Bockhop F, Covic A, Cunitz K, Polinder S, Haagsma JA, von Steinbuechel N, on behalf of the CENTER-TBI Participants and Investigators. Reference Values for the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) from General Population Samples in the United Kingdom, Italy, and The Netherlands. Journal of Clinical Medicine. 2022; 11(16):4658. https://doi.org/10.3390/jcm11164658

Chicago/Turabian Style

Zeldovich, Marina, Fabian Bockhop, Amra Covic, Katrin Cunitz, Suzanne Polinder, Juanita A. Haagsma, Nicole von Steinbuechel, and on behalf of the CENTER-TBI Participants and Investigators. 2022. "Reference Values for the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) from General Population Samples in the United Kingdom, Italy, and The Netherlands" Journal of Clinical Medicine 11, no. 16: 4658. https://doi.org/10.3390/jcm11164658

APA Style

Zeldovich, M., Bockhop, F., Covic, A., Cunitz, K., Polinder, S., Haagsma, J. A., von Steinbuechel, N., & on behalf of the CENTER-TBI Participants and Investigators. (2022). Reference Values for the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) from General Population Samples in the United Kingdom, Italy, and The Netherlands. Journal of Clinical Medicine, 11(16), 4658. https://doi.org/10.3390/jcm11164658

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop