Next Article in Journal
“I RUN CLEAN Project”—An Innovative and Self-Sustainable Approach to Develop Clean Sport Behaviours in Grassroots Athletes
Next Article in Special Issue
The Evolution of Psychological Distress Levels in University Students in Spain during Different Stages of the COVID-19 Pandemic: Risk and Protective Factors
Previous Article in Journal
Physical Exercise Methods and Their Effects on Glycemic Control and Body Composition in Adults with Type 2 Diabetes Mellitus (T2DM): A Systematic Review
Previous Article in Special Issue
The 5Cs of Positive Youth Development and Risk Behaviors in a Sample of Spanish Emerging Adults: A Partial Mediation Analysis of Gender Differences
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
EJIHPE
Volume 13
Issue 11
10.3390/ejihpe13110177
ejihpe-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessment and Psychometric Properties of the 21-Item Depression Anxiety Stress Scale (DASS-21) among Portuguese Higher Education Students during the COVID-19 Pandemic

by
Carlos Laranjeira
1,2,3,*,
Ana Querido
1,2,4,
Pedro Sousa
2,5 and
Maria Anjos Dixe
1,2
1
School of Health Sciences, Polytechnic University of Leiria, Campus 2, Morro do Lena, Alto do Vieiro, Apartado 4137, 2411-901 Leiria, Portugal
2
Centre for Innovative Care and Health Technology (ciTechCare), Rua de Santo André—66–68, Campus 5, Polytechnic University of Leiria, 2410-541 Leiria, Portugal
3
Comprehensive Health Research Centre (CHRC), University of Évora, 7000-801 Évora, Portugal
4
Center for Health Technology and Services Research (CINTESIS), NursID, University of Porto, 4200-450 Porto, Portugal
5
The Health Sciences Research Unit: Nursing (UICISA: E), Nursing School of Coimbra (ESEnfC), Polo A, Av. Bissaya Barreto, 3004-011 Coimbra, Portugal
*
Author to whom correspondence should be addressed.
Eur. J. Investig. Health Psychol. Educ. 2023, 13(11), 2546-2560; https://doi.org/10.3390/ejihpe13110177
Submission received: 8 September 2023 / Revised: 20 October 2023 / Accepted: 30 October 2023 / Published: 6 November 2023
(This article belongs to the Special Issue Mental Health during COVID-19 Pandemic: What Do We Know So Far?)
Browse Figure
Versions Notes

Abstract

:
The COVID-19 pandemic has caused substantial disruptions in the lives of higher education students, with detrimental repercussions for academic performance and overall mental health. Therefore, we aimed to evaluate the prevalence of depression, anxiety, and stress symptoms among Portuguese higher education students during the first wave of the coronavirus pandemic and investigate DASS-21’s psychometric characteristics and whether it functions effectively during a pandemic. A convenience sampling procedure was used to recruit 1522 participants (75.1% women and 79.2% undergraduate students) for this cross-sectional research. Participants completed an e-survey created using DASS-21. The results revealed a considerable prevalence of symptoms of depression [≥10] (N = 434, 28.5%), anxiety [≥7] (N = 551, 36.2%), and stress [≥11] (N = 544, 35.7%). Confirmatory factor analysis (CFA) revealed the scale’s three-factor structure, which matched the three DASS-21 subscales. Subsequently, the heterotrait–monotrait (HTMT) correlation ratio evaluated the scale’s discriminant validity, which was relatively good. Cronbach’s alpha measured the internal consistency of the DASS subscales, which was excellent (Cronbach’s α > 0.90). DASS-21 was shown to be a reliable and appropriate measure for assessing students’ mental health. Furthermore, DASS-21 is recommended for use by academics and healthcare professionals in measuring students’ psychological distress. Further validation studies of this scale are needed with larger and more representative samples.

1. Introduction

Depression and anxiety are two of the most difficult mental health challenges faced by young people [1]. The COVID-19 pandemic has heightened this concern [2,3,4]. Several studies have suggested that youths endured more of the pandemic’s tribulations than the general population [5,6]. The prevalence of psychopathological symptoms increased threefold in several countries due to the COVID-19 pandemic’s sanitary crisis [7]. Research shows that the SARS-CoV-2 virus promoted anxiety and depression in COVID-19 survivors, causing direct brain cell invasion, cytokine storms, and neurodegenerative processes [8]. Anosmia, ageusia, and headache are the most frequently reported neurologic symptoms among COVID-19 patients. However, severe adverse events have also been documented, including stroke, loss of consciousness, seizures, cognitive impairment, and increased psychiatric disorders [9,10,11]. Furthermore, the widespread propagation of the coronavirus, its severe effects, and the lockdown measures to reduce infection rates implied restricted social connections and widespread feelings of loneliness, as inferred by increasing internet use (a negative coping strategy) [8].
Several international studies documented negative psychological effects among university students because of the COVID-19 pandemic [12,13,14]. Portugal followed this tendency, exhibiting increased rates of anxiety and depression during the pandemic [5,15,16,17,18]. The combination of pandemic-related distress and restrictive preventative measures could have exacerbated pre-existing mental health conditions, giving rise to new symptoms in individuals who had not previously experienced mental health concerns [19]. Accurate psychological tools for screening anxiety and depression symptoms applicable to diverse groups are critical for identifying individuals requiring the attention of mental health professionals [20]. However, the selection of a mental health status assessment instrument may be influenced by several criteria, such as the clinical situation, practitioner type, patient characteristics, and how the findings will be used [21].
The 21-item Depression Anxiety Stress Scale (DASS-21) is a brief self-report scale that measures emotional states of depression, anxiety, and stress (within the last 7 days) [20]. This scale is based on DASS-42, initially developed by Lovibond and Lovibond [22] to cover the full range of core symptoms of emotional distress. Afterward, Antony et al. [23] used seven questions from each original instrument’s subscale to illustrate DASS-21’s reliability and validity.
DASS-21 is a widely used mental health assessment tool that is simple to use and an accurate tool for research on adults (its intended audience) [24]. This instrument has already been used in research related to virus outbreaks [25]. The three DASS-21 subscales have strong internal consistency and construct validity [26], distinguishing between clinical [27,28,29,30] and nonclinical groups [26,31,32,33]. A tripartite model was proposed to distinguish depression from anxiety and stress [23].
While not a clinical diagnostic measure, “DASS-21 is often used in research and practice in clinical and non-clinical samples to identify individuals in high distress who may be more prone to develop psychopathologies” [34] (p. 4). The original English version of DASS-21 has been assessed in many different countries and 56 languages [35]. DASS-21 has also been used in various situations and age groups, with conflicting findings [36]. There are presently several models, including the three-factor—like in the original DASS-21 research [37,38]—second-order three-factor [39], two-factor [31], one-factor [40], and four-factor [29] models. Several factors may be responsible for differences, including the type of sample (clinical/non-clinical), age of participants, and cultural aspects [41,42].
Considering that the DASS-21 scale has conflicting psychometric properties, we undertook a psychometric investigation to establish the factor structure and reliability of the Portuguese version of DASS-21 when psychological distress becomes ubiquitous, as during the COVID-19 pandemic. Although validated across a variety of demographics, DASS-21 has not been examined with higher education students in Portugal. Therefore, evaluating the use of DASS-21 to assess students’ mental health is essential. To overcome this gap, three specific aims were defined: (a) to evaluate the prevalence of depression, anxiety, and stress symptoms in a sample of Portuguese higher education students during the first wave of the coronavirus pandemic; (b) to determine the reliability, factor structure, and discriminant validity of the DASS-21 scale; and (c) to establish the associations between the DASS-21 subscales and sex, age, marital status, and academic qualifications.
We expected a three-factor model would best match the data. We also anticipated that the DASS-21 scores would vary by age, sex, marital status, and educational background. Finally, we expected significant correlations between the DASS-21 subscales and self-perceptions of physical, mental, and global health.

2. Materials and Methods

2.1. Study Design

This methodological study targeted higher education students during lockdown in Portugal as a response to the COVID-19 pandemic. It employed cross-sectional data gathered between April and October 2020 (the early phase of the coronavirus pandemic).

2.2. Recruitment and Sample

Our participants were Portuguese higher education students enrolled at four educational institutions for the 2020/2021 academic year. These institutions are medium-sized facilities with similar activity profiles located in Portugal’s central region. The sample size was calculated using the formula “sample size = number of items × number of participants”, which is often used in survey development research. We computed “the sample size based on one item and ten participants” [43] (p. 1). Based on the number of DASS-21 items, the minimum intended sample size was 210 answers.
Eligible participants were adults (over the age of 18) with an education level above high school and the ability to read and comprehend Portuguese. International mobility students were excluded because the e-survey was only developed in Portuguese.
All participants received the necessary information to provide informed consent regarding their participation in the study. They were informed about the scales and the purpose of the research. Given the COVID-19 pandemic, an e-survey was conducted using a convenience sample approach. Google Forms was used to distribute the survey. Participants were contacted using institutional mailing lists sent by the participating institutions’ communication offices and provided a link to access the online questionnaire. Participants could only reply once (multiple responses were blocked), thus reducing the risk of selection bias and anonymizing their identities. We received 1522 valid responses during the survey activation period.

2.3. Instruments

The e-survey was composed of three parts:
(1)
Basic and academic information, including age, sex, marital status, level of education, and type of educational institution.
(2)
Self-perceived physical, mental, and global health, rated on a ten-point scale (1 = poor health to 10 = excellent health), using the previous month as a reference.
(3)
Participants’ mental health status, using the Portuguese version of the Depression, Anxiety, and Stress Scale (DASS-21) [38]. Respondents indicated “how much each statement applied to them over the last week on a four-point Likert scale (0 = did not apply to me at all; to 3 = applied to me very much or most of the time)” [44] (p. 3). The scores were added together to provide a total score (0–63) or subtotals for each of the seven subscales (0–21). As recommended for DASS procedures [45], up to one missing item per subscale was permitted for computing the mean score, with missing items replaced by the mean. As DASS-21 is a reduced version of the main scale, which has 42 items, the final score of each subscale must be doubled [45]. The cut-off scores for clinical depression, anxiety, and stress were 10 or more, 7 or more, and 11 or more, respectively [22].

2.4. Data Analysis

Frequencies, percentages, medians, ranges, means, and standard deviations (SDs) were used in descriptive data analysis. The data were screened for missing values and normal distribution before analysis. No missing data were detected. The multivariate normality of the variables was assessed using skewness (coefficient of asymmetry) and kurtosis. Severe univariate normality violations to normality were considered when skewness was greater than three and kurtosis greater than seven [46].
Psychometric characteristics of the DASS-21 scale were determined based on fidelity and validity tests [46]. To examine the scale’s reliability, the following criteria were considered: (a) each item’s Pearson correlation with the total scale, wherein correlations greater than 0.20 were desired and other correlations were corrected [47]; (b) the Cronbach’s alpha of all items in each subscale and the scale as a whole, after excluding each item individually. “Cronbach’s alpha allowed us to evaluate the instrument’s internal consistency, which can range from 0 to 1, with higher values indicating better internal consistency” [48] (p. 1). A Cronbach’s alpha value greater than 0.8 indicates good internal consistency, while a value lower than 0.5 is unacceptable [48]. An exploratory factor analysis procedure was carried out, with loadings based on a principal axis factoring with varimax rotation. Factor loadings below 0.5 were suppressed, while item cross-loadings over 0.2 were systematically eliminated, one by one [48].
Construct validity was performed using confirmatory factorial analysis (CFA). Chi-square tests are widely used to evaluate how well models fit the data. However, since the χ2 statistic is sensitive to sample size, other goodness of fit metrics were applied, namely the goodness of fit index (GFI), the comparative fit index (CFI), the root mean square residual (RMR) index, parsimony-adjusted measures (PNFI and PCFI), the HOELTER index, the expected cross-validation index (ECVI), and Akaike information criteria (AIC) [48]. In this study, two models were tested: Model 1, Lovibond’s original three-factor structure; and Model 2, a three-factor structure of the 21-item DASS based on eliminating items with lower loadings and item–total correlations.
The present study investigated all the correlations among the depression, anxiety, and stress subscales and self-rated perceptions of health to examine the discriminant validity. A heterotrait–monotrait (HTMT) ratio of correlation close to one indicated a lack of discriminant validity. A Pearson correlation coefficient (r) lower than 0.85 indicated a variable’s discriminant validity [49].
An independent-sample t-test was used to determine whether there were statistically significant differences between male and female participants in the DASS subscales. The relationship between scale scores and age was calculated using Pearson correlation coefficients. A one-way ANOVA was carried out to examine the relationship between the DASS-21 scale and other sociodemographic characteristics. Post hoc comparisons were performed using the Bonferroni method.
The Statistical Package for Social Sciences (SPSS version 28 for Windows, IBM Corp., Chicago, IL, USA) was used to perform the statistical tests. JASP Version 0.13.1.0 [50] was used for confirmatory factor analysis.

2.5. Ethics

The study protocol was authorized by the Ethics Committee of the Polytechnic University of Leiria (CE/IPLEIRIA/22/2020), and the investigation was conducted following the Declaration of Helsinki criteria. Students’ permission was obtained before each e-survey. Participants who agreed to participate completed an informed consent form by ticking the “Yes, I Agree” box, rather than the “No thanks” option, on the online form. To promote truthful replies, the responses were completely anonymized. A unique identification code was assigned to each participating student to associate the questions asked with the corresponding student. Participation was entirely voluntary and unrewarded.

3. Results

3.1. Sample Description

The study sample included 1522 individuals, 24.9% male and 75.1% female. The mean age of the participants was 22.85 ± 6.95 (ranging from 18 to 59). Most of the students were single (91.2%; n = 1388), undergraduates (79.2%; n = 1205), and from the public education system (94.1%; n = 1435). Participants reported a mean of 6.51 ± 1.99 for physical health, 6.16 ± 2.14 for mental health, and 6.69 ± 1.79 for global health when asked about their health perceptions during the previous month.

3.2. Base Statistics

Table 1 depicts the base statistics of the DASS-21. Each item’s distribution exhibited positive skewness (ranging from 0.187 to 1.204) and mixed (positive and negative) kurtosis (ranging from −1.081 to 0.377). We assumed that skewness between −2 and +2 and kurtosis between −7 and +7 suggested a normally distributed variable [44]. The DASS-21 scores revealed a considerable prevalence of symptoms of depression [≥10] (N = 434, 28.5%), anxiety [≥7] (N = 551, 36.2%), and stress [≥11] (N = 544, 35.7%).

3.3. Reliability of DASS-21

Cronbach’s alpha was used to calculate internal consistency reliability. The overall alpha was 0.961, and the reliability scores of the three subscales varied from 0.900 (Anxiety) to 0.923 (Stress). Corrected item–total r values varied from 0.536 (item 2) to 0.815 (item 12); factor loadings ranged from 0.569 (item 2) to 0.862 (item 12) (Table 2). Only anxiety item 2, “mouth dryness”, was at the limit of 0.5; however, it was acceptable. The average of the participants’ depression was at the normal level (0–9), as was their anxiety (0–6) and stress (0–10).

3.4. Construct Validity

Construct validity was determined using CFA based on the structure of the original DASS-21 scale. Three latent variables, 21 observable variables, and 21 error terms constituted the model framework. The goodness of fit values were reasonable (CFI = 0.931, PNFI = 0.818, PCFI = 0.825, and RMR = 0.038).
Model refinement was based on modification indices whenever they were adequate from statistical and substantive points of view [51]. Correlation trajectories were established between the errors of items 6 and 18, 8 and 14, 11 and 12, 4 and 7, and 17 and 21.
Both estimated models were compared by analyzing comparative measures of fit with AIC (Akaike information criteria) and ECVI (the expected cross-validation index). The Model 1 scores were AIC = 1955.215 and ECVI = 1.285. The Model 2 scores were AIC = 1455.204 and ECVI = 0.957. Model 2 was deemed the best-fitting model (Table 3) since lower values suggest a better fit.
The goodness of fit values in Model 2 yielded a three-factor structure and were adequate (Figure 1), attesting to the factorial validity of the Portuguese version of DASS-21: GFI = 0.919, CFI = 0.952, PNFI = 0.814, PCFI = 0.820, and RMSEA = 0.065.

3.5. Discriminant Validity

Except for the correlation between the anxiety and stress subscales (r = 0.889), the variables all exhibited correlations lower than 0.85. The influence of mental health status, physical health status, and overall health status on the DASS-21 subscale scores was also evaluated. Correlations were statistically significant (p < 0.001) and negative, ranging between −0.275 (anxiety–physical health) and −0.539 (stress–mental health) (Table 4).

3.6. Associations between DASS-21 Subscales and Sociodemographic Variables

Associations between DASS-21 and sociodemographic variables were assessed by comparing the mean scores across age, sex, marital status, and academic qualifications. The influence of age on the DASS-21 subscale scores demonstrated that correlations were statistically significant (p < 0.001) and negative for stress (r = −0.119), anxiety (r = −0.163), and depression (r = −0.151).
The mean scores of the stress and anxiety subscales differed among the sexes. Females reported greater stress (8.909 + 5.716 vs. 6.982 + 5.636; t = 5.708, p < 0.001) and anxiety than males (6.086 + 5.618 vs. 4.464 + 4.880; t = 5.392, p < 0.001) (Table 5).
Concerning marital status, statistically significant differences (p < 0.001) were found among the three DASS-21 subscales (Table 6). A one-way ANOVA with Bonferroni post hoc correction showed significant differences between marital status (between the single and married groups) regarding stress (p < 0.001), anxiety (p < 0.001), and depression (p < 0.001).
Concerning academic qualifications, the one-way ANOVA test revealed statistically significant differences (p < 0.05) in the stress and anxiety subscales (Table 7). However, with the Bonferroni post hoc correction, the test only showed significant differences between the Graduation and Other groups regarding stress (p = 0.012).

4. Discussion

This study endeavored to evaluate the prevalence of anxiety, depression, and stress symptoms, to examine the psychometric properties of the DASS-21 scale, and to determine the associations between the DASS-21 subscales and sex, age, marital status, and academic qualifications in a large sample of Portuguese adults.
According to Pieh et al. [54], the COVID-19 pandemic and subsequent lockdown measures induced considerable levels of stress among younger people. University students faced significant challenges due to the transition to online study, social isolation from peers, limited access to leisure activities, and the inability to engage in regular student relationships [55]. In our study, the prevalence of mild to extremely severe anxiety, depression, and stress was found in 36.2%, 28.5%, and 35.7% of students, respectively. Similar results were found in previous international studies involving university students during the early stages of the pandemic [56,57]. Previous studies conducted in Portugal reported a lower prevalence of psychopathological symptoms [15,58]. Other studies reported levels of depression and stress higher than our findings, 37.5% and 49.0%, respectively [59]. Maia and Dias [60] also found a notable escalation in psychological distress during the pandemic period in comparison to non-pandemic times. However, the aforementioned data are cross-sectional; hence, a definitive conclusion regarding the causal or direct impact of the pandemic is precluded.
The DASS scale has become an increasingly popular assessment tool for evaluating psychological distress in clinical and non-clinical populations. According to our findings, the Portuguese version of DASS-21 has good psychometric properties. In line with prior research [32,38,61,62,63,64,65,66,67], our results support the DASS-21’s three-factor structure, providing additional evidence of DASS-21’s ability to adjust to many cultures. In addition, these findings will help cross-cultural comparisons [38]. Other studies conducted in Portugal found a different structure [31,68]. In a recent systematic review, the best-fitting models were consistently bifactor models, indicating an underlying unidimensional construct [69].
The reliability of DASS-21 was very good: the reliability of the three subscales ranged from 0.900 (anxiety) to 0.923 (stress). Similar results were found in other studies indicating a three-factor structure [34,38,39,61,63,70], wherein Cronbach’s alpha for depression, anxiety, and stress was ≥0.80 overall, implying good internal consistency reliability. Furthermore, the CFA model’s indexes demonstrated that DASS-21′s three-dimensional structure achieved an adequate fit for the data, and the results were consistent with earlier validation studies [71,72].
Item analysis showed that each scale’s items had good discrimination indices (corrected item–total correlation), indicating that, although the items in each subscale were logically homogeneous, they were not identical or comparable in form and substance. These findings are consistent with those of earlier research [73].
In terms of discriminant validity, the stress and anxiety subscales were substantially correlated, with values higher than those recommended by Kline [49]. These higher correlations may reflect a considerable overlap in the DASS-21 scale content, suggesting a generic concept, such as emotional distress. A similar study found a strong link between these subscales [74]. Discriminant validity was also evaluated between mental health status, physical health status, and overall health status and DASS-21 subscale scores. Correlations were statistically significant and negative for anxiety–physical health and stress–mental health. However, determining validity with other instruments and gold standard criteria, as has been achieved in other studies, is important [26,38,39,61,75].
The associations between personal characteristics and the Portuguese DASS-21 scale scores were also assessed. The examination of differential item functioning (DIF) indicated that the DASS-21 scores varied between the sexes. Our findings revealed that both sexes responded similarly in only three items in the depression subscale and one in the stress subscale. Female participants outperformed males on all DASS-21 subscales, in line with results found in Portugal (depression [38]) and other countries (anxiety and stress) [25,76]. Our results might be explained by the fact that women are more expressive than males and are more sensitive to mental health issues. Unlike our study, other studies found that men exhibit more mental health symptoms [77]. Correlations between age and the DASS-21 subscales were statistically significant and negative for stress, anxiety, and depression, suggesting that DASS-21 is substantially sensitive to age. As people get older, they probably experience fewer mental health issues because their social lives are more restricted than those of younger people. Consequently, they were less negatively impacted by the pandemic than young people. Similar results were obtained via other studies [78].
Concerning academic qualifications, the stress subscale correlated positively with education [26]. In our study, we found statistically significant differences in the stress and anxiety subscales. In terms of marital status, statistically significant differences were found among the three DASS-21 subscales. Our results demonstrated significant differences in stress, anxiety, and depression according to marital status.
Overall, our findings provide evidence that demographic variables seem to influence DASS-21 scores in the Portuguese version. These social determinants have been recognized as psychosocial risk factors that contribute to the occurrence of mental illnesses [79,80]. Nevertheless, more research is required to validate the link between these characteristics and mental disorders.
Our research included a large sample size and improved the comprehension and use of the scale in a particular historic moment (the COVID-19 pandemic). Nonetheless, some limitations of this research should be addressed when interpreting the findings, such as the manner of administration and the fact that we collected data over the internet. Additional research should be conducted to examine whether there are any significant differences when using alternative modalities of delivery (i.e., clinician interviews and paper-and-pencil questionnaires). DASS-21 was used as a self-assessment tool to detect depression, anxiety, and stress, and no objective clinical examination was performed to validate whether students were in distress. DASS-21 provides a snapshot of a current mental health status, and participants can choose not to disclose their symptoms in order to counteract the prejudice and stigma associated with mental health difficulties [81]. Similarly, self-assessments might be influenced by social desirability bias. The selection of a convenient, non-representative sample of the Portuguese population may impair conclusions and psychometric properties. Our sample included an unbalanced number of female and male participants, as well as a significant age gap between the youngest and oldest students. The present results predominantly reflect female undergraduate students from the public education system. Future research in this area should concentrate on sampling Portuguese percentile norms with a more representative sample in terms of age and gender [31], as well as investigating DASS-21’s longitudinal qualities for test–retest reliability. Further research may also include the study of personality traits and environmental variables that might potentially have a direct influence on the occurrence of depression, anxiety, and stress symptoms among students.
Despite these limitations, our findings have practical implications. While the pandemic no longer constitutes a public health emergency, detecting enduring long-term psychological disorders among students is necessary. Educational institutions and faculty members must prioritize the provision of enhanced mental health support and assistance for students. Online psychological counseling has proven to be especially advantageous during the most critical stages of the COVID-19 pandemic [82,83]. Valid and easily available instruments for psychiatric screening and assessment may considerably assist mental healthcare. DASS-21 is an effective screening tool for stress, anxiety, and depression symptoms. Furthermore, the scale may be used to assess the impacts of health promotion initiatives. The utility of DASS-21 extends beyond the scope of the COVID-19 pandemic and may apply to many other local or global emergencies.

5. Conclusions

The present study revealed a higher prevalence of mild-to-extreme symptoms of stress compared to complaints of anxiety and depression, so stress management training might be of high value to support students. Despite significant changes in environmental conditions since the COVID-19 pandemic, the current research found that DASS-21 maintained its reliability and validity, justifying its usage as a mental health screening tool among Portuguese students. The results demonstrated that DASS-21’s three-factor structure has excellent internal consistency. This survey might be expanded to include a broader national representation of students since it only included higher education students in Portugal’s central area.

Author Contributions

Conceptualization, A.Q., C.L. and M.A.D.; methodology, A.Q. and C.L.; software, C.L. and M.A.D.; validation, A.Q., C.L. and M.A.D.; formal analysis, P.S. and M.A.D.; investigation, A.Q., C.L. and M.A.D.; resources, A.Q. and C.L.; data curation, A.Q., C.L. and M.A.D.; writing—original draft preparation, M.A.D., P.S. and C.L.; writing—review and editing, C.L., A.Q., P.S. and M.A.D.; visualization, C.L., A.Q., P.S. and M.A.D.; supervision, A.Q. and C.L.; project administration, A.Q. and C.L.; funding acquisition, C.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by FCT—Fundação para a Ciência e a Tecnologia, I.P. (UIDB/05704/2020 and UIDP/05704/2020) and the Scientific Employment Stimulus—Institutional Call [CEECINST/00051/2018].

Institutional Review Board Statement

This study was conducted following the Declaration of Helsinki and approved by the Institutional Ethics Committee of the Polytechnic University of Leiria (protocol approval no. 22/2020).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Participation in the study was completely voluntary and anonymous. Implicit consent for the project was assumed when study participants completed the survey. Participants received no compensation.

Data Availability Statement

All data generated or analyzed during this study are included in this article.

Acknowledgments

We acknowledge all the volunteers who participated in the online survey to make this study possible.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.

References

  1. Jurewicz, I. Mental health in young adults and adolescents–supporting general physicians to provide holistic care. Clin. Med. 2015, 15, 151–154. [Google Scholar] [CrossRef] [PubMed]
  2. Penninx, B.W.J.H.; Benros, M.E.; Klein, R.S.; Vinkers, C.H. How COVID-19 shaped mental health: From infection to pandemic effects. Nat. Med. 2022, 28, 2027–2037. [Google Scholar] [CrossRef] [PubMed]
  3. Hossain, M.M.; Nesa, F.; Das, J.; Aggad, R.; Tasnim, S.; Bairwa, M.; Ma, P.; Ramirez, G. Global burden of mental health problems among children and adolescents during COVID-19 pandemic: An umbrella review. Psychiatry Res. 2022, 317, 114814. [Google Scholar] [CrossRef] [PubMed]
  4. Racine, N.; McArthur, B.A.; Cooke, J.E.; Eirich, R.; Zhu, J.; Madigan, S. Global Prevalence of Depressive and Anxiety Symptoms in Children and Adolescents During COVID-19: A Meta-analysis. JAMA Pediatr. 2021, 175, 1142–1150. [Google Scholar] [CrossRef] [PubMed]
  5. Bell, I.H.; Nicholas, J.; Broomhall, A.; Bailey, E.; Bendall, S.; Boland, A.; Robinson, J.; Adams, S.; McGorry, P.; Thompson, A. The impact of COVID-19 on youth mental health: A mixed methods survey. Psychiatry Res. 2023, 321, 115082. [Google Scholar] [CrossRef]
  6. Stock, C.; Helmer, S.M.; Heinrichs, K. COVID-19 related disruption in higher education students’ health and wellbeing: Implications for university action. Front. Public Health 2022, 10, 1015352. [Google Scholar] [CrossRef] [PubMed]
  7. Zhang, Z.; Feng, Y.; Song, R.; Yang, D.; Duan, X. Prevalence of psychiatric diagnosis and related psychopathological symptoms among patients with COVID-19 during the second wave of the pandemic. Glob. Health 2021, 17, 44. [Google Scholar] [CrossRef]
  8. Siste, K.; Hanafi, E.; Sen, L.T.; Murtani, B.J.; Christian, H.; Limawan, A.P.; Siswidiani, L.P.; Adrian. Implications of COVID-19 and Lockdown on Internet Addiction Among Adolescents: Data From a Developing Country. Front. Psychiatry 2021, 12, 665675. [Google Scholar] [CrossRef]
  9. Zubair, A.S.; McAlpine, L.S.; Gardin, T.; Farhadian, S.; Kuruvilla, D.E.; Spudich, S. Neuropathogenesis and neurologic manifestations of the coronaviruses in the age of coronavirus disease 2019: A review. JAMA Neurol. 2020, 77, 1018–1027. [Google Scholar] [CrossRef]
  10. Liu, L.; Ni, S.Y.; Yan, W.; Lu, Q.-D.; Zhao, Y.-M.; Xu, Y.-Y.; Mei, H.; Shi, L.; Yuan, K.; Han, Y.; et al. Mental and neurological disorders and risk of COVID-19 susceptibility, illness severity and mortality: A systematic review, meta-analysis and call for action. EClinicalMedicine 2021, 40, 101111. [Google Scholar] [CrossRef]
  11. Dehghani, A.; Zokaei, E.; Kahani, S.M.; Alavinejad, E.; Dehghani, M.; Meftahi, G.H.; Afarinesh, M.R. The potential impact of Covid-19 on CNS and psychiatric sequels. Asian J. Psychiatry 2022, 72, 103097. [Google Scholar] [CrossRef] [PubMed]
  12. La Rosa, V.L.; Commodari, E. University Experience during the First Two Waves of COVID-19: Students’ Experiences and Psychological Wellbeing. Eur. J. Investig. Health Psychol. Educ. 2023, 13, 1477–1490. [Google Scholar] [CrossRef] [PubMed]
  13. Islam, M.S.; Sujan, M.S.H.; Tasnim, R.; Sikder, M.T.; Potenza, M.N.; van Os, J. Psychological responses during the COVID-19 outbreak among university students in Bangladesh. PLoS ONE 2020, 15, e0245083. [Google Scholar] [CrossRef] [PubMed]
  14. Di Consiglio, M.; Merola, S.; Pascucci, T.; Violani, C.; Couyoumdjian, A. The Impact of COVID-19 Pandemic on Italian University Students’ Mental Health: Changes across the Waves. Int. J. Environ. Res. Public Health 2021, 18, 9897. [Google Scholar] [CrossRef]
  15. Laranjeira, C.; Dixe, M.A.; Querido, A. Mental Health Status and Coping among Portuguese Higher Education Students in the Early Phase of the COVID-19 Pandemic. Eur. J. Investig. Health Psychol. Educ. 2023, 13, 429–439. [Google Scholar] [CrossRef]
  16. Peixoto, M.M.; Cunha, O. COVID-19 fear and anxiety as mediators in the relationship between repetitive negative thinking and psychopathology: A community-based study during the 2nd confinement in Portugal. Curr. Psychol. 2023, 1–7. [Google Scholar] [CrossRef]
  17. Paulino, M.; Dumas-Diniz, R.; Brissos, S.; Brites, R.; Alho, L.; Simões, M.R.; Silva, C. COVID-19 in Portugal: Exploring the immediate psychological impact on the general population. Psychol. Health Med. 2021, 26, 44–55. [Google Scholar] [CrossRef]
  18. Laranjeira, C.; Querido, A.; Marques, G.; Silva, M.; Simões, D.; Gonçalves, L.; Figueiredo, R. COVID-19 pandemic and its psychological impact among healthy Portuguese and Spanish nursing students. Health Psychol. Res. 2021, 9, 24508. [Google Scholar] [CrossRef]
  19. Cullen, W.; Gulati, G.; Kelly, B.D. Mental health in the COVID-19 pandemic. QJM 2020, 11, 311–312. [Google Scholar] [CrossRef]
  20. Lee, J.; Lee, E.H.; Moon, S.H. Systematic review of the measurement properties of the Depression Anxiety Stress Scales-21 by applying updated COSMIN methodology. Qual. Life Res. 2019, 28, 2325–2339. [Google Scholar] [CrossRef]
  21. Peters, L.; Peters, A.; Andreopoulos, E.; Pollock, N.; Pande, R.L.; Mochari-Greenberger, H. Comparison of DASS-21, PHQ-8, and GAD-7 in a virtual behavioral health care setting. Heliyon 2021, 7, e06473. [Google Scholar] [CrossRef] [PubMed]
  22. Lovibond, P.F.; Lovibond, S.H. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav. Res. Ther. 1995, 33, 335–343. [Google Scholar] [CrossRef] [PubMed]
  23. Antony, M.M.; Bieling, P.J.; Cox, B.J.; Enns, M.; Swinson, R.P. Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychol. Assess. 1998, 10, 176–181. [Google Scholar] [CrossRef]
  24. Shaw, T.; Campbell, M.A.; Runions, K.C.; Zubrick, S.R. Properties of the DASS-21 in an Australian Community Adolescent Population. J. Clin. Psychol. 2017, 73, 879–892. [Google Scholar] [CrossRef]
  25. McAlonan, G.M.; Lee, A.M.; Cheung, V.; Cheung, C.; Tsang, K.W.T.; Sham, P.C.; Chua, S.E.; Wong, J.G. Immediate and sustained psychological impact of an emerging infectious disease outbreak on health care workers. Can. J. Psychiatry 2007, 52, 241–247. [Google Scholar] [CrossRef]
  26. Bottesi, G.; Ghisi, M.; Altoè, G.; Conforti, E.; Melli, G.; Sica, C. The Italian version of the Depression Anxiety Stress Scales-21: Factor structure and psychometric properties on community and clinical samples. Compr. Psychiatry 2015, 60, 170–181. [Google Scholar] [CrossRef]
  27. Alfonsson, S.; Wallin, E.; Maathz, P. Factor structure and validity of the Depression, Anxiety and Stress Scale-21 in Swedish translation. J. Psychiatr. Ment. Health Nurs. 2017, 24, 154–162. [Google Scholar] [CrossRef]
  28. Wong, D.; Dahm, J.; Ponsford, J. Factor structure of the Depression Anxiety Stress Scales in individuals with traumatic brain injury. Brain Inj. 2013, 27, 1377–1382. [Google Scholar] [CrossRef]
  29. Johnson, C.E.; Bennett, K.S.; Newton, J.; McTigue, J.; Taylor, S.; Musiello, T.; Lau, P.K.H. A pilot study to assess the validity of the DASS-21 subscales in an outpatient oncology population. Psychooncology 2018, 27, 695–699. [Google Scholar] [CrossRef]
  30. Rusli, B.N.; Amrina, K.; Trived, S.; Loh, K.P.; Shashi, M. Construct validity and internal consistency reliability of the Malay version of the 21-item depression anxiety stress scale (Malay-DASS-21) among male outpatient clinic attendees in Johor. Med. J. Malays. 2017, 72, 264–270. [Google Scholar]
  31. Apóstolo, J.L.; Mendes, A.C.; Azeredo, Z.A. Adaptation to Portuguese of the Depression, Anxiety and Stress Scales (DASS). Rev. Lat. Am. Enferm. 2006, 14, 863–871. [Google Scholar] [CrossRef] [PubMed]
  32. Henry, J.D.; Crawford, J.R. The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Br. J. Clin. Psychol. 2005, 44, 227–239. [Google Scholar] [CrossRef] [PubMed]
  33. Sinclair, S.J.; Siefert, C.J.; Slavin-Mulford, J.M.; Stein, M.B.; Renna, M.; Blais, M.A. Psychometric evaluation and normative data for the Depression, Anxiety, and Stress Scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Eval. Health Prof. 2012, 35, 259–279. [Google Scholar] [CrossRef] [PubMed]
  34. Ali, A.M.; Alkhamees, A.A.; Hori, H.; Kim, Y.; Kunugi, H. The Depression Anxiety Stress Scale 21: Development and Validation of the Depression Anxiety Stress Scale 8-Item in Psychiatric Patients and the General Public for Easier Mental Health Measurement in a Post COVID-19 World. Int. J. Environ. Res. Public Health 2021, 18, 10142. [Google Scholar] [CrossRef]
  35. Psychology Foundation of Australia. Depression, Anxiety and Stress Scales. 2022. Available online: https://www2.psy.unsw.edu.au/dass/translations.htm (accessed on 7 August 2023).
  36. Osman, A.; Wong, J.L.; Bagge, C.L.; Freedenthal, S.; Gutierrez, P.M.; Lozano, G. The Depression Anxiety Stress Scales-21 (DASS-21): Further examination of dimensions, scale reliability, and correlates. J. Clin. Psychol. 2012, 68, 1322–1338. [Google Scholar] [CrossRef]
  37. Allen, J.; Annells, M. A literature review of the application of the Geriatric Depression Scale, Depression Anxiety Stress Scales and Post-traumatic Stress Disorder Checklist to community nursing cohorts. J. Clin. Nurs. 2009, 18, 949–959. [Google Scholar] [CrossRef]
  38. Apóstolo, J.L.; Tanner, B.A.; Arfken, C.L. Confirmatory factor analysis of the Portuguese Depression Anxiety Stress Scales-21. Rev. Lat. Am. Enferm. 2012, 20, 590–596. [Google Scholar] [CrossRef]
  39. Daza, P.; Novy, D.M.; Stanley, M.A.; Averill, P. The Depression Anxiety Stress Scale-21: Spanish translation and validation with a Hispanic sample. J. Psychopathol. Behav. 2002, 24, 195–205. [Google Scholar] [CrossRef]
  40. Camacho, Á.; Cordero, E.D.; Perkins, T. Psychometric Properties of the DASS-21 Among Latina/o College Students by the US-Mexico Border. J. Immigr. Minor. Health 2016, 18, 1017–1023. [Google Scholar] [CrossRef]
  41. Antúnez, Z.; Vinet, E.V. Escalas de Depresión, Ansiedad y Estrés (DASS- 21): Validación de la versión abreviada en estudiantes universitarios Chilenos [Depression Anxiety Stress Scales (DASS-21): Validation of the abbreviated version in Chilean university students]. Ter. Psicol. 2012, 30, 49–55. [Google Scholar] [CrossRef]
  42. González-Rivera, J.A.; Pagán-Torres, O.M.; Pérez-Torres, E.M. Depression, Anxiety and Stress Scales (DASS-21): Construct Validity Problem in Hispanics. Eur. J. Investig. Health Psychol. Educ. 2020, 10, 375–389. [Google Scholar] [CrossRef] [PubMed]
  43. Suresh, K.; Chandrashekara, S. Sample size estimation and power analysis for clinical research studies. J. Hum. Reprod. Sci. 2012, 5, 7–13. [Google Scholar] [CrossRef] [PubMed]
  44. Tubić, T.; Živanović, B.; Lakićević, N.; Zenić, N.; Gilić, B.; Rudas, E.; Eliseev, S.; Trivić, T.B.; Bianco, A.; Drid, P. Psychological Distress in Elite Sambo and Recreational Athletes. Front. Psychol. 2022, 13, 781880. [Google Scholar] [CrossRef] [PubMed]
  45. Lovibond, P.F. Depression Anxiety and Stress Scales (DASS). 2014. Available online: https://www2.psy.unsw.edu.au/groups/dass/ (accessed on 7 August 2023).
  46. Marôco, J. Análise de Equações Estruturais: Fundamentos Teóricos, Software & Aplicações, 3rd ed.; Report Number: Lisboa, Portugal, 2021. [Google Scholar]
  47. Hajjar, S. Statistical analysis: Internal-consistency reliability and construct validity. Int. J. Quan. Qualit. Res. Meth. 2018, 6, 46–57. [Google Scholar]
  48. Pestana, M.; Gageiro, J. Análise de Dados Para Ciências Sociais: A Complementaridade do SPSS, 6th ed.; Edições Sílabo: Lisboa, Portugal, 2014. [Google Scholar]
  49. Kline, R.B. Principles and Practice of Structural Equation Modeling, 5th ed.; The Guilford Press: New York, NY, USA, 2023. [Google Scholar]
  50. JASP Team. JASP, version 0.13.1.0; [Computer software]; JASP Team: Amsterdam, The Netherlands, 2020. [Google Scholar]
  51. Byrne, B.M. Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, 2nd ed.; Routledge/Taylor & Francis Group: Oxford, UK, 2010. [Google Scholar]
  52. Biza-Khupe, S. An exploration of new frontiers in personal finance using structural equation modeling (SEM)—Towards improved research. Rev. Financ. Account. Stud. 2012, 2, 17–27. [Google Scholar]
  53. Hair, J.; Anderson, R.; Tatham, R.; Black, W. Multivariate Data Analysis, 7th ed.; Prentice-Hall: Upper Saddle River, NJ, USA, 2010. [Google Scholar]
  54. Pieh, C.; Budimir, S.; Probst, T. The effect of age, gender, income, work, and physical activity on mental health during coronavirus disease (COVID-19) lockdown in Austria. J. Psychos. Res. 2020, 136, 110186. [Google Scholar] [CrossRef]
  55. Butnaru, G.I.; Haller, A.P.; Dragolea, L.L.; Anichiti, A.; Tacu Hârșan, G.D. Students’ Wellbeing during Transition from Onsite to Online Education: Are There Risks Arising from Social Isolation? Int. J. Environ. Res. Public Health 2021, 18, 9665. [Google Scholar] [CrossRef]
  56. Wang, C.; Pan, R.; Wan, X.; Tan, Y.; Xu, L.; Ho, C. Immediate psychological responses and associated factors during the initial stage of the 2019 Coronavirus Disease (COVID-19) epidemic among the general population in China. Int. J. Environ. Res. Public Health 2020, 17, 1729. [Google Scholar] [CrossRef]
  57. Cao, W.; Fang, Z.; Hou, G.; Han, M.; Xu, X.; Dong, J.; Zheng, J. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res. 2020, 287, 112934. [Google Scholar] [CrossRef]
  58. Ferreira, M.J.; Sofia, R.; Carreno, D.F.; Eisenbeck, N.; Jongenelen, I.; Cruz, J. Dealing with the Pandemic of COVID-19 in Portugal: On the Important Role of Positivity, Experiential Avoidance, and Coping Strategies. Front. Psychol. 2021, 12, 647984. [Google Scholar] [CrossRef]
  59. Machado, B.C.; Pinto, E.; Silva, M.; Veiga, E.; Sá, C.; Kuhz, S.; Silva, P.O.; Pimenta, A.; Gomes, A.; Almeida, A.; et al. Impact of the COVID-19 pandemic on the mental and physical health and overall wellbeing of university students in Portugal. PLoS ONE 2023, 18, e0285317. [Google Scholar] [CrossRef] [PubMed]
  60. Maia, B.; Dias, P. Anxiety, depression and stress in university students: The impact of COVID-19. Estud. Psicol. 2020, 37, e200067. (In Portuguese) [Google Scholar] [CrossRef]
  61. Asghari, A.; Saed, F.; Dibajnia, P. Psychometric properties of the Depression Anxiety Stress Scales-21 (DASS-21) in a non-clinical Iranian sample. Int. J. Psychol. 2008, 2, 82–102. [Google Scholar]
  62. Gomez, R.; Summers, M.; Summers, A.; Wolf, A.; Summers, J. Depression Anxiety Stress Scales-21: Measurement and structural invariance across ratings of men and women. Assessment 2014, 21, 418–426. [Google Scholar] [CrossRef]
  63. Jafari, P.; Nozari, F.; Ahrari, F.; Bagheri, Z. Measurement invariance of the Depression Anxiety Stress Scales-21 across medical student genders. Int. J. Med. Educ. 2017, 8, 116–122. [Google Scholar] [CrossRef]
  64. Nanthakumar, S.; Bucks, R.S.; Skinner, T.C.; Starkstein, S.; Hillman, D.; James, A.; Hunter, M. Assessment of the Depression Anxiety and Stress Scale (DASS-21) in untreated obstructive sleep apnea (OSA). Psychol. Assess. 2017, 29, 1201–1209. [Google Scholar] [CrossRef]
  65. Shea, T.L.; Tennant, A.; Pallant, J.F. Rasch model analysis of the Depression, Anxiety and Stress Scales (DASS). BMC Psychiatry 2009, 9, 21. [Google Scholar] [CrossRef]
  66. Pezirkianidis, C.; Karakasidou, E.; Lakioti, A.; Stalikas, A.; Galanakis, M. Psychometric properties of the depression, anxiety, stress scales-21 (DASS-21) in a Greek sample. Psychology 2018, 9, 2933–2950. [Google Scholar] [CrossRef]
  67. Yıldırım, A.; Boysan, M.; Kefeli, M.C. Psychometric properties of the Turkish version of the depression anxiety stress Scale-21 (DASS-21). Br. J. Guid. Couns. 2018, 46, 582–595. [Google Scholar] [CrossRef]
  68. Vasconcelos-Raposo, J.; Fernandes, H.M.; Teixeira, C.M. Factor structure and reliability of the depression, anxiety and stress scales in a large Portuguese community sample. Span. J. Psychol. 2013, 16, E10. [Google Scholar] [CrossRef]
  69. Yeung, A.Y.; Yuliawati, L.; Cheung, S.-H. A systematic review and meta-analytic factor analysis of the depression anxiety stress scales. Clin. Psychol. Sci. Pr. 2020, 27, e12362. [Google Scholar] [CrossRef]
  70. Wang, K.; Shi, H.S.; Geng, F.L.; Zou, L.Q.; Tan, S.P.; Wang, Y.; Neumann, D.L.; Shum, D.H.; Chan, R.C. Cross-cultural validation of the Depression Anxiety Stress Scale-21 in China. Psychol. Assess. 2016, 28, e88–e100. [Google Scholar] [CrossRef] [PubMed]
  71. Teo, Y.C.; Yusuf, A.H.; Lim, W.P.A.; Ghazali, N.B.; Abd Rahman, H.; Lin, N.; Koh, D. Validation of DASS-21 among nursing and midwifery students in Brunei. J. Public Health 2019, 27, 387–391. [Google Scholar] [CrossRef]
  72. Sarıçam, H. The psychometric properties of Turkish version of depression anxiety stress Scale-21 (DASS-21) in health control and clinical samples. J. Cogn. Behav. Psychother. Res. 2018, 7, 19–30. [Google Scholar]
  73. Thiyagarajan, A.; James, T.G.; Marzo, R.R. Psychometric properties of the 21-item Depression, Anxiety, and Stress Scale (DASS-21) among Malaysians during COVID-19: A methodological study. Humanit. Soc. Sci. Commun. 2022, 9, 220. [Google Scholar] [CrossRef]
  74. Ahmed, O.; Faisal, R.A.; Alim, S.M.A.H.M.; Sharker, T.; Hiramoni, F.A. The psychometric properties of the Depression Anxiety Stress Scale-21 (DASS-21) Bangla version. Acta Psychol. 2022, 223, 103509. [Google Scholar] [CrossRef]
  75. Tran, T.D.; Tran, T.; Fisher, J. Validation of the depression anxiety stress scales (DASS) 21 as a screening instrument for depression and anxiety in a rural community-based cohort of northern Vietnamese women. BMC Psychiatry 2013, 13, 24. [Google Scholar] [CrossRef]
  76. Fonseca-Pedrero, E.; Paino, M.; Lemos-Giráldez, S.; Muñiz, J. Propriedades psicométricas de la Depression Anxiety and Stress Scales-21 (Dass-21) en universitarios españoles. Ansiedad Estrés 2010, 16, 215–226. (In Portuguese) [Google Scholar]
  77. Verma, S.; Mishra, A. Depression, anxiety, and stress and socio-demographic correlates among general Indian public during COVID-19. Int. J. Soc. Psychiatry 2020, 66, 756–762. [Google Scholar] [CrossRef]
  78. Erdoğan, S.; Can, A.A.; Abiç, A.; Yilmaz, D.V. Examination of individuals’ depression, anxiety, and stress levels during the COVID-19 pandemic in Turkey. Arch. Psychiatr. Nurs. 2022, 41, 96–102. [Google Scholar] [CrossRef]
  79. Nahar, Z.; Sohan, M.; Supti, K.F.; Hossain, M.J.; Shahriar, M.; Bhuiyan, M.A.; Islam, M.R. Prevalence and associated risk factors for mental health problems among female university students during COVID-19 pandemic: A cross-sectional study findings from Dhaka, Bangladesh. Heliyon 2022, 8, e10890. [Google Scholar] [CrossRef] [PubMed]
  80. Reile, R.; Sisask, M. Socio-economic and demographic patterns of mental health complaints among the employed adults in Estonia. PLoS ONE 2021, 16, e0258827. [Google Scholar] [CrossRef] [PubMed]
  81. Vaughan, R.S.; Edwards, E.J.; MacIntyre, T.E. Mental Health Measurement in a Post Covid-19 World: Psychometric Properties and Invariance of the DASS-21 in Athletes and Non-athletes. Front. Psychol. 2020, 11, 590559. [Google Scholar] [CrossRef] [PubMed]
  82. Pais-Ribeiro, J.; Ferreira-Valente, A.; Jarego, M.; Sánchez-Rodríguez, E.; Miró, J. COVID-19 Pandemic in Portugal: Psychosocial and Health-Related Factors Associated with Psychological Discomfort. Int. J. Environ. Res. Public Health 2022, 19, 3494. [Google Scholar] [CrossRef] [PubMed]
  83. Laranjeira, C.; Dixe, M.A.; Valentim, O.; Charepe, Z.; Querido, A. Mental Health and Psychological Impact during COVID-19 Pandemic: An Online Survey of Portuguese Higher Education Students. Int. J. Environ. Res. Public Health 2021, 19, 337. [Google Scholar] [CrossRef] [PubMed]
Ejihpe 13 00177 g001
Figure 1. The three-factor model (F1—stress, F2—anxiety, and F3—depression) of the Portuguese DASS-21.
Figure 1. The three-factor model (F1—stress, F2—anxiety, and F3—depression) of the Portuguese DASS-21.
Ejihpe 13 00177 g001
Table 1. Mean scores for DASS-21 and distribution parameters (n = 1522).
Table 1. Mean scores for DASS-21 and distribution parameters (n = 1522).
DASS-21 Items [22] (p. 179)MinMaxMSDMdnskku
“1. I found it hard to wind down”031.2290.9891.00.344−0.917
“2. I was aware of dryness of my mouth”030.7450.9520.01.013−0.148
“3. I couldn’t seem to experience any positive feeling at all”030.7020.8360.01.0050.227
“4. I experienced breathing difficulty” 030.6590.9220.01.2040.318
“5. I found it difficult to work up the initiative to do things”031.2681.0081.00.284−1.013
“6. I tended to over-react to situations”031.1850.9961.00.331−0.988
“7. I experienced trembling” 030.6370.9190.01.2410.377
“8. I felt that I was using a lot of nervous energy”031.3511.0131.00.187−1.064
“9. I was worried about situations in which I might panic and make a fool of myself”031.0461.0611.00.540−1.025
“10. I felt that I had nothing to look forward to”030.9130.9871.00.725−0.637
“11. I found myself getting agitated”031.1720.9851.00.341−0.956
“12. I found it difficult to relax”031.3391.0071.00.188−1.052
“13. I felt down-hearted and blue”031.2751.0141.00.278−1.032
“14. I was intolerant of anything that kept me from getting on with what I was doing”030.9240.9271.00.617−0.658
“15. I felt I was close to panic”030.8061.0250.00.957−0.409
“16. I was unable to become enthusiastic about anything”030.9270.9911.00.730−0.615
“17. I felt I wasn’t worth much as a person”030.8040.9930.00.945−0.336
“18. I felt I was rather touchy”031.2291.0371.00.322−1.081
“19. I was aware of the action of my heart in the absence of physical exertion” 030.9081.0491.00.770−0.743
“20. I felt scared without any good reason”030.8811.0011.00.782−0.617
“21. I felt that life was meaningless”030.6850.9610.01.2020.244
Min: minimum; max: maximum; M: mean; SD: standard deviation; Mdn: Median; sk: skewness; ku: kurtosis.
Table 2. Descriptive statistics, corrected item–total correlation, factor loadings, and Cronbach’s alpha of the DASS-21 Portuguese version.
Table 2. Descriptive statistics, corrected item–total correlation, factor loadings, and Cronbach’s alpha of the DASS-21 Portuguese version.
StressAnxietyDepressionTotal
Cronbach’s alpha (α)0.9230.9000.9220.961
Mean8.4295.6826.574-
SD5.7565.4875.613-
Corrected item–total correlationFactor loading
Item 10.747 0.800
Item 60.722 0.732
Item 80.778 0.838
Item 110.813 0.852
Item 120.815 0.862
Item 140.700 0.720
Item 180.743 0.756
Item 2 0.536 0.569
Item 4 0.738 0.759
Item 7 0.702 0.725
Item 9 0.730 0.791
Item 15 0.771 0.838
Item 19 0.758 0.789
Item 20 0.713 0.775
Item 3 0.7340.770
Item 5 0.6700.710
Item 10 0.8080.848
Item 13 0.7680.833
Item 16 0.7940.821
Item 17 0.7880.811
Item 21 0.7420.762
Table 3. Summary of the adjustment indices of the DASS-21 Portuguese version.
Table 3. Summary of the adjustment indices of the DASS-21 Portuguese version.
MeasureRecommended Cutoffs
[51,52,53]		DASS-21
Model 1		DASS-21
Model 2
CFI>0.80.9310.952
GFI>0.80.8860.919
RMR<0.050.0380.032
PNFI>0.50.8180.814
PCFI>0.50.8250.820
HOELTER (0.1)>200191257
AIC-1955.2151455.204
ECVI-1.2850.957
Comparative fit index (CFI); goodness of fit index (GFI); root mean square residual (RMR) index; parsimony-adjusted measures (PNFI and PCFI); HOELTER index; Akaike information criteria (AIC); and expected cross-validation index (ECVI).
Table 4. Pearson correlations between DASS-21 subscales, mental health status, physical health status, and overall health status.
Table 4. Pearson correlations between DASS-21 subscales, mental health status, physical health status, and overall health status.
DASS StressDASS AnxietyDASS Depression
rrr
DASS stress-0.889 *0.844 *
DASS anxiety--0.807 *
DASS depression---
Mental health perception−0.539 *−0.475 *−0.551 *
Physical health perception−0.299 *−0.275 *−0.287 *
Overall health perception−0.425 *−0.376 *−0.417 *
Person correlation coefficient (r); *: p < 0.001.
Table 5. Descriptive statistics of the DASS-21 subscales according to sex.
Table 5. Descriptive statistics of the DASS-21 subscales according to sex.
SubscalesMale (n = 379)Female (n = 1143)t
MSDMSD
DASS stress6.9825.6368.9095.7165.708 *
DASS anxiety4.4644.8806.0865.6185.392 *
DASS depression6.1615.6426.7105.6001.652
M: Mean; SD: standard deviation; independent t-test (t); *: p < 0.001.
Table 6. Descriptive statistics of the DASS-21 subscales by marital status.
Table 6. Descriptive statistics of the DASS-21 subscales by marital status.
SubscaleMarital StatusnMSDFp
DASS stressSingle13888.6445.7507.681p < 0.001
Married1186.0595.347
Divorced137.0005.859
Widowed38.3332.517
DASS anxietySingle13885.9455.52912.480p < 0.001
Married1182.8643.989
Divorced133.7695.904
Widowed33.0002.000
DASS depressionSingle13886.8605.61514.031p < 0.001
Married1183.5514.509
Divorced134.1546.296
Widowed33.6674.726
M: Mean; SD: standard deviation; one-way ANOVA (F-test).
Table 7. Descriptive statistics of the DASS-21 subscales by academic qualification.
Table 7. Descriptive statistics of the DASS-21 subscales by academic qualification.
SubscaleAcademic QualificationsnMSDFp
DASS stressGraduation12058.6445.7083.5230.015
Post-graduation157.3335.486
Master’s degree1548.1305.928
Other1487.1015.841
DASS anxietyGraduation12055.8945.5382.9190.033
Post-graduation154.5334.704
Master’s degree1544.8315.279
Other1484.9595.233
DASS depressionGraduation12056.7585.6242.1330.094
Post-graduation155.5335.475
Master’s degree1545.9945.503
Other1485.7845.589
M: mean; SD: standard deviation; one-way ANOVA (F-test).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Laranjeira, C.; Querido, A.; Sousa, P.; Dixe, M.A. Assessment and Psychometric Properties of the 21-Item Depression Anxiety Stress Scale (DASS-21) among Portuguese Higher Education Students during the COVID-19 Pandemic. Eur. J. Investig. Health Psychol. Educ. 2023, 13, 2546-2560. https://doi.org/10.3390/ejihpe13110177

AMA Style

Laranjeira C, Querido A, Sousa P, Dixe MA. Assessment and Psychometric Properties of the 21-Item Depression Anxiety Stress Scale (DASS-21) among Portuguese Higher Education Students during the COVID-19 Pandemic. European Journal of Investigation in Health, Psychology and Education. 2023; 13(11):2546-2560. https://doi.org/10.3390/ejihpe13110177

Chicago/Turabian Style

Laranjeira, Carlos, Ana Querido, Pedro Sousa, and Maria Anjos Dixe. 2023. "Assessment and Psychometric Properties of the 21-Item Depression Anxiety Stress Scale (DASS-21) among Portuguese Higher Education Students during the COVID-19 Pandemic" European Journal of Investigation in Health, Psychology and Education 13, no. 11: 2546-2560. https://doi.org/10.3390/ejihpe13110177

APA Style

Laranjeira, C., Querido, A., Sousa, P., & Dixe, M. A. (2023). Assessment and Psychometric Properties of the 21-Item Depression Anxiety Stress Scale (DASS-21) among Portuguese Higher Education Students during the COVID-19 Pandemic. European Journal of Investigation in Health, Psychology and Education, 13(11), 2546-2560. https://doi.org/10.3390/ejihpe13110177

Article Metrics

Back to TopTop