Next Article in Journal
Is the Triage System Welcomed in the Tertiary Hospital of the Limpopo Province? A Qualitative Study on Patient’s Perceptions
Previous Article in Journal
Examining Students’ Online Learning Satisfaction during COVID-19 to Inform Post-Pandemic Program Planning
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Problematic Internet Use and Resilience: A Systematic Review and Meta-Analysis

by
Sergio Hidalgo-Fuentes
1,2,
Manuel Martí-Vilar
2,* and
Yolanda Ruiz-Ordoñez
3
1
Departamento de Psicología y Salud, Universidad a Distancia de Madrid (UDIMA), Crta. De la Coruña Km. 38,500, vía de Servicio Número 15, Collado Villalba, 28400 Madrid, Spain
2
Departamento de Psicología Básica, Universitat de València, Avgda. Blasco Ibañez 21, 46010 Valencia, Spain
3
Departamento de Psicología Básica, Neuropsicología y Social, Universidad Católica de Valencia, 46100 Burjassot, Spain
*
Author to whom correspondence should be addressed.
Nurs. Rep. 2023, 13(1), 337-350; https://doi.org/10.3390/nursrep13010032
Submission received: 20 December 2022 / Revised: 20 February 2023 / Accepted: 23 February 2023 / Published: 24 February 2023

Abstract

:
Problematic Internet use has become a major problem worldwide due to its numerous negative correlates in the field of health, both mental and physical, and its increasing prevalence, making it necessary to study both its risk and protective factors. Several studies have found a negative relationship between resilience and problematic Internet use, although the results are inconsistent. This meta-analysis assesses the relationship between problematic Internet use and resilience, and analyses its possible moderating variables. A systematic search was conducted in PsycInfo, Web of Science and Scopus. A total of 93,859 subjects from 19 studies were included in the analyses. The results show that there is a statistically-significant negative relationship (r = −0.27 (95% CI [−0.32, −0.22])), without evidence of publication bias. This meta-analysis presents strong evidence of the relationship between the two variables. Limitations and practical implications are discussed.

1. Introduction

Internet use has grown substantially over the last few decades, with the number of users increasing by 1331.9% between 2000 and 2021 [1], when a total of 4.66 billion users were counted, representing approximately 60% of the world’s population [2]. The benefits associated with using the Internet, especially concerning information search and communication, have led people to rely more and more on this technology for their work, study, social interaction and access to various entertainment options [3]. However, excessive and uncontrolled use of this technology can lead to what has been termed problematic Internet use (PIU), which is defined as Internet use that causes psychological, social, educational and/or occupational difficulties in an individual’s life [4]. Although the term Internet addiction, conceptualized as an impulse control disorder whereby the person loses control over their use of the Internet to the extent that they experience numerous negative consequences, as proposed by Young [5], is widely used in the scientific literature [6], a considerable number of authors recommend the use of PIU as more appropriate [7,8,9], since it is not recognised as an addictive disorder in either the DSM-5 [10] or ICD-11 [11].
PIU has been associated with numerous negative variables related to both mental and physical health, such as anxiety and depression [12], low self-esteem [13], poor sleep quality [14], alexithymia [15], risk of obesity [16], high impulsivity [17] and problematic alcohol consumption [18], among others, and the World Health Organization has declared PIU a major public health concern, emphasizing the need to intensify international research on this problem to generate the information required to develop policies and interventions to prevent and treat PIU [19]. A recent meta-analysis, which was conducted on a total sample of 2,123,762 people, has estimated the prevalence of PIU among the general population at 14.22% [20], having increased in recent years [21]. The high number of detrimental variables associated with PIU, as well as its increasing prevalence, makes it necessary to emphasize the study of both potential risk factors and protective factors for PIU.
How to conceptualize resilience is a widely debated topic in the field of psychology [22]. Resilience can be defined as an individual’s ability to maintain or regain psychological well-being in the face of a challenging situation [23]; it is a dynamic process that encompasses positive adaptation in the face of significant adversity, which would include feedback, learning and making changes to remain positive and recover from frustration caused by stressful events [24]. Resilience is an important factor in personal well-being, being negatively correlated to negative indicators of mental health, such as depression and anxiety, and positively correlated to positive indicators of mental health, such as life satisfaction and positive affect [25]. Several studies have examined the role of resilience in various types of addictive behaviors, and have found that resilience serves as a protective factor against addiction to gambling [26], alcohol [27,28], drugs of abuse [29,30], and video games [31,32]. Likewise, the relationship between resilience and PIU has also been evaluated, and has found negative relationship between both variables [33,34,35]. However, to date there has been no meta-analysis specifically focused on the relationship between PIU and resilience that synthesizes the results found. The aim of this paper is therefore to synthesize the evidence from those studies that have examined the association between PIU and resilience by answering the following research questions: (1) what is the strength of the association between PIU and resilience?; and (2) is the association between PIU and resilience moderated by the methodological and socio-demographic variables of the studies analyzed?

2. Materials and Methods

2.1. Systematic Search

This meta-analysis (registered in PROSPERO database #CRD42022382337) was conducted according to the criteria of the PRISMA statement [36] (Appendix A, Table A1). A systematic search was conducted during November 2022 in three databases (PsycINFO, Scopus and Web of Science) using the terms (resilience OR resiliency OR resilient) AND (internet addiction OR problematic internet use OR internet abuse OR internet overuse OR internet dependence). Searches were restricted to papers published in English or Spanish. Moreover, the references of the selected articles were manually checked for other relevant studies that were not retrieved during the electronic search. The systematic reviews software Covidence (http://www.covidence.org accessed on 14 November 2022) was used to manage the study selection process.

2.2. Inclusion Criteria

The retrieved studies were selected based on the following inclusion criteria: (1) original empirical and quantitative cross-sectional or longitudinal studies; (2) published in peer-reviewed scientific journals; (3) published in English or Spanish; (4) include assessments of PIU and resilience; (5) present Pearson’s correlation coefficient between PIU and resilience or the statistical data necessary to calculate it: (6) present the sample size; and (7) the full text was available. In case of studies with partially duplicated samples, the study with the largest sample size was selected.

2.3. Methodological Quality of Included Studies

Conducting a meta-analysis without taking into consideration the methodological quality of the included studies may lead to biased results. Therefore, an assessment of the methodological quality of the studies analyzed in a meta-analysis is essential to be able to draw reliable conclusions. The risk of individual bias of the studies included in the meta-analysis was assessed using the short version of the Newcastle-Ottawa scale developed by Deng et al. [37]. The scale consists of a total of five items: (1) representativeness of the sample (inclusion of the entire population or random sampling); (2) sample size justified by methods such as power analysis; (3) response rate greater than 80%; (4) valid PIU and resilience assessment tests; and (5) appropriate and correctly described statistical analyses. Each item is scored as one point if it meets the criterion and zero points if it does not meet the criterion or the information is not available. The total score ranges from zero to five points, with studies scoring three or more points being considered at low risk of individual bias and those scoring less than three points being considered at high risk of individual bias. Assessments were performed by two reviewers working independently. Discrepancies were resolved by consensus.

2.4. Data Coding

A recording sheet was prepared to code the following information for the studies included: author(s), year of publication, country in which the study was conducted, continent, sample size, mean age of participants, gender (coded as the percentage of males in the sample), test used to assess PIU, test used to assess resilience, risk of individual bias and Pearson’s correlation between PIU and resilience. Data coding was performed by two reviewers working independently. The reviewers matched their data after extraction and revisited papers in case of disagreements. In the event of missing data, we contacted the authors of the study to request the necessary information; where we received no response or the authors refused to provide it, the information is listed as missing. To meet the independence assumption, in the case of longitudinal studies only the first correlation between PIU and resilience was coded.

2.5. Data Analysis

Most of the studies had Pearson correlations. For those studies with χ2, this result was converted to Pearson correlations using the formula r = √(χ2/n). Subsequently, to normalize their distributions, all Pearson correlations were converted to Fisher’s Z-scores using the formula Z = 0.5 × ln[(1 + r)/(1 − r)]. All analyses were performed with Z-scores, although the overall effect size and its confidence interval were transformed back to Pearson correlations for better interpretation following the recommendation of Borenstein et al. [38].
Due to the variability observed in the selected studies in terms of the countries in which they were conducted, the number of subjects and tests used, a random-effects meta-analysis with the restricted maximum likelihood method was chosen. Random-effects models generally produce more precise estimates and allow for greater generalizability of results [39,40,41]. The existence of statistically significant heterogeneity among the effect sizes of the analyzed studies was examined using Cochran’s Q test, while the degree of true heterogeneity not explained by random sampling error was assessed using the I2 statistic. I2 values of 25%, 50% and 75% are interpreted respectively as low, moderate and high heterogeneity [42].
The validity of a meta-analysis may be challenged by the presence of publication bias, a phenomenon whereby studies with statistically significant results or high effect sizes are more likely to be published [43]. Publication bias is a particularly important problem when conducting meta-analyses, since it can lead to overestimated effect sizes. In this study, and as recommended by Botella and Sánchez-Meca [44], the risk of publication bias was assessed by several methods: visual inspection of the funnel plot, Egger’s regression test [45], Begg and Mazumdar’s rank correlation test [46], and calculating the safety number according to Rosenthal’s method. In the absence of publication bias, the funnel plot will be symmetrical around the average effect size, while Egger’s test and Begg and Mazumdar’s test will show non-significant results. Rosenthal’s method makes it possible to estimate missing studies to calculate how many studies would be required for the estimated effect size to be non-significant.
A jacknife sensitivity analysis was performed, estimating the pooled effect size while eliminating each study alternatively, to assess the individual influence on the overall effect size of each of the studies included in the meta-analysis.
We examined the possible moderating role of the following variables: sex and age of participants, measures for assessing PIU and resilience, the continent in which the studies were conducts, individual risk of bias and year of publication. For continuous variables, meta-regression analyses were conducted, while for categorical variables, subgroup analyses were conducted. For subgroup analysis, and as recommended by Fu et al. [47], each subgroup should be composed of a minimum of four studies. When this was not possible due to fewer studies having been performed, the remaining studies were grouped into the subgroup others and included in the analyses under this heading if they comprised at least four studies. The percentage of variance explained by the moderators was assessed using the R2 index.
Analyses were performed in R Studio using the metafor statistical package [48].

3. Results

As can be seen in Figure 1, the search and selection process ended with the inclusion of 19 studies that met the inclusion criteria. The selected articles were published between 2015 and 2022 (see Table 1). Eight of the studies were conducted in China, four in South Korea, two in the United States and Turkey, and one each in Australia, Hungary and Iran. The combined sample was 93,859 subjects, with the sample sizes of the various studies ranging from 96 to 58,756 participants.
The estimated overall effect size for the correlation between PIU and resilience was Zr = −0.28 (95% CI [−0.33, −0.22]), which transformed back to Pearson’s correlation gives a result of r = −0.27 (95% CI [−0.32, −0.22]), and which, following the interpretation criteria proposed by Cohen [65], can be classified as a moderate intensity correlation. The forest plot of the effect sizes and 95% confidence intervals of the 19 studies are shown in Figure 2. As can be seen in the figure, the effect sizes of the studies ranged from Zr = −0.12 to Zr = −0.55. The Cochran’s Q test result was 281.4128, p < 0.0001, hence the homogeneity hypothesis is rejected, while the I2 value reached a value of 97.46%, which is considered high according to Higgins and Thompson’s criteria [42].
Although the funnel plot is not fully symmetrical (see Figure 3), both the Egger regression test (z = 0.2996, p = 0.76) and the Begg and Mazumdar rank correlation test (τ = −0.0292, p = 0.89) show non-significant results, thus ruling out the presence of publication bias. Likewise, the calculation of the number of safety according to Rosenthal’s method yielded a value of n = 18,877 (p < 0.001), making 18,877 unpublished studies with an effect size equal to zero necessary to make the p-value non-significant, exceeding the critical value which, for this meta-analysis, is set at 105 studies, according to the formula (5 × k) + 10, and k being the number of studies included in the meta-analysis [44].
The sensitivity analysis, performed using the jackknife method, did not show excessive individual influence of any of the studies on the estimated overall effect size, with the effect size ranging from Zr = −0.26 to Zr = −0.29 when alternately omitting each of the studies.
Meta-regression analyses were conducted to examine the possible moderating effect of year of publication, mean age of participants and percentage of males in the sample on the correlation between PIU and resilience. Both mean age (b = −0.0028, p = 0.72) and percentage of males among participants (b = −0.0027, p = 0.26) did not show up as moderating variables, while year of publication (b = −0.0283, p = 0.04) does moderate the relationship between the two variables, with a total explained variability of 16.55%, with more recent studies showing lower correlations.
For categorical variables, subgroup analyses were performed (see Table 2), and no moderating effect was found for any of the variables analyzed.

4. Discussion

The first aim of this paper was to estimate the magnitude of the association between PIU and resilience. Additionally, we examined the possible moderating role of gender and age of participants, the continent on which the studies were conducted, the tests used to assess both PIU and resilience, the year of publication of the studies, and the risk of individual bias.
The systematic search identified a total of studies that met the inclusion criteria with a total sample of 93,859 subjects. The results of the meta-analyses showed a statistically significant negative correlation of moderate intensity (r = −0.27) between the two variables, whereby those who showed higher levels of resilience had lower levels of PIU. Sensitivity analysis reveals that this result is consistent, with none of the studies having an excessive influence on the overall effect size. Furthermore, the various tests performed to assess the risk of publication bias ruled out the presence of bias. Despite the high degree of heterogeneity found, only the year of publication proved to be a moderating variable in the correlation between PIU and resilience, explaining 16.55% of the observed heterogeneity.
The result found has important implications for the prevention of PIU, a phenomenon with significant negative repercussions on mental and physical health, as well as significant associated economic costs [66]. Resilience may function as a protective factor for PIU by mitigating the negative impact of adverse situations or environments, causing individuals to suffer lower levels of depression or anxiety [67], two variables that have been consistently linked in the scientific literature to PIU [12,68,69,70]. Additionally, in theoretical terms, the negative association found between resilience and PIU could be explained in relation to the I-PACE model, which explains the onset and development of PIU by the interaction of personal, affective, cognitive and executive variables [6]. This theoretical model holds that stress is an important factor operating on addictive behaviors and that excessive and uncontrolled use of the Internet can sometimes be a coping style that attempts to cope with this stress. Resilience also improves people’s ability to cope with stressful situations, which are also a risk factor for PIU [71], as individuals with high levels of stress often use the Internet as a maladaptive coping strategy because, although it does not offer long-term improvement, Internet use can serve as a temporary relief from stressful symptoms. Thus, from this perspective, resilience, which is taken to be the ability to cope with adverse and stressful situations, may lead to a lesser need to use the Internet to reduce stress levels, since resilience itself will act as a protective factor. Thus, people with higher levels of resilience have and make use of adaptive coping strategies in stressful situations, which may prevent them from engaging in compulsive behaviors such as PIU. Therefore, the results obtained, together with the fact that resilience can be increased through appropriate programs [72], allow us to state that interventions aimed at increasing resilience can be an effective method of reducing the risk of PIU. Besides preventing the onset of PIU, resilience has also shown benefits when IPU has already developed, serving as a protective factor against the negative psychological effects of PIU [73].
Among the possible moderating variables of the relationship between PIU and resilience examined, the only statistically significant moderator was the studies’ year of publication, with more recent articles showing a smaller effect size among the variables studied. One possible explanation for this is that the more recent studies, conducted during the pandemic when many countries were in lockdown, show a lower relationship between PIU and resilience since individuals during this period suffered greater stress that could not be compensated for by their resilience levels, leading to excessive internet use to reduce this stress. By contrast, participants’ gender and age, as well as the geographical area in which the studies were conducted, are not statistically significant moderators of the relationship between PIU and resilience. The fact that there is little heterogeneity regarding these variables, especially age and geographic area, in the included studies could be influencing this result.
The results of this meta-analysis should be interpreted with caution due to certain limitations. Firstly, the number of studies that met the inclusion criteria is limited, so it would be advisable for future systematic reviews or meta-analyses to extend the search to other databases. Secondly, only studies published in Spanish or English were included, which could be considered a selection bias, despite English being the most widely used language in the scientific literature. Thirdly, only one of the possible moderating variables was found to have a significant effect and it could not explain a significant percentage of the heterogeneity found. It would therefore be important for future meta-analyses to examine the role of new potential moderators of the correlation between PIU and resilience, such as the population in which the studies were carried out or the scores obtained. Fourth, given the cross-sectional design of most of the included studies, it is not possible to establish causal relationships between the variables analyzed or to examine their evolution over time, hence it would be desirable to conduct further longitudinal or experimental design research in the future to examine these matters. Finally, most of the studies were conducted in Asian countries and with adolescent and young participants, with very limited research in other geographical areas and with subjects in other age groups.

5. Conclusions

PIU has become a growing problem in recent years, especially among adolescents and young people, being associated with many harmful variables, mainly psychological, hence studying its risk and protective factors to help to prevent and treat it should be a priority, bearing in mind both its negative effects and the number of people who suffer from this problem. This meta-analysis has synthesized the results on PIU and resilience. The results of this review, despite its limitations, indicate the existence of a significant negative relationship of moderate intensity between both variables that does not appear to depend on age, gender, geographical area or the tests used. This result has implications that go beyond the theoretical field by supporting the fact that working on people’s resilience can reduce the risk of PIU. Moreover, increasing resilience levels through appropriate training programs would have beneficial effects beyond reducing the risk of IPU, since resilience has also been shown to be a protective factor against other addictive behaviors such as alcohol consumption [27], gambling [26], drug abuse [29] and Internet gaming disorder [32]. Likewise, increasing resilience would also have a positive impact on other variables not directly related to problematic use of new technologies or addictions, improving both physical and mental health [72].

Author Contributions

Conceptualization, S.H.-F. and M.M.-V.; methodology, S.H.-F., M.M.-V. and Y.R.-O.; software, S.H.-F.; validation, S.H.-F., M.M.-V. and Y.R.-O.; formal analysis, S.H.-F.; investigation, S.H.-F., M.M.-V. and Y.R.-O.; resources, M.M.-V. and Y.R.-O.; data curation, S.H.-F.; writing—original draft preparation, S.H.-F.; writing—review and editing, M.M.-V. and Y.R.-O.; visualization, M.M.-V. and Y.R.-O.; project administration, M.M.-V. and Y.R.-O. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Search Strings; PRISMA Checklist.
Table A1. Search Strings; PRISMA Checklist.
Section/Topic#Checklist Item Reported on Page #
TITLE
Title 1Identify the report as a systematic review, meta-analysis, or both. 1
ABSTRACT
Structured summary 2Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number. 1
INTRODUCTION
Rationale 3Describe the rationale for the review in the context of what is already known. 1–2
Objectives 4Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS). 2
METHODS
Protocol and registration 5Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number. 2
Eligibility criteria 6Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. 2
Information sources 7Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched. 2
Search 8Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated. Table A1
Study selection 9State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis). 2
Data collection process 10Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators. 2–3
Data items 11List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made. 2–3
Risk of bias in individual studies 12Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis. 2
Summary measures 13State the principal summary measures (e.g., risk ratio, difference in means). 3
Synthesis of results 14Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I2) for each meta-analysis. 3
Risk of bias across studies 15Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies). 3
Additional analyses 16Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified. 3
RESULTS
Study selection 17Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram. 3–4
Study characteristics 18For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations. 4–5
Risk of bias within studies 19Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). 4–5
Results of individual studies 20For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot. 5
Synthesis of results 21Present results of each meta-analysis done, including confidence intervals and measures of consistency. 5–6
Risk of bias across studies 22Present results of any assessment of risk of bias across studies (see Item 15). 5–6
Additional analysis 23Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). 6–7
DISCUSSION
Summary of evidence 24Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers). 7
Limitations 25Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias). 7
Conclusions 26Provide a general interpretation of the results in the context of other evidence, and implications for future research. 7–8
FUNDING
Funding 27Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review. N/A
From: [36]. For more information, visit: www.prisma-statement.org (accessed on 17 December 2022).

References

  1. Miniwatts Marketing Group. Internet World Stats: Usage and Population Statistics. Available online: https://www.intenetworldstats.com/stats.htm (accessed on 25 October 2022).
  2. Statista Research Department. Global Digital Population as of January 2021. Available online: https://www.statista.com/statistics/617136/digital-population-worldwide/ (accessed on 25 October 2022).
  3. Brooks, S.; Wang, X.; Schneider, C. Technology Addictions and Technostress: An Examination of the U.S. and China. J. Organ. End User Comput. 2020, 32, 1–19. [Google Scholar] [CrossRef]
  4. Beard, K.W.; Wolf, E.M. Modification in the Proposed Diagnostic Criteria for Internet Addiction. Cyberpsychol. Behav. 2001, 4, 377–383. [Google Scholar] [CrossRef]
  5. Young, K.S. Internet Addiction: The Emergence of a New Clinical Disorder. Cyberpsychol. Behav. 1998, 1, 237–244. [Google Scholar] [CrossRef] [Green Version]
  6. Brand, M.; Young, K.S.; Laier, C.; Wölfling, K.; Potenza, M.N. Integrating Psychological and Neurobiological Considerations Regarding the Development and Maintenance of Specific Internet-Use Disorders: An Interaction of Person-Affect-Cognition-Execution (I-PACE) Model. Neurosci. Biobehav. Rev. 2016, 71, 252–266. [Google Scholar] [CrossRef] [Green Version]
  7. Kardefelt-Winther, D. Conceptualizing Internet Use Disorders: Addiction or Coping Process?: Conceptualizing Internet Use Disorders. Psychiatry Clin. Neurosci. 2017, 71, 459–466. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Spada, M.M. An Overview of Problematic Internet Use. Addict. Behav. 2014, 39, 3–6. [Google Scholar] [CrossRef]
  9. Starcevic, V.; Aboujaoude, E. Internet Addiction: Reappraisal of an Increasingly Inadequate Concept. CNS Spectr. 2017, 22, 7–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  10. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5, 5th ed.; American Psychiatric Association, Ed.; American Psychiatric Association: Washington, DC, USA, 2013; ISBN 9780890425541. [Google Scholar]
  11. World Health Organization. International Classification of Diseases 11th Revision (ICD-11). Available online: https://icd.who.int/ (accessed on 27 October 2022).
  12. Li, G.; Hou, G.; Yang, D.; Jian, H.; Wang, W. Relationship between Anxiety, Depression, Sex, Obesity, and Internet Addiction in Chinese Adolescents: A Short-Term Longitudinal Study. Addict. Behav. 2019, 90, 421–427. [Google Scholar] [CrossRef]
  13. Mo, P.K.; Chan, V.W.; Wang, X.; Lau, J.T. Gender difference in the association between internet addiction, self-esteem and academic aspirations among adolescents: A structural equation modelling. Comput. Educ. 2020, 155, 103921. [Google Scholar] [CrossRef]
  14. Alimoradi, Z.; Lin, C.-Y.; Broström, A.; Bülow, P.H.; Bajalan, Z.; Griffiths, M.D.; Ohayon, M.M.; Pakpour, A.H. Internet Addiction and Sleep Problems: A Systematic Review and Meta-Analysis. Sleep Med. Rev. 2019, 47, 51–61. [Google Scholar] [CrossRef] [Green Version]
  15. Hidalgo-Fuentes, S. Uso Problemático de Internet y Alexitimia: Un Meta-Análisis. Apunt. Univ. 2022, 12, 162–180. [Google Scholar] [CrossRef]
  16. Aghasi, M.; Matinfar, A.; Golzarand, M.; Salari-Moghaddam, A.; Ebrahimpour-Koujan, S. Internet Use in Relation to Overweight and Obesity: A Systematic Review and Meta-Analysis of Cross-Sectional Studies. Adv. Nutr. 2020, 11, nmz073. [Google Scholar] [CrossRef] [PubMed]
  17. Gecaite-Stonciene, J.; Saudargiene, A.; Pranckeviciene, A.; Liaugaudaite, V.; Griskova-Bulanova, I.; Simkute, D.; Naginiene, R.; Dainauskas, L.L.; Ceidaite, G.; Burkauskas, J. Impulsivity Mediates Associations Between Problematic Internet Use, Anxiety, and Depressive Symptoms in Students: A Cross-Sectional COVID-19 Study. Front. Psychiatry 2021, 12, 634464. [Google Scholar] [CrossRef] [PubMed]
  18. Ramón-Arbués, E.; Granada-López, J.M.; Martínez-Abadía, B.; Echániz-Serrano, E.; Antón-Solanas, I.; Nash, M. Prevalence and Factors Associated with Problematic Internet Use in a Population of Spanish University Students. Int. J. Environ. Res. Public Health 2021, 18, 7620. [Google Scholar] [CrossRef] [PubMed]
  19. World Health Organization. Public Health Implications of Excessive Use of the Internet, Computers, Smartphones and Similar Electronic Devices: Meeting Report; Main Meeting Hall, Foundation for Promotion of Cancer Research, National Cancer Research Centre: Tokyo, Japan, 2015; ISBN 9789241509367. [Google Scholar]
  20. Meng, S.-Q.; Cheng, J.-L.; Li, Y.-Y.; Yang, X.-Q.; Zheng, J.-W.; Chang, X.-W.; Shi, Y.; Chen, Y.; Lu, L.; Sun, Y.; et al. Global Prevalence of Digital Addiction in General Population: A Systematic Review and Meta-Analysis. Clin. Psychol. Rev. 2022, 92, 102128. [Google Scholar] [CrossRef]
  21. Pan, Y.-C.; Chiu, Y.-C.; Lin, Y.-H. Systematic Review and Meta-Analysis of Epidemiology of Internet Addiction. Neurosci. Biobehav. Rev. 2020, 118, 612–622. [Google Scholar] [CrossRef]
  22. Fletcher, D.; Sarkar, M. Psychological Resilience: A Review and Critique of Definitions, Concepts, and Theory. Eur. Psychol. 2013, 18, 12–23. [Google Scholar] [CrossRef] [Green Version]
  23. Ryff, C.D.; Friedman, E.M.; Morozink, J.A.; Tsenkova, V. Psychological Resilience in Adulthood and Later Life: Implications for Health. Annu. Rev. Gerontol. Geriatr. 2012, 32, 73–92. [Google Scholar] [CrossRef]
  24. Luthar, S.S.; Cicchetti, D.; Becker, B. The Construct of Resilience: A Critical Evaluation and Guidelines for Future Work. Child Dev. 2000, 71, 543–562. [Google Scholar] [CrossRef]
  25. Hu, T.; Zhang, D.; Wang, J. A Meta-Analysis of the Trait Resilience and Mental Health. Personal. Individ. Differ. 2015, 76, 18–27. [Google Scholar] [CrossRef]
  26. Oei, T.P.S.; Goh, Z. Interactions Between Risk and Protective Factors on Problem Gambling in Asia. J. Gambl. Stud. 2015, 31, 557–572. [Google Scholar] [CrossRef]
  27. Tudehope, L.; Lee, P.; Wiseman, N.; Dwirahmadi, F.; Sofija, E. The Effect of Resilience on the Relationship between Perceived Stress and Change in Alcohol Consumption during the COVID-19 Pandemic in Queensland, Australia. J. Health Psychol. 2022, 27, 2696–2713. [Google Scholar] [CrossRef] [PubMed]
  28. Van Gils, Y.; Dom, G.; Dierckx, E.; Van Alphen, S.P.J.; Franck, E. Resilience, Depression and Anxiety and Hazardous Alcohol Use Behaviour among Community Dwelling Older Adults. Aging Ment. Health 2022, 26, 56–64. [Google Scholar] [CrossRef] [PubMed]
  29. Yang, C.; Xia, M.; Han, M.; Liang, Y. Social Support and Resilience as Mediators between Stress and Life Satisfaction among People with Substance Use Disorder in China. Front. Psychiatry 2018, 9, 436. [Google Scholar] [CrossRef] [Green Version]
  30. Yang, C.; Zhou, Y.; Cao, Q.; Xia, M.; An, J. The relationship between self-control and self-efficacy among patients with substance use disorders: Resilience and self-esteem as mediators. Front. Psychiatry 2019, 10, 388. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Lin, P.-C.; Yen, J.-Y.; Lin, H.-C.; Chou, W.-P.; Liu, T.-L.; Ko, C.-H. Coping, Resilience, and Perceived Stress in Individuals with Internet Gaming Disorder in Taiwan. J. Environ. Res. Public Health 2021, 18, 1771. [Google Scholar] [CrossRef]
  32. Yen, J.-Y.; Lin, H.-C.; Chou, W.-P.; Liu, T.-L.; Ko, C.-H. Associations Among Resilience, Stress, Depression, and Internet Gaming Disorder in Young Adults. J. Environ. Res. Public Health 2019, 16, 3181. [Google Scholar] [CrossRef] [Green Version]
  33. Dinc, M.; Topcu, F. The Relation between Resilience and Problematic Internet Use among Youth. Dusunen. Adam. 2021, 34, 337–345. [Google Scholar] [CrossRef]
  34. Öztürk, A.; Kundakçı, N. Perceived Social Support, and Psychological Resilience as Predictors of Internet Addiction: A CrossSectional Study with a Sample of Turkish Undergraduates. Psychiatry Clin. Psychopharmacol. 2021, 31, 449–456. [Google Scholar] [CrossRef]
  35. Robertson, T.W.; Yan, Z.; Rapoza, K.A. Is Resilience a Protective Factor of Internet Addiction? Comput. Hum. Behav. 2018, 78, 255–260. [Google Scholar] [CrossRef]
  36. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. The PRISMA Group Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Green Version]
  37. Deng, J.; Zhou, F.; Hou, W.; Silver, Z.; Wong, C.Y.; Chang, O.; Huang, E.; Zuo, Q.K. The Prevalence of Depression, Anxiety, and Sleep Disturbances in COVID-19 Patients: A Meta-analysis. Ann. N. Y. Acad. Sci. 2021, 1486, 90–111. [Google Scholar] [CrossRef] [PubMed]
  38. Borenstein, M.; Hedges, L.V.; Higgins, J.P.T.; Rothstein, H.R. Introduction to Meta-Analysis, 2nd ed.; Wiley & Sons: Hoboken, NJ, USA, 2021; ISBN 9780470057247. [Google Scholar]
  39. Field, A.P. Meta-Analysis of Correlation Coefficients: A Monte Carlo Comparison of Fixed- and Random-Effects Methods. Psychol. Methods 2001, 6, 161–180. [Google Scholar] [CrossRef]
  40. Hedges, L.V.; Vevea, J.L. Fixed- and Random-Effects Models in Meta-Analysis. Psychol. Methods 1998, 3, 486–504. [Google Scholar] [CrossRef]
  41. Schmidt, F.L.; Oh, I.-S.; Hayes, T.L. Fixed- versus Random-Effects Models in Meta-Analysis: Model Properties and an Empirical Comparison of Differences in Results. Br. J. Math. Stat. Psychol. 2009, 62, 97–128. [Google Scholar] [CrossRef] [Green Version]
  42. Higgins, J.P.T.; Thompson, S.G. Quantifying Heterogeneity in a Meta-Analysis. Statist. Med. 2002, 21, 1539–1558. [Google Scholar] [CrossRef]
  43. Polanin, J.R.; Tanner-Smith, E.E.; Hennessy, E.A. Estimating the Difference Between Published and Unpublished Effect Sizes: A Meta-Review. Rev. Educ. Res. 2016, 86, 207–236. [Google Scholar] [CrossRef]
  44. Botella-Ausina, J.; Sánchez-Meca, J. Meta-Análisis en Ciencias Sociales y de la Salud, 1st ed.; Síntesis: Madrid, España, 2015; ISBN 9788490771242. [Google Scholar]
  45. Egger, M.; Smith, G.D.; Schneider, M.; Minder, C. Bias in Meta-Analysis Detected by a Simple, Graphical Test. BMJ 1997, 315, 629–634. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Begg, C.B.; Mazumdar, M. Operating Characteristics of a Rank Correlation Test for Publication Bias. Biometrics 1994, 50, 1088. [Google Scholar] [CrossRef]
  47. Fu, R.; Gartlehner, G.; Grant, M.; Shamliyan, T.; Sedrakyan, A.; Wilt, T.J.; Griffith, L.; Oremus, M.; Raina, P.; Ismaila, A.; et al. Conducting Quantitative Synthesis When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. J. Clin. Epidemiol. 2011, 64, 1187–1197. [Google Scholar] [CrossRef]
  48. Viechtbauer, W. Conducting Meta-Analyses in R with the Metafor Package. J. Stat. Soft. 2010, 36, 1–48. [Google Scholar] [CrossRef] [Green Version]
  49. Cao, Q.; An, J.; Yang, Y.; Peng, P.; Xu, S.; Xu, X.; Xiang, H. Correlation among Psychological Resilience, Loneliness, and Internet Addiction among Left-behind Children in China: A Cross-Sectional Study. Curr. Psychol. 2020, 41, 4566–4573. [Google Scholar] [CrossRef]
  50. Choi, S.-W.; Kim, D.-J.; Choi, J.-S.; Ahn, H.; Choi, E.-J.; Song, W.-Y.; Kim, S.; Youn, H. Comparison of Risk and Protective Factors Associated with Smartphone Addiction and Internet Addiction. J. Behav. Addict. 2015, 4, 308–314. [Google Scholar] [CrossRef]
  51. Cui, X.; Chi, X. The Relationship Between Social Support and Internet Addiction Among Chinese Adolescents during the COVID-19 Pandemic: A Multiple Mediation Model of Resilience and Post-Traumatic Stress Disorder Symptoms. Psychol. Res. Behav. Manag. 2021, 14, 1665–1674. [Google Scholar] [CrossRef]
  52. Dong, Y.; Li, H. The Relationship between Social Support and Depressive Symptoms among the College Students of Liaoning, China: A Moderated Mediated Analysis. Psychol. Health Med. 2020, 25, 368–378. [Google Scholar] [CrossRef] [PubMed]
  53. Hsieh, Y.-P.; Hwa, H.-L.; Shen, A.C.-T.; Wei, H.-S.; Feng, J.-Y.; Huang, C.-Y. Ecological Predictors and Trajectory of Internet Addiction from Childhood through Adolescence: A Nationally Representative Longitudinal Study. J. Environ. Res. Public Health 2021, 18, 6253. [Google Scholar] [CrossRef]
  54. Jin, S.W.; Jones, T.V.; Lee, Y. Does Resilience Mediate the Link between Depression and Internet Addiction among African American University Students? J. Negro Educ. 2019, 88, 114. [Google Scholar] [CrossRef]
  55. Kiss, H.; Fitzpatrick, K.M.; Piko, B.F. The Digital Divide: Risk and Protective Factors and the Differences in Problematic Use of Digital Devices among Hungarian Youth. Child. Youth Serv. Rev. 2020, 108, 104612. [Google Scholar] [CrossRef]
  56. Lee, J.; Won, S.; Chang, S.M.; Kim, B.-S.; Lee, S.J. Prevalence of Addictive Behaviors in Medical Students and Their Association With Stress. Psychiatry Investig. 2022, 19, 44–53. [Google Scholar] [CrossRef] [PubMed]
  57. Mak, K.K.; Jeong, J.; Lee, H.-K.; Lee, K. Mediating Effect of Internet Addiction on the Association between Resilience and Depression among Korean University Students: A Structural Equation Modeling Approach. Psychiatry Investig. 2018, 15, 962–969. [Google Scholar] [CrossRef]
  58. Nam, C.; Lee, D.; Lee, J.; Choi, A.; Chung, S.; Kim, D.-J.; Bhang, S.-Y.; Kwon, J.-G.; Kweon, Y.-S.; Choi, J.-S. The Role of Resilience in Internet Addiction among Adolescents between Sexes: A Moderated Mediation Model. J. Clin. Med. 2018, 7, 222. [Google Scholar] [CrossRef] [Green Version]
  59. Peng, C.; Wang, M.; Cheng, J.; Tan, Y.; Huang, Y.; Rong, F.; Kang, C.; Ding, H.; Wang, Y.; Yu, Y. Mediation of Internet Addiction on Association between Childhood Maltreatment and Suicidal Behaviours among Chinese Adolescents. Epidemiol. Psychiatr. Sci. 2021, 30, e64. [Google Scholar] [CrossRef]
  60. Saeed, M. The role of loneliness as a predictor of internet addiction with a mediating effect of self esteem and moderating effect of resilience among international students in China. Int. J. Psychosoc. Rehabil. 2020, 24, 4729–4748. [Google Scholar] [CrossRef]
  61. Salek Ebrahimi, L.; Ahmadi, G.; Masjedi Arani, A.; Mousavi, S.E. Predicting Internet Addiction in Medical Students by General Self-Efficacy, Difficulty in Emotion Regulation, and Resilience. Pract. Clin. Psychol. 2019, 7, 167–174. [Google Scholar] [CrossRef]
  62. Yilmaz, R.; Karaoglan Yilmaz, F.G. Problematic Internet Use in Adults: The Role of Happiness, Psychological Resilience, Dispositional Hope, and Self-Control and Self-Management. J. Ration.-Emot. Cogn.-Behav. Ther. 2022. [Google Scholar] [CrossRef] [PubMed]
  63. Zhang, H.; Li, D. Stressful Life Events and Problematic Internet Use in Adolescence: Mediation of Psychological Capital and Moderation of School Level. J. Adolesc. 2022, 94, 718–727. [Google Scholar] [CrossRef]
  64. Zhou, P.; Zhang, C.; Liu, J.; Wang, Z. The Relationship Between Resilience and Internet Addiction: A Multiple Mediation Model Through Peer Relationship and Depression. Cyberpsychol. Behav. Soc. Netw. 2017, 20, 634–639. [Google Scholar] [CrossRef] [PubMed]
  65. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; L. Erlbaum Associates: New York, NY, USA, 1988; ISBN 9780805802832. [Google Scholar]
  66. Rumpf, H.-J.; Effertz, T.; Montag, C. The Cost Burden of Problematic Internet Usage. Curr. Opin. Behav. Sci. 2022, 44, 101107. [Google Scholar] [CrossRef]
  67. Davydov, D.M.; Stewart, R.; Ritchie, K.; Chaudieu, I. Resilience and Mental Health. Clin. Psychol. Rev. 2010, 30, 479–495. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. Carli, V.; Durkee, T.; Wasserman, D.; Hadlaczky, G.; Despalins, R.; Kramarz, E.; Wasserman, C.; Sarchiapone, M.; Hoven, C.W.; Brunner, R.; et al. The Association between Pathological Internet Use and Comorbid Psychopathology: A Systematic Review. Psychopathology 2013, 46, 1–13. [Google Scholar] [CrossRef] [Green Version]
  69. Marciano, L.; Camerini, A.-L.; Schulz, P.J. Neuroticism and Internet Addiction: What Is next? A Systematic Conceptual Review. Personal. Individ. Differ. 2022, 185, 111260. [Google Scholar] [CrossRef]
  70. Prizant-Passal, S.; Shechner, T.; Aderka, I.M. Social Anxiety and Internet Use—A Meta-Analysis: What Do We Know? What Are We Missing? Comput. Hum. Behav. 2016, 62, 221–229. [Google Scholar] [CrossRef]
  71. Yadav, P.; Banwari, G.; Parmar, C.; Maniar, R. Internet Addiction and Its Correlates among High School Students: A Preliminary Study from Ahmedabad, India. Asian J. Psychiatry 2013, 6, 500–505. [Google Scholar] [CrossRef] [PubMed]
  72. Joyce, S.; Shand, F.; Tighe, J.; Laurent, S.J.; Bryant, R.A.; Harvey, S.B. Road to Resilience: A Systematic Review and Meta-Analysis of Resilience Training Programmes and Interventions. BMJ Open 2018, 8, e017858. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  73. Wisniewski, P.; Jia, H.; Wang, N.; Zheng, S.; Xu, H.; Rosson, M.B.; Carroll, J.M. Resilience mitigates the negative effects of adolescent internet addiction and online risk exposure. In Proceedings of the Conference on Human Factors in Computing Systems, Seoul, Republic of Korea, 18–23 April 2015; pp. 4029–4038. [Google Scholar]
Figure 1. PRISMA diagram of the search and selection process.
Figure 1. PRISMA diagram of the search and selection process.
Nursrep 13 00032 g001
Figure 2. Effect size for the relationship between PIU and resilience [33,34,35,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64].
Figure 2. Effect size for the relationship between PIU and resilience [33,34,35,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64].
Nursrep 13 00032 g002
Figure 3. Funnel plot for assessing publication bias.
Figure 3. Funnel plot for assessing publication bias.
Nursrep 13 00032 g003
Table 1. Summary of studies included in the meta-analysis.
Table 1. Summary of studies included in the meta-analysis.
StudyCountryContinentSampleAgeSex (% Men)PIU TestResilience TestRisk of Biasr
Cao et al., 2020 [49]ChinaAsia121811.855.25YDQCD-RISC 10Under−0.214
Choi et al., 2015 [50]South KoreaAsia44820.8939.7IATCD-RISCUnder−0.12
Cui & Chi, 2021 [51]ChinaAsia254416.4942.7YDQCD-RISC 10Under−0.267
Dinc & Topcu, 2021 [33]AustraliaOceania22014.1644.5CIUSCYRM-28High−0.29
Dong & Li, 2020 [52]ChinaAsia1362 53.9IAIICD-RISC 10Under−0.25
Hsieh et al., 2021 [53]ChinaAsia6233 51CIASCD-RISC 10Under−0.17
Jin et al., 2019 [54]USAAmerica32623.420.6IATBRSUnder−0.121
Kiss et al., 2020 [55]HungaryEurope24922.537.8PIU-QCD-RISC 10High−0.274
Lee et al., 2022 [56]South KoreaAsia866 70.8IAPSCD-RISCHigh−0.39
Mak et al., 2018 [57]South KoreaAsia83722.1343.13IATCD-RISCHigh−0.4
Nam et al., 2018 [58]South KoreaAsia519 51.64IATCD-RISCHigh−0.122
Öztürk & Kundakçı, 2021 [34]TurkeyEurope102820.1739.7IATBRSUnder−0.498
Peng et al., 2021 [59]ChinaAsia16,13015.2251.9IATRSCAUnder−0.252
Robertson et al., 2018 [35]USAAmerica24025.0565IATCD-RISCHigh−0.36
Saeed, 2020 [60]ChinaAsia43623.81 IATBRSHigh−0.15
Salek-Ebrahimi et al., 2019 [61]IranAsia9619.7321.1IATCD-RISCUnder−0.222
Yilmaz et al., 2022 [62]TurkeyEurope112346.758YIAT-SFBRSUnder−0.346
Zhang & Li, 2022 [63]ChinaAsia1228 YDQPPQHigh−0.38
Zhou et al., 2017 [64]ChinaAsia58,75610.8354.5YDQRRSHigh−0.218
YDQ: Young’s Diagnostic Questionnaire for Internet Addiction; IAT: Young’s Internet Addiction Test; CIUS: Compulsive Internet Use Scale; IAII: Internet Addiction Impairment Index; CIAS: Chen Internet Addiction Scale; PIU-Q: Problematic Internet Use Questionnaire; IAPS: Korean Internet Addiction Proneness Scale for Youth; YIAT-SF: Young’s Internet Addiction Test-Short Form; CD-RISC 10: Connor-Davidson Resilience Scale Short Form; CD-RISC: Connor-Davidson Resilience Scale; CYRM-28: Child and Youth Resilience Measure; BRS: Brief Resilience Scale; RSCA: Resilience Scale for Chinese Adolescents; PPQ: PsyCap Questionnaire; RRS: Revised Resilience Scale.
Table 2. Relationship between PIU and resilience: moderation analysis for categorical variables.
Table 2. Relationship between PIU and resilience: moderation analysis for categorical variables.
Zr95% CIpp Subgroup
Risk of individual bias 0.48
High−0.30 −0.38, −0.22<0.001
Under−0.26 −0.33, −0.18<0.001
Continent 0.15
Asia−0.25−0.32, −0.19<0.001
Other−0.34−0.44, −0.24<0.001
PIU test 0.90
IAT−0.27−0.35, −0.18<0.001
YDQ−0.28−0.40, −0.16<0.001
Other−0.30−0.40, −0.19<0.001
Resilience test 0.87
BRS−0.30−0.43, −0.18<0.001
CD-RISC−0.28−0.39, −0.18<0.001
CD-RISC 10−0.24−0.35, −0.13<0.001
Other−0.29−0.42, −0.17<0.001
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hidalgo-Fuentes, S.; Martí-Vilar, M.; Ruiz-Ordoñez, Y. Problematic Internet Use and Resilience: A Systematic Review and Meta-Analysis. Nurs. Rep. 2023, 13, 337-350. https://doi.org/10.3390/nursrep13010032

AMA Style

Hidalgo-Fuentes S, Martí-Vilar M, Ruiz-Ordoñez Y. Problematic Internet Use and Resilience: A Systematic Review and Meta-Analysis. Nursing Reports. 2023; 13(1):337-350. https://doi.org/10.3390/nursrep13010032

Chicago/Turabian Style

Hidalgo-Fuentes, Sergio, Manuel Martí-Vilar, and Yolanda Ruiz-Ordoñez. 2023. "Problematic Internet Use and Resilience: A Systematic Review and Meta-Analysis" Nursing Reports 13, no. 1: 337-350. https://doi.org/10.3390/nursrep13010032

APA Style

Hidalgo-Fuentes, S., Martí-Vilar, M., & Ruiz-Ordoñez, Y. (2023). Problematic Internet Use and Resilience: A Systematic Review and Meta-Analysis. Nursing Reports, 13(1), 337-350. https://doi.org/10.3390/nursrep13010032

Article Metrics

Back to TopTop