Next Article in Journal
Reviewing Sustainability Measurement Methods for Enterprises
Previous Article in Journal
Consumer Profiles of Sustainable Fruit and Vegetable Consumption in the European Union
Previous Article in Special Issue
Cultural Competence among Healthcare Professional Educators: A Mixed-Methods Study
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Enhancing Disaster Triage Competencies through Simulation-Based Training: An Interventional Study among Undergraduate Nursing Students

by
Amal Hamdi
1 and
Abdulellah Al Thobaity
2,*
1
Abha MaternSity and Children Hospital, Abha 62562, Saudi Arabia
2
Medical Surgical Nursing Department, College of Nursing, Taif University, Taif 21944, Saudi Arabia
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(21), 15513; https://doi.org/10.3390/su152115513
Submission received: 1 September 2023 / Revised: 27 October 2023 / Accepted: 30 October 2023 / Published: 1 November 2023
(This article belongs to the Special Issue Medical Education: The Challenges and Opportunities of Sustainability)

Abstract

:
This pre–post interventional study explores the effectiveness of simulation-based training in enhancing disaster nursing skills among nursing students at Taif University, Saudi Arabia. The training, which uses a realistic train accident simulation and involves a response team of healthcare professionals, aims to improve knowledge and performance in crisis management and triage during mass casualty incidents. The study’s necessity stems from the critical role nurses play in disaster response, requiring a comprehensive understanding of challenges, collaboration among stakeholders, and improved capabilities. A random sample of 101 nursing students voluntarily participated in the study, with the necessary approvals obtained. We measured their emergency management skills and knowledge using a detailed questionnaire (27 items) and conducted pretest and posttest evaluations. Data analysis was performed using SPSS. The results indicate the training’s effectiveness, as a significant portion of participants achieved high performance levels in the posttest, contrasting with a higher percentage of fail-level grades in the pretest. These findings underscore the potential to improve disaster management protocols and nursing professionals’ preparedness in Saudi Arabia. The study emphasizes the importance of comprehensive education in disaster nursing in enhancing emergency response and patient outcomes.

1. Introduction

Nurses hold a pivotal role in disaster response, necessitating a comprehensive understanding of typical disaster patterns. The discipline of disaster nursing aims to offer patient care to affected populations while also participating in disaster planning and preparedness at all levels. One study conducted on nurses dispatched to the Ya’an earthquake in 2013 underscored the necessity for well-articulated disaster plans and emergency in-service education [1]. Research conducted in Saudi Arabia revealed that emergency nurses often lack the required knowledge concerning disaster planning and management [1]. To enhance their disaster response capabilities, the study participants suggested three key training initiatives. However, nurses often encounter several challenges in the field of disaster nursing. These include a lack of preparedness, inadequate formal education, insufficient research, ethical and legal dilemmas, and lack of exposure to disaster situations. Therefore, to foster growth in disaster nursing, concerted efforts from educators, researchers, and practitioners are essential [2,3,4,5,6].
It is recommended that nursing programs incorporate disaster nursing components, thereby equipping nurses with the necessary skills and knowledge for effective disaster response and patient care [7]. Well-trained nurses play a pivotal role in disaster response, utilizing their skills to provide critical care and to manage resources effectively. Their readiness can significantly lessen the negative repercussions of disasters, contributing to community resilience and faster recovery [8,9,10]. By participating in educational programs and disaster response exercises, nurses can acquire expertise and competence in managing catastrophic situations [11]. The use of simulated drills and exercises, complemented by regular assessments and including practical drills, enhances nurses’ understanding of essential disaster management skills [12]. A solid understanding of the core principles of crisis management is crucial for nurses to perform effectively during catastrophic events [11,13,14]. Hence, equipping nurses with in-depth knowledge in crisis management is integral to fostering confident decision-making and a timely, efficient response during disaster situations [15]. Despite a growing need for disaster preparedness and healthcare responders in Saudi Arabia, there remains a significant gap in education and training in this sector [16]. A study conducted in the region identified incident management systems, disaster triage, and disaster drills as essential elements of education and training for emergency nurses, especially those with less than three years of experience [17,18,19]. Therefore, the emphasis on incident management systems, disaster triage, and disaster drills in education and training can significantly enhance the preparedness and response capabilities of early-career emergency nurses [20,21].
The existing framework for disaster nursing training includes the Homeland Security Exercise and Evaluation Program (HSEEP) and the International Council of Nurses (ICN) framework [22]. A novel educational method, likely involving simulation-based training, improved perceived disaster nursing knowledge and confidence among undergraduate nursing students [22]. However, disaster nursing knowledge scores varied significantly according to factors such as the students’ universities, gender, and disaster education [23]. An eight-module education program was proven to effectively increase knowledge of general disaster preparedness, disaster response self-efficacy, and psychological resilience in nursing students [24]. Yet, there are a lack of disaster education opportunities available to South Australian nurses [25], necessitating standardized disaster education to increase their disaster awareness and preparedness [25]. This need is also evident in China [26]. The proposed ‘SINCHI education model’, developed based on expert nurses’ experiences in international disaster response, emphasized simulation exercises and small-group work for effective disaster nursing education [27]. Evidence supported the simulation-based training’s efficacy in disaster nursing education [27]. A study comparing theme game-based teaching and multi-station disaster simulation found the former to be more effective in improving nursing students’ disaster nursing competency [28]. Despite the lack of statistically significant improvement, virtual simulation of an earthquake was found to be enjoyable and beneficial by students [29]. Virtual Reality Simulation (VRS) was also positively received by students, indicating its effectiveness in disaster procedure training [30]. Comparisons between VRS and traditional mannequin-based simulation found no significant differences in learning outcomes, suggesting VRS’s equal effectiveness [31]. The studies concluded that a need exists for optimal simulation-based education experiences aligning with learning outcomes and leveraging emerging technologies like VRS [30,31].
Disaster triage refers to the process of prioritizing medical care for the ill and injured during a disaster situation [17,32,33]. It involves segregating patients according to their medical needs and is generally carried out in these three stages: at the disaster scene, during transit to a medical facility, and upon arrival at the hospital [32,34,35]. Triage categories typically encompass emergency, urgent, non-urgent, and dying or deceased [36]. The nurse assigned to triage plays a vital role in categorizing patients based on their medical priority, necessitating the possession of appropriate knowledge and skills [28,37,38]. Simulation training methods, such as tabletop exercises, can aid in enhancing the clinical decision-making skills of nursing students [37,39]. These exercises, which can range from simple to more complex scenarios, involve reviewing operational plans, identifying potential areas of improvement, and promoting constructive dialogue [6,40].
In the context of Saudi Arabia, disaster preparedness studies tend to focus on the following areas: (a) general preparedness assessment, (b) core competencies, and (c) disaster planning [41,42,43,44,45,46]. However, these studies have a limited scope, particularly in the realm of education and training. The current methodologies for preparing nursing professionals in Saudi Arabia are deemed insufficient, with a notable lack of disaster drills, especially simulation exercises, being a key shortcoming in the education and training of emergency nurses [36,45,46,47,48]. This study principally aims to assess how simulation-based training enhances nursing students’ crisis management skills, particularly triage during mass casualty incidents. Using a scenario modeled on such an event, we identify challenges to refine disaster management protocols. Our flexible program adapts to evolving disaster scenarios, promoting sustainable education. By using reusable simulation models, it efficiently employs resources. The program boosts healthcare professionals’ capabilities, fostering resilient and sustainable practices. Offering immediate feedback, it encourages continuous learning in a controlled environment. Moreover, its scalability allows for wide-scale training across various contexts. This manuscript highlights the sustainable benefits of our disaster management training program.

2. Materials and Methods

2.1. Study Design and Procedure

This research utilized a quantitative interventional pre–post design, aiming to assess the effect of a training program on the disaster nursing abilities of students at the College of Nursing, Taif University, Saudi Arabia. A random sample of 101 students was selected from a pool of 135, using the Open Epi sample calculator to ensure a 95% confidence level for the study. All participants were sixth-level nursing students who agreed voluntarily to participate and complete both the pre- and posttest evaluations as required by the study. Figure 1 illustrates the conceptual process that starts with Selection of Participants, followed by the Pretest Evaluation, then proceeds to the Training Intervention, succeeded by the Posttest Evaluation, and concludes with Data Analysis. A disaster scenario simulation, a train accident with 80 passengers requiring immediate care and transportation, was devised for the study. The response team consisted of ten nurses, five doctors, and ten paramedics, guided by an emergency and disaster specialist. The team was equipped with the essential tools necessary for triage, transportation, and disability assistance. Additional backup was provided by several other hospitals, contributing 30 ambulances for the exercise. The simulation included law enforcement personnel to maintain security, reflecting a real disaster response scenario.

2.2. Questionnaire Design

The questionnaire was divided into the following three sections:
  • Socio-Demographic Data: This segment collected essential details about the respondents, such as gender, clinical experience, and previous training related to triage or RR procedures.
  • Knowledge Pertaining to Mass Casualty Incident (MCI): This section evaluated the participant’s comprehension and ability to handle a disaster situation, with questions based on a 5-point Likert scale.
  • Mass Casualty Incident (MCI) Simulation: In this segment, participants were presented with various disaster scenarios, and they had to categorize the cases based on a provided scale (1-Expectant (Black), 2-Immediate (Red), 3-Delayed (Yellow), 4-Minor (Green)).
The pretest was used to evaluate the students’ initial knowledge levels, whereas the posttest measured the effectiveness of the training intervention. The study was completely voluntary, and students could withdraw at any stage. Permissions to use the scales were obtained from the original authors, and the study was approved by the Institutional Review Board (IRB). There were no expected risks, costs, or payments associated with participation in the research project.

2.3. Simulation Methodology and Analysis

The initial step of the simulation setup involved conducting a pretest evaluation to gauge the baseline disaster nursing skills of the students. This evaluation served to ascertain the students’ initial understanding of the subject matter. Each participant was assigned a unique identifier number to ensure their anonymity. After the pretest evaluation, the training intervention was executed. This involved the creation of a detailed disaster scenario simulation, which was designed to mimic a train accident involving 80 passengers in need of immediate medical care and transportation. The students were divided into groups and each group was given an opportunity to participate in the simulation over two consecutive days. The response team for the simulation consisted of experienced medical professionals including nurses, doctors, and paramedics, who guided the students through the process.
Following the training intervention, a posttest evaluation was conducted to assess the effectiveness of the training. This evaluation was designed to measure the improvement in disaster nursing skills of the students after undergoing the training program. The same five-point Likert scale questionnaire was used, allowing for direct comparison of pre- and post-intervention data. The data collected from the pretest and posttest evaluations were then analyzed using the Statistical Package for the Social Sciences (SPSS), version 29. Descriptive analysis was used to understand the overall trends and patterns in the data. Additionally, a paired sample t-test was conducted to identify any statistically significant differences in the pre- and posttest scores. A p-value of ≤0.05 was considered to indicate statistical significance.

3. Results

The gender distribution was approximately balanced, with 50.4% (62 students) identifying as male and 49.6% (61 students) as female. (Table 1) The majority of the students, 59.3% (73 students), had accrued clinical experience, whereas 40.7% (50 students) had yet to amass such experience. Regarding prior training courses, Basic Life Support (BLS) training had been undertaken by a considerable proportion of students, totaling 59.3% (73 students) of the group. A small subset, 1.6% (2 students), had completed Advanced Cardiovascular Life Support (ACLS) training, and a marginally larger group, 2.4% of the total (3 students), had been trained in Advanced Trauma Life Support and Advanced Trauma Care for Nurses (ATLS, ATCN). Significantly, there existed a substantial fraction, 36.6% (45 students), who had not partaken in any of these specified training courses.
Test scores (Table 2) generally improved from pretest to posttest, with a wider score range in the latter suggesting varied individual performance. The Red category saw a substantial rise in the mean score, with a broader score dispersion. The Green category had a slight decrease in the mean score, but a wider score spread. The Black category experienced a minor mean score increase and a reduced score variability. The Yellow category showed improvement with a more closely clustered score distribution in the posttest.
The data (Table 3) show a general trend of increased understanding in several areas, such as using the SALT tool and identifying mass casualties. However, decreased understanding is seen in areas like disaster management phases and prioritizing patient survival. The largest knowledge gain was in using the SALT tool (mean increase from 3.52 to 4.27), while the largest decrease was in prioritizing patient survival (mean decrease from 2.63 to 1.83). The highest pre- and post-assessment scores were in understanding the SALT triage algorithm and using the SALT tool, respectively. The lowest scores in both assessments were in recognizing disasters. Increased standard deviations in the post-assessment suggest greater score variability.
The paired-samples t-test (Table 4) demonstrated a statistically noteworthy advancement in scores from pretest to posttest. The average posttest scores were higher, and this improvement was statistically significant. Additionally, the effect size, gauged via Cohen’s d and Hedges’ g, registered at approximately −0.31, indicating a small to medium effect. This suggests that our intervention had a measurable impact on the enhancement of students’ disaster nursing skills.
The analysis identifies several correlations among the different posttest scores (Table 5). Weak positive correlations exist between Red and Black scores, and Green and Yellow scores, indicating that higher scores in one tend to accompany higher scores in the other. Moderately positive correlations are observed between Green posttest scores and both Green pretest and Black posttest scores, suggesting a stronger relationship. However, these relationships, while statistically significant, do not imply causation, and only indicate scoring trends across different tests.
The regression analysis revealed a relationship between the Black posttest score and the scores of Red, Green, and Yellow posttests. A model incorporating Red, Green, and Yellow posttests as predictors significantly enhanced the prediction of Black posttest scores. However, it explained only about 14.4% of the variance in Black posttest scores, suggesting that other factors may be influencing these scores. Green and Red posttest scores significantly predicted Black posttest scores, while Yellow posttest scores did not. This indicates that increases in Red and Green posttest scores were associated with increases in Black posttest scores, but the same did not hold true for Yellow posttest scores. The moderate correlation coefficient of 0.406 in the model further underscores these relationships (Table 6).
Significant correlations were identified among various pairs of posttest variables, as detailed in Table 7. Notably, the posttest scores of Black showed significant correlations with those of Red, Green, and Yellow. Additionally, Green posttest scores also exhibited significant correlations with Green pretest scores and Yellow posttest scores.

4. Discussion

In high-stakes situations like catastrophic disasters, where split-second decisions can mean the difference between life and death, the importance of diligence, skill, and quick thinking cannot be overstated. Practice plays a crucial role in developing practitioners who can excel in such critical moments. Since disasters are inevitable, being prepared is essential for effective disaster management. Nursing educators and numerous national and international nursing organizations have recognized the significance of preparedness and have worked towards incorporating it into the curriculum by identifying a set of optimal competencies [40,49,50]. The aim of this study was to assess the impact of a training intervention on disaster nursing students, with the goal of better preparing them for such situations in the future.
The participant demographics and training background were characterized by a fairly equal distribution of gender, with males and females nearly equally represented. This study presents a notable departure from previous research in Saudi Arabia, where the majority of participants were female nurses. In contrast, this study observed an equal number of male and female student nurses at Taif University. This finding highlights a shift in participant demographics and underscores the importance of inclusivity in re-search to capture a more comprehensive understanding of the nursing profession and healthcare landscape. The majority of participants had prior clinical experience, while a decent proportion did not. Basic Life Support (BLS) was the most common training course completed, and a small percentage of participants had advanced training in courses such as ACLS, ATLS, and ATCN. Notably, a significant portion (36.6%, 45 individuals) had not received any of these specific training courses. These demographics and training background variations provide valuable context for understanding participants’ preparedness and expertise in the field of triage and Rapid Response procedures. Therefore, considering the diverse backgrounds and previous training experiences of participants, it is essential to develop a training program that caters to individual needs [11,17,42]. Customizing the program to address specific gaps in knowledge and skills identified through the analysis can help improve participants’ preparedness in triage and Rapid Response procedures [51]. Additionally, an advantageous strategy may be to offer advanced training courses like ACLS, ATLS, and ATCN to enhance expertise in managing complex situations [52] and collaboration with healthcare institutions to facilitate access to these courses.
It is crucial for nursing schools to enhance nursing students’ understanding and abilities in prioritizing patients based on the severity of their condition [43,44,53,54]. This can be achieved by providing supplementary training, education, resources, and mentorship, helping students develop the necessary knowledge and skills, and ensuring they are well-prepared for their careers in hospitals [55,56]. Comprehensive education and training should balance theoretical and practical aspects, emphasizing vital skills such as communication and decision-making [17,57,58]. An overall improvement in test scores suggests the effectiveness of training, but varied individual performances and a slight mean score decrease in certain categories underscore the need for differentiated instruction or additional support, hinting at potential training inadequacies or the complexity of some concepts. These findings reinforce the importance of nursing schools preparing students for patient prioritization tasks in their future hospital careers.
The data indicate improved understanding of the SALT tool and identification of mass casualties, yet they also highlight the decreased comprehension in disaster management and the prioritization of patient survival. The most significant knowledge increase was observed in use of the SALT tool, while the most substantial decrease was seen in prioritizing patient survival. This system, by quickly categorizing victims based on the severity of their injuries, allows first responders to efficiently allocate resources and provide immediate care to those most in need during disaster situations [59,60]. Lower scores and higher post-assessment variability in recognizing disasters suggest potential areas for improvement. These areas, potentially encompassing aspects like disaster management protocols, emergency response strategies, and inter-agency coordination, necessitate further exploration through research to enhance the effectiveness of disaster response [16,17,46]. Emphasis must be shifted towards practical, scenario-based learning experiences, fostering a more robust understanding of disaster management and patient prioritization within a disaster nursing context. Moreover, formative assessments should be employed throughout the course, enabling early identification and correction of comprehension gaps in disaster nursing concepts, thus enhancing post-assessment results and preparing students more effectively for real-world disaster scenarios [16,17,46,61]. In essence, these findings necessitate a shift towards a more dynamic, responsive, and practice-centered educational model in disaster nursing.
Our study findings reveal a statistically significant improvement in disaster nursing skills following the intervention, as evidenced by increased posttest scores corresponding to a small to medium effect size. This underscores the effectiveness of our intervention, yet also suggests potential for further enhancement. This aligns with previous literature which showed that using simulations to enhance the knowledge and skills of disaster nurses are very effective [62] Despite these results emphasizing the value of continuous evaluation and iterative improvement in disaster nursing education to ensure students are well-prepared for their professional roles, we must consider that our sample was collected from a single university. Therefore, it’s crucial to replicate this study in diverse settings to ensure broader applicability and generalizability.
The correlations observed among different test scores suggest a degree of overlap in the underlying skills assessed, highlighting the potential benefits of a holistic, multi-faceted approach to disaster nursing education. These findings indicate the interconnectedness of disaster nursing skills, underscoring the need for a holistic educational approach. They also suggest the possible utility of reevaluation of assessment design and restructuring of training programs, and highlight the importance of future research to further explore these correlations and their implications for disaster nursing practice [63,64]. However, the weak to moderate strength of these correlations indicates that each test likely evaluates distinct competencies, necessitating a comprehensive curriculum. These findings underscore the complexity of disaster nursing education and the importance of both targeted and well-rounded educational strategies.
The regression analysis indicates a significant relationship between Black (Posttest) scores and the Red (Posttest) and Green (Posttest) scores. Despite the model only accounting for 14.4% of variance in the Black (Posttest) scores, it significantly enhances prediction of these scores, suggesting that other factors may also be at play. Although Red (Posttest) and Green (Posttest) scores predict Black (Posttest) scores, no such prediction is seen with Yellow (Posttest) scores. As a result, the training program has successfully improved participants’ understanding and performance in triage and Rapid Response procedures [65]. By providing nursing students with training in disaster triage, they will be better equipped to handle emergency situations, assess and prioritize patients, make informed decisions, communicate effectively with the team, and respond swiftly to critical situations [66,67]. Given the importance of training nursing students in the domain of disaster triage and rapid response, investing in training will result in tangible benefits by improving emergency response capabilities and potentially enhancing patient outcomes [68,69].

5. Limitation

Limitations exist regarding nursing students’ understanding and competency development in tragedy care. The descriptive and post-analysis research design has limitations, and a qualitative research design is needed for in-depth understanding. Finally, there is a limitation in understanding confounding variables, such as environmental consequences, situational factors, and event concerns, which can affect patient care.

6. Recommendation

In our study, we propose a multifaceted approach to improve disaster nursing competencies. We recommend customizing training programs to cater to individual needs, taking into account participants’ diverse backgrounds and previous experiences. We suggest offering advanced training courses such as Advanced Cardiac Life Support (ACLS), Advanced Trauma Life Support (ATLS), and Advanced Trauma Care for Nurses (ATCN) through collaborations with healthcare institutions. Our strategy also emphasizes enhancing nursing students’ understanding and abilities in triage, including patient prioritization based on severity, and through supplementary training, resources, and mentorship. We advocate for comprehensive education in disaster triage, integrating theoretical knowledge with practical skills, including communication and decision-making. Finally, we underline the need for further investment in disaster triage training for nursing students, a step we believe is crucial for improving emergency response capabilities and patient outcomes.

7. Conclusions

This study reiterates the pivotal role of preparation and specialized training in disaster nursing. Tailored training programs combined with advanced courses can significantly elevate disaster preparedness and proficiency among nurses. There is a clear need for additional support to enhance the knowledge and skills in disaster triage among nursing students. Our findings show that the implemented training program successfully improved participants’ knowledge and performance. Investments in disaster triage training are critical for better patient outcomes and more effective responses to mass casualty incidents. This study highlights disaster nursing education as a key factor in improving emergency response and patient outcomes. From a societal perspective, enhancing disaster nursing competencies has far-reaching implications. It can boost community safety and public health by facilitating effective crisis management, potentially saving lives, and mitigating injury severity. This contributes to building more resilient communities and robust healthcare systems, especially in disaster-prone areas. In terms of research implications, our study paves the way for future investigations into the effectiveness of different teaching methods in disaster nursing education, their scalability, and their long-term impact on professional development and patient outcomes. Our research illuminates the need for studies that delve deeper into the social implications of improved disaster nursing competencies, focusing on how these competencies can shape public health outcomes, community resilience, and overall societal wellbeing in the face of disasters.

Author Contributions

Conceptualization, A.H.; Methodology, A.H.; Formal analysis, A.H. and A.A.T.; Writing—original draft, A.H.; Writing—review & editing, A.H. and A.A.T.; Supervision, A.A.T. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki. Ethical approval for this study was granted by the Taif University Human Research Ethics Committee, with the Certificate of Approval issued on 21/08/2022 (Approval 44-001)-(HAO-02-T-105).

Informed Consent Statement

Before the intervention, we obtained informed consent from all participants. The principal investigator explained the study’s aims, potential risks, and benefits, emphasizing that participation was voluntary, confidential, and cost-free. No payments were offered for involvement in this study. Each participant received a unique code to complete the online questionnaire, preserving anonymity. Signed consents were collected by a research assistant after the researchers left the room to ensure participants’ privacy.

Data Availability Statement

Data is unavailable due to privacy or ethical restrictions.

Acknowledgments

The authors express their gratitude to the deanship of scientific research at Taif University for funding this study. They also acknowledge the continuous support provided by the Deanship of Postgraduate Studies at Taif University.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Alzahrani, F.; Kyratsis, Y. Emergency nurse disaster preparedness during mass gatherings: A cross-sectional survey of emergency nurses’ perceptions in hospitals in Mecca, Saudi Arabia. BMJ Open 2017, 7, e013563. [Google Scholar] [CrossRef] [PubMed]
  2. Al Harthi, M.; Al Thobaity, A.; Al Ahmari, W.; Almalki, M. Challenges for nurses in disaster management: A scoping review. Risk Manag. Healthc. Policy 2020, 13, 2627–2634. [Google Scholar] [CrossRef] [PubMed]
  3. Li, Y.; Li, S.; Chen, S.; Xie, X.; Song, Y.; Jin, Z.; Zheng, X. Disaster nursing experiences of Chinese nurses responding to the Sichuan Ya’an earthquake. Int. Nurs. Rev. 2017, 64, 309–317. [Google Scholar] [CrossRef] [PubMed]
  4. MIURA, S.; KONDO, A.; TAKAMURA, Y. Practices and challenges of disaster nursing for Japanese nurses sent to Nepal following the 2015 earthquake. Health Emerg. Disaster Nurs. 2020, 7, 46–54. [Google Scholar] [CrossRef]
  5. Pourvakhshoori, N.; Norouzi, K.; Ahmadi, F.; Hosseini, M.; Khankeh, H. Nursing in disasters: A review of existing models. Int. Emerg. Nurs. 2017, 31, 58–63. [Google Scholar] [CrossRef]
  6. Powers, R.; Daily, E. International Disaster Nursing; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
  7. Latif, M.; Abbasi, M.; Momenian, S. The effect of educating confronting accidents and disasters on the improvement of nurses’ professional competence in response to the crisis. Q. J. Health Emergencies Disasters 2019, 4, 147–156. [Google Scholar] [CrossRef]
  8. Bell, S.A.; Inloes, J.B.; Donnelly, J.; Wasserman, M.; Wyte-Lake, T. Improving Nursing Home Disaster Readiness Through Implementation Science. J. Am. Med. Dir. Assoc. 2023. [Google Scholar] [CrossRef]
  9. Lucci, S.; Taliaferro, D. Readiness to Respond: Nursing Students’ Perspectives From a Virtual Disaster-Simulated Experience. Int. J. Hum. Caring 2023, 27, 70–80. [Google Scholar] [CrossRef]
  10. Veenema, T.G.; Lavin, R.P.; Griffin, A.; Gable, A.R.; Couig, M.P.; Dobalian, A. Call to action: The case for advancing disaster nursing education in the United States. J. Nurs. Scholarsh. 2017, 49, 688–696. [Google Scholar] [CrossRef]
  11. Kalanlar, B. Effects of disaster nursing education on nursing students’ knowledge and preparedness for disasters. Int. J. Disaster Risk Reduct. 2018, 28, 475–480. [Google Scholar] [CrossRef]
  12. Farokhzadian, J.; Farahmandnia, H.; Tavan, A.; Taskiran Eskici, G.; Soltani Goki, F. Effectiveness of an online training program for improving nurses’ competencies in disaster risk management. BMC Nurs. 2023, 22, 334. [Google Scholar] [CrossRef] [PubMed]
  13. Loke, A.Y.; Guo, C.; Molassiotis, A. Development of disaster nursing education and training programs in the past 20 years (2000–2019): A systematic review. Nurse Educ. Today 2021, 99, 104809. [Google Scholar] [CrossRef] [PubMed]
  14. Schultz, C.H.; Koenig, K.L.; Whiteside, M.; Murray, R.; National Standardized All-Hazard Disaster Core Competencies Task Force. Development of national standardized all-hazard disaster core competencies for acute care physicians, nurses, and EMS professionals. Ann. Emerg. Med. 2012, 59, 196–208.e191. [Google Scholar] [CrossRef] [PubMed]
  15. Goniewicz, M.; Khorram-Manesh, A.; Timler, D.; Al-Wathinani, A.M.; Goniewicz, K. Hospital Disaster Preparedness: A Comprehensive Evaluation Using the Hospital Safety Index. Sustainability 2023, 15, 13197. [Google Scholar] [CrossRef]
  16. Al Thobaity, A.; Plummer, V.; Innes, K.; Copnell, B. Perceptions of knowledge of disaster management among military and civilian nurses in Saudi Arabia. Australas. Emerg. Nurs. J. 2015, 18, 156–164. [Google Scholar] [CrossRef]
  17. Brinjee, D.; Al Thobaity, A.; Almalki, M.; Alahmari, W. Identify the disaster nursing training and education needs for nurses in Taif City, Saudi Arabia. Risk Manag. Healthc. Policy 2021, 14, 2301–2310. [Google Scholar] [CrossRef]
  18. Noh, J.; Oh, E.G.; Kim, S.S.; Jang, Y.S.; Chung, H.S.; Lee, O. Development and evaluation of a multimodality simulation disaster education and training program for hospital nurses. Int. J. Nurs. Pract. 2020, 26, e12810. [Google Scholar] [CrossRef]
  19. Wong, L.C.; Glauberman, G.H.; Katz, A.R.; Loos, J.R.; Bray, M.; Arndt, R.G.; Teruya, K.; Peterman, K.; Masaki, K. Interprofessional disaster simulation during the COVID-19 pandemic: Adapting to fully online learning. Clin. Simul. Nurs. 2022, 63, 10–15. [Google Scholar] [CrossRef]
  20. Wang, J.; Sun, X.; Lu, S.; Wang, F.; Wan, M.; Chen, H.; Tan, Y. Disaster preparedness and associated factors among emergency nurses in guangdong province, china: A descriptive cross-sectional study. Disaster Med. Public Health Prep. 2023, 17, e65. [Google Scholar] [CrossRef]
  21. Abu Hasheesh, M.O. Jordanian Nurses’ Perceived Disaster Preparedness: Factors Influencing Successful Planning. Sci. World J. 2023, 2023, 5473777. [Google Scholar] [CrossRef]
  22. Levoy, K.; DeBastiani, S.D.; McCabe, B.E. Evaluation of a novel disaster nursing education method. Disaster Med. Public Health Prep. 2018, 12, 703–710. [Google Scholar] [CrossRef] [PubMed]
  23. Ozpulat, F.; Kabasakal, E. Knowledge Levels of Nursing Students on Disaster Nursing and Their State of Disaster Preparedness. Int. J. Med. Res. Health Sci. 2018, 7, 165–174. [Google Scholar]
  24. Yildiz, C.C.; Yildirim, D. The Effects of Disaster Nursing Education Program on Beliefs in General Disaster Preparedness, Disaster Response Self-Efficacy, and Psychological Resilience in Nursing Students: A Single-Blind, Randomized Controlled Study. Nurs. Educ. Perspect. 2022, 43, 287–291. [Google Scholar] [CrossRef] [PubMed]
  25. Duong, K. Disaster education and training of emergency nurses in South Australia. Australas. Emerg. Nurs. J. 2009, 12, 86–92. [Google Scholar] [CrossRef]
  26. Zhou, W.J.; Turale, S.; Stone, T.E.; Petrini, M.A. Chinese nurses’ relief experiences following two earthquakes: Implications for disaster education and policy development. Nurse Educ. Pract. 2015, 15, 75–81. [Google Scholar] [CrossRef]
  27. Shinchi, K.; Matsunaga, H.; Fukuyama, Y. Proposal of a Model of Disaster Medical Education for Practical Risk Management and Disaster Nursing: The SINCHI Education Model. Prehospital Disaster Med. 2019, 34, 438–441. [Google Scholar] [CrossRef]
  28. Ma, D.H.; Shi, Y.X.; Zhang, G.; Zhang, J. Does theme game-based teaching promote better learning about disaster nursing than scenario simulation: A randomized controlled trial. Nurse Educ. Today 2021, 103, 104923. [Google Scholar] [CrossRef]
  29. Foronda, C.L.; Shubeck, K.; Swoboda, S.M.; Hudson, K.W.; Budhathoki, C.; Sullivan, N.; Hu, X.G. Impact of Virtual Simulation to Teach Concepts of Disaster Triage. Clin. Simul. Nurs. 2016, 12, 137–144. [Google Scholar] [CrossRef]
  30. Ulrich, D.; Farra, S.; Smith, S.; Hodgson, E. The Student Experience Using Virtual Reality Simulation to Teach Decontamination. Clin. Simul. Nurs. 2014, 10, 546–553. [Google Scholar] [CrossRef]
  31. Smith, S.; Farra, S.L.; Hodgson, E. Evaluation of two simulation methods for teaching a disaster skill. Bmj Simul. Technol. Enhanc. Learn. 2021, 7, 92–96. [Google Scholar] [CrossRef]
  32. Bazyar, J.; Farrokhi, M.; Khankeh, H. Triage systems in mass casualty incidents and disasters: A review study with a worldwide approach. Open Access Maced. J. Med. Sci. 2019, 7, 482. [Google Scholar] [CrossRef] [PubMed]
  33. Vassallo, J.; Beavis, J.; Smith, J.E.; Wallis, L.A. Major incident triage: Derivation and comparative analysis of the Modified Physiological Triage Tool (MPTT). Injury 2017, 48, 992–999. [Google Scholar] [CrossRef] [PubMed]
  34. Dadashzadeh, A.; Khani, J.; Soleymani, M. Triage in the mass casualties in pre-hospital emergency. J. Inj. Violence Res. 2019, 11, 146. [Google Scholar]
  35. Glauberman, G.H.; Wong, L.C.; Bray, M.L.; Katz, A.R. Disaster aftermath interprofessional simulation: Promoting nursing students’ preparedness for interprofessional teamwork. J. Nurs. Educ. 2020, 59, 353–356. [Google Scholar] [CrossRef]
  36. Ranse, J.; Zeitz, K. Disaster Triage. In International Disaster Nursing; Cambridge University Press: Cambridge, UK, 2010; pp. 57–80. [Google Scholar]
  37. Farhadloo, R.; Kashani Nejad, M.; Haji Mohammad Hoseini, M.; Vahedian, M.; Parvaresh Masoud, M. Investigating the effect of training with the method of simulation on the knowledge and performance of nursing students in the pre-hospital Triage. Health Emergencies Disasters Q. 2018, 3, 123–130. [Google Scholar] [CrossRef]
  38. Hu, F.; Yang, J.; Yang, B.X.; Zhang, F.-J.; Yu, S.-H.; Liu, Q.; Wang, A.-L.; Luo, D.; Zhu, X.-P.; Chen, J. The impact of simulation-based triage education on nursing students’ self-reported clinical reasoning ability: A quasi-experimental study. Nurse Educ. Pract. 2021, 50, 102949. [Google Scholar] [CrossRef]
  39. Olsson, M.; Svensson, A.; Andersson, H.; Dehre, A.; Elmqvist, C.; Rask, M.; Wireklint, S.; Boysen, G.N. Educational intervention in triage with the Swedish triage scale RETTS©, with focus on specialist nurse students in ambulance and emergency care–A cross-sectional study. Int. Emerg. Nurs. 2022, 63, 101194. [Google Scholar] [CrossRef]
  40. Hwang, W.J.; Lee, J. Effectiveness of the infectious disease (COVID-19) simulation module program on nursing students: Disaster nursing scenarios. J. Korean Acad. Nurs. 2021, 51, 648–660. [Google Scholar] [CrossRef]
  41. Mills, B.; Dykstra, P.; Hansen, S.; Miles, A.; Rankin, T.; Hopper, L.; Brook, L.; Bartlett, D. Virtual reality triage training can provide comparable simulation efficacy for paramedicine students compared to live simulation-based scenarios. Prehospital Emerg. Care 2020, 24, 525–536. [Google Scholar] [CrossRef]
  42. Hung, M.S.Y.; Lam, S.K.K.; Chow, M.C.M.; Ng, W.W.M.; Pau, O.K. The effectiveness of disaster education for undergraduate nursing students’ knowledge, willingness, and perceived ability: An evaluation study. Int. J. Environ. Res. Public Health 2021, 18, 10545. [Google Scholar] [CrossRef]
  43. Alshehri, B. Emergency nurses’ preparedness for disaster in the Kingdom of Saudi Arabia. J. Nurs. Educ. Pract. 2017, 7, 101–114. [Google Scholar] [CrossRef]
  44. Baker, O.G.; Alamri, A.A.; Aboshaiqah, A.E. A descriptive study to analyse the disaster preparedness among Saudi nurses through self-regulation survey. J. Nurs. Manag. 2019, 27, 1479–1484. [Google Scholar] [CrossRef] [PubMed]
  45. Al Thobaity, A.; Alamri, S.; Plummer, V.; Williams, B. Exploring the necessary disaster plan components in Saudi Arabian hospitals. Int. J. Disaster Risk Reduct. 2019, 41, 101316. [Google Scholar] [CrossRef]
  46. Al Thobaity, A.; Williams, B.; Plummer, V. A new scale for disaster nursing core competencies: Development and psychometric testing. Australas. Emerg. Nurs. J. 2016, 19, 11–19. [Google Scholar] [CrossRef]
  47. James, L.S.; Williams, M.L.; Camel, S.P.; Slagle, P. (Eds.) Nursing Student’s Attitudes toward Teams in an Undergraduate Interprofessional Mass Casualty Simulation. Nurs. Forum 2021, 56, 500–512. [Google Scholar] [CrossRef]
  48. Demir, S.; Tunçbilek, Z.; Alinier, G. The effectiveness of online Visually Enhanced Mental Simulation in developing casualty triage and management skills of paramedic program students: A quasi-experimental research study. Int. Emerg. Nurs. 2023, 67, 101262. [Google Scholar] [CrossRef]
  49. Jose, M.M.; Dufrene, C. Educational competencies and technologies for disaster preparedness in undergraduate nursing education: An integrative review. Nurse Educ. Today 2014, 34, 543–551. [Google Scholar] [CrossRef]
  50. Uslu, Y.; Unver, V.; Karabacak, U. Hybrid simulation in triage training. Int. J. Caring Sci. 2019, 12, 1626. [Google Scholar]
  51. King, R.V.; North, C.S.; Larkin, G.L.; Downs, D.L.; Klein, K.R.; Fowler, R.L.; Swienton, R.E.; Pepe, P.E. Attributes of effective disaster responders: Focus group discussions with key emergency response leaders. Disaster Med. Public Health Prep. 2010, 4, 332–338. [Google Scholar] [CrossRef]
  52. Franklin, B.E.; Carr, K.V.; Padden, D.L. Self-assessment of trauma competencies among army family nurse practitioners. Mil. Med. 2008, 173, 759–764. [Google Scholar] [CrossRef]
  53. Delnavaz, S.; Hassankhani, H.; Roshangar, F.; Dadashzadeh, A.; Sarbakhsh, P.; Ghafourifard, M.; Fathiazar, E. Comparison of scenario based triage education by lecture and role playing on knowledge and practice of nursing students. Nurse Educ. Today 2018, 70, 54–59. [Google Scholar] [CrossRef] [PubMed]
  54. Yuan, H.B.; Williams, B.A.; Man, C.Y. Nursing students’ clinical judgment in high-fidelity simulation based learning: A quasi-experimental study. J. Nurs. Educ. Pract. 2014, 4, 7. [Google Scholar] [CrossRef]
  55. Cruz, J.P.; Alshehry, A.; Almazan, J.U.; Alquwez, N.; Balay-odao, E.M.; Bajet, J.B. Psychometric assessment of the learning needs for disaster nursing scale Arabic version among baccalaureate nursing students in Saudi Arabia. Int. J. Disaster Risk Reduct. 2023, 91, 103689. [Google Scholar] [CrossRef]
  56. Alotaibi, R.M.; Alkhaldi, R.M.; Turkistani, A.A.; Alfaqih, M.M.; Alzaidi, L.S.; Alosaimi, Z.S.; Almutairi, M.H.; Al Thobaity, A. Exploring Factors and Challenges Influencing Nursing Interns’ Training Experiences in Emergency Departments in Saudi Arabia. Int. Med. Educ. 2023, 2, 131–140. [Google Scholar]
  57. Alharbi, N.S.; Youssef, H.A.; Felemban, E.M.; Alqarni, S.S.; Alharbi, N.M.; Alsayed, A.A.O.; Alshomrani, A.Z.A.; Shahbal, S. Saudi Emergency Nurses Preparedness For Biological Disaster Management At The Governmental Hospitals. J. Posit. Sch. Psychol. 2022, 6, 1218–1235. [Google Scholar]
  58. Bajow, N.; Mortelmans, L.J.; Maghraby, N.; Alatef Sultan, S.A.; Mani, Z.A.; Aloraifi, S. Disaster health education framework for short and intermediate training in Saudi Arabia: A scoping review. Front. Public Health 2022, 10, 932597. [Google Scholar] [CrossRef]
  59. Mehralian, G.; Pazokian, M.; Shahrestanaki, Y.A.; Salari, A.; Saberinia, A.; Soltani, S. Development and validation of SALT Triage method to facilitate the identification and classification of patients in Mass Casualty Incidents. J. Inj. Violence Res. 2023, 15. [Google Scholar]
  60. Deluhery, M.R.; Lerner, E.B.; Pirrallo, R.G.; Schwartz, R.B. Paramedic accuracy using SALT triage after a brief initial training. Prehospital Emerg. Care 2011, 15, 526–532. [Google Scholar] [CrossRef]
  61. Wang, J.; Li, C.; Zou, S.; Chen, H.; Xiang, J.; Hu, Y.; Huang, H.; Tan, Y. Psychometric evaluation of undergraduate student nurses’ learning perceived needs in disaster nursing: Two cross-sectional studies. Nurse Educ. Today 2020, 84, 104208. [Google Scholar] [CrossRef]
  62. Geng, C.; Luo, Y.; Pei, X.; Chen, X. Simulation in disaster nursing education: A scoping review. Nurse Educ. Today 2021, 107, 105119. [Google Scholar] [CrossRef]
  63. Huh, S.S.; Kang, H.Y. Effects of an educational program on disaster nursing competency. Public Health Nurs. 2019, 36, 28–35. [Google Scholar] [CrossRef] [PubMed]
  64. Jang, I.; Kim, J.S.; Lee, J.; Seo, Y. Educational needs and disaster response readiness: A cross-sectional study of clinical nurses. J. Adv. Nurs. 2021, 77, 189–197. [Google Scholar] [CrossRef] [PubMed]
  65. Sultan, M.A.S.; Khorram-Manesh, A.; Sørensen, J.L.; Berlin, J.; Carlström, E. Disaster Collaborative Exercises for Healthcare Teamwork in a Saudi Context. Int. J. Disaster Risk Sci. 2023, 14, 183–193. [Google Scholar] [CrossRef]
  66. Shirazi, F.B.; Moslehi, S.; Rasouli, M.R.; Masoumi, G. Identifying the Challenges of Simulating the Hospital Emergency Department in the Event of Emergencies and Providing Effective Solutions: A Qualitative Study. Disaster Med. Public Health Prep. 2023, 17, e202. [Google Scholar] [CrossRef] [PubMed]
  67. Liu, J.; Huang, Y.; Li, B.; Gui, L.; Zhou, L. Development and evaluation of innovative and practical table-top exercises based on a real mass-casualty incident. Disaster Med. Public Health Prep. 2023, 17, e200. [Google Scholar] [CrossRef]
  68. Adini, B.; Goldberg, A.; Laor, D.; Cohen, R.; Zadok, R.; Bar-Dayan, Y. Assessing levels of hospital emergency preparedness. Prehospital Disaster Med. 2006, 21, 451–457. [Google Scholar] [CrossRef]
  69. Yousefian, S.; Sohrabizadeh, S.; Jahangiri, K. Identifying the components affecting intra-organizational collaboration of health sector in disasters: Providing a conceptual framework using a systematic review. Int. J. Disaster Risk Reduct. 2021, 57, 102146. [Google Scholar] [CrossRef]
Figure 1. Conceptual Design.
Figure 1. Conceptual Design.
Sustainability 15 15513 g001
Table 1. Distribution of Personal Data for Students.
Table 1. Distribution of Personal Data for Students.
VariableValueNumberPercentage
GenderMale6250.4%
Female6149.6%
Clinical ExperienceYes7359.3%
No5040.7%
Training CourseBLS7359.3%
ACLS21.6%
ATLS, ATCN32.4%
None4536.6%
Table 2. Comparison of Mean Scores and Standard Deviations at Pretest and Posttest Across Different Categories.
Table 2. Comparison of Mean Scores and Standard Deviations at Pretest and Posttest Across Different Categories.
Variable PairPretest MeanPosttest MeanPretest Std. DeviationPosttest Std. Deviation
Test2.753.140.801.07
Red2.663.361.161.39
Green2.502.461.581.73
Black4.144.261.881.77
Yellow2.522.962.512.46
Table 3. Comparison of Mean and Standard Deviation for Knowledge Items in Pre- and Post Assessments.
Table 3. Comparison of Mean and Standard Deviation for Knowledge Items in Pre- and Post Assessments.
Knowledge ItemsPre (Mean)Pre (SD)Post (Mean)Post (SD)
Know the mass causality disaster START & SALT triage algorithm sequence3.550.943.582.27
Can prioritize sorting and assessing patient based on SALT tool3.520.954.271.77
Can describe the four Phases of Disaster Management3.501.022.562.51
Can identify mass casualty & multicausality3.430.983.622.25
Able to demonstrate how the mass casualty triage protocol is used during disasters3.420.892.972.47
Able to produce a high-level description of disaster events3.060.953.72.2
Can explain the difference between the objectives of emergency triage and disaster triage3.041.024.111.71
Able to assess the disaster2.80.912.932.47
Can prioritize patients to achieve maximum survival2.630.941.832.42
Can identify potential disaster risks to self and others2.480.881.912.44
Can recognize the disaster2.360.781.912.44
Table 4. Paired-Samples T-Test Analysis of Pretest and Posttest Score Differences.
Table 4. Paired-Samples T-Test Analysis of Pretest and Posttest Score Differences.
StatisticValueInterpretation
Mean (pretest–posttest)−0.39The average difference between pretest and posttest scores. Posttest scores were higher on average.
Standard Deviation1.26The variability of the differences between pretest and posttest scores.
t-value−3.46The test statistic. A negative value indicates that the pretest score mean is smaller than the posttest score mean.
Degrees of Freedom (df)122The number of independent pieces of information used in the estimates.
Significance (Two-Sided p-value)<0.01The p-value. Being less than 0.05, we can conclude there is a statistically significant increase in scores from pretest to posttest.
Cohen’s d−0.31An effect size measure, indicating a small to medium effect size according to Cohen’s guidelines.
Hedges’ correction−0.31Another measure of effect size that corrects for small sample bias, suggesting a small to medium effect size.
Table 5. Correlation Analysis Among Different Posttest Scores.
Table 5. Correlation Analysis Among Different Posttest Scores.
Variable 1Variable 2Pearson Correlationp-ValueInterpretation
Red (Posttest )Black (Posttest )0.260.01Weak positive correlation between Red (Posttest) and Black (Posttest). As scores on Red (Posttest) increase, there is a tendency for scores on Black (Posttest) to also increase.
Green (Posttest)Green (Pretest)0.29<0.01Moderate positive correlation between Green (Posttest) and Green (Pretest). Individuals who scored higher on Green (Pretest) tend to also score higher on Green (Posttest).
Green (Posttest)Black (Posttest)0.30<0.01Moderate positive correlation between Green (Posttest) and Black (Posttest ). As scores on Green (Posttest) increase, there is a tendency for scores on Black (Posttest ) to also increase.
Green (Posttest)Yellow (Posttest)0.220.01Weak positive correlation between Green (Posttest) and Yellow (Posttest). Individuals who scored higher on Green (Posttest) tend to also score higher on Yellow (Posttest).
Yellow (Posttest)Black (Posttest)0.210.01Weak positive correlation between Yellow (Posttest) and Black (Posttest). As scores on Yellow (Posttest) increase, there is a tendency for scores on Black (Posttest) to also increase.
Table 6. Regression Analysis of Posttest Scores.
Table 6. Regression Analysis of Posttest Scores.
VariableCoefficient (B)p-ValueSignificance
Yellow (Posttest)0.090.13Not Significant
Green (Posttest)0.260.01Significant
Red (Posttest)0.280.01Significant
Table 7. Correlation Coefficients of Paired Posttest Variables.
Table 7. Correlation Coefficients of Paired Posttest Variables.
Variable PairCorrelation Coefficient
Red Posttest-Black Posttest0.26 **
Green Posttest-Green Pretest0.29 **
Green Posttest-Black Posttest0.30 **
Yellow Posttest-Green Posttest0.22 *
Yellow Posttest-Black Posttest0.21 *
** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Hamdi, A.; Al Thobaity, A. Enhancing Disaster Triage Competencies through Simulation-Based Training: An Interventional Study among Undergraduate Nursing Students. Sustainability 2023, 15, 15513. https://doi.org/10.3390/su152115513

AMA Style

Hamdi A, Al Thobaity A. Enhancing Disaster Triage Competencies through Simulation-Based Training: An Interventional Study among Undergraduate Nursing Students. Sustainability. 2023; 15(21):15513. https://doi.org/10.3390/su152115513

Chicago/Turabian Style

Hamdi, Amal, and Abdulellah Al Thobaity. 2023. "Enhancing Disaster Triage Competencies through Simulation-Based Training: An Interventional Study among Undergraduate Nursing Students" Sustainability 15, no. 21: 15513. https://doi.org/10.3390/su152115513

APA Style

Hamdi, A., & Al Thobaity, A. (2023). Enhancing Disaster Triage Competencies through Simulation-Based Training: An Interventional Study among Undergraduate Nursing Students. Sustainability, 15(21), 15513. https://doi.org/10.3390/su152115513

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop