Next Article in Journal
Geographic Medical Overview of Noncommunicable Diseases (Cardiovascular Diseases and Diabetes) in the Territory of the AP Vojvodina (Northern Serbia)
Previous Article in Journal
Lifestyle Interventions for Prevention and Management of Diet-Linked Non-Communicable Diseases among Adults in Arab Countries
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 11
Issue 1
10.3390/healthcare11010046
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings?

by
Parisa Moll-Khosrawi
1,*,
Wolfgang Hampe
2,
Leonie Schulte-Uentrop
1,
Christian Zöllner
1,
Stefan Zimmermann
2 and
Thorben Huelmann
2
1
Department of Anaesthesiology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany
2
Institute of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany
*
Author to whom correspondence should be addressed.
Healthcare 2023, 11(1), 46; https://doi.org/10.3390/healthcare11010046
Submission received: 17 November 2022 / Revised: 13 December 2022 / Accepted: 22 December 2022 / Published: 23 December 2022
Browse Figure
Review Reports Versions Notes

Abstract

:
Non-technical skills (NTS) in medical care are essential to ensure patient safety. Focussing on applicants’ NTS during medical school admission could be a promising approach to ensure that future physicians master NTS at a high level. Next to pre-university educational attainment, many selection tests have been developed worldwide to facilitate and standardise the selection process of medical students. The predictive validity of these tests regarding NTS performance in clinical settings has not been investigated (yet). Therefore, we explored the predictive validities and prognosis of the Hamburg MMI (HAM-Int), HAM-Nat, PEA, and waiting as well as other quota (as example) designated by the Federal Armed Forces) for NTS performance in clinical emergency medicine training of medical students. During 2017 and 2020, N = 729 second, third, and fourth year students were enrolled within the study. The mean age of participants was 26.68 years (SD 3.96) and 49% were female students. NTS of these students were assessed during simulation scenarios of emergency training with a validated rating tool. Students admitted via waiting quota and designated by the Armed Forces performed significantly better than students admitted by excellent PEA (p = 0.026). Non-EU students performed significantly inferior (p = 0.003). Our findings provide further insight to explain how and if admission to medical school could predict NTS performance of further physicians.

1. Introduction

Traditionally, pre-university educational attainment (PEA) was used to select medical students. PEA has a high prognostic validity regarding academic success in medical school [1,2,3,4,5]. However, to facilitate and standardise the selection process of medical students, many selection tests have been developed worldwide [6]. These tests are expected to provide an incremental gain of knowledge in addition to the PEA. [7] The majority of these tests focus mainly on cognitive abilities, considering endpoints like examination results, drop-out rate, and duration of studies [1,4,8,9]. The “Hamburg Natural Science Test” (HAM-Nat) [10], “Test für Medizinische Studiengänge” (TMS) [3], “United Kingdom Clinical Aptitude Test” (UKCAT) [11], “BioMedical Admission Test (BMAT)” [12], or the “Medical College Admission Test” (MCAT) [13], are the most common tests in Europe which target cognitive skills. Highest incremental validities and correlations with study success have been shown for MCAT and HAM-Nat [14,15].
Next to cognitive tests, classical interviews [6], Multiple Mini Interviews (MMIs) [16] and Situational Judgement Tests (SJTs) [17] have gained a widespread popularity in medical school admission [18]. Although these tests are detached from medical or academic knowledge and mainly focus on psychosocial competencies, their predictive validity regarding non-cognitive abilities in clinical settings needs further investigation [18,19,20].
However, to ensure good patient care, it is important that the admission tests predict outcomes of interest, stretching beyond endpoints like semantic knowledge and recalling factual knowledge [21]. The performance of an individual in a clinical setting might be of greater significance than their results in a written test [21]. In accordance with this imperative, recent studies demonstrated the predictive validities for the BMAT and UKCAT regarding clinical practical and examination skills [21,22]. Further relevant outcomes need to be distinguished in future doctors are non-cognitive abilities in medical high-risk settings, such as emergency medicine [23,24,25]. The importance of these abilities has been emphasised repeatedly over the past years due to their essential impact on patient safety [26,27,28]. In the context of emergency medicine, these non-cognitive abilities are referred to as non-technical skills (NTS) [29]. NTS are defined as “the cognitive, social and personal resource skills that complement technical skills and contribute to safe and efficient task performance” [30]. They are detached from medical knowledge or technical skills and do not enhance with growing clinical experience [31]. Therefore, the early training of NTS already in undergraduate medical education has been demanded and supported [24,32,33].
Every physician is at risk of being confronted with life-threatening emergencies among their patients. Therefore, in order to prevent adverse events, NTS are essential for many medical specialties and not only limited to high-risk medical fields, such as anaesthesiology or emergency medicine [34].
Despite many educational efforts, the recent European Resuscitation Council Guidelines 2021 (ERC) identified a gap in knowledge and evidence regarding the best way to train and convey NTS [27]. NTS are acquired among the socialisation process of every human and are non-homogeneously dispersed [35]. Furthermore, NTS are skills which might be detached from being gained through deep learning or semantic efforts [36]. For high levels of NTS, knowledge has to be integrated and transferred into action and hereby, the expression of personality traits might be of even greater significance than educational strategies [36]. To ensure that future physicians master NTS at a high level, focusing on applicants’ NTS during medical school admission could be a promising approach [35]. Therefore, interest in the early identification of future doctors’’ NTS, already during admission to medical school, is increasingly growing [6,37]. So far, no published study has investigated the predictive validity of admission tests regarding NTS performance in a clinical setting [8].
To summarise, the early identification of students with NTS attributes is a promising approach, but knowledge about the predictive validity of admission tests that ensure these skills is scarce. Therefore, this prospective longitudinal study explored the predictive validities and prognosis of the Hamburg MMI (HAM-Int), HAM-Nat, PEA, and waiting quota for NTS performance in the clinical emergency medicine training of medical students in different years of medical school.

2. Methods

2.1. Study Design and Setting

We performed this prospective longitudinal study at University Medical Center Hamburg-Eppendorf from 2017 to 2020. Medical students who participated in the mandatory simulation-based emergency trainings of the Department of Anaesthesiology in their second, third and fourth years of medical school were sampled.
The undergraduate medical curriculum of the medical faculty of Hamburg is structured based on the spiral curriculum design described by Harden [38]. The learning contents of preceding teaching units are revisited and expanded in the subsequent teaching units. Aligned to this curriculum design, the anaesthesiology/emergency medicine teaching units take place from the first to the fourth year of medical school and build upon each other. Basic Life Support (lecture, seminar, mannequin-based training) is part of the first year, followed by Trauma training and Advanced Cardiac Life Support I (lecture, seminar, high fidelity simulation-based training) in the second year. Advanced Cardiac Life Support II (lecture, seminar, high fidelity simulation-based training) and operating room simulation (OR-SIM, adverse events in the operating theatre) (lecture, seminar, high fidelity simulation-based training) are part of the third year. In their fourth year of medical school, the students participate in Advanced Cardiac Life Support III (lecture, seminar, high fidelity simulation-based training).
A total of twelve–eighteen students are assigned to each high-fidelity simulation-training. Each training has a predefined set of standardised scenarios, through which the students rotate in randomly chosen small groups. The scenarios are standardised and solely for each type of training session. Each scenario is executed in a different room of the teaching facility. Three students of each small group form the emergency response team: one of the students partakes the role of the emergency physician/anaesthesiologist; two students partake the roles of paramedics/anaesthetic co-workers. Each scenario is supervised by an instructor (anaesthesiologist), who is experienced in medical teaching and emergency medicine. High-fidelity simulators (Resusci Anne, Laerdal, Scavenger, Norway) are used, which are suitable to create real-life settings and to train technical skills such as drug administration, endotracheal intubation, or defibrillation. The scenarios are composed of emergency situations, leading to cardiac arrest. With each subsequent simulation-based training the scenarios become more complex, requiring higher levels of technical and non-technical skills. To rule out the theory that NTS are influenced and biased by a lack of theoretical knowledge, each training session starts with a short 30 min seminar to refresh the theoretical knowledge of emergency medicine and cardiac life support among participants. Principles of Crew Resource Management (CRM) are addressed briefly; the role distribution and the corresponding tasks are clarified (physician, paramedic/anaesthetic co-workers).
At the beginning of each scenario, the emergency response team (small group of students) is provided with information on the emergency. This information is in accordance with a real-life setting and consists of hard facts, such as “cardiac arrest on the ward number 4”. The student in the role of the physician is supposed to lead the response team. This task includes to indicate and delegate necessary medical procedures and provide the medical care alongside with the team. Each scenario is followed by a debriefing which is conducted by the instructor, to provide emotional support and to talk through the medical facts. The debriefing is composed of three conceptual phases: gathering, analysing, and summarizing. The role of the instructor is that of a teacher, meaning that the debriefing is held in a conventional way.

2.2. Participants

Second, third and fourth year students who participated in the mandatory anaesthesiology teaching units of their medical school year were enrolled within the study (n = 729). Eligibility criteria were previous participation in “Basic Life Support” (1st year) and in “Trauma training” (2nd year), to rule out that unfamiliarity with the teaching format caused emotional stress and cognitive overload and hence caused bias in the study outcomes [39]. An overview of the demographic data of the study participants is provided in Table 1.
The study population was selected to medical school prior to 2020, before the changes of the German Federal Constitutional Court regarding admission to German medical schools were implemented. During the study period, medical schools had to select 40% of the applicants based on quotas such as waiting list, excellent PEA, students from non-EU nations, and other groups (hardship, medical officers from the Federal Armed Forces). The waiting list includes the applicants that have waited longest for admittance since high-school graduation. For the remaining 60% of the study places, the German medical schools were free to select the students by the procedure of their own choice. At Hamburg Medical School the first 100 places of this 60% quota were assigned to those applicants who had received the best PEA in combination with the best result of the HAM-Nat. The following about 100 places were assigned to applicants who also took part in the selection interviews (HAM-Int).
Demographic data of the study participants were collected independently from the study in a database after acceptance to medical school. Prior to each semester of the study period, the students were contacted with information on the study. Participation in the study was voluntary and written informed consent was obtained from each participant.

2.3. Outcomes

NTS were assessed using the “Anaesthesiology students’ non-technical skills” (AS-NTS) [40], which was developed to rate medical students’ NTS during simulation-based training. The AS-NTS has been broadly validated regarding feasibility, inter-rater reliability and has proven to be applicable independently from the rater’s experience in emergency medicine or medical education. Reliability of the AS-NTS has also been proven to be high (mean Cronbach’s alpha 0.88 for all subscales). Furthermore, it has been reported that the AS-NTS covers all the necessary NTS for student emergency education [40]. For each scenario of each training session, the instructor filled out the AS-NTS for the student partaking the physicians’ role. The AS-NTS is composed of three dimensions:
  • Planning tasks, prioritising, and problem-solving;
  • Teamwork and leadership;
  • Team orientation.
Performance is rated on a five-point Likert scale (1 = very good; 5 = very poor). An underlying skill structure is used to give behaviourally anchored rating examples to clarify what a “good” or “poor” performance on each dimension might look like.

2.4. Statistical Analysis

For all data descriptive statistics (standard deviations, means) were calculated.
The data were analysed using linear regression models. For the admission variable average treatment effects were calculated with Excellent PEA as the reference group. Modelfits were established via a global F test and the inspection of the adjusted R2. Furthermore ANOVAS (analysis of variances with multiple testing) were carried out as global tests. All calculations were performed in R version 4.1.2. [41].

3. Results

3.1. Participants

A total of n = 729 students participated in the study. Data from n = 620 students were included in the final analysis, who at least took the role of the physician one time during the simulation scenarios (the AS-NTS was only filled out for the physicians of each simulation scenario). From these students, 104 were admitted via waiting quota, 83 via excellent PEA, 192 via HAM-Nat, and 133 via HAM-Int. Further 18 non-EU students, 22 students who were designated by the Armed Forces, and 68 through the “other” quota were included.

3.2. Outcomes

A total of n = 1172 AS-NTS assessments were conducted. n = 72 assessments had to be excluded due to missing data. Table 2 summarises the admission of participants and the number of included AS-NTS ratings for each admission group as well as their mean age.
The overall performance of the whole group of students is depicted in Table 2.
Overall, the students improved their NTS during the study period, beginning with their worst performance on all AS-NTS dimensions at the first assessment (ACLS I). The best performance of NTS was assessed during the third training (OR-SIM) which declined slightly at the last assessment (ACLS III). The analysis of variances (ANOVA) showed significant differences of NTS performance between the training (p < 0.01) and significant differences between the points of the three dimensions of the AS-NTS (p = 0.016).
The comparison of NTS performance of the different cohorts of students indicated that students admitted via waiting quota and designated by the Armed Forces, performed significantly better than students admitted by excellent PEA (p = 0.026). Non-EU students showed significantly inferior performance on all AS-NTS dimensions (p = 0.003). The students admitted via HAM-Int performed slightly better on all the three NTS dimensions and the students admitted via HAM-Nat performed slightly inferior to the students admitted via excellent PEA. These differences were not statistically significant (p = 0.956 and p = 0.604) but relevant based on the global testing, which revealed more variances than there might be solely due to coincidence: Adjusted R² = 0.05, DF = 1087, F-Statistic = 7.055, p < 0.001 (Figure 1, Table 3).
To rule out the age of the participants as a confounding factor (on average, a.e. students from the waiting quota are older than students from the excellent PEA Quota), it was included in the statistical model but did not show a significant impact.

4. Discussion

In this prospective longitudinal study, the predictive validities of MMIs, HAM-Nat, PEA and waiting quota, regarding NTS performance in a simulated, real-life clinical setting were explored. We found that students admitted by waiting quota and designated by the Armed Federal Forces performed significantly better and non-EU students performed significantly inferior to the other students.
So far, only few admissions have been validated regarding professional success [21,22] and most investigations targeted endpoints which are linked to intellectual or cognitive abilities [1,4,21,42]. However, next to cognitive abilities, modelling studies have demonstrated that non-cognitive abilities, such as personality traits (conscientiousness) mediate academic performance [43,44]. Cognitive abilities might be even inferior predictors of clinical performance than the amount of relevant semantic knowledge [45]. In the context of emergency care, these non-cognitive skills are referred to as NTS. However, evidence on the predictive validity of medical school admission and performance of NTS in a simulated clinical emergency setting is scarce. It has been demonstrated that MMI results and supervisor ratings on communication skills during emergency medicine do correlate [46], but no published study targeted NTS as a whole construct. As NTS have been identified to be an important component ensuring and increasing patient safety [28], it would be preferable to target NTS of future doctors already in the selection process of medical schools [24]. Although the differentiation between non-cognitive and cognitive skills is challenging [47], MMIs are specifically designed to target non-cognitive skills [48]. In particular, the Hamburg MMI assesses psychosocial competencies, predominantly in simulation scenarios which are specially designed to target interpersonal skills (a.e. communication) [49]. Furthermore, since 2016, teamwork skills have been integrated into the dimensions of the MMI. Teamwork skills are composed of leadership and team orientation [50], which are important components of required NTS in emergency medicine. Therefore, they are targeted by many NTS taxonomies, as well as the AS-NTS [40,51]. Based on the design of the MMIs, it could be deduced that the MMI-selected students [8,52] would also display good NTS performance in the simulated clinical emergency settings. Contrary to our postulations, our results indicate no superior NTS performance of MMI-selected students. Therefore, the MMI seems not to measure competencies that are relevant in the practical context of emergency situations. These findings can be explained with several considerations. Based on interactionism, a social psychology theory, peoples’ behaviour in a situation is influenced by the environment, as well as on inter-individual interactions [53]. Therefore, the characteristics of situations determine how traits are expressed in behaviour. As the emotional stress of emergency simulation training is different than the stress during the MMI, the variability in trait-expressive behaviour within the two settings is plausible [54]. The same explanation can be transferred to the positive findings of Knorr et al., who—contrary to our study—did find positive predictive validities of MMIs and general practitioner evaluations on psychosocial competencies of students [48]. However, although NTS are the backbone of good patient care in critical situations, the question how to convey and train these skills remains unanswered [27]. It could be assumed, that NTS relevant for emergency medicine might underlie a far more complex structure of behavioural traits, than daily psychosocial competencies. Therefore, the MMI may assess a more subjective impression of general psychosocial competencies while the AS-NTS targets more objective competencies in a more complex scenario.
Excellent PEA, HAM-Nat, or HAM-Int scores require diligence and learning efforts and therefore, it could be reasonably assumed that these students have good NTS performance in emergency simulations. Nevertheless, we did not find superior NTS performance of neither MMI-nor PEA or HAM-Nat selected students. A possible explanation might be that academic success and performance of practical skills are positively influenced by diligence and learning effort but NTS are detached from these efforts [36]. In support of this argument, recent approaches from genetic research, that identified the genetic basis for non-cognitive skills by separating educational success from intellectual levels, merits consideration [55].
Other factors may influence the individual expression of non-cognitive skills and these personal skills are non-homogeneously distinctive as they are acquired among the socialisation progress of every individual [35]. The expression of NTS might depend on personal traits like conscientiousness [30], which is a strongly predictive personality factor for clinical performance, as well as NTS [9,56,57]. Conscientiousness increases with age [56]; therefore, we assumed that the students who were admitted via waiting quote, would perform better due to higher levels of conscientiousness and consecutively NTS. Although those students showed a superior performance, the statistical analysis of influencing factors showed that age itself is not determining the expression of NTS and the relationship between age and NTS is not linear. The determining factor for NTS expression showed to be the admission quota to medical school.
As stated before, it is important that admission tests predict relevant postgraduate outcomes of future doctors and ideally, they are assessed in postgraduate training during the actual clinical phase [21]. Therefore, one might question our study design. We targeted NTS in undergraduate training, while using simulation-based training as a proxy for a clinical setting. Linking admission results to actual clinical performance is defying. Nevertheless, our results are transferable to postgraduate performance to a certain extent. The argument against the study setting can be ruled out, when considering how standardly NTS are trained and assessed. In high-risk settings, such as aviation and nuclear power plants, NTS are key elements for licensing and are assessed in simulation-based training [58,59,60]. Much the same applies for emergency medicine. Even in specialty training, NTS are assessed and trained in simulation-based settings, because the real clinical setting has not proven to be adequate for this purpose [34]. Therefore, using a simulation-based setting is a plausible proxy to assess actual clinical behaviour. The question if high levels of NTS assessed in undergraduates, also predict the levels of NTS in their later professional life, remains unanswered. Therefore, the generalization of our results is not entirely possible. Nevertheless, as NTS do not enhance with clinical experience [31], it can be concluded that the personality traits and their expression already at student age might have a significance for later professional life. As an example, it seems unlikely that a student who is conscientious becomes a less conscientious doctor. This postulation is supported by findings from behavioural psychology, which state that people tend to become even more conscientious as they age [61]. Conscientiousness is directly linked to NTS [30]; hence, our results become more generalizable. Furthermore, even philosophical views of personality traits conclude that pieces of personality cannot be changed, only the change of the whole identity would affect the expression of personality traits. These identity changes would only take place under special circumstances and are not the normative pattern [61,62].
A further possible limitation of our study should be acknowledged. We restricted our analytic samples as they were preselected. At Hamburg Medical School, the first 100 places of the volatile 60% quota were assigned to those applicants who had received the best PEA in combination with the best result of the HAM-Nat. The following about 100 places were assigned to applicants who also took part in the selection interviews (HAM-Int). For example, applicants with best PEA in combination with best HAM-Nat results, did not participate in the HAM-Int. While our results are likely to be generalizable to the admission tests, we cannot rule out different results if every applicant would have had participated in all admission tests.
One might argue that the comparison of AS-NTS performance of medical students in different years of medical school is questionable. However, this argument can be mitigated, because we compared the cohorts of medical students in each year of their training. Furthermore, the assessment of NTS with the AS-NTS, as it is described by the authors, allows the comparison of different and repeated measurements at different time points of training [40]. Additionally, the scenarios of the emergency training are designed aligned to the spiral curriculum [38], meaning that with each proceeding training, the amount of necessary technical as well as non-technical skills are greater. Hereby, the different training or the NTS of different students in different years of medical training can be compared. Through this, one important strength of our study is derived: while exploring the predictive validities of different admissions regarding NTS performance, we systematically measured NTS with a validated tool, which has been developed specifically for undergraduate education.

5. Conclusions

Our findings open new perspectives regarding the complex structure and predictive validities of different admissions to medical school regarding NTS performance. However, it remains unexplained how and if admission to medical school can predict NTS performance in clinical emergency settings. Therefore, more research is needed to determine how admission to medical school can predict clinical NTS performance and whether admission tests need to be adapted accordingly.

Author Contributions

P.M.-K. made substantial contributions to conception and design, as well as the acquisition, analysis, and interpretation of the data. She has been involved in drafting and revising the manuscript and given final approval of the version to be published. She agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. W.H. made substantial contributions to conception and design. He has been involved in drafting the manuscript and revising it critically for important intellectual content. He has given final approval of the version to be published. He agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. L.S.-U. made substantial contributions analysis and interpretation of the data. She has been involved in revising the manuscript critically for important intellectual content and given final approval of the version to be published. She agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. C.Z. made substantial contributions to conception and design. He has been involved in drafting the manuscript and revising it critically for important intellectual content. He has given final approval of the version to be published. He agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. S.Z. made substantial contributions to conception and design. He has been involved in drafting the manuscript and revising it critically for important intellectual content. He has given final approval of the version to be published. He agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. T.H. made substantial contributions to conception and design, as well as acquisition, analysis, and interpretation of the data. He drafted the initial manuscript and has been involved in revising the manuscript and given final approval of the version to be published. She agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The local Ethic Committee of Hamburg (Ethikkommission der Ärztekammer Hamburg, Hamburg, Germany) was contacted with a detailed project description. The project was rated as a study with-but not on humans (Declaration of Helsinki, article 6) and therefore, no necessity of deliberation of the project was seen and the project was approved (§ 9 des Hamburgischen Kammergesetzes für Heilberufe).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Most data are presented in the manuscript. Further information can be provided due to reasonable request.

Acknowledgments

We thank all the students for participating in the study. We thank Jens Kubitz who was supportive in the first steps of the study design.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Trapmann, S.; Hell, B.; Weigand, S.; Schuler, H. Die Validität von Schulnoten zur Vorhersage des Studienerfolgs-eine Metaanalyse. Z. Für Pädagogische Psychol. 2007, 21, 11–27. [Google Scholar] [CrossRef] [Green Version]
  2. McManus, I.; Smithers, E.; Partridge, P.; Keeling, A.; Fleming, P.R. A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study. BMJ 2003, 327, 139–142. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Trost, G.; Begabungsforschung, B.I.f.T.-u. Evaluation des Tests für Medizinische Studiengänge (TMS): Synopse der Ergebnisse; Institut für Test- und Begabungsforschung: Bonn, Germany, 1998. [Google Scholar]
  4. Kadmon, G.; Resch, F.; Duelli, R.; Kadmon, M. Der Vorhersagewert der Abiturdurchschnittsnote und die Prognose der unterschiedlichen Zulassungsquoten für Studienleistung und-kontinuität im Studiengang Humanmedizin-eine Längsschnittanalyse. GMS Z. Für Med. Ausbild. 2014, 31, page. [Google Scholar]
  5. Ferguson, E.; James, D.; Madeley, L. Factors associated with success in medical school: Systematic review of the literature. BMJ 2002, 324, 952–957. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Patterson, F.; Knight, A.; Dowell, J.; Nicholson, S.; Cousans, F.; Cleland, J. How effective are selection methods in medical education? A systematic review. Med. Educ. 2016, 50, 36–60. [Google Scholar] [CrossRef] [Green Version]
  7. Rindermann, H.; Oubaid, V. Auswahl von Studienanfängern durch Universitäten-Kriterien, Verfahren und Prognostizierbarkeit des Studienerfolgs. Z. Für Differ. Und Diagn. Psychol. 1999, 20, 172–191. [Google Scholar] [CrossRef]
  8. Schwibbe, A.; Lackamp, J.; Knorr, M.; Hissbach, J.; Kadmon, M.; Hampe, W. Medizinstudierendenauswahl in Deutschland. Bundesgesundheitsblatt-Gesundh.-Gesundh. 2018, 61, 178–186. [Google Scholar] [CrossRef] [Green Version]
  9. Ferguson, E.; Sanders, A.; O’Hehir, F.; James, D. Predictive validity of personal statements and the role of the five-factor model of personality in relation to medical training. J. Occup. Organ. Psychol. 2000, 73, 321–344. [Google Scholar] [CrossRef]
  10. Werwick, K.; Winkler-Stuck, K.; Hampe, W.; Albrecht, P.; Robra, B.-P. Introduction of the HAM-Nat examination–applicants and students admitted to the Medical Faculty in 2012–2014. GMS Z. Für Med. Ausbild. 2015, 32, page. [Google Scholar]
  11. McManus, I.; Dewberry, C.; Nicholson, S.; Dowell, J.S. The UKCAT-12 study: Educational attainment, aptitude test performance, demographic and socio-economic contextual factors as predictors of first year outcome in a cross-sectional collaborative study of 12 UK medical schools. BMC Med. 2013, 11, 1–25. [Google Scholar] [CrossRef] [Green Version]
  12. Emery, J.L.; Bell, J.F.; Vidal Rodeiro, C.L. The BioMedical admissions test for medical student selection: Issues of fairness and bias. Med. Teach. 2011, 33, 62–71. [Google Scholar] [CrossRef] [PubMed]
  13. McGaghie, W.C. Assessing readiness for medical education: Evolution of the medical college admission test. Jama 2002, 288, 1085–1090. [Google Scholar] [CrossRef] [PubMed]
  14. Donnon, T.; Paolucci, E.O.; Violato, C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Acad. Med. 2007, 82, 100–106. [Google Scholar] [CrossRef] [PubMed]
  15. Hissbach, J.; Feddersen, L.; Sehner, S.; Hampe, W. Eignung von HAM-Nat und TMS-Modul „Medizinisch-naturwissenschaftliches Grundverständnis “für die Studienbewerberauswahl in der Medizin. GMSZMedAusbild29: Doc72 2012. [Google Scholar]
  16. Eva, K.W.; Reiter, H.I.; Rosenfeld, J.; Norman, G.R. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad. Med. 2004, 79, S40–S42. [Google Scholar] [CrossRef] [Green Version]
  17. Patterson, F.; Zibarras, L.; Ashworth, V. Situational judgement tests in medical education and training: Research, theory and practice: AMEE Guide No. 100. Med. Teach. 2016, 38, 3–17. [Google Scholar] [CrossRef]
  18. Knorr, M.; Hissbach, J. Multiple mini-interviews: Same concept, different approaches. Med. Educ. 2014, 48, 1157–1175. [Google Scholar] [CrossRef]
  19. Rees, E.L.; Hawarden, A.W.; Dent, G.; Hays, R.; Bates, J.; Hassell, A.B. Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Med. Teach. 2016, 38, 443–455. [Google Scholar] [CrossRef] [Green Version]
  20. Pau, A.; Jeevaratnam, K.; Chen, Y.S.; Fall, A.A.; Khoo, C.; Nadarajah, V.D. The multiple mini-interview (MMI) for student selection in health professions training–a systematic review. Med. Teach. 2013, 35, 1027–1041. [Google Scholar] [CrossRef]
  21. Paton, L.W.; McManus, I.C.; Cheung, K.Y.F.; Smith, D.T.; Tiffin, P.A. Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study. BMJ Open 2022, 12, e056129. [Google Scholar] [CrossRef]
  22. Ellis, R.; Brennan, P.; Scrimgeour, D.S.; Lee, A.J.; Cleland, J. Performance at medical school selection correlates with success in part A of the intercollegiate membership of the Royal College of surgeons (MRCS) examination. Postgrad. Med. J. 2022, 98, e19. [Google Scholar] [CrossRef] [PubMed]
  23. Greif, R.; Bhanji, F.; Bigham, B.L.; Bray, J.; Breckwoldt, J.; Cheng, A.; Duff, J.P.; Gilfoyle, E.; Hsieh, M.-J.; Iwami, T. Education, Implementation, and Teams: 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations. Circulation 2020, 142 (Suppl. 1), S222–S283. [Google Scholar] [CrossRef]
  24. Kiesewetter, J.; Gutmann, J.; Drossard, S.; Salas, D.G.; Prodinger, W.; Mc Dermott, F.; Urban, B.; Staender, S.; Baschnegger, H.; Hoffmann, G. The Learning Objective Catalogue for Patient Safety in Undergraduate Medical Education—A Position Statement of the Committee for Patient Safety and Error Management of the German Association for Medical Education. GMS J. Med. Educ. 2016, 33, page. [Google Scholar]
  25. World Health Organization. Patient Safety: Making Health Care Safer; World Health Organization: Geneva, Switzerland, 2017.
  26. Flin, R.; Patey, R. Improving patient safety through training in non-technical skills. Br. Med. J. Publ. Group 2009, 339, b3595. [Google Scholar] [CrossRef] [PubMed]
  27. Greif, R.; Lockey, A.; Breckwoldt, J.; Carmona, F.; Conaghan, P.; Kuzovlev, A.; Pflanzl-Knizacek, L.; Sari, F.; Shammet, S.; Scapigliati, A. European Resuscitation Council Guidelines 2021: Education for resuscitation. Resuscitation 2021, 161, 388–407. [Google Scholar] [CrossRef] [PubMed]
  28. Gordon, M.; Darbyshire, D.; Baker, P. Non-technical skills training to enhance patient safety: A systematic review. Med. Educ. 2012, 46, 1042–1054. [Google Scholar] [CrossRef] [PubMed]
  29. Gawronski, O.; Thekkan, K.R.; Genna, C.; Egman, S.; Sansone, V.; Erba, I.; Vittori, A.; Varano, C.; Dall’Oglio, I.; Tiozzo, E. Instruments to evaluate non-technical skills during high fidelity simulation: A systematic review. Front. Med. 2022, 9, 986296. [Google Scholar] [CrossRef]
  30. Flin, R.H.; O’Connor, P.; Crichton, M. Safety at the Sharp End: A Guide to Non-Technical Skills; Ashgate Publishing, Ltd.: Surrey, UK, 2008. [Google Scholar]
  31. Weinger, M.B. Experience≠ ExpertiseCan Simulation Be Used to Tell the Difference? J. Am. Soc. Anesthesiol. 2007, 107, 691–694. [Google Scholar] [CrossRef] [Green Version]
  32. WHO. WHO Patient Safety Curriculum Guide for Medical Schools. 2009. Available online: https://www.who.int/patientsafety/activities/technical/who_ps_curriculum.pdf (accessed on 20 January 2019).
  33. Frank, J.R.; Danoff, D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med. Teach. 2007, 29, 642–647. [Google Scholar] [CrossRef]
  34. Gaba, D.M.; Howard, S.K.; Fish, K.J.; Smith, B.E.; Sowb, Y.A. Simulation-based training in anesthesia crisis resource management (ACRM): A decade of experience. Simul. Gaming 2001, 32, 175–193. [Google Scholar] [CrossRef] [Green Version]
  35. Stewart, G.L.; Barrick, M.R. Team structure and performance: Assessing the mediating role of intrateam process and the moderating role of task type. Acad. Manag. J. 2000, 43, 135–148. [Google Scholar] [CrossRef]
  36. Schulte-Uentrop, L.; Cronje, J.S.; Zöllner, C.; Kubitz, J.C.; Sehner, S.; Moll-Khosrawi, P. Correlation of medical students’ situational motivation and performance of non-technical skills during simulation-based emergency training. BMC Med. Educ. 2020, 20, 1–7. [Google Scholar] [CrossRef] [PubMed]
  37. Niessen, A.S.M.; Meijer, R.R. Selection of medical students on the basis of non-academic skills: Is it worth the trouble? Clin. Med. 2016, 16, 339. [Google Scholar] [CrossRef] [Green Version]
  38. Harden, R.M. What is a spiral curriculum? Med. Teach. 1999, 21, 141–143. [Google Scholar] [CrossRef] [PubMed]
  39. Sweller, J. Cognitive load theory. In Psychology of Learning and Motivation; Elsevier: Amsterdam, The Netherland, 2011; Volume 55, pp. 37–76. [Google Scholar]
  40. Moll-Khosrawi, P.; Kamphausen, A.; Hampe, W.; Schulte-Uentrop, L.; Zimmermann, S.; Kubitz, J.C. Anaesthesiology students’ Non-Technical skills: Development and evaluation of a behavioural marker system for students (AS-NTS). BMC Med. Educ. 2019, 19, 205. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Team, R. Core. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Team, R. Core: Vienna, Austria, 2013; Available online: http://www.R-project.org/ (accessed on 20 January 2019).
  42. Werwick, K.; Winkler-Stuck, K.; Robra, B.-P. From HAM-Nat to the “Physikum”—Analysis of the study success parameters before and after the introduction of a science test in the approval procedure. GMS J. Med. Educ. 2018, 35, page. [Google Scholar]
  43. Tiffin, P.A.; Finn, G.M.; McLachlan, J.C. Evaluating professionalism in medical undergraduates using selected response questions: Findings from an item response modelling study. BMC Med. Educ. 2011, 11, 1–9. [Google Scholar] [CrossRef] [Green Version]
  44. Conrad, N.; Patry, M.W. Conscientiousness and academic performance: A mediational analysis. Int. J. Scholarsh. Teach. Learn. 2012, 6, n1. [Google Scholar] [CrossRef] [Green Version]
  45. Patterson, F.; Tiffin, P.A.; Lopes, S.; Zibarras, L. Unpacking the dark variance of differential attainment on examinations in overseas graduates. Med. Educ. 2018, 52, 736–746. [Google Scholar] [CrossRef] [Green Version]
  46. Burkhardt, J.C.; Stansfield, R.B.; Vohra, T.; Losman, E.; Turner-Lawrence, D.; Hopson, L.R. Prognostic value of the multiple mini-interview for emergency medicine residency performance. J. Emerg. Med. 2015, 49, 196–202. [Google Scholar] [CrossRef]
  47. MacCann, C.; Joseph, D.L.; Newman, D.A.; Roberts, R.D. Emotional intelligence is a second-stratum factor of intelligence: Evidence from hierarchical and bifactor models. Emotion 2014, 14, 358. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  48. Knorr, M.; Schwibbe, A.; Ehrhardt, M.; Lackamp, J.; Zimmermann, S.; Hampe, W. Validity evidence for the Hamburg multiple mini-interview. BMC Med. Educ. 2018, 18, 1–12. [Google Scholar] [CrossRef] [Green Version]
  49. Knorr, M.; Meyer, H.; Sehner, S.; Hampe, W.; Zimmermann, S. Exploring sociodemographic subgroup differences in multiple mini-interview (MMI) performance based on MMI station type and the implications for the predictive fairness of the Hamburg MMI. BMC Med. Educ. 2019, 19, 1–12. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  50. Baker, D.P.; Day, R.; Salas, E. Teamwork as an essential component of high-reliability organizations. Health Serv. Res. 2006, 41, 1576–1598. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Flowerdew, L.; Brown, R.; Vincent, C.; Woloshynowych, M. Development and validation of a tool to assess emergency physicians’ nontechnical skills. Ann. Emerg. Med. 2012, 59, 376–385.e374. [Google Scholar] [CrossRef] [PubMed]
  52. Albanese, M.A.; Snow, M.H.; Skochelak, S.E.; Huggett, K.N.; Farrell, P.M. Assessing personal qualities in medical school admissions. Acad. Med. 2003, 78, 313–321. [Google Scholar] [CrossRef] [Green Version]
  53. Ekehammar, B. Interactionism in personality from a historical perspective. Psychol. Bull. 1974, 81, 1026. [Google Scholar] [CrossRef]
  54. Lievens, F.; De Koster, L.; Schollaert, E. Current theory and practice of assessment centers: The importance of trait activation. In Oxford Handbook of Personnel Psychology; Oxford University Press: Oxford, UK, 2008. [Google Scholar]
  55. Demange, P.A.; Malanchini, M.; Mallard, T.T.; Biroli, P.; Cox, S.R.; Grotzinger, A.D.; Tucker-Drob, E.M.; Abdellaoui, A.; Arseneault, L.; Van Bergen, E. Investigating the genetic architecture of noncognitive skills using GWAS-by-subtraction. Nat. Genet. 2021, 53, 35–44. [Google Scholar] [CrossRef]
  56. Jerant, A.; Griffin, E.; Rainwater, J.; Henderson, M.; Sousa, F.; Bertakis, K.D.; Fenton, J.J.; Franks, P. Does applicant personality influence multiple mini-interview performance and medical school acceptance offers? Acad. Med. 2012, 87, 1250–1259. [Google Scholar] [CrossRef]
  57. Salas, E.E.; Fiore, S.M. Team Cognition: Understanding the Factors that Drive Process and Performance; American Psychological Association: Washington, DC, USA, 2004. [Google Scholar]
  58. Salas, E.B.C.; Bowers, C.A. Team training in the skies: Does crew resource management (CRM) training work? Hum. Factors 2001, 43, 641–674. [Google Scholar] [CrossRef]
  59. Flin, R.; Martin, L.; Goeters, K.-M.; Hormann, H.; Amalberti, R.; Valot, C.; Nijhuis, H. Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Hum. Factors Aerosp. Saf. 2003, 3, 97–120. [Google Scholar]
  60. Flin, R. Safe in Their Hands?: Licensing and Competence Assurance for Safety-Critical Roles in High Risk Industries; Department of Health: London, UK, 2005; p. 38.
  61. Hudson, N.W.; Fraley, R.C. Volitional personality trait change: Can people choose to change their personality traits? J. Personal. Soc. Psychol. 2015, 109, 490. [Google Scholar] [CrossRef] [PubMed]
  62. McCrae, R.R. Seeking a Philosophical Basis for Trait Psychology. Available online: https://pubmed.ncbi.nlm.nih.gov/36269570/ (accessed on 21 October 2022).
Healthcare 11 00046 g001 550
Figure 1. NTS performance of the investigated cohorts for each training session and each dimension of the AS-NTS. Abbreviations. MMIs = Multiple Mini Interviews; AS-NTS = Anaesthesiology students’ non-technical skills. ACLS = Advanced Cardiac Life Support. OR = Operating room. D = Dimension of AS-NTS. D1: Planning tasks, prioritising and problem-solving; D2: Teamwork and leadership; D3: Team orientation. Note: A low numeric AS-NTS score complies with a better NTS performance.
Figure 1. NTS performance of the investigated cohorts for each training session and each dimension of the AS-NTS. Abbreviations. MMIs = Multiple Mini Interviews; AS-NTS = Anaesthesiology students’ non-technical skills. ACLS = Advanced Cardiac Life Support. OR = Operating room. D = Dimension of AS-NTS. D1: Planning tasks, prioritising and problem-solving; D2: Teamwork and leadership; D3: Team orientation. Note: A low numeric AS-NTS score complies with a better NTS performance.
Healthcare 11 00046 g001
Table
Table 1. Demographic data of study participants.
Table 1. Demographic data of study participants.
AdmissionNumber of Students Enrolled (n = 674)Mean Age
in Years		Standard Deviation of Mean AgePercentage (%) of Female Students
All admissions67426.683.9649
Excellent PEA9324.652.0859
Ham-NAT20725.042.1040
Ham-INT15325.102.5369
Waiting quota10832.783.5557
Other quota7228.103.2353
Designated by armed forces2224.902.6544
Non-EU
Students		1926.272.6822
Note: Demographic data of n = 55 students was not available. These students were not included in the final analysis.
Table
Table 2. Overall NTS performance of all students.
Table 2. Overall NTS performance of all students.
AS-NTS ScoreACLS I
MV (SD)		ACLS II
MV (SD)		OR-SIM
MV (SD)		ACLS III
MV (SD)
Sum_score of AS-NTS2.18 (0.80)2.16 (0.83)1.84 (0.68)2.01 (0.71)
Dimension one
Planning tasks, prioritising and problem-solving		2.18 (0.91)2.23 (0.93)1.93 (0.75)2.04 (0.85)
Dimension two
Teamwork and leadership		2.19 (0.88)2.16 (0.92)1.79 (0.78)2.03 (0.82)
Dimension three
Team orientation		2.18 (0.91)2.08 (0.91)1.81 (0.79)1.97 (0.79)
Note: Abbreviations. AS-NTS = Anaesthesiology students’ non-technical skills. ACLS = Advanced Cardiac Life Support. OR = Operating room.
Table
Table 3. NTS performance of the admission cohorts for each training session and each dimension of the AS-NTS.
Table 3. NTS performance of the admission cohorts for each training session and each dimension of the AS-NTS.
AdmissionACLS IACLS IIOR-SIMACLS III
D1D2D3D1D2D3D1D2D3D1D2D3
Excellent PEA2.282.282.282.072.142.021.801.701.602.102.011.95
Ham-NAT2.192.172.192.302.212.212.031.921.862.052.011.98
Ham-INT2.232.232.182.362.211.951.691.621.922.041.961.93
Waiting quota1.861.952.001.751.731.751.731.691.651.861.931.79
Other quota2.872.532.602.362.142.062.222.002.002.112.181.87
Designated by the Armed Forces1.751.751.832.101.901.702.501.502.501.531.671.87
Non-EU Students2.672.671.672.893.113.222.002.002.502.382.382.23
Note. Abbreviations. AS-NTS = Anaesthesiology students’ non-technical skills. ACLS = Advanced Cardiac Life Support. OR = Operating room. D = Dimension of AS-NTS. D1: Planning tasks, prioritising and problem-solving; D2: Teamwork and leadership; D3: Team orientation.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Moll-Khosrawi, P.; Hampe, W.; Schulte-Uentrop, L.; Zöllner, C.; Zimmermann, S.; Huelmann, T. Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings? Healthcare 2023, 11, 46. https://doi.org/10.3390/healthcare11010046

AMA Style

Moll-Khosrawi P, Hampe W, Schulte-Uentrop L, Zöllner C, Zimmermann S, Huelmann T. Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings? Healthcare. 2023; 11(1):46. https://doi.org/10.3390/healthcare11010046

Chicago/Turabian Style

Moll-Khosrawi, Parisa, Wolfgang Hampe, Leonie Schulte-Uentrop, Christian Zöllner, Stefan Zimmermann, and Thorben Huelmann. 2023. "Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings?" Healthcare 11, no. 1: 46. https://doi.org/10.3390/healthcare11010046

APA Style

Moll-Khosrawi, P., Hampe, W., Schulte-Uentrop, L., Zöllner, C., Zimmermann, S., & Huelmann, T. (2023). Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings? Healthcare, 11(1), 46. https://doi.org/10.3390/healthcare11010046

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop