An Exploratory Study of Small-Group Learning Interactions in Pre-Clerkship Medical Education: Uncovering a Mismatch Between Student Perceptions and Real-Time Observations

Abstract

Small-group learning is a mainstay of medical education, and group functioning can have a major influence on these learning experiences. Our objective was to explore verbal exchange patterns within small-group learning sessions and examine how different patterns related to tutor involvement, tutor expertise, and participants’ perceptions. A non-participant observer collected group interactivity data using a real-time mobile device-based system. Verbal interaction patterns were visualized and analyzed using social network analysis and correlated with participant survey data and aggregate course grades. There were 46 observations across 30 separate groups. Group interactions clustered into four patterns defined by (1) tutor involvement (high vs. low) and (2) interactivity (high vs. low). Interaction patterns were largely stable for a given group and groups with content expert facilitators were generally less interactive. Students reported objectively fewer interactive groups as more interactive and enjoyable. There were no significant intergroup differences in aggregate course grades. Paradoxically, student perceptions were not aligned with observed interactivity data, and tutor content expertise influenced group interactivity. These findings suggest the need to better manage learner expectations of small-group learning, and to explicitly reflect on and develop skills for effective collaborative learning with both faculty and students.

1. Introduction

Over the past four decades, there has been an increasing shift in medical education away from lecture-based teaching towards more student-centred learning strategies that focus on critical thinking and complex problem-solving. These more active learning strategies have been shown to result in enhanced knowledge retention and skills application [1,2,3,4].

Small-group problem-based learning (PBL) is one example of an active learning strategy where inquiry-based interactive sessions are designed to engage students in critical thinking and complex problem-solving in preparation for professional settings [5,6,7]. One mechanism by which PBL supports the development of a range of knowledge and skills essential for professional practice can be conceptualized through the lens of Bandura’s social learning theory. This theory identifies four key elements essential for learning: (1) experience; (2) modelling; (3) social persuasion; and (4) physiological factors [8]. In keeping with social learning theory, the emphasis on group interaction in PBL requires students to draw on existing knowledge and prior experiences and relate them to new cases and information. Crucially, the interactive nature of these sessions allows peers and facilitators to practice and model clinical reasoning strategies and obtain feedback on their approaches to clinical cases. Interactive small groups capitalize on the discussion to explore cognitive conflict within the group, adding an element of social persuasion to achieve conceptual change [5]. This type of teaching strategy also has the added benefit of supporting the development of collaborative teamwork skills [9,10,11,12,13].

While there are many factors involved in successful PBL, dialogue, and interaction among students and faculty have been highlighted as key contributors to the learning process, and cooperative group interactions have been shown to positively influence students’ learning [14,15,16]. Engagement of individuals within the group is a key factor: several studies of group interactions in PBL have noted variations in student participation and highlighted the importance of individual engagement to both group functioning and effective learning [17,18,19,20]. Effective facilitation of PBL sessions requires that the tutor foster a non-threatening atmosphere and promote group interaction and clinical reasoning [18,21,22]. There is debate about the degree to which tutors should provide specific information and guidance versus attending primarily to the process and leaving the content for the students to work through among themselves [23,24]. Focus groups of undergraduate medical students have found that too much guidance from the tutor is silencing and obstructs the integration of knowledge; too little, on the other hand, may create uncertainty about the accuracy of the information discussed and the direction the discussion should take [25]. Furthermore, there is ongoing controversy about the relative merits of expert versus non-expert tutors [26]. Some authors have argued that in-depth tutor content expertise is essential to ensure that students identify and achieve the appropriate learning objectives [23,27,28]. Others have argued that content experts stifle the student-directed nature of PBL activities and that expertise in the PBL process is preferable [29]. Regardless, less is known about how tutor expertise may influence group interactivity and whether it supports or detracts from learning in the small-group setting [24,25].

Although there is no known “ideal” pattern of interaction for small-group PBL sessions and no universally accepted strategy for studying interactivity in small-group collaborative learning, several authors have used focus groups to explore the features that support effective learning experiences [21,30,31]. However, the self-reported nature of these types of data calls into question their ability to see the full picture and the required iterative qualitative analysis in these designs necessarily limits the extent to which data can be collected, analyzed, and fed back to participants in a timely fashion. Others have used time and resource-intensive video recording and coding analysis [18,32], which have provided valuable insights into the types of interactions that occur but are again impractical for providing tutors and groups with timely feedback to facilitate reflection and process improvement. In contrast, the collection of real-time data on verbal interactions and using methods of social network analysis to generate visual representations of group interactions has been successfully used in other small-group activities in medical education [33,34]. The insights obtained from the examination of these patterns of interaction and how they may be influenced by factors such as tutor expertise could be useful both to facilitate student reflection on the group process as well as to inform future faculty development initiatives.

This study aimed to use real-time observations of PBL to explore patterns of verbal interaction in small groups, and to examine the relationships between interactivity, tutor involvement, tutor content expertise, and participants’ perceptions of the learning experience. While the primary purpose was to explore patterns of group interaction, we also explored whether there were links between these factors and student performance on course-based exams.

2. Materials and Methods

2.1. Context

This exploratory study was conducted at a Canadian medical school with a curricular structure that includes both lecture-based and small-group instruction. For the vast majority of students, entry into this undergraduate medical MD degree program follows the completion of an initial bachelor’s degree. The program includes two initial pre-clerkship years with predominantly classroom-based learning, followed by two years of clinical clerkship training. In the pre-clerkship years, there are six system-based courses in each of years one and two; all of these courses include an element of PBL. Each PBL cycle typically consists of three small-group sessions within one week, with sessions one and two lasting two hours and session three lasting one hour. In each course, PBL is used to teach core topics within the subject area; for example, in the musculoskeletal course in year two, PBL topics include approach to polyarthritis, approach to monoarthritis, approach to a limping child, approach to back pain, approach to multi-trauma, etc. The class is typically divided into fifteen PBL groups of ten to twelve students each. Students and facilitators are assigned to groups by the course coordinator at the start of the course and all students were required to participate in the PBL component of each course. Group composition is consistent throughout a course, but groups are different in each course.

Data were obtained from a convenience sample of PBL groups over two academic years to obtain a broad sample that included students in various courses in both the first and second years. A subset of groups was observed on multiple occasions to determine whether patterns of interactions within a group were stable across sessions.

2.2. Data Collection

Data were collected from three different sources; (1) direct observations of PBL sessions by a non-participant observer, (2) written questionnaires completed by tutors and medical students, and (3) final course marks of students in observed PBL groups.

2.3. Direct Observations

Observations of each group were conducted during the second session of the PBL cycle, which was typically devoted to the discussion of independent research on learning objectives, followed by the presentation of new case material and identification of further learning objectives. The decision to observe the second sessions of the week was deliberate as these sessions were where students reconvened with their completed individual research for discussion, debate, and integration into the existing and new case materials; as such, we anticipated that there would be more significant student interaction during this session. Further, we expected that groups would be somewhat more familiar with each other and with PBL expectations than they might be in the initial sessions of the week, ensuring that findings would be less influenced by time spent on introductions and orientation to tasks. The unit of data collection was defined as the utterance, an element of conversation spoken by a single individual [33]. For this study, we defined interactivity as the frequency and sequence of verbal interactions between group members. Data were collected by one of two trained non-participant observers (one PI and one paid research assistant) using a handheld device running customized database forms developed with HanDBase v.4.8 (DDH Software, Wellington, FL, USA). This method allowed the observer to record the identity of a speaker in real-time with automatic time stamping to enable the calculation of the duration of each utterance [33,34]. Interobserver reliability was determined by calculating the kappa value for agreement on speaker identity for data collected simultaneously by both observers during one observed session.

2.4. Survey

At the time of the study, there were no validated instruments to measure students’ perceptions of their small-group PBL learning, so a novel nine-item, self-administered questionnaire (see Appendix A) was developed to assess participant perceptions of the small-group experience. The questionnaires were piloted by a group of medical educators familiar with this context prior to implementation. A paper questionnaire was used because of students’ familiarity with completing paper-based forms for other activities such as session evaluations. The questionnaire was distributed to students and tutors immediately following each observed session. Participants were asked to report their perspectives on the overall learning experience including group function and interactivity on a 5-point scale. In addition, tutors were asked about their perceptions of group interactivity and their self-reported expertise. To mitigate the risk of students being reluctant to rate their experiences honestly, these forms were returned to the independent observer rather than the tutor.

2.5. Course Marks

Final course marks based on examinations, quizzes, and assignments were obtained for all participating students by group. The marks were provided in aggregate, de-identified form to preserve students’ anonymity. To account for variability in mean final grades between different courses, grades for each group were normalized by dividing by the average overall class mark for that course.

2.6. Data Analysis

Descriptive statistics (midpoint and variance) were calculated for session duration, number of utterances, and proportion of session tutors spent talking, based on total utterance duration. Adjustments for repeated measures were performed when tutors and groups were observed on more than one occasion. We report mean and standard deviation (SD) where data were normally distributed. We report mode and range where standard comparative statistics (Student’s t-test, Mann–Whitney U, and Fisher exact test) were used to determine whether there were differences in session duration, the proportion of time the tutor spent talking, social network analysis (SNA) measures of group interactivity, or survey results related to tutor content expertise, course, or year. For these tests, we considered p values ≤ 0.05 to be statistically significant.

Data from group observations were analyzed using SNA techniques, which aim to describe and measure relationships between individuals. This approach allows for visualization and statistical analysis of patterns of group interactions, as well as the involvement of group members, and facilitates comparisons between different groups [35]. Analyses were conducted using an SNA software package (UCINET 6 for Windows, Analytic Technologies, Harvard MA, USA) and the associated NetDraw utility to present a visual representation of the relative contributions of each group member reflecting the frequency of utterances (depicted by line thickness) and the sequence of exchanges (reflected by connections) during each session [36].

Tutor involvement was quantified by calculating the normalized flow betweenness (NFB) for the frequency of tutor utterances for each session. This value expresses the amount of information that flows through an individual relative to information flow that does not. A higher NFB denotes a more involved individual, while a lower NFB denotes a less involved individual [35].

Overall group interactivity based on the frequency of utterances was calculated using the Freeman normalized centrality index (NCI) for each session. This index measures the degree of variation in the involvement of group members. It is expressed as a percentage of the score of a hypothetical group where all information passes through a single individual. Higher NCI values indicate less interactivity; lower NCI values indicate that interactions are more distributed amongst group members, and the group is more interactive [35].

Questionnaire data were analyzed by calculating mean Likert scores for each question. Correlations between observational data, student grades, and survey responses were calculated using the Pearson correlation statistic for normally distributed data and the Spearman rank statistic for non-parametric data.

2.7. Consent and Ethics

All participants were informed of the presence and role of the observer and the nature of the data to be collected, and signed consent was obtained prior to data collection. All members of the group had to consent to participate for the observer to join the group. Questionnaire completion was voluntary and anonymous. This study was reviewed and approved by the University of Alberta Research Ethics Board (no. Pro00010740).

3. Results

3.1. Demographics

Data from 46 of the 49 observed sessions were included in the final analysis. All 49 groups that were approached agreed to participate. Two sessions were excluded as these were scheduled as PBL sessions; however, once the observations were underway, it became clear that the time was being used for a different purpose that had not been updated on the schedule. One session was excluded due to loss of data after a technical failure. The final dataset included observations of 30 different groups: 11 from the first year and 19 from the second year. A total of 27 different tutors were observed. The mean number of students present for observed sessions was 10.3 per group (range 8–11, SD 0.85). We did not collect identifying demographic data from individual students and, thus, cannot report the number of distinct students observed, as some may have been in more than one observed group over the study period.

3.2. Observations

The average observed session length was 85.5 min (SD 20). There was an average of 538 utterances per session (SD 156), and the mean duration of each utterance was 10 s (SD 2). There were no statistically significant differences between first- and second-year groups or between different courses with respect to any of these variables, so these data were combined for subsequent analyses.

3.3. Reliability of Data Collection Instrument

In order to determine the inter-observer reliability of the data collection instrument and coding scheme, one session was attended by two observers who independently coded each utterance. When tested for agreement on who was speaking for all recorded interactions (n = 831), the kappa value was 0.60 (SE 0.02, 95% CI 0.56–0.63), indicating good reliability. Furthermore, there was less than a 5% difference in any measures of tutor involvement and group interactivity between datasets collected by the two observers.

3.4. Interaction Patterns

Interaction patterns ranged from highly interactive with all group members actively involved in the discussions, to groups where interactions were primarily between a small number of students and the tutor (Figure 1 and Figure 2). The distribution of patterns of interaction did not differ significantly between first- and second-year groups, so all data were combined for subsequent analyses.

There were bimodal distributions of both group interactivity and tutor involvement (Figure 1). Based on these distributions, we categorized groups with NCI of <4.2 as “MORE interactive” and those with NCI > 4.2 as “LESS interactive”. Similarly, groups were dichotomized as either “high tutor involvement” (NFB > 28) or “low tutor involvement” (NFB < 28). The most common group dynamic observed was high interactivity with low tutor involvement, and the least common was high interactivity with high tutor involvement (Figure 2). Groups with highly involved tutors (n = 9) were significantly less interactive than groups with less involved tutors (Fisher Exact, p = 0.02). This does not seem to be due simply to the presence of a highly involved participant, as there was no association between the presence of a highly involved student (n = 7) and overall group interactivity (Fisher Exact, p = 0.39).

3.5. Group and Tutor Consistency

In order to determine whether the groups had stable patterns of interaction, ten groups were observed more than once during the same course (four groups for two sessions, and six groups for three sessions). Of these groups, 7/10 were classified as “more interactive” during all observations (Figure 2). The remaining three groups had a mix of more and less interactive sessions.

Of the 10 tutors who were observed on multiple occasions, none self-identified as content experts. Eight were consistently classified as “less involved” across all sessions, while two were classified as “less involved” in some sessions, and “more involved” in others. Of groups facilitated by these tutors, seven were consistently more interactive, while three had a mix of more and less interactive sessions.

3.6. Impact of Tutor Expertise

A total of 7 of the 27 participating tutors self-identified as content experts, and 20 as content non-experts. Of the 46 observed sessions, 10 were led by content experts and 36 were led by content non-experts. There were no differences in session duration between groups facilitated by content experts versus non-experts, however content expert tutors spent a greater proportion of each session talking than did non-expert tutors (22.9% versus 15.5%, t = 2.23, p = 0.04). In addition, groups with content-expert facilitators were significantly less interactive than with non-content experts (t = 2.70, p = 0.02). There were no statistically significant relationships between self-reported tutor facilitation experience and observed interactions.

3.7. Student and Tutor Perceptions of Interactions, Group Function, and Enjoyment

Tutor perceptions of group interactivity, overall group function, or enjoyment of the sessions did not correlate with objective measures of group interactivity or tutor involvement. Paradoxically, on post-observation surveys, students perceived less interaction in objectively more interactive groups (r = −0.48, p < 0.01) and also rated these objectively more interactive groups as less enjoyable (r = −0.43, p < 0.05). Regardless of objective interactivity, students rated groups with content expert facilitators as more interactive (t = 2.14, p = 0.05) as well as more enjoyable (t = 2.04, p = 0.05); although the strength of these correlations is low to moderate. There were no other significant relationships found with other items in the survey.

3.8. Exploratory Analysis of Student Final Course Marks

There were no significant correlations between student final course marks and either objective group interactivity or overall tutor involvement. Similarly, whether the PBL tutor was a content expert had no bearing on student marks (t = 0.15, p = 0.43).

4. Discussion

In contrast to the bulk of the PBL literature, which has focused on comparing this learning strategy to traditional lectures [37], we used real-time observations of PBL to explore patterns of verbal interactions and examined how these related to tutor expertise, involvement, and participant perceptions of group function. We observed a range of interaction patterns across groups, ranging from highly interactive with multiple interactions between most group members, to highly facilitator-centric, with most interactions flowing through the tutor. These patterns were generally stable when groups were observed on multiple occasions and seemed to be influenced by tutor content expertise.

In alignment with Bandura’s social learning theory, it would be reasonable to assume that more interactive groups—where participants actively model thinking by drawing on their existing knowledge and collaborate to work through clinical scenarios—would be more effective for student learning. As such, we anticipated that students would recognize the benefits of increased interactivity during PBL sessions. We were surprised to find that students expressed a preference for objectively less interactive groups, while simultaneously perceiving these as more interactive.

Several factors may have contributed to this mismatch between student perceptions of group interactivity and objective observations. It may be that students simply have difficulty accurately assessing their interactions in small-group settings. This hypothesis is supported by a study that used video-recorded observations to examine group dynamics in PBL and also noted marked discrepancies between student self-report of group interactivity and those documented in video-recorded observations [30]. Another possibility is that, as our survey did not provide a specific definition of “interactivity”, and students have likely been socialized that interactivity within small groups is important for learning, they may, therefore, have conflated “interactivity” with positive perceptions of the learning experience. Thus, they may have simply been expressing a preference for a less interactive dynamic. This is consistent with our finding that students preferred objectively fewer interactive sessions.

Despite growing evidence to support the value of active learning strategies, a number of studies have also documented student preferences for more directive learning experiences. A randomized controlled study of students in applied sciences demonstrated that, while students in active learning instructional arms had objectively higher knowledge scores and better retention, learners believed they learned more in passive instructional arms [1]. This negative correlation between objective evidence of learning and students’ perception of learning was found to be related to the increased cognitive effort required during active learning. The importance of effort for effective learning is further supported by Steenhof et. al., who demonstrated better longer-term outcomes for students who struggled with problems compared to those who were given worked examples [38]. These findings align with the literature on desirable difficulty [39]. At the same time, regardless of ultimate learning value, students do not always take kindly to learning strategies that include a planned element of struggle. Tsang and Harris surveyed medical students and faculty regarding their perceptions of the value of different learning strategies and found that students believed that passive strategies, such as lectures, were more effective for their learning, while faculty endorsed active and collaborative learning strategies as more effective [3]. White et al. found students to be reluctant to engage in active learning strategies and suggested that while implementation design issues may have contributed, student developmental factors also played a significant role [40]. Finally, the team-based learning (TBL) literature has demonstrated that while students participating in this active learning strategy had better learning outcomes, their reaction to the experience was often negative [41].

We wondered if one reason for students’ preference for less interactive groups may relate to the tendency of these groups to fall into a ‘mini-lecture’ pattern, with more interactions flowing from and through the tutor. This experience may feel more comfortable than the work of active learning, particularly if the group is struggling to move forward. In addition to a general preference for less cognitively taxing learning, students may have believed that they benefited from the input of an “expert” facilitator who was well-positioned to share their knowledge. A recent focus group study of PBL group interactions supports this hypothesis as they found that students experienced tension between conflicting desires for increased interactivity and academic inquisition versus efficiency in more directive group processes and independent studies [31]. Another focus group study found that students’ expectations of the facilitator role were not aligned with the intended role of facilitating active learning: students wanted to be told when they were “correct” and given a summary lecture after the PBL session [42]. We found that groups with content-expert tutors were more likely to fall into a pattern where the tutor spent more time talking and with more interactions flowing through the tutor. These content expert tutors may have found it difficult to watch the group struggle with concepts that were within their area of expertise and may have found it challenging to resist the temptation to intervene and clear up areas of uncertainty. This is supported by a study of tutor performance in PBL, which demonstrated a correlation between decreased group productivity and the need for tutor input [14]. While it is possible that the groups with increased tutor involvement in our study were struggling with productivity, groups were randomly assigned to tutors, so it is unlikely that there was such a predominance of struggling groups assigned to content expert tutors, which would fully explain the decreased interactivity in these groups. Further, exploratory analysis of students’ final course scores suggests that the average course performance of less interactive groups was no different than more interactive groups. These observations suggest that the differences in interactivity had more to do with tutor factors than student or group characteristics.

Our findings should be considered in light of certain limitations. Our exploration of group interactions was limited to frequency and sequence of verbal interactions, without capturing content or elements of non-verbal communication. Furthermore, any time an observer is present in a group setting there is a risk of Hawthorne effect or participant reactivity. While this may have been a factor, the variety of patterns observed suggests that we were still able to detect differences in group interactions. In addition, for groups observed on more than one occasion, we found no consistent changes in interactivity, which is in keeping with suggestions that observer effects may be less of a concern in HPE [43]. While convenience sampling may have resulted in inadvertent selection bias of the observed groups, there was wide sampling across courses, groups, and tutors. All groups approached agreed to be observed, and there were no differences between the included groups’ final marks and the overall posted course marks. With respect to student performance, we acknowledge that the overall course assessment was only partly aligned with PBL objectives, so the use of final course grades would be unlikely to detect subtle changes due to differences in PBL group interactivity. It is important to note that tutor content expertise was self-reported, and while we feel it is unlikely that tutors would exaggerate their skills, it is possible that some may have underreported their expertise. Finally, this was a study of PBL as it is implemented at a single institution, which may limit the transferability of results.

Our findings raise interesting questions about the influences on interactions in small-group learning, as well as student perceptions of active learning in general. We demonstrated how the use of a simple data collection tool can be used to provide a visual representation of observed interactions in small-group learning and can then be triangulated with other data about group performance. There is a recognized need for better tools to provide students with objective feedback on their contributions to small-group learning [44]. While technical limitations at the time of this study hindered the ability to provide groups with timely interaction data, the recent emergence of sophisticated data-visualization platforms means that it could now be possible to create interaction maps in real-time. When combined with a recently developed validated tool for critical thinking, and group processes such as CLeD-EX, providing students with “interaction dashboards” may support effectively guided reflection on collaborative learning behaviours [45]. Given that efforts to enhance tutor skills in facilitating appropriate group interactions have been successful [18,46,47], the provision of visual feedback on group interactions could be used both for tutor training as well as feedback. Finally, it may also be interesting to explore whether the use of these sorts of tutor performance metrics that de-emphasize student ratings might empower tutors to apply these strategies.

Future studies may consider a deeper exploration of how the content and nature of discussions relate to interactivity and tutor involvement. In addition, while we were able to perform an exploratory analysis of final course scores, we did not have access to detailed assessment breakdowns to measure how group interaction may have affected small-group content-specific knowledge. Further exploration of students’ perceptions of interactivity and group dynamics in small-group learning may enhance our understanding of the noted mismatch between student impressions and objective measures of interactivity.

5. Conclusions

In summary, examining the patterns of interactivity of small groups during active learning sessions has provided interesting insights into factors that may influence group interactions, including the impact of tutor content expertise. We demonstrated the feasibility of a real-time observation tool that could be used to provide groups and individuals with timely feedback. Finally, we identified a key mismatch between pedagogical intentions and student perceptions of small-group active learning. While social learning theory and the educational literature support learning strategies that actively engage and challenge students, our findings suggest that students report that they prefer activities that are objectively more passive and expert-oriented. Acknowledging this disconnect and providing groups with objective observation-based feedback could facilitate data-informed guided reflection and may help direct efforts to enhance faculty development for tutors, as well as inform strategies to better orient students to, and manage expectations of small-group learning.