Next Article in Journal
A Learning Community Involving Collaborative Course-Based Research Experiences for Foundational Chemistry Laboratories
Previous Article in Journal
Learning Processes in Mechanics of Structures: Allying Analytical and Numerical Approaches
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 10
Issue 4
10.3390/educsci10040115
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Meta-Analysis of the Cognitive, Affective, and Interpersonal Outcomes of Flipped Classrooms in Higher Education

by
Hwan Young Jang
1 and
Hye Jeong Kim
2,*
1
Department of Education, College of Education, Dongguk University, Seoul 04620, Korea
2
Department of Education, Graduate School of Education, Chung-Ang University, Seoul 06974, Korea
*
Author to whom correspondence should be addressed.
Educ. Sci. 2020, 10(4), 115; https://doi.org/10.3390/educsci10040115
Submission received: 7 March 2020 / Revised: 30 March 2020 / Accepted: 18 April 2020 / Published: 20 April 2020
(This article belongs to the Section Higher Education)
Browse Figures
Versions Notes

Abstract

:
This paper aims to quantify the effects of flipped classrooms in higher education by reviewing 43 empirical studies of students’ cognitive, affective, and interpersonal outcomes. The innovative pedagogy of a flipped classroom in higher education fosters a sustainable, interactive, and student-centered learning environment (as opposed to the traditional lecture style, in which there is little room for interaction). This study’s results show the positive effects of flipped classrooms and highlight the improvement in students’ educational outcomes between 2012 and 2017. Overall, effect sizes were medium—effect size (ES) = 0.35, 95% confidence interval (CI) = 0.24 to 0.47—across three outcome domains using a random effects model. In the outcomes, affective (ES = 0.59), interpersonal (ES = 0.53), and cognitive (ES = 0.24) domains were of a higher order than the effect sizes. However, the results indicated that flipped classrooms benefitted students studying chemistry, engineering, mathematics, and physics less than they did students studying other subjects.

1. Introduction

The flipped classroom is an innovative instructional model that is gaining popularity in higher education because it provides active and student-centered learning and enhances students’ educational outcomes [1]. Rahman, Mohamed, Aris, and Zaid [2] state that flipped classrooms were initially introduced in college-level technology classes. In the flipped classroom, students study instructional materials before class, typically online lectures, and apply what they learned in in-class activities [3]. Unlike teacher-centered teaching (e.g., the traditional college lecture style), flipped classrooms provide students with engaging, interactive learning experiences in which they can develop complex reasoning, written communication, and critical thinking skills [4].
The needs of students and society often evolve faster than traditional teaching methods. Thus, there is an urgent need to reconstruct college education [5]. An increasing number of stakeholders, including students and instructors, see the traditional, teacher-centered lecture style as obsolete. Consequently, universities are responding by developing, systematizing, and delivering courses and programs in new and innovative ways, which they hope will engage students as well as meet their educational needs and demands. However, transitioning from traditional lecture-based learning to a new classroom model requires a paradigm shift from teacher-centered to student-centered learning [6]. Although some scholars debate about whether the dichotomy of lectures versus active learning is meaningful in today’s higher education classrooms [7,8], this paper assumes that flipped classrooms represent a different instructional model that can complement, rather than replace, traditional approaches to education.

2. Literature Review

2.1. Defining the Flipped Classroom

The flipped classroom model does away with most teacher-centered instructional activities to create time for student-centered learning activities, such as computer-based individual instruction prior to in-class sessions [3,9]. Despite the substantial variation in the definitions of a flipped classroom [10], most flipped classrooms involve students watching instructional videos online and before class, and then participating in related interactive educational activities during the class [10,11]. Some researchers define flipped classrooms as those in which students receive computer-based instruction [12] or read materials before class [3]. Thus, in flipped classrooms, instructors perform various roles: they are curricular designers, instructors, and media developers.
A flipped classroom is also called a “backwards classroom” because teachers sometimes record lectures in advance and upload them online for students to study on their own time. This frees up class time for student-centered educational works, such as inquiry-oriented strategies, problem-based learning, quizzes, or assignments [4]. The essential point is that, within a flipped classroom, active learning activities are expanded. The reciprocal actions that occur during the class distinguish flipped classrooms from traditional models of education [13].
Flipped classrooms are generally understood through the concept of blended learning [14]. Blended learning refers to how instructors and other educational leaders integrate new technologies to adopt and develop problem-based, active learning approaches to engage students [15]. Some researchers have asserted that flipped classrooms not only enhance educational outcomes but also inspire students to excel [11]. Thus, flipped classrooms are being implemented on university campuses [16].
However, there are some barriers to the implementation of flipped classrooms in practice. Professors may be hesitant to adopt new approaches to teaching and learning [6]. Students have diverse learning needs, and some students may want instructors to provide more (or less) challenging material [6]. Other students find it difficult to keep up with the fast pace of traditional, lecture-style learning [17]. Flipped classrooms appear to provide a great solution to the major problem faced by universities: satisfying the educational needs and demands of thousands of students who study and learn at different levels.

2.2. Effects of Flipped Classroom Studies

Previous studies on flipped classrooms have found that this instructional model has positive, negative, and mixed results.

2.2.1. Positive Outcomes of Flipped Classrooms

By measuring students’ cognitive, affective, and interpersonal outcomes, a growing body of studies has found that flipped classrooms have positive effects. Cognitive outcomes are not limited to learning outcomes (e.g., learning performance and test scores); they also refer to educational outcomes in which students develop and acquire meta-cognitive abilities, such as critical thinking skills. Flipped-classroom instruction improves students’ meta-cognition and collaborative learning strategies [18], their domain knowledge and critical thinking skills [19], and their understanding of content [20]. Hsieh, Wu, and Marek [11] note that flipped classrooms enhance student learning performance and improve learning outcomes. Flipped classrooms have also been shown to spark statistical improvements in students’ vocabulary and grammar [21] as well as boost their subject matter test scores [20]. However, several studies have reported that flipped classrooms have a negative effect on students’ cognitive outcomes. Jensen, Kummer, and Godoy [16] reported that introducing a flipped classroom does not improve learning gains, scientific reasoning ability, higher level conceptual knowledge, or understanding any more than traditional learning and teaching styles.
Affective outcomes refer to educational outcomes regarding students’ satisfaction, confidence, motivation, emotions, attitudes, and feelings toward learning, the subject matter itself, or educational activities [22]. Affective outcomes have been shown to strongly influence cognitive outcomes [23]. Flipped classrooms have been shown to improve student motivation [24], student satisfaction [21,25], and confidence [21]. However, some studies have shown that flipped classrooms had a negative impact on students’ satisfaction and attitudes [16,26].
Interpersonal outcomes refer to learning that aims to improve student action and performance, including interaction and engagement (e.g., active learning). Flipped classrooms have been found to improve student–teacher interaction, student engagement, student-to-student interaction, individual education, active learning, and debate competence [6,21,27].

2.2.2. Negative Outcomes of Flipped Classrooms

Not all studies on flipped classrooms report positive results. Some report mixed or negative results. Ryan and Reid [28] demonstrated that low-achieving students in flipped classrooms performed better on exams. However, Jensen, Kummer, and Godoy [16] indicated that flipped classrooms did not improve student performance outcomes regardless of whether students were high achievers or low achievers. Missildine, Fountain, Summers, et al. [26] showed that introducing flipped classrooms improved learning gains but did not improve students’ satisfaction. Lucke [29] indicated that students enjoyed their flipped classes but showed no improvement in cognition and understanding. Vliet, Winnips, and Brouwer [18] pointed out that positive learning gains from flipped classroom environments were only temporary.
Few meta-analyses exist on the effects of flipped classrooms. Further, there is little empirical evidence regarding flipped classrooms’ utility in improving student performance in higher education [30]. This study is the first to examine the effects of flipped classrooms in higher education using a meta-analysis.

3. Research Problem

This study conducts a meta-analysis to explore the effects of flipped classrooms on cognitive, affective, and interpersonal educational outcomes. The meta-analysis synthesizes the effects of flipped classrooms in higher education and attempts to answer the following research questions: (a) what is the overall effect of the flipped classroom approach in the context of higher education? (b) What outcome variables have the most influence on measurable flipped classroom effect size? And (c) are any effects of the flipped classroom approach moderated by studies’ characteristics or variables (e.g., department, subject area, and publication year)?

4. Method

Meta-analysis involves formulating a problem, collecting data, coding data, analysis, and interpretation [31]. This study’s meta-analysis followed the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) guidelines [32].

4.1. Literature Search

This paper examines journal articles and dissertations about flipped classrooms in the context of higher education that were published between 2012 and 2017. The authors searched five electronic databases for empirical articles: The Education Resources Information Center (ERIC), PROQUEST, Web of Science, PsychInfo, and Google Scholar. To capture a range of potential eligible studies, we employed the following search keywords in titles and abstracts: “flipped classroom,” “flipped class,” “flipped learning,” “inverted class,” “inverted classroom,” “smart learning,” and “blended learning.” The authors found forty-three meaningful studies that met the study’s inclusion and exclusion criteria (Figure 1).

4.2. Inclusion and Exclusion Criteria

Studies with the following features met this study’s inclusion and exclusion criteria: they must be quantitative studies on student learning or reasoning processes in flipped classrooms; they must provide sufficient information to calculate effect sizes; they must define the flipped classroom approach as including the use of video or audio materials before class and featuring in-class activities; they must compare flipped classrooms’ effects with those of traditional classrooms; they must feature students in higher education settings; they must have been published between January 2012 and June 2017; and they must be an empirical, peer-reviewed journal article or dissertation.

4.3. Coding Studies

The data were extracted from studies that met the inclusion criteria (Table 1). The studies’ characteristics were coded as possible moderating variables to investigate the variance of flipped classrooms’ effects. Two researchers independently coded each study. We developed a coding manual to maintain reliability of the coding procedures, which included study characteristics, effect size calculation, and report characteristics. Discrepancies between the two coders were resolved prior to data analysis without exception and were resolved by an independent third expert if no agreement could be reached between the two coders.

4.4. Computation of Effect Sizes

The effect size of this meta-analysis includes three different data formats: treatment vs. control group design, pre–post design, and standardized mean change difference (pre–post measure with both treatment and control group), where the pooled estimate of standard deviation was used to consider different sample sizes between flipped and non-flipped classroom groups. All effect sizes were calculated using the Comprehensive Meta-Analysis (CMA) program to estimate a mean effect size [67]. Effect sizes were reported as positive when flipped classroom students performed better than students in the control groups. The effect size was evaluated as follows: 0.20 = small effect, 0.50 = medium effect, and 0.80 = large effect [68].

4.5. Analysis

4.5.1. Combining Effect Sizes

We employed a two-step process to synthesize the effects of flipped classroom outcomes. First, it calculated the effect size and variance of each outcome in the primary study. Second, it calculated the weighted mean effect size (ES) using inverse variance weight. To select its analysis model, the study conducted a homogeneity test using two measures of variability: Q and I2. The Q test examined whether the variability in an average weighted ES exceeds sampling error alone [69]. I2 is an alternative measure of homogeneity, which is less sensitive to sample size than Q. I2 shows whether the proportion of the observed variance reflects differences in true effect sizes [67]. To evaluate I2 statistics, this study followed Higgins and Green’s [70] guidelines: 0% to 40% might not be important; 30% to 60% may represent moderate heterogeneity; 50% to 90% may represent substantial heterogeneity; and 75% to 100% may represent considerable heterogeneity. The null hypothesis of the homogeneity test was that all outcomes came from the same population. If homogeneous, this study used a fixed effects model that had a common effect size and only considered sampling variance. If heterogeneous, this study used a random effects model that had no common effect size and considered sampling variance and true difference between studies [71]. Based on the homogeneity test and investigation of flipped classroom primary studies, this study used random effect models to synthesize the main effects and sub-group analyses.

4.5.2. Publication Bias

Publication bias happens when the results of published studies are different from the results of unpublished studies because studies with positive results, large effects, and large sample sizes are overrepresented in the literature [67,72]. To examine publication bias, this study adopts a funnel plot, exploring symmetrical distributions around the weighted mean effect sizes [73]. Funnel plots are scatter plots of effect sizes from studies in the meta-analysis, where the horizontal axis represents effect sizes and the vertical axis represents standard errors [72]. An asymmetrical pattern in the results of the funnel plot indicates a possible publication bias.

4.5.3. Analyzing Variances in Effect Sizes Across Studies

Finally, this study examined the variances in the effect sizes using sub-group analysis and meta-regression [74]. Meta-analyzers should prove whether the effect sizes are homogenous in order to calculate the overall effect size in a meta-analysis. This study used homogeneity test results to select an analysis model and decide whether reviewers would perform a sub-group analysis. Q-statistics were used to assess the heterogeneous structure of the average effect sizes. When the Q statistic is significant (p < 0.05), it suggests that the studies in the meta-analysis are heterogeneous effects. A random effects model was adopted to calculate the overall effect size in this study. The homogeneity calculation formula is as follows:
Q = ∑ i = 1 k ( g i − g . ) 2 v ( g i ) = ∑ w i ( g i − g . ) 2
where   w i = 1 / v ( g i ) and wi is an inverse variance weight. The Q statistic is used to determine whether the primary results are homogeneous for subgroup analysis. The magnitude of effect sizes interpreted 0.2 as small, 0.5 as medium, and 0.8 as large according to Cohen’s rule of thumb [68].

4.5.4. Dependence

This meta-analysis included a total of 43 studies and 218 effect sizes. When a primary study has more than one effect size, reviewers should explain the assumption of independence because multiple effect sizes have dependence within the study. To maintain the assumption of independence, the reviewers should select only one effect size per study, which will cause information loss. To keep multiple effect sizes within the study, this choice will cause a violation of independence assumption. To avoid this violation, this study adopted the “shifting unit of analysis” method [75]. This method proposes a compromise between the issues of information loss and violation of independence assumptions. To calculate the overall effect size, “study” will be used as an analysis unit to determine the independence assumption. To perform sub-group analysis, the effect size of each sub-group will be used as a unit of analysis.

5. Results

As mentioned earlier, the 43 studies included in the meta-analysis synthesized a total of 218 effect sizes: an average of 5.1 effect sizes per study. As multiple effect sizes existed within studies, the reviewers considered the dependence of effect sizes in each study. Figure 2 shows the study characteristics for all 43 studies, including effect size (i.e., standard difference in means), standard error, variance, confidence interval, Z-value, and p-value in a forest plot. Black squares in the forest plot’s horizontal lines show the effect size of an individual study, and the horizontal lines indicate the confidence interval for each estimate. The small diamond shape at the bottom represents the overall effect size of all studies. According to the forest plot, the smallest effect size value is −0.933, and the highest effect size value is 1.666. Thirty-nine studies had positive effect sizes, while four had negative effect sizes. Consequently, the implementation of flipped classrooms had a significant effect in 39 of the 43 studies.

5.1. Overall Effect of Flipped Classroom (Research Question 1)

Table 2 provides the following statistics: number of studies; Q statistic assessing the heterogeneity of the distributions of the effect sizes; statistical significance of ES; and confidence interval (CI) for each ES, ES, and standard error (SE). The effect sizes were heterogeneous (Q [42] = 1022.9, p < 0.05), I2 = 96.0, among the 43 studies. Thus, all studies in the analysis did not share a common effect size, which means the null hypothesis of the homogeneity test can be rejected. We used the random effects model to estimate the overall effect size and compare sub-group differences using the study characteristics (e.g., outcome variables, report characteristics variables, and study characteristics variables). The results of the homogeneity test show that the effect sizes are heterogeneous (Table 2).
The results of the random effects model analysis are displayed in Table 3. The overall effect size of flipped classrooms was 0.35, indicating that flipped classrooms had a medium effect in terms of the Cohen’s rule of thumb [68]. The effect size showed an overall significant difference in outcomes from flipped classrooms and traditional lecture-based classrooms in higher education (ES = 0.35, 95% CI = 0.24 to 0.47).

5.2. Outcomes of Flipped Classroom (Research Question 2)

This meta-analysis used a random effects model to investigate the differences between sub-groups, as the results from each sub-group were heterogeneous. The categorical variables are as follows: outcome domains (cognitive, affective, and interpersonal), department, subject, data format, and publication status. We conducted a meta-regression analysis using publication year as a covariate. In the random effects categorical analysis by outcome, shown in Table 4, the results of implementing flipped classrooms varied. In the outcomes, the respective effect sizes of affective (ES = 0.59), interpersonal (ES = 0.53), and cognitive (ES = 0.24) domains were in descending order.
In the context of higher education, flipped classrooms appear to have more significant effects on students’ affective and interpersonal outcomes than on their cognitive outcomes. Regarding affective outcomes, students’ immersion (ES = 1.52), motivation (ES = 0.76), attitude (ES = 0.64), independence (ES = 0.57), impression (ES = 0.28), and confidence (ES = 0.25) were in descending order of effect sizes. Regarding interpersonal outcomes, students’ participation (ES = 0.63), interaction (ES = 0.54), and response (ES = 0.32) were in descending order of effect sizes. In the cognitive domain, understanding (ES = 0.60), achievement (ES = 0.59), synthesis (ES = 0.49), analysis (ES = 0.46), meta-cognition (ES = 0.26), application (ES = 0.25), knowledge (ES = 0.15), and evaluation (ES = −0.93) were in descending order of effect sizes.

5.3. Effects of Characteristics (Research Question 3)

Table 5 and Table 6 list the effect sizes measured by this study, separated by department and subject area. The highest proportion of effect sizes (k = 71) applies to students in mixed departments. The effect sizes by department were English, ES = 0.83, Education, ES = 0.58, Animal Science, ES = 0.46, Psychology, ES = 0.44, Medical, ES = 0.37, Pharmaceutical Medicine, ES = 0.29, Management, ES = 0.25, Mixed, ES = 0.21, Business, ES = 0.19, Nursing, ES = 0.13, and Chemistry, ES = 0.11.
This study investigated a variety of subject areas to determine whether the flipped classroom approach is more beneficial in some contexts or subjects than it is in others. The effect sizes by subject were Computer Science, ES = 0.96, Education, ES = 0.80, Psychology, ES = 0.45, Nursing, ES = 0.44, Mixed subject, ES = 0.44, Science, ES = 0.43, Medicine, ES = 0.37, Pharmaceutical Medicine, ES = 0.29, Physiology, ES = 0.22, Management, ES = 0.19, Engineering, ES = 0.17, Physics, ES = 0.15, Math, ES = 0.13, English, ES = 0.12, Chemistry, ES = 0.10, and Business, ES = −0.93. However, the effect sizes of Business, Nursing, and Psychology require careful interpretation because they showed one effect size (k = 1) per subject.
In the primary studies reviewed in this research, the data are generally represented in three different formats: pre–post design, treatment vs. control group design, and pre–post with treatment vs. control group (standardized mean change difference). The effect sizes for each type are as follows: treatment vs. control, ES = 0.25 (95% CI = 0.21 to 0.28), pre–post design, ES = 0.38 (95% CI = 0.35 to 0.42), and standardized mean change difference, ES = 0.47 (95% CI = 0.41 to 0.53). The difference was not small, and study design may factor into this difference in effect sizes. Regarding publication type, the effect size of dissertations (ES = 0.61, 95% CI = 0.54 to 0.68) was larger than the effect size of journal articles (ES = 0.29, 95% CI = 0.26 to 0.31), but the difference was not significant (Table 7).
Regarding year of publication, this study conducted a meta-regression analysis in which the regressing effect sizes of flipped classrooms on year of publication served as a moderator. The slope of the meta-regression by publication year is negative overall, but it is statistically significant (Table 8) and has a significant moderating effect on the relationship between flipped classrooms and a study’s year of publication.

5.4. Publication Bias

The funnel plot (Figure 3) shows the symmetry of effect size distribution in the mean effect size whether publication bias in the overall effect size exists, providing no evidence for publication bias. This meta-analysis shows no missing studies and finds no imputations of effect size for publication bias.

6. Discussion

This study conducted a meta-analysis of the effects of flipped classrooms on students’ cognitive, affective, and interpersonal outcomes in higher education. It extends the discussions and findings from recent meta-analyses that found that flipped classrooms had a significant effect on students’ cognitive outcomes in higher education: for example, by improving their test scores, grade, knowledge, skills, and self-directed learning (e.g., [9,76,77]). This study expands the evidence for flipped classroom effectiveness in improving college students’ academic outcomes as compared to traditional, lecture-based classrooms.
The first research question was regarding the overall effect of flipped classrooms on students’ cognitive, affective, and interpersonal outcomes. The study found that flipped classrooms had a medium effect on academic outcomes; the average scores of students in flipped classrooms were 0.35 standard deviations above the average scores of students in traditional, lecture-based classrooms. It also confirmed the results of previous, related studies (e.g., ES = 0.36 [3]; ES = 0.35 [9]; ES = 0.53 [77]; ES = 0.21 [78]). In short, its findings demonstrate that flipped classrooms can improve college students’ academic outcomes in various ways, could provide an effective way to inculcate essential 21st-century skills in students [79], and may assist students with special educational needs in performing better than they would in traditional, lecture-based classrooms.
The second research question was regarding the outcomes influenced by the introduction of the flipped classroom method. The overall effect sizes of the affective outcomes (ES = 0.59, SE = 0.03, 95% CI = 0.53 to 0.65]), interpersonal outcomes (ES = 0.53, SE = 0.31, CI = 0.47 to 0.59), and cognitive outcomes (ES = 0.24, SE = 0.24, 95% CI = 0.19 to 0.36) were the descending order of the overall effect sizes. This study’s results suggest that flipped classrooms improve college students’ cognitive, affective, and interpersonal outcomes and that flipped classrooms have more significant effects on affective and interpersonal outcomes than on cognitive outcomes. This result can be explained by the features of the flipped classroom that encourage active engagement and learner-centered interactions. Furthermore, this study’s findings indicate that flipped classrooms indirectly affect cognitive outcomes because affective outcomes have a strong influence on cognitive outcomes [23], in part by improving students’ motivation and willingness to learn [80]. However, affective outcomes (e.g., attitudes and satisfaction) in the flipped classroom are not necessarily positive in higher education.
This study’s results regarding the high effect sizes of interpersonal outcomes in flipped classrooms are consistent with the results of Shi, Ma, Macleod, et al. [77]. Further, the results can be explained by the instructors’ tendency to design active in-class activities in flipped classrooms to increase student participation and interaction [61] through discussion, small group activities, feedback, group discussion, collaborative group work, and group projects [81]. These active, in-class activities enhance students’ interpersonal skills and encourage them to become active and self-directed learners who are deeply involved in the learning process [82,83].
This study’s third research question addressed the effects of study characteristics on how the effect sizes of flipped classrooms were measured. To answer this question, the study performed subgroup analyses using subject area, department, publication year, and study design as moderators. These moderators accounted for a small amount of the relatively large levels of heterogeneity between studies. The results indicated that flipped classrooms can be applied in a variety of subject areas and still effectively improve educational outcomes, as discussed in Rahman, Mohamed, Aris, and Zaid [2]. Although instructors’ individual approaches can influence the success of flipped classrooms, this study found that English, Engineering, Math, Physics, and Chemistry classrooms showed small effect sizes. These results are in line with other meta-analyses of flipped classrooms (e.g., [3,78]).
Regarding publication bias and publication type, this study found that the primary literature on flipped classrooms did not indicate publication bias, even though dissertations (ES = 0.61) had a greater effect size than journal articles (ES = 0.29). This study also performed a funnel plot to examine the possibility of publication bias but did not find evidence for publication bias. Thus, publication type can be treated as a moderator in future flipped classroom interventions.

7. Limitations and Future Research Directions

This meta-analysis has several limitations. First, the meta-analysis gains ecological validity by including only quantitative field studies (experimental or quasi-experimental research), which examine whether the study results can be generalized to real-life settings. However, some internal validity relative to more controlled laboratory studies is sacrificed: for example, randomized controlled trials [84]. Second, this meta-analysis includes only quantitative findings despite the fact that there are many flipped classroom studies that employ qualitative research methods [31,85]. Because this study excluded qualitative studies from its analysis, its results should be interpreted with caution. Qualitative findings help researchers arrive at deeper understandings [86] and generate new knowledge [87]. Some studies show that flipped classrooms have been particularly effective among the learner demographic [28] because low achievers require more interaction and motivation to attain good learning outcomes. We recommend and encourage researchers to implement flipped classrooms with various student bodies in a variety of academic settings to better define the degree to which these results are transferrable [16].
The flipped classroom is not a panacea, and its effectiveness depends in large part on whether students actually use the available pre-class time effectively [30]. We therefore propose repeated use of flipped classrooms and related, modified strategies on a trial-and-error basis. Ratta [6] insisted that flipped classroom instruction is congruent with today’s digital-savvy college student; moreover, it is also important to understand the various influences of today’s student culture, study style, study habits, and use of devices. Further study may be warranted to allow more detailed conclusions about student performance to be drawn [88].

8. Conclusions

This study synthesized the results of 43 studies regarding the effects of flipped classrooms on students’ cognitive, affective, and interpersonal outcomes in higher education. It examined the overall effect sizes of flipped classrooms compared to traditional, lecture-based classrooms and found that flipped classrooms had a medium effect on various student learning outcomes. Particularly, the study identified that the flipped classroom shows a more significant effect on affective and interpersonal outcomes than on cognitive outcomes. This result can be explained by the features of the flipped classroom that encourage active engagement and learner-centered interactions. Instructors and other educational leaders in higher education institutions can pursue instruction redesigns and educational supports to implement flipped classrooms as an effective pedagogical practice. Additionally, the mixed results of adopting the flipped classroom instruction in departments and subjects show that various instructional forms and strategies are factors that determine the effectiveness on educational outcomes. Thus, future research must explore the relationship between various forms of flipped classrooms and educational outcomes to arrive at pedagogical decisions for instructional development.

Author Contributions

H.Y.J. conceived the research idea and designed the research framework; H.Y.J. analyzed the data; and H.Y.J. and H.J.K. wrote the draft and approved the final manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

The authors acknowledge support from the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (project NRF-2017S1A5A8021541). The views expressed in this article are those of the authors and do not necessarily reflect those of the grant agencies.

Acknowledgments

The authors want to thank their anonymous reviewers.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Keengwe, J.; Onchwari, G.; Agamba, J. Promoting effective e-learning practices through the constructivist pedagogy. Educ. Inf. Technol. 2014, 19, 887–898. [Google Scholar] [CrossRef]
  2. Rahman, A.A.; Mohamed, H.; Aris, B.; Zaid, N.M. The Influences of Flipped Classroom: A Meta-Analysis Approach Every Student Capability in Every Class. In Proceedings of the EEE 6th International Conference on Engineering Education, Kuala Lumpur, Malaysia, 9–10 December 2014; pp. 24–28. [Google Scholar]
  3. Van Alten, D.C.D.; Phielix, C.; Janssen, J.; Kester, L. Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis. Educ. Res. Rev. 2019, 28, 1–18. [Google Scholar] [CrossRef]
  4. McLaughlin, J.E.; Roth, M.T.; Glatt, D.M.; Gharkholonarehe, N.; Davidson, C.A.; Griffin, L.M.; Esserman, D.A.; Mumper, R.J. The flipped classroom: A course redesign to foster learning and engagement in a health professions school. Acad. Med. 2014, 89, 236–243. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Mason, G.S.; Shuman, T.R.; Cook, K.E. Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Trans. Educ. 2013, 56, 430–435. [Google Scholar] [CrossRef]
  6. Ratta, C. Flipping the classroom with team-based learning in undergraduate nursing education. Nurse Educ. 2015, 40, 71–74. [Google Scholar] [CrossRef]
  7. Hora, M.T.; Ferrare, J.J. Remeasuring postsecondary teaching: How singular categories of instruction obscure the multiple dimensions of classroom practice. J. Coll. Sci. Teach. 2014, 43, 36–41. [Google Scholar] [CrossRef]
  8. Campbell, C.M.; Cabrera, A.F.; Ostrow Michel, J.; Patel, S. From comprehensive to singular: A latent class analysis of college teaching practices. Res. High. Educ. 2017, 58, 581–604. [Google Scholar] [CrossRef]
  9. Låg, T.; Sæle, R.G. Does the flipped classroom improve student learning and satisfaction? A systematic review and meta-analysis. AERA Open 2019, 5, 1–17. [Google Scholar] [CrossRef]
  10. Abeysekera, L.; Dawson, P. Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. High. Educ. Res. Dev. 2015, 34, 1–14. [Google Scholar] [CrossRef] [Green Version]
  11. Hsieh, J.S.C.; Wu, W.-C.V.; Marek, M.W. Using the flipped classroom to enhance EFL learning. Comput. Assist. Lang. Learn. 2017, 30, 1–21. [Google Scholar] [CrossRef]
  12. Lo, C.K.; Hew, K.F. A critical review of flipped classroom challenges in K-12 education: Possible solutions and recommendations for future research. Res. Pract. Technol. Enhanc. Learn. 2017, 12. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Siegle, D. Technology: Differentiating instruction by flipping the classroom. Gift. Child Today 2014, 37, 51–55. [Google Scholar] [CrossRef]
  14. Karagöl, İ.; Esen, E. The effect of flipped learning approach on academic achievement: A meta-analysis study. Hacettepe Egit. Derg. 2019, 34, 708–727. [Google Scholar] [CrossRef]
  15. Chen, Y.; Wang, Y.; Kinshuk; Chen, N.S. Is FLIP enough? or should we use the FLIPPED model instead? Comput. Educ. 2014, 79, 16–27. [Google Scholar] [CrossRef] [Green Version]
  16. Jensen, J.L.; Kummer, T.A.; Godoy, P.D. Improvements from a flipped classroom may simply be the fruits of active learning. CBE Life Sci. Educ. 2015, 14, 1–12. [Google Scholar] [CrossRef] [PubMed]
  17. Zhang, Z. Construction of Online Course Based on FCM Concept. In Proceedings of the 2nd International Conference on Information, Electronics and Computer, Wuhan, China, 7–9 March 2014; pp. 157–160. [Google Scholar]
  18. Van Vliet, E.A.; Winnips, J.C.; Brouwer, N. Flipped-class pedagogy enhances student metacognition and collaborative-learning strategies in higher education but effect does not persist. CBE Life Sci. Educ. 2015, 14, 1–10. [Google Scholar] [CrossRef] [Green Version]
  19. Kong, S.C. Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing flipped classroom strategy. Comput. Educ. 2014, 78, 160–173. [Google Scholar] [CrossRef]
  20. Huelskamp, D. Flipping the collegiate science classroom: A review of the research. Glob. Educ. J. 2015, 2015, 61–72. [Google Scholar]
  21. Kang, N. The comparison between regular and flipped classrooms for EFL Korean adult learners. Multimedia-Assisted Lang. Learn. 2015, 18, 41–72. [Google Scholar]
  22. Clore, G.L.; Wyer, R.S.; Dienes, B.; Gasper, K.; Gohm, C.; Isbell, L. Affective feelings as feedback: Some cognitive consequences. In Theories of Mood and Cognition: A User’s Guidebook; Martin, L.L., Clore, G.L., Eds.; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2001; pp. 27–62. [Google Scholar]
  23. Isen, A.M. Positive affect decision making. In Handbook of Emotions; Lewis, M., Haviland-Jones, J.M., Eds.; Guilford Press: New York, NY, USA, 2000; pp. 417–435. [Google Scholar]
  24. Strayer, J.F. How learning in an inverted classroom influences cooperation, innovation and task orientation. Learn. Environ. Res. 2012, 15, 171–193. [Google Scholar] [CrossRef]
  25. Kim, H.R.; Jang, Y.K. Flipped learning with simulation in undergraduate nursing education. J. Nurs. Educ. 2017, 56, 329–336. [Google Scholar] [CrossRef] [PubMed]
  26. Missildine, K.; Fountain, R.; Summers, L.; Gosselin, K. Flipping the classroom to improve student performance and satisfaction. J. Nurs. Educ. 2013, 52, 597–599. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Chun, K.H.; Lee, Y.H. Improvement of debate competence: An outcome of an introductory course for medical humanities. Korean J. Med. Educ. 2016, 28, 87–93. [Google Scholar] [CrossRef] [PubMed]
  28. Ryan, M.D.; Reid, S.A. Impact of the flipped classroom on student performance and retention: A parallel controlled study in general chemistry. J. Chem. Educ. 2016, 93, 13–23. [Google Scholar] [CrossRef]
  29. Lucke, T. Using Learning Analytics to Evaluate the Effectiveness of the Flipped Classroom Approach. In Proceedings of the 25th Annual Conference of the Australasian Association for Engineering Education, Wellington, New Zealand, 8–10 December 2014; pp. 1156–1164. [Google Scholar]
  30. Pierce, R.; Fox, J. Vodcasts and active-learning exercises in a “flipped classroom” model of a renal pharmacotherapy module. Am. J. Pharm. Educ. 2012, 76, 1–5. [Google Scholar] [CrossRef] [Green Version]
  31. Cooper, H.; Hedges, L.V. The Handbook of Research Synthesis; SAGE Publications: New York, NY, USA, 1994. [Google Scholar]
  32. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Altman, D.; Antes, G.; Atkins, D.; Barbour, V.; Barrowman, N.; Berlin, J.A.; et al. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009. [Google Scholar] [CrossRef] [Green Version]
  33. Choi, Y. The Effects of Flipped-Learning on the Self-Directed Learning, Academic Motivation, and Class Satisfaction of the College Students. Master’s Thesis, Konkuk University, Seoul, Korea, 2016. Available online: http://www.riss.kr/link?id=T13997527 (accessed on 18 April 2020).
  34. Kim, N.; Chun, B.A.; Choi, J.-I. A case study of flipped learning at college: Focused on effects of motivation and self-efficacy. J. Educ. Technol. 2014, 30, 467–492. [Google Scholar] [CrossRef]
  35. Lee, H. The effects of English teaching-learning on learners’ English improvement and affective factors through flipped learning in a university ESP class. J. Linguist. Stud. 2015, 20, 73–99. [Google Scholar] [CrossRef]
  36. Lee, S. A case study of flipped learning at college English: Focused on failure tolerance and preference. English 21 2015, 28, 289–316. [Google Scholar]
  37. Lim, J. The effects of flipped classroom on middle level of college students’ essay performance. J. Soc. Teach. Engl. Media 2015, 16, 197–214. [Google Scholar]
  38. Lim, J. The effects of flipped procedural focused writing teaching on college students’ essay performance. J. Korea Engl. Educ. Soc. 2015, 14, 155–180. [Google Scholar]
  39. Oh, J.-S. University students’ experiences and perceptions towards flipped classroom. Korean Educ. Inq. 2015, 33, 1–23. [Google Scholar]
  40. Son, E.; Park, J.; Im, I.; Lim, Y.; Hong, S. Impact of flipped learning applied at a class on learning motivation of college students. J. Cogn. Enhanc. Interv. 2015, 6, 97–117. [Google Scholar]
  41. Souza, M.D.; Rodrigues, P. Investigating the effectiveness of the flipped classroom in an introductory programming course. New Educ. Rev. 2015, 40, 129–139. [Google Scholar] [CrossRef]
  42. Mattis, K.V. Flipped classroom versus traditional textbook instruction: Assessing accuracy and mental effort at different levels of mathematical complexity. Technol. Knowl. Learn. 2015, 20, 231–248. [Google Scholar] [CrossRef]
  43. Sletten, S.R. Investigating Self-Regulated Learning Strategies in the Flipped Classroom. In Proceedings of the Society for Information Technology & Teacher Education International Conference, Las Vegas, NV, USA, 2 March 2015; pp. 497–501. [Google Scholar]
  44. Mortensen, C.J.; Nicholson, A.M. The flipped classroom stimulates greater learning and is a modern 21st century approach to teaching today’s undergraduates. J. Anim. Sci. 2015, 93, 3722–3731. [Google Scholar] [CrossRef]
  45. Kim, E. The Effect of Flipped Learning Strategy on Motivation, Academic Achievement, and Self-Directed Learning of Junior College Students. Master’s Thesis, Incheon University, Incheon, Korea, 2016. Available online: http://www.riss.kr/link?id=T14026718 (accessed on 18 April 2020).
  46. Park, W.; Kim, H. A case study of flipped classroom applied in the classroom: Focusing on academic and nonacademic experiences, and classroom engagement of university students. J. Learn. Curric. Instr. 2016, 16, 525–546. [Google Scholar]
  47. Sun, J.C.Y.; Wu, Y.T.; Lee, W.I. The effect of the flipped classroom approach to OpenCourseWare instruction on students’ self-regulation. Br. J. Educ. Technol. 2017, 48, 713–729. [Google Scholar] [CrossRef]
  48. Davies, R.S.; Dean, D.L.; Ball, N. Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educ. Technol. Res. Dev. 2013, 61, 563–580. [Google Scholar] [CrossRef]
  49. Brooks, A.W. Information literacy and the flipped classroom: Examining the impact of a one-shot flipped class on student learning and perceptions. Commun. Inf. Lit. 2014, 8, 225–235. [Google Scholar] [CrossRef] [Green Version]
  50. Sengel, E. Using the “flipped classroom” to enhance physics achievement of the prospective teacher impact of flipped classroom model on physics course. J. Balk. Tribol. Assoc. 2014, 20, 488–497. [Google Scholar]
  51. Kim, Y. The effect of the flipped class on the affective experience, learning achievement, and class satisfaction of college English language learners. Foreign Lang. Educ. 2015, 22, 227–254. [Google Scholar] [CrossRef]
  52. Suh, M. The effect and awareness of the flipped classroom approach through mixed methods. J. Educ. Technol. 2016, 32, 535–570. [Google Scholar] [CrossRef]
  53. Overmyer, G.R. The Flipped Classroom Model for College Algebra: Effects on Student Achievement. Ph.D. Thesis, Colorado State University, Fort Collins, CO, USA, 2014. [Google Scholar]
  54. Winter, J.B. The Effect of the Flipped Classroom Model on Achievement in an Introductory College Physics. Ph.D. Thesis, Mississippi State University, Starkville, MS, USA, 2013. [Google Scholar]
  55. Lee, S. Effects of Flipped Learning on the Learning Motivation, Learning Flow, Class Participation in the University Class. Master’s Thesis, Chung Nam University, Daejeon, Korea, 2016. Available online: http://www.riss.kr/link?id=T14163208 (accessed on 18 April 2020).
  56. Talley, C.P. The enhanced flipped classroom: Increasing academic performance with student-recorded lectures and practice testing in a “flipped” STEM course. J. Negro Educ. 2013, 82, 339–347. [Google Scholar] [CrossRef]
  57. Albert, M.; Beatty, B.J. Flipping the classroom applications to curriculum redesign for an introduction to management course: Impact on grades. J. Educ. Bus. 2014, 89, 419–424. [Google Scholar] [CrossRef]
  58. Al-Zahrani, A.M. From passive to active: The impact of the flipped classroom through social learning platforms on higher education students’ creative thinking. Br. J. Educ. Technol. 2015, 46, 1133–1148. [Google Scholar] [CrossRef]
  59. Osman, S.Z.M.; Jamaludin, R.; Mokhtar, N.E. Flipped classroom and traditional classroom: Lecturer and student perceptions between two learning cultures, a case study at Malaysian Polytechnic. Int. Educ. Res. 2014, 2, 16–25. [Google Scholar] [CrossRef] [Green Version]
  60. Sahin, A.; Cavlazoglu, B.; Zeytuncu, Y.E. Flipping a college calculus course: A case study. Educ. Technol. Soc. 2015, 18, 142–152. [Google Scholar]
  61. Leicht, R.M.; Zappe, S.E.; Messner, J.I.; Litzinger, T. Employing the classroom flip to move “lecture” out of the classroom. J. Appl. Pract. Eng. Educ. 2012, 3, 19–31. [Google Scholar]
  62. Winquist, J.R.; Carlson, K.A. Flipped statistics class results: Better performance than lecture over one year later. J. Stat. Educ. 2014, 22, 1–10. [Google Scholar] [CrossRef] [Green Version]
  63. Flynn, A.B. Structure and evaluation of flipped chemistry courses: Organic & spectroscopy, large and small, first to third year, English and French. Chem. Educ. Res. Pract. 2015, 16, 198–211. [Google Scholar]
  64. Prashar, A. Assessing the flipped classroom in operations management: A pilot study. J. Educ. Bus. 2015, 90, 126–138. [Google Scholar] [CrossRef]
  65. Fraga, L.M.; Harmon, J. The flipped classroom model of learning in higher education: An investigation of preservice teachers’ perspectives and achievement. J. Digit. Learn. Teach. Educ. 2014, 31, 18–27. [Google Scholar] [CrossRef]
  66. ALRowais, A.S. The impact of flipped learning on achievement and attitudes in higher education. Int. J. Cross-Disciplinary Subj. Educ. 2014, 4, 1914–1921. [Google Scholar] [CrossRef]
  67. Borenstein, M.; Hedges, L.V.; Higgins, J.P.; Rothstein, H.R. Introduction to Meta-Analysis; Wiley: Hoboken, NJ, USA, 2011. [Google Scholar]
  68. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Lawrence Earlbaum Associates: Hillsdale, NJ, USA, 1988. [Google Scholar]
  69. Shadish, W.R.; Haddock, C.K. Combining estimates of effect size. In The Handbook of Research Synthesis and Meta-Analysis; Cooper, H., Hedges, L.V., Eds.; Russell Sage Foundation: New York, NY, USA, 2009; pp. 261–281. [Google Scholar]
  70. Higgins, J.P.T.; Green, S. The Cochrane Handbook for Systematic Reviews of Interventions; The Cochrane Collaboration: Oxford, UK, 2011. [Google Scholar]
  71. Hedges, L.V.; Olkin, I. Statistical Methods for Meta-Analysis; Academic Press: Orlando, FL, USA, 1985. [Google Scholar]
  72. Duval, S.; Tweedie, R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000. [Google Scholar] [CrossRef] [PubMed]
  73. Egger, M.; Smith, G.D.; Schneider, M.; Minder, C. Bias in meta-analysis detected by a simple, graphical test. Br. Med. J. 1997, 315, 629–634. [Google Scholar] [CrossRef] [Green Version]
  74. Schmidt, F.L.; Hunter, J.E. Methods of Meta-Analysis: Correcting Error and Bias in Research Findings, 3rd ed.; SAGE Publications: Thousand Oaks, CA, USA, 2015. [Google Scholar]
  75. Cooper, H. Research Synthesis and Meta-Analysis: A Step-by-Step Approach, 5th ed.; SAGE Publications: Thousand Oaks, CA, USA, 2017. [Google Scholar]
  76. Chen, K.S.; Monrouxe, L.; Lu, Y.H.; Jenq, C.C.; Chang, Y.J.; Chang, Y.C.; Chai, P.Y.C. Academic outcomes of flipped classroom learning: A meta-analysis. Med. Educ. 2018, 52, 910–924. [Google Scholar] [CrossRef] [Green Version]
  77. Shi, Y.; Ma, Y.; MacLeod, J.; Yang, H.H. College students’ cognitive learning outcomes in flipped classroom instruction: A meta-analysis of the empirical literature. J. Comput. Educ. 2020, 7, 79–103. [Google Scholar] [CrossRef]
  78. Cheng, L.; Ritzhaupt, A.D.; Antonenko, P. Effects of the Flipped Classroom Instructional Strategy on Students’ Learning Outcomes: A Meta-Analysis; Springer: Berlin/Heidelberg, Germany, 2019; Volume 67. [Google Scholar] [CrossRef]
  79. Binkley, M.; Erstad, O.; Herman, J.; Raizen, S.; Ripley, M.; Miller-Ricci, M.; Rumble, M. Defining twenty-first century skills. In Assessment and Teaching of 21st Century Skills; Springer: Berlin/Heidelberg, Germany, 2012; pp. 17–66. [Google Scholar]
  80. Al-Samarraie, H.; Shamsuddin, A.; Alzahrani, A.I. A flipped classroom model in higher education: A review of the evidence across disciplines. Educ. Technol. Res. Dev. 2019. [Google Scholar] [CrossRef]
  81. Akçayır, G.; Akçayır, M. The flipped classroom: A review of its advantages and challenges. Comput. Educ. 2018, 126, 334–345. [Google Scholar] [CrossRef]
  82. Gillette, C.; Rudolph, M.; Kimble, C.; Rockich-Winston, N.; Smith, L.; Broedel-Zaugg, K. A meta-analysis of outcomes comparing flipped classroom and lecture. Am. J. Pharm. Educ. 2018, 82, 6898. [Google Scholar] [CrossRef] [PubMed]
  83. Prince, M. Does active learning work? A review of the research. J. Eng. Educ. 2004, 93, 223–231. [Google Scholar] [CrossRef]
  84. Springer, L.; Stanne, M.E.; Donovan, S.S. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Rev. Educ. Res. 1999, 69, 21–51. [Google Scholar] [CrossRef]
  85. Glass, G.V.; Green, J.L.; Camilli, G.; Elmore, P.B. Meta-Analysis: The Quantitative Synthesis of Research Findings. In Handbook of Complementary Methods in Education Research; Green, J.L., Green, J.L., Camilli, G., Elmore, P.B., Eds.; Lawrence Earlbaum Associates: Washington, DC, USA, 2006; pp. 427–438. [Google Scholar]
  86. Fyrenius, A.; Wirell, S.; Silén, C. Student approaches to achieving understanding—Approaches to learning revisited. Stud. High. Educ. 2007, 32, 149–165. [Google Scholar] [CrossRef]
  87. Pearson, P.D.; Ferdig, R.E.; Blomeyer, J.R.L.; Moran, J.; North Central Regional Educational Laboratory (NCREL). The Effects of Technology on Reading Performance in the Middle-School Grades: A Meta-Analysis with Recommendations for Policy; Learning Point Associates: Naperville, IL, USA, 2005. [Google Scholar]
  88. Ranalli, J.; Moore, J. Targeted Flipped Classroom Technique Applied to a Challenging Topic. In Proceedings of the IEEE Frontiers in Education Conference (FIE), Erie, PA, USA, 12 October 2016; pp. 1–4. [Google Scholar]
Education 10 00115 g001 550
Figure 1. PRISMA flow diagram of article selection.
Figure 1. PRISMA flow diagram of article selection.
Education 10 00115 g001
Education 10 00115 g002 550
Figure 2. Forest plot showing the distribution of 43 studies’ effect sizes.
Figure 2. Forest plot showing the distribution of 43 studies’ effect sizes.
Education 10 00115 g002
Education 10 00115 g003 550
Figure 3. Funnel plot of effect sizes.
Figure 3. Funnel plot of effect sizes.
Education 10 00115 g003
Table
Table 1. Characteristics of studies.
Table 1. Characteristics of studies.
First AuthorYearPublicationEffect SizeData FormatMajorSubjectOutcome
Choi, Y. [33]2016Thesis0.566PPEducationEducationA
Chun, K. [27]2016Journal0.368PPMedicalMedicineA
Kim, N. [34]2014Journal0.220PPEducationPhysiologyC
Lee, H. [35]2015Journal0.634PPBusinessEnglishC
Lee, S. [36]2015Journal1.321PPEnglishEnglishN/A
Lim, J. [37]2015Journal−0.017PPMultipleEnglishC
Lim, J. [38]2015Journal−0.353PPMultipleEnglishC
Oh, J. [39]2015Journal0.059PPMultipleMultipleC
Son, E. [40]2015Journal1.666PPMultipleMultipleA
Pierce, R. [30]2012Journal0.856PPPharmacyPharmacyC
McLaughlin, J. [4]2014Journal0.270PPPharmacyPharmacyC
Souza, M. [41]2015Journal1.098PPEngineeringComputerC
Mattis, K. [42]2015Journal−0.079PPNursingMathA
Sletten, S. [43]2015Journal0.382PPMultipleMultipleC
Mortensen, C. [44]2015Journal0.459PPAnimal ScienceScienceC
Kang, N. [21]2015Journal0.196SMCDMultipleEnglishC
Kim, E. [45]2016Thesis0.641SMCDEducationEducationA
Park, W. [46]2016Journal0.261SMCDMultipleMultipleAC
Sun, J. [47]2017Journal0.128SMCDMultiplePhysicsA
Hsieh, J. [11]2016Journal0.709SMCDEnglishEnglishC
Davies, R. [48]2013Journal0.229SMCDManagementComputerC
Brooks, A. [49]2014Journal0.225SMCDMultipleEnglishC
Sengel, E. [50]2014Journal0.145SMCDEducationPhysicsC
Kim, Y. [51]2015Journal0.780SMCDMultipleEnglishA
Suh, M. [52]2016Journal0.467SMCDEducationEducationAC
Overmyer, G. [53]2014Dissertation0.192SMCDMultipleMathC
Winter, J. [54]2013Dissertation0.242SMCDMultiplePhysicsC
Lee, S. [55]2016Thesis0.953SMCDEducationEducationA
Jensen, J. [16]2015Journal0.076SMDMultipleScienceC
McLaughlin, J. [4]2014Journal0.270SMDPharmacyPharmacyACI
Mason, G. [5]2013Journal0.033SMDEngineeringEngineeringC
Souza, M. [41]2015Journal1.098SMDEngineeringComputerC
Missildine, K. [26]2013Journal0.436SMDNursingNursingC
Ryan, M. [28]2016Journal0.024SMDMultipleChemistryC
Talley, C. [56]2013Journal0.447SMDPsychologyPsychologyC
Albert, M. [57]2014Journal0.168SMDBusinessManagementC
Al-Zahrani, A. [58]2015Journal0.484SMDEducationEducationAC
Mason, G. [5]2013Journal0.033SMDEngineeringEngineeringACI
Osman, S. [59]2014Journal−0.933SMDBusinessBusinessC
Sahin, A. [60]2015Journal0.509SMDEngineeringEngineeringC
Leicht, R. [61]2012Journal0.080SMDEngineeringEngineeringC
Winquist, J. [62]2014Journal0.427SMDPsychologyMathC
Kim, Y. [51]2015Journal0.780SMDMultipleEnglishAC
Suh, M. [52]2016Journal0.467SMDEducationEducationC
Mattis, K. [42]2015Journal−0.079SMDNursingMathC
Flynn, A. [63]2015Journal0.107SMDChemistryChemistryC
Prashar, A. [64]2015Journal0.165SMDManagementManagementC
Fraga, L. [65]2014Journal0.360SMDEducationEnglishC
Overmyer, G. [53]2014Dissertation0.192SMDMultipleMathC
Winter, J. [54]2013Dissertation0.242SMDMultiplePhysicsC
Mortensen, C. [44]2015Journal0.459SMDAnimal ScienceScienceC
ALRowais, A. [66]2014Journal0.405PPEducationEducationC
Note: SMES = standardized mean difference; PP = pre–post; SMCES = standardized mean change difference; A = affective, C = cognitive, I = interpersonal.
Table
Table 2. Results of the homogeneity test.
Table 2. Results of the homogeneity test.
NQp-Value−95% CIES+95% CISE
431022.9<0.0000.300.320.340.011
Note: N = number of studies; Q = homogeneity statistics; ES = effect size; SE = standard error; CI = confidence interval.
Table
Table 3. Overall result of meta-analysis.
Table 3. Overall result of meta-analysis.
N−95% CIES+95% CISE
430.240.350.470.060
Note: N = number of studies; ES = effect size; CI = confidence interval; SE = standard error.
Table
Table 4. Effect size by outcome.
Table 4. Effect size by outcome.
Outcomesk−95% CIES+95% CISE
Affective Outcomes
Attitude110.520.640.750.058
Confidence70.130.250.360.059
Immersion31.291.521.760.122
Impression50.080.280.480.28
Independence20.330.570.810.123
Motivation110.650.760.880.057
Subtotal390.530.590.650.030
Cognitive Outcomes
Achievement80.440.590.750.080
Application80.160.250.340.046
Analysis30.280.460.630.092
Evaluation1−1.46−0.93−0.400.270
Knowledge1090.120.150.180.017
Metacognition160.170.260.350.046
Synthesis40.260.490.710.115
Understanding40.490.600.710.057
Subtotal1530.190.240.360.014
Interpersonal Outcomes
Interaction30.410.540.670.066
Participation100.540.630.730.048
Response50.180.320.450.068
Subtotal180.470.530.590.031
Note: k = number of effect size; CI = confidence interval; ES = effect size; SE = standard error.
Table
Table 5. Effect sizes by subgroup related to the department.
Table 5. Effect sizes by subgroup related to the department.
SubgroupCategoriesk−95% CIES+95% CISE
DepartmentAnimal Science40.350.460.570.055
Business60.120.190.260.038
Chemistry1−0.010.110.230.062
Education370.510.580.640.032
Engineering380.330.420.500.044
English20.460.831.200.189
Management150.110.250.390.072
Medical90.270.370.470.052
Nursing7−0.010.130.280.074
Pharmaceutical Med.170.240.290.340.025
Psychology20.150.440.720.147
Mixed710.170.210.250.021
Note: k = number of effect size; CI = confidence interval; ES = effect size; SE = standard error.
Table
Table 6. Effect sizes by subgroup related to the subject area.
Table 6. Effect sizes by subgroup related to the subject area.
SubgroupCategoriesk−95% CIES+95% CISE
SubjectBusiness1−1.46−0.93−0.410.270
Chemistry190.020.100.180.039
Computer Science70.800.961.110.077
Education260.720.800.880.041
Engineering320.070.170.270.051
English340.040.120.200.041
Humanities21.751.952.140.098
Management170.120.190.260.036
Mathematics130.030.130.230.051
Medicine90.270.370.470.052
Nursing10.210.440.660.116
Pharmaceutical Med.170.240.290.340.025
Physics90.040.150.250.054
Physiology90.110.220.330.054
Psychology10.000.450.890.226
Science50.330.430.540.053
Mixed90.370.440.510.037
Note: k = number of effect size; CI = confidence interval; ES = effect size; SE = standard error.
Table
Table 7. Effect sizes by subgroup: data format and publication type.
Table 7. Effect sizes by subgroup: data format and publication type.
Sub-GroupCategoriesk−95% CIES+95% CISE
Data FormatPre–post670.350.380.420.018
Treatment vs. control1060.210.250.280.017
Mean change difference450.410.470.530.029
Publication TypeDissertation290.540.610.680.036
Journal1880.260.290.310.012
Note: k = number of effect size; CI = confidence interval; ES = effect size; SE = standard error.
Table
Table 8. Results of the random-effects regression analysis by publication year.
Table 8. Results of the random-effects regression analysis by publication year.
Standard ParameterEstimateErrorz-Valuep-Value
Intercept1021.4141.87.200.001
Publication year−0.510.070−7.200.001

Share and Cite

MDPI and ACS Style

Jang, H.Y.; Kim, H.J. A Meta-Analysis of the Cognitive, Affective, and Interpersonal Outcomes of Flipped Classrooms in Higher Education. Educ. Sci. 2020, 10, 115. https://doi.org/10.3390/educsci10040115

AMA Style

Jang HY, Kim HJ. A Meta-Analysis of the Cognitive, Affective, and Interpersonal Outcomes of Flipped Classrooms in Higher Education. Education Sciences. 2020; 10(4):115. https://doi.org/10.3390/educsci10040115

Chicago/Turabian Style

Jang, Hwan Young, and Hye Jeong Kim. 2020. "A Meta-Analysis of the Cognitive, Affective, and Interpersonal Outcomes of Flipped Classrooms in Higher Education" Education Sciences 10, no. 4: 115. https://doi.org/10.3390/educsci10040115

APA Style

Jang, H. Y., & Kim, H. J. (2020). A Meta-Analysis of the Cognitive, Affective, and Interpersonal Outcomes of Flipped Classrooms in Higher Education. Education Sciences, 10(4), 115. https://doi.org/10.3390/educsci10040115

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop